Presentation given by Jonathon W. Lowe, GIS Specialist, IBM Consulting Services.

2. Tonight’s Agenda, in no particular order: What’s the problem? Enterprises such as DEFRA want to manage their spatiotemporal data using consolidation, quality and access strategies. Who cares? Introducing a typical cast of stakeholders (and their diverse cultures) in a shared spatiotemporal data management programme… Laws of the jungle. Pure “greenfields” are ever more rare – what technical constraints do implementors encounter in large projects? Meet the Beast. What is IBM’s architectural approach when answering spatiotemporal data management and systems integration challenges? Where next? With a baseline for data exchange in place, what happens next for an integrated spatiotemporal enterprise, and for our industry? Questions? Heckle during the presentation, or save up for the end.

3. What’s the spatial problem? For GI practitioners, the DEFRA challenge is to make spatiotemporal data management more efficient for both data providers and data recipients. This increased efficiency will in turn lead to higher quality data and improved policy making.

4. Drawn from RASTER DATA Drawn from VECTOR DATA Drawn from NUMERIC, TEXTUAL, or TEMPORAL ATTRIBUTES ORTHOPHOTO CONTEXTUALBASE MAP THEMATIC or CHOROPLETH MAP Spatiotemporal Data

5. DEFRA’s 400+ thematic business datasets Ordnance Survey’s MasterMap & Rasters UK Perspectives’ Aerial Photography Spatiotemporal Data at DEFRA

6. Spatiotemporal Data Capture and Maintenance Spatiotemporal Application Development ? Spatiotemporal Data Management & Systems Integration

7. Spatiotemporal Data Capture and Maintenance Spatiotemporal Application Development Spatial Database (PostgreSQL, Oracle, Informix, DB2, MySQL Microsoft SQL Server) Traditional GIS Traditional GIS Mainstream IT applied to a traditional GIS?

8. Data Provider Data Provider Data Provider Data Provider Data User Data User Data User Data User Repository Data Provider Data Provider Data Provider Data Provider Data User Data User Data User Data User Spatiotemporal Data Consolidation However, central repositories reduce direct contact between people, the same people who used to tell you whether their data was fit for your purposes…or not. How can data users evaluate a warehouse’s data quality without talking to the people creating and providing the data? Time formerly spent managing our data’s distribution now goes toward improving and maintain our data instead. Benefit Deliver the data once to a central Repository from which multiple internal and external users download data on demand. Repeatedly distributing spatial data to multiple internal and external users in a point-to-point fashion takes time. Remedy Data Provider Pain By pulling data from the Repository, I save time formerly spent managing that data myself. Benefit The Repository supplies all the required current data from one access point. I have to find and manage the data I need for my application, and must ensure that I keep up to date data. Remedy Data User Pain

9. Spatiotemporal Data Quality Validation Unclosed polygons Closed polygons Duplicate vertices Clockwise exterior and interior rings Proper exterior and interior ring rotation No duplicate vertices Examples of potentially problematic geometries The same geometries with fixes applied But … the consolidated datasets in a central repository and the associated quality reports are useless unless they can be searched by both data providers and potential data recipients.

10. Spatiotemporal Data Storage and Inventory

11. Spatiotemporal Multi-format Data Access Instead of managing our data’s distribution formats, we now have more time to improve and maintain our data. Benefit The Repository accepts the data in one standard submission format but distributes it to users in a variety of formats. It’s time consuming to distribute our spatial data in the many different formats that our users’ heterogenious systems require. Remedy Pain Instead of reformatting data to suit our system’s needs, we can begin analysing that data immediately upon receiving it. Benefit The Repository provides data in the formats most often requested by data recipients. The data we need is not available in the required format, so we spend time reformatting the data ourselves. Remedy Pain

13. Spatiotemporal Data Capture and Maintenance Spatiotemporal Application Development ? Spatiotemporal Data Management & Systems Integration

14. Spatiotemporal Data Capture and Maintenance Spatiotemporal Application Development Do these 4 services comprise Spatio-temporal Data Manage-ment? Spatiotemporal Data Consolidation Spatiotemporal Data Quality Validation Spatiotemporal Data Storage and Inventory Spatiotemporal Multi-format Data Access

15. Spatiotemporal Infrastructure

16. Spatiotemporal Security Data Provider Data Provider Data Provider Data Provider Data User Data User Data User Data User Repository Data Provider Data Provider Data Provider Data Provider Data User Data User Data User Data User Point-to-point security model Centralised, administered security model

17. Offering #9: Usage Metrics and Metering

18. Spatiotemporal Data Capture and Maintenance Spatiotemporal Application Development Spatiotemporal Data Management Systems Integration Spatiotemporal Data Consolidation Spatiotemporal Data Quality Validation Spatiotemporal Data Storage and Inventory Spatiotemporal Multi-format Data Access Spatiotemporal Infrastructure Spatiotemporal Security Spatiotemporal Usage Metrics and Metering

19. Technical Constraints Pure “greenfields” are ever more rare – what technical constraints do implementors encounter in large projects?

20. IBM’s technical constraints IBM’s p-series hardware (64-cpu server) IBM’s AIX 5.3 Operating System IBM’s Websphere Application Server (with 64-bit Java Virtual Machine) IBM’s Websphere Portal Server Java Development Environment (preferred) Websphere Application Server Websphere Portal Server DEFRA’s technical constraints GI Clients: CadCorp and ESRI and MapInfo Oracle Spatial Oracle 9i or 10g Spatial Database

21. Evaluating DEFRA’s constraints Version compatibilities of ESRI, Oracle Spatial and OGC Web Services

22. IBM’s technical constraints IBM’s p-series hardware (64-cpu server) IBM’s AIX 5.3 Operating System IBM’s Websphere Application Server (with 64-bit Java Virtual Machine) IBM’s Websphere Portal Server Java Development Environment (preferred) Websphere Application Server Websphere Portal Server DEFRA’s technical constraints GI Clients: CadCorp and ESRI and MapInfo Oracle Spatial Oracle 9i or 10g Spatial Database Geospatial Rendering Engine ? ?

23. The tension between traditional GIS, mainstream IT and free open-source geospatial software products

24. “ How is it that groups of computer programmers (sometimes very large groups) made up of individuals separated by geography, corporate boundaries, culture, language, and other characteristics, and connected mainly via telecommunications bandwidth, manage to work together over time and build complex, sophisticated software systems outside the bounaries of a corporate structure and for no direct monetary compensation? And why does the answer to that question matter to anyone who is not a computer programmer?” “ This book explains how the open source software process works. It is broadly a book about technology and society, in the sense that changes in technology uncover hidden assumptions of inevitability in production systems and the social arrangements that accompany them. It is also about computers and software, because the success of open source rests ultimately on computer code, code that people often find more functional, reliable, and faster to evolve than most proprietary software built inside a conventional corporate organization. It is a business and legal story as well. Open source code does not obliterate profit, capitalism, or intellectual property rights. Companies and individuals are creating intellectual products and making money from open source software code, while inventing new business models and notions about property along the way.”

26. FOSS4G2006 (Free Open Source Software for Geospatial, 2006 Conference

27. What’s the next (bigger) problem? For Information Technology practitioners in general, the DEFRA problem is systems integration in support of a more “joined-up” government organisation. Evidence of success is the British public’s level of satisfaction with government services – for DEFRA, this public is often farmers or anyone involved with food, livestock, disease or the rural enviroment. What’s the spatial problem? For GI practitioners, the DEFRA challenge is to make spatiotemporal data management more efficient for both data providers and data recipients. This increased efficiency will in turn lead to higher quality data and improved policy making.

30. Spatiotemporal Data Quality Validation Unclosed polygons Closed polygons Duplicate vertices Clockwise exterior and interior rings Proper exterior and interior ring rotation No duplicate vertices

31. Note: Basic Validation does not test Spatiotemporal Contextual Data Quality. Although their edges should match, polygon data representing moorland (purple diagonal cross hatching) may extend beyond government-established boundaries for ecosystem protection (solid pink) due to data capture at different scales.

32. No Payment (outside of scheme) Payment = £16.10/ha Disadvantaged Ecosystem Payment = £29.78/ha Severely Disadvantaged Ecosystem Payment = £11.26/ha Moorland Invalid data! RULE: Moorland Line must exist within LFA Moorland Severely Disadvantaged Ecosystem Disadvantaged Ecosystem Land Parcel This farmer’s land parcel may be subject to FOUR different payment tiers, as well as an erroneous payment due to contextually invalid geometric data . Note: Basic Validation does not test Spatiotemporal Contextual Data Quality. Although their edges should match, polygon data representing moorland (purple diagonal cross hatching) may extend beyond government-established boundaries for ecosystem protection (solid pink) due to data capture at different scales.

33. Spatiotemporal Data Capture and Maintenance Spatiotemporal Application Development Spatial Database (PostgreSQL, Oracle, Informix, DB2, MySQL Microsoft SQL Server) Traditional GIS Traditional GIS Mainstream IT applied to a traditional GIS?

34. Spatiotemporal Data Capture and Maintenance Spatiotemporal Application Development Spatiotemporal Data Management Systems Integration Spatiotemporal Data Consolidation Spatiotemporal Data Quality Validation Spatiotemporal Data Storage and Inventory Spatiotemporal Multi-format Data Access Spatiotemporal Infrastructure Spatiotemporal Security Spatiotemporal Usage Metrics and Metering

35. Spatiotemporal Data Capture and Maintenance Spatiotemporal Application Development Spatiotemporal Data Management Systems Integration Spatiotemporal Data Consolidation Spatiotemporal Data Quality Validation Spatiotemporal Data Storage and Inventory Spatiotemporal Multi-format Data Access Spatiotemporal Data Processing Spatiotemporal Infrastructure Spatiotemporal Security Spatiotemporal Usage Metrics and Metering

36. Step 1 : An inspector discovers a diseased animal and reports its position. The State Veterinary Svc. creates an initial buffer zone (circle) around the point of disease discovery. Step 2 : SVS then expands the buffer zone (manually) to include intersected and epidemiologically relevant contiguous land known to host livestock management activities. Step 3 : SVS then identifies the livestock keepers contained within the zone and enlists their cooperation in controlling animal movement within it, enforced through a permitting process. In past outbreaks, this series of steps has taken several hours of desktop processing each night, accompanied by a manual results distribution process. Disease Mapping Scenario for State Veterinary Service (SVS) Farmer Jones Farmer Blair Farmer Smith

37. Disease Mapping Scenario for State Veterinary Service (SVS) An alternative : Following disease point discovery, SVS places a point on a map, and sets a radius. The tool automatically selects underlying intersecting fields, allows for easy one-click expansion and automatically identifies the links to a land ownership database. Hours of processing are reduced to minutes and results are automatically available to all members of the service.

38. Possible technical solutions for Disease Zone polygon extension scenario Land boundaries via (A) remote spatial database connection, or (B) “throttled” Web Feature Services connection Local Client Local Client Local Client Database & Web Server Database & Citrix Server Database & Web Server 1 2 3 Users’ mouse clicks are sent to a remote web application that is built for a single functional purpose. Desktop Tools CadCorp/ESRI/MapInfo Client Spatial Layers Disease Zones Desktop Tools Citrix Client simulates CacCorp/ ESRI/MapInfo Client Spatial Layers [None] Desktop Tools Web Browser Client Spatial Layers [None] Users’ mouse clicks are sent to Citrix server which simulates a local GIS desktop on the local client. Local GI Client & Remote Spatial Data : Use a local GI client to snap locally stored Disease Zone polygons to remote land boundary geometry via (A) network access to a remote spatial database holding PBL data (e.g. a “mart”), or, (B) a “throttled” PBL Web Feature Service. Remote Solution-specific GI Application & Remote Spatial Data : Use a web browser and online custom application to add or remove land boundary polygons from the Disease Zone polygon by clicking desired land boundary polygons. Remote ArcGIS & Remote Spatial Data : Use a remote GI client, provided via Citrix, to snap (remotely stored) Disease Zone polygons to underlying (remotely stored) land boundary polygons. The GI client accesses the remote spatial data via spatial database connections or “throttled” Web Feature Service connections. Connection Connection Connection Pros : Power users can apply full suite of desktop GI capabilities. Access to features enables local analysis and copy/snap. Cons : Performance limited by network bandwidth. New data must go back through QC system. Desktop GI client training time and software/licence costs. Pros : Same pros as (1) above, plus good performance and manageable deployment, licencing, and version control of GI Client tools. Cons : Citrix licence is an extra expense. Cost of spatial database software & licence. Cost of GI client tool training. Pros : Same pros as (1) above, plus ease of deployment, no licencing, minimal training, good performance and wide accessibility. Cons : Time/cost of capturing requirements and building custom solution. Heavy CPU load Server Spatial Layers Land boundaries Aerial Photos Server Spatial Layers Land boundaries Disease Zones Aerial Photos Server Spatial Layers Land boundaries Disease Zones Aerial Photos Heavy Network traffic Heavy CPU load Light CPU load Moderate Network traffic Heavy CPU load Light CPU load Light Network traffic Heavy CPU load

39. Spatiotemporal Data Capture and Maintenance Spatiotemporal Data Consolidation Spatiotemporal Data Quality Validation Spatiotemporal Data Storage and Inventory Spatiotemporal Multi-format Data Access Spatiotemporal Data Processing Spatiotemporal Infrastructure Spatiotemporal Security Spatiotemporal Usage Metrics and Metering Spatiotemporal Application Development

40. Tonight’s Agenda, in no particular order: What’s the problem? Enterprises such as DEFRA want to manage their spatiotemporal data using consolidation, quality and access strategies. Who cares? Introducing a typical cast of stakeholders (and their diverse cultures) in a shared spatiotemporal data management programme… Laws of the jungle. Pure “greenfields” are ever more rare – what technical constraints do implementors encounter in large projects? Meet the Beast. What is IBM’s architectural approach when answering spatiotemporal data management and systems integration challenges? Where next? With a baseline for data exchange in place, what happens next for an integrated spatiotemporal enterprise, and for our industry? Questions? Heckle during the presentation, or save up for the end.