Optimizing Rail Data for Google Earth Mashup
•Télécharger en tant que PPTX, PDF•
0 j'aime•2,140 vues
Presented by Lawrence Chong of Transport Canada Abstract: Displaying a huge data set for Google Earth can be a challenge. The data must be properly processed for efficient display. Using FME, the KML output is structured using grouping, networklink, tiling and region for optimization.
Recommandé
Recommandé
Contenu connexe
En vedette
En vedette (20)
Plus de Safe Software
Plus de Safe Software (20)
Dernier
Dernier (20)
Optimizing Rail Data for Google Earth Mashup
- 1. Optimizing Rail Data for Google Earth Mashup TM Lawrence Chong Technical Data Support Transport Canada April 24, 2012
- 2. Optimizing Rail Data for Google Earth Mashup TM OVERVIEW Preparing huge dataset for display in Google Earth can be a challenge. The data must be properly structured for efficient display. Recent release of FME 2012 includes several transformer enhancements specifically for KML translation. This presentation is to demonstrate how to achieve KML optimization using Grouping, Tiling, NetworkLink, Folder, Region and Styling with FME.
- 3. Optimizing Rail Data for Google Earth Mashup TM BACKGROUND Transport Canada (TC) is responsible for transportation policies and programs. TC regulations, standards and programs work to ensure that air, marine, road and rail transportation are safe, secure, efficient and environmentally responsible. With 48,000 kilometres of tracks and over 25,000 road crossings, Canada has one of the largest rail networks in the world. OBJECTIVE Implement a cost effective, efficient and portable system for viewing Rail Data. Provide Rail Inspectors the visualization tool to access rail information using Google EarthTM and Google Street ViewTM for planning, review and analysis, anywhere, anytime. Google EarthTM and Google Street ViewTM are trademarks of Google Inc.
- 4. Optimizing Rail Data for Google Earth Mashup TM Rail Data Mashup Layers Road Crossings Main Tracks Yards Spurs Crossing Collisions Source: Google EarthTM and Google StreetTM View
- 5. Optimizing Rail Data for Google Earth Mashup TM Basic FME Workflow Multi-steps parallel processing to create • Grouping data for various LOD • Tiling utilizing thinning and •NetworkLink cluster sampling • Folder MS Access Database technique •Regioning Excel Spreadsheet • Date formatting Output to 5 folders ESRI SHP Files • Label formatting Level 1 KML grouped by TC MapInfo TAB Files Validation • Attribute cleanup (Low LOD) Optimization Symbolization Regions Pass Rail Data KML KML Verification Level 2 Styling Data Integrity Optimization Output Fail Level 3 KML (High LOD) Optimization Terminate Display Error
- 6. Optimizing Rail Data for Google Earth Mashup TM FME Workbench for Road Crossings Data Integrity Loading KML Optimization Output Style Definition LOD 1 Cascading Folders (Levels 1 & 2) Cluster Sampling LOD 2 Cascading NetworkLink (Level 3) Tiling Level 1 (Low LOD) LOD 3 Level 2 Level 3 (High LOD) Grouping Styling Regioning
- 7. Optimizing Rail Data for Google Earth Mashup TM FME Workbench for Road Crossings Data Integrity Original: yyyymmdd00000000000 Formatted: yyyy-mm-dd
- 8. Optimizing Rail Data for Google Earth Mashup TM FME Workbench for Road Crossings KML Optimization – Cluster Sampling & Tiling Loading LOD 1
- 9. Optimizing Rail Data for Google Earth Mashup TM FME Workbench for Road Crossings KML Optimization – Cluster Sampling & Tiling LOD 2
- 10. Optimizing Rail Data for Google Earth Mashup TM FME Workbench for Road Crossings KML Optimization – Tiling LOD 3
- 11. Optimizing Rail Data for Google Earth Mashup TM FME Workbench for Road Crossings KML Optimization – Step 1 Cascading Folder •kml_document=Destination KML document •kml_id=kml_name=Folder Label Cascading NetworlLink •kml_link_href=Output KML file •kml_id=kml_name=NetworkLink label Cascading Folder
- 12. Optimizing Rail Data for Google Earth Mashup TM Cascading Folder •kml_document=Destination KML document FME Workbench for Road Crossings •kml_parent=Previous Grouping (fanout) •kml_id=kml_name=Current Grouping Cascading NetworlLink KML Optimization – Step 2 •kml_document=Previous Grouping (fanout) •kml_link_href=Output KML file •kml_id=kml_name=NetworkLink label Cascading Folder Cascading NetworkLink
- 13. Optimizing Rail Data for Google Earth Mashup TM FME Workbench for Road Crossings KML Optimization – Cascading Folder
- 14. Optimizing Rail Data for Google Earth Mashup TM FME Workbench for Road Crossings KML Optimization – Cascading NetworkLink
- 15. Optimizing Rail Data for Google Earth Mashup TM FME Workbench for Road Crossings KML Optimization – Step 3 KML Output LOD 1 (High Altitude) Single file-Folder structure LOD 2 Single file-Folder structure Cascading Folder •kml_document=Destination KML document •kml_parent=Previous Grouping (fanout) LOD 3 •kml_name=Placemark Label (Street Level) Multiple files- Cascading NetworlLink NetworkLink •kml_document=Previous Grouping (fanout) structure •kml_name=Placemark label Styling Regioning
- 16. Optimizing Rail Data for Google Earth Mashup TM FME Workbench for Road Crossings Style Definition
- 17. Optimizing Rail Data for Google Earth Mashup TM FME Workbench for Crossing Collisions Data Integrity Loading KML Optimization Output Style Definition Cascading NetworkLink Tiling > Grouping Styling Regioning
- 18. Optimizing Rail Data for Google Earth Mashup TM FME Workbench for Rail Network Pre-processing KML Optimization Output Style Definition NetworkLink Cascading Level 1 (Low LOD) Main Tracks Tiling Level 2 (High LOD) Main Tracks Main Tracks Spurs Yards Level 3 (Street Level) Yards & Spurs Grouping Regioning Styling
- 19. Optimizing Rail Data for Google Earth Mashup TM Output
- 20. Thank You! Questions? For more information: Lawrence Chong Lawrence.chong@tc.gc.ca Transport Canada
Notes de l'éditeur
- Presentation overv
- Brief background and the project objective.
- Workbench Workflow:1. Data Integrity Lat/Long validation, attribute verification and text replacement2. Loading Loading 2D point data3. KML Optimization A 2-tier cascading Folder/NetworkLink tree structure is generated using a 3-steps parallel process. Data with various Level of Detail (LOD) are created for viewing at various zoom levels. Level 1 being the highest and Level 3 being the lowest (street level). Optimization is achieved using the following: Cluster Sampling – Levels 1 and 2 are processed using this technique to reduce data density. Attributes are also removed to reduce file size for quick loading. Tiling – Using NTDB mapsheet of various scales as overlay, map index is added to the attribute used in tiling for consistent and uniform display.NetworkLink – To maintain small file size, data is fanned out using cascading NetworkLink to improve display performance.Regioning – Data is only visible visible when the bounding box is within the display and the LOD requirements are met. Grouping – The data is split into 5 groups based on TC Regions. Styling – Style is defined in the root document and referenced in other KML files to maintain small file size. 4. KML Output Using cascading Folder for a single file output to maintain an organized KML structure. Using cascading NetworkLink for multiple files output to maintain small file size.
- 1. Data Integrity Lat/Long validationRecords with invalid Lat/Long values and data outside Canada extent are discarded using Tester transformer. Date Formatting Using SubstringExtractor and StringConcatenator, dates are reformatted to read yyyy-mm-dd (original format=yyyymmdd0000000000) Attribute cleanup Using StringReplacer, some attribute texts are replaced.
- 2. LoadingUsing 2DPointReplacer , the data is loaded using the Lat/Long values. 3. KML Optimization – Cluster Sampling and Tiling (LOD 1) To optimized display, data displayed at high altitude are thinned using cluster sampling technique to reduce data density. This is done at the request of the end user to be able to pin point the location based on milepost before zooming into the area of interest. The workflow is as follow:Using NeighborhoodAggregator, a neighborhood bounding box with a value of .25 is defined for LOD 1 data sampling. CentrePointReplacer is used to extract the group centroid. Using NeighborhoodFinder, a search tolerance of .1 is performed against the complete dataset to extract the closest element to the centroid. Using PointOnArea, 1M Mapsheet index is added to the element attribute for use in Level 1 tiling. The output is reduced by 90%.
- 3. KML Optimization – Cluster Sampling and Tiling (LOD 2) Level 2 optimization is identical to Level 1 except for the sampling parameters.Using NeighborhoodAggregator, a neighborhood bounding box with a value of .1 defined for data sampling. CentrePointReplacer is used to extract the group centroid.Using NeighborhoodFinder, a search tolerance of .03 is performed against the dataset to extract the closest element to the centroid.Using PointOnArea, 250K Mapsheet index is added to the element attribute for use in tiling later. The output is reduced by about 80%.
- 3. KML Optimization – Cluster Sampling and Tiling (LOD 3)Only tiling is performed for Street Level data. Using 50K Mapsheet as overlay, map index is added tothe element attribute for use in tiling.
- 3. KML Optimization – Step 1 of 3 An organized KML tree structure is achieved using CascadingNetworkLink and Cascading Folder. This allows for speedy search and locating data. First, the data is grouped using the Aggregator transformer. For Folder/NetworkLink feature type, required KML tags are kml_id, kml_document, kml_link_href/kml_parent and kml_name. The tag values for the Step 1 are as follow: Cascading Folderkml_document=Destination KML documentkml_id=kml_name=Folder name=Current GroupingCascading NetworlLinkkml_link_href=Output KML filekml_id=kml_name=NetworkLink name=Current Grouping The output port connects to the Folder feature type andNetworkLink feature type respectively.
- 3. KML Optimization – Step 2 of 3KMLRegion is used to control the display behavior of placemarks using view location and LOD. Itsparameters are defined usingKMLRegionSetter In order to maintain a consistent and uniform display when zooming in and out, the Region boundary extent is assigned a fixed value of the mapsheet extent. Min/Max for the Region is calculated using the centroid and the extent of the mapsheet. View setting parameter is as shown. Folder/NetworkLink parameters are as follows:Cascading Folderkml_document=Destination KML documentkml_parent=Previous Grouping (fanout)kml_id=kml_name=Folder name=Current GroupingCascading NetworlLinkkml_document=Previous Grouping (fanout)kml_link_href=Output KML filekml_id=kml_name=Current Grouping=NetworkLink nameThe output port connects to the Folder feature type andNetworkLink feature type respectively.
- 3. KML Optimization - Cascading FolderSteps 1 & 2 For cascading Folder, create a feature type called Folder and connect it to Step 1 & 2. There must be an instance of Folder feature type for each KML output.
- 3. KML Optimization - Cascading NetworkLinkSteps 1 & 2 For cascading NetworkLink, create a feature type called NetworkLink and connect it to Step 1 & 2. There must be an instance of NetworkLink feature type for each KML output.
- 3. KML Optimization - Step 3 of 3This is the main output. Workflow is as follow: Using BoundingBoxExtractor to extract boundary extent.Define the fanout string using previous grouping. The value is assigned to kml_document. Using AttributeCreator to create KML tag called kml_target style for unique styling. The data is fanned into 3 sets of output based on LOD. Mapsheet extent is used as the Region bounding box for a consistent and uniform display. Set the values for: Region View PlacemarkCascading Folder/NetworkLink parameters are defined as follows:Cascading Folderkml_document=Destination KML documentkml_parent=Previous Grouping (fanout)kml_name=Placemark LabelCascading NetworlLinkkml_document=Previous Grouping (fanout)kml_name=Placemark Label4. KML OutputOutput to 5 groups based on TC Region each with 3 datasets.Single file output for each of LOD 1 and 2.Multiple files output using group fanout for LOD 3
- 3. KML Optimization - Custom StylingStyle is defined once in the root document and referenced in other KML files to reduce repetition and thus minimizes file size.Create a null geometry instance. Create unique KML style for each Protection Type using KmlStyler and assign it to kml_id. Create an instance of the Style feature type for each KML output. Connect each Style feature to every style defined.
- Crossing Collision Workflow:The workflow is very identical to Road Crossing and uses the same input file. The data is filtered to process records with Accident Count > 0 only. 1. Data IntegrityLat/Long validation, attribute verification and text replacement2. Loading Loading 2D points3. KML Optimization Tiling – Using PointOnArea, map index attribute is added to the schema to be used for tiling to maintain uniform display. Cascading NetworkLink – To maintain small file size, the data is broken down into the smallest denomination. Region – Placemark is only visible when its bounding box is within the viewing extent and the prescribed resolution. Grouping – The data is split in 5 groups based on TC Regions. Styling – Style is defined in the root document and referenced in other KML files, thus minimize file size.4. KML Output Using cascading NetworkLink for multiple files output to maintain small file size.
- Rail Network Workflow:Rail Network consists of Main Tracks, Spurs and Yards.1. Pre-processingJoin short segments using LineJoiner to reduce fragmentation. The joined lines are then generalized using Generalizer for 2 zoom levels. 2. Tiling Using NTDB mapsheet as overlay, map index attribute is added to the attribute using LineOnArea . The bounding box is used for tiling to maintain uniform display.3. KML Optimization 3 datasets for viewing at different altitudes are generated. Low resolution Main Track data is generalized and attributes removed to reduce file size for efficient display. Cascading NetworkLink – To maintain small file size, the data is fanned out using cascading NetworkLink. Region – Placemarks are only visible when its bounding box is within the viewing extent and the prescribed resolution. Grouping – The data is split in 5 groups based on TC Regions. Styling – Style is defined in the root document and referenced in other KML files, thus minimize file size.4. KML Output Using cascading Folder for a single file output to maintain an organized KML structure. Using cascading NetworkLink for multiple files output to maintain small file size.
- KML Output: Anorganized and orderly KML tree structure enables quick content search.