This document discusses how rasters from satellites, drones, and other sources can be processed and analyzed using FME. It provides examples of bringing raster data into workflows from various sources like Landsat 8, Sentinel 2, and Planet. It also discusses how FME can be used to automate raster processing, mosaicking, styling, combining, and outputting tiles. Examples are given for creating up-to-date hybrid basemaps and monitoring and analyzing wildfire changes daily. Additional examples discuss building georeferenced satellite videos and using FME for natural hazard workflows like flood risk assessment and search and rescue with UAVs. Automation and accessing data are emphasized as important for meeting challenges of natural disaster response.
7. first
Get data from Planet and
other sources
Read imagery and
vector layers.
Process
Mosaic rasters,
style vector data,
combine, tile.
Output as .png
Store raster tiles in the
cloud. Make available
everywhere to
anyone.
third
Automate
Do this whenever new
images become
available.
last
DEMO: UP-TO-DATE HYBRID
BASEMAP
second
10. first
Get images
Read Planet
imagery of California
wildfires.
Process Store
Save to local or
cloud storage.
third
Analyze
Make tools for visual
and analytical change
comparison.
last
DEMO: MONITOR AND
ANALYZE CHANGE DAILY
Derive new products
like NIR, NDVI, NDWI.
second
12. first
Get images from Planet
Collect images for a
selected timeframe.
Process
Clip to extents of
desired area.
Add video to map
Prepare HTML output
with LeafletJS.
MAKE A GEOREFERENCED
SATELLITE VIDEO
third lastsecond
Output as .mp4
Integrate with FFmpeg
and output a video.
15. NATURAL HAZARDS: CHALLENGES
● Diverse data types needed.
○ Vector and raster
○ Open and proprietary
○ Spatial and non-spatial
● Limited IT infrastructure. Need to make
data accessible.
● Urgency! Disasters mean short timelines.
● Need to automate processing and
distribution.
16. KEY WORKFLOWS
● Impact assessment.
○ population and transportation
○ identify critical infrastructure
○ hazard sources and extents
● Data integration.
● Risk, probability, scenarios, time.
● UAV surveys, automation.
21. UAVs FOR DAMAGE ASSESSMENT
● UAVs are easy and safe to deploy.
● Rapidly assess impact to people and
infrastructure.
● Data guides the level of response.
Story: Renato Salvaleon develops UAV
systems at Southern Co, supported by
FME Server automation. safe.com/uav
UAV storm damage assessment by Southern Co.
22. MEETING NATURAL HAZARD CHALLENGES
● Diverse data types needed.
○ Vector and raster
○ Open and proprietary
○ Spatial and non-spatial
● Limited IT infrastructure. Need to make data accessible.
● Urgency! Disasters mean short timelines.
● Need to automate data processing and distribution.
Rapid prototyping
24. RESOURCES
● San Francisco Transit demo: http://fme.ly/sftransit
● Blog - earthquake notifications: http://fme.ly/earthquake
● Christchurch earthquake story: http://fme.ly/christchurch
● Flood notification tutorial: http://fme.ly/flood
● Risk analyzer by con terra: georiskanalyzer.com
● Blog - get started with drones: http://fme.ly/drones
● FME Server for Fort McMurray wildfire: http://fme.ly/atco
● Search “remote sensing” or “UAV” on safe.com
We don’t care about rasters; we care about rasters mean - is it a picture, elevation model, etc. A raster is just a means to achieving the solution we want - we care about the environment or the city or whatever else it represents.
Rasters are coming to us in large volumes and in real time, so processing this data automatically is key. That’s why FME Server and FME Cloud are such an important part of a lot of raster workflows.
Planet is a huge archive - billions of images - humans can’t manage all of this manually. We need automation.
Some useful ones worth mentioning.
FeatureReader is not a raster transformer, but we want to mention it here because it plays an important role when reading satellite data. Instead of a traditional approach - read data, then figure out what to keep for further processing, we first figure out what we need (coverage, weather, quality, date range), and then read it using all those pre-processed values as parameters.MapnikRasterizer - for making a really nice output - maps or anything (chocolate packages)
FeatureReader - let features drive what data you get from Planet. e.g. GIS boundaries
RasterMosaicker - combine all the images we get
MapnikRasterizer - we have lots of mapnik resources & webinars
NDVI - the key is to say it’s a custom transformer - can add functionality that’s missing
http://fme.ly/sftransit or http://demos.fmeserver.com/planet-satellite-map/
server checks regularly for new images that meet criteria
https://blog.safe.com/2018/01/elastic-processing-cloud-docker-fme-server/
Generating millions of tiles normally takes a full day… we were able to do it in 25 mins.
We’re talking about assessing risks and damage from floods, earthquakes, forest fires, climate change, hurricanes, and more.
Image 1: Alpine Shire Council wanted to improve bushfire risk assessments. With FME, they designed an app that gives real-time analysis of risk. (If you search “alpine shire council” on safe.com you can find this story.)
Image 2: I85 Atlanta Bridge Collapse: a guy lit a grocery cart on fire and the fire grew and ultimately collapsed the bridge. This was a Southern Co project to assess the damage to infrastructure.
Image 3: was an FME UC presentation - “Helping rebuild a city with FME” - after the Christchurch earthquake
__
Other examples:
Critical infrastructure protection: FortisBC
Humanitarian and disaster response: Hurricane, Syria, Myanmar (OGC Testbed13)
UAVs allow for rapid assessment and response: UAV damage assessment and cell network support in Puerto Rico
Image of SFO flooding scenario from OGC TestBed 11. FME was used to read NetCDF data from a high resolution flood forecast model and convert this into a KML time series of raster images for display in Google Earth.
Several key points from this slide are based on findings from the following study:
https://unstats.un.org/unsd/geoinfo/RCC/docs/rccap20/16_Robert%20Deakin_New%20Zealand%20-%20Canterbury%20SDI%20Paper.pdf
Twentieth United Nations Regional Cartographic Conference for Asia and the Pacific Jeju, 6 - 9 October 2015 Item 7(b) of the provisional agenda Invited Papers Canterbury SDI: lessons learned from post-earthquake recovery
The root causes underlying these issues were identified as:
The lack of data sharing agreements in place;
No data sharing channels (such as web services), or standardisation of formats and data models
No catalogue or registry of available data sources
A lack of training or practice on how to pull together data sources
Problems manifested as:
• Inability to find key people who could supply data;
• Inability to access the data;
• Reluctance to supply data because of perceived poor quality;
• Reluctance to supply data because the requestor was perceived as not needing it;
• Reluctance to supply data because it was not known how the requestor might use it;
• Privacy issues;
• Supplying spatial data in unhelpful formats.
Problems related to integrating data from multiple agencies included:
• No plans on how to integrate data representing aspects such as people, places, things, events and concepts, from multiple agencies;
• No plans on how to integrate data from a single agency into a recipient’s business system;
• Data-models were different and the attributes contained different data. Schemas were not available;
• Abstractions were different (e.g. mobile toilet delivery being recorded as a number against a street name by some people, and against an address by others);
Geometry data types were different (whether the feature was stored as a point, a line, or a polygon); • Datasets were at different levels of completion; • Data was in different formats, e.g. WFS, ArcGIS, MS Access, Excel, CSV files, PDFs, paper forms and maps. Getting expertise was also a problem: • People were thrust into positions they were not equipped to deal with; • Information was captured by people without experience of spatial data; • Data was managed by people without experience of spatial data; • No access to technical expertise; • Out-of-town resources were unfamiliar with the city (e.g. common road names and place names); • Out-of-town resources were unfamiliar with the systems they were working in.
__
Opportunities with FME
Common data models, spatial reference
Open standards
Cloud based, decentralized
Mobile devices
Crowd sourcing
APIs, loosely coupled
Web sources – leave in place
Real time, non-spatial
Model based, rapid prototyping
Easy updates
FME Capabilities
Formats R&W
Web Access, Publish
Processing Geometry: Vector, raster, point clouds
Processing records: ETL, schema mapping
https://www.safe.com/solutions/for-industries/emergency-services-and-policing/
Increase Disaster Response Efficiency
Build Situational AwarenessSwiftly gather, combine, and analyse tabular and location-aware information from 350+ sources, providing facts for critical decision making in real-time.Leverage Citizen Engagement Harvest intelligence from citizens via social media, including geotagged photos, and return life-saving information back to the public.Hands-Free Data Flow Reduce manual efforts during emergencies by creating automated workflows that leverage complex event processing (CEP) technology to immediately deliver critical intelligence using real-time notifications.
Integrating various sources is key.
Impact assessment requires 4D basemap (time) to fully evaluate impacts.
Hazard impact extents
Real time updates from the field, often non-spatial (messages, spreadsheets)
Intersect impact zones with infrastructure to determine severity
Our example demo scenario - flood risk assessment. Open in Workbench.
Vancouver Flood Hazard Risk Assessment:
Read infrastructure layers from geodatabase and assign criticality
Load elevation model for Vancouver region
Read live water level feed from nearest NOAA monitoring station
Generate flood severity polygons with FloodAreaExtractor
Assess flood impact on infrastructure
Calculate HazardRisk = severity * criticality, and use to style output
FloodRiskAreas:
Subtract flood level from each cell
Classify cells by flood severity
Convert severity levels to areas
Hazard impact extents
Real time updates from the field, often non-spatial (messages, spreadsheets)
Map extents in 4D (time)
Intersect impact zones with infrastructure to determine severity
UAVs are a help for marine search and rescue because you can search large areas quickly and safely. Consider near-shore areas that are hard to access by land or sea – UAVs can fill this gap.
RCMSAR used FME to build the route, i.e. generate search grid waypoints. Then UAV flies the grid (autonomously, to reduce human error and maximize coverage given limited flight time). Then they used FME to assemble the results into a photomosaic.
Automation is key for flight planning and post-processing. Need to optimize the route, make sure you comply with regulations, process the data that’s gathered, build a catalogue or dashboard, distribute the data, generate flight statistics.
Telemetry based georeferencingExport to KML for Google Earth review
See Presentation by Renato, Steve, Dean at FME 2017 UC:
https://www.safe.com/presentation/uavs-and-fme-powering-your-drone-and-data-with-fme/
UAS at Southern Co:
https://www.southerncompany.com/innovation/unmanned-aircraft-systems.html
http://proceedings.esri.com/library/userconf/geoconx16/papers/geoconx_51.pdf
Another Key Automation Workflow: Resilience Scenario Analysis
Modelling / simulation (FME supports NetCDF 4, multi-D raster)
Scenarios stress / test response systems
Requires automation to iterate through impact analysis
Big users are analyzers of climate change and extreme weather impacts
Again: rasters are coming to us from satellites & UAVs in large volumes and in real time, so processing this data automatically is key. That’s why FME Server and FME Cloud are such an important part of a lot of raster workflows.
Image: California earthquake story
Powered by FME Notification Services
trigger workflows to run when new data becomes available
serve up custom reports automatically by email
FME cloud based
Stephanie Halpin
See FME UC 2017:
UAV presentation - Southern Co. (Renato, Steve & Dean)
https://www.safe.com/presentation/uavs-and-fme-powering-your-drone-and-data-with-fme/
FME and Disaster Response
https://www.safe.com/presentation/fme-for-disaster-response/