Ce diaporama a bien été signalé.
Le téléchargement de votre SlideShare est en cours. ×

Modernizing Hazard Data Feeds at the PDC

Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité

Consultez-les par la suite

1 sur 25 Publicité

Modernizing Hazard Data Feeds at the PDC

Télécharger pour lire hors ligne

In this presentation, we will highlight how moving from a custom build-in-house solution for aggregating data feeds to using FME Server and Desktop helped to support disaster risk reduction efforts.

In this presentation, we will highlight how moving from a custom build-in-house solution for aggregating data feeds to using FME Server and Desktop helped to support disaster risk reduction efforts.

Publicité
Publicité

Plus De Contenu Connexe

Plus par Safe Software (20)

Plus récents (20)

Publicité

Modernizing Hazard Data Feeds at the PDC

  1. 1. Modernizing Hazard Data Feeds at the Pacific Disaster Center
  2. 2. 20 22 FME User Conference ● Introduction ● About Kontur and the Pacific Disaster Center (PDC) ● Review of current data processing methods at PDC ● Data modernization with FME ● Examples of FME in use at PDC ● Additional infrastructure changes with FME ● The path forward for PDC and FME Agenda
  3. 3. 20 22 FME User Conference Jorma Rodieck o Worked in the IT field for the last 20+ years o Took my first GIS class at UC Santa Cruz in 1996 o Worked with Kontur last seven years primarily with the PDC o Strong technical background with a focus on GIS related projects the last 12 years o Bachelor’s degree in Biology from UC Santa Cruz o Post-Baccalaureate Certificate in Geographic Information Systems from Penn State University o AWS Certified Developer – Associate o From the Pacific NW and currently living in Maui, Hawaii Project Manager / Developer at Kontur
  4. 4. 20 22 FME User Conference
  5. 5. 20 22 FME User Conference Kontur is a company specialized in disaster management and geoinformation. Our solutions include datasets and program products that are used in emergencies. Kontur helps customers: ● track events in real-time ● estimate risks and impact ● get notified about changes in the course of events About Kontur
  6. 6. 20 22 FME User Conference
  7. 7. 20 22 FME User Conference © Copyright PDC 2006-2022 Who is PDC? Staff and industry experts of disasters 8 Global Locations Thousands More than 60 Managed by University of Hawaii since 2006 200+ Partner projects and engagements 25 YEARS OF SERVICE + Since 1996 Over 70 Partner Countries PDC Headquarters www.pdc.org
  8. 8. 20 22 FME User Conference © Copyright PDC 2006-2022 www.pdc.org Who uses our systems? Our partners at-a-glance National Governments United States Agencies International & Regional Orgs
  9. 9. 20 22 FME User Conference © Copyright PDC 2006-2022 Our Global Mission www.pdc.org To provide evidence-based research, applied science, information, and cutting-edge technology solutions for more effective disaster risk reduction (DRR) policies, practices, and humanitarian assistance and disaster relief (HA/DR) operations in the Asia Pacific region and beyond.
  10. 10. 20 22 FME User Conference © Copyright PDC 2006-2022 www.pdc.org Disasters happen fast. Make better decisions faster. Real-time and historical hazards Anticipated impacts Collaboration and sharing Live cameras Smart AlertTM early warning Asset protection Unrivaled Situational Awareness
  11. 11. 20 22 FME User Conference © Copyright PDC 2006-2022 Disasters happen fast. Make better decisions faster. 6,500+ DATA LAYERS
  12. 12. 20 22 FME User Conference © Copyright PDC 2006-2022 Disasters happen fast. Make better decisions faster. Disaster Alert™ disasteralert.pdc.org DisasterAWARE Pro® disasteraware.pdc.org Customizable Solutions Unlock the power of your own data. Disaster Alert is a free, public application that provides individuals, families, and their loved ones with the information they need to stay safe anywhere in the world. Built on PDC s DisasterAWARE® platform, Disaster Alert offers real-time updates on hazards as they are unfolding around the globe. DisasterAWARE is the most powerful and reliable early warning and multi- hazard monitoring platform. It is used by thousands of the most demanding government customers and humanitarian assistance organizations around the globe. It supports disaster risk reduction and best practices throughout all phases of disaster management—providing early warning, multi-hazard monitoring, modeled impact assessments, and the largest collection of scientifically verified geospatial data. We provide out-of-the box and customized disaster management solutions for early warning and decision support, built on our industry-leading disaster risk intelligence platform, DisasterAWARE. Learn more about our tailored DisasterAWARE solutions. Informing Decision-Making Worldwide
  13. 13. 20 22 FME User Conference How is PDC using FME? PDC’s Implementation: o PDC aggregates various hazard and data feeds from a large variety of sources o The data aggregation work is primarily done by a custom Java application built by PDC 10+ years ago o Issues arise with this process due to: o Complexity of the Java application o Monolithic architecture o Scalability issues o Limited knowledge of how the application is put together and operates o High level of programming experience needed to make changes to the application
  14. 14. 20 22 FME User Conference How is PDC using FME? Modernization of PDC’s Hazard Data Aggregation: o Installation of FME Desktop and FME Server o Test and Production Environments running FME Server o Move automation process from legacy Java application to FME Server o Review of all logic and algorithms used in data collection and aggregation for each hazard type o Rewrite data aggregation and processing logic using FME o Review of FME Desktop results to confirm they conform to expected known values o Publish to FME Server Test o Integrate into PDC products and services
  15. 15. 20 22 FME User Conference 2) Publish to FME Server FME Data Process at PDC 1) Workbench creation using FME Desktop 3) FME Server writes data to PostgreSQL DB 4) Map document creation using data in SQL DB 5) Publish map document to create map service in ArcGIS Server From FME to PDC’s DisasterAWARE Application 6) Create new layer in DisasterAWARE
  16. 16. 20 22 FME User Conference Example: Wildfire Exposure Providing real-time hazard information 1. Connect to NIFC’s API for real-time fire perimeter information 2. Connect the fire perimeter to real-time fire hazard information 3. Build products for information decision-makers
  17. 17. 20 22 FME User Conference Example: Public Weather Alerts for Canada 1. PDC Monitors an RSS feed and processes the data every time a new alert is issued 2. Each hazard received contains polygon geometry and attributes such as title, author, description, event, affected areas, severity. 3. Processing is done within FME and a new hazard is created within DisasterAWARE if the specific criteria is met for that hazard type. 4. Additional processing done in FME to create exposure impact data (see next slide)
  18. 18. 20 22 FME User Conference Example: FME Integration into PDC’s Hazard Brief 1. Hazard Brief is a DisasterAWARE component that contains the latest hazard impact data for a specific hazard 2. Hazard Brief updates the hazard data shown as new data comes available 3. FME is utilized in Hazard Brief processing for calculating the exposure impact from the hazard polygon and providing the exposure data to the Hazard Brief application
  19. 19. 20 22 FME User Conference Additional FME Infrastructure Changes Monitoring and Alerting o Leveraging the FME Server API and Google Sheets: o Daily FME job to compare Test and Prod FME Servers to make sure that workbenches, automations, schedules, etc. are in sync o Daily FME job to confirm that workspaces on the server are the same as the files checked into the PDC Git repo o Run checks on critical jobs to report if they have failed x times in a row o Example: It is ok for a job that runs every 10min to fail once, but not ok for a job that runs once a month to fail once o Run regular checks on queues to ensure jobs are processed efficiently o Integration into PDC’s system monitoring tool (Site24x7) to provide immediate alerting for critical issues o Slack and email notifications for job failures and configuration changes
  20. 20. 20 22 FME User Conference How has FME improved life at PDC? o FME helps remove much of the complexity when making changes to existing code o Less dependency on highly technical Java developers o No longer need to redeploy the whole application when making changes o Improved reliability: A bug in an FME workspace will not impact other workspaces o Separation of jobs into individual workbenches makes it easier to understand each component o Knowledge of how each workbench operates can be easily shared o Lowers the barrier to entry for skills required to develop o Easier to debug and rerun jobs when something is not working
  21. 21. 20 22 FME User Conference The Path Forward for PDC and FME PDC plans to migrate all existing data processes from their existing monolithic Java application to FME. The logic and data science used for aggregating hazard data for each hazard type will be reviewed, and then after the logic and algorithms are confirmed all data processing for the hazard type will be moved to FME. Once completed for all hazard types, the existing Java application will be removed from production.
  22. 22. 20 22 FME User Conference Questions? Comments?
  23. 23. 20 22 FME User Conference Resources Pacific Disaster Center: https://pdc.org Kontur, Inc: https://kontur.io DisasterAlert (PDC’s public DisasterAWARE application): https://disasteralert.pdc.org DisasterAlert Mobile:
  24. 24. Thank You! jorma@kontur.io
  25. 25. 20 22 FME User Conference Please Share Your Session Feedback Log in & navigate to the agenda. Select the session you attended Scroll down to “Session Feedback” Download EventMobi app Event code: fmeuc Click “Launch App” Fill out the survey and submit

×