SlideShare une entreprise Scribd logo
Modernizing Hazard Data Feeds
at the Pacific Disaster Center
20
22
FME
User
Conference
● Introduction
● About Kontur and the Pacific Disaster Center (PDC)
● Review of current data processing methods at PDC
● Data modernization with FME
● Examples of FME in use at PDC
● Additional infrastructure changes with FME
● The path forward for PDC and FME
Agenda
20
22
FME
User
Conference
Jorma Rodieck
o Worked in the IT field for the last 20+ years
o Took my first GIS class at UC Santa Cruz in 1996
o Worked with Kontur last seven years primarily with the PDC
o Strong technical background with a focus on GIS related
projects the last 12 years
o Bachelor’s degree in Biology from UC Santa Cruz
o Post-Baccalaureate Certificate in Geographic Information
Systems from Penn State University
o AWS Certified Developer – Associate
o From the Pacific NW and currently living in Maui, Hawaii
Project Manager / Developer at Kontur
20
22
FME
User
Conference
20
22
FME
User
Conference
Kontur is a company specialized in disaster management and geoinformation.
Our solutions include datasets and program products that are used in
emergencies.
Kontur helps customers:
● track events in real-time
● estimate risks and impact
● get notified about changes in the course of events
About Kontur
20
22
FME
User
Conference
20
22
FME
User
Conference
© Copyright PDC 2006-2022
Who is PDC?
Staff and industry experts
of disasters
8 Global
Locations
Thousands
More than 60
Managed by
University of
Hawaii
since 2006
200+
Partner projects and
engagements
25
YEARS
OF SERVICE
+
Since 1996
Over 70
Partner Countries
PDC
Headquarters
www.pdc.org
20
22
FME
User
Conference
© Copyright PDC 2006-2022 www.pdc.org
Who uses our systems?
Our partners at-a-glance
National Governments
United States Agencies International & Regional Orgs
20
22
FME
User
Conference
© Copyright PDC 2006-2022
Our Global Mission
www.pdc.org
To provide evidence-based research, applied science, information, and cutting-edge technology
solutions for more effective disaster risk reduction (DRR) policies, practices, and humanitarian
assistance and disaster relief (HA/DR) operations in the Asia Pacific region and beyond.
20
22
FME
User
Conference
© Copyright PDC 2006-2022 www.pdc.org
Disasters happen fast. Make better decisions faster.
Real-time and
historical hazards
Anticipated
impacts
Collaboration and
sharing
Live
cameras
Smart AlertTM
early warning
Asset
protection
Unrivaled
Situational
Awareness
20
22
FME
User
Conference
© Copyright PDC 2006-2022
Disasters happen fast. Make better decisions faster.
6,500+
DATA LAYERS
20
22
FME
User
Conference
© Copyright PDC 2006-2022
Disasters happen fast. Make better decisions faster.
Disaster Alert™
disasteralert.pdc.org
DisasterAWARE Pro®
disasteraware.pdc.org
Customizable Solutions
Unlock the power of your own data.
Disaster Alert is a free, public
application that provides individuals,
families, and their loved ones with the
information they need to stay safe
anywhere in the world. Built on PDC s
DisasterAWARE® platform, Disaster Alert
offers real-time updates on hazards as
they are unfolding around the globe.
DisasterAWARE is the most powerful and
reliable early warning and multi-
hazard monitoring platform. It is used
by thousands of the most demanding
government customers and humanitarian
assistance organizations around the globe.
It supports disaster risk reduction and best
practices throughout all phases of disaster
management—providing early warning,
multi-hazard monitoring, modeled impact
assessments, and the largest collection of
scientifically verified geospatial data.
We provide out-of-the box and customized
disaster management solutions for early
warning and decision support, built on our
industry-leading disaster risk intelligence
platform, DisasterAWARE.
Learn more about our tailored DisasterAWARE
solutions.
Informing Decision-Making Worldwide
20
22
FME
User
Conference
How is PDC using FME?
PDC’s Implementation:
o PDC aggregates various hazard and data feeds
from a large variety of sources
o The data aggregation work is primarily done by a
custom Java application built by PDC 10+ years ago
o Issues arise with this process due to:
o Complexity of the Java application
o Monolithic architecture
o Scalability issues
o Limited knowledge of how the application is put
together and operates
o High level of programming experience needed
to make changes to the application
20
22
FME
User
Conference
How is PDC using FME?
Modernization of PDC’s Hazard Data Aggregation:
o Installation of FME Desktop and FME Server
o Test and Production Environments running FME Server
o Move automation process from legacy Java application
to FME Server
o Review of all logic and algorithms used in data
collection and aggregation for each hazard type
o Rewrite data aggregation and processing logic using
FME
o Review of FME Desktop results to confirm they
conform to expected known values
o Publish to FME Server Test
o Integrate into PDC products and services
20
22
FME
User
Conference
2) Publish to FME Server
FME Data Process at PDC
1) Workbench creation using FME Desktop
3) FME Server writes data
to PostgreSQL DB
4) Map document creation
using data in SQL DB
5) Publish map document to create map
service in ArcGIS Server
From FME to PDC’s DisasterAWARE Application
6) Create new layer in
DisasterAWARE
20
22
FME
User
Conference
Example: Wildfire Exposure
Providing real-time hazard information
1. Connect to NIFC’s API for real-time fire perimeter information
2. Connect the fire perimeter to real-time fire hazard information
3. Build products for information decision-makers
20
22
FME
User
Conference
Example: Public Weather Alerts for Canada
1. PDC Monitors an RSS feed and
processes the data every time
a new alert is issued
2. Each hazard received contains
polygon geometry and
attributes such as title, author,
description, event, affected
areas, severity.
3. Processing is done within FME
and a new hazard is created
within DisasterAWARE if the
specific criteria is met for that
hazard type.
4. Additional processing done in
FME to create exposure
impact data (see next slide)
20
22
FME
User
Conference
Example:
FME Integration into PDC’s Hazard Brief
1. Hazard Brief is a DisasterAWARE
component that contains the latest
hazard impact data for a specific
hazard
2. Hazard Brief updates the hazard
data shown as new data comes
available
3. FME is utilized in Hazard Brief
processing for calculating the
exposure impact from the hazard
polygon and providing the
exposure data to the Hazard Brief
application
20
22
FME
User
Conference
Additional FME Infrastructure Changes
Monitoring and Alerting
o Leveraging the FME Server API and Google Sheets:
o Daily FME job to compare Test and Prod FME Servers to make sure that
workbenches, automations, schedules, etc. are in sync
o Daily FME job to confirm that workspaces on the server are the same as the
files checked into the PDC Git repo
o Run checks on critical jobs to report if they have failed x times in a row
o Example: It is ok for a job that runs every 10min to fail once, but not ok for a job that
runs once a month to fail once
o Run regular checks on queues to ensure jobs are processed efficiently
o Integration into PDC’s system monitoring tool (Site24x7) to provide immediate
alerting for critical issues
o Slack and email notifications for job failures and configuration changes
20
22
FME
User
Conference
How has FME improved life at PDC?
o FME helps remove much of the complexity when making
changes to existing code
o Less dependency on highly technical Java developers
o No longer need to redeploy the whole application when
making changes
o Improved reliability: A bug in an FME workspace will not
impact other workspaces
o Separation of jobs into individual workbenches makes it
easier to understand each component
o Knowledge of how each workbench operates can be
easily shared
o Lowers the barrier to entry for skills required to develop
o Easier to debug and rerun jobs when something is not
working
20
22
FME
User
Conference
The Path Forward for PDC and FME
PDC plans to migrate all existing data processes
from their existing monolithic Java application to
FME. The logic and data science used for
aggregating hazard data for each hazard type will
be reviewed, and then after the logic and
algorithms are confirmed all data processing for
the hazard type will be moved to FME. Once
completed for all hazard types, the existing Java
application will be removed from production.
20
22
FME
User
Conference
Questions? Comments?
20
22
FME
User
Conference
Resources
Pacific Disaster Center: https://pdc.org
Kontur, Inc: https://kontur.io
DisasterAlert (PDC’s public DisasterAWARE application): https://disasteralert.pdc.org
DisasterAlert Mobile:
Thank You!
jorma@kontur.io
20
22
FME
User
Conference
Please Share Your Session Feedback
Log in & navigate to the
agenda. Select the
session you attended
Scroll down to “Session
Feedback”
Download EventMobi app
Event code: fmeuc
Click “Launch App”
Fill out the survey
and submit

Contenu connexe

Similaire à Modernizing Hazard Data Feeds at the PDC

Resume_Hegel_updated_Network (1)
Resume_Hegel_updated_Network (1)Resume_Hegel_updated_Network (1)
Resume_Hegel_updated_Network (1)
Hegel Durocher
 
DSD-INT 2019 The Incident Management Forecasting System (IMFS) for England - ...
DSD-INT 2019 The Incident Management Forecasting System (IMFS) for England - ...DSD-INT 2019 The Incident Management Forecasting System (IMFS) for England - ...
DSD-INT 2019 The Incident Management Forecasting System (IMFS) for England - ...
Deltares
 
Resume_Padmaja
Resume_PadmajaResume_Padmaja
Resume_Padmaja
Padmaja Samantara
 
Automated Application Integration with FME & Cityworks Webinar
Automated Application Integration with FME & Cityworks WebinarAutomated Application Integration with FME & Cityworks Webinar
Automated Application Integration with FME & Cityworks Webinar
Safe Software
 
How Utilities can Improve Operations using Data Integration Workflows
How Utilities can Improve Operations using Data Integration WorkflowsHow Utilities can Improve Operations using Data Integration Workflows
How Utilities can Improve Operations using Data Integration Workflows
Safe Software
 
Mud Reporting: Streamlined Process and Data Management
Mud Reporting: Streamlined Process and Data ManagementMud Reporting: Streamlined Process and Data Management
Mud Reporting: Streamlined Process and Data Management
pvisoftware
 
Michael Haftel Resume-2016
Michael Haftel Resume-2016Michael Haftel Resume-2016
Michael Haftel Resume-2016
Michael Haftel
 
Amer-CV-2014_10
Amer-CV-2014_10Amer-CV-2014_10
Amer-CV-2014_10
amer salameh
 
8 Ways Utility Networks Can Meet Data Demands
8 Ways Utility Networks Can Meet Data Demands8 Ways Utility Networks Can Meet Data Demands
8 Ways Utility Networks Can Meet Data Demands
Safe Software
 
Amit Chaubey
Amit ChaubeyAmit Chaubey
Amit Chaubey
Amit Chaubey
 
Application Specialist
Application SpecialistApplication Specialist
Application Specialist
Brian Machado
 
cv15
cv15cv15
Resume2_Roy Anderson_online version
Resume2_Roy Anderson_online versionResume2_Roy Anderson_online version
Resume2_Roy Anderson_online version
Roy Anderson
 
Torben Verdich Resume TS-SCI
Torben Verdich Resume TS-SCITorben Verdich Resume TS-SCI
Torben Verdich Resume TS-SCI
Torben Verdich, MCITP
 
Current_CV
Current_CVCurrent_CV
Mindtree's performance engineering services.
Mindtree's performance engineering services.Mindtree's performance engineering services.
Mindtree's performance engineering services.
Mindtree Ltd.
 
Avoiding disaster recovery disasters
Avoiding disaster recovery disastersAvoiding disaster recovery disasters
Avoiding disaster recovery disasters
Alexandra Matthiesen
 
Avoiding disaster recovery disasters
Avoiding disaster recovery disastersAvoiding disaster recovery disasters
Avoiding disaster recovery disasters
Veritas Technologies LLC
 
Production Deployment of the New Integrator Project C.docx
Production Deployment of the New Integrator  Project C.docxProduction Deployment of the New Integrator  Project C.docx
Production Deployment of the New Integrator Project C.docx
briancrawford30935
 
Chuck_Roden_Resume
Chuck_Roden_ResumeChuck_Roden_Resume
Chuck_Roden_Resume
Chuck Roden
 

Similaire à Modernizing Hazard Data Feeds at the PDC (20)

Resume_Hegel_updated_Network (1)
Resume_Hegel_updated_Network (1)Resume_Hegel_updated_Network (1)
Resume_Hegel_updated_Network (1)
 
DSD-INT 2019 The Incident Management Forecasting System (IMFS) for England - ...
DSD-INT 2019 The Incident Management Forecasting System (IMFS) for England - ...DSD-INT 2019 The Incident Management Forecasting System (IMFS) for England - ...
DSD-INT 2019 The Incident Management Forecasting System (IMFS) for England - ...
 
Resume_Padmaja
Resume_PadmajaResume_Padmaja
Resume_Padmaja
 
Automated Application Integration with FME & Cityworks Webinar
Automated Application Integration with FME & Cityworks WebinarAutomated Application Integration with FME & Cityworks Webinar
Automated Application Integration with FME & Cityworks Webinar
 
How Utilities can Improve Operations using Data Integration Workflows
How Utilities can Improve Operations using Data Integration WorkflowsHow Utilities can Improve Operations using Data Integration Workflows
How Utilities can Improve Operations using Data Integration Workflows
 
Mud Reporting: Streamlined Process and Data Management
Mud Reporting: Streamlined Process and Data ManagementMud Reporting: Streamlined Process and Data Management
Mud Reporting: Streamlined Process and Data Management
 
Michael Haftel Resume-2016
Michael Haftel Resume-2016Michael Haftel Resume-2016
Michael Haftel Resume-2016
 
Amer-CV-2014_10
Amer-CV-2014_10Amer-CV-2014_10
Amer-CV-2014_10
 
8 Ways Utility Networks Can Meet Data Demands
8 Ways Utility Networks Can Meet Data Demands8 Ways Utility Networks Can Meet Data Demands
8 Ways Utility Networks Can Meet Data Demands
 
Amit Chaubey
Amit ChaubeyAmit Chaubey
Amit Chaubey
 
Application Specialist
Application SpecialistApplication Specialist
Application Specialist
 
cv15
cv15cv15
cv15
 
Resume2_Roy Anderson_online version
Resume2_Roy Anderson_online versionResume2_Roy Anderson_online version
Resume2_Roy Anderson_online version
 
Torben Verdich Resume TS-SCI
Torben Verdich Resume TS-SCITorben Verdich Resume TS-SCI
Torben Verdich Resume TS-SCI
 
Current_CV
Current_CVCurrent_CV
Current_CV
 
Mindtree's performance engineering services.
Mindtree's performance engineering services.Mindtree's performance engineering services.
Mindtree's performance engineering services.
 
Avoiding disaster recovery disasters
Avoiding disaster recovery disastersAvoiding disaster recovery disasters
Avoiding disaster recovery disasters
 
Avoiding disaster recovery disasters
Avoiding disaster recovery disastersAvoiding disaster recovery disasters
Avoiding disaster recovery disasters
 
Production Deployment of the New Integrator Project C.docx
Production Deployment of the New Integrator  Project C.docxProduction Deployment of the New Integrator  Project C.docx
Production Deployment of the New Integrator Project C.docx
 
Chuck_Roden_Resume
Chuck_Roden_ResumeChuck_Roden_Resume
Chuck_Roden_Resume
 

Plus de Safe Software

Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
Safe Software
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
Safe Software
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
Safe Software
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action:  Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action:  Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
Safe Software
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Safe Software
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data Streams
Safe Software
 
The Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemThe Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data Ecosystem
Safe Software
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Safe Software
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Safe Software
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
Safe Software
 
Mastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISMastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GIS
Safe Software
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Safe Software
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Safe Software
 
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Safe Software
 
Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...
Safe Software
 

Plus de Safe Software (20)

Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action:  Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action:  Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data Streams
 
The Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemThe Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data Ecosystem
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Mastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISMastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GIS
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & Esri
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
 
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI Technology
 
Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...
 

Dernier

AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
IndexBug
 
CAKE: Sharing Slices of Confidential Data on Blockchain
CAKE: Sharing Slices of Confidential Data on BlockchainCAKE: Sharing Slices of Confidential Data on Blockchain
CAKE: Sharing Slices of Confidential Data on Blockchain
Claudio Di Ciccio
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
Aftab Hussain
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
Wouter Lemaire
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
Claudio Di Ciccio
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
Jason Packer
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
Ivanti
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
DianaGray10
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
Mariano Tinti
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
Daiki Mogmet Ito
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Matthew Sinclair
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
Zilliz
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Speck&Tech
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
innovationoecd
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
SitimaJohn
 
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdfAI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
Techgropse Pvt.Ltd.
 

Dernier (20)

AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
 
CAKE: Sharing Slices of Confidential Data on Blockchain
CAKE: Sharing Slices of Confidential Data on BlockchainCAKE: Sharing Slices of Confidential Data on Blockchain
CAKE: Sharing Slices of Confidential Data on Blockchain
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
 
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdfAI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
 

Modernizing Hazard Data Feeds at the PDC

  • 1. Modernizing Hazard Data Feeds at the Pacific Disaster Center
  • 2. 20 22 FME User Conference ● Introduction ● About Kontur and the Pacific Disaster Center (PDC) ● Review of current data processing methods at PDC ● Data modernization with FME ● Examples of FME in use at PDC ● Additional infrastructure changes with FME ● The path forward for PDC and FME Agenda
  • 3. 20 22 FME User Conference Jorma Rodieck o Worked in the IT field for the last 20+ years o Took my first GIS class at UC Santa Cruz in 1996 o Worked with Kontur last seven years primarily with the PDC o Strong technical background with a focus on GIS related projects the last 12 years o Bachelor’s degree in Biology from UC Santa Cruz o Post-Baccalaureate Certificate in Geographic Information Systems from Penn State University o AWS Certified Developer – Associate o From the Pacific NW and currently living in Maui, Hawaii Project Manager / Developer at Kontur
  • 5. 20 22 FME User Conference Kontur is a company specialized in disaster management and geoinformation. Our solutions include datasets and program products that are used in emergencies. Kontur helps customers: ● track events in real-time ● estimate risks and impact ● get notified about changes in the course of events About Kontur
  • 7. 20 22 FME User Conference © Copyright PDC 2006-2022 Who is PDC? Staff and industry experts of disasters 8 Global Locations Thousands More than 60 Managed by University of Hawaii since 2006 200+ Partner projects and engagements 25 YEARS OF SERVICE + Since 1996 Over 70 Partner Countries PDC Headquarters www.pdc.org
  • 8. 20 22 FME User Conference © Copyright PDC 2006-2022 www.pdc.org Who uses our systems? Our partners at-a-glance National Governments United States Agencies International & Regional Orgs
  • 9. 20 22 FME User Conference © Copyright PDC 2006-2022 Our Global Mission www.pdc.org To provide evidence-based research, applied science, information, and cutting-edge technology solutions for more effective disaster risk reduction (DRR) policies, practices, and humanitarian assistance and disaster relief (HA/DR) operations in the Asia Pacific region and beyond.
  • 10. 20 22 FME User Conference © Copyright PDC 2006-2022 www.pdc.org Disasters happen fast. Make better decisions faster. Real-time and historical hazards Anticipated impacts Collaboration and sharing Live cameras Smart AlertTM early warning Asset protection Unrivaled Situational Awareness
  • 11. 20 22 FME User Conference © Copyright PDC 2006-2022 Disasters happen fast. Make better decisions faster. 6,500+ DATA LAYERS
  • 12. 20 22 FME User Conference © Copyright PDC 2006-2022 Disasters happen fast. Make better decisions faster. Disaster Alert™ disasteralert.pdc.org DisasterAWARE Pro® disasteraware.pdc.org Customizable Solutions Unlock the power of your own data. Disaster Alert is a free, public application that provides individuals, families, and their loved ones with the information they need to stay safe anywhere in the world. Built on PDC s DisasterAWARE® platform, Disaster Alert offers real-time updates on hazards as they are unfolding around the globe. DisasterAWARE is the most powerful and reliable early warning and multi- hazard monitoring platform. It is used by thousands of the most demanding government customers and humanitarian assistance organizations around the globe. It supports disaster risk reduction and best practices throughout all phases of disaster management—providing early warning, multi-hazard monitoring, modeled impact assessments, and the largest collection of scientifically verified geospatial data. We provide out-of-the box and customized disaster management solutions for early warning and decision support, built on our industry-leading disaster risk intelligence platform, DisasterAWARE. Learn more about our tailored DisasterAWARE solutions. Informing Decision-Making Worldwide
  • 13. 20 22 FME User Conference How is PDC using FME? PDC’s Implementation: o PDC aggregates various hazard and data feeds from a large variety of sources o The data aggregation work is primarily done by a custom Java application built by PDC 10+ years ago o Issues arise with this process due to: o Complexity of the Java application o Monolithic architecture o Scalability issues o Limited knowledge of how the application is put together and operates o High level of programming experience needed to make changes to the application
  • 14. 20 22 FME User Conference How is PDC using FME? Modernization of PDC’s Hazard Data Aggregation: o Installation of FME Desktop and FME Server o Test and Production Environments running FME Server o Move automation process from legacy Java application to FME Server o Review of all logic and algorithms used in data collection and aggregation for each hazard type o Rewrite data aggregation and processing logic using FME o Review of FME Desktop results to confirm they conform to expected known values o Publish to FME Server Test o Integrate into PDC products and services
  • 15. 20 22 FME User Conference 2) Publish to FME Server FME Data Process at PDC 1) Workbench creation using FME Desktop 3) FME Server writes data to PostgreSQL DB 4) Map document creation using data in SQL DB 5) Publish map document to create map service in ArcGIS Server From FME to PDC’s DisasterAWARE Application 6) Create new layer in DisasterAWARE
  • 16. 20 22 FME User Conference Example: Wildfire Exposure Providing real-time hazard information 1. Connect to NIFC’s API for real-time fire perimeter information 2. Connect the fire perimeter to real-time fire hazard information 3. Build products for information decision-makers
  • 17. 20 22 FME User Conference Example: Public Weather Alerts for Canada 1. PDC Monitors an RSS feed and processes the data every time a new alert is issued 2. Each hazard received contains polygon geometry and attributes such as title, author, description, event, affected areas, severity. 3. Processing is done within FME and a new hazard is created within DisasterAWARE if the specific criteria is met for that hazard type. 4. Additional processing done in FME to create exposure impact data (see next slide)
  • 18. 20 22 FME User Conference Example: FME Integration into PDC’s Hazard Brief 1. Hazard Brief is a DisasterAWARE component that contains the latest hazard impact data for a specific hazard 2. Hazard Brief updates the hazard data shown as new data comes available 3. FME is utilized in Hazard Brief processing for calculating the exposure impact from the hazard polygon and providing the exposure data to the Hazard Brief application
  • 19. 20 22 FME User Conference Additional FME Infrastructure Changes Monitoring and Alerting o Leveraging the FME Server API and Google Sheets: o Daily FME job to compare Test and Prod FME Servers to make sure that workbenches, automations, schedules, etc. are in sync o Daily FME job to confirm that workspaces on the server are the same as the files checked into the PDC Git repo o Run checks on critical jobs to report if they have failed x times in a row o Example: It is ok for a job that runs every 10min to fail once, but not ok for a job that runs once a month to fail once o Run regular checks on queues to ensure jobs are processed efficiently o Integration into PDC’s system monitoring tool (Site24x7) to provide immediate alerting for critical issues o Slack and email notifications for job failures and configuration changes
  • 20. 20 22 FME User Conference How has FME improved life at PDC? o FME helps remove much of the complexity when making changes to existing code o Less dependency on highly technical Java developers o No longer need to redeploy the whole application when making changes o Improved reliability: A bug in an FME workspace will not impact other workspaces o Separation of jobs into individual workbenches makes it easier to understand each component o Knowledge of how each workbench operates can be easily shared o Lowers the barrier to entry for skills required to develop o Easier to debug and rerun jobs when something is not working
  • 21. 20 22 FME User Conference The Path Forward for PDC and FME PDC plans to migrate all existing data processes from their existing monolithic Java application to FME. The logic and data science used for aggregating hazard data for each hazard type will be reviewed, and then after the logic and algorithms are confirmed all data processing for the hazard type will be moved to FME. Once completed for all hazard types, the existing Java application will be removed from production.
  • 23. 20 22 FME User Conference Resources Pacific Disaster Center: https://pdc.org Kontur, Inc: https://kontur.io DisasterAlert (PDC’s public DisasterAWARE application): https://disasteralert.pdc.org DisasterAlert Mobile:
  • 25. 20 22 FME User Conference Please Share Your Session Feedback Log in & navigate to the agenda. Select the session you attended Scroll down to “Session Feedback” Download EventMobi app Event code: fmeuc Click “Launch App” Fill out the survey and submit