SlideShare une entreprise Scribd logo
1  sur  13
Simplifying the
Complex:
Serving data from
Pipeline Data Models
Peter Veenstra
TRC
Introduction
Peter Veenstra
Problem or Challenge?
GIS is difficult, users want data …
Non-GIS Professionals need access to
data to do their jobs  in XLS
Industry Standard Pipeline Data Models
are complex  Linear Referencing
Data stored in Silos – hard to synchronize
or coordinate …
Model, Load, As-Built – Re-build Silo 1 Silo 2
Big Gnarly Mess
1000’s if not 10’s of 10000’s miles of data
overlaid on a single route.
Linear events (in 100’s of tables) related
to each other by common position along
the pipeline but without formal relates.
Potentially 7+ million segments to process.
X
Good
Well Drained
Test 1 - Current
MAOP
T
MFL
Type 1
Type 2b Type 2a
X X X X X X
Poor
Poorly Drained
Test 2 - Current
Solution
Build Toolboxes using
FME
Move toolboxes to
server.
Let the data be
consumed.
Toolboxes
FME Toolkit for Pipeline
Data Models (PDM)
FME Toolkit for Linear
Referencing (LRM)
FME Toolkit for
Database Integration
(DIM)
XYZM
PL-AAA
XYZM
XYZM
Sum
Latest
Merge
Move to FME
Server
What is Server?
Why is it better than
scheduled tasks and
scripts?
Planned Migration
(build, then optimize)
Aggregation
Data Warehouse
FME Server
Build a data ware-house
aggregating data stored
in different silos …
- Shared coordinate
plane
- Hierarchy to
Hierarchy translation
- Dynamic
Segmentation into
massive table
PODS Silo 1 Silo 2 Silo 3
X
Good
Well Drained
Test 1 - Current
MAOP
T
MFL
Type 1
Type 2b Type 2a
X X X X X X
Poor
Poorly Drained
Test 2 - Current
FME Server
Services Bus
Query Engine
Tableau
Hierarchy Finder
Data Extractor
Data Delta Finder/Synchronizer
Data Health
2016: Status and Progress Tracker
A lot to get
your head
around …
Subscribe, publish,
consume … respond.
Services-oriented rather
than task-oriented
Event-driven rather than
procedural …
GIS is difficult, users
want data …
"Good data leads to
good knowledge, and
good knowledge solves
problems."
Good data eh!
Thank you!
Peter Veenstra, TRC
pveenstra@trcsolutions.com
816-820-7841

Contenu connexe

En vedette

El Primado de Pedro
El Primado de PedroEl Primado de Pedro
El Primado de PedroMiguel Angel
 
Planta y secciones de una cubierta de igual pendiente.
Planta y secciones de una cubierta de igual pendiente.Planta y secciones de una cubierta de igual pendiente.
Planta y secciones de una cubierta de igual pendiente.Jose Antonio Fuentes
 
Beautiful Nature
Beautiful NatureBeautiful Nature
Beautiful NatureRenny
 
Trabajo final Proyeto Emf 2013 J.T
Trabajo final Proyeto Emf 2013 J.TTrabajo final Proyeto Emf 2013 J.T
Trabajo final Proyeto Emf 2013 J.TJöhn Cädc
 
Alibaba: The Figures
Alibaba: The FiguresAlibaba: The Figures
Alibaba: The FiguresStartup China
 
La maquina de vapor. Rosario Carmona Marín.
La maquina de vapor. Rosario Carmona Marín.La maquina de vapor. Rosario Carmona Marín.
La maquina de vapor. Rosario Carmona Marín.RosarioCarmonaMarin98
 
Web 2.0: It's All about Social Networking
Web 2.0: It's All about Social NetworkingWeb 2.0: It's All about Social Networking
Web 2.0: It's All about Social Networkingeoshea
 
هلا النجار
هلا النجارهلا النجار
هلا النجارHala Najjar
 
Couleurs Du Monde
Couleurs Du MondeCouleurs Du Monde
Couleurs Du MondeRuper
 
Mind map title sequnace ideas
Mind map title sequnace ideasMind map title sequnace ideas
Mind map title sequnace ideasJazmine Sims
 
Coperti sortam agnes
Coperti sortam agnesCoperti sortam agnes
Coperti sortam agnesNicole France
 

En vedette (20)

El Primado de Pedro
El Primado de PedroEl Primado de Pedro
El Primado de Pedro
 
Por las avenidas del modelo educativo...
Por las avenidas del modelo educativo...Por las avenidas del modelo educativo...
Por las avenidas del modelo educativo...
 
Planta y secciones de una cubierta de igual pendiente.
Planta y secciones de una cubierta de igual pendiente.Planta y secciones de una cubierta de igual pendiente.
Planta y secciones de una cubierta de igual pendiente.
 
Albañilería
AlbañileríaAlbañilería
Albañilería
 
Llocs 3r
Llocs 3rLlocs 3r
Llocs 3r
 
HOJA PARROQUIAL Nº 105
HOJA PARROQUIAL Nº 105HOJA PARROQUIAL Nº 105
HOJA PARROQUIAL Nº 105
 
Beautiful Nature
Beautiful NatureBeautiful Nature
Beautiful Nature
 
Trabajo final Proyeto Emf 2013 J.T
Trabajo final Proyeto Emf 2013 J.TTrabajo final Proyeto Emf 2013 J.T
Trabajo final Proyeto Emf 2013 J.T
 
Alibaba: The Figures
Alibaba: The FiguresAlibaba: The Figures
Alibaba: The Figures
 
La maquina de vapor. Rosario Carmona Marín.
La maquina de vapor. Rosario Carmona Marín.La maquina de vapor. Rosario Carmona Marín.
La maquina de vapor. Rosario Carmona Marín.
 
Técnico LEÂO
Técnico LEÂOTécnico LEÂO
Técnico LEÂO
 
Web 2.0: It's All about Social Networking
Web 2.0: It's All about Social NetworkingWeb 2.0: It's All about Social Networking
Web 2.0: It's All about Social Networking
 
Apresentação1
Apresentação1Apresentação1
Apresentação1
 
Unit 16 tractors
Unit 16 tractorsUnit 16 tractors
Unit 16 tractors
 
هلا النجار
هلا النجارهلا النجار
هلا النجار
 
Web Link Analysis
Web Link AnalysisWeb Link Analysis
Web Link Analysis
 
Couleurs Du Monde
Couleurs Du MondeCouleurs Du Monde
Couleurs Du Monde
 
Mind map title sequnace ideas
Mind map title sequnace ideasMind map title sequnace ideas
Mind map title sequnace ideas
 
Coperti sortam agnes
Coperti sortam agnesCoperti sortam agnes
Coperti sortam agnes
 
Venice_la_nuite
Venice_la_nuiteVenice_la_nuite
Venice_la_nuite
 

Similaire à Simplifying the Complex: Serving Data from Pipeline Data Models

High performance digital predistortion for wideband RF power amplifiers
High performance digital predistortion for wideband RF power amplifiersHigh performance digital predistortion for wideband RF power amplifiers
High performance digital predistortion for wideband RF power amplifiersLei Guan (Phd, SM-IEEE)
 
Scalability20140226
Scalability20140226Scalability20140226
Scalability20140226Nick Kypreos
 
GoFFish - A Sub-graph centric framework for large scale graph analytics
GoFFish - A Sub-graph centric framework for large scale graph analyticsGoFFish - A Sub-graph centric framework for large scale graph analytics
GoFFish - A Sub-graph centric framework for large scale graph analyticscharithwiki
 
Update on Trinity System Procurement and Plans
Update on Trinity System Procurement and PlansUpdate on Trinity System Procurement and Plans
Update on Trinity System Procurement and Plansinside-BigData.com
 
TeraGrid Communication and Computation
TeraGrid Communication and ComputationTeraGrid Communication and Computation
TeraGrid Communication and ComputationTal Lavian Ph.D.
 
Broadband tech 2005
Broadband tech 2005Broadband tech 2005
Broadband tech 2005eaiti
 
Big Data Everywhere Chicago: High Performance Computing - Contributions Towar...
Big Data Everywhere Chicago: High Performance Computing - Contributions Towar...Big Data Everywhere Chicago: High Performance Computing - Contributions Towar...
Big Data Everywhere Chicago: High Performance Computing - Contributions Towar...BigDataEverywhere
 
Ling liu part 02:big graph processing
Ling liu part 02:big graph processingLing liu part 02:big graph processing
Ling liu part 02:big graph processingjins0618
 
Hw09 Hadoop Based Data Mining Platform For The Telecom Industry
Hw09   Hadoop Based Data Mining Platform For The Telecom IndustryHw09   Hadoop Based Data Mining Platform For The Telecom Industry
Hw09 Hadoop Based Data Mining Platform For The Telecom IndustryCloudera, Inc.
 
Digital Signal Processing Quiz
Digital Signal Processing QuizDigital Signal Processing Quiz
Digital Signal Processing QuizMelissa Luster
 
Streaming exa-scale data over 100Gbps networks
Streaming exa-scale data over 100Gbps networksStreaming exa-scale data over 100Gbps networks
Streaming exa-scale data over 100Gbps networksbalmanme
 
Experiences with High-bandwidth Networks
Experiences with High-bandwidth NetworksExperiences with High-bandwidth Networks
Experiences with High-bandwidth Networksbalmanme
 
FME Around the World (FME Trek Part 1): Ken Bragg - Safe Software FME World T...
FME Around the World (FME Trek Part 1): Ken Bragg - Safe Software FME World T...FME Around the World (FME Trek Part 1): Ken Bragg - Safe Software FME World T...
FME Around the World (FME Trek Part 1): Ken Bragg - Safe Software FME World T...IMGS
 
Optical Switching in the Datacenter
Optical Switching in the DatacenterOptical Switching in the Datacenter
Optical Switching in the DatacenterKostas Katrinis
 
Data on the Move: Transitioning from a Legacy Architecture to a Big Data Plat...
Data on the Move: Transitioning from a Legacy Architecture to a Big Data Plat...Data on the Move: Transitioning from a Legacy Architecture to a Big Data Plat...
Data on the Move: Transitioning from a Legacy Architecture to a Big Data Plat...MapR Technologies
 
Big learning 1.2
Big learning   1.2Big learning   1.2
Big learning 1.2Mohit Garg
 
2021 itu challenge_reinforcement_learning
2021 itu challenge_reinforcement_learning2021 itu challenge_reinforcement_learning
2021 itu challenge_reinforcement_learningLASSEMedia
 
Reflections on Almost Two Decades of Research into Stream Processing
Reflections on Almost Two Decades of Research into Stream ProcessingReflections on Almost Two Decades of Research into Stream Processing
Reflections on Almost Two Decades of Research into Stream ProcessingKyumars Sheykh Esmaili
 

Similaire à Simplifying the Complex: Serving Data from Pipeline Data Models (20)

High performance digital predistortion for wideband RF power amplifiers
High performance digital predistortion for wideband RF power amplifiersHigh performance digital predistortion for wideband RF power amplifiers
High performance digital predistortion for wideband RF power amplifiers
 
Scalability20140226
Scalability20140226Scalability20140226
Scalability20140226
 
GoFFish - A Sub-graph centric framework for large scale graph analytics
GoFFish - A Sub-graph centric framework for large scale graph analyticsGoFFish - A Sub-graph centric framework for large scale graph analytics
GoFFish - A Sub-graph centric framework for large scale graph analytics
 
Update on Trinity System Procurement and Plans
Update on Trinity System Procurement and PlansUpdate on Trinity System Procurement and Plans
Update on Trinity System Procurement and Plans
 
TeraGrid Communication and Computation
TeraGrid Communication and ComputationTeraGrid Communication and Computation
TeraGrid Communication and Computation
 
Broadband tech 2005
Broadband tech 2005Broadband tech 2005
Broadband tech 2005
 
Big Data Everywhere Chicago: High Performance Computing - Contributions Towar...
Big Data Everywhere Chicago: High Performance Computing - Contributions Towar...Big Data Everywhere Chicago: High Performance Computing - Contributions Towar...
Big Data Everywhere Chicago: High Performance Computing - Contributions Towar...
 
Ling liu part 02:big graph processing
Ling liu part 02:big graph processingLing liu part 02:big graph processing
Ling liu part 02:big graph processing
 
Hw09 Hadoop Based Data Mining Platform For The Telecom Industry
Hw09   Hadoop Based Data Mining Platform For The Telecom IndustryHw09   Hadoop Based Data Mining Platform For The Telecom Industry
Hw09 Hadoop Based Data Mining Platform For The Telecom Industry
 
Digital Signal Processing Quiz
Digital Signal Processing QuizDigital Signal Processing Quiz
Digital Signal Processing Quiz
 
Streaming exa-scale data over 100Gbps networks
Streaming exa-scale data over 100Gbps networksStreaming exa-scale data over 100Gbps networks
Streaming exa-scale data over 100Gbps networks
 
Experiences with High-bandwidth Networks
Experiences with High-bandwidth NetworksExperiences with High-bandwidth Networks
Experiences with High-bandwidth Networks
 
FME Around the World (FME Trek Part 1): Ken Bragg - Safe Software FME World T...
FME Around the World (FME Trek Part 1): Ken Bragg - Safe Software FME World T...FME Around the World (FME Trek Part 1): Ken Bragg - Safe Software FME World T...
FME Around the World (FME Trek Part 1): Ken Bragg - Safe Software FME World T...
 
Optical Switching in the Datacenter
Optical Switching in the DatacenterOptical Switching in the Datacenter
Optical Switching in the Datacenter
 
Data on the Move: Transitioning from a Legacy Architecture to a Big Data Plat...
Data on the Move: Transitioning from a Legacy Architecture to a Big Data Plat...Data on the Move: Transitioning from a Legacy Architecture to a Big Data Plat...
Data on the Move: Transitioning from a Legacy Architecture to a Big Data Plat...
 
Big learning 1.2
Big learning   1.2Big learning   1.2
Big learning 1.2
 
2021 itu challenge_reinforcement_learning
2021 itu challenge_reinforcement_learning2021 itu challenge_reinforcement_learning
2021 itu challenge_reinforcement_learning
 
Exascale Capabl
Exascale CapablExascale Capabl
Exascale Capabl
 
PV 2014 - Montali - Verification of Parameterized Data-Aware Dynamic Systems
 PV 2014 - Montali - Verification of Parameterized Data-Aware Dynamic Systems PV 2014 - Montali - Verification of Parameterized Data-Aware Dynamic Systems
PV 2014 - Montali - Verification of Parameterized Data-Aware Dynamic Systems
 
Reflections on Almost Two Decades of Research into Stream Processing
Reflections on Almost Two Decades of Research into Stream ProcessingReflections on Almost Two Decades of Research into Stream Processing
Reflections on Almost Two Decades of Research into Stream Processing
 

Plus de Safe Software

From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action:  Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action:  Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationSafe Software
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsSafe Software
 
The Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemThe Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemSafe Software
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataSafe Software
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightSafe Software
 
Mastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISMastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISSafe Software
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriSafe Software
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfSafe Software
 
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologySafe Software
 
Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Safe Software
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataSafe Software
 
New Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s FoundersNew Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s FoundersSafe Software
 
Taking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsTaking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsSafe Software
 
Initiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategyInitiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategySafe Software
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Safe Software
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...Safe Software
 
Mastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FMEMastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FMESafe Software
 

Plus de Safe Software (20)

From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action:  Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action:  Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data Streams
 
The Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data EcosystemThe Critical Role of Spatial Data in Today's Data Ecosystem
The Critical Role of Spatial Data in Today's Data Ecosystem
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Mastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GISMastering MicroStation DGN: How to Integrate CAD and GIS
Mastering MicroStation DGN: How to Integrate CAD and GIS
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & Esri
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
 
Breaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI TechnologyBreaking Barriers & Leveraging the Latest Developments in AI Technology
Breaking Barriers & Leveraging the Latest Developments in AI Technology
 
Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...Best Practices to Navigating Data and Application Integration for the Enterpr...
Best Practices to Navigating Data and Application Integration for the Enterpr...
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
New Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s FoundersNew Year's Fireside Chat with Safe Software’s Founders
New Year's Fireside Chat with Safe Software’s Founders
 
Taking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New HeightsTaking Off with FME: Elevating Airport Operations to New Heights
Taking Off with FME: Elevating Airport Operations to New Heights
 
Initiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance StrategyInitiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance Strategy
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
Geospatial Synergy: Amplifying Efficiency with FME & Esri ft. Peak Guest Spea...
 
Mastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FMEMastering DevOps-Driven Data Integration with FME
Mastering DevOps-Driven Data Integration with FME
 

Dernier

Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersThousandEyes
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024Scott Keck-Warren
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesSinan KOZAK
 
Pigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions
 
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksSoftradix Technologies
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):comworks
 
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada
 
APIForce Zurich 5 April Automation LPDG
APIForce Zurich 5 April  Automation LPDGAPIForce Zurich 5 April  Automation LPDG
APIForce Zurich 5 April Automation LPDGMarianaLemus7
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Mattias Andersson
 
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxMaking_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxnull - The Open Security Community
 
Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024Neo4j
 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024BookNet Canada
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupFlorian Wilhelm
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...shyamraj55
 

Dernier (20)

Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen Frames
 
Pigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping Elbows
 
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other Frameworks
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):
 
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
 
APIForce Zurich 5 April Automation LPDG
APIForce Zurich 5 April  Automation LPDGAPIForce Zurich 5 April  Automation LPDG
APIForce Zurich 5 April Automation LPDG
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
 
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxMaking_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
 
Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024
 
Vulnerability_Management_GRC_by Sohang Sengupta.pptx
Vulnerability_Management_GRC_by Sohang Sengupta.pptxVulnerability_Management_GRC_by Sohang Sengupta.pptx
Vulnerability_Management_GRC_by Sohang Sengupta.pptx
 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food Manufacturing
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
 

Simplifying the Complex: Serving Data from Pipeline Data Models

  • 1. Simplifying the Complex: Serving data from Pipeline Data Models Peter Veenstra TRC
  • 4. GIS is difficult, users want data … Non-GIS Professionals need access to data to do their jobs  in XLS Industry Standard Pipeline Data Models are complex  Linear Referencing Data stored in Silos – hard to synchronize or coordinate … Model, Load, As-Built – Re-build Silo 1 Silo 2
  • 5. Big Gnarly Mess 1000’s if not 10’s of 10000’s miles of data overlaid on a single route. Linear events (in 100’s of tables) related to each other by common position along the pipeline but without formal relates. Potentially 7+ million segments to process. X Good Well Drained Test 1 - Current MAOP T MFL Type 1 Type 2b Type 2a X X X X X X Poor Poorly Drained Test 2 - Current
  • 6. Solution Build Toolboxes using FME Move toolboxes to server. Let the data be consumed.
  • 7. Toolboxes FME Toolkit for Pipeline Data Models (PDM) FME Toolkit for Linear Referencing (LRM) FME Toolkit for Database Integration (DIM) XYZM PL-AAA XYZM XYZM Sum Latest Merge
  • 8. Move to FME Server What is Server? Why is it better than scheduled tasks and scripts? Planned Migration (build, then optimize)
  • 9. Aggregation Data Warehouse FME Server Build a data ware-house aggregating data stored in different silos … - Shared coordinate plane - Hierarchy to Hierarchy translation - Dynamic Segmentation into massive table PODS Silo 1 Silo 2 Silo 3
  • 10. X Good Well Drained Test 1 - Current MAOP T MFL Type 1 Type 2b Type 2a X X X X X X Poor Poorly Drained Test 2 - Current FME Server Services Bus Query Engine Tableau Hierarchy Finder Data Extractor Data Delta Finder/Synchronizer Data Health 2016: Status and Progress Tracker
  • 11. A lot to get your head around … Subscribe, publish, consume … respond. Services-oriented rather than task-oriented Event-driven rather than procedural …
  • 12. GIS is difficult, users want data … "Good data leads to good knowledge, and good knowledge solves problems." Good data eh!
  • 13. Thank you! Peter Veenstra, TRC pveenstra@trcsolutions.com 816-820-7841