SlideShare une entreprise Scribd logo
1  sur  14
Télécharger pour lire hors ligne
© Fraunhofer-Institut für Intelligente
Analyse- und Informationssysteme IAIS
Spatio-temporal analysis of flows
in CDC 2013 data
Gennady Andrienko
Natalia Andrienko
http://geoanalytics.net
© Fraunhofer-Institut für Intelligente
Analyse- und Informationssysteme IAIS
Data processing procedures
1. Initial processing in Database
• Eliminating duplicates (same ID and time stamp)
• Eliminating stationary points (speed<2km/h)
• Dividing into days (by 3AM)
• Further dividing by 30min stops and 1km gaps
• Eliminating trajectories consisting of less than 5 points, shorter than 5
minutes, within 100m bounding rectangle
2. Further processing attempts in main memory
• Removing segments with speed > 75km/h
• Removing segments with high tortuosity (>2 over 1min), sinuosity (>5
over 1min) or being within 100m radius over 10-15 minutes
3. Still, the data are far from being perfect
• Wrong hardware / software / settings?
© Fraunhofer-Institut für Intelligente
Analyse- und Informationssysteme IAIS
Data quality
• Jumping around stops;
• Systematically wrong positions
© Fraunhofer-Institut für Intelligente
Analyse- und Informationssysteme IAIS
Summarization and aggregation of trajectories
• Density-driven Voronoi polygons, r=100m: 14,033 polygons country-wide
• Correctly reflect the street network
© Fraunhofer-Institut für Intelligente
Analyse- und Informationssysteme IAIS
Flows between adjacent polygons
• 14,033 polygons => 26,094 directed connections
• 5,723 used by at least 5 different trajectories
• Attribute “N different trajectories” compensates for “hairball” structures @stops
© Fraunhofer-Institut für Intelligente
Analyse- und Informationssysteme IAIS
Hourly time series of flows: transformation and clustering
• Only connections used by
at least 5 trajectories
1. Hourly time series
2. Smoothing by 3 hours
windows
3. Mean-normalization of
each time series
4. Clustering by k-Means
with different K
© Fraunhofer-Institut für Intelligente
Analyse- und Informationssysteme IAIS
Major clusters
© Fraunhofer-Institut für Intelligente
Analyse- und Informationssysteme IAIS
Cluster 5
© Fraunhofer-Institut für Intelligente
Analyse- und Informationssysteme IAIS
Cluster 3
© Fraunhofer-Institut für Intelligente
Analyse- und Informationssysteme IAIS
Cluster 1
© Fraunhofer-Institut für Intelligente
Analyse- und Informationssysteme IAIS
Cluster 2
© Fraunhofer-Institut für Intelligente
Analyse- und Informationssysteme IAIS
Cluster 4
© Fraunhofer-Institut für Intelligente
Analyse- und Informationssysteme IAIS
Conclusions
• Different roads have different temporal signatures
• Especially bridges
• Too few trajectories per person / per road segment for more sophisticated
analysis
• Data quality issues
© Fraunhofer-Institut für Intelligente
Analyse- und Informationssysteme IAIS
What we can do:
• Analysis of flows and their temporal dynamics
Times
Locations
Movers
Spatial events
Spatial event data Spatial time series
Movement data Local time series
Spatial distributions
Trajectories
Details:
Visual Analytics of Movement: an Overview of
Methods, Tools, and Procedures
Information Visualization, 12(1), pp.3-24, 2013
and
Visual Analytics of Movement
Springer-Verlag 2013
ISBN 978-3-642-37582-8
Due: July 5, 2013

Contenu connexe

Similaire à Spatio temporal analysis of flows in cdc 2013 data

Improving Traffic Prediction Using Weather Data with Ramya Raghavendra
Improving Traffic Prediction Using Weather Data  with Ramya RaghavendraImproving Traffic Prediction Using Weather Data  with Ramya Raghavendra
Improving Traffic Prediction Using Weather Data with Ramya Raghavendra
Spark Summit
 
SharkFest16_Palm_Online
SharkFest16_Palm_OnlineSharkFest16_Palm_Online
SharkFest16_Palm_Online
Brad Palm
 
Improving Traffic Prediction Using Weather Datawith Ramya Raghavendra
Improving Traffic Prediction Using Weather Datawith Ramya RaghavendraImproving Traffic Prediction Using Weather Datawith Ramya Raghavendra
Improving Traffic Prediction Using Weather Datawith Ramya Raghavendra
Spark Summit
 
2014-04-easteros
2014-04-easteros2014-04-easteros
2014-04-easteros
Jack Wang
 

Similaire à Spatio temporal analysis of flows in cdc 2013 data (20)

OSMC 2016 | Friends and foes in API Monitoring by Heinrich Hartmann
OSMC 2016 | Friends and foes in API Monitoring by Heinrich HartmannOSMC 2016 | Friends and foes in API Monitoring by Heinrich Hartmann
OSMC 2016 | Friends and foes in API Monitoring by Heinrich Hartmann
 
OSMC 2016 - Friends and foes by Heinrich Hartmann
OSMC 2016 - Friends and foes by Heinrich HartmannOSMC 2016 - Friends and foes by Heinrich Hartmann
OSMC 2016 - Friends and foes by Heinrich Hartmann
 
Crash course on data streaming (with examples using Apache Flink)
Crash course on data streaming (with examples using Apache Flink)Crash course on data streaming (with examples using Apache Flink)
Crash course on data streaming (with examples using Apache Flink)
 
Strel streaming
Strel streamingStrel streaming
Strel streaming
 
Av 738 - Adaptive Filtering Lecture 1 - Introduction
Av 738 - Adaptive Filtering Lecture 1 - IntroductionAv 738 - Adaptive Filtering Lecture 1 - Introduction
Av 738 - Adaptive Filtering Lecture 1 - Introduction
 
Improving Traffic Prediction Using Weather Data with Ramya Raghavendra
Improving Traffic Prediction Using Weather Data  with Ramya RaghavendraImproving Traffic Prediction Using Weather Data  with Ramya Raghavendra
Improving Traffic Prediction Using Weather Data with Ramya Raghavendra
 
The data streaming processing paradigm and its use in modern fog architectures
The data streaming processing paradigm and its use in modern fog architecturesThe data streaming processing paradigm and its use in modern fog architectures
The data streaming processing paradigm and its use in modern fog architectures
 
MC Lecture 8 67875667767777775677887.pptx
MC Lecture 8 67875667767777775677887.pptxMC Lecture 8 67875667767777775677887.pptx
MC Lecture 8 67875667767777775677887.pptx
 
Tsinghua University: Two Exemplary Applications in China
Tsinghua University: Two Exemplary Applications in ChinaTsinghua University: Two Exemplary Applications in China
Tsinghua University: Two Exemplary Applications in China
 
Chap2 slides
Chap2 slidesChap2 slides
Chap2 slides
 
L4 volume studies
L4 volume studiesL4 volume studies
L4 volume studies
 
SharkFest16_Palm_Online
SharkFest16_Palm_OnlineSharkFest16_Palm_Online
SharkFest16_Palm_Online
 
Network visibility and control using industry standard sFlow telemetry
Network visibility and control using industry standard sFlow telemetryNetwork visibility and control using industry standard sFlow telemetry
Network visibility and control using industry standard sFlow telemetry
 
A Deep Learning use case for water end use detection by Roberto Díaz and José...
A Deep Learning use case for water end use detection by Roberto Díaz and José...A Deep Learning use case for water end use detection by Roberto Díaz and José...
A Deep Learning use case for water end use detection by Roberto Díaz and José...
 
An Introduction to Distributed Data Streaming
An Introduction to Distributed Data StreamingAn Introduction to Distributed Data Streaming
An Introduction to Distributed Data Streaming
 
Improving Traffic Prediction Using Weather Datawith Ramya Raghavendra
Improving Traffic Prediction Using Weather Datawith Ramya RaghavendraImproving Traffic Prediction Using Weather Datawith Ramya Raghavendra
Improving Traffic Prediction Using Weather Datawith Ramya Raghavendra
 
A Platform for Data Intensive Services Enabled by Next Generation Dynamic Opt...
A Platform for Data Intensive Services Enabled by Next Generation Dynamic Opt...A Platform for Data Intensive Services Enabled by Next Generation Dynamic Opt...
A Platform for Data Intensive Services Enabled by Next Generation Dynamic Opt...
 
Khatibi lecture cov.uni
Khatibi lecture cov.uniKhatibi lecture cov.uni
Khatibi lecture cov.uni
 
A Platform for Data Intensive Services Enabled by Next Generation Dynamic Opt...
A Platform for Data Intensive Services Enabled by Next Generation Dynamic Opt...A Platform for Data Intensive Services Enabled by Next Generation Dynamic Opt...
A Platform for Data Intensive Services Enabled by Next Generation Dynamic Opt...
 
2014-04-easteros
2014-04-easteros2014-04-easteros
2014-04-easteros
 

Plus de cdc2013workshop

Plus de cdc2013workshop (8)

Tracking daily mobilities: GPS based bicycle data collection, processing, and...
Tracking daily mobilities: GPS based bicycle data collection, processing, and...Tracking daily mobilities: GPS based bicycle data collection, processing, and...
Tracking daily mobilities: GPS based bicycle data collection, processing, and...
 
Cycling in ghent objective and subjective evaluation of civitas policy measures
Cycling in ghent objective and subjective evaluation of civitas policy measuresCycling in ghent objective and subjective evaluation of civitas policy measures
Cycling in ghent objective and subjective evaluation of civitas policy measures
 
Application of gps tracking in bicycle research
Application of gps tracking in bicycle researchApplication of gps tracking in bicycle research
Application of gps tracking in bicycle research
 
Relating mobility patterns to socio demographic profiles
Relating mobility patterns to socio demographic profilesRelating mobility patterns to socio demographic profiles
Relating mobility patterns to socio demographic profiles
 
Analyzing cyclists’ behaviors and exploring the environments from cycling tracks
Analyzing cyclists’ behaviors and exploring the environments from cycling tracksAnalyzing cyclists’ behaviors and exploring the environments from cycling tracks
Analyzing cyclists’ behaviors and exploring the environments from cycling tracks
 
Reconstructing movement traces throug a hybrid map matching algorithm
Reconstructing movement traces throug a hybrid map matching algorithmReconstructing movement traces throug a hybrid map matching algorithm
Reconstructing movement traces throug a hybrid map matching algorithm
 
Extraction of bicycle commuter trips from day long gps trajectories
Extraction of bicycle commuter trips from day long gps trajectoriesExtraction of bicycle commuter trips from day long gps trajectories
Extraction of bicycle commuter trips from day long gps trajectories
 
Cyclist's waiting: identifying road signal patterns
Cyclist's waiting: identifying road signal patternsCyclist's waiting: identifying road signal patterns
Cyclist's waiting: identifying road signal patterns
 

Dernier

IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
Enterprise Knowledge
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
Earley Information Science
 

Dernier (20)

08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 

Spatio temporal analysis of flows in cdc 2013 data

  • 1. © Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS Spatio-temporal analysis of flows in CDC 2013 data Gennady Andrienko Natalia Andrienko http://geoanalytics.net
  • 2. © Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS Data processing procedures 1. Initial processing in Database • Eliminating duplicates (same ID and time stamp) • Eliminating stationary points (speed<2km/h) • Dividing into days (by 3AM) • Further dividing by 30min stops and 1km gaps • Eliminating trajectories consisting of less than 5 points, shorter than 5 minutes, within 100m bounding rectangle 2. Further processing attempts in main memory • Removing segments with speed > 75km/h • Removing segments with high tortuosity (>2 over 1min), sinuosity (>5 over 1min) or being within 100m radius over 10-15 minutes 3. Still, the data are far from being perfect • Wrong hardware / software / settings?
  • 3. © Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS Data quality • Jumping around stops; • Systematically wrong positions
  • 4. © Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS Summarization and aggregation of trajectories • Density-driven Voronoi polygons, r=100m: 14,033 polygons country-wide • Correctly reflect the street network
  • 5. © Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS Flows between adjacent polygons • 14,033 polygons => 26,094 directed connections • 5,723 used by at least 5 different trajectories • Attribute “N different trajectories” compensates for “hairball” structures @stops
  • 6. © Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS Hourly time series of flows: transformation and clustering • Only connections used by at least 5 trajectories 1. Hourly time series 2. Smoothing by 3 hours windows 3. Mean-normalization of each time series 4. Clustering by k-Means with different K
  • 7. © Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS Major clusters
  • 8. © Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS Cluster 5
  • 9. © Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS Cluster 3
  • 10. © Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS Cluster 1
  • 11. © Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS Cluster 2
  • 12. © Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS Cluster 4
  • 13. © Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS Conclusions • Different roads have different temporal signatures • Especially bridges • Too few trajectories per person / per road segment for more sophisticated analysis • Data quality issues
  • 14. © Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS What we can do: • Analysis of flows and their temporal dynamics Times Locations Movers Spatial events Spatial event data Spatial time series Movement data Local time series Spatial distributions Trajectories Details: Visual Analytics of Movement: an Overview of Methods, Tools, and Procedures Information Visualization, 12(1), pp.3-24, 2013 and Visual Analytics of Movement Springer-Verlag 2013 ISBN 978-3-642-37582-8 Due: July 5, 2013