Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Luigi Selmi - The Big Data Integrator Platform

1 136 vues

Publié le

BDE SC4 webinar 3 presentation

Publié dans : Technologie
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

Luigi Selmi - The Big Data Integrator Platform

  1. 1. Pilot SC4 SC4 Final Hangout 2018-02-02 Big Data Europe Empowering Communities with Data Technologies
  2. 2. Objective of the Pilot SC4 SC4 Final Hangout 2018-02-02 A scalable, fault-tolerant and flexible platform based on open source frameworks that can process unbounded data sets.
  3. 3. Microservice Architecture SC4 Final Hangout 2018-02-02
  4. 4. The FCD Pipeline SC4 Final Hangout 2018-02-02
  5. 5. Pilot Architecture SC4 Final Hangout 2018-02-02
  6. 6. BDE Components SC4 Final Hangout 2018-02-02
  7. 7. Parallelization: input and output data SC4 Final Hangout 2018-02-02 device_id timestamp lat lon speed orientation transit The mapMatch subtask keeps the time order so that the next task keyBy(road_seg)/window(15’)/apply(avera ge_speed) will return the correct result within the time window for each road segment. road_seg_id start_date num_vehicles avg_speed
  8. 8. Parallelization: map-match subtasks SC4 Final Hangout 2018-02-02 1. source() 2. mapMatch() 3. keyBy()/window()/apply() 4. sink() The subtasks can be distributed in slots with different parallelism (e.g. from 1 to 12)
  9. 9. Parallelization: map-match subtasks SC4 Final Hangout 2018-02-02 A slot can process all the subtasks in a pipeline
  10. 10. SC4 Pilot Pipeline SC4 Final Hangout 2018-02-02
  11. 11. Data Upload SC4 Final Hangout 2018-02-02
  12. 12. Producer and Consumer SC4 Final Hangout 2018-02-02
  13. 13. Visualization SC4 Final Hangout 2018-02-02 The pilot SC4 can process real-time FCD data for map-matching and classify a road segment according to the traffic level.
  14. 14. Short-Term Traffic Forecast SC4 Final Hangout 2018-02-02 Algorithm: Feedforward ANN
  15. 15. Short-Term Traffic Forecast SC4 Final Hangout 2018-02-02 Algorithm: Feedforward ANN Hyperparameters (spatial and temporal correlation): Input layer units: (Dd*24*60*Cr)/Tw Dd = number of days (e.g. working days, 5 days) Tw = time window (e.g. 30’) Cr = connected road segments (e.g. 3) -> 720 input units
  16. 16. SANSA-Stack: Big Data + Machine Learning + Semantic Technologies SC4 Final Hangout 2018-02-02 SANSA-Stack, part of the BDE project, and RDF data sets based on semantic technologies such as LinkedGeoData, will enable more use cases related to SC4
  17. 17. Thanks SC4 Final Hangout 2018-02-02 BDE project website: https://www.big-data-europe.eu/ Code repository: https://github.com/big-data-europe?q=sc4 Contact: luigi.selmi@iais.fraunhofer.de

×