Presentation of a of real world data science solution that turns an open data set of Paris bike sharing system that showcases Cortana Intelligent suite. Vélib is a large scale public service of rent bike in Paris, launched on 15 July 2007, the service offers today about 14500 bicycles and 1230 stations. Through Cortana Azure Analytics Suite, we will set up a data analysis solution to : - Obtain descriptive analysis in real time. - Provide predictive analysis to predict the availability of bicycles and stations. In the presentation we will show how we have implemented the solution in Azure using the following services: Azure Event Hubs, Azure Stream Analytics, Azure Data Factory, Azure Machine Learning, SQL Azure Database, Power BI.

1. ##SQLSatMadrid
Project [Vélib’ by Cortana]

2. ##SQLSatMadrid
BIG Thanks to SQLSatMadrid Sponsors

3. ##SQLSatMadrid
Speakers

4. ##SQLSatMadrid
Agenda
 Presentation of the Project
 Cortana Intelligent Suite
 Creation of the architecture …

5. ##SQLSatMadrid
Purpose of the Project
 Get a descriptive analysis in real time of the
Vélib.
 Provide predictive analysis to predict the
availability of the number of bike and stations
available.

6. ##SQLSatMadrid
Objectives of the solution
300 000 Users – 18 000 Bikes – 2 000 Stations

7. ##SQLSatMadrid
1500 € Per bikes

8. ##SQLSatMadrid
Transform data into intelligent actions

9. ##SQLSatMadrid
[Vélib’ by Cortana] - Architecture
1 min
Cloud
service
Processing
Data flow

10. ##SQLSatMadrid
API JCDecaux

11. ##SQLSatMadrid
Azure Cloud Service

12. ##SQLSatMadrid
Azure
Event Hub
1 min
Cloud
service
Processing
Data flow
[Vélib’ by Cortana] - Architecture

13. ##SQLSatMadrid
Azure Event Hub

14. ##SQLSatMadrid
Azure Stream
Analytics Power BI
Azure
Event Hub
1 min
Cloud
service
Processing
Data flow
5 min
SQL
Database
[Vélib’ by Cortana] - Architecture

15. ##SQLSatMadrid
Azure Stream Analytics
 Processing continuous data
 Collecting millions of events per second
 Built-in Scalable solution
 Transform, enrich, temporal operations
 Detecting patterns and anomalies
 Data flow correlated with baseline data
Point of
Service Devices
Self Checkout
Stations
Kiosks
Smart
Phones
Slates/
Tablets
PCs/
Laptops
Servers
Digital
Signs
Diagnostic
EquipmentRemote Medical
Monitors
Logic
Controllers
Specialized
DevicesThin
Clients
Handhelds
Security
POS
Terminals
Automation
Devices
Vending
Machines
Kinect
ATM

16. ##SQLSatMadrid
Azure Stream Analytics

17. ##SQLSatMadrid
Lambda Architecture
Retrieving data from stations that follow two processes:
• A stream will be processed in real time
• A stream will be stored to be processed and analyzed

18. ##SQLSatMadrid
Azure SQL Database

19. ##SQLSatMadrid
Azure
Machine Learning
Azure Stream
Analytics Power BI
Azure
Event Hub
1 min
Cloud
service
Processing
Data flow
5 min
1 heure
SQL
Database
[Vélib’ by Cortana] - Architecture

20. ##SQLSatMadrid
Exploratory Data Analysis

21. ##SQLSatMadrid

22. ##SQLSatMadrid
Raw Training set
Temporal features
Forecast horizon
Hour
Minutes
day
month
Lag t-1 à t-4
Distirbuted lag model
1st & 2nd order derivative
Moving average
exponentially weighted moving average
Learner Decision Forest
regression
Cross validation
hyper Parameters optimisation
Output
Statistical features
Mean
Standard deviation
Variance
Skewness
Kurtosis
Min
Max
Cumulative frequency
Median absolute deviation
Model Architecture

23. ##SQLSatMadrid
Azure
Machine Learning
Azure Stream
Analytics Power BI
Azure
Event Hub
1 min
Azure Data Factory
Cloud
service
Processing
Data flow
5 min
SQL
Database
[Vélib’ by Cortana] - Architecture

24. ##SQLSatMadrid
Azure Data Factory

25. ##SQLSatMadrid
Azure
Machine Learning
Azure Stream
Analytics Power BI
Azure
Event Hub
1 min
Azure Data Factory
Cloud
service
Processing
Data flow
5 min
Cortana
SQL
Database
[Vélib’ by Cortana] - Architecture

26. ##SQLSatMadrid
Power BI

27. ##SQLSatMadrid
Cortana
Cortana understands and shows the
state of stations for now and for the
following hour :
« Prévisions pour la station madeleine »
« Forecasts for the station madeleine »
« Mostrar próxima disponibilidad en
Madeleine »

28. ##SQLSatMadrid
Azure
Machine Learning
Azure Stream
Analytics Power BI
Azure
Event Hub
1 min
Azure Data Factory
Cloud
service
Processing
Data flow
5 min
Cortana
SQL
Database
[Vélib’ by Cortana] - Architecture
Bots

29. ##SQLSatMadrid
Cortana Intelligence Suite
“on the shelf”
Intelligence
Dashboards &
Visualizations
Information
Management
Big Data Stores Machine Learning
and Analytics
CortanaEvent Hubs
HDInsight
(Hadoop and
Spark)
Stream
Analytics
Data Intelligence Action
People
Automated
Systems
Apps
Web
Mobile
Bots
Bot
Framework
SQL Data
WarehouseData Catalog
Data Lake
Analytics
Data Factory
Machine
Learning
Data Lake
Store
Cognitive
Services
Power BI
Data
Sources
Apps
Sensors
and devices
Data

30. ##SQLSatMadrid
logic
Web service LUIS
Bot Framework

31. ##SQLSatMadrid
Bot Connector
Your bot

32. ##SQLSatMadrid
Language Understanding Intelligent Service
Reduce labeling effort with interactive featuring
Seamless integration to Speech API
Deploy using just a few examples with active learning
Supports 5 languages (English, Chinese, Italian, French, Spanish)

33. {
“entities”: [
{
“entity”: “flight_delays”,
“type”: “Topic”
}
],
“intents”: [
{
“intent”: “FindNews”,
“score”: 0.99853384
},
{
“intent”: “None”,
“score”: 0.07289317
},
{
“intent”: “ReadNews”,
“score”: 0.0167122427
},
{
“intent”: “ShareNews”,
“score”: 1.0919299E-06
}
]
}

34. ##SQLSatMadrid
Gracias a los organizadores
• https://mva.microsoft.com/
• https://blogs.msdn.microsoft.com/big_data_france/
Links :
Gracias, Merci, Thank You !
• velibteam@microsoft.com
Alias :