Transport for London (TfL) and WSO2 have been working together on integration projects that focus on making the most use of London road networks and public transport. TfL collects and brings together a wide range of data from multiple disparate systems. Then they use this data for operational purposes and also make it open and available to everyone in real-time. This webinar will explore how TfL, with the help of the WSO2 analytics platform, Uses IoT and real-time streaming techniques to understand the current and predicted transport network status. Innovates heterogeneous data sources by combining the TfL Unified API with traffic, air quality, and passenger flow data Provides better travel time and transit suggestions for Londoners.

1. Transport for London
Using data to keep London moving
Sriskandarajah Suhothayan
Associate Director / Architect
WSO2

2. OVERVIEW
• Based on the presentation presented at
– Strata Data Conference, London, 2017.
• Co-presented with :
– Roland Major
Enterprise Architect
Formerly Transport of London
2

3. • Introduction of Transport for London
• Surface Intelligent Transport System
• Conceptual Architecture
• Introduction to WSO2 technologies
• Data in Motion - Hack Week
• ‘Live Journey Planner’ prototype
• How It’s Done?
• Data Driven Operational Applications
• Game Changing Visualizations
• Learning Outcomes
• What's next in WSO2 Analytics
AGENDA

4. • 30 million journeys daily
• In addition to all road and rail
transport, they look after rivers,
assisted travel, taxi and private hire
regulation
• They also do much more from
education to running the world’s
largest out of home advertising estate
of its type
• They are 150 years old and a chock full
of heritage and design assets
BREADTH OF TRANSPORT FOR LONDON

5. Sponsorship
Own the improvement strategy for the Transport for London Road Network
Engage with a wide range of customers and external stakeholders
Deliver user benefits that are clearly defined and measured
Outcome Delivery
Highways design and engineering
Traffic modelling capability
Intelligent control of traffic signals
Monitor, analyse and optimise road network performance
Operations
24/7 Operation to keep London moving
Real-time incident management through LSTCC
Assess and coordinate works and schemes to minimise disruption on TfL’s roads
ROAD SPACE MANAGEMENT

6. SITS will
• provide the capability to unlock significant additional effective capacity on
the road network for the future
• enable and support delivery of a multi-modal approach to transport
management by using and allocating existing and new capacity
• enable and support delivery of a Balanced Scorecard approach to transport
management by using and allocating existing and new capacity based on
local modal demands
SURFACE INTELLIGENT TRANSPORT SYSTEM

7. Key Characteristics
Bringing information and insight
closer to decisions
Locale, map based UI
Timely, event driven not batch
Trusted, consistent and accurate
Public cloud hosted
Reusable commodity platforms
Open Standards
Middleware Platform - WSO2
CONCEPTUAL
ARCHITECTURE
Integration
Data Hub
Collaboration
and Innovation
Analytics and
Visualisation
Data Driven
Operational
Applications
Secure Enclave

8. • Enabler for Digital Transformation
• 100% Open Source Middleware Platform
• Offices in : Mountain View, New York, London, Sao Paolo, Colombo
• 350+ Customers
• 450 People, 300 Engineers
WSO2

9. OPEN TECHNOLOGY FOR AGILE DIGITAL BUSINESS
100% open source

10. September 26-29, 2016
Objective :
Managing the Capacity of London’s Transport
Network
• Maximizing capacity on the public transport
network
• Maximizing capacity on the roads network
• Improving air quality
Datasets
• TfL APIs (Realtime and Historical)
• SCOOT sensor reading (Realtime)
• Passenger flow (Historical)
• Air quality (from KCL) (Historical)
Solution
• ‘Live Journey Planner’ Prototype
DATA IN MOTION - HACK WEEK

11. TfL has about 14,000 sensors measuring roads approaching junctions
This data is currently used by the Real Time control to manage optimization
Requirements to process
780 Million events per day
Latency to data center circa 1 Second
At data capturing resolution of 250ms scans
TRAFFIC CONTROL SENSORS
Junction
SCOOT Sensor

13. CHANCE OF GET A SEAT IN THE TRAIN?
Summarized Oyster Card Data

14. REALTIME TRAFFIC
HighTraffic
LowTraffic

16. Collect SCOOT Data
Learning traffic patterns using R
Building Random Forest Classification (it’s 88% accurate for this usecase) !
Exported the model as PMML
Use the model to predict traffic in realtime with WSO2 DAS
LOOKING INTO THE FUTURE

17. INCIDENT !
Via Local and Waze

18. WALKING AND CYCLING

19. Via Local and Waze

20. ARCHITECTURE

21. • Realtime data processing pipeline
HOW IT’S DONE ?
Detect
Headway
and
Vehicle
Length
Traffic
and
Flow
Calculation
Integrating
Historic
Summarization
Predicting
Traffic
Potential
Incident
Analysis

22. On raw SCOOT data stream
Sample stream: 1111100000111111100000
define stream ScootStream (scootId string, time long, reading int, seqId long);
from every e1=ScootStream[reading==1], e2=ScootStream[reading==0]+,
e3=ScootStream[reading==1]+, e4=ScootStream[reading==0]
select e3[0].seqId - e2[0].seqId as headway,
e4.seqId - e3[0].seqId as vehicleLength, ...
insert into DetectorStream;
DETECT HEADWAY AND VEHICLE LENGTH
Detect
Headway
and
Vehicle
Length
Traffic
and
Flow
Calculation
Integrating
Historic
Summarization
Predicting
Traffic
Potential
Incident
Analysis
Pattern Matching

23. On all SCOOT Detectors in London region.
from DetectorStream[str:split(scootId, "-", 0) == ‘London’]#window.time(’1 min’)
select count(*)/60 as flow,
avg(vehicleLength)/60 as traffic,
1/avg(headway) as density, ...
group by scootId
insert into TrafficStream;
The results are mapped to links and presented via APIs for visual representation
TRAFFIC AND FLOW CALCULATION
Detect
Headway
and
Vehicle
Length
Traffic
and
Flow
Calculation
Integrating
Historic
Summarization
Predicting
Traffic
Potential
Incident
Analysis
Filtering
SlidingTimeWindow
Group By
Aggregations

24. Historic data analysis with Apache Spark
Joining With Summarized Data
@from(table=‘rdbms’, url=‘...’, ...)
define table TrafficSummery (scootId string, week int, day string, traffic
long);
from TrafficStream as ts join TrafficSummery as tt
on ts.week == tt.week and ts.day == tt.day
select ts.traffic as currentTraffic, tt.traffic as usualTraffic, ...
insert into SummeryTrafficStream;
INTEGRATING WITH HISTORIC SUMMARIZATION
Detect
Headway
and
Vehicle
Length
Traffic
and
Flow
Calculation
Integrating
Historic
Summarization
Predicting
Traffic
Potential
Incident
Analysis
Join

25. Predicting traffic in next 15 minutes
from SummeryTrafficStream
#pmml:predict(’wso2das-3.1.0/marbel_model.pmml')
select *
insert into PredictedTrafficStream;
PREDICTIONS
Detect
Headway
and
Vehicle
Length
Traffic
and
Flow
Calculation
Integrating
Historic
Summarization
Predicting
Traffic
Potential
Incident
Analysis
Predict Function

26. Increasing trend in traffic hikes
from PredictedTrafficStream
select currentTraffic - historicTraffic as currentHike,
predictedTraffic – currentTraffic as predictedHike, ...
having currentTrafficHike > 0 and predictedTrafficHike > 0
insert into TrafficHikeStream;
from every e1=TrafficHikeStream ->
e2=TrafficHikeStream[ (e2.currentHick – e1.currentHike)*100.0 / e1.currentHike > 20 and
(e2.predictedHike – e1.predictedHike)*100.0 / e1.predictedHike > 20]
with in 15 min
insert into PotentialIncidentStream;
POTENTIAL INCIDENT ANALYSIS
Detect
Headway
and
Vehicle
Length
Traffic
and
Flow
Calculation
Integrating
Historic
Summarization
Predicting
Traffic
Potential
Incident
Analysis
Pattern Matching

27. London Works 2
• Central Register – a pan London system enabling visibility and management of works and
related activities in London
• Traffic Management Act Notifications (TMAN) - A dedicated interface between London
boroughs and TfL enabling the balanced delivery of major schemes and works on the TLRN
and SRN
• Forward Planning Tool - An advance planning tool that allows promoters to provide early
visibility of road and street works
Data Driven Operational Applications

28. GAME CHANGING VISUALIZATIONS

31. Situational Awareness

32. Situational Awareness

33. • Realtime analytics on data provides edge
• Keep focus on usability
• Use the right tool for the right task
• Skills are the biggest hurdle
• Bringing information sets together encourages new thinking
• Using Agile approaches has transformed outcomes
• Removing system fragmentation has a big impact on organization
• Flattening delivery structures & small staged initiatives
• Platform approach
• Early indications of making data easily shared and integrated is improving
decisions
LEARNING POINTS

34. Available by end of
year 2017
34
WHATS NEXT IN WSO2 ANALYTICS

35. • Support for Machine Learning model execution
• Online machine learning, PMML, and Java models
• Support to consume and publish data to and from various transports and
in multiple data formats
• HTTP, Kafka, Thrift, TCP, JMS, MQTT, RabbitMQ, Email
• XML, JSON, Text, CSV, Binary
• Support for connecting to various data stores
• RDBMs, MongoDB, Cassandra, HBase, Solr
• Incremental long running data aggregation
• Without a need of spark cluster
35
SUPPORTED FUNCTIONALITY

36. • Rich developer tool
• With auto completion, simulation and debugging support
• System monitoring support
• Business user friendly configurations
• Realtime dashboard
• With gadget generation support
36
TOOLING SUPPORT

37. 37
DEVELOPER STUDIO

38. 38
BUSINESS RULES

39. 39
DASHBOARD

40. • Scalable deployment with Apache Kafka
• Exactly once data processing
• Zero data loss
• Minimum high available deployment with 2 nodes
• Multi data center support
• Zero downtime
40
SCALABILITY AND HIGH AVAILABILITY

41. • Strata Talk on “Transport for London: Using data to keep London moving”
https://conferences.oreilly.com/strata/strata-eu-
2017/public/schedule/detail/57582
• WSO2 Analytics https://wso2.com/analytics
• WSO2 Data Analytics Server https://wso2.com/analytics/features
• WSO2 Stream Processor (Will be released on end of 2017)
https://github.com/wso2/product-sp/releases
• WSO2 Siddhi https://wso2.github.io/siddhi/
• WSO2 Support https://wso2.com/support/
41
USEFUL LINKS

42. THANK YOU
wso2.com