Whatever the size or type of organization, Big Data has permeated our transportation industry. It is no longer a question of IF Big Data will be useful, but instead WHY is it useful and HOW can we best apply it. This presentation aims to address how we can leverage existing services and available partnerships in transportation, consider new and emerging technologies, and determine strategy for what’s to come in transportation, including connected and autonomous vehicles. While it may be a huge challenge to solve transportation problems with Big Data, it can help us make better travel decisions today and plan for better infrastructure tomorrow.
2. london.ca 2
• Jon Kostyniuk, P.Eng.
• Primarily transportation
modelling and forecasting
background.
• All models are wrong, but
some are useful.
• TripChain.Org project lead,
trip generation data using
blockchain technology.
• Interested in data, its value,
and how we can better apply
its usefulness.
Who am I?
3. london.ca 3
• It’s not just coming, it’s here.
• It will demand more attention with:
oConnected and Autonomous Vehicles (CAVs)
oMobility-as-a-Service (MaaS)
oDistributed Systems
• We’re not all going to be data experts, but we should
have some basic data literacy.
• Help increase data-driven decision making (DDDM).
• Data like a currency, can provide value when timely
transmitted; therefore, communication is key.
Why should I care about Big Data?
4. london.ca 4
• Volume: Vast amounts generated every second.
• Velocity: Not only speed generated, but speed
at which it moves around.
• Variety: Types of data to work with, not just
structured data anymore.
• Veracity: What is the trustworthiness of the
data?
• Value: How does the data bring business value?
Defining “Big Data”
6. london.ca 6
Existing Traffic Signal System
• Existing system from 2005, updated regularly,
becoming dated with limited abilities with RT aspirations.
• 401 existing signal locations connected to a central
system at City Hall.
• Limited real-time awareness at intersections without
context.
7. london.ca 7
• Video data collection – TMCs, ATRs.
• Bluetooth readers for travel time and
OD studies, limited use.
• Radar detection, including many of
over 40 permanent ATRs.
• Traffic data via Google Maps and
APIs, limited use/confidence
• Limitations to “snapshot in time”
data collection, but demand is
dynamic.
Existing Data Services
8. london.ca 8
• Our TIMMS project, may include:
oTSP to support RT,
o“Adaptive” Signals at key corridors,
oA modern TMC,
oCCTV monitoring, and/or
oTravel time monitoring and feedback.
• Need to manage expectations for new
system, not a panacea.
• Goals and objectives to include
development of a Data Strategy.
New System Planned
9. london.ca 9
• Public web app, road closure and
disruption information.
• Lots of detailed information, used
by many business units.
• Need to “get” information from
website, not everyone knows
about.
• Custom-build, used for past
decade, but somewhat dated.
• Currently reviewing its use cases
and future direction.
Renew London Web App
11. london.ca 11
• Better strategy to “push” data to
popular apps and services.
• Enhanced Renew London, added
data feed.
oWaze’s CIFS XML format
• Agnostic on third-party usage of
feed.
• Joined Waze’s CCP in April 2018,
now live.
• Use Waze’s web interfaces, third
party tools, or build your own.
Renew London Integration with Waze
Construction
Road
Closures
Collisions
Road
Hazards
12. london.ca 12
WARP Project
• Open-source traffic data solution by Louisville KY,
supported by OGC – QR to GitHub repo.
• Polls traffic data, dumps in database for historical
use.
• Processor features, in-development:
oHosted cloud service, database
oAPI endpoints, integration
oTraffic study tool, analytics
oInteractive map, visualization
13. london.ca 13
Train-Delay Warning System
• Testing sensors at 3 crossing locations.
• Participating in BCIP pilot program with
TRAINFO in 2019.
• Provide crossing insights along with
Bluetooth sensors.
• Gathers data, predicts when a train may
be present
• Early warning system via VMS to allow
drivers to choose better routes or other
“push” services.
14. london.ca 14
A blockchain is a distributed database
that is updated in near real-time, stored in
decentralized locations, and easy to
monitor. It has a level of security that
insures that no one party can modify a
database entry, because, in a sense,
everyone is watching.
Blockchain Technology
16. london.ca
Process
Characteristics
Blockchain Benefits
Consensus between
Multiple Parties
Enhanced coordination and choreography between
parties through a shared view of the latest data
status.
Reconciliation Master source of data instead of disparate data
sources that require constant validation and
reconciliation.
Data Lineage
(Temporal or 4D Data)
Complete traceability, ensuring integrity of data that is
continuously updated and maintained by multiple
parties.
Auditability Reliable and accurate audit trail with transparency of
the party responsible for each data change.
16
Blockchain Technology
18. london.ca
• CAV Technical
Background report,
May 2018
oQR Code report
link above.
• Council resolution,
develop a CAV
Strategic Plan.
18
Connected and Autonomous Vehicles
19. london.ca 19
• Some municipalities early pilots in SPaT and MAP
V2I connectivity.
oSignal phase and timing
oPhysical geometry of intersection
• Data partnerships with OEMs, other non-traditional
stakeholders.
oE.g., vehicle manufacturers, communications, ride
sharing, etc.
• Each CAV a valuable data source.
oHow can municipalities leverage the data?
oImpacts on liability, responsibility, etc.?
Connected and Autonomous Vehicles
20. london.ca 20
• Key points on data:
oThere are significant
privacy issues.
oTechnology
expertise is urgently
needed.
oFocus on digital
infrastructure.
Connected and Autonomous Vehicles
• PPSC Report, The Future of Automated Vehicles in
Canada, January 2018 – QR Code report link above.
• Governments should build data expertise and capacity.
22. london.ca 22
A plan designed to
improve all of the ways
you acquire, store,
manage, share, and
use data.
What is a Data Strategy?
23. london.ca 23
• Each component
independent and can
evolve as needed.
• Each component has an
individual set of skills and
capabilities.
• “Enterprise-class” strategy
not always needed.
• Complexity increases with
organizational scope.
• Establish goals for each
component.
Essential Data Strategy Components
Source: The 5 essential Components of a Data Strategy, SAS Institute Inc.
24. london.ca 24
Identify data and understand its meaning.
Provision data to be made available while
respecting rules and access guidelines.
Govern through policies and mechanisms to
ensure effective data usage.
Store persistent data in a structure and location
that supports access and processing.
Integrate by moving and combining data to
provide a unified view.
Core Components Defined
25. london.ca 25
• Do we currently partner with the right people?
• What are our data bottlenecks?
• How do we optimize our data collection and usage?
• Are we getting the most out of our equipment / services?
• Which data services are the most unreliable and why?
• Do we have the right IT systems in place?
• What parts of our operations could be more efficient?
• What are our core data competencies?
• What data skills gaps exist in our municipality?
• What are our key skills needed in the next two years?
Example Strategic Questions
26. london.ca 26
Big Data Skills
• Business Skills: Understanding
transportation services, including
communication and interpersonal.
• Analytical Skills: Spot patterns, cause and
effect, build models, etc.
• Computer Science: Hardware, software,
AI, programming, etc.
• Statistics and Mathematics: Determine
relevant data, sample sizes, algorithms, etc.
• Creativity: Ability to convey insights
effectively to an audience.
27. london.ca 27
• Well, maybe not completely… but, hello visualizations!!
• Consider best way to communicate data with audience.
• Insightful to add 4th dimension… time.
Goodbye Spreadsheets
28. london.ca 28
• Challenging to have staff and expertise.
• Organizational challenges – operations, structure, size.
• BDaaS viable alternative to do-it-yourself.
• Collect, store, analyze, and provide access
• Does service provider support your strategy?
• Cost of Big Data vs. traditional approach.
Big Data as a Service (BDaaS)
29. london.ca 29
Key Takeaways
• Like promises of the past, Big Data is
not a panacea, manage expectations.
• Plan ahead, use Big Data effectively.
• Beware of too many purpose-built apps.
• Informed travellers = better decisions
and less frustration.
“Fewer red lights, less stop and start and a
minimum of delays, those are the goals of a
computerized traffic control system in which London
may participate” ~ London Free Press, 1962
Thank You
I appreciate you all attending and listening to this talk.
I’d like to extend a special thank you to the OTC staff and directors, and in particular Scott Godwin and Doug Green, for giving me the opportunity to speak today.
Ask Questions to the Audience
How do you feel when you just get a green light only to be stopped again at a red light at the next intersection?
How do you feel when you turn the corner and see those flashing lights and railroad crossing bells ringing, not knowing how long you’ll be stuck there?
Who am I?
[Cover bullets on slide]
Why am I telling you about Big Data?
Big Data really is this undefined “blob” that means many things to many people – ongoing challenge.
Essentially, I wish to advocate that:
All municipalities are different (e.g. size, staffing, abilities, needs, etc.), so we should consider our Strategic Needs;
When data insights are provided to travellers in near real-time, better decisions can be made to reduce frustration; and
When presented effectively, DDDM can help us make better transportation planning and design decisions.
I will briefly cover:
Some of the conventional approaches we are taking at the City of London (which I’m sure is similar with many of us);
Look at some emerging and even fanciful technologies; and
Most importantly, overview how we can pursue developing a Data Strategy for our own municipalities.
Why should I care about Big Data?
It’s not just coming, it’s here.
It’s only going to demand more attention with the advent of:
Connected and Autonomous Vehicles (CAVs);
Mobility-as-a-Service (MaaS); and
Distributed Systems (i.e. Blockchain and Smart Contracts).
We’re not all going to be data experts (don’t be scared), but we should have some basic data literacy.
If applied effectively, Big Data can help increase data-driven decision making.
Data is like a currency and can provide value when it is timely transmitted to where it’s needed. Therefore, communication of data is key.
Defining “Big Data”
Generally, big data can be defined by one or more of these 5 “Vs”.
Volume
[Read slide…] We’re not just talking about gigabytes – but terabytes, petabytes, and even zettabytes. How do we comprehend let alone work with this?
Velocity
[Read slide…] Data, and especially real time data, is most useful when it gets to where it’s needed in a timely manner.
Variety
[Read slide…] While structured data, like SQL databases, will continue to retain importance, unstructured data such as audio, video, and photos are becoming more important.
Veracity
[Read slide…] Given the other Vs, we need to have a certain level of trust in our data sources to help us drive appropriate decisions.
Value
[Read slide…] If the data we are collecting does not help us gain relevant insights, why are we doing it?
Existing Traffic Signal System
Existing traffic signal system in-place since 2005 and updated regularly.
We have 401 traffic signal locations in current system, including both full signals and IPS.
All signals connected to a central system, can change timings from City Hall.
System becoming dated, coming to end of lifespan, and limited considering RT aspirations.
Limited real-time awareness at each intersection without context. [Talk about February 2018 flood.]
Existing Data Services
Typical practices compared with many medium-sized cities currently.
Video data collection for most common TMCs and ATRs.
Use four (4) “portable” Bluetooth units to perform travel time and OD studies, but this is of limited use.
In recent years, radar detection has become our “go-to” technology, including many of our over 40 permanent ATR locations.
On occasion we have used Google Maps and APIs to obtain traffic data, but still limited use and confidence in this approach.
While these services provide a good, basic data background, there are still limitations to the “snapshot in time” approach whereas travel demands are dynamic.
New System Planned
We do have a new system planned called our Transportation Integrated Mobility Management System (TIMMS) project. Still under development, but may include TSP, “Adaptive” Signals, TMC, CCTVs, and/or Travel Time Sensors.
Despite this we need to manage expectations for the new system.
One of the main components of this new system I wish to discuss today will be the development of a Data Strategy.
Renew London Web App
The City curates a web app that provides up-to-date road closure and disruption information.
Lots of detailed information available within the app, used may many business units within the City.
However, one needs to “get” the information from the website.
What if you don’t know about the website or are a visitor to the City?
This is frustrating to travellers and goes to the point that data is like a currency that needs to provide timely value when it is needed.
While there are similar third-party services available, Renew London is a custom-build, but is becoming somewhat dated.
We are currently assessing the use cases and future direction of Renew London.
Renew London Integration
First step: Instead to going to the Renew London website to “get” data, we’ll create a “push” data feed any third-party can use.
Our feed uses Waze’s Closure and Incident Feed Specification (CIFS) format as de-facto standard.
Generally, we have an agnostic stance on third-party usage:
Working with internal stakeholders such as London Transit Commission, Emergency Services, and Corporate Security.
Open to third-party external apps and services, e.g. TomTom, Garmin, Apple Maps, Navmii,
We joined Waze’s Connected Citizen’s Program (CCP) as of April 2018.
Free, two-way agreement.
Provide CIFS data every 2 minutes and obtain crowdsource and congestion data from Waze.
Push common data such as construction, collisions, road closures, and road hazards.
Ability to push out Emergency Shelter information in event of emergency.
Can access Waze via their web interface, third-party tools, or build/integrate your own.
WARP Project
Unique, less conventional ways are being explored to capture traffic data.
Open-source Waze Analytics Relational-database Platform (WARP) is on project currently under development by Louisville KY, the Open Government Coalition, and various other partners.
Includes four man areas of development:
A database to process and store historical data;
Data hooks via an API to integrate with other third-party products;
A traffic study tool to provide analytics to assist DDDM; and
An interactive map to visualize events and patterns over periods of time.
I’ve heard claims this could effectively provide data for FREE* traffic studies.
Time will tell, but caution that FREE likely means “Some conditions may apply”.
However, crowdsourced data could provide a preliminary snapshot of problematic areas of interest.
Source
https://docs.google.com/presentation/d/1loAV4BDAUyXdrn44QoLmYiwZdLmL59C4jvJGlZ1a-AY/edit
Train-Delay Warning System
Who likes getting stuck at train crossings?
We are currently testing train presence sensors at three (3) London railway crossing locations.
In 2019, we will be participating in a pilot program with TRAINFO through the Build in Canada Innovation Program (BCIP).
Our current project with this system is to gather train disruption data and determining the impacts to traffic patterns.
Over time, the algorithms build a crossing profile to “predict” when likely train disruption events will occur.
The intent is to provide timely, early warning systems to inform drivers and allow them to choose alternate routes.
Use Variable Message Signs (VMS) and/or travel apps, such as Waze.
Still limitations in propagation time from sensor detection to “push” notifications in apps - the train disruption could be over.
Blockchain Technology
Who here has heard of blockchain technology or smart contracts?
Who here has heard about Bitcoin or Ethereum?
Many people seem to know Bitcoin, but don’t (literally) buy into the hype!
Many people are similarly less familiar with Blockchain Technology.
I would argue that this technology is more important and has more of a potential future in Big Data and transportation.
Let’s start with a definition… [Read Definition].
In other words, a blockchain is distributed, immutable, and auditable.
Blockchain Technology
To visualize how blockchains interact, consider a centralized system, such as a central traffic signal system or ATMS.
Blockchain generally acts as a distributed system, but has characteristics of a decentralized system – a hybrid between the two.
Effectively, new data propagates through the network in near real-time as in the distributed model.
However, individuals in isolation can access the network as in the decentralized model.
Redundancy in the network – several, if not most, of the network nodes could fail while continuing to maintain functionality.
Potential Use Cases
Link together Mobility-as-a-Service (MaaS) partners for seamless data exchange.
Transparency and incentive tracking for Public-Private-Partnerships (P3s) – MOBI.
Help the AI in Autonomous Vehicle algorithms “learn” quicker by propagating new data in near real-time.
Break down data silos and give credit and value to data producers – Ocean Protocol.
Shameless plug for my pet project, TripChain.Org – propagate trip generation data to improve transportation planning decisions.
Many other Blockchain projects, constant churn, still emerging.
Prognosis for Blockchain
Overall still an emerging, promising technology, but watch progress in coming 5-10 years.
Technology still not proven in production, no “killer app” yet to emerge.
Although data storage getting cheaper and cheaper, still beware of data bloat.
Connected and Autonomous Vehicles
Briefly touch upon Connected and Autonomous Vehicles (CAVs).
While still several years out, we wish to take a more proactive approach to preparing for CAVs as opposed to reacting when they show up on our roads.
Developed a CAV Technical Background report in May 2018 and obtained a Council resolution to prepare a CAV strategy for the City of London.
Focus on recommendations including Infrastructure, Land Use, Transit, Parking, Accessibility, Safety, Privacy and Security, and Public Awareness and Education.
Connected and Autonomous Vehicles
Getting an early start, some municipalities are already piloting Vehicle-to-Infrastructure (V2I) communication, including SPaT and MAP data.
To pull CAVs off, we will need to explore and leverage non-traditional data partnerships.
Each CAV itself will likely be a valuable data source.
How can municipalities leverage this data for planning and operations?
Will vehicle self-report pot holes, road conditions, etc.?
How will municipalities respond?
What are the impacts on liability.
May questions to explore in the near future.
Connected and Autonomous Vehicles
The report included several key points related to data and municipalities:
There are significant privacy issues: As more and more vehicle data is generated, collected and shared, governments must act to protect the privacy rights and security of individuals.
Technology expertise is urgently needed: It will be crucial for regulators to develop expertise in data science and computer science. CAVs will generate large volumes of data which currently have no clear ownership rights. Governments will need to effectively address these concerns and protect the public interest.
Physical infrastructure modifications can wait, focus on digital infrastructure: CAVs are being designed to work with the physical or “hard” infrastructure that exists today. However, governments will need to be aware of which technologies automakers and suppliers are using. Digital interfaces such as sensor data, maps, etc. warrant more immediate attention to help support CAV systems.
Recommends governments should build data expertise and capacity as part of the interdisciplinary effort to make CAVs a reality in the coming years and decades.
Example Strategic Questions
Thinking at a high level, here are some example strategic questions municipalities could consider.
[Review question list]
Big Data Skills
Five (5) essential data science skills to develop.
[Review slides]
Goodbye Spreadsheets
I have a love/hate relationship with spreadsheets.
Tables and charts will always be valuable to gain insights, but depending on the audience, data visualizations are a great tool.
Observing data over time also gives a sense to guide decision making.
Big Data as a Service (BDaaS)
Of course, it can be challenging for many municipalities to have staff and expertise move from data to insights.
Organizational challenges such as operations, structure, and size may provide hurdles to gaining insights.
BDaaS is a growing and viable alternative to do-it-yourself data analysis. Depending on the service can collect, store, analyze, and provide access to data.
Ultimately need to consider whether a BDaaS provider supports your strategy and look at the cost vs. traditional approaches.
These are not an endorsement nor an extensive list, but here are several companies in our industry which may offer BDaaS to varying degrees.
I’m sure there are many others that I have not yet considered.
Key Takeaways
I like this quote… [Read Quote]. Does this ring true today, nearly 60 years later?
Like promises of the past, Big Data is not a panacea, so manage expectations with key stakeholders and the public.
By creating a Data Strategy and planning ahead, your municipality can use Big Data effectively.
Beware of too many purpose-built apps.
Look integration opportunities with prominent or popular apps or services to maximize your impact.
Not just serving your municipality, but also visitors and travellers passing through.
And lastly, informed travellers have the ability to make better travel decisions, hopefully experiencing less frustration.
Help to use the transportation network more effectively.
Knowledge is power and can help minimize the frustration the comes from the unexpected.