Semantics-empowered Smart City applications: today and tomorrow

Semantics-empowered Smart City
Applications: Today and Tomorrow
Keynote at
The 6th Workshop on Semantics for Smarter Cities (S4SC 2015), October 11-12, 2015
Prof. Amit Sheth
LexisNexis Ohio Eminent Scholar; Executive Director, Kno.e.sis
Wright State University
Special Thanks: Pramod Anantharam
http://www.ict-citypulse.eu/

• Smart Cities as envisioned in the past and current state
• Smart City initiatives from governments
• Significance of digital infrastructure for Smart Cities
– Variety of smart city projects creating the digital infrastructure
• CityPulse: Large-scale data analytics for smart cities
• Key to Develop Future Robust Smart City Applications
• Future of smart cities
– P1: Increased use of multimodal observations and knowledge for enhanced explanation and
prediction of city events to enable data driven policy making
– P2: Innovative smart city solutions for situations with low/no instrumentation through
seamless citizen participation
• ComSmart Cities: opportunities and challenges
– P1: Integrating Multimodal Observations: Transportation Domain
• Combining textual traffic events with traffic sensor data
– P2: Smart city solutions for situations with low/no instrumentation
• Coordination during Disasters
• Smart Health for Situations with Low/No Instrumentation
• Smart Cities: opportunities and challenges
Outline

4
Source LAT Times, http://documents.latimes.com/la-2013/
Future Cities: A View from 1998
Thanks to Dr. Payam Barnaghi for sharing the slide

5
Image courtesy: http://robertluisrabello.com/denial/traffic-in-la/#gallery[default]/0/
Source: wikipedia
Back to the Future: 2013

7
https://www.whitehouse.gov/the-press-office/2015/09/14/fact-sheet-administration-announces-new-smart-cities-initiative-help
Image credit: https://www.whitehouse.gov/administration/president-obama
Smart Cities: Significance and Impact

8
Smart Cities: Significance and Impact
Image credit: https://commons.wikimedia.org/wiki/File:Narendra_Damodardas_Modi.jpg

10
Smart Cities: A Historical Perspective
Economic development on trade routesCivilizations on river banks
Economic development now increasingly rely on digital infrastructure
10
Image credit: http://www.rcet.org/twd/students/socialstudies/ss_extensions_1intro.html
Image credit: http://www.shutterstock.com/pic-157118819/stock-vector-conceptual-tag-cloud-containing-words-related-to-smart-city-digital-city-infrastructure-ict.html

11
Smart City Applications: Proliferation of Digital Infrastructure
http://postscapes.com/internet-of-things-award/2014/smart-city-application.html

12
Smart City Applications: Proliferation of Digital Infrastructure
http://postscapes.com/internet-of-things-award/2014/smart-city-application.html

14
Industrial SIE, ERIC
SME AI,
Higher
Education
UNIS, NUIG,
UASO, WSU
City BR, AA
Partners:
Duration: 36 months
CityPulse: Large-scale Data Analytics for Smart Cities

16
Data:
Data Processing Pipeline

17http://www.ict-citypulse.eu/scenarios/
101 Smart City Use-case Scenarios

19
http://www.ict-citypulse.eu/scenarios/
Use-case Scenarios

22
- Programmable devices
- Off-the-shelf gadgets/tools
Physical: Sensors Monitoring the Physical World

23
Cyber: Observations Pushed to the Cyber World

24
Motion sensor
Motion sensor
Motion sensor
ECG sensor
World Wide Web
Road block, A3
Road block, A3
Social: People Interacting with the Physical World

25
http://wiki.knoesis.org/index.php/PCS
Amit Sheth, Pramod Anantharam, Cory Henson, 'Physical-Cyber-Social Computing: An Early 21st Century Approach,' IEEE Intelligent Systems, vol. 28,
no. 1, pp. 78-82, Jan.-Feb., 2013. http://doi.ieeecomputersociety.org/10.1109/MIS.2013.20
Physical
Cyber
Social*
Developers need to Consider observations from Physical-Cyber-Social
systems in building future Smart City applications
*http://www.ichangemycity.com/
Key to Develop Future Robust Smart City Applications

Image credit: http://www.ibm.com/smarterplanet/us/en/smarter_cities/overview/index.html
Public Safety Urban planning Gov. & agency
admin.
Energy &
water
Environmental Transportation Social Programs Healthcare Education
Some Smart City Components
28

Future of Smart Cities
29
Public Safety Urban planning Gov. & agency
admin.
Energy &
water
Environmental Transportation Social Programs Healthcare Education
• Increased use of multimodal observations and knowledge for enhanced
explanation and prediction of city events to enable data driven policy
making
• Innovative smart city solutions for situations with low/no
instrumentation through seamless citizen participation

Increased use of multimodal observations and knowledge for enhanced
explanation and prediction of city events to enable data driven policy making
Multimodal observations and knowledge
Enhanced explanation and prediction
Data driven policy decisions
Public Safety Urban planning
Gov. & agency
admin.
Energy &
water
Environmental Transportation
Social Programs Healthcare
Education
https://www.oracle.com/applications/enterprise-resource-planning/roles/chief-financial-officer.html

31
Integrating Multimodal Observations: Transportation Domain Motivation

Link
Description
Scheduled
Event
Scheduled
Event
511.org
511.org
Schedule Information
511.org
32
Integrating Multimodal Observations: Transportation Domain Motivation
Slow moving
traffic

• Why?
– Explain/Interpret average speed and link travel time data using event
schedule provided by city authorities and real-time traffic events
shared on Twitter
– Past work: Predict congestion using single modality such as sensor
data
• What?
– Combine
• 511.org data about Bay Area Road Network Traffic
– E.g., Average speed and link travel time data stream
– E.g., (Happened or planned) event reports
• Tweets that report events including ad hoc ones
33
Integrating Multimodal Observations: Transportation Domain
Thanks to Dr. Krishnaprasad Thirunarayan for sharing the slide

• How?
– Extract events from textual tweets stream
– Build statistical models of normalcy, and thereby anomaly, from
numerical sensor data streams
– Correlate multimodal streams, using spatio-temporal information, to
annotate “anomalies” in sensor data time series with textual events
34

• How?
35

• Are people talking about city traffic events on
twitter?
• Can we extract city traffic related events from
twitter?
• How can we leverage event and location knowledge
bases for event extraction?
• How well can we extract city events?
Research Questions
36

37
Twitter as a Source of City Events
Public Safety
Urban planning
Gov. & agency
admin.
Energy & water
Environmental
TransportationSocial Programs
Healthcare
Education

Some Challenges in Extracting Events from Tweets
• No well accepted definition of ‘events related to a city’
• Tweets are short (140 characters) and its informal nature
make it hard to analyze
– Entity, location, time, and type of the event
• Multiple reports of the same event and sparse report of some
events (biased sample)
– Numbers don’t necessarily indicate intensity
• Validation of the solution is hard due to the open domain
nature of the problem
38

39
Pramod Anantharam, Payam Barnaghi, Krishnaprasad Thirunarayan, and Amit Sheth. 2015. Extracting City Traffic Events from Social Streams.
ACM Trans. Intell. Syst. Technol. 6, 4, Article 43 (July 2015), 27 pages. DOI=10.1145/2717317 http://doi.acm.org/10.1145/2717317
Extracting City Events from Textual Data

• City Event Annotation
– Automated creation of training data
– Annotation task (our CRF model vs. baseline CRF model)
• City Event Extraction
– Use aggregation algorithm for event extraction
– Extracted events AND ground truth
• Dataset (Aug – Nov 2013) ~ 8 GB of data on disk
– Over 8 million tweets
– Over 162 million sensor data points
– 311 active events and 170 scheduled events
First Evaluation
40

41
Distribution of Extracted Events Over Locations
• Evaluation Metric For Comparing Events with Ground Truth
– Complementary Events
• Additional information e.g., slow traffic from sensor data and
accident from textual data
– Corroborative Events
• Additional confidence e.g., accident event supporting a accident
report from ground truth
– Timeliness
• Early detection e.g., knowing poor visibility before its formal
report

42
Complementary Events

43
Corroborative Events

44
Timeliness
Timeliness
Evaluating Timeliness

• How?
45
Traffic Domain Use-case (Open Data)

Image credit: http://traffic.511.org/index
Multiple events
Varying influence
interact with each other
46
Challenge: Non-linearity in Traffic Dynamics

• Causes of non-linearity in sensor data streams
– Temporal landmarks : peak hour vs off-peak traffic vs
weekend traffic
– Effect of location
– Scheduled events such as road construction, baseball
game, or music concert
– Unexpected events such as accidents or heavy rains
– Random variations (viz., stochasticity)
47
Traffic Dependencies

• Disclaimer
"All models are wrong, but some are useful.” - George Box
• Normalcy Models
– Gaussian Mixture Model (GMM)
• Captures multiple co-existing events and its impact on traffic
• Doesn’t capture temporal dependencies
– Auto Regressive (AR) Models
• Captures temporal dependencies in traffic dynamics
• Doesn’t capture hidden aspects of the domain (e.g., volume of traffic)
– Linear Dynamical System (LDS)
• Captures temporal dependencies and hidden aspects of a domain
• Anomaly Model
– Cf. Box and Whisker plots
48
Abstracting Traffic Behavior: Traffic Data Model

Image credit: http://tourontap.com/us-open-2012/courses-and-more-by-the-bay/
AT&T Park
49

Histogram of speed values
collected from June 1st 12:00 AM to June 2nd 12:00 AM
Histogram of travel time values
collected from June 1st 12:00 AM to June 2nd 12:00 AM
50
Traffic Data: First Peek

Most of the drivers tend to
go 5 km/h over the posted speed limit
There are relatively few drivers who
go more than 10 km/h over the
posted speed limit
There are situations in a day where the
drivers are going (forced) below the
speed limit e.g., rush hour traffic
Do these histograms resemble any probability distribution?
51
Traffic Data: Possible Explanation

52
Multiple Gaussian Distributions: A Better Fit for Speed Observations?
This distribution resembles a
Gaussian Mixture Model (GMM)

Assume Normalcy to be uninterrupted traffic flow
July 2014 has no events so, we
hypothesize higher log-likelihood
score
June 2014 has many events so, we
hypothesize lower log-likelihood
score
-115655.8
(Closer to Normalcy)
-125974.3
53
Golden Gate Fields: Comparing Months with Varying Event Occurrences

54
Hourly Traffic Dynamics Over a Day

56
Tagging Anomalies with LDS Models

• How?
57
Traffic Domain Use-case (Open Data)

• If an anomaly is detected on a link L and during time
period [tst, tet], then the anomaly is explained by an event
if the event occurred in the vicinity within 0.5km radius
and during [tst-1, tet+1].
• CAVEAT: An anomaly may not be explained because of
missing data.
58
Spatio-temporal Co-occurrence Criteria

• Data collected from San Francisco Bay Area between May 2014 to May
2015
– 511.org:
• 1,638 traffic incident reports
• 1.4 billion speed and travel time observations
– Twitter Data: 39,208 traffic related incidents extracted from over 20 million
tweets1
• Naïve implementation for learning normalcy models for 2,534 links
resulted in 40 minutes per link (~ 2 months of processing time for our
data)
– 2.66 GHz, Intel Core 2 Duo with 8 GB main memory
• Scalable implementation by exploiting the nature of the problem resulted
in learning normalcy models within 24 hours
– The Apache Spark cluster used in our evaluation has 864 cores and 17TB main
memory.
59
1Anantharam, P. 2014. Extracting city traffic events from social streams. https://osf.io/b4q2t/wiki/home/
Experimental Data Statistics and Infrastructure

Innovative smart city solutions for situations with low/no instrumentation through
enhanced citizen participation
Coordination during Disasters

• May lead to second disaster to be managed:
– Under-supply of required demands
– Over-supply of not required resources
• Hurricane Sandy example,
“Thanks, but no thanks”,
NPR, Jan 12 2013
Story link:
http://www.npr.org/2013/01/09/168946170/thanks-
but-no-thanks-when-post-disaster-donations-
overwhelm
Uncoordinated Engagement

63
Image: http://www.gizmodo.com.au/2012/04/how-we-identify-single-
voices-in-a-crowd/
BIG QUESTION: Can these needles be identified in the
haystack of massive datasets?
Me and @CeceVancePR are
coordinating a clothing/food drive for
families affected by Hurricane Sandy. If
you would like to donate, DM us
Does anyone know how to donate
clothes to hurricane #Sandy victims?
[REQUEST/DEMAND]
[OFFER/SUPPLY]
Coordination teams
want to hear!
[BIG] Ad-hoc Community with Varying but [FEW] Important Intents

Really sparse Signal to Noise:
• 2M tweets during the first 48 hrs. of #Oklahoma-tornado-2013
- 1.3% as the precise resource donation requests to help
- 0.02% as the precise resource donation offers to help
64
• Anyone know how to get involved to help the
tornado victims in Oklahoma??#tornado
#oklahomacity (OFFER)
• I want to donate to the Oklahoma cause
shoes clothes even food if I can (OFFER)
Disaster Response Coordination:
Finding Actionable Nuggets for Responders to act
• Text REDCROSS to 909-99 to donate to
those impacted by the Moore tornado!
http://t.co/oQMljkicPs (REQUEST)
• Please donate to Oklahoma disaster
relief efforts.: http://t.co/crRvLAaHtk
(REQUEST)
For responders, most important information is the scarcity and
availability of resources
Blog by our colleague Patrick Meier on this analysis: http://irevolution.net/2013/05/29/analyzing-tweets-tornado/

Want to help animals in
#Oklahoma? @ASPCA
tells how you can help:
http://t.co/mt8l9PwzmO
x
RESPONSE TEAMS
(including humanitarian
org. and ‘pseudo’
responders)
VICTIM SITE
Where do I go
to help out for
volunteer work
around
Moore?
Anyone know?
Anyone know
where to donate to
help the animals
from the Oklahoma
disaster? #oklaho
ma #dogs
Matchable
Matchable
If you would like
to volunteer
today, help is
desperately
needed in
Shawnee. Call
273-5331 for
more info
65
CITIZEN SENSORS
DEMAND SUPPLY
Match-making: Assisting Coordination
Image: http://offthewallsocial.com/tag/social-media/

Image Credit: http://www.webmd.com/news/breaking-news/future-of-health/#advances-in-vision-toc/advances-in-vision
http://www.georgeinstitute.org/philanthropic-opportunities/smart-health-india
http://transmissionsmedia.com/wp-content/uploads/2013/02/NGO.jpg
https://developer.ibm.com/apimanagement/2014/11/20/government-apis-less/
Better understanding for policy decisions
Technology enabled enhanced citizen participation
Smart Health for Situations with Low/No Instrumentation
Innovative smart city solutions for situations with low/no instrumentation through
enhanced citizen participation
Seamless interactions b/w citizens, NGOs, and Governments
• Localize symptoms and disease
prevalence
• Localize environmental
conditions e.g., presence of
mosquitos, stagnant water
• Understand epidemics and its
propagation
• Measures to curb epidemics by
timely supply of medication and
advice
Heterogeneous data acquisition from healthcare workers
and low-cost sensors (e.g., SMS, and images).

Proposed Ecosystem
- Kno.e.sis Center
- Manav Sadhna
- eMoksha
- Government
My son was on
Cloraquine for 2 days and
is not showing any
improvements on malaria
symptoms.
Our resources will not last
long if the malaria cases
increase in a few days. We
are in need of medications
and volunteers.
Small water pools
around the
neighborhood are
creating mosquito
problems.
User 1
User 2
User 3
Chloraquine is not a
suggested solution for
malaria in India. Please
see a provider ASAP. The
closest healthcare facility
is on street X.
Received several
comments
from this area regarding
malaria symptoms.
Please send your
volunteer to check.
Query from User 1: classified
as an “active care” type query.
Response needs to be sent to
the user. Further analysis
showed similar queries from
same region.
Query from User 2
and 3: classified as a
“preventive care”
query. The message
needs to be sent to
an NGO.
Water pool is
breeding site for
Anopheles
mosquitos, so
preventive
measures need
to be taken.
SMS, e-mails,
tweets, Web
People from various
locations
Ontology/Knowledg
e Base

• Smart Cities as envisioned in the past and current state
• Smart City initiatives from governments
• Significance of digital infrastructure for Smart Cities
– Variety of smart city projects creating the digital infrastructure
• CityPulse: Large-scale data analytics for smart cities
• Key to Develop Future Robust Smart City Applications
• Future of smart cities
– P1: Increased use of multimodal observations and knowledge for enhanced explanation and
prediction of city events to enable data driven policy making
– P2: Innovative smart city solutions for situations with low/no instrumentation through
seamless citizen participation
• ComSmart Cities: opportunities and challenges
– P1: Integrating Multimodal Observations: Transportation Domain
– Combining textual traffic events with traffic sensor data
– P2: Smart city solutions for situations with low/no instrumentation
• Coordination during Disasters
• Smart Health for Situations with Low/No Instrumentation
• Smart Cities: opportunities and challenges
Outline

• Utilizing multimodal and heterogeneous observations for
enhanced understanding and prediction of city events
• Create better governance of our cities and better public
services through data driven policy making
• Empower citizens for active participation in shaping the
development of a city
• Provide more business opportunities for companies (and
SMEs) and private sector services
• Improve energy efficiency, create greener environments…
• Create better healthcare, elderly-care…
70
Smart Cities: Opportunities

• Dealing with massive heterogeneity in observations from a
city spanning physical, cyber, and social domains
• Dealing with missing, sparse, and noisy observations from
machine sensors and people
• Seamless integration of citizens in shaping city policies
(reliability and quality of citizen reporting of city events)
• Reliability and dependability of the massive infrastructure of
connected devices, services, and people
• Transparency and data management issues (privacy, security,
trust, …)
71
Smart Cities: Challenges

Thank You
http://knoesis.org/amit, http://personal.ee.surrey.ac.uk/Personal/P.Barnaghi/
@amit_p, @pbarnaghi
amit@knoesis.org, p.barnaghi@surrey.ac.uk
Acknowledgement: CityPulse Consortium
http://www.ict-citypulse.eu
Annual Report:
http://www.ict-citypulse.eu/page/sites/default/files/citypulse_annual_report.pdf
Kno.e.sis – Ohio Center of Excellence in Knowledge-enabled Computing
Wright State University, Dayton, Ohio, USA

Semantics-empowered Smart City applications: today and tomorrow

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Semantics-empowered Smart City applications: today and tomorrow

Similaire à Semantics-empowered Smart City applications: today and tomorrow (20)

Dernier

Dernier (20)

Semantics-empowered Smart City applications: today and tomorrow