Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
Arpan pal u world2012
1. 1
Personal and Community Context
Discovery
17 May 2015
Arpan Pal
Principal Scientist and Research Head
Innovation Lab, Cyber physical Systems
Tata Consultancy Services (TCS)
2. 2
Tata Consultancy Services Ltd. (TCS)
Pioneer & Leader in Indian IT
TCS was established in 1968
One of the top ranked global software service provider
Largest Software service provider in Asia
250,000+ associates
USD 10B + annual revenue
Global presence
First Software R&D Center in India
- 2 -
6. 6
Personal and Community Context Discovery
Context - patterns of individual, group and societal behaviours.
Broadly classified into three categories –
Personal Physical Network Discovery
Who is interacting with whom? What is the level of interaction? Who all are
part of similar-interest networks?
Individual Context Discovery
Who is doing what?
Community Context Discovery
Can we discover how a community / group behaves as a whole?
8. 8
Other Example Use Cases
Organizational Behavior Analysis
Team Efficiency Study
Best Practice Study
Workspace Ergonomics Study
Customer Behavior Study in Retail Stores
Customer movement pattern
Customer interaction pattern with shelves / merchandize
Crowdedness measure in public places
Efficient scheduling of public transport
Crowd Behavior analysis
Evacuation planning during disaster
Ref. - Alex Pentland et. al., MIT media Lab
9. 9
What do we need to Sense
Location
Proximity
Activity
Identity
Provide Context discovery as a
Service
10. 10
How to Sense
Needs to be Ubiquitous and Unobtrusive
There should not be any new hardware / device to carry
for an individual
Proposal
o Use smartphone-based sensors (GPS, accelerometer,
compass, microphone)
o Use 3D surveillance cameras (like Kinect)
o Augment with social network data and email data analytics
o Multimodal fusion of all the above
Privacy can be an issue – needs to be handled on an use
case-by-use case basis
o Privacy vs. Utility
12. 12
Mobile Phone Based Sensing
Proximity / presence
– Using Bluetooth for finding nearby mobiles
– Using Wi-Fi to discover other mobiles nearby
Location
– Using ultrasound beacon
– Using GPS (outdoors)
– Using Accelerometer / compass
Activity
– Using Accelerometer
Interaction Level
– Using Microphone Audio
Identity
– From Network ID
On-board sensors
Accelerometer, GPS,
Compass
Camera, Microphone
Network
Bluetooth, WiFi, 2G/GPRS, 3G
Network
2G/GPRS, Bluetooth
On-board sensors
Microphone, Camera
13. 13
Sensor Penetration and power consumption in
Mobile Phones
0 20 40 60 80 100
Bluetooth
USB
Edge
GPRS
Wifi
3G
Camera
GPS
Accelarometer
Digital Compass
Consolidated Market Penetration
Source: Nericell: Rich Monitoring of of Road and Traffic
Conditions using Mobile Smartphones, Prashant Mohan et.
al., Microsoft Research, SenSys 2008, North Carolina, USA
14. 14
Kinect Based Sensing
Human Identification
– Skeleton Model Based
– External Stimulus based refinement
Network Discovery
– Network discovery through proximity
– Level of Interaction through Audio
• 2D Camera with
IR depth sensor
• Excitation by IR
light pattern
• Directional Mic.
15. 15
Kinect Based Sensing (contd …)
Working on a public Kinect dataset
• People Discussion
• Give/Put/Take an object
• Enter/leave a room
• Leave baggage unattended
• Handshaking
• Typing on a keyboard
• Telephone conversation (Mobile, landline)
Image and the corresponding 3D cloud point
Human Interaction
– Activity Detection on 3D Point Cloud
– Physical object Identification
– Interaction with Objects
Human activities recognition and localization competition (HARL), ICPR 2012
16. 16
Soft sensing from Web
Unstructured Data
• Social network posts
such as tweets, facebook
• Blog posts
• Email bodies
Structured Data
• Social network profiles
and network information
• Email headers
• Tweet Attributes
Personal
Network
17. 17
IoT Platform from TCS
Internet
End Users
Administrators
Device Integration & Management Services
Analytics Services
Application Services
Storage
Messaging & Event Distribution Services
ApplicationServices
Presentation Services
Application Support Services
Middleware
Edge Gateway
Sensors
Internet
Back-end on Cloud
RIPSAC – Real-time Integrated Platform for Services & AnalytiCs
Traditional
Internet
Service Delivery
Platform & App
Development
Platform
Security/Privacy
Framework
Lightweight M2M
Protocols
Analytics-as-a-
Service
Social Network
Integration
SDKs and APIs for
App developer
18. 18
Summary
Introduced the concept of personal and community context
discovery as a service with help of example use cases
Proposed an unobtrusive and ubiquitous way to gather the
context through mobile phone based sensors, 3D camera
and web data
Each method has its own limitation both from application
and technology perspective
Need for a multimodal fusion for improved accuracy
A generic IoT platform to implement and deploy the services
for application developers