Presentation from Chris Haddad - VP Platform Evangelism, WSO2 #gigaomlive More at http://events.gigaom.com/structuredata-2014/

1. 1

2. Last Updated: Jan. 2014
VP
Pla&orm
Evangelism
@cobiacomm
on
Twi6er
h6p://blog.cobia.net/cobiacomm/
Chris
Haddad
Tracking
a
Soccer
Game
with
Big
Data

3. Big
Data
Improves
Team
Performance
๏ Sports
Teams
๏ Your
Business
Team
3

4. Improvement
Requires
A
Feedback
Loop
4

5. Internet
of
Things
(IoT)
Bridges
Physical
and
Virtual
Worlds
๏ Sensors,
actuators
everywhere
๏ Internet
connecOvity
Improvement
=
IoT,
Apps,
and
APIs
๏ VisualizaOon
Apps
๏ Control
APIs

6. DEBS
Challenge
• Bridge
game
play
to
Internet
of
Things
• Sensors
in
shoes,
ball,
goalie
hands
• Analyze
Soccer
Game
• SpaOal
and
Temporal
Processing
• Visualize
Game
Play
• AcOvity
Heat
maps
• Recommend
Performance
Improvements
http://srinathsview.blogspot.com/2013/05/solving-debs-2013-grand-challenge-with.html

7. Bridging
Game
Play
to
IoT
7
Temporal
Tracking
SpaOal
Tracking
• Each
event
includes
the
locaOon
(x,y,z),
Ome
stamp,
velocity
and
acceleraOon
• Throughput
15,000
events
per
second

8. Analyze
Game
8
๏ BALL
POSSESSION
STATE
DIAGRAM
๏ SHOT
ON
GOAL
STATE
DIAGRAM

9. Analyze
Game
9
๏ RUNNING
LOCATION
STATES

10. 10
SoluOon
Architecture
๏ Capture
millions
of
events
with
Common
Events
Collector
๏ Derive
real-­‐Ome
decisions
with
temporal
processing
๏ Further
opOmize
recommendaOons
with
spaOal
Map-­‐Reduce
processing
analyOcs
๏ View
feedback
dashboards

11. Batch
AnalyOc
FoundaOon
11

12. Business
AcOvity
Monitor

13. Realizing
Real-­‐Ome
AnalyOcs
๏ Processing
Data
on
the
fly,
while
storing
a
minimal
amount
of
informaOon
and
responding
fast
(from
<1
ms
to
few
seconds)
๏ Idea
of
Event
streams
๏ A
series
of
events
in
Ome
๏ Enabling
technologies
๏ Stream
Processing
(Storm)
๏ Complex
Event
processing
(Siddhi)

14. Complex
Event
Processing

16. Complex
Event
Processing
Stream
Processing
• SQL
like
language
• Supports
powerful
temporal
operators
(e.g.
windows,
event
paberns)
• Focus
on
speed
• Harder
to
scale
• e.g.
WSO2
CEP,
Streambase,
Esper
• Operators
connected
in
a
network,
but
you
have
to
write
the
logic
• Distributed
by
design
• Focus
on
reliability
(do
not
loose
messages),
has
transacOons
• e.g.
Storm,
S4

17. Complex
Event
Processing
Operators
๏ Filters
or
transformaOons
(process
a
single
event)
๏ from Ball[v>10] select .. insert into ..
๏ Windows
+
aggregaOon
(track
window
of
events:
Ome,
length)
๏ from Ball#window.time(30s) select avg(v) ..
๏ Joins
(join
two
event
streams
to
one)
๏ from Ball#window.time(30s) as b join Players as p on p.v
< b.v
๏ Paberns
(state
machine
implementaOon)
๏ from Ball[v>10], Ball[v<10]*,Ball[v>10] select ..
๏ Event
tables
(map
a
database
as
an
event
stream)
๏ Define table HitV (v double) using .. db info ..

18. Usecase
1:
Running
Analysis
๏ Detect
when
speed
crosses
threshold
limits
define partition player by Players .id;
from s = Players [v <= 1 or v > 11] ,
t = Players [v > 1 and v <= 11]+ ,
e = Players [v <= 1 or v > 11]
select s.ts as tsStart , e.ts as tsStop ,s.id as
playerId ,
‘‘trot" as intensity , t [0].v as
instantSpeed ,
(e.ts - s.ts )/1000000000 as unitPeriod
insert into RunningStats partition by player;

19. Usecase
2:
Ball
possession
๏ Ball
possession
๏ Defined
as
possessing
the
ball
from
Ome
you
hit
it
unOl
someone
else
hit
it
or
ball
leaves
the
ground)

20. Usecase
3:
Heatmap
of
AcOvity
๏ Show
where
acOons
happened
(via
cells
defined
by
a
grid
of
64X100
etc.),
need
updates
once
every
second
๏ Can
solved
via
cell
change
boundaries,
but
does
not
work
if
one
player
stays
more
than
1
sec
in
the
same
cell.
So
need
to
join
with
a
Omer.

21. Usecase
4:
Detect
kicks
on
goal
๏ Main
Idea:
Detect
kicks
on
the
ball,
calculate
direcOon
aker
1m,
and
keep
giving
updates
as
long
as
it
is
in
right
direcOon

22. Results
for
DEBS
Scenarios

23. Big
Data
and
IoT
Changes
the
Game
๏ Real-­‐Ome
Status
Dashboard
๏ Player
game
play
๏ Momentum
indicators
๏ PredicOons
about
the
next
move
๏ Improvement
Dashboard
๏ Study
of
game
and
players
effecOveness
๏ Monitor
player
health
and
body
funcOons

24. Success
Target
Photo by John Trainoron Flickr http://www.flickr.com/photos/
trainor/2902023575/, Licensed under CC

25. SituaOonal
Context:
Traffic
Example
{incidents: [
{impacting: false,
eventCode: 0,
iconURL: "http://...",
lat: 38.743999,
shortDesc: "Exit ramp closed on I 95...",
type: 1,
severity: 0,
fullDesc: "In NEWINGTON exit ramp
closed on I-95 ...",
startTime: "2010-02-21T00:14:07",
lng: -77.188004,
id: "368598263",
endTime: "2010-02-27T05:04:19"
},
Image source: http://www.directoryofnewyorkcity.com/blog/2009/05/how-to-find-parking-in-new-york-city/
Real-time traffic map: http://www.mapquestapi.com/traffic/

26. Accelerate
interac6ons
inside
and
outside
the
organiza6on
Reduce
interac6on
fric6on
and
cost
Increase
engagement
and
enhance
produc6vity
Sense
business
ac6vity
and
automa6cally
adapt
Become
a
More
Connected
Business
http://wso2.com/landing/enabling-the-connected-business

27. Connected
Business
Reference
Architecture

28. Conceptual
Architecture

29. CEP
High
Availability

30. Siddhi
Storm
Bolt
(next
Major
Release)
๏ We
have
wriben
a
Siddhi
bolt
that
would
let
users
run
distributed
Siddhi
Queries
using
Storm
SiddhiBolt siddhiBolt1 = new SiddhiBolt( .. siddhi queries .. );
SiddhiBolt siddhiBolt2 = new SiddhiBolt( .. siddhi queries .. );
TopologyBuilder builder = new TopologyBuilder();
builder.setSpout("source", new PlayStream(), 1);
builder.setBolt("node1", siddhiBolt1, 1)
.shuffleGrouping("source", "PlayStream1");
..
builder.setBolt("LeafEacho", new EchoBolt(), 1)
.shuffleGrouping("node1", "LongAdvanceStream");
..
cluster.submitTopology("word-count", conf, builder.createTopology());

31. 31

32. Contact
us
!

33. 33
About
the
Presenter
๏ Chris
Haddad
๏ VP
Plaoorm
Evangelism
๏ Learn
more
about
me
๏ www.linkedin.com/in/cobiacomm/
๏ Follow
me
๏ @cobiacomm
on
Twiber
๏ hbp://blog.cobia.net/cobiacomm
๏ On
Google+
too