1. Gaining
Opera+onal
Intelligence
with
WSO2
BAM
Director,
Research,
WSO2
Inc.
Visi6ng
Faculty,
University
of
Moratuwa
Member,
Apache
SoEware
Founda6on
Research
Scien6st,
Lanka
SoEware
Founda6on
Srinath
Perera

2. About
WSO2
๏ Global
enterprise,
founded
in
2005
by
acknowledged
leaders
in
XML,
web
services
technologies,
standards
and
open
source
๏ Provides
only
open
source
plaHorm-­‐as-­‐a-­‐service
for
private,
public
and
hybrid
cloud
deployments
๏ All
WSO2
products
are
100%
open
source
and
released
under
the
Apache
License
Version
2.0.
๏ Is
an
Ac+ve
Member
of
OASIS,
Cloud
Security
Alliance,
OSGi
Alliance,
AMQP
Working
Group,
OpenID
Founda+on
and
W3C.
2
๏ Driven
by
Innova+on
๏ Launched
first
open
source
API
Management
solu+on
in
2012
๏ Launched
App
Factory
in
2Q
2013
๏ Launched
Enterprise
Store
and
first
open
source
Mobile
solu+on
in
4Q
2013

3. 3
What
WSO2
delivers

4. Business
Model
4

5. More
Informa+on
!
๏ Include
links
to
product
downloads,
white
paper
downloads
,
etc.
5

6. Outline
§ Connected
Business
and
Big
data
analy+cs
§ Big
Data
Technologies
from
WSO2
§ BAM
–
Batch
analy+cs
§ CEP
–
Real
+me
analy+cs
§ Lambda
Architecture
to
combine
§ From
your
business
to
insights
§ Iden+fy
KPIs
§ Calculate
KPIs
§ Build
Dashboards
§ Add
Drilldown
§ Alerts
and
Ac+ons

7. About
Connected
Business
webinar
series
§ March 05:Introduction to
the Connected Business
§ March 12: Enterprise
Integration made easy
with WSO2 ESB
§ ..
§ April 30: Gaining
Operational Intelligence
‪with WSO2 BAM
§ May 7: The WSO2
Advantage for a
Connected Business

8. Adap+ve
Connected
Business

9. Connected
Business

10. Be
Adap+ve
๏ Capture
business
ac+vity
(iden+fied
by
messages,
transac+on
execu+on,
and
data
state
changes)
and
store
data
points
for
future
analy+cs
๏ Deliver
automated
no+fica+ons
to
stakeholders
and
systems
based
on
business
ac+vity,
stakeholder
accountability,
and
authority.
๏ Automa+cally
adapt
business
process
execu+on
based
on
events
and
current
condi+ons

11. Big
Picture

12. Collec+ng
Data
๏ Data
collected
at
sensors
and
sent
to
big
data
system
via
events
or
flat
files
๏ Event
Streams:
we
name
the
events
by
its
content/
originator
• Get
data
through
– Point
to
Point
– Event
Bus
• E.g.
Data
bridge
– a
thrib
based
transport
we
did
that
do
about
400k
events/
sec

13. Making
Sense
of
Data
๏ To
know
(what
happened?)
๏ Basic
analy+cs
+
visualiza+ons
(min,
max,
average,
histogram,
distribu+ons
…
)
๏ Interac+ve
drill
down
๏ To
explain
(why)
๏ Data
mining,
classifica+ons,
building
models,
clustering
๏ To
forecast
๏ Neural
networks,
decision
models

14. Dashboards
and
last
Mile
§ Presenting information
o To end user
o To decision takers
o To scientist
§ Interactive exploration
§ Sending alerts
http://www.flickr.com/photos/
stevefaeembra/3604686097/

16. Big
Data
Architecture

17. Data
Collec+on
• Can
receive
events
via
SOAP,
HTTP,
JMS,
..
• WSO2
Events
is
highly
op+mized
version
(400K
events
TPS)
• Default
Agents
and
you
can
write
custom
agents.
Agent agent = new
Agent(agentConfiguration);
publisher = new AsyncDataPublisher(
"tcp://localhost:7612", .. );
StreamDefinition definition =
new StreamDefinition(STREAM_NAME,
VERSION);
definition.addPayloadData("sid",
STRING);
...
publisher.addStreamDefinition(definition
);
...
Event event = new Event();
event.setPayloadData(eventData);
publisher.publish(STREAM_NAME, VERSION,
event);

18. Business
Ac+vity
Monitor

19. BAM
Hive
Query
Find
how
much
+me
spent
in
each
cell.
CREATE EXTERNAL TABLE IF NOT EXISTS PlayStream …
select sid,
ceiling((y+33000)*7/10000 + x/10000) as cell,
count(sid)
from PlayStream
GROUP BY sid, ceiling((y+33000)*7/10000 + x/10000);

20. Complex
Event
Processor

21. CEP
Query
define partition sidPrt by PlayStream.sid, LocBySecStream.sid
from PlayStream#window.timeBatch(1sec)
select sid, avg(x) as xMean, avg(y) as yMean, avg(z) as zMean
insert into LocBySecStream partition by sidPrt
from every e1 = LocBySecStream ->
e2 = LocBySecStream [e1.yMean + 10000 > yMean
or yMean + 10000 > e1.yMean]
within 2sec select e1.sid
insert into LongAdvStream partition by sidPrt ;
Calculate the mean
location of each player
every second
Detect more
than 10m run

22. Lambda
Architecture

24. Extrac+ng
Opera+onal
Intelligence
from
your
Business

25. Key
Performance
Indicators
(KPIs)
๏ To
monitor,
you
need
to
measure
๏ KPIs
are
the
bojom
line
๏ For
a
organiza+on
it
may
be
profit,
or
revenue
๏ For
marke+ng,
it
may
be
LEADs
generated
๏ They
are
oben
domain
specifics
๏ For
each
KPI,
we
need
to
see
history,
find
rela+onships,
and
some+me
possible
future
trends.

26. Many
Stake
Holders,
Many
Views

27. Decision
Process

28. Dashboard
๏ Idea
is
to
get
the
“Overall
idea”
in
a
glance
๏ Like
your
car
dashboard
๏ Support
for
personaliza+on,
you
can
build
your
own
dashboard.
๏ This
is
the
entry
point
for
Drill
down
and
ac+vity
monitoring.

29. Drill
Down
๏ Find
the
problem,
and
find
what
caused
it.
๏ It
is
like
“Finding
CPU
bojleneck”
๏ Idea
is
to
recursively
find
categories
which
are
responsible
for
the
problem
๏ E.g.
See
year’s
sales,
then
drill
down
to
a
par+cular
year
to
see
per
month
data.
๏ See
country
level
data,
and
drill
down
to
see
state
wide
data.
๏ Ac+vity
monitoring
is
a
key
ac+vity
here

30. Case
Study:
Sobware
Support
Services
๏ KPIs
๏ Number
of
open
Issues
๏ Mean
and
Max
+me
to
close
a
issue
๏ SLA
viola+ons
๏ List
of
long
running
issues
๏ Issue
by
last
update
+me
๏ Data
๏ Collect
data
when
issue
is
created
or
updated
or
closed.
๏ E.g.
IssueChange(id,
ac+on,
+me,
userID)

31. From
data
to
KPI
๏ Group
issues
by
ID
grouping
each
issue
into
one
ac+vity
(Ac+vity
Monitoring)
๏ Calculate
KPIs
for
each
ac+vity
define partition IssueParition IssueChange.id;
from IssueChange[type=“open”] as s
->IssueChange[“closed”] as e
select e.time-s.time ..
Aggregate
data
in
different
level
to
support
drill
down
๏ hour,
weekly,
and
monthly
๏ Ver+cal,
product,

32. Aggregate
to
support
Drilldown
Dimension 1
Dimension 2

33. Dashboard
(Build
with
UES)
Click to
drill down
Click to
drill down
Click to
see details
about
Issues

34. Drill
Down
๏ Click
and
drill
down
issues
in
each
category
๏ Can
select
a
issue
and
see
all
ac+vi+es
happened
within
the
issue
๏ See
who
takes
most
+me
on
issues
๏ Look
at
day,
week,
month
trends

35. Analy+cs
Process

36. Ques+ons?

37. THANK
YOU!!!
37