When an enterprise gets connected, the success of the business relies on how quickly and effectively they can identify game-changing events from the avalanche of events that are produced by the eco system. Enterprise systems, such as enterprise resource planning, system monitoring, stock transaction, and financial transaction systems, generate an enormous amount of events that can contain useful data. Processing this data efficiently in real-time can provide a competitive edge. It is not hard to “analytics” wash your organization, and use analytics in a few places. However, it’s not easy to use analytics to generate actionable insights, make a difference, and achieve competitive advantage. There are many pitfalls, and you need to have a fundamental understanding about your goals. A common mistake is starting too big, and trying to design the end system before understanding the problem. In this session, Srinath will discuss some key fundamentals of moving beyond hype to gaining insightful information by effectively analyzing an enterprise’s data with the use of technology.

1. Big Data: Beyond Hype to Insight
Srinath Perera (@srinath_perera)
VP – Research
WSO2

2. Success Stories
• Money Ball ( Baseball draDing)
• Nate Silver predicted outcomes in 49 of the 50
states in the 2008 U.S. PresidenQal elecQon
• Cancer detecQon from Biopsy cells ( Big Data
find 12 paUerns while we only knew 9),
hUp://go.ted.com/CseS
• Bristol-Myers Squibb reduced the Qme it takes
to run clinical trial simulaQons by 98%
• Xerox used big data to reduce the aUriQon rate
in its call centers by 20%.
• Kroger Loyalty programs ( growth in 45
consecuQve quarters)

3. Premise of Big Data
If you collect data about your business, and feed it to a Big Data
system, you will find useful insights that will provide competitive
advantage
– (e.g. Analysis of data sets can find new correlations to "spot business
trends, prevent diseases, combat crime and so on”. [Wikipedia])
Underline assumption is that way we
operate, and organizations are
inefficient.

4. Big Data as a
Way to OpQmize
• Assumptions: Once you
identify your sickness, you are
halfway cured
• You must know what is worth
Optimizing
premature
opQmizaQon is
the root of all
evil

5. “Big Data Washing”
You can tick yes, but unlikely to make a difference

6. How to Big Data Wash
your System in 24 hours?
• Publish collect the data you can with
minimal effort
• Do lot of simple aggregaQons
• Figure out what data combinaQons makes
predest pictures
• Throw in some machine learning
algorithms, predict something but don’t
compare
• Create a cool dashboard and do a cool
demo, and say that you are just scratching
the surface!!

7. Are Insights are
automatic?
• I wish
• Only if we have right data
• Only if we look at the right place
• Only if such insights are there
• Only if we found the insights

8. Value Preposition

9. Big Data Tools
• KPIs
• AnalyQcs ( Batch, Real-Qme,
InteracQve, PredicaQve)
• VisualizaQons, Dashboards
• Alerts
• Sensors ( and other data
collecQon plumbing)

10. KPIs and their Role
• KPIs (Key Performance Indicators) are numbers
that can give you an idea about performance
of something
– E.g. Countries have them ( GDP, Per Capita
Income, HDI index etc)
• Examples
– Company Revenue
– LifeQme value of a customer
– Revenue per Square foot ( in retail industry)
• Idea is to define them and monitor them. But
defining them is hard work!!
• ODen one indicator tells half the story, and you
need several that cover different angles

11. What is a Dashboard?
• Think a car dashboard
• It give you idea about
overall system in a glance
• It is boring when all is
good, and grab aUenQon
when something is wrong
• ODen have support for
drill down and find root
cause

12. Alerts
• NoQficaQons ( sent via email,
SMS, Pager etc.)
• Goal is to give you peace of mind
( not having to check all the Qme)
• They should be specific
• They should be infrequent
• They should have very low false
posiQves
• Let users control sensiQvity

13. You need a Human in the Loop
Systems that digest your data, take decisions, and run the system by itself, they can only
be used with limited applications Yet
(e.g. Algorithmic trading, Showing Advertisements, or War)

14. Decisions, Actions, and
Drill down
• Operators need to see the data in
context, and drill down into detail to
understand the root cause
• Typical model is to start from an alert
or dashboard, see data in context
(other transacQons around same
Qme, what does same user did before
and aDer etc.) and then let the user
drill down
• For example,
hUp://wso2.com/videos/wso2-fraud-
detecQon-soluQon

15. Role of Realtime Analytics
• Use to detect something very fast!
Within few milliseconds to few
seconds.
• Very powerful in detecQng
condiQons over Qme (e.g. ball
possession in a football game)
• Alerts are done through RealQme
analyQcs

16. Role of Predictive Analytics
• PredicQve analyQcs learn a problem from
examples
– E.g. learn to drive
• Two main cases are
– PredicQng next value or values (e.g. electricity load
predicQon)
– PredicQng category (e.g. SPAM or not for a email)
• Used to grouping, to generate alerts, or to
augment visualizaQons
• Need lot of experQse to create correct models
and use them.

17. Big Data Pipeline

18. Doing it Once is Cheap,
Sedng up a system to do it
conQnuously is Expensive
Do your scenarios ad-hoc first (hire some experQse if you must),
before sedng up a system that does it every day

19. Keeping it running is Even Harder
● Incorporate ConQnuous data
o Integrate data conQnuously
o We get feedback about effecQveness
of decisions (e.g. Accuracy of Fraud)
● Track and update models
o Trends change
o Generate models in batch mode and
update

20. Templates for Big Data Projects
• Use existing Dataset: I already have a data set, and list of
potential problems, and figure out how to fix it.
• **Fix a known Problem: Find a problem, collect data about it,
analyze, visualize, build a model and improve. Then build a
dashboard to monitor.
• Improve Overall Process: Instrument processes ( start with
most crucial), find KPIs, analyze and visualize the processes, and
improve
• Find Correlations: Collect all available data, data mine the data
or visualize, find interesting correlations.

21. Actionable Insights
are the Key!!
• Insights are about significant event that
warrant aUenQon ( e.g. more than two
technical issues would lead customer to
churn)
• Decision makers can idenQfy the
context associated with the insight
( e.g. operators can see though history
of customers who qualify)
• Decision makers can do something
about the insight ( e.g. can work with
customers to reassures and fix)

22. Think Deeply about Who is using your
System and How?

23. Summary
• Big Data provide a way to OpQmize
• Tools
– KPIs
– AnalyQcs ( Batch, Real-Qme, InteracQve, PredicaQve)
– VisualizaQons, Dashboards
– Alerts
– Sensors ( and other data collecQon plumbing)
• Start small
• Try out with data sets before setup a system
• Find a high impact problem and make it work
end to end
• Pay aUenQon to user Experience

24. Thank You!
#WSO2ConEU
Share your feedback for this session
wso2con.com/app