On- demand webcast ‘Building Real-time Streaming Apps in Minutes’ available at http://bit.ly/1PwhobK

1. © 2015 Impetus Technologies
1
Building Real-time Streaming Applications in
minutes
Apache Storm made easy
Anand Venugopal
Head of Product for StreamAnalytix
Panelists: Punit Shah (Architect), Syed Bilal (Data Scientist)
Recorded version available at http://bit.ly/1PwhobK

2. © 2015 Impetus Technologies
2
BRIEF INTRO
• Big Data Solutions & Services company
• Unique in depth, expertise – started Big Data implementations in 2008
• Proven with customer success
• IP and Products
• We deliver - Business Impact from Big Data Solutions
• Technology expertise
• Data Science
• BusinessAnalytics
• Serving Fortune 1000 companies since 1996
• Large-scale and mission critical software platforms
• HQ: Los Gatos,CA; 1500 people
• Offshore operations in 3 cities in India
Recorded version available at http://bit.ly/1PwhobK

3. © 2015 Impetus Technologies
3
AGENDA
• Modern Agile Enterprise
• Business Drivers
• Lambda Architecture
• Enterprise “Nervous System”
• StreamAnalytix Demo
• Building real-time data pipelines on Apache Storm – easy and fast
• Blending real-time and batch analytics in Social Media Analytics
• Announcements
• Q&A
Recorded version available at http://bit.ly/1PwhobK

4. © 2015 Impetus Technologies
4
DRIVERS FORTHE MODERN AGILE ENTERPRISE
Business
Operations
Business
Analytics
Real-time
Streaming Analytics
Recorded version available at http://bit.ly/1PwhobK

5. © 2015 Impetus Technologies
5
BUSINESS OPERATIONS INCREASINGLY REAL-TIME IN NATURE
Fleet Operations & Logistics Security
Mobile Devices andApps Energy Industry IT Operations
Recorded version available at http://bit.ly/1PwhobK

6. © 2015 Impetus Technologies
6
DRIVERS FOR A MODERN AGILE ENTERPRISE –THE CUSTOMER
You and II need it when
I need it
Stop sending
me blanket
Ads
My income
went up I can
buy a house
now
I already
bought it and
wont need it
for a few years
Dear Call center agent – I wish
you knew who I am and my
recent issues – I am about to
pull the plug!
Recorded version available at http://bit.ly/1PwhobK

7. © 2015 Impetus Technologies
7
CONTEXT UNAWARE  BAD CUSTOMER EXPERIENCE
Recorded version available at http://bit.ly/1PwhobK

8. © 2015 Impetus Technologies
8
CONTEXT AWARE  POSITIVE CUSTOMER EXPERIENCE
Multi-channel
engagement in
real-time
Context
Sensitive service
Happy customers,
Loyalty, Revenue,
Profits, Growth
Recorded version available at http://bit.ly/1PwhobK

9. © 2015 Impetus Technologies
9
MODERN AGILE ENTERPRISE = IMPACT FROM DATA ANALYTICS
DATA ANALYTICS IMPACT
Recorded version available at http://bit.ly/1PwhobK

10. © 2015 Impetus Technologies
10
MODERN AGILE ENTERPRISE – DATA MANAGEMENT
DATA ANALYTICS IMPACT
• Sense/ Capture/Transport
• Ingest,Transformation,
Pre-process
• Volume,Variety,Velocity –
Big Data +Traditional
merge
• Quality, Governance,
Security, Lineage, Life-
cycle
Recorded version available at http://bit.ly/1PwhobK

11. © 2015 Impetus Technologies
11
MODERN AGILE ENTERPRISE – ANALYTICS MATURITY
DATA ANALYTICS IMPACT
• Descriptive  Predictive 
Prescriptive (ML ?)
• Search and Exploration,
Sandbox – Data Scientists
• Across silos, Includes external
data, Fast blending
• Speed – Latency of access and
Recency of data
Recorded version available at http://bit.ly/1PwhobK

12. © 2015 Impetus Technologies
12
MODERN AGILE ENTERPRISE – REAL IMPACT
DATA ANALYTICS IMPACT
• Converting Insights to
Impact, Decision culture
• Measuring, closed-loop
feedback, Improving
• Customers Experience a
difference
• Financial measures are
clearly impacted
Recorded version available at http://bit.ly/1PwhobK

13. © 2015 Impetus Technologies
13
MODERN AGILE ENTERPRISE – SUMMARY
DATA ANALYTICS IMPACT
• Sense/ Capture/Transport
• Ingest,Transformation,
Pre-process
• Volume,Variety,Velocity –
Big Data +Traditional
merge
• Quality, Governance,
Security, Lineage, Life-
cycle
• Descriptive  Predictive 
Prescriptive (ML ?)
• Search and Exploration,
Sandbox – Data Scientists
• Across silos, Includes external
data, Fast blending
• Speed – Latency of access and
Recency of data
• Converting Insights to
Impact, Decision culture
• Measuring, closed-loop
feedback, Improving
• Customers Experience a
difference
• Financial measures are
clearly impacted
Recorded version available at http://bit.ly/1PwhobK

14. © 2015 Impetus Technologies
14
BATCHVS. REAL-TIME BUSINESS PROCESS
SENSE Days ANALYSE Weeks ACT
SENSE ANALYSE ACT
Sec/ ms
Batch
Real time
Sec/ ms
Recorded version available at http://bit.ly/1PwhobK

15. © 2015 Impetus Technologies
15
BARRIERTO BEING “AGILE” –THE “BATCH GAP”
The batch workflow is too slow
Views are out of date
Not yet
absorbed.Data absorbed into BatchViews
Now
Time
Just a few hours of data.
Recorded version available at http://bit.ly/1PwhobK

16. © 2015 Impetus Technologies
16
t
now
Hadoop works great back
here
RT-Ax works
here
BLENDED VIEW – HISTORICAL AND NOW
Blended viewBlended viewBlended View
Recorded version available at http://bit.ly/1PwhobK

17. © 2015 Impetus Technologies
17
LAMBDA ARCHITECTURE : BIGAND FAST DATA COMBINED
Batch Layer
All data
Pre-computed
information
Batch re-compute
Speed Layer
All data
Pre-computed
information
Real time increment
Batch view
Serving Layer
Batch view
Merge
Real time view
Real time view
All
Incoming
Data Query
Recorded version available at http://bit.ly/1PwhobK

18. © 2015 Impetus Technologies
18
THE FULL PICTURE OF A MODERN AND AGILE
ARCHITECTURE
Landing and
ingestion
Structured
Unstructured
External
Social
Machine
Geospatial
Time Series
Streaming
Provisioning,Workflow, Monitoring and Security
Enterprise
Data Lake
Predictive
applications
Exploration &
discovery
Enterprise
applications
Real-Time applications
Traditional
data
repositories
RDBMS MPP
Compliance, Governance, Information Lifecycle, Data
Lineage, Enterprise Meta Data Management
Recorded version available at http://bit.ly/1PwhobK

19. © 2015 Impetus Technologies
19
SMART ENTERPRISE BIG DATA BUS
Recorded version available at http://bit.ly/1PwhobK

20. © 2015 Impetus Technologies
20
Enterprise Class Real time Streaming Analytics Platform
A Product developed and offered by
Recorded version available at http://bit.ly/1PwhobK

21. © 2015 Impetus Technologies
21
AT A GLANCE
StreamAnalytix is a software platform that enables enterprises to analyze and
respond to events in real-time at Big Data scale. It is designed to rapidly build and
deploy streaming analytics applications for any industry vertical, any data format,
and any use-case
Recorded version available at http://bit.ly/1PwhobK

22. © 2015 Impetus Technologies
22
STREAMANALYTIX BLOCK DIAGRAM
Recorded version available at http://bit.ly/1PwhobK

23. © 2015 Impetus Technologies
23
DEMO – SOCIAL MEDIA ANALYTICS
Recorded version available at http://bit.ly/1PwhobK

24. © 2015 Impetus Technologies
24
TEXT ANALYTICS ALGORITHMS
TEXT CLASSIFICATION TOPIC MODELINGSENTIMENTANALYSIS
Recorded version available at http://bit.ly/1PwhobK

25. © 2015 Impetus Technologies
25
SENTIMENT ANALYSIS
• Lexicon based approach; Unsupervised model
• Lexicon prepared offline with Matrix
decomposition based approach.
• Custom language rules added:
• Surrounding words could negate, amplify etc.
• Dealing with Slang, abbreviations.
Recorded version available at http://bit.ly/1PwhobK

26. © 2015 Impetus Technologies
26
TEXT CLASSIFICATION
• 20 categories; Multiple labels if applicable
• Semantic similarity approach based on
Matrix Decomposition
• Classification Model trained offline. Can be
extended to more domains.
Recorded version available at http://bit.ly/1PwhobK

27. © 2015 Impetus Technologies
27
BATCH PROCESS –TOPIC MODELING
• Identification of main topics of conversation
within text.
• Model based on exploiting co-occurrence of
words across documents.
• Current model is specifically tuned for
informal text like twitter to deal with issues
like misspellings, slangs.
Recorded version available at http://bit.ly/1PwhobK

28. © 2015 Impetus Technologies
28
DEMO – SOCIAL MEDIA ANALYTICS
Recorded version available at http://bit.ly/1PwhobK

29. © 2015 Impetus Technologies
29
REAL-TIME MODEL BUILDING
read Kafka
Channel (Storm
Spout)
eventsKafka
Topic
Custom
Processor
(Model scoring
in real-time)
Kafka
Channel
(Storm Spout)
Custom
Processor
(Model Building
in real-time)
read
events
Update model for scoring in real-time
using the platform orchestration layer
(functionality of StreamAnalytix)
Scored
event sent
downstream
Two copies of the event are read into two
different topologies.
One Topology is responsible for
building/enhancing the model in real-time. This
component will also access historical data to
ensure model creation uses “all” the data.
The second topology is responsible for applying
the model in real-time for scoring.
At a certain threshold, the ‘newer’ version of the
model can be applied to the scoring component
by using the platform orchestration layer
without any down-time
Another possible scenario:
The real-time scoring and learning can be done by the same
component. In such situations, a fork of the incoming
message is not required. The system design/ configuration of
doing learning and scoring in the same component is a
matter of scale and performance tuning
Historical/
training
data
read
Recorded version available at http://bit.ly/1PwhobK

30. © 2015 Impetus Technologies
30
POLL
Functionality Cores/ Capacity Duration
Free 1 Limited Unlimited Forever
Free 2 ALL Limited Forever
Trial ALL Unlimited 60 days
Which of these options for trying StreamAnalytix would you download ?
Recorded version available at http://bit.ly/1PwhobK

31. © 2015 Impetus Technologies
31
Q&A
(Use the chat/Q&A panel)
Email us at inquiry@streamanalytix.com
www.StreamAnalytix.com
?
Request: On-premise and Cloud based trial and/or Proof of
concept