SlideShare a Scribd company logo
1 of 37
Download to read offline
Finding the Signal in the Noise
June 15, 2015
Webinar Presentation
Nova Spivack, CEO
sales@bottlenose.com
What is Bottlenose For?
Bottlenose discovers the threats and
opportunities that impact your business
Bottlenose does this using patented stream
intelligence technology
2
Key Stream Intelligence Use-Cases
Threats
• Risk detection
• Crisis mitigation
• Competitive threats
• Reputational threats
• Cyber threat detection
Opportunities
• Audience and customer insights
• Innovation and research
• New business and market opportunities
• Competitive intelligence
• Product and marketing intelligence
3
Vision
Stream Intelligence
Our mission is to build the leading business intelligence company for stream data
Stream data is the fastest growing segment of data. It includes all types of live or historical, unstructured or
structured, time-stamped data, such as: email and messaging data, social media, mobile data, news, IT log
data, CRM data, support data, sales data, Web and app analytics data, financial data, sensor and device data.
We have built the first unified platform and application for automating the discovery of actionable
intelligence across any stream data sources – We call this stream intelligence.
4
... the future belongs to raw unstructured or semi-
structured data from both internal and external
sources - increasingly delivered in (near) real-time.
This data has great value yet most organizations
do not have the tech infrastructure to handle all
this data.” - IDC
Problem: Massive growth of unstructured data cannot be
managed effectively with existing Tech infrastructure
Real-Time Discovery Against Streaming
Data is Required:
5
● There are never going to be
enough data scientists or
analysts to cope with the rise
of unstructured stream data in
the enterprise
● Analysts need automated
stream intelligence tools to
help them deal with the
volume, velocity and variety
of stream data
Analysts Are
Drowning in Streams
Solution: Bottlenose Automates Stream Intelligence
• Bottlenose provides the most advanced
automated stream intelligence that
automatically finds patterns such as trends,
anomalies, threats, opportunities and
correlations in stream data
• Bottlenose is extremely easy to use and easy to
derive value from right away without extensive
engineering and IT involvement or long
professional solutions
• The platform combines both internal enterprise
data and external data from social, broadcast,
web and other areas.
We are In The Stream Intelligence Sweet Spot
The Bottlenose solution is a new generation of tools that automates the production
of actionable intelligence from stream data
VarietyVelocity
Volume
&
ELK Stack
7
Stream Intelligence is “BI 3.0”
(Source: HBR)
8
Competitive Advantage from Coping with Stream Data 9
Competitive Advantage from Coping with Stream Data 10
Competitive Advantage from Coping with Stream Data 11
Competitive Advantage from Coping with Stream Data 12
Bottlenose
Platform
Social & traditional media
(social networks, blogs,
Forums, newswires)
98% of all live TV & Radio
Broadcasts
Enterprise Data
(Sales, financials, Web
analytics, IT systems, email,
internal databases, etc.) Web Data, commercial data
sources, financial market
data sources, public data
sources
Machine and sensor data
(Internet-of-things, machine
data, weather data, etc.)
13
Generate Actionable Intelligence from ANY Stream Data
13
Stream Intelligence Pipeline
Applications
Rules & Agents
Alerts/Actions Based on Business Interests
Stream Data
Storage & API
Long-term storage,
real-time access,
search & APIs
Trend Detection
Extrapolation, Correlation/Clustering
Data Mining & Analytics
~30 Entity Types and ~150 Metrics
Ingestion & Enrichment
Push/Pull of Unstructured/Structured Data
Data in Motion
Alerts &
Actions
New
Patterns
Entities &
Metrics
14
Breaking News
Automatically Discover Threats and Opportunities
Known Issues
Unknown/Emerging
Issues
Customer Problems
Enterprise Risk Factor
Fraud Risk
Product Recall
Competitive Threat
Cyber Attack
Focal
Interest
Power Outage
Natural DisasterTraffic Congestion
Device Failure
Financial Trading
Anomaly
Reputation Risk
Intellectual Property
Violation
15
Continuous High-Volume Stream Analytics
• 3 billion live + historical messages analyzed every hour
• 72 billion records analyzed per day + predictive analytics on 7.2 billion
• 67,000,000 new messages ingested every day
• Trend detection at a rate of 1 million events per second
• 30 entity types recognized * 150 metrics per entity * 10’s of millions of entities =
~50 to 100 billion time series monitored and analyzed continuously
• Growing to 200 Terabytes of data stored & analyzed continuously in 2015
1000s of High-Level Detected Trends Per Hour
• Automated data science layer applies machine learning, statistics, predictive
analytics to correlate, cluster, predict and analyze emergent trends
We See the Near Future Before Anyone Else
• 80% of the time, our system detects breaking news and emerging threats,
opportunities and keywords up to 10’s to 100’s of minutes ahead of the media,
Twitter, ad networks, etc. Similar advantages against non-text data sources
Key Metrics
Bottlenose
analyzes 72
billion data
records
every day
16
Demo: Data Agnostic Stream Intelligence
17
Customer Facing Products
● Analytics, intelligence, and
discovery engine
○ Nerve Center
○ Full-stack offering
● Streaming data services to
applications
○ Bottlenose API (Platform)
18
‣ Advanced filtering & aggregations using
simple OLAP interface
‣ “Interactive Analytics” thanks to sub-second
query response time
‣ Add new data sources using central mapping
system
Analytics Engine 19
A sophisticated Semantic approach is required to make sense of the raw data. The
structure of data can be derived based on entities/dimensions the system has a pattern for.
Machine learning techniques can begin to make inferences and match to known profiles as
data flows in.
One of the most powerful capabilities is when different data sources need to be compared.
A system like ours automatically normalizes them. For example, when the data has
different time granularity, we automatically align different time periods in order to find
overlaps.
Of course the Semantic engine can also be adjusted with a vertical industries unique facts,
relationships, and jargon.
Need for a Radical New Form of Information Retrieval:
Semantic meaning bottoms up from raw data
20
Application - Nerve Center
Our application provides a powerful suite of tools to find business insights in
streaming data:
● Monitor: Real-time monitoring with powerful live visualizations.
● Analyze: Fast interactive analytics to dig deep into the data.
● Discover: Automated insight discovery. Get notified when new patterns
are detected.
● Customize: Reports and live dashboards can be created for any vertical
by mix-and-matching insights & visualizations across any combination of
data streams
21
Bottlenose Platform
Ingest Augment
Analytics
Engine
Discovery Engine
Nerve Center®
Store
22
Augmentation Engine
Analytics
Engine
Detection Engine
Correlations Engine
Rules & Agent Engine
Processing Layers
Depth of Insight
23
Augmentation Engine
Analytics
Engine
Detection Engine
Correlations Engine
Rules & Agent Engine
Processing Layers
Depth of Insight
24
Augmentation Engine
Analytics
Engine
Detection Engine
Correlations Engine
Rules & Agent Engine
Processing Layers
Depth of Insight
25
‣ Typical topic stream like “Beyonce” (Pepsi)
‣ 4M new events (data records) per month
‣ ~8M unique entities tracked per month
‣ ~8M unique entities x 150 metrics x many
time buckets = A lot of data points
‣ And this is just 1 stream. We have thousands
of these running at all times...
Data Points 26
Augmentation Engine
Analytics
Engine
Detection Engine
Correlations Engine
Rules & Agent Engine
Depth of Insight
Processing Layers 27
‣ Systematically walk through all data points.
‣ Continuous stream of categorized signals.
Searchable.
Detection Engine 28
29
Detection Engine
Anticipate
Python servers
for trend
detection &
extrapolation
Detector
Workers that continuously aggregate
entities and fetch corresponding metrics
Context Gathering
Finding additional meta-data around
detections
Time Series Extrapolation
Clustering
Rolling clustering of trends based on
overlapping meta-data and a variety of
distance functions
Analytics Requests
Entities & Time Series
Analytics Requests
Related Entities
Find Related Trends
Related Trends
New and updated trends
32
‣ Python library, using SciPy
‣ Algorithms for detection & extrapolation in
time series data
‣ Includes tooling for debugging, training and
simulating
‣ ~500 detections/CPU-core/second
Anticipate 33
Augmentation Engine
Analytics
Engine
Detection Engine
Correlations Engine
Rules & Agent Engine
Depth of Insight
Processing Layers 34
Augmentation Engine
Analytics
Engine
Detection Engine
Correlations Engine
Rules & Agent Engine
Depth of Insight
Processing Layers 35
Our automated insight discovery on streaming data
enables “intelligence as a service for every
organization”
Intelligence as a Service 36
sales@bottlenose.com
www.bottlenose.com
http://twitter.com/bottlenoseapp
Contact
37

More Related Content

What's hot

Data Governance and Data Science to Improve Data Quality
Data Governance and Data Science to Improve Data QualityData Governance and Data Science to Improve Data Quality
Data Governance and Data Science to Improve Data QualityDATAVERSITY
 
DataEd Slides: Growing Practical Data Governance Programs
DataEd Slides: Growing Practical Data Governance ProgramsDataEd Slides: Growing Practical Data Governance Programs
DataEd Slides: Growing Practical Data Governance ProgramsDATAVERSITY
 
Introduction to Data Governance
Introduction to Data GovernanceIntroduction to Data Governance
Introduction to Data GovernanceJohn Bao Vuu
 
Data-Ed: Monetizing Data Management
Data-Ed: Monetizing Data Management  Data-Ed: Monetizing Data Management
Data-Ed: Monetizing Data Management Data Blueprint
 
Data-Ed Online: Trends in Data Modeling
Data-Ed Online: Trends in Data ModelingData-Ed Online: Trends in Data Modeling
Data-Ed Online: Trends in Data ModelingDATAVERSITY
 
ADV Slides: Organizational Change Management in Becoming an Analytic Organiza...
ADV Slides: Organizational Change Management in Becoming an Analytic Organiza...ADV Slides: Organizational Change Management in Becoming an Analytic Organiza...
ADV Slides: Organizational Change Management in Becoming an Analytic Organiza...DATAVERSITY
 
Implementing Big Data, NoSQL, & Hadoop - Bigger Is (Usually) Better
Implementing Big Data, NoSQL, & Hadoop - Bigger Is (Usually) BetterImplementing Big Data, NoSQL, & Hadoop - Bigger Is (Usually) Better
Implementing Big Data, NoSQL, & Hadoop - Bigger Is (Usually) BetterDATAVERSITY
 
A Modern Approach to DI & MDM
A Modern Approach to DI & MDMA Modern Approach to DI & MDM
A Modern Approach to DI & MDMDATAVERSITY
 
The Value of Metadata
The Value of MetadataThe Value of Metadata
The Value of MetadataDATAVERSITY
 
Data Structures - The Cornerstone of Your Data’s Home
Data Structures - The Cornerstone of Your Data’s HomeData Structures - The Cornerstone of Your Data’s Home
Data Structures - The Cornerstone of Your Data’s HomeDATAVERSITY
 
Master Data Management - Aligning Data, Process, and Governance
Master Data Management - Aligning Data, Process, and GovernanceMaster Data Management - Aligning Data, Process, and Governance
Master Data Management - Aligning Data, Process, and GovernanceDATAVERSITY
 
ADV Slides: Strategies for Fitting a Data Lake into a Modern Data Architecture
ADV Slides: Strategies for Fitting a Data Lake into a Modern Data ArchitectureADV Slides: Strategies for Fitting a Data Lake into a Modern Data Architecture
ADV Slides: Strategies for Fitting a Data Lake into a Modern Data ArchitectureDATAVERSITY
 
How to Create Controlled Vocabularies for Competitive Intelligence
How to Create Controlled Vocabularies for Competitive IntelligenceHow to Create Controlled Vocabularies for Competitive Intelligence
How to Create Controlled Vocabularies for Competitive IntelligenceIntelCollab.com
 
Using Data Governance to Protect Sensitive Data
Using Data Governance to Protect Sensitive DataUsing Data Governance to Protect Sensitive Data
Using Data Governance to Protect Sensitive DataDATAVERSITY
 
DAS Webinar: Emerging Trends in Data Architecture – What’s the Next Big Thing?
DAS Webinar: Emerging Trends in Data Architecture – What’s the Next Big Thing?DAS Webinar: Emerging Trends in Data Architecture – What’s the Next Big Thing?
DAS Webinar: Emerging Trends in Data Architecture – What’s the Next Big Thing?DATAVERSITY
 
DAMA International Symposium San Diego CA 03-17-2008
DAMA International Symposium San Diego CA 03-17-2008DAMA International Symposium San Diego CA 03-17-2008
DAMA International Symposium San Diego CA 03-17-2008Robert J. Abate, CBIP, CDMP
 
Metadata Strategies - Data Squared
Metadata Strategies - Data SquaredMetadata Strategies - Data Squared
Metadata Strategies - Data SquaredDATAVERSITY
 
Data-Ed: Data Architecture Requirements
Data-Ed: Data Architecture Requirements  Data-Ed: Data Architecture Requirements
Data-Ed: Data Architecture Requirements Data Blueprint
 
Data governance, Information security strategy
Data governance, Information security strategyData governance, Information security strategy
Data governance, Information security strategyvasanthi4ever
 
Slides: Data Governance Reality Check
Slides: Data Governance Reality CheckSlides: Data Governance Reality Check
Slides: Data Governance Reality CheckDATAVERSITY
 

What's hot (20)

Data Governance and Data Science to Improve Data Quality
Data Governance and Data Science to Improve Data QualityData Governance and Data Science to Improve Data Quality
Data Governance and Data Science to Improve Data Quality
 
DataEd Slides: Growing Practical Data Governance Programs
DataEd Slides: Growing Practical Data Governance ProgramsDataEd Slides: Growing Practical Data Governance Programs
DataEd Slides: Growing Practical Data Governance Programs
 
Introduction to Data Governance
Introduction to Data GovernanceIntroduction to Data Governance
Introduction to Data Governance
 
Data-Ed: Monetizing Data Management
Data-Ed: Monetizing Data Management  Data-Ed: Monetizing Data Management
Data-Ed: Monetizing Data Management
 
Data-Ed Online: Trends in Data Modeling
Data-Ed Online: Trends in Data ModelingData-Ed Online: Trends in Data Modeling
Data-Ed Online: Trends in Data Modeling
 
ADV Slides: Organizational Change Management in Becoming an Analytic Organiza...
ADV Slides: Organizational Change Management in Becoming an Analytic Organiza...ADV Slides: Organizational Change Management in Becoming an Analytic Organiza...
ADV Slides: Organizational Change Management in Becoming an Analytic Organiza...
 
Implementing Big Data, NoSQL, & Hadoop - Bigger Is (Usually) Better
Implementing Big Data, NoSQL, & Hadoop - Bigger Is (Usually) BetterImplementing Big Data, NoSQL, & Hadoop - Bigger Is (Usually) Better
Implementing Big Data, NoSQL, & Hadoop - Bigger Is (Usually) Better
 
A Modern Approach to DI & MDM
A Modern Approach to DI & MDMA Modern Approach to DI & MDM
A Modern Approach to DI & MDM
 
The Value of Metadata
The Value of MetadataThe Value of Metadata
The Value of Metadata
 
Data Structures - The Cornerstone of Your Data’s Home
Data Structures - The Cornerstone of Your Data’s HomeData Structures - The Cornerstone of Your Data’s Home
Data Structures - The Cornerstone of Your Data’s Home
 
Master Data Management - Aligning Data, Process, and Governance
Master Data Management - Aligning Data, Process, and GovernanceMaster Data Management - Aligning Data, Process, and Governance
Master Data Management - Aligning Data, Process, and Governance
 
ADV Slides: Strategies for Fitting a Data Lake into a Modern Data Architecture
ADV Slides: Strategies for Fitting a Data Lake into a Modern Data ArchitectureADV Slides: Strategies for Fitting a Data Lake into a Modern Data Architecture
ADV Slides: Strategies for Fitting a Data Lake into a Modern Data Architecture
 
How to Create Controlled Vocabularies for Competitive Intelligence
How to Create Controlled Vocabularies for Competitive IntelligenceHow to Create Controlled Vocabularies for Competitive Intelligence
How to Create Controlled Vocabularies for Competitive Intelligence
 
Using Data Governance to Protect Sensitive Data
Using Data Governance to Protect Sensitive DataUsing Data Governance to Protect Sensitive Data
Using Data Governance to Protect Sensitive Data
 
DAS Webinar: Emerging Trends in Data Architecture – What’s the Next Big Thing?
DAS Webinar: Emerging Trends in Data Architecture – What’s the Next Big Thing?DAS Webinar: Emerging Trends in Data Architecture – What’s the Next Big Thing?
DAS Webinar: Emerging Trends in Data Architecture – What’s the Next Big Thing?
 
DAMA International Symposium San Diego CA 03-17-2008
DAMA International Symposium San Diego CA 03-17-2008DAMA International Symposium San Diego CA 03-17-2008
DAMA International Symposium San Diego CA 03-17-2008
 
Metadata Strategies - Data Squared
Metadata Strategies - Data SquaredMetadata Strategies - Data Squared
Metadata Strategies - Data Squared
 
Data-Ed: Data Architecture Requirements
Data-Ed: Data Architecture Requirements  Data-Ed: Data Architecture Requirements
Data-Ed: Data Architecture Requirements
 
Data governance, Information security strategy
Data governance, Information security strategyData governance, Information security strategy
Data governance, Information security strategy
 
Slides: Data Governance Reality Check
Slides: Data Governance Reality CheckSlides: Data Governance Reality Check
Slides: Data Governance Reality Check
 

Similar to SmartData Webinar Slides: How to analyze 72 billion messages a day to find trends

Streaming and Visual Data Discovery for the Internet of Things
Streaming and Visual Data Discovery for the Internet of ThingsStreaming and Visual Data Discovery for the Internet of Things
Streaming and Visual Data Discovery for the Internet of ThingsDatawatchCorporation
 
Blocks & Bots - Digital Summit Harvard Business School 2015
Blocks & Bots - Digital Summit Harvard Business School 2015Blocks & Bots - Digital Summit Harvard Business School 2015
Blocks & Bots - Digital Summit Harvard Business School 2015Mona M. Vernon
 
Big Data & Analytics, Peter Jönsson
Big Data & Analytics, Peter JönssonBig Data & Analytics, Peter Jönsson
Big Data & Analytics, Peter JönssonIBM Danmark
 
Impacto del Big Data en la empresa española
Impacto del Big Data en la empresa españolaImpacto del Big Data en la empresa española
Impacto del Big Data en la empresa españolaParadigma Digital
 
How to make your data scientists happy
How to make your data scientists happy How to make your data scientists happy
How to make your data scientists happy Hussain Sultan
 
Ai design sprint - Finance - Wealth management
Ai design sprint  - Finance - Wealth managementAi design sprint  - Finance - Wealth management
Ai design sprint - Finance - Wealth managementChinmay Patel
 
How Big Data Changes Our World
How Big Data Changes Our WorldHow Big Data Changes Our World
How Big Data Changes Our WorldKim Escherich
 
Big data analytics, research report
Big data analytics, research reportBig data analytics, research report
Big data analytics, research reportJULIO GONZALEZ SANZ
 
Liberating data power of APIs
Liberating data power of APIsLiberating data power of APIs
Liberating data power of APIsBala Iyer
 
SAP Forum Ankara 2017 - "Verinin Merkezine Seyahat"
SAP Forum Ankara 2017 - "Verinin Merkezine Seyahat"SAP Forum Ankara 2017 - "Verinin Merkezine Seyahat"
SAP Forum Ankara 2017 - "Verinin Merkezine Seyahat"MDS ap
 
Big data-analytics-changing-way-organizations-conducting-business
Big data-analytics-changing-way-organizations-conducting-businessBig data-analytics-changing-way-organizations-conducting-business
Big data-analytics-changing-way-organizations-conducting-businessAmit Bhargava
 
Neo4j GraphTalk Copenhagen - Next Generation Solutions using Neo4j
Neo4j GraphTalk Copenhagen - Next Generation Solutions using Neo4j Neo4j GraphTalk Copenhagen - Next Generation Solutions using Neo4j
Neo4j GraphTalk Copenhagen - Next Generation Solutions using Neo4j Neo4j
 
Certus Accelerate - Building the business case for why you need to invest in ...
Certus Accelerate - Building the business case for why you need to invest in ...Certus Accelerate - Building the business case for why you need to invest in ...
Certus Accelerate - Building the business case for why you need to invest in ...Certus Solutions
 
Five Trends in Real Time Applications
Five Trends in Real Time ApplicationsFive Trends in Real Time Applications
Five Trends in Real Time Applicationsconfluent
 
Big Data : Risks and Opportunities
Big Data : Risks and OpportunitiesBig Data : Risks and Opportunities
Big Data : Risks and OpportunitiesKenny Huang Ph.D.
 
Introducing Smart Data Discovery
Introducing Smart Data DiscoveryIntroducing Smart Data Discovery
Introducing Smart Data DiscoveryPanorama Software
 
“Real Time Machine Learning Architecture and Sentiment Analysis Applied to Fi...
“Real Time Machine Learning Architecture and Sentiment Analysis Applied to Fi...“Real Time Machine Learning Architecture and Sentiment Analysis Applied to Fi...
“Real Time Machine Learning Architecture and Sentiment Analysis Applied to Fi...Quantopian
 

Similar to SmartData Webinar Slides: How to analyze 72 billion messages a day to find trends (20)

Streaming and Visual Data Discovery for the Internet of Things
Streaming and Visual Data Discovery for the Internet of ThingsStreaming and Visual Data Discovery for the Internet of Things
Streaming and Visual Data Discovery for the Internet of Things
 
Blocks & Bots - Digital Summit Harvard Business School 2015
Blocks & Bots - Digital Summit Harvard Business School 2015Blocks & Bots - Digital Summit Harvard Business School 2015
Blocks & Bots - Digital Summit Harvard Business School 2015
 
Big Data & Analytics, Peter Jönsson
Big Data & Analytics, Peter JönssonBig Data & Analytics, Peter Jönsson
Big Data & Analytics, Peter Jönsson
 
Impacto del Big Data en la empresa española
Impacto del Big Data en la empresa españolaImpacto del Big Data en la empresa española
Impacto del Big Data en la empresa española
 
How to make your data scientists happy
How to make your data scientists happy How to make your data scientists happy
How to make your data scientists happy
 
uae views on big data
  uae views on  big data  uae views on  big data
uae views on big data
 
Ai design sprint - Finance - Wealth management
Ai design sprint  - Finance - Wealth managementAi design sprint  - Finance - Wealth management
Ai design sprint - Finance - Wealth management
 
How Big Data Changes Our World
How Big Data Changes Our WorldHow Big Data Changes Our World
How Big Data Changes Our World
 
Big data analytics, research report
Big data analytics, research reportBig data analytics, research report
Big data analytics, research report
 
Liberating data power of APIs
Liberating data power of APIsLiberating data power of APIs
Liberating data power of APIs
 
SAP Forum Ankara 2017 - "Verinin Merkezine Seyahat"
SAP Forum Ankara 2017 - "Verinin Merkezine Seyahat"SAP Forum Ankara 2017 - "Verinin Merkezine Seyahat"
SAP Forum Ankara 2017 - "Verinin Merkezine Seyahat"
 
Big data-analytics-changing-way-organizations-conducting-business
Big data-analytics-changing-way-organizations-conducting-businessBig data-analytics-changing-way-organizations-conducting-business
Big data-analytics-changing-way-organizations-conducting-business
 
Data Mining With Big Data
Data Mining With Big DataData Mining With Big Data
Data Mining With Big Data
 
Neo4j GraphTalk Copenhagen - Next Generation Solutions using Neo4j
Neo4j GraphTalk Copenhagen - Next Generation Solutions using Neo4j Neo4j GraphTalk Copenhagen - Next Generation Solutions using Neo4j
Neo4j GraphTalk Copenhagen - Next Generation Solutions using Neo4j
 
Certus Accelerate - Building the business case for why you need to invest in ...
Certus Accelerate - Building the business case for why you need to invest in ...Certus Accelerate - Building the business case for why you need to invest in ...
Certus Accelerate - Building the business case for why you need to invest in ...
 
Five Trends in Real Time Applications
Five Trends in Real Time ApplicationsFive Trends in Real Time Applications
Five Trends in Real Time Applications
 
Big Data : Risks and Opportunities
Big Data : Risks and OpportunitiesBig Data : Risks and Opportunities
Big Data : Risks and Opportunities
 
Introducing Smart Data Discovery
Introducing Smart Data DiscoveryIntroducing Smart Data Discovery
Introducing Smart Data Discovery
 
Big Data analytics usage
Big Data analytics usageBig Data analytics usage
Big Data analytics usage
 
“Real Time Machine Learning Architecture and Sentiment Analysis Applied to Fi...
“Real Time Machine Learning Architecture and Sentiment Analysis Applied to Fi...“Real Time Machine Learning Architecture and Sentiment Analysis Applied to Fi...
“Real Time Machine Learning Architecture and Sentiment Analysis Applied to Fi...
 

More from DATAVERSITY

Architecture, Products, and Total Cost of Ownership of the Leading Machine Le...
Architecture, Products, and Total Cost of Ownership of the Leading Machine Le...Architecture, Products, and Total Cost of Ownership of the Leading Machine Le...
Architecture, Products, and Total Cost of Ownership of the Leading Machine Le...DATAVERSITY
 
Data at the Speed of Business with Data Mastering and Governance
Data at the Speed of Business with Data Mastering and GovernanceData at the Speed of Business with Data Mastering and Governance
Data at the Speed of Business with Data Mastering and GovernanceDATAVERSITY
 
Exploring Levels of Data Literacy
Exploring Levels of Data LiteracyExploring Levels of Data Literacy
Exploring Levels of Data LiteracyDATAVERSITY
 
Building a Data Strategy – Practical Steps for Aligning with Business Goals
Building a Data Strategy – Practical Steps for Aligning with Business GoalsBuilding a Data Strategy – Practical Steps for Aligning with Business Goals
Building a Data Strategy – Practical Steps for Aligning with Business GoalsDATAVERSITY
 
Make Data Work for You
Make Data Work for YouMake Data Work for You
Make Data Work for YouDATAVERSITY
 
Data Catalogs Are the Answer – What is the Question?
Data Catalogs Are the Answer – What is the Question?Data Catalogs Are the Answer – What is the Question?
Data Catalogs Are the Answer – What is the Question?DATAVERSITY
 
Data Catalogs Are the Answer – What Is the Question?
Data Catalogs Are the Answer – What Is the Question?Data Catalogs Are the Answer – What Is the Question?
Data Catalogs Are the Answer – What Is the Question?DATAVERSITY
 
Data Modeling Fundamentals
Data Modeling FundamentalsData Modeling Fundamentals
Data Modeling FundamentalsDATAVERSITY
 
Showing ROI for Your Analytic Project
Showing ROI for Your Analytic ProjectShowing ROI for Your Analytic Project
Showing ROI for Your Analytic ProjectDATAVERSITY
 
How a Semantic Layer Makes Data Mesh Work at Scale
How a Semantic Layer Makes  Data Mesh Work at ScaleHow a Semantic Layer Makes  Data Mesh Work at Scale
How a Semantic Layer Makes Data Mesh Work at ScaleDATAVERSITY
 
Is Enterprise Data Literacy Possible?
Is Enterprise Data Literacy Possible?Is Enterprise Data Literacy Possible?
Is Enterprise Data Literacy Possible?DATAVERSITY
 
The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...
The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...
The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...DATAVERSITY
 
Emerging Trends in Data Architecture – What’s the Next Big Thing?
Emerging Trends in Data Architecture – What’s the Next Big Thing?Emerging Trends in Data Architecture – What’s the Next Big Thing?
Emerging Trends in Data Architecture – What’s the Next Big Thing?DATAVERSITY
 
Data Governance Trends - A Look Backwards and Forwards
Data Governance Trends - A Look Backwards and ForwardsData Governance Trends - A Look Backwards and Forwards
Data Governance Trends - A Look Backwards and ForwardsDATAVERSITY
 
Data Governance Trends and Best Practices To Implement Today
Data Governance Trends and Best Practices To Implement TodayData Governance Trends and Best Practices To Implement Today
Data Governance Trends and Best Practices To Implement TodayDATAVERSITY
 
2023 Trends in Enterprise Analytics
2023 Trends in Enterprise Analytics2023 Trends in Enterprise Analytics
2023 Trends in Enterprise AnalyticsDATAVERSITY
 
Data Strategy Best Practices
Data Strategy Best PracticesData Strategy Best Practices
Data Strategy Best PracticesDATAVERSITY
 
Who Should Own Data Governance – IT or Business?
Who Should Own Data Governance – IT or Business?Who Should Own Data Governance – IT or Business?
Who Should Own Data Governance – IT or Business?DATAVERSITY
 
Data Management Best Practices
Data Management Best PracticesData Management Best Practices
Data Management Best PracticesDATAVERSITY
 
MLOps – Applying DevOps to Competitive Advantage
MLOps – Applying DevOps to Competitive AdvantageMLOps – Applying DevOps to Competitive Advantage
MLOps – Applying DevOps to Competitive AdvantageDATAVERSITY
 

More from DATAVERSITY (20)

Architecture, Products, and Total Cost of Ownership of the Leading Machine Le...
Architecture, Products, and Total Cost of Ownership of the Leading Machine Le...Architecture, Products, and Total Cost of Ownership of the Leading Machine Le...
Architecture, Products, and Total Cost of Ownership of the Leading Machine Le...
 
Data at the Speed of Business with Data Mastering and Governance
Data at the Speed of Business with Data Mastering and GovernanceData at the Speed of Business with Data Mastering and Governance
Data at the Speed of Business with Data Mastering and Governance
 
Exploring Levels of Data Literacy
Exploring Levels of Data LiteracyExploring Levels of Data Literacy
Exploring Levels of Data Literacy
 
Building a Data Strategy – Practical Steps for Aligning with Business Goals
Building a Data Strategy – Practical Steps for Aligning with Business GoalsBuilding a Data Strategy – Practical Steps for Aligning with Business Goals
Building a Data Strategy – Practical Steps for Aligning with Business Goals
 
Make Data Work for You
Make Data Work for YouMake Data Work for You
Make Data Work for You
 
Data Catalogs Are the Answer – What is the Question?
Data Catalogs Are the Answer – What is the Question?Data Catalogs Are the Answer – What is the Question?
Data Catalogs Are the Answer – What is the Question?
 
Data Catalogs Are the Answer – What Is the Question?
Data Catalogs Are the Answer – What Is the Question?Data Catalogs Are the Answer – What Is the Question?
Data Catalogs Are the Answer – What Is the Question?
 
Data Modeling Fundamentals
Data Modeling FundamentalsData Modeling Fundamentals
Data Modeling Fundamentals
 
Showing ROI for Your Analytic Project
Showing ROI for Your Analytic ProjectShowing ROI for Your Analytic Project
Showing ROI for Your Analytic Project
 
How a Semantic Layer Makes Data Mesh Work at Scale
How a Semantic Layer Makes  Data Mesh Work at ScaleHow a Semantic Layer Makes  Data Mesh Work at Scale
How a Semantic Layer Makes Data Mesh Work at Scale
 
Is Enterprise Data Literacy Possible?
Is Enterprise Data Literacy Possible?Is Enterprise Data Literacy Possible?
Is Enterprise Data Literacy Possible?
 
The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...
The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...
The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...
 
Emerging Trends in Data Architecture – What’s the Next Big Thing?
Emerging Trends in Data Architecture – What’s the Next Big Thing?Emerging Trends in Data Architecture – What’s the Next Big Thing?
Emerging Trends in Data Architecture – What’s the Next Big Thing?
 
Data Governance Trends - A Look Backwards and Forwards
Data Governance Trends - A Look Backwards and ForwardsData Governance Trends - A Look Backwards and Forwards
Data Governance Trends - A Look Backwards and Forwards
 
Data Governance Trends and Best Practices To Implement Today
Data Governance Trends and Best Practices To Implement TodayData Governance Trends and Best Practices To Implement Today
Data Governance Trends and Best Practices To Implement Today
 
2023 Trends in Enterprise Analytics
2023 Trends in Enterprise Analytics2023 Trends in Enterprise Analytics
2023 Trends in Enterprise Analytics
 
Data Strategy Best Practices
Data Strategy Best PracticesData Strategy Best Practices
Data Strategy Best Practices
 
Who Should Own Data Governance – IT or Business?
Who Should Own Data Governance – IT or Business?Who Should Own Data Governance – IT or Business?
Who Should Own Data Governance – IT or Business?
 
Data Management Best Practices
Data Management Best PracticesData Management Best Practices
Data Management Best Practices
 
MLOps – Applying DevOps to Competitive Advantage
MLOps – Applying DevOps to Competitive AdvantageMLOps – Applying DevOps to Competitive Advantage
MLOps – Applying DevOps to Competitive Advantage
 

Recently uploaded

Edukaciniai dropshipping via API with DroFx
Edukaciniai dropshipping via API with DroFxEdukaciniai dropshipping via API with DroFx
Edukaciniai dropshipping via API with DroFxolyaivanovalion
 
April 2024 - Crypto Market Report's Analysis
April 2024 - Crypto Market Report's AnalysisApril 2024 - Crypto Market Report's Analysis
April 2024 - Crypto Market Report's Analysismanisha194592
 
Halmar dropshipping via API with DroFx
Halmar  dropshipping  via API with DroFxHalmar  dropshipping  via API with DroFx
Halmar dropshipping via API with DroFxolyaivanovalion
 
Data-Analysis for Chicago Crime Data 2023
Data-Analysis for Chicago Crime Data  2023Data-Analysis for Chicago Crime Data  2023
Data-Analysis for Chicago Crime Data 2023ymrp368
 
Unveiling Insights: The Role of a Data Analyst
Unveiling Insights: The Role of a Data AnalystUnveiling Insights: The Role of a Data Analyst
Unveiling Insights: The Role of a Data AnalystSamantha Rae Coolbeth
 
(PARI) Call Girls Wanowrie ( 7001035870 ) HI-Fi Pune Escorts Service
(PARI) Call Girls Wanowrie ( 7001035870 ) HI-Fi Pune Escorts Service(PARI) Call Girls Wanowrie ( 7001035870 ) HI-Fi Pune Escorts Service
(PARI) Call Girls Wanowrie ( 7001035870 ) HI-Fi Pune Escorts Serviceranjana rawat
 
VIP Call Girls Service Miyapur Hyderabad Call +91-8250192130
VIP Call Girls Service Miyapur Hyderabad Call +91-8250192130VIP Call Girls Service Miyapur Hyderabad Call +91-8250192130
VIP Call Girls Service Miyapur Hyderabad Call +91-8250192130Suhani Kapoor
 
VidaXL dropshipping via API with DroFx.pptx
VidaXL dropshipping via API with DroFx.pptxVidaXL dropshipping via API with DroFx.pptx
VidaXL dropshipping via API with DroFx.pptxolyaivanovalion
 
Carero dropshipping via API with DroFx.pptx
Carero dropshipping via API with DroFx.pptxCarero dropshipping via API with DroFx.pptx
Carero dropshipping via API with DroFx.pptxolyaivanovalion
 
CebaBaby dropshipping via API with DroFX.pptx
CebaBaby dropshipping via API with DroFX.pptxCebaBaby dropshipping via API with DroFX.pptx
CebaBaby dropshipping via API with DroFX.pptxolyaivanovalion
 
Week-01-2.ppt BBB human Computer interaction
Week-01-2.ppt BBB human Computer interactionWeek-01-2.ppt BBB human Computer interaction
Week-01-2.ppt BBB human Computer interactionfulawalesam
 
VIP High Profile Call Girls Amravati Aarushi 8250192130 Independent Escort Se...
VIP High Profile Call Girls Amravati Aarushi 8250192130 Independent Escort Se...VIP High Profile Call Girls Amravati Aarushi 8250192130 Independent Escort Se...
VIP High Profile Call Girls Amravati Aarushi 8250192130 Independent Escort Se...Suhani Kapoor
 
Introduction-to-Machine-Learning (1).pptx
Introduction-to-Machine-Learning (1).pptxIntroduction-to-Machine-Learning (1).pptx
Introduction-to-Machine-Learning (1).pptxfirstjob4
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...apidays
 
VIP Call Girls in Amravati Aarohi 8250192130 Independent Escort Service Amravati
VIP Call Girls in Amravati Aarohi 8250192130 Independent Escort Service AmravatiVIP Call Girls in Amravati Aarohi 8250192130 Independent Escort Service Amravati
VIP Call Girls in Amravati Aarohi 8250192130 Independent Escort Service AmravatiSuhani Kapoor
 
BigBuy dropshipping via API with DroFx.pptx
BigBuy dropshipping via API with DroFx.pptxBigBuy dropshipping via API with DroFx.pptx
BigBuy dropshipping via API with DroFx.pptxolyaivanovalion
 
VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...
VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...
VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...Suhani Kapoor
 
Log Analysis using OSSEC sasoasasasas.pptx
Log Analysis using OSSEC sasoasasasas.pptxLog Analysis using OSSEC sasoasasasas.pptx
Log Analysis using OSSEC sasoasasasas.pptxJohnnyPlasten
 
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al BarshaAl Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al BarshaAroojKhan71
 
100-Concepts-of-AI by Anupama Kate .pptx
100-Concepts-of-AI by Anupama Kate .pptx100-Concepts-of-AI by Anupama Kate .pptx
100-Concepts-of-AI by Anupama Kate .pptxAnupama Kate
 

Recently uploaded (20)

Edukaciniai dropshipping via API with DroFx
Edukaciniai dropshipping via API with DroFxEdukaciniai dropshipping via API with DroFx
Edukaciniai dropshipping via API with DroFx
 
April 2024 - Crypto Market Report's Analysis
April 2024 - Crypto Market Report's AnalysisApril 2024 - Crypto Market Report's Analysis
April 2024 - Crypto Market Report's Analysis
 
Halmar dropshipping via API with DroFx
Halmar  dropshipping  via API with DroFxHalmar  dropshipping  via API with DroFx
Halmar dropshipping via API with DroFx
 
Data-Analysis for Chicago Crime Data 2023
Data-Analysis for Chicago Crime Data  2023Data-Analysis for Chicago Crime Data  2023
Data-Analysis for Chicago Crime Data 2023
 
Unveiling Insights: The Role of a Data Analyst
Unveiling Insights: The Role of a Data AnalystUnveiling Insights: The Role of a Data Analyst
Unveiling Insights: The Role of a Data Analyst
 
(PARI) Call Girls Wanowrie ( 7001035870 ) HI-Fi Pune Escorts Service
(PARI) Call Girls Wanowrie ( 7001035870 ) HI-Fi Pune Escorts Service(PARI) Call Girls Wanowrie ( 7001035870 ) HI-Fi Pune Escorts Service
(PARI) Call Girls Wanowrie ( 7001035870 ) HI-Fi Pune Escorts Service
 
VIP Call Girls Service Miyapur Hyderabad Call +91-8250192130
VIP Call Girls Service Miyapur Hyderabad Call +91-8250192130VIP Call Girls Service Miyapur Hyderabad Call +91-8250192130
VIP Call Girls Service Miyapur Hyderabad Call +91-8250192130
 
VidaXL dropshipping via API with DroFx.pptx
VidaXL dropshipping via API with DroFx.pptxVidaXL dropshipping via API with DroFx.pptx
VidaXL dropshipping via API with DroFx.pptx
 
Carero dropshipping via API with DroFx.pptx
Carero dropshipping via API with DroFx.pptxCarero dropshipping via API with DroFx.pptx
Carero dropshipping via API with DroFx.pptx
 
CebaBaby dropshipping via API with DroFX.pptx
CebaBaby dropshipping via API with DroFX.pptxCebaBaby dropshipping via API with DroFX.pptx
CebaBaby dropshipping via API with DroFX.pptx
 
Week-01-2.ppt BBB human Computer interaction
Week-01-2.ppt BBB human Computer interactionWeek-01-2.ppt BBB human Computer interaction
Week-01-2.ppt BBB human Computer interaction
 
VIP High Profile Call Girls Amravati Aarushi 8250192130 Independent Escort Se...
VIP High Profile Call Girls Amravati Aarushi 8250192130 Independent Escort Se...VIP High Profile Call Girls Amravati Aarushi 8250192130 Independent Escort Se...
VIP High Profile Call Girls Amravati Aarushi 8250192130 Independent Escort Se...
 
Introduction-to-Machine-Learning (1).pptx
Introduction-to-Machine-Learning (1).pptxIntroduction-to-Machine-Learning (1).pptx
Introduction-to-Machine-Learning (1).pptx
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
VIP Call Girls in Amravati Aarohi 8250192130 Independent Escort Service Amravati
VIP Call Girls in Amravati Aarohi 8250192130 Independent Escort Service AmravatiVIP Call Girls in Amravati Aarohi 8250192130 Independent Escort Service Amravati
VIP Call Girls in Amravati Aarohi 8250192130 Independent Escort Service Amravati
 
BigBuy dropshipping via API with DroFx.pptx
BigBuy dropshipping via API with DroFx.pptxBigBuy dropshipping via API with DroFx.pptx
BigBuy dropshipping via API with DroFx.pptx
 
VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...
VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...
VIP High Class Call Girls Jamshedpur Anushka 8250192130 Independent Escort Se...
 
Log Analysis using OSSEC sasoasasasas.pptx
Log Analysis using OSSEC sasoasasasas.pptxLog Analysis using OSSEC sasoasasasas.pptx
Log Analysis using OSSEC sasoasasasas.pptx
 
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al BarshaAl Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha
 
100-Concepts-of-AI by Anupama Kate .pptx
100-Concepts-of-AI by Anupama Kate .pptx100-Concepts-of-AI by Anupama Kate .pptx
100-Concepts-of-AI by Anupama Kate .pptx
 

SmartData Webinar Slides: How to analyze 72 billion messages a day to find trends

  • 1. Finding the Signal in the Noise June 15, 2015 Webinar Presentation Nova Spivack, CEO sales@bottlenose.com
  • 2. What is Bottlenose For? Bottlenose discovers the threats and opportunities that impact your business Bottlenose does this using patented stream intelligence technology 2
  • 3. Key Stream Intelligence Use-Cases Threats • Risk detection • Crisis mitigation • Competitive threats • Reputational threats • Cyber threat detection Opportunities • Audience and customer insights • Innovation and research • New business and market opportunities • Competitive intelligence • Product and marketing intelligence 3
  • 4. Vision Stream Intelligence Our mission is to build the leading business intelligence company for stream data Stream data is the fastest growing segment of data. It includes all types of live or historical, unstructured or structured, time-stamped data, such as: email and messaging data, social media, mobile data, news, IT log data, CRM data, support data, sales data, Web and app analytics data, financial data, sensor and device data. We have built the first unified platform and application for automating the discovery of actionable intelligence across any stream data sources – We call this stream intelligence. 4
  • 5. ... the future belongs to raw unstructured or semi- structured data from both internal and external sources - increasingly delivered in (near) real-time. This data has great value yet most organizations do not have the tech infrastructure to handle all this data.” - IDC Problem: Massive growth of unstructured data cannot be managed effectively with existing Tech infrastructure Real-Time Discovery Against Streaming Data is Required: 5
  • 6. ● There are never going to be enough data scientists or analysts to cope with the rise of unstructured stream data in the enterprise ● Analysts need automated stream intelligence tools to help them deal with the volume, velocity and variety of stream data Analysts Are Drowning in Streams
  • 7. Solution: Bottlenose Automates Stream Intelligence • Bottlenose provides the most advanced automated stream intelligence that automatically finds patterns such as trends, anomalies, threats, opportunities and correlations in stream data • Bottlenose is extremely easy to use and easy to derive value from right away without extensive engineering and IT involvement or long professional solutions • The platform combines both internal enterprise data and external data from social, broadcast, web and other areas. We are In The Stream Intelligence Sweet Spot The Bottlenose solution is a new generation of tools that automates the production of actionable intelligence from stream data VarietyVelocity Volume & ELK Stack 7
  • 8. Stream Intelligence is “BI 3.0” (Source: HBR) 8
  • 9. Competitive Advantage from Coping with Stream Data 9
  • 10. Competitive Advantage from Coping with Stream Data 10
  • 11. Competitive Advantage from Coping with Stream Data 11
  • 12. Competitive Advantage from Coping with Stream Data 12
  • 13. Bottlenose Platform Social & traditional media (social networks, blogs, Forums, newswires) 98% of all live TV & Radio Broadcasts Enterprise Data (Sales, financials, Web analytics, IT systems, email, internal databases, etc.) Web Data, commercial data sources, financial market data sources, public data sources Machine and sensor data (Internet-of-things, machine data, weather data, etc.) 13 Generate Actionable Intelligence from ANY Stream Data 13
  • 14. Stream Intelligence Pipeline Applications Rules & Agents Alerts/Actions Based on Business Interests Stream Data Storage & API Long-term storage, real-time access, search & APIs Trend Detection Extrapolation, Correlation/Clustering Data Mining & Analytics ~30 Entity Types and ~150 Metrics Ingestion & Enrichment Push/Pull of Unstructured/Structured Data Data in Motion Alerts & Actions New Patterns Entities & Metrics 14
  • 15. Breaking News Automatically Discover Threats and Opportunities Known Issues Unknown/Emerging Issues Customer Problems Enterprise Risk Factor Fraud Risk Product Recall Competitive Threat Cyber Attack Focal Interest Power Outage Natural DisasterTraffic Congestion Device Failure Financial Trading Anomaly Reputation Risk Intellectual Property Violation 15
  • 16. Continuous High-Volume Stream Analytics • 3 billion live + historical messages analyzed every hour • 72 billion records analyzed per day + predictive analytics on 7.2 billion • 67,000,000 new messages ingested every day • Trend detection at a rate of 1 million events per second • 30 entity types recognized * 150 metrics per entity * 10’s of millions of entities = ~50 to 100 billion time series monitored and analyzed continuously • Growing to 200 Terabytes of data stored & analyzed continuously in 2015 1000s of High-Level Detected Trends Per Hour • Automated data science layer applies machine learning, statistics, predictive analytics to correlate, cluster, predict and analyze emergent trends We See the Near Future Before Anyone Else • 80% of the time, our system detects breaking news and emerging threats, opportunities and keywords up to 10’s to 100’s of minutes ahead of the media, Twitter, ad networks, etc. Similar advantages against non-text data sources Key Metrics Bottlenose analyzes 72 billion data records every day 16
  • 17. Demo: Data Agnostic Stream Intelligence 17
  • 18. Customer Facing Products ● Analytics, intelligence, and discovery engine ○ Nerve Center ○ Full-stack offering ● Streaming data services to applications ○ Bottlenose API (Platform) 18
  • 19. ‣ Advanced filtering & aggregations using simple OLAP interface ‣ “Interactive Analytics” thanks to sub-second query response time ‣ Add new data sources using central mapping system Analytics Engine 19
  • 20. A sophisticated Semantic approach is required to make sense of the raw data. The structure of data can be derived based on entities/dimensions the system has a pattern for. Machine learning techniques can begin to make inferences and match to known profiles as data flows in. One of the most powerful capabilities is when different data sources need to be compared. A system like ours automatically normalizes them. For example, when the data has different time granularity, we automatically align different time periods in order to find overlaps. Of course the Semantic engine can also be adjusted with a vertical industries unique facts, relationships, and jargon. Need for a Radical New Form of Information Retrieval: Semantic meaning bottoms up from raw data 20
  • 21. Application - Nerve Center Our application provides a powerful suite of tools to find business insights in streaming data: ● Monitor: Real-time monitoring with powerful live visualizations. ● Analyze: Fast interactive analytics to dig deep into the data. ● Discover: Automated insight discovery. Get notified when new patterns are detected. ● Customize: Reports and live dashboards can be created for any vertical by mix-and-matching insights & visualizations across any combination of data streams 21
  • 23. Augmentation Engine Analytics Engine Detection Engine Correlations Engine Rules & Agent Engine Processing Layers Depth of Insight 23
  • 24. Augmentation Engine Analytics Engine Detection Engine Correlations Engine Rules & Agent Engine Processing Layers Depth of Insight 24
  • 25. Augmentation Engine Analytics Engine Detection Engine Correlations Engine Rules & Agent Engine Processing Layers Depth of Insight 25
  • 26. ‣ Typical topic stream like “Beyonce” (Pepsi) ‣ 4M new events (data records) per month ‣ ~8M unique entities tracked per month ‣ ~8M unique entities x 150 metrics x many time buckets = A lot of data points ‣ And this is just 1 stream. We have thousands of these running at all times... Data Points 26
  • 27. Augmentation Engine Analytics Engine Detection Engine Correlations Engine Rules & Agent Engine Depth of Insight Processing Layers 27
  • 28. ‣ Systematically walk through all data points. ‣ Continuous stream of categorized signals. Searchable. Detection Engine 28
  • 29. 29
  • 30.
  • 31.
  • 32. Detection Engine Anticipate Python servers for trend detection & extrapolation Detector Workers that continuously aggregate entities and fetch corresponding metrics Context Gathering Finding additional meta-data around detections Time Series Extrapolation Clustering Rolling clustering of trends based on overlapping meta-data and a variety of distance functions Analytics Requests Entities & Time Series Analytics Requests Related Entities Find Related Trends Related Trends New and updated trends 32
  • 33. ‣ Python library, using SciPy ‣ Algorithms for detection & extrapolation in time series data ‣ Includes tooling for debugging, training and simulating ‣ ~500 detections/CPU-core/second Anticipate 33
  • 34. Augmentation Engine Analytics Engine Detection Engine Correlations Engine Rules & Agent Engine Depth of Insight Processing Layers 34
  • 35. Augmentation Engine Analytics Engine Detection Engine Correlations Engine Rules & Agent Engine Depth of Insight Processing Layers 35
  • 36. Our automated insight discovery on streaming data enables “intelligence as a service for every organization” Intelligence as a Service 36