CTAM Making Analytics Actionable RJA FINAL

Making Analytics
ActionableActionable
CTAM Conference
Marriott Marquis New YorkMarriott Marquis, New York
Robert J. Abate, CBIP, CDMP
October 6th, 2011
Solutions Principal, EMC Consulting
1© Copyright 2011 EMC Corporation. All rights reserved.

Making Analytics ActionableMaking Analytics Actionable
•Topic IntroductionTopic Introduction
•Data Sources &
Rationalization
•Actionable Analytics
•Open Exchange Of•Open Exchange Of
Ideas

Topic Introduction:p
Analytics & Big Data…
You Are In The “Data” Business…

The Analytics Business Opportunity
“Through 2015, organizations integrating high value, diverse new
Through 2015, organizations integrating high value, diverse new
information sources and types into a coherent information
management infrastructure will outperform industry peers financially
by more than 20%”
Gartner July 2011
"The New Value
Integrator, Insights
from the CFO Study”

The Information Issue Is…
•33% of business leaders make critical decisions without the information they
need
•50% of business leaders don’t have access to the information across their50% of business leaders don t have access to the information across their
organization needed to do their jobs
•75% of business leaders say more predictive information would drive better
decisions IBM Institute for Business Value, March 2009

Information Trust & Alignment Is Missing…Information Trust & Alignment Is Missing…
• Harris Interactive recently polled 23,000 U.S. employees
in key industries and functional areas about EIM & EII:in key industries and functional areas about EIM & EII:
– Only 37% said they have a clear understanding of what
their organization is trying to achieve and why
– Only one in five was enthusiastic about their team and
the organization’s / corporation’s goals
– Only one in five said they have a clear “line of sight”– Only one in five said they have a clear line of sight
between their tasks and their team and organization’s
goals
Only 15% felt that their organization fully enables them– Only 15% felt that their organization fully enables them
to execute key goals
– Only 20% fully trusted the organization they work for

View Another Way…
If a football team had
these players on the field:these players on the field:
• Only 4 of the 11 players on the field
would know which goal is theirs
• Only 6 of the 11 would care
• Only 3 of the 11 would know what
position they play and what they are
supposed to do
• 9 players out of 11 would in some way• 9 players out of 11 would, in some way,
be competing against their own team
rather than the opponent

We Are Awash In A “Sea Of Data”1We Are Awash In A Sea Of Data
• In the information age, every organization is in the “data”
businessbusiness
• Data is growing exponentially, so are the challenges
• Complexity is causing insight to be lost• Complexity is causing insight to be lost
Source: IDC Digital Universe White Paper, Sponsored by EMC, May
2009
9
1 “Big Opportunities In Big Data”, Forrester , Brian Hopkins, 5/18/2011

Pictorial Representation Of Information

Time Really Is Money!

Data Is Coming At Us FasterData Is Coming At Us Faster
• In a recent TDWI survey of 450 CIO’s
– 17% have a real time data warehouse
– 90% plan on having a real time warehousep g
– 75% will replace to get to a real-time solution
“REAL -TIME IS A RAPIDLY BECOMING A
NECESSARY FOUNDATION TO A VALUABLENECESSARY FOUNDATION TO A VALUABLE
ANALYTICS SOLUTION!”

Data Is Coming From All DirectionsData Is Coming From All Directions
• Data is now commonly entering into the enterprise
from external sourcesfrom external sources
– Marketing Lists
– Neilson / AcxiomNeilson / Acxiom
– Government
– Bloombergg
– TransUnion, Experian, Equifax
– Radian 6
– Biz360
– Etc.

The Data Integration Imperative

Summation Of ChallengesSummation Of Challenges
• Business mandate to obtain more value out
of the data (get answers)of the data (get answers)
• Need to adapt and become agile to
information and industry-wide changesinformation and industry wide changes
• Variety of sources, amount and granularity of
data that customers want to integrate isdata that customers want to integrate is
growing exponentially
• Need to shrink the latency between theeed to s t e late cy bet ee t e
business event and the data availability for
analysis and decision-making

Data Sources &
RationalizationRationalization
How Do We Make Sense Of The Data…

What is Information?
• Data = Documents, Pages Files,…
• Structured Data / Metadata = Data + Syntax
• Semantics = Meaning of Structured Data Elements
• Information = Structured Data + SemanticsWhat is Information? Information Structured Data Semantics
• Knowledge = Purposeful Combination of Information
ABATE Wisdom Triangle
Options =
“Benefit Driven
Usage”
Wisdom
Business Performance Management
Making coherent judgments and inferences from the knowledge
gained by evaluating all the possible outcomes
Understanding
Patterns = “Context
& Function”
Wisdom
Knowledge
Business Intelligence & D/W
Making sense of these facts so that we can now derive conclusions
from related and non-related information
Data In Context =
“Related” Information
Performance Reporting
Placing these facts into the context of your business
model and operations process
Unstructured
Data
Raw Data Reporting
Structured and Unstructured raw elements or
images (structured vs. unstructured)
g ( )

Where is the data?
• Unstructured Data
– Emails and text-based documentation, contract documentation, social
media outlets
– Web sentiment
• Network Performance DataNetwork Performance Data
– Gigabytes of event and fault data captured every day from networks
– Data from various network layers (service, network and transmission) that
are rarely brought together
– Stores rich insights into root causes of alarms, network performance
i d h kissues and other network events
• Network Traffic Data
– Captures type of data and traffic passing over network, level of BW usage,
type of user
– Could enable better traffic management and network investment decisions
• Call / IP Detail Records
– Typically archived and rarely used for detailed analysis due to volume
– Provides insights on call patterns data usage and perceived QoSProvides insights on call patterns, data usage and perceived QoS
• Web logs and Call Center Records
– Traffic analysis common but rarely look at from specific customer’s journey
– Can help move customers from labor-intensive channels to automation
From article: “Unlocking the value in telecommunications data”

Analytics Maturity
“What Is The Best Thing That Could Happen?”“What Is The Best Thing That Could Happen?”
Business Optimization – Moving Up The Maturity Curve
BP Optimization
“What If These Trends Continue?”“What If These Trends Continue?”
“What Will Happen Next?”“What Will Happen Next?”
Forecasting
Predictive
Analytics
“Why Is This Happening?”“Why Is This Happening?”
What If These Trends Continue?What If These Trends Continue?
dvantage
Statistical
Analysis
Forecasting
petitiveAd
Drill-Down Queries
Exception Alerts
Com
Standard “Operational”
Or “System” Reports
Ad-Hoc Reporting
Business Intelligence / Analytics Maturity
Or System Reports

Why is Analytics Maturity Critical?
ll d h lImmature organizations allocate a majority of time to data capture, while
mature organizations action on data already captured
Decision Making
& Action
Decision Making
Analyze
Information
%TimeSpent
Collect Data
TimeSpent
& Action
Analyze
Information
%
Analyze
Information
Predictive Analytics
Collect Data Collect Data
Classic Reporting BI & Analytic Tools First Generation EIM
Less Mature More MatureLess Mature More Mature

New Technologies Have Changed Game…
Recent Technology Advancements Analytics RamificationsRecent Technology Advancements
• MPP data platforms
• Advanced in-database analytics
• Analytics (Hadoop MapReduce) on
Analytics Ramifications
• Tackle more complex analytical
opportunities to yield new insights
• Empower users by delivering actionable• Analytics (Hadoop, MapReduce) on
unstructured data
• Integrate analytic results with operational
systems
Empower users by delivering actionable
insights directly
• Extend reach of BI/Analytics to new devices
and next generation user experiences
• Data visualization across multiple
mediums and devices
• Linking structured & unstructured data
using semantics & linking
• Enable self-service access to data
• Increase visibility into data assets
facilitating audits and compliance
• Elastic architectures that simplifyusing semantics & linking
• Self-service BI and data marts
• New structured & unstructured “big data”
sources
• Elastic architectures that simplify
management and administration
• Collaboration around analytics and data
assets associated with specific subject
• Agile data warehousing concepts
• ELT & data enrichment capabilities
• Self-provisioned analytic sandboxes
• Analytics collaboration

Big Data, Big Picture Analytics
Portal
(Collaboration &(Collaboration &
Sandboxes
New Data WatchesNew Data Watches Continuous
Query
Correlation &Correlation &
NotificationsNotifications
Deep Analytic
Perspectives• Set-Top Box Analytics
h l i i
p
– Channel Viewing Data
• Advertizing Analytics
– Interactive Services
• Shopping, Home, Auto
• Information Services & Web
Vid O D d A l ti
Deep “Data Pool”
Historic, Federated, Multi-Participant, Cleansed …
• Video On Demand Analytics
– On Demand Programming Sales
& Selections
• Internet Analytics
– IPDR Statistics, Bandwidth
Utili ti
Utilization, …

Actionable Information:
Turning Data Into DecisionsTurning Data Into Decisions
Turning Info Into Intelligence

Companies are telling us that they’re trying to gain key insights
about their customers, products, and operations, p , p
Who are my most
valuable customers?
What are my Quality Of Service
(QOS) metrics and are they
i i ?”
Where should I make revenue
h i ?
aluable custo e s?
What are my most
important products?
What customers are most
likely to churn?
improving?” growth investments?
How do I minimize
What are my most
successful campaigns?
Where can I optimize
network effectiveness?
How do I minimize
portfolio risk?
What channels are
most effective?
What business areas are
at greatest risk?
Where do I have
compliance risk?
“77% of respondents are aware of bad decisions that managers have made
within their organizations because they did not have access to accurate
information”
“The Fact Gap: The Disconnect Between Data and Decisions”
BusinessWeek Research Services

Evolution to “Continuous Intelligence”
• Industry moving from historical and near-
time reporting to real-time analytics that
enable real-time actions / decisions
• Ability to predict future trends using large
volumes of historical / real-time data
gaining traction with customers
• Challenging economic times are driving
service providers to focus on cost-cuttingp g
programs (e.g. improved operations,
avoiding violations)
• Retaining customers is critical as new
competition enters market and priceco pet t o e te s a et a p ce
becomes second to customer experience
• Improving access to real-time, actionable
intelligence is a growing trend in the
service assurance and sales areas
se v ce assu a ce a d sales a eas
Heavy Reading

Large-Scale, Real-World Analytics
• How do I segment my clients?
• Which trial is working better?
• How can I predict fraudulent or other
• K-means clustering
• Mann-Whitney U Test
• Logistic regressionHow can I predict fraudulent or other
negative events?
• Is this user likely to be interested in
this ad?
Logistic regression
• Conjugate Gradient, SVM
this ad?
• Does this product appeal to some
segments more than others?
• How do I do hyper targeting of my
• Log-likelihood
• Cohort analysis• How do I do hyper-targeting of my
high-value frequent customers?
• Which features of a campaign result
in user revisits?
• Cohort analysis
• Linear regression
in user revisits?
• Which campaigns have the best
return on investment?
• Financial models

New industry shift underway, and key to success is defining a
process where organizations can continuously uncover and
publish new insights about the businesspublish new insights about the business
1) Business
A na l y t i cs L i fe c ycle
1
2) Bus. & IT
Acquires and
1) Business
Defines mandate
and requirements
17% of1
2
q
integrates data
5) Business
Consumes insights
enterprises feel
they have a
strategic
shortage of
5
Strategic
Business
Initiative 3) Data
Scientists
g
and measures
effectiveness
Data
Scientists… a
role that many
did not even
3
4
Scientists
Build and refine
analytic models
4) IT
Publishes new
insights
know existed
12 months
ago1
1Source: Enterprise Strategy Group, 2011

CSP Analytics Hot SpotsCSP Analytics Hot Spots
Two major focus areas identified by the TM Forum include:
• How can we understand the impact of product
changes in days rather than months?
• How do we value the revenues and costs of each
customer?
• How do we build a 360o view of the customer’s
interaction with us?
• How do we get a complete view across multipleHow do we get a complete view across multiple
networks (e.g. broadband, wireless, voice)?
Customer Analytics

Example: IP TV Performance Analytics
Moving from Reactive…ov g o eact ve…
• Network Operations Centers monitor availability and performance from different
systems
• Correlation is traditionally slow, manual and not integrated with customer care
• Care is typically contacted by frustrated customer before they understand customer
impacted
• Result is slow response and inability to be proactive with customer
Report trouble
• Sorry, we did not know you were having an issue…
• We do not see any issues from this end…
• It must be due to something out of our control…
NEGATIVE
experience
Ticketing System
experience
Customer Data
Network Operations
Network Monitoring
T di i l i i f k
Traditional monitoring of network
availability / health for devices,
connections, app servers, etc.

Example: IP TV Performance Analytics
…to Proactive…to oact ve
• Availability, Performance, Customer, Ticket data fed into analytics engine
• Data analyzed in real-time and customer experience impacting events identified
• Information available to customer care in Service Experience Dashboard
Phone call, email, social media…
p
• Information can be used for credits, auto-notification or to support customer calls
• Data can be used for off-line analytics to identify trends / opportunities
Confirm
Satisfaction
• We know you were having issues and resolved it for you…
• We appreciate your service and want to give you a credit…
• <email sent indicating service issues identified and fixed>
POSITIVE
experience
Ticketing System Customer Data
Service Experience
Dashboard
Data
Scientists
Other Data
Network Operations
Network Monitoring
Real-time
Analytics Engine
Analytics
Modeling
Constantly analyze data to
identify trends and
h h
Network Monitoring
Traditional monitoring of network
availability / health for devices,
connections, app servers, etc.
improvements. Which
regions have the most
video issues? What
channels do customers
use for specific issues?

Example: Customer Retention Analytics
• Customer Retention
– Identify those at risk ofy
abandoning their accounts.
– Use logistic regression models,
GAM models built in R, or SAS
d lscoring models.
– Build and execute models in the
database, and reduce time to act.
Al d di
Credit Card Number Probability of Fraud
• Also used to predict…
– Usage fraud
– Customer churn
7125 6289 5972 9510 15%
3955 8125 1327 7120 22%
2190 6379 9218 9290 7%
SSN Probability of Churn
611‐43‐2435 15%
812‐35‐1035 22%
– Customer profitability
– Total customer experience
(contract negotiations)
2760 1924 2864 0950 47%
4915 1908 8302 9940 12%
3534 7203 6200 4010 3%
253‐23‐2943 7%
732‐62‐1435 47%
483‐32‐5298 12%
821‐90‐8574 3%

Example: “What are our customers saying?”
sql# SELECT cid, sum(tfxidf)/count(*) AS centroid
FROM (
SELECT id, tfxidf, cid,
row_number() OVER (
PARTITION BY id
ORDER BY distance, cid) rank
FROM blog_distance
) blog_rank
WHERE rank = 1
GROUP BY cid;
-MAP:
NAME t t tNAME: extract_terms
PARAMETERS: [id integer, body text]
RETURNS: [id int, title text, doc _text]
FUNCTION: |
if 'parser' not in SD:
import ...
class MyHTMLParser(HTMLParser):
...
SD['parser'] = MyHTMLParser()
parser = SD['parser']
parser.reset()
parser.feed(body)
id | tfxidf
-----+-------------------------------------------------------------------
2482 | {3,1,37,1,18,1,29,1,45,1,...}:{0,8.25206814635817,0,0.34311110...}
1 | {41,1,34,1,22,1,125,1,387,...}:{0,0.771999985977529,0,1.999427...}
10 | {3,1,4,1,30,1,18,1,13,1,4,...}:{0,2.95439664949608,0,3.2006935...}
...

What are the largest analytics challenges?
Data quality and a clear business case are critical to success
Analytics programs need to…y p g
• Enable an Information
Mgmt strategy that will
make accurate data a
critical business objectivecritical business objective
• Focus, focus, focus.
Define a clear business
case, show small wins,
b ldbuild momentum.
• Realize this is different
than “reporting” and
ensure the right platformensure the right platform
and support org are put in
place day one
Challenges – creating data driven relationships from legacy infrastructure
TM Forum’s Exploiting Analytics, Sep 2010
TM Forum s Exploiting Analytics, Sep 2010

“Think Different”“Think Different”“Think Different”“Think Different”
Apple Ad Campaign, 1997Apple Ad Campaign, 1997Apple Ad Campaign, 1997Apple Ad Campaign, 1997

Q&A

THANK YOUTHANK YOU
Robert J. Abate, CBIP, CDMPJ , ,
Solutions Principal, EMC Consulting
robert.abate@emc.com
Blog: infocus.emc.com/author/robert_abate/
(201) 745-7680

CTAM Making Analytics Actionable RJA FINAL

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (13)

Similaire à CTAM Making Analytics Actionable RJA FINAL

Similaire à CTAM Making Analytics Actionable RJA FINAL (20)

Plus de Robert J. Abate, CBIP, CDMP

Plus de Robert J. Abate, CBIP, CDMP (6)

CTAM Making Analytics Actionable RJA FINAL