DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
1120 track1 grossman
1. What is the Analytic Maturity of
Your Company and Five Ways That
You Can Improve It
Robert L. Grossman
University of Chicago,
Analytic Strategy Partners &
Open Data Group
Predictive Analytics World
Chicago, June 20, 2017
2. A little about myself
• I have been building predictive models over big data and consulting in
the strategy and practice of analytics for over 21 years:
• 15 years at Open Data Group (2002-2017)
• 6 years at Magnify (1995-2001).
• I am active in standards for predictive modeling and big data, such as
the Portable Format for Analytics (PFA) and the Predictive Model
Markup Language (PMML).
• I’m a Professor at the University of Chicago and the Director of the
Center for Data Intensive Science. I lead a research group on
translational data science and its applications to biology, medicine
and health care.
4. The Analytic Diamond and Your Role in All This
• No matter where you are
in the company, if you help
the right analytics get built
and deployed, you will be
eventually be recognized
and rewarded.
• The more you understand
about the overall process,
the easier this will be.
Analytic strategy,
governance, security &
compliance.
Analytic algorithms
& models
Analytic operations
Analytic Infrastructure
*Adapted from Robert L. Grossman, The Strategy and Practice of Analytics, O’Reilly, 2018, to appear.
6. Software Capability Maturity Model – Process Model
• Capability Maturity Model (CMM)
is a model for the maturity of
processes that support a
contracted project.
• Developed at the CMU Software
Engineering Institute - Version 1.0
in 1991 and Version 1.1 in 1993.
• Maturity Levels, Key Process Areas,
Goals, Key Practices & Common
Features.
• I have been working on a Analytic
Processes Maturity Model
(APMM).
7. Analytic Processes Maturity Model vs
Software Processes Maturity Model
*Adapted from Robert L. Grossman, The Strategy and Practice of Analytics, O’Reilly, 2018, to appear.
Data scientist
Data engineer
Product owner or manger
Business owner or manger
Software Process Model Analytic Process Model
Software Data + software
Software quality Data + model + software quality
3 sides to the SPMM 4 sides to the APMM
8. 2. A Quick Introduction to the Analytic Maturity Model*
*Source: This section is based upon Robert L. Grossman, Quantifying the Analytic Maturity of an Organization, submitted for publication.
9. Analytic Maturity Levels
1. Reports from data 2. Predictive models from data
3. Repeatable process
for analytics
4. Consistent & integrated
analytics across the enterprise
5. Strategy driven enterprise analytics
10. Analytic Maturity Levels
Level 1. Ability to analyze data and build reports.
Level 2. Ability to build predictive models.
Level 3. Repeatable analytics. Repeatable process for building and
deploying analytic models.
Level 4. Enterprise level analytics. Analytics are used throughout an
enterprise, built with a common infrastructure & process and
integrated together.
Level 5. Strategy driven analytics. Does your company, have an
analytic strategy that is linked to the corporate strategy and with
consistent and integrated analytics throughout the enterprise.
11. Key Differences Between AM 1 & AM 2
• Analytic Maturity Level 2
organizations make predictions
about future events instead of
summarizing past events.
• Organizations at Analytic
Maturity Level 2 know the
difference between (business)
rules and analytics and integrate
both of them into deployed
systems. They also know the
difference between reports and
models.
Past FutureToday
Use data to make
predictions about
future events (e.g. who
will respond to an offer)
Use data to summarize
past behavior (e.g. total
sales by segment by
quarter)
Predictive analyticsReporting
12. Analytic algorithms
& models
Analytic Infrastructure
Analytic Operations
Level 3
Level 3
Level 3
AM Level 3 Organization has a
repeatable process for:
1. Getting the required data
2. Building models and
3. Deploying models.
4. Evaluating models.
13. Analytic algorithms
& models
Analytic Infrastructure
Analytic Operations
Level 4 have
enterprise wide
analytic & modeling
infrastructure
AM Level 4 organizations
integrate the results from
multiple models to
optimize operations
AM Level 4 have an enterprise wide
infrastructure and process for:
1. Data management & integration
2. Building models
3. Deploying models
4. Integrating the results
5. Evaluating the results
Level 4
14. Key Differences Between AM 2 & AM 3
• Analytic Maturity Level 3
organizations remove
barriers to building models,
such as when modelers to do
not have easy access to the
data.
• Analytic Maturity Level 3
organizations remove
barriers to deploying models.
Data scientists
build models
Data engineers
deploy models
Data engineers provide
the required analytic
infrastructure
SOPs
15. Key Differences Between AM 3 & AM 4
• Analytic Maturity Level 4 organizations
use a consistent and repeatable process
& an enterprise-wide infrastructure to
produce analytic models across the
organization.
• Analytic Maturity Level 4 organizations
integrate analytic models from across the
organization to improve decision making.
• The analytic governance for Analytic
Maturity Level 4 organizations extends
across an organization and optimizes
operations for the organization as a
whole.
• Analytic Maturity Level 4 organizations
have a culture of analytics.Division A
Division B
Division E…
• Analytic governance
• Optimize analytics for
company as a whole
18. Deploying analytic models and analytic workflows
Analytic
Engines
Analytic
Workflow
Producers
Export
analytic
workflows
Import
analytic
workflows
PFA
PFA is a collection of analytic primitives
approved by the PFA Working Group that is
rich enough to define analytic workflows. See
www.dmg.org.
19. Case Study 2
• Set up an analytic
governance
committee that the
data scientists direct
with the involvement
of the other parts of
the organization.
• Start with one
committee.
Analytics
Governance
Committee
Analytics Compliance
& Security Review
Committee
Analytics
Technical Policy
Committee
Data Committee
Corporate
Governance
20. Analytic Infrastructure Analytic Modeling Analytic Operations Analytic Strategy
Scores
Actions
Measures
Cleaning &
enriching data
Candidate Model
Validated Model
Dashboards
Operational data
Data Warehouse
Analytic
Datamart
ETL Deployment
Execute strategy
Develop strategy
Strategic Alignment
Governance
Exploratory Data
Analysis
Building features
Source: Robert L. Grossman, The Strategy
and Practice of Analytics, O’Reilly, 2018.
Adapted from Robert L. Grossman, The Strategy and Practice of Analytics, O’Reilly, 2018, to appear.
21. Goals of IT Governance
1. Assure that the investments in IT generate business value.
2. Mitigate the risks that are associated with IT.
3. Operate in such a way as to make good long term decisions with
accountability and traceability to those funding IT resources, those
developing and support IT resources, and those using IT resources.
22. Goals of Analytic Governance*
1. Assure that good long term decisions about analytics are reached
and that investments in analytics generate business value.
2. Manage the risk and liability associated with data & analytics.
3. Operate in such a way as to make sure that there is accountability,
transparency, and traceability to those funding analytic resources,
to those developing and supporting analytic resources, and to those
using analytic resources.
4. Provide an organization structure to ensure that the necessary
analytic resources are available, that data is available to those
building analytic models, that analytic models can be deployed, and
that the impact of analytic models is quantified and tracked.
*Adapted from Robert L. Grossman, The Strategy and Practice of Analytics, O’Reilly, 2018, to appear.
24. Five Ways to Get Started
1. Set up a committee to quantify the analytic maturity of your company.
If you cannot measure it, you cannot improve it.
2. Set up (or improve) the environment for deploying analytic models,
using a model interchange format, analytic engines, or similar
technology. (Helpful for AM Level 2)
3. Set up your first analytic governance committee or improve the
operational efficiency of your current analytic governance. (Helpful for
AM Level 2)
4. Set up SOPs for building and/or deploying analytic models so that the
process is faster, repeatable & replicable. (AM Level 3 Requirement)
5. Volunteer to lead a process to integrate two different models from
two different parts of your company to improve the relevant actions.
(Helpful for AM Level 4)
25. Seven Process Areas to Focus on for Improving the
Analytic Maturity of Your Company
1. Develop processes to consistently deploy analytics.
2. Develop a standard set of features that can be used across divisions.
3. Embed people into the organization to spread analytics enterprise wide.
4. Develop standard processes for getting the data to the modelers.
5. Develop standard processes for evaluating the effectiveness of deployed
models.
6. Develop standard processes for iteratively improving a deployed model.
7. Develop a culture of analytics.
26. Summary
Level Description
1. Analytic reporting Ability to analyze data & build
reports.
2. Analytic modeling Ability to build predictive models.
3. Repeatable analytics Repeatable process for building,
deploying & evaluating analytics.
4. Enterprise level analytics Common analytics processes used
throughout an enterprise,
integrated together & optimized.
5. Strategy driven analytics There is an analytic strategy that
drives analytics.
28. For more information
For more information, please see:
• Robert L. Grossman, A Framework for Evaluating the Analytic Maturity
of an Organization, International Journal on Information Management,
to appear.
• Robert L. Grossman and Kevin P. Siegel, Organizational Models for Big
Data and Analytics, Journal of Organization Design, Volume 3, Number
1, 2014, pages 20-25.
• Robert L. Grossman, The Strategy and Practice of Analytics, O’Reilly,
2018, to appear.
analyticstrategy.com
Robert L. Grossman
rgrossman.com
@BobGrossman
