It is not easy to succeed with self-service analytics. Besides a governed self-service architecture, it requires well-designed governance processes, a standard analytics and data platform, a federated organizational structure with co-located Bl developers, and continuous training and support. This report examines the evolution of self-service BI and the necessary foundation for its success and then presents a reference architecture to support self-service analytics.
1. The Rise of Self-service
Business Intelligence
Stephen Kaiser, CBIP CDP-DM
Servus Credit Union
2. Agenda
• Objectives of Self-service BI
• Evolution in Analytics Technology
• Self-service Data Discovery and Data Preparation Tools
• Business Impacts and Consequences
• Top-down and Bottom-up BI
• Self-service BI Success Factors
• Governed Self-service BI
• Reference Architecture and Information Supply Chain
3. Objectives of Self-service BI
SS BI
Make BI tools
easy to use
Make BI tools
easy to use
Make BI
results easy
to consume
and enhance
Make BI
results easy
to consume
and enhance
Make DW
solutions fast
to deploy and
easy to
manage
Make DW
solutions fast
to deploy and
easy to
manage
Make it easy
to access
source data
Make it easy
to access
source data
Courtesy of BI Research and Intelligent Solutions Inc.
• Speed
• Agility
• Freedom to address local needs
quickly
• Enable BI to be adopted across
the organization
• Leverage data assets to better
deliver competitive advantage
4. Evolution in Analytics Technology
• New self-service technologies now
enable business to generate
insights without assistance from IT
o Visual discovery tools
o Data preparation and catalog
tools
o Open source platforms & data
pipelining tools
• “By 2018, data discovery and data management evolution will drive
most organizations to augment centralized analytic architectures with
decentralized approaches.” - Cindi Howson, Gartner, Inc., June 2015.
5. Visual Discovery Tools
• Designed for open-ended exploration and the end point or potential
deliverable is unknown
• Enable executives/managers to explore and customize data views w/o
waiting for IT development
• Enable easy diagnosis of root
cause problems and determine
changes
• Smarter finance, budgeting and
forecasting
• Scenario analysis
• New insights
6. Data Preparation Tools
• Stand alone vs. integrated tools
• Visual or scripting interface
• Business analyst/data scientist
• Interactive visual exploration
• Data mashup capabilities are now table
stakes
• Advanced capabilities focus on reducing
time and complexity of preparing data for
analysis in governed way
7. Machine Learning and Data Preparation Tools
• Automated machine-learning algorithms with guided
business-user-orientated tools
o Data typing
o Data structure
o Data transformation
o Join and blend
opportunities
o Enrichment
opportunities
8. Business Impacts of Self-serve Tools
• Self-service data discovery/preparation tools
were developed in response to IT’s difficulty in
providing data in a timely manner
• Agility and speed to respond to opportunities
have been key benefits
• Drivers of self-service analytics include:
adaptability, affordability, autonomy, and
management
• Data preparation tools reverse the traditional
split between preparation and analysis to 20%
data preparation and 80% data analysis
9. Unanticipated/Unwanted Consequences
• Self-service introduces a major challenge in terms of governance
• Scalability issues arise and suddenly it takes too long to get results
• Analysts define their own calculations/metrics leading to conflicting
results and a new version of spreadmart/dashboard chaos
• Data silos proliferate as every business unit
assembles its own data/metrics
• Departments must maintain
self-managed data marts
• Cross-functional analysis
becomes impossible
10. State of the Market
“The business intelligence (BI) and
analytics market has passed a tipping
point as it shifts away from IT-centric,
reporting-based platforms and toward
modern BI and analytics platforms that
enable smarter analytics and greater
agility.” – Gartner Inc., February 2016.
11.
12. Self-service BI Success Factors
• Top-down and bottom-up approaches must
converge
• IT needs to move from a “doer” to an “enabler”
and resist the desire to play the gatekeeper role
• Create a two-tiered organizational model with a centralized BI team
working collaboratively with a collection of decentralized teams under
the leadership of a BI/analytics CoE or competency centre
• Classify and administer users by skills and needs
• Architect a governed self-service data environment that
maps people, technology and developers to an information
supply chain
13. Classify Users by Skills and Needs
• Numerous types of IT professionals: Developers, DBA’s and System
analysts, and Data engineers
• Business users are made up of casual users and power users
• Casual users are comprised of two subclasses: Data consumers and
Data explorers
• Power users are comprised of three subclasses: Data analysts, Data
scientists and Statisticians
• All users have different skills and needs thus leading to different data
access needs and different application toolsets
• Granular permissions define access/distribution rights
15. Governed Self-service BI
• Need for organizations to refocus on governance has
emerged
• New model must promise not only governance but
also ability to easily source, share and manage data
with agility
• Realization that in some cases “imperfect but fast is better than perfect
but slow” thus the level and ownership of governance depends the use
case and data required
• Business must take responsibility for governing data and balance the
levels required in the two tiers
16. Governed Self-service BI
• Governance is both top-down using governance committees and
bottom-up using tribal knowledge
• Data catalogs crawl databases to profile data while data preparation
tools track data lineage and metadata
• Grassroots governance augments top-down governance
• “Data governance gateway” manages flow of
information from bottom-up self-service area
to top-down centralized area
• Affirmative data governance check results in
seal of approval and potential conversion to
production report
18. Self-service workflows
• Two process flows:
• Top-down – IT manages both data and report/dashboard creation in
fully governed environment
o Casual users consume and lightly modify using interactive
dashboards and self-service tools on governed data
o Power users can provide enhancements
• Bottom-up – power users use sophisticated tools in a lightly governed
environment to perform adhoc analyses
o Create models/reports/dashboards using lightly modeled/raw
non-aggregated data from earlier in the information supply chain
o Casual users can request prototypes/changes from power users/IT
19. Self-service workflows
• Distribution privileges are defined by
user group
• Artifacts at any stage can be
promoted/submitted to governance
committee for verification/approval
• Once approved reports receive a
watermark/seal of approval
• Enables users to distinguish between
curated and non-curated data
Courtesy of the Eckerson Group
20. Reference Architecture
• Three main sections:
• Operations
o Applications, systems and services that feed data into the
analytical environment
• Self-service
o Lightly governed (no IT assistance) for power users that require
adhoc capabilities with sophisticated tools
• Managed service
o Fully governed for casual users with high quality data
o Provisioned with interactive dashboards/search-based tools
21. Information Supply Chain
• Top-down governed supply chain managed by IT department
• Source data – transactional, structured, semi-structured, unstructured
data
• Landing/Staging area – raw data in landing area which undergoes data
quality checks and cleaning prior to transfer to staging area
• Data Hub – summarized, joined data with relevant metrics merged into
linked subject area tables
• Business Views – either physical or virtual views in a abstraction or
semantic layer that displays data in user friendly terms
• Analytic applications – reports, dashboards, and custom layer analytic
applications that often only use a subset of the business view
22. Users and the Information Supply Chain
• Access to respective areas delegated by IT team based on user group
and user skills/needs
• Data scientists – Staging area – require raw non-aggregated data for
creating predictive models
• Data analysts – Data hub – access data from wide subject specific
tables to populate visual discovery tool
• Data explorers – Business views – modify and extend existing reports
and create new simple reports
• Data consumer – snapshots – explore and analyze data within a report
or dashboard
23.
24. Self-service Business Intelligence redux
• Self-service is not easy – users will require
continuous education, training and
support to be successful
• Self-service can deliver not only
governance but also ability to easily
source, share and manage data with agility
• Self-service can integrate governance, user
groups, application tools, and workflows
to achieve a blend of both governance and
user empowerment
