This presentation contains strategies that BI groups within IT can use to maximize their productivity and value to the business. It contains an overview of why and how ‘agile BI’ is used at direct-marketing leader Valpak, and several other strategies that can be employed to help deliver timely, effective BI solutions.
1. Practical BI
Strategies that IT can use to
maximize its productivity and
value to the business
Tom Spetnagel
Director of Business Intelligence, Valpak
Tom Spetnagel
2. 2
Summary of Strategies
• Use agile practices for BI development
• Determine actual requirements and
design to them
• Utilize appropriate requirements-
gathering techniques
• Implement and wield a BI charter
Tom Spetnagel
4. Good News for BI:
It‟s Taking Off!
Web Search
Semantic Search
Mobile BI
BI Applications
Mobile CPU Optimization
Data Mining
Inline BI
Enterprise Search
Social Analytics
Self-Service
Data
Technology Social Sentiment Monitoring
Access
Real-Time BI
Big Data
Web Analytics
In-Memory BI
Public Data
Cloud Master-Data-Management
In-Database Analytics
Unstructured Data
Data Replication
Web Services
Tom Spetnagel
5. 5
The BI Explosion Is Scary, Too
• BI is getting bigger and more
complex, but BI budgets aren‟t
keeping pace!
• Access to information is the
root of recent evolution:
Google, Facebook, mobile
• Self-Service BI is continuously
on-the-rise
• IT is therefore becoming less
central in BI!
Tom Spetnagel
6. 6
Users Won‟t Wait on IT
http://xkcd.com/
BI users can’t wait on IT;
they create their own
solutions, and they aren’t
always good ones!
Tom Spetnagel
7. BI‟s Biggest Challenge:
7
Prioritization!
• Since BI supply can‟t keep
up with demand,
continuously producing
„something good at the
right time‟ is critical
Tom Spetnagel
8. 8
BI Stakeholders:
Managing BI stakeholders
is a lot like trying to keep
chickens under a blanket!
-They’re not aligned
-They want it all
-They want it now
-They always want
something different
Tom Spetnagel
9. 9
What‟s A Solution?
(Practical BI, Strategy #1)
Tom Spetnagel
10. 10
What is „Agile‟?
• The application of common sense to
software development*
• A set of concepts developed by
people frustrated with the
application of „traditional‟ project
management to software
* I wish I could trademark this!
Tom Spetnagel
11. 11
Agile Evolution
• Agile Manifesto conceived at an
informal drink-and-ski weekend in 2001
• Reaction to fundamental differences
of building software and building
physical items (like aircraft carriers)
Tom Spetnagel
12. 12
Unlike with physical construction, since it’s only
‘zeros-and-ones’, software can be changed quickly!
Tom Spetnagel
13. 13
4 Main Agile Principles
More Important Less Important
• Individuals and • Processes and tools
interactions • Comprehensive
• Working software documentation
• Customer collaboration • Contract negotiation
• Responding to change • Following a plan
http://agilemanifesto.org/
Tom Spetnagel
14. 14
Primary Intentions of Agile
• Deliver the most valuable thing at
the right time
• Deliver working software quickly!
• Embrace but manage change
• Establish short-term predictability
• Eliminate surprises from both the IT
and business sides
Tom Spetnagel
15. 15
Some Agile
„Methodologies‟
Scrum
Extreme Programming (XP)
Unified Process (UP)
Feature Driven Development (FDD)
Lean Software Development
Crystal Clear
Tom Spetnagel
16. 16
Agile at Valpak:
“Scrum”
• Multiple scrum teams, each team having:
– 1 Scrum Master, 1 Product Owner, 5 to 7 Team
Members
• 2 week iterations, executing several „stories‟
per team, bounded by:
– Sprint planning (1st Monday)
– Sprint demo and review (2nd Friday)
• Daily stand-up status meetings
Tom Spetnagel
17. 17
Types of BI „Stories‟ Include:
• ETLs • Performance
• Metadata Mapping • Data Quality
• Formal Reports / • Security
Dashboards • Upgrade/patch
• Alerts
• Automated Report
Distribution
Tom Spetnagel
18. 18
Why Is Agile Great for BI?
• Creates a practical method for
handling crucial BI challenges which
drive scope and affect success
• Gives ownership and flexibility to the
business, not IT
Tom Spetnagel
19. Crucial Scope-Drivers in BI (1)
• “Data Quality”
– A catch-all term for numerous different problems:
• Unclear definitions
• Missing data / duplicated data
• Unexpected data
• Unreconciling data
• Performance/Speed
– People expect reports to run as fast as business
„transactions‟ (create 1 order, save 1 order, etc.)
• And it‟s even worse with mobile devices!
Tom Spetnagel
20. 20
Data Quality vs. Effort
Data quality is a function
of effort; increasing effort
has diminishing returns
and it is never possible to
reach 100% data quality
Tom Spetnagel
21. Crucial Scope-Drivers in BI (2)
• “Terminology”/Definitions
The cultural hurdles that
come with defining or
redefining terms for BI take
much time to overcome!
• Historical Data
Stakeholders often want
‘history’, not just information
from this point forward!
Tom Spetnagel
22. 22
Agile Handling of BI Scope-Drivers
• Data Quality
– Iterate to provide additional quality checks where
desired
• Performance
– Iterate to achieve better performance where desired
• Terminology
– Iterate to update definitions where needed; within an
iteration, make a decision and go!
• History
– Load the history in a separate iteration after new data
collection has been activated
Tom Spetnagel
23. 23
Business Accountability
• Let the business decide what they
want most in the next iteration
(based on what IT tells them it can
get done in that timeframe)
Tom Spetnagel
24. 24
1 More Reason
Agile Is Great for BI
• It‟s tough for BI stakeholders to know
what something is worth!
• Example: What is it worth to you to
have a timely, accurate bank
balance?
Tom Spetnagel
25. 1 Last Reason
25
Agile Is Great for BI
“Walking on water and developing
software from a specification are easy
if both are frozen”
- Edward V. Berard, "Life-Cycle Approaches"
BI Stakeholders can rarely
know what they really need
(or need next) until they’re
using it!
Tom Spetnagel
26. 26
Challenges of Agile
• High ratio of planning &
communicating time to coding time
• High amount of time discussing &
refining the agile process; some
danger of over-analysis
• High % of time collaborating; IT folks
need to be good communicators
Tom Spetnagel
27. 27
Agile „In Their Own Words‟:
http://www.youtube.com/watch?v=A
0As88akpXs
Tom Spetnagel
28. 28
Practical BI Strategy #2:
Determine, and Deliver to,
the Actual BI Requirements
• Don‟t deliver just what is (initially)
requested; scientifically deconstruct it
into what is actually needed
• „Requirements‟ and „design‟ are
different in BI, just as in application
development
Tom Spetnagel
29. 29
BI Requirements vs. Design:
Example #1
Analyst Questions Designer Questions
1. Do users expect the 1. What should the data
new data to reconcile sources be? Should the
with anything existing? output have any built-in
2. How many people will validations, reconciliations,
need access to the or subtotals?
same info at the same 2. What mechanism is best for
time? How often? providing shared data
(web page? email or text
alert? printed poster?)?
Tom Spetnagel
30. 30
BI Requirements vs. Design:
Example #2
Analyst Questions Designer Questions
1. How recent/up-to-date 1. Does the solution require
does information need access to real-time
to be? transaction data? Or can it
2. What is the acceptable be data warehouse data,
timeframe for accessing updated/frozen on a
information? What are schedule?
the response-time 2. Should data be stored in-
requirements? database or in-memory?
What summarization or
indexing is needed?
Tom Spetnagel
31. 31
Mockups
• A report or dashboard mockup is nice
but does not constitute either
comprehensive requirements or design
• Mockups are a great starting point for
a requirements conversation, though!
Tom Spetnagel
32. 32
Practical BI Strategy #3:
Use the Best Requirements-
Gathering Technique for
the BI Assignment
• There are a number of
different and effective ways to
gather requirements
• Review, implement, and
combine these however
necessary
Tom Spetnagel
34. 34
Practical BI Strategy #4:
Implement and Wield
a BI Charter
• Gather a set of goals,
principles, and strategies that IT
and the business can agree on
• Use this to focus discussions
and overcome objections to IT
proposals and decisions
Tom Spetnagel
35. 35
BI Charter:
Example #1
• Goals
– Limit confusion around „what numbers are right/best‟
• Principles
– Data in the data warehouse are the „official‟ figures
unless specifically documented otherwise
• Strategies
– Get „official‟ figures into the data warehouse
– Avoid storing both official and unofficial figures for the
same metric in the data warehouse
– Restrict access to unofficial data in 3rd party tools
Tom Spetnagel
36. 36
BI Charter:
Example #2
• Goals
– Minimize the amount of BI tool-training required
• Principles
– IT will not support unofficial tools which users have self-
provided
– Access to 3rd-party BI platforms will be supported on an
exception basis when unique value is provided
• Strategies
– Minimize the number of tools users must know how to use
– Use a BI platform which scores highly on ease-of-use and
which has multi-purpose tools
Tom Spetnagel
