8. Preparing Your Organization for
BI Automation
Kalido Summer Series: Find Hidden
Costs in Your Business Before They
Find You
9. Framing the Conversation & The Power of Relativity
“…humans rarely choose things in absolute terms. We don’t
have an internal value meter that tells us how much things
are worth. Rather, we focus on the relative advantage of
one thing over another, and estimate value accordingly.” –
Predictably Irrational, Dan Ariely
Which one of these circles is bigger than the other?
12. Nik: “Do you have slowly-changing dimensions?”
Local DW Guru: “What’s that?”
YOU DON’T HAVE A DATA WAREHOUSE!!!
Nik: “How do you handle surrogate key management?”
Local DW Guru: “Why would I use surrogate keys?”
Nik: “How do you manage referential integrity?”
Local DW Guru: “The database handles it.”
Nik: “How many subject areas do you manage?”
Local DW Guru: “One.”
Nik: “Are your dimensions conformed?”
Local DW Guru: “Heh?”
A Data Warehouse Is Not a Relative Concept
13. “I don’t mean Data Warehouse the way you do.”
Offering to automate what I don’t do today or even understand is a
silly proposition.
We Have to Baseline Our Organizations on a Definition
“Can I go see it, the Data Warehouse?”
“Teradata, our Data Warehouse…”
“Nik, I know you’re going to get
mad, but we should just…”
“You know, Data Warehouse, a big
database”
14. Modeling
Star and Snowflake Schema
Physical Schema Management
Slowly Changing Dimensions
Data Mart and Aggregates
Data Load and Index
Management
Rollup Path Awareness
Incremental Summary
Generation
Convert Existing Logical Models
Name & Label Management
Release to Production
Version Management
Object Level Change
Management
Model Migration
Generic Export/Import for Data
Migration
Model Comparison Report
Object Level Dependency for
Migration Versions
Testing
Built-in Integrity Checking
Aggregate Task Results
Excel Integration for User
Reconciliation
Data rollback and Batch Reload
User Interface for Data BrowsingOperations
Task Execution & Monitoring
Deployment & Migration
Audit and Logging
Process Automation
Archiving
Job Definition with Dependency
Data Integration
Data Sourcing and Field Mapping
Data Detection
Data Validation
Code Management and Lookup
Suspense and Exception
Handling
Currency and Units of Measure
System Key Management Post Processing Housekeeping
BI Delivery
Native XLS Pivot Table
Generation
Metadata Management
Report-Time Formula
Management
These capabilities
enable building a
scalable Business
Intelligence platform.
19. …humans have consistently
demonstrated an ability to
find new things to do that
are of greater value when
jobs have been outsourced
or automated.
– Philip Rosedale (Creator of Second Life),
Abundance
22. • Frame the conversation…make sure everyone
is talking about the same thing.
• Reinvest your time-savings in developing a
better solution.
• Re-tool your people.
These three variables seem nearly always to be in a discussion about failed or sub-optimal data warehouse implementations. Too long to get to value. Too slow to adapt to change. And too expensive over a long period of time.Time to value is often measured in years rather than months. In other words that measuring stick of value-----business improvement------which is enabled by the best information being available for consumption by business processes or analytics, takes too long to get to. If ever. Responding quickly to change is an aspirational hallmark of most organizations today. Whether the change is market driven, or merger/divestiture driven, or driven by a change in the fundamental structure of a business it is the speed that effects of change can be interpreted and acted on that reflects a key element of value in a data warehouse. If the reports or analytics can’t be updated in time to assess and exploit opportunity or challenge while it is there, the value is diminished.And finally, TCO. Hand built warehouses are expensive to build and maintain, because adapting one to accommodate change in the business is pretty much the same as implementing one. So you need the same talent pool of ETL programmers and support people that you needed to build it in the first place. You don’t accrue economies over time through incremental changes and adjustments made by two or three people--------you accrue costs to rebuild each time you make a change.Experts agree that hand building data warehouses is expensive, time consuming and complex. Speeding the process up in the words of one of those experts Ralph Hughes is “simple but not easy”.(NEXT SLIDE)
That Gap is still alive and well when we talk about creating a data warehouse with value for the business. In other words, one that meets CURRENT requirements rather than those that were generated years ago.We wanted to find out how prevalent this gap was, so we went and asked. We polled nearly 550 practitioners and business people at TDWI events throughout the year.Data validates what we intuitively know. Business change occurs at a pace that IT capabilities have trouble matching. Some of the comments that we picked up along the way included:“We are pretty well aligned with our IT group, but either the technologies or processes they are using can’t provide the response we need.”“We use the word “agile” a lot, but in reality if we can’t extensively plan for a change, it is difficult to make one.”“We set up our own data engine to support our most critical needs.” (shadow IT)In the research we’ve conducted and sponsored and the interactions throughout the year with clients, three themes really do continue to come up over and over again.We refer to these as the Achilles Heels of Data Warehousing. (NEXT SLIDE)
Talk About Ralph’s 3-2-1We actually questioned the responses on this slide a bit. Many of the people we polled shared with us anecdotally:“I’ll be guessing at costs, because a lot of the people aren’t in my department.”“I’m pretty sure that we’re way over a million dollars/year, but I’m embarrassed to say that.”“We set up our own data mart because IT couldn’t deliver and we have two guys running it.”“I know there are a whole bunch of consultants in the building but I don’t know what they get paid.”“I’m pretty sure the warehouse was never finished so I guess we don’t spend anything anymore.”Scary stuff. A lot of people just don’t know what their costs are, and then there are those who have divorced themselves entirely from an IT sponsored and delivered data warehouse because it doesn’t meet requirements. In either case it’s a net loss for the organization.Here’s some sobering data though….. The costs of maintaining a hand built data warehouse generally do NOT reduce over time. There are very few efficiencies built into the warehouse because it must be hand built every time a change occurs. The frequency of changes directly drive those costs. There is no such thing as an incremental change and incremental expense.The simple fact is that through better use of automation the dependency on staff is reduced and the dependency on 3rd party tools to support hand built development is reduced.Here are two data points from two studies.First is a poll that Kalido ran. We asked the questionThrough another piece of research we sponsored with Nucleus Research, we found that for a company of 2500 employees who spend 15% of their time locating, assembling and analyzing data and where the efficiency of the processes is “fair” where “fair” is just below the midpoint for efficiency, the negative economic impact on OPERATIONS is $2.8M per year. That’s on business operations alone without any data warehousing operations or technology costs factored in. So making incremental corrections to how efficiently your company manages its data, integrates new sources, adapts quickly to change…….all that compounds into some rather large numbers.
Kalido customers are nearly 4X as likely to spend less than $100K on additional software for the warehouse than the TDWI World Conference survey respondents.TDWI World Conference respondents are nearly 4X more to spend over $1M.
Without a frame of reference, the conversation has to be different. This guy is unfamiliar with the plight of the …, so first you have to explain the problem that you are trying to solve, and then you can discuss solving that problem faster and cheaper.Transition: Now that we have framed the conversation, we can talk about automation.
Relay story of current Data Warehouse project.Started with a scope that did not include Customer or CRMBudget was already lockedBrought in automation capability and expanded the scope.Enabled us to move in lock-step with the development of requirements