The term Business Intelligence was first used widely by Howard Dresner from Gartner who proposed "business intelligence" as an umbrella term to describe "concepts and methods to improve business decision making by using fact-based support systems” In the late 1990s this usage was widely accepted and most of the other terms like management information, decision support systems, executive information systems began to disappear. Despite the 25 years experience we have with Business Intelligence the results have been disappointing, there have been some spectacular successes but in general the majority of organizations have not achieved the results that they expected. This mini-presentation is one of a series where we try to explain the basics of Business Intelligence to the people who will pay for it – the Business.
More than Just Lines on a Map: Best Practices for U.S Bike Routes
Building blocks of Business Intelligence
1. The building blocks of
Business Intelligence
Norman Manley – Decision Support Systems
Business Intelligence, if you get it right, gives you the
chance to improve your organization significantly – but
improvement means change and that isn’t easy to achieve.
The term Business Intelligence was first used widely by Howard Dresner from Gartner who proposed "business intelligence" as an umbrella term to describe "concepts and methods to improve business decision making by using fact-based support systems” In the late 1990s this usage was widely accepted and most of the other terms like management information, decision support systems, executive information systems began to disappear. Despite the 25 years experience we have with Business Intelligence the results have been disappointing, there have been some spectacular successes but in general the majority of organizations have not achieved the results that they expected. This mini-presentation is one of a series where we try to explain the basics of Business Intelligence to the people who will pay for it – the Business.
In the last 25 years there have been many definitions of Business Intelligence, mostly from vendors of either software or services who altered the definition to suit their products. Here we have returned to Howard Dresner’s original concept – to improve business decision-making and as a result improve the performance of the organization. The other major change here has been the emphasis on the fact that it is a continuing process. Business Intelligence has often been seen as a project – like building a bridge, and as such the financial people in our organizations expect it to have a planning, a budget and an end date when it is delivered and then no more costs except maintenance. Successful Business Intelligence is much more like having a child than building a bridge, you invest large amounts before the birth and afterwards the child grows bigger, has to be fed and clothed and educated and hopefully the satisfaction of being a parent rises – but so do the costs. Basically it isn’t a project it’s a life’s work, it can bring spectacular results but it requires constant care and attention.
Almost all organizations have many different applications, sometimes hundreds, that perform some administrative task like producing invoices or inventory control or managing human resources. In most cases these applications were either bought separately or home built without fully integrating them either in terms of definitions of the underlying data (the metadata) or in terms of the hardware and software platforms they use – the result is we have many different sources of data, none of which actually “talk” to each other. The second problem is that most of these applications have their own reporting solutions that generate their own “Business Intelligence” completely separate from the other applications, this results in every manager having their own sources of data which they have often copied into a spreadsheet to try and produce some sort of sensible information which you can more or less guarantee will be different from the information that their colleague has.
The first step of real Business Intelligence is to access all the various applications and extract as much data as we can, volume of data used to be a problem, but now storage costs so little it shouldn’t be a problem.
The second stage is to store the data somewhere central, separate from the applications because when we start processing this data we don’t want to affect the performance of the production systems.
This data is NOT clean, it will contain duplicates, there will be bits missing and some pieces will be differently formatted than similar data from other applications.
We must start with the definitions because Data and Information are not the same thing. Data and Information are two different building blocks of Business Intelligence and it is important that we understand the difference:
Data is a collection of things that we have either registered or measured, examples are the price of something we sold, the number of that article that we sold, the date that somebody called in sick, the salary a person has on a given date or the length of a telephone call.
To Change Data into Information we need to decide how we are going to define things, for example we could define Profit as being the difference between the sales price and the cost of materials plus 10% manufacturing costs expressed as US dollars, we then need to combine all the data from all the sources we have, both internal and external and then we need to apply the definitions and the business rules to make sure we are integrating them correctly.
The Information we produce is something that has been deduced or calculated from the available data, revenue is a calculation based on number of items sold and price, number of sickness days is based on calculating the days between somebody registering as sick and returning to work, salary increase is based on comparing the salary at one date with the salary at another date and the cost of a telephone call is based on a number of factors including how long the call was, where it was and what time of day it was made. This is all information that can be generated form data.
The intention of the information as defined here is to be “One source of the Truth” in other words this information should be (or become) the one place where everybody in the organization goes when they need to know about what is happening in terms of performance.
Again we must start with the definitions; Knowledge implies that something is true, so what we are trying to find is the truth behind the information that we are given, in fact if there are less people receiving unemployment benefit this year than there were last year we will want to know why. There could be (and usually are) a number of different causes, we need to understand what they are to be knowledgeable.
Seeing that 10% of gearboxes have to replaced under warranty (information) does not give us the knowledge that in 90% of the cases there was nothing wrong with them and the warranty claims were fraudulent, which means that whilst improving the quality of the gearbox may cost 5 million dollars it will not solve 90% of the problem. Knowledge is the basis of taking good decisions and that comes from information.
Having the knowledge is the first step, but getting it accepted as being true is much more difficult. Particularly if the knowledge we produce does not agree with the current thinking at management level we are often told that our conclusions must be wrong. Ideas that are different are often considered as dangerous, to accept new concepts we may have to rethink our existing beliefs and possibly modify our policies – all of which leads to change, and research shows us that people in general are resistant to change.
The second step when we have established that the knowledge we have generated is (probably) accurate is to decide what we need to do to produce the desired results. We know from the definition that Business Intelligence is about performance improvement, and improvement means change. Different organizations have different cultures so who decides “what we need to do” will vary per organization, it might be the board of directors, it might be an individual manager or it might be some sort of steering committee, but whoever it is it needs to happen – we need to take action and then the loop begins again, we generate data, transform it into information, the information becomes knowledge and we decide if we made the right changes, if they are sufficient or possibly that we need to do something else.
To summarize the 4 steps:
We start with collecting the data, both from inside and outside of our organizations, as much as we can access because storing data is no longer expensive and the more data we have the more useful information we can generate
We clean the data and combine it using the definitions that we have agreed and we apply the business rules to provide information.
We use the information and both internal and external experts to gain insight into what is really happening in our organization – this insight is what we call knowledge.
We plan and carry out the actions that we think are most likely to produce the results we want taking account of the organizations’ published mission and strategies
…. and then we start all over again, we collect the data that has been generated in the new situation and we go through the whole process again to generate insights which will tell us if we took the right actions and what other changes we will need to make.