Slowly changing dimension in dimensional Modeling

1. Slowly Changing
Dimension: Categories
By:
Prof. Sunita Sahu
Assistant Prof, VESIT,Mumbai

2. Slowly Changing Dimension: Categories
 Dimensions that change slowly over time, rather
than changing on regular schedule, time-base.
 In Data Warehouse there is a need to track
changes in dimension attributes in order to report
historical data.
 The usual changes to dimension tables are
classified into three types
 Type 1
 Type 2
 Type 3
2

3. Example
3
Order fact
Product Key
Time Key
Customer Key
Salesperson Key
Order Dollars
Cost Dollars
Margin Dollars
Sale Units
Customer
Customer Key
Customer Name
Customer Code
Martial Status
Address
State
Zip
Salesperson
Salesperson Key
Salesperson
Name
Territory Name
Region Name
Product
Product Key
Product Name
Product Code
Product Line
Brand
Time
Time Key
Date
Month
Quarter
Year

4. Type 1 Changes: Error Correction
 Usually relate to corrections of errors in the source
system.
 For example, the customer dimension: change in
name because of spelling mistake
4

5. Type 1 Changes, cont.
General Principles for Type 1 changes:
 Usually, the changes relate to correction of errors in
the source system
 Sometimes the change in the source system has no
significance
 The old value in the source system needs to be
discarded
 The change in the source system need not be
preserved in the DWH
5

6. Applying Type 1 changes
 Overwrite the attribute value in the dimension table
row with the new value
 The old value of the attribute is not preserved
 No other changes are made in the dimension table
row.
 The key of this dimension table or any other key
values are not affected.
 Easiest to implement.
6

7.  Before the change:
Customer_ID Customer_Name Customer_Type
1 Cust_1 Corporate
 After the change:
Customer_ID Customer_Name Customer_Type
1 Cust_1 Retail

8. Type 2 Changes:
 Let’s look at the martial status of customer.
 One the DWH’s requirements is to track orders by
martial status
 All changes before 11/10/2004 will be under Martial
Status = Single, and all changes after that date will be
under Martial Status = Married
 We need to aggregate the orders before and after the
marriage separately
8

9. Type 2 Changes, cont.
 General Principles for Type 2 changes:
 They usually relate to true changes in source
systems.
 There is a need to preserve history in the DWH.
 This type of change partitions the history in the DWH.
 Every change for the same attributes must be
preserved.
9

10. Type 2 Implementation
 The steps:
 Add a new dimension table row with the new value of
the changed attribute
 An effective date will be included in the dimension
table
 There are no changes to the original row in the
dimension table
 The key of the original row is not affected
 The new row is inserted with a new surrogate key
10

11.  Before the change:

Custo
mer_ID
Customer_N
ame
Customer_T
ype
Start_Date End_Date
1 Cust_1 Corporate 22-07-2010
31-12-9999
Custo
mer_ID
Customer_N
ame
Customer_T
ype
Start_Date End_Date
1 Cust_1 Corporate 22-07-2010
31-12-9999
2 Cust_1 Retail 22-07-2010 31-12-9999
Type 2 Example

12. Type 3 Changes
 Type 3 Slowly Changing Dimension, there will be two
columns to indicate the particular attribute of interest, one
indicating the original value, and one indicating the current
value.
 There will also be a column that indicates when the current
value becomes active.
 Not common at all
 Time-consuming
 We want to track history without lifting heavy burden.
 There are many soft changes and we don’t care for the
“far” history
12

13. Type 3 Changes
 General Principles:
 They usually relate to “soft” or tentative changes in
the source systems
 There is a need to keep track of history with old and
new values of the changes attribute
 They are used to compare performances across the
transition
 They provide the ability to track forward and backward
13

14. Type 3
 No new dimension row is needed
 The existing queries will seamlessly switch to the
current value.
 Any queries that need to use the old value must be
revised accordingly.
 The technique works best for one soft change at a
time.
 If there is a succession of changes, more
sophisticated techniques must be advised
14

15. Customer Key Name State
1001 Williams New York
 After Williams moved from New York to Los Angeles, the
original information gets updated, and we have the following
table (assuming the effective date of change is February 20,
2010):
Customer Key Name Original State Current State Effective Date
1001 Williams New York Los Angeles 20-FEB-2010

Type 3

16.  Advantages
 This does not increase the size of the table, since new
information is updated.
 This allows us to keep some part of history.
 Disadvantages
 Type 3 will not be able to keep all history where an attribute is
changed more than once. For example, if Williams later
moves to Texas on December 15, 2003, the Los Angeles
information will be lost.
Type 3

17. Large Dimension Table
 Dimension table is large based on two factors.
 very deep: that is, the dimension has a very large
number of rows.
 Very wide: that is, the dimension may have a large
number of attributes or columns.
 In a data warehouse, typically the customer and
product dimensions are likely to be large.
 Such customer dimension tables may have as
many as 100 million rows.
 The product dimension of large retailers is also
quite huge.

18. Junk Dimension
 The junk dimension is simply a structure that provides a convenient
place to store the junk attributes. It is just a collection of random
transactional codes, flags and/or text attributes that are unrelated to
any particular dimension.
 In OLTP tables that are full of flag fields and yes/no attributes, many
of which are used for operational support and have no
documentation except for the column names and the memory banks
of the person who created them. Not only do those types of attributes
not integrate easily into conventional dimensions such as Customer,
Vendor, Time, Location, and Product, but you also don’t want to carry
bad design into the data warehouse.However, some of the
miscellaneous attributes will contain data that has significant
business value, so you have to do something with them.


19. Junk Dimension
 Advantage of junk dimension:
 It provides a recognizable location for related codes,
indicators and their descriptors in a dimensional
framework.
 This avoids the creation of multiple dimension tables.
 Provide a smaller, quicker point of entry for queries
compared to performance when these attributes are
directly in the fact table.
 An interesting use for a junk dimension is to capture the
context of a specific transaction. While our common,
conformed dimensions contain the key dimensional
attributes of interest, there are likely attributes about the
transaction that are not known until the transaction is
processed.


20. Junk Dimension

21. Rapidly Changing Dimensions
 If one or more of its attributes changes frequently.
 when you deal with a type 2 change, you create an
additional dimension table row with the new value of
the changed attribute. By doing so, you are able to
preserve the history.
 consider customer dimension. Here the number of
rows tends to be large, sometimes in the range of
even a million or more rows. But significant attributes
in a customer dimension may change many timesin a
year. Rapidly changing large dimensions can be too
problematic for the type 2 approach.

22. Rapidly Changing Dimensions
 One effective approach is to break the large
dimension table into one or more simpler dimension
tables. How can you accomplish this?
 Obviously, you need to break off the rapidly
changing attributes into another dimension table,
leaving the slowly changing attributes behind in the
original table.

23. Solution to rapidly changing dimension
 Large dimensions call for special considerations.
 Because of the sheer size, many data warehouse
functions involving large dimensions may be slow
and inefficient.
 You need to address the following issues by using
effective design methods, by choosing proper
indexes, and by applying other optimizing
techniques: