DB2 10 for z/OS introduced temporal data support which allows applications to query data as it existed at different points in time. The document discusses system temporal tables, business temporal tables, and bi-temporal tables. It provides examples of temporal DDL, SELECT extensions for querying historical data, and discusses early experiences and performance considerations with temporal data in DB2 10.
2. Topics
• Introduction
• DB2 10 for z/OS Overview
• Temporal Data Support
• Early Experiences
• Further Reading
• Summary
The Information Management Specialists
3. Introduction
• Director and Principal Consultant at Triton Consulting
• 24 years DB2 experience, 19 as a consultant working with
customers in UK, Europe and the US
• IBM Gold Consultant since 1999
• IBM Information Champion
• Former IDUG (International DB2 User Group) President
• Author of IBM Redbooks, white papers and more recently
“flashbooks”
The Information Management Specialists
4. Topics
• Introduction
• DB2 10 for z/OS Overview
• Temporal Data Support
• Early Experiences
• Further Reading
• Summary
The Information Management Specialists
5. DB2 10 for z/OS
• Extensive beta program running throughout
2009/10, with customers from all around the world
• Generally available since October 2010
• Upgrade path provided from DB2 8 or DB2 9
• Many customers are currently planning their DB2 10
upgrades, to begin in the next 12-24 months
DB2 8 for z/OS End of Support in April 2012
The Information Management Specialists
6. Top New Features
• CPU/Performance Improvements • In-memory object support
• Temporal Data • Optimiser enhancements
• Virtual Storage Enhancements • MEMBER CLUSTER for UTS
• High performance DBATs • Backup and recovery
• Security Extensions enhancements
• Improved Catalog Concurrency • Enhanced audit
• Access Path Management • Include additional index
columns
• pureXML enhancements
• Currently Committed semantics
• Enhanced SQL OLAP functions
• Automated statistics
• Skip Migration
• Dynamic schema change
enhancements • And many more….
The Information Management Specialists
7. Topics
• Introduction
• DB2 10 for z/OS Overview
• Temporal Data Support
• Early Experiences
• Further Reading
• Summary
The Information Management Specialists
8. Why Temporal Data?
• Most IT systems need to keep historical as well as current
information
• Industry regulations and competitive pressures are
prompting IT managers to maintain even more data for
longer periods of time
• Most warehousing / analytics applications require a
historical perspective of data
• Implementation requires lots of effort by DBA and
developer to design, test and implement
Lots of “reinventing the wheel”
The Information Management Specialists
9. Why Temporal Data?
• DB2 10 provides this functionality as part of the database
engine – the first major RDBMS vendor to do so
Improve DBAs and developer productivity
Reduce errors in application code
Reduce time-to-market for new applications
Improve performance by driving function into the database
engine
Improve application consistency
Support compliance / audit requirements
The Information Management Specialists
10. Why Temporal Data?
• Solve common business problems
Ensure that a customer only has one financial position at a
given time
Was an insured covered for a procedure on a specific date?
► Was that information correct at the time the claim was processed?
Establish prices for a catalog ahead of time, so that they
are completed before the change needs to be made
Answer a customer complaint about an old bill
… and many, many more
The Information Management Specialists
11. Temporal Data Concepts
• Temporal Table - a table that supports “data
versioning” to allow point-in-time queries to be
executed
Multiple versions of each row are kept by DB2 as they
change over time
Additional metadata is kept, recording the period in time
when a given version of the row was valid
Contrast to traditional non-temporal (aka “snapshot”)
tables, where only the current version of a row is available
unless developer/DBA have taken additional steps to
support historical perspective
The Information Management Specialists
12. Temporal Data Concepts
• Period – the time during which a given version of a row
is/was valid
Period is defined by special start timestamp and end timestamp
columns in the temporal table
Note that in current DB2 implementation start timestamp is
inclusive (>=), but end timestamp is not inclusive (<)
► Feature request to provide ZPARM and make this a site decision
CustNo … Start_TS End_TS
Period #1
123 … 2011-05-01 11:03 2011-05-26 23:51
123 … 2011-05-26 23:51 9999-12-31 23:59
Period #2
The Information Management Specialists
13. Temporal Data Concepts
• System Temporal Table – a temporal table that uses
DB2 system-defined periods to record row history
Associated with a DB2 system event (INSERT / UPDATE /
DELETE) against table – all changes captured
DB2 automatically sets start and end timestamp for a
period
History is maintained in a separate “history” table
No gaps between periods – end timestamp of previous row
version = start timestamp of new version
Useful for audit/compliance requirements
Typically implemented by DBA
The Information Management Specialists
14. Temporal Data Concepts
• Business Temporal Table – a temporal table that uses
business-defined periods to record row history
Aka “application temporal”
Associated with a business event such as a change of address
Useful for tracking business events over time
History is maintained in the base table
Application has control over start and end timestamp for a
given period
► Can have gaps between periods
Typically implemented by developer and DBA
The Information Management Specialists
15. Temporal Data Concepts
• Bi-Temporal Table – a temporal table that supports
both business and system time periods
Best of both worlds – application-defined versioning with
complete audit trail of system changes
Two sets of start/end timestamps defined
The Information Management Specialists
16. Implementation Choices
• User-managed
Only choice prior to DB2 10
Typically implemented using triggers to catch “before” version of row
during INSERT/UPDATE/DELETE
► Need application logic to determine if a given series already exists (update old
row + INSERT in same UOW) or not (INSERT only)
Lots of scope for inconsistency / errors in application code (e.g.
overlapping periods)
► Need audit function to ensure rows are valid, SQL can be horrible
Lots of scope for performance issues – especially with index design
• DB2-managed
Designed to avoid the pitfalls above
Some potential performance gains – see later
The Information Management Specialists
17. Implementing DB2 Temporal Tables
• DBA indicates which tables need temporal support at CREATE/ALTER
time
Support for UTS, classic partitioned and single table segmented
tablespaces
• For system temporal or bi-temporal tables
DBA also creates separate history table with identical definition to
base table (same columns, data types, null attributes, etc but no
SYSTEM_TIME specification) and associates with base table via ALTER
DB2 automatically copies old version of row to history table whenever
row in main table is changed via DELETE or UPDATE
DB2 automatically UNIONs base and history table as required
The Information Management Specialists
18. Select Extensions
• Elegant extensions to SELECT statement allow historical
perspective to be seen via standard SQL
Same syntax for SYSTEM_TIME and BUSINESS_TIME
SELECT *
FROM BASE_TABLE
FOR SYSTEM_TIME AS OF :TSTAMP
WHERE CUST_NO = 123
Single - Inclusive/Exclusive
SELECT * SELECT *
FROM BASE_TABLE FROM BASE_TABLE
FOR SYSTEM_TIME BETWEEN :ST_TSTAMP FOR SYSTEM_TIME FROM : ST_TSTAMP
AND :EN_TSTAMP TO : EN_TSTAMP
WHERE CUST_NO = 123 WHERE CUST_NO = 123
Multiple - Inclusive/Inclusive Multiple - Inclusive/Exclusive
The Information Management Specialists
19. Implementing System Temporal
• SYSTEM_TIME row begin column
TIMESTAMP(12) NOT NULL GENERATED ALWAYS AS ROW BEGIN
Composed of 9-digit timestamp, plus 3 digits for data sharing member
DB2 ensures uniqueness of generated values across transactions
Not updateable by application
• SYSTEM_TIME row end column
TIMESTAMP(12) NOT NULL GENERATED ALWAYS AS ROW END
Not updateable by application
• Data types for both begin and end columns must be the same
• INSERT, UPDATE, DELETE use standard syntax, all versioning handled by
DB2
• Query rewrite for SELECT statements to UNION ALL history table
The Information Management Specialists
20. System Temporal DDL
CREATE TABLE CUSTOMER
(
CUST_NO INTEGER NOT NULL,
…
SYS_START TIMESTAMP(12) NOT NULL GENERATED ALWAYS AS ROW BEGIN,
SYS_END TIMESTAMP(12) NOT NULL GENERATED ALWAYS AS ROW END,
SYS_CRT TIMESTAMP(12) NOT NULL GENERATED ALWAYS AS TRANSACTION START ID,
PERIOD SYSTEM_TIME(SYS_START, SYS_END)
);
CREATE TABLE CUSTOMER_HIST
(
CUST_NO INTEGER NOT NULL,
…
SYS_START TIMESTAMP(12) NOT NULL,
SYS_END TIMESTAMP(12) NOT NULL
);
ALTER TABLE CUSTOMER ADD VERSIONING USE HISTORY TABLE CUSTOMER_HIST;
The Information Management Specialists
21. Example – System Temporal
System INSERT INTO EMP UPDATE EMP
Event VALUES (321, 20000) WHERE EMPNO = 321
SET SALARY = 25000
T1 T2 Time T3
EmpNo Salary Start_TS End_TS EmpNo Salary Start_TS End_TS
Main 300 10000 T0 31-12-9999 300 10000 T0 31-12-9999
Table
321 20000 T1 31-12-9999 321 25000 T3 31-12-9999
SELECT SALARY EmpNo Salary Start_TS End_TS
FROM EMP = 25000
WHERE EMPNO = 321 321 20000 T1 T3
SELECT SALARY
FROM EMP History Table
FOR SYSTEM TIME AS OF T2 = 20000
WHERE EMPNO = 321
The Information Management Specialists
22. Implementing Business Temporal
• BUSINESS_TIME row begin and row end columns can be defined as either:
TIMESTAMP(6) NOT NULL
DATE NOT NULL
• BUSINESS_TIME periods have a check constraint in which end > begin
column
• New optional unique BUSINESS_TIME WITHOUT OVERLAPS index is useful
in having a built-in mechanism for enforcing keys being unique over a
period of time
Adds start and end columns to index
• UPDATE and DELETE statements have been enhanced to allow updating
and deleting based upon a period of time
FOR PORTION OF BUSINESS_TIME FROM expression1 TO expression2
The Information Management Specialists
23. FOR PORTION OF…
UPDATE DELETE
Rows Contained
FOR PORTION OF
Business Time Row
Result
Span FROM or TO
FOR PORTION OF
Business Time Row
Result
Span FROM and TO
FOR PORTION OF
Business Time Row
Result
The Information Management Specialists
24. Business Temporal DDL
CREATE TABLE CUSTOMER
(
CUST_NO INTEGER NOT NULL,
…
BUS_START TIMESTAMP(6) NOT NULL GENERATED ALWAYS AS ROW BEGIN,
BUS_END TIMESTAMP(6) NOT NULL GENERATED ALWAYS AS ROW END,
PERIOD BUSINESS_TIME(BUS_START, BUS_END)
);
CREATE UNIQUE INDEX CUST_IX
ON CUSTOMER
(
CUST_NO,
BUSINESS_TIME WITHOUT OVERLAPS
);
The Information Management Specialists
25. Topics
• Introduction
• DB2 10 for z/OS Overview
• Temporal Data Support
• Performance
• Early Experiences
• Further Reading
• Summary
The Information Management Specialists
26. Performance – System Temporal
• Redbook measurement
• Transaction mix of 70% read
(SELECT), 30% write (10% INSERT
+ 20% UPDATE/DELETE)
• Compare:
Baseline (no history)
History with triggers
History with system temporal
• Result: 56% less CPU overhead
using system temporal when
compared to triggers
Source: DB2 10 Performance Topics Redbook
The Information Management Specialists
27. Performance – System Temporal
• Redbook measurement
• Compare, for varying number of
rows
UPDATE/DELETE with triggers
UPDATE/DELETE with system
temporal
• Result
30-39% less CPU overhead using
system temporal for UPDATE
when compared to triggers
10-19% less CPU for DELETE
Source: DB2 10 Performance Topics Redbook
The Information Management Specialists
28. Performance – Business Temporal
• Redbook measurement
• Compare performance of UPDATE
which results in two or three rows
being split
UPDATE with business temporal
INSERT/UPDATE via stored procedure
Source: DB2 10 Performance Topics Redbook
The Information Management Specialists
29. Experiences – System Temporal
• System temporal is not a watertight audit strategy
Can ALTER TABLE to remove history relationship if you have the necessary
authority (no record in SYSCOPY if versioning added or dropped)
• Schema change is painful (e.g. adding a column)
Need to prevent access, drop versioning, add column to both tables, add
versioning again
PM31313 addresses this – new function to support ALTER ADD COLUMN and
keep history table in step
• Need to integrate with archival strategy (if you have one…)
• Need to treat base and history table as if they are referentially connected
– backup and recover as a set
• Need to be able to supress history data from being replicated – open PMR
(PM31315)
The Information Management Specialists
30. Experiences – Business Temporal /
General
• Some applications with user-managed temporal functionality
can be ported relatively easily
As long as DB2 implementation is consistent with your site/application
approach
DSNZPARM to set inclusive/inclusive or inclusive/exclusive is a
common request
Benefits must be clear
• Non-temporal UPDATE/DELETE still allowed – care is required
• Limited 3rd party support for SQL extensions
The Information Management Specialists
31. Topics
• Introduction
• DB2 10 for z/OS Overview
• Temporal Data Support
• Performance
• Early Experiences
• Further Reading
• Summary
The Information Management Specialists
32. Further Reading
• IBM DB2 10 Home Page
http://www-01.ibm.com/software/data/db2/zos/db2-10/
• White Paper – DB2 10: A Smarter Database for a Smarter Planet
https://www14.software.ibm.com/webapp/iwm/web/signup.do?source=sw-
infomgt&S_PKG=wp-z-db2-smarter
Also available as part of a “flashbook” - ISBN: 1583473610
• DB2 10 for z/OS Performance Topics Redbook (SG24-7942)
• IDUG – International DB2 User Group
http://www.idug.org/
The Information Management Specialists
33. Topics
• Introduction
• DB2 10 for z/OS Overview
• Temporal Data Support
• Further Reading
• Summary
The Information Management Specialists
34. Summary
• DB2 10 contains a long list of significant enhancements to improve
performance, scalability and productivity
• Temporal support is an industry first, and offers some compelling
advantages for new applications than need a historical data
perspective
Improve DBAs and developer productivity
Reduce errors in application code
Reduce time-to-market for new applications
Improve performance by driving function into the database engine
Improve application consistency
Support compliance / audit requirements
The Information Management Specialists