1. Informix Warehouse
Accelerator
Jacques Roy
IBM, Informix development
April 6, 2011 © 2010 IBM Corporation

2. Agenda
■ Data warehouse industry trends
■
■ Data warehouse on Informix
■
■ Infomrix warehouse accelerator
■
2 © 2010 IBM Corporation

3. Sate of Data Warehousing 2011
DBMS Market in 2011:
■ DBMS market at the close of 2009 was approximately $21.2
billion (2010 data not yet available)
■ Data Warehouse DBMS market was approximately 35% of the
DBMS market or $7.42 billion
Key Findings:
■ Data warehouse DBMSs have evolved to a broader analytics
infrastructure supporting operational analytics, corporate
performance management and other new applications and uses.
■ Cost is driving interest in alternative architectures but
performance optimization is driving multi-tiered data architectures
and a variety of deployment options - notably a strong interest in
in-memory data mart deployments.
3 © 2010 IBM Corporation

4. Sate of Data Warehousing (cont.)
Market Dynamics for 2011
■ Today, smaller data warehouses, those less than 5 TB's of source
system extracted data (SSED) are the only "data warehouse" for the
entire organization and are commonly solving organizations' analytic
needs.
Analysis:
■ Gartner only rarely encounters an organization which has actually
delivered on the Enterprise Data Warehouse (EDW) vision. The EDW
remains a design principle, but it is rarely if ever actually deployed.
Gartner estimates that between 70% and 75% of all systems referred to
as EDW are actually single business departments in nature.
■ Optimization techniques such as summaries, aggregates and indexes
are simply the result of performance restrictions inherent to normalized
data and the way the RDBMS manages rows and columns.
4 © 2010 IBM Corporation

5. State of Data Warehousing (Cont.)
A Glimpse Into the Future
■ Vendor solutions began to focus even more on the ability to
isolate and prioritize workload types including strategies for dual
warehouse deployments and mixing OLTP and OLAP on the
same platform.
■ In-memory DBMS solutions provide a technology which enables
OLTP/OLAP combined solutions. Organizations should increase
their emphasis on financial viability during 2011 and even into
2012 as well as aligning their analytics strategies with vendor
road maps when choosing a solution.
5 © 2010 IBM Corporation

7. Existing Informix Warehouse Features
■ Performance & Scalability
– Inherent SMP Multi-threading
– Parallel Data Query (PDQ)
– Light Scan for fast table scans
– Online Index build
– Efficient Hash Joins
– Auto Fragment Elimination
– Memory Grant Manager (MGM)
– High Performance Loader
– Optimistic Concurrency
■
■ Easy of Management
– Time cyclic data management using Range Partitioning
– Sophisticated Query Optimizer for OLTP and Warehousing
7 © 2010 IBM Corporation

8. Informix Warehousing Moving Forward
■ Goal is to provide a comprehensive warehousing platform that
is highly competitive in the marketplace
–
– Incorporating the best features of XPS and Red Brick into IDS for
OLTP/Warehousing and Mixed-Workload
–
– Using the latest Informix technology in:
• Continuous Availability and Flexible Grid
• Data Warehouse Accelerator using latest industry technology
–
– Integration of IBM’s BI software stack
8 © 2010 IBM Corporation

13. BI Tools for Informix
The Performance Management Framework
Cognos 10 provides a comprehensive set Cognos identifies best-practice decision areas, or
of BI tools for: information sweet spots by business function:
Reporting
Analysis
Dashboards
Scorecards
Performance Management Framework for:
Solutions for different areas of the
organization
13 © 2010 IBM Corporation

14. Third Generation of Database Technology
According to IDC’s Article (Carl Olofson) – Feb. 2010
1st Generation:
- Vendor proprietary databases of IMS, IDMS, Datacom
2nd Generation:
- RDBMS for Open Systems, dependent on disk layout, limitations in scalability
and disk I/O
- Database tuning by adding updating stats, creating/dropping indexes, data
partitioning, summary tables & cubes, force query plans, resource governing
3rd Generation: IDC Predicts that within 5 years:
■ Most data warehouses will be stored in a columnar fashion
■ Most OLTP database will either be augmented by an in-memory database
(IMDB) or reside entirely in memory
■ Most large-scale database servers will achieve horizontal scalability through
clustering
14 © 2010 IBM Corporation

15. Market Data: Key Drivers of Change
15 © 2010 IBM Corporation

16. Informix Warehouse Accelerator
How is it different?
What is it?
• Performance: Unprecedented response
The Informix Warehouse Accelerator (IWA) is a times to enable 'train of thought' analysis
workload optimized, appliance-like, add-on, that enables frequently blocked by poor query
the integration of business insights into operational performance.
processes to drive winning strategies. It accelerates
• Integration: Connects to IDS through deep
select queries, with unprecedented response times.
integration providing transparency to all
applications.
• Self-managed workloads: queries are
executed in the most efficient way
• Transparency: applications connected to
IDS, are entirely unaware of IWA
• Simplified administration: appliance-like
hands-free operations, eliminating many
database tuning tasks
Breakthrough Technology Enabling New Opportunities
16 © 2010 IBM Corporation

17. Breakthrough Technologies for Performance
Extreme Compression Row & Columnar Database
Required because RAM is the limiting factor. Row format within IDS for transactional workloads
and columnar data access via accelerator for
OLAP queries.
Multi-core and Vector In Memory Database
Optimized Algorithms 3 generation database technology avoids
rd
Avoiding locking or synchronization 7 1 I/O. Compression allows huge databases
to be completely memory resident
6 2
5 3
Predicate evaluation on 4 Frequency Partitioning
compressed data Enabler for the effective parallel access of
Often scans w/o decompression the compressed data for scanning.
during evaluation Horizontal and Vertical Partition
Elimination.
Massive Parallelism
All cores are used within used for queries
17 © 2010 IBM Corporation

18. IWA: Characteristics
• A dedicated SMP system (Linux on Intel x86_64)
• No changes to the applications
– Applications continue to attach to IDS.
– When applicable query needs to be executed, IDS exploits the accelerator
transparently to the applications
– Fencing and protection of IDS against possible accelerator failures
• Order of magnitude performance improvement
• Reducing need for tedious tuning of IDS (partitioning, indexes, etc.)
• Appliance-like form-factor
– Hands free operations
• Significantly improved price/performance and TCO as a combined effect
of:
– Accelerating intensive & complex analytics queries
– Orders of magnitude performance improvement for accelerated queries
– Reduced DBA effort for tuning accelerated queries
18 © 2010 IBM Corporation

19. Sample Customer Results: Case Study #1
Query Description Informix Informix w ISAO Notes Improvement
1 Find Top 100 Entities 1:28:22 0:01:28 Fact Table Scan 6023.23%
2 Find Top 100 Members 1:22:32 0:01:05 Fact Table Scan 7640.45%
Summarize all transactions by State
3 and County 1:34:37 0:00:14 Fact Table Scan 41708.49%
IWA did not
support this
Summarize the top 10 Commodities subquery
4 by State and County 1:05:03 1:03:35 query 102.29%
Detailed Report on Specific
Programs, States, Counties and
5 Years 0:00:00 0:00:00 Index Read 83.45%
Detailed Report on Specific
6 Programs in a Date Range 0:00:06 0:00:06 Index Read 108.41%
Summarize all transactions by
State, County, City, State, Zip,
Program, Program Year,
7 Commodity and Fiscal Year 1:48:58 0:00:41 Fact Table Scan 15800.89%
Failed - I did not configure
Find Entities where the payments do Long enough logs to
not equal total Member Transac Failed - Long support the
8 Transaction Amounts tion Transaction query
Totals 7:19:37 1:07:09 654.69%
19 © 2010 IBM Corporation

20. Government Agency Datamart
Performance expectation goals were up to 20X OLAP-style Queries
Tests were done on a Intel x86_64 SMP box running Linux RHEl
Microstrategy Report was used, which generates 667 SQL statements
537 are SELECT statements.
Datamart for this report has 250 Tables and 30 GB Data size
Informix Panther and IWA run this report in 67 seconds.
7 seconds in IWA and 60 seconds in Informix (TEMP table processing, etc)
Without IWA, total runtime on Informix 11.70 on the same HW is 40 Minutes!
The same report today runs on XPS & SUN HW (Sparc M9000) and takes 90 mins.
Performance gain for the customer would be ~90x !!!
20 © 2010 IBM Corporation

21. IWA Referenced Hardware Configuration
Intel(R) Xeon(R) CPU X7560 @ 2.27GH 4 X 8
Memory 512G
6 disks 300 GB SAS hard disk
drives each
Options:
- 4-processor, 4U rack-optimized enterprise server with Intel® Xeon®
processors.
- 8-core, 6-core and 4-core processor options with up to 2.26 GHz (8-
core), 2.66 GHz (six-core) and 1.86 GHz (four-core) speeds with up to
16 MB L3 cache
- Scalable from 4 sockets and 64 DIMMs to 8 sockets and 128 DIMMs
- Optional MAX5 32-DIMM memory expansion
- 16x 1.8" SAS SSDs with eXFlash or 8x 2.5" SAS HDDs
21 © 2010 IBM Corporation

22. IWA Software Components
■ Linux on Intel x86_64 (RHEL 5 or SUSE SLES 11)
■
■ IDS 11.70 + IWA code modules including IDS Stored Procedures
(Informix Ultimate Warehouse Edition)
■
■ ISAO Studio Plug-in – GUI for Mart definition
■
■ OnIWA – On Utilities for Monitoring IWA
22 © 2010 IBM Corporation