SlideShare une entreprise Scribd logo
1  sur  46
Télécharger pour lire hors ligne
Ed Lynch – Executive Client Technical Professional
Data Warehousing and Business Analytics for System z
10 October 2012




A Hybrid Technology Platform for Increasing
the Speed of Operational Analytics




                                                       © 2009 IBM Corporation
Speaker Biography
 Ed Lynch is an Executive Client Technical Professional specializing in IBM’s System z Data
 Warehousing, Business Analytics, and Information Integration software products. Ed’s twenty-
 eight year career with IBM has spanned many areas of IBM, and has always involved IBM's
 Information Management (IM) products. His previous roles have included DB2 for z/OS
 Development and Management, DB2 Technical Marketing, Development and Delivery of IM
 Product Education, a Principal of Information Integration Design and Implementation
 Consulting Services, and DB2 Tools & Information Integration Technical Sales.
 Currently, Ed is the lead Technical Specialist for North America’s System z Data Warehousing
 and Business Analytics, and Information Integration Software Technical Sales team.
 Ed has worked extensively with DB2 across the various operating system platforms,
 InfoSphere Data Replication, InfoSphere Classic Replication Server, DB2 Analytics
 Accelerator, DB2 DataPropagator, InfoSphere Federation Server, InfoSphere Classic Federation
 Server, DB2 Connect, IBM Information Server, and IBM’s Business Analytics solutions. In his
 current role, he frequently works with product development in identifying and prioritizing
 product requirements, and developing product strategy. He also provides software technical
 sales support and works extensively with customers to create architectures using these
 products.
 edlynch@us.ibm.com
 1-972-561-9975




                                                                                     © 2009 IBM Corporation
Abstract

   With the wealth of data available today, organizations are no longer willing to relegate
   information to the back office. Modern organizations are demanding access to information.
   However, it is not enough to capture information, users must be quickly able to sift through
   these massive amounts of data, extract information and transform it into actionable
   knowledge. Systems today are enabling organizations to anticipate risk, identify threats,
   assess readiness, and match the risk assessment to the resources required to address
   them; all at the time of decision. They use a platform that provides the ability to react to
   changes decisively, based upon the facts of the situation, not in hours or days- but at the
   moment of opportunity. They optimize decisions based upon current weather conditions,
   past threats and behaviors and current resource availability to assure a successful
   operation.This session will review the architecture and benefits of a hybrid system of MPP
   and SMP technologies enabling the merging of fit for purpose and mixed workload
   capabilities into a single system. See how this hybrid system facilitates both transaction-
   oriented applications and analytics into a single platform for operational analytics. Find out
   why these enhancements are the next logical steps in creating a highly optimized
   environment, both in price and performance, that is designed to meet the wide range of
   analytic workloads that today's organizations need to accommodate.




                                                                                        © 2009 IBM Corporation
DB2 Analytics Accelerator V3                   More insight from your data
Further extending the features
                                             • Unprecedented response times for
                                               “right-time” analysis

                                             • Complex queries in seconds rather
                                               than hours

                                             • Transparent to the application

                                             • Inherits all System z DB2 attributes

                                             • No need to create or maintain indices

                                             • Eliminate query tuning
     Blending System z and Netezza
                                             • Fast deployment and time-to-value
     technologies to deliver unparalleled,
     mixed workload performance for
     complex analytic business needs.


 4                                                                       © 2009 IBM Corporation
DB2 Analytics Accelerator
      Train-of-thought Analytics




    FAST                     Cost Saving          Appliance
    Complex queries run      Eliminate costly     No applications to
    up to 2000x faster       query tuning while   change, just plug it
    while retaining single   offloading complex   in, load the data,
    record lookup speed      query processing     and gain the value




5                                                                        © 2009 IBM Corporation
Introducing
     DB2 Analytics Accelerator V3
     Reducing the Cost of High Speed Analytics




    Improve Productivity       Lower Host Costs           Consolidate
    Eliminate query tuning     Reduce storage costs       Reduced complexity

    Eliminate table indexing   Offload query processing   Reduced software costs

    Minimize storage admin     Defer system upgrades      Reduced hardware costs



6                                                                      © 2009 IBM Corporation
Fast Time to Value
   IBM DB2 Analytics Accelerator
          Production ready - 1 person, 2 days
   Table Acceleration Setup             2 Hours
     –   DB2 “Add Accelerator”
     –   Choose a Table for “Acceleration”
     –   Load the Table (DB2 copy to Netezza)
     –   Knowledge Transfer
     –   Query Comparisons
   Initial Load Performance
         400 GB “Loaded” in 29 Min
         570 million rows (Loads of 800GB to 1.3TB/Hr)
   Actual Query Acceleration            1908x faster
         2 Hours 39 Minutes to 5 Seconds
   CPU Utilization Reduction
         35% to ~0%
Actual customer results, October 2011




                                                         © 2009 IBM Corporation
Performance & Savings
                                                                  DB2 w ith     Times
                                                                                Faster
                                                                                         Queries run faster
                                               DB2 O nly            IDAA
                Total    Total                                                             • Save CPU resources
               Row s    Rows
     Query    Reviewed Returned               Hours    Sec(s)    Hours Sec(s)              • People time
    Query 1   2,813,571 853,320                2:39     9,540      0.0      5    1,908
    Query 2   2,813,571 585,780                2:16     8,220      0.0      5    1,644     • New Business
    Query 3   8,260,214      274               1:16     4,560      0.0      6      760       opportunities
    Query 4   2,813,571 601,197                1:08     4,080      0.0      5      816
    Query 5   3,422,765      508               0:57     4,080      0.0     70       58
    Query 6   4,290,648      165               0:53     3,180      0.0      6      530
    Query 7     361,521   58,236               0:51     3,120      0.0      4      780
    Query 8    3,425.29      724               0:44     2,640      0.0      2    1,320
    Query 9   4,130,107      137               0:42     2,520      0.1   193        13

    DB2 Analytics Accelerator: “we had this up and
    running in days with queries that ran over 1000
    times faster”
    DB2 Analytics Accelerator: “we expect ROI in
    less than 4 months”
                                                                                      Advance to 32 minute mark for DB2
                                                                                     Analytics Accelerator section of keynote


                  Accelerating decisions to the speed of business
8                                              12 October 2012
      Actual customer results, October 2011                                                                    © 2009 IBM Corporation
IBM DB2 Analytics Accelerator V3 Product Components

                                                                 Netezza
                           zEnterprise                         Technology
         CLIENT




       Data Studio
       Foundation
      DB2 Analytics
       Accelerator                                   Network
      Admin Plug-in
                                OSA-                 Primary
                              Express3                 10Gb
                               10 GbE
                                                     Backup


                                                               IBM DB2
                      Data Warehouse application
                                                               Analytics
                      DB2 for z/OS enabled for IBM
      Users/                                                   Acelerator
                       DB2 Analytics Accelerator
    Applications


9                                                                    © 2009 IBM Corporation
Deep DB2 Integration within zEnterprise
                  Applications                 DBA Tools, z/OS Console, ...
             Application Interfaces                   Operational Interfaces
              (standard SQL dialects)                  (e.g. DB2 Commands)


                                DB2 for z/OS

                                                                         IBM
       Data             Buffer                         Log               DB2
      Manager          Manager
                                    ...    IRLM
                                                     Manager           Analytics
                                                                      Accelerator



       Superior availability                                                            Superior
       reliability, security,                     z/OS on                           performance on
      Workload management                         System z                          analytic queries




                                                                       Netezza

10                                                                                    © 2009 IBM Corporation
TM
Accelerator powered by Netezza 1000 Appliance
                                       Slice of User Data
                                       Swap and Mirror partitions
                                       High speed data streaming
                                       High compression rate
                                        EXP3000 JBOD Enclosures
                                          12 x 3.5” 1TB, 7200RPM, SAS (3Gb/s)
   Disk Enclosures                        max 116MB/s (200-500MB/s compressed data)
                                        e.g. 1000-12: 8 enclosures → 96 HDDs(32/128 TB)




                                       Accelerator Server
  SMP Hosts                            SQL Compiler, Query Plan, Optimize,
                                       Administration
                                         2 front/end hosts, IBM 3650M3 or 3850X5
                                          clustered active-passive
                                         2 Nehalem-EP Quad-core 2.4GHz per host
  Snippet BladesTM
  (S-Blades, SPUs)
                                        Processor & streaming DB logic
                                        High-performance database engine
                                        streaming joins, aggregations, sorts, etc.


                                                                    © 2009 IBM Corporation
S-Blade™ Components                          Dual-Core FPGA
                                                 8 FPGA Processors/Blade




                                        Netezza DB Accelerator




                      Intel Quad-Core
                       8 Cores/Blade



                            IBM BladeCenter Server
                                                        © 2009 IBM Corporation
Eliminating the I/O Bottleneck
Move the SQL to the hardware        to where the data lives




                       “Just send
                      the Answer,
                        not Raw
                         Data”
                                                      © 2009 IBM Corporation
select DISTRICT,PRODUCTGRP,
The Key to the Speed                                    sum(NRX)
                                                        from   MTHLY_RX_TERR_DATA
                                                        where MONTH = '20091201'
                                                        and    MARKET = 509123
                                                        and    SPECIALTY = 'GASTRO'
                                            FPGA
              CPU Core
                                            Core                               Zone Map




            Complex ∑          Restrict,    Project
          Joins, Aggs, etc.                           Uncompress
                               Visibility
                                                                     Slice of table
                                                                   MTHLY_RX_TERR_DATA
                                                                      (compressed)


   sum(NRX)       where   MONTH = '20091201'          select DISTRICT,
                  and     MARKET = 509123                    PRODUCTGRP,
                  and     SPECIALTY = 'GASTRO'               sum(NRX)
                                                                           © 2009 IBM Corporation
Bringing Netezza AMPPTM Architecture to DB2 for z/OS
AMPP = Asymmetric Massively
       Parallel Processing


                                                  CPU     FPGA

Advanced                                             Memory
Analytics



   BI                           SMP               CPU     FPGA
                                Host
                 DB2 for z/OS                        Memory

 Legacy
Reporting

                                                  CPU     FPGA

  DBA                                                Memory



                                       Network                         Disk
                                       Fabric      S-Blades™           Enclosures


                                       IBM DB2 Analytics Accelerator
                                                                       © 2009 IBM Corporation
Query Execution Process Flow

              Application     Optimizer
               Interface
                                                                                                                       SPU
                                                                                                                 CPU         FPGA

                                                                                                                    Memory




                                                                    Accelerator DRDA Requestor
                                                                                                                       SPU
                                                                                                                 CPU         FPGA




                                                                                                 SMP Host
                                                                                                                    Memory


Application                                                                                                            SPU
                               Query execution run-time for                                                      CPU     FPGA

                              queries that cannot be or should                                                      Memory
                              not be off-loaded to Accelerator
                                                                                                                       SPU
                                                                                                                 CPU     FPGA

                                                                                                                    Memory




                                    DB2 for z/OS                                                            DB2 Analytics Accelerator



                            Queries executed without DB2 Analytics Accelerator
                            Queries executed with DB2 Analytics Accelerator
                                                                                                                             © 2009 IBM Corporation
Workload-Optimized Query Execution
                                                                                    • Single and unique system
                                                         DB2 for z/OS and             for mixed query workloads
                                                    IBM DB2 Analytics Accelerator
                                                                                    • Dynamic decision for most
 OLTP-like query
 OLTP-like query
                                                                                      efficient execution platform
                   User control and DB2 heuristic


                                                                                    • New special register
                                                               DB2 Native
                                                                                      QUERY ACCELERATION
                                                               DB2 Native
   Light ODS-
   Light ODS-                                                  Processing
                                                               Processing              – NONE
      query
      query
                                                                                       – ENABLE
                                                                                       – ENABLE WITH FAILBACK
                                                                                    • New heuristic in DB2
 Light BI Query
 Light BI Query                                                                       optimizer



 Heavy BI Query
 Heavy BI Query                                           Optimized processing
                                                             for BI Workload




                                                                                                       © 2009 IBM Corporation
Accelerator Data Load
                                                             DB2 for z/OS                                Accelerator

                                                             Table A
                                                   Table B

                                                                                                         CPU   FPGA
                                                              Part 1   Unload   USS Pipe
               Accelerator Administrative Stored




                                                                                                          Memory
                                                   Table C



Accelerator                                                                                              CPU   FPGA
                         Procedures




  Studio                                           Table D
                                                              Part 2   Unload   USS Pipe




                                                                                           Coordinator
                                                                                                          Memory

                                                   Part 1
                                                               .          .        .
                                                                                                         CPU   FPGA
                                                               .          .        .
                                                                                                          Memory
                                                   Part 2
                                                               .          .        .
                                                                                                         CPU   FPGA
                                                   Part 3              Unload   USS Pipe
                                                             Part m                                       Memory




              • 1 TB / h – can vary, depending on CPU resources, table partitioning,
              • Update on table partition level, concurrent queries allowed during load
              • V2.1 & V3 unload in DB2 internal format, single translation by accelerator
                                                                                                                      © 2009 IBM Corporation
DB2 Analytics Accelerator V3
Lowering the Costs of Trusted Analytics


 What’s New?                                  • zEnterprise EC12 Support
 • High Performance Storage                     Version 3 will support the zEnterprise
   Saver                                        EC12, z196 and z114 System z
                                                platforms
     Store a DB2 table or partition of data
     solely on the Accelerator. Removes       • Query Prioritization
     the requirement for the data to be
                                                Brings System z workload
     replicated on both DB2 and the
                                                management down to the individual
     Accelerator
                                                query being routed to the Accelerator
 • Incremental Update
                                              • High Capacity
     Enables tables within the Accelerator
                                                Support has been extended to include
     to be continually updated throughout
                                                the entire Netezza 1000 line (1.28 PB)
     the day.
                                              • UNLOAD Lite
                                                Reduces z/OS MIPS consumption, by
                                                moving the preparation off System z.




19                                                                        © 2009 IBM Corporation
Build a System z Trusted Analytic System
Reduce the cost of host storage for historical data by 95%!




Historical                     High Performance            Low Latency Data
Most data in an analytic       All aggregate queries run   Tables and partitions that
system is historical and not   at the same high speed as   require updating will be
subject to change. Most        any accelerator supported   able to be updated by
data can be in a Storage       query                       incremental update, table
Saver and maintain trusted                                 load or partition load
performance and security



                                                                             © 2009 IBM Corporation
High Performance Storage Saver
Reducing the cost of high speed storage

                      Store historic data on the Accelerator only
                                     Applications
                                                                        Tables can be resident on:
                                                                           1. DB2 Only
                                                                           2. DB2 and Accelerator
                                                                           3. Accelerator Only
                               SQL

                                                                          When data no longer
                                                                       requires updating, reclaim
         DB2                   DB2
                                                                            the DB2 storage
                                                    Accelerator
        Table A               Table A                Table A

                                                                         Special Registers control behavior
     High speed                                    High speed               CURRENT QUERY ACCELERATION
     indexed                                       aggregate
     lookups, best          Accelerator            lookups, best for        CURRENT GET_ACCEL_ARCHIVE
     for OLTP
                              Table A              complex DSS type
     type queries                                  queries               Managed by zParms

21                   Mixed workload type queries
                                                                                               © 2009 IBM Corporation
Save Over 95% of Host Disk Space for Historical Data
                                    Historical Data
      Year   Year -1     Year -2      Year -3         Year -4   Year -5   Year -7


      1Q       1Q          1Q            1Q             1Q       1Q         1Q


       2Q      2Q           2Q           2Q             2Q       2Q         2Q



      3Q       3Q          3Q           3Q              3Q       3Q        3Q


               4Q          4Q            4Q             4Q       4Q         4Q




                                     Current Data

       4Q        One Quarter = 3.57% of 7 years of data
                 One Month = 1.12% of 7 years of data
                 One month = 2.78% of 3 years of data
                                                                            © 2009 IBM Corporation
High Performance Storage Saver
  Reducing the cost of high speed storage
              Time-partitioned tables where:
                 – only the recent partitions are used in a transactional context (frequent data
                   changes, short running queries)
                 – the entire table is used for analytics (data intensive, complex queries).
              DB2 partitions are deleted after the High Performance Storage Saver are created
              on the accelerator




                      DB2                                     No longer present on DB2 Storage
Query from
Application
                       #1

               Or

                    Accelerator   Accelerator   Accelerator     Accelerator       Accelerator    Accelerator    Accelerator
                       #1            #2            #3                #4                #5           #6                #7


  23                                                                                                           © 2009 IBM Corporation
The Evolution of a High Performance Storage Saver
High Speed Access to Historical Data


      Table / Data          Accelerator       Accelerator   Archive
       Creation           Load / Update /IU      Only        Only

          DB2                 DB2             Accelerator
         Table A             Table A           Table A




                           Accelerator
                             Table A


        Backup               Backup




24
                                                               © 2009 IBM Corporation
Storage options to match data needs
Optimized in both price and performance for differing workloads


           High Performance Storage Saver               Database Resident Partitions
                       Single Disk Store                            Dual Disk Store
       •   Only stored on Accelerator storage (Less   • Stored on both DB2 and Accelerator
           Cost)                                        storage
       •   Optimized performance for                  • Mixed query workload with transactions,
           deep analytics, multifaceted, reporting      single record queries and record updates
           and complex queries                          with deep analytics, multifaceted,
       •   Only full table update or full partition     reporting and complex queries.
           update from backup                         • Full table, full partition update, Incremental
       •   Same high speed query access                 update from DB2 data
           transparently through DB2                  • Same high speed query access
                                                        transparently through DB2




       Cost                  The right mix of cost and functionality                 Functionality
                                                                                                © 2009 IBM Corporation
25
The zEnterprise Hybrid Solution
Mixed Workloads for Next Generation Business Analytics
          Operational                  Analytic                Mixed Workload
          Applications               Applications               Applications

     Transaction Processing      Data warehousing               Operational BI




     Shared Everything DB       Shared Nothing DB                 Hybrid DB

       High volume business       Low volume complex          High volume business
       transactions and batch   queries context switching     transactions and batch
          reporting running                                      reporting running
            concurrently                                    concurrently with complex
                                                                      queries


26                                                                         © 2009 IBM Corporation
Incremental Update


                                                  Table or
                           ELT or ETL         Partition Update
         OLTP                    Data              DB2 Analytics
       Application             Warehouse            Accelerator




                               Data              Incremental
                            Replication            Update




                 Synchronizing data to lower data latency
                     from days to minutes/seconds
27                                                                 © 2009 IBM Corporation
Option 1: Full Table Refresh
 Changes in data warehouse tables typically
 driven by scheduled (nightly or more
 frequently) ETL process
 Data used for complex reporting based on
 consistent and validated content (e.g., weekly
                                                                   Operational Analytics, Reports, OLAP,
                                                                   Operational Analytics, Reports, OLAP,
 transaction reporting to the central bank)
 Multiple sources or complex transformations                                         Continuous
                                                                                     Continuous
                                                                                       Query
 prevent propagation of incremental changes                                             Query
                                                                                     Processing
                                                                                     Processing

 Full table refresh triggered through DB2 stored
 procedure (scheduled, integrated into ETL                                  DB2 z/OS Query Optimizer
                                                                            DB2 z/OS Query Optimizer
 process or through GUI)

                                                                     DB2 native
                                                                     DB2 native                        Accelerator
                                                                                                       Accelerator
                                                                     processing
                                                                     processing                        processing
                                                                                                       processing
                             ETL Process




 Queries may continue
                             ETL Process




 during full table refresh
                                                                                  Full table refresh
 for accelerator
                                                                            DB2 for z/OS database
                                                                            DB2 for z/OS database
                                           Changes / Replacement                                         © 2009 IBM Corporation
Option 2: Table Partition Refresh
 Changes in data warehouse table typically driven by “delta” ETL process (considering only
 changes in source tables compared to previous runs) or by more frequent changes to most
 recent data
 Optimization of Option 1 when target data warehouse table is partitioned and most recent
 updates are only applied to the latest partition

                                                       Operational Analytics, Reports, OLAP,
                                                       Operational Analytics, Reports, OLAP,
 Table partition refresh triggered through DB2
 stored procedure (scheduled, integrated into
                                                                             Continuous
                                                                             Continuous
 ETL process or through GUI)                                                   Query
                                                                                Query
                                                                             Processing
                                                                             Processing


 Maintains snapshot
                                                                    DB2 z/OS Query Optimizer
                                                                    DB2 z/OS Query Optimizer
 semantics for consistent
 reports
 Queries may continue                                    DB2 native
                                                         DB2 native                            Accelerator
                                                                                               Accelerator
                              Replication
                              Replication




                                                         processing
                                                         processing                            processing
                                                                                               processing
 during table partition
 refresh for accelerator                                 January


                                                         February


                                                         March
                              ETL Process
                              ETL Process




                                                         April


                                                         May               Partition refresh

                                            Changes
                                                                    DB2 for z/OS database © 2009 IBM Corporation
                                                                    DB2 for z/OS database
Option 3: Incremental Update
 Changes in data warehouse tables typically
 driven by replication or manual updates
  – Corrections after a bulk-ETL-load of a data warehouse
    table
  – Continuously changing data (e.g. trickle-feed updates from
    a transactional system to an ODS)

 Reporting and analysis based on most recent
                                                                 Operational Analytics, Reports, OLAP,
                                                                 Operational Analytics, Reports, OLAP,
 data
 May be combined with Option 1 & 2 (first table                                     Continuous
                                                                                    Continuous
 refresh and then continue with incremental                                           Query
                                                                                       Query
                                                                                    Processing
                                                                                    Processing
 updates)

                                                                          DB2 z/OS Query Optimizer
                                                                          DB2 z/OS Query Optimizer
                                      Application
                                      Application




                                                                   DB2 native
                                                                   DB2 native                        Accelerator
                                                                                                     Accelerator
 Incremental update can be                                         processing
                                                                   processing                        processing
                                                                                                     processing
 configured per database
 table
                                      Replication
                                      Replication




                                                                                Incremental Update
                                                    Changes
                                                                          DB2 for z/OS database
                                                                          DB2 for z/OS database © 2009 IBM Corporation
Now expandable to 960 cores and 1.28 petabytes
                                                                                                        1                  10



                                                                                                               .......




                        002      005         010        015     020          030        040       060   060              100


      Cabinets           1/4      1/2         1         1 1/2   2             3          4         6     8                10



  Processing Units       24       48         96         144     192          288        384       576   768              960



    Capacity (TB)        8        16         32          48     64            96        128       192   256              320


     Effective
                         32       64         128        192     256          384        512       768   1024             1280
   Capacity (TB)*

                                              PureData System for Analytics


              Predictable, Linear Scalability throughout entire family

  Capacity           = User Data space
  Effective Capacity = User Data Space with compression               *: 4X compression assumed
                    Low Latency, High Capacity Update                                                          © 2009 IBM Corporation
Connectivity Options                                               DB2                              DB2
 Multiple DB2 systems can connect to a single Accelerator




 A single DB2 system can connect to multiple Accelerators                            DB2




 Multiple DB2 systems can connect to multiple Accelerators           DB2                                  DB2




 The same table can be stored in the multiple Accelerators
 (except High Performance Storage Saver tables)                  Full flexibility for DB2 systems:
                                                         •   residing in the same LPAR
     Better utilization of Accelerator resources         •   residing in different LPARs
                                                         •   residing in different CECs
     Scalability                                         •   being independent (non-data sharing)
     High availability                                   •   belonging to the same data sharing group
                                                         •   belonging to different data sharing groups


                                                                                                © 2009 IBM Corporation
32
Analytics Accelerator Table Definition and Deployment
       IBM Data Studio Client                             DB2 for z/OS                                 DB2 Analytics
                                                                                                        Accelerator
                                                           Accelerator
               Accelerator                                Administrative                               Netezza Catalog
                 Studio                                 Stored Procedures

                                                           DB2 Catalog




     The tables need to be defined and deployed to the Accelerator before data is loaded and queries sent to it for
     processing.
       Definition: identifying tables for which queries need to be accelerated
       Deployment: making tables known to DB2, i.e. storing table meta data in the DB2 and Netezza catalog.
     IBM DB2 Analytics Accelerator Studio guides you through the process of defining and deploying tables, as well as
     invoking other administrative tasks.
     IBM DB2 Analytics Accelerator Stored Procedures implement and execute various administrative operations such
     as table deployment, load and update, and serve as the primary administrative interface to the Accelerator from
     the outside world including Accelerator Studio.



33                                                                                                                © 2009 IBM Corporation
Shielding Against Disk Failures


     Primary

      Mirror

      Temp




 •   All user data and temp space mirrored
 •   Disk failures transparent to queries and transactions
 •   Failed drives automatically regenerated
 •   Bad sectors automatically rewritten or relocated



                                                             © 2009 IBM Corporation
Shielding Against S-BladeTM Failures




                    .       .        .           .   .
                    .       .        .           .   .
                    .       .        .           .   .



     S-Blades




 •   S-Blade failure is automatically detected




                                                         © 2009 IBM Corporation
Shielding Against S-BladeTM Failures




                   .        .        .                .        .
                   .        .        .                .        .
                   .        .        .                .        .



     S-Blades



 •   Drives automatically reassigned to active S-Blades within a chassis
 •   Read-only queries (that have not returned data yet) automatically restarted
 •   Transactions and loads interrupted
 •   Loads automatically restarted from last successful checkpoint


                                                                          © 2009 IBM Corporation
Disaster Recovery Option 1 – Table Loaded in One Accelerator (1 of 2)
                                                      SYSPLEX
App 1              DSG Member 1                                                       DSG Member 2

                                               Tables       Tables                                                        App 4
                                              of App 4     of App 5
App 2

                                          Tables      Tables     Tables                                                   App 5
App 3                                    of App 1    of App 2   of App 3


                   Short Range                                                         Short Range
                                                      Long
                     Switch                           Range
                                                                                          Switch

                   Short Range                                                         Short Range


          Accelerator Instance 1                                             Accelerator Instance 2




                                                    Tables Created
                                                    but Not Loaded          Tables       Tables       Tables
                                                                           of App 1     of App 2     of App 3

         Tables      Tables        Tables                                      Tables           Tables
        of App 1    of App 2      of App 3                                    of App 4         of App 5


                                                                                                                © 2009 IBM Corporation
Disaster Recovery Option 1 – Table Loaded in One Accelerator (2 of 2)
                                                                                                                         App 1
                                                     SYSPLEX
App 1              DSG Member 1                                                      DSG Member 2
                                                                                                                         App 2
                                               Tables      Tables
                                              of App 4    of App 5
App 2
                                                                                                                         App 3
                                          Tables     Tables     Tables
App 3                                    of App 1   of App 2   of App 3
                                                                                                                         App 4
                   Short Range                                                        Short Range
                                                     Long                                                                App 5
                     Switch                          Range
                                                                                         Switch

                   Short Range                                                        Short Range


          Accelerator Instance 1                                           Accelerator Instance 2




                                                                           Tables       Tables       Tables
                                                                          of App 1     of App 2     of App 3
                                                    Already Created
         Tables      Tables        Tables           Must LOAD                Tables            Tables
        of App 1    of App 2      of App 3                                  of App 4          of App 5


                                                                                                               © 2009 IBM Corporation
Disaster Recovery Option 2– Table Loaded in Two Accelerators (1 of 2)
                                                       SYSPLEX
 App 1              DSG Member 1                                                       DSG Member 2

                                                 Tables      Tables                                                        App 4
                                                of App 4    of App 5
 App 2

                                            Tables     Tables     Tables                                                   App 5
 App 3                                     of App 1   of App 2   of App 3


                    Short Range                                                         Short Range
                                                       Long
                      Switch                           Range
                                                                                           Switch

                    Short Range                                                         Short Range


           Accelerator Instance 1                                             Accelerator Instance 2




          Tables      Tables        Tables                                   Tables      Tables        Tables
         of App 1    of App 2      of App 3                                 of App 1    of App 2      of App 3

              Tables             Tables                                         Tables           Tables
             of App 4           of App 5                                       of App 4         of App 5


                                                                                                                 © 2009 IBM Corporation
Disaster Recovery Option 2 – Table Loaded in Two Accelerators (2 of 2)
                                                                                                                           App 1
                                                       SYSPLEX
 App 1              DSG Member 1                                                       DSG Member 2
                                                                                                                           App 2
                                                 Tables      Tables
                                                of App 4    of App 5
 App 2
                                                                                                                           App 3
                                            Tables     Tables     Tables
 App 3                                     of App 1   of App 2   of App 3
                                                                                                                           App 4
                    Short Range                                                         Short Range
                                                       Long                                                                App 5
                      Switch                           Range
                                                                                           Switch

                    Short Range                                                         Short Range


           Accelerator Instance 1                                            Accelerator Instance 2




                                                       Data Already
                                                       Available
          Tables      Tables        Tables                                   Tables       Tables       Tables
         of App 1    of App 2      of App 3                                 of App 1     of App 2     of App 3

              Tables             Tables                                        Tables            Tables
             of App 4           of App 5                                      of App 4          of App 5


                                                                                                                 © 2009 IBM Corporation
Why Both?
Marrying the best of both worlds
                      IBM                                  IBM
                PureData N1001                           System z




                  Focused Appliance                 Mixed Workload System

        Capitalizing on the strengths of both platforms while driving to the most
        cost effective, centralized solution - destroying the myth that transaction
                 and decision systems had to be on separate platforms

       Very focused workload                               Very diverse workload

                                                                                      © 2009 IBM Corporation
Tailored to your needs
A Hybrid Solution
                                                                        IBM
                         IBM                                        System z with
                        Netezza                            IBM DB2 Analytics Accelerator

                      Focused Appliance                        Mixed Workload System
                                                      • Mixed workload system z with operational
          • Appliance with a streamlined                transaction systems, data warehouse,
            database and HW acceleration for            operational data store, and consolidated
            performance critical functionality          data marts.
          • Price/performance leader                  • Unmatched availability, security and
                                                        recoverability
          • Speed and ease of deployment and
                                                      • Natural extension to System z to enable
            administration                              pervasive analytics across the
          • Optimized performance for                   organization.
            deep analytics, multifaceted, reporting   • Speed and ease of deployment and
            and complex queries                         administration


     True Appliance                    Flexible Integrated System                Custom Solution




    Simplicity           The right mix of simplicity and flexibility                  Flexibility

                                                                                          © 2009 IBM Corporation
Next Steps
                                       Opportunity




      Operational           New Workload                   Data
       Reporting                                      Consolidation        Operational BI
     (Accelerator)           or Winback
                               (ISAS +                   (ISAS +              (ISAS +
          OR
                             Accelerator)              Accelerator)         Accelerator)
       Existing z
      Warehouse




                                                            Warehouse
                Workload                                   Collaboration
               Assessment                                   Workshop



                                            Optional POC
                                                                                © 2009 IBM Corporation
43
The Ultimate Consolidation Platform
                      Data Mart Data Mart Data Mart Data Mart                         Bringing it all together
                                                                                      • Better Business Response
                           Data Mart Consolidation                                    • Reduced Costs
                              System z PR/SM
                             Recognized leader in mixed                               • More Available
                        virtualization and workload isolation
                                                                                      • More Secure
Transaction Systems                                                                   • Reduced Data Movement
      (OLTP)
                                                                                      • Better Governance
                                                                                      • Reduced Data Latency
                                                                                      • Reduced Complexity

  Data Warehousing
                                                                                      • Reduced Resources
                                z/OS:                           Netezza:
 Business Intelligence Recognized leader in mixed         Recognized leader in
  Predictive Analytics  workloads with security,           cost-effective high
                                 availability             speed deep analytics
                              and recoverability

                                                   Together:
                        Destroying the myth that transactional and decision support
                               workloads have to be on separate platforms
44                                                                                                    © 2009 IBM Corporation
Learn More




                  Visit the Data Warehousing &
                  Business Analytics Webpage
         http://www.ibm.com/software/data/businessintelligence/systemz/




                                                                          © 2009 IBM Corporation
Ed Lynch
     System z Data Warehousing & Business Analytics
                 edlynch@us.ibm.com



46                                                    © 2009 IBM Corporation

Contenu connexe

Tendances

IBM Systems solution for SAP NetWeaver Business Warehouse Accelerator
IBM Systems solution for SAP NetWeaver Business Warehouse AcceleratorIBM Systems solution for SAP NetWeaver Business Warehouse Accelerator
IBM Systems solution for SAP NetWeaver Business Warehouse AcceleratorIBM India Smarter Computing
 
How a Cloud Computing Provider Reached the Holy Grail of Visibility
How a Cloud Computing Provider Reached the Holy Grail of VisibilityHow a Cloud Computing Provider Reached the Holy Grail of Visibility
How a Cloud Computing Provider Reached the Holy Grail of Visibilityeladgotfrid
 
Fi nf068c73aef66f694f31a049aff3f4
Fi nf068c73aef66f694f31a049aff3f4Fi nf068c73aef66f694f31a049aff3f4
Fi nf068c73aef66f694f31a049aff3f4Shawn D'souza
 
Rvrbd steelhead family_brochure_web_101013
Rvrbd steelhead family_brochure_web_101013Rvrbd steelhead family_brochure_web_101013
Rvrbd steelhead family_brochure_web_101013mglatts
 
Oracle en Entel Summit 2010
Oracle en Entel Summit 2010Oracle en Entel Summit 2010
Oracle en Entel Summit 2010Entel
 
Solix Corporate Overview
Solix Corporate OverviewSolix Corporate Overview
Solix Corporate OverviewKunal Grover
 
Development of concurrent services using In-Memory Data Grids
Development of concurrent services using In-Memory Data GridsDevelopment of concurrent services using In-Memory Data Grids
Development of concurrent services using In-Memory Data Gridsjlorenzocima
 
2016 August POWER Up Your Insights - IBM System Summit Mumbai
2016 August POWER Up Your Insights - IBM System Summit Mumbai2016 August POWER Up Your Insights - IBM System Summit Mumbai
2016 August POWER Up Your Insights - IBM System Summit MumbaiAnand Haridass
 
IBM Spectrum Scale and Its Use for Content Management
 IBM Spectrum Scale and Its Use for Content Management IBM Spectrum Scale and Its Use for Content Management
IBM Spectrum Scale and Its Use for Content ManagementSandeep Patil
 
Liquidity Risk Management powered by SAP HANA
Liquidity Risk Management powered by SAP HANALiquidity Risk Management powered by SAP HANA
Liquidity Risk Management powered by SAP HANASAP Technology
 
Oracle Database 11g Lower Your Costs
Oracle Database 11g Lower Your CostsOracle Database 11g Lower Your Costs
Oracle Database 11g Lower Your CostsMark Rabne
 
Iod session 3423 analytics patterns of expertise, the fast path to amazing ...
Iod session 3423   analytics patterns of expertise, the fast path to amazing ...Iod session 3423   analytics patterns of expertise, the fast path to amazing ...
Iod session 3423 analytics patterns of expertise, the fast path to amazing ...Rachel Bland
 
Real-Time Loading to Sybase IQ
Real-Time Loading to Sybase IQReal-Time Loading to Sybase IQ
Real-Time Loading to Sybase IQSybase Türkiye
 
MT129 Isilon Data Lake Overview
MT129 Isilon Data Lake OverviewMT129 Isilon Data Lake Overview
MT129 Isilon Data Lake OverviewDell EMC World
 
Track 2, session 6 best practices to virtualize mission critical sap environm...
Track 2, session 6 best practices to virtualize mission critical sap environm...Track 2, session 6 best practices to virtualize mission critical sap environm...
Track 2, session 6 best practices to virtualize mission critical sap environm...EMC Forum India
 
Hadoop Twelve Predictions for 2012
Hadoop Twelve Predictions for 2012Hadoop Twelve Predictions for 2012
Hadoop Twelve Predictions for 2012Cloudera, Inc.
 
Performance Management in ‘Big Data’ Applications
Performance Management in ‘Big Data’ ApplicationsPerformance Management in ‘Big Data’ Applications
Performance Management in ‘Big Data’ ApplicationsMichael Kopp
 
Faster, Cheaper, Easier... and Successful Best Practices for Big Data Integra...
Faster, Cheaper, Easier... and Successful Best Practices for Big Data Integra...Faster, Cheaper, Easier... and Successful Best Practices for Big Data Integra...
Faster, Cheaper, Easier... and Successful Best Practices for Big Data Integra...DataWorks Summit
 

Tendances (20)

IBM Systems solution for SAP NetWeaver Business Warehouse Accelerator
IBM Systems solution for SAP NetWeaver Business Warehouse AcceleratorIBM Systems solution for SAP NetWeaver Business Warehouse Accelerator
IBM Systems solution for SAP NetWeaver Business Warehouse Accelerator
 
How a Cloud Computing Provider Reached the Holy Grail of Visibility
How a Cloud Computing Provider Reached the Holy Grail of VisibilityHow a Cloud Computing Provider Reached the Holy Grail of Visibility
How a Cloud Computing Provider Reached the Holy Grail of Visibility
 
Fi nf068c73aef66f694f31a049aff3f4
Fi nf068c73aef66f694f31a049aff3f4Fi nf068c73aef66f694f31a049aff3f4
Fi nf068c73aef66f694f31a049aff3f4
 
Rvrbd steelhead family_brochure_web_101013
Rvrbd steelhead family_brochure_web_101013Rvrbd steelhead family_brochure_web_101013
Rvrbd steelhead family_brochure_web_101013
 
Oracle en Entel Summit 2010
Oracle en Entel Summit 2010Oracle en Entel Summit 2010
Oracle en Entel Summit 2010
 
Solix Corporate Overview
Solix Corporate OverviewSolix Corporate Overview
Solix Corporate Overview
 
Ingres Products
Ingres Products Ingres Products
Ingres Products
 
Development of concurrent services using In-Memory Data Grids
Development of concurrent services using In-Memory Data GridsDevelopment of concurrent services using In-Memory Data Grids
Development of concurrent services using In-Memory Data Grids
 
2016 August POWER Up Your Insights - IBM System Summit Mumbai
2016 August POWER Up Your Insights - IBM System Summit Mumbai2016 August POWER Up Your Insights - IBM System Summit Mumbai
2016 August POWER Up Your Insights - IBM System Summit Mumbai
 
IBM Spectrum Scale and Its Use for Content Management
 IBM Spectrum Scale and Its Use for Content Management IBM Spectrum Scale and Its Use for Content Management
IBM Spectrum Scale and Its Use for Content Management
 
Liquidity Risk Management powered by SAP HANA
Liquidity Risk Management powered by SAP HANALiquidity Risk Management powered by SAP HANA
Liquidity Risk Management powered by SAP HANA
 
Oracle Database 11g Lower Your Costs
Oracle Database 11g Lower Your CostsOracle Database 11g Lower Your Costs
Oracle Database 11g Lower Your Costs
 
Iod session 3423 analytics patterns of expertise, the fast path to amazing ...
Iod session 3423   analytics patterns of expertise, the fast path to amazing ...Iod session 3423   analytics patterns of expertise, the fast path to amazing ...
Iod session 3423 analytics patterns of expertise, the fast path to amazing ...
 
Lsi Nytro flash ssd PCIe controllers product training value proposition and k...
Lsi Nytro flash ssd PCIe controllers product training value proposition and k...Lsi Nytro flash ssd PCIe controllers product training value proposition and k...
Lsi Nytro flash ssd PCIe controllers product training value proposition and k...
 
Real-Time Loading to Sybase IQ
Real-Time Loading to Sybase IQReal-Time Loading to Sybase IQ
Real-Time Loading to Sybase IQ
 
MT129 Isilon Data Lake Overview
MT129 Isilon Data Lake OverviewMT129 Isilon Data Lake Overview
MT129 Isilon Data Lake Overview
 
Track 2, session 6 best practices to virtualize mission critical sap environm...
Track 2, session 6 best practices to virtualize mission critical sap environm...Track 2, session 6 best practices to virtualize mission critical sap environm...
Track 2, session 6 best practices to virtualize mission critical sap environm...
 
Hadoop Twelve Predictions for 2012
Hadoop Twelve Predictions for 2012Hadoop Twelve Predictions for 2012
Hadoop Twelve Predictions for 2012
 
Performance Management in ‘Big Data’ Applications
Performance Management in ‘Big Data’ ApplicationsPerformance Management in ‘Big Data’ Applications
Performance Management in ‘Big Data’ Applications
 
Faster, Cheaper, Easier... and Successful Best Practices for Big Data Integra...
Faster, Cheaper, Easier... and Successful Best Practices for Big Data Integra...Faster, Cheaper, Easier... and Successful Best Practices for Big Data Integra...
Faster, Cheaper, Easier... and Successful Best Practices for Big Data Integra...
 

Similaire à A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

DB2 Real-Time Analytics Meeting Wayne, PA 2015 - IDAA & DB2 Tools Update
DB2 Real-Time Analytics Meeting Wayne, PA 2015 - IDAA & DB2 Tools UpdateDB2 Real-Time Analytics Meeting Wayne, PA 2015 - IDAA & DB2 Tools Update
DB2 Real-Time Analytics Meeting Wayne, PA 2015 - IDAA & DB2 Tools UpdateBaha Majid
 
Reliability and performance with ibm db2 analytics accelerator
Reliability and performance with ibm db2 analytics acceleratorReliability and performance with ibm db2 analytics accelerator
Reliability and performance with ibm db2 analytics acceleratorbupbechanhgmail
 
TDWI San Diego 2014: Wendy Lucas Describes how BLU Acceleration Delivers In-T...
TDWI San Diego 2014: Wendy Lucas Describes how BLU Acceleration Delivers In-T...TDWI San Diego 2014: Wendy Lucas Describes how BLU Acceleration Delivers In-T...
TDWI San Diego 2014: Wendy Lucas Describes how BLU Acceleration Delivers In-T...IBM Analytics
 
IMS01 IMS Keynote
IMS01   IMS KeynoteIMS01   IMS Keynote
IMS01 IMS KeynoteRobert Hain
 
Refactoring to Microservices
Refactoring to MicroservicesRefactoring to Microservices
Refactoring to MicroservicesJacinto Limjap
 
J1 - Keynote Data Platform - Rohan Kumar
J1 - Keynote Data Platform - Rohan KumarJ1 - Keynote Data Platform - Rohan Kumar
J1 - Keynote Data Platform - Rohan KumarMS Cloud Summit
 
Has Your Data Gone Rogue?
Has Your Data Gone Rogue?Has Your Data Gone Rogue?
Has Your Data Gone Rogue?Tony Pearson
 
PHP Apps on the Move - Migrating from In-House to Cloud
PHP Apps on the Move - Migrating from In-House to Cloud  PHP Apps on the Move - Migrating from In-House to Cloud
PHP Apps on the Move - Migrating from In-House to Cloud RightScale
 
Converged Everything, Converged Infrastructure delivering business value and ...
Converged Everything, Converged Infrastructure delivering business value and ...Converged Everything, Converged Infrastructure delivering business value and ...
Converged Everything, Converged Infrastructure delivering business value and ...NetAppUK
 
Bridging the Last Mile: Getting Data to the People Who Need It (APAC)
Bridging the Last Mile: Getting Data to the People Who Need It (APAC)Bridging the Last Mile: Getting Data to the People Who Need It (APAC)
Bridging the Last Mile: Getting Data to the People Who Need It (APAC)Denodo
 
Partner Event with ChunTai Tech Industry in Taipei - Oct 2017
Partner Event with ChunTai Tech Industry in Taipei - Oct 2017Partner Event with ChunTai Tech Industry in Taipei - Oct 2017
Partner Event with ChunTai Tech Industry in Taipei - Oct 2017Christoph Adler
 
Grizzard webinar final 082510
Grizzard webinar final 082510Grizzard webinar final 082510
Grizzard webinar final 082510Sean O'Connell
 
Simplifying Real-Time Architectures for IoT with Apache Kudu
Simplifying Real-Time Architectures for IoT with Apache KuduSimplifying Real-Time Architectures for IoT with Apache Kudu
Simplifying Real-Time Architectures for IoT with Apache KuduCloudera, Inc.
 
IBM Cloud Day January 2021 - A well architected data lake
IBM Cloud Day January 2021 - A well architected data lakeIBM Cloud Day January 2021 - A well architected data lake
IBM Cloud Day January 2021 - A well architected data lakeTorsten Steinbach
 
Data Virtualization Manager for z/OS
Data Virtualization Manager for z/OS Data Virtualization Manager for z/OS
Data Virtualization Manager for z/OS Gustav Lundström
 
How to Increase Performance in IBM Cognos
How to Increase Performance in IBM CognosHow to Increase Performance in IBM Cognos
How to Increase Performance in IBM CognosCresco International
 

Similaire à A Hybrid Technology Platform for Increasing the Speed of Operational Analytics (20)

DB2 Real-Time Analytics Meeting Wayne, PA 2015 - IDAA & DB2 Tools Update
DB2 Real-Time Analytics Meeting Wayne, PA 2015 - IDAA & DB2 Tools UpdateDB2 Real-Time Analytics Meeting Wayne, PA 2015 - IDAA & DB2 Tools Update
DB2 Real-Time Analytics Meeting Wayne, PA 2015 - IDAA & DB2 Tools Update
 
VendorReview_IBMDB2
VendorReview_IBMDB2VendorReview_IBMDB2
VendorReview_IBMDB2
 
Reliability and performance with ibm db2 analytics accelerator
Reliability and performance with ibm db2 analytics acceleratorReliability and performance with ibm db2 analytics accelerator
Reliability and performance with ibm db2 analytics accelerator
 
TDWI San Diego 2014: Wendy Lucas Describes how BLU Acceleration Delivers In-T...
TDWI San Diego 2014: Wendy Lucas Describes how BLU Acceleration Delivers In-T...TDWI San Diego 2014: Wendy Lucas Describes how BLU Acceleration Delivers In-T...
TDWI San Diego 2014: Wendy Lucas Describes how BLU Acceleration Delivers In-T...
 
IMS01 IMS Keynote
IMS01   IMS KeynoteIMS01   IMS Keynote
IMS01 IMS Keynote
 
Refactoring to Microservices
Refactoring to MicroservicesRefactoring to Microservices
Refactoring to Microservices
 
Ibm db2update2019 icp4 data
Ibm db2update2019   icp4 dataIbm db2update2019   icp4 data
Ibm db2update2019 icp4 data
 
J1 - Keynote Data Platform - Rohan Kumar
J1 - Keynote Data Platform - Rohan KumarJ1 - Keynote Data Platform - Rohan Kumar
J1 - Keynote Data Platform - Rohan Kumar
 
Has Your Data Gone Rogue?
Has Your Data Gone Rogue?Has Your Data Gone Rogue?
Has Your Data Gone Rogue?
 
PHP Apps on the Move - Migrating from In-House to Cloud
PHP Apps on the Move - Migrating from In-House to Cloud  PHP Apps on the Move - Migrating from In-House to Cloud
PHP Apps on the Move - Migrating from In-House to Cloud
 
Converged Everything, Converged Infrastructure delivering business value and ...
Converged Everything, Converged Infrastructure delivering business value and ...Converged Everything, Converged Infrastructure delivering business value and ...
Converged Everything, Converged Infrastructure delivering business value and ...
 
Tech Tuesdays Session 1
Tech Tuesdays Session 1Tech Tuesdays Session 1
Tech Tuesdays Session 1
 
Bridging the Last Mile: Getting Data to the People Who Need It (APAC)
Bridging the Last Mile: Getting Data to the People Who Need It (APAC)Bridging the Last Mile: Getting Data to the People Who Need It (APAC)
Bridging the Last Mile: Getting Data to the People Who Need It (APAC)
 
Partner Event with ChunTai Tech Industry in Taipei - Oct 2017
Partner Event with ChunTai Tech Industry in Taipei - Oct 2017Partner Event with ChunTai Tech Industry in Taipei - Oct 2017
Partner Event with ChunTai Tech Industry in Taipei - Oct 2017
 
Grizzard webinar final 082510
Grizzard webinar final 082510Grizzard webinar final 082510
Grizzard webinar final 082510
 
Neelesh it assignment
Neelesh it assignmentNeelesh it assignment
Neelesh it assignment
 
Simplifying Real-Time Architectures for IoT with Apache Kudu
Simplifying Real-Time Architectures for IoT with Apache KuduSimplifying Real-Time Architectures for IoT with Apache Kudu
Simplifying Real-Time Architectures for IoT with Apache Kudu
 
IBM Cloud Day January 2021 - A well architected data lake
IBM Cloud Day January 2021 - A well architected data lakeIBM Cloud Day January 2021 - A well architected data lake
IBM Cloud Day January 2021 - A well architected data lake
 
Data Virtualization Manager for z/OS
Data Virtualization Manager for z/OS Data Virtualization Manager for z/OS
Data Virtualization Manager for z/OS
 
How to Increase Performance in IBM Cognos
How to Increase Performance in IBM CognosHow to Increase Performance in IBM Cognos
How to Increase Performance in IBM Cognos
 

Plus de IBMGovernmentCA

Cge leadership summit ibm presentation public sector analytics
Cge leadership summit   ibm presentation public sector analyticsCge leadership summit   ibm presentation public sector analytics
Cge leadership summit ibm presentation public sector analyticsIBMGovernmentCA
 
Security Trends and Risk Mitigation for the Public Sector
Security Trends and Risk Mitigation for the Public SectorSecurity Trends and Risk Mitigation for the Public Sector
Security Trends and Risk Mitigation for the Public SectorIBMGovernmentCA
 
Investigating, Mitigating and Preventing Cyber Attacks with Security Analytics
Investigating, Mitigating and Preventing Cyber Attacks with Security AnalyticsInvestigating, Mitigating and Preventing Cyber Attacks with Security Analytics
Investigating, Mitigating and Preventing Cyber Attacks with Security AnalyticsIBMGovernmentCA
 
CEO Study Insights; Career Resiliency In Time of Change
CEO Study Insights; Career Resiliency In Time of ChangeCEO Study Insights; Career Resiliency In Time of Change
CEO Study Insights; Career Resiliency In Time of ChangeIBMGovernmentCA
 
Overview of IBM Capabilities
Overview of IBM CapabilitiesOverview of IBM Capabilities
Overview of IBM CapabilitiesIBMGovernmentCA
 
Business Process Management
Business Process ManagementBusiness Process Management
Business Process ManagementIBMGovernmentCA
 
Information Governance for Smarter Government Strategy and Solutions
Information Governance for Smarter Government Strategy and SolutionsInformation Governance for Smarter Government Strategy and Solutions
Information Governance for Smarter Government Strategy and SolutionsIBMGovernmentCA
 
Smarter Computing Integrated Systems
Smarter Computing Integrated SystemsSmarter Computing Integrated Systems
Smarter Computing Integrated SystemsIBMGovernmentCA
 
Smarter Software for Smarter Governments
Smarter Software for Smarter GovernmentsSmarter Software for Smarter Governments
Smarter Software for Smarter GovernmentsIBMGovernmentCA
 
Perspectives and Case Studies on Effective Theatre Base Service Management
Perspectives and Case Studies on Effective Theatre Base Service ManagementPerspectives and Case Studies on Effective Theatre Base Service Management
Perspectives and Case Studies on Effective Theatre Base Service ManagementIBMGovernmentCA
 
Reducing IT Costs and Improving Security with Purpose Built Network Appliances
Reducing IT Costs and Improving Security with Purpose Built Network AppliancesReducing IT Costs and Improving Security with Purpose Built Network Appliances
Reducing IT Costs and Improving Security with Purpose Built Network AppliancesIBMGovernmentCA
 
Improving Defence Program Execution
Improving Defence Program ExecutionImproving Defence Program Execution
Improving Defence Program ExecutionIBMGovernmentCA
 
Social Networks the Next Emerging Spectrum in Asymmetric Warfare and Counter ...
Social Networks the Next Emerging Spectrum in Asymmetric Warfare and Counter ...Social Networks the Next Emerging Spectrum in Asymmetric Warfare and Counter ...
Social Networks the Next Emerging Spectrum in Asymmetric Warfare and Counter ...IBMGovernmentCA
 
Defense Intelligence & The Information Challenge
Defense Intelligence & The Information ChallengeDefense Intelligence & The Information Challenge
Defense Intelligence & The Information ChallengeIBMGovernmentCA
 
Analytics for Smarter Defence
Analytics for Smarter DefenceAnalytics for Smarter Defence
Analytics for Smarter DefenceIBMGovernmentCA
 
Keynote phaedra boinodiris serious games beyond training from process optim...
Keynote phaedra boinodiris   serious games beyond training from process optim...Keynote phaedra boinodiris   serious games beyond training from process optim...
Keynote phaedra boinodiris serious games beyond training from process optim...IBMGovernmentCA
 

Plus de IBMGovernmentCA (20)

Cge leadership summit ibm presentation public sector analytics
Cge leadership summit   ibm presentation public sector analyticsCge leadership summit   ibm presentation public sector analytics
Cge leadership summit ibm presentation public sector analytics
 
Security Trends and Risk Mitigation for the Public Sector
Security Trends and Risk Mitigation for the Public SectorSecurity Trends and Risk Mitigation for the Public Sector
Security Trends and Risk Mitigation for the Public Sector
 
Investigating, Mitigating and Preventing Cyber Attacks with Security Analytics
Investigating, Mitigating and Preventing Cyber Attacks with Security AnalyticsInvestigating, Mitigating and Preventing Cyber Attacks with Security Analytics
Investigating, Mitigating and Preventing Cyber Attacks with Security Analytics
 
Security Intelligence
Security IntelligenceSecurity Intelligence
Security Intelligence
 
Reputational Risk
Reputational RiskReputational Risk
Reputational Risk
 
CEO Study Insights; Career Resiliency In Time of Change
CEO Study Insights; Career Resiliency In Time of ChangeCEO Study Insights; Career Resiliency In Time of Change
CEO Study Insights; Career Resiliency In Time of Change
 
Overview of IBM Capabilities
Overview of IBM CapabilitiesOverview of IBM Capabilities
Overview of IBM Capabilities
 
Business Process Management
Business Process ManagementBusiness Process Management
Business Process Management
 
Information Governance for Smarter Government Strategy and Solutions
Information Governance for Smarter Government Strategy and SolutionsInformation Governance for Smarter Government Strategy and Solutions
Information Governance for Smarter Government Strategy and Solutions
 
Smarter Computing Integrated Systems
Smarter Computing Integrated SystemsSmarter Computing Integrated Systems
Smarter Computing Integrated Systems
 
Smarter Software for Smarter Governments
Smarter Software for Smarter GovernmentsSmarter Software for Smarter Governments
Smarter Software for Smarter Governments
 
Perspectives and Case Studies on Effective Theatre Base Service Management
Perspectives and Case Studies on Effective Theatre Base Service ManagementPerspectives and Case Studies on Effective Theatre Base Service Management
Perspectives and Case Studies on Effective Theatre Base Service Management
 
Reducing IT Costs and Improving Security with Purpose Built Network Appliances
Reducing IT Costs and Improving Security with Purpose Built Network AppliancesReducing IT Costs and Improving Security with Purpose Built Network Appliances
Reducing IT Costs and Improving Security with Purpose Built Network Appliances
 
Improving Defence Program Execution
Improving Defence Program ExecutionImproving Defence Program Execution
Improving Defence Program Execution
 
Social Networks the Next Emerging Spectrum in Asymmetric Warfare and Counter ...
Social Networks the Next Emerging Spectrum in Asymmetric Warfare and Counter ...Social Networks the Next Emerging Spectrum in Asymmetric Warfare and Counter ...
Social Networks the Next Emerging Spectrum in Asymmetric Warfare and Counter ...
 
Defense Intelligence & The Information Challenge
Defense Intelligence & The Information ChallengeDefense Intelligence & The Information Challenge
Defense Intelligence & The Information Challenge
 
Analytics for Smarter Defence
Analytics for Smarter DefenceAnalytics for Smarter Defence
Analytics for Smarter Defence
 
Keynote phaedra boinodiris serious games beyond training from process optim...
Keynote phaedra boinodiris   serious games beyond training from process optim...Keynote phaedra boinodiris   serious games beyond training from process optim...
Keynote phaedra boinodiris serious games beyond training from process optim...
 
Where Ideas Come From
Where Ideas Come FromWhere Ideas Come From
Where Ideas Come From
 
What Are The Chances
What Are The ChancesWhat Are The Chances
What Are The Chances
 

Dernier

Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 

Dernier (20)

Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 

A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

  • 1. Ed Lynch – Executive Client Technical Professional Data Warehousing and Business Analytics for System z 10 October 2012 A Hybrid Technology Platform for Increasing the Speed of Operational Analytics © 2009 IBM Corporation
  • 2. Speaker Biography Ed Lynch is an Executive Client Technical Professional specializing in IBM’s System z Data Warehousing, Business Analytics, and Information Integration software products. Ed’s twenty- eight year career with IBM has spanned many areas of IBM, and has always involved IBM's Information Management (IM) products. His previous roles have included DB2 for z/OS Development and Management, DB2 Technical Marketing, Development and Delivery of IM Product Education, a Principal of Information Integration Design and Implementation Consulting Services, and DB2 Tools & Information Integration Technical Sales. Currently, Ed is the lead Technical Specialist for North America’s System z Data Warehousing and Business Analytics, and Information Integration Software Technical Sales team. Ed has worked extensively with DB2 across the various operating system platforms, InfoSphere Data Replication, InfoSphere Classic Replication Server, DB2 Analytics Accelerator, DB2 DataPropagator, InfoSphere Federation Server, InfoSphere Classic Federation Server, DB2 Connect, IBM Information Server, and IBM’s Business Analytics solutions. In his current role, he frequently works with product development in identifying and prioritizing product requirements, and developing product strategy. He also provides software technical sales support and works extensively with customers to create architectures using these products. edlynch@us.ibm.com 1-972-561-9975 © 2009 IBM Corporation
  • 3. Abstract With the wealth of data available today, organizations are no longer willing to relegate information to the back office. Modern organizations are demanding access to information. However, it is not enough to capture information, users must be quickly able to sift through these massive amounts of data, extract information and transform it into actionable knowledge. Systems today are enabling organizations to anticipate risk, identify threats, assess readiness, and match the risk assessment to the resources required to address them; all at the time of decision. They use a platform that provides the ability to react to changes decisively, based upon the facts of the situation, not in hours or days- but at the moment of opportunity. They optimize decisions based upon current weather conditions, past threats and behaviors and current resource availability to assure a successful operation.This session will review the architecture and benefits of a hybrid system of MPP and SMP technologies enabling the merging of fit for purpose and mixed workload capabilities into a single system. See how this hybrid system facilitates both transaction- oriented applications and analytics into a single platform for operational analytics. Find out why these enhancements are the next logical steps in creating a highly optimized environment, both in price and performance, that is designed to meet the wide range of analytic workloads that today's organizations need to accommodate. © 2009 IBM Corporation
  • 4. DB2 Analytics Accelerator V3 More insight from your data Further extending the features • Unprecedented response times for “right-time” analysis • Complex queries in seconds rather than hours • Transparent to the application • Inherits all System z DB2 attributes • No need to create or maintain indices • Eliminate query tuning Blending System z and Netezza • Fast deployment and time-to-value technologies to deliver unparalleled, mixed workload performance for complex analytic business needs. 4 © 2009 IBM Corporation
  • 5. DB2 Analytics Accelerator Train-of-thought Analytics FAST Cost Saving Appliance Complex queries run Eliminate costly No applications to up to 2000x faster query tuning while change, just plug it while retaining single offloading complex in, load the data, record lookup speed query processing and gain the value 5 © 2009 IBM Corporation
  • 6. Introducing DB2 Analytics Accelerator V3 Reducing the Cost of High Speed Analytics Improve Productivity Lower Host Costs Consolidate Eliminate query tuning Reduce storage costs Reduced complexity Eliminate table indexing Offload query processing Reduced software costs Minimize storage admin Defer system upgrades Reduced hardware costs 6 © 2009 IBM Corporation
  • 7. Fast Time to Value IBM DB2 Analytics Accelerator Production ready - 1 person, 2 days Table Acceleration Setup 2 Hours – DB2 “Add Accelerator” – Choose a Table for “Acceleration” – Load the Table (DB2 copy to Netezza) – Knowledge Transfer – Query Comparisons Initial Load Performance 400 GB “Loaded” in 29 Min 570 million rows (Loads of 800GB to 1.3TB/Hr) Actual Query Acceleration 1908x faster 2 Hours 39 Minutes to 5 Seconds CPU Utilization Reduction 35% to ~0% Actual customer results, October 2011 © 2009 IBM Corporation
  • 8. Performance & Savings DB2 w ith Times Faster Queries run faster DB2 O nly IDAA Total Total • Save CPU resources Row s Rows Query Reviewed Returned Hours Sec(s) Hours Sec(s) • People time Query 1 2,813,571 853,320 2:39 9,540 0.0 5 1,908 Query 2 2,813,571 585,780 2:16 8,220 0.0 5 1,644 • New Business Query 3 8,260,214 274 1:16 4,560 0.0 6 760 opportunities Query 4 2,813,571 601,197 1:08 4,080 0.0 5 816 Query 5 3,422,765 508 0:57 4,080 0.0 70 58 Query 6 4,290,648 165 0:53 3,180 0.0 6 530 Query 7 361,521 58,236 0:51 3,120 0.0 4 780 Query 8 3,425.29 724 0:44 2,640 0.0 2 1,320 Query 9 4,130,107 137 0:42 2,520 0.1 193 13 DB2 Analytics Accelerator: “we had this up and running in days with queries that ran over 1000 times faster” DB2 Analytics Accelerator: “we expect ROI in less than 4 months” Advance to 32 minute mark for DB2 Analytics Accelerator section of keynote Accelerating decisions to the speed of business 8 12 October 2012 Actual customer results, October 2011 © 2009 IBM Corporation
  • 9. IBM DB2 Analytics Accelerator V3 Product Components Netezza zEnterprise Technology CLIENT Data Studio Foundation DB2 Analytics Accelerator Network Admin Plug-in OSA- Primary Express3 10Gb 10 GbE Backup IBM DB2 Data Warehouse application Analytics DB2 for z/OS enabled for IBM Users/ Acelerator DB2 Analytics Accelerator Applications 9 © 2009 IBM Corporation
  • 10. Deep DB2 Integration within zEnterprise Applications DBA Tools, z/OS Console, ... Application Interfaces Operational Interfaces (standard SQL dialects) (e.g. DB2 Commands) DB2 for z/OS IBM Data Buffer Log DB2 Manager Manager ... IRLM Manager Analytics Accelerator Superior availability Superior reliability, security, z/OS on performance on Workload management System z analytic queries Netezza 10 © 2009 IBM Corporation
  • 11. TM Accelerator powered by Netezza 1000 Appliance Slice of User Data Swap and Mirror partitions High speed data streaming High compression rate EXP3000 JBOD Enclosures 12 x 3.5” 1TB, 7200RPM, SAS (3Gb/s) Disk Enclosures max 116MB/s (200-500MB/s compressed data) e.g. 1000-12: 8 enclosures → 96 HDDs(32/128 TB) Accelerator Server SMP Hosts SQL Compiler, Query Plan, Optimize, Administration 2 front/end hosts, IBM 3650M3 or 3850X5 clustered active-passive 2 Nehalem-EP Quad-core 2.4GHz per host Snippet BladesTM (S-Blades, SPUs) Processor & streaming DB logic High-performance database engine streaming joins, aggregations, sorts, etc. © 2009 IBM Corporation
  • 12. S-Blade™ Components Dual-Core FPGA 8 FPGA Processors/Blade Netezza DB Accelerator Intel Quad-Core 8 Cores/Blade IBM BladeCenter Server © 2009 IBM Corporation
  • 13. Eliminating the I/O Bottleneck Move the SQL to the hardware to where the data lives “Just send the Answer, not Raw Data” © 2009 IBM Corporation
  • 14. select DISTRICT,PRODUCTGRP, The Key to the Speed sum(NRX) from MTHLY_RX_TERR_DATA where MONTH = '20091201' and MARKET = 509123 and SPECIALTY = 'GASTRO' FPGA CPU Core Core Zone Map Complex ∑ Restrict, Project Joins, Aggs, etc. Uncompress Visibility Slice of table MTHLY_RX_TERR_DATA (compressed) sum(NRX) where MONTH = '20091201' select DISTRICT, and MARKET = 509123 PRODUCTGRP, and SPECIALTY = 'GASTRO' sum(NRX) © 2009 IBM Corporation
  • 15. Bringing Netezza AMPPTM Architecture to DB2 for z/OS AMPP = Asymmetric Massively Parallel Processing CPU FPGA Advanced Memory Analytics BI SMP CPU FPGA Host DB2 for z/OS Memory Legacy Reporting CPU FPGA DBA Memory Network Disk Fabric S-Blades™ Enclosures IBM DB2 Analytics Accelerator © 2009 IBM Corporation
  • 16. Query Execution Process Flow Application Optimizer Interface SPU CPU FPGA Memory Accelerator DRDA Requestor SPU CPU FPGA SMP Host Memory Application SPU Query execution run-time for CPU FPGA queries that cannot be or should Memory not be off-loaded to Accelerator SPU CPU FPGA Memory DB2 for z/OS DB2 Analytics Accelerator Queries executed without DB2 Analytics Accelerator Queries executed with DB2 Analytics Accelerator © 2009 IBM Corporation
  • 17. Workload-Optimized Query Execution • Single and unique system DB2 for z/OS and for mixed query workloads IBM DB2 Analytics Accelerator • Dynamic decision for most OLTP-like query OLTP-like query efficient execution platform User control and DB2 heuristic • New special register DB2 Native QUERY ACCELERATION DB2 Native Light ODS- Light ODS- Processing Processing – NONE query query – ENABLE – ENABLE WITH FAILBACK • New heuristic in DB2 Light BI Query Light BI Query optimizer Heavy BI Query Heavy BI Query Optimized processing for BI Workload © 2009 IBM Corporation
  • 18. Accelerator Data Load DB2 for z/OS Accelerator Table A Table B CPU FPGA Part 1 Unload USS Pipe Accelerator Administrative Stored Memory Table C Accelerator CPU FPGA Procedures Studio Table D Part 2 Unload USS Pipe Coordinator Memory Part 1 . . . CPU FPGA . . . Memory Part 2 . . . CPU FPGA Part 3 Unload USS Pipe Part m Memory • 1 TB / h – can vary, depending on CPU resources, table partitioning, • Update on table partition level, concurrent queries allowed during load • V2.1 & V3 unload in DB2 internal format, single translation by accelerator © 2009 IBM Corporation
  • 19. DB2 Analytics Accelerator V3 Lowering the Costs of Trusted Analytics What’s New? • zEnterprise EC12 Support • High Performance Storage Version 3 will support the zEnterprise Saver EC12, z196 and z114 System z platforms Store a DB2 table or partition of data solely on the Accelerator. Removes • Query Prioritization the requirement for the data to be Brings System z workload replicated on both DB2 and the management down to the individual Accelerator query being routed to the Accelerator • Incremental Update • High Capacity Enables tables within the Accelerator Support has been extended to include to be continually updated throughout the entire Netezza 1000 line (1.28 PB) the day. • UNLOAD Lite Reduces z/OS MIPS consumption, by moving the preparation off System z. 19 © 2009 IBM Corporation
  • 20. Build a System z Trusted Analytic System Reduce the cost of host storage for historical data by 95%! Historical High Performance Low Latency Data Most data in an analytic All aggregate queries run Tables and partitions that system is historical and not at the same high speed as require updating will be subject to change. Most any accelerator supported able to be updated by data can be in a Storage query incremental update, table Saver and maintain trusted load or partition load performance and security © 2009 IBM Corporation
  • 21. High Performance Storage Saver Reducing the cost of high speed storage Store historic data on the Accelerator only Applications Tables can be resident on: 1. DB2 Only 2. DB2 and Accelerator 3. Accelerator Only SQL When data no longer requires updating, reclaim DB2 DB2 the DB2 storage Accelerator Table A Table A Table A Special Registers control behavior High speed High speed CURRENT QUERY ACCELERATION indexed aggregate lookups, best Accelerator lookups, best for CURRENT GET_ACCEL_ARCHIVE for OLTP Table A complex DSS type type queries queries Managed by zParms 21 Mixed workload type queries © 2009 IBM Corporation
  • 22. Save Over 95% of Host Disk Space for Historical Data Historical Data Year Year -1 Year -2 Year -3 Year -4 Year -5 Year -7 1Q 1Q 1Q 1Q 1Q 1Q 1Q 2Q 2Q 2Q 2Q 2Q 2Q 2Q 3Q 3Q 3Q 3Q 3Q 3Q 3Q 4Q 4Q 4Q 4Q 4Q 4Q Current Data 4Q One Quarter = 3.57% of 7 years of data One Month = 1.12% of 7 years of data One month = 2.78% of 3 years of data © 2009 IBM Corporation
  • 23. High Performance Storage Saver Reducing the cost of high speed storage Time-partitioned tables where: – only the recent partitions are used in a transactional context (frequent data changes, short running queries) – the entire table is used for analytics (data intensive, complex queries). DB2 partitions are deleted after the High Performance Storage Saver are created on the accelerator DB2 No longer present on DB2 Storage Query from Application #1 Or Accelerator Accelerator Accelerator Accelerator Accelerator Accelerator Accelerator #1 #2 #3 #4 #5 #6 #7 23 © 2009 IBM Corporation
  • 24. The Evolution of a High Performance Storage Saver High Speed Access to Historical Data Table / Data Accelerator Accelerator Archive Creation Load / Update /IU Only Only DB2 DB2 Accelerator Table A Table A Table A Accelerator Table A Backup Backup 24 © 2009 IBM Corporation
  • 25. Storage options to match data needs Optimized in both price and performance for differing workloads High Performance Storage Saver Database Resident Partitions Single Disk Store Dual Disk Store • Only stored on Accelerator storage (Less • Stored on both DB2 and Accelerator Cost) storage • Optimized performance for • Mixed query workload with transactions, deep analytics, multifaceted, reporting single record queries and record updates and complex queries with deep analytics, multifaceted, • Only full table update or full partition reporting and complex queries. update from backup • Full table, full partition update, Incremental • Same high speed query access update from DB2 data transparently through DB2 • Same high speed query access transparently through DB2 Cost The right mix of cost and functionality Functionality © 2009 IBM Corporation 25
  • 26. The zEnterprise Hybrid Solution Mixed Workloads for Next Generation Business Analytics Operational Analytic Mixed Workload Applications Applications Applications Transaction Processing Data warehousing Operational BI Shared Everything DB Shared Nothing DB Hybrid DB High volume business Low volume complex High volume business transactions and batch queries context switching transactions and batch reporting running reporting running concurrently concurrently with complex queries 26 © 2009 IBM Corporation
  • 27. Incremental Update Table or ELT or ETL Partition Update OLTP Data DB2 Analytics Application Warehouse Accelerator Data Incremental Replication Update Synchronizing data to lower data latency from days to minutes/seconds 27 © 2009 IBM Corporation
  • 28. Option 1: Full Table Refresh Changes in data warehouse tables typically driven by scheduled (nightly or more frequently) ETL process Data used for complex reporting based on consistent and validated content (e.g., weekly Operational Analytics, Reports, OLAP, Operational Analytics, Reports, OLAP, transaction reporting to the central bank) Multiple sources or complex transformations Continuous Continuous Query prevent propagation of incremental changes Query Processing Processing Full table refresh triggered through DB2 stored procedure (scheduled, integrated into ETL DB2 z/OS Query Optimizer DB2 z/OS Query Optimizer process or through GUI) DB2 native DB2 native Accelerator Accelerator processing processing processing processing ETL Process Queries may continue ETL Process during full table refresh Full table refresh for accelerator DB2 for z/OS database DB2 for z/OS database Changes / Replacement © 2009 IBM Corporation
  • 29. Option 2: Table Partition Refresh Changes in data warehouse table typically driven by “delta” ETL process (considering only changes in source tables compared to previous runs) or by more frequent changes to most recent data Optimization of Option 1 when target data warehouse table is partitioned and most recent updates are only applied to the latest partition Operational Analytics, Reports, OLAP, Operational Analytics, Reports, OLAP, Table partition refresh triggered through DB2 stored procedure (scheduled, integrated into Continuous Continuous ETL process or through GUI) Query Query Processing Processing Maintains snapshot DB2 z/OS Query Optimizer DB2 z/OS Query Optimizer semantics for consistent reports Queries may continue DB2 native DB2 native Accelerator Accelerator Replication Replication processing processing processing processing during table partition refresh for accelerator January February March ETL Process ETL Process April May Partition refresh Changes DB2 for z/OS database © 2009 IBM Corporation DB2 for z/OS database
  • 30. Option 3: Incremental Update Changes in data warehouse tables typically driven by replication or manual updates – Corrections after a bulk-ETL-load of a data warehouse table – Continuously changing data (e.g. trickle-feed updates from a transactional system to an ODS) Reporting and analysis based on most recent Operational Analytics, Reports, OLAP, Operational Analytics, Reports, OLAP, data May be combined with Option 1 & 2 (first table Continuous Continuous refresh and then continue with incremental Query Query Processing Processing updates) DB2 z/OS Query Optimizer DB2 z/OS Query Optimizer Application Application DB2 native DB2 native Accelerator Accelerator Incremental update can be processing processing processing processing configured per database table Replication Replication Incremental Update Changes DB2 for z/OS database DB2 for z/OS database © 2009 IBM Corporation
  • 31. Now expandable to 960 cores and 1.28 petabytes 1 10 ....... 002 005 010 015 020 030 040 060 060 100 Cabinets 1/4 1/2 1 1 1/2 2 3 4 6 8 10 Processing Units 24 48 96 144 192 288 384 576 768 960 Capacity (TB) 8 16 32 48 64 96 128 192 256 320 Effective 32 64 128 192 256 384 512 768 1024 1280 Capacity (TB)* PureData System for Analytics Predictable, Linear Scalability throughout entire family Capacity = User Data space Effective Capacity = User Data Space with compression *: 4X compression assumed Low Latency, High Capacity Update © 2009 IBM Corporation
  • 32. Connectivity Options DB2 DB2 Multiple DB2 systems can connect to a single Accelerator A single DB2 system can connect to multiple Accelerators DB2 Multiple DB2 systems can connect to multiple Accelerators DB2 DB2 The same table can be stored in the multiple Accelerators (except High Performance Storage Saver tables) Full flexibility for DB2 systems: • residing in the same LPAR Better utilization of Accelerator resources • residing in different LPARs • residing in different CECs Scalability • being independent (non-data sharing) High availability • belonging to the same data sharing group • belonging to different data sharing groups © 2009 IBM Corporation 32
  • 33. Analytics Accelerator Table Definition and Deployment IBM Data Studio Client DB2 for z/OS DB2 Analytics Accelerator Accelerator Accelerator Administrative Netezza Catalog Studio Stored Procedures DB2 Catalog The tables need to be defined and deployed to the Accelerator before data is loaded and queries sent to it for processing. Definition: identifying tables for which queries need to be accelerated Deployment: making tables known to DB2, i.e. storing table meta data in the DB2 and Netezza catalog. IBM DB2 Analytics Accelerator Studio guides you through the process of defining and deploying tables, as well as invoking other administrative tasks. IBM DB2 Analytics Accelerator Stored Procedures implement and execute various administrative operations such as table deployment, load and update, and serve as the primary administrative interface to the Accelerator from the outside world including Accelerator Studio. 33 © 2009 IBM Corporation
  • 34. Shielding Against Disk Failures Primary Mirror Temp • All user data and temp space mirrored • Disk failures transparent to queries and transactions • Failed drives automatically regenerated • Bad sectors automatically rewritten or relocated © 2009 IBM Corporation
  • 35. Shielding Against S-BladeTM Failures . . . . . . . . . . . . . . . S-Blades • S-Blade failure is automatically detected © 2009 IBM Corporation
  • 36. Shielding Against S-BladeTM Failures . . . . . . . . . . . . . . . S-Blades • Drives automatically reassigned to active S-Blades within a chassis • Read-only queries (that have not returned data yet) automatically restarted • Transactions and loads interrupted • Loads automatically restarted from last successful checkpoint © 2009 IBM Corporation
  • 37. Disaster Recovery Option 1 – Table Loaded in One Accelerator (1 of 2) SYSPLEX App 1 DSG Member 1 DSG Member 2 Tables Tables App 4 of App 4 of App 5 App 2 Tables Tables Tables App 5 App 3 of App 1 of App 2 of App 3 Short Range Short Range Long Switch Range Switch Short Range Short Range Accelerator Instance 1 Accelerator Instance 2 Tables Created but Not Loaded Tables Tables Tables of App 1 of App 2 of App 3 Tables Tables Tables Tables Tables of App 1 of App 2 of App 3 of App 4 of App 5 © 2009 IBM Corporation
  • 38. Disaster Recovery Option 1 – Table Loaded in One Accelerator (2 of 2) App 1 SYSPLEX App 1 DSG Member 1 DSG Member 2 App 2 Tables Tables of App 4 of App 5 App 2 App 3 Tables Tables Tables App 3 of App 1 of App 2 of App 3 App 4 Short Range Short Range Long App 5 Switch Range Switch Short Range Short Range Accelerator Instance 1 Accelerator Instance 2 Tables Tables Tables of App 1 of App 2 of App 3 Already Created Tables Tables Tables Must LOAD Tables Tables of App 1 of App 2 of App 3 of App 4 of App 5 © 2009 IBM Corporation
  • 39. Disaster Recovery Option 2– Table Loaded in Two Accelerators (1 of 2) SYSPLEX App 1 DSG Member 1 DSG Member 2 Tables Tables App 4 of App 4 of App 5 App 2 Tables Tables Tables App 5 App 3 of App 1 of App 2 of App 3 Short Range Short Range Long Switch Range Switch Short Range Short Range Accelerator Instance 1 Accelerator Instance 2 Tables Tables Tables Tables Tables Tables of App 1 of App 2 of App 3 of App 1 of App 2 of App 3 Tables Tables Tables Tables of App 4 of App 5 of App 4 of App 5 © 2009 IBM Corporation
  • 40. Disaster Recovery Option 2 – Table Loaded in Two Accelerators (2 of 2) App 1 SYSPLEX App 1 DSG Member 1 DSG Member 2 App 2 Tables Tables of App 4 of App 5 App 2 App 3 Tables Tables Tables App 3 of App 1 of App 2 of App 3 App 4 Short Range Short Range Long App 5 Switch Range Switch Short Range Short Range Accelerator Instance 1 Accelerator Instance 2 Data Already Available Tables Tables Tables Tables Tables Tables of App 1 of App 2 of App 3 of App 1 of App 2 of App 3 Tables Tables Tables Tables of App 4 of App 5 of App 4 of App 5 © 2009 IBM Corporation
  • 41. Why Both? Marrying the best of both worlds IBM IBM PureData N1001 System z Focused Appliance Mixed Workload System Capitalizing on the strengths of both platforms while driving to the most cost effective, centralized solution - destroying the myth that transaction and decision systems had to be on separate platforms Very focused workload Very diverse workload © 2009 IBM Corporation
  • 42. Tailored to your needs A Hybrid Solution IBM IBM System z with Netezza IBM DB2 Analytics Accelerator Focused Appliance Mixed Workload System • Mixed workload system z with operational • Appliance with a streamlined transaction systems, data warehouse, database and HW acceleration for operational data store, and consolidated performance critical functionality data marts. • Price/performance leader • Unmatched availability, security and recoverability • Speed and ease of deployment and • Natural extension to System z to enable administration pervasive analytics across the • Optimized performance for organization. deep analytics, multifaceted, reporting • Speed and ease of deployment and and complex queries administration True Appliance Flexible Integrated System Custom Solution Simplicity The right mix of simplicity and flexibility Flexibility © 2009 IBM Corporation
  • 43. Next Steps Opportunity Operational New Workload Data Reporting Consolidation Operational BI (Accelerator) or Winback (ISAS + (ISAS + (ISAS + OR Accelerator) Accelerator) Accelerator) Existing z Warehouse Warehouse Workload Collaboration Assessment Workshop Optional POC © 2009 IBM Corporation 43
  • 44. The Ultimate Consolidation Platform Data Mart Data Mart Data Mart Data Mart Bringing it all together • Better Business Response Data Mart Consolidation • Reduced Costs System z PR/SM Recognized leader in mixed • More Available virtualization and workload isolation • More Secure Transaction Systems • Reduced Data Movement (OLTP) • Better Governance • Reduced Data Latency • Reduced Complexity Data Warehousing • Reduced Resources z/OS: Netezza: Business Intelligence Recognized leader in mixed Recognized leader in Predictive Analytics workloads with security, cost-effective high availability speed deep analytics and recoverability Together: Destroying the myth that transactional and decision support workloads have to be on separate platforms 44 © 2009 IBM Corporation
  • 45. Learn More Visit the Data Warehousing & Business Analytics Webpage http://www.ibm.com/software/data/businessintelligence/systemz/ © 2009 IBM Corporation
  • 46. Ed Lynch System z Data Warehousing & Business Analytics edlynch@us.ibm.com 46 © 2009 IBM Corporation