Oracle GoldenGate Performance Tuning
•Download as PPTX, PDF•
3 likes•3,074 views
Oracle GoldenGate Performance Tuning presentation given at IOUG Collaborte 18.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Oracle GoldenGate Performance Tuning
Similar to Oracle GoldenGate Performance Tuning (20)
More from Bobby Curtis
More from Bobby Curtis (20)
Recently uploaded
Recently uploaded (20)
Oracle GoldenGate Performance Tuning
- 2. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted Extreme Replication: Performance Tuning Oracle GoldenGate for the Real World Bobby Curtis Director of Product Management Oracle, GoldenGate April 2018
- 3. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Confidential – Oracle Internal/Restricted/Highly Restricted 3
- 4. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Agenda Overview and Objectives Architecture and Functionality Changes Tuning Areas Configuration Items Questions 1 2 3 4 5 Confidential – Oracle Internal/Restricted/Highly Restricted 4
- 5. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Oracle GoldenGate Real time Performance Extensible & Flexible Proven & Reliable Oracle GoldenGate provides low-impact capture, routing, transformation, and delivery of database transactions across homogeneous and heterogeneous environments in real-time with no distance limitations. Most Databases Data Events Transaction Streams Cloud DBs Big Data Supports Databases, Big Data and NoSQL: * The most popular enterprise integration tool in history
- 6. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Use cases Oracle GoldenGate New DB/ HW/OS/APP Fully Active Distributed DB Application Offloading Big Data, DW & Marts Message Bus & Data Grid Global Data Centers Data Streaming GoldenGate Oracle Confidential – Internal/Restricted/Highly Restricted1/05/2018 Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Databases Applications Big Data and No SQL Mainframes Mobile Streaming & Messaging Capture Deliver Distribute Receive Secure Oracle SQL Server MySQL IBM DB2 Z IBM DB2 i IBM DB2 LUW HP NonStop Informix Sybase Messaging High Availability Zero Downtime Migration Data Synchronization Big Data Ingestion Query/Report Offloading Stream Analytics Data Bus (Lambda/Kappa)
- 7. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 7 Oracle GoldenGate Modular Architecture Capture (extract): committed transactions are captured (and can be filtered) as they occur by reading the transaction logs. As of V.11.2.1, GoldenGate offers two options for capture for Oracle; Classic & Integrated Capture Trail: stages and queues data for routing. Pump: distributes data for routing to target(s). Route: data is compressed, encrypted for routing to target(s). Delivery: applies data with transaction integrity. New with GoldenGate 12c, Integrated Delivery & Non- Integrated/Integrated Prallel Delivery. 20 YRS
- 8. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Objectives for 12.3 Confidential – Oracle Internal/Restricted/Highly Restricted 8 Useability Manageability Performance
- 9. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Agenda Overview and Objectives Architecture and Functionality Changes Tuning Areas Configuration Items Questions 1 2 3 4 5 Confidential – Oracle Internal/Restricted/Highly Restricted 9
- 10. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Microservices Architecture Confidential – Oracle Internal/Restricted/Highly Restricted 10 New way to administer GoldenGate Capture Trail Files Trail Files DeliveryDistribution Service Network Receiver Service Administration Service Metrics Service >HTML>>SQL >>CLI >>API Customizable Interactive Experience Proxy/Reverse Proxy ServiceManager
- 11. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Performance Metric Service Confidential – Oracle Internal/Restricted/Highly Restricted 11 Built In Performance Monitor Single Host View/Deployment Repository NoSQL Berkley Database (BDB) Lightweight Memory Database (LMDB) Requires 2 ports Accessed through HTML5 or REST API* Enterprise View - OEM
- 12. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Parallel Replicat Confidential – Oracle Internal/Restricted/Highly Restricted 12 Built for speed Mapper . . . Mapper . . . Mapper Dependency Calculation & Scheduling Parallel Replicat Database Applier . . . Applier . . . Applier . . . Applier Trail Files Trail Files single Mapper Oracle Database Integrated Replicat Dependency Calculation & Scheduling Applier Applier Applier Parallel Mapping Dependency Calculation and Scheduling outside the database Parallel Appliers Multiple Pipes into the database Option to Offload all Processes to middle-tier
- 13. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Agenda Overview and Objectives Architecture and Functionality Changes Tuning Areas Configuration Items Questions 1 2 3 4 5 Confidential – Oracle Internal/Restricted/Highly Restricted 13
- 14. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Tuning Approach Confidential – Oracle Internal/Restricted/Highly Restricted 14 Host Host Database Database GoldenGate GoldenGate - MPSTAT, VMSTAT, IOSTAT, STRACE, TOP - AWR, ASH, UTL_SPADV, TRACE GG tools: REPORTCOUNT, LAG, Streams Adv., STATS Waterfall Method Source Target Note: LAG in Microservices is found under the reports REST API.
- 15. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | • Parameters currently running • Table/Column Mappings • Runtime messages and errors • Runtime statistics • REPORTCOUNT - REPORTCOUNT EVERY [ # ] [ SECONDS | MINUTES | HOURS ], RATE Example: REPORTCOUNT EVERY 15 MINUTES, RATE • Files located in $DEPLOYMENT_HOME/var/lib/report GoldenGate Performance Tools Reports
- 16. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Automatic Workload Repository (AWR) • Good starting point within the database • Help determine potiential bottlenecks • Extract/Replicat are given unique SQL Module IDs - Used in both AWR and ASH • Use Active Session History (ASH) for more details on identified session Database Performance Tools
- 17. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Note: Will not be in 11.2.x AWR reports AWR Output - Replication Stats
- 18. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | • Used to provide information on the performance of integrated processes • Both Extract and Replicat • Provides real-time statistics • Has to be installed - Installed under GoldenGate User - $ORACLE_HOME/rdbms/admin/utlspadv.sql Database Performance Tools Streams Performance Advisor (Integrated Extract/Replicat Only)
- 19. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Run Time Views • V$GOLDENGATE_CAPTURE • V$GG_APPLY_RECEIVER • V$GG_APPLY_READER • V$GG_APPLY_COORDINATOR • V$GG_APPLY_SERVER • V$GOLDENGATE_TABLE_STATS • V$GOLDENGATE_CAPABILITIES • V$DBA_APPLY_ERRORS Configuration Views • DBA_GOLDENGATE_PRIVILEGES • DBA_GOLDENGATE_SUPPORT_MODE • DBA_CAPUTRE • DBA_CAPTURE_PARAMETERS • DBA_GOLDENGATE_INBOUND • DBA_GG_INBOUND_PROGRESS • DBA_APPLY • DBA_APPLY_PARAMETERS • DBA_APPLY_REPERROR_HANDLERS • DBA_APPLY_HANDLE_COLLISIONS • DBA_APPLY_DML_CONF_HANDLERS Associated GoldenGate Monitoring Scripts GoldenGate Views Integrated Processes
- 20. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | • SQL based scripts that create HTML output - Different scripts for database releases - MOS:1448324.1 - Microservices HC Scripts: $OGG_HOME/healthcheck • Summary - Overview of environment - Advice/Warnings of potential issues of configuration • Analysis - Compare configurations - Performance Recommendations - Detail information for diagnostic purposes • Statistics - Runtime information of Streams (GoldenGate) processing Health Checks
- 21. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | CPU Performance (mpstat/top) • Looking to see if there are any processes limited by CPU (top) • Looking to see what average cpu spike is (mpstat) I/O Performance (iostat) • Gather for database disks and trail file locations • Critical before increasing parallelism Memory Performance (vmstat) • Not needed for integrated processes • Needed if tuning classic or coordinated processes Host Performance Tools
- 22. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Agenda Overview and Objectives Architecture and Functionality Changes Tuning Areas Configuration Items Questions 1 2 3 4 5 Confidential – Oracle Internal/Restricted/Highly Restricted 23
- 23. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Database Object Tuning • GoldeGate uses SQL to apply changes • AWR highlights SQL and SEGMENT issues • ADDM recommendations as needed • Redo Logs, ensure they are large enough for expected workload • Gather statistics regularly, especially after initial initialization • Target side: Indexes can cause slowness in apply process as well.
- 24. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Database Configurations Parameter Description ENABLE_GOLDENGATE_REPLICATION • New in DB 11.2.0.4/12.1.0.2 • Boolean: Default to False, set to True to activate STREAMS_POOL_SIZE • Size appropriately for number of concurrent IE & IR processes • Min. 1G per IE/IR then add 25%
- 25. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Integrated Extract Parameters Parameter Description LogAllSupCols • Instructs extract to write supplemental logged columns to trail file UpdateRecordFormat • Single Logical Change Record (LCR) with BEFORE and AFTER images • COMPACT - reduces the amount of data sent with LCR Parallelism • Controls number of preparers for processing logs • Increase parallelism if preparers are CPU bound • EE allows parallelism of preparers (default = 2/ Max=4) • SE does not allow parallelism of preparers (default = 1/ Max=2) Max_SGA_Size • Controls amount of streams pool configured per extract • Min: 1024 (1G)/Max: 3584 (3.5G)
- 26. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Integrated Replicat Parameters Parameter Description Parallelism • Controls number of appliers (default = 4) • Setting to 1 disables parallism Max_Parallelism • Controls maximum number of appliers • Default is 50 in OGG 12.1.2.1 (30 in OGG 12.1.2.0) Commit_Serialization • Default = DEPENDENT_TRANSACTIONS • Set to FULL, if apply in source commit order is required Eager_Size • Threshold to begin apply of large transactions (default = 15011, old default=9500) • Serializes apply processing • Make sure streams_pool_size is configured correctly to avoid Waiting for Memory issues Max_SGA_Size • Controls memory resource for IR • Default is INFINITE BatchSQL • Controls number of appliers (default = 4) • Setting to 1 disables parallelism • Use with caution - degradation can be seen in some cases
- 27. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | (megabits / gigabits) * RTT = Bandwidth-Delay Product (BDP in bytes) BDP * 3 = TCPBUFSIZE/TCPFLUSHBYTES (8192 / 8) * 0.021 = 21.5 bytes 21.5 bytes * 3 = 64.5 bytes Since BDP result is less than 1MB, no need to set TCPBUFSIZE or TCPFLUSHBYTES *Default is 1MB Network Tuning
- 28. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Host Evaluate if you need bigger/newer hardware if needed!
- 29. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 30
- 30. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Further Reading • MOS Note: 2193391.1: Latest GoldenGate/Database Patch Recommendations • MOS Note: 1557031.1: Oracle GoldenGate/Oracle RDBMS Server Recommended Patches • MOS Note: 1448324.1: OGG Integrated Healthcheck Script • MOS Note: 1485620.1: Best Practices for Downstream Capture • MOS Note: 1488668.1: GoldenGate Performance Data Gathering • Oracle GoldenGate Best Practices: http://www.oracle.com/technetwork/database/availability/maa-gg-performance- 1969630.pdf
Editor's Notes
- This is a Title Slide with Picture and Logo slide ideal for including a picture and partner or product logo with a brief title, subtitle and presenter information. To customize this slide with your own picture: Right-click the slide area and choose Format Background from the pop-up menu. From the Fill menu, click Picture and texture fill. Under Insert from: click File. Locate your new picture and click Insert. To Replace the LOGO on this sample slide: Right-click the sample LOGO and choose Change Picture. Navigate to the location where the new logo is stored, select desired logo file and click on the Open button to replace the sample logo. To copy the Customized Background from Another Presentation on PC Click New Slide from the Home tab's Slides group and select Reuse Slides. Click Browse in the Reuse Slides panel and select Browse Files. Double-click the PowerPoint presentation that contains the background you wish to copy. Check Keep Source Formatting and click the slide that contains the background you want. Click the left-hand slide preview to which you wish to apply the new master layout. Apply New Layout (Important): Right-click any selected slide, point to Layout, and click the slide containing the desired layout from the layout gallery. Delete any unwanted slides or duplicates. To copy the Customized Background from Another Presentation on Mac Click New Slide from the Home tab's Slides group and select Insert Slides from Other Presentation… Navigate to the PowerPoint presentation file that contains the background you wish to copy. Double-click or press Insert. This prompts the Slide Finder dialogue box. Make sure Keep design of original slides is unchecked and click the slide(s) that contains the background you want. Hold Shift key to select multiple slides. Click the left-hand slide preview to which you wish to apply the new master layout. Apply New Layout (Important): Click Layout from the Home tab's Slides group, and click the slide containing the desired layout from the layout gallery. Delete any unwanted slides or duplicates.
- This is a Safe Harbor Front slide, one of two Safe Harbor Statement slides included in this template. One of the Safe Harbor slides must be used if your presentation covers material affected by Oracle’s Revenue Recognition Policy To learn more about this policy, e-mail: Revrec-americasiebc_us@oracle.com For internal communication, Safe Harbor Statements are not required. However, there is an applicable disclaimer (Exhibit E) that should be used, found in the Oracle Revenue Recognition Policy for Future Product Communications. Copy and paste this link into a web browser, to find out more information. http://my.oracle.com/site/fin/gfo/GlobalProcesses/RevRec/Policies/cnt2321729.pdf For all external communications such as press release, roadmaps, PowerPoint presentations, Safe Harbor Statements are required. You can refer to the link mentioned above to find out additional information/disclaimers required depending on your audience.
- Virtually every company will approach its journey to Cloud from a distinct point of view, based on an individual history and circumstances Our own four-decade history means Oracle has seen it all And when it comes to enabling the journey to Cloud, we can provide a path from any starting point Supports new cloud, existing environments, and hybrid Supports all workloads, all developers, and all data Connects all applications and business practices Goes beyond migrate - Run, Manage, Support Provide Lifetime Value What do we mean when we talk about providing lifetime value? ADDITIONAL Also, innovation goes to HOW we delivered this. I would say that you have to look at the challenges that enterprises are facing in adopting this new delivery model of the public cloud. First and foremost, is the challenge that the technology stack of the cloud is completely different from what enterprises have invested in on prem. Innovation is how we can leverage a customer’s prior investment in technology while allowing them to enjoy the benefits of the new model seamlessly, painlessly. No writing of applications, no modifications of VMs while providing bi-directional portability to the cloud and back. That is innovation. Secondly, allowing customers to leverage their existing IT processes, their architectural blueprints, their complicated network topology in the cloud is innovation. And thirdly, operational excellence in our data centers, allowing incredibly fast performance in all cloud service operations is innovation that only a few if any companies can match Oracle’s ability on. Not everyone is starting fresh in the Cloud, and in fact, few—if any—customers have this luxury. Most organizations are currently taking some initial steps or just contemplating their first steps. Their first steps they do make In the Cloud are services that need to co-exist within current environments and investments. New applications need to interoperate with on-premises applications. New forms of doing business need to co-mingle with existing business processes. Just as important as providing solutions is providing a path. Otherwise you and me are no different from many of the Cloud competitors out there. There are multiple roads a customer can take in the journey to the Cloud. With Oracle there isn’t just one solution or one path to the Cloud. Only Oracle offers a full technology continuum to enable customers wherever they might be on that journey. We can… Support new cloud, existing environments and hybrid Support all workloads, all developers, and all data Connect all applications and business practices Go beyond migrate – Architect, Run, Manage, Support Provide Lifetime Value And what do I mean by provide lifetime value??
- Performance tuning of Oracle GoldenGate is built upon established approaches used for host systems and databases. Additional turning can be done directly upon Oracle GoldenGate by using tools like TRACE/TRACE2 and LAG reporting features. Keep in mind that some latency will be due to network traffice going from source to target. Most tuning will be done at the process level.
- Performance tuning of Oracle GoldenGate is built upon established approaches used for host systems and databases. Additional turning can be done directly upon Oracle GoldenGate by using tools like TRACE/TRACE2 and LAG reporting features. Keep in mind that some latency will be due to network traffice going from source to target. Most tuning will be done at the process level.
- Performance tuning of Oracle GoldenGate is built upon established approaches used for host systems and databases. Additional turning can be done directly upon Oracle GoldenGate by using tools like TRACE/TRACE2 and LAG reporting features. Keep in mind that some latency will be due to network traffice going from source to target. Most tuning will be done at the process level.
- Performance tuning of Oracle GoldenGate is built upon established approaches used for host systems and databases. Additional turning can be done directly upon Oracle GoldenGate by using tools like TRACE/TRACE2 and LAG reporting features. Keep in mind that some latency will be due to network traffice going from source to target. Most tuning will be done at the process level.