This document discusses how Oracle Real Application Clusters (RAC) works on Oracle's Engineered Systems. It notes that Engineered Systems provide well-known configurations that allow for specialization and optimization. They provide hardware-assisted resilience and enable optimized software utilization through Oracle-owned operating systems and customized hardware. The document provides several examples of how Engineered Systems improve performance and availability for Oracle databases.

2. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Oracle Real Application Clusters (RAC)
on Engineered Systems – Quick Tip
Markus Michalewicz – Senior Director of Product Management, Database HA & Scalability
April 23, 2018
Markus.Michalewicz@oracle.com
@OracleRACpm
http://www.linkedin.com/in/markusmichalewicz
http://www.slideshare.net/MarkusMichalewicz

3. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
3

4. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Super Cluster
4
Oracle Database Appliance (ODA) Oracle Exadata Database Machine
Oracle’s Engineered Systems (ES)
Mini Cluster

5. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 5
Journey to Autonomous Database
Database
Appliance
Exadata
Thousands of engineer years automating and optimizing database infrastructure

6. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Engineered Systems
are well known configurations
allowing for specialization
6
Engineered Systems provide
hardware-assisted resilience
Engineered Systems
enable optimized
software utilization
Engineered Systems – Designed for Success
“ “

7. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 7
The Problem with Choices…
Venus
Oracle GI
Oracle RAC
Virtualization
Operating System
Earth
Oracle GI
Oracle RAC
Virtualization
Operating System
Storage Layer
Storage Access Layer
Virtualization Layer
Operating System Layer
Software Layer
Server Hardware Layer
Network Layer

8. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 8
The Problem with Choices … A Simple Solution
Venus
Oracle GI
Oracle RAC
Virtualization
Operating System
Earth
Oracle GI
Oracle RAC
Virtualization
Operating System
Storage Layer
Storage Access Layer
Virtualization Layer
Operating System Layer
Software Layer
Server Hardware Layer
Network Layer

9. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Well Known Configurations
• Oracle Database code uses less than 5% of OS-dependent code
– This makes the Oracle Database very portable, but limits specialization
• Engineered Systems use Oracle owned OS and well known (to the firmware) hardware
– Highly specialized configurations enable optimized software and hardware utilization
9
Protocol / Hardware (HW) Infiniband (IB) Converged Ethernet (CE)
UDP
• UDP over IP over IB
is generically supported on all HW
• UDP over CE will be supported
generically as part of CE support
RDS
• RDS over IB requires
• Oracle Linux
+ certain UEK versions
• Oracle branded HCA
N/A
RoCE N/A
• RDMA over CE (RoCE) will be
supported on Engineered Systems only

10. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Engineered Systems
are well known configurations
allowing for specialization
10
Engineered Systems provide
hardware-assisted resilience
Engineered Systems
enable optimized
software utilization
Engineered Systems – Designed for Success
“ “

11. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Hardware-Assisted Resilience
• Exadata uses IB Subnet Manager to inform higher
level software layers about component failures
– Callouts eliminate waiting for timeouts
– Examples include, but are not limited to:
• Fast Node Death Detection
• Fast Cell Death Detection
• Exadata Database Machines use a special I/O Fencing
mechanism based on the “diskmon” process, which
– Monitors and handles storage cell failures and I/O fencing
– Broadcasts intra database IORM (I/O Resource Manager)
plans from databases to storage cells
11

12. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Engineered Systems
are well known configurations
allowing for specialization
12
Engineered Systems provide
hardware-assisted resilience
Engineered Systems
enable optimized
software utilization
Engineered Systems – Designed for Success
“ “

13. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 13
Exadata – Unique Smart Platform for Database
Optimized Software Utilization
Smart Storage
• Columnar Compressed Data reduces
space usage by 10x
• Database aware Flash Caching gives
speed of flash with capacity of disk
• DB backups optimized in storage
Smart OLTP
• Optimized InfiniBand protocol
achieves > 4 million DB I/Os per sec
• Ultra-fast transactions by using DB
optimized flash logging algorithms
• Highly Available In-Memory DB by
mirroring memory across servers
Smart Consolidation
• Workload prioritization from CPU to
Network to Storage ensures QOS
• Critical DB messages always jump to
head of queue for ultra-fast latency
• I/O issued by interactive users and
important workloads is prioritized
Smart Analytics
• Move queries to storage,
not storage to queries
– Achieves 263 GB/s data throughput
– Requires > 250 10Gb Links on others
• Automatically parallelizes queries
across all storage servers

14. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 14
Exafusion and Cache Fusion Accelerator reduce context switches on RAC Systems
Optimized Software Utilization
• Exafusion
– Exafusion provides lower latency & higher throughput via
direct to wire block transfers between Oracle RAC instances
– Data is transferred directly from user space to the Infiniband
network, leading to reduced CPU utilization and better scale-
out performance
• Cache Fusion Accelerator
– The Cache Fusion Accelerator (CFA) is an OS kernel (Linux &
Solaris only) module which can respond directly to certain lock
requests via RDSv3
– CFA saves user/kernel context switches, frees up CPU cycles in
LMS, and “speeds up” messages

15. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 15
RDMA decreases messaging between RAC instances and CPU time
Optimized Software Utilization
Instance 1 Instance 2 Instance 3
UNDO UNDO
RDMA RDMA
• Undo Block RDMA-read
– In some workloads, more than half of the remote reads
are for Undo Blocks to satisfy read consistency
– Undo Block RDMA-read uses RDMA to directly and
rapidly access UNDO blocks in remote instances
• Commit Cache
– The Commit Cache maintains an in-memory table
on each instance which records the commit time
of transactions
– Remote LMS directly reads the commit cache and sends
back commit times for requested transactions
• Replaces having to send entire 8K transaction table block

16. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 16
In-Memory Duplication between RAC Instances
Optimized Software Utilization
• Similar to storage mirroring
• Duplicate in-memory columns on another node
• Enabled per table/partition
– E.g. only recent data
• Application transparent
• Downtime eliminated by using duplicate after
failure
• Improved scalability by reading from both sides
of the mirror

17. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 17
… Oracle’s expertise built in
Engineered Systems Benefit from …
http://oracle.com/goto/maa

18. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 18