Internal architecture of Exadata database server

1. Architecture of Exadata Database
Machine – Part II
By Paresh Nayak
Enterprise Db Architect & Technical Leader
1/10

2. Architecture of Exadata Database Machine – Part II
• Exadata Database machine provides a high performance, high
availability and plenty of storage space platform for oracle database
.The high availability clustering is provided by Oracle RAC and ASM
will be responsible for storage mirroring . Infiniband technology
provides high bandwidth and low latency cluster interconnect and
storage networking. The powerful compute nodes joins in the RAC
cluster to offers the great performance.
• In this article, we will see the
• Exadata Database Machine Network architecture
• Exadata Database Machine Storage architecture
• Exadata Database Machine Software architecture.
• How to scale up the Exadata Database Machine
2/10

3. Key components of the Exadata Database Machine
o Shared storage: Exadata Storage servers
Database Machine provides intelligent, high-performance shared storage to both single-instance and RAC
implementations of Oracle Database using Exadata Storage Server technology. The Exadata storage servers is designed
to provide the storage to oracle database using the ASM (Automatic Storage Management). ASM keeps the redundant
copies of data on separate Exadata Storage Servers and it protects against the data loss if you lost the disk or entire
storage server.
o Shared Network – Infiniband
Database machine uses the infiniband network for interconnect between database servers and exadata storage servers.
The infiniband network provides 40Gb/s speed.So the latency is very low and offers the high bandwidth. In Exadata
Database machine, multiple infiniband switches and interface boning will be used to provide the network redundancy.
o Shared cache:
The database machine’s RAC environment, the database instance buffer cache are shared. If one instance has kept
some data on cache and that required by another instance,the data will be provided to the required node via infiniband
cluster interconnect. It increases the performance since the data is happening between memory to memory via cluster
interconnect.
o Database Server cluster:
The Exadata database machine’s full rack consists , 8 compute nodes and you can able to build the 8-n0de cluster using
the oracle RAC. The each compute nodes has up to 80 CPU cores and 256GB memory.
o Cluster interconnect:
By default, the database machine is configured use the infiniband storage network as cluster interconnect.
3/10

4. Database Machine – Network Architecture
• Management Network – ILOM:
ILOM(Integrated lights out manager) is the default remote hardware management on all oracle servers It uses the
traditional Ethernet network to manage the exadata database machine remotely. ILOM provides the graphical remote
administration facility and it also helps the system administrators to monitor the hardware remotely.
• Client Access:
The database servers will be accessed by application servers via Ethernet network. Bonding will be created using
multiple ethernet adapters for network redundancy and load balancing. Note: This database machine consists Cisco
switch to provide the connectivity to ethernet networks.
There are three different networks has been shown on the above diagram.
4/10

5. InfiniBand Network Architecture
The below diagrams shows that how the infiniband links are connected to different components on X3-2
Half/Full Rack setup.
The spine switch will be exists only on half rack and full rack exadata database configuration only. The spine
switch will help you to scale the environment by providing the Inifiniband links to multiple racks. In the quarter
rack of X3-2 model, you will get leaf switches . You can scale up to 18 rack by adding the infiniband cables to the
infiniband switches.
switch x3-2 half-full rack
5/10

6. InfiniBand Network Architecture
How we can interconnect two racks ? Have a look at the below diagram closely. Single InfiniBand network formed
based on a Fat Tree topology
Scale two Racks
Six ports on each leaf switch are reserved for external connectivity.These ports are used for Connecting to
media servers for tape backup, Connecting to external ETL servers,Client or application access Including Oracle
Exalogic Elastic Cloud
6/10

7. Database Machine Software Architecture
Software architecture - exadata
7/10

8. Database Machine Software Architecture
 CELLSRV, MS,RS & IORM are the important process of the exadata storage cell servers. In the DB servers ,
these storage’s griddisks are used to create the ASM diskgroup.In the database server, there will be special
library called LIBCELL. In combination with the database kernel and ASM, LIBCELL transparently maps
database I/O to exadata storage server.
 There is no other filesystems are allowed to create in Exadata storage cell. Oracle Database must use the ASM
for volume manager and filesystem.
 Customers has option to choose the database servers operating system between oracle Linux and oracle
Solaris x86 . Exadata will support the oracle database 11g release 2 and laster versions of database.
8/10

9. Database Machine Storage Architecture
Exadata Storage cell
Exadata storage servers has above mentioned software components. Oracle Linux is the default operating
system for exadata storage cell software . CELLSRV is the core exadata storage component which provides the
most of the services. Management Server (MS) provides Exadata cell management and configuration.MS is
responsible for sending alerts and collects some statistics in addition to those collected by CELLSRV.Restart
Server (RS) is used to start up/shut down the CELLSRV and MS services and monitors these services to
automatically restart them if required.
How the disks are mapped to Database from the Exadata storage servers ?
Exadata Disks overview
9/10

10. If you look the below image, you can observe that database servers is considering the each cell nodes as
failure group.
Exadata DG
10/10