Oracle Exadata Cloud Services guide from practical experience - OOW19

Oracle Exadata Cloud Services
Guide from practical experience
Nelson Calero
Oracle Open World 2019
DEV3121 - Moscone South R201

Today’s topics
• Exadata oﬀering (quick review)
– On-Premises, Oracle Cloud, Cloud at customer
• Operation compared with On-premises
• Examples – migration, patching, maintenances
2 © 2018 Pythian
Intended audience: DBAs interested in Cloud and Exadata
Disclaimer: Oracle cloud changes fast, this is from Q1 2019

• Principal Consultant at Pythian – several roles since 2014
• Working with Oracle tools and Linux environments since 1996
• DBA Oracle (2001) & MySQL (2005)
• Co-founder and President of the Oracle user Group of Uruguay (2009)
• LAOUC Director of events (2013)
• Computer Engineer (1998)
• Oracle ACE (2014), Oracle ACE Director (2017)
• Oracle Certiﬁed Professional 10g/11g/12c, OCE, Cloud DB & Infra
• Amazon Solutions Architect – Associate (2016)
• Google Cloud Architect (2017), Google Cloud Data Engineer (2017)
• Oracle University Instructor (2011)
• Blogger and speaker: Oracle Open World, Collaborate, OTN Tour, Regional conferences
About me
3 © 2019 Pythian Services
http://www.linkedin.com/in/ncalero @ncalerouy

Cloud Automation/DevOps Machine Learning and MLOps
Data Lakes / Platforms / DataOps
BI - Analytics - Visualizations
Consulting - Strategy & Data Management
Data Warehouse Migration/Modernization
Database Troubleshooting
Database Strategy & Estate Planning
Database Migration
Operational Data & Cloud
Infrastructure
Analytics & Cloud
Data Solutions
Plan
Deploy
ManageDatabase Mgt

3 Membership Tiers
• Oracle ACE Director
• Oracle ACE
• Oracle ACE Associate
bit.ly/OracleACEProgram
500+ Technical Experts
Helping Peers Globally
Connect:
@oracleace
Facebook.com/oracleaces
oracle-ace_ww@oracle.com

Exadata oﬀering
7
https://www.oracle.com/technetwork/database/availability/exadata-deep-dive-3961445.pdf
© 2019 Pythian Services
On-Premises
- Customer data center
- Managed by Customer
- Purchased
Public Cloud Service
- Oracle Cloud
- Managed by Oracle
- Subscription
Cloud at Customer
- Customer data center
- Managed by Oracle
- Subscription
Autonomous
Database

OCI Regions
8
https://cloud.oracle.com/regions

https://www.oracle.com/technetwork/database/exadata/exadataservice-ds-2574134.pdf
Oracle Exadata Cloud Service (ExaCS)
9

10 https://docs.cloud.oracle.com/iaas/Content/Database/Concepts/exaoverview.htm
•Configurations:
o base (2 compute, 3 storage cells - but less cores and RAM than quarter)
o quarter rack (2 compute, 3 storage cells)
o half rack (4 compute, 6 storage cells)
o full rack (8 compute, 12 storage cells)
•Compute node VMs with root access
•NO administrative access to the components administered by Oracle:
o physical compute node hardware (dom0)
o network switches
o power distribution units (PDUs)
o integrated lights-out management (ILOM) interfaces
o Exadata Storage Servers
•Cost: first month is billed in advance (to be changed soon)
ExaCS - key differences with On-Premises

OCI ExaCS vs Exadata On-Prem
• Several database versions available (11g,12c,18c,19c), latest PSUs only
• System version (GI) is latest available (based on DB version), not choosable
• Pre-installed tools and maintenance scripts: dbaascli, dbaasapi, cleandb, etc.
• Automated backups (managed) or manual using bkup_api
• Oracle Home provisioned as ﬁlesystem – for each version, can be shared
• Compute node access only, but no dom0 access
• Fixed /u01 size, cannot be increased
• Able to scale OCPU easily
NOTE: This only apply to Oracle Exadata Cloud Services.
Other Oracle Cloud database services (VM DB Systems and Bare Metal)
have diﬀerences in tools, procedures and options.

OCI ExaCS vs Exadata On-Prem
• Several database versions available (11g,12c,18c,19c), latest PSUs only
• System version (GI) is latest available (based on DB version), not choosable
• Pre-installed tools and maintenance scripts: dbaascli, dbaasapi, cleandb, etc.
• Automated backups (managed) or manual using bkup_api
• Oracle Home provisioned as ﬁlesystem – for each version, can be shared
• Compute node access only, but no dom0 access
• Fixed /u01 size, cannot be increased
• Able to scale OCPU easily
NOTE: This only apply to Oracle Exadata Cloud Services.
Other Oracle Cloud database services (VM DB Systems and Bare Metal)
have diﬀerences in tools, procedures and options.
New features and improvements are frequent - constant
development, so always check docs to see latest information

$ oci db system-shape list -c $COMPID --availability-domain DMXj:US-ASHBURN-AD-1 --query "data [?contains("name",'Exadata')]" --output
table
+------------------+-----------------+----------------+----------------+----------------+---------------------+---------------------+
| avail-core-count | core-count-incr | max-node-count | min-core-count | min-node-count | name | shape |
+------------------+-----------------+----------------+----------------+----------------+---------------------+---------------------+
| 84 | 2 | 2 | 22 | 2 | Exadata.Quarter1.84 | Exadata.Quarter1.84 |
| 168 | 4 | 4 | 44 | 4 | Exadata.Half1.168 | Exadata.Half1.168 |
| 368 | 8 | 8 | 0 | 8 | Exadata.Full2.368 | Exadata.Full2.368 |
| 48 | 2 | 2 | 0 | 2 | Exadata.Base.48 | Exadata.Base.48 |
+------------------+-----------------+----------------+----------------+----------------+---------------------+---------------------+
DB System shapes (as of August 2019)
• X7 shapes are *2: Quarter2.92 / Half2.184 / Full2.368

calero@H6RMYZ1:~$ oci db version list -c $COMPID --db-system-shape "VM.Standard2.4" --all --output table
+-----------------------------+--------------+-----------------+
| is-latest-for-major-version | supports-pdb | version |
+-----------------------------+--------------+-----------------+
| True | False | 11.2.0.4 |
| False | False | 11.2.0.4.181016 |
| False | False |11.2.0.4.190115 |
| False | False |11.2.0.4.190416 |
| True | True | 12.1.0.2 |
| False | True | 12.1.0.2.181016 |
| False | True |12.1.0.2.190115 |
| False | True |12.1.0.2.190416 |
| True | True | 12.2.0.1 |
| False | True | 12.2.0.1.181016 |
| False | True |12.2.0.1.190115 |
| False | True |12.2.0.1.190416 |
| True | True | 18.0.0.0 |
| False | True | 18.4.0.0 |
| False | True |18.5.0.0 |
| False | True |18.6.0.0 |
| True | True |19.0.0.0 |
| False | True |19.3.0.0.190416 |
+-----------------------------+--------------+-----------------+
DBCS versions available (August vs April 2019)
+-----------------+
| version |
+-----------------+
| 11.2.0.4 |
| 11.2.0.4.180417 |
| 11.2.0.4.180717 |
| 11.2.0.4.181016 |
| 12.1.0.2 |
| 12.1.0.2.180417 |
| 12.1.0.2.180717 |
| 12.1.0.2.181016 |
| 12.2.0.1 |
| 12.2.0.1.180417 |
| 12.2.0.1.180717 |
| 12.2.0.1.181016 |
| 18.0.0.0 |
| 18.2.0.0 |
| 18.3.0.0 |
| 18.4.0.0 |
+-----------------+

Database homes
[root@myexa01 ~]# dbaascli dbhome info
DBAAS CLI version 18.2.3.1.0
Executing command dbhome info
Location of inventory.xml is set to /u01/app/oraInventory/ContentsXML/inventory.xml
Enter a homename or just press enter if you want details of all homes
1.HOME_NAME=OraHome103_12102_dbbp181016_0
HOME_LOC=/u02/app/oracle/product/12.1.0/dbhome_4
PATCH_LEVEL=12102_dbbp181016
DBs installed= CTEST
DBs installed=
DBs installed= CDBDEV
DBs installed= TEST

Database homes
[root@myexa01 ~]# dbaascli dbhome info
Executing command dbhome info
Location of inventory.xml is set to /u01/app/oraInventory/ContentsXML/inventory.xml
Enter a homename or just press enter if you want details of all homes
DBs installed= CTEST
DBs installed=
DBs installed= CDBDEV
DBs installed= TEST
[oracle@myexa01 ~]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VGExaDb-LVDbSys1
24G 19G 5G 86% /
/dev/xvda1 488M 32M 431M 7% /boot
/dev/mapper/VGExaDb-LVDbOra1
20G 16G 3.4G 82% /u01
/dev/xvdb 50G 11G 36G 24% /u01/app/12.2.0.1/grid
/dev/xvdc 50G 4.0G 43G 9% /u01/app/oracle/product/12.2.0.1/dbhome_1
/dev/xvdd 50G 8.6G 39G 19% /u01/app/oracle/product/12.1.0.2/dbhome_1
/dev/xvde 50G 5.4G 42G 12% /u01/app/oracle/product/11.2.0.4/dbhome_1
/dev/xvdg 1.1T 149G 880G 15% /u02
/dev/asm/acfsvol01-132
800G 248G 552G 45% /acfs01

Pre-installed tools
• dbaasapi - DB lifecycle management (LM), but recommended to
use OCI command line (oci) or console for this
• dbaascli - TDE + OH management + lsnrctl + PDB LM + patching
• exacli - storage cells metrics and diagnostic information
• bkup_api - backup and recovery operations
dbaastools RPM - can be conﬁgured to be updated automatically
$ rpm -qa | grep dbaastools
dbaastools_exa-1.0-1+18.2.3.1.0_190415.1015.x86_64
$ dbaascli patch tools list
$ dbaascli patch tools apply --patchid LATEST
$ dbaascli patch tools auto enable

Pre-installed tools
• dbaasapi - DB lifecycle management (LM), but recommended to
use OCI command line (oci) or console for this
• dbaascli - TDE + OH management + lsnrctl + PDB LM + patching
• exacli - storage cells metrics and diagnostic information
• bkup_api - backup and recovery operations
dbaastools RPM - can be conﬁgured to be updated automatically
$ rpm -qa | grep dbaastools
dbaastools_exa-1.0-1+18.2.3.1.0_190415.1015.x86_64
$ dbaascli patch tools list
$ dbaascli patch tools apply --patchid LATEST
$ dbaascli patch tools auto enable
[oracle@myexa01 ~]$ ls /var/opt/oracle
lrwxrwxrwx 1 root root 18 Sep 12 2018 dbaas_acfs ->
/acfs01/dbaas_acfs
drwxrwx--- 2 oracle oinstall 4096 Sep 12 2018 cprops
drwxr-xr-x 2 oracle oinstall 4096 Nov 19 05:58 tmp
drwxr-xr-x 3 oracle oinstall 4096 Nov 19 05:58 creg
drwxrwx--- 30 oracle oinstall 4096 Jan 30 11:45 log
-rw-r--r-- 1 root root 36 Feb 22 06:21 dbcs-version.txt
drwxr-xr-x 5 oracle oinstall 4096 Mar 4 11:18 bkup_api
drwxr-xr-x 6 oracle oinstall 4096 Mar 4 11:18 bundled_perllibs
drwxr-xr-x 2 oracle oinstall 4096 Mar 4 11:18 cleandb
drwxr-xr-x 11 oracle oinstall 4096 Mar 4 11:18 cns
drwxr-xr-x 5 oracle oinstall 4096 Mar 4 11:18 dbaasapi
drwxr-xr-x 6 oracle oinstall 4096 Mar 4 11:18 dbaascli
drwxr-xr-x 2 root root 4096 Mar 4 11:18 dg
drwxr-xr-x 6 root root 4096 Mar 4 11:18 ibackup
drwxr-xr-x 4 root root 4096 Mar 4 11:18 managed
drwxr-xr-x 3 oracle oinstall 4096 Mar 4 11:18 misc
drwxr-xr-x 6 root root 4096 Mar 4 11:18 odgda
drwxr-xr-x 3 oracle oinstall 4096 Mar 4 11:18 orec
drwxr-xr-x 3 oracle oinstall 4096 Mar 4 11:18 oss
drwxr-xr-x 4 oracle oinstall 4096 Mar 4 11:18 perl_lib
drwxr-xr-x 3 oracle oinstall 4096 Mar 4 11:18 rpms
drwxr-xr-x 2 oracle oinstall 4096 Mar 4 11:18 exapatch
drwxr-xr-x 3 oracle oinstall 4096 Mar 4 11:18 ocde
-rw------- 1 oracle oinstall 75626 Mar 15 10:56 cstate.xml_old
-rw------- 1 oracle oinstall 623 Mar 18 13:04 cstate.xml

Log maintenance scripts
[grid@myexa01 ~]$ cat /var/opt/oracle/cleandb/cleandblogs.cfg
#
# cleandblogs.cfg
#
# all values are in days
# ##
AlertRetention=14
ListenerRetention=14
AuditRetentionDB=1
CoreRetention=7
TraceRetention=7
shortpRetention=7
longpRetention=30
LogDirRetention=14
gfLogRetention=14
LogRetention=30
ScratchRetention=7
CstateRetention=3
# Crontab
# ***** user-name command to be executed
15 03 * * 6 oracle /var/opt/oracle/cleandb/cleandblogs.pl
15 04 * * 6 grid /var/opt/oracle/cleandb/cleandblogs.pl
[grid@myexa01 ~]$ ls -lrt /var/opt/oracle/log/cleandblogs/cleandblogs
cleandblogs_2019-02-23_04:15:01.log cleandblogs-cmd_2019-03-02_03:15:01.log
cleandblogs_2019-03-02_03:15:01.log cleandblogs-cmd_2019-03-02_04:15:01.log
cleandblogs_2019-03-02_04:15:01.log
cleandblogs-cmd_2019-03-09_03:15:01.9279777015.log
cleandblogs_2019-03-09_03:15:01.9279777015.log
cleandblogs-cmd_2019-03-09_04:15:01.668107159756.log
cleandblogs_2019-03-09_04:15:01.668107159756.log cleandblogs-cmd.log
cleandblogs-cmd_2019-02-23_04:15:01.log cleandblogs.log
[grid@myexa01 app]$ tail
/var/opt/oracle/log/cleandblogs/cleandblogs_2019-03-09_04:15:01.668107159756.log
2019-03-09 04:16:05.462929 - /u01/app/grid/crsdata/myexa01/cvu/cvulog/cvu20190206155335.log is over
30.7639814814815 days old...removing
2019-03-09 04:16:05.463141 - /u01/app/grid/crsdata/myexa01/cvu/cvulog/cvu20190206215336.log is over
30.5139351851852 days old...removing
2019-03-09 04:16:05.464719 - Completed maintenance activities on instance +ASM1.
2019-03-09 04:16:05.465125 - Removing files like /var/opt/oracle/log/cleandblogs/* older than 14 days.

Tales from the ﬁeld
• Migrating DB from On-Prem
• Creating non-CDB using dbaasapi
• Upgrading shapes (X6 to X7)
• Patching
• Planned maintenances

• All migration options in www.oracle.com/goto/move, more speciﬁcs in
https://www.oracle.com/database/technologies/high-availability/oracle-cloud-maa.html
• Example: Creating standby on OCI same version.
MOS note 2386116.1 with all steps
– Create cloud host
– Create Cloud standby using RMAN duplicate
– Enable TDE and encrypt standby
– Switchover
If upgrade is needed, similar procedure but with extra
steps - MOS note 2326901.1 for that, using a transient
logical standby to perform rolling upgrade, or switchover and upgrade if not possible
Migrating DB from On-Prem

Migrating DB from On-Prem
Some of the non-intuitive steps:
– Few latest DB PSU available, meaning we could need to patch source as first step
– Need to drop OCI DB to create standby: web console is not aware of changes
– Registering the new DB with OCI tooling - dbaascli regdb
– PDB is recommended: we need to convert from non-cdb as last step after switchover
– OCI requires TDE: it can be enabled while OCI is standby from On-Prem, saving the
downtime required for this task if done after switchover using Fast Offline Conversion
method: https://www.oracle.com/technetwork/database/availability/tde-conversion-dg-3045460.pdf
– Migration leaves DB without standby on OCI if only doing above steps
• Creating the standby on OCI is a manual procedure
• Automated only for same region (different AD)
• MOS note 2273304.1 and Peter post about keeping STB when converting to PBD
(more details later)

Registering the new DB with OCI (1/3)
[root@myexa01 ~]# dbaascli regdb prereqs --dbname CTEST
Executing command regdb prereqs
INFO: Logfile Location: /var/opt/oracle/log/CTEST/regdb/regdb_2019-04-02_12:02:19.040884215892.log
WARNING : Database service name not matches : From DB: CTEST.mydomain.com, From creg:
CTEST_iad3kc.prdoc1.ashvcn.oraclevcn.com
WARNING : Database is not in flashback mode, Found: NO
ERROR : Database block change tracking not enabled, Found: DISABLED
WARNING: Database redo destination set to ASM Disk groups +DATAC1 +RECOC1 Expected: +RECOC1
WARNING : tde_wallet status mismatch. Found: OPEN:PASSWORD Expected: OPEN:AUTOLOGIN
ERROR: Prereqs failed. Please check log:
/var/opt/oracle/log/CTEST/regdb/regdb_2019-04-02_12:02:19.040884215892.log
[root@myexa01 ~]#
After dropping pre-created OCI DB and manually creating DB as standby
- Use dbaascli to validate DB has all prerequisites to be registered
=> Manually ﬁx every error reported before continuing

When all changes are made, register it:
[root@myexa01 ~]# dbaascli regdb begin --dbname CTEST
Executing command regdb begin
Logfile Location: /var/opt/oracle/log/CTEST/regdb/regdb_2019-04-02_12:20:14.051075262852.log
Running prereqs
Executing command regdb prereqs
...
Prereqs completed
WARNING : Updating the service_names
Running OCDE .. will take time ..
OCDE Completed successfully.
Database CTEST registered as Cloud database
[root@myexa01 ~]#

[root@myexa01 ~]# dbaascli database status --dbname CTEST
Executing command database status
Database Status:
PRCD-1120 : The resource for database CTEST_iad3kc could not be found.
PRCR-1001 : Resource ora.ctest_iad3kc.db does not exist
Database name: CTEST
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
PL/SQL Release 12.1.0.2.0 - Production
CORE 12.1.0.2.0 Production
TNS for Linux: Version 12.1.0.2.0 - Production
NLSRTL Version 12.1.0.2.0 - Production
[root@myexa01 ~]# ls -lrt /var/opt/oracle/creg
total 52
drwxrwx--- 2 oracle oinstall 4096 Mar 20 12:20 grid
lrwxrwxrwx 1 oracle oinstall 30 Mar 28 10:29 CTEST_iad3kc.ini -> /var/opt/oracle/creg/CTEST.ini
-rw-rw---- 1 oracle oinstall 4322 Apr 1 04:38 CTEST.ini
[root@myexa01 ~]# head /var/opt/oracle/creg/CTEST.ini
#################################################################
# This file is automatically generated by database as a service #
# #
# Do not edit contents in this file, Thanks! #

Database Status:
PRCD-1120 : The resource for database CTEST_iad3kc could not be found.
PRCR-1001 : Resource ora.ctest_iad3kc.db does not exist
[root@myexa01 ~]# ls -lrt /var/opt/oracle/creg
total 52
drwxrwx--- 2 oracle oinstall 4096 Mar 20 12:20 grid
lrwxrwxrwx 1 oracle oinstall 30 Mar 28 10:29 CTEST_iad3kc.ini -> /var/opt/oracle/creg/CTEST.ini
-rw-rw---- 1 oracle oinstall 4322 Apr 1 04:38 CTEST.ini
[root@myexa01 ~]# head /var/opt/oracle/creg/CTEST.ini
#################################################################
# This file is automatically generated by database as a service #
# #
# Do not edit contents in this file, Thanks! #
[root@myexa01 ~]# grep -c CTEST_iad3kc /var/opt/oracle/creg/CTEST.ini
4
[root@myexa01 ~]# sed -i 's/CTEST_iad3kc/CTEST/g' /var/opt/oracle/creg/CTEST.ini
[root@myexa01 ~]# grep -c CTEST_iad3kc /var/opt/oracle/creg/CTEST.ini
0
Database Status:
Instance CTEST1 is running on node myexa01. Instance status:
Open,HOME=/u02/app/oracle/product/12.1.0/dbhome_4.
Instance CTEST2 is running on node myexa02. Instance status:
Open,HOME=/u02/app/oracle/product/12.1.0/dbhome_4.
[root@myexa01 ~]#

Enable TDE in the OCI standby
• White paper: Converting to Transparent Data Encryption with Oracle Data
Guard using Fast Offline Conversion – Enabling TDE for Oracle 12.1
https://www.oracle.com/technetwork/database/availability/tde-conversion-dg-3045460.pdf
• Different procedure in 11.2 vs 12.x (because of KEYSTORE usage in 12.x)
– Create autologin keystore (wallet) on primary (several steps)
– Copy files to standby
– Configure wallet on standby
– Restart standby in mount mode
– Stop recovery
– Encrypt each tablespace manually:
•
• Excluding SYSTEM, SYSAUX, UNDO and TEMP
– Start recovery
alter database datafile '+DATA/TDESTBY/DATAFILE/users.1163.910254789' encrypt;

Enable TDE in the OCI standby
Encrypted blocks can be seen using DBVERIFY:
[oracle@myexa01 TDE_convert]$ dbv file='+DATA/TDESTBY/DATAFILE/mydata.313.1005593357' userid=/
DBVERIFY: Release 12.1.0.2.0 - Production on Tue Apr 16 01:59:23 2019
Copyright (c) 1982, 2014, Oracle and/or its affiliates. All rights reserved.
DBVERIFY - Verification starting : FILE = +DATA/TDESTBY/DATAFILE/mydata.313.1005593357
DBVERIFY - Verification complete
Total Pages Examined : 1048576
Total Pages Processed (Data) : 0
Total Pages Failing (Data) : 0
Total Pages Processed (Index): 0
Total Pages Failing (Index): 0
Total Pages Processed (Other): 1494
Total Pages Processed (Seg) : 0
Total Pages Failing (Seg) : 0
Total Pages Empty : 1
Total Pages Marked Corrupt : 0
Total Pages Influx : 0
Total Pages Encrypted : 1047081
Highest block SCN : 0 (0.0)

Convert to PDB
• Well known procedure, but requires downtime
– Several possible issues. Good blog serie about this from Mike Dietrich:
https://mikedietrichde.com/2019/07/22/database-migration-from-non-cdb-to-pdb-overview/
• No cloud DR option just after it completed - extra time to create standby
• Can we have standby in place for DR before conversion?
– This is: non-cdb standby conﬁguration, that stays after converting to PDB
– Reusing the Source Standby Database Files When Plugging a non-CDB as a
PDB into the Primary Database of a Data Guard Conﬁguration (Doc ID
2273304.1)
• Lot of details, good example on this blog post from Pieter Van Puymbroeck
https://vanpupi.stepi.net/2019/06/08/to-cdb-or-not-to-cdb-thats-the-question/

Creating non-CDB using dbaasapi
● Cloud databases are CDB by default (12.1)
● This procedure is for particular cases where PDB cannot be used
● MOS note 2528257.1 has all details
○ Update local software library to include non-CDB images
○ Create a JSON ﬁle with parameters to create the DB
# download the 12.1 January 2019 non-CDB software image:
$ dbaascli cswlib download --version 12102 --bp JAN2019 --cdb no
# prepare JSON file and launch DB creation
$ /var/opt/oracle/dbaasapi/dbaasapi -i createdb.json
# check status
$ tail /home/oracle/dbinput/createdb.out
# monitor progress using another JSON file
$ /var/opt/oracle/dbaasapi/dbaasapi -i createdbStatus.json
$ cat /home/oracle/dbinput/createdbStatus.out

Upgrading shape X6 to X7
• This is hardware upgrade: need to move data between servers
• Easy solution (new in 12.2): Refreshable clone PDB
– No need for dataguard, this is another feature but using DB link
(despite automatic DG is also possible)
– New in 18c: Refreshable PDB Switchover
# on target:
CREATE PLUGGABLE DATABASE pdb_copy FROM mypdb@remoteDB
REFRESH MODE EVERY 60 MINUTES;
# switchover: open RO target
ALTER PLUGGABLE DATABASE pdb_copy OPEN READ ONLY;
# perform switchover in source:
ALTER SESSION SET CONTAINER = mypdb;
ALTER PLUGGABLE DATABASE
REFRESH MODE MANUAL
FROM pdb_copy@remoteDB
SWITCHOVER;
https://docs.oracle.com/en/database/oracle/oracle-database/18/multi/administering-pdbs-with-sql-plus.html#GUID-B505C234-FAF4-4BAB-8B59-59276E0EA128

Patching
• We (customer) should patch Compute nodes: OS, DB and Grid homes
• Oracle patches Dom0, cells and IB, but not the Compute nodes
• Planned patch window is not ﬂexible (only which weekend of the
patching month to use, asking on an SR)
• Several sites at patched at the same time, cannot opt to leave some out
for availability
• After maintenance ends we need to validate all went well
– some system could be patched and others no (i.e: window not large enough
to deal with all errors (Standby, or some nodes))
– make sure all services we had running are up now. SRs to follow up if you
discover that. SLA does not include those.
• X6 DBs uses APEX, while X7 does not (i.e.: longer patch time)

# list available patches
dbaascli patch db list --oh exacs-node1:/u02/app/oracle/product/18.0.0.0/dbhome_1
# Validate patch prereqs
dbaascli patch db prereq --patchid 12345678 --dbnames MYDB
# Apply a patch
# NOTE: you run the SQL part only on the last node
dbaascli patch db apply --patchid 12345678 --dbnames MYDB --run_datasql 1
# rollback a patch
dbaascli patch db switchback --patchid 12345678 --dbnames MYDB --run_datasql 1
--instance1 exacs-node1:/u02/app/oracle/product/18.0.0.0/dbhome_1
35
Patching using CLI - Exadata
https://docs.cloud.oracle.com/iaas/Content/Database/Tasks/exapatching.htm
Note exadbcpatchmulti utility has been incorporated into dbaascli

ExaCS DB backups
• Managed by Oracle: Object storage and local FRA disk
https://docs.cloud.oracle.com/iaas/Content/Database/Tasks/exabackingup.htm
• unmanaged: RMAN or dbcli to local storage
• logs: /opt/oracle/dcs/log/<nodename>/rman/bkup/<db_unique_name>
• Conﬁgure/customize automatic backups
– Most RMAN options + cloud storage options (bkup_oss_url, bkup_oss_user, etc.)
https://docs.oracle.com/en/cloud/paas/exadata-cloud/csexa/customize-backup-configuration-using-bkup_api.html
# /var/opt/oracle/bkup_api/bkup_api set config --file=filename --dbname=dbname

Maintenances
• Planned maintenances
– Email notification in advance, we need to check after it completes
• Outages (unplanned) examples requiring our investigation
– dom0 errors causing ora-600 and reboots, cannot fix as we don't have access to
dom0. Ended up being a known issue, fixed by Oracle after following up with an SR
– Node reboots: failing hardware, software issues or planned maintenance?
– Automatic RMAN backup failed with no apparent reason - ended up being intermittent
network issues breaking the communication with Object store
– /u01 filled up with traces (kdump) after node crashes
• Tools changes/upgrades
– Behaviour changes affecting our own scripts if we process their output/logs

What’s next?
• Oracle PMs help a lot to understand what’s needed to move forward our
issues and requirements - don’t be afraid to reach out to them
• Explore the APIs/CLI to automate interventions without using the console
• Take advantage of Oracle Management Cloud
– more than just monitoring: https://cloud.oracle.com/management
– Setup and use example from Oracle Learning Library:
https://apexapps.oracle.com/pls/apex/f?p=44785:50:7039277249749:::50:P50_COURSE_ID,P50_EVENT_ID:196,5791
• Explore hybrid cloud deployments to expand your on-premises capacity
for speciﬁc use cases

THANK YOU
Questions?
39
calero@pythian.com
@ncalerouy
http://www.linkedin.com/in/ncalero

References - documentation
Oracle Exadata Cloud Services concepts https://docs.cloud.oracle.com/iaas/Content/Database/Concepts/exaoverview.htm
Oracle Cloud tooling: https://docs.oracle.com/en/cloud/paas/exadata-cloud/csexa/update-cloud-tooling.html
Patching: https://docs.cloud.oracle.com/iaas/Content/Database/Tasks/exapatching.htm
Backups: https://docs.cloud.oracle.com/iaas/Content/Database/Tasks/exabackingup.htm
Reusing the Source Standby Database Files When Plugging a non-CDB as a PDB into the Primary Database of a
Data Guard Conﬁguration (Doc ID 2273304.1)
Creating non-CDB databases using Oracle Database 12c on the Exadata Cloud Service 2528257.1
Migration to Exadata Cloud using Simple Data Guard Approach with Minimal Downtime(Doc ID 2386116.1)
Oracle Database Tablespace Encryption Behavior in Oracle Cloud (Doc ID 2359020.1)
Converting to Transparent Data Encryption with Oracle Data Guard using Fast Oﬄine Conversion
https://www.oracle.com/technetwork/database/availability/tde-conversion-dg-3045460.pdf

Oracle Exadata Cloud Services guide from practical experience - OOW19

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Oracle Exadata Cloud Services guide from practical experience - OOW19

Similar to Oracle Exadata Cloud Services guide from practical experience - OOW19 (20)

More from Nelson Calero

More from Nelson Calero (20)

Recently uploaded

Recently uploaded (20)

Oracle Exadata Cloud Services guide from practical experience - OOW19