1.
© Pythian Services Inc. 2020 1
Nelson Calero
25 Agosto 2020
Oracle Groundbreakers Tour 2020 LATAM
Automatizando patcheo y upgrades
de BD Oracle con Fleet
Provisioning and Patching (FPP)

2.
2© Pythian Services Inc. 2020© Pythian Services Inc. 2020
 Motivation
 Architecture
 Key functionalities
 Examples / demos
 Using Vagrant VMs
 Patching a Database
Fleet Provisioning and Patching (FPP)
Audiencia: DBAs / DevOps

3.
3© Pythian Services Inc. 2020© Pythian Services Inc. 2020
Nelson Calero
http://www.linkedin.com/in/ncalero @ncalerouy
• Principal Consultant at Pythian (several roles since 2014)
• Working with Oracle tools and Linux environments (since 1996)
• Co-founder and President, Uruguay Oracle user Group (2009)
• LAOUC board (since 2013)
• Computer Engineer (1998)
• Oracle ACE (2014), Oracle ACE Director (2017)
• Certified in Oracle, Amazon and Google
• Oracle University Instructor (2011)
• Blogger and speaker

4.
© Pythian Services Inc. 2020 4
Founded in 1997, Pythian is a global IT services company that helps
organizations transform how they compete and win by helping them
turn data into valuable insights, predictions and products. From cloud
automation to machine learning, Pythian designs, implements and
supports customized solutions to the toughest data challenges.

5.
5© Pythian Services Inc. 2020© Pythian Services Inc. 2020
“Oracle FPP is a software lifecycle management method for provisioning and maintaining Oracle homes”
https://docs.oracle.com/en/database/oracle/oracle-database/19/cwadd/rapid-home-provisioning.html

Spend less time dealing with Oracle software updates (no OS)

Proprietary automation approach (no ansible or similar tools) with specific features for Oracle

Since 12.1.0.2 no extra cost for targets running RAC (also no license required for server)
https://docs.oracle.com/en/database/oracle/oracle-database/19/dblic/Licensing-Information.html#GUID-0F9EB85D-4610-4EDF-89C2-4916A0E7AC87
Motivation

6.
6© Pythian Services Inc. 2020© Pythian Services Inc. 2020
 Using product features to minimize downtime
 Zero-Downtime Oracle Grid Infrastructure Patching possible since 19.6 RU (ZDOGIP – MOS ID 2635015.1)

Out of place deployments
 Implementing automation for all required steps behind a single command option
 Zero-Downtime Oracle Database upgrade (ZDU) using OGG or DG, all automated. OGG example:
 Storing templates (gold images) of software (Oracle Homes or third party) that can be reused
 Efficient space usage when using Series as only delta are stored

ACFS snapshots to provision copies
 Tracking in a central database (MGMTDB) what is deployed where – homes, databases, patches.
How?
[grid@fpps ~]$ rhpctl zdtupgrade database -dbname MYDB -destwc OH197 -ggsrcwc GG11g -ggdstwc GG12c
-targetnode host121 -root

7.
7© Pythian Services Inc. 2020© Pythian Services Inc. 2020
Architecture
Server
- ASM, ACFS, GNS
- Can be central mode or local
Target
- Host with OS installed
- Clients 12.2 and up have more
features
Clients
- Part of Grid Infrastructure
- Local FS or ACFS to store software
(working copies of gold images)
Repository
- Gold images
- Metadata (users, roles, permissions,
identities)
https://www.oracle.com/technetwork/database/database-technologies/cloud-storage/acfs/learnmore/oraclefpp-19c-datasheet-5486895.html

8.
8© Pythian Services Inc. 2020© Pythian Services Inc. 2020
• Provision and manage any software home – GI/DB o custom
• Provision, scale, patch, and upgrade Oracle Grid Infrastructure and DB
– Oracle RAC, single instance, Oracle RAC One Node
– On managed (clients) and unmanaged (target) servers
• Zero-downtime upgrade (ZDU) of databases and GI homes (check version restrictions)
• Batch mode patching
– Parallel patching nodes in RAC
– Integration with Application continuity
• Error prevention
– Evaluate commands without implementing
– Resume failed operations automatically when re-run
– Rollback
– Abort to allow manual actions to finish
• Many details, good technical documentation and presentations
– https://www.oracle.com/a/tech/docs/fpp-technical-brief-19c.pdf
– https://es.slideshare.net/MarkusMichalewicz/managing-gold-images-for-easy-patching-and-upgrading
Key functionalities

9.
9© Pythian Services Inc. 2020© Pythian Services Inc. 2020
Important clarifications
• FPP is the product name since 19c, it was RHP (Rapid Home Provisioning) on 12.1 and 18c
• CDB Fleet Management in Oracle Database 18c is another thing
https://docs.oracle.com/en/database/oracle/oracle-database/19/multi/configuring-cdb-fleet.html#GUID-50B535A6-4746-4FFD-B40A-7986DB994D3C
• OEM feature too - several MOS notes can be confusing as they describe only using emcli:
– Master Note for Database Patching Using Fleet Maintenance. (Doc ID 2435251.1)
– How to Patch Or Upgrade RAC Database Using Fleet Maintenance (Doc ID 2448341.1)
– How To Patch Or Upgrade Standalone Database Using Fleet Maintenance (Doc ID 2434260.1)
• FPP is a feature of Oracle Grid Infrastructure, allowing two modes of use: local and server
– details in next slide
• ACFS is required by server – clients can store software (workingcopies) in local storage
• Some OS versions require patches – check RHP Support Matrix (Doc ID 2233390.1)

10.
10© Pythian Services Inc. 2020© Pythian Services Inc. 2020
Local vs Server deployment

Local mode (AKA Independent Automaton)
 Only handles local host without having to register or deploy gold images
 Installed by default since 19c (to support the local switch home capability)
 rhpserver should be running as a service anyway
 Good blog posts describing its usage:

http://www.ludovicocaldara.net/dba/gi18c-patching-part1/

http://www.ludovicocaldara.net/dba/gi19c-local-mode-automaton/

Server

Needs Grid Infrastructure (GIMR + ASM + ACFS)

check MOS 2126710.1 for minimum requirements, 1369107.1 for ACFS OS Certification Matrix

1-click install – MOS Doc 2124960.1
All activity is stored in logs:
[grid@fpps ~]$ ls -lrt /u01/app/grid/crsdata/fpps/rhp/rhpserver.log*
-rw-r--r--. 1 grid oinstall 0 Aug 16 13:52 /u01/app/grid/crsdata/fpps/rhp/rhpserver.log.0.lck
-rw-r--r--. 1 grid oinstall 10485846 Aug 18 15:39 /u01/app/grid/crsdata/fpps/rhp/rhpserver.log.2
-rw-r--r--. 1 grid oinstall 10485762 Aug 18 22:56 /u01/app/grid/crsdata/fpps/rhp/rhpserver.log.1
-rw-r--r--. 1 grid oinstall 4303760 Aug 19 10:38 /u01/app/grid/crsdata/fpps/rhp/rhpserver.log.0

11.
11© Pythian Services Inc. 2020© Pythian Services Inc. 2020
FPP default local mode in 19c
[oracle@ol7-19-rac1 ~]$ srvctl status rhpserver
Rapid Home Provisioning Server is enabled
Rapid Home Provisioning Server is running on node ol7-19-rac2
[oracle@ol7-19-rac1 ~]$ srvctl config rhpserver
Storage base path: /opt/oracle/rhp_images
Disk Groups:
Port number: 8896
Transfer port range:
Rapid Home Provisioning Server is enabled
Rapid Home Provisioning Server is individually enabled on nodes:
Rapid Home Provisioning Server is individually disabled on nodes:
Email address:
Mail server address:
Mail server port:
Transport Level Security disabled
HTTP Secure is enabled

12.
12© Pythian Services Inc. 2020© Pythian Services Inc. 2020
FPP server
[oracle@ol7-19-rac1 ~]$ srvctl status rhpserver
Rapid Home Provisioning Server is enabled
Rapid Home Provisioning Server is running on node ol7-19-rac2
[oracle@ol7-19-rac1 ~]$ srvctl config rhpserver
Storage base path: /opt/oracle/rhp_images
Disk Groups:
Port number: 8896
Transfer port range:
Rapid Home Provisioning Server is enabled
Rapid Home Provisioning Server is individually enabled on nodes:
Rapid Home Provisioning Server is individually disabled on nodes:
Email address:
Mail server address:
Mail server port:
Transport Level Security disabled
HTTP Secure is enabled
[grid@fpps ~]$ srvctl status rhpserver
Rapid Home Provisioning Server is enabled
Rapid Home Provisioning Server is running on node fpps
[grid@fpps ~]$ srvctl config rhpserver
Storage base path: /rhp_storage
Disk Groups: DATA
Port number: 8896
Transfer port range:
Rapid Home Provisioning Server is enabled
Rapid Home Provisioning Server is individually enabled on nodes:
Rapid Home Provisioning Server is individually disabled on nodes:
Email address:
Mail server address:
Mail server port:
Transport Level Security disabled
HTTP Secure is enabled
[grid@fpps ~]$ rhpctl query server
Rapid Home Provisioning Server (RHPS): vgt-ol7-fpp-c
Storage base path: /rhp_storage
Disk Groups: DATA
Port number: 8896
[grid@fpps ~]$ srvctl status rhpserver
Rapid Home Provisioning Server is enabled
Rapid Home Provisioning Server is running on node fpps
[grid@fpps ~]$ srvctl config rhpserver
Storage base path: /rhp_storage
Disk Groups: DATA
Port number: 8896
Transfer port range:
Rapid Home Provisioning Server is enabled
Rapid Home Provisioning Server is individually enabled on nodes:
Rapid Home Provisioning Server is individually disabled on nodes:
Email address:
Mail server address:
Mail server port:
Transport Level Security disabled
HTTP Secure is enabled
[grid@fpps ~]$ rhpctl query server
Rapid Home Provisioning Server (RHPS): vgt-ol7-fpp-c
Storage base path: /rhp_storage
Disk Groups: DATA
Port number: 8896

13.
13© Pythian Services Inc. 2020© Pythian Services Inc. 2020
Gold images
- Created on FPP server as read-only from working software or installation files
- It can be for Oracle homes (Database, Grid Infrastructure or GoldenGate), or custom types
- Homes deployed by FPP can only be based on images
- Stored in ACFS
- Example: creating an image from a database home installed in a host (target) not running FPP
[grid@fpps ~]$ rhpctl import image -image db_19700 -imagetype ORACLEDBSOFTWARE -targetnode fppc -path
/u01/app/oracle/product/193000/dbhome_1 -root
Enter user "root" password:
fpps.localdomain: Adding storage for image ...
fpps.localdomain: Creating a new ACFS file system for image "db_19700" ...
fpps.localdomain: Creating export file system ...
fpps.localdomain: Starting export file system...
fpps.localdomain: Mounting file system...
fpps.localdomain: Copying files...
fpps.localdomain: Removing export file system ...
[grid@fpps ~]$

14.
14© Pythian Services Inc. 2020© Pythian Services Inc. 2020
Gold images
- Created on FPP server as read-only from working software or installation files
- It can be for Oracle homes (Database, Grid Infrastructure or GoldenGate), or custom types
- Homes deployed by FPP can only be based on images
- Stored in ACFS
- Example: creating an image from a database home installed in a host (target) not running FPP
[grid@fpps ~]$ rhpctl import image -image db_19700 -imagetype ORACLEDBSOFTWARE -targetnode fppc -path
/u01/app/oracle/product/193000/dbhome_1 -root
Enter user "root" password:
fpps.localdomain: Adding storage for image ...
fpps.localdomain: Creating a new ACFS file system for image "db_19700" ...
fpps.localdomain: Creating export file system ...
fpps.localdomain: Starting export file system...
fpps.localdomain: Mounting file system...
fpps.localdomain: Copying files...
fpps.localdomain: Removing export file system ...
[grid@fpps ~]$
[grid@fpps ~]$ rhpctl query image
fpps.localdomain: Audit ID: 33
Image name: db_19300
Image name: gi_19300
Image name: db_19700
[grid@fpps ~]$ rhpctl query image
fpps.localdomain: Audit ID: 33
Image name: db_19300
Image name: gi_19300
Image name: db_19700

15.
15© Pythian Services Inc. 2020© Pythian Services Inc. 2020
Gold images
- Created on FPP server as read-only from working software or installation files
- It can be for Oracle homes (Database, Grid Infrastructure or GoldenGate), or custom types
- Homes deployed by FPP can only be based on images
- Stored in ACFS
- Example: creating an image from a database home installed in a host not running FPP
[grid@fpps ~]$ rhpctl import image -image db_19700 -imagetype ORACLEDBSOFTWARE -targetnode fppc -path
/u01/app/oracle/product/193000/dbhome_1 -root
Enter user "root" password:
fpps.localdomain: Adding storage for image ...
fpps.localdomain: Creating a new ACFS file system for image "db_19700" ...
fpps.localdomain: Creating export file system ...
fpps.localdomain: Starting export file system...
fpps.localdomain: Mounting file system...
fpps.localdomain: Copying files...
fpps.localdomain: Removing export file system ...
[grid@fpps ~]$
[grid@fpps ~]$ rhpctl query image
fpps.localdomain: Audit ID: 33
Image name: db_19300
Image name: gi_19300
Image name: db_19700
[grid@fpps ~]$ rhpctl query image
fpps.localdomain: Audit ID: 33
Image name: db_19300
Image name: gi_19300
Image name: db_19700
[grid@fpps ~]$ rhpctl query image -image db_19700
fpps.localdomain: Audit ID: 34
Image name: db_19700
Owner: grid@vgt-ol7-fpp-c
Site: vgt-ol7-fpp-c
Access control: USER:grid@vgt-ol7-fpp-c
Access control: ROLE:OTHER
Access control: ROLE:GH_IMG_PUBLISH
Access control: ROLE:GH_IMG_ADMIN
Access control: ROLE:GH_IMG_VISIBILITY
Parent Image:
Software home path: /rhp_storage/images/idb_19700859372/swhome
Image state: PUBLISHED
Image size: 9054 Megabytes
Image Type: ORACLEDBSOFTWARE
Image Version: 19.0.0.0.0:19.7.1.0.0
Groups configured in the image: OSDBA=dba,OSBACKUP=dba,OSDG=dba,OSKM=dba,OSRAC=dba
Image platform: Linux_AMD64
Interim patches installed: 31204483,29585399
Contains a non-rolling patch: FALSE
Complete: TRUE
[grid@fpps ~]$ rhpctl query image -image db_19700
fpps.localdomain: Audit ID: 34
Image name: db_19700
Owner: grid@vgt-ol7-fpp-c
Site: vgt-ol7-fpp-c
Access control: USER:grid@vgt-ol7-fpp-c
Access control: ROLE:OTHER
Access control: ROLE:GH_IMG_PUBLISH
Access control: ROLE:GH_IMG_ADMIN
Access control: ROLE:GH_IMG_VISIBILITY
Parent Image:
Software home path: /rhp_storage/images/idb_19700859372/swhome
Image state: PUBLISHED
Image size: 9054 Megabytes
Image Type: ORACLEDBSOFTWARE
Image Version: 19.0.0.0.0:19.7.1.0.0
Groups configured in the image: OSDBA=dba,OSBACKUP=dba,OSDG=dba,OSKM=dba,OSRAC=dba
Image platform: Linux_AMD64
Interim patches installed: 31204483,29585399
Contains a non-rolling patch: FALSE
Complete: TRUE

16.
16© Pythian Services Inc. 2020© Pythian Services Inc. 2020
Gold images
- Created on FPP server as read-only from working software or installation files
- It can be for Oracle homes (Database, Grid Infrastructure or GoldenGate), or custom types
- Homes deployed by FPP can only be based on images
- Stored in ACFS
- Example: creating an image from a database home installed in a host not running FPP
[grid@fpps ~]$ rhpctl import image -image db_19700 -imagetype ORACLEDBSOFTWARE -targetnode fppc -path
/u01/app/oracle/product/193000/dbhome_1 -root
Enter user "root" password:
fpps.localdomain: Adding storage for image ...
fpps.localdomain: Creating a new ACFS file system for image "db_19700" ...
fpps.localdomain: Creating export file system ...
fpps.localdomain: Starting export file system...
fpps.localdomain: Mounting file system...
fpps.localdomain: Copying files...
fpps.localdomain: Removing export file system ...
[grid@fpps ~]$
[grid@fpps ~]$ rhpctl query image
fpps.localdomain: Audit ID: 33
Image name: db_19300
Image name: gi_19300
Image name: db_19700
[grid@fpps ~]$ rhpctl query image
fpps.localdomain: Audit ID: 33
Image name: db_19300
Image name: gi_19300
Image name: db_19700
[grid@fpps ~]$ rhpctl query image -image db_19700
fpps.localdomain: Audit ID: 34
Image name: db_19700
Owner: grid@vgt-ol7-fpp-c
Site: vgt-ol7-fpp-c
Access control: USER:grid@vgt-ol7-fpp-c
Access control: ROLE:OTHER
Access control: ROLE:GH_IMG_PUBLISH
Access control: ROLE:GH_IMG_ADMIN
Access control: ROLE:GH_IMG_VISIBILITY
Parent Image:
Software home path: /rhp_storage/images/idb_19700859372/swhome
Image state: PUBLISHED
Image size: 9054 Megabytes
Image Type: ORACLEDBSOFTWARE
Image Version: 19.0.0.0.0:19.7.1.0.0
Groups configured in the image: OSDBA=dba,OSBACKUP=dba,OSDG=dba,OSKM=dba,OSRAC=dba
Image platform: Linux_AMD64
Interim patches installed: 31204483,29585399
Contains a non-rolling patch: FALSE
Complete: TRUE
[grid@fpps ~]$ rhpctl query image -image db_19700
fpps.localdomain: Audit ID: 34
Image name: db_19700
Owner: grid@vgt-ol7-fpp-c
Site: vgt-ol7-fpp-c
Access control: USER:grid@vgt-ol7-fpp-c
Access control: ROLE:OTHER
Access control: ROLE:GH_IMG_PUBLISH
Access control: ROLE:GH_IMG_ADMIN
Access control: ROLE:GH_IMG_VISIBILITY
Parent Image:
Software home path: /rhp_storage/images/idb_19700859372/swhome
Image state: PUBLISHED
Image size: 9054 Megabytes
Image Type: ORACLEDBSOFTWARE
Image Version: 19.0.0.0.0:19.7.1.0.0
Groups configured in the image: OSDBA=dba,OSBACKUP=dba,OSDG=dba,OSKM=dba,OSRAC=dba
Image platform: Linux_AMD64
Interim patches installed: 31204483,29585399
Contains a non-rolling patch: FALSE
Complete: TRUE
[grid@fpps ~]$ df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 7.7G 0 7.7G 0% /dev
tmpfs 7.8G 845M 6.9G 11% /dev/shm
tmpfs 7.8G 25M 7.7G 1% /run
tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup
/dev/mapper/vg_main-lv_root 32G 1.7G 31G 6% /
/dev/sda1 497M 109M 389M 22% /boot
vagrant 930G 796G 134G 86% /vagrant
/dev/mapper/VolGroupU01-LogVolU01 100G 7.5G 93G 8% /u01
tmpfs 1.6G 0 1.6G 0% /run/user/1002
/dev/asm/ghchkpt-455 5.2G 788M 4.5G 15% /rhp_storage/chkbase
tmpfs 1.6G 0 1.6G 0% /run/user/1000
/dev/asm/ghvol809083-455 16G 8.4G 7.7G 52% /rhp_storage/images/idb_19300926564
/dev/asm/ghvol478-455 14G 7.6G 6.5G 54% /rhp_storage/images/igi_19300152127
/dev/asm/ghvol305408-455 20G 11G 9.7G 52% /rhp_storage/images/idb_19700859372
[grid@fpps ~]$ df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 7.7G 0 7.7G 0% /dev
tmpfs 7.8G 845M 6.9G 11% /dev/shm
tmpfs 7.8G 25M 7.7G 1% /run
tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup
/dev/mapper/vg_main-lv_root 32G 1.7G 31G 6% /
/dev/sda1 497M 109M 389M 22% /boot
vagrant 930G 796G 134G 86% /vagrant
/dev/mapper/VolGroupU01-LogVolU01 100G 7.5G 93G 8% /u01
tmpfs 1.6G 0 1.6G 0% /run/user/1002
/dev/asm/ghchkpt-455 5.2G 788M 4.5G 15% /rhp_storage/chkbase
tmpfs 1.6G 0 1.6G 0% /run/user/1000
/dev/asm/ghvol809083-455 16G 8.4G 7.7G 52% /rhp_storage/images/idb_19300926564
/dev/asm/ghvol478-455 14G 7.6G 6.5G 54% /rhp_storage/images/igi_19300152127
/dev/asm/ghvol305408-455 20G 11G 9.7G 52% /rhp_storage/images/idb_19700859372

17.
17© Pythian Services Inc. 2020© Pythian Services Inc. 2020
Gold images
- Created on FPP server as read-only from working software or installation files
- It can be for Oracle homes (Database, Grid Infrastructure or GoldenGate), or custom types
- Homes deployed by FPP can only be based on images
- Stored in ACFS
- Example: creating an image from a database home installed in a host not running FPP
[grid@fpps ~]$ rhpctl import image -image db_19700 -imagetype ORACLEDBSOFTWARE -targetnode fppc -path
/u01/app/oracle/product/193000/dbhome_1 -root
Enter user "root" password:
fpps.localdomain: Adding storage for image ...
fpps.localdomain: Creating a new ACFS file system for image "db_19700" ...
fpps.localdomain: Creating export file system ...
fpps.localdomain: Starting export file system...
fpps.localdomain: Mounting file system...
fpps.localdomain: Copying files...
fpps.localdomain: Removing export file system ...
[grid@fpps ~]$
[grid@fpps ~]$ rhpctl query image
fpps.localdomain: Audit ID: 33
Image name: db_19300
Image name: gi_19300
Image name: db_19700
[grid@fpps ~]$ rhpctl query image
fpps.localdomain: Audit ID: 33
Image name: db_19300
Image name: gi_19300
Image name: db_19700
[grid@fpps ~]$ rhpctl query image -image db_19700
fpps.localdomain: Audit ID: 34
Image name: db_19700
Owner: grid@vgt-ol7-fpp-c
Site: vgt-ol7-fpp-c
Access control: USER:grid@vgt-ol7-fpp-c
Access control: ROLE:OTHER
Access control: ROLE:GH_IMG_PUBLISH
Access control: ROLE:GH_IMG_ADMIN
Access control: ROLE:GH_IMG_VISIBILITY
Parent Image:
Software home path: /rhp_storage/images/idb_19700859372/swhome
Image state: PUBLISHED
Image size: 9054 Megabytes
Image Type: ORACLEDBSOFTWARE
Image Version: 19.0.0.0.0:19.7.1.0.0
Groups configured in the image: OSDBA=dba,OSBACKUP=dba,OSDG=dba,OSKM=dba,OSRAC=dba
Image platform: Linux_AMD64
Interim patches installed: 31204483,29585399
Contains a non-rolling patch: FALSE
Complete: TRUE
[grid@fpps ~]$ rhpctl query image -image db_19700
fpps.localdomain: Audit ID: 34
Image name: db_19700
Owner: grid@vgt-ol7-fpp-c
Site: vgt-ol7-fpp-c
Access control: USER:grid@vgt-ol7-fpp-c
Access control: ROLE:OTHER
Access control: ROLE:GH_IMG_PUBLISH
Access control: ROLE:GH_IMG_ADMIN
Access control: ROLE:GH_IMG_VISIBILITY
Parent Image:
Software home path: /rhp_storage/images/idb_19700859372/swhome
Image state: PUBLISHED
Image size: 9054 Megabytes
Image Type: ORACLEDBSOFTWARE
Image Version: 19.0.0.0.0:19.7.1.0.0
Groups configured in the image: OSDBA=dba,OSBACKUP=dba,OSDG=dba,OSKM=dba,OSRAC=dba
Image platform: Linux_AMD64
Interim patches installed: 31204483,29585399
Contains a non-rolling patch: FALSE
Complete: TRUE
[grid@fpps ~]$ df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 7.7G 0 7.7G 0% /dev
tmpfs 7.8G 845M 6.9G 11% /dev/shm
tmpfs 7.8G 25M 7.7G 1% /run
tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup
/dev/mapper/vg_main-lv_root 32G 1.7G 31G 6% /
/dev/sda1 497M 109M 389M 22% /boot
vagrant 930G 796G 134G 86% /vagrant
/dev/mapper/VolGroupU01-LogVolU01 100G 7.5G 93G 8% /u01
tmpfs 1.6G 0 1.6G 0% /run/user/1002
/dev/asm/ghchkpt-455 5.2G 788M 4.5G 15% /rhp_storage/chkbase
tmpfs 1.6G 0 1.6G 0% /run/user/1000
/dev/asm/ghvol809083-455 16G 8.4G 7.7G 52% /rhp_storage/images/idb_19300926564
/dev/asm/ghvol478-455 14G 7.6G 6.5G 54% /rhp_storage/images/igi_19300152127
/dev/asm/ghvol305408-455 20G 11G 9.7G 52% /rhp_storage/images/idb_19700859372
[grid@fpps ~]$ df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 7.7G 0 7.7G 0% /dev
tmpfs 7.8G 845M 6.9G 11% /dev/shm
tmpfs 7.8G 25M 7.7G 1% /run
tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup
/dev/mapper/vg_main-lv_root 32G 1.7G 31G 6% /
/dev/sda1 497M 109M 389M 22% /boot
vagrant 930G 796G 134G 86% /vagrant
/dev/mapper/VolGroupU01-LogVolU01 100G 7.5G 93G 8% /u01
tmpfs 1.6G 0 1.6G 0% /run/user/1002
/dev/asm/ghchkpt-455 5.2G 788M 4.5G 15% /rhp_storage/chkbase
tmpfs 1.6G 0 1.6G 0% /run/user/1000
/dev/asm/ghvol809083-455 16G 8.4G 7.7G 52% /rhp_storage/images/idb_19300926564
/dev/asm/ghvol478-455 14G 7.6G 6.5G 54% /rhp_storage/images/igi_19300152127
/dev/asm/ghvol305408-455 20G 11G 9.7G 52% /rhp_storage/images/idb_19700859372
[grid@fpps ~]$ acfsutil info fs
/rhp_storage/images/idb_19700859372
ACFS Version: 19.0.0.0.0
on-disk version: 49.0
compatible.advm: 19.0.0.0.0
ACFS compatibility: 19.0.0.0.0
flags: MountPoint,Available,KiloSnap
creation time: Tue Aug 18 21:29:59 2020
mount time: Tue Aug 18 21:30:27 2020
mount sequence number: 5
number of nodes: 1
allocation unit: 4096
metadata block size: 4096
volumes: 1
total size: 21474836480 ( 20.00 GB )
total free: 10403278848 ( 9.69 GB )
file entry table allocation: 201588736
primary volume: /dev/asm/ghvol305408-455
label:
state: Available
major, minor: 248, 232964
logical sector size: 512
size: 21474836480 ( 20.00 GB )
free: 10403278848 ( 9.69 GB )
metadata read I/O count: 330354
metadata write I/O count: 303807
total metadata bytes read: 2707853312 ( 2.52 GB )
total metadata bytes written: 3313106944 ( 3.09 GB )
ADVM diskgroup: DATA
ADVM resize increment: 67108864
ADVM redundancy: unprotected
ADVM stripe columns: 8
ADVM stripe width: 1048576
[grid@fpps ~]$ acfsutil info fs
/rhp_storage/images/idb_19700859372
ACFS Version: 19.0.0.0.0
on-disk version: 49.0
compatible.advm: 19.0.0.0.0
ACFS compatibility: 19.0.0.0.0
flags: MountPoint,Available,KiloSnap
creation time: Tue Aug 18 21:29:59 2020
mount time: Tue Aug 18 21:30:27 2020
mount sequence number: 5
number of nodes: 1
allocation unit: 4096
metadata block size: 4096
volumes: 1
total size: 21474836480 ( 20.00 GB )
total free: 10403278848 ( 9.69 GB )
file entry table allocation: 201588736
primary volume: /dev/asm/ghvol305408-455
label:
state: Available
major, minor: 248, 232964
logical sector size: 512
size: 21474836480 ( 20.00 GB )
free: 10403278848 ( 9.69 GB )
metadata read I/O count: 330354
metadata write I/O count: 303807
total metadata bytes read: 2707853312 ( 2.52 GB )
total metadata bytes written: 3313106944 ( 3.09 GB )
ADVM diskgroup: DATA
ADVM resize increment: 67108864
ADVM redundancy: unprotected
ADVM stripe columns: 8
ADVM stripe width: 1048576

18.
18© Pythian Services Inc. 2020© Pythian Services Inc. 2020
Gold images - note
- Importing from a client or target requires NFS server to be running. If not:
- Fix on server, and retry above import:
[grid@fpps ~]$ rhpctl import image -image db_19700 -imagetype ORACLEDBSOFTWARE -targetnode fppc -path
/u01/app/oracle/product/193000/dbhome_1 -root
...
fpps.localdomain: Mounting file system...
PRGO-1257 : failed to import image "db_19700"
PRGO-1257 : failed to import image "db_19700"
PRCZ-2105 : failed to execute command "/bin/mount" as user "root" using root credentials within 300
seconds on nodes "fppc"
PRCZ-2103 : Failed to execute command "/bin/mount" on node "fppc" as user "root". Detailed error:
mount.nfs: Connection refused
[root@fpps ~]# rpc.nfsd
[root@fpps ~]# service nfs start
Redirecting to /bin/systemctl start nfs.service

19.
19© Pythian Services Inc. 2020© Pythian Services Inc. 2020
Workingcopies
- Is a deployed image into a client or target – installed version of the software, ready to be used
- ‘add workingcopy’ executes all steps required to complete the installation
- This copy is on client storage, can be on ACFS or local storage
[grid@fpps ~]$ rhpctl add workingcopy -workingcopy wc_db_193_orig -image db_19300 -targetnode fppc
-user oracle -groups OSBACKUP=dba,OSDG=dba,OSKM=dba,OSRAC=dba -oraclebase /u01/app/oracle -path
/u01/app/oracle/product/193/dbhome_1 -root
Enter user "root" password:
fpps.localdomain: Audit ID: 37
fpps.localdomain: Option storagetype is set to the following default value: LOCAL.
fpps.localdomain: Storing metadata in repository for working copy "wc_db_193_orig" ...
fpps.localdomain: Connecting to node fppc ...
fpps.localdomain: Starting transfer for remote copy ...
fpps.localdomain: Starting clone operation...
fpps.localdomain: Using inventory file /etc/oraInst.loc to clone ...
fppc:
fppc:
fppc: [INFO] [INS-32183] Use of clone.pl is deprecated in this release. Clone operation is
equivalent to performing a Software Only installation from the image.

20.
20© Pythian Services Inc. 2020© Pythian Services Inc. 2020
Workingcopies
- Is a deployed image into a client or target – installed version of the software, ready to be used
- ‘add workingcopy’ executes all steps required to complete the instalation
- This copy is on client storage, can be on ACFS or local storage
[grid@fpps ~]$ rhpctl add workingcopy -workingcopy wc_db_193_orig -image db_19300 -targetnode fppc
-user oracle -groups OSBACKUP=dba,OSDG=dba,OSKM=dba,OSRAC=dba -oraclebase /u01/app/oracle -path
/u01/app/oracle/product/193/dbhome_1 -root
Enter user "root" password:
fpps.localdomain: Audit ID: 37
fpps.localdomain: Option storagetype is set to the following default value: LOCAL.
fpps.localdomain: Storing metadata in repository for working copy "wc_db_193_orig" ...
fpps.localdomain: Connecting to node fppc ...
fpps.localdomain: Starting transfer for remote copy ...
fpps.localdomain: Starting clone operation...
fpps.localdomain: Using inventory file /etc/oraInst.loc to clone ...
fppc:
fppc:
fppc: [INFO] [INS-32183] Use of clone.pl is deprecated in this release. Clone operation is
equivalent to performing a Software Only installation from the image.
fppc: You must use /u01/app/oracle/product/193/dbhome_1/runInstaller script available to perform the Software Only
install. For more details on image based installation, refer to help documentation.
fppc:
fppc: Starting Oracle Universal Installer...
fppc:
fppc: You can find the log of this install session at:
fppc: /u01/app/oraInventory/logs/cloneActions2020-08-18_10-13-44PM.log
fppc: .................................................. 5% Done.
fppc: .................................................. 10% Done.
fppc: .................................................. 50% Done.
fppc: ..........
fppc: Finish Setup successful.
fppc: The cloning of wc_db_193_orig was successful.
fppc: Please check '/u01/app/oraInventory/logs/cloneActions2020-08-18_10-13-44PM.log' for more details.
fppc:
fppc: Setup Oracle Base in progress.
fppc:
fppc: Setup Oracle Base successful.
fppc: .................................................. 95% Done.
fppc:
fppc: As a root user, execute the following script(s):
fppc: 1. /u01/app/oracle/product/193/dbhome_1/root.sh
fppc:
fppc: .................................................. 100% Done.
fpps.localdomain: Successfully executed clone operation.
fpps.localdomain: Executing root script on nodes fppc.
fppc: Check /u01/app/oracle/product/193/dbhome_1/install/root_fppc_2020-08-18_22-16-01-218802424.log for the output
of root script
fpps.localdomain: Successfully executed root script on nodes fppc.
fpps.localdomain: Working copy creation completed.
[grid@fpps ~]$
fppc: You must use /u01/app/oracle/product/193/dbhome_1/runInstaller script available to perform the Software Only
install. For more details on image based installation, refer to help documentation.
fppc:
fppc: Starting Oracle Universal Installer...
fppc:
fppc: You can find the log of this install session at:
fppc: /u01/app/oraInventory/logs/cloneActions2020-08-18_10-13-44PM.log
fppc: .................................................. 5% Done.
fppc: .................................................. 10% Done.
fppc: .................................................. 50% Done.
fppc: ..........
fppc: Finish Setup successful.
fppc: The cloning of wc_db_193_orig was successful.
fppc: Please check '/u01/app/oraInventory/logs/cloneActions2020-08-18_10-13-44PM.log' for more details.
fppc:
fppc: Setup Oracle Base in progress.
fppc:
fppc: Setup Oracle Base successful.
fppc: .................................................. 95% Done.
fppc:
fppc: As a root user, execute the following script(s):
fppc: 1. /u01/app/oracle/product/193/dbhome_1/root.sh
fppc:
fppc: .................................................. 100% Done.
fpps.localdomain: Successfully executed clone operation.
fpps.localdomain: Executing root script on nodes fppc.
fppc: Check /u01/app/oracle/product/193/dbhome_1/install/root_fppc_2020-08-18_22-16-01-218802424.log for the output
of root script
fpps.localdomain: Successfully executed root script on nodes fppc.
fpps.localdomain: Working copy creation completed.
[grid@fpps ~]$

21.
21© Pythian Services Inc. 2020© Pythian Services Inc. 2020
Workingcopies
- Is a deployed image into a client or target – installed version of the software, ready to be used
- ‘add workingcopy’ executes all steps required to complete the instalation
- This copy is on client storage, can be on ACFS or local storage
[grid@fpps ~]$ rhpctl add workingcopy -workingcopy wc_db_193_orig -image db_19300 -targetnode fppc
-user oracle -groups OSBACKUP=dba,OSDG=dba,OSKM=dba,OSRAC=dba -oraclebase /u01/app/oracle -path
/u01/app/oracle/product/193/dbhome_1 -root
Enter user "root" password:
fpps.localdomain: Audit ID: 37
fpps.localdomain: Option storagetype is set to the following default value: LOCAL.
fpps.localdomain: Storing metadata in repository for working copy "wc_db_193_orig" ...
fpps.localdomain: Connecting to node fppc ...
fpps.localdomain: Starting transfer for remote copy ...
fpps.localdomain: Starting clone operation...
fpps.localdomain: Using inventory file /etc/oraInst.loc to clone ...
fppc:
fppc:
fppc: [INFO] [INS-32183] Use of clone.pl is deprecated in this release. Clone operation is
equivalent to performing a Software Only installation from the image.
fppc: You must use /u01/app/oracle/product/193/dbhome_1/runInstaller script available to perform the Software Only
install. For more details on image based installation, refer to help documentation.
fppc:
fppc: Starting Oracle Universal Installer...
fppc:
fppc: You can find the log of this install session at:
fppc: /u01/app/oraInventory/logs/cloneActions2020-08-18_10-13-44PM.log
fppc: .................................................. 5% Done.
fppc: .................................................. 10% Done.
fppc: .................................................. 50% Done.
fppc: ..........
fppc: Finish Setup successful.
fppc: The cloning of wc_db_193_orig was successful.
fppc: Please check '/u01/app/oraInventory/logs/cloneActions2020-08-18_10-13-44PM.log' for more details.
fppc:
fppc: Setup Oracle Base in progress.
fppc:
fppc: Setup Oracle Base successful.
fppc: .................................................. 95% Done.
fppc:
fppc: As a root user, execute the following script(s):
fppc: 1. /u01/app/oracle/product/193/dbhome_1/root.sh
fppc:
fppc: .................................................. 100% Done.
fpps.localdomain: Successfully executed clone operation.
fpps.localdomain: Executing root script on nodes fppc.
fppc: Check /u01/app/oracle/product/193/dbhome_1/install/root_fppc_2020-08-18_22-16-01-218802424.log for the output
of root script
fpps.localdomain: Successfully executed root script on nodes fppc.
fpps.localdomain: Working copy creation completed.
[grid@fpps ~]$
fppc: You must use /u01/app/oracle/product/193/dbhome_1/runInstaller script available to perform the Software Only
install. For more details on image based installation, refer to help documentation.
fppc:
fppc: Starting Oracle Universal Installer...
fppc:
fppc: You can find the log of this install session at:
fppc: /u01/app/oraInventory/logs/cloneActions2020-08-18_10-13-44PM.log
fppc: .................................................. 5% Done.
fppc: .................................................. 10% Done.
fppc: .................................................. 50% Done.
fppc: ..........
fppc: Finish Setup successful.
fppc: The cloning of wc_db_193_orig was successful.
fppc: Please check '/u01/app/oraInventory/logs/cloneActions2020-08-18_10-13-44PM.log' for more details.
fppc:
fppc: Setup Oracle Base in progress.
fppc:
fppc: Setup Oracle Base successful.
fppc: .................................................. 95% Done.
fppc:
fppc: As a root user, execute the following script(s):
fppc: 1. /u01/app/oracle/product/193/dbhome_1/root.sh
fppc:
fppc: .................................................. 100% Done.
fpps.localdomain: Successfully executed clone operation.
fpps.localdomain: Executing root script on nodes fppc.
fppc: Check /u01/app/oracle/product/193/dbhome_1/install/root_fppc_2020-08-18_22-16-01-218802424.log for the output
of root script
fpps.localdomain: Successfully executed root script on nodes fppc.
fpps.localdomain: Working copy creation completed.
[grid@fpps ~]$
[grid@fpps ~]$ rhpctl query workingcopy
fpps.localdomain: Audit ID: 38
Working copy name: wc_db_19300
Working copy name: wc_db_193_orig
[grid@fpps ~]$
[grid@fpps ~]$ rhpctl query workingcopy
fpps.localdomain: Audit ID: 38
Working copy name: wc_db_19300
Working copy name: wc_db_193_orig
[grid@fpps ~]$

22.
22© Pythian Services Inc. 2020© Pythian Services Inc. 2020
Patches and upgrades
- All is performed by single commands using rhpctl:
 Patching: move { database | gihome } / movepdb database
 Upgrade: { upgrade | zdtupgrade } database / upgrade gihome
 Can combine both, adding more parameters to the command
- Patching example:
[grid@fpps ~]$ rhpctl move database -patchedwc wc_db_19700 -sourcewc wc_db_193 -dbname OLD19 -root
Enter user "root" password:
fpps.localdomain: Audit ID: 44
fpps.localdomain: verifying versions of Oracle homes ...
fpps.localdomain: verifying owners of Oracle homes ...
fpps.localdomain: verifying groups of Oracle homes ...
fpps.localdomain: starting to move the following databases: "OLD19"
fpps.localdomain: restarting databases: "OLD19" ...
fppc:
fppc: SQL*Plus: Release 19.0.0.0.0 - Production on Tue Aug 18 23:37:11 2020
fppc: Version 19.3.0.0.0
Fppc:
...
fppc: Patch installation complete. Total patches installed: 3
...

23.
23© Pythian Services Inc. 2020© Pythian Services Inc. 2020
Test using vagrant
server (FPPS): 16Gb RAM client (FPPC) - optional: 8Gb RAM, unmanaged
https://blogs.oracle.com/db_maintenance/oracle-fleet-patching-and-provisioning-meets-vagrantvirtualbox
$ git clone https://github.com/oracle/vagrant-boxes
$ cd vagrant-boxes/OracleFPP

24.
24© Pythian Services Inc. 2020© Pythian Services Inc. 2020
Examples using vagrant boxes
If on Windows, some old issues can be fixed easily (on WLS 10):
$ git clone https://github.com/oracle/vagrant-boxes
$ cd vagrant-boxes/OracleFPP
$ cd scripts
$ dos2unix *.sh
Examples:
- Validate initial deployment – 2m
- Create gold image of DB 19.3 using OTN binaries from
https://www.oracle.com/database/technologies/oracle19c-linux-downloads.html – 14m (video 2:07s)
- Provision gold image (creating working copy) in host not managed by FPP (target) – 9m (video 1:41s)
- Provision DB in that client home (one command BD creation) – 22m (video 1:43s)

25.
25© Pythian Services Inc. 2020© Pythian Services Inc. 2020
Examples using vagrant boxes
If on Windows, some old issues can be fixed easily (on WLS 10):
$ git clone https://github.com/oracle/vagrant-boxes
$ cd vagrant-boxes/OracleFPP
$ cd scripts
$ dos2unix *.sh
Examples:
- Validate initial deployment – 2m
- Create gold image of DB 19.3 using OTN binaries from
https://www.oracle.com/database/technologies/oracle19c-linux-downloads.html – 14m (video 2:07s)
- Provision gold image (creating working copy) in host not managed by FPP (target) – 9m (video 1:41s)
- Provision DB in that client home (one command BD creation) – 22m (video 1:43s)
DEMODEMO

26.
26© Pythian Services Inc. 2020© Pythian Services Inc. 2020
More examples in docs
Several scenarios covered in public documentation:
https://docs.oracle.com/en/database/oracle/oracle-database/19/cwadd/rapid-home-provisioning.html#GUID-4A987DBE-1BBC-4FEE-B576-A06308AA05C
8

Creating an Oracle Grid Infrastructure 12c Release 2 Deployment

Provisioning an Oracle Database Home and Creating a Database

Provisioning a Pluggable Database

Upgrading to Oracle Grid Infrastructure 12c Release 2

Patching Oracle Grid Infrastructure Without Changing the Grid Home Path

Patching Oracle Grid Infrastructure and Oracle Databases Simultaneously

Patching Oracle Database 12c Release 1 Without Downtime

Upgrading to Oracle Database 12c Release 2

Adding a Node to a Cluster and Scaling an Oracle RAC Database to the Node

Adding Gold Images for Fleet Patching and Provisioning

User Actions for Common Fleet Patching and Provisioning Tasks

27.
27© Pythian Services Inc. 2020© Pythian Services Inc. 2020
Patching a database home
FPP single command operation Manual task
- Create a working copy of the DB to be patched
- Apply patches to the working copy
- Test and validate the patched home
- Create a gold image using the patching working copy
- Delete the patched working copy with the patched DB
- Create a working copy from the patched gold image
- Move the DB to the working copy
- Test and validate DB on patched home
- Delete the working copy

28.
28© Pythian Services Inc. 2020© Pythian Services Inc. 2020
Patching a database home
FPP single command operation Manual task
- Create a working copy of the DB to be patched
- Apply patches to the working copy
- Test and validate the patched home
- Create a gold image using the patching working copy
- Delete the patched working copy with the patched DB
- Create a working copy from the patched gold image
- Move the DB to the working copy
- Test and validate DB on patched home
- Delete the working copy
DEMODEMO

29.
29© Pythian Services Inc. 2020© Pythian Services Inc. 2020
More features to explore not covered here
- Adaptive RAC Rolling OJVM Patching
- Zero-Downtime database upgrades
- Exadata Targets (Storage Cells, DB Nodes, IB switch)
- Peer-to-Peer Services
- Scan Fleet
- RESTful API
- Authentication plug-in
- Notifications
- FPP server backup and restore

30.
© Pythian Services Inc. 2020 30© Pythian Services Inc. 2020
THANK YOU
Questions?
calero@pythian.com
@ncalerouy
http://www.linkedin.com/in/ncalero

31.
31© Pythian Services Inc. 2020© Pythian Services Inc. 2020
References
• https://www.oracle.com/technetwork/database/database-technologies/cloud-storage/acfs/learnmore/oraclefpp-19c-datasheet-54
86895.html
• https://blogs.oracle.com/db_maintenance/oracle-fleet-patching-and-provisioning-meets-vagrantvirtualbox
• ACFS Support On OS Platforms (Certification Matrix). (Doc ID 1369107.1)
• RHP: Rapid Home Provisioning Server - Minimum Requirement (Doc ID 2126710.1)
• Pre-requisite Checks For Setting Up Fleet Environment (Doc ID 2434264.1)
• How To Setup a Rapid Home Provisioning (RHP) Server and Client (Doc ID 2097026.1)
• Rapid Home Provisioning (RHP) setup in "1-Click" (Doc ID 2124960.1)
• FPPBR Fleet Patching and Provisioning - Backup & Restore Utility (Doc ID 2566782.1)