Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Effective Oracle Home Management
with the New Release Model
Ludovico Caldara
Computing Engineer
IT Department, CERN
ludovi...
About Ludovico Caldara
■ 18+ Years DBA (Not Only Oracle)
■ ITOUG co-founder
■ OCP (11g, 12c, MySQL) & OCE
■ Italian living...
CERN
• CERN - European Laboratory for Particle Physics
• Founded in 1954 by 12 Countries for fundamental
physics research ...
Fundamental Research
• What is 95% of the Universe made of?
• Why do particles have mass?
• Why is there no antimatter
lef...
Effective Oracle Home Management
#UKOUG_Tech18
6
The Large Hadron Collider (LHC)
Largest machine in the world
27km, 6000+ ...
“Patch soon, patch often” vs reality
05.12.2018
Effective Oracle Home Management
#UKOUG_Tech18
7
Frequent patching?
• Patching is time consuming
• It is complex
• It introduces some risks
• It is not always really neces...
How many bugs could you encounter?
$ opatch lspatches
27923320;Database PSU 12.1.0.2.180717, Oracle JavaVM Component (JUL2...
Quick overview of the
New Release Model
05.12.2018
Effective Oracle Home Management
#UKOUG_Tech18
10
What does it change?
Jan 18 Apr Jul Oct Jan 19 Apr Jul Oct
RU 18.1.0 18.2.0 18.3.0 18.4.0 18.5.0 18.6.0 18.7.0 18.8.0
RUR1...
What does it change?
05.12.2018
Effective Oracle Home Management
#UKOUG_Tech18
12
Short
Development
cycle
Frequent
Upgrade...
Which patches should I apply?
05.12.2018
Effective Oracle Home Management
#UKOUG_Tech18
13
Brave vs Fearful
12.1 >= 12.2
Effective Oracle Home Management
#UKOUG_Tech18
14
How
do I
feel?
PSU
Bundle
Patch
BraveFearf...
Brave vs Fearful
12.1 >= 12.2
Effective Oracle Home Management
#UKOUG_Tech18
15
How
do I
feel?
PSU
Bundle
Patch
BraveFearf...
Common patching patterns
05.12.2018
Effective Oracle Home Management
#UKOUG_Tech18
16
Common worst practices in maintaining homes
• Installing software by hand
• Different patch levels for Oracle Homes with t...
Worst approach (big bang)
1. Stop everything
2. In-place binaries patching
3. Database patching, all at once
4. Start ever...
Worst approach (big bang)
1. Stop everything
2. In-place binaries patching
3. Database patching, all at once
4. Start ever...
Different approach (still bad)
1. Install new binaries manually
2. Patch the new binaries
3. Stop, change OH, patch
databa...
Different approach (still bad)
1. Install new binaries manually
2. Patch the new binaries
3. Stop, change OH, patch
databa...
Automate the installation!
• Automated creation
05.12.2018
Effective Oracle Home Management
#UKOUG_Tech18
22
Soft
ware Pat...
Image repo
Automate the installation!
• Automated creation • Software cloning
05.12.2018
Effective Oracle Home Management
...
Image repo
Automate the installation!
• Automated creation • Software cloning
05.12.2018
Effective Oracle Home Management
...
Less paths
Effective Oracle Home Management
#UKOUG_Tech18
2505.12.2018
18.3.0 18.4.0 18.4.0
+ one-off X
18.5.0
+ one-off X...
Workarounds vs one-offs
• Sometimes through underscore parameters
• Mitigate the number of one-offs
• Less merge requests
...
Rapid Home provisioning
• It might be the perfect product…
• Requires Lifecycle Management Pack
• The technology stack is ...
Custom approach
• Simple script
• Golden images
• Solid naming conventions
• I blogged about it!
http://www.ludovicocaldar...
Implementation at Etat de Vaud
05.12.2018
Effective Oracle Home Management
#UKOUG_Tech18
29
Naming Conventions
• Central Inventory contains OH name
• OH name matches OH location
• No dots (.) allowed (inventory lim...
Golden images
05.12.2018
Effective Oracle Home Management
#UKOUG_Tech18
31
Golden Images Store
18_3_0.zip
18_4_0.zip
18_3_...
Image store
Typical workflow
05.12.2018
Effective Oracle Home Management
#UKOUG_Tech18
33
SERVER1
18_3_0
SERVER2…
First In...
Image store
Typical workflow
05.12.2018
Effective Oracle Home Management
#UKOUG_Tech18
34
18_3_0.zip
SERVER1
18_3_0
SERVER...
Image store
Typical workflow
05.12.2018
Effective Oracle Home Management
#UKOUG_Tech18
35
18_3_0.zip
SERVER1
18_3_0
SERVER...
Image store
Typical workflow
05.12.2018
Effective Oracle Home Management
#UKOUG_Tech18
36
18_3_0.zip
SERVER1
18_3_0
SERVER...
Image store
Typical workflow
05.12.2018
Effective Oracle Home Management
#UKOUG_Tech18
37
18_3_0.zip
SERVER1
18_3_0
SERVER...
Image store
Typical workflow
05.12.2018
Effective Oracle Home Management
#UKOUG_Tech18
38
18_3_0.zip
SERVER1
18_3_0
SERVER...
Image store
Typical workflow
05.12.2018
Effective Oracle Home Management
#UKOUG_Tech18
39
18_3_0.zip
SERVER1
18_3_0
SERVER...
What it actually does
Creation workflow
• Copy the Home
• Clean it up
• Create the zip
• Store the zip in the repo
• Store...
Effective Oracle Home Management
#UKOUG_Tech18
4105.12.2018
DEMO
Further development at CERN
05.12.2018
Effective Oracle Home Management
#UKOUG_Tech18
42
Read-only Oracle Home
$ roohctl –enable
Enabling Read-Only Oracle home.
Update orabasetab file to enable Read-Only Oracle ...
AGILE INFRASTRUCTURE
SERVER
Insertion in the Agile Infrastructure
05.12.2018
Effective Oracle Home Management
#UKOUG_Tech1...
Grid Infrastructure Installation
• Puppet + Rundeck for automatic GI install
• Patched Grid Infrastructure versions beside...
Independent Local-mode Automaton (18c)
05.12.2018
Effective Oracle Home Management
#UKOUG_Tech18
46
# $GI_HOME/srvctl enab...
Independent Local-mode Automaton (18c)
• Damn-easy move of GI HOME
• Automatic resource relocation
• Enable Grid Infrastru...
Back to the demo…
05.12.2018
Effective Oracle Home Management
#UKOUG_Tech18
48
Questions?
05.12.2018
Effective Oracle Home Management
#UKOUG_Tech18
49
Thank you!
Ludovico Caldara
Computing Engineer
IT Department, CERN
ludovico.caldara@cern.ch
@ludodba
05.12.2018
Effective ...
Effective Oracle Home Management in the new Release Model era
Prochain SlideShare
Chargement dans…5
×

Effective Oracle Home Management in the new Release Model era

59 vues

Publié le

How many companies can afford patching regularly their environments?
Patching and maintaining a big amount of Oracle Databases is perceived as complex by most companies. Is there a way to make patching simpler and more controlled? What are the best (and worst) practices for Oracle Home maintenance?
What are the challenges of the new release model that will bring us one new major release per year?
In this session, we will explain some ideas to improve Oracle Home management and database patching, as well as practical examples of automated environments, live demos included!

Publié dans : Technologie
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

Effective Oracle Home Management in the new Release Model era

  1. 1. Effective Oracle Home Management with the New Release Model Ludovico Caldara Computing Engineer IT Department, CERN ludovico.caldara@cern.ch @ludodba 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 2
  2. 2. About Ludovico Caldara ■ 18+ Years DBA (Not Only Oracle) ■ ITOUG co-founder ■ OCP (11g, 12c, MySQL) & OCE ■ Italian living in Switzerland ■ http://www.ludovicocaldara.net ■ @ludodba ■ ludovicocaldara 05.12.2018 3 Effective Oracle Home Management #UKOUG_Tech18
  3. 3. CERN • CERN - European Laboratory for Particle Physics • Founded in 1954 by 12 Countries for fundamental physics research in the post-war Europe • Today 22 member states and world-wide collaborations • About ~1100 MCHF yearly budget • 2’300 CERN personnel • 10’000 users from 110 countries 405.12.2018 Effective Oracle Home Management #UKOUG_Tech18
  4. 4. Fundamental Research • What is 95% of the Universe made of? • Why do particles have mass? • Why is there no antimatter left in the Universe? • What was the Universe like, just after the "Big Bang"? 505.12.2018 Effective Oracle Home Management #UKOUG_Tech18
  5. 5. Effective Oracle Home Management #UKOUG_Tech18 6 The Large Hadron Collider (LHC) Largest machine in the world 27km, 6000+ superconducting magnets Emptiest place in the solar system High vacuum inside the magnets Hottest spot in the galaxy During Lead ion collisions create temperatures 100 000x hotter than the heart of the sun Fastest racetrack on Earth Protons circulate 11245 times/s (99.9999991% the speed of light)
  6. 6. “Patch soon, patch often” vs reality 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 7
  7. 7. Frequent patching? • Patching is time consuming • It is complex • It introduces some risks • It is not always really necessary • It leads to human errors 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 8
  8. 8. How many bugs could you encounter? $ opatch lspatches 27923320;Database PSU 12.1.0.2.180717, Oracle JavaVM Component (JUL2018) 22243983; 27762253;OCW PATCH SET UPDATE 12.1.0.2.180717 (27762253) 27547374;Database Bundle Patch : 12.1.0.2.180717 (27547374) OPatch succeeded. $ cd $ORACLE_HOME/inventory $ grep -r "bug description" * | wc -l 2132 $ grep -r "bug description" * | grep -i "wrong result" | wc -l 66 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 9
  9. 9. Quick overview of the New Release Model 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 10
  10. 10. What does it change? Jan 18 Apr Jul Oct Jan 19 Apr Jul Oct RU 18.1.0 18.2.0 18.3.0 18.4.0 18.5.0 18.6.0 18.7.0 18.8.0 RUR1 18.2.1 18.3.1 18.4.1 18.5.1 18.6.1 18.7.1 RUR2 18.2.2 18.3.2 18.4.2 18.5.2 18.6.2 RU 19.1.0 19.2.0 19.3.0 19.4.0 RUR1 19.2.1 19.3.1 RUR2 19.2.2 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 11 2
  11. 11. What does it change? 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 12 Short Development cycle Frequent Upgrades And Updates
  12. 12. Which patches should I apply? 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 13
  13. 13. Brave vs Fearful 12.1 >= 12.2 Effective Oracle Home Management #UKOUG_Tech18 14 How do I feel? PSU Bundle Patch BraveFearful How do I feel? Release Update Revision Release Update BraveFearful • 05.12.2018
  14. 14. Brave vs Fearful 12.1 >= 12.2 Effective Oracle Home Management #UKOUG_Tech18 15 How do I feel? PSU Bundle Patch BraveFearful How do I feel? Release Update Revision Release Update BraveFearful Ludovico(*) Ludovico(*) (*) yes, it’s Comic Sans https://mikedietrichde.com/2017/10/24/differences-psu-bp-ru-rur/ • 05.12.2018
  15. 15. Common patching patterns 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 16
  16. 16. Common worst practices in maintaining homes • Installing software by hand • Different patch levels for Oracle Homes with the same path • Not tracking the installed patches • Oracle Home paths hard-coded in the operational scripts • Not minding about naming conventions 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 17
  17. 17. Worst approach (big bang) 1. Stop everything 2. In-place binaries patching 3. Database patching, all at once 4. Start everything Effective Oracle Home Management #UKOUG_Tech18 18 UP UP UP UP DOWN DOWN DOWN DOWN UP UP UP UP UP DOWN UP STOP ALL START ALLPATCH BINARIES PATCH DBs • 05.12.2018
  18. 18. Worst approach (big bang) 1. Stop everything 2. In-place binaries patching 3. Database patching, all at once 4. Start everything Effective Oracle Home Management #UKOUG_Tech18 1905.12.2018 UP UP UP UP DOWN DOWN DOWN DOWN UP UP UP UP UP DOWN UP STOP ALL START ALLPATCH BINARIES PATCH DBs No easy rollback Long downtime Complex planning
  19. 19. Different approach (still bad) 1. Install new binaries manually 2. Patch the new binaries 3. Stop, change OH, patch databases one by one 4. Decommission old binaries Effective Oracle Home Management #UKOUG_Tech18 2005.12.2018 UP UP UP UP DOWN DOWN DOWN DOWN UP UP UP UP PATCH BINARIES STOP, MOVE, PATCH ONE BY ONE
  20. 20. Different approach (still bad) 1. Install new binaries manually 2. Patch the new binaries 3. Stop, change OH, patch databases one by one 4. Decommission old binaries Effective Oracle Home Management #UKOUG_Tech18 2105.12.2018 UP UP UP UP DOWN DOWN DOWN DOWN UP UP UP UP PATCH BINARIES STOP, MOVE, PATCH ONE BY ONE Complex to do manually Repetitive task when many servers
  21. 21. Automate the installation! • Automated creation 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 22 Soft ware Patch Patch Patched Software • runInstaller -silent • opatch apply
  22. 22. Image repo Automate the installation! • Automated creation • Software cloning 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 23 Soft ware Patch Patch Patched Software • runInstaller -silent • opatch apply 18.3.1 18.4.0 Server 18.3.1
  23. 23. Image repo Automate the installation! • Automated creation • Software cloning 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 24 Soft ware Patch Patch Patched Software • runInstaller -silent • opatch apply 18.3.1 18.4.0 Server 18.3.1 RUNDECK ANSIBLE RHP OTHER
  24. 24. Less paths Effective Oracle Home Management #UKOUG_Tech18 2505.12.2018 18.3.0 18.4.0 18.4.0 + one-off X 18.5.0 + one-off X 18.5.0 + one-off X + one-off Y 18.3.0 18.4.0 18.4.0 + one-off X 18.5.0 + one-off X 18.5.0 + one-off X + one-off Y 18.4.1 18.4.1 + one-off Y GOOD: BAD:
  25. 25. Workarounds vs one-offs • Sometimes through underscore parameters • Mitigate the number of one-offs • Less merge requests • Document it! (why, when, who) 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 26 1 2 3 alter system set "_px_groupby_pushdown"=off comment='Ludo, 03.05.16: W/A for bug 18499088' scope=both sid='*‘;
  26. 26. Rapid Home provisioning • It might be the perfect product… • Requires Lifecycle Management Pack • The technology stack is complex (Clusterware, ASM, ACFS, NFS, GNS…) 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 27
  27. 27. Custom approach • Simple script • Golden images • Solid naming conventions • I blogged about it! http://www.ludovicocaldara.net/dba/oh-mgmt-[1..7]/ 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 28
  28. 28. Implementation at Etat de Vaud 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 29
  29. 29. Naming Conventions • Central Inventory contains OH name • OH name matches OH location • No dots (.) allowed (inventory limitation) 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 30 Central Inventory <HOME NAME=“18_3_0" LOC="/u01/app/oracle/product/18_3_0" TYPE="O" IDX="2"> • 1 to 1 mapping: easy to code • 18c read-only OH uses the name as subdir in `orabasehome`: consistent naming!
  30. 30. Golden images 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 31 Golden Images Store 18_3_0.zip 18_4_0.zip 18_3_1.zip SERVER1 18_3_0.zip 18_3_1.zip SERVER2 18_3_1.zip SERVER3 18_4_0.zip Golden Image Metadata Name Path ------- ----------------- 18_3_0 /store/18_3_0.zip 18_3_1 /store/18_3_1.zip 18_4_0 /store/18_4_0.zip
  31. 31. Image store Typical workflow 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 33 SERVER1 18_3_0 SERVER2… First Install
  32. 32. Image store Typical workflow 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 34 18_3_0.zip SERVER1 18_3_0 SERVER2… $ ohctl -c -n 18_3_0
  33. 33. Image store Typical workflow 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 35 18_3_0.zip SERVER1 18_3_0 SERVER2… $ ohctl -i 18_3_018_3_0
  34. 34. Image store Typical workflow 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 36 18_3_0.zip SERVER1 18_3_0 SERVER2… $ ohctl -i 18_3_0 -n 18_4_0 18_3_0 18_4_0
  35. 35. Image store Typical workflow 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 37 18_3_0.zip SERVER1 18_3_0 SERVER2… Manual patching 18_3_0 18_4_0
  36. 36. Image store Typical workflow 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 38 18_3_0.zip SERVER1 18_3_0 SERVER2… $ ohctl -c 18_3_0 18_4_0 18_4_0.zip
  37. 37. Image store Typical workflow 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 39 18_3_0.zip SERVER1 18_3_0 SERVER2… $ ohctl -i 18_4_0 18_3_0 18_4_0 18_4_0.zip 18_4_0
  38. 38. What it actually does Creation workflow • Copy the Home • Clean it up • Create the zip • Store the zip in the repo • Store metadata Installation workflow • Security checks • Create new path • Unzip from the store • Attach home • Relink (opt. with RAC) • Run setasmgid • Other stuff (e.g. TNS) Effective Oracle Home Management #UKOUG_Tech18 4005.12.2018
  39. 39. Effective Oracle Home Management #UKOUG_Tech18 4105.12.2018 DEMO
  40. 40. Further development at CERN 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 42
  41. 41. Read-only Oracle Home $ roohctl –enable Enabling Read-Only Oracle home. Update orabasetab file to enable Read-Only Oracle home. Orabasetab file has been updated successfully. Create bootstrap directories for Read-Only Oracle home. Bootstrap directories have been created successfully. Bootstrap files have been processed successfully. Read-Only Oracle home has been enabled successfully. Check the log file /u01/app/oracle/cfgtoollogs/roohctl/roohctl- 180822AM102911.log. 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 43
  42. 42. AGILE INFRASTRUCTURE SERVER Insertion in the Agile Infrastructure 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 44 Image Store Scripts REST client Oracle Home
  43. 43. Grid Infrastructure Installation • Puppet + Rundeck for automatic GI install • Patched Grid Infrastructure versions beside the first one https://www.markusdba.net/?p=294 • Out-of-place patching with 18c independent local-mode automaton 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 45 $ rhpctl move gihome –sourcehome old_path -destinationhome new_path
  44. 44. Independent Local-mode Automaton (18c) 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 46 # $GI_HOME/srvctl enable volume -volume GHCHKPT -diskgroup mgmt # $GI_HOME/srvctl enable filesystem -volume GHCHKPT -diskgroup mgmt # $GI_HOME/srvctl start filesystem -volume GHCHKPT -diskgroup mgmt $ $GI_HOME/srvctl start rhpserver Local Resources --------------------------------------------------------------- ora.MGMT.GHCHKPT.advm ora.mgmt.ghchkpt.acfs ora.helper Cluster Resources --------------------------------------------------------------- ora.rhpserver
  45. 45. Independent Local-mode Automaton (18c) • Damn-easy move of GI HOME • Automatic resource relocation • Enable Grid Infrastructure Golden Images • Patching (apply and rollback) is done automatically 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 47
  46. 46. Back to the demo… 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 48
  47. 47. Questions? 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 49
  48. 48. Thank you! Ludovico Caldara Computing Engineer IT Department, CERN ludovico.caldara@cern.ch @ludodba 05.12.2018 Effective Oracle Home Management #UKOUG_Tech18 50

×