Oracle Automatic Storage Management has proven to be one of the most widely adopted new features in Oracle Database 10g and it has been dramatically improved in the later 11g releases. This presentation will explain what changes are solved by ASM, how these challenges are solved, what barriers there are to ASM adoptions, and how 11g Release 2 addresses these barriers.

1. Oracle ASM 11g
The Evolution
Alex Gorbachev
RMOUG Training Days 2010
Denver, 18-Feb-10

2. Alex Gorbachev
• CTO, The Pythian Group
• Blogger
• OakTable Network member
• Oracle ACE Director
• BattleAgainstAnyGuess.com
• Vice-president, Oracle RAC SIG
2 © 2009/2010 Pythian

3. Why Companies Trust Pythian
• Recognized Leader:
• Global industry-leader in remote database administration services and consulting for Oracle,
Oracle Applications, MySQL and SQL Server
• Work with over 150 multinational companies such as Forbes.com, Fox Interactive media, and
MDS Inc. to help manage their complex IT deployments
• Expertise:
• One of the world’s largest concentrations of dedicated, full-time DBA expertise.
• Global Reach & Scalability:
• 24/7/365 global remote support for DBA and consulting, systems administration, special
projects or emergency response
3 © 2009/2010 Pythian

4. ASM adoption poll
• How many use ASM?
• Plan to use?
• Plan NOT to use?
• Adoption rates amongst Pythian customers - very high
4 © 2009/2010 Pythian

5. Pre-ASM - simple file system on JBOD
• Resilience?
• multiplexing
• redo + controlfiles
Server
• hardware mirroring
/u01 /u02
• Scaleability?
• Manual datafile layout
• separate indexes and tables?
• Hardware striping
• Manageability?
Disk Disk
• poor
• resize? move?
5 © 2009/2010 Pythian

6. Pre-ASM - raw devices
Server
/dev/sdb /dev/sdc • One file => disk or partition
• Why raw devices?
• remove overhead of FS
• Later CIO, direct IO, etc
Disk Disk
6 © 2009/2010 Pythian

7. Pre-ASM - raw devices (cluster)
Server 1 • One file => disk or partition
• Why raw devices?
/dev/sdb /dev/sdc
• remove overhead of FS
• Later CIO, direct IO, etc
Disk Disk
• Clusters - shared storage
/dev/sdb /dev/sdc
Server 2
7 © 2009/2010 Pythian

8. Pre-ASM - volume manager
• FS or “raw device”
• Logical volume
• Volume groups
• Physical disks
8 © 2009/2010 Pythian

9. Pre-ASM - volume manager
• FS or “raw device”
• Logical volume
• Volume groups
• Physical disks
Storage box
8 © 2009/2010 Pythian

10. Pre-ASM - cluster VM and FS
• Cluster Volume Manager (CVM)
• Synchronizes volume metadata
• Still lots of manageability overhead
• 1 volume = one file
• Cluster File System (CFS)
• On top of CVM
• Synchronize file metadata & concurrent IO
• Network Attached Storage (NAS)
• FS and VM on storage box
• ZFS opens the whole new dimension in manageability
9 © 2009/2010 Pythian

11. Pre-ASM storage issues
10 © 2009/2010 Pythian

12. Pre-ASM storage issues
• Too complex
• 3rd party vendors
• Additional licensing cost
• Many groups involved
• Storage engineers
• SA’s
• DBA’s
• Loosing layout visibility
10 © 2009/2010 Pythian

13. Pre-ASM storage issues
• Too complex • Too simple
• 3rd party vendors • Doesn’t scale
• Not reliable
• Additional licensing cost
• Not flexible
• Many groups involved
• Storage engineers
• SA’s
• DBA’s
• Loosing layout visibility
10 © 2009/2010 Pythian

14. ASM introduction
• Simplify
• Fewer vendors involved
• Fewer layers
• Move many operations under DBA umbrella
• oh well, Storage Administrator role
• Cost efficient
• No need to license volume manager
• Cheap storage boxes are good for ASM
• Manageable
11 © 2009/2010 Pythian

15. ASM principles
• Organizing Oracle files on physical disks
• Mirroring
• reliability & failure tolerance
• Striping
• performance & scaleability
• Rebalancing
• manageability
12 © 2009/2010 Pythian

16. ASM striping 10g
13 © 2009/2010 Pythian

17. ASM striping 10g
13 © 2009/2010 Pythian

18. ASM striping 10g
13 © 2009/2010 Pythian

19. ASM striping 10g
13 © 2009/2010 Pythian

20. ASM striping 10g
13 © 2009/2010 Pythian

21. ASM striping 10g
13 © 2009/2010 Pythian

22. ASM striping 10g
file1
13 © 2009/2010 Pythian

23. ASM striping 10g
file1 file2
13 © 2009/2010 Pythian

24. ASM striping 10g
• Allocation unit (AU) 1MB
• Coarse striping - whole AU
• Fine striping - 128K inside AU
file1 file2
13 © 2009/2010 Pythian

25. ASM striping 10g
• Allocation unit (AU) 1MB
• Coarse striping - whole AU
• Fine striping - 128K inside AU
• One extent = 1 AU
file1 file2
13 © 2009/2010 Pythian

26. ASM striping 10g
• Allocation unit (AU) 1MB
• Coarse striping - whole AU
• Fine striping - 128K inside AU
• One extent = 1 AU
• Random striping -> equal distribution
• No auto-magic re-layout based on performance
file1 file2
13 © 2009/2010 Pythian

27. ASM rebalancing
14 © 2009/2010 Pythian

28. ASM rebalancing
14 © 2009/2010 Pythian

29. ASM rebalancing
14 © 2009/2010 Pythian

30. ASM rebalancing
14 © 2009/2010 Pythian

31. ASM rebalancing
14 © 2009/2010 Pythian

32. ASM rebalancing
14 © 2009/2010 Pythian

33. ASM rebalancing
• Add / remove disks
• Migrate to another storage
• Extend / shrink
• asm_power_limit - rebalancing power/overhead
• No mirroring (external redundancy)
• planned maintenance
14 © 2009/2010 Pythian

34. ASM mirroring
15 © 2009/2010 Pythian

35. ASM mirroring
15 © 2009/2010 Pythian

36. ASM mirroring
15 © 2009/2010 Pythian

37. ASM mirroring
15 © 2009/2010 Pythian

38. ASM mirroring
15 © 2009/2010 Pythian

39. ASM mirroring
15 © 2009/2010 Pythian

40. ASM mirroring
15 © 2009/2010 Pythian

41. ASM mirroring
15 © 2009/2010 Pythian

42. ASM mirroring
15 © 2009/2010 Pythian

43. ASM mirroring
• Primary extent & mirror extent
• Failure tolerance
• two or three way mirroring
15 © 2009/2010 Pythian

44. ASM failure groups
• Group of disks can fail at once
• Mirror extents between failure groups
SAN 1 SAN 2
16 © 2009/2010 Pythian

45. ASM failure groups
• Group of disks can fail at once
• Mirror extents between failure groups
SAN 1
16 © 2009/2010 Pythian

46. ASM failure groups
• Group of disks can fail at once
• Mirror extents between failure groups
SAN 1
16 © 2009/2010 Pythian

47. ASM instance
• Special kind of Oracle instance
• Clustered in RAC
• ASM disks and diskgroups
• ASM extent map
• Rebalancing operations
• Must be started before database instance
17 © 2009/2010 Pythian

48. ASM 10g issues
• ASM upgrade requires full RAC outage
• Fixed 1MB allocation unit
• Overhead with multi-TB databases
• Recovery from failure is inefficient
• Extended clusters read IO is inefficient
• Limited FS functionality and tools
• No snapshot-like functionality
18 © 2009/2010 Pythian

49. ASM 11gR1 - diskgroup compatibility
• Default
SQL> select name, compatibility, database_compatibility
from v$asm_diskgroup;
NAME COMPATIBIL DATABASE_C
---- ---------- ----------
DG1 10.1.0.0.0 10.1.0.0.0
• New features
SQL> ALTER DISKGROUP dg1 SET ATTRIBUTE 'compatible.asm'='11.1';
Diskgroup altered.
SQL> ALTER DISKGROUP dg1 SET ATTRIBUTE 'compatible.rdbms'='11.1';
Diskgroup altered.
SQL> select name, compatibility, database_compatibility
from v$asm_diskgroup;
NAME COMPATIBIL DATABASE_C
---- ---------- ----------
DG1 11.1.0.0.0 11.1.0.0.0
19 © 2009/2010 Pythian

50. ASM 11g - rolling updates for RAC
ALTER SYSTEM START ROLLING MIGRATION TO
11.2.0.0.0;
• Limited services from ASM
• Normal database operation
• No diskgroup configuration changes
• Mount / unmount diskgroups
ALTER SYSTEM STOP ROLLING
MIGRATION;
© 2009/2010 Pythian

51. ASM 11gR1 - variable AU size
• Allocation Unit - 1/2/4/.../64 MB
• 10g - 1MB (_asm_ausize)
• Data sits closer together and can be read in bigger chunks
• Can improve sequential IO
• Reduces striping over SAN striped LUN’s
• Configuring Oracle ASM hidden parameters for EVA8000, HP
Knowledge Brief
• Reduces SGA for metadata for large files
• 11g - can be done per diskgroup
© 2009/2010 Pythian

52. ASM 11gR1 - variable extent size
• 10g
• 1 extent = 1 AU • 1 TB = 1+ mil. extents
• 11g
• 1-20,000 extents • 1MB AU:
1 extent = 1 AU • 1 TB = 53,572 extents
• 20,001 - 40,000 • 64MB AU:
1 extent = 8 AU’s • 1 TB = 16,384 extents
• 40,001 - ... • 100 TB = 66,788 extents
1 extent = 64 AU’s
22 © 2009/2010 Pythian

53. ASM 11gR1 - recovery from failures
• Fast Mirror Resync
• OFFLINE disks
• disk_repair_time attribute
• ASM extent change tracking
• Suitable for transient failures and maintenance
• Fast Rebalancing in restricted mount
• Rebalancing => many lock/unlock extent map
• Restrict mode rebalancing => no locks
• Restricted mounted DG => service outage
© 2009/2010 Pythian

54. ASM 11gR1 - Preferred Mirror Read
• For extended clusters
• Storage mirrored across 2 or 3 datacenters
• 10g
• Read is first done on primary extent
• 11g
• Preferred read failure groups (to local disks)
• asm_preferred_read_failure_groups in init.ora
© 2009/2010 Pythian

55. ASM Dynamic Volume Manager (ADVM)
• Clustervolume manager
• Can place any filesystem on top of an ASM Volume
• Oracle ACFS or third-party like ext3
• Unix/Linux & Windows
• Dynamically resizable volumes
• Leveraging all standard ASM features (like mirroring)
• Extent size 64MB - allocation unit
• Number of columns - 1 to 8 configurable
• Striping across columns
• Dirty Region Logging (DRL)
25 © 2009/2010 Pythian

56. ADVM on Linux
Loadable kernel modules
[root@epa01 init.d]# ./acfsload start -s
[root@epa01 init.d]# lsmod | grep ora
oracleacfs 787460 0
oracleadvm 177792 0
oracleoks 226656 2 oracleacfs,oracleadvm
[root@epa01 init.d]#
26 © 2009/2010 Pythian

57. ADVM: Creating a volume
ASMCMD & SQL Interfaces (+GUI Tools: ASMCA & OEM)
ASMCMD> volcreate -G DG_A1 -s 50M DV_A2
Volume Name: DV_A2
Volume Device: /dev/asm/dv_a2-354
State: ENABLED
Size (MB): 256
Resize Unit (MB): 256
Redundancy: UNPROT
Stripe Columns: 4
Stripe Width (K): 128
Usage:
Mountpath:
SQL> ALTER DISKGROUP DG_A1 ADD VOLUME DV_A2 SIZE 50M;
27 © 2009/2010 Pythian

58. ADVM in the OS
OS devices
[oradb@epa01 ~]$ ls -l /dev/asm/
total 0
brwxrwx--- 1 root dba 252, 12289 Nov 16 02:15 dv_a1-24
brwxrwx--- 1 root dba 252, 111617 Nov 16 02:15 dv_main_db-218
[oradb@epa01 ~]$
Dynamically resizable
ASMCMD> volresize -G DG_A1 -s 400M DV_A2
28 © 2009/2010 Pythian

59. ADVM: creating filesystems
# /sbin/mkfs -t ext4 /dev/asm/dv_a3-461
# /sbin/mkfs -t acfs -b 4k /dev/asm/dv_a3-461
mkfs.acfs: version = 11.2.0.1.0.0
mkfs.acfs: on-disk version = 39.0
mkfs.acfs: volume = /dev/asm/dv_a3-461
mkfs.acfs: volume size = 268435456
mkfs.acfs: Format complete.
# mount -t acfs /dev/asm/dv_a3-461 /a03
# df –h /a03
Filesystem Size Used Avail Use% Mounted on
/dev/asm/dv_a3-461 256M 37M 220M 15% /a03
29 © 2009/2010 Pythian

60. Oracle ASM Cluster Filesystem (ACFS)
• General purpose cluster
filesystem
• Journaling, variable extent-based
• Windows/Linux/Unix
• Oracle homes, dumps, traces,
export files and etc - *anything*
• Native ASM files (i.e. no datafiles,
Diagram from Oracle ASM 10gR2 Documentation
controlfiles and etc) unsupported
• Direct path IO or cached • ASM tools
• Dynamic resize SQL/ASMCMD/ASMCA/OEM
• Mount registry (like /etc/fstab)
• Standard OS commands
mkfs / mount / fsck / fuser / cp / chmod / chown / ...
30 © 2009/2010 Pythian

61. ACFS Snapshots
• Filesystem technology
• No snapshots for ASM diskgroups and ADVM volumes
• Read-only
• Space-efficient
• Initially sparse with copy-on-write technology
• Up to 63 snapshots per filesystem
• Access via .ACFS/snaps hidden directory
• CLI acfsutil snap and OEM support
31 © 2009/2010 Pythian

62. ASM 11gR2 - “small” enhancements
• ASM supports placement of OCR and voting disks
• Intelligent Data Placement (hot/cold -> outer/inner)
• Disk Group rename
• renamedg utility
• ASMCMD enhancements
• New ASMCA tool, OEM support
• Access control for ASM files (owner/group/others)
• 4K sector-size for diskgroups
32 © 2009/2010 Pythian

63. ASM Overhead?
• ASM is not in the IO code path for Oracle Database
• ASM only operates extent maps and disk headers
• Database -> thousands IO’s per second vs ASM -> few IO’s per minute
• Additional ASM instance - memory and processes but not
much
• Database instance keeps ASM extent maps in SGA
• Mirroring generates additional write IO
• ADVM - another layer
• ACFS - yet another layer
33 © 2009/2010 Pythian

64. What’s still missing?
34 © 2009/2010 Pythian

65. What’s still missing?
• Everything ZFS has
34 © 2009/2010 Pythian

66. What’s still missing?
• Read-write ACFS snapshots / clones
• ASM diskgroup level snapshots for database files
• ADVM snapshots (do we need them?)
• ACFS or ADVM compression
• Diskgroup mirroring conversion
• Split mirror functionality
• De-duplication capabilities
• Still need multipathing
• Proper throttling of rebalancing operations
• Is ADVM and ACFS are production ready?
35 © 2009/2010 Pythian

67. What’s next?
• Book draw
• Leave me your cards or email+phone
Q&A
Email me - gorbachev@pythian.com
Read my blog - http://www.pythian.com
Follow me on Twitter - @AlexGorbachev
Join Pythian fan club on Facebook & LinkedIn
36 © 2009/2010 Pythian