TSM for Virtual Environments Jim Smith, Software Architect, Tivoli Storage Manager

1. Tivoli Storage Manager
Data Protection for VMware
Jim Smith
TSM Architect
smithjp@us.ibm.com
May 24, 2011
© IBM Svenska AB

2. Disclaimer
• The information on the new product is intended to outline
our general product direction and it should not be relied
on in making a purchasing decision.
• The information on the new product is for informational
purposes only and may not be incorporated into any
contract.
• The information on the new product is not a commitment,
promise, or legal obligation to deliver any material, code
or functionality.
• The development, release, and timing of any features or
functionality described for our products remains at our
sole discretion.
2

3. Agenda – Session 1
• Backup in virtual environments
– in-guest, on-host, and off-host
• Why you should be excited about vStorage APIs for Data
Protection
– VMware Consolidated Backup vs. vStorage APIs
• TSM Solution Overview
• Potential Future Work Items
3

4. Agenda – Session 2
• Backup
– How VMware backup data is stored in nodes and filespaces
– Virtual vs. Physical deployment
– Incremental backup
– Versioning and Policy
– Scheduling
– Tape considerations
– Permissions
– Deployment considerations
• Recovery
– File-level recovery options
• Reporting
• Demo
4

5. SESSION 1
5

6. Analyst Predictions
How do you manage data protection and recovery in your virtual servers?
Sources – Gartner, IDC, ESG
6

7. VMware Infrastructure
C: D:
Guest
Machines
vCenter
Host
Machines
(ESX, ESXi)
VMFS VMDK NAS/local
SAN
TSM Storage
datastore datastore datastore
Hierarchy
RDM
TSM Server
7

8. In-Guest Backup Philosophy: treat virtual
machines as physical
machines
TSM Agents placed in each
Guest
Machines guest; traditional backup
over LAN
a vCenter
Host Cumulative Result: Individual files
Machines backup load stored in TSM server for
(ESX, ESXi) on host hard operational recovery
to measure
LA
N
on
ly
VMFS NAS/local a
files
SAN
TSM Storage
datastore datastore datastore
Hierarchy
This method is generally
supported by TSM for
VMware ESX/ESXi, Microsoft
Hyper-V, Linux KVM
TSM Server
8

9. On-Host Backup Philosophy: treat hypervisor
as general purpose OS
Optional: manual
procedure to backup
Guest data from a snapshot;
Machines no integrated solution
a for making data
consistent
vCenter
ESX only;
VMware TSM Linux agent Result: Individual files
Host representing virtual disks and
moving away placed on ESX host;
Machines
from ESX user selectively backs control files stored in TSM
(ESX, ESXi)
up data stores at server for DR of virtual
directory level machine
VMFS VMDK NAS/local LA
No
a
rS
AN vmdk(s)
SAN
TSM Storage
datastore datastore datastore Hierarchy
This method is generally
supported by TSM for
VMware ESX and Microsoft
Hyper-V
TSM Server
9

10. Off-Host Backup Philosophy: exploit access
to shared storage from
another host
Guest
Machines
a
vCenter
Backup Agent orchestrates
Result: blocks representing
Host snapshots of virtual
virtual machine stored in
Machines machines and moves data
(ESX, ESXi) server for DR of virtual
from snapshot to backup
machine
storage
VMDK
VMFS NAS/local a
SAN LAN or SAN
TSM Storage
datastore datastore datastore
Hierarchy
Backup Agent
Can also exploit hardware-based snapshots
as backup versions TSM Server
10

11. Virtual Machine Backup Methods
Summary
Method Philosophy Advantages Disadvantages
In-Guest Treat virtual machines as •Re-use existing •Typically cannot
physical machines processes and exploit SAN
products without •Hard to quantify
modification cumulative load on
host
On-Host Treat hypervisor as •Re-use existing •Not scalable to
general purpose OS processes and large number of vms
products with little per host
modification •Hypervisor vendors
•SAN access moving away from
general purpose OS
as hypervisor
Off-Host Exploit access to shared •Off-load backup •Need to modify
storage from another host from hosts backup products
•SAN access
11

12. How VCB Works - Backup
C: D: 1. VCB interacted
Guest with infrastructure to
Machines vCenter
vm1 take snapshot
(vSphere API)
Host
Machines
(ESX, ESXi)
3. TSM Backup-Archive
client moved flat files to
TSM Server
VMFS VMDK
SAN
TSM Storage
datastore datastore
Hierarchy
datastore
VC
B
Proxy = TSM + VCB +
2. VCB “clones” machine to disk staging (VDDK) Disk staging TSM Server
12

13. How VCB Works - Recovery
C: D: 1. TSM Backup-
Guest Archive client
Machines vCenter
vm1 restored flat files
from TSM Server
Host
Machines
(ESX, ESXi)
2. VMware Converter
used to convert
recovered files to
virtual machine
VMFS VMDK
SAN
TSM Storage
datastore datastore
Hierarchy
datastore
VC
B
Proxy = TSM + VCB +
Disk staging TSM Server
13

14. VCB Limitations
• Extra product to install, configure, and maintain separate
from backup product
• Needed extra disk space to “clone” copies of virtual
machine for backup
• VCB doesn’t do restore (B=Backup); need to use
VMware Converter
• Any new function provided in virtual disk development kit
(VDDK) may not be available through VCB
implementation
• VMware tried to move away from VCB in vSphere 4.1;
probably will not be supported in future releases
14

15. vStorage APIs for Data Protection -
Discovery and Mapping
Based on backup request,
Guest TSM agent queries host or
Machines vCenter to find virtual guests
that are backup candidates,
e.g., all guests on an ESX vCenter
host or all guests in a
specific folder
Host
Machines
(ESX, ESXi)
VMFS VMDK NAS/local
vSphere SDK
SAN
datastore datastore datastore
vStorage Server
(TSM agent) TSM Server
15

16. vStorage APIs for Data Protection -
Snapshot and Preparation
Guest
Machines 2. Virtual machines given
control for backup event to put vCenter
data in a consistent state either
through Microsoft Volume
Host
Shadow Copy Services (VSS),
Machines custom scripts or VMware
(ESX, ESXi) synch driver
VMFS VMDK NAS/local 1. TSM invokes snapshots of
vSphere guest machines through
SDK vSphere SDK; saves
SAN machine configuration
information
datastore datastore datastore
vStorage Server
3. VMware takes software snapshot, new .vmdk created, (TSM agent)
virtual machine passed backup complete event TSM Server
16

17. vStorage APIs for Data Protection –
Data Movement
TSM agent reads data
directly from snapshot
Guest VMDK and sends data to
Machines TSM server
Change block tracking vCenter
feature allows to backup
only changed blocks since
Host last backup OR only used
Machines blocks on first backup
(ESX, ESXi)
VMFS VMDK NAS/local
SAN Virtual Disk
Development Kit
(VDDK)
datastore datastore datastore
vStorage Server
Snapshot VMDK accessed via SAN,
LAN, or HotAdd transport
(TSM agent)
TSM Server
17

18. vStorage APIs for Data Protection -–
Termination
Guest
Machines A
vCenter
Host TSM Server contains block-
Machines level image of guest
(ESX, ESXi) machine(s) which can be
used for operational
recovery of individual files
or DR of entire machine
VMFS VMDK NAS/local
A
SAN
TSM Storage
datastore datastore datastore
Hierarchy
vStorage Server
(TSM agent)
Software snapshot removed TSM Server
18

19. vStorage APIs for Data Protection –
Recovery
1. Guest machine is
redefined by creating
Guest new machine with same
Machines properties (vSphere SDK)
2. VMDK files written back vCenter
to virtual machine
(VDDK)
Host
Machines
(ESX, ESXi)
VMFS VMDK NAS/local
1
vSphere
SDK
SAN
2
datastore datastore datastore
VDDK
vStorage Server
(TSM agent)
TSM Server
19

20. vStorage APIs for Data Protection
Comparison
vStorage APIs for Data
VCB Protection
• Extra product to install, configure, and • Underlying APIs shipped with backup
maintain separate from backup product
product • No staging area; data streamed
• Needed extra disk space to “clone” directly to/from VMware storage
copies of virtual machine for backup • Supports both backup and recovery;
• VCB doesn’t do restore (B=Backup); no need for VMware Converter
need to use VMware Converter • Can fully exploit underlying VDDK
• Any new function provided in virtual function
disk development kit (VDDK) may not • Not dependent on host (ESX) software
be available through VCB
implementation
• VMware tried to move away from VCB
in vSphere 4.1; probably will not be
supported in future releases
20

21. TSM Data Protection for VMware
Backup-Archive Client DP for VMware
• Full-VM backup and restore of • Uses B-A client for base
any guest OS using vStorage function at left
APIs for Data Protection • Full + Incremental backup
• Backup of used blocks only (also via CBT)
through VMware’s changed – B-A client provides this
block tracking function through license file
• Individual file recovery for
Windows and Linux guests
• Near-instant volume recovery
for Windows and Linux guests
21

22. Components of DP for VMware
TSM Backup-Archive Client
Guest responsibilities:
Machines
1. Full-VM backups and vCenter
Full-VM restores
Host 2. Main UI for DP for
Machines VMware
(ESX, ESXi)
3. Can be placed on
physical or virtual machine
VMFS VMDK 4. License file enables
incremental backup
capability
SAN
TSM Storage
datastore datastore
Hierarchy
datastore RDM
vStorage Server
TSM Server
22

23. Components of DP for VMware
TSM DP for VMware
Guest Recovery Agent
Machines
1. Mount for individual file vCenter
recovery and near-instant
volume restore
Host
Machines 2. Can be placed on
(ESX, ESXi) vStorage Server or inside
target virtual machine
VMFS VMDK
SAN
TSM Storage
datastore datastore
Hierarchy
datastore RDM
vStorage Server
TSM Server
23

24. POTENTIAL FUTURE
ENHANCEMENTS
24

25. Potential Future
vSphere Client Plug-in Enhancement
25

26. Potential Future
vSphere Client Plug-in Enhancement
Datastores
What is FlashCopy Manager?
IBM Tivoli Storage FlashCopy Manager software
provides fast application-aware backups and
restores using the snapshot technologies of IBM
storage systems.
What do you want to do?
Define a backup task ...
Run a restore ...
View current backup status ...
Understanding backups
Understanding restores
26

27. Potential Future
FlashCopy Manager – Full-VM Backup Enhancement
1. FCM initiates a software
snapshot of virtual guest volumes
VM1 (vSphere API)
C: D: Linux vStorage
Server
(physical or
virtual machine)
/a VM2
FlashCopy Manager
SAN ESXESXi Server
VMFS
Snaps
vmdk vmdk vmdk VM1
TSM Server
SAN Storage Subsystem
27

28. Potential Future
FlashCopy Manager – Full-VM Backup Enhancement
1. FCM initiates a software
snapshot of virtual guest volumes
VM1 (vSphere API)
C: D: Linux vStorage
2. FCM determines which LUN(s) Server
are associated with virtual
machines (physical or
virtual machine)
/a VM2
FlashCopy Manager
SAN ESXESXi Server
VMFS
LUN 2 LUN 1
Snaps
vmdk vmdk vmdk VM1
TSM Server
SAN Storage Subsystem
28

29. Potential Future
FlashCopy Manager – Full-VM Backup Enhancement
1. FCM initiates a software
snapshot of virtual guest volumes
VM1 (vSphere API)
C: D: Linux vStorage
2. FCMdetermines which LUN(s) Server
are associated with virtual
machines (physical or
3. FCM invokes hardware copy virtual machine)
/a VM2
services to create a persistent
snapshot copy of the LUN(s) hosting
the .vmdk and software snapshot FlashCopy Manager
SAN ESXESXi Server
VMFS
LUN 2 LUN 1
Snaps
vmdk vmdk vmdk VM1
LUN 2’ LUN 1’
Snaps
vmdk vmdk vmdk VM1
TSM Server
SAN Storage Subsystem
29

30. Potential Future
FlashCopy Manager – Full-VM Backup Enhancement
1. FCM initiates a software
snapshot of virtual guest volumes
VM1 (vSphere API)
C: D: Linux vStorage
2. FCMdetermines which LUN(s) Server
are associated with virtual
machines (physical or
3. FCM invokes hardware copy virtual machine)
/a VM2
services to create a persistent
snapshot copy of the LUN(s) hosting
the .vmdk and software snapshot FlashCopy Manager
SAN ESXESXi Server
VMFS
LUN 2 LUN 1 4. Hardware snapshot is
Snaps
persisted for use as
vmdk vmdk vmdk VM1 source for recovery
operation, software
snapshots are deleted.
LUN 2’ LUN 1’
Snaps
vmdk vmdk vmdk VM1
TSM Server
SAN Storage Subsystem
30

31. Potential Future
Unified Recovery for VMware Enhancement
vStorage API for
Data Protection
DP for FlashCopy
VMware or Manager
Common UI using
VMware vSphere
Client plug-in
Hardware
Snapshot
Tivoli Storage Manager
Storage Pool
31

32. DP for VMware – Official Wiki Page
http://www.ibm.com/developerworks/wikis/display/tivolistoragemanager/IBM+Tivoli+Storage+Manager+for+Virtual+Environments
32

33. SESSION 2
33

34. Disclaimer
• The information on the new product is intended to outline
our general product direction and it should not be relied
on in making a purchasing decision.
• The information on the new product is for informational
purposes only and may not be incorporated into any
contract.
• The information on the new product is not a commitment,
promise, or legal obligation to deliver any material, code
or functionality.
• The development, release, and timing of any features or
functionality described for our products remains at our
sole discretion.
34

35. Agenda – Session 2
• Backup
– How VMware backup data is stored in nodes and filespaces
– Virtual vs. Physical deployment
– Incremental backup
– Versioning and Policy
– Scheduling
– Tape considerations
– Permissions
– Deployment considerations
• Recovery
– File-level recovery options
• Reporting
• Demo
35

36. Reference Deployment
dcsmc –
node=vmbldesx1 – vm1 vm2 vm3
asnode=dc1
dcsmc – vm4 vm5 vm6
node=vmbldesx2 –
asnode=dc1
NODENAME DC1
dcsmc –node=vmep1
Datacenter = DC1 –asnode=dc2 vm10 vm11 vm12
vm13 vm14 vm15
dcsmc –node=vmep2
–asnode=dc2
NODENAME DC2
vStorage Server TSM Server
(TSM agent)
Each data mover is a proxy Each data mover stores data
agent which moves data on behalf of a proxy target
from a ESX host and has its which represents a
Datacenter = DC2 own node name datacenter.
36

37. Data Paths
•Transport is independent of Backup
Path
•VDDK picks best transport in order of
SAN, Hotadd, nbdssl, nbd
Guest
Machines •Can override VMware transport with
vmvstortransport Backup-Archive
client option
•Hotadd transport not available for
physical vStorage Server
Host •SAN Backup Path not available when
Machines using virtual vStorage Server
(ESX, ESXi) TSM Storage
Hierarchy
VMFS VMDK
Transport Backup Path
Datastore -> vStorage vStorage Server -> TSM Server
SAN
Servecr
datastore datastore
SAN, SAN
Hotadd, (LAN-free),
nbdssl, TCP/IP
nbd
vStorage Server TSM Server
(TSM agent)
37

38. Virtual vs. Physical Deployment
Physical vStorage Server Virtual vStorage Server
• Off-loads backup • Backup workload on host
workload from host • Supports Hotadd
• Supports IP and SAN transport
data path to TSM Server • Only supports IP data
storage path to TSM Server
• No client-side data storage
deduplication • Client-side data
deduplication
38

39. Incremental Backup
•Current model =
traditional full +
incremental
Guest Disk Storage Pool
Machines
•Recovery = restore
full + incremental in
chronological order,
0
1
2
1
2
e.g., D1 + D2 + D3 + D4
Host 3 3
4
Machines •VMware Change
(ESX, ESXi) Day 1 (Full) Day 3 Block Tracking (CBT)
used to track incr.
changes
2
3 1
4 2
VMFS VMDK •Must use VMware
Day 2 HW 7 for CBT
Day 4
SAN
datastore datastore
TSM Storage
Hierarchy
39

40. Versioning
• Version defined as the set of 1 full
+ supporting incremental backups
• Honors VEREXISTS and
RETAINEXTRA parameters
• Example at left – VEREXISTS=2;
third full backup will cause the first
S M T W T F S
week of backups to be expired
F I I I I I • No concept of deleting/expiring
F I I I I I backup versions if a virtual
F machine is excluded from backup
domain
• Client option VMMC to specify
non-default management class
40

41. Scheduling – Recommended Starting
Point
• Set backup domain for each vStorage Server (data mover) to protect
an ESX host
– Use Domain.vmfull Backup-Archive client option or via Preferences Editor
– Can specify “all vms”, Folder, Host, or individual virtual machine
– Can remove individual virtual machines from backup processing
• Full backups “batched” together
– All VM’s full backup during extended time (e.g., weekend)
– All VM’s incremental backup during weekdays
• Advantages:
– This is the easiest approach to configure/schedule
– Least number of TSM schedule definitions required (one weekend schedule +
one weekday schedule)
TSM Data Protection for VWware “Schedule Recommendations” Whitepaper:
https://www.ibm.com/developerworks/wikis/display/tivolistoragemanager/Recommendations+for+Scheduling+with+TSM+for+Virtual+Environments
41

42. Data Representation
File Types
•BITMAP – master
Guest MBLK0000
Disk Storage Pool bitmap for image (1)
Machines
•OVF = machine
MBLK0001
definition (1)
MBLK0002 •CTL/DAT = control
DISK 0
JOB1 JOB1 JOB1 + 128 MB data
MBLK0000.CTL MBLKnnnn.CTL BITMAP.DAT
Host
Machines
JOB2 JOB2 JOB2
(ESX, ESXi)
MBLK0000.CTL MBLK0003.CTL BITMAP.DAT
MBLKnnnn
…vmname.OVF
JOB1. JOB1 JOB2 = incr. backup
MBLK0000.DAT MBLKnnnn.DAT
(fewer CTL/DAT files)
VMFS VMDK JOB2 JOB2
…vmname.OVF
MBLK0000.DAT MBLK0003.DAT
SAN
datastore datastore
TSM Storage
Hierarchy
42

43. Tape Configuration
Disk Storage Pool = Meta-Data
Guest JOB1 JOB1 JOB1
Machines
MBLK0000
MBLK0000.CTL MBLKnnnn.CTL BITMAP.DAT
VMCTLMC
MBLK0001
JOB2 JOB2 JOB2
MBLK0000.CTL MBLK0003.CTL BITMAP.DAT
MBLK0002
DISK 0
Host
Machines
(ESX, ESXi)
MBLKnnnn Tape Storage Pool = Data
JOB1. JOB1
…vmname.OVF
MBLK0000.DAT MBLKnnnn.DAT VMMC
VMFS VMDK
JOB2 JOB2
…vmname.OVF
MBLK0000.DAT MBLK0003.DAT
SAN
datastore datastore
Best Practice Recommendation when using disk / tape - Set
the MIGDELAY (e.g. 10 days) parameter for the disk storage pool TSM Storage
to enable the majority of mount requests to be satisfied with Hierarchy
snapshots that are on disk.
43

44. VMware Permissions
Windows
Active
Directory
Linux
User
-
or + Set of Privileges Object = Permission
>
Group
•DP for VMware can perform backup and recovery with restricted set of VMware permissions
•Recommended to use Active Directory to manage users
•For recovery, need permissions at datacenter which encompasses datastore, virtual machines, and
network
•In general, do not need permissions to interact with virtual machine (e.g., Power on, Power off)
•Technote describing minimal permissions:
https://www-304.ibm.com/support/docview.wss?uid=swg21497028
44

45. Summary – Backup-Archive Client
Preference Editor for VMware
45

46. Other Deployment Considerations
• Virtual machines should be migrated to
hardware version 7 to take advantage of change
block tracking/incremental backup
• Plan on 1 – 1.5 processors per instance of the
Backup-Archive client (1.5 if you are planning on
using TSM client-side data deduplication)
46

47. Estimated Per-Process Throughput
Estimated Per-Process Throughput for Proxy Host Sizing
NOTE: These values are estimates intended for proxy host sizing only and are not intended to
be a guarantee of actual performance.
Deduplication? Low Estimate Medium Estimate High Estimate
No 65 GB/Hour 120 GB/Hour 170 GB/Hour
Yes 40 GB/Hour 60 GB/Hour 85 GB/Hour
Factors to consider when selecting an estimate:
• Number of concurrent backup processes within proxy
• Datastore I/O capacity
• SAN and LAN bandwidth speed and utilization
• TSM Server throughput capacity
• Deduplication and compression results
• Selection of secure transport (nbdssl)
• Physical vs. virtual vStorage Server (data mover)
47

48. RECOVERY
48

49. General Disk Terminology
KEY:
C: D: VMDK
Virtual representation of
physical disk as stored
in VMware data store
E: F: G:
disk
Physical disk or block
device or LUN as
viewed from VMware
/boot / guest operating system,
e.g., DISK 0 or /dev/sda
partition
/dvlp /test /prod Logical partition where
file system is mounted
49

50. Individual File Recovery – Windows Partition
(via Windows DP for VMware Mount)
Supported:
Windows 2003
Windows 2008
Windows XP
Windows Vista
vStorage Backup Server TSM
or Windows Guest Machine API
Software required: TSM Server
Windows DP for VMware Mount
TSM Storage
Hierarchy
X: (snapshot of E: at time t3)
3
C: D:
Desired partition -> E: F: G:
50

51. Individual File Recovery – Linux Partition (via
Linux DP for VMware Mount)
Software required: TSM
Supported:
Windows DP for VMware Mount API
RHEL 5.2-5.4
CygWin (provides SSH)
SLES 10 SP2
vStorage Backup Server
iSCSI target
or Windows Guest
(snapshot of /sdb at time t3)
Machine 3
TSM Server
SSH – TCP/IP
iSCS
I TSM Storage
Hierarchy
Linux Guest Machine
/boot /
Software required:
Linux DP for VMware Mount iSCSI initiator
iSCSI initiator, SSH, Perl, (/dpmount =
mdadm snapshot of /dvlp at time t3)
3
/dvlp /test /prod
Desired partition ->
51

52. Individual File Recovery – Linux Disk (via
manual iSCSI Initiator) Supported:
Platforms
supported in
previous slides
TSM
API Best Effort Support
Several Linux
distributions and
vStorage Backup Server Windows 7 -
or Windows Guest Machine iSCSI target Search “TSM
(snapshot of /sdb at time t3)
3 LINUX ISCSI BEST
TSM Server EFFORT”
iSCS
I TSM Storage
Hierarchy
Linux Guest Machine
iSCSI initiator /boot /
(/dev/sdc =
snapshot of /dev/sdb at time t3)
3
User issues:
mount /dev/sdc1 /dpmount /dvlp /test /prod
Desired partition ->
52

53. Individual File Recovery – Linux Disk (via
manual iSCSI Initiator)
Windows Linux
53

54. Linux File Recovery - Comparison
Partition Mount Disk Mount
• Interface contained to Linux DP for • Two interfaces: Windows DP for
VMware Mount VMware Mount and system tools in
• In addition to Windows DP for VMware Linux guest
Mount, requires Cygwin (Windows), • Requires only Windows DP for
Linux DP for VMware Mount (Linux VMware package on Windows
guest), iSCSI initiator (Linux guest) machine; no Tivoli software installed in
• iSCSI configuration is not externalized Linux guest
to customer • Manual iSCSI target and initiatior
• Support for RHEL 5.2-5.4; SLES 11 configuration
• Support for major fs types (e.g., ext3, • Same support limitations as partition
reiserfs) mount
• No support for mounting LVM,
dynamic volumes, etc.
54

55. REPORTING
55

56. Reporting – Backup Summary
Activity log Messages 4141I -4144I and 4149I provide summary information on virtual machines processed
56

57. Reporting – Identifying Errors
Check activity log for other failures by virtual machine name
57

58. Reporting – Identifying Missed Backups
Check filespace table for virtual machines that don’t have current backups
58

59. Reporting – Other Notes
• In general, use “asnodename” for queries
• QUERY OCCUPANCY for storage used per vm
• Notes on summary records
– Summary record created for proxynode agent (not target or
“asnodename” like previous queries)
– Currently no filespace name in summary records
– Restore summary records cut for incremental backups
59

60. DEMO
60