SlideShare a Scribd company logo
1 of 21
Download to read offline
How can Hypervisors leverage
              Advanced Storage features?



Author:                        Shriram Pore, Solutions Architect, Calsoft Inc.
Presenter:                     Dr. Anupam Bhide, CEO, Founder Calsoft Inc.

2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.
Introduction

  Useful for storage vendors who are considering
   implementing hypervisor storage APIs like VAAI
  Understand how hypervisors interact with storage
   today
     Limitations in that interaction today

  Need for a standard that is both hypervisor and
   storage agnostic.
  Some Areas (that today’s hypervisor-specific

   standards do not cover)


 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.
                                                                          2
Virtual Environment Challenges
        Challenges

  Virtual environments (Hypervisors) with NAS/SAN
    arrays have high storage bandwidth usage
    (IP/FC/etc.) as compared to DAS
  SAN/NAS arrays have matured technologies such as
    snapshot, clone, server copy, range locking, etc.
  Hypervisors can leverage these matured technologies
   and improve storage / network utilization.
  However, hypervisors themselves have developed
   sophisticated storage virtualization layers

      Solution
 The goal here is to –
       Offload file/bulk-block operations to NAS/SAN arrays/servers to reduce storage bandwidth
           and increase I/O performance, storage utilization, etc.
  2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.
Hypervisors use of storage

  Classification – how hypervisors use storage
     Hypervisor using local disks (e.g. VMware VMFS on

      local disk)
     Hypervisor creating proprietary file system on SAN

      (e.g. VMware VMFS on a SAN array)
     Hypervisor using LUNs of SAN array in RDM mode
     Hypervisor using NAS storage box over NFS/CIFS
  Better integration can have big benefits in many

   cases
     E.g. Vendors have reported 99% savings in
      network bandwidth for cloning operations and 10x
      to 20x in efficiency
 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.
                                                                          4
Need for standard to optimize hypervisor
interaction with storage
                                                                                               Why the need?
                                                                                   • Storage is the biggest bottleneck for
                                                                                     hypervisor performance
                                                                                   • Many common hypervisor operations
                                                                                     can be optimized by delegating
                                                                                     operations to storage boxes
  Hypervisors         ESX(i)              Hyper-V              Xen                 • Storage vendors cannot afford to
                                                                                     conform to multiple hypervisor
                                                                                     standards for this delegation


                         SNIA interface       SNIA interface
                                                                                   • Standard needs to be both hypervisor
 Interface layer
                            for SAN              for NAS                             independent and storage independent
                                                                                   • Identify in-efficient file/ Block operations
                                                                           Identify on VMs in virtual environments

                                                                                  • Define set of SAN/ NAS primitives
                                                                                    (standard APIs) based on above
                                                                           Define   identification including capability
   Storage                                                                          exchange.

                               SAN                         NAS             Complia • Levels of compliance depend on interface
                                                                             nce     implementations for storage and
                                                                                     hypervisor

  2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.
SAN based Architecture


                                             Application                   Application
                                               (VM)                       (Web Services)


         Set of files (disk                            Virtual Disk Library
         metadata)

                                                           User Kernel
           Files and file                                  Boundary
           segments                        Hypervisor proprietary FS e.g. VMFS

              Logical                        Block Layer (Block Devices, logical
              Blocks                                      volume)

                                                  Physical SCSI Device Access
              Device
                                             Vendor Specific SCSI extensions on
              Blocks
                                                        SAN array

 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.
Representative
SAN Primitives and Use Cases
     SAN                                  Goal                                                     Use Cases
  Primitives
    Block            •    Avoid multiple WRITE                              •   Block Zero feature speeds up deployment of
   Zeroing           •    Use SCSI WRITE SAME                                   thick provision eager zeroed virtual disks
                                                                            •   Full Copy feature speeds up storage vMotion
                     •    Avoid multiple READ & WRITE                           and cloning of VMs
                          and network bandwidth to                          •   Hardware-Assisted Locking feature is useful
  Full Copy
                          copy complete VMDK                                    for operations that require VM locking for
                     •    use SCSI EXTENDED COPY                                power on/off etc. and cluster wide operations
  Hardware                                                                      like vMotion, storage vMotion etc.
                     •    BLOCK/EXTENT LEVEL locking                        •   Power On Storm (100s of VMs being powered
   Assisted
                     •    use SCSI ATOMIC TEST & SET                            up which are using same LUN) has huge
   Locking
                                                                                latency which is resolved by hardware
                     •    Errors to indicate soft and                           assisted locking
                          hard out of space conditions
     Thin            •    Pause VM in case of hard                          •   Dead Space Reclamation enables reclamation
 Provisioning             errors                                                of blocks from a thin-provisioned LUN on the
                     •    use UNMAP for Space                                   SAN based arrays
                          Reclamation
Offloaded Data Transfer to the arrays
                                                                                       (illustrations from VMware SAN VAAI and Hyper-V ODX)
   2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.
Types of VMware virtual disks


       Thick provisioned
              All backing storage immediately allocated
              Not zeroed immediately

       Thin provisioned
              Backing storage not fully allocated

       Thick provisioned with eager zeroing
              Like thick provisioned, but also zeroed immediately



 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.
                                                                          8
Efficiencies gained

     Block zeroing
        Storage vendor can lazily zero extent in background

        Can implement proprietary mechanisms to mark extent as
         zeroed out
        In thin provisioned LUNs, storage vendor can unmap the

         extent
     Full copy
        Doing the copy within the storage box is more efficient
        Furthermore, writeable snapshot(i.e. clone) technology can

         be used to do the full copy without actually copying blocks
        Dedupe performance can be improved by recognizing that
         two extents are just copies of each other

  2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.
                                                                           9
Efficiencies gained

      Atomic Test and Set
        Many VM operations such as powering on or off
          

        VMs require getting or releasing VM-specific lock.
        This requires either getting SCSI-2 or SCSI-3
        persistent group reservations – both inefficient.
       Storage vendors can optimize by implementing
        atomic test and set.
    UNMAP

       Thin provisioned LUNs can re-use deleted space




  2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.
                                                                           10
Integration of Hypervisors
and SAN capabilities
 Steps Involved

              • Identify in-efficient bulk-block operations on VMs in virtual environments/hypervisors
  Identify

              • Define set of SAN primitives (unused existing SCSI commands to be overloaded or new SCSI
                commands to be adopted) based on above identification including capability exchange (supported
  Define        primitives) - can be added to SNIA specifications for standardization


         • SAN vendors to implement SCSI commands to deliver desired functionality, which will use its own
 Impleme   technologies to achieve maximum performance
    nt


              • Hypervisor to invoke SCSI commands to leverage SAN capabilities
    Call

              • SAN arrays/servers to adopt proven/efficient technologies for increased performance and reduced
                usage of network bandwidth (features like Dynamic LUN Provisioning, Thin Provisioning, concurrent
  Adopt         provisioning, Space reclamation, Dynamic Snapshots, LUN migration, etc.)


   2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.
NAS based Architecture – Plugin Approach


                                                                          Application (Web
                                           Application (VM)
                                                                             Services)




                                                                      Virtual disk library    Vendor Specific NFS
          Set of files (disc               Virtual Disk Library
                                                                          plugin API         or Custom RPC plugin
          metadata)


                                                          User Kernel
                                                          Boundary                              Plugin Approach
                                                                                              • Plugin approach
            Files and file                     NAS proprietary FS over NFS/CIFS                 allows vendors to
            segments                                                                            use their own
                                                Block Layer (Block Devices, logical             communication
              Logical                                        volume)                            mechanism
              Blocks                                                                          • Many vendors use
              Device Blocks                         Physical SCSI Device Access                 unused NFS
                                                                                                commands


 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.
Representative
NAS Primitives and Use Cases
    NAS                             Goal                                                Use Cases
 Primitives
                  •    Monitor space utilization in                •    Thick provisioning is the normal standard in
                       sparse file and guarantee                        enterprise deployments of hypervisors
  File Space           adequate space for VMs                      •    With NAS file systems, POSIX lseek is the only
 Reservation •         Allows future I/O ops of VM                      way for hypervisors to efficiently create large
                       NOT to fail due to space                         files – but no backing storage will be created
                       unavailability                                   with lseek
                                                                   •    File Space Reservation and Extended Stats
                  •    Clone VMs in faster and
                                                                        features enable hypervisors to quickly reserve
 File Cloning          storage space efficient way
                                                                        space on NAS server over NFS/CIFS to create
   (Full and      •    Also can be used for faster &
                                                                        thick provisioned virtual disk for a VM
     Lazy)             storage efficient snapshots &
                       restores (VM snapshot)
                                                                   •    VM cloning/deployment and storage vMotion
                  •    Retrieve accurate              space             are examples of operations that are offloaded
                       utilization of VMs - data that                   to NAS server.
                       cannot be retrieved using
  Extended
                       NFS calls                                    •   Lazy clones typically used for instant
  Statistics
                   • Monitor space utilization in                       operations. E.g. in VMs Linked Clones and VDI
                       sparse file and guarantee                    •   Full Clones – used for operations across data
   2012 Storage Developer Conference. space for VMs
                       adequate © Calsoft Inc. All Rights Reserved.     stores
Efficiencies gained

    File Space Reservation
       Thick provisioned semantics are hard to achieve in NAS due

         to NFS protocol limitations
    Cloning – Full and Lazy
       Doing the cloning within the NAS box is more efficient

       Furthermore, file level writeable snapshot(i.e. clone)

         technology can be used to do the lazy copy without
         actually copying blocks
       Dedupe performance can be improved by recognizing that

         two extents are just copies of each other
    Extended Attributes
       Understand exactly how much space a lazy clone uses
       Understand how much space a VMDK file is actually using


  2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.
                                                                           14
Additional Observations

    Delegate creation of VM snapshots to the storage
     box. These snapshots need to be hypervisor-
     consistent.
       For NAS, it is inefficient for hypervisors to do file

        snapshots
       Flash-based storage boxes do not write in place
        and can do snapshots much more efficiently.
       VDI applications could use the writeable

        snapshots(clone) already provided by most
        storage vendors

 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.
                                                                          15
Additional Observations

  With storage box provided snapshots, backups can
   be taken without involving the server
  Primitive for disaster recovery that can use
   replication or remote mirroring (similar to SRA/SRM)
  UNMAP for even thick provisioning for flash based

   SAN arrays – helps reduce the size of book-keeping
   data structures.
  Storage arrays can demultiplex IO streams from

   hypervisor and provide VM level IO stream QoS - vVol



 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.
                                                                          16
Microsoft Offloaded Data Transfer
-ODX
  Available Windows Server 8 onwards
  Works for both SAN storage(VHDs) and NAS using

   SMB protocol
  Provides Full copy semantics for both SAN and NAS

  Protocol works as follows:

     Send offload read request to source device
     Return with a token
     Send offload write request with token to

      destination device


 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.
                                                                          17
Author Biography




 Shriram Pore - Solutions Architect, Calsoft Inc.
  A veteran of storage industry
  More than 11 years of experience in architecting and developing products

  Key strength lies in quickly understanding product requirements and translating

    them into architectural and engineering specs for implementation.
  Architected, designed and implemented solutions for NAS-VMware integrated
    Backup-Recovery and Clone of VM
  Led FS and replication teams on the file-server and also led the CORE component

    team from system management perspective (configuration, provision, and
    manage various facilities of file-server).
  Master Of Computer Science from India, Pune University

  Bachelor of Computer Science from India, Pune University



 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.
Presenter Biography




  Dr. Anupam Bhide – CEO, Co-Founder, Calsoft Inc.
   Storage industry veteran
   More than 21 years of industry experience

   Senior Architect in the RDBMS development group at Oracle Corp, designed

     some of the key features of Oracle8
   Founder-member of the DB2/6000 Parallel Edition team at IBM Research
     Center
   Visiting Faculty at University of California – Berkeley

   Ph.D. in Computer Science, University of California-Berkeley

   BS in Computer Science: Indian Institute of Technology, Bombay and MS:
     University of Wisconsin-Madison



 2012 Storage Developer Conference. © Calsoft Inc.. All Rights Reserved.
                                                                               19
Thank You



                                     Questions & Answers


                                                      Contact info
                                                       Dr. Anupam Bhide
                                                CEO, Co-Founder, Calsoft Inc.
                                                 Email: anupam@calsoftinc.com
                                                   Phone: +1 (408) 834 7086
                                                      Twitter: @Calsoftinc



 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.
                                                                                20
Integration of Hypervisors
and NAS capabilities

 Steps Involved

            • Identify in-efficient file-operations on VMs in virtual environments
 Identify

        • Define set of NAS primitives (standard APIs) based on above identification including capability exchange
 Define   (supported primitives) This may be added to SNIA specs for standardization


            • Provide a framework to implement APIs as plugin on hypervisors
 Provide


            • NAS vendor to implement the plugin to use its technologies to achieve maximum performance
 Implem
   ent

            • Hypervisor to call into the plugin via defined framework APIs to leverage NAS capabilities
   Call

        • Modify NFS/CIFS to accommodate NAS primitives desired, else provide a new interface which would be easily
 Modify integrated into the plugin with no security compromise

       • NAS server to adopt proven/efficient technologies for increased performance and reduced usage of network
 Adopt   bandwidth (e.g.: file-clone, server-copy, FS snapshots, Thin Versioning, de-dupe, etc.)
   2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.

More Related Content

What's hot

VMworld 2015: Just Because You COULD, Doesn’t Mean You SHOULD – vSphere 6.0 A...
VMworld 2015: Just Because You COULD, Doesn’t Mean You SHOULD – vSphere 6.0 A...VMworld 2015: Just Because You COULD, Doesn’t Mean You SHOULD – vSphere 6.0 A...
VMworld 2015: Just Because You COULD, Doesn’t Mean You SHOULD – vSphere 6.0 A...VMworld
 
VMworld 2015: VMware NSX Deep Dive
VMworld 2015: VMware NSX Deep DiveVMworld 2015: VMware NSX Deep Dive
VMworld 2015: VMware NSX Deep DiveVMworld
 
VMware VSAN Technical Deep Dive - March 2014
VMware VSAN Technical Deep Dive - March 2014VMware VSAN Technical Deep Dive - March 2014
VMware VSAN Technical Deep Dive - March 2014David Davis
 
Hybrid cloud overview and VCF on VxRAIL
Hybrid cloud overview and VCF on VxRAILHybrid cloud overview and VCF on VxRAIL
Hybrid cloud overview and VCF on VxRAILDavid Pasek
 
VMware Site Recovery Manager (SRM) 6.0 Lab Manual
VMware Site Recovery Manager (SRM) 6.0 Lab ManualVMware Site Recovery Manager (SRM) 6.0 Lab Manual
VMware Site Recovery Manager (SRM) 6.0 Lab ManualSanjeev Kumar
 
VMworld 2015: Site Recovery Manager and Policy Based DR Deep Dive with Engine...
VMworld 2015: Site Recovery Manager and Policy Based DR Deep Dive with Engine...VMworld 2015: Site Recovery Manager and Policy Based DR Deep Dive with Engine...
VMworld 2015: Site Recovery Manager and Policy Based DR Deep Dive with Engine...VMworld
 
C3 Citrix Cloud Center
C3 Citrix Cloud CenterC3 Citrix Cloud Center
C3 Citrix Cloud CenterRui Lopes
 
VSAN – Architettura e Design
VSAN – Architettura e DesignVSAN – Architettura e Design
VSAN – Architettura e DesignVMUG IT
 
VMworld 2014: vCloud Hybrid Service Networking Technical Deep Dive
VMworld 2014: vCloud Hybrid Service Networking Technical Deep DiveVMworld 2014: vCloud Hybrid Service Networking Technical Deep Dive
VMworld 2014: vCloud Hybrid Service Networking Technical Deep DiveVMworld
 
VMware Advance Troubleshooting Workshop - Day 5
VMware Advance Troubleshooting Workshop - Day 5VMware Advance Troubleshooting Workshop - Day 5
VMware Advance Troubleshooting Workshop - Day 5Vepsun Technologies
 
Net1674 final emea
Net1674 final emeaNet1674 final emea
Net1674 final emeaVMworld
 
VMworld 2015: Networking Virtual SAN's Backbone
VMworld 2015: Networking Virtual SAN's BackboneVMworld 2015: Networking Virtual SAN's Backbone
VMworld 2015: Networking Virtual SAN's BackboneVMworld
 
VMware vSphere 6.0 - Troubleshooting Training - Day 1
VMware vSphere 6.0 - Troubleshooting Training - Day 1VMware vSphere 6.0 - Troubleshooting Training - Day 1
VMware vSphere 6.0 - Troubleshooting Training - Day 1Sanjeev Kumar
 
HBC8292 vCloud Air Recovery as a Service (RaaS) Deep Dive
HBC8292 vCloud Air Recovery as a Service (RaaS) Deep DiveHBC8292 vCloud Air Recovery as a Service (RaaS) Deep Dive
HBC8292 vCloud Air Recovery as a Service (RaaS) Deep Divedavehill99
 
VMware Advance Troubleshooting Workshop - Day 2
VMware Advance Troubleshooting Workshop - Day 2VMware Advance Troubleshooting Workshop - Day 2
VMware Advance Troubleshooting Workshop - Day 2Vepsun Technologies
 
VMworld 2015: Virtual Volumes Technical Deep Dive
VMworld 2015: Virtual Volumes Technical Deep DiveVMworld 2015: Virtual Volumes Technical Deep Dive
VMworld 2015: Virtual Volumes Technical Deep DiveVMworld
 
VMware Advance Troubleshooting Workshop - Day 3
VMware Advance Troubleshooting Workshop - Day 3VMware Advance Troubleshooting Workshop - Day 3
VMware Advance Troubleshooting Workshop - Day 3Vepsun Technologies
 
HBC9363 Virtualization 2.0 How the Cloud is Evolving the Modern Data Center
HBC9363 Virtualization 2.0 How the Cloud is Evolving the Modern Data CenterHBC9363 Virtualization 2.0 How the Cloud is Evolving the Modern Data Center
HBC9363 Virtualization 2.0 How the Cloud is Evolving the Modern Data Centerdavehill99
 

What's hot (20)

VMworld 2015: Just Because You COULD, Doesn’t Mean You SHOULD – vSphere 6.0 A...
VMworld 2015: Just Because You COULD, Doesn’t Mean You SHOULD – vSphere 6.0 A...VMworld 2015: Just Because You COULD, Doesn’t Mean You SHOULD – vSphere 6.0 A...
VMworld 2015: Just Because You COULD, Doesn’t Mean You SHOULD – vSphere 6.0 A...
 
VMworld 2015: VMware NSX Deep Dive
VMworld 2015: VMware NSX Deep DiveVMworld 2015: VMware NSX Deep Dive
VMworld 2015: VMware NSX Deep Dive
 
VMware VSAN Technical Deep Dive - March 2014
VMware VSAN Technical Deep Dive - March 2014VMware VSAN Technical Deep Dive - March 2014
VMware VSAN Technical Deep Dive - March 2014
 
Hybrid cloud overview and VCF on VxRAIL
Hybrid cloud overview and VCF on VxRAILHybrid cloud overview and VCF on VxRAIL
Hybrid cloud overview and VCF on VxRAIL
 
VMware Site Recovery Manager (SRM) 6.0 Lab Manual
VMware Site Recovery Manager (SRM) 6.0 Lab ManualVMware Site Recovery Manager (SRM) 6.0 Lab Manual
VMware Site Recovery Manager (SRM) 6.0 Lab Manual
 
VMworld 2015: Site Recovery Manager and Policy Based DR Deep Dive with Engine...
VMworld 2015: Site Recovery Manager and Policy Based DR Deep Dive with Engine...VMworld 2015: Site Recovery Manager and Policy Based DR Deep Dive with Engine...
VMworld 2015: Site Recovery Manager and Policy Based DR Deep Dive with Engine...
 
C3 Citrix Cloud Center
C3 Citrix Cloud CenterC3 Citrix Cloud Center
C3 Citrix Cloud Center
 
Xrm xensummit
Xrm xensummitXrm xensummit
Xrm xensummit
 
VSAN – Architettura e Design
VSAN – Architettura e DesignVSAN – Architettura e Design
VSAN – Architettura e Design
 
VMworld 2014: vCloud Hybrid Service Networking Technical Deep Dive
VMworld 2014: vCloud Hybrid Service Networking Technical Deep DiveVMworld 2014: vCloud Hybrid Service Networking Technical Deep Dive
VMworld 2014: vCloud Hybrid Service Networking Technical Deep Dive
 
VMware Advance Troubleshooting Workshop - Day 5
VMware Advance Troubleshooting Workshop - Day 5VMware Advance Troubleshooting Workshop - Day 5
VMware Advance Troubleshooting Workshop - Day 5
 
Net1674 final emea
Net1674 final emeaNet1674 final emea
Net1674 final emea
 
VMworld 2015: Networking Virtual SAN's Backbone
VMworld 2015: Networking Virtual SAN's BackboneVMworld 2015: Networking Virtual SAN's Backbone
VMworld 2015: Networking Virtual SAN's Backbone
 
VMware vSphere 6.0 - Troubleshooting Training - Day 1
VMware vSphere 6.0 - Troubleshooting Training - Day 1VMware vSphere 6.0 - Troubleshooting Training - Day 1
VMware vSphere 6.0 - Troubleshooting Training - Day 1
 
HBC8292 vCloud Air Recovery as a Service (RaaS) Deep Dive
HBC8292 vCloud Air Recovery as a Service (RaaS) Deep DiveHBC8292 vCloud Air Recovery as a Service (RaaS) Deep Dive
HBC8292 vCloud Air Recovery as a Service (RaaS) Deep Dive
 
VMware Advance Troubleshooting Workshop - Day 2
VMware Advance Troubleshooting Workshop - Day 2VMware Advance Troubleshooting Workshop - Day 2
VMware Advance Troubleshooting Workshop - Day 2
 
Hcx intro preso v2
Hcx intro preso v2Hcx intro preso v2
Hcx intro preso v2
 
VMworld 2015: Virtual Volumes Technical Deep Dive
VMworld 2015: Virtual Volumes Technical Deep DiveVMworld 2015: Virtual Volumes Technical Deep Dive
VMworld 2015: Virtual Volumes Technical Deep Dive
 
VMware Advance Troubleshooting Workshop - Day 3
VMware Advance Troubleshooting Workshop - Day 3VMware Advance Troubleshooting Workshop - Day 3
VMware Advance Troubleshooting Workshop - Day 3
 
HBC9363 Virtualization 2.0 How the Cloud is Evolving the Modern Data Center
HBC9363 Virtualization 2.0 How the Cloud is Evolving the Modern Data CenterHBC9363 Virtualization 2.0 How the Cloud is Evolving the Modern Data Center
HBC9363 Virtualization 2.0 How the Cloud is Evolving the Modern Data Center
 

Similar to Sdc 2012-how-can-hypervisors-leverage-advanced-storage-features-v7.6(20-9-2012)

What is coming for VMware vSphere?
What is coming for VMware vSphere?What is coming for VMware vSphere?
What is coming for VMware vSphere?Duncan Epping
 
Building a Distributed Block Storage System on Xen
Building a Distributed Block Storage System on XenBuilding a Distributed Block Storage System on Xen
Building a Distributed Block Storage System on XenThe Linux Foundation
 
Veritas Dynamic Multi-Pathing for VMware 6
Veritas Dynamic Multi-Pathing for VMware 6Veritas Dynamic Multi-Pathing for VMware 6
Veritas Dynamic Multi-Pathing for VMware 6Symantec
 
Architecting a Private Cloud - Cloud Expo
Architecting a Private Cloud - Cloud ExpoArchitecting a Private Cloud - Cloud Expo
Architecting a Private Cloud - Cloud Exposmw355
 
Datasheet Virbak Abio V32
Datasheet Virbak Abio V32Datasheet Virbak Abio V32
Datasheet Virbak Abio V32powerguy73
 
V mware v sphere advanced administration
V mware v sphere advanced administrationV mware v sphere advanced administration
V mware v sphere advanced administrationbestip
 
Storage Options in Windows Server 2012
Storage Options in Windows Server 2012Storage Options in Windows Server 2012
Storage Options in Windows Server 2012Lai Yoong Seng
 
Best Practices for Virtualizing Hadoop
Best Practices for Virtualizing HadoopBest Practices for Virtualizing Hadoop
Best Practices for Virtualizing HadoopDataWorks Summit
 
Xen server 6.1 technical sales presentation
Xen server 6.1 technical sales presentationXen server 6.1 technical sales presentation
Xen server 6.1 technical sales presentationNuno Alves
 
Caching technology comparison
Caching technology comparisonCaching technology comparison
Caching technology comparisonRohit Kelapure
 
Windows Server 2012 - Dynamische opslag met Storage Pools
Windows Server 2012 - Dynamische opslag met Storage PoolsWindows Server 2012 - Dynamische opslag met Storage Pools
Windows Server 2012 - Dynamische opslag met Storage PoolsCompuTrain. De IT opleider.
 
Cloud Computing & Scaling Web Apps
Cloud Computing & Scaling Web AppsCloud Computing & Scaling Web Apps
Cloud Computing & Scaling Web AppsMark Slingsby
 
Pm 01 bradley stone_openstorage_openstack
Pm 01 bradley stone_openstorage_openstackPm 01 bradley stone_openstorage_openstack
Pm 01 bradley stone_openstorage_openstackOpenCity Community
 
Openstorage with OpenStack, by Bradley
Openstorage with OpenStack, by BradleyOpenstorage with OpenStack, by Bradley
Openstorage with OpenStack, by BradleyHui Cheng
 
End of RAID as we know it with Ceph Replication
End of RAID as we know it with Ceph ReplicationEnd of RAID as we know it with Ceph Replication
End of RAID as we know it with Ceph ReplicationCeph Community
 
V mware v sphere boot camp
V mware v sphere boot campV mware v sphere boot camp
V mware v sphere boot campbestip
 
CloudStack-Developer-Day
CloudStack-Developer-DayCloudStack-Developer-Day
CloudStack-Developer-DayKimihiko Kitase
 

Similar to Sdc 2012-how-can-hypervisors-leverage-advanced-storage-features-v7.6(20-9-2012) (20)

What is coming for VMware vSphere?
What is coming for VMware vSphere?What is coming for VMware vSphere?
What is coming for VMware vSphere?
 
VMworld2011 Recap
VMworld2011 RecapVMworld2011 Recap
VMworld2011 Recap
 
Building a Distributed Block Storage System on Xen
Building a Distributed Block Storage System on XenBuilding a Distributed Block Storage System on Xen
Building a Distributed Block Storage System on Xen
 
Veritas Dynamic Multi-Pathing for VMware 6
Veritas Dynamic Multi-Pathing for VMware 6Veritas Dynamic Multi-Pathing for VMware 6
Veritas Dynamic Multi-Pathing for VMware 6
 
Architecting a Private Cloud - Cloud Expo
Architecting a Private Cloud - Cloud ExpoArchitecting a Private Cloud - Cloud Expo
Architecting a Private Cloud - Cloud Expo
 
Datasheet Virbak Abio V32
Datasheet Virbak Abio V32Datasheet Virbak Abio V32
Datasheet Virbak Abio V32
 
V mware v sphere advanced administration
V mware v sphere advanced administrationV mware v sphere advanced administration
V mware v sphere advanced administration
 
Storage Options in Windows Server 2012
Storage Options in Windows Server 2012Storage Options in Windows Server 2012
Storage Options in Windows Server 2012
 
Best Practices for Virtualizing Hadoop
Best Practices for Virtualizing HadoopBest Practices for Virtualizing Hadoop
Best Practices for Virtualizing Hadoop
 
Xen server 6.1 technical sales presentation
Xen server 6.1 technical sales presentationXen server 6.1 technical sales presentation
Xen server 6.1 technical sales presentation
 
Caching technology comparison
Caching technology comparisonCaching technology comparison
Caching technology comparison
 
2018 jk
2018 jk2018 jk
2018 jk
 
Windows Server 2012 - Dynamische opslag met Storage Pools
Windows Server 2012 - Dynamische opslag met Storage PoolsWindows Server 2012 - Dynamische opslag met Storage Pools
Windows Server 2012 - Dynamische opslag met Storage Pools
 
Cloud Computing & Scaling Web Apps
Cloud Computing & Scaling Web AppsCloud Computing & Scaling Web Apps
Cloud Computing & Scaling Web Apps
 
Pm 01 bradley stone_openstorage_openstack
Pm 01 bradley stone_openstorage_openstackPm 01 bradley stone_openstorage_openstack
Pm 01 bradley stone_openstorage_openstack
 
Openstorage with OpenStack, by Bradley
Openstorage with OpenStack, by BradleyOpenstorage with OpenStack, by Bradley
Openstorage with OpenStack, by Bradley
 
End of RAID as we know it with Ceph Replication
End of RAID as we know it with Ceph ReplicationEnd of RAID as we know it with Ceph Replication
End of RAID as we know it with Ceph Replication
 
V mware v sphere boot camp
V mware v sphere boot campV mware v sphere boot camp
V mware v sphere boot camp
 
CloudStack-Developer-Day
CloudStack-Developer-DayCloudStack-Developer-Day
CloudStack-Developer-Day
 
ebk EVO-RAIL v104
ebk EVO-RAIL v104ebk EVO-RAIL v104
ebk EVO-RAIL v104
 

Recently uploaded

How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 

Recently uploaded (20)

How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 

Sdc 2012-how-can-hypervisors-leverage-advanced-storage-features-v7.6(20-9-2012)

  • 1. How can Hypervisors leverage Advanced Storage features? Author: Shriram Pore, Solutions Architect, Calsoft Inc. Presenter: Dr. Anupam Bhide, CEO, Founder Calsoft Inc. 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.
  • 2. Introduction  Useful for storage vendors who are considering implementing hypervisor storage APIs like VAAI  Understand how hypervisors interact with storage today  Limitations in that interaction today  Need for a standard that is both hypervisor and storage agnostic.  Some Areas (that today’s hypervisor-specific standards do not cover) 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved. 2
  • 3. Virtual Environment Challenges Challenges  Virtual environments (Hypervisors) with NAS/SAN arrays have high storage bandwidth usage (IP/FC/etc.) as compared to DAS  SAN/NAS arrays have matured technologies such as snapshot, clone, server copy, range locking, etc.  Hypervisors can leverage these matured technologies and improve storage / network utilization.  However, hypervisors themselves have developed sophisticated storage virtualization layers Solution  The goal here is to –  Offload file/bulk-block operations to NAS/SAN arrays/servers to reduce storage bandwidth and increase I/O performance, storage utilization, etc. 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.
  • 4. Hypervisors use of storage  Classification – how hypervisors use storage  Hypervisor using local disks (e.g. VMware VMFS on local disk)  Hypervisor creating proprietary file system on SAN (e.g. VMware VMFS on a SAN array)  Hypervisor using LUNs of SAN array in RDM mode  Hypervisor using NAS storage box over NFS/CIFS  Better integration can have big benefits in many cases  E.g. Vendors have reported 99% savings in network bandwidth for cloning operations and 10x to 20x in efficiency 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved. 4
  • 5. Need for standard to optimize hypervisor interaction with storage Why the need? • Storage is the biggest bottleneck for hypervisor performance • Many common hypervisor operations can be optimized by delegating operations to storage boxes Hypervisors ESX(i) Hyper-V Xen • Storage vendors cannot afford to conform to multiple hypervisor standards for this delegation SNIA interface SNIA interface • Standard needs to be both hypervisor Interface layer for SAN for NAS independent and storage independent • Identify in-efficient file/ Block operations Identify on VMs in virtual environments • Define set of SAN/ NAS primitives (standard APIs) based on above Define identification including capability Storage exchange. SAN NAS Complia • Levels of compliance depend on interface nce implementations for storage and hypervisor 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.
  • 6. SAN based Architecture Application Application (VM) (Web Services) Set of files (disk Virtual Disk Library metadata) User Kernel Files and file Boundary segments Hypervisor proprietary FS e.g. VMFS Logical Block Layer (Block Devices, logical Blocks volume) Physical SCSI Device Access Device Vendor Specific SCSI extensions on Blocks SAN array 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.
  • 7. Representative SAN Primitives and Use Cases SAN Goal Use Cases Primitives Block • Avoid multiple WRITE • Block Zero feature speeds up deployment of Zeroing • Use SCSI WRITE SAME thick provision eager zeroed virtual disks • Full Copy feature speeds up storage vMotion • Avoid multiple READ & WRITE and cloning of VMs and network bandwidth to • Hardware-Assisted Locking feature is useful Full Copy copy complete VMDK for operations that require VM locking for • use SCSI EXTENDED COPY power on/off etc. and cluster wide operations Hardware like vMotion, storage vMotion etc. • BLOCK/EXTENT LEVEL locking • Power On Storm (100s of VMs being powered Assisted • use SCSI ATOMIC TEST & SET up which are using same LUN) has huge Locking latency which is resolved by hardware • Errors to indicate soft and assisted locking hard out of space conditions Thin • Pause VM in case of hard • Dead Space Reclamation enables reclamation Provisioning errors of blocks from a thin-provisioned LUN on the • use UNMAP for Space SAN based arrays Reclamation Offloaded Data Transfer to the arrays (illustrations from VMware SAN VAAI and Hyper-V ODX) 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.
  • 8. Types of VMware virtual disks  Thick provisioned  All backing storage immediately allocated  Not zeroed immediately  Thin provisioned  Backing storage not fully allocated  Thick provisioned with eager zeroing  Like thick provisioned, but also zeroed immediately 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved. 8
  • 9. Efficiencies gained  Block zeroing  Storage vendor can lazily zero extent in background  Can implement proprietary mechanisms to mark extent as zeroed out  In thin provisioned LUNs, storage vendor can unmap the extent  Full copy  Doing the copy within the storage box is more efficient  Furthermore, writeable snapshot(i.e. clone) technology can be used to do the full copy without actually copying blocks  Dedupe performance can be improved by recognizing that two extents are just copies of each other 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved. 9
  • 10. Efficiencies gained  Atomic Test and Set Many VM operations such as powering on or off  VMs require getting or releasing VM-specific lock. This requires either getting SCSI-2 or SCSI-3 persistent group reservations – both inefficient.  Storage vendors can optimize by implementing atomic test and set.  UNMAP  Thin provisioned LUNs can re-use deleted space 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved. 10
  • 11. Integration of Hypervisors and SAN capabilities Steps Involved • Identify in-efficient bulk-block operations on VMs in virtual environments/hypervisors Identify • Define set of SAN primitives (unused existing SCSI commands to be overloaded or new SCSI commands to be adopted) based on above identification including capability exchange (supported Define primitives) - can be added to SNIA specifications for standardization • SAN vendors to implement SCSI commands to deliver desired functionality, which will use its own Impleme technologies to achieve maximum performance nt • Hypervisor to invoke SCSI commands to leverage SAN capabilities Call • SAN arrays/servers to adopt proven/efficient technologies for increased performance and reduced usage of network bandwidth (features like Dynamic LUN Provisioning, Thin Provisioning, concurrent Adopt provisioning, Space reclamation, Dynamic Snapshots, LUN migration, etc.) 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.
  • 12. NAS based Architecture – Plugin Approach Application (Web Application (VM) Services) Virtual disk library Vendor Specific NFS Set of files (disc Virtual Disk Library plugin API or Custom RPC plugin metadata) User Kernel Boundary Plugin Approach • Plugin approach Files and file NAS proprietary FS over NFS/CIFS allows vendors to segments use their own Block Layer (Block Devices, logical communication Logical volume) mechanism Blocks • Many vendors use Device Blocks Physical SCSI Device Access unused NFS commands 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.
  • 13. Representative NAS Primitives and Use Cases NAS Goal Use Cases Primitives • Monitor space utilization in • Thick provisioning is the normal standard in sparse file and guarantee enterprise deployments of hypervisors File Space adequate space for VMs • With NAS file systems, POSIX lseek is the only Reservation • Allows future I/O ops of VM way for hypervisors to efficiently create large NOT to fail due to space files – but no backing storage will be created unavailability with lseek • File Space Reservation and Extended Stats • Clone VMs in faster and features enable hypervisors to quickly reserve File Cloning storage space efficient way space on NAS server over NFS/CIFS to create (Full and • Also can be used for faster & thick provisioned virtual disk for a VM Lazy) storage efficient snapshots & restores (VM snapshot) • VM cloning/deployment and storage vMotion • Retrieve accurate space are examples of operations that are offloaded utilization of VMs - data that to NAS server. cannot be retrieved using Extended NFS calls • Lazy clones typically used for instant Statistics • Monitor space utilization in operations. E.g. in VMs Linked Clones and VDI sparse file and guarantee • Full Clones – used for operations across data 2012 Storage Developer Conference. space for VMs adequate © Calsoft Inc. All Rights Reserved. stores
  • 14. Efficiencies gained  File Space Reservation  Thick provisioned semantics are hard to achieve in NAS due to NFS protocol limitations  Cloning – Full and Lazy  Doing the cloning within the NAS box is more efficient  Furthermore, file level writeable snapshot(i.e. clone) technology can be used to do the lazy copy without actually copying blocks  Dedupe performance can be improved by recognizing that two extents are just copies of each other  Extended Attributes  Understand exactly how much space a lazy clone uses  Understand how much space a VMDK file is actually using 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved. 14
  • 15. Additional Observations  Delegate creation of VM snapshots to the storage box. These snapshots need to be hypervisor- consistent.  For NAS, it is inefficient for hypervisors to do file snapshots  Flash-based storage boxes do not write in place and can do snapshots much more efficiently.  VDI applications could use the writeable snapshots(clone) already provided by most storage vendors 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved. 15
  • 16. Additional Observations  With storage box provided snapshots, backups can be taken without involving the server  Primitive for disaster recovery that can use replication or remote mirroring (similar to SRA/SRM)  UNMAP for even thick provisioning for flash based SAN arrays – helps reduce the size of book-keeping data structures.  Storage arrays can demultiplex IO streams from hypervisor and provide VM level IO stream QoS - vVol 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved. 16
  • 17. Microsoft Offloaded Data Transfer -ODX  Available Windows Server 8 onwards  Works for both SAN storage(VHDs) and NAS using SMB protocol  Provides Full copy semantics for both SAN and NAS  Protocol works as follows:  Send offload read request to source device  Return with a token  Send offload write request with token to destination device 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved. 17
  • 18. Author Biography Shriram Pore - Solutions Architect, Calsoft Inc.  A veteran of storage industry  More than 11 years of experience in architecting and developing products  Key strength lies in quickly understanding product requirements and translating them into architectural and engineering specs for implementation.  Architected, designed and implemented solutions for NAS-VMware integrated Backup-Recovery and Clone of VM  Led FS and replication teams on the file-server and also led the CORE component team from system management perspective (configuration, provision, and manage various facilities of file-server).  Master Of Computer Science from India, Pune University  Bachelor of Computer Science from India, Pune University 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.
  • 19. Presenter Biography Dr. Anupam Bhide – CEO, Co-Founder, Calsoft Inc.  Storage industry veteran  More than 21 years of industry experience  Senior Architect in the RDBMS development group at Oracle Corp, designed some of the key features of Oracle8  Founder-member of the DB2/6000 Parallel Edition team at IBM Research Center  Visiting Faculty at University of California – Berkeley  Ph.D. in Computer Science, University of California-Berkeley  BS in Computer Science: Indian Institute of Technology, Bombay and MS: University of Wisconsin-Madison 2012 Storage Developer Conference. © Calsoft Inc.. All Rights Reserved. 19
  • 20. Thank You Questions & Answers Contact info Dr. Anupam Bhide CEO, Co-Founder, Calsoft Inc. Email: anupam@calsoftinc.com Phone: +1 (408) 834 7086 Twitter: @Calsoftinc 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved. 20
  • 21. Integration of Hypervisors and NAS capabilities Steps Involved • Identify in-efficient file-operations on VMs in virtual environments Identify • Define set of NAS primitives (standard APIs) based on above identification including capability exchange Define (supported primitives) This may be added to SNIA specs for standardization • Provide a framework to implement APIs as plugin on hypervisors Provide • NAS vendor to implement the plugin to use its technologies to achieve maximum performance Implem ent • Hypervisor to call into the plugin via defined framework APIs to leverage NAS capabilities Call • Modify NFS/CIFS to accommodate NAS primitives desired, else provide a new interface which would be easily Modify integrated into the plugin with no security compromise • NAS server to adopt proven/efficient technologies for increased performance and reduced usage of network Adopt bandwidth (e.g.: file-clone, server-copy, FS snapshots, Thin Versioning, de-dupe, etc.) 2012 Storage Developer Conference. © Calsoft Inc. All Rights Reserved.