2. Agenda
Why IP Storage
IP storage NFS iSCSI SMB3
iSCSI concept
iSCSI architecture
iSCSI name
hba or software iscsi
iSCSI dependant independant hardware
iSCSI Configuration
Binding or not binding that is the question
Stacked or not stacked that is the question
Masking on the array with iqn
Multipathing, Policy & Pluggable Storage Architecture
VMFS 5
Miscellaneous
ALUA
Jumbo Frame
Delay Ack
Equallogic
Equallogic configurations
Host Software for VMware
Host Software for Windows
Questions ?
3. Why IP Storage
• Low cost infrastructure with 1 GB Ethernet
• No switch and special connectivity to
implement it
• For small infrastructure with less 50 VMs
• BUT not so easy to implement (a lot of best
practice to find perf and throughput)
4. IP Storage NFS iSCSI SMB3
• Two storage family
• Block (SCSI, iSCSI, FC, FCoE, AoE)
• NAS Network Attached storage ( NFS,
SMB3 => Hyper-V)
6. iSCSI – What is it?
• iSCSI specifies a way to “encapsulate”
SCSI commands in a TCP/IP network
connection:
IP TCP iSCSI
Gigabit Header Header SCSI commands and data
Header
Ethernet
Explains how to extract
SCSI commands and data
Provides information necessary to
Provides Link
guarantee delivery
Addressing, large
frame sizes, and Contains “routing” information
flow control so the message can find its
way through the network
7. iSCSI concept
• Internet Small Computer Systems Interface
(iSCSI)
• Physical network isolation (dedicated sw)
Logical network isolation (VLAN)
• Default Port Number: 3260
• An optional CHAP Secret (e.g.,
"secretsarefun") do not use for masking
• LUN (Logical unit number)
12. Network software iSCSI Configuration
• A VMkernel port must be created for ESXi to access software
iSCSI.
– The same port can be used to access NAS/NFS storage.
• To optimize your vSphere networking setup:
Separate iSCSI networks from NAS/NFS networks.
• Physical separation is preferred.
• If physical separation is not possible, use VLANs.
13. iSCSI Target-Discovery Methods
• Two discovery methods are
supported:
• Static
• Dynamic (also known as
SendTargets)
By default Dynamic
This functionality is used by
Equallogic to redirect the host to 192.168.36.101:3260
the right target
14. iSCSI Security: CHAP
• iSCSI initiators use CHAP for Software iSCSI properties > General tab
authentication purposes.
• By default, CHAP is not
configured.
• ESXi supports two types of
CHAP authentication:
• Unidirectional
• Bidirectional:
– Software iSCSI only
Target
authenticates
host.
• CHAP is not masking !!!!
Host
authenticates
target.
15. Multipathing with iSCSI
• Hardware iSCSI:
• Use two or more hardware iSCSI
adapters.
• Software or dependent hardware
iSCSI:
• Use multiple network interface
cards (NICs).
• Connect each NIC to a separate
VMkernel port.
• Associate VMkernel ports with
iSCSI initiator.
16. Configuration iSCSI 1/4
• Create a
dedicated vSwitch
with 2 VMKernel
and 2 VMNIC
connected on the
IP storage network
• If same subnet
dedicate a VMNIC
per VMKernel
• If not do nothing
21. Binding or not binding VMKernel on iSCSI sw
• Rescan times take longer than usual Incorrect
• number of paths are seen per device
• Unable to see any storage from the storage device
• CISCO NEXUS with LACP impose this config. with AA array without ALUA
22. Stacked or not stacked that is the question
The architecture of the array can have two
developement
Multi target: for each Ethernet port you have
a iSCSI target => do not stack the switch like
the FC word. (HDS or VNX)
Uni target: only one target is generated for
all the Ethernet port => stack the switch to
have a failover in case of lost connectivity
(NetApp, Equallogic)
23. Dedicated network or not …
For small network yes it’s better
Think IOS update
With 10GB and Nexus… no choice
Generaly network infrastructure is
stacked and it’s not a good thinks in case
of lost of the master (old switch 15 sec
without network)
26. What is Asymmetric Logical Unit
(ALUA)?
ALUA Allows for paths to be
profiled
Active (optimized)
Active (non-optimized)
27. Le VMFS 5
• VMFS-5:
• Allows concurrent access to shared storage
• Can be dynamically expanded
• Uses a 1MB block size, good for storing large virtual disk files
• Uses subblock addressing, good for storing small files:
– The subblock size is 8KB.
• Provides on-disk, block-level locking
host host
VMFS datastore
28. VAAI
• vStorage APIs for Array Integration is a
feature introduced ESX/ESXi 4.1 that
provides hardware acceleration
functionality.
• It enables your host to offload specific
virtual machine and storage management
operations to compliant storage hardware.
• Used for lock file, clone & copie, svMotion
• NFS is supported with a ESXi agent
• Need license Ent & Ent +
29. Datastore size
• Updating the metadata of VMFS requires a
SCSI reserve and release
• 1. Starting and stopping a VM
• 2. vMotioning a VM
• 3. Using VMware snapshots
• 4. Using vAPI Dataprotection (use VMware snapshots)
• . => 15 VMs / block datastore
• VMware Best Practice 300 – 700 GB
• But actually with VAAI => no limit
• Use Large capacity datastores Avoid RDMs
• NFS: 16TB
• VMFS: vSphere 5 = 64TB
• vSphere 4 = 2TB
• Avoid extents
30. Miscellaneous
• Never route network IP storage !!! Warning with the loop
!!!!
• Ask for compliant subnets with the Network team
• Use always the vCenter Plug-In to apply the best practice
on the ESXi host
• Use always the vCenter Plug-In to create and mount the
datastore
31. Jumbo Frame
• What is an Ethernet Jumbo Frame? Ethernet frames with
more than 1500 bytes of payload (9000 is common – FCoE
is 2240)
• Commonly ‘thought of’ as having better performance
• Should I use Jumbo Frames?
• Adds complexity & performance gains (while existent) are
relatively marginal with common block sizes
• Stick with the defaults when you can
32. Delayed Ack mandatory (EMC et HDS hyper-V)
TCP delayed acknowledgment is a technique used by some implementations of
the Transmission Control Protocol in an effort to improve network performance. In
essence, several ACK responses may be combined together into a single
response, reducing protocol overhead. However, in some circumstances, the
technique can reduce application performance.
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1
002598
http://support.microsoft.com/kb/981482
http://www.axiomdynamics.com/it-tweaks-and-tricks/iSCSI-best-practices/html/iSCSI-Best-Practices-
VMware-EMC-Microsoft.html
33. Advanced network configurations
• Enable Rapid spanning-tree (portfast)
• Enable flow control on all ports in the switch and array
• Disable port storm control (broadcast- unicast)
• Enable jumbo frames if possible
34. Alignment is Optimal I/O
Misalignment of filesystems results in additional work on storage controller to
satisfy IO request
Affects every protocol, and every storage array VMFS & NFS Datastores
VMDKs & RDMs with NTFS, EXT3, LUN, etc
Filesystems exist in the datastore and VMDK
Create VMFS with GUI not vCLI
Windows NT, 2000, 2003, & XP (use diskpart to set to 1MB)
Linux (use fdisk expert mode and align on 2048 = 1MB)
35. Fixing Misalignment
Guest OS filesystem is misaligned
• Step 1: Take an array snapshot/backup
• Step 2: Use offline tools to realign EMC
UBERAlign (open, works with all, scheduler, and in-
guest reclaim)
vSphere Migrator
Alternate: Use online tool to align NetApp
Migrate & Optimize (VSC feature)
36. SIOC NIOC SDRS
• QoS on storage and network dVS
• NIOC offert QoS compatible 802.1p for Virtual machine
traffic, Management traffic, iSCSI traffic, NFS traffic, Fault-
tolerant traffic, VMware vMotion traffic, User-defined
traffic, vSphere replication traffic
• SIOC use queue depth to manage contention. SIOC
monitors the latency of I/Os to datastores at each ESX
host sharing that device. When the average normalized
datastore latency exceeds a set threshold (30ms by
default), the datastore is considered to be congested, and
SIOC kicks in to distribute the available storage resources
to virtual machines in proportion to their shares.
• Need license Ent+ …
37. iSCSI Storage array
iSCSI Storage Switch Multi target Jumbo Frame 10GB ALUA
array stacked
EMC VNX no yes yes yes yes
depend Need different
NetApp subnet
yes yes FC only
HDS HUS no yes 1GB ? yes AA
HDS AMS no yes 1GB no Only AMS AA
2500
Equallogic yes no yes yes -
38. Equallogic
• Can be combined with other EqualLogic PS
Series arrays in the same SAN group,
online. Up to 16 total members per group
• Automatic RAID 5, RAID 6, RAID 10 and
RAID 50
• 1 mgmt target per SAN Group but each
stack has these own tragets.
• The mgmt target redirect to the target with
the LUN
• Need to use de MPIO of Dell Equallogic for
Windows
39. Equallogic configurations
• There are three load balancers that operate within
a pool:
• The NLB (Network Load Balancer) manages the
assignment of individual iSCSI connections to
Ethernet ports on the pool members
• The CLB (Capacity Load Balancer) manages the
utilization of the disk capacity in the pool
• The APLB (Automatic Performance Load
Balancer) manages the distribution of high I/O
data within the pool.
40. Host Software for VMware
• Host Integration Tools for VMware®
• Auto-Snapshot Manager/VMware Edition (ASM/VE): Integrates with
VMware VirtualCenter and PS Series snapshots to allow administrators to
enable Smart Copy protection of VirtualCenter folders, datastores, and
virtual machines.
• EqualLogic Datastore Manager: Allows administrators to create and
manage datastores on EqualLogic PS Series arrays from within vCenter.
• Virtual Desktop Deployment Utility: Automates the deployment of virtual
desktops in a VMware View™ environment.
• Storage Adapter for Site Recovery Manager (SRM): Allows SRM to
understand and recognize PS Series replication for full SRM integration.
• Multipathing Extension Module for VMware® vSphere™: Provides
connection awareness enhancements to the existing VMware multipathing
functionality that understands PS Series network load balancing and
facilitates host connections to PS Series volumes.
41. Host Software for Windows
• Host Integration Tools for Microsoft®
• Remote Setup Wizard (RSW): Initializes new PS Series arrays, configures
host connections to PS Series SANs, and configures and manages
multipathing.
• Multipath I/O Device Specific Module (MPIO DSM): Includes a connection
awareness-module that understands PS Series network load balancing and
facilitates host connections to PS Series volumes.
• VSS and VDS Provider Services: Allows 3rd party backup software
vendors to perform off-host backups.
• Auto-Snapshot Manager/Microsoft Edition (ASM/ME): Provides point-in-
time SAN protection of critical application data using PS Series snapshots,
clones, and replicas of supported applications such as SQL Server®,
Exchange Server®, Hyper-V™, and NTFS file shares.
• PowerShell Tools: Provides a comprehensive set of PowerShell cmdlets for
managing one or many PS Series groups.
