SAN Best Practices and Trends

1© Copyright 2013 EMC Corporation. All rights reserved.
STORAGE AREA
NETWORKING
Erik Smith
SAN Technology update and best practice deep
dive for FC, FCoE and iSCSI SANs

Roadmap Information Disclaimer
 EMC makes no representation and undertakes no obligations with
regard to product planning information, anticipated product
characteristics, performance specifications, or anticipated release
dates (collectively, “Roadmap Information”).
 Roadmap Information is provided by EMC as an accommodation to the
recipient solely for purposes of discussion and without intending to be
bound thereby.
 Roadmap information is EMC Restricted Confidential and is provided
under the terms, conditions and restrictions defined in the EMC Non-
Disclosure Agreement in place with your organization.

Objectives For This Session
1. Describe the recent changes in the SAN space.
2. Describe EMC’s best practices and how they help
reduce risk.
3. Describe trends with Cloud (IaaS) and how they do
(and do not) intersect with traditional SANs.

Agenda
 Recent FC and FCoE product changes
 EMC Best practice deep dive
 Cloud (IaaS) trends and the impact to SAN

Recent FC and FCoE product changes
 Products
– Support for Brocade VDX and VCS
 Topologies
– VCS: End-to-End FCoE and support for FIP Snooping
Bridges (FSB)
 Firmware
– FOS: v6.4.3d and FOS v7.0.2c (target releases June)
– NOS: 3.0.1a (recommended)
Connectrix-B / Brocade

 Recent new features:
– In-flight Encryption and Compression on 16G capable
Inter-Switch Links (ISLs)
– BB_Credit Loss detection and recovery on 16G ISLs
– Enhanced optical Inter-Chassis Link (ICL) topology support
for DCX 8510
– D_Ports
– Fabric Asssigned WWN
Connectrix-B / Brocade (cont)

 Products
– MDS-9710 is now supported by EMC
– New 8G modules for the MDS 9513 with fabric 3
▪ 32 port full line rate
▪ 48 port 1.5:1 oversubscribed
 Topologies
– UCS FI module “Direct attach” to EMC Storage
Connectrix-MDS / Cisco

 Firmware (target releases)
– MDS
▪ 3.3(5b)
▪ 4.2(7e)
▪ 5.0(7) – Gen 3
▪ 5.2(1) – Gen 4
Connectrix-MDS / Cisco (cont)

 Products
– QFX3500, QFX3600 and QFabric
 Topologies
– Gateway mode (NPIV)
▪ A connection to a Brocade or Cisco switch is required for FCoE
– Transit Switch mode (FSB)
▪ Connectivity to Brocade, Cisco or Juniper switches supported
Juniper

Agenda
– Single Initiator Zoning
– Dynamic Interface Management
– Monitor for congestion
– Periodic SAN Health checks
– Monitoring for bit errors
– Cable hygiene
– Target releases

SAN Topology Overview

Best Practice – Single Initiator Zoning
 Best practice
– Each zone should only contain one initiator and the targets
it will access
 Reason
– Originally
▪ Concern about initiators logging in to other initiators
– Today
▪ Limits the number of name server queries that are sent to the switch
▪ Recommended due to its use in E-Lab test configurations

The response will include all ports that
are logged in, have registered with the
NS and are in the same zone as Host 1

Without zoning, the response would
include information about all host and
storage ports in the fabric

For each port returned in the NS
Response, a host will query the Name
Server for additional information.

 Initiator login with Single Initiator Zoning
– 4 hosts have been zoned to 1 target – 4 zones - (4:1)

 Initiator login without Zoning
– 32 hosts and 8 targets in the same “zone” (4:1)

Host will attempt to PLOGI/PRLI all
discovered ports and perform REPORT
LUNs with all Targets.

Single Initiator Zoning Summary
 Use single initiator / single target zoning if at all
possible
 If not possible or practical
– e.g., some VMware and RecoverPoint environments
– Learn to watch for signs of trouble
▪ Randomly missing host or storage ports during power on or after link
events
▪ Elongated host boot times
 Target Driven Zoning will help when available!

Agenda
– Cable hygiene
– Target releases

Dynamic Interface Management
 Best practice
– Enable “Dynamic Interface Management”
▪ Brocade – Port Fencing
▪ Cisco – Port Guard
– Tested settings coming soon to the EMC FC SAN Topologies
TechBook
 Reason
– Can prevent SAN outage by shutting down a misbehaving
host or storage port

Agenda
– Cable hygiene
– Target releases

Monitor For Congestion
 Best practice
– Monitor for congestion in the fabric
 Reason
– Unresolved congestion can spread throughout the fabric
impacting completely unrelated operating environments

 Oversubscription – A condition
that exists when the potential
demand exceeds the capacity of
the system

 Congestion - A condition
that exists when the actual
demand exceeds the
capacity of the system

 Oversubscription and Congestion
– On an Inter Switch Link (ISL)
▪ By design
▪ After failure
– At the N_Port
▪ Slow drain

Example Topology
Monitor For Congestion (continued)
Switch A
Switch B
1 2
1 2
H H H H H H HH
T T T T T T TT
250 250 250 250 250 250 250 250
2 @ 16 Gb/s
~3200 MB/s
8 @ 250 MB/s
~2000 MB/s
 By design
– 8:1 considered normal
– 8 Gb/s potential per 1Gb/s
capacity
 Example
– 8 HBAs @ 250 MB/s = 2000
MB/s
– 2 ISLs capable of 3200 MB/s
total
– 2000/3200 = ~63% (OK)
ISL Oversubscription

 After failure
– Demand may exceed capacity
 Example
– 8 HBAs demand 250 MB/s =
2000 MB/s
– 1 ISL capable 1600 MB/s total
– 2000/1600 = 125% (NOT
OK)
– Each host will experience
backpressure as port 1
becomes congested
Example TopologyISL Congestion
Switch A
Switch B
1 2
1 2
H H H H H H HH
T T T T T T TT
250 250 250 250 250 250 250 250
1 @ 16 Gb/s
~1600 MB/s
8 @ 250 MB/s
~2000 MB/s
X

 Under normal conditions
– Monitor ISLs for utilization exceeding 80%
 Failure conditions
– Monitor environment for loss of ISLs
– Monitor ISLs for utilization exceeding 80%
Detecting Problems Due to ISL Oversubscription

 Practice prevention!
– Plan for failures, have spare capacity available in case an ISL fails
 Add additional capacity
– If exceeding 80% frequently
 Topology change
– Investigate host and storage placement
– Storage on the core only when it makes sense
 Ensure features are enabled to make ISL use as
efficient as possible
Resolving Problems Due to ISL Oversubscription

 Slow drain or Busy node
– May be unable to keep up with line rate due to workload
– Could cause congestion spreading
Detecting Problems at the N_Port

 Slow drain

 Poor performance
 80% rule does not apply here
– A slow drain is unable to run at line rate and will not hit 80%
 Monitor F_Ports for time spent at zero Tx Credit
– Careful, some number of these per day are normal
– Need to find out what is normal in your environment and look for exception cases
 Transmit LR may be incrementing on switch ports in
severe cases
Detecting Problems Due to a Slow Drain

1.Draw the fabric topology
2.Determine which ISLs are in use by the impacted
devices
3.Look at all FC Switch ports involved and gather
baseline statistics
4.Look at all FC Switch ports involved and gather
statistics (2nd time)
5.Analyze
Identifying a Slow Drain (Option 1)

 Brocade - Enable bottleneck detection
– See the “Brocade Network Advisor SAN User Manual”
 Cisco – Configure Slow Drain Device Detection
– See “Configuring Average Credit Non-Available Duration Threshold and
Action” in the MDS Interface Configuration guide
Identifying a Slow Drain (Option 2)

 When the slow drain is a storage port
– Not as common as you would think
– SCSI typically throttles bandwidth, not link level flow control
 When the slow drain is a host port
– SCSI incast (similar to TCP incast)
– Likelihood of encountering this type of problem may increase as the number of VMs
per Hypervisor increase.
 When the slow drain is a congested ISL
– Add additional capacity
– Your 80% rule should have caught this!
Slow Drain scenarios

 Brocade
– Enable bottleneck detection
– Configure Edge Hold time
 Cisco – Configure Slow Drain Device Detection and
Congestion Avoidance
– Stuck Frame timeout value
– No-Credit timeout value
Resolving Slow Drain related problems automatically

Agenda
– Cable hygiene
– Target releases

Periodic SAN Health Checks
 Best practice
– Perform periodic SAN Health checks
 Reason
– An ounce of prevention is worth a pound of cure

Periodic SAN Health Checks
 Brocade
– EMC SANity (Brocade SAN Health)
– A free utility that you download and run in your environment
▪ Provides a spreadsheet including historical performance monitoring
data
▪ Also provides a visio diagram of your fabric!
 Brocade and Cisco
– EMC E-Lab Advisor
– Upload the output from “supportshow” or “show tech-support”
and a report will be produced.
▪ Validates that you are running the latest versions and checks for error conditions

Agenda
– Cable hygiene
– Target releases

Monitor For Bit Errors
 Best practice
– Monitor for bit errors
 Reason
– Periodic bit errors can lead to BB_Credit Loss
▪ Each bit error has ~ 1 in 263 chance of landing in the middle of an
R_RDY
– Becoming less of a problem as BB_Credit recovery is supported
on more platforms.
▪ Brocade – Default on when portcfglongdistance is set
▪ Cisco – Automatically enabled on E or TE ports
▪ You still need to monitor F_Ports (host and storage ports)

Agenda
– Cable hygiene
– Target releases

Practice Good Cable Hygiene
 Best practice
– Clean your cable ends before insertion
 Reason
– See previous discussion about monitoring for bit errors
– Cleaning cable ends resolves 80%+ of the physical
connectivity problems that we see in the field.

Bad Good

 Cleaning information:
– Cisco
▪ See “Inspection and Cleaning Procedures for Fiber-Optic Connections”
– Finisar
▪ See “PREVENTIVE MAINTENANCE OF FIBER OPTIC CABLES AND
FINISAR OPTICS”

 40G and MTP Connectors
– Today 40G is actually a parallel connection
(4x10G)
– MTP Connectors contain 12 fiber strands
▪ Only 8 are used with 40G.
– Cleaning these connectors (both sides) is
very important

Agenda
– Cable hygiene
– Target releases

Run Target Releases
 Best practice
– Target firmware versions are listed in the EMC Support
Matrix and are denoted with a (T)
– Run a Target (T) release if possible
 Reason
– Target releases have a proven track record for being stable
and free of major defects.

Agenda

Cloud IaaS Trends And The Impact To SANs
Infrastructure as a Service (IaaS)
• Compute, Network and
storage resources
• Typically:
I. Web orderable
II. Automatically
configured

Infrastructure as a Service (IaaS)
• External array based
storage is implied in this
model
• Connectivity via FC,
FCoE, iSCSI or NAS

Storage A Storage B
Host Host Host Host
VM VM VM VM VM
L2 / Switch / Ethernet SAN BSAN ALAN
Pod

Storage A Storage B
Host Host Host Host
VM VM VM VM VM
LAN
Pod
L2 / Switch / DCB Ethernet

Compute
Network (L2)
Storage
Pod A
Network (L3)

Compute
Network (L2)
Storage
VM VM
Pod A
Network (L3)

In Summary
 Follow Best practices
– Unless you have a compelling reason not to
– Understand the risks if you deicide to follow the path less traveled
 FC and FCoE will be around for a while
 Cloud IaaS environment could present a challenge to FC and
FCoE.

EMC Total Customer Experience
Voice of Customer:
• Loyalty Input from our Customers
• Transactional Satisfaction on steps of the CJM
• Benchmarking against best practices
Voice of Partner:
• Loyalty input from our Partner Ecosystem
• Improving how we enable our partners
Product Quality Scorecard:
• Focus on hardware, software, and services
• Analysis of quality metrics
Voice of Field:
• Formalize feedback from EMC Field Leaders
• Their view on how we are doing with our
Customers and Partners
Buying Deploying Servicing
Using
(Product)
CUSTOMER JOURNEY MAP
Unified Analytics Platform
A Customer & Partner focused data-driven approach to continuous improvement

Additional information
 Please check out my blog
www.brasstacksblog.typepad.com

Additional information
 Or follow me on twitter!
@ErikSmith7

Learn More!
Activity
Keynote • Brian Gallagher: (title of Brian’s keynote TBD)
Lectures • Title
• Title
• And so forth (title only… speaker names are
not published, by the way)
Birds-of-a-Feather • Title
Hands on Labs • Title
Partner Lectures • Title
Solutions Pavilion • Name of Demo or solution area

QUESTIONS?

Visit us at the Mission Critical
Storage & Solutions Booth
Play our Cloud Defender game, win cool
prizes, and get your picture taken with the
Data City Defenders!

SAN Best Practices and Trends

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à SAN Best Practices and Trends

Similaire à SAN Best Practices and Trends (20)

Plus de EMC

Plus de EMC (20)

Dernier

Dernier (20)

SAN Best Practices and Trends