3. 1
Introduction
In Seagate’s and Xyratex’s last paper “Achieving Rapid Scale in Enterprise and Cloud Data Centers with SAS” it
was discussed how:
1 Storage requirements are continuing to grow exponentially with no slowdown in sight.
2 Desktop drive designs do not accommodate data center and multi-spindle environments with sustained
performance, scalability, and reliability as forefront drivers.
3 Not all data centers and infrastructures are necessarily the same and the related performance ability of
any environment must match the requirements necessary for critical applications.
4 Organizations expecting to purchase or lease reliable sustained storage performance at scale are
advised to seek out and demand the better-sustained match of high-performance enterprise drives
(nearline SAS) to the needs.
While it is important to discuss how Serial AT Attachment (SATA) drives are not inherently designed to provide the
best option for enterprise class workloads, it is equally important to understand why Serial Attached SCSI (SAS)
provides a better high performance storage solution today and represents a critical enabler to lowering data
center total cost of ownership (TCO) over the next decade. The case for SAS in the enterprise becomes evident
when assessing the emerging performance and technology directions influencing next generation enterprise
data centers and cloud deployments, costs, and operational power and cooling demands.
Historical Perspective
Consider the historical evolution of enterprise-class storage devices over the years. Indeed, in the early days,
storage related decisions were simpler for organizations and related IT departments. If the desire was for
enterprise class disk drives for servers or storage arrays, there was only one choice: a 3.5-inch (3.5”) 10,000-
RPM SCSI drive. As servers and storage evolved, drive performance became critical, which resulted in the
development and introduction of 15,000-RPM drives.
In addition to the need for performance, the overall
power requirements within datacenter and equipment
asset(s) was a large concern. This provided the platform
for the development of the 2.5-inch (2.5”) drive.
Moving to a smaller form factor, 2.5” drives
enabled more efficient power consumption and
greater input-output (I/O) throughput performance per
drive, enabling enterprise applications requiring high
transaction rates and best available input-output per
second (IOPS) capability. Overall, the transition to the
2.5” form factor served the specific power efficiency
and high performance needs, but did not allow for the
same capacity growth equivalent of 3.5” drives. This
caused a distinct separation between “enterprise high
performance” drives, which continued to focus on the
2.5” form factor featuring higher I/O rates operating
at either 10,000-RPM or 15,000-RPM. These drives
catered to high-end enterprise applications requiring
the full set of system robustness attributes, including
numerous data integrity features, multi-spindle robustness characteristics, and improved power efficiency.
However, with strict focus on high performance and power consumption, the tradeoff was a lower data storage
capacity per drive.
The continuing industry wide explosion of data creation and volume created an opportunity leading to the
ultimate development of more than one drive class focused on enterprise applications requiring the highest
Figure 1: Interface Evolution (Source: SCSI Trade Association)
4. 2 attainable data storage capacity with moderate performance and varying degrees in system robustness
attributes. This led to the introduction of the 7,200-RPM enterprise drives, or the “nearline” drive class.
Today, nearline enterprise drives make up a majority of drive usage and capacity demand in the enterprise
space. Indeed, high capacity with moderate performance per drive for enterprise applications are primary
characterizations of nearline drives. Further separation within the nearline drive class relates to the levels of
system robustness attributes detailed in the table below, including data integrity features and suitability for
multi-spindle integration.
Dual Port SAS based Enterprise Capacity
Nearline is closely associated with the higher end of the
enterprise spectrum, featuring highest available capacity,
very good to moderate performance per drive and the
highest levels in system robustness attributes.
This combination of attributes supports excellent multi-drive
suitability built-within the core of the system, starting with
the drives themselves. Using best in class system design
and integration methods, end users benefit from enhanced
internal system quality, high performance and excellent
system availability, resulting in significantly less hardware
to satisfy requirements, lower power consumption, lower
hardware failure rates and lower operational costs.
Drive Type Interface
Interface
Speed
per port
Form
Factor(s)
Pricing Capacity Performance
Data
Integrity
Rotational
Vibration
Tolerance
Multi-spindle
Bit Error
Rate
Annual
Failure Rate
Desktop Drive
SATA
(Single Port)
3 & 6 Gb/s 3.5 $ High
Low
7,200-RPM
N/A Low Not suitable
1 sector per
10E14
High*
Enterprise Value
(Nearline Lite)
SATA
(Single Port)
3 & 6 Gb/s 3.5 $$ High
Low to Moderate
≤7,200-RPM
Minimal Moderate Minimal suitability
1 sector per
10E15
Moderate
Enterprise Capacity
(Nearline)
SAS
(Dual Port)
3 & 6 Gb/s 2.5 & 3.5 $$$ High
Very Good to
Moderate
7,200-RPM
Full High Excellent suitability
1 sector per
10E15
Low
Enterprise
Performance
SAS
(Dual Port)
3 & 6 Gb/s 2.5 $$$$ Moderate
High
10,000-RPM
15,000-RPM
Full Highest Excellent suitability
1 sector per
10E16
Lowest
The decision of where to utilize one drive type over another is not abundantly or immediately clear, since all types
share the same interface speeds, form factors, and are plug-in compatible with similar connector assemblies. The
primary purpose of this paper is providing perspectives on future trends where the separation between value
and usage widens the distinction between drive types. Future system architects, organizations, and end users
will need to pay close attention to these developments.
Market and End User Requirements
There are many catalysts to the widespread use of nearline, but probably none more than Cloud Computing
– more specifically Cloud Storage. The cloud ushers in a new era of highly scalable data centers. Indeed, the
emergence of sophisticated virtualization capabilities drove a complete rethinking, and redesign, of large-scale
data center infrastructures as well as computing and storage. These factors drove the inception of what the
industry now knows as “Anything and Everything as a Service” deployments. These
new philosophies abstract the user experience and application from the underlying
hardware with the goal of reducing acquisition, design, deployment, and operational
costs. While the premise of using hardware with lowest possible cost has worked well
in the short term, will this strategy continue to make sense long term?
One area where SATA nearline lite does make sense relates to the industry trend
known as “Cold Archival Storage,” which relates to the storage of older data, still
requiring accessibility, in less expensive and “as needed” platforms. In a cold storage
framework, infrequent or rarely accessed data resides in less expensive, archival
storage assets required to store and/or retrieve the data. These assets receive power
and attention only when needed based on user-required access to the data. How does
this relate to storage requirements and disk drive capabilities? Overall, this sounds
remarkably similar to the present day single I/O requests inherent with SATA, which
already allows for significantly more storage capacity with the least overhead infrastructure. The net result, there
is no need to improve the SATA interface as what is available today is sufficient for future needs.
Even though the ability to store and utilize significantly more data, with less infrastructure, is a driving
engine of change in the cloud industry, there are other factors to consider in gaining maximum productivity for
future needs. In the very near future, when the underlying hardware components deliver higher levels of “multi-
core” processing capabilities within a single chip, the demands for increased storage performance becomes more
critical as indicated by the industry and the community.
* Based on a general enterprise workload
5. 3
Figure 2: SAS Roadmap (Source: SCSI Trade Association 2013)
Storage Performance Considerations
What is regarded today as supercomputing will ultimately be common in everyday computing. Why is this the
case? Consider how multi-core processing, networks, and memory technologies all share a similar characteristic:
“lanes.” Multi-core development increases independent threads, or processing lanes, while network and memory
bandwidth expands communication lanes, or channels. The natural tendency is increasing, geometrically,
independent initiator threads and communication lanes as these areas grow together in
unison. These mutually synergistic technologies become more important when considering
how to support next generation virtualization platforms. In order to gain high levels of
productivity in the future, the requirement for consistent underlying and built-in “hyper-
multilane” capabilities, suited for scale, is a crucial facet.
To meet the demands of cloud data centers from a TCO and workload perspective,
cloud service providers (“CSP”s) and data center managers require cost effective upgrades
to technology much faster and more often. With SATA reaching its apogee at 6Gb/s, the
architecture is at its maximum signaling rate.
With SAS, the technology roadmap is
more robust with 12Gb/s and 24Gb/s
SAS signaling rates on the horizon. For
data center infrastructures requiring
increased throughput, SAS offers the
flexibility to switch out 6Gb/s SAS drives
with the latest 12Gb/s SAS drives at
a lower technology refresh cost than
afforded by SATA. When considering
the current, and ongoing, drive industry
investment around SAS innovation, there
is really no comparison with SATA. SAS comes with more
features than available with SATA now, and the SAS roadmap
indicates higher signaling rates positioned to yield more
pronounced throughput and robustness gains coupled with
broader high performance feature-set additions over the
course of the next 10 years.
Single Port SATA based Enterprise
Value Nearline “Lite” is focused on
high capacity and affordability, featuring the
highest available capacity, lowest 3.5-inch hard
drive power consumption, low to moderate
performance, and a minimized set of enterprise-
class features in comparison to traditional
Nearline SAS hard drives, which makes it more
suitable to cloud based infrastructures.
This combination of attributes may not inherently
support all enterprise workloads; therefore, the
application of other system design and integration
practices is necessary to compensate and work
around any limitations. Using adequate work
around techniques, end users will benefit from
favorable initial system affordability and power
savings, with acceptable system performance;
however, these systems will inherently require
more hardware or software to satisfy more robust
enterprise application requirements resulting in
higher maintenance actions and operational cost.
Example “System Integration
Techniques” An example design method,
applicable to either SAS nearline or SATA nearline
“Lite” based storage systems, is extensive use
of middleware and virtualization to abstract
lower level hardware errors from the user level
experience. Even though, this provides required
levels of performance and availability measured
at the storage system level, it does not decrease
the incidence of hardware failures. On the
contrary, end users may enjoy what appears to
be transparent use of the system with little or
no impact, in the presence of numerous ongoing
failover events.
This is where the user needs to be careful since
ongoing failover events are not representative of a
“no impact” scenario. Indeed, the replacement of
failed drives raises operational costs. Depending
on the system design, drive rebuild events
cause performance slow down and increased
vulnerability to application outage or data loss.
In the end, users pay for the full system and all
operational costs, since there is no “abstracting”
net-realized flow of expenses to the purchaser.
Selecting drives with lower incidence of failure
helps reduce operational costs. Selecting drives
with higher incidence of failure may appear to have
“no impact” due to the system design, but this
increases operational costs. Application of state of
the art system integration techniques may enable
acceptable results for SATA nearline lite based
systems, whereas these same techniques result in
vastly superior overall gains when applied to SAS
nearline based systems. The point being storage
systems embody many design trades. The choices
of disk drive type and system design methods
represent key factors that end users should take
into account in their deployment decisions.
6. 4 According to the SCSI Trade Association (STA) presentation “SAS Standards and Technology Update1
” the
STA commits to preserving existing SAS architectures as well as 3Gb/s SAS & 6Gb/s SAS usage models while
maintaining backward compatibility with current 3Gb/s & 6Gb/s SATA via SAS backplane device connectors. In
addition, the STA stresses focus on 6Gb/s SAS and continued SAS compatibility encouraging improved storage
system reliability, availability, and serviceability (“RAS”) attributes, doubling transfer rates, improving cost-to-
performance and power-
to-bandwidth ratios, and
reducing the number of
per Gb/s connections. This
focus aids in maximizing
link utilization when using
devices operating at less
than 12Gb/s. All of this, in
conjunctionwithmaintaining
and supporting the SAS
Advanced Connectivity
roadmap, promotes broad
adoption of Mini-SAS HD and
supporting MultiLink SAS™
implementations.
According to another
STA presentation2
, by Marty
Czekalski, Senior Staff
Program Manager at Seagate Technology and President of STA, the performance benefits of SAS theoretically
double with each improvement in interface speed. For example, with 6Gb/s SAS in full duplex, four (4) ports
could, in essence, deliver 48Gb/s of available bandwidth. The beauty of this architecture is that 12Gb/s SAS is,
on the short-term horizon, capable of delivering 96Gb/s per every four (4) port duplex configuration resulting in
even higher I/O per second (IOPS) and lower latency storage.
The SAS feature-set, depicted in Figure 3 and created by Seagate Technology, is the step up to SAS pyramid.
As indicated, SAS has enterprise features, which are impractical on a SATA drive including:
• Full Duplex I/O – Full duplex link capability supports concurrent writes and reads.
• Two concurrent data channels – SAS drives allow execution of two data commands simultaneously.
Read/Read or Write/Write or Read/Write using both ports or Read/Write on one port.
• Enterprise Command Queuing – The command queue supports up to 128 commands from up to 16
separate hosts/initiators. Problems with the execution of any one command will not affect any other
command in the queue.
• End-to-end data integrity – Supports data integrity checks in tandem with the host. The host appends
an 8-byte check field to the data it sends to the drive. The drive checks the data it receives against the
check field and signals the host if an error condition exists. This check field writes to medium with the
data and returns to the host for validation on a Read.
• Full SCSI command set – SAS drives support the same comprehensive command set and inherently
compatible with available and future SAS controllers ensuring compatibility among disk drive
vendors for years to come.
• Variable sector size from 512 bytes to 4160 bytes in 8-byte increments. This has beneficial impact on
advanced protocol, security, and data integrity.
1 SAS Standards and Technology Update, 2011- www.scsita.org
2 SAS & SATA Change the Face of Enterprise Storage, 2010 - www.scsita.org
Figure 2: Seagate Step up to SAS
7. 5• Full IOECC – Error Correction for all drive electronic errors (silicon failures) is available on both Read
and Write operations.
• Dual port - SAS provides dual path capability without the need for a multiplexer or SAS bridge.
• Signal Integrity - SAS provides more robust signalling
• Cost - SAS based RAID controllers are the same cost as SATA
What about using a SAS to SATA bridge to take advantage of the SAS controller or backplane, but
utilizing SATA drives for the extra cost savings? The use of SAS-SATA bridge does not provide an exact
implementation of native SAS is many areas:
• Aborting a task will have a much greater impact on the outstanding commands from different initiators.
• Non-512B LBA size and use of PI will have a negative performance impact.
• Limited queue depth not improved (split across two (2) paths.)
• Still half duplex communications
• Enterprise SAS versus SATA drive cost differential is less than cost of SAS to SATA bridge card
System Customization
When customizing server and storage systems in a cloud data center, SATA based architecture, as mentioned
previously has limitations. With SATA, data center and cloud service architects are limited to lower performance
spin-speeds: 7,200-RPM nearline, as well as 5,900 and 5,400-RPM enterprise hard drives. With a SAS based
architecture, cloud architects can leverage single system architecture into multiple server and storage
configurations based on workload demand simply by changing out the hard disk drives (HDDs).
Enterprise-class HDD SAS SATA
5,400-RPM No Yes
5,900-RPM No Yes
7,200-RPM Yes Yes
10,000-RPM Yes No
15,000-RPM Yes No
Further, a SAS-based server and storage architecture allows for optimal customization, especially in consideration
of application requirements and changes in terms of workload or performance IOPS.
With a SAS based design, a cloud architect can switch out 7,200-RPM drives for 10,000 or 15,000-RPM drives
based on the needs of the application. This flexibility allows data centers and service providers to customize
standardized system designs for multiple use cases and is only possible through the utilization of an all SAS design.
System Integration – The Bigger Picture
The existing quandary for CSPs and enterprise organizations is providing high performance, availability, reliability,
and interoperability to their clients and infrastructure, but maintaining a low TCO. Indeed, availability, agility, and
low cost are crucial components for CSPs in order to compete in the marketplace. Many CSP and organizations,
that can no longer effectively integrate traditional enterprise IT vendor storage solutions, now incorporate a
“build their own” structure opting, in many cases, to use SATA drives with higher capacities and lower cost per
drive metrics to meet their TCO goals. However, this approach does not take the significant need for higher
availability, reliability, flexibility, scalability, and drive longevity into consideration, especially within larger CSPs
maintaining critical “always on” real time performance and service level agreements (SLAs) to their customers.
SAS nearline drives offer CSPs and enterprise infrastructures benefits not capable with SATA nearline lite
and lower cost drives. The inherent multi-port capability of SAS drives allows a redundant, “fail-over” path for
8. 6 each drive and SAS based storage systems support active-active frameworks. In addition, the design of SAS
drives takes heavy loads and use into consideration with Mean Time Between Failure (“MTBF”) ratings over 1
million hours and warranties between 3 to 5 years. These factors alone, over SATA MTBFs and warranties, allow for
a better availability and TCO in the long term. In terms of flexibility and interoperability, SAS offers the capability
to utilize SATA drives as well as SAS drives in the same environment.
While SATA has its place in the cloud and the enterprise, the SAS benefits of backward compatibility with the
SCSI command set and interoperability as well as SATA bring to bear higher levels of cost savings and adaptability.
Further, SAS leverages the continuing roadmap evolution with increased performance.
Conclusion - Key Takeaways
Seagate’s and Xyratex’s first white paper focused on how SAS can help achieve rapid scale in enterprise and cloud
data centers. This second paper focuses on how a SAS based storage architecture can potentially lower TCO in
both the near-term and, but more importantly, the long-term as it relates to system upgrades, customization, and
integration.
• In order to meet data center demands from a TCO and workload perspective, data center managers require
cost effective upgrades to technology much faster and more often; best achieved deploying all SAS
architecture.
• The flexibility of SAS based storage allows data center operators to customize standardized system
designs for multiple use cases thereby extending the life of the system and thus, lowering TCO.
• When designing a data center, system architects and end users need to pay close attention to the
development of the SAS interface and how it best addresses availability, reliability, flexibility, scalability,
and longevity requirements; especially within large CSPs maintaining critical “always on” real time
performance and SLAs with their customers.
Stay connected to Xyratex and Seagate for future white papers focused on designing, building, deploying and
operating scalable storage architectures for cloud, high performance computing and big data environments.
Xyratex & Seagate Partnership
Xyratex and Seagate are recognized leaders in high quality, enterprise-class data storage technologies.
Xyratex’s research and development expertise in storage and SAS technology creates solutions with
unmatched price-to-performance and TCO benefits for its OEM clients. Xyratex solutions span storage enclosures
and application storage servers with its OneStor™ family of products as well as integrated, high performance,
scale out solutions with its high-performance ClusterStor™ product line; specifically suited for data intensive
applications in the research, energy, defense, and life sciences markets. Xyratex’s rigorously tested storage
platforms and solutions provide increased levels of reliability, availability, and serviceability (RAS) as well as
TCO not available with other manufacturers. Further, this experience and patented testing methodologies allow
Xyratex to satisfy the requirements of Cloud Service Providers and enterprise datacenters for superior and robust
storage infrastructure, which is critical to sustain successful SLAs and meet application uptime guarantees. Learn
more at http://www.xyratex.com
Seagate contends that as storage capacity continues to shift to cloud architectures, data centers must
evolve to deliver the highest quality of service at the lowest possible total cost of ownership (TCO). Seagate has
the industry’s broadest selection of enterprise storage devices for cloud data centers. From high-performance
solid state drives to high-capacity hard disk drives, choosing the right storage device or mix of storage devices
can have a direct effect on data center TCO. Seagate and our partners help data center customers choose the
right storage device or mix of storage devices to best meet their requirements through Seagate’s Cloud Builder
Alliance. The Cloud Builder Alliance focuses on strategic relationships with “leading edge” partners building the
next-generation cloud using Seagate technology, devices, features, and services. Learn more at http://www.
seagate.com/solutions/cloud/.
“SAS is the mainstream
server storage interface of
choice, assuring investment
protection by guaranteeing
at least two generations
of accessibility,” said Chris
Lyon, Executive Director,
SCSI Trade Association.
“Companies such as Xyratex
and Seagate are enabling
the robust capabilities
of SAS with new storage
enclosures and application
storage servers that
continue to propel the
market ahead as well as
extending SAS architecture
deeper into enterprise
storage.“
