At Red Hat Storage Day New York on 1/19/16, Red Hat partner Seagate presented on how to implement dense storage using HDDs with SSDs and PCIe flash accelerator cards.
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Implementation of Dense Storage Utilizing HDDs with SSDs and PCIe Flash Accelerator Cards
1. Rick Stehno, Sr. Database and Ceph Storage Performance Architect
Applications and Ecosystem Solutions Development
Implementation of Dense Storage Utilizing
HDDs with SSDs and PCIe Flash Accelerator
Cards
NY, USA
January 2016 1
2. Seagate Confidential 2
Big Data is Bigger than MORE Data
Demand GROWING at
Exponential RATE
Bigger Different $-Constrained
Created 5%
31%
56%
8%
62%
25%
Consumer Electronics Client Compute Cloud Computing Tablets & Smartphones
1,121
Exabytes
2010
77
Exabytes
Location SHIFTING to the
CLOUD
Stored
Capacity
$0B
$20B
$40B
$60B
$80B
$100B
$120B
$140B
$160B
2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018
Servers External Controller-Based Storage
Enterprise Network Equipment Enterprise Communications Applications
Data Center System Spend
Stable but SLOW growth
3. Seagate Confidential 3
Seagate is Evolving to Address Trends
SYSTEMS
DEVICES
ARRAYS
Converged Infrastructure
High Performance Storage
Hybrid Data
Systems
Software-defined
Storage
Flash Arrays
NAND PCIe
Traditional Enterprise
NAND SASHyperscale
Desktop
Notebook / Tablet
Branded NAND SATAHybrid
Kinetic
Still Growing
Expanded
Offerings
Started with Our
Core Products Flash
Controllers
4. Seagate Confidential 4
Seagate Enterprise Device PortfolioHighestPerformance/$
Highest GB/$
Supported by end-to-end silicon capabilities across Flash and HDDs; >1 Billion ARM CPUs shipped
Silicon technology/IP spans Flash and HDD Portfolio across PCIe, SAS and SATA
Archival HDDs
Capacity
HDDs
Performance
HDDs
with TurboBoost™
Flash Caching
Nytro® PCIe
Flash
Accelerators
with SandForce®
Technology
SAS
SSDs
Latency
Density
Thermal
PCIe M.2, 2.5
SAS 2.5
15K/10K 2.5
NL 2.5
NL/Value 3.5
Kinetic 3.5
Archive 3.5
Perf. 3.5
Nytro PCIe
SSDs
Markets
Hyperscale
Traditional-IT
HPC
Mission-Critical/Compute
Business-Critical/Near-Line
Archival
JBODs
Storage Arrays
Servers
5. Seagate Confidential 5
DISK
FS
DISK DISK
OSD
DISK DISK
OSD OSD OSD OSD
FS FS FSFS btrfs
xfs
ext4
MMM
Advanced Storage:
• Shingled Magnetic Recording (SMR) Drives
• Excellent density, cost, read performance, and
sequential write streaming
• Poor random write performance
• PCIe Flash
• Excellent performance, in-server integration
• Poor cost per GB
Object Storage Daemons (OSDs):
• 10s to 10000s of OSDs in a cluster
• One per disk or SSD
• Serves stored objects to clients
• Intelligent peers perform replication and recovery
tasks
• Support for erasure coding of objects
• Support for strong consistency leveraging write
journals
Ceph Deployment
6. Seagate Confidential 6
Featuring:
• Nytro XP6302 flash accelerator card
• Nytro XD - tuned Write Back caching SW
• 12x 8TB SMR HDDs
Benefits
• Accelerate Ceph journals and Metadata
with flash
• By moving this data onto flash, cost-
effective SMR drives are well-suited for
extremely dense object storage systems
reducing storage costs over 40%
• Experienced 11x Ceph Object Storage
Performance Gains
Storage Component Deployment in Ceph
8. Seagate Confidential 8
Ceph Benchmarks – 4k writes
SMR only
SMR with
SSD
SMR Cache
Tier
SMR with
Nytro XD
MB/s 2.53 2.56 5.9 28.19
0
5
10
15
20
25
30
MB/s
4K writes - 16 threads
SMR only
SMR with
SSD
SMR Cache
Tier
SMR with
Nytro XD
Latency/s 0.0246 0.024 0.01 0.0022
0
0.005
0.01
0.015
0.02
0.025
0.03
Seconds
4K writes - 16 threads
11x Gain in MB/s 11x drop in Latency
9. Seagate Confidential 9
Ceph Benchmarks – 4M writes
SMR only SMR with SSD
SMR Cache
Tier
SMR with Nytro
XD
MB/s 135.532 194.556 590.4 777.05
0
100
200
300
400
500
600
700
800
900
MB/s
4M writes - 16 threads
SMR only
SMR with
SSD
SMR Cache
Tier
SMR with
Nytro XD
Latency/s 0.471 0.328 0.108 0.082
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
Seconds
4M writes - 16 threads
7x gain in MB/s 6x decrease in Latency
10. Seagate Confidential 10
Ceph Storage Costs
SMR + Nytro XD Price per MB/s: Cost of ((Total SMR HDD cost + Nytro PCIe Card) / MB/s for each test)
HDD Price per GB Price MB/s 4k writes Price MB/s 4M writes
CMR 4TB HDD $0.08250 CMR 4TB HDD $589.90 CMR 4TB HDD $11.89
SMR 8TB HDD $0.03250 SMR 8TB HDD + Nytro XD $315.46 SMR 8TB HDD + Nytro XD $11.98
These prices do not include savings from electrical/cooling costs, reducing datacenter floor space, from the reduction of HDD
11. Seagate Confidential 11
Scaled-Up Hardware Configuration
Example Ceph configuration
using fully Populated Open Rack:
• 18 Winterfell Servers
• 33.5 TBs of Flash
• 2.2 PBs of SMR Storage
12. Seagate Confidential 12Seagate Confidential
Thank You!
Questions?
Learn how Seagate accelerates storage with one of
the broadest SSD and Flash portfolios in the market