Introduction to Solid State Drives

Introduction to
Solid State Drive
Technology

Saturday, November 16, 13

Class Overview
• The Evolution of Storage Technology
• Spinning Disks
• Storage Metrics
• Solid State Technology


better understanding of spinning disks, understand high & low level ﬂash, issues with SSDs

The Evolution of Storage
Technology


Pre-History

Density/Time


Speed/Time

Spinning Disks


The Parts of
a Hard Drive


The Parts of
a Hard Drive

Platters


The Parts of
a Hard Drive


Actuator Arms
and Heads

The Parts of
a Hard Drive


Controller
and Interface

Voltron Force Assemble!


Disk Interface
• Removable (USB/CF)
• SATA1 / II / III
• Nearline SAS
• SAS
• Fibre Channel
• PCI-e

Disk
Interface

•
•


USB

Spinning Disk
Removable Media
Advantages:

•

Nigh Universal

Disadvantages:

•
•
•
•

Slower
Fragile
Easily lost.
Abstraction Layer

Disk
Interface

SATA I / II / III
• Speeds: 1.5 / 3 / 6Gb/s
• Requires AHCI for things like NCQ
• Subset of SAS
• Shares IDE command set

AHCI - Advanced Host Controller Interface (IDE is ok for TRIM)
NCQ on SSD ensure SSD has things to do while host is latent
(Intel can queue 32 requests) - logo from SATA-IO (intnl org)

Disk
Interface

SATA 3.1
(This time, it’s personal)

• Approved July 2011
• Universal Storage Module
• mSATA
• QTRIM

QTRIM - queued TRIM commands, USM is a mobile drive standard

SAS / Nearline SAS
• SAS
• Enhanced CRC checking
• 512/520/528 bit blocks
• Low density, high reliability
• Nearline SAS
• ...not so much

Serially Attached SCSI - 16 bits of CRC

Disk Geometry


Disk
Geometry

Platters


Disk
Geometry

Tracks


Disk
Geometry

Cylinders


Disk
Geometry

Sectors


Disk
Geometry

Logical Block Addressing
• First introduced as an abstraction layer
• Replaced CHS addressing
• Address Space is Linear (block 0 - n)
• Size of address space depends on the
standard at time of manufacture.


Currently at 48-bit LBA -

Variables Affecting
Spinning Disk IO Rate


Platter Rotational Speed


Seek Speed


Data Density


Variables
Affecting
Spinning
Disk Speed

Controller Cache


Size / battery backed / hybrid drives

Spinning Disk
Damage Vectors


Spinning Disk
Damage Vectors

Movement
• Movement vertical or parallel to platter
• Measured in G forces
• Head Crashes
• Spinning Down
• Head uses “Landing Strip”
• Repeated platter contact causes damage
to the read/write head


Used to manually park the heads | Putting your computer to sleep can cause the head to park
| nanocoating on the bumpy landing strip

Protection against
movement
• “Active Drive Protection”: Free-fall sensor
• Has a lift arm to lift the head away from the
platter

• Some protection systems are in the drive,
some are in the controller

• Don’t mix the two

Apple: Sudden motion sensor, Lenovo/IBM: Hard Drive Active Protection System
Next slide: “You know, vibrations are movement...”

You know, vibrations
are movement...


Yes, vibrations are
important, too


Spinning Disk
Damage Vectors

Shelf Life
• Oil / Lubricants in bearings
• Temperature ﬂuctuations
• Magnetic “events” (bit rot)
• Outgassing / vapor removal

Long-term “archival quality” drives with long-life lubricant
Long term “cold storage” arrays which periodically spin up drives every few weeks to clean &
scrub the data

Spinning Disks in RAID
• Redundant Array of Inexpensive Disks
• Common RAID levels:
• 0,1,5,6,10
• Software / Hardware

Spinning
Disks
In RAID

Important
Considerations
• Redundancy
• Capacity
• Speed
• Robust


Speed: Dedicated hardware? Single point of failure? Parity Calculation? How long to rebuild a
drive? NUMBER OF SPINDLES!! Redundancy: How many drive failures? URE errors? Capacity:
Parity stripe or mirrors? (harder better faster stronger)

Spinning
Disks
In RAID

Advantages
• Linear Speed
• Price (Per Gigabyte)
• Well-Understood


Spinning
Disks
In RAID

Disadvantages
• Random Speed
• Price (per IOPS)
• Failure Rate
• Rebuild Speed


Storage Metrics


IOPS
• What are they?
• What aren’t they?


The Simpliﬁed Equation
IOPS = 1/(((R+W)/2)/1000) + (L/1000)
R = Average Read Time
W = Average Write Time
L = Average Latency


Rule of Thumb
Assumptions
RPM
5400

50-80

7200

80-100

10k

130-150

15k


IOPS

180-200

Determining IOPS
• Per Drive
• Manufacturer’s Stated Numbers
• Rule of Thumb
• Per RAID Array
• Write penalty

IO Proﬁling
•

•

Active Tools

•
•
•

Bonnie++
dd
Intel NAS Toolkit

Passive Tools

•
•

io(stat/meter/top), atop
Resource Monitor / Process Explorer


http://www.intel.com/products/server/storage/NAS_Perf_Toolkit.htm

Solid State Drive
Technology


NOR Flash
• Reads and writes are atomic single-bit
• Expensive
• Small speciﬁc use cases


Won’t talk about NOR much.

NAND Flash
• Reads are based on “read blocks” (4k)
• Writes are based on “erasure blocks”
• Cheap (and getting cheaper)
• Broad use cases

Read / Write Proﬁles
• Logical addresses abstracted from LBA
• No seek time
• Reads are generally very fast
• Writes are typically slower

Random and Linear IO have identical access time
Next slide: The magic

The Magic
Insulating
Barrier

Pure
Silicon

Doped silicon capable of
holding an electrical charge


Barrier is a dielectric ﬁlm (silicon oxide)

Quantum Tunneling

(transmission coefﬁcient for a particle tunneling through a single potential barrier)


Hot Carrier Injection
Storage / Erase uses Fowler-Nordheim Tunnel Injection / Release

Doped Silicon
Single Layer Cell (SLC)

Multi-Layer Cell (MLC)

Triple-Layer Cell (TLC)

Use charge pumps to get through the barrier
Each charge level has a binary state - 1 or 0

Gradual Destruction
Multiple cells need
multiple writes
Energy increases
with cell layers

Barrier accumulates
electrons

Electrical potential
difference of barrier
and cells disappears

Difﬁculty Going Forward
SLC

MLC

0

00

000

100

0000

0100

1000

1100

1

01

001

101

0001

0101

1001

1101

10

010

110

0010

0110

1010

1110

11

011

111

0011

0111

1011

1111


TLC

4LC

Density
• 3-Dimensional
• Charge levels
• Size of cells
• “Dot Pitch” (Cells Per Inch)
• 5nm, 3nm, 2nm
• Varies with “level” count

SLC / ESLC
• Low Density
• Single (bit) Level Cell
• Quick: 25µs Read / 200-300µ Write
• More robust & long wear time
• Write endurance near 100,000 cycles

Capacity expensive | only in 5nm / 3nm densities |

MLC / EMLC
• Reasonably High Density
• Two (bit) Level Cell
• Decently fast: 50µs Read / 600-900µs Write
• Medium lifetime
• Write endurance near 3,000 cycles

TLC
• Very High Density
• Three (bit) Level Cell
• Decently fast: 75µs Read / 900-1350µs Write
• Not very robust or durable :-(
• Write endurance ~ 1,000 cycles

Write Ampliﬁcation and
Garbage Collection


Block Sizes
• Read Block
• 4k (aka “page”)
• Erasure Block
• (Large) multiple of 4k
• aka “block”

e-ink parallel

256KB
erasure
block size

Write Ampliﬁcation

Written Data
Empty Cell


next - want to change the data in the upper right quadrant

Written Data
Empty Cell
New Data
Old Data


next - big chunk of new data to write

Written Data
Empty Cell
Old Data

New Data

Where does this go? We’re out of empty erasure blocks!

Written Data
Empty Cell
New data written
over old cell


w/o
TRIM

Written Data
Empty Cell
New data written
over old cell


w/
TRIM

Garbage Collection


IO Performance Proﬁles


Remember:
• Spinning Disks
• Linear is fast
• Random is slow
• Read marginally faster than writes
(sometimes)


writes slower when switching tracks

With SSDs:
• Reads are fast
• Writes are slow(ish)
• Random or linear doesn’t matter (as much)


SSD Performance
Overview
•


Depends on

•
•
•
•
•
•
•

Number of ﬂash chips in use
Number of busses from the processor
Performance of controller CPU
Contention
Bus speed
Number of erasure blocks used
Number of previous writes to ﬂash cells

• Chips
• IO Busses
• CPU Cores


Causes of Contention
• Legitimate use
• Garbage collection
• Legitimate (but latent) useage
• IO Blender!
(Bender Blender: http://bit.ly/10vc7Sf)

Latent: updatedb? atime? app-level garbage collection?
(t-shirt at threadless)

Bus Speed
• SATA - 3 or 6 Gb/s?
• IOPS Calc
• Can your controller handle your disks?


Read
• Very fast
• No seek time
• moderately improved over spinning
disk (linear - random greatly)

• Causes no damage to the media
• Generally scales up with capacity

Write
• Usually fast (depending on drive usage)
• No seek time
• highly improved over spinning disk
• Causes no damage to the media
• Generally scales up with capacity

Spinning Disk
Read/Write Matrix
Read
Linear
Random

Write

SSD
Read/Write Matrix
Read
Linear
Random

Write

Solid State in Practice


Solid State Form Factors


Removable Media


Drives


PCI Cards


Next slide: parts of an SSD

Parts of an SSD


Interface

USB

PCI

IDE (sadly?)

SATA/SAS

Controller
Main
Processor

RAM
Cache

Battery /
SuperCapacitor

I/O Bus Lanes

Flash Chips


If individual chip capacity is ﬁnite, how do bigger drives increase capacity? What does this
mean for performance?

Flash Controllers
• Flash Translation Layer (FTL)
• Stripe Writes
• Interpret bus instructions
• Wear Leveling
• Garbage Collection

Do the heavy lifting - single largest problem with ﬂash drives, without a doubt.

Flash Translation Layer
LBA (0...n blocks)

F

L

A

S

H

C

H

I

P

S

SSD Aspects & Concerns


Longevity
• Primarily determined by the class of ﬂash
• (e)SLC, (e)MLC, TLC
• Related to wear-leveling
• Under-reported capacity
• Short-stroking improves lifetime (not speed)

Partition Alignment
• Performance and longevity
• As big (or bigger) issue than it was in
spinning disks

• Native 4k read blocks
• Far larger erasure blocks
• larger than is practical for alignment

TRIM
• As a command, refers to ATA-8 spec
• SCSI equivalent is UNMAP, but both are
often referred to as TRIM.

• Does not immediately delete unused blocks
• Allows for GC

Linux calls this “discard” - TRIM refers

Linux TRIM Support
• EXT4 / XFS / JFS / BTRFS - Native using
‘discard’ option

• Consider NOOP or Deadline IO scheduler
• fstrim (part of util-linux) for R/W vols
• zerofree for R/O vols

fstrim & zerofree - userland - important for thin-provisioned volumes on SAN arrays which
support it. Check docs on schedulers for details - deadline prefers read queues (under /sys/
block)

OSX Trim Support
• Comes by default on factory-installed SSDs
• Trim-Enabler
• http://www.groths.org/trim-enabler/


ZFS and SSDs
• ZFS Intent Log (ZIL)
• Adaptive Replacement Cache (ARC)
• arc_summary can help you decide


ZIL is almost like a journal - ARC is a RAM cache that has disk backing it. SSDs can be L2ARC
- https://code.google.com/p/jhell/wiki/arc_summary

Filesystems in General
•
•
•

Standard journaling ﬁlesystems
Mount options (atime/relatime, etc), /tmp->tmpfs
Next-Gen

•
•
•

ZFS / BTRFS
Distributed Filesystems
DRBD


ZFS - SSD cache pool | ZFS/BTRFS are COW | DRBD no trim

Monitor Health w/
S.M.A.R.T.
• S.M.A.R.T. information
• vendor-speciﬁc
• Includes ﬂash erase count
• smartctl on Linux and Mac
• Dozens of tools on Windows (check wiki)

Forensics
...Our results lead to three conclusions:

built-in commands are effective, but
manufacturers sometimes implement them
incorrectly.
First,

overwriting the entire visible address space of
an SSD twice is usually, but not always, sufﬁcient to
Second,

sanitize the drive.

none of the existing hard drive-oriented
techniques for individual ﬁle sanitization are
effective on SSDs
Third,

Reliably Erasing Data From Flash-Based Solid State Drives
Michael Wei∗, Laura M. Grupp∗, Frederick E. Spada†, Steven Swanson∗
∗Department of Computer Science and Engineering, University of California, San Diego
†Center for Magnetic Recording and Research, University of California, San Diego

(http://bit.ly/fast11-wei-paper)

SSD-enhanced RAID
Array Considerations


Hardware / Software
Hardware RAID Controllers

• Dedicated CPU Power • Commercial Support
• Battery-backed storage • Proprietary Tech
Software RAID Controllers

• Trust (eyes on code) • Portability
• Excessive cost of HW • Spare CPU Cycles

single point of failure

TRIM / GC?
• Does the RAID software/device know
enough to pass along TRIM?

• Will the array eventually crawl because of
ongoing GC issues?


No software RAID that I know of supports it. Intel chipset for RAID0 with TRIM

Access Bandwidth
• How much data can a single drive transmit?
• How many drives are in the array?
• What is the aggregate bus speed to the
array controller?

• What is the bus speed to the host(s)?

SSD Throughput
Example
4.15Gb/s

From Tech Radar: http://bit.ly/100UhvY

Controller / Bus
• Speed / Ports
• How mature / reliable / tested?
Remember
Me?


Just because buses exist in a storage array oesn’t make them magic and inﬁnite in size

Tiering / Caching
Very fast SDRAM
SSD tier - hot blocks
Faster spinning disks
Very slow, cheap disks

Future Technology


Enhanced Capacity


Kowloon Walled City

Enhanced Longevity


Telomeres in chromosomes

Smart SSDs


Active Flash

What should I buy?


Questions?


Introduction to Solid State Drives

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (20)

Similaire à Introduction to Solid State Drives

Similaire à Introduction to Solid State Drives (20)

Dernier

Dernier (20)

Introduction to Solid State Drives