Storage Geeks - Enterprise Storage 101 for 2019. Introduction to the current technologies in storage hardware, firmware and enterprise storage networking.

1. STORAGE GEEKS
101 - 2019
Andrew McGee
NOV 2019
@Andrew_McGee

2. STORAGE GEEKS
A Short History of
Electromagnetic Disk Drives

3. The Hard Disk Drive
Diagram: I, Surachit, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=4275514
Two main avenues of technology
improvement:
● Capacity/Areal Density
● Performance
HDD Advancement

4. Engineers VS The Laws of Nature
Engineering Corner
● Nano tech
● Miniaturization
● Materials Science
Engineering
Thermal instability at nanoscale
causes magnetic particles to flip.
Nature
Superparamagnetic
Effect

5. Perpendicular - PMR
Increased storage density over
longitudinal recording.
Increased platter thickness &
complexity.
● 2005 - Toshiba
● 2006 - Seagate
● 2006 - Hitachi
● 2006 - Western Digital
● 2006 - Fujitsu
HDD Capacity Advancement
Diagram: Public Domain image by Luca Cassioli 2005, https://commons.wikimedia.org/wiki/File:Perpendicular_Recording_Diagram.svg

6. Shingled - SMR
Increased storage density by 25%.
New tracks overlap old ones and must
overwrite them. Write penalty!
Host or Device managed.
libzbc - Zoned Block Device library
manages write performance.
● 2013 - Seagate
● 2014 - HGST
HDD Capacity Advancement

7. Helium Filled
30% less weight to data ratio
25% higher MTBF. 23% less power.
Using SMR or PMR hermetically
sealed and filled with helium.
● 2013 - HGST 7 platter 6TB
● 2016 - Seagate 7 platter 10TB
● 2017 - Toshiba 9 platter 14TB
● 2019 - Seagate 9 platter 16TB
● 2019 - HGST 9 platter 18TB
HDD Capacity Advancement

8. Heat Assisted - HAMR
Increased storage density by 10x.
Laser heats the magnetic material
before writing.
Heat, write and cool in 1 nano second.
Seagate
● 2019 - 16TB
● 2020 - 20TB
● 2024 - 48TB
HDD Capacity Advancement
"The technology is very, very difficult, and
there has been a lot of skepticism” - IDC 2013

9. Microwave Assisted - MAMR
Alternative to HAMR backed by HGST
and Toshiba.
Spin Torque Oscillators (STO) to
generate 20-40 GHz microwaves
HGST
● 2019 - 16TB
● 2019 - 18TB
● 2020 - 20TB
HDD Capacity Advancement

10. Bit Patterned Media - BPM
A future technology to increase areal
density demonstrated by Toshiba.
Media is pre-patterned into magnetic
islands using nano-lithography.
1 grain per bit instead of 20-30 today
to avoid superparamagnetism.
Possible areal densities up to 20-300
Tb/in2 as opposed to the 1 Tb/in2.
● 2010 - Toshiba PoC
HDD Capacity Advancement

11. Common Connection Protocols
SAS 12Gbps Serial Attached SCSI - Enterprise drives based on SCSI command set
NL-SAS 12Gbps SAS Controller with SATA based media, disk and performance
SATA 6Gbps SATA - Mostly consumer based on Parallel ATA command set

12. HDD Summary
Researchers fight for areal density against superparamagnetism
Techniques such as PMR, SMR, HAMR and MAMR increase areal density
Helium filled HDD’s allow for more platters in same form factor

13. STORAGE GEEKS
A Short History of Solid
State Drives

14. The Solid State Drive
Diagram: Toniperis, CC BY-SA 4.0, https://commons.wikimedia.org/wiki/File:NAND-ssd.svg
NAND Flash Memory storage
DWPD instead of MTBF
Two main avenues of technology
improvement:
● Capacity/Density
● Reliability
SSD Advancement

15. Flash NAND Types
SLC Single Level Cell 1 bit per cell - Enterprise class, expensive - 100,000 e/w cycles
MLC Multi Level Cell - enterprise & consumer, higher capacity, 10,000 e/w cycles
TLC Triple Level Cell - cheapest, highest capacity, consumer, 5,000 e/w cycles
3DXP 3D XPoint (Intel Optane) not based on NAND does not store a current

16. Flash Controllers
Embedded processor with firmware
Critical to drive performance
FTL provides logical to physical
mapping and LBA
● Bad block mapping
● Wear leveling
● Queuing/Caching
● Garbage collection
Flash Storage Brain
Diagram: Dmitry Nosachev, CC BY-SA 4.0, https://en.wikipedia.org/wiki/File:NAND_Flash_Pages_and_Blocks.svg

17. Common Connection Protocols
SAS 12Gbps Serial Attached SCSI - Enterprise drives based on SCSI command set
M.2 Form factor that can use SATA or PCIe (NVMe)
SATA 6Gbps SATA - Mostly consumer based on Parallel ATA command set
PCIe 31.5 Gbps PCIe 3.0 x4 - typically uses NVMe command set

18. SSD Summary
Flash resiliency based on SLC, MLC or TLC NAND
Flash controllers use sophisticated techniques for endurance and performance
Traditional or new connection types and form factors

19. STORAGE GEEKS
A Short Evolution of
Enterprise Storage

20. Block Storage Network - SAN
Modular or monolithic, Intelligent
controllers
Redundant multi pathing, dedicated
network
FC, iSCSI, FCoE, NVMe, IB
Block Characteristics:
● Very fast
● Data written in blocks
● Not human friendly
● Complex addressing
Storage Array
Ethernet Switch
FC Switch
Server
Clients
Storage Array

21. File Storage Network - NAS
Usually Modular, Intelligent controllers
Redundancy via dedicated or existing
ethernet network
NFS, CIFS
File Characteristics:
● Pretty fast
● Data written in files
● Human friendly paths
● Standard networking
Storage Array (Filer)
Ethernet Switch
Servers
Clients
NAS Storage

22. Object Storage - Cloud
Usually massively scale-out
Redundancy via multiple data copies
REST API (http), Swift, S3
Object Characteristics:
● Not usually fast
● Data written in objects
● Eventually consistent
● Human friendly metadata
● Cloud networking
Content Addressable Storage
Ethernet Switch
Servers
Clients
Object Storage
WAN

23. Data Protection Techniques
RAID Redundant Array of Independent Disks
RAID 0 - striping only, no protection
RAID 1 - exact mirroring
RAID 5 - 5D+1P, parity blks striped
RAID 6 - 4D+2P, parity blks striped
Performance only
Least capacity
1 disk fail
2 disk fail

24. Data Protection Techniques
RAID Parity and striping across block devices to create sets of redundancy
Multipath Redundant network paths from host to storage (dual HBA/NIC at host)
EC Erasure coding saves data in fragments with parity across different locations
Mirror Storage array level synchronous and asynchronous mirroring of data (DR/BC)
Cache Battery or super-capacitor backed up cache

25. Data Optimisation/Reduction
Compression Algo to reduce redundant blocks, whitespace etc (unstructured data)
Deduplication Avoids storing duplicate blocks (CPU intensive, rehydration, mapping)
● RAW Storage
● Usable storage
● Effective storage
● Decimal GB vs GiB
Storage Vendor Terms

26. Storage Array Summary
Block, File and Object storage characteristics
Data protection techniques employed by storage - RAID, EC, Mirroring
Data reduction techniques employed by storage - Compression and Dedupe

27. STORAGE GEEKS
A Quick Look at CEPH

28. Ceph Storage
Object Built on scale out object storage but also provides interfaces to block and file
REST Uses http to provide S3 or Swift compatibility
Metadata Metadata servers to control data mapping and redundancy
CRUSH Hashing algo to calculate storage and retrieval of data in distributed cluster
SDS Software defined storage abstracts the intelligence from the hardware

29. STORAGE GEEKS
Andrew McGee
NOV 2019
@Andrew_McGee
THANKS!

30. Reference
03 - The HDD
https://en.wikipedia.org/wiki/Hard_disk_drive
04 - Engineers VS Nature
https://en.wikipedia.org/wiki/Superparamagnetism
05 - PMR
https://en.wikipedia.org/wiki/Perpendicular_recording
06 - SMR
https://en.wikipedia.org/wiki/Shingled_magnetic_recording
https://www.anandtech.com/show/7290/seagate-to-ship-5tb-hdd-in-2014-using-shingled-magnetic-recording
07 - Helium
https://www.backblaze.com/blog/helium-filled-hard-drive-failure-rates/
https://www.anandtech.com/show/10106/western-digital-introduces-its-consumer-helium-drives
08 - HAMR
https://blog.seagate.com/intelligent/hamr-next-leap-forward-now/
https://en.wikipedia.org/wiki/Heat-assisted_magnetic_recording
https://fstoppers.com/originals/hamr-and-mamr-technologies-will-unlock-hard-drive-capacity-year-326328
https://www.backblaze.com/blog/hamr-hard-drives/
09 - MAMR
https://www.theregister.co.uk/2019/03/07/hamr_and_mamr_hdd_direction_debate/
https://blog.westerndigital.com/mamr-technology-breakthrough/
https://www.anandtech.com/show/13764/western-digital-2019-16tb-hdd-mamr-hamr
https://www.anandtech.com/print/11925/western-digital-stuns-storage-industry-with-mamr-breakthrough-for-nextgen-hdds
10 - BPM
https://www.toshiba.co.jp/rdc/rd/fields/11_e09_e.htm
11 - Connection Protocols
https://en.wikipedia.org/wiki/Serial_Attached_SCSI
https://en.wikipedia.org/wiki/Serial_ATA

31. Reference
14 - The SSD
https://en.wikipedia.org/wiki/Solid-state_drive
15 - Flash NAND
https://www.mydigitaldiscount.com/everything-you-need-to-know-about-slc-mlc-and-tlc-nand-flash.html
16 - Flash Controllers
https://en.wikipedia.org/wiki/Wear_leveling
https://en.wikipedia.org/wiki/Write_amplification
17 - Connection protocols
https://en.wikipedia.org/wiki/M.2