In this presentation from the DDN User Meeting at SC13, Jeff Denworth provides a product update, Watch the video presentation: http://insidehpc.com/2013/11/13/ddn-user-meeting-coming-sc13-nov-18/

1. Store
Process
Analyze
Collaborate
Archive
Cloud
Product Update
#BurstForward
Jeff Denworth
VP, Marketing
Invent
Discover
Compete
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
1

2. DDN End-to-End HPC Cluster Expertise
Applications
Ex. 2x-5x 3DSRMME
Gains
Cluster Mgr.
Dynamic Cluster
Optimization (End-End)
Compute
SFA-E, IME
Converged Systems
ddn.com
Networking
Ethernet, InfiniBand,
WAN/Cloud
Schedulers
SFX APIs: Prefetch,
User Priority
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
2

3. Product Portfolio: Three Strong Pillars
Analytics
Platforms
[x]Scalers: Accelerating
Time to Insight Across
Markets and Industries
SFA: Block and File,
SSD and Disk,
Processors and Storage
Object Storage
WOS: Cloud and Data Centers,
Internet, Web and Telcos,
Massive Scale and Low Latency
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
3

4. Big Data & Cloud Infrastructure
DDN 2013 Announced Product Portfolio
Cloud
Tiering
Petascale
Lustre* Storage
Analytics Reference
Architectures
Enterprise
Scale-Out File Storage
Cloud Foundation
WOS® 3.0
Big Data Platform
Management
EXAScaler™
GRIDScaler™
10Ks of Clients
1TB/s+, HSM
Linux HPC Clients
NFS & CIFS
~10K Clients
1TB/s+, HSM
Linux/Windows HPC Clients
NFS & CIFS
32 Trillion Unique Objects
GeoReplicated Cloud Storage
256 Million Objects/Second
Self-Healing Cloud
Parallel Boolean Search
Storage Fusion Architecture™ Core Storage Platforms
DirectMon
SFA12Kx
SFA7700
48GB/s/1.4M IOPS
1,680 Drives:
16U-2 Racks
Embedded Computing
13GB/s, 400K IOPS
60 Drives in 4U;
396 Drives in 20U
WOS7000
Flexible Drive Configuration
SATA
SAS
SSD
SFX Automated Flash Caching
•
•
Read
Write*
•
•
60 Drives in 4U
Self-Contained Servers
Context Commit
Instant Commit
* Future Release
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
4

5. Big Data & Cloud Infrastructure
DDN 2013 Announced Product Portfolio
Cloud
Tiering
Petascale
Lustre* Storage
Analytics Reference
Architectures
Enterprise
Scale-Out File Storage
Cloud Foundation
Lots of Amazing Products
WOS® 3.0
32 Trillion Unique Objects
GeoReplicated Cloud Storage
256 Million Objects/Second
Self-Healing Cloud
Parallel Boolean Search
GRIDScaler™
More Than We Can Cover Today...
Big Data Platform
Management
EXAScaler™
10Ks of Clients
1TB/s+, HSM
Linux HPC Clients
NFS & CIFS
~10K Clients
1TB/s+, HSM
Linux/Windows HPC Clients
NFS & CIFS
Focus For This Talk:
Storage Fusion Architecture™ Core Storage Platforms
• DDN’s HPC Engineering Agenda
SFA12Kx
SFA7700
• SC’13 New Product Update
DirectMon
48GB/s/1.4M IOPS
1,680 Drives:
16U-2 Racks
Embedded Computing
13GB/s, 600K IOPS
60 Drives in 4U;
396 Drives in 20U
WOS7000
Flexible Drive Configuration
SATA
SAS
SSD
SFX Automated Flash Caching
•
•
Read
Write*
•
•
60 Drives in 4U
Self-Contained Servers
Context Commit
Instant Commit
* Future Release
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
5

6. Our HPC Focus
Price
/Performance
Infinite
Research
Collaboration
& Data Ubiquity
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
Scale
Simple
Infrastructure
6

7. Razor-Focused On The Cost of Computing
Fully-Parallel,
State Machine I/O
HPC’s Most Aggressive
Caching Strategy
Best HW Efficiency
Least Electronics / perf. unit
Price
/Performance
• HW: RAM, Flash
• SW: APIs, Buffers
Save on HDDs, Racks, Power
Increase Cluster Efficiency
Data Center Density
& Storage Capacity
Compute/Store
Convergence
•
•
•
•
ddn.com
Best Slot per RU
Least Electronics / perf
Most Disks / system
WOS = 99% platter util.
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
•
•
Eliminate I/O Server Layer
Buffers can save $MMs
7

8. Breaking Down Barriers To Data Access.
Protocol Barriers
Research
Collaboration
& Data Ubiquity
Geographic Barriers
Aggressively Enabling
Multi-Protocol Platforms
Cloud Storage Technology
Can Enable Rapid Insights
Single Namespace, Ubiquitous Data
• Parallel File Systems
• Cross-Protocol NAS
• Even Object Storage
DDN is actively engineering multiprotocol namespace management
•
•
•
•
New: DDN GRIDScaler
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
Massive Namespace
Easy OID Management
Robust Sync Tools
Advanced, WAN Data Protection
Add Parallel File Systems, and Presto!
New: DDN WOS Bridge
8

9. Infinite Scalability Delivers True Power To Your Apps
Namespace Level
Infinite
Scale
Storage Level
Engineering Around
POSIX Locking
Real-Time Data Delivery (HDD)
•
•
Totally Distributed Algorithms
Buffering to Relieve I/O Pressure
•
•
Non-Deterministic Data
Placement & Retrieval
•
•
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
Guaranteed Response SLAs
Eliminate Rebuild Latency
Eliminate File Layout Hostpots (Buffer)
Load Aware Write & Read (WOS, Buffer)
9

10. Sophisticated I/O Technology, Management Simplicity.
One Tool;
Any Platform
SFA, GridScaler,
ExaScaler, Clusters,
Hadoop, WOS
Simple
Infrastructure
Self-Healing Storage
•
•
ddn.com
SFA: Drive Power Cycling
WOS: Data Protection
Policies Always-Conform
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
Compute/Store
Convergence
DDN SFA Appliances eliminate
I/O servers & networking to
reduce systems by <90%
Self-Notifying System
Management
Now supporting Call Home!
10

11. Proven Blueprints; Optimized for TCO Scale and Speed
And more...
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
11

12. DDN Delivers the Most Powerful &
Comprehensive End to End Stack for HPC
Parallel File
System Clients
NAS-Attached
Clients
DirectMon™
Infinite Memory Engine™
Burst Buffer System
Multi-Protocol Access
DDN Scale-Out Parallel File Systems
ReACT™
and
SFX™
Storage Caching
WOS™
Bridge
Cloud Connector
and File System
Data Replicator
SFA™ Storage Appliances
One Integrated HPC Storage Stack from The HPC Leader
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
12

13. The World HPC Champion is Now Even Faster!
Introducing The SFA12KX Family
§
§
§
§
§
World-Leading Parallel File I/O Foundation
Best-In-Class IOPS & B/W for All Workloads
Multi-Level Write and Read Cache, Cache APIs
Deep Capacity, Leading Data Center Density
Server-Virtualized To Embed HPC File Systems
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
2 New Models:
SFA12KX
48GB/s
SFA12KE
23GB/s
13

14. SFA7700: World’s Fastest Scale-Out Hybrid Platform
DDN’s Departmental Storage Powerhouse
The Speed of Flash, The Cost of Disk.
The Power of SFA-OS, All in Only 4U
Fully-Featured Like the SFA12KX
§ SFX Flash Cache and API-Enabled Caching
§ Next Gen DirectMon Appliance Management
§ World Leading Hybrid Price Performance
§ 13 GB/s, 400K Cache IOPS, 900TB in 12U
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
SFA7700
14

15. DDN Revolutionary Caching Boosts HPC Application Performance
Write Distribution for Multi-Disciplinary HPC Cluster
Sampling from ORNL Testing
Caching Systems Are HPC’s Shock Absorbers
• DeBlends Unaligned Writes
• Preserve Small I/O Performance
• Smoothes Out Mixed I/O Inconsistency
• Significantly Improves Storage Utilization
Source: ORNL
70% of I/Os are <512KB, 50% are <16KB
58% write I/Os vs. 42% read I/Os
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
15

16. SFX Improves Application Performance by Up to 700%
Context Commit API
In-Band - Out Of Band
Usage & Priority-Based Caching
Aligned Writes To Flash
Partial Writes to RAM
ReACT
DRAM Cache
SFX Write* Instant Commit
SFX Read
HDD Tier
Cache Flush
* 2014
Cache Warm
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
16

17. DDN Application-Defined Caching Eliminates Cluster Idle Time
In-Band File System Hint Awareness; Out-of-Band Middleware & DB Structure Caching
Out-of-Band API Control
Enables Dramatic Efficiency Gains
Application
I/O Idle
I/O Idle
Load
Operating System
File System Client
Time
CPU
Scheduler
File System ‘Hint Sniffer’ [Technical Term J]
API
Out of Band Ethernet
I/O
Load
CPU
I/O
SFX
SFX
Prefetch
Prefetch
Time
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
SFX is equally valuable in telling the
system what not to cache
17

18. DDN Boosts File System Speed, Access and Reliability
Our Value-Add Approach To Accelerate I/O,
Minimize TCO & Facilitate Collaboration.
GRIDScaler™
EXAScaler™
ddn.com
§
§
§
§
§
Open-Platform File System Agenda
Data Ubiquity: NAS, Parallel IO, Cloud
Caching Tools To Accelerate I/O, APIs
Collaborate and Archive Via The Cloud
Easy To Manage Any Platform, Any Scale
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
18

19. WOS 3.0: The World’s Most Scalable Web Storage
Simple, Hyper-Efficient Object Storage from The Performance Leader
32 Trillion
Object Scalability
10TB/s
256M
throughput
objects/sec
Up To 64 Remote Sites
API, REST, NAS Access
99.99999%+
Best-In-Class Cloud Availability
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
19

20. WOS Delivers True Cross-Campus and Global Collaboration
Compute
Grid
WOS™ Storage Cloud
Site 1
Compute
Grid
Site 3
• Eliminate Research Data Silos
• ‘Push Model’ Data Distribution
Compute •
Synchronize Independent File
Grid
Systems for Simple Replication
• Access Data From A Variety of
Site 4
Protocols
Compute
Grid
Site 2
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
20

21. DirectMon Next Generation Monitoring Tool
•
•
•
•
•
•
•
ddn.com
New Web Interface
Deep Statistical Monitoring
SFA Configuration Tools
‘Scaler Dashboard & Config
Log & Config Management
Call Home Enabled in Q4
Scale-Out Cluster Manager
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
21

22. DDN Delivers the Most Powerful &
Comprehensive End to End Stack for HPC
Parallel File
System Clients
NAS-Attached
Clients
DirectMon™
Infinite Memory Engine™
Burst Buffer System
Multi-Protocol Access
DDN Scale-Out Parallel File Systems
ReACT™
and
SFX™
Storage Caching
WOS™
Bridge
Cloud Connector
and File System
Data Replicator
SFA™ Storage Appliances
One Integrated HPC Storage Stack from The HPC Leader
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
22

23. Store
Process
Analyze
Collaborate
Archive
Cloud
#BurstForward
Invent
Discover
Compete
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
Exascale I/O Platform
Technology Introduction

24. Evolution of Data-Intensive Computing
CPU
Average Top500
Cluster Concurrency
2010
12 Threads per Node
13,000 Cores
2018
(est.)
O(1,000) Threads per Node
57,772,000 Cores
4,400x Increase
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
24

25. File Systems Will Not Survive Hyper-Concurrency
File System Degradation vs. Concurrency
Million and Billion Core
Computers Need Smarter
Solutions To Scale Applications
and Meet Uptime Requirements
Concurrency Degree (Percent)
100
90
80
<Today>
<~2015>
<Exascale>
70
60
50
40
30
Wall-Clock
Degradation
20
10
0
1
10
100
1000
10000
100000
Source: DDN Testing; Exascale Research
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
25

26. NVRAM Presents New Architectural Opportunity
NVRAM Is A Viable HPC Tier, Today
$/MB/s: $0.29 (91% less)
$/GB: $1 (1000% more)
W/GB/s: 0.003 (98% less)
And Performance Gains Outpace HDD
1E+10
Widening
GHz
100000000
106 to 108
1000000
10000
$/MB/s: $3.19
$/GB: $0.1
W/GB/s: 0.13
IOPS
100
IOPS
1
2000
HDDs (IOPS)
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
2010
Flash
2018
CPU Operations
26

27. Store
Process
Analyze
Collaborate
Archive
Cloud
#BurstForward
Invent
Discover
Compete
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
27

28. Introducing a Revolutionary Exascale/Everyscale Architecture
Leveraging 15 Years of HPC Expertise,
Engineering & Deployment Leadership
•
•
•
•
•
Dramatically Reduces HPC Computing Cost
Enhances Performance & Power Usage by Orders of Magnitude
Reduces Network & Server Amount & Complexity by Orders of Magnitude
Patented Algorithms Deliver Unlimited Scale
Liberates Applications From File System Bottlenecks
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
28

29. Next-Generation Distributed Memory
System from the Worldwide HPC Leader
•
•
•
•
•
•
ddn.com
Speed Up Cluster I/O 1000x
Divide Number of Disk Drives by a Factor of 10
Divide Rack Space by a Factor of 10
Divide Power Needs by a Factor of 10
Break Through POSIX Scaling Barriers
Achieve Unlimited File System Scalability
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
29

30. Infinite Memory Engine Delivers Unparalleled Scale
IME Metadata Scaling (24 Concurrent Threads)
Frags/Sec (Million)
200
150
Linear Scalability!
100
50
0
0
1
2
3
Proven IME
Scaling
IM 'Max'
ddn.com
4
5
Number of IME Nodes
6
7
8
9
200 PFLOPS Bid
Requirement
Trinity-Pessimistic-Req
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
30

31. The Infinite Memory Advantage
Limitless Scalability
Patented DDN Algorithms
Fully POSIX & HPC Compatible
No Application Modifications
Scale-Out Data Protection
Distributed Erasure Coding
Non-Deterministic System
Write Anywhere, No Layout Needed
Integrated With File Systems
Designed to Accelerate Lustre*, GPFS
No Code Modification Needed
Writes: Fast. Reads: They’re Fast Too.
No other system offers both at scale.
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
31

32. IME Completes DDN’s HPC-Leading Caching Strategy
The Ultimate I/O Deblender
API
SFA
Storage Fusion
Architecture
Control
RAM
7μs
SFX
Flash Tiering
SSD
50μs
HDD
17ms
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
32

33. IME HW Savings Grow As Moore’s Law Advances
IME Savings Grow
Greater over Time
Average HPC Cluster Example (2018)
Machine Size/Checkpoint
100M Threads, 2PB Checkpoint in 5m
100%
Disk Only
Node Local IME + SFA
80%
7 TB/s
8 TB/s
60%
33,400
3,240
40%
100
12
$33M
$7M
The Growing HDD Gap
Peak Performance
Spinning Disks
Racks
20%
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
0%
62%
80%
2016
Storage Acquisition Cost
2018
33

34. Store
Process
Analyze
Collaborate
Archive
Cloud
Live Demo Running In Denver!
#BurstForward
Invent
Discover
Compete
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
34

35. Worldwide Debut!!
Running Live In DDN’s SC’13 Booth.
Node Local Buffer Storage Topology
Hybrid Compute & Storage
14 x Commodity Compute Nodes
Each Node with 7 Low-Cost, Node-Local SSDs
High-Speed Persistence Layer
File System Replay to SFA7700 w/ Lustre*
Near-Perfect Disk Utilization; Fully-Ordered I/O
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
35

36. Demo Comparative Testing: Shared Writes
14 IME Clients (Dell nodes); 98 SSDs
Linear Cluster Scaling
CLUSTER LEVEL TESTING
DDN GRIDScaler™
IME( overall)
6,225 Concurrent Write Requests
43 GB/s
49 GB/s
12,250,000 Concurrent Write Requests
17 MB/s
49 GB/s
DDN GRIDScaler™
(per SSD)
IME
(per SSD)
62.5 Concurrent Write Requests
438 MB/s
500 MB/s
125,000 Concurrent Write Requests
170 KB/s
500 MB/s
DISK LEVEL TESTING
SSDs behind a PFS don’t help
IME is at line rate to scale with SSD rates
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
2,669x More
Metadata
Scalability
Simulates System 1/4 th
the size of The Average
Top500 in 2018.
Avg. 2018 Top500
Cluster Concurrency
57,772,000 Cores (est)
36

37. From Exascale to Everyscale, DDN Is THE HPC Leader
The Worldwide HPC Storage Leader
▶ Liberating Applications From
Infrastructure & File System Scaling Limits
▶ The Most Advanced HPC Stack To
Minimize The Cost of Computing
▶ Unleashing Global Collaboration
▶ Simplifying Scale-Out for Everyscale
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
37

38. See You On The Floor...
• We Have Terrific Speakers Lined Up Today!
• Over 90 People From DDN At SC’13
• CTOs & Engineers Are Here In Force
• Discussing Anything/Everything HPC
• ExaScale to EveryScale
• Geek Out On Our $M Demos
• WOS, SFA, IME, Lustre*, Analytics, Etc.
• Plan The Future of HPC In Our Whisper Suites
See You @ Booth 1307
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
38