Database acceleration with flash memory

1. Journey to the Open Grid
Soumik Sinharoy – Orange Silicon Valley
October 25, 2011, IBM Information on Demand – Las Vegas

2. Orange is ...
 world-leading communications
brand owned by France Telecom
 181 000 employees and 50.9 bn €
revenues in 2010
 serving over 216 million
customers in 32 countries over 5
continents - addl100m customers
from Asia and Africa by 2015
plus ...
 Over 2000 distribution outlets in
Europe
a leading retailer of premium content
& games with ~ 20 million downloads
per year in Europe
50 Orange application ~ 9 million
downloads from apps stores

3. 216 million customers worldwide…
United Kingdom
Poland
Belgium
LuxembourgSlovakia
Austria Moldova
France Romania
Switzerland
Spain
Armenia
Tunisia
Morocco
Jordan
Western Sahara Egypt
Mali
Niger
Senegal
Dominican Republic
Guinea Central African Republic
Guadeloupe Ivory Coast
Martinique Cameroon
Uganda Kenya
Guyana
Madagascar
Vanuatu
Botswana Mauritius
Countries where we provide services for Reunion Island
residential customers
Countries where we provide services for
business customers
our Group provides services for residential customers in 35 countries and for
businesses

4. 3,500 employees working on innovation
United Kingdom Poland
London Warsaw
8 cities in
France
Spain Japan
Madrid/Barcelona Tokyo
USA China
San Francisco Beijing
Jordan
Amman
Ivory Coast
Orange Innovation centers Abidjan
Egypt
Cairo

5. Orange Silicon Valley: disruptive
innovations for the future
 Participate in the disruptive innovations
 Partnership and business development with
companies, startups, and universities
 Introduce our business leaders to the latest
solutions from Silicon Valley
 Co-development with the ecosystem
benchmarking and diligence on new
technologies in order to frame
recommendations on technology and
business strategy for France Telecom
Group
 Executive Briefing -global customer
management teams, and policy-makers
about trends in the IT and communications
business

6. The Journey
 Open standards
 Open architecture
 Vendor neutral – level the playing field
for competition
 Carbon footprint- Reduce power
usage
 Improve end user experience
 Compact infrastructure footprint
 Dense Computing
– Compute density
– IO density
– Storage density
6

7. Infrastructure snapshot
 More than 30,000 of servers (EMEA)
 More than 50% deployments are 4-way
to 64-way
 Enterprise storage
– NAS (file sharing)
– FC SAN (for most purposes)
 Challenges
– Business growth demands for
bigger platform sizing
– Scalability barrier
– Compute density
– footprint
– Application performance
limitations
– Architecture limitations
– Interconnect technology
7

8. SMP => Grid
InfiniBand
App Servers
Database as Service Cluster
 Migration large databases from
SMP to GRID (IBM SystemX
Blades)
 90% reduction in server TCO
 InfiniBand QDR for server
interconnect
 8G/4G FC SAN for Storage
8

9. 2008 research study at IBM Montpellier: SystemX
blades with Intel Xeon processors
ib0 Sql*net ( IPoIB) + IO (SRP) IBM HS21 XM
IO (SRP)
Oracle RAC Cluster Apps Servers
Oracle RAC interconnect + IO (SRP)
ib1
IO (SRP)
Inter Switch Link
ISR9024M_DDR
ISR9024M_DDR
IB Storage
Achieved
# of nodes TPM Average ms Max IOPs
IBM LSI DS5000
4 763,038 44 27,880
6 1,162,022 22 36,870
8 1,665,183 33 46,067
9

10. Open Grid
Application / Database Server - INFINBAND  28 blades in IB QDR cluster
 FCoIB -> FC : Low Latency bridging
 FC SAN : capacity
IP to Fiber Channel - Gateway
Fiber Channel Storage - FIBER CHANNEL
10

11. 2010 study with IBM Sillicon Valley Labs : Integrate FC SAN with InfiniBand Cluster using DB2
PureScale
11

12. 2010 study: DB2 pureScale benchmark results
Scaling Results
60,000
50,000
– 10,000 TPS per
member (>3x 2008
40,000
study)
TPS
30,000 Average TPS
– 99% linear scalability
20,000
– data dips : transaction
10,000
anomaly
0
1 MBR 2 MBR 3 MBR 4 MBR 5 MBR
Members
Experiments 1-5 members
12,000
10,000
8,000 – Standard table contained 2,500,000 rows
1 MBR
2 MBR – Transaction mix 80% read, 20% write
TPS
6,000 3 MBR
– 40+ connections to each member with zero
4 MBR
5 MBR
think time
4,000
– Read transaction consists of 10 random
2,000
select statements
– Write transaction consists of 3 random
0 update statements
1 35 69 103 137 171 205 239 273 307 341 375 409 443 477 511
Seconds
12

13. Storage server equation
Need too many
servers for a
single disk
HDD: too many
spinning for few
servers!
The Inversion Phenomenon
13

14. SSD based Video Storage at a 4KM distance
 16 diskless 8-12 core servers
 4 KM InfiniBand WAN
 2780 DVD quality videos streamed
 Only 13Gbps bandwidth used
 This is only 60% of pipe capacity
 only 2 SSD – ioDrive Duo
 Demo
QSFP
SSD transceiver
4km
SSD optical
InfiniBand Edge QSFP InfiniBand Edge InfiniBand Core
Switch
link
transceiver Switch Switch

15. Flash Memory Adoption Trends, forecasts
Source IDC
Source :Flash Memory Summit „11
Worldwide Enterprise SSD Share by Interface, 2010–2015
IDC
View
IDC: 2011
Note:
+PCIe 2.5” will vie for SATA and SAS
Source Gartner share Source IDC
15

16. OPEN GRID: Scale out with Flash
Performance
Flash
T0
SAN
T1
Source : nutanix
Capacity
Server side scale out flash – Tier 0
Low cost capacity of existing SAN with HDD
16

17. Scale on Flash - unify fabric
 Scale out tier – 0 with PCIe flash
 ISV : manage cache ?
 PCIe : secondary buffer pool
 Unified IO
– IO Virtualization
– reduce interconnect cost
– transparent bridging
HBA
HBA
HBA
HBA
HBA
HBA
HBA
Unified IO :
17

18. 18

19. IBM SVL OSV : Experiment Results
 Stream Computing:  OLTP Cluster
– Processing transactions “In- – Rates measured using SAN
and IBM HighIOPS for logs
Motion”
– DB2 pureScale avg tps
– Fundamental Validations – using 2 x Data Members
Account, Merchant, Limit, Ra and SAN: 47K TPS
ting(AMT) – DB2 pureScale avg tps
using 2 x Data Members
– Able to handle “Burst” traffic and FLASH: 84K TPS
and cache database
– 2X Improvement using
operations FLASH storage for database
logs!
– performance threshold – not
reached yet : need to add
more stream servers
scalability effort continues : results to be published
19

20. LD JCTD Operational Utility Assessment
CREATE LOCALLY
Large
Data LD
LD
LD
LD
LD
“While the first of about 10 files was
still being transferred to the legacy
work station, Large Data had all ten SHARE GLOBALLY
files.”
LD JCTD IOUA Report, Nov 09, JHU/APL
Globally Synchronized, Shared Data and High Resolution Collaboration

21. Operational Demonstration Results
Operational Impact
• GEOINT access & web services for warfighters
• UNCLAS US Gov’t and NGO support
• Remote access to large, distributed ISR files
• Data virtualization & near real-time failover
“Simply put, the system NRL has in place for delivering
large, AOI detailed imagery is outstanding and truly a model for
the DoD/IC.”
– Sean Wohltman, Google Inc.
Performance Results Summary as a
Fraction of Theoretical Maximum Data Transfer Rate
RDMA/IB TCP/IP/Ethernet
NETWORK
Single Multiple Single Multiple
(Bandwidth
Efficiency) Stream Streams Stream Streams
Transition Metric
75% / 90% 80% / 90% 75% / 90% 80% / 90%
Threshold/Objective
Wide Area Network
Suitability 94% ++ 98% ++ 86% + 17%
(2,000 fiber miles)
• Demonstrated TRL-7/8/9 Long-Haul WAN
89% + 83% + 42% 22%
• Cost effective, open source (13,000 fiber miles)
• Commodity components FILE SYSTEM Multiple Multiple
Single File Single File
• Stability/availability on par with (Read/Write) Files Files
operational systems Transition Metric
60% / 80% 60% / 80% 60% / 80% 60% / 80%
Threshold/Objective
“A quick overview of the system was all that Long-Haul WAN
72% ++ 94% ++ 34% 6%
was required for ease of use” (13,000 fiber miles)
-- LD User Meets Threshold + / Meets Objective ++
Effectiveness: LD met or exceeded Transition Thresholds

22. RDMA transport : Long Haul ?
RDMA transport : Global Range
?

23. developing the networks of the future
the challenges ahead
 respond to the massive increase in traffic: by 2015, we expect the standard level of
mobile data traffic to have risen by 26 times
 expand coverage and increase our speed to keep pace with the digital revolution
 constantly improve the quality of our services
 renew our infrastructures while respecting the environment

24. 40G InfiniBand over 370 KM : ESNet ANI Tested
 Orange Silicon Valley qualify
world’s first long haul 40G
InfiniBand solution
 Collaboration with US
Department of Energy
 Field trial on DoE 370KM
circuit
90%+ transport efficiency
under concurrent streaming
Maximum bandwidth
Unidirectional 3.8 GB/s,
Bidirectional 7.6 GB/s

25. Open Grid:
Cross boundaries : global scale
 Dissemination of
information throughout
Grid Grid Grid
the enterprise
WAN  Synchronize distributed
enterprise data centers
Storage Storage Storage
Server Server Server
Data
 Ultra Low Latency
Data Data
Center Center Center Fabric across long haul
Storage Network Backbone :
CONVERGENCE PHENOMENON
25

26. Seattle Nov - 2011
OC 768
Streaming (?)
SSD
Video streams
SEA
CHI NYC
MANLAN at AoA
6,000 miles !
SLC
BNL
continental US
World’s first 40Gbps Long Haul InfiniBand !!

27. Thank You
27