Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1zBtGDa. Dave McCrory talks about what is Data Gravity, how it affects performance and portability and why these effects are amplified when there are larger volumes of data. Filmed at qconlondon.com. Dave McCrory most recently served as SVP of engineering at Warner Music Group, where he led over 100 engineers building the company’s new Digital Services Platform, based on an open source enterprise platform as a service. His extensive experience in the cloud and virtualization industry included positions as a senior architect in Cloud Foundry while at VMware and as a cloud architect at Dell.

1. Don’t let Data Gravity
CRUSH your infrastructure
Dave @McCrory
CTO @Basho

2. InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
http://www.infoq.com/presentations
/data-gravity-infrastructure

3. Presented at QCon London
www.qconlondon.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide

4. Data Gravity
App

5. App
Network / Bus

6. App
Data
W
rites

7. App
Data
Reads

8. App
Data
App
App
App
W
rites
W
rites
W
rites
W
rites

9. App
Data
App
App
App
Data

10. App
Data
App
App
App
Data

11. App
Data
App
App
App
Data
App
App App
Which App is Most Advantaged?
App App

12. App
Data
App
App
App
Data
App
App App
Most Advantaged App
App App

13. Why?
Data
App
Most Advantaged App

14. Why?
Data
App
Most Advantaged App
Distance Matters on
the Network

15. Bit rate and Latency
App
Bit rate is measured in Bits / time
credit - http://en.wikipedia.org/wiki/List_of_device_bit_rates
10 Gigabit Ethernet (10GBASE-X) 10 Gbit/s 1.25 GB/s
Infiniband SDR 1× 2 Gbit/s 250 MB/s
Gigabit Ethernet (1000BASE-T) 1,000,000.0 kbit/s 125 MB/s 1998
OC-3072/STM-1024 159,252,000.0 kbit/s 19.907 GB/s
OC-192/STM-64 9,953,000.0 kbit/s 1.244 GB/s
DOCSIS v3.0 (Cable modem) 222,480.0 kbit/s 122.88 MB/s
Modem 28.8 (3200 baud)(V.34-1994) 28.8 kbit/s 3.6 kB/s 1994
Modem 2400 (600 baud) (V.22bis) 2.4 kbit/s 0.3 kB/s 1984
Morse code (skilled operator) 0.021 kbit/s .00265 kB/s 1844

16. -64.000.000
-48.000.000
-32.000.000
-16.000.000
0
16.000.000
32.000.000
48.000.000
64.000.000
80.000.000
1975 1980 1985 1990 1995 2000 2005 2010 2015 2020
Computer Bus Memory Local Area Net Wide Area Net Storage
KBps

17. -64.000.000
-48.000.000
-32.000.000
-16.000.000
0
16.000.000
32.000.000
48.000.000
64.000.000
80.000.000
1975 1980 1985 1990 1995 2000 2005 2010 2015 2020
Computer Bus Memory Local Area Net Wide Area Net Storage
KBps

18. -64.000.000
-48.000.000
-32.000.000
-16.000.000
0
16.000.000
32.000.000
48.000.000
64.000.000
80.000.000
1975 1980 1985 1990 1995 2000 2005 2010 2015 2020
Computer Bus Memory Local Area Net Wide Area Net Storage
KBps

19. -64.000.000
-48.000.000
-32.000.000
-16.000.000
0
16.000.000
32.000.000
48.000.000
64.000.000
80.000.000
1975 1980 1985 1990 1995 2000 2005 2010 2015 2020
Computer Bus Memory Local Area Net Wide Area Net Storage
KBps

20. -64.000.000
-48.000.000
-32.000.000
-16.000.000
0
16.000.000
32.000.000
48.000.000
64.000.000
80.000.000
1975 1980 1985 1990 1995 2000 2005 2010 2015 2020
Computer Bus Memory Local Area Net Wide Area Net Storage
KBps

21. -64.000.000
-48.000.000
-32.000.000
-16.000.000
0
16.000.000
32.000.000
48.000.000
64.000.000
80.000.000
1975 1980 1985 1990 1995 2000 2005 2010 2015 2020
Computer Bus Memory Local Area Net Wide Area Net Storage
Storage Bandwidth
Bandwidth
KBps

22. Local Area Net Wide Area Net Storage
TheSeparationisIncreasing

23. -36.000.000
0
36.000.000
72.000.000
108.000.000
144.000.000
180.000.000
1975 1980 1985 1990 1995 2000 2005 2010 2015 2020
Computer Bus Memory Local Area Net Wide Area Net Storage
75M
51M
21M 20M
3M - KBps

24. Bit rate and Latency
App
Latency is measured in Time
L1 cache reference 0.5ns
Branch mispredict 5ns
L2 cache reference 7ns 14x L1 cache
Mutex lock/unlock 25ns
Main memory reference 100ns 20x L2 cache, 200x L1 cache
Compress 1K bytes with Zippy 3,000ns
Send 1K bytes over 1 Gbps network 10,000ns 0.01ms
Read 4K randomly from SSD* 150,000ns 0.15ms
Read 1 MB sequentially from memory 250,000ns 0.25ms
Round trip within same datacenter 500,000ns 0.5ms
Read 1 MB sequentially from SSD* 1,000,000ns 1ms 4X memory
Disk seek 10,000,000ns 10ms 20x datacenter roundtrip
Read 1 MB sequentially from disk 20,000,000ns 20ms 80x memory, 20X SSD
Send packet CA->Netherlands->CA 150,000,000ns 150ms
credit - https://gist.github.com/jboner/2841832

25. 0,1
1
10
100
1.000
10.000
100.000
1.000.000
L1CacheReference
Branchmispredict
L2CacheReference
L3CacheReference
MutexLock/Unlock
QuickPathInterconnect
MainMemoryReference
Send1Kover1GbpsNetwork
Read4KrandomlyfromSSD
Read1MBsequentiallyfrommemory
RoundTripwithinthesameDC
Read1MBsequentiallyfromSSD
0,5
5 7
12
25
40
100
10.000
150.000
250.000
500.000
1.000.000
Latencies in nanoseconds
CPU BUS MEM NET SSD
(Logarithmic Scale)

26. 0,1
1
10
100
1.000
10.000
100.000
1.000.000
L1CacheReference
Branchmispredict
L2CacheReference
L3CacheReference
MutexLock/Unlock
QuickPathInterconnect
MainMemoryReference
Send1Kover1GbpsNetwork
Read4KrandomlyfromSSD
Read1MBsequentiallyfrommemory
RoundTripwithinthesameDC
Read1MBsequentiallyfromSSD
0,5
5 7
12
25
40
100
10.000
150.000
250.000
500.000
1.000.000
CPU BUS MEM NET SSD
Latencies in nanoseconds (Logarithmic Scale)

27. 25
40
100
10.000
150.000
250.000
500.000
1.000.000
CPU
BUS
MEM
NET
SSD
Latencies in nanoseconds (Logarithmic Scale)
1,500TimesSlower
1 ms

28. CRUSHWhy the ?

29. Data Creation
• Human Driven and Machine Driven
• Humans use Human Interfaces
(UIs are an example)
• Machines use Machine Interfaces
(APIs are an example)

30. Moore’s Law meets Big Data
• IDC estimates Data will double every 2 years through 2020
http://www.emc.com/collateral/analyst-reports/idc-the-digital-universe-in-2020.pdf
• McKinsey estimates Data will increase by 40% yearly
http://www.mckinsey.com/insights/business_technology/big_data_the_next_frontier_for_innovation
• Data production will be 44 times greater in 2020 than it
was in 2009
http://www.csc.com/big_data/flxwd/83638-big_data_just_beginning_to_explode_interactive_infographic
Data is doubling at an equal or greater
rate than CPU transistor counts!

31. Causes of Data Growth
• Big Data
• Internet of Things (Sensors)
• Machine to Machine
• Machine Learning

32. The Data Entropy
Problem

33. • Data is stored on a physical device (subject to
degradation)
• The device must consume power to make data
available
• Data is subject to “bitrot” aka error propagation
• The more data you have, the more costly it is to
maintain

34. Data Lake

35. • Data warehousing?
• Data needs to have context (structure)
• Data Entropy problem
• Junk Data is still Junk Data, no matter how long you
store it
• Each storage type has trade-offs
• When have this ever gone well?

36. Data Gravity’s
relationship with Latency

37. Amazon found every 100ms of latency cost them
Google found an extra .5 seconds in search page generation time
dropped traffic by 20%
A broker could lose
if their electronic trading platform is 5 milliseconds behind the competition
Businesses care about Latency
source: http://blog.gigaspaces.com/amazon-found-every-100ms-of-latency-cost-them-1-in-sales/
Shopzilla -
increase in revenue, a 50% reduction in hardware, and a 120% increase traffic
from Google
Bing found that a 2 second slowdown changed queries/user by -1.8% and revenue/
user by -4.3%

38. Moving Data to the
Processing

39. AppData

40. AppData

41. Moving Data to the
Processing
• How fast is the Bus/Network?
• How much data is being moved?
• How variable is the latency?
• How much latency is acceptable?
Considerations

42. Moving Processing to
the Data

43. AppData

44. AppData

45. Moving Processing to the
Data
• How fast is the Bus/Network?
• How much processing power is near the Data?
• How variable is the latency?
• How much latency is acceptable?
Considerations

46. Store multiple copies
of your Data

47. Clustering

51. Data Center 1 Data Center 2

52. Data Center 1 Data Center 2
Replication
and
Synchronization

53. Caching your Data and
using in Memory
Backends

54. Store Data in Memory (Sharded)

55. Store Data in Memory (Cluster)

56. Checkpointing and
Derivative Data

57. Data Proc

58. Data Proc

59. Data Proc

60. Data Proc
1

61. Data Proc
1

62. Data Proc
1
+1

63. Data Proc
2

64. Data Proc
2
+3

65. Data Proc
5

66. Data Proc
5
Check
point

67. Data Proc
5
Check
point
5

68. Data Proc
55

69. Data Proc
55
Error

70. Data Proc
Error5

71. Data Proc
5
5

72. Data Proc
55
5

73. Realtime (inbound)
processing of Data

74. Data DataProc
Queue Queue
Inbound

75. Data DataProc
Proc
Proc
Queue

76. Data
Data
Proc
Inbound
Queue
Data Archive
Queue
Queue
Proc

77. Lambda Architecture

78. Data
DataProc
Proc
Batch
Realtime
Queue
Queue
Data
Query
&
Merge

79. Archival

80. Data DataProc Archive
Inbound
“Raw” “Refined” “Expired”

81. Command Query
Responsibility
Segregation (CQRS)

82. Data Data
Writes Queries
Replication

83. Where from here?

84. Data
Data
Archive
Proc Data
Proc
Queries
Cache
Inbound Writes
ReadsQueue
Replication
Replication
Checkpoints
Derived Data

85. DataArchive Data
Writes
Reads
Geo
Replicated
Custer
Geo
Replicated
Custer
Geo
Replicated
Custer

86. Questions?
@mccrory on Twitter

87. Watch the video with slide synchronization on
InfoQ.com!
http://www.infoq.com/presentations/data-
gravity-infrastructure