Contenu connexe Similaire à Terracotta Ditch the Disk webcast (20) Terracotta Ditch the Disk webcast1. © 2013 Terracotta Inc. | Internal Use Only
Ditch the Disk:
Designing great
in-memory
architectures
2. © 2013 Terracotta Inc. | Internal Use Only 2
Your speakers
Gagan Mehra
Chief Evangelist
Terracotta
Orion Letizi
Co-founder
Terracotta
3. © 2013 Terracotta Inc. | Internal Use Only 3
What we’ll cover in this webcast
• Why enterprises are ditching their disks
• The top challenges in designing
great in-memory architectures
• Emerging best practices
• Case study: AdJuggler
• How to start ditching your disks
• Q & A
4. © 2013 Terracotta Inc. | Internal Use Only 4
4© 2013 Terracotta Inc. | Internal Use Only
Why enterprises are
ditching the disks
5. © 2013 Terracotta Inc. | Internal Use Only 5
The Internet has revealed weaknesses in the
standard disk-based architecture
6. © 2013 Terracotta Inc. | Internal Use Only 6
Ad-hoc data management built into applications
results in inconsistent speed, scale, and reliability
7. © 2013 Terracotta Inc. | Internal Use Only 7
The in-memory data management revolution
From disk to RAM
Memory
90% of Data in
Database
Database
90% of Data in
Memory
Slow
Expensive
Difficult to scale
Ultra fast
Cost-efficient
TB-scale servers
modernize
8. © 2013 Terracotta Inc. | Internal Use Only 8
In-Memory
Explosion in
volume of
business data
Big Data
Why in-memory now?
Low-cost RAM meets Big Data
Unlock the value in your dataMaximize inexpensive memory
Steep drop in
price of RAM
9. © 2013 Terracotta Inc. | Internal Use Only 9© 2013 Terracotta Inc. | Internal Use Only 9
Who’s ditching the disk?
FINANCIAL
SERVICES
GOVERNMENT TELECOMMUNICATIONS
MEDIA
ENTERTAINMENT
ECOMMERCE
FRAUD
DETECTION
TRANSPORTATION HEALTHCARE TRAVEL TECHNOLOGY
10. © 2013 Terracotta Inc. | Internal Use Only 10© 2013 Terracotta Inc. | Internal Use Only 10
The business case for ditching the disk (ROI)
Additional revenue/profit
$10 million to $2 billion (based on media, financial services, e-commerce)
Ability to handle more customers (speed at scale)
Smarter selling and cross-selling
Faster insights
Database license savings:
Oracle Enterprise edition per processor = $47,500 + 20% license support
Other ROI opportunities:
Reduced monitoring and management overhead (people and tools)
Reduced penalties for failing to meet SLAs
Smarter business decisions with faster access to data
11. © 2013 Terracotta Inc. | Internal Use Only 11© 2013 Terracotta Inc. | Internal Use Only 11
A clear correlation with Big Data success
44% of best-in-class Big Data
performers are already using
in-memory data management, and
more are planning to deploy it.
0% of Big Data laggards use in-
memory data management.
According to Aberdeen Group*:
*In-memory Computing: Lifting the Burden of Big Data (Jan 2012)
12. © 2013 Terracotta Inc. | Internal Use Only 12
12© 2013 Terracotta Inc. | Internal Use Only
The top 6 challenges in designing
great in-memory architectures
13. © 2013 Terracotta Inc. | Internal Use Only 13
PERFORMANCE
Achieving predictable low latency to Big Data1
Obstacles
• Network latency (for distributed in-memory data sets)
• Marshalling and unmarshalling of data structures
• Garbage collection pauses (Java)
Time
Latency
NOT PREDICTABLE!
14. © 2013 Terracotta Inc. | Internal Use Only 14
SCALE
Minimal server footprint with large data sets2
Obstacles
• Limits on in-memory storage per node
• Data replication overhead
• Other management overhead
GREAT NOT SO GREAT
1 TB IN-MEMORY DATA
15. © 2013 Terracotta Inc. | Internal Use Only 15
RELIABILITY
Fault tolerance and high availability3
Obstacles
• RAM is volatile
• Replicating data across nodes can become complex and
expensive
• Failover must be immediate and seamless
DISTRIBUTED IN-MEMORY DATA
X
16. © 2013 Terracotta Inc. | Internal Use Only 16
RESILIENCY
Fast Restartability4
Obstacles
• Large data sets can require very long reload times
• Traditional databases are not well suited as persistent
storage for in-memory data (slow reloads)
X
17. © 2013 Terracotta Inc. | Internal Use Only 17
CONSISTENCY
Synching data across distributed data sets5
Obstacles
• Network latency
• Consistency flexibility (eventual, strong, transactional)
• WAN replication (across regional data centers)
X=1 X=2 X=?
DISTRIBUTED IN-MEMORY DATA
18. © 2013 Terracotta Inc. | Internal Use Only 18
CONTROL
Monitoring and Management6
Obstacles
• Few standardized tools
• Many in-memory data management tools ship without
management and monitoring dashboards
19. © 2013 Terracotta Inc. | Internal Use Only 19
19© 2013 Terracotta Inc. | Internal Use Only
Emerging best practices
20. © 2013 Terracotta Inc. | Internal Use Only 20
Emerging best practices around in-memory
data management challenges
Performance: Off-heap storage
Storing data off the Java heap lets you keep massive amounts of data in-process by
increasing predictability and decreasing latency.
Scale: Tiers, not grids
Classic P2P data grids require as many as 5x the number of servers due to
management overhead. (More if off-heap storage unavailable.)
Reliability: Mirrored stripes
With an active and mirror for each server stripe in your array, you can failover
automatically to increase availability and reliability.
3
2
1
21. © 2013 Terracotta Inc. | Internal Use Only 21
Emerging best practices around in-memory
data management challenges (cont.)
Resiliency: Fast restartable stores
The best in-memory architectures optimize persistent transaction storage for very fast
reload. Loading a terabyte should take minutes, not days.
Consistency: Configurable guarantees
Allow your data management team to set consistency guarantees for each data set:
eventual, strong, transactional.
Control: In-memory dashboards
Build or buy a dashboard for advanced in-memory views and controls showing latency,
utilization, and capacity over time.
6
5
4
22. © 2013 Terracotta Inc. | Internal Use Only 22
22© 2013 Terracotta Inc. | Internal Use Only
Case study: AdJuggler
23. © 2013 Terracotta Inc. | Internal Use Only 23© 2013 Terracotta Inc. 23
- AdJuggler VP of Technology Ben Lindquist
“At AdJuggler, we’re building a 1 million
transaction-per-second advertising
marketplace. Speed at scale is everything,
and we are past the point where we can do
things in traditional ways.”
24. © 2013 Terracotta Inc. | Internal Use Only 24
AdJuggler in-memory architecture
I wanted to throw out
the database and, with
it, the disks.
— AdJuggler VP of
Technology Ben
Lindquist
25. © 2013 Terracotta Inc. | Internal Use Only 25
25© 2013 Terracotta Inc. | Internal Use Only
How to start
ditching your disks
26. © 2013 Terracotta Inc. | Internal Use Only 26© 2013 Terracotta Inc. | Internal Use Only 26
How to start ditching your disks?
• Start with a low-risk, high-return opportunity
with potential for a quick win
• Get early buy-in from senior executives
• Define and track success metrics so you can
expand your “ditch the disk” project
27. © 2013 Terracotta Inc. | Internal Use Only 27
Ditch the Disk: Q & A
Questions
Please type yours in the “Questions” panel or in the chat window.
28. © 2013 Terracotta Inc. | Internal Use Only 28
Want to learn more about designing in-memory
architectures?
1. Download “Ditch the Disk” white paper
Visit: www.terracotta.org (Resources > White Papers)
2. Contact Gagan to discuss your in-memory
architecture challenges
Email: gagan@terracotta.org
3. Follow us on Twitter
Follow: @big_memory
Notes de l'éditeur In-Memory Technologies can help e-commerce companies keep pace.Over last several decades there has been a huge drop in memory prices and massive increase in the size of commodity serversIt’s time to ditch the disk… to stop locking data away in slow, disk-bound databases which are expensive and difficult to runInstead, you can store data in memory, right where the application runs for ultra-fast access (at least 100x faster) A few years back, building an in-memory solution was too expensive. But now the good news is that the explosion in data – combined with a steep drop in RAM prices - is creating some exciting new opportunities to rethink the way we look at data management. Brief writeup on AdJuggler.