2. Lead DBA @ Data Services / ObjectRocket by Rackspace
15+ years in data and information systems, ranging from application develop,
data architecture, system design, and more.
Primary focus – Helping business focus on using data not managing
and storing it.
David Murphy
@davidmurphy_data
www.linkedin.com/in/davidbmurphy/
3. True genius resides in the capacity for
evaluation of uncertain, hazardous,
and conflicting information.
- Winston Churchill
EVERYONE’S GOT TO HAVE A
GREAT DATA QUOTE RIGHT?!
5. This
is
NOT…
a deep dive on any technology
a comprehensive list
a roadmap discussion
the end of the journey
What We Will Cover
6. What We’ll Cover
Concepts
What are the lifecycle stages
How to classify your workloads
Terminology
Actions What technologies are there
When to use them
Fitting them together
Why is this better
7. What are the lifecycle stages
Transient
• Sessions
• Logins
• Shop Cart
Short - Medium
• Feeds
• E-Commerce
• Video Game Stats
Analytics
• Reports
• Summary Data
• Dash boards
Archival
• Cold Storage
• Seldom Access
• Governances
L i f e C y c l e
8. What are the lifecycle stages
Transient
• Sessions
• Logins
• Shop Cart
Short - Medium
• Feeds
• E-Commerce
• Video Game Stats
Analytics
• Reports
• Summary Data
• Dash boards
Archival
• Cold Storage
• Seldom Access
• Governances
L i f e C y c l e
9. What are the lifecycle stages
Transient
• Sessions
• Logins
• Shop Cart
Short - Medium
• Feeds
• E-Commerce
• Video Game Stats
Analytics
• Reports
• Summary Data
• Dash boards
Archival
• Cold Storage
• Seldom Access
• Governances
L i f e C y c l e
10. What are the lifecycle stages
Transient
• Sessions
• Logins
• Shop Cart
Short - Medium
• Feeds
• E-Commerce
• Video Game Stats
Analytics
• Reports
• Summary Data
• Dash boards
Archival
• Cold Storage
• Seldom Access
• Governances
L i f e C y c l e
11. Updated frequently
Ultra fast retrieval
If missing is OK
IS IS NOT
Workloads - Transient
Rich Query-able
Durable
Point of truth
12. Some to many updates
Rich Query-able
Durable + Point of Truth
IS IS NOT
Workloads - Short to Medium
Built for short term
99% Write 1% Reads
Heavy Aggregations
23. Transient
• Memcache
• CouchBase
• Redis
• SQLite
Medium
• MySQL
• Maria
• PostgreSQL
• Mongo DB
• XtraCluster
• NDB
Analytics
• Hadoop
• InfoBright
• Cassandra
• Teradata
Archival
•Hadoop +
External
•Hadoop
Snapshots
•Cassandra
using S3
Technologies
24. Fitting it together
• What is the fewest technologies we can use
• What will for new requests
• Do I have plans to handle each stage of data?
• If not can the technologies do a decent job on the
odd case?
• Have talent now? Can I get a service or person easily?
25. Fitting it together - tools
Build a matrix with
• Features needs ( Transactions, Persistent , Geo,…)
• Importance ( 1- 5)
• Current or Attainable Talent ( 1 -5 )
• Does its Licensing work for this project ( 0 or 1)
(Features * Importance * Talent * License) = Combined Rank
29. How to scale – focus on what you know
You scale your app by letting someone else
• Build the hardware
• Know the Ops side for the technology
• Make the technologies pass data as its ages vs duplicating
the data
• Be the experts
• You just focus on the features of your app and make $$$
30. Questions?
WE ARE HIRING! ( DBA, DevOps, and more)
https://rackertalent.com
https://www.objectrocket.com/careers
Twitter: @dmurphy_data @rackspace @objectrocket
Email: david@objectrocket.com
Github: https://github.com/dbmurphy
SlideDeck: https://github.com/dbmurphy/presentations
Notes de l'éditeur
Atomic: Everything in a transaction succeeds or the entire transaction is rolled back.
Consistent: A transaction cannot leave the database in an inconsistent state.
Isolated: Transactions cannot interfere with each other.
Durable: Completed transactions persist, even when servers restart etc.
Atomic: Everything in a transaction succeeds or the entire transaction is rolled back.
Consistent: A transaction cannot leave the database in an inconsistent state.
Isolated: Transactions cannot interfere with each other.
Durable: Completed transactions persist, even when servers restart etc.
Atomic: Everything in a transaction succeeds or the entire transaction is rolled back.
Consistent: A transaction cannot leave the database in an inconsistent state.
Isolated: Transactions cannot interfere with each other.
Durable: Completed transactions persist, even when servers restart etc.
Atomic: Everything in a transaction succeeds or the entire transaction is rolled back.
Consistent: A transaction cannot leave the database in an inconsistent state.
Isolated: Transactions cannot interfere with each other.
Durable: Completed transactions persist, even when servers restart etc.