SlideShare a Scribd company logo
1 of 25
Real world capacity planning: Cassandra on blades and big iron   July 2011
About me ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Media6 Degrees  ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
What is this talk about?  ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Using puppet, chef... from day 1 ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Calculating Data size on disk ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Speed of disk ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Disk formatting ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Memory ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
CPU ,[object Object],[object Object],[object Object]
Different workloads ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Large Disk / Big Iron key points ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Blade server key points ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Schema lessons ,[object Object],[object Object],[object Object],[object Object]
Capacity Planning rule #1 Know your hard drive limits
Capacity Planning rule #2 Writes are fast, until c* flushes and  compacts so much, that they are not
Capacity Planning rule #3 ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Capacity Planning rule #4 ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Capacity Planning rule #5 ,[object Object]
The use case: Dr. Real Time and Mr. Batch
Dr. Real Time ,[object Object],[object Object],[object Object],[object Object]
Dr. Real Time's Lab ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Mr. Batch ,[object Object],[object Object],[object Object],[object Object]
Mr. Batch rampaging  through the data ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Questions ,[object Object]

More Related Content

What's hot

Hardware Provisioning for MongoDB
Hardware Provisioning for MongoDBHardware Provisioning for MongoDB
Hardware Provisioning for MongoDBMongoDB
 
Edge performance with in memory nosql
Edge performance with in memory nosqlEdge performance with in memory nosql
Edge performance with in memory nosqlLiviu Costea
 
Cassandra in Operation
Cassandra in OperationCassandra in Operation
Cassandra in Operationniallmilton
 
MongoDB Deployment Checklist
MongoDB Deployment ChecklistMongoDB Deployment Checklist
MongoDB Deployment ChecklistMongoDB
 
Top 10 database optimization tips
Top 10 database optimization tipsTop 10 database optimization tips
Top 10 database optimization tipsraviwriter
 
RedisConf17- Redis as a Primary Data Store
RedisConf17- Redis as a Primary Data StoreRedisConf17- Redis as a Primary Data Store
RedisConf17- Redis as a Primary Data StoreRedis Labs
 
Ceph Day Santa Clara: Ceph Performance & Benchmarking
Ceph Day Santa Clara: Ceph Performance & Benchmarking Ceph Day Santa Clara: Ceph Performance & Benchmarking
Ceph Day Santa Clara: Ceph Performance & Benchmarking Ceph Community
 
Speed up R with parallel programming in the Cloud
Speed up R with parallel programming in the CloudSpeed up R with parallel programming in the Cloud
Speed up R with parallel programming in the CloudRevolution Analytics
 
Lessons Learned Migrating 2+ Billion Documents at Craigslist
Lessons Learned Migrating 2+ Billion Documents at CraigslistLessons Learned Migrating 2+ Billion Documents at Craigslist
Lessons Learned Migrating 2+ Billion Documents at CraigslistJeremy Zawodny
 
Ceph Days 2014 Paul Evans Slide Deck
Ceph Days 2014 Paul Evans Slide DeckCeph Days 2014 Paul Evans Slide Deck
Ceph Days 2014 Paul Evans Slide DeckDaystromTech
 
DHT2 - O Brother, Where Art Thou with Shyam Ranganathan
DHT2 - O Brother, Where Art Thou with 	Shyam RanganathanDHT2 - O Brother, Where Art Thou with 	Shyam Ranganathan
DHT2 - O Brother, Where Art Thou with Shyam RanganathanGluster.org
 
Hardware Provisioning
Hardware ProvisioningHardware Provisioning
Hardware ProvisioningMongoDB
 
Redis Modules - Redis India Tour - 2017
Redis Modules - Redis India Tour - 2017Redis Modules - Redis India Tour - 2017
Redis Modules - Redis India Tour - 2017HashedIn Technologies
 
Capacity Planning For Your Growing MongoDB Cluster
Capacity Planning For Your Growing MongoDB ClusterCapacity Planning For Your Growing MongoDB Cluster
Capacity Planning For Your Growing MongoDB ClusterMongoDB
 

What's hot (19)

Hardware Provisioning for MongoDB
Hardware Provisioning for MongoDBHardware Provisioning for MongoDB
Hardware Provisioning for MongoDB
 
Hdfs internals
Hdfs internalsHdfs internals
Hdfs internals
 
Edge performance with in memory nosql
Edge performance with in memory nosqlEdge performance with in memory nosql
Edge performance with in memory nosql
 
Cassandra in Operation
Cassandra in OperationCassandra in Operation
Cassandra in Operation
 
Hive at booking
Hive at bookingHive at booking
Hive at booking
 
MongoDB Deployment Checklist
MongoDB Deployment ChecklistMongoDB Deployment Checklist
MongoDB Deployment Checklist
 
Redis database
Redis databaseRedis database
Redis database
 
Top 10 database optimization tips
Top 10 database optimization tipsTop 10 database optimization tips
Top 10 database optimization tips
 
RedisConf17- Redis as a Primary Data Store
RedisConf17- Redis as a Primary Data StoreRedisConf17- Redis as a Primary Data Store
RedisConf17- Redis as a Primary Data Store
 
Ceph Day Santa Clara: Ceph Performance & Benchmarking
Ceph Day Santa Clara: Ceph Performance & Benchmarking Ceph Day Santa Clara: Ceph Performance & Benchmarking
Ceph Day Santa Clara: Ceph Performance & Benchmarking
 
Speed up R with parallel programming in the Cloud
Speed up R with parallel programming in the CloudSpeed up R with parallel programming in the Cloud
Speed up R with parallel programming in the Cloud
 
Lessons Learned Migrating 2+ Billion Documents at Craigslist
Lessons Learned Migrating 2+ Billion Documents at CraigslistLessons Learned Migrating 2+ Billion Documents at Craigslist
Lessons Learned Migrating 2+ Billion Documents at Craigslist
 
Ceph Days 2014 Paul Evans Slide Deck
Ceph Days 2014 Paul Evans Slide DeckCeph Days 2014 Paul Evans Slide Deck
Ceph Days 2014 Paul Evans Slide Deck
 
DHT2 - O Brother, Where Art Thou with Shyam Ranganathan
DHT2 - O Brother, Where Art Thou with 	Shyam RanganathanDHT2 - O Brother, Where Art Thou with 	Shyam Ranganathan
DHT2 - O Brother, Where Art Thou with Shyam Ranganathan
 
Hardware Provisioning
Hardware ProvisioningHardware Provisioning
Hardware Provisioning
 
4 use cases for C* to Scylla
4 use cases for C*  to Scylla4 use cases for C*  to Scylla
4 use cases for C* to Scylla
 
Redis Modules - Redis India Tour - 2017
Redis Modules - Redis India Tour - 2017Redis Modules - Redis India Tour - 2017
Redis Modules - Redis India Tour - 2017
 
Capacity Planning For Your Growing MongoDB Cluster
Capacity Planning For Your Growing MongoDB ClusterCapacity Planning For Your Growing MongoDB Cluster
Capacity Planning For Your Growing MongoDB Cluster
 
HDFS Deep Dive
HDFS Deep DiveHDFS Deep Dive
HDFS Deep Dive
 

Viewers also liked

Viewers also liked (20)

Adco teaser
Adco teaserAdco teaser
Adco teaser
 
My life
My lifeMy life
My life
 
Unit 7 y 8
Unit 7 y 8Unit 7 y 8
Unit 7 y 8
 
Successes2009
Successes2009Successes2009
Successes2009
 
Lecture2
Lecture2Lecture2
Lecture2
 
Outland res. brochure 6 30-11 brown
Outland res. brochure 6 30-11 brownOutland res. brochure 6 30-11 brown
Outland res. brochure 6 30-11 brown
 
Desktop support qua
Desktop support quaDesktop support qua
Desktop support qua
 
베트남 노동법 주요내용
베트남 노동법 주요내용베트남 노동법 주요내용
베트남 노동법 주요내용
 
SBPS Staff Survey
SBPS Staff SurveySBPS Staff Survey
SBPS Staff Survey
 
Jennifer h. jenny and timmy
Jennifer h. jenny and timmyJennifer h. jenny and timmy
Jennifer h. jenny and timmy
 
Themes ways of the world
Themes ways of the worldThemes ways of the world
Themes ways of the world
 
Avanta Brochure Presentation 2011
Avanta Brochure Presentation 2011Avanta Brochure Presentation 2011
Avanta Brochure Presentation 2011
 
Linkedin
LinkedinLinkedin
Linkedin
 
1 ea5ea59 39b4-4e4c-a0cd183077e7b0aa
1 ea5ea59 39b4-4e4c-a0cd183077e7b0aa1 ea5ea59 39b4-4e4c-a0cd183077e7b0aa
1 ea5ea59 39b4-4e4c-a0cd183077e7b0aa
 
Module 1
Module 1Module 1
Module 1
 
CV VI
CV VICV VI
CV VI
 
Da rtn 11_jan2013
Da rtn 11_jan2013Da rtn 11_jan2013
Da rtn 11_jan2013
 
Andrés bio
Andrés   bioAndrés   bio
Andrés bio
 
Cd y ci
Cd y ciCd y ci
Cd y ci
 
แนะนำทุน พสวท.
แนะนำทุน พสวท.แนะนำทุน พสวท.
แนะนำทุน พสวท.
 

Similar to Real world capacity

Web20expo Filesystems
Web20expo FilesystemsWeb20expo Filesystems
Web20expo Filesystemsroyans
 
Beyond the File System: Designing Large-Scale File Storage and Serving
 	Beyond the File System: Designing Large-Scale File Storage and Serving 	Beyond the File System: Designing Large-Scale File Storage and Serving
Beyond the File System: Designing Large-Scale File Storage and Servingmclee
 
Web20expo Filesystems
Web20expo FilesystemsWeb20expo Filesystems
Web20expo Filesystemsguest18a0f1
 
Web20expo Filesystems
Web20expo FilesystemsWeb20expo Filesystems
Web20expo Filesystemsroyans
 
Web20expo Filesystems
Web20expo FilesystemsWeb20expo Filesystems
Web20expo Filesystemsroyans
 
Beyond the File System - Designing Large Scale File Storage and Serving
Beyond the File System - Designing Large Scale File Storage and ServingBeyond the File System - Designing Large Scale File Storage and Serving
Beyond the File System - Designing Large Scale File Storage and Servingmclee
 
Filesystems
FilesystemsFilesystems
Filesystemsroyans
 
CS 542 Putting it all together -- Storage Management
CS 542 Putting it all together -- Storage ManagementCS 542 Putting it all together -- Storage Management
CS 542 Putting it all together -- Storage ManagementJ Singh
 
MongoDB Sharding
MongoDB ShardingMongoDB Sharding
MongoDB Shardinguzzal basak
 
Pilot Hadoop Towards 2500 Nodes and Cluster Redundancy
Pilot Hadoop Towards 2500 Nodes and Cluster RedundancyPilot Hadoop Towards 2500 Nodes and Cluster Redundancy
Pilot Hadoop Towards 2500 Nodes and Cluster RedundancyStuart Pook
 
Scalable Web Architectures: Common Patterns and Approaches - Web 2.0 Expo NYC
Scalable Web Architectures: Common Patterns and Approaches - Web 2.0 Expo NYCScalable Web Architectures: Common Patterns and Approaches - Web 2.0 Expo NYC
Scalable Web Architectures: Common Patterns and Approaches - Web 2.0 Expo NYCCal Henderson
 
Cloud computing UNIT 2.1 presentation in
Cloud computing UNIT 2.1 presentation inCloud computing UNIT 2.1 presentation in
Cloud computing UNIT 2.1 presentation inRahulBhole12
 
Building modern data lakes
Building modern data lakes Building modern data lakes
Building modern data lakes Minio
 

Similar to Real world capacity (20)

Cassandra as Memcache
Cassandra as MemcacheCassandra as Memcache
Cassandra as Memcache
 
Cassandra admin
Cassandra adminCassandra admin
Cassandra admin
 
Web20expo Filesystems
Web20expo FilesystemsWeb20expo Filesystems
Web20expo Filesystems
 
Beyond the File System: Designing Large-Scale File Storage and Serving
 	Beyond the File System: Designing Large-Scale File Storage and Serving 	Beyond the File System: Designing Large-Scale File Storage and Serving
Beyond the File System: Designing Large-Scale File Storage and Serving
 
Web20expo Filesystems
Web20expo FilesystemsWeb20expo Filesystems
Web20expo Filesystems
 
Web20expo Filesystems
Web20expo FilesystemsWeb20expo Filesystems
Web20expo Filesystems
 
Web20expo Filesystems
Web20expo FilesystemsWeb20expo Filesystems
Web20expo Filesystems
 
Beyond the File System - Designing Large Scale File Storage and Serving
Beyond the File System - Designing Large Scale File Storage and ServingBeyond the File System - Designing Large Scale File Storage and Serving
Beyond the File System - Designing Large Scale File Storage and Serving
 
Filesystems
FilesystemsFilesystems
Filesystems
 
Ce202 Storage
Ce202 StorageCe202 Storage
Ce202 Storage
 
CS 542 Putting it all together -- Storage Management
CS 542 Putting it all together -- Storage ManagementCS 542 Putting it all together -- Storage Management
CS 542 Putting it all together -- Storage Management
 
Nachos 2
Nachos 2Nachos 2
Nachos 2
 
Nachos 2
Nachos 2Nachos 2
Nachos 2
 
MongoDB Sharding
MongoDB ShardingMongoDB Sharding
MongoDB Sharding
 
Pilot Hadoop Towards 2500 Nodes and Cluster Redundancy
Pilot Hadoop Towards 2500 Nodes and Cluster RedundancyPilot Hadoop Towards 2500 Nodes and Cluster Redundancy
Pilot Hadoop Towards 2500 Nodes and Cluster Redundancy
 
Scalable Web Architectures: Common Patterns and Approaches - Web 2.0 Expo NYC
Scalable Web Architectures: Common Patterns and Approaches - Web 2.0 Expo NYCScalable Web Architectures: Common Patterns and Approaches - Web 2.0 Expo NYC
Scalable Web Architectures: Common Patterns and Approaches - Web 2.0 Expo NYC
 
Cloud computing UNIT 2.1 presentation in
Cloud computing UNIT 2.1 presentation inCloud computing UNIT 2.1 presentation in
Cloud computing UNIT 2.1 presentation in
 
Building modern data lakes
Building modern data lakes Building modern data lakes
Building modern data lakes
 
Linux Huge Pages
Linux Huge PagesLinux Huge Pages
Linux Huge Pages
 
Hadoop Research
Hadoop Research Hadoop Research
Hadoop Research
 

More from Edward Capriolo

Nibiru: Building your own NoSQL store
Nibiru: Building your own NoSQL storeNibiru: Building your own NoSQL store
Nibiru: Building your own NoSQL storeEdward Capriolo
 
Web-scale data processing: practical approaches for low-latency and batch
Web-scale data processing: practical approaches for low-latency and batchWeb-scale data processing: practical approaches for low-latency and batch
Web-scale data processing: practical approaches for low-latency and batchEdward Capriolo
 
Intravert Server side processing for Cassandra
Intravert Server side processing for CassandraIntravert Server side processing for Cassandra
Intravert Server side processing for CassandraEdward Capriolo
 
Cassandra NoSQL Lan party
Cassandra NoSQL Lan partyCassandra NoSQL Lan party
Cassandra NoSQL Lan partyEdward Capriolo
 
M6d cassandrapresentation
M6d cassandrapresentationM6d cassandrapresentation
M6d cassandrapresentationEdward Capriolo
 
Breaking first-normal form with Hive
Breaking first-normal form with HiveBreaking first-normal form with Hive
Breaking first-normal form with HiveEdward Capriolo
 
Hadoop Monitoring best Practices
Hadoop Monitoring best PracticesHadoop Monitoring best Practices
Hadoop Monitoring best PracticesEdward Capriolo
 
Whirlwind tour of Hadoop and HIve
Whirlwind tour of Hadoop and HIveWhirlwind tour of Hadoop and HIve
Whirlwind tour of Hadoop and HIveEdward Capriolo
 
Counters for real-time statistics
Counters for real-time statisticsCounters for real-time statistics
Counters for real-time statisticsEdward Capriolo
 

More from Edward Capriolo (15)

Nibiru: Building your own NoSQL store
Nibiru: Building your own NoSQL storeNibiru: Building your own NoSQL store
Nibiru: Building your own NoSQL store
 
Web-scale data processing: practical approaches for low-latency and batch
Web-scale data processing: practical approaches for low-latency and batchWeb-scale data processing: practical approaches for low-latency and batch
Web-scale data processing: practical approaches for low-latency and batch
 
Big data nyu
Big data nyuBig data nyu
Big data nyu
 
Cassandra4hadoop
Cassandra4hadoopCassandra4hadoop
Cassandra4hadoop
 
Intravert Server side processing for Cassandra
Intravert Server side processing for CassandraIntravert Server side processing for Cassandra
Intravert Server side processing for Cassandra
 
M6d cassandra summit
M6d cassandra summitM6d cassandra summit
M6d cassandra summit
 
Apache Kafka Demo
Apache Kafka DemoApache Kafka Demo
Apache Kafka Demo
 
Cassandra NoSQL Lan party
Cassandra NoSQL Lan partyCassandra NoSQL Lan party
Cassandra NoSQL Lan party
 
M6d cassandrapresentation
M6d cassandrapresentationM6d cassandrapresentation
M6d cassandrapresentation
 
Breaking first-normal form with Hive
Breaking first-normal form with HiveBreaking first-normal form with Hive
Breaking first-normal form with Hive
 
Casbase presentation
Casbase presentationCasbase presentation
Casbase presentation
 
Hadoop Monitoring best Practices
Hadoop Monitoring best PracticesHadoop Monitoring best Practices
Hadoop Monitoring best Practices
 
Whirlwind tour of Hadoop and HIve
Whirlwind tour of Hadoop and HIveWhirlwind tour of Hadoop and HIve
Whirlwind tour of Hadoop and HIve
 
Cli deep dive
Cli deep diveCli deep dive
Cli deep dive
 
Counters for real-time statistics
Counters for real-time statisticsCounters for real-time statistics
Counters for real-time statistics
 

Real world capacity

Editor's Notes

  1. 10/01/10