LSM Trees
•Télécharger en tant que PPTX, PDF•
19 j'aime•6,867 vues
Lightning talk on LSM trees. A high level overview given at Cassandra meetup
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (20)
Dernier
Dernier (20)
LSM Trees
- 2. www.pythian.com twitter: cnlohfink | e-mail: clohfink@blackbirdit.com PAGE© Pythian Cassandra ● Senior Engineer at Pythian (recently acquired Blackbird) o Lead of Cassandra practice ● Experience o Worked with Cassandra since 0.8 (2011) o Java/Python developer About Me Chris Lohfink 1
- 3. www.pythian.com twitter: cnlohfink | e-mail: clohfink@blackbirdit.com PAGE© Pythian Cassandra • Fault tolerance • Multiple DCs • Large data sets • Data model gotchas • Unstable releases • Bootstrapping • Repairs • Unit tests - My Opinions - 2 Cassandra
- 4. www.pythian.com twitter: cnlohfink | e-mail: clohfink@blackbirdit.com PAGE© Pythian Cassandra ● Can be good, bad, or ugly ● Same as with everything else Where is Performance? 3
- 5. www.pythian.com twitter: cnlohfink | e-mail: clohfink@blackbirdit.com PAGE© Pythian Cassandra Lies, Dirty Lies, & Benchmarks 4
- 6. www.pythian.com twitter: cnlohfink | e-mail: clohfink@blackbirdit.com PAGE© Pythian Cassandra 5
- 7. www.pythian.com twitter: cnlohfink | e-mail: clohfink@blackbirdit.com PAGE© Pythian Cassandra 6
- 8. www.pythian.com twitter: cnlohfink | e-mail: clohfink@blackbirdit.com PAGE© Pythian Cassandra ● An LSM-tree is composed of two or more tree-like components, each optimized for their type of storage o A small in-memory tree o One or more on disk trees ● Used in Cassandra, HBase, LevelDB, Google Big Table, SQLite4 & more Log Structured Merge Tree LSM-Tree 7
- 9. www.pythian.com twitter: cnlohfink | e-mail: clohfink@blackbirdit.com PAGE© Pythian Cassandra Log Structured Merge Tree One tree in memory, one or more on disk Memory Disk ReadsWrites 8
- 10. www.pythian.com twitter: cnlohfink | e-mail: clohfink@blackbirdit.com PAGE© Pythian Cassandra Log Structured Merge Tree Match Cassandra nomenclature Memtable SSTable ReadsWrites 9
- 11. www.pythian.com twitter: cnlohfink | e-mail: clohfink@blackbirdit.com PAGE© Pythian Cassandra • Very Fast Writes – Cost of IO amortized across large batches – Append only • Comparable Reads – Comparable best-case – Potential for very slow Log Structured Merge Tree 10 Characteristics
- 12. www.pythian.com twitter: cnlohfink | e-mail: clohfink@blackbirdit.com Pythian PAGE© Pythian© Pythian Cassandra Write Path Represent numbers of shovels in stock Memtable SSTables Temp -20 Shovels 0 State MN 11
- 13. www.pythian.com twitter: cnlohfink | e-mail: clohfink@blackbirdit.com Pythian PAGE© Pythian© Pythian Cassandra Write Path Decide we need to include humidity Memtable SSTables Temp -20 Shovels 0 State MN Humidity 10 12
- 14. www.pythian.com twitter: cnlohfink | e-mail: clohfink@blackbirdit.com Pythian PAGE© Pythian© Pythian Cassandra Write Path Its Minnesota so the Temperature changed Memtable SSTables Temp -20 Shovels 0 State MN Humidity 10 Temp 80 13
- 15. www.pythian.com twitter: cnlohfink | e-mail: clohfink@blackbirdit.com Pythian PAGE© Pythian© Pythian Cassandra Write Path Shovel demand drops so its time to put some in stock again Memtable SSTables Temp -20 Shovels 0 State MN Humidity 10 Temp 80 Shovels 25 14
- 16. www.pythian.com twitter: cnlohfink | e-mail: clohfink@blackbirdit.com Pythian PAGE© Pythian© Pythian Cassandra Write Path Reached memtable limit, flush to disk Memtable SSTables Temp -20 Shovels 0 State MN Humidity 10 Temp 80 Shovels 25 Humidity 10 Temp 80 Shovels 25 15
- 17. www.pythian.com twitter: cnlohfink | e-mail: clohfink@blackbirdit.com Pythian PAGE© Pythian© Pythian Cassandra Write Path We still in Minnesota and its been a few days so adjust Temp Memtable SSTables Temp -20 Shovels 0 State MN Humidity 10 Temp 80 Shovels 25 Temp 10 16
- 18. www.pythian.com twitter: cnlohfink | e-mail: clohfink@blackbirdit.com Pythian PAGE© Pythian© Pythian Cassandra Read Path Check the current temp, take most recent from all SSTables and the Memtable Memtable SSTables Temp -20 Shovels 0 State MN Humidity 10 Temp 80 Shovels 25 Temp 10 17
- 19. www.pythian.com twitter: cnlohfink | e-mail: clohfink@blackbirdit.com Pythian PAGE© Pythian© Pythian Cassandra Compaction Create new SSTable thats the merged result Memtable SSTables Temp -20 Shovels 0 State MN Humidity 10 Temp 80 Shovels 25 Temp 10 Humidity 10 Shovels 25 Temp 80 State MN 18
- 20. www.pythian.com twitter: cnlohfink | e-mail: clohfink@blackbirdit.com Pythian PAGE© Pythian© Pythian Cassandra Compaction Delete old SSTables Memtable SSTables Temp 10 Humidity 10 Shovels 25 Temp 80 State MN 19
- 21. www.pythian.com twitter: cnlohfink | e-mail: clohfink@blackbirdit.com Pythian PAGE© Pythian© Pythian Cassandra Questions? 20
Notes de l'éditeur
- Attention! Before you open this template be sure what you have the following fonts installed: Novecento Sans wide font family (6 free styles) http://typography.synthview.com Abattis Cantarell http://www.fontsquirrel.com/fonts/cantarell Icon Sets Fonts: raphaelicons-webfont.ttf from this page: http://icons.marekventur.de iconic_stroke.ttf from this page: http://somerandomdude.com/work/open-iconic modernpics.otf from this page: http://www.fontsquirrel.com/fonts/modern-pictograms general_foundicons.ttf, social_foundicons.ttf from this page: http://www.zurb.com/playground/foundation-icons accessibility_foundicons_new.ttf fixed for PowerPoint Mac from here: http://docs.google.com/open?id=0Byf3seS60YFLR3VJN3dMbGp0UDA fontawesome-webfont.ttf from this page: http://fortawesome.github.io/Font-Awesome Entypo.otf from this page: http://www.fontsquirrel.com/fonts/entypo sosa-regular-webfont.ttf from this page: http://tenbytwenty.com/?xxxx_posts=sosa All fonts are permitted free use in commercial projects. If you have difficulties to install those fonts or have no time to find all of them, please follow the FAQs: http://graphicriver.net/item/six-template/3626243/support