SlideShare a Scribd company logo
1 of 22
Download to read offline
LEARNING TO BUILD
DISTRIBUTED SYSTEMS
THE HARD WAY
@iconara
NEW
and improved!
LEARNING TO BUILD
DISTRIBUTED SYSTEMS
THE HARD WAY
@iconara
NEW
and improved!
speakerdeck.com/iconara
Theo / @iconara
chief architect at BURT
FAILURE
embrace it
SCALE
how hard can it be? let’s worry about that later.
KNOW YOUR LIMITS
who’s the largest customer you could sign?
what would happen if you did?
BALANCE
“customer” is a really bad shard key,
find something that distributes evenly & uniformly
SCALE OUT, NOT UP
bigger boxes aren’t going to save you
<
START WITH TWO
OF EVERYTHING
going from one to two is the hardest,
force yourself to solve the scaling problem up front
START WITH TWO
OF EVERYTHING
you’ll solve the scaling problem,
and need less overcapacity
THREE
LIMITS
we’ll probably never run out of memory
BACK PRESSURE
what happens when the system is working at full
capacity? what happens next?
PRODUCTION = QA
production is where the weird shit happens,
can you test production traffic without
deploying to production?
=
MONOLITHS
running all the things on the same box is really fast.
what could ever go wrong?
1:4:9
DECOUPLE
UNTIL IT BREAKS
moving things to separate services means that you
will be able to scale them independently
PROCESSING
& STORAGE
separate processing from storage,
they almost never scale together.
SCALE
exponential scaling is also scaling,
but you want it as cheaply as possible
SCALE
your CFO may not agree that O(2n) = O(n)
GÖTEBORG,
DISTRIBUTED
@gbgdistr
meetup.com/gbgdistributed
KTHXBAI
@iconara
github.com/iconara
architecturalatrocities.com
burtcorp.com

More Related Content

Viewers also liked

Red cross 72 hour survival kit
Red cross 72 hour survival kitRed cross 72 hour survival kit
Red cross 72 hour survival kitheu_av
 
พนักงานราชการ กศน
พนักงานราชการ กศนพนักงานราชการ กศน
พนักงานราชการ กศนPaweena Kumpor
 
95944832006
9594483200695944832006
95944832006nahilis
 
It’s for your health nutrition
It’s for your health nutritionIt’s for your health nutrition
It’s for your health nutritionJodi Dingman
 
Understanding Brand
Understanding BrandUnderstanding Brand
Understanding Brandmanumaxnanda
 
Schedule 08 feb11
Schedule 08 feb11Schedule 08 feb11
Schedule 08 feb11heu_av
 
Building a CQL driver
Building a CQL driverBuilding a CQL driver
Building a CQL driverTheo Hultberg
 
Cloud expo John Darlington - Imperial College
Cloud expo   John Darlington - Imperial CollegeCloud expo   John Darlington - Imperial College
Cloud expo John Darlington - Imperial CollegeCloudExpoEurope
 
Zimolvediba uznemuma attistibai
Zimolvediba uznemuma attistibaiZimolvediba uznemuma attistibai
Zimolvediba uznemuma attistibaiMartins Baranovs
 

Viewers also liked (12)

Mundo In tacto
Mundo In tactoMundo In tacto
Mundo In tacto
 
Red cross 72 hour survival kit
Red cross 72 hour survival kitRed cross 72 hour survival kit
Red cross 72 hour survival kit
 
พนักงานราชการ กศน
พนักงานราชการ กศนพนักงานราชการ กศน
พนักงานราชการ กศน
 
95944832006
9594483200695944832006
95944832006
 
It’s for your health nutrition
It’s for your health nutritionIt’s for your health nutrition
It’s for your health nutrition
 
CCI Brochure
CCI BrochureCCI Brochure
CCI Brochure
 
Understanding Brand
Understanding BrandUnderstanding Brand
Understanding Brand
 
Schedule 08 feb11
Schedule 08 feb11Schedule 08 feb11
Schedule 08 feb11
 
Building a CQL driver
Building a CQL driverBuilding a CQL driver
Building a CQL driver
 
Cloud expo emer coleman
Cloud expo   emer colemanCloud expo   emer coleman
Cloud expo emer coleman
 
Cloud expo John Darlington - Imperial College
Cloud expo   John Darlington - Imperial CollegeCloud expo   John Darlington - Imperial College
Cloud expo John Darlington - Imperial College
 
Zimolvediba uznemuma attistibai
Zimolvediba uznemuma attistibaiZimolvediba uznemuma attistibai
Zimolvediba uznemuma attistibai
 

Similar to Learning to build distributed systems the hard way

Y Combinator Startup Class #12 : Building for the Enterprise
Y Combinator Startup Class #12 : Building for the EnterpriseY Combinator Startup Class #12 : Building for the Enterprise
Y Combinator Startup Class #12 : Building for the EnterpriseFabien Grenet
 
Mind the Gap - State of the Browser 2015
Mind the Gap - State of the Browser 2015Mind the Gap - State of the Browser 2015
Mind the Gap - State of the Browser 2015Christian Heilmann
 
Docker Inside/Out: The 'Real' Real- World World of Stacking Containers in pro...
Docker Inside/Out: The 'Real' Real- World World of Stacking Containers in pro...Docker Inside/Out: The 'Real' Real- World World of Stacking Containers in pro...
Docker Inside/Out: The 'Real' Real- World World of Stacking Containers in pro...Sonatype
 
A People's History of Microservices
A People's History of MicroservicesA People's History of Microservices
A People's History of MicroservicesCamille Fournier
 
Alex Bolboacă: Usable Software Design at I T.A.K.E. Unconference 2015
Alex Bolboacă: Usable Software Design at I T.A.K.E. Unconference 2015Alex Bolboacă: Usable Software Design at I T.A.K.E. Unconference 2015
Alex Bolboacă: Usable Software Design at I T.A.K.E. Unconference 2015Mozaic Works
 
Rapid growth | 10 Challenges and Opportunities
Rapid growth | 10 Challenges and OpportunitiesRapid growth | 10 Challenges and Opportunities
Rapid growth | 10 Challenges and OpportunitiesNick Caldwell
 
Usability Testing
Usability TestingUsability Testing
Usability TestingAndy Budd
 
Getting Business Exec Buy-in for Architecture Change
Getting Business Exec Buy-in for Architecture ChangeGetting Business Exec Buy-in for Architecture Change
Getting Business Exec Buy-in for Architecture ChangeDominica DeGrandis
 
The Most Important Thing: How Mozilla Does Security and What You Can Steal
The Most Important Thing: How Mozilla Does Security and What You Can StealThe Most Important Thing: How Mozilla Does Security and What You Can Steal
The Most Important Thing: How Mozilla Does Security and What You Can Stealmozilla.presentations
 
Be a SPOC but dont be a SPOF
Be a SPOC but dont be a SPOFBe a SPOC but dont be a SPOF
Be a SPOC but dont be a SPOFSethMcBean
 
Bilot 3mode
Bilot 3modeBilot 3mode
Bilot 3modeBilot
 
Improving Forecasts with Monte Carlo Simulations
Improving Forecasts with Monte Carlo Simulations  Improving Forecasts with Monte Carlo Simulations
Improving Forecasts with Monte Carlo Simulations Michael Wallace
 
Introduction to DevOps with Lego and Chocolate simulation game.
Introduction to DevOps with Lego and Chocolate simulation game.Introduction to DevOps with Lego and Chocolate simulation game.
Introduction to DevOps with Lego and Chocolate simulation game.Dana Pylayeva
 

Similar to Learning to build distributed systems the hard way (20)

The alignment
The alignmentThe alignment
The alignment
 
Y Combinator Startup Class #12 : Building for the Enterprise
Y Combinator Startup Class #12 : Building for the EnterpriseY Combinator Startup Class #12 : Building for the Enterprise
Y Combinator Startup Class #12 : Building for the Enterprise
 
You are Not Your Customer
You are Not Your CustomerYou are Not Your Customer
You are Not Your Customer
 
Windows 7
Windows 7Windows 7
Windows 7
 
Rewrites in Real Life
Rewrites in Real LifeRewrites in Real Life
Rewrites in Real Life
 
Mind the Gap - State of the Browser 2015
Mind the Gap - State of the Browser 2015Mind the Gap - State of the Browser 2015
Mind the Gap - State of the Browser 2015
 
Docker Inside/Out: The 'Real' Real- World World of Stacking Containers in pro...
Docker Inside/Out: The 'Real' Real- World World of Stacking Containers in pro...Docker Inside/Out: The 'Real' Real- World World of Stacking Containers in pro...
Docker Inside/Out: The 'Real' Real- World World of Stacking Containers in pro...
 
A People's History of Microservices
A People's History of MicroservicesA People's History of Microservices
A People's History of Microservices
 
Alex Bolboacă: Usable Software Design at I T.A.K.E. Unconference 2015
Alex Bolboacă: Usable Software Design at I T.A.K.E. Unconference 2015Alex Bolboacă: Usable Software Design at I T.A.K.E. Unconference 2015
Alex Bolboacă: Usable Software Design at I T.A.K.E. Unconference 2015
 
Rapid growth | 10 Challenges and Opportunities
Rapid growth | 10 Challenges and OpportunitiesRapid growth | 10 Challenges and Opportunities
Rapid growth | 10 Challenges and Opportunities
 
Usability Testing
Usability TestingUsability Testing
Usability Testing
 
Getting Business Exec Buy-in for Architecture Change
Getting Business Exec Buy-in for Architecture ChangeGetting Business Exec Buy-in for Architecture Change
Getting Business Exec Buy-in for Architecture Change
 
SEO and Accessibility
SEO and AccessibilitySEO and Accessibility
SEO and Accessibility
 
The Most Important Thing: How Mozilla Does Security and What You Can Steal
The Most Important Thing: How Mozilla Does Security and What You Can StealThe Most Important Thing: How Mozilla Does Security and What You Can Steal
The Most Important Thing: How Mozilla Does Security and What You Can Steal
 
New Rules Of Engagement
New Rules Of EngagementNew Rules Of Engagement
New Rules Of Engagement
 
Meetup ssc w_ms_slides
Meetup ssc w_ms_slidesMeetup ssc w_ms_slides
Meetup ssc w_ms_slides
 
Be a SPOC but dont be a SPOF
Be a SPOC but dont be a SPOFBe a SPOC but dont be a SPOF
Be a SPOC but dont be a SPOF
 
Bilot 3mode
Bilot 3modeBilot 3mode
Bilot 3mode
 
Improving Forecasts with Monte Carlo Simulations
Improving Forecasts with Monte Carlo Simulations  Improving Forecasts with Monte Carlo Simulations
Improving Forecasts with Monte Carlo Simulations
 
Introduction to DevOps with Lego and Chocolate simulation game.
Introduction to DevOps with Lego and Chocolate simulation game.Introduction to DevOps with Lego and Chocolate simulation game.
Introduction to DevOps with Lego and Chocolate simulation game.
 

More from Theo Hultberg

AWS Cost Optimization
AWS Cost OptimizationAWS Cost Optimization
AWS Cost OptimizationTheo Hultberg
 
Cassandra for all the Things
Cassandra for all the ThingsCassandra for all the Things
Cassandra for all the ThingsTheo Hultberg
 
Chasing the elephant
Chasing the elephantChasing the elephant
Chasing the elephantTheo Hultberg
 
Learning to build distributed systems the hard way
Learning to build distributed systems the hard wayLearning to build distributed systems the hard way
Learning to build distributed systems the hard wayTheo Hultberg
 
A Guide to the Post Relational Revolution
A Guide to the Post Relational RevolutionA Guide to the Post Relational Revolution
A Guide to the Post Relational RevolutionTheo Hultberg
 
Concurrency and Distributed Systems Using JRuby
Concurrency and Distributed Systems Using JRubyConcurrency and Distributed Systems Using JRuby
Concurrency and Distributed Systems Using JRubyTheo Hultberg
 
Shortcuts around the mistakes I've made scaling MongoDB
Shortcuts around the mistakes I've made scaling MongoDB Shortcuts around the mistakes I've made scaling MongoDB
Shortcuts around the mistakes I've made scaling MongoDB Theo Hultberg
 
Standing on the shoulders of giants with JRuby
Standing on the shoulders of giants with JRubyStanding on the shoulders of giants with JRuby
Standing on the shoulders of giants with JRubyTheo Hultberg
 

More from Theo Hultberg (8)

AWS Cost Optimization
AWS Cost OptimizationAWS Cost Optimization
AWS Cost Optimization
 
Cassandra for all the Things
Cassandra for all the ThingsCassandra for all the Things
Cassandra for all the Things
 
Chasing the elephant
Chasing the elephantChasing the elephant
Chasing the elephant
 
Learning to build distributed systems the hard way
Learning to build distributed systems the hard wayLearning to build distributed systems the hard way
Learning to build distributed systems the hard way
 
A Guide to the Post Relational Revolution
A Guide to the Post Relational RevolutionA Guide to the Post Relational Revolution
A Guide to the Post Relational Revolution
 
Concurrency and Distributed Systems Using JRuby
Concurrency and Distributed Systems Using JRubyConcurrency and Distributed Systems Using JRuby
Concurrency and Distributed Systems Using JRuby
 
Shortcuts around the mistakes I've made scaling MongoDB
Shortcuts around the mistakes I've made scaling MongoDB Shortcuts around the mistakes I've made scaling MongoDB
Shortcuts around the mistakes I've made scaling MongoDB
 
Standing on the shoulders of giants with JRuby
Standing on the shoulders of giants with JRubyStanding on the shoulders of giants with JRuby
Standing on the shoulders of giants with JRuby
 

Recently uploaded

IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)wesley chun
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...Neo4j
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProduct Anonymous
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Igalia
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEarley Information Science
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slidevu2urc
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘RTylerCroy
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUK Journal
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherRemote DBA Services
 
Evaluating the top large language models.pdf
Evaluating the top large language models.pdfEvaluating the top large language models.pdf
Evaluating the top large language models.pdfChristopherTHyatt
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfsudhanshuwaghmare1
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityPrincipled Technologies
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoffsammart93
 

Recently uploaded (20)

IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Evaluating the top large language models.pdf
Evaluating the top large language models.pdfEvaluating the top large language models.pdf
Evaluating the top large language models.pdf
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 

Learning to build distributed systems the hard way