SlideShare a Scribd company logo
1 of 38
@michaelneale
www.CloudBees.com
@michaelneale
github.com/michaelneale
currently: trying to make less reliable things more
reliable.
Some things we use to get it done:
- scala (back end apps)
- ruby (chef, rails, proxymachine, lots)
- java (jenkins and plugins)
- erlang (“agents” and lots more..)
Exceptions. Errors.
What The Hell Is That Process Doing?
and how it relates to..
On Call PTSD
Exception handling is wrong...
But it seems harmless...
Erlang says no no no...
Question: a process on server not responding
Answer:
1) panic
2) take time thinking about underlying cause
3) bounce the bastard and pray **
FACT: Answer#3 will often work so well, you
often don’t really spend time to think about
doing #2**



** FibreError?
Imagine a programming environment:
 - unlimited process spawning
 - concurrency someone else’s problem
 - don’t try to handle errors - let it fail
Erlang
Built for stability, and not bothering me.
Anything else is gravy


I wasn’t an erlang programmer
Process isolation
- it works so well
- nothing shared
- has worked so well for so long with OSes
- in erlang - each ‘process’ is like a process !
nginx    Example: My Host OS

     haproxy               Erlang process

                         supervisor
                            ...
                       p1         p999



Each: own heap, garbage collector
Erlang is:
- A vm based runtime (light !)
- language with prolog heritage (!)
- compiler
- interpreter
- debug tools - batteries included
- OTP - an application (server?) platform
Erlang is:
- Not OO
- no shared state **
- state kept in “term stores” - mini DBs
- processes easy to hot swap
- Quasi functional programming
    (immutable variables, higher order)
- distributed (if you want)
OTP
- Open Telecommunications Platform
- not about telco !
- network stacks/apis, supervisor trees
- toolkit for reliable servers (middleware)
- behaviours
State in erlang




files
         Host + real kernel
                        Erlang process

DETS                supervisor
             ...
ETS                p1                    p999



              Send                       Receive
Erlang processes, like OS processes


- Can only send messages to each other
- Can fail independently
- Can be restarted, safely, independently
- Easy to write apps for - Old School !
Better


- Tiny overhead (can have 100K+ of them !)
- perfect for supervisory agents
- distribute: run on every box if you like !
- lots of processes == lots of concurrency
- if you need to block, block !
Have used erlang for


- Supervisory agents - controllers
- Autoscaling services (realtime data analytics)
- misc backend systems
- fun command line tools !
- is “lite” VM (startup, footprint)
Supervisor configuration


- Declare rules for restarting, failure
- So you don’t have to intervene !
- can hot reload changes
- can have dependent processes
Restart just busted process
Program 1 (module with behaviour)
Program 2 (module with behaviour)
Anyway, that is why we use Erlang, also...
- RabbitMQ
- Riak (nosql DB)
- CouchDB (nosql DB)
- Probably your telephone exchange
Alternative approaches:
- Use “native” with daemontools/monit etc
- Akka (Scala) has similar supervisor
- library for your favourite language
- ephemeral “state less” apps
- Write software without bugs
- Or just restart everything 400 times a day
What is not awesome:
- OTP boiler plate
- VM not as fast as some
- No one seems to like the language
- tools? IDEs?
Thank you

More Related Content

What's hot

Cassandra: Not Just NoSQL, It's MoSQL
Cassandra: Not Just NoSQL, It's MoSQLCassandra: Not Just NoSQL, It's MoSQL
Cassandra: Not Just NoSQL, It's MoSQL
Eric Evans
 
Spark intro by Adform Research
Spark intro by Adform ResearchSpark intro by Adform Research
Spark intro by Adform Research
Vasil Remeniuk
 

What's hot (19)

Application Profiling for Memory and Performance
Application Profiling for Memory and PerformanceApplication Profiling for Memory and Performance
Application Profiling for Memory and Performance
 
Async and Await on the Server
Async and Await on the ServerAsync and Await on the Server
Async and Await on the Server
 
Task parallel library presentation
Task parallel library presentationTask parallel library presentation
Task parallel library presentation
 
Debugging ansible modules
Debugging ansible modulesDebugging ansible modules
Debugging ansible modules
 
Cassandra: Not Just NoSQL, It's MoSQL
Cassandra: Not Just NoSQL, It's MoSQLCassandra: Not Just NoSQL, It's MoSQL
Cassandra: Not Just NoSQL, It's MoSQL
 
Celery introduction
Celery introductionCelery introduction
Celery introduction
 
Testing in Scala. Adform Research
Testing in Scala. Adform ResearchTesting in Scala. Adform Research
Testing in Scala. Adform Research
 
Ansible: What, Why & How
Ansible: What, Why & HowAnsible: What, Why & How
Ansible: What, Why & How
 
Spark intro by Adform Research
Spark intro by Adform ResearchSpark intro by Adform Research
Spark intro by Adform Research
 
Play Framework
Play FrameworkPlay Framework
Play Framework
 
Gabriele Santomaggio - Inside Elixir/Erlang - Codemotion Milan 2018
Gabriele Santomaggio - Inside Elixir/Erlang - Codemotion Milan 2018Gabriele Santomaggio - Inside Elixir/Erlang - Codemotion Milan 2018
Gabriele Santomaggio - Inside Elixir/Erlang - Codemotion Milan 2018
 
Continuous testing and deployment in Perl (London.pm Technical Meeting Octobe...
Continuous testing and deployment in Perl (London.pm Technical Meeting Octobe...Continuous testing and deployment in Perl (London.pm Technical Meeting Octobe...
Continuous testing and deployment in Perl (London.pm Technical Meeting Octobe...
 
Automating Perl deployments with Hudson
Automating Perl deployments with HudsonAutomating Perl deployments with Hudson
Automating Perl deployments with Hudson
 
Async/Await
Async/AwaitAsync/Await
Async/Await
 
Open stack and_vagrant-os-meetup-2015
Open stack and_vagrant-os-meetup-2015Open stack and_vagrant-os-meetup-2015
Open stack and_vagrant-os-meetup-2015
 
Coroutines in Kotlin
Coroutines in KotlinCoroutines in Kotlin
Coroutines in Kotlin
 
The art of concurrent programming
The art of concurrent programmingThe art of concurrent programming
The art of concurrent programming
 
Gatling @ Scala.Io 2013
Gatling @ Scala.Io 2013Gatling @ Scala.Io 2013
Gatling @ Scala.Io 2013
 
Async await
Async awaitAsync await
Async await
 

Viewers also liked (7)

裝飾設計
裝飾設計裝飾設計
裝飾設計
 
Errors and handling them. YOW nights Sydney 2011
Errors and handling them. YOW nights Sydney 2011Errors and handling them. YOW nights Sydney 2011
Errors and handling them. YOW nights Sydney 2011
 
Devoxx 2014 michael_neale
Devoxx 2014 michael_nealeDevoxx 2014 michael_neale
Devoxx 2014 michael_neale
 
Scala sydoct2011
Scala sydoct2011Scala sydoct2011
Scala sydoct2011
 
Cors michael
Cors michaelCors michael
Cors michael
 
Legal counsel it_cv
Legal counsel it_cvLegal counsel it_cv
Legal counsel it_cv
 
Cross site calls with javascript - the right way with CORS
Cross site calls with javascript - the right way with CORSCross site calls with javascript - the right way with CORS
Cross site calls with javascript - the right way with CORS
 

Similar to Osdc 2011 michael_neale

0.5mln packets per second with Erlang
0.5mln packets per second with Erlang0.5mln packets per second with Erlang
0.5mln packets per second with Erlang
Maxim Kharchenko
 
Joe armstrong erlanga_languageforprogrammingreliablesystems
Joe armstrong erlanga_languageforprogrammingreliablesystemsJoe armstrong erlanga_languageforprogrammingreliablesystems
Joe armstrong erlanga_languageforprogrammingreliablesystems
Sentifi
 
PuppetDB: Sneaking Clojure into Operations
PuppetDB: Sneaking Clojure into OperationsPuppetDB: Sneaking Clojure into Operations
PuppetDB: Sneaking Clojure into Operations
grim_radical
 
Spark Autotuning - Strata EU 2018
Spark Autotuning - Strata EU 2018Spark Autotuning - Strata EU 2018
Spark Autotuning - Strata EU 2018
Holden Karau
 
Chelberg ptcuser 2010
Chelberg ptcuser 2010Chelberg ptcuser 2010
Chelberg ptcuser 2010
Clay Helberg
 
DevOps Days Vancouver 2014 Slides
DevOps Days Vancouver 2014 SlidesDevOps Days Vancouver 2014 Slides
DevOps Days Vancouver 2014 Slides
Alex Cruise
 
MongoDB Use Cases: Healthcare, CMS, Analytics
MongoDB Use Cases: Healthcare, CMS, AnalyticsMongoDB Use Cases: Healthcare, CMS, Analytics
MongoDB Use Cases: Healthcare, CMS, Analytics
MongoDB
 

Similar to Osdc 2011 michael_neale (20)

Repeating History...On Purpose...with Elixir
Repeating History...On Purpose...with ElixirRepeating History...On Purpose...with Elixir
Repeating History...On Purpose...with Elixir
 
Erlang
ErlangErlang
Erlang
 
0.5mln packets per second with Erlang
0.5mln packets per second with Erlang0.5mln packets per second with Erlang
0.5mln packets per second with Erlang
 
An introduction to erlang
An introduction to erlangAn introduction to erlang
An introduction to erlang
 
Joe armstrong erlanga_languageforprogrammingreliablesystems
Joe armstrong erlanga_languageforprogrammingreliablesystemsJoe armstrong erlanga_languageforprogrammingreliablesystems
Joe armstrong erlanga_languageforprogrammingreliablesystems
 
Puppet for Sys Admins
Puppet for Sys AdminsPuppet for Sys Admins
Puppet for Sys Admins
 
REX about installing Kubernetes on Premise
REX about installing Kubernetes on PremiseREX about installing Kubernetes on Premise
REX about installing Kubernetes on Premise
 
PuppetDB: Sneaking Clojure into Operations
PuppetDB: Sneaking Clojure into OperationsPuppetDB: Sneaking Clojure into Operations
PuppetDB: Sneaking Clojure into Operations
 
You shall not get excited
You shall not get excitedYou shall not get excited
You shall not get excited
 
Understanding Spark Tuning: Strata New York
Understanding Spark Tuning: Strata New YorkUnderstanding Spark Tuning: Strata New York
Understanding Spark Tuning: Strata New York
 
Spark Autotuning Talk - Strata New York
Spark Autotuning Talk - Strata New YorkSpark Autotuning Talk - Strata New York
Spark Autotuning Talk - Strata New York
 
Spark Autotuning - Strata EU 2018
Spark Autotuning - Strata EU 2018Spark Autotuning - Strata EU 2018
Spark Autotuning - Strata EU 2018
 
Fault Tolerance in Spark: Lessons Learned from Production: Spark Summit East ...
Fault Tolerance in Spark: Lessons Learned from Production: Spark Summit East ...Fault Tolerance in Spark: Lessons Learned from Production: Spark Summit East ...
Fault Tolerance in Spark: Lessons Learned from Production: Spark Summit East ...
 
Chelberg ptcuser 2010
Chelberg ptcuser 2010Chelberg ptcuser 2010
Chelberg ptcuser 2010
 
DevOps Days Vancouver 2014 Slides
DevOps Days Vancouver 2014 SlidesDevOps Days Vancouver 2014 Slides
DevOps Days Vancouver 2014 Slides
 
Introduction to Erlang/(Elixir) at a Webilea Hands-On Session
Introduction to Erlang/(Elixir) at a Webilea Hands-On SessionIntroduction to Erlang/(Elixir) at a Webilea Hands-On Session
Introduction to Erlang/(Elixir) at a Webilea Hands-On Session
 
Concurrent Programming with Ruby and Tuple Spaces
Concurrent Programming with Ruby and Tuple SpacesConcurrent Programming with Ruby and Tuple Spaces
Concurrent Programming with Ruby and Tuple Spaces
 
MongoDB Use Cases: Healthcare, CMS, Analytics
MongoDB Use Cases: Healthcare, CMS, AnalyticsMongoDB Use Cases: Healthcare, CMS, Analytics
MongoDB Use Cases: Healthcare, CMS, Analytics
 
Beam me up, Scotty
Beam me up, ScottyBeam me up, Scotty
Beam me up, Scotty
 
Elixir
ElixirElixir
Elixir
 

More from Michael Neale

More from Michael Neale (11)

Jenkins X intro (from google app dev conference)
Jenkins X intro (from google app dev conference)Jenkins X intro (from google app dev conference)
Jenkins X intro (from google app dev conference)
 
Cd syd
Cd sydCd syd
Cd syd
 
Microservices and functional programming
Microservices and functional programmingMicroservices and functional programming
Microservices and functional programming
 
Java one 2011_michaelneale
Java one 2011_michaelnealeJava one 2011_michaelneale
Java one 2011_michaelneale
 
Sjug aug 2010_cloud
Sjug aug 2010_cloudSjug aug 2010_cloud
Sjug aug 2010_cloud
 
SJUG March 2010 Restful design
SJUG March 2010 Restful designSJUG March 2010 Restful design
SJUG March 2010 Restful design
 
On Scala Slides - OSDC 2009
On Scala Slides - OSDC 2009On Scala Slides - OSDC 2009
On Scala Slides - OSDC 2009
 
Osdc Complex Event Processing
Osdc Complex Event ProcessingOsdc Complex Event Processing
Osdc Complex Event Processing
 
Scala Sjug 09
Scala Sjug 09Scala Sjug 09
Scala Sjug 09
 
Jaoo Michael Neale 09
Jaoo Michael Neale 09Jaoo Michael Neale 09
Jaoo Michael Neale 09
 
Osdc Michael Neale 2008
Osdc Michael Neale 2008Osdc Michael Neale 2008
Osdc Michael Neale 2008
 

Recently uploaded

Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
UXDXConf
 

Recently uploaded (20)

IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and Planning
 
How we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdfHow we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdf
 
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
 
The Metaverse: Are We There Yet?
The  Metaverse:    Are   We  There  Yet?The  Metaverse:    Are   We  There  Yet?
The Metaverse: Are We There Yet?
 
Speed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in MinutesSpeed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in Minutes
 
WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM Performance
 
Oauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftOauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoft
 
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptxWSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
 
The UX of Automation by AJ King, Senior UX Researcher, Ocado
The UX of Automation by AJ King, Senior UX Researcher, OcadoThe UX of Automation by AJ King, Senior UX Researcher, Ocado
The UX of Automation by AJ King, Senior UX Researcher, Ocado
 
Syngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdfSyngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdf
 
ECS 2024 Teams Premium - Pretty Secure
ECS 2024   Teams Premium - Pretty SecureECS 2024   Teams Premium - Pretty Secure
ECS 2024 Teams Premium - Pretty Secure
 
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
 
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi IbrahimzadeFree and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
 
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya HalderCustom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder
 
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdfWhere to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
 
Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024
 
What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
 

Osdc 2011 michael_neale

  • 2. @michaelneale github.com/michaelneale currently: trying to make less reliable things more reliable.
  • 3. Some things we use to get it done: - scala (back end apps) - ruby (chef, rails, proxymachine, lots) - java (jenkins and plugins) - erlang (“agents” and lots more..)
  • 4. Exceptions. Errors. What The Hell Is That Process Doing? and how it relates to..
  • 7. But it seems harmless...
  • 8.
  • 9.
  • 10. Erlang says no no no...
  • 11. Question: a process on server not responding Answer: 1) panic 2) take time thinking about underlying cause 3) bounce the bastard and pray **
  • 12. FACT: Answer#3 will often work so well, you often don’t really spend time to think about doing #2** ** FibreError?
  • 13.
  • 14. Imagine a programming environment: - unlimited process spawning - concurrency someone else’s problem - don’t try to handle errors - let it fail
  • 15. Erlang Built for stability, and not bothering me. Anything else is gravy I wasn’t an erlang programmer
  • 16. Process isolation - it works so well - nothing shared - has worked so well for so long with OSes - in erlang - each ‘process’ is like a process !
  • 17. nginx Example: My Host OS haproxy Erlang process supervisor ... p1 p999 Each: own heap, garbage collector
  • 18. Erlang is: - A vm based runtime (light !) - language with prolog heritage (!) - compiler - interpreter - debug tools - batteries included - OTP - an application (server?) platform
  • 19. Erlang is: - Not OO - no shared state ** - state kept in “term stores” - mini DBs - processes easy to hot swap - Quasi functional programming (immutable variables, higher order) - distributed (if you want)
  • 20.
  • 21.
  • 22.
  • 23. OTP - Open Telecommunications Platform - not about telco ! - network stacks/apis, supervisor trees - toolkit for reliable servers (middleware) - behaviours
  • 24.
  • 25.
  • 26.
  • 27. State in erlang files Host + real kernel Erlang process DETS supervisor ... ETS p1 p999 Send Receive
  • 28. Erlang processes, like OS processes - Can only send messages to each other - Can fail independently - Can be restarted, safely, independently - Easy to write apps for - Old School !
  • 29. Better - Tiny overhead (can have 100K+ of them !) - perfect for supervisory agents - distribute: run on every box if you like ! - lots of processes == lots of concurrency - if you need to block, block !
  • 30. Have used erlang for - Supervisory agents - controllers - Autoscaling services (realtime data analytics) - misc backend systems - fun command line tools ! - is “lite” VM (startup, footprint)
  • 31. Supervisor configuration - Declare rules for restarting, failure - So you don’t have to intervene ! - can hot reload changes - can have dependent processes
  • 33. Program 1 (module with behaviour)
  • 34. Program 2 (module with behaviour)
  • 35. Anyway, that is why we use Erlang, also... - RabbitMQ - Riak (nosql DB) - CouchDB (nosql DB) - Probably your telephone exchange
  • 36. Alternative approaches: - Use “native” with daemontools/monit etc - Akka (Scala) has similar supervisor - library for your favourite language - ephemeral “state less” apps - Write software without bugs - Or just restart everything 400 times a day
  • 37. What is not awesome: - OTP boiler plate - VM not as fast as some - No one seems to like the language - tools? IDEs?

Editor's Notes

  1. \n
  2. \n
  3. \n
  4. \n
  5. \n
  6. \n
  7. \n
  8. \n
  9. \n
  10. \n
  11. \n
  12. \n
  13. \n
  14. \n
  15. \n
  16. \n
  17. \n
  18. \n
  19. \n
  20. \n
  21. \n
  22. \n
  23. \n
  24. \n
  25. \n
  26. \n
  27. \n
  28. \n
  29. \n
  30. \n
  31. \n
  32. \n
  33. \n
  34. \n
  35. \n
  36. \n
  37. \n
  38. \n