Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (16)
Similaire à Your Code is Wrong
Similaire à Your Code is Wrong (20)
Plus de nathanmarz
Plus de nathanmarz (12)
Dernier
Dernier (20)
Your Code is Wrong
- 1. Your Code is Wrong Nathan Marz @nathanmarz 1
- 2. Let’s start with an example
- 4. (Storm is a realtime computation system, like Hadoop but for realtime)
- 6. Storm architecture Master node (similar to Hadoop JobTracker)
- 7. Storm architecture Used for cluster coordination
- 8. Storm architecture Run worker processes
- 10. Used to show errors in the Storm UI
- 11. Error info is stored in Zookeeper
- 12. What happens when a user deploys code like this?
- 13. Denial-of-service on Zookeeper and cluster goes down
- 14. Robust! Designed input space Actual input space
- 15. Your code is wrong
- 16. Your code is literally wrong
- 17. Your code is wrong
- 19. Why do you believe your code is correct?
- 20. Your code Dependency 1 Dependency 2 Dependency 3
- 22. Dependency 4 Dependency 6 Dependency 9 Dependency 7 Dependency 8
- 24. Electronics
- 25. Chemistry
- 26. Atomic physics
- 28. I think I can safely say that nobody understands quantum mechanics. Richard Feynman
- 29. Your code is wrong
- 30. Your code ...
- 31. All the software you’ve used has had bugs in it
- 32. Including the software you’ve written
- 33. Your code is sometimes correct
- 36. Treat code as nondeterministic
- 37. Embrace “your code is wrong” to design better software
- 38. Robust! Designed input space Actual input space
- 39. Robust! Designed input space Actual input space
- 40. An example
- 44. Your code is wrong
- 45. So your processes will crash
- 46. Storm’s daemons are process fault-tolerant
- 52. Robust! Designed input space Actual input space
- 53. Robust! Designed input space Actual input space
- 54. The impact of code being wrong
- 55. Robust! Designed input space Actual input space Failures! Bad performance! Security holes! Irrelevant!
- 56. Design principle #1 Measuring and monitoring are the foundation of solid engineering
- 57. Measuring: Under what range of inputs does my software function well?
- 58. Monitoring: What’s the actual input space of my software?
- 59. Measure & Monitor Latency Throughput Stack traces Buffer sizes Memory usage CPU usage #threads spawned ...
- 60. How you monitor your software is as important as its functionality
- 61. Design principle #2 Embrace immutability
- 63. MySQLApplication
- 65. RiakApplication
- 67. HBaseApplication
- 68. Your code is wrong
- 69. So data will be corrupted
- 70. And you may not know why
- 71. Views Immutable, ever-growing data Application Architecture based on immutability
- 73. Design principle #3 Minimize dependencies
- 74. The less that can go wrong, the less that will go wrong
- 75. Example: Storm’s usage of Zookeeper
- 76. Worker locations stored in Zookeeper
- 77. All workers must know locations of other workers to send messages
- 78. Two ways to get location updates
- 79. 1. Poll Zookeeper Worker Zookeeper
- 80. 2. Use Zookeeper “watch” feature to get push notifications Worker Zookeeper
- 81. Method 2 is faster but relies on another feature
- 82. Storm uses both methods Worker Zookeeper
- 83. If watch feature fails, locations still propagate via polling
- 84. Eliminating dependence justified by small amount of code required
- 85. Design principle #4 Explicitly respect functional input ranges
- 87. Implement self-throttling to avoid overloading other systems
- 88. Design principle #5 Embrace recomputation
- 89. “Your code is wrong” meanings 1. Design input space differs from actual input space 2. The logic of your code is wrong 3. Requirements are constantly changing
- 90. You must be able to change your code to match shifting requirements
- 93. Have to build a search index
- 95. Recomputation gives you so much more
- 97. Building software no different than any other engineering
- 98. The underlying challenges are the same
- 101. What will break it?
- 102. What are limits of my dependencies?
- 103. How can I add redundancy to increase robustness?
- 104. Can I isolate failures?
- 105. Our raw materials are ideas instead of matter
- 106. Thank you