Building beyond billions
•Télécharger en tant que PPTX, PDF•
0 j'aime•323 vues
A presentation on how to build High Throughput Systems with Node.js
Recommandé
Contenu connexe
Similaire à Building beyond billions
Similaire à Building beyond billions (20)
Dernier
Dernier (20)
Building beyond billions
- 1. Building Beyond Billions High Throughput Systems w/ Node.js
- 2. Overview Where should we spend time? 10 principles of high throughput systems Stress testing An inside look into our system Node.js pros and cons
- 3. 104,000 reqs / sec Augur 4,000 reqs / sec Building Beyond Billions almost … there...
- 4. # of consumer devices 2016 5 B 50 Billion 20202017 2018 2019
- 5. Web tracking, via API
- 6. Dieter Rams Nawar Alsafar Ten Principles of Good Design Software Architecture
- 7. Ten principles of good software architecture 1) Good architecture is embarrassingly parallel 2) Good architecture has known bounds 3) Good architecture has wire time bottleneck 4) Good architecture reduces cost 5) Good architecture uses application logic that is understandable and elementary in style 1) Good architecture self-heals 2) Good architecture is rapidly updated 3) Good architecture consistently performs 4) Good architecture is always communicating 5) Good architecture uses less memory
- 8. Ten principles of good software architecture Good architecture is embarrassingly parallel Good architecture self-heals
- 9. Ten principles of good software architecture Good architecture has wiretime bottleneck Good architecture consistently performs
- 10. Ten principles of good software architecture Good architecture has known bounds Good architecture is rapidly updated
- 11. Ten principles of good software architecture Good architecture reduces cost Good architecture is always communicating
- 12. Ten principles of good software architecture Good architecture uses application logic that is understandable and elementary in style Good architecture uses less memory
- 14. DNS Pub / Sub Web iOS Android Clients Global LB U.S. API server U.S. E.U. C.H. Recon workers Events server Analytics worker ETL Front- end server Site / web app Firewall C.H.E.U. Aerospike clusters U.S. Firewall Google NoSQL Firewall Google Fiber request Normal net request
- 15. Concurrency model Stack: context Heap: mem. alloc. Queue: things to do Special properties Run-to-completion Non-blocking execution JIT compiler Full compiler Optimized compiler Node.js - under the hood
- 16. Node.js - pros / cons Pros Biggest ecosystem Fast, thanks to v8, crankshaft, turbofan Getting faster every week One language can be used for all web dev Cons No central memory share is available GC could be better Special knowledge needed to max perf
Notes de l'éditeur
- less elaboration
- Good architecture is embarrassingly parallel where little or no effort is needed to separate the problem into a number of parallel tasks.[1] This is often the case where there is little or no dependency, or need for communication between, those parallel tasks, or for results between them. story: by definition, you want as many workers to handle incoming requests as possible. this is pure parallel computing, where receiving and responding to an incoming request is independent of any other incoming request. For us, we call these our recon workers, and the workhorses of our fleet Good architecture is self-healing Now our workhorses arent always healthy, we keep them alive as long as possible, but in their current design, they may randomly explode, for a currently unknown reason, which is under investigation. That being said, we know this limitation exists, and our system is built to check the health the pulse of each horse, so when a horse is dead, a new horse is created to take its steed. this self-healing process is important, since you never what can occur. this makes both the application and databases resliant to changes. also, when our incoming traffic increases beyond the current capcity, then the system heals itself by adding more capacity, or if we have too many horses, it heals itself by down-scaling. this is also a self-healing system
- Good architecture has wiretime bottleneck This really points at the hardware and software code not bottlenecking you, instead, the biggest source of lag is wiretime / download latency. when you have a high throughput system, then rule is true, otherwise its false Good architecture consistently performs If you system has high variability, then some is wrong with your hardware, or mostly likely, your application’s logic. for us, we try to keep our high-throughput, real-time system to under 10ms response times. at the time of this writing, our median is 8ms, and average is about double that. the goal of a high throughput system is have a set of expected behavior, that remains under the designers control. this is part of that … if your system behaves erratically, that should be by design, not for unknown reasons
- Good architecture has known bounds You need to know what the capacity is per server, or when shit will begin to melt. Some months ago we onboarded a real-time bidding company, basically someone who does a billion plus transactions a month, or thousands of requests per a second. we were stoked, they were going to pay big bucks….and we saw those sweet little hits come in, ooh yeah...that was money...and then our Aerospike database failed, both nodes, the entire cluster, that everything depended on failed (we think due to the high throughput) and we couldnt get it back on, as all the hits kept coming in...it was a holy shit, everything is melting and we’re fucked moment. Knowing the capacity of your system prevents this, since you can know, where your bottleneck is, vs being suprised by it. This allows you to be growing and modify archiecture at certain growth leaps and is very improtant Good architecture is rapidly updated Software sucks when its gets slowed updates...it doesnt care to stay in sync with customer needs, it doesnt adapt fast enough to survive climate change. The ability to rapidly release allows the architecture to stay relevant. When our customers ask for features, and I feel like I cant push out an update, that makes them sad, meanwhile when I push out updates several times a day to address any of their concerns, even though I caused the issue, I am the hero that saved the day, building customer satisfaction from negative scenarios
- Good architecture reduces cost if you dont do this, as you grow, the economic unit you sell gets more expensive. on the contrary, growing makes everything cheaper and cheaper, you have a high efficiency industrial operation. story: for example, we had over 200-600 million events floating in our Pub / Sub, waiting to be processed. Our analytics worker couldn’t keep up with the growth of events, so we changed its logic...simplifying it, knowing it has a certain set of faults that could be addressed next iteration / update. with the upgrade, each server now had 200-300 micro-workers, which increased our throughput to 100,000 messages processed per sec. as a result, we decreased the size of the server in half, and still had more than sufficient capacity. i think we can do this again, so we went from >$600 to handle billions of events, to about $300. I think we can further reduce the cost down to $100. good architecture reduces cost Good architecture is always communicating In that the same example, we had put a dashboard which displayed the number of Pub / Sub messages that were accumulated in the Pub / Sub pipe. We noticed there were over 200 Million, using our instrumentation / dashboard, we monitored the number of messages, throughput, and could see when we our architectural approach was good enough to bring the 200 million to 0. Thanks to our architecture communicating with us, we saw a problem and could see how our rapid updates effected the problem
- Good architecture uses application logic that is understandable and elementary in style Good logic simple and straightforward to any intermediate developer. The best code is no code. The more code, the worse the code. Accomplishing a set of logic using 10 lines is better than 100 lines. This makes code maintainable and easy to change. When you dont follow this law, you get “Page’s Law” Page's Law is the inverse of Moore’s Law: It says software gets twice as slow every 18 months. This helps explain why your computer seems to get slower as it ages, even though the hardware inside remains unchanged. When you follow this principle, Page’s Law is untrue, and you’re application logic gets fully optimized at every iteration...so every release has the opportunity to speeds things up, never slow things down. This also prevents over-engineering Good architecture uses less memory In the world of physics, we have space-time. You less execution time? Then use more space. You want more space? Then use up more time. This relationship of memory time is interesting. What’s more interesting to approach the archiecture in a more clever fashion. Examples of this approach can be seen in large-scale reinforcement learning. For example when you type into Google, Yahoo, or Bing a query, a relevant banner ad is displayed. The choice of what ad to render is based on your consumer insights, search history, and current query, with milliseconds the goal is to find the optimal set of ads to display to you, as well as the most relevant set of search results. This is a really interesting problem to solve, since you have so much data in memory, that to do the math, would require a lot of parelling processing. Instead of doing that, in machine learning, we use an online algorithm that uses a small amount of memory to take a best guess at which ad will best match your query. We use less memory and still accomplish the goal. This same approach should be applied to high throughput systems, since we have so much data, its not about big data, its about the right small data. We’re about to design a system that collects billions of unique datapoints, and keeps its history, we plan to use clever tricks to keep the state of the information in the system, meanwhile we throw most of the actual data each month. in this way, we use memory, while speeding things up. its all about how you approach modeling the math