This is a 15 minute presentation I did
at work recently about threads,
polling and node.js.

It rests heavily on the shoul...
non-blocking I/O,
event loops,
javascript and
node.js.
Threads suck
Why do we
use threads?
Policy p = cmServer.getPolicy(...);

System.out.println(p.getContentId().toString());
Policy p = cmServer.getPolicy(...);

System.out.println(p.getContentId().toString());
Policy p = cmServer.getPolicy(...);
Blocking implies multiple execution stacks. Where else to go?

System.out.println(p.ge...
How to deal with blocking?
Processes
    No shared memory
    Heavy

OS Threads
    Can share memory
    Relatively cheap
...
Are threads cheaper
than processes?
HTTP request latency; glibc 2.3.2 on Linux 2.6
                     HTTP-anropslatens; glibc 2.3.2 på Linux 2.6




      ...
HTTP request latency; glibc 2.3.2 on Linux 2.6




                    1 thread per call




                             ...
Are threads
cheap at all?
nginx vs apache
       http://blog.webfaction.com/a-little-holiday-present
nginx vs apache
           http://blog.webfaction.com/a-little-holiday-present




  ∆ = Apaches context switching
lmbench
         ./lat_ctx -N 10 -s 4096 0 1 .. 20

80

74

68

62

56

50
     2     3   4   5   6   7   8   9   10 11 12...
lmbench
         ./lat_ctx -N 10 -s 4096 0 1 .. 20

80

74

68

62

56

50
     2     3   4   5   6   7   8   9   10 11 12...
65 µs x 2 x 4000 = 520 ms
How about memory?
> ulimit -a | grep stack
stack size     (kbytes, -s) 8192
...but at least 512+1 kb
For OS X, source: ADC
nginx vs apache
       http://blog.webfaction.com/a-little-holiday-present
nginx vs apache
              http://blog.webfaction.com/a-little-holiday-present




Difference: Threads and processes co...
What about ngnix?
The cost of I/O

L1-cache             3 cycles
L2-cache            14 cycles
RAM                250 cycles
Disk        41 ...
The weight of I/O
The weight of I/O

L1-cache   A big squirrel
The weight of I/O

L1-cache   A big squirrel
L2-cache   A medium-sized cat
The weight of I/O

L1-cache   A big squirrel
L2-cache   A medium-sized cat
RAM        Mattias, basically
The weight of I/O

L1-cache   A big squirrel
L2-cache   A medium-sized cat
RAM        Mattias, basically
Disk       Like a...
The weight of I/O

L1-cache   A big squirrel
L2-cache   A medium-sized cat
RAM        Mattias, basically
Disk       Like a...
Servers pretty I/O-intensive
             =
   Squirrelʼs burried in
   whales fat and wheat
Policy p = cmServer.getPolicy(...);

System.out.println(p.getContentId().toString());
cmServer.getPolicy(function (Policy p) {
  System.out.println(
     p.getContentId().toString()
  );
});
cmServer.getPolicy(function (Policy p) {
  System.out.println(
           Sleeps until p’s available
     p.getContentId()...
cmServer.getPolicy(function (Policy p) {
  System.out.println(
            Sover tills dess p finns
     p.getContentId()....
Non-blocking I/O...

All code is run in a single thread
Functions listen on events,
act and sleep
...is pretty demanding


All I/O has to be non-blocking
Callbacks are pretty ugly without HOF
V8 + node.js
“a purely evented,
 non-blocking
 infrastructure to script
 highly concurrent
 programs.”
V8 Chromes JS engine
JS Built to be event driven
node.js Queues built on /dev/epoll
        amongst other things
var http = require('http');

http.createServer(function (req, res) {
  setTimeout(function () {
    res.sendHeader(200, {'...
% node hello.js
% ab -n 1000 -c 200 http://127.0.0.1:8080/

...

Time per request:      2010.070 [ms] (mean)
             ...
var stat = require('posix').stat,
      puts = require('sys').puts;


var promise = stat('/etc/passwd');


promise.addCall...
var web = require('./web');


web.server(function (route) {
    route.get("^/([a-z]+)$", function(parms, req, res) {
     ...
% node web.test.js
% ab -n 10000 -c 250 http://127.0.0.1:8080/marcus
...

Requests per second:    4851.93 [#/sec] (mean)
T...
var web   = require('./web'),
    posix = require('posix');

web.server(function (route) {
    route.get("^/([a-z]+)$", fu...
throughput
80

74

68

62

56

50
     1   5   13   25   50   100   250
HEALTH
Distributed ab

One configuration, multiple nodes
REST interface
Streams JSON over HTTP
Decent performance
demo!
Prochain SlideShare
Chargement dans…5
×

Non-blocking I/O, Event loops and node.js

27 395 vues

Publié le

A lightning talk I held at a geek night at work about my recent experience with node.js, and my fledgling love story with javascript.

Publié dans : Technologie
0 commentaire
59 j’aime
Statistiques
Remarques
  • Soyez le premier à commenter

Aucun téléchargement
Vues
Nombre de vues
27 395
Sur SlideShare
0
Issues des intégrations
0
Intégrations
375
Actions
Partages
0
Téléchargements
0
Commentaires
0
J’aime
59
Intégrations 0
Aucune incorporation

Aucune remarque pour cette diapositive

Non-blocking I/O, Event loops and node.js

  1. 1. This is a 15 minute presentation I did at work recently about threads, polling and node.js. It rests heavily on the shoulders of giants, especially Ryan Dahls original node.js talk at jsconf.eu.
  2. 2. non-blocking I/O, event loops, javascript and node.js.
  3. 3. Threads suck
  4. 4. Why do we use threads?
  5. 5. Policy p = cmServer.getPolicy(...); System.out.println(p.getContentId().toString());
  6. 6. Policy p = cmServer.getPolicy(...); System.out.println(p.getContentId().toString());
  7. 7. Policy p = cmServer.getPolicy(...); Blocking implies multiple execution stacks. Where else to go? System.out.println(p.getContentId().toString());
  8. 8. How to deal with blocking? Processes No shared memory Heavy OS Threads Can share memory Relatively cheap Green threads / Co-routines
  9. 9. Are threads cheaper than processes?
  10. 10. HTTP request latency; glibc 2.3.2 on Linux 2.6 HTTP-anropslatens; glibc 2.3.2 på Linux 2.6 1 thread per call 1 process per call http://bulk.fefe.de/scalable-networking.pdf Open connections
  11. 11. HTTP request latency; glibc 2.3.2 on Linux 2.6 1 thread per call 1 process per call Which is the reason why we use thread pools! Open connections
  12. 12. Are threads cheap at all?
  13. 13. nginx vs apache http://blog.webfaction.com/a-little-holiday-present
  14. 14. nginx vs apache http://blog.webfaction.com/a-little-holiday-present ∆ = Apaches context switching
  15. 15. lmbench ./lat_ctx -N 10 -s 4096 0 1 .. 20 80 74 68 62 56 50 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
  16. 16. lmbench ./lat_ctx -N 10 -s 4096 0 1 .. 20 80 74 68 62 56 50 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 On a MBP, OS X 10.6 Context switch around 65 µs
  17. 17. 65 µs x 2 x 4000 = 520 ms
  18. 18. How about memory?
  19. 19. > ulimit -a | grep stack stack size (kbytes, -s) 8192
  20. 20. ...but at least 512+1 kb For OS X, source: ADC
  21. 21. nginx vs apache http://blog.webfaction.com/a-little-holiday-present
  22. 22. nginx vs apache http://blog.webfaction.com/a-little-holiday-present Difference: Threads and processes cost.
  23. 23. What about ngnix?
  24. 24. The cost of I/O L1-cache 3 cycles L2-cache 14 cycles RAM 250 cycles Disk 41 000 000 cycles Network 240 000 000 cycles
  25. 25. The weight of I/O
  26. 26. The weight of I/O L1-cache A big squirrel
  27. 27. The weight of I/O L1-cache A big squirrel L2-cache A medium-sized cat
  28. 28. The weight of I/O L1-cache A big squirrel L2-cache A medium-sized cat RAM Mattias, basically
  29. 29. The weight of I/O L1-cache A big squirrel L2-cache A medium-sized cat RAM Mattias, basically Disk Like a 100 blue whales
  30. 30. The weight of I/O L1-cache A big squirrel L2-cache A medium-sized cat RAM Mattias, basically Disk Like a 100 blue whales Network Belarus yearly wheat import
  31. 31. Servers pretty I/O-intensive = Squirrelʼs burried in whales fat and wheat
  32. 32. Policy p = cmServer.getPolicy(...); System.out.println(p.getContentId().toString());
  33. 33. cmServer.getPolicy(function (Policy p) { System.out.println( p.getContentId().toString() ); });
  34. 34. cmServer.getPolicy(function (Policy p) { System.out.println( Sleeps until p’s available p.getContentId().toString() ); });
  35. 35. cmServer.getPolicy(function (Policy p) { System.out.println( Sover tills dess p finns p.getContentId().toString() ); }); // And down here we can do other stuff
  36. 36. Non-blocking I/O... All code is run in a single thread Functions listen on events, act and sleep
  37. 37. ...is pretty demanding All I/O has to be non-blocking Callbacks are pretty ugly without HOF
  38. 38. V8 + node.js
  39. 39. “a purely evented, non-blocking infrastructure to script highly concurrent programs.”
  40. 40. V8 Chromes JS engine JS Built to be event driven node.js Queues built on /dev/epoll amongst other things
  41. 41. var http = require('http'); http.createServer(function (req, res) { setTimeout(function () { res.sendHeader(200, {'Content-Type': 'text/plain'}); res.sendBody('Hello World'); res.finish(); }, 2000); }).listen(8080);
  42. 42. % node hello.js % ab -n 1000 -c 200 http://127.0.0.1:8080/ ... Time per request: 2010.070 [ms] (mean) min mean[+/-sd] median max Connect: 0 2 1.9 1 8 Processing: 2001 2006 2.8 2005 2013 Waiting: 2000 2003 2.0 2003 2009 Total: 2002 2008 3.5 2006 2015 Percentage of the requests served within a certain time (ms) 50% 2006 66% 2008 75% 2012 80% 2013 90% 2013 95% 2014 98% 2014 99% 2014 100% 2015 (longest request)
  43. 43. var stat = require('posix').stat, puts = require('sys').puts; var promise = stat('/etc/passwd'); promise.addCallback(function (s) { puts('modified: ' + s.mtime); });
  44. 44. var web = require('./web'); web.server(function (route) { route.get("^/([a-z]+)$", function(parms, req, res) { res.sendHeader(200, {'Content-Type':'text/plain'}); res.sendBody("Hello " + parms[0]); res.finish(); }); }).listen(8080);
  45. 45. % node web.test.js % ab -n 10000 -c 250 http://127.0.0.1:8080/marcus ... Requests per second: 4851.93 [#/sec] (mean) Time per request: 51.526 [ms] (mean) Time per request: 0.206 [ms] (mean, across all c requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 14 111.8 1 999 Processing: 4 35 15.8 31 103 Waiting: 4 35 15.8 30 103 Total: 14 49 112.5 32 1035 Percentage of the requests served within a certain time (ms) 50% 32 66% 34 75% 35 80% 36 90% 57 95% 92 98% 102 99% 986 100% 1035 (longest request)
  46. 46. var web = require('./web'), posix = require('posix'); web.server(function (route) { route.get("^/([a-z]+)$", function(parms, req, res) { posix.cat(parms[0]).addCallback(function(text) { res.sendHeader(200, {'Content-Type':'text/plain'}); res.sendBody(text); res.finish(); }); }); }).listen(8080);
  47. 47. throughput 80 74 68 62 56 50 1 5 13 25 50 100 250
  48. 48. HEALTH
  49. 49. Distributed ab One configuration, multiple nodes REST interface Streams JSON over HTTP Decent performance
  50. 50. demo!

×