JavaScript used to be confined to the browser. But these days, it becoming increasingly popular in server-side applications in the form of NodeJS. NodeJS provides event-driven, non-blocking I/O model that supposedly makes it easy to build scalable network application. In this talk you will learn about the consequences of combining the event-driven programming model with a prototype-based, weakly typed, dynamic language. We will share our perspective as a server-side Java developer who wasn’t entirely happy about JavaScript in the browser, let alone on the server. You will learn how to use NodeJS effectively in modern, polyglot applications.
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
NodeJS: the good parts? A skeptic’s view (oredev, oredev2013)
1. NodeJS: the good parts?
A skeptic’s view
Chris Richardson
Author of POJOs in Action
Founder of the original CloudFoundry.com
@crichardson
chris@chrisrichardson.net
http://plainoldobjects.com
@crichardson
2. Presentation goal
How a grumpy, gray-haired
server-side Java developer
discovered an appreciation
for NodeJS and JavaScript
@crichardson
8. Agenda
Overview of NodeJS
JavaScript: Warts and all
The Reactor pattern: an event-driven architecture
NodeJS: There is a module for that
Building a front-end server with NodeJS
@crichardson
14. Dynamic and weakly-typed
Dynamic:
Types are associated with values - not variables
Define new program elements at runtime
Weakly typed:
Leave out arguments to methods
Read non-existent object properties
Add new properties by simply setting them
@crichardson
16. JavaScript is a prototypal
language
Person
__proto__
sayHello
...
Prototype
function
...
inherited
Chris __proto__
name
nickname
“Chris”
“CER”
overrides
object specific
@crichardson
17. Prototypal code
Not defined
here
$ node
> var person = { sayHello: function () { console.log("Hello " + this.name); }};
[Function]
> var chris = Object.create(person, {name: {value: "Chris"}});
undefined
> var sarah = Object.create(person, {name: {value: "Sarah"}});
undefined
> chris.sayHello();
Hello Chris
create using
undefined
properties
> sarah.sayHello();
prototype
Hello Sarah
undefined
> chris.sayHello = function () { console.log("Hello mate: " + this.name); };
[Function]
> chris.sayHello();
Hello mate: Chris
undefined
@crichardson
18. JavaScript is Functional
function makeGenerator(nextFunction) {
Return a function
closure
var value = 0;
return function() {
var current = value;
value = nextFunction(value);
return current;
};
Pass function as an
argument
}
var inc = makeGenerator(function (x) {return x + 1; });
> inc()
0
> inc()
1
@crichardson
19. But JavaScript was created
in a hurry
The ‘Java...’ name creates expectations that it can’t satisfy
Fake classes: Hides prototypes BUT still seems weird
global namespace
scope of vars is confusing
Missing return statement = confusion
‘function’ is really verbose
‘this’ is dynamically scoped
Unexpected implicit conversions: 99 == “99”!
truthy and falsy values
52-bit ints
@crichardson
20. Dynamic + weakly-typed (+ event-driven)
code
+
misspelt property names
lots of time spent in the abyss
Essential: Use IDE integrated
with JSLint/JSHint + tests
21. Prototypal languages have
benefits BUT
Developers really like classes
JavaScript prototypes lack the powerful features from
the Self language
e.g. Multiple (and dynamic) inheritance
http://www.cs.ucsb.edu/~urs/oocsb/self/papers/papers.html
@crichardson
22. Verbose function syntax
> var numbers = [1,2,3,4,5]
> numbers.filter(function (n) { return n % 2 == 0; } ).map(function (n) { return n * n; })
[ 4, 16 ]
>
Versus
Prelude> let numbers = [1,2,3,4,5]
Prelude> map (n -> n * n) (filter (n -> mod n 2 == 0) numbers)
[4,16]
Or
scala> val numbers = 1..5
scala> numbers filter { _ % 2 == 0} map { n => n * n }
Vector(4, 16)
@crichardson
23. Verbose DSLs
describe('SomeEntity', function () {
beforeEach(function () { ... some initialization ... });
Jasmine
it('should do something', function () {
...
expect(someExpression).toBe(someValue);
});
});
Versus
class SomeScalaTest ...{
Scalatest
before { ... some initialization ... }
it should "do something" in {
...
someExpression should be(someValue)
}
@crichardson
24. JavaScript is the language
of the web
“You have to use the programming
language you have, not the one that you
might want”
@crichardson
25. It works but the result is
lost opportunities
and
impeded progress
@crichardson
26. Martin Fowler once said:
"...I'm one of those who despairs that a
language with such deep flaws plays such an
important role in computation. Still the
consequence of this is that we must take
javascript seriously as a first-class language
and concentrate on how to limit the damage
its flaws cause. ...."
http://martinfowler.com/bliki/gotoAarhus2012.html
@crichardson
27. Use just the good parts
Douglas
Crockford
http://www.crockford.com/
@crichardson
28. Use a language that
compiles to JavaScript
TypeScript
Classes and interfaces (dynamic structural typing)
Typed parameters and fields
Dart
Class-based OO
Optional static typing
Bidirectional binding with DOM elements
Less backwards compatibility with JavaScript
Also has it’s own VM
@crichardson
29. CoffeeScript Hello World
Classes :-)
http = require('http')
Concise
class HttpRequestHandler
constructor: (@message) ->
Bound method
handle: (req, res) =>
res.writeHead(200, {'Content-Type': 'text/plain'})
res.end(@message + 'n')
handler = new HttpRequestHandler "Hi There from CoffeeScript"
server = http.createServer(handler.handle)
server.listen(1338, '127.0.0.1')
console.log('Server running at http://127.0.0.1:1338/')
@crichardson
32. About the Reactor pattern
Defined by Doug Schmidt in 1995
Pattern for writing scalable servers
Alternative to thread-per-connection model
Single threaded event loop dispatches events on
handles (e.g. sockets, file descriptors) to event handlers
@crichardson
34. Benefits
Separation of concerns - event handlers separated
from low-level mechanism
More efficient - no thread context switching
Simplified concurrency - single threaded = no
possibility of concurrent access to shared state
@crichardson
35. Drawbacks
Non-pre-emptive - handlers must not take a long time
Difficult to understand and debug:
Inverted flow of control
Can’t single step through code easily
Limited stack traces
No stack-based context, e.g. thread locals, exception handlers
How to enforce try {} finally {} behavior?
@crichardson
36. NodeJS app = layers of event
handlers
Recurring
events
from
Event
Emitters
Application code
Event
listener
HTTP
Callback
function
DB driver
...
One time
events:
async
operation
completion
Basic networking/file-system/etc.
NodeJS event loop
@crichardson
37. Async code = callback hell
Scenarios:
Sequential: A
B
C
Scatter/Gather: A and B
C
Code quickly becomes very messy
@crichardson
38. Messy callback code
The result of
getProductDetails
getProductDetails = (productId, callback) ->
productId = req.params.productId
result = {productId: productId}
Propagate error
makeCallbackFor = (key) ->
(error, x) ->
if error
callback(error)
else
result[key] = x
if (result.productInfo and result.recommendations and result.reviews)
callback(undefined, result)
Update result
Gather
getProductInfo(productId, makeCallbackFor('productInfo'))
getRecommendations(productId, makeCallbackFor('recommendations'))
getReviews(makeCallbackFor('reviews'))
Scatter
@crichardson
39. Simplifying code with
Promises (a.k.a. Futures)
Functions return a promise - no callback parameter
A promise represents an eventual outcome
Use a library of functions for transforming and
composing promises
Promises/A+ specification - http://promises-aplus.github.io/promises-spec
when.js (part of cujo.js by SpringSource) is a popular
implementation
Crockford’s RQ library is another option
@crichardson
41. Not bad but lacks Scala’s
syntactic sugar
class ProductDetailsService .... {
def getProductDetails(productId: Long) = {
for (((productInfo, recommendations), reviews) <getProductInfo(productId) zip
getProductInfo(productId) zip
getRecommendations(productId) zip
getRecommendations(productId) zip
getReviews(productId)
getReviews(productId))
yield
ProductDetails(productInfo, recommendations, reviews)
}
}
@crichardson
42. Long running computations
Long running computation
blocks event loop for
other requests
Need to run outside of main event loop
Options:
Community: web workers threads
Built-in: NodeJS child processes
@crichardson
43. Using child processes
parent.js
var child = require('child_process').fork('child.js');
function sayHelloToChild() {
child.send({hello: "child"});
}
Create child process
Send message to child
setTimeout(sayHelloToChild, 1000);
child.on('message', function(m) {
console.log('parent received:', m);
});
function kill() {
child.kill();
}
setTimeout(kill, 2000);
child.js
process.on('message', function (m) {
console.log("child received message=", m);
process.send({ihateyou: "you ruined my life"})
});
@crichardson
44. Modern multi-core machines
vs. single-threaded runtime
Many components of many applications
Don’t need the scalability of the Reactor pattern
Request-level thread-based parallelism works fine
There are other concurrency options
Actors, Software transactional memory, ...
Go goroutines, Erlang processes, ...
Imposing a single-threaded complexity tax on the
entire application is questionable
@crichardson
47. Thousands of community
developed modules
https://npmjs.org/
web frameworks, SQL/NoSQL
database drivers, messaging, utilities...
@crichardson
48. What’s a module?
foo.js
One or more JavaScript files
exports.sayHello = function () {
console.log(“Hello”);
}
Optional native code:
Compiled during installation
JavaScript != systems programming language
Package.json - metadata including dependencies
@crichardson
50. Easy to use
Core module OR
Path to file OR
module in node_modules
Module’s exports
var http = require(“http”)
var server = http.createServer...
@crichardson
52. There is a module for that...
Modules + glue code
=
rapid/easy application development
AWESOME!...
@crichardson
53. ... BUT
Variable quality
Multiple incomplete/competing modules, e.g. MySQL
drivers without connection pooling!
Often abandoned
No notion of a Maven-style local repository/cache =
repeated downloads
...
@crichardson
56. Agenda
Overview of NodeJS
JavaScript: Warts and all
The Reactor pattern: an event-driven architecture
NodeJS: There is a module for that
Building a front-end server with NodeJS
@crichardson
57. So why care about
NodeJS?
Easy to write scalable network services
Easy to push events to the browser
Easy to get (small) stuff done
It has a role to play in modern
application architecture
@crichardson
58. Evolving from a monolithic
architecture....
WAR
StoreFrontUI
Product Info
Service
Recommendation
Service
Review
Service
Order
Service
@crichardson
59. ... to a micro-service architecture
product info application
Product Info
Service
recommendations application
Store front application
StoreFrontUI
Recommendation
Service
reviews application
Review
Service
orders application
Order
Service
@crichardson
61. ...Presentation layer evolution
Browser
View
Web application
Static
content
Controller
JSON-REST
Model
HTML 5/JavaScript
IOS/Android clients
Events
RESTful
Endpoints
Event
publisher
@crichardson
62. Directly connecting the front-end to the backend
Chatty API
View
REST
Product Info
service
REST
Recommendation
Service
AMQP
Controller
Review
service
Model
Traditional web
application
View
Controller
Model
Browser/Native App
Web unfriendly
protocols
@crichardson
63. NodeJS as an API gateway
Single entry point
Browser
View
Controller
NodeJS
Model
HTML 5 - JavaScript
Product Info
service
REST
Recommendation
Service
AMQP
Review
service
REST
proxy
Native App
View
API
Gateway
REST
Controller
Model
Event
publishing
Optimized Client
specific APIs
Protocol
translation
@crichardson
64. Serving static content with
the Express web framework
var express = require('express')
, http = require('http')
, app = express()
, server = http.createServer(app)
;
From public
sub directory
app.configure(function(){
...
app.use(express.static(__dirname + '/public'));
});
server.listen(8081);
@crichardson
67. Implementing coarsegrained mobile API
var express = require('express'),
...;
app.get('/productdetails/:productId', function (req, res) {
getProductDetails(req.params. productId).then(
function (productDetails) {
res.json(productDetails);
}
});
@crichardson
69. Socket.io server-side
var express = require('express')
, http = require('http')
, amqp = require(‘amqp’)
....;
server.listen(8081);
...
var amqpCon = amqp.createConnection(...);
Handle
socket.io
connection
io.sockets.on('connection', function (socket) {
function amqpMessageHandler(message, headers, deliveryInfo) {
Republish
var m = JSON.parse(message.data.toString());
socket.emit(‘tick’, m);
as socket.io
};
event
amqpCon.queue(“”, {},
function(queue) {
queue.bind(“myExchange”, “”);
queue.subscribe(amqpMessageHandler);
Subscribe to
});
});
AMQP queue
https://github.com/cer/nodejs-clock
@crichardson
70. Socket.io - client side
<html>
<body>
The event is <span data-bind="text: ticker"></span>
<script src="/socket.io/socket.io.js"></script>
<script src="/knockout-2.0.0.js"></script>
<script src="/clock.js"></script>
</body>
</html>
Bind to model
Connect to
socket.io
clock.js
var socket = io.connect(location.hostname);
function ClockModel() {
self.ticker = ko.observable(1);
socket.on('tick', function (data) {
self.ticker(data);
});
};
ko.applyBindings(new ClockModel());
Subscribe
to tick event
Update
model
@crichardson
71. NodeJS is also great for writing
backend micro-services
“Network elements”
Simply ‘route, filter and transform packets’
Have minimal business logic
@crichardson
72. NodeJS-powered home security
FTP Server
Log file
FTP Server
Upload directory
Upload2S3
S3
SQS Queue
UploadQueue
Processor
DynamoDB
@crichardson
73. Summary
JavaScript is a very flawed language
The asynchronous model is often unnecessary; very
constraining; and adds complexity
BUT despite those problems
Today, NodeJS is remarkably useful for building networkfocussed components
@crichardson