Various factors to consider when trying to adopt a defensive programming mindset, methodology and process. This is especially useful for teams working with node.js.
2. INTRODUCTION
Ruben Tan Long Zheng 陈龙正
VP of Engineering, OnApp CDN KL
Lead Engineer, 40 Square Sdn Bhd
Javascript > 5 years
@roguejs
Organizer of Nodehack KL
Wednesday, May 29, 13
6. NEVER ASSUME!
Never assume a dependency is reliable!
var db = require(‘database’);
db.open();
db.write(‘foo bar’, function (err, data) {
// ... do something ...
});
Wednesday, May 29, 13
7. NEVER ASSUME!
var db = require(‘database’);
db.open();
db.write(‘foo bar’, function (err, data) {
// ... do something ...
});
What if this failed?
will write() throw an error? will open() throw
an exception?
Wednesday, May 29, 13
8. NEVER ASSUME!
var db = require(‘database’);
db.open(function (err) {
db.write(‘mr-big’, bigData, function (err, data) {
// ... unrelated logic
db.close();
});
db.read(‘foo2’, function (err, data) {
// ... some work done
});
});
Accidents happen...
Wednesday, May 29, 13
9. NEVER ASSUME!
var db = require(‘database’);
db.open(function (err) {
db.write(‘mr-big’, bigData, function (err, data) {
// ... unrelated logic
db.close();
});
db.read(‘foo2’, function (err, data) {
// ... some work done
});
});
close() might affect read()
Wednesday, May 29, 13
13. DEPENDENCY AWARENESS
What can fail, WILL FAIL!
Never assume a dependency is reliable!
Contingency plans - failover, redundancy, fail-fast, etc
Pro-active monitoring
Load test, stress test, chaos monkey, etc
Remember, what can fail, WILL FAIL!
Wednesday, May 29, 13
17. EXECUTION ORDER
var mq = require(‘mq’);
mq.conn(...);
mq.on(‘ready’, function () {
mq.send(‘batman’);
mq.on(‘message’, function (msg) {
console.log(msg);
mq.close();
});
});
mq is never closed!
send() executes before on()
Wednesday, May 29, 13
18. DOIN’ IT RIGHT!
var mq = require(‘mq’);
mq.conn(...);
mq.on(‘ready’, function () {
mq.on(‘message’, function (msg) {
console.log(msg);
mq.close();
});
mq.send(‘batman’);
});
Swap places
Wednesday, May 29, 13
19. SANITIZATION & VALIDATION
function foodForKittens(num) {
return num * 10;
}
foodForKittens();
num is not validated, is undefined
this will fail!
Wednesday, May 29, 13
23. var db = require(‘database’);
var conn = db.open(...);
function writeToDb(conn, cb) {
conn.write(bigData, function (err, res) {
if (err) {
cb(err);
return;
}
cb(null, res);
});
});
writeToDb(conn, ghostCallback);
what if open() returned undefined?
this will throw an exception!
Wednesday, May 29, 13
24. var db = require(‘database’);
var conn = db.open(...);
function writeToDb(conn, cb) {
conn.write(bigData, function (err, res) {
if (err) {
cb(err);
return;
}
cb(null, res);
});
});
writeToDb(conn, ghostCallback);
What if ghostCallback is undefined?
These will fail too!
Wednesday, May 29, 13
25. DOIN’ IT RIGHT!
var db = require(‘database’);
var conn = db.open(...);
function writeToDb(conn, cb) {
if (typeof conn !== ‘object’) {
// ... handle error ...
}
if (typeof cb !== ‘function’) {
// ... handle error ...
}
conn.write(bigData, function (err, res) {
if (err) {
cb(err);
return;
}
cb(null, res);
});
});
writeToDb(conn, ghostCallback);
Validate your input,
especially when they
involve functions or
methods that you need to
invoke in your code.
These are not the time to
fail-fast!
Wednesday, May 29, 13
26. DON’T GO OVERBOARD...
Validate only necessary parameters
Method invocations (anObject.method())
Function invocations (aFunction())
Have a proper error/exception handling policy
Validate for correctness, not existence
Correctness: typeof a === ‘object’
Existence: a !== undefined
Wednesday, May 29, 13
27. SCOPE AWARENESS
Plagues most callback-based code
Bad practice leads to costly debugging waste
New JS programmers not aware of scoping
JS scoping is a simple but weird thing (to non-JS
programmers)
Wednesday, May 29, 13
28. SCOPE!!!
var a = ‘outside’;
if (true) {
var a = ‘inside’;
console.log(a);
}
console.log(a);
What is the output?
> node test.js
inside
inside
Wednesday, May 29, 13
29. SCOPE!!!
Non-JS programmers:
a inside the if block is “inside”
a outside the if block is “outside”
JS programmers:
they are both “inside”
JS scope by function
Wednesday, May 29, 13
30. SCOPE CHAINS!!!
var avar = 1;
(function outer1() {
var avar = 2;
(function inner1() {
var avar = 3;
console.log(avar); // outputs 3
})();
(function inner2() {
console.log(avar); // outputs 2
})();
})();
(function outer2() {
(function inner3() {
console.log(avar); // outputs 1
})();
})();
inner1()
local - found!
inner2()
local - nope
outer1() - found!
inner3()
local - nope
outer2() - nope
global - found!
Wednesday, May 29, 13
31. HOISTING VARIABLES
function () {
for (var i = 0; i < 10; i++) {
for (var j = 0; j < 10; j++) {
// ... do something
}
}
}
function () {
var i, j; // now the scope is clear for i & j
for (i = 0; i < 10; i++) {
for (j = 0; j < 10; j++) {
// ... do something
}
}
}
Below is far clearer what individual variable scopes are:
Wednesday, May 29, 13
32. CONTROL FLOW
Node.js’ async nature makes it unintuitive to predict
control flow
I <3 async (github.com/caolan/async)
Control flow is ugly. Welcome to Javascript.
Async will save your life. Use it.
Wednesday, May 29, 13
33. CONTROL FLOW
var fs;
fs = require(‘fs’);
fs.readFile(‘./myfile.txt’, function (err, data) {
if (err) {
console.log(err);
return;
}
fs.writeFile(‘./myfile2.txt’, data, function (err) {
if (err) {
console.log(err);
return;
}
// ... do stuff ...
});
})
Wednesday, May 29, 13
36. CONTROL FLOW
var async, fs;
async = require(‘async’);
fs = require(‘fs’);
async.waterfall([
function step1(callback) {
fs.readFile(‘./myfile.txt’, callback);
},
function step2(data, callback) {
fs.writeFile(‘./myfile2.txt’, data, callback);
}
], function (err) {
// ... execute something in the end ...
});
Wednesday, May 29, 13
41. ERROR HANDLING
Standardize error handling in the app
Log to error DB
Output to error file
Output error to a stream
Use a logging library
Ask a leprechaun to manage it
etc
Wednesday, May 29, 13
42. LOGGING
How do you feel if your “log” looks like this?
> tail -f error.log
[12:01:55] ERROR - General error detected
[12:01:56] ERROR - General error detected
[12:01:57] ERROR - General error detected
[12:01:58] ERROR - General error detected
[12:01:59] ERROR - General error detected
[12:02:00] ERROR - General error detected
[12:02:01] ERROR - General error detected
Wednesday, May 29, 13
44. LOGGING
Logs are the first place you go to find out what
happened
Standardize a log location for each app
Make logs easy to access for developers
Wednesday, May 29, 13
45. DESIGN FOR FAILURE
Common steps to designing software:
1 - what should it do?
2 - how do I do it?
3 - how do I deploy?
4 - done
Wednesday, May 29, 13
46. DESIGN FOR FAILURE
Proper steps in defensive programming:
1 - what should it do?
2 - how many ways can it fail?
3 - how do I know when it fails?
4 - how do I prevent it from failing?
5 - write code accordingly
Wednesday, May 29, 13
47. DESIGN FOR FAILURE
Nothing is reliable
TCP can fail
Network can go down
Servers can run out of memory
Cows might fly through the sky crashing into your
datacenter and flooding the server rooms with milk
and destroying everything
Wednesday, May 29, 13
48. DESIGN FOR FAILURE
Designing for failure mindset & methodologies:
Identify SPOF (single point of failures)
Redundancy, failover, monitoring
Fail-fast, start-fast
Persist important data
Reliability & Consistency > Speed
Code is liability
Wednesday, May 29, 13