Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2qGqydZ. Jamund Ferguson discusses tackling challenges with large-scale Node.js deployments, using async/await to handle errors, and strategies to embrace latest JavaScript technologies. Filmed at qconlondon.com. Jamund Ferguson is a JavaScript architect at PayPal. He loves to look at how following patterns consistently can prevent bugs in applications. He’s previously contributed to the ESLint and StandardJS open-source projects and has as of late become a fan of FlowType and TypeScript.

1. NODE.JS APPLICATIONS
BUILDING SCALABLE AND DEPENDABLE JAMUND FERGUSON

2. InfoQ.com: News & Community Site
Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
enterprise-nodejs-apps
• Over 1,000,000 software developers, architects and CTOs read the site world-
wide every month
• 250,000 senior developers subscribe to our weekly newsletter
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• 2 dedicated podcast channels: The InfoQ Podcast, with a focus on
Architecture and The Engineering Culture Podcast, with a focus on building
• 96 deep dives on innovative topics packed as downloadable emags and
minibooks
• Over 40 new content items per week

3. Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide
Presented at QCon London
www.qconlondon.com

4. The Mystery of the
Missing Stack Trace🕵

5. THE MYSTERY OF THE MISSING STACK TRACE
Error: Can't set headers after they are sent.
at ServerResponse.OutgoingMessage.setHeader (_http_outgoing.js:356:11)
at ServerResponse.header (/web/mynodeapp/node_modules/express/lib/response.js:767:10)
at ServerResponse.send (/web/mynodeapp/node_modules/express/lib/response.js:170:12)
at ServerResponse.res.send (/web/mynodeapp/node_modules/pplogger/index.js:225:18)
at done (/web/mynodeapp/node_modules/express/lib/response.js:1004:10)
at Stub.callback (/web/mynodeapp/node_modules/adaro/lib/engine.js:137:22)
at Stub.flush (/web/mynodeapp/pp/node_modules/dustjs-linkedin/lib/dust.js:513:10)
at Chunk.end (/web/mynodeapp/node_modules/dustjs-linkedin/lib/dust.js:612:15)
at /web/mynodeapp/node_modules/adaro/lib/patch/index.js:89:53
at /web/mynodeapp/node_modules/adaro/lib/reader/js.js:39:13
at /web/mynodeapp/node_modules/engine-munger/lib/munger.js:85:13
at /web/mynodeapp/node_modules/engine-munger/lib/cache.js:65:13
at /web/mynodeapp/node_modules/graceful-fs/graceful-fs.js:78:16
at /web/mynodeapp/node_modules/async-listener/glue.js:188:31
at FSReqWrap.readFileAfterClose [as oncomplete] (fs.js:380:3)

6. 😢

7. We had no idea why or where our apps
were failing

8. WHAT CAN WE DO?
▸ Compare a diff between last known working code
▸ Look through other logs for more information (nginx logs, access logs, etc)
▸ Look at system metrics (is there a memory leak or CPU spike?)
▸ Add console.log statements somewhere???
▸ Advanced debugging techniques (post-mortem debugging, heapdumps, etc.)
THE MYSTERY OF THE MISSING STACK TRACE

11. DECRYPT RETURNS A PROMISE
ADDUSER EXPECTS A STRING
LOG EXPECTS A SMALL OBJECT OR A STRING

12. 💣

13. THE MYSTERY OF THE MISSING STACK TRACE
LESSONS LEARNED
▸ We need better static analysis
▸ We need better debugging tools
▸ We need a consistent way to handle errors
▸ We need to better understand our logging & monitoring
💣

14. FLOWTYPE & ESLINT
STATIC ANALYSIS WITH

15. TYPE CHECKING COULD HAVE CAUGHT
THAT BUG WITH 2-LINES OF CODE
STATIC ANALYSIS

16. STATIC ANALYSIS
PREVENTING BUGS WITH TYPES

17. STATIC ANALYSIS
PREVENTING BUGS WITH TYPES

18. STATIC ANALYSIS
FLOW WON’T LET THAT SLIDE

19. STATIC ANALYSIS
WHY ADD TYPES TO YOUR JS?
▸ Prevents large % of bugs
▸ Helps surface architectural problems
▸ Both Flow and TypeScript are well maintained, high quality tools
▸ Both syntaxes are light-weight and easy to use
▸ Both allow for gradual adoption

21. 🙋 🍩🚫

22. Type systems have a lot in common with linters

23. Each JS File Parser (Acorn) Abstract Syntax Tree (AST) Rules
Warning
Success
Errors
Architecture of JavaScript Linter (ESLint)

24. Linters can only think about one file at a time

25. All Your JS FilesFlow
Types & Relationships Graph
Architecture of JavaScript Type System (Flow)
Single JS File
Warning
Success
Errors
Flow
Graph

26. COULD A LINTER HAVE HELPED
US WITH OUR MYSTERY BUG?

27. STATIC ANALYSIS

28. STATIC ANALYSIS

29. STATIC ANALYSIS
STATIC ANALYZERS AND FORMATTERS ARE PRETTY COOL
FlowType ESLint Prettier

30. Unfortunately, we still get bugs from time to time

31. INSPECTOR MODULE
DEBUGGING USING THE

32. THE BUILT-IN INSPECTOR MODULE

34. THE BUILT-IN INSPECTOR MODULE
SETTING UP A DEBUG MODE
TURN IT ON
TURN IT OFF

35. THE BUILT-IN INSPECTOR MODULE

36. THE BUILT-IN INSPECTOR MODULE
Set Breakpoint

37. THE BUILT-IN INSPECTOR MODULE
PAUSE ON UNCAUGHT EXCEPTIONS

38. THE BUILT-IN INSPECTOR MODULE
PAUSE ON CAUGHT EXCEPTIONS

39. THE BUILT-IN INSPECTOR MODULE

40. THE BUILT-IN INSPECTOR MODULE

41. THE BUILT-IN INSPECTOR MODULE

42. THE BUILT-IN INSPECTOR MODULE

43. Don’t try this in production
STOP

44. THE BUILT-IN INSPECTOR MODULE

45. THE BUILT-IN INSPECTOR MODULE

46. THE BUILT-IN INSPECTOR MODULE
https://chromedevtools.github.io/devtools-protocol/

47. THE BUILT-IN INSPECTOR MODULE
Default Node Error
Inspector Based Error

48. DEBUGGING NODE.JS APPS
FIND A DEBUGGING APPROACH THAT
WORKS FOR YOU AND YOUR TEAM

49. ASYNC/AWAIT
ERROR HANDLING USING

50. ERROR HANDLING WITH ASYNC/AWAIT

51. Errors thrown inside async
functions get converted into
rejected Promises
💡

52. ERROR HANDLING WITH ASYNC/AWAIT

53. Async Middleware Pattern

54. ERROR HANDLING WITH ASYNC/AWAIT
T H I S I S P R E T T Y N I C E

55. ERROR HANDLING WITH ASYNC/AWAIT
B U T W E D O N ’ T A C T U A L LY C AT C H I T 💣
E R R O R S W I L L B U B B L E U P

56. ERROR HANDLING WITH ASYNC/AWAIT
💥

57. ERROR HANDLING WITH ASYNC/AWAIT
PASS IN YOUR ASYNC MIDDLEWARE
RETURN A STANDARD MIDDLEWARE FUNCTION
CATCH ANY ERRORS
PASS THOSE TO THE EXPRESS ERROR HANDLER
EXECUTE THE ASYNC MIDDLEWARE
APPLY AS NEEDED

58. ERROR HANDLING WITH ASYNC/AWAIT
💥

59. Make it easy for your engineers
to do the right thing

60. ERROR HANDLING WITH ASYNC/AWAIT

61. Custom Error Classes

62. CUSTOM ERRORS

63. CUSTOM ERRORS

64. CUSTOM ERRORS
1.
2.
3.

65. CUSTOM ERRORS

66. CUSTOM ERRORS

67. CUSTOM ERRORS

68. CUSTOM ERRORS
SUMMARY
▸ Don’t use object literals or strings for errors (missing stack trace)
▸ Use the Error built-in object
▸ Subclass Error to add statusCodes or to convert error codes into user-
friendly error messages for localization, etc
▸ We basically have one error class per micro-service to handle parsing the
errors out of the response….

69. The Mystery of the
Client-Side Errors)

70. THE MYSTERY OF THE CLIENT-SIDE ERRORS
CLIENT-SIDE MONITORING
BUTTON DOESN’T WORK
REAL ISSUE USUALLY IN DEV TOOLS

71. THE MYSTERY OF THE CLIENT-SIDE ERRORS
CLIENT-SIDE MONITORING
window.onerror = function (msg, url, line, col, error) {
// 1. clean up the data
// 2. log to server w/AJAX or sendBeacon() API
}

72. THE MYSTERY OF THE CLIENT-SIDE ERRORS
WE NOTICED A SPIKE DURING DEPLOY

73. THE MYSTERY OF THE CLIENT-SIDE ERRORS
WE CONGRATULATED OURSELVES…THEN ACTUALLY LOOKED INTO THE BUG
🏆 😮

74. THE MYSTERY OF THE CLIENT-SIDE ERRORS

75. THE MYSTERY OF THE CLIENT-SIDE ERRORS
WE HAVE A LOT OF SERVERS

76. THE MYSTERY OF THE CLIENT-SIDE ERRORS
WE HAVE A LOT OF SERVERS

77. THE MYSTERY OF THE CLIENT-SIDE ERRORS
WE HAVE A LOT OF SERVERS

78. THE MYSTERY OF THE CLIENT-SIDE ERRORS
WE HAVE A LOT OF SERVERS

79. THE MYSTERY OF THE CLIENT-SIDE ERRORS
WE HAVE A LOT OF SERVERS

80. THE MYSTERY OF THE CLIENT-SIDE ERRORS
UI Code
Server code
A
B

81. THE MYSTERY OF THE CLIENT-SIDE ERRORS
ROLLING BACK MADE THINGS WORSE

82. THE MYSTERY OF THE CLIENT-SIDE ERRORS
LESSONS LEARNED
▸ UI is a huge monitoring blind-spot
▸ Be aware of how the deploy process affects users
▸ Try to make your code changes backwards compatible
▸ Consider separating UI and server deploys

83. BUILDING SCALABLE AND DEPENDABLE NODE.JS APPLICATIONS
▸Use static analysis (including types) to catch bugs early
▸Have a plan for debugging apps in production
▸Adopt a consistent approach to error handling
▸Know how to access all of your logs
▸Don’t forget to monitor client-side errors

84. THE END

85. Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
enterprise-nodejs-apps