Everyone's had to endure the "last mile" of developing an application, but what happens if you consider those tasks from day 1? This talk centres around an application we released at Sky Bet earlier this year, the approaches we took, and how we benefited.

1. The Last Mile
Disclaimer: I’m only using Windows for PowerPoint. Please don’t judge.
@StephenMelrose
Lead Software Engineer - BSkyB
Photo by Ousseynou Cissé
https://www.flickr.com/photos/afuelcalledlove/

2. Are Recruiting!
Developers, DevOps, Scala/Hadoop,
Node.js Super Heroes, Principle Test Engineers,
an Engineering Manager, and more!
victoria.howling@bskyb.com
For more info, please see/contact:

3. SPIN Connector

4. Context

9. Image source: http://blog.burningman.com/wp-content/uploads/2012/09/blog4.jpg

10. (a.k.a. SPIN)

11. Image source: http://www.psdgraphics.com/file/adhesive-plaster.jpg

12. Image source: http://patriciaknight.files.wordpress.com/2011/08/change-architect-sign1.jpg

14. Image source: http://3.bp.blogspot.com/-Kdzzg4pimz0/Udk6bE5tDvI/AAAAAAAAFes/MxIGspcwu3M/s1600/photo.JPG

16. Testing
Performance
Extendibility
Supportability

17. Logging

18. Request failed
Connection lost
Failed to perform action

20. Request failed
Connection lost
Failed to perform action

21. Request failed
Connection lost
Failed to perform action

22. Request failed
Connection lost
Failed to perform action

23. Useful
Informative

24. Failed to start-up controller for fixture
[gC6RFPZ0IW7Zv_vqFR78Aj6_05c]: Failed to get AMQP
details from SPIN Connect API: Failed to get stream
AMQP details: HTTP request failed: ETIMEDOUT - Request:
{"url":"http://<api-
domain>/UnifiedDataAPI/stream/IceHockey/gC6RFPZ0IW7Zv_v
qFR78Aj6_05c","options":{"headers":{"Accept-
Encoding":"gzip","X-Auth-
Token":"XXXXXX"},"timeout":30000,"attempt":2,"uri":"htt
p://<api-
domain>/UnifiedDataAPI/stream/IceHockey/gC6RFPZ0IW7Zv_v
qFR78Aj6_05c"}}

25. Failed to start-up controller for fixture
[gC6RFPZ0IW7Zv_vqFR78Aj6_05c]: Failed to get AMQP
details from SPIN Connect API: Failed to get stream
AMQP details: HTTP request failed: ETIMEDOUT - Request:
{"url":"http://<api-
domain>/UnifiedDataAPI/stream/IceHockey/gC6RFPZ0IW7Zv_v
qFR78Aj6_05c","options":{"headers":{"Accept-
Encoding":"gzip","X-Auth-
Token":"XXXXXX"},"timeout":30000,"attempt":2,"uri":"htt
p://<api-
domain>/UnifiedDataAPI/stream/IceHockey/gC6RFPZ0IW7Zv_v
qFR78Aj6_05c"}}

26. Failed to start-up controller for fixture
[gC6RFPZ0IW7Zv_vqFR78Aj6_05c]: Failed to get AMQP
details from SPIN Connect API: Failed to get stream
AMQP details: HTTP request failed: ETIMEDOUT - Request:
{"url":"http://<api-
domain>/UnifiedDataAPI/stream/IceHockey/gC6RFPZ0IW7Zv_v
qFR78Aj6_05c","options":{"headers":{"Accept-
Encoding":"gzip","X-Auth-
Token":"XXXXXX"},"timeout":30000,"attempt":2,"uri":"htt
p://<api-
domain>/UnifiedDataAPI/stream/IceHockey/gC6RFPZ0IW7Zv_v
qFR78Aj6_05c"}}

27. Failed to start-up controller for fixture
[gC6RFPZ0IW7Zv_vqFR78Aj6_05c]: Failed to get AMQP
details from SPIN Connect API: Failed to get stream
AMQP details: HTTP request failed: ETIMEDOUT - Request:
{"url":"http://<api-
domain>/UnifiedDataAPI/stream/IceHockey/gC6RFPZ0IW7Zv_v
qFR78Aj6_05c","options":{"headers":{"Accept-
Encoding":"gzip","X-Auth-
Token":"XXXXXX"},"timeout":30000,"attempt":2,"uri":"htt
p://<api-
domain>/UnifiedDataAPI/stream/IceHockey/gC6RFPZ0IW7Zv_v
qFR78Aj6_05c"}}

28. Failed to start-up controller for fixture
[gC6RFPZ0IW7Zv_vqFR78Aj6_05c]: Failed to get AMQP
details from SPIN Connect API: Failed to get stream
AMQP details: HTTP request failed: ETIMEDOUT - Request:
{"url":"http://<api-
domain>/UnifiedDataAPI/stream/IceHockey/gC6RFPZ0IW7Zv_v
qFR78Aj6_05c","options":{"headers":{"Accept-
Encoding":"gzip","X-Auth-
Token":"XXXXXX"},"timeout":30000,"attempt":2,"uri":"htt
p://<api-
domain>/UnifiedDataAPI/stream/IceHockey/gC6RFPZ0IW7Zv_v
qFR78Aj6_05c"}}

29. Log Tail Video

30. Nothing

33. The Syslog Protocol (RFC 5424)
DEBUG Verbose and raw debug information. Use in development only.
e.g. request/response bodies
INFO 95% of our logging. Stuff we want to monitor, e.g. HTTP request
complete, took X seconds, performed Y actions
NOTICE Similar to INFO, but significant events.
Also, errors that CAN happen and we HAVE HANDLED
WARN Errors that SHOULDN’T happen, but we HAVE HANDLED
ERROR Errors that SHOULDN’T happen and we CAN’T HANDLE
CRITICAL Fatal errors that cause the application to crash

37. Graphs

38. Node.js Log
Processor
Graphitestatsd

41. Monitoring

44. Load Testing

45. SPIN’s API OpenBetSPIN Connector
SPIN Recorder

46. SPIN’s API OpenBetSPIN Connector
Fake SPIN API
Our Entire
Frontend Stack

47. Released to Live

48. Information exposure is paramount
Know what your app is doing
Automate the monitoring
Make sure it can do it

49. Thank you
Any questions?
@StephenMelrose
http://stephenmelrose.co.uk/