Building applications for the IaaS Cloud is easy, right? "Sure, no problem - just lift and shift!" all the Cloud vendors shout in unison. However, the reality of building and deploying Cloud applications can often be different. This talk will introduce lessons learnt from the trenches during two years of designing and implementing cloud-based Java applications, which we have codified into our Cloud developer’s 'DHARMA' rules; Documented (just enough); Highly cohesive/loosely coupled (all the way down); Automated from code commit to cloud; Resource aware; Monitored thoroughly; and Antifragile.
We will look at these lessons from both a theoretic and practical perspective using a real-world case study from Instant Access Technologies (IAT) Ltd. IAT recently evolved their epoints.com(http://epoints.com/) customer loyalty platform from a monolithic Java application deployed into a data centre on a 'big bang' schedule, to a platform of loosely-coupled JVM-based components, all being continuously deployed into the AWS IaaS Cloud
7. Moving to the Cloud
• IAT chose Amazon Web Services (AWS) IaaS
• IaaS was great, but…
– Made a few mistakes
– Learnt a lot of lessons
• Bonus! Russ Miles view on PaaS
– bit.ly/1neXzaf
07/05/2014 @danielbryantuk
10. Common Cloud Problems
• Components
– Who does what now?
– Configuration issues
• Deployment topology
– Cloud networking
– “Is that DB local?”
07/05/2014 @danielbryantuk
11. Common Cloud Problems
• Unknown failure modes
– Bang!
– Now you see me…
– Can…you…hear…me?…
– A to B, but not B to A
07/05/2014 @danielbryantuk
12. Common Cloud Problems
• High Availability (HA)
– Self-inflicted wounds…
• Snowflake servers
• Monitoring / Diagnostics
07/05/2014 @danielbryantuk
13. Common Cloud Problems
• Not respecting the
underlying hardware
• Not testing in the Cloud
– Here be dragons!
07/05/2014 @danielbryantuk
14. We’ve created a “Cloud Developer’s DHARMA”
to act as a checklist when building Cloud apps
07/05/2014 @danielbryantuk
15. dharma
/ˈdɑˈmə,ˈdəˈmə/
noun
1. Signifies behaviors that are considered to be in
accord with order that makes life and universe
possible (Hinduism)
2. "cosmic law and order”, but is also applied to
the teachings of the Buddha (Buddhism)
07/05/2014 @danielbryantuk
16. Documented (just enough)
Highly cohesive/loosely coupled (all the way down)
Automated from commit to Cloud
Resource aware
Monitored thoroughly
Antifragile
07/05/2014 @danielbryantuk
17. Documented (just enough)
Highly cohesive/loosely coupled (all the way down)
Automated from commit to Cloud
Resource aware
Monitored thoroughly
Antifragile
07/05/2014 @danielbryantuk
26. Our Build Pipeline
• Component Build
– Compile
– Unit Tests (surefire)
– Integration Tests (failsafe)
• Deployment onto QA Cloud
– Python Scripts + Chef to provision
– Verify success using Python
07/05/2014 @danielbryantuk
27. Our Build Pipeline
• Acceptance Tests
– Cucumber and Selenium
– Work in progress…
• Performance Tests
– Jmeter
– Jenkins Jmeter performance plugin
• Live Deployment
– “Human-based conditional operation”
07/05/2014 @danielbryantuk
28. Automating QA
• Intra-component integration testing
– Spock is awesome (code.google.com/p/spock)
– Utilise embedded datastore/middleware
• Inter-component integration testing
– The hardest part of SOA…
• Consumer-based Testing
– Brandon Byars (bit.ly/1lmcoaD)
07/05/2014 @danielbryantuk
29. Infrastructure: Say No To Snowflakes!
• Automate all provisioning
– Chef, Puppet, SaltStack
– Bash, Python
– AWS API / CLI
• “Infrastructure as Code”
– Version control everything
07/05/2014 @danielbryantuk
30. Infrastructure: Say No To Snowflakes!
• Doing “Proper Development”
– Gareth Rushgrove at Craft Conf (bit.ly/1njuc49)
– Chef Conf (www.youtube.com/user/getchef)
• Local tooling/testing
– Vagrant (www.vagrantup.com)
– Docker (www.docker.io)
07/05/2014 @danielbryantuk
31. Documented (just enough)
Highly cohesive/loosely coupled (all the way down)
Automated from commit to Cloud
Resource aware
Monitored thoroughly
Antifragile
07/05/2014 @danielbryantuk
34. What you get…
07/05/2014 @danielbryantuk
Fact: 9 out of 10 cheetahs prefer the
taste of an Ops team over tinned food
35. Thou Shalt Know thy Cloud…
• AWS EBS 100 IOPS (by default)
– My Mac SSD does 49K IOPS
• 1000Mbps network max transfer ~125MB/s
– My Mac does 400+ MB/s Sequential Write to SSD
• “Noisy *virtual+ Neighbours”
Reference for Mac statistics: bit.ly/1ftJZH8
07/05/2014 @danielbryantuk
36. Thinking/Acting Operationally
• Cultivate “Mechanical Sympathy”
• Virtualisation
– Tech Target (bit.ly/1kDVqyG)
• Networking
– ‘Unix and Linux System Administration Handbook’
– aws.amazon.com/documentation
07/05/2014 @danielbryantuk
37. Thinking/Acting Operationally
• Learn Linux fundamentals
• Diagnostic skills
– top, netstat, vmstat, tcpdump
– Java utils: jps, jstat, jmap, jhat
– “DevOps Troubleshooting” by K. Rankin
• Maybe grow a beard…
07/05/2014 @danielbryantuk
38. Documented (just enough)
Highly cohesive/loosely coupled (all the way down)
Automated from commit to Cloud
Resource aware
Monitored thoroughly
Antifragile
07/05/2014 @danielbryantuk
46. Documented (just enough)
Highly cohesive/loosely coupled (all the way down)
Automated from commit to Cloud
Resource aware
Monitored thoroughly
Antifragile
07/05/2014 @danielbryantuk
47. Antifragile
• The opposite of fragile?
– Robust…
– Antifragile…
• Netflix are best-in-class
– bit.ly/1gs5n3q
• System must be robust first!
07/05/2014 @danielbryantuk
57. Documented (just enough)
Highly cohesive/loosely coupled (all the way down)
Automated from commit to Cloud
Resource aware
Monitored thoroughly
Antifragile
07/05/2014 @danielbryantuk
So, Cloud Apps are ‘done’ when…
58. Thanks For Listening
• Massive thanks to all the IAT team!
• Questions / comments?
– d.bryant@iatltd.com
– @danielbryantuk
• Join us at Devoxx UK!
– www.devoxx.co.uk
07/05/2014 @danielbryantuk
Notes de l'éditeur
Code doesn’t tell whole storycontext, containers, components and classes
James Gough’s “The benefits are more than just the tests”Mash Badar’s “TDD at Scale” (slidesha.re/19P7kzS)