1. Massively
Continuous Integration
From 3 days to 30 minutes

2. •Software as a Service
•Project Management
•Collaborative Work Management

3. How we use Selenium
•1,100 Selenium 1 tests
•800 Selenium 2 tests
•40,000 hours per month
•Release Acceptance + CI

4. Then
Scripted scenarios
Slow, brittle
Monthly Releases
3-5 days acceptance
Manual administration

5. Now
Tight unit-like UI tests
Fast, Robust
Daily releases
30-45 minutes
Sign-off and go

6. “True” Continuous Integration
•How often do I integrate?
•What tests tell me I’m integrated?
•What must I know to release?

7. Pipeline Priorities
•Performance (Cycle Time)
•Scalability
•Visibility
•Accountability

8. ATTASK’S PIPE
BUILD -- DEPLOY -- TEST -- DESTROY

9. Commit
ATTASK’S PIPE
BUILD -- DEPLOY -- TEST -- DESTROY

10. Commit
Build Installer
Build Tests
ATTASK’S PIPE
BUILD -- DEPLOY -- TEST -- DESTROY

11. Selenium Grid
Grid + AtTask
Commit
Build Installer
Build Tests
ATTASK’S PIPE
BUILD -- DEPLOY -- TEST -- DESTROY

12. Selenium Grid
Grid + AtTask
Commit Install
Build Installer
Build Tests
ATTASK’S PIPE
BUILD -- DEPLOY -- TEST -- DESTROY

13. Selenium Grid
Grid + AtTask
Commit Install
Build Installer
Build Tests
Unit
Integration
1/4 3/4
2/4 4/4
ATTASK’S PIPE
BUILD -- DEPLOY -- TEST -- DESTROY

14. Selenium Grid Selenium IE
Grid + AtTask
Commit Install Selenium FF
Build Installer
Build Tests Other tests
Unit
Integration
1/4 3/4
2/4 4/4
ATTASK’S PIPE
BUILD -- DEPLOY -- TEST -- DESTROY

15. Selenium Grid Selenium IE
Grid + AtTask
Commit Install Selenium FF Tear Down
Build Installer
Build Tests Other tests
Unit
Integration
1/4 3/4
2/4 4/4
ATTASK’S PIPE
BUILD -- DEPLOY -- TEST -- DESTROY

16. AIRPORT SECURITY
NOBODY LIKES TO WAIT IN LINE

17. Selenium Grid Selenium IE
Grid + AtTask
Commit Install Selenium FF Tear Down
Build Installer
Build Tests Other tests
Unit
Integration
1/4 3/4
2/4 4/4
SERIAL VS PARALLEL

18. Selenium Grid Selenium IE
Grid + AtTask
Commit Install Selenium FF Tear Down
Build Installer
Build Tests Other tests
Unit
Integration
1/4 3/4
2/4 4/4
SERIAL VS PARALLEL

19. CYCLE TIME

20. CYCLE TIME

21. CYCLE TIME

22. JENKINS, EC2, AND ATTASK
A MATCH MADE IN HEAVEN

23. JENKINS, EC2, AND ATTASK
A MATCH MADE IN HEAVEN

24. JENKINS DYNAMIC SLAVE ALLOCATION

25. JENKINS DYNAMIC SLAVE ALLOCATION

26. SLAVE
JENKINS DYNAMIC SLAVE ALLOCATION

27. SLAVE
SLAVE
SLAVE
SLAVE
JENKINS DYNAMIC SLAVE ALLOCATION

28. JENKINS DYNAMIC SLAVE ALLOCATION

29. From 3 days to 30 minutes
with Selenium 2

30. SELENIUM 1 (1800 TESTS)
TESTING WITH SELENIUM 1

31. SELENIUM 1 (1800 TESTS)
4 HOURS
SLOW
TESTING WITH SELENIUM 1

32. TRANSITIONING TO SELENIUM 2

33. TRANSITIONING TO SELENIUM 2

34. TRANSITIONING TO SELENIUM 2

35. SELENIUM 1 (1800 TESTS)
4 HOURS
TRANSITIONING TO SELENIUM 2

36. SELENIUM 1 (1800 TESTS)
4 HOURS
SELENIUM 2 (750 TESTS)
30 MINUTES
WHERE DID THE
TESTS GO?
TRANSITIONING TO SELENIUM 2

37. TRANSITIONING TO SELENIUM 2

38. 2.1 MINUTES
TRANSITIONING TO SELENIUM 2

39. 2.1 MINUTES 34 SECONDS
TRANSITIONING TO SELENIUM 2

40. 2.1 MINUTES 34 SECONDS
WITH GRID...
TRANSITIONING TO SELENIUM 2

41. 2.1 MINUTES 34 SECONDS
WITH GRID...
8 SECONDS
TRANSITIONING TO SELENIUM 2

42. 2.1 MINUTES 34 SECONDS
WITH GRID...
8 SECONDS 2.4 SECONDS
TRANSITIONING TO SELENIUM 2

43. Scalability
•To the cloud with Amazon Cloud Formations
•Dynamic Selenium Grids
•Dynamic AtTask Environments

44. JSON
PARTLY CLOUDY WITH A CHANCE OF RAIN

45. JSON
PARTLY CLOUDY WITH A CHANCE OF RAIN

46. Scalability
Selenium Grid Selenium IE
Grid + AtTask
Commit Install Selenium FF Tear Down
Build Installer
Build Tests Other tests
Unit
Integration
1/4 3/4
2/4 4/4

47. Scalability
Selenium Grid
Grid + AtTask
Commit
Build Installer
Build Tests

48. SLAVE
IT’S RAINING GRIDS

49. SLAVE
IT’S RAINING GRIDS

50. SELENIUM GRID
SLAVE
SELENIUM GRID
IT’S RAINING GRIDS

51. SELENIUM GRID
SLAVE
SELENIUM GRID
IT’S RAINING GRIDS

52. SELENIUM GRID
SLAVE
SELENIUM GRID
IT’S RAINING GRIDS

53. SLAVE
SELENIUM GRID
SLAVE
SLAVE
SELENIUM GRID
SLAVE SLAVE
IT’S RAINING GRIDS

54. Visibility
•Selenium results in Jenkins
•Description plugin
•Theres my commit!
•View plugin
•All the jobs on one line, oh the humanity!

55. Visibility
•Selenium results in Jenkins
•Description plugin
•Theres my commit!
•View plugin
•All the jobs on one line, oh the humanity!

56. Visibility
•Selenium results in Jenkins
•Description plugin
•Theres my commit!
•View plugin
•All the jobs on one line, oh the humanity!

57. Accountability
•Claim / Blame test results using AtTask

58. Accountability
•Claim / Blame test results using AtTask

59. Accountability
•Claim / Blame test results using AtTask

60. Gotchas
•Watch your integration points
•Get the right skill set
•Align process w/ tools
•Continuous Investment

61. What does it cost?
•3 FTE
•Two human, one machine(s)
•Keeping costs down
•Efficient machine sizes
•Rock-solid tear down
•Elasticity

62. Return on investment
•5 days (40 hours) to certify one release...
•10 engineers
•$40 per hour
•$20,000
•30 minutes to certify one release...
•10 engineers
•$40 per hour
•$250

63. To release 12 times in a year...
$240,000 TO CERTIFY $3000 TO TEST
$160,000 FOR TWO FTE
$77,000 FOR CI
To release 24 times in a year...
$480,000 TO CERTIFY $6000 TO TEST
$160,000 FOR TWO FTE
$77,000 FOR CI
To release 100 times in a year...
$2,000,000 TO CERTIFY $25000 TO TEST
$160,000 FOR TWO FTE
$77,000 FOR CI

64. 100 deployments annually
$2,000,000

65. 100 deployments annually
$265,000 $2,000,000

66. 100 deployments annually
$265,000

67. Where to go from here
•Continuous Delivery
•Thoughtworks
•Application Lifecycle Management
•Infrastructure CI
•OpenStack

68. Tools Manifest
•AtTask contributions...
•EC2 - wiki.jenkins-ci.org/display/JENKINS/Amazon+EC2+Plugin
•CloudFormation - wiki.jenkins-ci.org/display/JENKINS/AWS+Cloudformation+Plugin
•Pipeline View - github.com/JoelJ/PipelineDashboard
•Description Setter - github.com/JoelJ/DescriptionSetter
•GIT - /wiki.jenkins-ci.org/display/JENKINS/Git+Plugin
•AtTask for Jenkins - Coming soon...

69. Q&A