Workshop Test Automation & Continuous Integration & Delivery at Agile Bodensee, Oct 2014, Konstanz Germany

1. AGILE BODENSEE 2014
TEST AUTOMATION &
CONTINUOUS INTEGRATION WORKSHOP
Konstanz
01.10.2014
1

2. 2
WHO AM I?
Gridfusion Software Solutions
Contact:
Michael Palotas
Gerbiweg 2
8853 Lachen
SWITZERLAND
Tel.: +41 79 6690708
Email: michael.palotas@gridfusion.net
Founder / Principal Consultant
Gridfusion Software Solutions
Head of Productivity & Test Engineering, eBay

3. SETTING THE STAGE
Tell me about yourself J
What are your expectations for today?
3

4. POSSIBLE AGENDA TOPICS
Introduction
CI – what is it, why do we use it, what are we trying to achieve
Automation / Code Quality
Tools
Unit tests
Cobertura
Sonar
E2E Tests
Selenium
Cucumber
Amazon cloud
Pipeline
Integrating mobile
Vagrant - Infrastructure as code
Management / organizational aspect
4

5. Your setup
How do you build software?
5

6. WHAT IS SO SPECIAL ABOUT AGILE?
6

7. WHY CI/CD?
7

8. WHAT IS IMPORTANT IN AGILE?
8

9. Agile
=?
Release working software anytime
9

10. Traditional waterfall model / tools do not support
“build and deploy anytime”
10

11. WHAT IS CI / CD?
CI and CD
=
Automated Build?
Automated Tests?
Automated Quality?
Automated Deployment?
Automated Feedback?
11

12. WHY CI / CD
Deliver value to the business more frequently
Better Quality
Early Bugs
Bug Prevention instead of late detection
Fast & frequent feedback
12

13. WHY CI / CD
Automated frequent builds
Automated frequent tests
Automated frequent code quality metrics
(Hopefully) Fewer bugs
Early feedback
Fast feedback
13

14. WITHOUT CI
Slow / long release cycles
Late testing
Waterfall (WaterScrum)
Bugs
Slow feedback
Complex integration
14

15. CORE PRINCIPLES
Every build could be a release
Everything should be automated
Stable and trustworthy automated tests
Build pipelines
15

16. RELEASING IN THE OLD WORLD
16
Coding
Deploy to
QA
QA
Deploy to
Production
Production
Smoke
Tests
Bug
Bashes

17. CI / CD - CORE WORKFLOW
17
Compile
Unit Test
Deploy to QA
Acceptance tests
Deploy to Production
Production Smoke Tests
Code Quality

18. THE MAIN TASKS
Automated build
Automated code quality
Automated testing
Automated deployment
18

19. Wakaleo.com
19

20. CAN YOU MEASURE AUTOMATED CODE
QUALITY? DOES THAT MAKE SENSE?
20

21. AUTOMATED CODE QUALITY?
Sonar gives you information on:
- Lines of code
- % of comments
- Duplications
- Complexity
- Rules compliance
- Unit test coverage
- Unit test success rate
- Unit test duration
- Hotspots
21

22. AUTOMATE EVERYTHING?
22

23. WHAT SHOULD YOU AUTOMATE?
23

24. WHAT ARE BARRIERS TO CI / CD?
24

25. WHAT IS CONTIUOUS INTEGRATION?
Continuous integration (CI) is the practice, in software engineering,
of merging all developer working copies with a shared mainline
several times a day. It was first named and proposed as part of
extreme programming (XP). Its main aim is to prevent integration
problems, referred to as "integration hell" in early descriptions of XP.
CI can be seen as an intensification of practices of periodic
integration advocated by earlier published methods of incremental
and iterative software development, such as the Booch method. CI
isn't universally accepted as an improvement over frequent
integration, so it is important to distinguish between the two as there
is disagreement about the virtues of each.
25

26. WHAT IS CONTINUOUS DELIVERY?
26
Continuous Delivery (CD) is a design practice used in software
development to automate and improve the process of software
delivery. Techniques such as automated testing, continuous
integration and continuous deployment allow software to be
developed to a high standard and easily packaged and
deployed to test environments, resulting in the ability to
rapidly, reliably and repeatedly push out enhancements and
bug fixes to customers at low risk and with minimal manual
overhead. The technique was one of the assumptions of
extreme programming but at an enterprise level has
developed into a discipline of its own, with job descriptions for
roles such as "buildmaster" calling for CD skills as mandatory.

27. THE MANAGEMENT / ORGANIZATIONAL ASPECT
What are the changes for developers and
testers?
What needs to be changed in the organization to
enable them to implement CI / CD?
What role has management in creating a devops
culture?
27

28. OUR TOOLS
Version Control System GIT
Build Tool MAVEN
Unit Test Framework JUNIT / TESTNG
End To End Test Framework SELENIUM
Build Server / Deployment JENKINS
28

29. VERSION CONTROL: GIT
Branching & Merging
Small and Fast
Distributed
Data Assurance
Staging Area
Free and Open Source
http://git-scm.com/about/
29

30. GIT: BRANCHING & MERGING
30
Git-scm.com

31. GIT: SMALL & FAST
31
Git-scm.com

32. GIT: THE REST
Distributed
Data Assurance
Staging Area
Free & Open Source
32

33. GIT
Distributed / local
Download: http://git-scm.com/
Initialize directory: git init
Status: git status
Add files and directories to git: git add file1 dir2
Commit: git commit –am “commit message”
33

34. SHARE YOUR CODE - GITHUB
Create repository on Github: https://github.com
Create remote: git remote add origin https://…
Push code to Github: git push origin master
Tag your code: git tag –a v0.1 –m “initial version”
Push tag to Github: git push origin v0.1
34

35. GITHUB
35

36. CONNECT GIT AND GITHUB
Silverpeas.org
36

37. MAVEN
37
http://maven.apache.org/guides/getting-started/maven-in-five-minutes.html
Apache Maven is a software project
management and comprehension tool. Based
on the concept of a project object model (POM),
Maven can manage a project's build, reporting
and documentation from a central piece of
information.

38. POM.XML
The pom.xml file is the core of a project's
configuration in Maven. It is a single
configuration file that contains the majority of
information required to build a project in just
the way you want.
38

39. POM.XML
39

40. MAVEN TARGETS
validate: validate the project is correct and all necessary information is available
compile: compile the source code of the project
test: test the compiled source code using a suitable unit testing framework. These tests should not
require the code be packaged or deployed
package: take the compiled code and package it in its distributable format, such as a JAR.
integration-test: process and deploy the package if necessary into an environment where integration
tests can be run
verify: run any checks to verify the package is valid and meets quality criteria
install: install the package into the local repository, for use as a dependency in other projects locally
deploy: done in an integration or release environment, copies the final package to the remote repository
for sharing with other developers and projects.
clean: cleans up artifacts created by prior builds
40

41. EXAMPLES
mvn clean
mvn compile
mvn test
41

42. OUR APPLICATION TODAY
42

43. APPLICATION STRUCTURE
It is a super complex application …
2 text fields
1 submit button
1 very complex calculation
43

44. OUR ENTRY PAGE
44

45. RESULT PAGE
45

46. CALCULATOR CLASS
46

47. CALCULATOR TESTS
47

48. RUN UNIT TESTS
1. Run in Eclipse
2. Run via maven
48

49. COBERTURA
mvn clean cobertura:cobertura –Dgroup=unit
49
TMFSydney2014/target/site/cobertura/index.html

50. SONAR
http://localhost:9000/dashboard/index/net.gridfusion.dobcalc:tmf2
50

51. END TO END TESTS (E2E)
Selenium
Selenium Grid
51

52. TEST AUTOMATION
Manual Tests
E2E
Tests
Integration Tests
Unit Tests

53. WHAT IS SELENIUM?
Selenium automates browsers
that’s it
…

54. E2E / UAT AUTOMATION WITH SELENIUM
54
CLIENT
SERVER
JSON Wire Protocol
BROWSER

55. SELENIUM 2 / WEBDRIVER
JSON WIRE
PROTOCOL
Client
Java
C#
Ruby
Python
Server
Server
Server
i.e. Selendroid, iOS-Driver

56. CLIENT
Is seen as „Selenium“ by the users
Generates HTTP requests which are received by the server
Is called by the test framework or the CI server
Supported languages: Java, C#, Python, Ruby, Perl, PHP,
JS

57. SERVER
Receives HTTP requests
Start and teardown of browser
Translates requests into browser specific commands
Communicates back to the client

58. SELENIUM GRID
Test 1 Test 2 Test
…
Test
4500
Test 3
Execution Time
Parallel Execution
Tes t Tes t Tes t Tes t
Tes t Tes t Tes t Tes t
Test Test Test
Execution Time
Test
Parallel Execution
Parallel Execution

59. SCALING – SELENIUM GRID
DEV
CI
….
SELENIUM GRID
HUB
IOS ANDROID
WINDOWS
LINUX
OSX

60. TEST INFRASTRUCTURE
AUT
API
DB
Browsers
Mobiles
CLIENT

61. A SIMPLE SELENIUM TEST
61

62. A SIMPLE SELENIUM TEST
62

63. SELENIUM – MORE ADVANCED TESTS
63

64. DATA PROVIDER
64

65. URL FACTORY
65

66. CONTINUOUS INTEGRATION - JENKINS
Download at http://jenkins-ci.org/
66

67. RECAP
WHAT DO WE EXPECT
THE CI SYSTEM TO DO?
67

68. 68

69. WHAT JENKINS DOES
Jenkins checks out the workspace from Github
Builds and runs tests locally according to POM
Runs maven targets according to POM
description
69

70. A SIMPLE BUILD JOB
70

71. A SIMPLE BUILD JOB
1. (Push to Github repository)
2. Let Jenkins pick up the change
3. Perform “mvn test –Dgroup=unit”
71

72. LET’S CREATE A BUILD / DEPLOYMENT PIPELINE
72

73. THE MASTER JOB
73
Unit Test
Deploy to QA
Acceptance tests
Deploy to Production
Production Smoke Tests
Code Coverage

74. UNIT TESTS
74

75. UNIT TESTS
75

76. UNIT TESTS
76

77. UNIT TESTS - RESULT
77

78. DEPLOY TO QA
78

79. DEPLOY TO QA
79

80. DEPLOY TO QA - RESULT
80

81. E2E TESTS
81

82. E2E TESTS
82

83. E2E TESTS - RESULT
83

84. E2E TESTS - RESULT
84

85. DEPLOY TO PRODUCTION
85

86. DEPLOY TO PRODUCTION
86

87. PRODUCTION SMOKE TESTS
87

88. LET’S GO MOBILE
88

89. INFRASTRUCTURE AS CODE
Consistent Infrastructure
Efficient Change Management
Simple to rebuild
89

90. TOOLS
90

91. VAGRANT EXAMPLE
91

92. BOOTSTRAP.SH
92

93. 93

94. THANK YOU!
94