Build FAST with parallel_calabash by Rajdeep Varma

1. Build FAST with
parallel_calabash
http://earthlymission.com/wp-content/uploads/2015/01/ants-
sewing_2160863k.jpg

2. About Myself
• Rajdeep Varma
• 5 years in Quality Engineering. I work at
ThoughtWorks
• Passionate about test automation and agile
practices
• Tested a number of web apps and recently
started testing mobile apps

3. I will talk about
• Our Problem: Long running automation
• Our Solution: parallel execution of tests by
multiprocessing on same machine.

4. Typical CI Pipeline Of Mobile Project
Git Push
Compile
+
Unit Test
Build App UI Tests Distribute
“Fast Feedback”

6. Stats
• 5 git repo
• Avg. 30 commits a day
• Approx. 100 Automated UI tests.
• “GitPush” TO “TestedOK” > 1 Hrs

7. With slow tests
Git Push
UI Tests
Deploy to
next
stage
Umm…slow tests—Chai Break—damn! test failed :(
Fixed!
<pain>its so
slow<pain/>
— Pray —

8. LOL

9. :(

10. Patience Is Not Always A Virtue

11. Choose One
a) Lots of automated tests for good coverage
b) Faster automation run
c) BOTH!

12. How?
http://sharpreflections.com/wp-content/uploads/2012/06/multi_core_cpu.png

13. Our tool-set
• MobileWeb: Cucumber + waitr-webdriver + parallel_tests
• AndroidApp: Calabash

14. Options
• Multiple parallel builds in CI
• Proprietary cloud based tools
• Create a tool that matches our need
http://images.clipartpanda.com/choice-clipart-fork-choice-hi.png

15. parallel_calabash

16. Demo

19. Single Device Paralle
l
Time
CPU
Usage

20. How it works?
<feature_grouper>
<feature_grouper>
1_digit_addition.feature
2_digit_addition.feature
3_digit_addition.feature
4_digit_addition.feature
5_digit_addition.feature
6_digit_addition.feature
7_digit_addition.feature
8_digit_addition.feature
Detect
Devices
Feature
Grouper
1_digit_addition
2_digit_addition
3_digit_addition
4_digit_addition
5_digit_addition
6_digit_addition
7_digit_addition
8_digit_addition

21. How it works?
<process spawner>
1_digit_addition
2_digit_addition
3_digit_addition
4_digit_addition
5_digit_addition
6_digit_addition
7_digit_addition
8_digit_addition
ENV[ADB_DEVICE_ARG] = emulator-
5554
ENV[TEST_PROCESS_NUMBER] = 1
Proces
s
Spawn
er
ENV[ADB_DEVICE_ARG] = emulator-
5556
ENV[TEST_PROCESS_NUMBER] = 2
ENV[ADB_DEVICE_ARG] = emulator-
5558
ENV[TEST_PROCESS_NUMBER] = 3
ENV[ADB_DEVICE_ARG] = emulator-
5560
ENV[TEST_PROCESS_NUMBER] = 4
S
U
M
M
A
R
I
Z
E
R
E
S
U
L
T
stdou
t

22. Reporting
config/cucumber.yml
default: --format html --out my_report_<%=ENV['DEVICE_INFO']%>.html
Reports from 3 devices:
my_report_GT_N7100.html
my_report_Samsung_Galaxy_Note_2___4_3___API_18___720x1280_1.html
my_report_Samsung_Galaxy_Note_2___4_3___API_18___720x1280_2.html

23. Optimization in test
distribution
• Issue with distribution based on cucumber
“featue” files
• Distribution based on “Scenario” instead of
features
• --group-by-scenarios

24. Advantages of fast tests
• Fast Feedback
• Faster development cycle for BDD environment
• Modern machines have multiple cores and
parallelization can make Maximum use of
Hardware
• Faster tests makes developers & Testers happy

25. what did we achieve?
• Speed
• Scalability
• Reporting
• Process specific action using
ENV[TEST_PROCESS_NUMBER]

27. Choose One
a) Lots of automated tests for good coverage
b) Faster automation run
c) BOTH!

28. TIME

29. https://github.com/rajdeepv/parallel_calabash
https://rubygems.org/gems/parallel_calabash
parallel_calabash

30. References:
• https://github.com/grosser/parallel_tests
• http://martinfowler.com/
• http://continuousdelivery.com/
• http://watirmelon.com/

31. ?