Everyone loves working on a greenfield project. You’re starting fresh and nothing holds you back. Unfortunately, for most testers, this is a rare occurrence. Chances are you will work on legacy applications. Because these often have no automated tests, developers are afraid to make bold changes. More testers than developers can be assigned to these projects. Changing one line of code may require multiple days of manual testing. Eventually work grinds to a halt. Sound familiar? Emanuil Slavov explains how to deal with this sticky situation without losing your mind. Start small and work outside in. Learn how to combine the best practices of automated acceptance tests, unit tests, static code analysis, continuous integration, and architecture for testability. Discover how to make your automated tests more reliable, easy to support, and a breeze to extend. Emanuil’s presentation is inspired by his real-life experience—working on legacy projects for more than five years.

1. AUTOMATION TESTING
LEGACY APPLICATIONS
@EmanuilSlavov
OF
emo@komfo.com
emanuilslavov.com

2. THE UGLYTHE GOOD THE BAD

3. GREENFIELD vs. BROWNFIELD

4. GREENFIELD PROJECT

5. BROWNFIELD PROJECT
BROWNFIELD PROJECT

6. WHY INVEST IN LEGACY
SYSTEM?

7. TEAM HAPPINESS

8. LOW TEAM MORALE

9. Regression Bugs
Fragile Software
Slow Feedback
Stupid Errors
Repetitive Work
Slow Progress

10. Quality software is team effort.
It needs system thinking.

11. SHIFT LEFT

12. THE THREE PILLARS
OF AUTOMATED TESTING

13. Static Code Analysis
Unit Tests
Black Box Tests

14. WHAT TO DO ABOUT IT

15. Start with basic acceptance tests

16. Functionality that makes money
Must have functionality - compliance, security
Repeating Manual Tests - Save Time
Pareto Principle - 80/20

17. Your Tests
Fast
Reliable
Maintainable

18. Do not test through the UI.
(if possible)

20. result = RestClient.post(
REGISTER_URL,
user_details.to_json,
{:content_type => :json}
)

21. 800 test x 10 seconds = 2h 13min
This saved us:

22. Set test data via API or DB.

23. Limit external dependencies calls.
(talked about this last year)

24. Need to Call
External System
Automation
Test?
Talk to the
real system
No
Fake the
response
Yes

25. Test should create the data they need.

26. Scenario: Client admin should not be able to
view master’s agencies
Given а master user
And master creates agency
And a client admin
When client admin views master's agency
Then client admin should get an error

27. Poll for results from API/DB operations.

28. sleeping(1).seconds.between_tries.failing_after(10).tries do
result = some_operation
raise 'No Data' if result['data'] == []
end

29. Run a test 20 times consecutively.
Commit only if the test does not fail.

30. for i in {1..20}; do your_test; done

31. Automatically rerun failed tests.

32. Same static code checks for tests code as
for production code.

33. CODE CHANGES

34. First Order of Business:
Remove Unused Code

35. Remove the commented code
Remove the code you know it’s not used
Instrument the code to check what’s really used

36. Second Order of Business:
Stop The Rot

38. CONTINUOUS
INTEGRATION

40. Run on every commit
Max execution time: 5 min.
Hook one by one all the checks
Run longer tests periodically

41. Developers need to receive feedback
about their new code within 5 minutes.

42. CHECKS ON COMMIT

43. The PHP Case

44. LINTER

46. php -l api/models/mobile_push_model.php
PHP Parse error: api/models/mobile_push_model.php on line 61
Errors parsing api/models/mobile_push_model.php

47. HHVM

49. UnknownObjectMethod in file:
api/models/mobile_push_model.php, line: 55, problem entry:
$pusher->reallyUnsubscribeDevice
($params['user_id'], $params['device_id'], $actions)

50. STATIC CODE QUALITY

51. CYCLOMATIC COMPLEXITY
function testPrint() {
echo('Hello World');
}
Complexity: 1
function testPrint($parameter) {
if($parameter) {
echo('Hello World');
}
}
Complexity: 2

52. Method complexity
should be less than 10.

53. Complexity 82
Complexity 10
Constantly refactor
to decrease complexity

54. Method size should be less than
100 lines (ideally less than 50).

55. Improve the code - then lower the
threshold on commit check.
Then repeat.

56. FIGHT LEGACY CODE
WRITE UNIT TESTS

57. Written by Developers
Fast, Independent
Test Technical Aspects
Cooperation between QA & Developers

58. [Demo]

59. 100% test coverage is not sufficient!

61. Missing assertions
Missing handling unlikely conditions
Don’t aim for specific coverage number

62. SECURITY TESTS

63. SQL Injection Detection
(PHP and ADOdb)
$dbConn->GetRow(“SELECT * FROM users WHERE id = $user_id”)
$dbConn->GetRow(“SELECT * FROM users WHERE id = ?”, array(‘$user_id’))

64. Those errors can be caught with code
analysis.
No need to run slow whole application
security scan.

65. There was no such tool.
So we developed one.

66. github.com/emanuil/php-reaper

68. Scans PHP ADOdb code for SQL injections
Command line
Suitable for CI, on-commit tests
Plans to expand, pull requests welcomed

69. MONITORING

70. Your second line of defence.

71. Show a lot with TV and Raspberry Pi.

73. Live Graphs + Deploys

74. Live Graphs + Deploys

75. CONCLUSION

76. Аutomatе the most important functionalities
Continuously improve static code quality
Write unit tests for changed/new code
Expand checks on commit
Enable monitoring

77. RECOMMENDED READING