Smart Gamma use case of implementation Continuous Integration/ Continuous Delivery for Laravel web app, tested by phpunit and Behat, build automation with Jenkins, blue-green deploy on AWS Beanstalk

1. Continuous Integration/
Continuous Delivery веб-приложения
Evgeniy Kuzmin, CTO Smart Gamma

2. Agile attributes
- Continuous integration with codebase
- Development with autotests
- Continuous feedback
- Bugfix first
- Feature planning & releases
Goals
- Increase development speed
- Increase product quality
- Decrease risk
- Improve feedback
- Increase time-to-market

3. Goal
Use CI/CD not only for big
teams
Initial Conditions
- workstationhealth.com
- Laravel framework
- 3 in house devs
- AWS
- Jira
- bitbucket
- QA/DevOps consulting

4. CI/CD Journey

5. Trunc-based flow - “pure” CI

6. Feature / task branching GIT flow
Is this CI?
- small team
- short-lived branches
- force integration with develop in the
task branch “in-progress” state
Requirements

7. Force developers for faster integration

8. Docker local development

9. docker-compose.yml

10. docker-compose-local.yml

11. https://gist.github.com/Evgenas/c6da523eb2462d848d93505ef
603208b
$ docker-compose -f docker-compose.yml -f docker-compose.local.yml up
Or use Makefile
$ make up

13. Auto-testing

14. “Одним только тестированием качества не
добиться. Рецепт получения высокого качества:
смешивайте разработку и тестирование в
блендере, пока они не станут единой
субстанцией.”
“Вся ответственность за качество лежит на
плечах тех, кто пишет код. Разработчик
отвечает за качество всего кода, к которому он
прикасается: пишет, исправляет или вносит
изменения.”
Роли: разработчик, разработчик в тестировании, инженер по тестированию
Типы тестов: лёгкие, средние, долгие, супердолгие

15. 100% покрытие vs избирательное
тестирование
Модульные тесты (лёгкие)
dependencies
amount
Algorithmic
complexity
Mediators
бизнес-логика
Простой код
без
зависимостей
God Object /
Spaghetti

16. Функциональные тесты (средние)
Attributes
Database connection
Request modeling
Response assertion
Database assertion
Goals
API layer stability
Business logic
Testing code coverage (over unit
testing coverage)
Laravel pros
Easy HOW TO START documentation
Phpunit wrapper
Transaction support “from the box”

20. Testing code coverage
phpdbg – allows get test code coverage faster then
xdebug
find tests/ -name '*Test.php' |
./vendor/liuggio/fastest/fastest -o -p 4 -x phpunit.xml
'phpdbg -qrr ./vendor/phpunit/phpunit/phpunit {} --
coverage-php ./tests/_reports/logs/{n}.cov;'
phpcov – allows merge test coverage from threads
phpdbg -qrr ./vendor/bin/phpcov merge tests/_reports/logs/
--clover tests/_reports/logs/clover.xml

21. Etalon Datas
Data loading variation:
- Inline in each test
- Preload before suite
Laravel:
- Seeds / factories
- Organize domain data builders
- Transactions

22. Domain Data Builders

23. Transactions in phpunit
Restore DB to the initial state after each test is completed

24. How to get faster feedback?
Goal-keep Limits:
- 5 mins for unit/functional tests
- 20 mins for UI tests
Benchmark for 1 thread
Tests: 632, Assertions: 6765:
5 minutes 21 seconds
Optimizations:
- tests multithreading execution
- seeds reload optimization
- selective build code coverage
(only in development branch)

25. Multithreading tests
Tool:
Fastest - https://github.com/liuggio/fastest
find tests/ -name '*Test.php' |
./vendor/liuggio/fastest/fastest
-o -p 4 -x phpunit.xml 'phpdbg -qrr
./vendor/phpunit/phpunit/phpunit {} --coverage-php
./tests/_reports/logs/{n}.cov;'

26. Seeds reload optimization

27. Optimization result
5 minutes 52 seconds vs 59 seconds

29. Phing - build automation tool
Goal: One step build
Tool: https://www.phing.info/
Sample: https://gist.github.com/Evgenas/97fc0bfb

31. Jenkins

32. Trigger light tests suite build (ergoblitz-build-minimal)
BitBucket Plugin: https://plugins.jenkins.io/bitbucket
Bitbucket Pullrequest Builder: https://plugins.jenkins.io/bitbucket-pullrequest-builder

34. Phing Plugin:
https://plugins.jenkins.io/phing

35. Tests feedback notifications
Slack Notification Plugin: https://plugins.jenkins.io/slack

36. Trigger next level tests
Parameterized Trigger Plugin:
https://plugins.jenkins.io/parameterized-trigger

38. UI/User Stories testing automation - end2end tests

39. Behat - what is inside?
BDD or not BDD?

44. Jenkins behat job config

45. Run upstream behat job from build-ci-minimal job

46. Jenkins behat job config

47. Clean up end2end. environment

48. Reseed / redeploy before testing

49. Run tests with fastest for multithreading

50. Success result

51. Trace failed tests

52. Linked screenshot

54. Jira releases

55. Jenkins prepare release job

56. How to stick Jenkins build to Jira release
Jira Plugin: https://plugins.jenkins.io/jira

59. Merge develop to master

60. Prepare docker images for production

61. Amazon ECR

62. Jenkins Docker build
DockerCommon Plugin:
https://plugins.jenkins.io/docker-commons
CloudBees Docker Build and Publish Plugin:
https://plugins.jenkins.io/docker-build-publish
Amazon ECR Plugin:
https://plugins.jenkins.io/amazon-ecr

63. Preproduction
deployment

64. Amazon Elastic Beanstalk
Multicontainer Docker Environment

65. Beanstalk Environments

67. Blue/Green deploy

68. BlueGreen
deploy settings

69. AWS Elastic Beanstalk Plugin: https://plugins.jenkins.io/awseb-deployment-plugin
How to connect Jenkins and AWS Beanstalk

71. AWS Beanstalk configuration files
App deployment: /Dockerrun.aws.json
DB Migrations: ./ebextensions/my-scripts.config
Cron: ./ebextensions/cronjob.config
https://gist.github.com/Evgenas/8bdf43a1

72. Dockerrun.aws.json

73. Cronjob setup on deploy

74. Run migration on deploy

75. Blue-Green Deployment execution

76. Migrations rules for blue-green deployments
https://wiki.openstack.org/wiki/DBMigrationBestPractices

79. Additional preproduction auto-testing with BrowserStack

80. Deploy to production

81. Production / preproduction errors monitoring with Slack

82. evgeniy.k@smart-gamma.com
https://www.linkedin.com/in/evgeniy-kuzmin-03816a52/