Modern organizations invest vastly on designing top-notch visual user interfaces that improve their credibility, strengthen their brand and help them win new customers, but how do they ensure that they look flawless when delivered to the end users? How do they prevent regressions and visual defects given all the possible combinations of devices, platforms, browsers, screen sizes and resolution that proliferate from day to day? Those who care enough often use dedicated, manual QA stages that inevitably introduce delays in the deployment pipelines and cost a great deal of resources and time. Those who, on the other hand, leave this aspect unaddressed, incur the risk of exposing defective interfaces that negatively impress their audience, often times losing them to competitors. We present an approach to visual testing that is completely automated and integrated into a continuous deployment pipeline, capable of blocking visual regressions and defects at early development stages thus saving resources, costs and reducing the time needed to deliver impeccable user interfaces. A live demo will highlight all the steps involved, from design to deployment, using GitLab, Python, Selenium and Docker.

1. Enabling continuous deployment
of pixel-perfect user interfaces
Stefano Rago

2. Stefano Rago
DevOps Engineer at
@helmoltz

3. Topics
• The importance of UX / UI
• Issues and challenges
• An approach based on TDD
• Integrating automated tests into CI/CD

4. UX / UI
• UX Design encompasses any and all
interactions between a potential or active
customer and a company.
• UI Design is responsible for the transference
of a brand’s strengths and visual assets to a
product’s interface as to best enhance the
user’s experience.
source: careerfoundry.com

5. UI/UX in the industry
• 88% of online consumers are less likely to return
to a site after a bad experience. – Econsultancy
• 48% of people cited a website’s design as the
number one factor in deciding the credibility of a
business. – Ironpaper Inc.
• A site with a well-designed user interface can have
up to 200% higher conversion rates than one that
was poorly designed. – Naturalinteraction
• “Every dollar invested in UX generates around
$100 dollars return” – IBM.

6. UI Defects

17. Challenges
i18n
dynamic content
icons by Pixel perfect

18. A simple website…
•10 pages
•3 supported desktop browsers
•2 mobile browsers
•2 orientations
•3 responsive breakpoints
… 360 variations

19. Manual testing
+ Can better explore cases previously unthought of
+ Can be more flexible
- Tedious and error prone
- Unfeasible to take into account all possible scenarios
- Tends to be executed only in specific development stages
- Introduces delays in delivery and high costs

20. Automated testing
+ Can be easily integrated in CI/CD pipelines
+ Fast feedback loop
+ Reduces time, effort and costs required to fix issues
+ Confidence and safety nets for developers
+ Tests can be replicated and run at different development stages
- Tests maintenance
- Time required to run tests

21. Support for automated tests
• Approach
• Languages
• Test frameworks and reporting
• Tolerance and false positive detection
• Coverage
• Aid for code reviews
• SaaS

22. Our approach
• Targeting web apps and websites
• Screenshot-based
• Test Driven, outside-in
• Python, pytest, needle
• Selenium grid

23. Our approach
GitLab runner
GitLab job runner #1
…
GitLab job runner #n
GitLab server
git repository
CI/CD
Selenium grid
Selenium Hub
Selenium Node
Google Chrome
dev
Production
web server
nginx
flask
end user
1
2
3
4 5
uwsgi

24. Demo

25. Considerations, trade-offs and
improvements
• Test duration / feedback loop
• Maintenance costs
• Time needed to reach the testable
state
• False positive detection
• Continuous testing/monitoring of
production environments

26. Thanks