Presentation from Ukranian Testing Days conference (August 2012) about techniques and approaches in UI testing automation.

1. Automated Web UI
Testing
Mikalai Alimenkou
http://xpinjection.com
@xpinjection

3. Do you need UI testing?

4. Ugly design = bad
user experience

5. Some sites can lose money

6. Think about design as
non-functional requirement

7. May be just do it manually?
Large number Many
of pages browsers

8. Not all layout bugs are so visible

9. Navigation through all pages is very useful for all types of testing
NAVIGATION TESTS WITH
WEBDRIVER

10. Why WebDriver?
• Works with many browsers
• Can handle window size and browser settings
• Clear and simple API
• Allow to execute JavaScript

11. How do navigation tests work?
Preconfigured Define all
environment pages from
requirements
Reference
Wait for
database Load in series
READY state

12. Types of UI testing
Image
analysis
Screenshot based
Static (HTML, CSS,
JavaScript)

13. Well structured page has better chances to be displayed correctly
STATIC ANALYSIS TECHNIQUES

16. How to access page?
• Filter in Java for static HTML
• Lightweight proxy server
• HtmlWebDriver
• Real browsers to make things realistic:
• work with user agents
• browser specific headers
• dynamic pages

17. Existing solutions for Java
• http://www.rexsl.com/rexsl-w3c/ - API for
W3C validators
• https://code.google.com/p/w3c-markup-
validation-filter/ - validation Filter

18. DEMO TIME!

21. DEMO TIME!

22. Useful techniques
• Disable everything you don’t want to validate
• Disable ads, banners, images to make
everything quicker
• Static CSS may be validated on server

23. HTTP proxy for statistics
https://github.com/webmetrics/browsermob-proxy

24. What can you get from
HTTP proxy?
• External resources integration status
(Facebook, Twitter, Ads, etc.)
• Images, banners, ads and other nonfunctional
resources
• Collect HTTP traffic for analysis (404, redirects,
loading time, etc.)

25. Net panel in Firebug

26. Export statistics and reports
http://www.softwareishard.com/blog/firebug/automate-page-
load-performance-testing-with-firebug-and-selenium/

27. DEMO TIME!

28. Compare page screenshots with history to limit amount of manual work
SCREENSHOT BASED TECHNIQUES

29. Regression screenshots
For each
navigation test
Compare
Take fresh
screenshots

30. How can it be stable?
• Disable or mock banners, GIFs and other
animation
• Limit tested regions for each page
• Improve screenshot comparison algorithms
• Use image comparison tools for manual
validation

31. Find panels by corners
Check
X and Y
Validate
content

32. Extract information about page elements for extended layout analysis
SCREENSHOT PROCESSING
TECHNIQUES

33. Fighting Layout Bugs
• http://code.google.com/p/fighting-layout-
bugs/
• Open source Java library
• Uses WebDriver
• Proof of concepts

34. Need to detect edges and text
Edges
Edges Text

35. Simple text detector
• Inject jQuery to the page for simple JavaScript
• jQuery('*').css('color', '#000000');
• Take screenshot
• jQuery('*').css('color', '#FFFFFF');
• Take screenshot again
• Compare screenshots – different pixels are
text

36. Simple edge detector
• Inject jQuery to the page for simple JavaScript
• jQuery('*').css('color', 'transparent');
• Take screenshot
• Analyze pixel sequences with the same or very
similar color and contrast pixels from left or
right
• Apply minimal length constraints

37. #1. Text overlapping edges
• Bug is where text
intersects edge
• Replace each buggy
pixel with circle to
define buggy areas
• Blend detected areas
with red line on the
screenshot

38. #2. Text with low contrast
• Bug is where text color is close to background
color
• Find areas with the same background color
• Blend detected areas with red line on the
screenshot

39. #3. Invalid images detection
• Missed, empty or invalid URL
• 4xx or 5xx HTTP response code
• Content-type is not image related

40. #4. Focus visibility detection
• Switch to the next element with TAB
• Take screenshot
• Check if something changed for element

41. Other types of layout bugs
• Different layouts in different browsers
• Unintended scrollbars for dialogs and frames
• Broken panel borders
• Text partly hidden
• Broken elements flow

42. We are capable to do more
with modern technologies
and our brain

43. @xpinjection
http://xpinjection.com
mikalai.alimenkou@xpinjection.com