We often come across situations when: > We cannot figure out why our Selenium/Appium scripts fail during execution either locally or on CI. > We need to debug scripts locally while writing them but find debugging painful and slow Debugging the scripts is often the approach to fix them. What all different ways we can do it? I shared about some of the good practices I have used for debugging Selenium and Appium scripts

1. 11th Meetup
18 July 2019
DBS, Singapore
Good practices for
debugging Selenium
and Appium tests

2. Abhijeet Vaikar
Senior Software Engineer Test @ Carousell
Co-organizer @ taqelah!
Testing web and mobile apps since 7 years
LinkedIn: https://www.linkedin.com/in/abhijeetvaikar/
Twitter: https://twitter.com/AbhijeetVaikar

3. Poll Time

4. Testing is a process of
Learning, Discovering, Exploring, Analysing a
system.

5. Automated testing, thus is
also a process of
Learning, Discovering, Exploring, Analysing a
system
where the system is:
● the AUT
● and your test code too!

6. Typical iteration of an automated UI test
Local Execution
● Running test locally
● Verifying it works fine
● Fix if it breaks
Continuous
Integration
● Run the test along with
other tests on a CI
server against an
in-house device farm
or a cloud provider
● Analyse test runs
Test Authoring and
Prototyping
● Acceptance Criteria
● Inspecting the UI
● Creating UI locators
● Writing the test code
● Adding assertions

7. Pass
Automated Test
Fail
What failed?
Why it failed?
Where it failed?
When it failed?
A tester’s reactions to each iteration
Authoring
Local Execution
CI execution

8. What
failed?
Where it
failed?
When it
failed?
Why it
failed? DEBUGGING!

10. Sources of debugging your Selenium/Appium code
● UI Inspectors and browser dev tools for debugging locators
● REPL tools for trying out Selenium/Appium client commands
● Debuggers in your IDE
● Test execution logs
● Selenium/Appium server logs
● Javascript console logs in browser
● Logs produced by native mobile apps
● Network logs

11. UI Inspectors and browser dev tools for
debugging locators

12. Appium Desktop for UI inspection

13. What can you do with Appium desktop?
● Inspect the UI hierarchy interactively
● Design and try out locators with different strategies
● Swipe by coordinates
● Tap by coordinates
● Record actions and convert them to script in the language of your choice
(Javascript, Java-JUnit, Ruby, Python, Robot framework) along with boiler
plate code
● Copy the XML UI hierarchy source which you can then inspect
● (New) Actions tab to execute device and session based actions

14. Macaca Inspector
https://macacajs.github.io/app-inspector/
Web based UI
inspector for both
iOS and Android
platforms

15. Android specific: UiAutomator Viewer
iOS specific: XCode’s Accessibility inspector
Want more?

16. Chrome dev tools
● Ctrl + F and paste your locator to evaluate
● Hit $x(“//*your-xpath-here”) in the console to evaluate a xpath
● Hit $(“your-css-selector”) in the console to evaluate a css selector
● Cheat sheets are your best friends (preferably avoid browser plugins that generate xpath/css selectors for
you):
https://www.red-gate.com/simple-talk/development/dotnet-development/xpath-css-dom-and-selenium-the-r
osetta-stone/

17. REPL tools for debugging
Selenium/Appium client commands

18. JShell - The REPL for Java (Since Java 9)
https://www.linkedin.com/pulse/test-automation-selenium-webdriver-java-cli-via-jshell-nir-tal/

19. You can also use the
evaluate expression
feature in IDEs like
Eclipse/IntelliJ

20. WebDriverIO REPL (Javascript) - for both Selenium and Appium
https://webdriver.io/docs/repl.html

21. from selenium import webdriver
browser = webdriver.Firefox()
browser.get('http://seleniumhq.org/'
)
Python shell - REPL for Python
Python being an interpreter, you can run each line of code separately in
Python shell
https://pypi.org/project/selenium/

22. Ruby
Build your own REPL for Selenium/Appium:
http://elementalselenium.com/tips/11-build-an-interactive-prompt
ARC (Appium Ruby Console):
https://github.com/appium/ruby_console
NodeJS
Appium REPL:
https://www.npmjs.com/package/appium-repl

23. Debuggers in your IDE

24. What do you prefer?

25. System.out.println("I'm here");
System.out.println("Inside the function");
System.out.println("Outside the loop");
Imagine debugging across
multiple classes by adding print
statements!

26. How to use a debugger - 101
1. Add a breakpoint against the
line of code you want to start
debugging from
2. Execute your test runner in
Debug mode

27. SAGE MODE ACTIVATED
● Step Over
● Step Into
● Force Step Into
● Step Out
● Evaluate Expression
● Resume normal execution
● Inspect all variables/objects
https://naruto.fandom.com/wiki/Sage_Mode

28. Test execution logs

29. Use Logging libraries instead of:
● System.out.println() - Java
● Console.log() - Javascript/NodeJS
● print() - Python
● Puts - Ruby
Test execution logs as a source for debugging test runs:
● Useful to understand what happened in your test
● Logs can be used for generating test reports

30. Why use a logging framework for your tests?
● Customized output - Easy formatting
● Logging levels (INFO, WARN, ERROR, DEBUG, TRACE)
● Can easily switch it on or off based on requirement
● Ability to persist log output on various storage
mechanisms (file, database)
● System.out.println() is an I/O operation

31. Examples
LOG.info("Initialising WebDriver on cloud provider: {}" ,
configuration. getCloudProvider ().toUpperCase());
LOG.info("TEST PASSED");
catch (WebDriverException e) {
LOG.error("Error creating web driver" , e);
System.exit(1);
}
LOG.warn("API call failed. Retrying again" );
LOG.warn("Unable to write the report json file : {}" ,
e);
Info - Logging important information
Error - Something went very wrong
Warn - Unstable but can be recovered
LOG.debug("Dismissing alert {}" , alert)
Debug - Information only meant for debugging purpose

32. Selenium/Appium server logs

33. Selenium WebDriver logs
System.setProperty("webdriver.chrome.logfile" , "D:chromedriver.log" );
System.setProperty("webdriver.chrome.verboseLogging" , "true");
System.setProperty(FirefoxDriver. SystemProperty .DRIVER_USE_MARIONETTE ,"true");
System.setProperty(FirefoxDriver. SystemProperty .BROWSER_LOGFILE ,"C:templogs.t
xt");

34. Appium server logs
--log /path/to/appium.log
Server argument (When you start appium server)
LogEntries logEntries =
driver.manage().logs().get("server”);
Programmatically

35. Javascript console logs in browser

36. How to keep Chrome dev tools open while you are
debugging your Selenium test locally
ChromeOptions options = new ChromeOptions();
options.addArguments("--disable-extensions" );
options.addArguments("--auto-open-devtools-for-tabs" );
driver = new ChromeDriver(options);
How to fetch browser console logs (including errors)
programmtically
LogEntries logEntries =
driver.manage().logs().get(LogType.BROWSER);

37. Logs produced by native mobile apps

38. Fetching ADB logs for Android device for debugging
List<LogEntry> logEntries =
driver.manage().logs().get("logcat").getAll();
Fetching iOS device logs
● idevicesyslog tool from the command line for real iOS Device
● Console app on MacOS to read console log from simulator

39. Network Logs

40. MITMProxy - free and open source HTTPS
proxy
MITMProxy Java - A java wrapper for MITM
Proxy
Fiddler for Windows and MacOS

41. What techniques or practices do
you follow to debug your
automation scripts?
Please share!

42. Thank you!