DevEX - reference for building teams, processes, and platforms
Automated Web App Performance Testing Using WebDriver
1. Web Performance Testing
with WebDriver
Tools, Tips, and Techniques
Michael Klepikov
Make the Web Faster Team, Google
@MTWFG
webpagetest.org
2. Why is it important?
■ +1s load => -7% conversion rate
■ Just better user experience
■ Mobile exacerbates slowness
■ 46% mobile users abandon after 10s
3. Page Load performance
■ WebPageTest.org
■ Waterfall chart
■ Simulated latency
■ Many iterations for statistical validity
4. Interactive scenarios
■ Multi-page
■ Login
■ Ordering sequence
■ AJAX is a better user experience
5. Marry WD + WebPageTest
■ Use functional tests for performance
■ Measure performance while test runs
■ Not quite as trivial as it seems
6. WPT + WD Security
■ Sandboxing Java is hard, JS is easy
■ WebDriverJS FTW
○ NodeJS VM API
■ Submit a WDJS script to WPT
7. Google Instant Search - WDJS
driver = new webdriver.Builder().build();
driver.get('http://www.google.com');
driver.findElement(webdriver.By.name('q'))
.sendKeys('webdriver');
driver.findElement(webdriver.By.name('btnG'))
.click();
driver.wait(function() {
return driver.getTitle();
});
9. Measure around WD calls?
■ Test-WD-browser: HTTP JSON RPC
■ WD has 200ms wait loops
■ Too granular, overhead, variability
■ We want single-ms precision
10. Page self-instrumentation
■ Frameworks exist
○ Boomerang, Episodes, others
■ Page "calls home" with numbers
■ Intercept these while the test runs
○ Parse from URLs in HAR
12. Browser Standards
■ Nav Timings in most major browsers
■ Resource Timings -- not yet
■ User Timings -- not yet
■ Chrome DevTools Timeline
■ Bowsers are moving fast...
13. Do's and Dont's
■ No WD within instrumented intervals
■ Aggregate over multiple iterations!
■ Run frontend in maximum isolation
○ Mock/stub/fake backends
○ Stable, repeatable results