Tackling Strategic
Engineering Challenges
Srikanth Rentachintala
Director of Engineering, eBay
Anthony Tang
Rob Mason
Director of Engineering, Walmart
Chief Technology Officer, Applause

BEFORE WE GET
STARTED…
The webinar will last approximately 45 minutes to
1 hour (including Q&A)
You can send your questions using the
“Questions” function on GoToWebinar, we will ask
them during the session.
We will send you a link to the recording as well as
this slide deck
If you tweet during the webinar, feel free to
mention @Applause, @eBay and @Walmart

Presenters of This Session
3
Srikanth
Rentachintala
Director of
Engineering
Rob
Mason
ChiefTechnology
Officer
Anthony
Tang
Director of
Engineering

Today’s Agenda
4
1
3
2
4
Test Automation
Accessibility
Test Data
Q&A

TEST AUTOMATIONRemember: you can send your questions using the
“questions” function on gotowebinar

What is ideal?What is practical?Where does your company fall?
What’s a Good Target for the Percentage of Test Cases to Automate?
6
0% 100%50%
Percentage of test automation
increases as product matures
100% is ideal, but isn’t realistic or practical

What Test Automation Metrics Do You Leverage?
7
Quality Metrics Operational Metrics Product/Team Health Metrics
Test Code Coverage Build CycleTime Development CycleTime
Functional Test Case Automation App/test cases out of sync Production Defect Creation Rate
App Size Automated Deployment Consistency ofVelocity
App LoadTime Deployment Frequency Integration Frequency
App Crash Rate Application Availability MeanTime Between Build Failures

What Is Your Approach to Automation Frameworks and Device/Browser
Providers?
8
eBay Walmart Applause
Build & Deployment (CI/CD
Pipelines) (with customizations)
Services Automation
Web Apps
iOS Apps
Android Apps
Web & Native Grid

How Do You Ensure Test Automation Is Applied In The Right Areas?
9
SpeedtoExecute
Static
Integration
Outof
Process/In
App
InProcess
Functional
E2E
(Horizontal &Vertical)TimetoDevelop
CosttoMaintain
Confidence
Flakiness/Stability
Unit
MultiplesolutionsOK,Toolsstandardsincrease
Walmart’sTesting Lava Lamp

How Do You Ensure Test Automation Is Applied In The Right Areas?
10
eBay’sTesting Pyramid

TEST DATARemember: you can send your questions using the
“questions” function on gotowebinar

eBay’s Method of Getting Good Test Data
How Do You Overcome the Challenges of Getting Good Test
Data in Pre-production Environments?
12
Take slice of
production data
Identify all relationships
using algorithms
Anonymize and
randomize the data
Push data to staging
environment

13
How Do You Ensure Tests Are Consistent
From Run-to-run?
1. Automate the Process
2. Mock the data
3. Curate the data

ACCESSIBILITYRemember: you can send your questions using the
“questions” function on gotowebinar

What’s Your Approach To Accessibility Testing?
15
Manual
• Catch critical issues that
require human eye
• Peace of mind for fullness of
coverage
• Extend coverage beyond
~20% of accessibility issues
• Test in multiple markets with
real users
Automated
• Immediate results
• Benchmarking to gauge
impact of recent changes
• Enhances percentage of
automation coverage
• Catch obvious accessibility
issues

What’s Your Approach to Accessibility Testing?
16
Jira
TestAutomation
(various)
Pre-Prod
Environment
Production
A11Y rules
checks (web)
A11Y unit/int tests
(web & native)
A11Y functional tests
(web & native)
Test Results Test Results Test Results
Looper
Output
Gate
Pass
A11Y manual testing
(web & native)
Jira Tickets
A11Y manual testing
(web & native)
Jira Tickets
Gate
Pass
Walmart’s Approach to Accessibility
YesYes
No
No

How Do You Leverage Automated Accessibility Tools?
17

Q&ARemember: you can send your questions using the
“questions” function on gotowebinar