In this webinar you will learn how Walmart migrated from many independent, siloed teams doing manual testing to integrated teams that run over 50k automated tests daily using shared resources like Test Armada and Sauce Labs.
Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...
Continuous Testing at Scale the Walmart Way with Test Armada
1. Pranav Verma
Sr. Engineering Manager
Developer Experience Tools
EMAIL:
pverma@walmartlabs.co
m
Testing Infrastructure at Scale
September 2018
E X P E R I E N C E T O O L S
CUSTO M ER
E X P E R I E N C E T O O L S
DEVELOPER
2. Agenda
• Test Armada Introduction
• Test Armada Scope
• Test Armada Infrastructure
• Continuous Integration
• Automation Cloud
• Data Insights
• Q&A
3. A Cultural Shift in Quality Automation
within Customer Experience
Engineering
2016
4. You have 6 months to
automate100% for your
P1/P2 test cases
5. Automate 2000+ Manual Test
Train 600 Associates
On-board 25+ number of Apps
Migrate FE to new Platform
THE DETAILS
7. The Before and After Transformation
2000+ Manual Test
100% P1/P2
Automated
Lack of accountability
around Quality
Shared ownership of
Quality
2 Deployments a
month
Daily deployments
Test limited to a
single browser
Multi-browser
combinations part of CI
process
9. End to end test platform for web, native, and
backend services with actionable reporting
capabilities
10. Scope Across the Company
TEAMS WE SUPPORT
85 projects in 2 years
200 projects by 2020
1M Development Hours Saved
16M Functional Tests Executed
149M Mocked API Calls
LOCATIONS WE SUPPORT NUMBERS AT SCALE
Bay Area
San Diego
Bangalore
Canada
Omaha Hoboken
Bentonville
UK
Mexico
Point of Sales RCT ASDA Growth
USGM Jet.com Pharmacy GIF Store Systems OmniChannel
Customer Promise Hayneedle
USGR
EXPO
Portland
12. How to run automation tests fast
at a massive scale reliably.
CONTINUOUS INTEGRATION
PROBLEM
Tests need to run against multiple OS/Browsers, which multiplies the numbers
of tests. Need a solution to run massive number of tests without compromising
execution time.
GOAL
14. BENEFITS
CART PR Verify
(228 Tests on 2 Browsers)
6
HOURS
Average Serial
Test ExecutionTime
25
MINUTES
Average Parallel
Test ExecutionTime
to 93%
Savings
=
15. How to test Walmart websites and native apps on
all supported browsers and devices.
AUTOMATION CLOUD
PROBLEM
Need a solution to support automation reliably atWalmart scale, and also
allows team to configure as per their testing needs.
GOAL
18. How to see health of your project in one place.
DATA INSIGHTS
PROBLEM
The application quality information is scattered between JIRA, CI,Test
frameworks and Automation clouds. Need a solution to visualize all
information in one place to make informed decisions.
GOAL
19. Data Insights Platform
Managed Services
SOLUTION
FunctionalTestAgent
PerfTestAgent
CI
…
DB
Functional Report Client Perf Report Backend Perf Report
Self-Service Portal
API
H
X
HYGIEIA
XRAY
CLIENT
21. r
Open Test Armada Platform
Resource Manager Self ServiceData Insights
TESTARMADA FEATURES
PROGRAMMING LANGUAGE
Magellan
TEST RUNNER
TestArmadaTDK
TESTARMADA LIBRERIES
OUT OFTHE BOX SOLUTION
Any Language
PROGRAMMING LANGUAGE
Any Runner
TEST RUNNER
TestArmada Service
TESTARMADAAPI
Resource Manager Self ServiceData Insights
TESTARMADA FEATURES
CUSTOMIZED SOLUTION