Learn how to accelerate your test execution pipeline with test frameworks, automation and parallel testing from our very own Bria Grangard, Product Marketing Manager.

1. Accelerating Your Test
Execution Pipeline
BriaGrangard

2. @Bria_Grangard
Who Am I?
• SmartBear Software
• Automated UI functional testing tools & test
management solutions
• Stay in Touch!
• @Bria_Grangard
• Education
• Dartmouth: AB in Engineering, BE in Biomedical
Engineering, MEM with a healthcare focus
• What do I love to do?
• Run, dance, play board games (Settlers of Catan
anyone?)

3. Page
Today’s Agenda
@Bria_Grangard

4. We provide tools for development, testing, and operations teams
to create great software, faster than ever.
AccelerateSDLCWorkflows | ImproveQualityatEveryStage | RealizeRapidTime-to-Value
• HQ in Boston, MA, USA, with 7 offices globally
• Founded in 2009
• Open Source Innovator (Swagger & SoapUI)
6.5M+
Users
194
Countries
22K+
Companies
TestComplete
SoapUI Pro
SwaggerHub
CrossBrowserTesting
QAComplete
AlertSite

5. UI
API
Create Great Software, Without Tradeoffs
Perform Code &
Doc Review
Collaborator
Design, Develop, &
Document APIs
SwaggerHub
DEV TEST OPS
Code
Create Automated UI
Functional Tests
(Web, Desktop, Mobile)
TestComplete Script
Run Tests On Real
Devices in the Cloud
CrossBrowserTesting
Create Web Load Tests
LoadComplete
Spec
Create Automated API
Functional Tests
(REST, SOAP, and more)
SoapUI Pro Script
Virtualize API &
Database Services
ServiceV Pro
Create API Load Tests
LoadUI Pro
Monitor Web & API Performance,
Availability, & Functional Correctness
AlertSite
Manage Manual & Automated Tests
QAComplete
Integrations …100 +

6. | SB Test
Page
What’s Going on in the
Testing World?
@Bria_Grangard
BDD
AI
Machine
Learning
DevOps
Shift Left
Agile
Automation

7. Page
There are bottlenecks in today’s development processes.
• Irontriangletrade-off
• Teamstodayareconstantlyfeelingpressuretodeliversoftwarefaster,withoutcompromisingquality
• Automationmightrampup
• Thereisonlyacertainpointastohowscalableautomationcanbe
• Testenvironmentsareoftentheroot-causeofthebottlenecks
• Theyareverytimeconsumingandcostly
@Bria_Grangard

8. Page
@Bria_Grangard
The promise of the new software delivery cycle
WaterfallAgile
Design Build Test Implement
DevOps
Week1 Week2 Week3 Week4

9. Page
@Bria_Grangard
Time Consuming Nature Of Web Testing
#ofTests
0
200
400
600
800
1000
1200
MVP Feature
Set 1
Feature
Set 2
Feature
Set 3
V2
More Features = More Testing
Age Of
Product
#ofBrowsers
0
2
4
6
8
10
12
MVP Feature
Set 1
Feature
Set 2
Feature
Set 3
V2
Popularity of
Product

10. Page
@Bria_Grangard
1
2
3
4
Test Less
Hire More Testers
Diversify Your
Testing
Run Tests In Parallel
More Bugs
Increased Cost

11. | SB Test
Page
Now… How Can We Go Faster?
@Bria_Grangard
1. Test Frameworks
2. Automation
3. Parallel Testing

12. Level 1: Test
Frameworks
@Bria_Grangard

13. Page
The Basics of a Test Framework
@Bria_Grangard
Requirements Tests Defects
What do we
make and how
should it
behave
Make sure it
works as
stated in the
Requirement
Definition Sets Environments
Actual Results
do not equal
Expected
Results

14. Page
@Bria_Grangard
What is a Test Framework?
A Test Framework:
Links tests to other SDLC items
Is NOT a Test Automation Framework but often contains one
Allows for rapid creation of tests from reusable components
Separates data from logic (REUSABILITY)
Provides a standardized test “language” and reporting structure for an application
under test

15. Page
@Bria_Grangard
Elements of a Test Framework
• Library: A repository of all your decomposed scripts, separated into their components
• Test Data Sources: A repository of all data sources
• Helper Functions: A repository of all decomposed test scripts, automated or manual,
that are inputs or checks
• Test Environments: A list of all covered testing environments, broken out by type (OS,
browsers…)
• Modules: The combination of library items with any helper functions and test data
sources–plus environments
• Structure / Hierarchies: The “folder” structure of modules

16. Level 2: Figure Out What
Tests Should be Automated
@Bria_Grangard

17. Page
There are many types of testing that need to be done…
@Bria_Grangard
Browser: Chrome HTML5, Angular JS Network Service/API/Database
End-to-End Testing

18. Page
@Bria_Grangard
Automated Testing Pyramid Approach
UI
Service
Unit
Manual

19. Page
@Bria_Grangard
A Little Manual v Automated Math
Product v2
# of Test Cases
# of Browsers Supported
TotalTest Cases
Avg Test Run Time
Product v2
1,000
10
10,000
.5 Min
TotalTestTime 83 Hrs
With 2 QA engineers,
That is 1 week of testing
# of Test Cases
# of Browsers Supported
TotalTest Cases
Avg Test Run Time
1,000
10
10,000
4 Min
TotalTestTime 666 Hrs
With 5 manual testers, that is
3.5 weeks of testing

20. Page
Decide on What to Automate
• EnvironmentSetup/Teardown
• DataEntry
• FormFilling
• Varyingdatainputsinarepetitiveprocess
• Exposingbackenddata(APIs,DBtable,etc.)
• Repetitive/boringtasksthatarepronetoinattentionerrors
• Taskswithhighreusevalueacrossmanyworkflows
• Testswithtimingorscreenresponsivenessasacriteriaforsuccess
• Manynon-functionaltesttypes,suchasperformancetesting
• CapturingResults
@Bria_Grangard

21. Page 21
@Bria_Grangard
Speeding Up Your Pipeline
Longer
Time To Test Behind Releases
Shorter
Manual
Testing
Record &
Replay
Unit
Testing
POM Atomic Testing Continuous
Testing

22. PagePage
But my Dev team says I
have days to test, not
weeks…
@Bria_Grangard

23. Level 3: Go Faster.
@Bria_Grangard

24. Page
Let’s Go Faster!
Wecan,with paralleltesting
@Bria_Grangard
Test 2Test 2Test 2
Test 3 Test 3 Test 3
Running tests sequentially, we were
able to run our tests in 1 week
Test 1Test 1 Test 1
Test 1
Test 1
Test 1
Test 2
Test 2
Test 2
Test 3
Test 3
Test 3
With 20 Parallel
executions, we can run our
entire test suite in only 4
hours

25. Page
Types of test to run in parallel
Testingacrossdifferentbrowsersanddevicesisoneofthemosttime
consumingaspectsoftestingthefrontendofyourwebsiteorweb
application.Runmoretests,againstmorebrowserconfigurationsby
runningtheminparallel.
Accordingtothetestingpyramid,Unittestsshouldbeyourmost
abundanttesttypeinyourentiretestingsuite.Becauseofthis,running
14,000unittestsinunderanhourisreallyonlypossiblewithamassive
paralleltestinginfrastructureinvestment.
Needtogetyourminimumtestingdoneinthenext20minuteswhileyou
pushahotfix?Onlywaytodothatistoruntheminparallel,allowingyou
togetthemosttestingdoneintheshortestamountoftime.
Becausedeploymentsarehappeningatsucharapidpace,regression
testingisonofthebestwaystohaveatypeof“testingversioncontrol”
makingsurethefunctionalityofthenewbuild,matchesthatofthelast
stablebuild.Runningthesetestsinparallelallowsmoretobetested.
Cross Browser Testing
Unit Testing
Regression Testing
Smoke Testing
Mosteffectivetests to seeROIfromforparalleltesting
@Bria_Grangard

26. Page
Benefits of Parallel Testing
1. Quick Deployments
2. Faster Feedback
3. Cross Browser Testing
4. Better Test coverage
5. Saves Valuable Time
@Bria_Grangard

27. | SB Test
Page
Key Takeaways
1. Testframeworksareimportantandakeytosuccess.
2. Whenautomating—choose whichteststoautomatewisely
3. Paralleltestinglet’syougofaster.
@Bria_Grangard

28. Questions?
@Bria_Grangard

29. Thank You!
@Bria_Grangard