Originally presented by Nick Germanotta of Verizon & Bala Thangaraj of IBM at SAP SAPPHIRE NOW 2015 & ASUG Annual Conference in Orlando, Florida.

1. Confidentialand proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
How Verizon Uses
Test Automation to Accelerate
SAP Implementations
Nick Germanotta, Verizon
Bala Thangaraj, IBM

2. Confidentialand proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 2
Agenda
• Overview
• Uni-Fi Program Overview
• Automation Testing Objectives
• Accelerated Implementation Approach
• Execution Strategy
• Benefits Summary
• Next Steps
• Questions

3. Confidentialand proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 3
Overview
This session explains how Verizon used Worksoft test automation suite to accelerate two
successive SAP convergence projects with a combination of IBM methods and Worksoft
best practices
Learn how we executed end-to-end business process testing to reduce cycle times by 35%,
save time with streamlined / reusable test scenarios, and improve business process quality
and test coverage.
Three main takeaways:
• Explore how Verizon used automated testing to effectively build & execute end-to-end testing early in the project. This
significantly reduced the traditional three testing cycles to two and enabled early detection of defects
45% of overall testing scope was automated in less than 90 days
• Learn how to reduce manual effort and improve staff efficiency and how to apply the efficiency model to future releases.
Executed automated test scenarios in 1.5 hours vs. 40 hours when done manually
• Develop plans to maintain high quality business process execution during a major SAP implementation to ensure
business operations and users are not adversely affected during a consolidation, migration, or upgrade.
Worksoft enabled Team without increasing the project resource pool

4. Confidentialand proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 4
Uni-Fi Program Overview
Uni-Fi, a financial systems unification program was approved by Verizon executive management and is sponsored by
Verizon’s Corporate Executive Vice President and CFO. Its objective is the consolidation of several legacy systems and
acquisitions into one of the select ERP systems that best fit the existing standard Verizon business processes.
By consolidating and standardizing systems and processes, Uni-Fi promotes:
Corporate
Agility
Improved
Financial
Controls and
Reporting
Rapid
Acquisitions
and Divestitures
Uni-Fi will consolidate to select SAP and PeopleSoft ERP systems
Consolidated
System
Architecture

5. Confidentialand proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 5
Uni-Fi Program Overview
Challenges:
• Inconsistent policies, processes and
metrics, and lack of controls
• Aging legacy systems and increased
maintenance costs
• Multiple reporting sources and increased
regulatory and reporting requirements
• Multiple transactional systems and support
strategies
Priority:

6. Confidentialand proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 6
Uni-Fi Program Overview
ERP System Consolidation: 2012 – 2017*
• Schedule and plan pending funding allocation
and scope determination
V1V
VNIC/VSSI/VZEDI
EdgeCast
Dispatch1
Corp. & VDSL
ECS
Uplynk
Hughes
VDSI
nSAP
vSAP
PS 9.1
CostPoint VFI
Terremark
OnCue
TBD







Legend
Select System
System / Org.
 Completed
 Underway



7. Confidentialand proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 7
Uni-Fi Program Overview
Release 3
NPRS & D1 to nSAP
Release 3.1
Edgecast to nSAP
Release 4
Corporate & Domestic
IT to PeopleSoft 9.1
Release 4.1*
VDSI to PeopleSoft 9.1
Release 5*
Empire City Subway
to vSAP
Hughes Conversion
Release # and target
system TBD
2014 2015 2016 2017
Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1
A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M
NRPS & D1
8/1/2014
Uplynk & OnCue went live in 1Q2014
*Go-Live date confirmation pending scope lock
TODAY
Edgecast
10/1/2014
ICE & VDS
8/3/2015
VDSI
2016 TBD
ECS
7/1/2015 (TBC)
Scheduled/Completed
Targeted/Go Live TBD
Under Assessment
Hughes
Being Assessed

8. Confidentialand proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 8
Automation Testing Objectives
• Testing is said to be one of the biggest pain-points of SAP Implementation and Upgrades
“For SAP implementations more than a third of total project effort is used for “Quality & Testing”
• Experience from large IBM SAP projects and projects in general show that level of automation is ”too” low
& this trend is changing
Improved Test Coverage Reduced TCO Accelerated Implementations
Objectives to be achieved
• Accelerate Uni-Fi project releases by reducing test cycle time
• Free-up business critical resources to drive strategic important projects without increasing budget
• Increase efficiency by reducing manual testing effort, email hand-offs and wait times
• Create an Automation framework that can be adopted for future Uni-Fi releases
• Early detection of defects – faster execution of tests. Improve time to react.
• Minimize the creation of automated test scripts by using re-use Test Libraries – create once, use many times
• Use well documented business processes for user training and regulatory requirements
IBM’s experience at large SAP clients shows
that Worksoft automation tool is 30-45% faster
in test build and execution than previously
deployed tools
TTQ

9. Confidentialand proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 9
Accelerated Implementation:
Blueprint Phase
Scenario
Process
Steps
Confirm the Business Process Hierarchy (BPH) in Solution Manager as soon as the
Business Blueprint phase is completed.
Design the Business Process Hierarchy in SOLMAN
BPH defines the “Scenario Level” scope of Testing
Expand the Scenarios in BPH to identify the Data Variant Scenarios to
define the Overall Test Scope
Ex: Procure to Pay – Procure to Pay for Capital Projects
Procure to Pay for Expense Projects
Procure to Pay for Limit Item Services
Classify the Scenarios under “Quad A” methodology
“Adopt” – will use process & system as designed
“Adapt” – some changes to accommodate requirements
“Add” – new process or functionality currently not available
“Abstain” – Process not applicable for this release

10. Confidentialand proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 10
Scope of Business Process
Validation
SRM
Leasing
Supply
Chain
Project &
Assets
Order to
Cash
Accounts
Payable,
T&E
nSAP
Finance
In Scope
Out of Scope
Procure to Pay Scenarios
[SRM, Portal & SAP ECC]
Modules Types of Scenarios
Order to Cash Scenarios
Project Creation to Settlement
Asset Maintenance
Travel & Expense Processes
AP Invoicing & Payment Processes
Tax&FinancialValidations

11. Confidentialand proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 11
Accelerated Implementation:
Realization Build Phase
Define the selection criteria for automation and identify the Scope of Automation
Testing in early Realization Phase
SAP Projects of such scale will use the following criteria to decide the Business
Scenarios that fall under Automation
• Business Critical Test Cases – Global Scenarios that have broader System and release Impact
• Data Driven Test Cases - Business Scenarios that use multiple data values for the same functions
• Complex and Time Consuming Business Cases - Multiple Hand-offs across different Functional Areas
• Business Scenarios that requiring great deal of precision on Qty or Dollars [High Financial Impacts]
• Testing Needed on Multiple Systems / SAP Clients or Portal
• Simple Test Cases involving repetitive tests or several iterations
• And those that can be Automated using the Tool technology
The following test cases will not fall under Automation
• One time / Ad-Hoc Testing
• Back End Testing that runs behind GUI applications
• Usability – If the Business Process is easy to execute and Test
• Nature of the Business Process does not affect future changes and releases to a broader degree
• Localized Business Processes

12. Confidentialand proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 12
Accelerated Implementation:
Realization Test Phase:
Categorizing processes using this framework helped manage scope and drive efficient work.
Functional Unit Test Cycle 1: End-to-End /
UAT Dry Run
Cycle 2: End-to-End /
User Acceptance Test
‘Adopt’
• Build Automation Component Transactions
needed for E2E Scenarios that most
regions will likely adopt
• Execute automated end-to-end scenarios using
organizational elements during Week 1 of the cycle
• Execute automated end-to-end scenarios
using organizational elements during week 1
‘Adapt’
• Adjust the Automation Components
Transactions needed for E2E Scenarios
• Execute automated end-to-end scenarios using to
accommodate regional and/or site relevant
adjustments
• Execute automated end-to-end scenarios
using to accommodate regional and/or site
relevant adjustments
‘Add’
• Manually Script and test all core SAP
transactions and development objects
during Global Build in HPQC
• Script and validate any additions to end-to-end tests
in HPQC.
• Manually execute E2E scripts using converted data
& security roles
• Manually execute approved end-to-end scripts
in HPQC using new security roles and
converted data.
‘Abstain’
• Identify and communicate transactions and
development that are out of scope for a
site
• Identify scenarios that are low risk or not applicable • Refrain from executing low risk scenarios that
passed during informal testing.
Build Automation Tests Execute Automated Tests as early as Week 1 of each testing cycle

13. Confidentialand proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 13
Functional
Design
Baseline
Configuration
Functional
Unit Testing
(FUT)
Move to QA
End User
Role Profiles
Integration
Cycle 1
UAT Dry Run
Integration
Cycle 2
UAT
Final
Regression
Test
Risk
Assessment
Go / No-Go
Decision
Realization Phase: Milestones
Automation Test
“BUILD”
Automation Test
“EXECUTION”Projects
GL
P2
P
- 45% of the Total Test Scope was automated in less
than 90 days
- 70 Core End to End Scenarios (~237 Scripts) were built
versus the original plan of 42 scenarios
- “Adopt” & “Adapt” Scenarios were automated with the
dual intent of utilizing them for Integration as well as
regression testing
- 40% of the Test Scope was executed during week 1 of each
integration test cycle.
- Fully validated implementation with 2 integration test cycles in
contrast to 3 cycle approach
- Reduced the overall test execution time for each E2E scenario
from ~35 Hrs to 1.5 Hrs and increased the number of Data
Variants per scenario

14. Confidentialand proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 14
Worksoft Enabled Team
• Subject Matter Experts (drawn from Business Team)
– Responsible for recording the business process using
Certify Capture, typically from Client’s Business Team
– Provides guidance to the Automation Test Engineers
on the Business Process
– Validate, Reviews & Approves automation Tests Build
– 2 to 4 Business SMEs are typically engaged
• Automation Test Engineers
– Responsible for enhancing the Business Process
recorded by SMEs, typically fits a Consultant Profile
– Defining Variables for use in the Test and setup
RecordSets
– Build continuity in E2E Scenarios by linking several
processes
– Build centralized processes for use in cross functional
integration scenarios
– 2 to 3 Automation engineers per process area
• Central Test Management
– Responsible for Administration of Worksoft & User
Access Management and HPQC
– Establishes Best Practices, Naming Conventions and
Project Structures for use
– Coordinates the Test Planning & Execution
Responsible for management of issues, Risks, Scope
and Schedule
– One Test Lead and 2 Test Coordinators
• Worksoft Support Consultant
– Responsible for Installation & Setup of Worksoft
– Provides Training & Guidance to the Automation Test
Team during Planning, Build & Execution
– Resolves complex Test Requirements, Collaborate,
mentor and assist in Build Process
– 1 Worksoft Support Engineer was onsite for the
duration of Build.
Business SMEs and Automation Test Engineers Commit 25% of their time towards the
Automation Test Build Efforts during the Realization Phase in parallel to the other Project
Tasks to which they have been assigned

15. Confidentialand proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 15
Benefits Summary
15
Key Project Approach Drivers
Minimal Verizon staff to support project & better utilization of Consulting
resources
Significant integration requirements to 3rd party and legacy systems
Multiple sites for manufacturing and distribution
Highly successful operations cannot be impacted
Project Results
45 % of the Total Test scope was automated
70 E2E Processes [~ 237 Scripts] reduced Testing Cycle time by 35%
Fully validated implementation with 2 Integration test cycles in contrast
to traditional 3 cycle testing approach.
Reduced overall test execution time for each E2E scenarios from `35
Hrs to 1.5 Hrs and increased the number of data variants per scenario
Company Overview / Background
SAP Users: 99,000 in 55 Countries
The Uni-Fi Convergence program was approved by Verizon executive
management. Its objective is the consolidation of all wireline ERP
processes into 2 instances of SAP using the current nSAP system and
processes as the mandatory go-forward solution.
164 Core Test Scenarios with ~ 500 Data Variants involving 137 Testers
positioned globally
Scope of Services
ERP Implementation and Integration
SAP ECC 6.0 / SRM / BW / Legacy
FICO
SD
MM
WM
PP-PI
Project Systems
Asset Management
Business Intelligence
Key Success Factors
45% E2E Automated Tests executed during Week 1 of each test cycle.
Relentless focus on accomplishing milestones on schedule
Early test execution reduced the risk with late defect detection
Eliminated the needed for Test Script Programming specialists. Test build
engineers drawn from existing functional team
Management support for adherence to the Worksoft Test Automation tool &
IBM’s “Quad A” methodology
Quick Adoption, Team Training & Ownership
Automation Test foundation laid for future releases & benefits are being
realized

16. Confidentialand proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 16
Next Steps
• The initial Worksoft capabilities built for Release 3 was leveraged significantly for a
follow-on EdgeCast Release, a new Verizon acquisition onto nSAP.
• Expand the automation framework to the vSAP ERP platform
• Increase automation capabilities in the SRM, Portal and Workflow notifications which
constantly changes due to policy updates and technical upgrades.
• Considering deployment of “Lights-out” Testing and Certify Impact for future
implementations

17. Confidentialand proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 17
References
Verizon: Nick Germanotta, Uni-Fi Program Director nick.e.germanotta@one.verizon.com
IBM:
Bala Thangaraj, Global Business Services bthanga@us.ibm.com
Armando Ortiz, Partner Comms Sector armando.ortiz@us.ibm.com
Michael Thelen, Delivery Partner mthelen@us.ibm.com
Worksoft:
Harrison Schultz, Partner hschultz@worksoft.com
Chris Lynch, Partner clynch@worksoft.com