Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
- 1 -
Automating End To End
Scenarios
Presenters:
Sandra Alequin (Senior Manager, Allstate Insurance)
Monika Mehrotra (Sen...
- 2 -
Enterprise testing (ETest) strategy was established in pursuit of a
strategic objective: to improve agency and cust...
- 3 -
Executive Summary
Java
Applications
.NET
Applications
Siebel
Applications
Mobile
Applications
Open Web
Applications
...
- 4 -
 Over 200+ application from a mix of legacy (AS400, Mainframes) and
new technologies (.NET, Java)
600+ services on...
- 5 -
Validation points introduced across different layers for deeper and wider test coverage
5
Executive Summary
Java
App...
- 6 -
Executive Summary
• Implementation of trouble
shooting at service layer to
reduce effort spent root
cause analysis
•...
- 7 -
Complexity, size and time to execute E-Test regression suite
warranted the need to automate the regression suite
A...
- 8 -
QTP Automation
Framework
Service Validation
Framework
Frontend Layer
UI Components
Presentation Layer Logic
Backend ...
- 9 -
Switch of test machine is
required to continue execution
Stop for Batch Cycle
Resume after Batch Cycle
Data needs to...
- 10 -
 Tests were run on specific workstations to mimic customer and user
experience. So how to switch test machines to ...
- 11 -
Test Automation challenges addressed
Data Extraction
& Persistence
Multiple Machines
Execution
Execution Inline
wit...
- 12 -
12
# 1-Data Extraction & Persistence
Data Persistence in XML Files
DataPersistenceacross
machines
DataPersistence
a...
- 13 -
13
Test Execution Dependency- Across Batches
Customer Inquiry, Quote & New Business
Customer Agent Search :
(Mobile...
- 14 -
14
1A
1B
1C
E Test Release 1
HP QC Test Lab Folders
at Scenario level
HP QC Test Lab Folder at Release level
Batch1...
- 15 -
15
# 4 - CA LISA integration with HPQTP & HPQC
CA LISA
- 16 -
16
HP Quality
Center
Mainframe Web Java
Enterprise Automation Components
(Reusable Functions, Keyword functions, Fr...
- 17 -
Test Case Modeling
17
Application
Workflow
Payment
Work
flow
Data
Data1
Data
Work
flow
Work
flow
Data
Data2
Data3
D...
- 18 -
Test Case Modeling
18
ETest Implementation Approach
Assessment of ETest
Environment and Test
Scenarios
ETest Automa...
- 19 -
Executive Summary
Current State Post Implementation Benefits
Quality
• Test steps scoped
based on
available effort
...
- 20 -
Executive Summary
Additional Benefits of Integrated Automation Solution
Predictability
 Validation at backend laye...
- 21 -
Questions & Answers
- 22 -
Thank you.
Prochain SlideShare
Chargement dans…5
×

Automating End-to-End Business Scenario Testing

1 072 vues

Publié le

Allstate Insurance had a problem. While thoroughly testing each of their more than thirty business systems, they were still failing to provide good service to their clients, agents, and internal customers. The reason was simple. Implementing end-to-end business processes requires more than just running data through a set of separate systems. While focusing on automating unit, integration, and system testing, they had failed to consider the need for system-to-system integration tests―tests that would verify that their business systems passed data correctly, met interface expectations, and synchronized properly. Monika Mehrotra and Sandra Alequin describe how Allstate, with the assistance of Infosys, supplemented their existing test suites with a set of end-to-end tests that provided deeper test coverage, demonstrating proper system operation from beginning to end. In addition, Allstate implemented a test environment that more closely resembled their production environment, discovering defects that had previously escaped into daily operation. Learn the importance of end-to-end, not just piecemeal testing.

Publié dans : Logiciels
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

Automating End-to-End Business Scenario Testing

  1. 1. - 1 - Automating End To End Scenarios Presenters: Sandra Alequin (Senior Manager, Allstate Insurance) Monika Mehrotra (Senior Manager, lnfosys Ltd)
  2. 2. - 2 - Enterprise testing (ETest) strategy was established in pursuit of a strategic objective: to improve agency and customer experience with a goal of minimizing production problems and reducing the risk due to major application releases on critical business functions. ETest strategy implementation included establishing new processes, tools, team and environment. ETest consists of 200+ applications with over 600+ backend services and equally high number of interacting databases. 33 key business functions, which are highly integrated, are tested end-to-end prior to being put in production under this strategy. Enterprise Testing (ETest) Strategy at Allstate
  3. 3. - 3 - Executive Summary Java Applications .NET Applications Siebel Applications Mobile Applications Open Web Applications Mainframe Presentation Layer Middle-Tier Data Layer Mainframe Web Servers Application Servers LDAP MQ TIBCO SOAP REST JMS EJBs JDBC Store Procedure FTP Mobile Web Browsers Applications Enterprise Testing Environment – An Overview DatabaseDatabaseDatabase
  4. 4. - 4 -  Over 200+ application from a mix of legacy (AS400, Mainframes) and new technologies (.NET, Java) 600+ services on different technologies like SOAP, REST etc Unresponsive Backend Services can cause failures in tests and hence need constant pinging to keep them alive Time consuming Defect Root Cause Analysis due to large number of systems and integrations Deactivation of Test IDs post Data Refresh activities Complexities in Enterprise Testing Environment
  5. 5. - 5 - Validation points introduced across different layers for deeper and wider test coverage 5 Executive Summary Java Applications .NET Applications Siebel Applications Mobile Applications Open Web Applications Mainframe Database Presentation Layer Middle-Tier Data Layer Mainframe Web Servers Application Servers LDAP MQ TIBCO SOAP REST JMS EJBs JDBC Store Procedure FTP Mobile Web Browsers Applications Existing validation points Newly introduced validation points Solution to address E-Test Environment challenges DatabaseDatabase
  6. 6. - 6 - Executive Summary • Implementation of trouble shooting at service layer to reduce effort spent root cause analysis • Reduction in time spent on debugging and enable faster fixes for defects • Validation of middle tier components that otherwise are not tested as part of GUI testing scenarios • Deeper and wider coverage introduced into architecture with middle tier validation • Validated integration and interaction points ensuring quality in systems integration • Data navigating from one end to another in network is tracked for quality and reliability Smoother Integration Better Coverage Defect Isolationa b c Users - Presentation layer Headless clients consuming services Data Access Logic Data Utilities Data Sources Presentation – Middle Tier Integration Middle Tier – Data Layer Integration Service Service Service Service Service Service Service b a a c c a b Addition of Middle Tier Validation in E-Test
  7. 7. - 7 - Complexity, size and time to execute E-Test regression suite warranted the need to automate the regression suite An assessment on then E-Test processes were performed and an Integrated (frontend and backend) automation solution was arrived at to target improvement in efficiency and effectiveness of E-Test regression suite execution by giving deeper and wider test coverage. De-risked and cost effective 2-Phased approach was adopted to build out the integrated automation. Phase 1 was performed for feasibility analysis and devise solution for automation requirements. Phase 2 was implementing the solution and rollout Necessity for Integrated Test Automation Solution
  8. 8. - 8 - QTP Automation Framework Service Validation Framework Frontend Layer UI Components Presentation Layer Logic Backend Service Layer Service Orchestration Services Backend Data Layer Data Access Components Data utilities Data Validation using CA LISA E Test Environment Leverage LISA for early validation at service/data layer Validating UI layer on a more stable Backend Integrated Automation Solution HP Quality Center HP Quick Test Professional CA LISA CA LISA Backend (Service and Data layer) validation will be automated with CA LISA and Frontend (UI layer) validation will be automated with HP QTP. The solution will be orchestrated using HP QC and provides flexibility to validate layers in integrated or silo mode.
  9. 9. - 9 - Switch of test machine is required to continue execution Stop for Batch Cycle Resume after Batch Cycle Data needs to be captured and transferred between the test scripts/test sets LISA Scripts integration with QTP and HPQC for backend scripts execution Typical ETest End-to-End Scenario for Automation ? – Questions to be answered and solutions to be proven out Customer does inquiry and requests for a Quote Mobile Search Web Search Agent contacts customer and the quote is converted to a request for new business New business is received and reviewed by underwriters Home Office Staff verifications Underwriters Underwriter provides approved , New business documents generated and Customer access documents Document Management System Customer Access Endorsement confirmation Policy EndorsementRegistration Renewal Validations Validate Renewals processing after payments Policy Endorsements, Renewals and billing Endorsement Documents Renewal of Policies Renewal Billing Renewal Documents: Endorsement processing Renewals Payments Sample ETest Scenario ? ? ? ? Exclusive Agent Applications Independent Agent Applications Home Office Staff
  10. 10. - 10 -  Tests were run on specific workstations to mimic customer and user experience. So how to switch test machines to continue automated execution?  How to handle the execution of a single test case across multiple job batches?  How would the data required for execution across batch of same test case maintained throughout the execution of the case, which may span across days?  How to integrate HPQC (Test Management), HPQTP (UI Automation) and CA LISA (Service layer validation) tools to perform an integrated test run? Automation Challenges in ETest End-to-End Scenarios
  11. 11. - 11 - Test Automation challenges addressed Data Extraction & Persistence Multiple Machines Execution Execution Inline with Daily Batch cycles LISA Scripts integration with QTP and HPQC 1 2 3 4 Customer does inquiry and requests for a Quote Mobile Search Web Search Agent contacts customer and the quote is converted to a request for new business New business is received and reviewed by underwriters Home Office Staff verifications Underwriters Underwriter provides approved , New business documents generated and Customer access documents Document Management System Customer Access Endorsement confirmation Policy EndorsementRegistration Renewal Validations Validate Renewals processing after payments Policy Endorsements, Renewals and billing Endorsement Documents Renewal of Policies Renewal Billing Renewal Documents: Endorsement processing Renewals Payments Sample ETest Scenario 2 1 3 4 Exclusive Agent Applications Independent Agent Applications Home Office Staff
  12. 12. - 12 - 12 # 1-Data Extraction & Persistence Data Persistence in XML Files DataPersistenceacross machines DataPersistence acrossbatchruns Customer Inquiry, Quote & New Business Customer Machine: Customer Inquiry and Begins Property Quote: (Mobile and Web) Agent Machine: New Business Purchase Under Writer Machine; New Business Agent Machine: Endorsement : Agent Customer Machine: Customer: Login to “Customer Account", Register Batch 1: Customer Inquiry and Begins Property Quote: (Web) Batch 2: New Business Purchase: Agent Batch 3: New Business : Agent Data is getting read by different machines from XML Reference # is persisted in XML and used by script for Policy # generation 1 1 Policy # is persisted used by script and later used other scripts2 2 1 2 •Data is persisted in Shared LAN drive which is accessible by all machines •Data is persisted in form of XMLs for each test case. •XPATH Queries are employed for faster data retrieval
  13. 13. - 13 - 13 Test Execution Dependency- Across Batches Customer Inquiry, Quote & New Business Customer Agent Search : (Mobile and Web) Customer creates Quote Customer Inquiry, Quote & New Business Manage Agency by Agent Purchases quote and email validation Validation of New Business Batch 2 • Automated Script generates Execution Control XML for each Test Case • Test Script execution status with other details are captured in XML structure • Execution sets Test Set_Status “Y” in case of Pass “N” for failure • Next Test Script reads XML data and determines if execution is required or not Invoke Next Batch test case if Status=“Y” Stop Execution for test case if Status=“N” STOP Determine Execution is required/not from XML Batch 1 # 2 - Execution Inline with Daily Batch cycles Cross Test Set Dependency with XMLs
  14. 14. - 14 - 14 1A 1B 1C E Test Release 1 HP QC Test Lab Folders at Scenario level HP QC Test Lab Folder at Release level Batch1 Batch2 BatchN… Batch1 Batch2 BatchN… Batch1 Batch2 BatchN… HP QC Test Sets at Batch run levels Each Test Set contains specific test scripts to be run as part of E Test Case HPQC identifies available machine in the host group and triggers execution on the available machine within the group. Upon execution completion, test results are reported back to quality center and execution is moved to next test script Machine Groups HP QC Test Cases Batch 1 for 1A contains scripts to be executed before batch cycle process runs #3 - Multiple Machines Execution Hosting in HPQC
  15. 15. - 15 - 15 # 4 - CA LISA integration with HPQTP & HPQC CA LISA
  16. 16. - 16 - 16 HP Quality Center Mainframe Web Java Enterprise Automation Components (Reusable Functions, Keyword functions, Framework Driver Engine) Platforms Under Test Automation Framework Remote Machines Testers Java Specific Components Web Specific Components Mainframe Specific Components Framework to handle Application Diversity
  17. 17. - 17 - Test Case Modeling 17 Application Workflow Payment Work flow Data Data1 Data Work flow Work flow Data Data2 Data3 Data1 Data2 Data3 Data1 Data2 Data3 Modularizing the application workflow by creating multiple reusable Business Process Components (BPCs) Workflow created in excel using action keywords Multiple data sets can be mapped to the same workflow to easily create different data oriented test cases Customer Registration Agent Search Diagrammatic representation of test case modeling Test Case modeling with the framework
  18. 18. - 18 - Test Case Modeling 18 ETest Implementation Approach Assessment of ETest Environment and Test Scenarios ETest Automation Feasibility Analysis ETest Implementation Design ETest SOA Validation Implementation ETest SOA validation rollout ETest UI Automation Feasibility Analysis ETest Automation Unique requirements identified ETest Automation Solution Development for requirements Proof Of Concept for ETest Automation Complete Automation Implementation in ETest Rollout of Automation Testing for ETest
  19. 19. - 19 - Executive Summary Current State Post Implementation Benefits Quality • Test steps scoped based on available effort • End to End Business scenario validated in base state • Volume test steps execution was increased by 3 times • End to End Business scenario implemented in multiple states • 200% improvement across Tests and ROCs • Improved coverage at backend layer • Increased and consistent validation • Multiple Regression Runs in current window Time • 3-week window to execute E2E tests • 1-week window to execute E2E tests • 33% reduction in the execution window • Creating window for more execution • Reduced turn around time on defect resolution Cost • High manual effort to rerun failed scenarios • Overall team is reduced by 40% • 30% reduction in execution cost • Early validation • Reduced issues and effort during End to End • Lessen manual effort with automation ETest Automation - Post Implementation Analysis
  20. 20. - 20 - Executive Summary Additional Benefits of Integrated Automation Solution Predictability  Validation at backend layer resulting in early defect detection  Post implementation UI execution happens on a validated backend layer Portability  Solution can be deployed on E-Test or other environments with minimal effort Coverage  Improve coverage by introducing more validations at all layers of validation Modularity & Scalability  Deploy solution in packages to enable selective run  Ability to run automation scripts for specific scenario category and/or applications.
  21. 21. - 21 - Questions & Answers
  22. 22. - 22 - Thank you.

×