The document discusses how testing tools have evolved along with the testing lifecycle to bring more structure and efficiency. It provides several examples of projects that transitioned from disorganized testing methods like spreadsheets to more integrated testing tools. This allowed them to better manage requirements, automate test execution, and provide reporting. While no single tool is perfect, selecting one suited to the project's needs and ensuring users are trained can formalize the testing process and improve outcomes.

1. ©2009Ness Technologies – Proprietary and Confidential
How testing and testers evolved within the time and benefits
of testing tools during mature testing lifecycle.
Testwarez 2009, Krakow
Testing Lifecycle: Don’t be a fool,
use a proper tool!
Zdeněk Grössl & Lucie Riedlová
20. 10. 2009

2. www.ness.com
Our aim: to use the mature Testing Lifecycle
Testing is strategic for us
Over 2.500 testers (18 countries worldwide) sharing their corporate
knowledge
Different modes – test labs, consulting, custom development, SAAS
Proof? Satisfied customers around the world
• Telco
• IT
• Public and Military
• In CZ customers from TOP 100
Let’s see that “Lifecycle”
Testing LifecycleTesting Lifecycle

3. www.ness.com
What do we mean by “Lifecycle” :-)What do we mean by “Lifecycle” :-)

4. www.ness.com
From random clicks to Testing LifecycleFrom random clicks to Testing Lifecycle
test coordinadion
progress of testing
connection to management
test cases
RQ coverage
test scripts and test data
management
automatization
managing test team
reporting about status
test functionality and
requirements before
release bug report
inefficient testing
difficult to reproduce tests
uncoordinatefd action
feedback on test process
imputs for next iteration
Tools can help a lot
!
How to handle this
?

5. www.ness.com
Test ManagementTest Management

6. www.ness.com
Testing tools – Test managementTesting tools – Test management
Dynamic changes on smaller projectDynamic changes on smaller project
Project:
Financial sector
Testing implemented from scratch, about 12 team members
Dynamic development process
Testing documentation in Excel sheets
Sudden change: Increase of RQs
40% new people hired
Issues with versions of Excel sheets
Information about changes was not
reliable
A need to formalize testing process
Solution: Tool for test Management
Easy to setup
Online access
Suitable for smaller teams
Keeps history of records
SPIRA TEAM was selected as pilot
Benefits:
Testing was formalized
History and more detailed information about testing process
SPIRA extended to other projects

7. www.ness.com
Testing tools – Test management – screenshotsTesting tools – Test management – screenshots
Example: SPIRA TEAM
Project Home
My Page

8. www.ness.com
Testing tools – Test managementTesting tools – Test management
Too much extra work at world’s HW leaderToo much extra work at world’s HW leader
Project:
UAT testing of world-wide application for Sales (12.000 users)
Two successive releases were tested, about 1 year between releases
Role: Regional Test Coordinator (40 testers), reporting about progress
Complete Worldwide team: 140 members
First release: Shared Excel files
Too much extra work
Three different locations to login into before
testing each day
Reporting for management: combine excel
sheets and reports from Bugzilla
Take care about testers (motivate, coach,
inform about bug fixes…)
Extremely challenging!
Second release: HP Mercury QC
A pro tool: all in one; no multiple logins
Simple administration of all modules
User-friendly testing
Easy overview of testing progress,
Bug tracking and traceability
Sophisticated reports for management.
Benefits:
More effective testing on such a large project.
Clear overview for management
Focus on testing, not on extra activities!

9. www.ness.com
Testing tools – Test management – screenshotsTesting tools – Test management – screenshots
Example: Evolution from Excel to Mercury QC

10. www.ness.com
Testing tools – Test managementTesting tools – Test management
SummarySummary

11. www.ness.com
Test AnalysisTest Analysis

12. www.ness.com
Testing tools – Test Analysis and DesignTesting tools – Test Analysis and Design
We got lost in Test CasesWe got lost in Test Cases
Project:
Financial sector (saving bank)
Deliver complete QA for new module of client’s internal system
One customer, three dislocated suppliers
Totally about 70 new RQs (about ¼ of the whole system)
All in Excel sheets
(RQ, TC, test data)
Excel sheet too complex
Difficult to review test coverage
Poor reporting options
Issues with access to files
Several errors occurred
Requirements on the tool
Achieve and review RQ coverage easily
Supports evaluation of TCs
User-friendly environment
Quick and easy setup
Accessible worldwide
SPIRA TEAM selected
Benefits:
Administrative / human errors avoided
Great overview for test analyst
Module delivered with remaining 6 minor bugs (totally over 140)
Transparent for client and increase of our credit

13. www.ness.com
Testing tools – Test Analysis – screenshotsTesting tools – Test Analysis – screenshots
Example: SPIRA TEAM
Map TC to RQ
RQ coverage overview

14. www.ness.com
Testing tools – Test Analysis & DesignTesting tools – Test Analysis & Design
SummarySummary

15. www.ness.com
Test DesignTest Design
15

16. www.ness.com
Testing tools – Test Design and AnalysisTesting tools – Test Design and Analysis
Regression testing under stressRegression testing under stress
Project:
SPIRA TEAM as testing tool on the project (Finance)
Releases 8x per year, hot-fixes need to be released within 8 hours
Much time spent on regression testing
Need of change:
Regression testing not 100%
effective under stress
An error was delivered to the
customer
More formalized regression tests
were not good solution
We need tool for automated testing
Options: Test Complete, Win Runner
Situation on project did not enable it.
Good luck: our SW-coder recommended
Selenium – add-on for Firefox
Benefits:
Future errors avoided
Connection with SPIRA to automatically record test runs
It is useful to discuss testing issues with developers as well

17. www.ness.com
Testing tools – TestTesting tools – Test DesignDesign –– screenshotsscreenshots
Example: SPIRA TEAM together with Selenium
► Selenium IDE – write and run tests in Firefox
► Selenium Remote Control – runs tests in multiple browsers and platforms
(IE7, Safari, Opera, Firefox, Windows, Linux, Solaris, others)

18. www.ness.com
Testing tools – Test Design and AnalysisTesting tools – Test Design and Analysis
More than was requested (by SilviaMore than was requested (by Silvia ValientovValientováá))
Project:
Biggest insurance company in CZ
33 methods, 6 web services were tested separately
What was requested?
Test the methods like a process
Customer: Buy us a new testing tool!
SoapUI was known tool for customer
Why SoapUI
The old tool has requested functionality
Property Transfer and Xpath Assertion
Wizard
Link responses with requests
Support of E2E testing
Benefits:
Client can now easily perform this testing
No extra costs required to purchase licenses
Easy upgrade from Integration testing to E2E testing
We found what customer really NEEDS, not what he just REQUESTS

19. www.ness.com
Testing tools – TestTesting tools – Test DesignDesign –– screenshotsscreenshots
Example: The SOAP UI Story
► Used for Web Service Testing

20. www.ness.com
Testing tools – Test Design and AnalysisTesting tools – Test Design and Analysis
Sometimes just look around for a tool!Sometimes just look around for a tool!
Project:
Internetbanking system (Lead Czech saving bank)
Load testing was newly required
Tested system was developed by NESS Czech
Requirements on the tool
Simulate simultaneous access of
300 users per 10 minutes
Minimum time and costs to set up a
tool
Restriction on client’s side
Why Visual Studio 2005 (for Testers)?
Tested application was programmed in this
environment
Developers were familiar with the tool
Quick implementation and results in one
week
Benefits:
Developers can be very good test designers
Scope of our services for the client broadened

21. www.ness.com
Testing tools – Test Design & AnalysisTesting tools – Test Design & Analysis
SummarySummary
automated
testing
language &
environment
independent
connection
to other tools

22. www.ness.com
Test ExecutionTest Execution
22

23. www.ness.com
Testing tools – Test ExecutionTesting tools – Test Execution
Testing scope broadened (by Jaroslav StrharskTesting scope broadened (by Jaroslav Strharskýý))
Project:
Major Telco company in CZ
Excel for RQs and TCs
Originally only 2 testers planned
Need of change:
25 testers involved
Excel files too complicated
Several bugs got misrouted
Testing with HP Mercury Quality Center
Testers complained about difficulty of the
tool and level of formalism
Monitoring of users was necessary
Formalized and free tests were put in place
Benefits:
Inform business testers about fixed bugs they discovered: they will be satisfied
Training for testers takes approx. the same time regardless of a tool
Testing with this tool is as quick as without a tool. But consider the benefits!

24. www.ness.com
Testing tools – TestTesting tools – Test ExecutionExecution –– screenshotsscreenshots
HP Mercury Quality Center

25. www.ness.com
Testing tools – Test ExecutionTesting tools – Test Execution
No time for new toolsNo time for new tools
Project:
TCs and bugs maintained in Word and Excel
Communication to developers was face-to-face
Bugs were fixed almost immediately
Need of change:
Model does not work under stress
Big volume of reported bugs (+300%)
Management required reporting
Only 3 weeks to deliver the testing!
Why JIRA from Atlassian?
JIRA is used in NESS as internal
tracking tool (over 750 users)
Every employee is familiar with it
Skillful administrators
Benefits :
No more misunderstandings
Reports for management
All set up in less than 1 day

26. www.ness.com
Testing tools – TestTesting tools – Test ExecutionExecution –– screenshotsscreenshots
JIRA by Atlassian

27. www.ness.com
Testing tools – Test ExecutionTesting tools – Test Execution
SummarySummary

28. www.ness.com
Summary? There is no perfect tool!
► Every tool suits something
► If the tool is TOO complex and TOO perfect: Is it still easy to maintain?
Is there anything else we should be aware of? Answer these questions
about testing tools:
► Easy to install?
► Friendly to use?
► Even for non-IT users?
► Any additional features?
• Notifications,
• Attachments, screenshots
• Reporting options
• Upgrades
• …
► Could you customize the tool?
► When you tried it, does it serve you well?
Overall summaryOverall summary

29. www.ness.com
What do we use?
► We – Lucie and Zdenek – like SPIRA (and several our customers too  )
We use tools from our partners (Atlassian, HP, IBM, Inflectra, Microsoft…)
Use tool best for the purpose and for the client
► Excel? OK, but educate clients! And remember their maturity!
Which tool should you use? See questions on previous slide!
► Summarize your needs
Don’t you know your needs?
► Talk to someone who helps you define them
Most powerful tool: OPEN MINDS!
And what about testing at NESS Czech?And what about testing at NESS Czech?

30. www.ness.com
Complete QA solution in one package
Server and thin-client-based application
Very user-friendly environment
Rich reporting options
Document repository
Option to connect to LDAP
Positive feedback from clients
Even from business (non-IT) people
After one year over 30 projects and over 60 external users
SPIRA TEAM by InflectraSPIRA TEAM by Inflectra
http://www.inflectra.com

31. www.ness.com
bug tracking
project management
agile software development
technical support and help-desk
used by more than 10 000 organizations
create any kind of issue in seconds
keep your projects and issues organized
track all activities in your project
designed with both business and technical users in mind
easy to administer and customize
Because you’ve got issuesBecause you’ve got issues

32. www.ness.com
Our question: What did you take from this presentation?
► There is no perfect tool!
► Discuss and ask questions before you decide to use certain tool!
► Ask those who know about this stuff…
We are ready for your questions…
… and for the chat about testing – later :-)
Questions and AnswersQuestions and Answers
It looks like end...
but it is rather start 