When an organization moves to agile software development methodology it becomes easier to forget about agile performance testing. Many organizations depend on PushToTest to educate agile teams on agile load and performance tests. Agile got developers and testers talking to one another. Agile performance testing gets business managers and technical team members communicating.

1. Managing Software Projects With Agile Methodology
AGILE PERFORMANCE TESTING
Open Source Software (OST) Adoption
Todd Bradfute, bradfute@pushtotest.com
Frank Cohen, +01 408-871-0122, fcohen@pushtotest.com
August 2, 2011

2. Open Source Test Automation
About PushToTest
‣Our Mission: We Test, For A More Reliable World
‣Make It Easier To Move From Manual To Automated Testing
‣Support Advanced Testing, including Agile, Load, Integration
‣Open Source Testing (OST) For Everyone
2

3. Open Source Test Automation
Intellectual Property of PushToTest. (c) 2009 All rights reserved.
Your Team
‣fcohen@pushtotest.comFounder, PushToTest
Frank Cohen, CEO and
‣bradfute@pushtotest.com Services, PushToTest
Todd Bradfute, Professional

4. Open Source Test Automation
Slides on Twitter
‣http://twitter.com/fcohen
‣Please retweet
‣Please follow me
4

5. Open Source Test Automation
Next let’s talk about
Agile Open Source Testing
‣Test Driven Development
‣Roles and Fungability
‣Testing and Test Management
5

6. Open Source Test Automation
Agile Is Seldom Achieved
‣Test-First development
‣Resource interchangeability (aka “fungibility”)
‣Commercial tools lock test tools to “testers”
‣OST lets everyone on the team be more involved.

7. Open Source Test Automation
Overview
‣Test Driven Development
‣Resource Fungibility
‣OST as the Lingua Franca
‣Certification using OST
‣Demonstrable Results
‣Detailed Results
‣Provable Results

8. Open Source Test Automation
Test Driven Development
‣Write the test before implementing the feature
‣To expand lookupZip(String addr) to return zip+4
• First: write test that expects #####-#### as output
• Second: Run it and watch it fail because output is #####
• Third: Change implementation
• Fourth: Run it and watch it pass
‣You’re finished adding the feature when this (and all other) tests pass
‣OST enables TDD

9. Open Source Test Automation
Resource Fungibility
‣Any person can work any task
‣Most Agile teams don’t even try
‣The prize is extreme velocity and flexibility!

10. Open Source Test Automation
OST - The Lingua Franca
‣Cross testing ... i.e. person A’s code is tested by person B
‣A and B are both members of the agile team
‣Both A and B may be developers, but not testing their own code
‣Using OST the team selects a standard tool set
‣On the current Best Buy project the primary OST is soapUI

11. Open Source Test Automation
Certification Using OST
‣At Best Buy the organization has a “certification” process
‣Starts with the sub-teams own test artifacts
‣Reviews plans and execution output
‣tests
Responsible for repurposing tests as concurrency/load/performance

12. Open Source Test Automation
Demonstrable Tests
‣Reports are saved to the database
‣The Agile process is enabled by using Rally
‣Process flow:
‣User Story -> Test Case -> Test Card -> Test Results -> Test Output

13. Open Source Test Automation
Rally/TestMaker integration

14. Open Source Test Automation
Easy Control For Everybody
PushToTest TestCards - Best Buy
fcohen signed-in, help, sign-out
Welcome To TestMaker TestCards
TestMaker Enterprise 6.0.1
3rd Party Connector View Results
Delete
Licensed for 50 virtual users
Associate Rally User Story
and 10 service monitors Create Copy
New TestCard
Assign To Rally
List TestCards
My TestCards
3rd Party Connector
TSH Mock Services
Address Lookup
Verizon Registration

15. Open Source Test Automation
Detailed Results
‣Seeing a pass is comforting, but how do you know it really did?
‣Use drill-down to see details on a passing case
‣Drill down from a fail to the actual error messages
‣Anyone on the team can review any pass/fail results

16. Open Source Test Automation
Transaction Report

17. Open Source Test Automation
Drilled Down Detail
‣Input:
‣http://server1.bestbuy.com/TSH/v1/offers HTTP/1.1
‣…
‣{ "variable1": "value1", "variable2": "value2" …}
‣Output:
‣HTTP/1.1 500 Internal Server Error
‣...
‣<response>
‣ <error>Unrecognised resource: offers</error>

18. Open Source Test Automation
Provable Results
‣Test results should be 100% reliable, but what’s the safety net?
‣Run it again, Sam.
‣Rally -> Test Case -> TestCard -> Execute

19. Open Source Test Automation
Next let’s talk about
Agile Performance Testing
‣Test Orchestration
‣Repurposing Methodology
‣Cloud and Grid Deployment
19

20. Open Source Test Automation
Why An OST Alternative?
‣the Internet Professional does a great job for testing a small part of
HP QuickTest
‣Functional Testing (Smoke, Regression, Integration)
‣Windows using ActiveX
‣Flash, Flex
‣MS Internet Explorer
‣Web 2.0 with Coding Effort
‣Best Money Can Buy
20

21. Open Source Test Automation
OST For Everywhere Else
‣Open Source Testing Tools
‣TestMaker Object Designer
‣Selenium
‣Sahi
‣Unit Tests (Java, Ruby, Python, PHP)
‣Record in WebKit based browsers (Chrome, Safari, Opera), IE, Firefox
‣Agile Testing, Agile Performance Testing and Production Monitoring
‣Rich Internet Applications (RIA,) Mobile, Pads
‣OST For Everyone (Developers, Testers, Business Managers)
21

22. Open Source Test Automation
OST Architecture
Application Under Test PTTMonitor
Web, RIA, BPM, SOA CPU, Net, Memory
Test Injector
Bundles ScriptRunners Data Production Libraries
MySQL, HtmlUnit Sahi, Selenium, SoapUI, Java csv files, rdbms queries
Test Operation Console
Designer Grid and Cloud Repository Results Analysis
Record/Playback Test Deployment MySQL RDBMS Business Intelligence Tool
Continuous Integration Test Management
22

23. Open Source Test Automation
Where To Go From Here
‣Watch A Screencast and Tutorial
‣http://www.pushtotest.com/screencastcentral
‣Attend An Open Source Test Workshop
‣http://workshop.pushtotest.com
‣Ask For A Proposal on Licenses, Training, Support, Consulting
‣Call +01 408 871 0122, sales@pushtotest.com
23