Within minutes, organizations can incur losses totaling tens of millions of dollars due to unplanned production outages that are the result of poor performance. This presentation will give you an understanding of critical success factors that will allow you to implement cost-effective and scalable performance test initiatives.

1. 1 © 2014 Method360, Inc. All rights reserved.
Performance Testing:
Eliminate System Outages
and Save Millions
PRESENT ED BY
Sunny Hiran

2. 2
©2014 Method360, Inc. All rightsreserved.
Method360
Method360
PERFORMANCE TESTING
Introduction
SUNNY HIRAN
Director
Quality Assurance
Through 2015, 80% of the outages impacting mission-critical services are expected to be caused by people and process issues, according to Gartner. More than half of these outages will be attributed to change, configuration, and other related issues.
(Top Seven Considerations for Configuration Management for Virtual and Cloud Infrastructure).
Only 22% of organizations surveyed by Gartner, however, have deployed the full complement of fault, performance, and configuration management capabilities. Performance and configuration capabilities are expected to lag through 2017 according to a recent Gartner research report.
(I& O Teams Must Proactively Develop Three Core Network Management Disciplines)

3. 3
©2014 Method360, Inc. All rightsreserved.
Method360
Method360
The Stakes Are High
In a matter of days if not hours, organizations can incur losses totaling tens of millions of dollars due to unplanned production outages that are the result of poor performance.
Even if software conforms to business and functional requirements, in the absence of rigorous performance and scalability validation, credibility and competitiveness are at stake. Once lost, these are difficult to recover.
PERFORMANCE TESTING

4. 4
©2014 Method360, Inc. All rightsreserved.
Method360
Method360
Agenda
•Key Takeaways
•Performance Nightmares
•What is Performance Testing?
•Locking down performance scope and understanding what to simulate
•Tool options in the performance marketplace, including proven open source solutions
•Key strategies that will ensure the responsiveness of your mission critical systems
PERFORMANCE TESTING

5. 5
©2014 Method360, Inc. All rightsreserved.
Method360
Method360
Key Takeaways
•Performance outages are preventable!!
•Understand that performance outages impact all production support landscape models, both hosted and in-house solutions. (Demand that your hosted provider demonstrate how your system will perform under load & DR)
•Performance failures have a real impact on organizations both in terms of credibility and lost revenue
•Performance testing should be a mandatory testing activity for any mission critical solution
PERFORMANCE TESTING

6. 6
©2014 Method360, Inc. All rightsreserved.
Method360
Method360
Performance Nightmares
TWITTERGOES DOWN FOR 'MOST USERS' IN LONGEST OUTAGE SINCE IPO
According to previous Status blog updates, it was the longest outage for Twitter's mobile and Web platforms in at least the past year, when most issues have been resolved in 25 minutes or less.
Twitter had trouble in its early years with frequent outages and improved the stability of its service even as users and tweet volume skyrocketed, helping to convince users and investors about the viability of the microblogging service.
GOOGLE APP ENGINE
For four hours between10:30-14:30 EST on October 26th, 2013 Google App Engine failed to deliver about 50% of its requests. As a service used by hundreds of thousands of developers to create applications, this outage was felt heavily across the web.
ROYAL BANK OF SCOTLAND
The IT staff was responsible for system failures that affected 17 million customers of RBS, NatWest and Ulster Bank. The problem occurred during maintenance on systems which caused an error in their automated batch scheduler and processor. This prevented millions of customers from receiving or making payments, and lasted for more than a week! The outage cost RBS a whopping £125 million!
PERFORMANCE TESTING

7. 7
©2014 Method360, Inc. All rightsreserved.
Method360
Method360
Performance Nightmares
BANK OF AMERICA
On September 14th, 2013 problems started with BofA's website with the message "some of our pages are temporarily unavailable" on the homepage. The issues only got worse. 72 hours later, the majority of users were unable to connect to Bank of America's website due to slowness and time-out failures. The problems weren’t resolved for another 24 hours
BofA attributed the outages to month end traffic along with a code release.
KOHL’S
Last year, Kohl's ran a massive online special for Black Friday shoppers, offering early bird specials, 20% off sales prices, and free order shipping. The bargains started the day before Thanksgiving and ran until 3pm on Black Friday. However, given the surge in traffic, the Kohl's website experienced an outage for several hourson Thanksgiving evening. As the heaviest online traffic week of the year, a fewhours of downtime can be incredibly costly for online retailers.
AMAZON WEB SERVICES (AWS)
On October 22nd, 2013 a memory leak and failed monitoring system caused Reddit, Foursquare, Minecraft, Airbnb, Heroku, GitHub, imgur, Pocket, HipChat, Coursera and a number of others to go down. The outage lasted for six hours until service was restored. Finally, on Christmas eve, Netflix went down until Christmas morning due to an elastic load balancing failure in AWS.
PERFORMANCE TESTING

8. 8
©2014 Method360, Inc. All rightsreserved.
Method360
Method360
What is Performance Testing and Why do it?
PERFORMANCE TESTING
“Performance testing is in general testing performed to determine how a system performs in terms of responsiveness and stability under a particular workload. It can also serve to investigate, measure, validate or verify other quality attributes of the system, such as scalability, reliability and resource usage.”

9. 9
©2014 Method360, Inc. All rightsreserved.
Method360
Method360
Performance Scope
PERFORMANCE TESTING

10. 10
©2014 Method360, Inc. All rightsreserved.
Method360
Method360
Performance Tools & Monitoring Options
Borland® Silk Performer®
Web Load
Performance Testing Tools
MS VSTS
Live Monitoring Tools
PERFORMANCE TESTING

11. 11
©2014 Method360, Inc. All rightsreserved.
Method360
Testing the ability of the system to process required datavolumes
Stressingthe system up to and beyond its breaking point
Captures performance of a newsystemto a knownstandard
Identify problems over a periodof timeby executing continuous repetitive transactions
Exercisingfailure points and verifying backup systems can be deployed
Measuring systemresponse times when measured under a specified load
Performance Testing Services
Load
Stress
Failover
Baseline
Endurance
Volume
Performance Strategies
PERFORMANCE TESTING

12. 12
©2014 Method360, Inc. All rightsreserved.
Method360
Method360
Implementing Performance Testing
•Library updates
•Lessons learned and best practices
•Ongoing validation and support
05_Closure
•End-to-end load/ performance test executions
•Monitoring, tuning & improvement
•Defect Management
•Test summary report
04_Test Execution
•Test scripts development
•Test data setup
•Test environment setup
•Test scenarios development
03_Test Artifacts Creation
•Test strategy, plan, tools setup
•Determine monitoring requirements
02_Preparation
•Scope, tools, transaction distributions resourcing, feasibility analysis, LOE
•Kickoff and KT sessions with business
•Finalize success criteria
01_Planning
PERFORMANCE TESTING

13. 13
©2014 Method360, Inc. All rightsreserved.
Method360
Method360
Q&A
PERFORMANCE TESTING

14. 14
©2014 Method360, Inc. All rightsreserved.
Method360
Contact Info
QUALITY ASSURANCE
SUNNY HIRAN
415.535.5453
shiran@method360.com
PERFORMANCE TESTING
To learn more about how Method360’s solution offerings can benefit your company, please contact us:

15. 15
©2014 Method360, Inc. All rightsreserved.
Method360
Thank You
END-TO-END SERVICES. HIGH VALUE SOLUTIONS.
PERFORMANCE TESTING