Dollar Thrifty Automotive Group, Inc. (DTG) customers are increasingly choosing the Internet as the primary way to rent vehicles from the Dollar Rent A Car and Thrifty Car Rental Web sites. Recently DTG undertook a significant redesign initiative for its two Web sites to optimize customer experience ahead of its busiest summer travel season and used Gomez’s web load and performance testing solution to validate their efforts. Attendees of this hands-on Webinar will see a Gomez Reality Load product demonstration and learn about the steps DTG took to validate peak performance for all internal and external components including Content Delivery Networks (CDNs), ads, analytics and ecommerce platforms, delivered across the Internet to its customers’ browsers. This Webinar will cover:
•How Dollar Thrifty geared up for their peak summer season.
•How a new style of load testing enables organizations to “walk in their customer’s shoes” and find problems before end-users find them.
•Best practices for identifying and resolving Web performance issues across the entire Web application delivery chain, inside and outside the firewall.
•Testing approaches that don’t require costly hardware or software investments.
•How to uncover geographical response time discrepancies that may surface under load.
Apidays New York 2024 - The value of a flexible API Management solution for O...
Best Practices for Identifying Web Performance Issues Before Your Customers Do- A Gomez/Dollar Thrifty Web Performance Testing Case Study
1. Best Practices for Identifying Web Performance Issues Before Your Customers DoA Gomez/Dollar Thrifty Web Performance Testing Case Study: Imad Mouline, CTO, Gomez Emanuel Daniele, Sr. Solution Engineer, Gomez Jim Arrowood, DTAG, Director, Web Development and Architecture
2. Agenda Reality Load Overview – Imad Mouline Dollar Thrifty Case Study – Jim Arrowood Product Demo – Emanuel Daniele
3. The Challenge of Delivering Quality Web Experiences Traditional testing: “OK” …user is NOT happy The Web Application Delivery Chain 3rd Party/Cloud Services Browsers and devices Users Users Local ISP Load Balancing Load Balancing Web Servers Web Servers App Servers App Servers Internet DB Servers DB Servers MajorISP Mobile Carrier Storage Storage Mobile Components Mobile Components Content DeliveryNetworks Traditional zone of control
28. Low cache hit rateThe Challenge of Ensuring Quality Web Experiences Traditional testing: “OK” …user is NOT happy Traditional zone of control Zone of customer expectation Zone of customer expectation
29. Gomez Load Testing: On-Demand Realistic Load Testing from Browser to Data Center Backbone Last Mile Real-world load Find user experience breaking points Accurately measure response time High volume load (HTTP, Browser) Find infrastructure breaking points Define capacity headroom 100,000+ consumer- grade desktops 168+ countries 2,500+ ISPs Major mobile carriers around the globe 100+ commercial-grade nodes & data centers
30. Gomez Network: The World’s Most Comprehensive Performance and Testing Network Backbone Virtual Test Bed Gomez Last Mile Web Performance Management and Load Testing 100+ locations Cross-Browser Testing 500+ browser/OS combo’s 5,000+ supported devices Web Performance Management and Load Testing 100,000+ locations
31.
32. Self-service, full turnkey solution, or tailored to meet your needs with Gomez Professional Services offerings
33. Tests from an “Outside-in” customer point of view, with drill down to all web application components
41. Dollar and Thrifty have over 600 corporate and franchised locations in the United States and Canada, operating in virtually all of the top U.S. and Canadian airport markets.
42. The Company’s approximately 6,400 employees are located mainly in North America, but global service capabilities exist through an expanding international franchise network. 9
46. Dollar Thrifty significantly redesigned both of our websites in 2008 leaving question as to how the sites will perform in the peak of the 2009 summer
47. Internal load tests using traditional methods in test and staging environments did not provide the needed confidence12
48. The Gomez Relationship Dollar Thrifty has utilized the Gomez ActiveXF platform for several years Synthetic Tests of Key Business Processes Monitors for response time (Home Page Load & Reservation Process) Monitors for successful execution (Home Page & Reservation Process) 13
49. Gomez Reality Load In Early 2009, Dollar Thrifty teamed with Gomez to launch the first external load test of the company’s infrastructure and key software platforms utilizing Gomez Reality Load 14
50. The Goals Regardless of how much load, where is our weakest point? Can we handle our previous year’s peak +25% traffic? 15
51. Identify Core Business Processes How do the vast majority of the consumers interact with the site? Use 80-20 rule Shop for Rates Make Reservation Modify Reservation Cancel Reservation 16
52. Distribute Core Business Processes By percentage, how do consumers interact with the site? Shop for Rates – 70%* Make Reservation – 15%* Modify Reservation – 10%* Cancel Reservation – 5%* *Hypothetical 17
53. Forecast Visitors Based on historical web analytic data and the goal of the test, compute the intended visitor capacity of the site within an hour 10,000 visitors per hour at peak historically* If we intend to support peak +25% then our visitor forecast is 12,500 visitors *Hypothetical 18
54. Identify the Needed Scripts Throughput Tests – Where are we weakest? One script per core business process with no “think-time” Capacity / Load Tests – How much load can we legitimately handle? One script per core business process with “think-time” 19
66. As tests are executing ensure all internal monitoring tools in all tiers of the application are capturing as much detail as possible without skewing results
74. We were able make a simple configuration change to resolve the “sticky sessions” issue
75. We were able to move Thrifty middle-tier boxes into the Thrifty.com front-end and Dollar.com middle-tier to better distribute our load with no additional expense
76. When time and budget allows, retest to ensure mitigating actions resolve the issue and ensure no additional bottlenecks have appeared25
78. Wrap Up Questions & Answers Check back on QA Forums To find out more about Reality Load: cmason@gomez.com mgil@gomez.com Product Information http://www.gomez.com/products-solutions/products/load-testing/ 2 Minute Explainer http://www.gomez.com/resources/video-library/gomez-reality-load-testing-two-minute-explainer/
81. Load tested DB to improve performance in anticipation of another traffic spike3rd Party/Cloud Services Browsers and devices Local ISP Users Load Balancing Load Balancing Web Servers Web Servers 1 App Servers App Servers Internet DB Servers DB Servers MajorISP Mobile Carrier Storage Storage Mobile Components Mobile Components Content DeliveryNetworks
90. The fact that applications often have many bottlenecks, that can only be uncovered one at a time.30
91.
92. Goal was to validate performance of entire application3rd Party/Cloud Services Browsers and devices Local ISP Users Load Balancing Load Balancing 2 Web Servers Web Servers App Servers App Servers Internet DB Servers DB Servers MajorISP Mobile Carrier Storage Storage Mobile Components Mobile Components Content DeliveryNetworks
93.
94.
95. Goal was to validate overall performance focusing in 2 key regions3rd Party/Cloud Services Browsers and devices Local ISP Users Load Balancing Load Balancing Web Servers Web Servers 3 App Servers App Servers Internet DB Servers DB Servers MajorISP Mobile Carrier Storage Storage Mobile Components Mobile Components Content DeliveryNetworks
96. No Performance Issues Detected From Data-Center Increase and hold load and not exceed response times of 4 seconds and Success Rate of 99% There was only 1 page error and 11 errors total out of 60000+ transactions Page response times stayed under 4 seconds, outside of one brief blip By traditional test standards the test passed
97. Performance Issues Detected From Real User Desktops Key geographies for this customer are New York and Pennsylvania Last Mile data showing substantial number of measurements greater than 4 seconds
98. Last Mile Case Study: Primary Geographies Key geographies for this customer are New York and Pennsylvania. The response time never met the 4 second average goal. By these standards the test failed. Availability was Less than 99%. 36
Notes de l'éditeur
Key themes:Find and resolve problems across the entire Web application delivery chainWorld’s largest load testing network reach means that you can test performance under load from browser to data centerFind and fix user experience and performance problems undetectable with traditional solutionsTalk trackThe answer to understanding and controlling the Web application delivery chain is to take a new view on your Web or mobile application: you need to adopt an outside-in, customer point of view. You need to “look back” at your web site and infrastructure the same way your customer does: by operating and running it “from the outside in”What does this mean?<click to animate>It means you need to test with real world load the same way your customers will. So, you need web load and performance testing to : Test across geographies, measuring web application performance at peak loads from your end-users’ perspective Find problems missed by lab-based “behind the firewall” solutionsHow does Reality Load help you do this?<click to animate>Reality Load XF’s Last Mile testing finds problems across the entire Web application delivery chain that cloud or data center-only solutions miss to accurately measure response time and find real end-user problems.With Gomez you can be fully informed about what’s going on with your web and mobile application and ensure quality experiences for your users.
Key themes:Gomez covers the globe with the most comprehensive testing networkWe are where your customers areTalk trackThis is a visual depiction of our global testing network.You can see where our Backbone and Last Mile testing locations are.Our Last Mile locations literally span the globe and allow you to test and monitor from any significant location in the world. And it’s growing every day.You can use these for a combination of monitoring and load testing.You can’t see the locations for the virtual test bed because it’s virtual – i.e. location independent.
ResolutionThe query was optimized and indexed to achieve greater performance with less CPU utilization.Business ImpactHad the application been launched with this query in place the login process would have bottlenecked at less than 10% of the anticipated traffic, resulting in:Higher response timesServer errors
All aspects of the user experience delivery chain must be testedBusiness ImpactCustomers negatively impacted by Higher response timesTime-out errors