DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
Why Average Response Time is not a right measure of your web application's performance Nishant Verma
1. +
Why “Average” Response Time is not a right measure of your web
application's performance?
By- Nishant Verma
2. +For the next 15 minutes
n Web Application Performance Testing
n Average Response Time
n Solution to Average Response Time
n Beyond Average Response Time
n Questions
4. +
Web Application Performance Testing
How to do
Design Test/
Scenarios
Establish
Performance
Acceptance
Criteria
Identify
Environment
Execute Test
Analyze
Reports / Tune/
Retests
What we
collect
Response Time
Throughput
CPU Utilization
Memory
5. +Average Response Time
n Response Time is defined as the time it takes for each Web page
to load.
n How we capture?
n Hit the server with a specified number of user (could be in pattern)
and then capture the response time of the server.
n Refer the plot of ART on next page.
7. +
Problems with Average Response
Time
n It’s a bad way of tracking website performance.
n It doesn’t indicate or hint about the outliers.
n We tend to average the number of samples than the quantity
itself.
n It represents a distorted picture in case of exceptional
scenario with extremely high response time.
9. + Apdex
n Created by Peter Sevcik of Apdex Alliance (apdex.org).
n How is it calculated?
n ApdexT = (# Satisfied requests + # Tolerating requests / 2)
(# Total requests)
APDEX: Application Performance Index
10. +
How it scores over Average Response
Time
n It help takes us Go-No-Go decision.
n It’s doesn’t hide outliers. It yells the number of users affected by the
problem.
n It doesn’t get distorted when there are extremely high response
times.
n It’s a measure towards Satisfaction score and can be derived based
on threshold. It’s not a fix number.
11. + One last thing…
Load Testing
Stress
Testing
Soak Testing
Performance
Tuning
Hi I am Nishant Verma and today I am here to present a lightening talk for you folks for the next 15 minutes. So let’s start: (Slide 1)Not sure how many of you have read this quote before. But I would expect you folks to relate to this quote at the end of the 15 minutes.[ENTER]How many of you have previously worked on Performance Testing? (Show of hand would be fine!) How many of you are aware of Average Response Time? (Show of hands!)All right Cool!
So at the high level agenda looks like: What it is? How we do it? What we collect?What is it? And What’s wrong with Average Response Time?Solution - What is it? How to use it? - - - - - - And then in the end we will try to take some questions if time permits.
(Slide 3)So what is Performance Testing? [Click]It’s a type of testing with focus on non-functional aspects of the application. It’s done and owned by a tester. [Click] What we typically measure is [Click] Responsiveness of the application [Click] Scalability [Click] Throughput Why we do it? [Click] To establish a baseline of application performance [Click] To determine compliance.Response time, or Responsiveness, is simply how quickly an interactive system responds to user input.Throughput is the amount of transactions produced over time during a test. It’s also expressed as the amount of capacity that a website or application can handle.
How to do:So we start with designing the test scenarios which could be a business case .Then we establish the acceptance criteria (basically defining exit criteria)Identifying a test environment, sometimes we even create a separate Performance Environment with scale down version of actual production environment.Put a tested build preferably a release candidate build. Execution- How we do it ? Typically what we do is we make a load profile which could be just bombarding the app server with What we collect: Responsiveness:Response time, or Responsiveness, is simply how quickly an interactive system responds to user input. Basically, “Throughput” is the amount of transactions produced over time during a test. It’s also expressed as the amount of capacity that a website or application can handle.
1. Response Time is defined as the time it takes for each Web page to load.2. There are bunch of tools in market like LR which suggests to go by 90% percentile.3. What we generally do is hit the server with a specified number of user (could be in pattern) and then capture the response time of the server.
Let’s say an E-commerce application like Flipkart
We need a performance Indicator Help take the go-no-go decision. Without looking at all possible URLs. Without parsing the request log to find any outliers.Indicator of the number of users affected by the performance problem.Indicator of over all health of the website, without missing outliers.
Let’s re look at the load scenario once again. And this time comparing it with Apdex Index.
This is just a tip of iceberg.Average Response Time or Apdex Index is just the tip of the so called Performance Testing ice berg….Load Testing – To verify application behavior under normal and peak load conditions.Stress Testing-To determine or validate an application’s behavior when it is pushed beyond normal or peak load conditions.Soak Testing provides a measure of a system's stability over an extended period of time.