So, Beyonce, unbeknownst to you, decides to wear your shirt. A paparazzi snaps her casually walking down Rodeo Drive with it.
Suddenly your site explodes and you’re getting angry emails from crazed Beyonce fans about not being able to access it.
What happened?! Was it the dreaded DDoS monster? Or did something even worse happen? You went viral…
When your WordPress site finally goes live, it’s likely that you’ve probably spent weeks or even months building, iterating and debating about it.
The last thing you’re thinking about is testing it.
But if you plan on succeeding on the most important days of your business and site, like a function room, you need to understand how many people can fit in it, otherwise you could be leaving thousands on the table when your site goes down.
Key Take-Away
============
This talk will cover a history of load testing, why it’s important, and a live demonstration with an open-source and free tool that everyone can access right now.
Presented by Robert Li at WordCamp Sydney 2019
2. Who is this idiot?
My name is Robert Li.
I’m a Solutions Engineer for WP Engine.
I’ve founded 3 startups and exited 1.
I have a 3 year old teacup pomeranian
named Chelsey.
I am a long suffering Knicks fan.
I love making WordPress do weird,
wonderful things.
3. Takeaways
1. Solutions
2. Why Load Test?
3. What do you need?
4. (Pray to the Demo Gods)
5. Demo
6. How do you prepare?
7. Let’s take a look
Degree of Difficulty: Nerd
4.
5.
6.
7.
8.
9. The problem
Your Site
You’ve created a site with
lots of cool features, that
has a great user
experience, and that
you’ve spend tonne’s of
money (or time) on.
Your Hosting
You’ve signed up to some
hosting that will fit your
new site, in its current
state.
It works great!
The Flood
You send a shirt to an
influencer, they wear it,
you actually start selling
stuff, which is great!
Then your site
goes down.
WHAT HAPPENED...
14. Functional Tests don’t
reflect the Real World
Testing the functionality of your site or
application doesn’t reflect how users will
use it in real life. They may run across
bottlenecks in complex situations your
testing can’t account for.
15. Your code can behave
differently under load
Any changes to code to an established
website can cause it to behave
differently under load. Bugs can appear
that you didn’t detect in other testing.
16. Servers perform
differently under stress
When scaling to 100s, 1000s or
potentially Millions of concurrent users,
you may get completely different results
for CPU load, error rate, I/O throughput,
and response times.
Houston we
have a
problem
17. The cost of failure is extremely high.
50% of users will leave if a page takes longer than 6.5s to load. 1 minute of
downtime during Black Friday costs US$4700 on average. Downtime during a
peak traffic event damages brand integrity and creates a snowball effect.
https://royal.pingdom.com/page-load-time-really-affect-bounce-rate/
https://www.blazemeter.com/blog/why-load-testing-is-important/
19. Execution
How do you execute a load test?
Reaction
How does the server react to the
load test?
20. Load Testing Terms Primer
Scenarios - A template or script of what a user might do on a site.
Virtual Users/User Agents - Simulated users
Concurrency - How many actions and users are happening and on the site at any
given time
504, 502, 429 errors - When the server is either crapping itself, or protecting itself
from “crapping itself”.
.yaml - The file format that holds scenarios
21. Execution
Ingredients:
Servings: 1000vu
● Taurus - Open source, creates
the Virtual Users that perform
scenarios.
● Blazemeter - Visualizes the data
(can also execute load tests)
● jMeter - Software to help you
build and tweak scenarios
● Blazemeter Chrome Extension -
Records your actions on a
website to start as the basis for
a scenario
22. Reaction
Ingredients:
Servings: 1 overloaded server
htop sar -q 1
● Stackdriver/Cloudwatch - Server
monitoring (GCP/AWS).
OpenNMS is agnostic & open
source.
● New Relic - Monitoring the
application (WordPress) itself.
● Pingdom/Uptime
Robot/freshping - Measures
whether the website is up.
● htop/sar -q 1 - Command
Line (CLI) Live monitoring.
23. CAVEATS
DO NOT LOAD TEST…
❏ On your production
environment
❏ On a shared environment
❏ Without telling your host’s
support/security team
❏ On a site or web
application you don’t have
permission to blast.
x
x
x
x
29. Do as much as
possible before testing
Decide what to complete and what to defer.
Bottlenecks found in functional testing?
Optimize to get a true to life response.
30. Deciding Test Type
Spike - How does a
system respond to a
sudden increase and
decrease in load? e.g.
A Live Online Event
Stress - How does a system
respond to consistent high
intensity loads? Where does
it fail? How does it recover?
e.g. A Flash Sale
Soak - How does a
system respond to
high load over time?
e.g. Black Friday
31. Creating Realistic
Scenarios
Start with the user journey
Your marketing and web team are
integral to generating realistic test
scenarios.
Have them map out ~5 user profiles
and journeys. Usually this is their
funnel
● From where and how do users
access the site?
● What do they do on the site?
● How do they arrive? In waves?
● How long do they stay?
32. Determine your KPIs
What is acceptable? Not what is
ideal. What level of existence are
you willing to accept?
Considerations:
● Response Time
● Error %
● Ratio of failed to successful
transactions
● Maximum requests per second
● Apdex Score
33. Start small before
scaling up big...
Do small scale testing to
discover errors and bottlenecks
in your own scenarios and with
your own tech stack.
Common Questions to ask
yourself:
● Is the script broken?
● Is it calling on a resource that
doesn’t exist or performing an
impossible action?
● Is the testing platform actually
the bottleneck?
● Are there optimizations I can
make to improve headroom?
34. What is the tipping point?
When does acceptable become unacceptable?
35. Under load is 80% usage
Test for 120% usage
When number of hits/s
plateaus this indicates:
● System can no longer
support extra concurrent
requests
● You are at max
concurrent users.