By Radha Nagaraja, National September 11 Memorial and Museum, USA The website of an organization is its global public face in the modern digital age, and smooth functioning of the website is key to meeting an organization's mission. The appropriate Web server infrastructure and configuration are, by common sense wisdom, a function of the expected traffic patterns and the content served on the website. However, a challenging wrinkle in Web server provisioning is posed by high spikes in traffic due to specific unique events. A particularly dramatic instance of this challenge is faced annually by the 9/11 Memorial and Museum on September 11th, when website traffic increases by 35x. To ensure smooth functioning of the website, the annual run-up to the expected high-spike day utilizes a myriad of tools, including best-estimate characterization of expected load; provisioning of a surrogate server infrastructure set up to best mimic the website configuration (and any special content/functionalities planned for the specific day when the high spike is expected) for the future high-spike time; incorporation or appropriate modeling of the caching and load balancing mechanisms expected to be utilized; user simulation and browser automation technologies to realistically mimic spike traffic patterns; and monitoring of quantitative metrics of website performance during load tests over adequate time intervals to uncover subtle issues that might manifest only under sustained load. Applying the set of strategies above has proven to be a time-tested and robust policy for the 9/11 Memorial and Museum to meet the huge spike in Web traffic on September 11th of each year. In this presentation, the strategies above will be covered in detail including overview of the techniques used to estimate future high-spike traffic, rig up surrogate servers, create realistic mock traffic, and best ensure that a website remains performant under high-spike conditions.

1. Art and Science of Load Testing MW18 Radha Nagaraja
Weighing the World Wide
Web: The Art and Science of
Load Testing
Radha Nagaraja
Email: rnagaraja@911memorial.org
National September 11 Memorial and Museum
Presentation at: Museums and the Web, 2018, Vancouver, Canada

2. Art and Science of Load Testing MW18 Radha Nagaraja
Live Streaming and Events
on 9/11 Anniversary
~ 35 X
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Challenge

3. Art and Science of Load Testing MW18 Radha Nagaraja
Website Content Management System (CMS)
Website Hosting Stack
Dedicated Database Servers (Primary
and Fallback)
Dedicated Web Servers
Dedicated Load Balancers
Varnish Caching Layer
DDOS Protection and Caching Layer
End-Users (Client-Side)
Client
Monitoring
Tools
Server
Monitoring
Tools
Technology Stack

4. Art and Science of Load Testing MW18 Radha Nagaraja
Website Hosting Stack
Dedicated Database Servers (Primary
and Fallback)
Dedicated Web Servers
Dedicated Load Balancers
Varnish Caching Layer
Server
Monitoring
Tools
Client
Monitoring
Tools
DDOS
Cache
Memcache
Drupal
Cache
Varnish
Cache
DDOS Protection and Caching Layer
End-Users (Client-Side)
Website Content Management System (CMS)
Technology Stack

5. Art and Science of Load Testing MW18 Radha Nagaraja
Website Content Management System (CMS)
Website Hosting Stack
Dedicated Database Servers (Primary
and Fallback)
Dedicated Web Servers
Dedicated Load Balancers
Varnish Caching Layer
DDOS Protection and Caching Layer
Server
Monitoring
Tools
End-Users (Client-Side)
Client
Monitoring
Tools
• Visitor Analytics
• Incoming and
Outgoing
Traffic Analytics
• CPU
• Memory
• Database and
CMS Analytics
Technology Stack

6. Art and Science of Load Testing MW18 Radha Nagaraja
Load Testing: High-Level Workflow
1. Estimate traffic.
2. Provision appropriate (best-estimate) infrastructure.
3. Create traffic (as realistic as possible).
4. Test website performance.
5. Repeat Steps 2-4 as needed.

7. Art and Science of Load Testing MW18 Radha Nagaraja
Variation of Traffic Patterns During Spike

8. Art and Science of Load Testing MW18 Radha Nagaraja
Identify the hardware
configuration of the
Production stack
Calculate the upsize
requirements for the
Anniversary by
estimating visitor traffic
Set up a cloned
Production
environment in
collaboration with
hosting provider
Setup a new test Virtual
IP on the DDOS and
caching platform for
routing traffic to the
cloned stack
Setup a new domain for
the load test
environment (e.g.
loadtest.911memorial.o
rg)
Configure the Test
Payment Gateway so
we can make test
transactions
Configure any external
API/database
connections to point to
the test servers.
Configure Analytics for
the cloned domain.
Finally, make test
transactions to test if
everything works as
expected
All set for starting
load testing!
Note: We usually start with running load tests with the current configuration of the Production environment and then run tests on upsized environments
as necessary and compare the difference in performance.
Setting up of the Load Test Environment
Preparing for Load Test

9. Art and Science of Load Testing MW18 Radha Nagaraja
Defining Load Test Scenario

10. Art and Science of Load Testing MW18 Radha Nagaraja
Defining and Modeling Use Cases
Sample use case with
anonymous and
authenticated sections
Modeling in the load
testing framework

11. Art and Science of Load Testing MW18 Radha Nagaraja
Load Test Time Profile and Latency Measurement
A sample time
profile of number of
concurrent users
Variation of latency. It is
seen that the latency
remains approximately
constant while ramping up
number of concurrent users
and during sustained load.

12. Art and Science of Load Testing MW18 Radha Nagaraja
Conclusions and Summary
• Load tests and performance tests are important to ensure the
website can handle spikes in traffic smoothly.
• Uncover issues that might manifest only under sustained load
• It is crucial to make the load test infrastructure and simulated
traffic as close to real as possible.
• As a precaution, it is still advisable to keep “human components”
online during the high-spike period.