The presentation on Unified APM: The new age performance monitoring for production systems was done during #ATAGTR2017, one of the largest global testing conference. All copyright belongs to the author. Author and presenter : Kaushik Raghavan

1. #ATAGTR2017
16th 17th March
Unified APM: The new age performance monitoring
for production systems
Kaushik Raghavan

2. Agile Testing Alliance Global Testing Retreat 2017
Agenda
• Introduction APM
• Walkthrough of APM features
• Synthetic User Monitoring
• Real User Monitoring
• Server Monitoring
• Application and Database Monitoring
• Deep dive diagnostics
• Log Processing
• Load Testing
• Unified APM
• A live demo of all the features using the Unified
APM Tool.

3. Agile Testing Alliance Global Testing Retreat 2017
Introduction to APM
• Collect performance metrics of an entire
application across the technology stack.
• Report and alert any abnormality into application
performance.
• Help development teams to find the root causes
for application performance issues.
• Provide analytical insights into application
performance to all the stakeholders and how they
are impacting business goals
• Generate bird’s-eye-view maps of the various
application components in production systems

4. Agile Testing Alliance Global Testing Retreat 2017
Gartner's 5 Dimensions of APM, A
Conceptual APM solution
• End-user experience monitoring
• Runtime application architecture discovery,
modelling and display
• User-defined transaction profiling
• Component deep-dive monitoring in application
context
• Log processing
• Analytics and Insights

5. Agile Testing Alliance Global Testing Retreat 2017
Source : First published in APM Digest in March 2012 by Larry Dragich

6. Agile Testing Alliance Global Testing Retreat 2017
Synthetic user monitoring (SUM)
• SUM is a way to monitor the online applications
performance from various geographies across the
world.
• Online applications will have users connecting from
different parts of world with wide range of internet
speed, different browsers and operating systems.
• SUM works by simulating the user navigations (single
page or a complete transaction) from SUM agents that
are placed across the world.
• Most SUM agents allow simulating the transactions
with different speeds, multiple browsers and devices.

7. Agile Testing Alliance Global Testing Retreat 2017
SUM Configuration

8. Agile Testing Alliance Global Testing Retreat 2017
SUM Insights  The output of SUM
SUM Answers the following Questions
• Is the Site up?
• Are the Pages loading faster?
• Are the transactions successful?
• Are my pages performing consistently
well across locations, browsers and
networks?
• Are there any third party components
delaying my pages?
The SUM Output  A HAR file
• Time taken to fetch the DNS
information.
• Time taken by each and every
request in the page.
• Time taken to connect to the server.
• Time taken to transfer the data from
the server to the browser.
• No of Requests blocked.
• No of Request failures.

9. Agile Testing Alliance Global Testing Retreat 2017
The HAR File
 ​HAR stands for HTTP Archive.
 A common format for recording HTTP (Web Request) tracing
information.
 This file contains a variety of information, but for our purposes, it
has a record of each object being loaded by a browser.
 Each of these objects’ timings is recorded as shown below.

10. Agile Testing Alliance Global Testing Retreat 2017
Your own Selenium Hack (goodies)
• You can generate HAR files from you selenium Automation
Code.
• You will need Fire fox, Net export and a har viewer.
• You will also need some java coding experience. (Selenium
Code will be delivered to the participants via github link)
• HAR Export Trigger by “Software is hard” blog.
• For Quick SUM Test using www.webpagetest.org

11. Agile Testing Alliance Global Testing Retreat 2017
Real User Monitoring (RUM)
• RUM is used to gain insights into actual end-user
experience with online applications.
• RUM is a minimal invasive monitoring technique that is
used records all user interaction with the online hosted
in internet.
• Front-end is responsible for 80% of web page’s load
time.
• RUM can be used to improve browser-side
performance by offering powerful performance
insights.
• RUM can be implemented by inserting a small,
lightweight java script agent into all the webpages.

12. Agile Testing Alliance Global Testing Retreat 2017
RUM Insights
• How many visitors are coming to my page and at
what frequency?
• How long each user spends on my website?
• Which page in my site is being visited often?
• How long does it take to load the assets of the
page?
• How long does it take to render the page on
different browsers?
• Are there any java scripts slowing down my page?
• Where are users coming from?

13. Agile Testing Alliance Global Testing Retreat 2017
RUM Insights

14. Agile Testing Alliance Global Testing Retreat 2017
Fixing your own Peg of rum using
boomerang JS
• Loading the Boomerang JS file in the Web pages.
• Adding the code snippet to measure the page response
time.
• Retrieving the information from beacon URL.
<script src="boomerang.js" type="text/javascript"></script>
<script type="text/javascript">
BOOMR.init({
user_ip: "<user's ip address>",
beacon_url: "http://yoursite.com/path/to/beacon.php",
autorun: false
});
</script>

15. Agile Testing Alliance Global Testing Retreat 2017
Server monitoring
• Server monitoring is used to get deep insights into
critical OS server metrics like CPU, memory usage, disk
utilization, services and processes and more.
• Server monitoring will help us to Identify and resolve
performance or availability issues promptly.
• Server monitoring is a proactive way to and identify
performance degradation.
• Agents are the most common way to collect
performance metrics from the servers.
• Agents are light-weight, low-footprint services running
on your servers. Agents can collect performance data
of Operating System, Web Servers and Databases

16. Agile Testing Alliance Global Testing Retreat 2017
Insights from Agents
• Summary of server status
and performance over
time.
• CPU usage, memory usage,
disk utilization
• Number of services and
processes running
• Network statistics
including bandwidth and
open ports
• Processor queue length,
context switches and
uptime

17. Agile Testing Alliance Global Testing Retreat 2017
Application and Database monitoring.
• It is very important to continuously monitor the applications and database
in production.
• Application servers and Database servers have to be tuned to optimal
settings so that end-users can have seamless interaction with online
applications.
• It is also important to watch the some of the key performance metrics of
these severs so that proactive decisions can be taken to avoid catastrophe
in production.
• Application monitoring brings out hidden performance issues quickly so
that they can be resolved quickly, thus enabling the teams to spend more
time on building the software and not troubleshooting it.
• App and DB monitoring metrics will provide insights into application
performance and will aid in taking right decisions to build, deploy, and
maintain great web software.
• Monitoring High-Volume Production Environment is key to any transaction
processing systems.

18. Agile Testing Alliance Global Testing Retreat 2017
Key Application and DB counters
Application Monitoring
• Number of Request
• Number of Threads
connected
• Number of connections
established
• Request processing time
• JVM heap size
• Number of Errors per
Second
• Number of Active Sessions
• Bytes sent and received
Database Monitoring
• User sessions
• Execution Plans
• Waited locks and
immediate locks
• Resource consumption
• Number of open tables.
• Database objects
• Schema statistics
• User sessions
• I/O read and writes.

19. Agile Testing Alliance Global Testing Retreat 2017
A snapshot of APP and DB monitoring

20. Agile Testing Alliance Global Testing Retreat 2017
Deep dive diagnostics (profiling)
Deep dive diagnostic aka profiling goes beyond the performance
metrics collected from application performance monitoring tools, and
peeks directly into the functions, methods, objects and classes of the
executing application.
When an Application is profiled the following things are recorded.
– The execution speeds of functions and methods
– The CPU and memory utilization by the objects
– Unhandled exceptions
– Number of times a function is being called.
– Profile databases to find expensive queries and stored procedures
– Find performance bottlenecks at both the code and the data access
layer
– Get rich performance insights right-down to line-level timings and
expensive database queries.

21. Agile Testing Alliance Global Testing Retreat 2017
Profiling Example
• Deep dive diagnostics helps the development teams with
exact performance bottlenecks at both method level and
query level. This helps development teams to quickly resolve
the performance issue and thus enhancing the customer
experience.

22. Agile Testing Alliance Global Testing Retreat 2017
Log processing and analytics
• An Ideal APM system provides log processing and analytics.
Whenever there are crashes on the server, logs help us to resolve
the issue quickly by pin-pointing the problem areas.
• Log processing eliminates the need for sysadmins to extract the
logs. Log processing allows you to view all logs at one place.
• With the advent of ELK (Elastics search, logstach, and Kibana)
visualizing the logs has become very easy.
• APM systems allow you to deep insights into your production logs
to take corrective and preventive actions.
• APM tools allow you to drill down the logs with various filters.
• Log processing systems collect gigabytes of logs and generate
meaningful insights about application performance.

23. Agile Testing Alliance Global Testing Retreat 2017
Birds Eye-view of ELK enabled APM

24. Agile Testing Alliance Global Testing Retreat 2017
Load Testing
• Load testing allow QA teams to measure application response
times, throughput rates, and resource-utilization levels.
• Load test also allows the Devops teams to identify the application’s
breaking point, assuming that the breaking point occurs below the
peak load condition.
• Load testing is way to simulate real time production systems, so
that the application performance can be predicted.
• Load testing repots help the IT teams to plan the infrastructure to
the optimum levels.
• When load testing is integrated with APM systems it allows the
stake holders to see how the production systems behave during
peak load conditions allowing them to take right decisions.
• Load tests help organizations to minimize the risk of application
crashes when launching web, mobile, and cloud-based applications,
and effectively plan for peak traffic spikes.

25. Agile Testing Alliance Global Testing Retreat 2017
Unified APM
• An ideal APM system should provide a unified view of production systems
under one screen.
• It will be very difficult to move across screens to correlate various
performance issues that happed in a particular environment
• Unified APM system helps organizations to detect and diagnoses problems
in real time, drilling down to the offending code right from browser to the
data access layer across all layers of technology stack.
• Analysing performance insights with unified APM is a sophisticated
approach. Unified APM approach helps IT teams to diagnose and resolve
performance issues by significantly reducing the time for communications
between different teams.
• The benefits of a unified approach are increased testing productivity,
better application performance, quick release of new applications and
new features.
• Most importantly not to forget the joy of making a customer happy every
time he visits your site.

26. Agile Testing Alliance Global Testing Retreat 2017
The Way forward
• According to Gartner technology, process and people changes are
disrupting the conventional implementation of application
performance monitoring.
• It is important for software organizations to select APM solutions
that can help them to solve the challenges of application scaling
and increasing dynamic resources.
• IT teams have to implement a life cycle strategy to implement APM
solutions so that the applications are monitored for their
performance in all the stages of SDLC and not in production systems
alone.
• There is huge demand for APM specialists who can create insightful
views in to production systems with the data being collected.
• APM solution vendors have to provide flexible pricing structure so
that they can accommodate a variety of costumers to increase their
value propositions.

27. Agile Testing Alliance Global Testing Retreat 2017
Thank You…. 
• Questions and Feedbacks are Welcome.