CA Technologies worked with IDG Research Services to tap the CIO Forum on LinkedIn to gain insight into how enterprise CIOs feel about their ability to monitor application performance and to guage the benefits of using an Application Performance Management (APM) solution. By tapping into this community, our goal was to explore the ability of companies to meet end user expectations with regard to application performance – more specifically we were looking to identify key challenges companies are facing with respect to application performance and to better understand how they might leverage APM solutions to address those challenges.
For more information on CA APM, visit http://www.ca.com/us/products/application-performance-management.aspx?mrm=425887
2. Market Pulse: APPLICATION PERFORMANCE MANAGEMENT
Background and Methodology
What did we measure?
1. Importance of monitoring/measuring
Online survey during August 2012 application performance
among members of the CIO
Forum on 2. Impact from application downtime/
performance problems
100 senior IT managers at
enterprise organizations 3. Confidence in ability to meet end-user
(ave. gross sales of $4.38B) expectations
responded to the survey.
4. Gap between capabilities needed and
capabilities companies can deliver
GOAL: to explore the ability of
companies in a cross-section
of verticals to meet end user 5. Planned investment in application
management solutions/services
expectations with regard to
application performance.
6. Building a business case for APM investment
2
4. Market Pulse: APPLICATION PERFORMANCE MANAGEMENT
For 8 out of 10 enterprise organizations, monitoring and
measuring app performance is an important IT priority.
Importance of monitoring/measuring application performance
Not important 2012
at all
7%
Not very
important
14%
Very
important
48%
Somewhat
important
31%
Q1: With the end-user experience in mind, how important is monitoring and measuring application performance as
an IT priority at your company?
4 Base: 100 respondents
5. Market Pulse: APPLICATION PERFORMANCE MANAGEMENT
Not surprising given more than half (57%) of respondents
have experienced a loss of employee productivity due to
application downtime/poor performance in the past 18 months
Outcomes experienced as a result of problems with critical applications
57% 41%
45% 30%
Inadequate or
Loss of employee incomplete information Escalating costs
productivity due to Too much time spent for decision making of resolving issues
application downtime/ resolving urgent issues and problems
poor performance limits ability to launch
new services and meet
changing business demands
Q.3: Which of the following has your company experienced in the past 18 months as a result of problems with your critical
applications that have impacted end users?
5 Base: 100 respondents
6. Market Pulse: APPLICATION PERFORMANCE MANAGEMENT
Yet only 16% are very confident in their current
ability to meet end-user expectations!
Confidence in ability to meet end-user expectations for application performance
Very confident 16%
Somewhat confident 50%
Not very confident 26%
Not confident at all 8%
Q2: How confident are you in your current ability to meet end-user expectations with regard to application performance?
6 Base: 100 respondents
7. Market Pulse: APPLICATION PERFORMANCE MANAGEMENT
Companies value quick insight and understanding of the
business impact when monitoring and measuring the
end-user application experience
Importance of capabilities for monitoring/measuring the end-user experience
% answering ―very" or ―somewhat" important
Ability to quickly and accurately identify
90%
problem root cause
Proactively identify, diagnose root cause
and fix problems before end-users are 90%
impacted
Prioritizing application issues and problems
85%
based on business impact
Understand the business impact of poorly
83%
performing end-user transactions
Ability to link end-user transaction issues to
failing infrastructure, application and 82%
network components
With the exception of “prioritizing application issues based on business impact” fewer than 50% of
respondents rate their company’s ability to meet these objectives as excellent or good.
Q.4: How important are the following capabilities when monitoring and measuring the end-user experience for your critical applications?
7 Base: 100 respondents
8. Market Pulse: APPLICATION PERFORMANCE MANAGEMENT
The largest gap between the capabilities companies need
and those they can deliver is in identifying and fixing problems
before end users are impacted
% rating factor ―very" or ―somewhat" important vs. % noting company is excellent/good at achieving that objective GAP %
Importance Ability
Proactively identify, diagnose root cause and fix 90%
problems, before end-users are impacted 37%
53%
76%
Having 100% visibility into all end-user transations across
IT environments (physical, virtual, cloud, mainframe) 28%
48%
Having a single source of truth on application health and 75% 48%
end-user experience 27%
Ability to link end-user transaction issues to failing 82%
infrastructure, application and network components 35% 47%
Prioritizing application issues and problems based on 85%
business impact 55% 30%
Q.4: How important are the following capabilities when monitoring and measuring the end-user experience for your critical applications?
Q.6: How would you rate your company’s ability to meet the following objectives with regard to monitoring and maintaining end users’
experiences?
8 Base: 100 respondents
9. Market Pulse: APPLICATION PERFORMANCE MANAGEMENT
When resolving end-user problems, the proactive detection of
issues and problems was rated as somewhat/very important by
the largest number of respondents (93%), with rapid root cause
analysis close behind (91%)
Importance of capabilities for resolving end-user problems with critical applications
% answering ―very" or ―somewhat" important
93%
91%
89%
Sixty-two percent of
organizations with <$500M
in revenue say eliminating
finger pointing between
teams is
important, compared with
89% of $500M+
organizations.
Proactively detecting Performing root
application issues and cause analysis quickly, Prioritizing application
problems to resolve problems before issues and problems
users are affected based on business impact
Q.5: How important are the following capabilities when resolving end-user problems with your critical applications?
9 Base: 100 respondents
10. Market Pulse: APPLICATION PERFORMANCE MANAGEMENT
On average, companies expect the level of investment
in APM to increase by 29% over the next year
Investment in APM solutions/services
$613,651
$474,011
29%
Average amount invested in Average amount of planned
% change in mean
2012 investment in 2013 investment
Application performance management (APM) is defined as a process and use of a related IT solution to detect, diagnose,
remedy and report application’s performance to ensure that it meets or exceeds end-users’ and businesses’ expectations.
Q.7: How much is your company investing in 2012 in application performance management solutions/services, and how much will your company
invest in 2013?
10 Base: Respondents providing a numerical response
11. Market Pulse: APPLICATION PERFORMANCE MANAGEMENT
Issues with the performance of critical applications
are driving/would be most likely to drive investment
in APM solutions
Factors driving investment in APM solutions/services
Top 3 responses
59%
49% 49%
A performance problem Demand for IT Improving end user/
with critical application accountability (SLA customer satisfaction
(revenue management, SaaS
generating, customer/partner vendors, business
facing, internal productivity) service management, etc.)
Q.8: Which of the following factors are or would drive an investment in application performance management
solutions/services at your company?
11 Base: 100 respondents
12. Market Pulse: APPLICATION PERFORMANCE MANAGEMENT
A majority of respondents cite spending less
time/resources on the resolution of urgent issues
as a key benefit to APM
Most appealing benefits of APM
Top responses
Less time and resources spent
resolving urgent issues 53%
Faster troubleshooting/Fewer
49%
escalations/Less finger pointing
Higher service quality/ability 45%
to meet SLAs
Ability to understand end-user
experience in complex virtualized 43%
and cloud environments
Increased efficiency 41%
Increased productivity 41%
Q.9: Which of the following benefits are most appealing as it pertains to APM?
12 Base: 100 respondents
13. Market Pulse: APPLICATION PERFORMANCE MANAGEMENT
When talking with non-IT stakeholders, a majority of
respondents would cite improved end-user experience
and increased productivity as key benefits of APM
APM benefits to highlight to non-IT stakeholders
Top 3 responses
53%
52%
43%
Improve end-user Increased Cost control/
experience productivity savings
Q.11: When discussing APM with non-IT internal stakeholders, which benefits would you highlight internally to the business?
13 Base: 100 respondents
14. Market Pulse: APPLICATION PERFORMANCE MANAGEMENT
Four out of ten (43%) companies surveyed utilize APM
solutions during pre-production phases of the application
development lifecycle
Production Development
46% 24%
Quality
Staging
assurance
15%
24%
Performance
testing
30%
Q.12: During which stages of the application development lifecycle is your company utilizing application
performance management solutions/services? (Please check all that apply.)
14 Base: 100 respondents
15. Market Pulse: APPLICATION PERFORMANCE MANAGEMENT
Improved service quality is a key business benefit of
APM in the pre-production phase of the application
development lifecycle
Business benefits associated with using APM
54% Top 3 responses
38% 38%
Improved service Reduced operational Improved
quality costs decision-making
Q.13: From a business perspective, what benefits do you associate with utilizing application performance management
solutions/services in the pre-production/testing phase of the application development lifecycle?
15 Base: 100 respondents
16. Market Pulse: APPLICATION PERFORMANCE MANAGEMENT
Service quality improvements to the business are also a key
IT benefit of utilizing APM solutions during pre-production
IT benefits associated with using APM
Top responses
48%
45%
39%
34%
Improved service quality Provides a realistic picture Improved Reduced IT costs
to the business of how the app will perform decision-making
in the market
Q.14: What IT benefits do you associate with utilizing application performance management solutions/services in the pre-
production testing phase of the application development lifecycle?
16 Base: 100 respondents
17. APM SaaS is at the very early stage of the adoption
curve, with the majority of companies reporting no
immediate plans to implement
Usage of APM SaaS
4 out of 10 of these companies are using
APM SaaS to monitor only some of their
critical business applications
Usa APM SaaS
in some
capacity, 24%
No plans to
Plan to
implement APM
implement APM
SaaS, 61%
SaaS in next 12
months, 15% Half of these companies will just be
starting to engage with their MSP on
this topic over the next 12 months as
opposed to a partial or full implementation
Q.16: How is your company currently using or how do you plan to use APM SaaS?
17 Base: 100 respondents
18. Market Pulse: APPLICATION PERFORMANCE MANAGEMENT
When asked about the importance of APM
capabilities, cost was rated as somewhat or
very important by nearly 9 in 10 respondents
Importance of APM capabilities
% answering ―very" or ―somewhat" important
89%
85% Perhaps not
82% surprisingly, scalability is
significantly less
important for organizations
with <$500M in revenue: only
57% rate it as important
compared with 89% of
$500M+ organizations.
Cost Ease of use Breadth of
APM capabilities
Q.15: How important are the following APM capabilities to your company?
18 Base: 100 respondents
19. Conclusions
Satisfying the user – whether that is a customer or an internal employee
user – is still a top priority.
IT managers understand the importance of monitoring and measuring
application performance, but are frustrated in their ability to do so.
Despite static budgets and constrained staffs, they recognize the
strategic value of APM.
More IT organizations are becoming proactive, preparing to invest more
in APM as well as embracing the dev-ops trend by moving APM earlier
in the life cycle.
APM products must equip IT to handle the escalating demands of
the future:
— Keep up with the growing number and complexity of applications
— Contend with different software distribution models
— Span the application life cycle and
— Empower IT staffs to be proactive rather than reactive
19
My name is Janet King and I am the General Manager/VP of Research for IDG Research Services, the research arm for CIO magazine. I am glad to be here today to share the results to a recent research study we conducted on behalf of CA.
This research was conducted in August of this year among members of the CIO forum on LinkedIn. CIO has one of the largest forums on LinkedIn with more than 50,000 members.By tapping into this community, our goal was to explore the ability of companies to meet end user expectations with regard to application performance – more specifically we were looking to identify key challenges companies are facing with respect to application performance and to better understand how they might leverage APM solutions to address those challenges. To gain those insights we asked questions around the central themes listed here.We had 100 senior IT managers respond to the survey. These managers , representing a cross section of vertical industries, were from enterprise organizations with average annual gross sales of $4.3 billion.
The research confirms that measuring and monitoring application performance is a key IT priority for the majority of organization’s today. In fact , roughly half note it is a very important priority for their organization!
And this level of attention is not surprising given 57% of companies have experienced loss of employee productivity due to poor application performance and more than 4 out of 10 note that too much time spent resolving application performance issues has limited their ability to launch new services or meet changing business requirements …This is pretty compelling evidence that poor application performance has real impact on the health and well-being of the business.
While we know that measuring and monitoring end-users’ application experience is a key priority for companies today, respondents report low levels of confidence in their organization’s ability to meet end-user expectations, with just 16% noting they are very confident. And these confidence levels have declined from 30% in last year’s survey- a drop of 50% year over year!
So what do companies need to be more effective in their quest to monitor and measure the end user experience for critical applications. The 5 most important capabilities are listed here but the common thread among them is that companies want the ability to quickly and accurately identify and diagnose problems and a better understanding of the business impact of poorly performing applications.
While capabilities around quick, efficient and unified views into application performance are highly valued, companies often struggle to deliver on those objectives. In fact, as this chart shows the gap between what companies need and what they can deliver is biggest for proactive identification and diagnosis of application problems before end-users are impacted, followed by visibility into end-user transactions across all IT environments and need for a single source of truth on application health and end-user experience.
Not surprisingly the top capabilities companies value for resolving end-user problems with critical applications follow a similar pattern to what they value for monitoring and measuring the end-user experience ...namely, the ability to quickly, proactively identify problems and their root cause before end users are impacted. The ability to prioritize application issues based on business impact is also a highly valued capability. And among larger companies with $500M or more in revenue, eliminating finger pointing between teams is seen as an important step to resolving end-user problems with critical applications.
So what have we learned so far? We know that most companies put measuring and monitoring application performance high on their IT priority list.Why – to minimize the negative impact of poor application performance on employee productivity, responsiveness of IT to changing business requirements, and costs, among other things.Yet the level of confidence companies have in their ability to meet end-users’ application expectations is low and declining, andcompanies want to find a path forward that will help them quickly, proactively identify, diagnose and resolve concerns around application performance.To address that companies are investing in application performance management solutions. For purposes of this survey we defined APM as: a process and use of a related IT solution to detect, diagnose, remedy and report application’s performance to ensure that it meets or exceeds end-users’ and businesses’ expectations. The research shows that companies are currently investing an average of $474,000 in APM solutions/services and they expect that level of investment to grow by 29% over the next year to an average of nearly $614,000.
These investments are being driven by a number of factors but chief among them are performance problems with critical applications, demand for IT accountability and the desire to improve end user/customer satisfaction.
APM solutions are perceived to offer many benefits to address those key business drivers including reducing the amount of time/resources companies spend resolving urgent issues, enabling faster troubleshooting and eliminating finger pointing, and helping companies improve service quality/meet SLAs.
In fact when it comes to building a business case for investment with non-IT stakeholders improved end-user experience and increased productivity are among the benefits from investing in APM solutions that IT execs most widely promote.
And while application performance management solutions are utilized by companies during all phases of the application lifecycle, there is conclusive evidence that companies are actively using these solutions during the pre-production phases of the lifecycle. In fact, more than 4 out of 10 are using APM solutions in one or more pre-production phases like development, staging, testing and/or quality assurance.
And in fact companies associate numerous business and IT benefits with using APM during the pre-production/testing phase of the app dev lifecycle…Top business benefits include…improved service quality, reducing operational costs and improved decision making.
On the IT side, service quality improvements to the business also factor in as an important benefit from using APM solutions during pre-production phases of the application development lifecycle.
The research shows increased market investment in APM and an understanding of the benefits from deploying these solutions earlier in the application development lifecycle.So what about APM solutions provided by managed service providers and other third party vendors? There is a lot of hype in the market today around APM as a service with many vendors entering the market. So we wanted to take a look at who is using APM as a service today and what companies future plans are in this area.
Regardless of the type of APM solution being deployed … what are companies looking for in an APM solution?It is no surprise that even while companies report spending an average of $140,00 more on APM solutions next year they are keeping an eye on the costs invested in these solutions. And given the nature of the problem is so large and so complex they need solutions that are both easy to use and offer a wide breadth of capabilities.For large businesses typically handling billions of transactions daily, scalability is key, those companies require APM solutions that will scale with the business without increasing costs.
The results of the survey confirm that satisfying the user – whether that is a customer or an internal employee user – is still a top priority. IT managers understand the importance of monitoring and measuring application performance, but are frustrated in their ability to do so. Yet despite static budgets and constrained staffs, they recognize the strategic value of APM and more IT organizations are becoming proactive, preparing to invest more in APM as well as embracing the dev-ops trend by moving APM earlier in the life cycle. What’s needed are APM products that help IT departments narrow the gap between what they want and need to do in order to maintain optimal application performance with limited resources. At the same time, APM products must equip IT to handle the escalating demands of the future, enabling IT to keep up with the growing number and complexity of applications, contend with different software distribution models, span the application life cycle and empower IT staffs to be proactive rather than reactive. Thank you!