Customers and service providers alike are struggling to keep costs down while supporting initiatives like Bring your own Device (BYOD). Advanced tools are needed to add new levels of serviceability and outpace the services demand. Avaya, which has long been a leader in the development of automated remote diagnostic service capabilities, is once again leading the way with a new generation of solutions designed to simplify and accelerate problem resolution.
Simplify and Accelerate Problem Resolution with Advanced Diagnostics
1. Introduction
The transformation taking place today in enterprise communications networks
requires a corresponding change in how they are supported and maintained.
The dramatic growth in network complexity has made management and
maintenance more time-consuming and expensive. Customers and service
providers alike are struggling to keep costs down while supporting initiatives
like Bring your own Device (BYOD). Advanced tools are needed to add new
levels of serviceability and outpace the services demand. Avaya, which has long
been a leader in the development of automated remote diagnostic service
capabilities, is once again leading the way with a new generation of solutions
designed to simplify and accelerate problem resolution.
Clients covered by Avaya Support Advantage Preferred have access to two new
diagnostic tools: Avaya Diagnostic Server and expanded access to Avaya’s extensive
testing and diagnostic routines through the Avaya Diagnostic Portal.
Supporting the New Enterprise Network
Today’s enterprise communications networks are undergoing more change
than at any time in their history.
In the past, network value was concentrated in a relatively small core of
infrastructure elements (switches, routers) and devices (phones, fax machines,
PCs) supporting a limited set of communications capabilities (primarily voice)
between a fairly narrow set of locations (office to office.)
avaya.com | 1
White Paper
Simplify and Accelerate Problem Resolution
with Advanced Diagnostics
Table of Contents
Supporting the New Enterprise
Network........................................... 1
Traditional Service
Paradigms...................................... 2
Advanced Remote
Diagnostics................................... 3
Changing the Value Equation
in Service....................................... 3
Simplicity, Security and
Flexibility.......................................4
Next Generation Remote
Access: The Avaya
Diagnostic Server....................... 5
Empowering Avaya
Partners: The Avaya
Diagnostic Portal........................ 6
The Future..................................... 6
2. avaya.com | 2
Today, none of the above is true. The infrastructure is more distributed. The
range of devices more diverse. More people work in more locations,
communicating and sharing greater amounts of information across a wider
range of formats—voice, text and video—than anyone ever previously thought
possible. Mobility, Bring Your Own Device (BYOD), SIP and virtualization are all
part of the mix. The technology supporting all of this has become increasingly
complex as technology stacks grow thicker and increasingly include elements
from different suppliers.
While previously troubleshooting involved tracing a connection between two end
points over a clearly identified set of components, it now requires looking at
multiple sessions across different call servers, feature servers, conferencing
bridges, and handoffs (none of which necessarily appear as if they are connected)
while correlating information from multiple logs with various time stamps.
The net result is that problem resolution is more time-consuming and costly:
A 2008 Gartner study of overall IT support costs showed few enterprises (less
than 10 percent) able to lower their hardware and software support costs, and
almost half seeing a significant rise. 1
This is consistent with internal Avaya
research showing that over the past decade, the amount of time—normalized
for various factors — required to resolve a real-time communications problem
has doubled.
Traditional Service Paradigms
As the service and support challenge intensifies, traditional service paradigms
struggle to keep up—and they will continue to do so as communications
elements become even more distributed across the core, edge and cloud.
On-site Support: A significant portion of service problems are still resolved by
sending a technician on-site to physically work on a system. This is costly, both
in terms of the impact on the customer due to the elapsed time for resolution
and in terms of labor and support expenses. According to the Technology
Services Industry Association, a typical truck roll today costs $1,011.172
.
1 Gartner; User Survey Analysis: The Cost of Hardware Support, North America, 2008, December, 2008
2 Technology Services Industry Association, presentation at the Field Service Power Hour - “Slow your Roll”
May 6, 2013, Ravi Naidu, Director – Research, Field Services
3. avaya.com | 3
Service Desk: Service Desk (a.k.a Help Desk) approaches have eclipsed on-site
service as the most common form of support. This can be more cost-effective
for the provider, but not always as beneficial for the customer: if a “catch and
dispatch” approach is followed—logging the call and passing it on—the elapsed
time until resolution can still be significant.
Self-service: Web-based self-service solutions have been successful at
reducing service costs and accelerating resolution. They have won acceptance
in the marketplace from customers who welcome the flexibility and overall cost
reduction. Self-service is rapidly evolving with capabilities such as:
• Enhanced knowledge bases including text and video-based information
• Search engines with intuitive, multilingual search capabilities
• Improved incident-status reporting capabilities
• Enhanced interaction capabilities such as virtual chat and video
• Incident status portals with personalized access
• Data mining of incidents (calls, chats, Web forms or e-mails)
Avaya has invested significantly in self-service options, for example posting an
average of 1,600 knowledge articles to its knowledge base each month. In 2012,
customers viewed more than 700 of Avaya’s instructional videos more than
175,000 times.
While self-service is very valuable, it is not a panacea for all support situations.
Also, it in effect shifts more of the support burden onto customer personnel,
leading to inevitable drawbacks (e.g., as companies operate globally, the wide
range in technical backgrounds and languages of their employees can make
error-identification difficult.)
Advanced Remote Diagnostics
Traditional service approaches are all reactive and as such can never entirely
eliminate the impact of performance disruption. They all tend to reinforce the
siloed, tactical, “fix what’s broken” approach that is increasingly ill-equipped to
support today’s complex and widely distributed networks.
4. avaya.com | 4
The only service delivery method that that can be effectively proactive, while
also minimizing both cost issues and human involvement, is automated remote
diagnostics. As Gartner notes, “proactive and predictive services that prevent
customer pain” are the foundation for the next generation of service delivery.3
Remote diagnostics of enterprise communications have evolved dramatically
over the last two decades: from remotely located technicians using data
networks to log in and gain administrative access to customer systems to
today’s fully-automated Avaya EXPERT SystemsSM
Diagnostic Tools.
Avaya relies on its EXPERT SystemsSM
to detect and resolve more than 600,000
trouble alarms each year, ranging from issues as simple as a power surge or an
inadvertent programming error to proactively identifying if trunks or processors
are nearing capacity, whether toll fraud occurred, or if there’s danger of a
system outage. These EXPERT SystemsSM
can also check for software and
hardware upgrades, identify password lapses, and even identify “chronic”
problems that need resolution before they escalate.
Remote technician support, together with Avaya EXPERT SystemsSM
,
deliver an overall resolution rate of 99% of the alarms on Avaya enterprise
communications solutions.
Avaya provides access to a range of tools for analyzing system behavior and performance
issues across the network.
3 Gartner, Market Trends: Product Support Service Evolution Slows, but Proactive Opportunities Exist Despite Market
Conditions, April 2013
5. avaya.com | 5
Changing the Value Equation in Service
The increasing effectiveness of remote diagnostics is bringing about a change
in how the value of service is calculated. Historically, it’s been measured in
terms of the speed with which a repair takes place (i.e., response-time metrics
such as time-to-respond or time-to-repair). This is still the case, but recent
years have seen an increasing emphasis on the value of prevention: stopping
problems before they occur.
A study conducted by the Data Analysis Research Department of Avaya Labs
demonstrated that the process of dealing with remote alarms (via Avaya
EXPERT SystemsSM
and/or remote technicians,) reduces the risks that can lead
to customer outages. The study showed that systems that experienced a major
problem had 74% fewer outages when monitored remotely by Avaya. This
general effect was detected consistently across all major product groups and
software releases.
To help customers understand the value of remote diagnostics, Avaya
introduced the While You Were Sleeping/EXPERT View®
Reports which show
the number of alarms and trouble tickets received during a specified timeframe
and estimate the cost that a business would have incurred if it had not been
protected by remote monitoring and trouble resolution. These reports enable
customers to fully comprehend the value of remote monitoring and trouble
resolution that is part of their maintenance agreement.
Simplicity, Security and Flexibility
As the value of remote diagnostics has expanded, the solutions have kept
evolving. (See the sidebar, Avaya Remote Diagnostics: Constant Evolution.) For
example, Avaya introduced Secure Access Link (SAL), a standardized remote
access solution, in response to customer requests for more control over who
had access to their system, when this access could take place, and what
information was accessible. They also wanted remote access tools that were
“channel-agnostic”—making it possible to obtain support from their in-house
staff or from preferred partners.
Avaya SAL provides controls that enable customers to determine who will
provide services to their products and to what degree. Customers may have
services provided by Avaya, their own internal support groups, authorized
Avaya partners, or any combination thereof.
6. avaya.com | 6
In addition, Avaya SAL egress mode provides no inherent mechanism to allow
Avaya or any service partner to remotely access customers’ products without
the TCP/IP connection first being initiated from customers’ networks. To help
customers remain compliant with Payment Card Industry (PCI) and other
industry regulations, all Avaya users are uniquely identified and authenticated.
Avaya SAL provides clear, auditable logging of any access attempt, as well as a
customer-managed authorization mechanism.
Avaya SAL also presents minimal connectivity requirements, as all connections
initiated by the SAL Gateway to the service provider use HTTPS, making
it possible to tunnel application-layer protocols, such as SSH, from the
engineer’s desktop to the products being serviced. Connectivity and
installation are simple and secure—generally requiring no firewall changes
on the customer network.
Establishing a standardized approach to remote access through Avaya SAL
enabled a range of new value-add services such as the Avaya Operations
Intelligence Suite and the While You Were Sleeping report.
Next Generation Remote Access: The Avaya
Diagnostic Server
In 2013, Avaya introduced a major enhancement to its remote diagnostic
capabilities: the Avaya Diagnostic Server.
The Avaya Diagnostic Server, which can be implemented at the client location
or a partner location, does everything Avaya SAL does, and much more. For
example, now it’s possible to:
• Remotely capture direct evidence of system behavior—static screen grabs or
even video of the solutions in action—without involving customer personnel.
• Remotely implement professional-level packet sniffing—which previously
required one more onsite visits—to monitor everything coming in and out of
the network interface, pinpointing bottlenecks and diagnosing intermittent
quality issues.
• Enable system engineers to remotely generate calls between locations,
pressing buttons and executing keypad sequences, the way they would if they
were physically at the customer’s site.
7. avaya.com | 7
• Remotely run stress tests, flooding the customer’s network, to simulate
extreme performance situations and measure scalability.
In the past, each of these tests would have required multiple truck rolls, special
probes, the installation of traffic generator equipment, and/or extensive
involvement of customer and support personnel. With the Avaya Diagnostic
Server it’s all done remotely.
No additional servers are needed beyond the Avaya Diagnostic Server itself,
alleviating complex firewall integration. (Many competitive solutions require
servers and sniffers to perform these functions, which require time and
resourcing.)
Avaya clients who now have Avaya SAL can move to the Avaya Diagnostic
Server at any time. It’s just a software upgrade, and it’s free. Avaya clients who
don’t currently have SAL are entitled to the Avaya Diagnostic Server as part of
Avaya Support Advantage Preferred maintenance coverage offer.
Embedded Intelligent Agents: A key capability of the Avaya Diagnostic Server
is its ability to leverage intelligent agents embedded in Avaya endpoints. These
intelligent agents continually collect relevant data and transmit it to the Avaya
Diagnostic Server to detect potential problems before they occur. Having the
embedded agents and now a service capability specifically designed to accept
agent-transmitted information creates a true 1 + 1 = 3 benefit—significantly
enhancing the ability to diagnose, analyze, and fix incidents without any human
assistance and without compromising the stability of the machine.
8. avaya.com | 8
Avaya is focused on extending these diagnostic capabilities across the wide
area network, making it possible to analyze performance on each hop—
including segments outside of the enterprise perimeter— gaining visibility into
all areas that affect voice/video/data quality. These extended capabilities will
continue to accelerate troubleshooting, maximize uptime, and further decrease
the resources required to pinpoint issues.
The Avaya Diagnostic Server, which can be implemented at a client or partner location,
does everything the current Secure Access Link Gateway does, and much more.
Empowering Avaya Partners:
The Avaya Diagnostic Portal
Customers have consistently sought service and support options that are
channel-agnostic. Avaya has responded by putting more communications and
collaboration capabilities in our partners’ hands via Avaya SAL and now the
Avaya Diagnostic Server.
In 2013 Avaya provided our customers and partners with another powerful tool
through the introduction of the Avaya Diagnostic Portal. Developed through
experience with thousands of support requests from the Avaya customer base,
the portal provides web-based access to a wide range of diagnostic tools and
tests that were previously only available to Avaya engineers.
Partners (and customers) can gain access to the Avaya Diagnostic Portal via the
Avaya Support Website with Support Advantage Preferred. Access to the
portal makes it possible to run hundreds of tests (typically in under two
minutes) and obtain a report highlighted with simple green, yellow, and red
indicators (reducing the time required for discovery, analysis and resolution)
and also containing detailed actionable recommendations.
DISCOVERY AGENTS ADMINISTRATIONREMOTE CONTROL
Help | Welcome: svci | Sign Out
Mode: Admin
Packet Sniffing
Agent(s) added to working set:135.27.114.184,135.27.114.240
Agent Working Set
IP / Extn. / MAC
Add Agent
Remove
Start Capture Start Capture
Refresh Delete Get All Captures
Packet Captures
Select Agent IP Extn. Gateway Type
Select Agent IPID Start Time Last Update
Packet
Count
File Size
(bytes)
135.27.114.184 662212@apac.avaya.com 135.27.114.1 SLAMON
135.27.114.240 662211@apac.avaya.com 135.27.114.1 SLAMON
3 135.27.114.184 06-Jun-2013 05:25:39 06-Jun-2013 05:26:38 22 11582
2 135.27.114.184 06-Jun-2013 02:32:57 06-Jun-2013 02:33:51 24 13221
1 135.27.114.184 01-Jun-2013 16:37:28 01-Jun-2013 16:38:21 24 13221
4 135.27.114.240 06-Jun-2013 05:25:39 06-Jun-2013 05:26:33 16 7998
Items 1 - 2 of 2
Items 1 - 4 of 4
9. avaya.com | 9
Avaya Remote Diagnostics: Constant Evolution
1. IP-based Remote Access: The use of modems enabled Avaya
professionals to gain the same level of administrative access as if they
were locally connected to the product terminal.
2. Avaya EXPERT SystemsSM
Diagnostic Tools: Supporting software that
could automatically perform regular tasks instead of requiring a
technician to log in manually to each product.
3. Advanced Services/Secure Access: Use of IPSec VPNs and a
customer-resident management server, the Avaya Secure Intelligent
Gateway, to provide advanced features such as patching and software
release management. Secure, isolated Enterprise Service Platform data
centers were established to contain and control access to the Avaya
Secure Intelligent Gateways and constrain access to any customer
sensitive data.
4. Enhanced Remote Access: IP-based solutions combined with
customer-resident servers, such as Avaya Secure Services Gateways
(SSG), to provide remote support capabilities similar to those
previously provided by modems, but with the added benefit of higher
bandwidth and improved customer control.
5. Channel-neutral Support: Customers can take advantage of self-
service, Avaya support, and/or authorized partner support via Avaya
Secure Access Link—high speed, secure SSL VPN session solution that
integrates the customer and Avaya at the application layer, with
advanced policy and audit capabilities. Customers can be in complete
control of when and how Avaya, or any other service partner, accesses
their equipment.
6. Avaya Operations Intelligence Suite: The Avaya Operations
Intelligence Suite is a unified dashboard that improves serviceability
with real time system status, reporting, and incident management.
With easy access to powerful system reports and tools, The Avaya
Operations Intelligence Suite can provide an end-to-end view of
network operations, enabling customers to reap the full benefits of
their Avaya solutions.
7. Advanced Remote Diagnostics: Avaya introduces the Avaya
Diagnostic Server and the Avaya Diagnostic Portal, significantly
extending the range of service capabilities that can be performed
remotely and/or by customers/partners onsite.
8. Avaya Support Website: Avaya continually adds new capabilities to its
support site (support.avaya.com) to speed and enhance responses. In
2013, the site underwent a complete redesign introducing a wide range
of new features to enable Avaya customers to locate critical
information more quickly and also customize the site to their specific
needs and solutions.