5 AI-Powered ITSM Myths Debunked

Valerie O’Connell
Research Director
Enterprise Management Associates
5 AI-Powered ITSM Myths Debunked
IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Varun Singh
Founder and VP Product
Moveworks

Watch the On-Demand Webinar
Slide 2
• 5 AI-Powered ITSM Myths Debunked:
http://info.enterprisemanagement.com/5-ai-powered-itsm-myth-
debunked-webinar-ws
• Check out upcoming webinars from EMA here:
http://www.enterprisemanagement.com/freeResearch

3 © 2019 Enterprise Management Associates
Today’s Speakers
Valerie O’Connell, Research Director, IT Service and Business
Management, EMA
Valerie came to EMA with decades of senior-level experience in the effective marketing of
technology. From strategic positioning with practical go-to-market planning and compelling
messaging, Valerie works with her clients to drive business. She has enjoyed many years as an
independent practitioner, serving industry giants such as Microsoft and EMC, as well as innovative
startups.
Varun Singh, Founder and VP Product, Moveworks
Varun leads the Moveworks product team. Before Moveworks, he was a lead product
manager at Facebook, where he designed an AI solution and chatbot to better recruit
engineering talent. Earlier, he was SVP of Product at Sefaira (acquired by Trimble
Buildings).

Valerie O’Connell
Research Director
Enterprise Management Associates
EMA research findings on AI/ML in IT Automation

1. High-level findings:
1. Across the board, automation correlates highly with improved business outcomes/metrics
and perceived competitive advantage.
2. Automation initiatives enjoy a very high success rate.
Common sense and research agree: doing more with less and doing it better is good
business.
The gap between enterprises with high and low levels of automation is a competitive gap.

The automation gap is a competitive gap:
While 30% have automation in the planning stage and only 4% have no plans,
26% are already at advanced levels of automation
6
To what degree have you implemented automation in your organization?
18%
8%
19%
22%
30%
4%
0% 5% 10% 15% 20% 25% 30% 35%
Automation is actively being expanded throughout the IT infrastructure
Complex processes have been automated
Some multi-step processes have been automated
Simple tasks have been automated
Automation is in the planning phase
Nothing is automated and we aren't planning to automate anytime soon
Sample Size = 400
High: 26% complex processes + automation
actively expanded
Medium: 41% simple tasks + some multi-
step processes
Low: 34% nothing is automated + planning
Note: Success breeds success!
Automation breeds a culture that can consume and capitalize on automation.
The competitive gap widens with time at an accelerating rate

INDUSTRY ANALYSIS & CONSULTING7
What is the primary use case that automation and/or AI/analytics is currently supporting?
22%
17%
17%
15%
8%
8%
6%
6%
3%
0% 5% 10% 15% 20% 25%
Integrated operations (for superior availability, performance, and change management)
Self-service capabilities for routine requests and services
Routine service desk capabilities: incident/problem/request/change
Integrated asset management/IT financial planning
DevOps/agile initiatives
Enterprise service management (ESM for HR, facilities, etc.)
Integrated security and operations (SecOps)
Major incident response
Internet of Things (IoT)
The combination
of self-service
and routine
service desk
capabilities led as
the primary use
case (34%)
On average, the primary use case is relatively new
(57%) with 31% “Just now fully deployed and in
production” and 26% “currently in deployment”
Sample Size = 400

INDUSTRY ANALYSIS & CONSULTING8
Which were the two most valuable benefits achieved from your automation investments?
28%
25%
21%
18%
12%
12%
10%
10%
8%
6%
6%
6%
6%
4%
4%
4%
4%
3%
2%
2%
2%
1%
0% 5% 10% 15% 20% 25% 30%
Cost savings across IT
Improved IT operational efficiencies
Improved end-user/customer satisfaction
Improved quality of delivered services
Reduced unplanned IT work
Faster incident and problem resolution
More efficient release management
Faster and more accurate change management
Improved business outcomes for IT services
Improved revenue due to reduced downtime
Improved IT-to-business alignment
Improved business/organizational productivity
DevOps process efficiencies
Superior lifecycle IT asset management effectiveness
Improved compliance/risk reduction
Faster deployment of applications
Able to focus more proactively on ITSM (improve/create services)
Improved ability to optimize cloud
Reduced support tickets/call volumes
Improved SecOps efficiencies
Faster deployment of infrastructure
Freed up existing resources to focus on higher-level/more complex tasks
Sample Size = 400, Valid Cases = 400, Total Mentions = 769
Note:
• Combined improved end-user/customer
satisfaction + quality of delivered services
(39%) led as most valuable benefit
• Benefits achieved were a close map to
initial drivers (benefits sought)

But there was no shortage of obstacles…

The major obstacles can be combined into three larger categories:
10
Cost/resources
• Cost/budgetary issues
• Lack of financial and human resources
• Product cost and complexity
Implementation and experience
• Lack of skillsets
• Data management issues
• Training the ML/AI capabilities
• Understanding and setting the use case priorities
• Products not fully baked
• Too much customization needed
Change 61%
• Resistance to change 15%
• Lack of trust in the technology 15%
• Fear of job loss 10%
• Organizational change 9%
• Organizational conflicts in IT 12%
(Note: Lack of executive support can manifest in any category)

IT automation and AI/ML conclusions
11
• IT automation/AI is an executive play because:
• High visibility of projects
• High level of business benefits and advantages
• Digital transformation
• High organizational level of control needed for processes to cross “natural” boundaries
• Automation is an acquired taste
• What begins as fear and resistance to change morphs into…
• Acceptance and extended use cases at an accelerated pace of adoption
• A cycle of continual improvement
• An organizational culture of innovation and automation
Which translate into competitive advantage

© 2019 Moveworks, Inc.
5 AI-Powered ITSM Beliefs
Varun Singh
Founder & VP Product | Moveworks

Company
Investors
Founded:
2016
Headquarters:
Mountain View, CA
Leadership Experience
Employees:
110+
PhDs:
12+

Slow IT support is limiting the productivity of your workforce
The Problem
14
5 hours
Average time an employee waits
before an agent looks at their ticket
Average time an employee waits
before their issue is resolved
1.1per employee

How can I order a new laptopI need a new laptop

© 2019 Moveworks, Inc. 16
New laptop! How do INeed help
What’s the process for ordering new
Lenovo replaceme
my macbook has died
ere do we go to order a new laptop?
Need assistance in ordering a new lap
how to order a laptop for a new hire
I need a new Latitude
how do I request a new laptop
Am I able to get a laptop refresh?
am I eligible for a laptop refresh
Can I get a laptop refresh?
Can I start the process for a laptop refresh?
Can you assist me with a laptop refresh?
get a laptop
I need to order a laptop
I think I am due for a laptop refresh
How can i get a laptop?
How can I order a new laptop
how do I order a new laptop
how to request a laptop refresh
how to request laptop compute
how to submit laptop refresh
How would an employee order a new laptop to replace on that is malfunctioning?
I am eligible for a laptop refresh. how do I initiate one?
I need to refresh the laptop/get an upgrade
i want to order new laptop
I want to request laptop for work purpose
I would like to order a new dell laptop
Laptop 3 year refresh
laptop refresh - macbook air/pro
Order a new laptop
Order New Laptop
request to order a new laptop
update laptop
I need a new laptop

Increasing resolution for Moveworks
customers
17
Serving
companies
across tech,
media, retail,
and finance
35%
of tickets solved autonomously
60sAverage resolution time
65mIT support tickets processed

Common misunderstandings
18
Data Chat bots Operating AI

Belief #1

Successful machine learning relies on huge data
1990s
Text Recognition
5 million handwritten characters
2007
Facial Recognition
14 million annotated images
in ImageNet
2019
Tesla Autopilot
10 billion+ miles of driving
sensor data

Successful NLP
typically
requires huge
data
Google Smart Reply
trained on
1 billion messages

HOW BIG IS YOUR
ENTERPRISE DATA?

For a company of 2,000 employees
2323
24,000 IT Support tickets per year

2424
30% Auto Alerts 16,800 Employee IT issues

2525
12,000 Training data 10% Eval20% Test

2626
Software 1,000
Network Hardware SecuritySoftware Hardware Security

2727
Software 1,000
Network Hardware Security
Wifi 100 VPN Router DNS …
Software Hardware Security

2828
Software 1,000
Network Hardware Security
Wifi 100 VPN Router DNS …
Provision Credential 10 Setup Connection …
Software Hardware Security
Wifi Router DNS …
Provision Credential Connection …

From data to decision boundary
Data Feature Selection Model Decision Boundary
discover

Too much uncertainty
I need zoom
Can I have zoom
How do I connect to VPN
VPN not working
30
The challenge with small data

Too much uncertainty
I need zoom
Can I have zoom
VPN not working
Decision Boundary??
31

I need zoom
Can I have zoom
Decision Boundary??
32
Difficult to estimate the true data distribution
VPN not working

I need zoom
Can I have zoom
33
Difficult to estimate the true data distribution
VPN not working
Decision Boundary
I managed to get on the VPN.
How do I connect to a zoom
meeting?
I need to get on a zoom
meeting. How do I connect to
the VPN?

HOW DO YOU GET BIG
DATAFROM SMALL DATA?

Data improvement techniques
35
Meta Learning Collective LearningTransfer Learning

36

Learn knowledge from tangential domains
37
SPECIFIC DOMAIN TANGENTIAL DOMAIN
Training
Data
g(w)
Model A
f(w)
Model B
Raw Data Raw Data
Training
Data

Learn knowledge from tangential domains
38
SPECIFIC DOMAIN TANGENTIAL DOMAIN
Training
Data
f(w)
Model B
Raw Data Raw Data
Training
Data
h(w)
Generic
Knowledge
g(w)
Model A

Generic
language
patterns can be
transferred
How can I disable the cache temporarily or
refresh the page in some way that I could
see the changes?

Unsupervised clustering of IT tickets
without Transfer Learning
Unsupervised clustering of IT tickets
with Transfer Learning
Transfer learning in action

I don’t have enough data for
machine learning
Belief
Reality
Tangential domains eliminate the need for
big data

Belief #2

You need labeled data to build and test models
Labeled data
discover

Clean data is correctly labeled data
Labeled data
discover

Labeled data
Matt
How often does my password expire? |
2:03 PM David
Can you get me access to zoom please? |
2:05 PM
Action InformationInformation Action

Labeled data
Matt
2:03 PM David
2:05 PM
Action Information ActionInformation

Labeled data
Matt
2:03 PM David
2:05 PM
Information Action Information Action

Supervised learning
given x, predict y
input / data
label

Supervised learning
given x, predict y
Matt
2:03 PM
Information

WHAT IF YOUR USE CASE
IS COMMONACROSS
THE INDUSTRY?

IT ticket data is homogenizing
53
— Finance
— Retail
— Manufacturing
— Technology
— Services

54

Collective learning across similar problems
55
I need Zoom meeting access |
When I connect to Zoom,
there’s no video |
Good Morning, can I have the
Zoom account? |
how to enable zoom |
Organization B Organization COrganization A
How do I log in to WebEx |
WebEx audio connection is
funky for the past two days |
how to attend webex call |
what is the size limit from
webex recordings |
How to setup Google hangout
meeting? |
TV and Google Hangouts not
working in room |
My mic in Google hangouts
doesn't work |

Abstract the language
56
I need Zoom meeting access |
When I connect to Zoom,
there’s no video |
Good Morning, can I have the
Zoom account? |
how to enable zoom |
Organization B Organization COrganization A
How do I log in to WebEx |
WebEx audio connection is
funky for the past two days |
how to attend webex call |
what is the size limit from
webex recordings |
How to setup Google hangout
meeting? |
TV and Google Hangouts not
working in room |
My mic in Google hangouts
doesn't work |
$VCA
$VCA
$VCA
$VCA
$VCA
$VCA
$VCA
$VCA
$VCA
$VCA
$VCA

Language generalization
57
Varun
I need a license for Zoom. |
3:09 PM
Jose
Can I get Confluence access? |
3:11 PM
Sarah
Please give John a Trello account |
3:27 PM
$PERSON need license $SOFTWARE
$PERSON get access$SOFTWARE
give $PERSON account$SOFTWARE

Language generalization
58
Varun
I need a license for Zoom. |
3:09 PM
Jose
Can I get Confluence access? |
3:11 PM
Sarah
Please give John a Trello account |
3:27 PM
$PERSON need license $SOFTWARE
$PERSON get access$SOFTWARE
give $PERSON account$SOFTWARE
$PERSON need a license for $SOFTWARE
$PERSON need a $SOFTWARE license
$PERSON need $SOFTWARE access
$PERSON need access to $SOFTWARE
$PERSON need a $SOFTWARE account
$PERSON need an account for $SOFTWARE
Can $PERSON get a license for $SOFTWARE
Can $PERSON get a $SOFTWARE license
Can $PERSON get $SOFTWARE access
Can $PERSON get access to $SOFTWARE
Can $PERSON get a $SOFTWARE account
Can $PERSON get an account for $SOFTWARE
Please give $PERSON a license for $SOFTWARE
Please give $PERSON a $SOFTWARE license
Please give $PERSON $SOFTWARE access
Please give $PERSON access to $SOFTWARE
Please give $PERSON a $SOFTWARE account
Please give $PERSON an account for $SOFTWARE

# Labeled Tickets
PredictionAccuracy
0 20
0
40
0
60
0
80
0
100
0
Better predictions with 5x less data

# Labeled Tickets
PredictionAccuracy
0 20
0
40
0
60
0
80
0
100
0
Using only your own
data

# Labeled Tickets
PredictionAccuracy Collective learning
with 10 other orgs’
data
0 20
0
40
0
60
0
80
0
100
0
Using only your own
data

Machine learning requires lots of clean data
Advanced machine learning works even with
dirty data
Belief
Reality

Belief #3

Google
search
today…

2019
Enterprise
search…
still broken!

VS
Internet pages
Company wiki pages

Keyword tagging is brittle
68
Help with common laptop issues
Last updated May 16, 2018
help issue laptop
Can someone help issue me a new laptop? |Query:
Keywords:
Document:

Keyword tagging ignores syntax
69
I can’t process a monitor order. |
I need to monitor the order process. |
What is the process to order a monitor? |

Keyword systems can’t disambiguate
70
How to order a cardboard box
Last updated April 8, 2015
What Box
What is Box? |Query:
Keywords:
Document:
is ?
(ignore stopwords and punctuation)

Enterprise language is symptomatic
71
Natalia
My laptop monitor turned blue! |
3:11 PM Marc
I need to request paternity leave |
3:12 PM
Windows Blue Screen of Death
(BSOD) Error troubleshooting guide
Last updated May 16, 2018
California Paid Family Leave claims
process
Last updated Feb 21, 2019

HOW DO YOU DELIVER
MODERN SEARCH?

73
Understand the
question
Understand the
answer
Match the
question to an
answer

How many attendees can a Zoom meeting support with two-way audio?
Focus Word:
Numbers in answer
Core Topic Entity:
Topic of answer
Context:
Mentions in answer
Machine Learning helps understand sentence
context

… you can have up to 100 video participants in any meetings. These participants
have two-way video, audio, and collaboration features …
participants
David
How many attendees can a
Zoom meeting support with
two-way audio? |
3:11 PM
Word embedding is a technique for finding
relationships between words
attendees

Nihal
My laptop won’t power up |
3:11 PM Original query
My computer won’t start
My Lenovo won’t turn on
Computer not starting
My laptop won’t start even when plugged in
Macbook won’t boot – has a black screen
Alternate query
Alternate query
Alternate query
Alternate query
Alternate query
Identify synonymous words and phrases

77
Understand the
question
Understand the
answer
Match the
question to an
answer

Document
Snippeting
Whole document
Factoid
Likely answer to:
“What emoji codes does Slack use”
How-to
Likely answer to:
“How do I use emojis in Slack”

79
Understand the
question
Understand the
answer
Match the
question to an
answer

Sequence to Sequence:
Originally developed for language translation

Sequence to Sequence:
Can be used to translate questions to answers
can someone help issue a laptop replacementme
request a new laptop this form ServiceNowusing on

Keyword tagging is the only option for
enterprise search
High quality enterprise search requires
natural language understanding and
machine learning
Belief
Reality

Belief #4

Shopping bot
• Rules based match
• No slot filling

Travel booking bot
• Basic slot filling
• Greedy match

Input Text
Start
with ‘Is’ ?
Yes
No
Yes
No
Yes
No
“Hardcoded” rules approach falls apart

Machine learning to predict responses
99
Combines data from
multiple backend
systems & NLU
inputs
Model-based, on-the-fly
dialog assembly
f(w)

I can use a bot toolkit to create a
conversational AI system
For a focused domain, a conversational
AI system must be driven by machine
learning models
Belief
Reality

Belief #5

From natural language to action
102
Incoming request Desired result

Ensemble of models to fulfill request
103

104
Spelling “Howdo” → “How do” Spelling correction models

105
How do I add $PERSON to $GROUPNER
Named entity models

EmailDL(ADD, Bhavin, marketing)LSP & SPA Statistical grammar models
106
Named entity models

107
(EmailDL: 0.7, HowTo: 0.19, …)Intent
Named entity models
Intent extraction models

108
Spelling “Howdo” → “How do”
EmailDL(ADD, bhavin@moveworks.ai, ?)Record Linkage
Spelling correction models
Named entity models
Record linkage models

109
EmailDL(ADD, bhavin@moveworks.ai,
marketingna@moveworks.ai)
Fuzzy Select
Named entity models
Disambiguation models

110
EmailDL(ADD, bhavin@moveworks.ai,
marketingna@moveworks.ai)
Fuzzy Select
https://www.googleapis.com/.../?group=marketing
na&member=bhavin
Action Engine
Named entity models
Disambiguation models

One machine learning model is sufficient
for making predictions
High performing machine learning systems rely
on an ensemble of models
Belief
Reality

Belief #6
Bonus

Machine learning models respond to data drift
True Labels: Provision NOT Provision
How do I get Lucidchart?
I am unable to save diagrams in Lucid Chart
True Positive
True Negative
Provision
NOT Provision

Machine learning data flow
115
Annotate
Deploy
TrainMonitor

A model is developed once
and used forever
Belief
Reality
With AI, you need to constantly train and
re-train your models. Data drift is real.

Recap of common AI beliefs
117
Belief Reality
I don’t have enough data for machine learning Tangential domains eliminate the need for big data
Machine learning requires clean data Advanced machine learning works even with dirty data
Keyword tagging is the only option for enterprise search
High quality enterprise search requires natural language
understanding and machine learning
One machine learning model is sufficient for making predictions
High performing machine learning systems rely on an
ensemble of models
A model is developed once and used forever
With AI, you need to constantly train and re-train your
models. Data drift is real.
I can use a bot toolkit to create a conversational AI system
For a focused domain, a conversational AI system must be
driven by machine learning models

For more information
Schedule a customized demo to see Moveworks in action
Follow us on social media
Read the EMA Impact Brief
https://www.moveworks.com/request-demo
https://www.linkedin.com/company/moveworksai/
@moveworks
http://bit.ly/ema-impact-brief

5 AI-Powered ITSM Myths Debunked

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à 5 AI-Powered ITSM Myths Debunked

Similaire à 5 AI-Powered ITSM Myths Debunked (20)

Plus de Enterprise Management Associates

Plus de Enterprise Management Associates (20)

Dernier

Dernier (20)

5 AI-Powered ITSM Myths Debunked