Identifying and Categorizing Flapping Events to reduce resource wastage and service loss

1. Machine Learning in Infrastructure Monitoring:
Identifying and Categorizing Flapping Events to
reduce resource wastage and service loss

2. Confidential & Restricted
Copyright © Prodapt Solutions
Background
This insight will focus on how machine learning helps data centers identify and categorize flapping events
in order to reduce resource wastage and service loss
Approaches to handling flapping events
Discussed in the previous
insight:
What data centers need to know
about flapping events
The impact of flapping events in infrastructure monitoring
Top flapping events by occurrence
Root causes of event flapping
• Addressing flapping events as and when they occurReactive
• Configuring/resetting the thresholds to reduce the volume of flapping
events
Proactive
• Predicting and categorizing the events as flapping or non- flapping
leveraging machine learning to minimize resource wastage and service
loss
Predictive

3. Confidential & Restricted
Copyright © Prodapt Solutions
Reactive approach: Addressing flapping events as and when they occur
A typical workflow in handling the events in a reactive approach method:
Step
In reactive approach, teams
start working immediately
after getting event
notification. Whether it’s a
flapping or non- flapping
event, the resource
assignment is done and
trouble shooting begins.
Hence, this approach involves
resource wastage as well as
service loss in some cases,
based on event’s nature.

4. Copyright © Prodapt Solutions
Proactive approach: Updating/resetting the thresholds to reduce the volume of
flapping events
Setting thresholds which are too low can cause unnecessary event triggers.
Storage-related alerts Timeout-related alerts
Identifying the events’ behavior and proactively increasing threshold limits for those events accordingly can
reduce the number of flapping events significantly. This decreases the resource wastage as monitoring teams get
fewer flapping events. However, service loss may still continue because of the fewer flapping events which still
affects the visibility of the genuine events.
It has been observed that typical storage-related alerts
are set to a very low threshold (e.g., 80%) causing a lot
of events. Reconfiguring the alarm thresholds to a
higher permissible number (e.g: 90%) based on the
field experience can reduce noise.
Setting timeout related alerts too low can cause the
system to throw alerts even as the response is not
really ‘timed-out’ but delayed. Low threshold timeout
triggers a lot of events like- site down, network
latency, port not responding etc.

5. Copyright © Prodapt Solutions
Predictive approach: Identifying and categorizing flapping events to minimize
resource wastage and service loss
Machine Learning-based solution to identify and categorize flapping events
Monitoring teams maintain
events log that contains
event related information.
Events log is in natural
language.
Using NLP techniques,
the Intelligence system
gains power to read
and process "Events
logs(Text)" as humans
do
With the combined
power of NLP and
machine learning
algorithm, system learns
to classify the events into
flapping/ non-flapping by
identifying hidden /
underlying patterns in the
events.
once the new event arrives
into the system, with the
previously acquired
intelligence, system reads
the events in textual
format and classifies them.
Event Logs
XGenuine
Event
Flapping
Event
Building Intelligence

6. Copyright © Prodapt Solutions
Big data frameworks help in gathering and feeding huge volumes of data in to
the Machine Learning system
Gathering
information with the
help of big data
frameworks
Output: Categorizing
events in
flapping/non-flapping
and extrapolate the
information
Enabling machine to
read information and
identify patterns
based on algorithms
3
1
2
Big data frameworks: Helps
the system in gathering a
variety of data including:
Event logs
Customer Relationship
Data Base (CRDB )
Network element
information
Inventory information
Machine Learning and Natural Language
Processing: System reads and analyses all
the information passed by big data
frameworks like:
Events log
Events nature, patterns, severity,
cascaded events
Client information and SLA level
With the help of pattern matching
algorithm and analysed data, the system
predicts future events, their nature,
severity, sub-events, associated client etc.
Event: XXXX
Nature: Flapping
Severity: Low
Cascading: No
Client: ABYZ

7. Copyright © Prodapt Solutions 7
Use Case: How Machine Learning analyzes and categorizes the nature of events
Objective: To demonstrate how
flapping events can be identified
and categorized leveraging
machine learning and natural
language processing
Event type: Device Failed
Availability Check: Component
Device XXXX is not available
Machine Learning
algorithm used:
Gradient Boosting
Machine
Component device
44676 is not available
UDP – SNMP
Event 2:
Device Failed
Availability
Check:
UDP – SNMP
Event 2:
Device Failed
Availability
Check: UDP – SNMP
Nature: Flapping
Action: No Action
Required
Device Failed
Availability
Check
Common Part
Event 1: Device
Failed Availability
Check: Component
device 44676
Is not available
Unique Part
OUTPUT
Event 1:
Device Failed
Availability Check:
Component device
44676 is not available
Nature: Non – Flapping
Action : Assign
Ticket
OUTPUT
Leveraging Machine
Learning to identify
and categorize events,
data centers can
reduce resource
wastage
by approximately 50%
and service loss by
approximately 10%
Unique Part Machine Learning
System
Gradient boost
Algorithm
Bag of
words for
Flapping
Events
Bag of words
for Non-
flapping
Events
Corpus
System checks for
unique
part in the
combined corpus
and categorizes
the events
(flapping or non –
flapping)
Sample events:

8. Chennai
Johannesburg
New York
Dallas
Tualatin
Amsterdam
London
THANK YOU!
Prodapt Solutions Pvt. Ltd.
INDIA
Chennai:
1. Prince Infocity II, OMR
Ph: +91 44 4903 3000
2. “Chennai One” SEZ, Thoraipakkam
Ph: +91 44 4230 2300
SOUTH AFRICAUSA
Prodapt North America
Tualatin: 7565 SW Mohawk St.,
Ph: +1 503 636 3737
Dallas: 222 W. Las Colinas Blvd., Irving
Ph: +1 972 201 9009
New York: 1 Bridge Street, Irvington
Ph: +1 646 403 8158
Prodapt SA (Pty) Ltd.
Johannesburg: No. 3,
3rd Avenue, Rivonia
Ph: +27 (0) 11 259 4000
THE NETHERLANDS
Prodapt Solutions Europe
Amsterdam: Zekeringstraat 17A, 1014 BM
Ph: +31 (0) 20 4895711
Prodapt Consulting BV
Rijswijk: De Bruyn Kopsstraat 14
Ph: +31 (0) 70 4140722
UK
Prodapt (UK) Limited
Reading: Davidson House,
The Forbury,
Reading RG1 3EU
Ph: +44 (0) 11 8900 1068
Bengaluru
Bangalore: “CareerNet Campus”
No. 53, Devarabisana Halli,
Outer Ring Road