Agile Mumbai 2019 Conference | Intelligent DevOps enabling Enterprise Agility through power of Automation and Cognitive Intelligence |Harsha Vardhan

@ Agile Network India , All Rights Reserved. www.agilenetworkindia.com
Intelligent DevOps enabling
Enterprise Agility
Harsha Vardhan
Director - Software Engineering
Optum Global Solutions
The contents in this presentation is prepared by Harsha Vardhan in his personal capacity and to the best of his knowledge. No lia bility
whatsoever for the accuracy and completeness of the above information ishowsoever assumed. The opinions/ contents expressed/
provided in this article are the author'sown and do not reflect the views of the Optum Global Solutions(India) Private Limited
(“company”). Neither company nor the author shall not be held liable for any improper or incorrect use of the information described and/or
contained herein and assumes no responsibility for anyone's use of the information.”

Agenda
DevOps relevance in Digital world
Scaling DevOps
DevOps Systems that Do -> Think -> Learn
Q&A

@ Agile Network India , All Rights Reserved. www.agilenetworkindia.com@ Agile Network India , All Rights Reserved. www.agilenetworkindia.com
Kuwait: A Desert on Fire © Sebastião Salgado
5 years to 18 months - it’s all about Agility ++

Business Value driven by DevOps

Raise your bar - Be contextual …
*ApplyingAgile andDevOpsprinciple at scale by Gary Gruver

Raise your bar !!

Automation +
Speed +
Collaboration
Adaptive
Automation +
Predictive outcomes
+ CX
Continuous journey (Continuous Integration, Continues Testing,
Continuous Deployment, Continuous Operations etc.) of DevOps to
become Intelligent i.e. Intelligent Integration, Intelligent Testing, Intelligent
Deployment and Intelligent Operations
Intelligent DevOps

Code
Management
Code Merge Packaging
Build (CI)
Server
Unit Testing
Code
Coverage
App Config Publishing Deploy
Auto Test/
ATDD
Dashboard
/ Reporting
Publishing
Dashboard/
Reporting
Smoke
Testing
System/
Integration
SecurityScan
Dashboard/Reporting
Network
Config
Continuous Testing
Continuous Delivery
Continuous Integration
Non-Functional/
Performance
Dev Tech Lead Tech Lead Architect Dev/Architect Dev/Architect
Architect Architect Dev/ArchitectSDETSDETSDETSDET
Architect Architect Architect Architect Architect ArchitectDev/SDET
Medium High One Time ContinuousLow
Efforts /
Complexity
Regression
1 1 1 1 3 1
2113
3
3 3 3
3 3 3 3 3 3
DevOps Systems that Do

In the spirit of the “Continuous” philosophy in DevOps, we envision a new
solution approach around “Continuous Insight” where analytical insights
are generated and acted upon continuously (and autonomously) as
processes execute
Incident Data
Log Data
Monitoring data
NPS data
Continuous Insight

DevOps Systems that Think

Traditional CAB ( Change Advisory Board)
Vs Change Predictors

Current Culture
“You are only as good as your last failure”
..Restrict
.. Inconsistent
..Govern
..Focusing on the Negative
Future
“Balancing risk and velocity & focus on outcomes”
..increased throughput & flexibility
..consistency
..focus on the right changes
..focus on success & learn from
failure
Turning the corner on Change Culture

How might we leverage machine learning to
identify changes with the highest potential risk for
negative outcomes?

@ Agile Network India , All Rights Reserved. www.agilenetworkindia.com 1
ITSM
Datamart
Data Ingestion
Completed Changes
(Scheduled Batch)
Model Building,
Training, Re-Training,
and Validation
Additional insights to
supplement model
Trained Model
ET
Data Lake
New change
request
Enhanced Risk
assessment
Machine Learning
Model API
Service Now API
Planned Changes
(real-time)
Web Service
Dashboard updated daily
result
input
Source code
Data Processing / Transformation
NLP
Solution Architecture

• Performance metrics, several models, pick the best
• Gradient-BoostedTrees implementation  XGBoost
• Ensemble  combines many “weak” learners to create strong learner
TPR
FPR
Algorithm AUC Precision Recall F1
Logistic Regression 0.825 0.162 0.135 0.147
Decision Tree 0.831 0.192 0.176 0.184
Random Forest 0.854 0.213 0.191 0.201
Gradient Boosted Trees 0.906 0.292 0.269 0.281
Neural Network - LSTM 0.848 0.089 0.176 0.142
Test-set performance metrics
Higher risk
Model Training & Selection – The Recipe

Via Machine Learning, we can drive down our cost per incident from Low
Risk Changes.
Scenario :
• We average about 10 incidents from low risk changes per month. Currently the machine
learning is upgrading 10% (1) of those changes to high or critical Risk. This higher risk
profile brings about more review and planning. This in turn leads to more positive
outcomes.
• Making those assumptions, we estimate that we can save approximately $17,000 per year
month or about $204,000 annually from reduction of P1/2 incidents.
Additional Insights

Availability
Traditional approach of uptime vs Raw Error Rate
An availability metric that is an accurate indicator of
true customer experience

HttpResponse RER Dashboards
RER
Dashboard
Inputs
Response Codes
URI
Elapsed Time
Error Rate measuring isn’t new:
•Companies like Google uses Error Rates as a key part of their Error Budgets for
managing risk and their SRE program
•Apps exceeding their ’budget’ aren’t allowed to release features
• In the simplest form, Raw Error Rate is:
[successful transaction count / total transaction count]
• Successful transaction do not include transactions that exceed the response
SLA
Raw Error Rate Example: (Example: outage@ 2 AM vs 2 PM with a 1 hour outage)
ADTM: 98.9% availability for the day where as
Raw Error Rate: An application that serves 10,000 requests in 24 hours
issues 30 errors over an hour at 2am. 99.7% availability for the day.
Continuous Monitoring using RER

Why enhance monitoring?
Customer Experience Monitoring

Sec + Ops
Proactively mitigation of security risks with
automation and holistic visibility

AI integration in DevOps yielding useful outcomes in SecOps (e.g.
preventing Denial of Service )
Bot to detect abnormal traffic
Captcha (technique to distinguish between humans and computers)
Preventing DoS

CX-driven DevOps (or CX-Ops) that requires data analytics and cognitive
approaches (such as natural language processing or NLP) to decode
meaningful insight
Transactional NPS feeding into log monitoring
Monitoring triggers drives Bots / Agents availability for chat / call
Peak Season readiness
DevOps System that learns

Intelligent DevOps through Continuous Insight
Investment
decision
analytics
CX-based
requirements
Code quality
analytics
Test Optimization
& Defect
Prediction
A/B Testing
Log & CX
Analytics
Portfolio backlog
prioritization & value
analysis.
Improved Scheduling
& Capacity
Validated backlog
requirements.
Improved estimates
Predict failures
based on code
commit.
Dynamic code
promotion criteria
Predict defects
Optimized tests
Scenario
identification based
on trends
CX scores
Anomaly detection
Better load
balancing
Capture
Phases/Tools
Details
Capture
Environment
Details
Map Phases/Tools
With
Environments
Create
Pipeline Job
DevOps PitStop Process Flow
Tools Assessment
& Selection
Tool Chain Framework (Set-up the tools using DevOps PitStop)
ExecutionFramework(Run the automated phases using DevOps PitStop)
Intelligent DevOps

Engineering Excellence
Optimize for iteration
speed
Push relentlessly toward
automation
Build the right software
abstractions
Develop a focus on high
code quality withcode
reviews
Maintain a respectful
work environment
Build shared
ownershipof code
Invest in automated
testing
Build a culture of
learning and
continuous
improvement
Hire the best talent
Agile Engineering Practices
Its all about Software Engineering Culture…

Thank You

Agile Mumbai 2019 Conference | Intelligent DevOps enabling Enterprise Agility through power of Automation and Cognitive Intelligence |Harsha Vardhan

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Agile Mumbai 2019 Conference | Intelligent DevOps enabling Enterprise Agility through power of Automation and Cognitive Intelligence |Harsha Vardhan

Similaire à Agile Mumbai 2019 Conference | Intelligent DevOps enabling Enterprise Agility through power of Automation and Cognitive Intelligence |Harsha Vardhan (20)

Plus de AgileNetwork

Plus de AgileNetwork (20)

Dernier

Dernier (20)

Agile Mumbai 2019 Conference | Intelligent DevOps enabling Enterprise Agility through power of Automation and Cognitive Intelligence |Harsha Vardhan