Agile Mumbai 2022 - Ashwinee Singh | Agile in AI or AI in Agile?

1. Agile in AI OR AI in Agile Ashwinee Singh – India Head of Business Agility ashwinee.singh@ust.com November 2022 Agile Mumbai 2022 www.agilemumbai.com

2. Proprietary © 2022 UST Inc 2 INTRODUCTION  Ashwinee is a seasoned Digital Transformation Leader with 23 years of IT Industry experience. Ashwinee has progressed in his career being a Software Engineer, Dev Lead, Technical Manager, Project Manager, Program Manager, Delivery Manager, Transformation Manager, Transformation Director, Agile Transformation Coach and Director.  He has been leading and contributing to some of large scale Agile-DevOps transformations for Fortune clients across the globe (USA, UK, Australia, Switzerland, France, Germany and India). He has been employed with IBM, Capgemini, Cognizant, and Infosys in past.  Ashwinee currently heads UST’s Business Agility Practice for India and Asia as Practice Director.

3. Proprietary © 2022 UST Inc 3 SESSION TOPICS  The AI Revolution  Challenges building AI applications  Does Agile have a solution  Building ML Model  Adopting Agile for building ML Model  Challenges applying Agile for ML  Industry approaches of applying Agile for ML  Evolving an Agile approach for ML  DevOps / MLOps for Model Lifecycle Management  How AI is helping with Agile-DevOps implementation

6. Proprietary © 2022 UST Inc 6 Source: www.forbes.com AI PROJECTS FAILURE RATE ! Gartner: 85% of all AI projects will fail to deliver outcomes in 2022 MIT-SMR: 70% of companies report minimal to no impact from AI MIT-SMR: Less than 2 out of 5 companies reported business gain in last 3 years VentureBeat: 87% of DS projects never make it to production Tom Siebel: 99% of internal AI projects fail

7. Proprietary © 2022 UST Inc 7 WHAT IS CAUSING AI PROJECTS TO FAIL ? Source : www.cognilytica.com Applying application development approaches to data-centric AI Lack of sufficient quantity of data Lack of sufficient quality of data Underestimating time and cost of the data component of AI projects Lack of planning for continued AI, model, data iteration and lifecycle Misalignment of real world data and interaction against training data and models Applying proof of concept thinking to real-world pilots ROI Misalignment of AI solution to problem 1.Vendor misalignment on promise vs. reality 1.Overpromising AI capabilities and underdelivering projects

8. Proprietary © 2022 UST Inc 8 WHAT ARE MAIN AI MODELLING ISSUES FACED BY BUSINESSES ? 66% Lack of clarity on provenance of training data 64% Lack of collaboration across roles involved in AI model development and deployment 63% Lack of AI policies 63% Monitoring AI across cloud and AI environments. Source: IBM’s Global AI Adoption Index 2021

9. Proprietary © 2022 UST Inc 9 WHATS SO DIFFERENT WITH DEVELOPING AI SOLUTION ? 1. Usually many additional layers of unpredictability and unknowns 2. The key focus remains data exploration and insight generation and not just the application development 3. Typical application development solution is built around business rules however data science solution solves problems differently (i.e., not dependent on business rules) 4. Highly evolving and exploratory nature makes it difficult to predict timelines 5. Most of the time goes into data engineering, feature exploration, etc., which is hard to measure tangibly 6. Change in data quality and sufficiency have a significant impact on the end result / outcome 7. Higher complexities of skill set requirements Data Analyst, Expert Analyst (SMEs), Data Engineer, Data Scientists, Analytics / DS lead, Product Manager / Owner, UX designer, ML engineer, etc. 8. The result of Proof of Concept/prototype may vary from the result of a real-world project due to many factors, and interdependencies involved. 9. Defining problem and approach is a complex and iterative process and hard to define clearly upfront 10. Success of AI projects requires planning for continuous management (i.e., monitoring, evaluation, continuous training, deployment, etc.) You may not need AI if the solution can be described in a flowchart or with a set of simple heuristics

10. Proprietary © 2022 UST Inc 10 IMPORTANCE OF WORKING WITH DATA Source - COGNILYTICA “Over 80% of the time enterprises spend on AI projects goes toward preparing, cleaning and labeling data.” Specifically, the report finds that the many steps involved in collecting, aggregating, filtering, cleaning, deduping, enhancing, selecting and labeling data far outnumber the steps on the data science, model building and deployment sides. A recent report from AI research and advisory firm Cognilytica finds that –

16. Proprietary © 2022 UST Inc 16 CHALLENGES ADOPTING AGILE IN BUILDING AI SOLUTION Could not leverage empirical data for estimation and general decision making 1.AI projects do not follow a linear, predictable path Could not forecast number of tasks or size of task / activity hence could not really apply traditional Scrum by estimating and planning Sprint Fixed duration Sprint timeboxing does not wok due to varying degree of logical work breakups requiring constant exploration and experimentation, hypothesis testing, requires research and learning ML modeling is stochastic where the outcomes are characterized by probabilities so often it is not possible to define end deliverable / outcome The measure of success is hard to define upfront. AI / ML models and their components (code, trained data, parameters, hyperparameters, etc.) are not end objective, but they are just enablers in delivery of a suitable ML solution The AI-ML Model isn’t built around expectation of satisfying customer / user needs rather to see how given data could be best utilized to help with any business needs which often is not known in advance

19. Proprietary © 2022 UST Inc 19 DESIGNING PRINCIPLES FOR BLENDED AGILE METHOD Flexible length Sprints working in Kanban fashion: o Sprints may be too short to work out a messy modeling algorithm, but too long if exploration quickly indicates a need to pivot o Small increments with systematic, frequent validation are required to assess the degree to which it is addressing the business problem. o If necessary, rapidly abandon one concept and pursue another, communicate and set the expectation from the solution, including the degree to which it is addressing the problem o Much of the work is far removed from end-users, so feedback must be gathered in many forms, not just sprint demos All the phases of the project should be mutually iterative; progression backward or forwards should be allowed (Unlike CRISP-DM and other approaches) o (e.g., it is highly likely that unknown problems of data understanding are identified while making the model, the approach should allow going back to the data understanding phase) AI data projects should be approached from the top-down and bottom-up directions. The outcomes of AI efforts depend on finding the overlap between o What is possible based on the information that can be extracted from the underlying data? o What is desirable, based on identifying actionable, timely decision support for business stakeholders?  We suggest a blended methodology that adopts the relevant and fitting aspects of Agile and other approaches  We can use phases and iterative approaches of the data-centric methodologies like CRISM-DM and extend it for each business requirement / customer need  Use of DataOps, MLOps for automation and management of data sets and algorithmic models and the entire process

20. Proprietary © 2022 UST Inc 20 EVOLVING AGILE METHOD FOR BUILDING ML MODEL Product Owner Data Scientist Agile-ML Process Lead Data Engineers Data Analysts Software Engineers Business Analysts Definition of Ready (DoR) Definition of Done (DoD)

21. Proprietary © 2022 UST Inc 21 EVOLVING AGILE METHOD FOR BUILDING ML MODEL Product Owner Data Scientist & Data Engineers Agile-ML Process Lead  Defines all data related backlog items  Defines acceptance criteria for data related PBIs  Defines all non-data related backlog items  Defines Business Value and prioritization  Works as process orchestrator for the entire Team  Manages flow of work and maintains Kanban Task Board Work-in-Progress (WIP) limits  Organises and moderates all team events and maintains team cohesion Definition of Done (DoD) Definition of Ready (DoR) Kanban Task Board

22. Proprietary © 2022 UST Inc 22 KEY PRINCIPLES AND PRACTICES OF AGILE METHOD  Team to practice Agile ScrumBan approach based on milestone activities which varies very much in terms of effort and time (Example – Data Cleaning, Data Labelling, Model Evaluation, etc) without any fixed timeboxed sprints  Team work focus effectively managed by close collaboration across Troika roles of Data Scientist, Product Owner and Agile-ML Process Lead  Integrated Product Item Backlog is managed by team which has both Data related items as well Business and other Technical task items added and tracked through Kanban Task Board  Data driven activities are performed more as exploratory / research activities with lots of unknowns under Data Scientist Leadership  Agile-Scrum events of daily stand-up and Retrospectives are followed with a cadence  Agile-Scrum events of Demo and Review are done on event basis  Business activities are planned and worked by team under Product Owner Leadership with clear focus on applicability of ML Solution for business  Model Definition of Done (DoD) is established which ensures that model is trained properly, data is correct to accurately predict the future or in a state to get a correct answer for the given business problem it is trying to solve Product Owner Data Engineers & Scientist Agile-ML Process Lead Role Troika

23. Proprietary © 2022 UST Inc 23 COMORBIDITY PREDICTION ML SOLUTION USING AGILE METHOD Client Context A large healthcare player based in Indiana, US Solution Objectives • Predict progression of comorbidities in members with: • Type-2 Diabetes Mellitus (T2DM), • Chronic Kidney Disease (CKD) • End Stage Renal Disease (ESRD) • Identify members at high risk of developing comorbidities Solution Approach • Solution developed using historical claims, member diagnosis conditions, drug refill history, member laboratory tests, and member demographics information to understand the risk of developing a comorbidity in different time periods • For each primary disease, set of relevant comorbidities are chosen based on exploratory data analysis and clinical research inputs • Developed 85 models to access risk of developing comorbidities for each combination of primary disease, comorbidity and time period • Solution provides top contributing factors towards risk score • Solution Development team comprised of Data Scientist, Product Owner, Process Lead, Data Engineers and Data Analysts • Solution built in about 16 weeks timeframe leveraging Agile-Kanban ways of working Benefits Realization - The developed solution helped our client to : • Accurately identify members with high risk of developing comorbidities and the underlying factors with 90% accuracy • Implement targeted focused interventions at a member-level • Reduce cost of expensive procedures Courtesy: Abzooba

24. Proprietary © 2022 UST Inc 24 FRAMEWORK FOR BUILDING ML SOLUTION Courtesy: xpresso.ai 1. Define the problem and get the data needed 2. Figure out how you will measure success 3. Figure out how you will evaluate that success 4. Prepare your data 5. Develop a model incrementally 6. Scale up and improve your model 7. Tune your parameters and regularize your model

27. Proprietary © 2022 UST Inc 27 APPLYING AI FOR AGILE-DEVOPS IMPLEMENTATION Product Development Product Discovery 1 2 3 4 1 2 3 4 NLP being used to write User Stories and establish initial Product Backlog (https://userstorygenerator.ai/). Initial Product Backlog word2vec, paragraph2vec, Long Short-Term Memory (used in Google Translate), or Convolutional Neural Networks (used in Facebook’s DeepText engine) can generate dense vector representations that produce superior results on various NLP tasks Deep Learning as Tool NLP component which performs automatic analysis on textual artifacts and then generates vector representations of those artifacts NLP as Tool AI generated data can be compiled and summarised to provide product owners and other business stakeholders insights, planning and prioritizing features and bug fixes for future releases Insights for PO Code Modelling component is responsible for learning vector representations which reflect the semantic and syntactic structure of source code and is used often in IDEs Code Modelling Machine learning applications can absorb data streams from various DevOps Telemetry Tools to find correlations & generating more insightful view of the application’s overall health and useful foresights Telemetry Analysis Applying AI or machine learning algorithms to these different type of testing results could identify patterns of poor coding practices that result in too many errors caught by the tests Testing Augmentation Sophisticated Code Generator tools like GitHub’s Copilot are generating as much as 30% of new code by using AI for some programming languages Code Generation Product Backlog

28. Proprietary © 2022 UST Inc 28 Summary KEY TAKEAWAYS ON COUPLING AI AND AGILE  AI-ML Applications development is too data-centric and hence does not get benefitted by just following the regular Agile methods which have worked well for Software Development  Evolving Agile model around ScrumBan approach inspired from CRISP-DM, Data-Driven-Scrum, IBM-DES and Microsoft’s TDSM seems promising and works better with contextualization  Formalizing key Troika roles of Data Scientist, Product Owner and Agile-ML Process Lead helps keeping the right focus and balance across Data and Application Development activities thus increasing the business benefits  Figuring out your measures of success while developing ML models is crucial  The practices - AIOps and MLOps - both play a significant role in aiding businesses in achieving operational efficiency. MLOps brings agility by bringing machine learning model into production and managing it, whereas AIOps brings agility by using AI-ML and big data to automate IT / project operations. Hence, they both contribute to “Agile in AI and AI in Agile.”)  Increasing number of AI-ML techniques like NLP are proving very useful in increasing the effectiveness of Agile-DevOps implementations

29. Thank you ust.com Ashwinee.Singh@ust.com

31. Proprietary © 2022 UST Inc 31 At UST, we believe in the power of technology to engineer a better future 22+ 30+ 30K Years in business We have been bringing technology to life for our clients for over 20 years Countries We operate in 30+ countries with over 34 delivery centers and 42 operating centers Employees We have over 30,000 associates committed to your success 140+ 7+ We are privately held with an investment from one of the largest PE funds in the world Clients We serve over 140 Global 1000 clients Industries Healthcare Life Sciences Retail & CPG Semiconductor Manufacturing Financial Services Technology, Media & Telecom Travel & Hospitality

32. Proprietary © 2022 UST Inc 32 Together, we create successful outcomes for our customers • We believe in building long term partnerships with clients • More attention paid to client success at all levels • Commitment beyond contract – to deliver business outcomes • Flexible contractual models • Consistent executive attention • Dedicated account team • Client feedback drives investments up front and throughout the journey 32 13Average tenure of client relationships

33. Proprietary © 2022 UST Inc 33 A digital leader with a robust portfolio of solutions Leader in digital services focusing on tangible outcomes through meaningful innovation CUSTOMER EXPERIENCE & AGILE • Business Agility Services along with distributed Agile Development • Digital solutions with human-centered design • Intelligent process automation ANALYTICS • World class data engineering skills • Artificial Intelligence powered solutions for deep insights and predictive capabilities • Machine Learning for automating functions and conversion of data to actionable insights CYBERSECURITY & BUSINESS RISK • AI driven vulnerability assessment for threat detection and scoring • Rule driven threat remediation recommendation and automated resolution • Selective business risk analysis and solutions LEGACY MODERNIZARION & CLOUD ENABLEMENT • AI powered application analysis and mapping tools • Infrastructure and application cloud migration capabilities as well as tools to assess and optimize efficiency of an environment • Methodology and tools for Cloud Native for rapid application delivery INNOVATION & EMERGING SOLUTIONS • Innovation Pods for rapid and repeatable innovation • Use of emerging technologies (blockchain, quantum computing, AR/VR) to address business needs • Comprehensive digital product design solutions for new products and services Comprehensive portfolio of services for DESIGN, BUILD and OPERATE (“DBO”)

34. Proprietary © 2022 UST Inc 34 Global business agility services • UST Agility Consulting centers are located in the US, Europe, Australia, and Latin America with engagements around the world • Agile Solution centers operate out 15+ countries • 150+ Agile/DevOps Coaches, 3500+ Scrum Master and thousands of practitioners • Agile leadership in each region works through a unified vision and integrated approach Aliso Viejo Calabasas New York Norfolk Atlanta Dallas Chicago Pittsburgh Bentonville Austin Toronto London Ireland Geneva Madrid Copenhagen Oldenburg Singapore Shanghai Penang Taiwan Hong Kong Tokyo Trivandrum Kochi Coimbatore Bangalore Chennai Pune Mumbai Delhi Sydney Active Agile engagements in: Chile Argentina Venezuela Peru Colombia Active Agile engagements in: Amsterdam London Leeds Glasgow Madrid Barcelona Budapest Copenhagen Active Agile engagement in: Trivandrum Kochi Chennai Bangalore Active Agile engagement in: Mexico City Guadalajara Leon Costa Rica Active Agile engagement across the US Turkey Costa Rica Active Agile engagement in : Sydney USA India China Spain Germany Mexico Singapore Philippines Malaysia Taiwan Turkey Colombia Australia UK

35. Proprietary © 2022 UST Inc 35 Copyright and confidentiality notice Copyright © 2022 by UST Global Inc. All rights reserved. This document is protected under the copyright laws of United States, India, and other countries as an unpublished work and contains information that shall not be reproduced, published, used in the preparation of derivative works, and/or distributed, in whole or in part, by the recipient for any purpose other than to evaluate this document. Further, all information contained herein is proprietary and confidential to UST Global Inc and may not be disclosed to any third party. Exceptions to this notice are permitted only with the express, written permission of UST Global Inc. UST® is a registered service mark of UST Global Inc. UST 5 Polaris Way Aliso Viejo, CA 92656 T +1 949 716 8757 F +1 949 716 8396 ust.com

37. Proprietary © 2022 UST Inc 37 WADING THROUGH COMPLEXITY From Stacey’s Complexity Matrix Simple Domain • Everything is Known • Sense – Categorize-Respond Complicated Domain • More known than unknown • Sense – Analyze-Respond Complex Domain • More unknowns than known • Probe – Sense – Respond Chaotic Domain • Very little is known • Act-Sense-Respond Requirements Technology Known Predictable Certain Uncertain Simple Complicated Complicated Complex Empirical Process Emergent Practices Agile Servant Leadership Good Practices Lean – Six Sigma Supervision Best Practices Waterfall C&C Unpredictable Chaotic Novel Practices Lean – Build-Measure- Learn Entrepreneurship

Agile Mumbai 2022 - Ashwinee Singh | Agile in AI or AI in Agile?

Vous êtes en train de lire un aperçu.

Consultez-les par la suite

Agile Mumbai 2022 - Ashwinee Singh | Agile in AI or AI in Agile?

Plus De Contenu Connexe

Similaire à Agile Mumbai 2022 - Ashwinee Singh | Agile in AI or AI in Agile? (20)

Plus par AgileNetwork (20)

Plus récents (20)