Ce diaporama a bien été signalé.
Le téléchargement de votre SlideShare est en cours. ×

Agile Mumbai 2022 - Adish Apte & Ashish Sharma | AI/ML Powered & Insights Fuelled Software Delivery Acceleration

Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité

Consultez-les par la suite

1 sur 22 Publicité

Plus De Contenu Connexe

Plus par AgileNetwork (20)

Plus récents (20)

Publicité

Agile Mumbai 2022 - Adish Apte & Ashish Sharma | AI/ML Powered & Insights Fuelled Software Delivery Acceleration

  1. 1. AI/ML Powered & Insights fueled software delivery acceleration Adish Apte & Ashish Sharma Dhairya Singhvi | Engineer | IIOT | Airoli – Title: Heroic Horse In Artist’s own voice: Idegorgeous animal! Horses have always been of great help to the humankind, in terms of transportation or even during wars. I try to capture the personality of this creature and relate it to the work we do!a is to reflect the resilience and speed this The content of this document is confidential and intended for the recipient specified in this message only. It is strictly forbidden to share any part of this message with any third party, without the written consent fo LTI. LTI has filed patend for this IP and reserves the right, title and interest in the software/solutions included and demonstrated herewith. AGILE MUMBAI 2022 CONFERENCE Theme – AI for Business Agility Date: 5 - 6 November, 2022 Venue: AC Patil CoE, Kharghar, Navi Mumbai
  2. 2. LTI Proprietary and Confidential ©Larsen & Toubro Infotech Ltd. Privileged and Confidential https://www.agilemumbai.com/ Takeaways of this Session 2 AI assisted software delivery is a very powerful idea whose time has come Urgency to act in introducing these new competencies within their software deliveries Quality data is critical to ensure the insights delivered are relevant and accurate That AI / ML is delivered probabilistic outcomes and needs human intervention to override as the case may be
  3. 3. LTI Proprietary and Confidential ©Larsen & Toubro Infotech Ltd. Privileged and Confidential https://www.agilemumbai.com/ AI in everyday life ? 3
  4. 4. LTI Proprietary and Confidential ©Larsen & Toubro Infotech Ltd. Privileged and Confidential https://www.agilemumbai.com/ AI in daily life 4 Fun quiz post lunch to shake you up and not stir ☺
  5. 5. LTI Proprietary and Confidential ©Larsen & Toubro Infotech Ltd. Privileged and Confidential https://www.agilemumbai.com/ How can we leverage AI in enhancing Agile Software Delivery Productivity? 5
  6. 6. LTI Proprietary and Confidential ©Larsen & Toubro Infotech Ltd. Privileged and Confidential https://www.agilemumbai.com/ I could see further since I was sitting on the shoulders of giants. Attributed to Isaac Newton 6 Compute Algos Data
  7. 7. LTI Proprietary and Confidential ©Larsen & Toubro Infotech Ltd. Privileged and Confidential https://www.agilemumbai.com/ Typical large program in an Enterprise Scrum Teams 20 Sprints / Year 20 User Stories / Sprint ~10 Program 2 years Total User Stories ~8000 Data 7
  8. 8. LTI Proprietary and Confidential ©Larsen & Toubro Infotech Ltd. Privileged and Confidential https://www.agilemumbai.com/ What does this mean in terms of data Defects Test Cases Code Files User Stories ~100 lines 10-20 lines 3-5 lines 8000 User Story 24,000 Test Cases 75,000 defects 1.25 Mn. LOC 6 Modules Scrum Teams 20 Sprints / Year 20 User Stories / Sprint ~10 Program 2 years Total User Stories ~8000 Data 8
  9. 9. LTI Proprietary and Confidential ©Larsen & Toubro Infotech Ltd. Privileged and Confidential https://www.agilemumbai.com/ Evolution of NLP techniques N-gram 1948 1982 RNN 1997 LSTM 2003 1st feedforward NN Language model 2011 IBM Watson beats Jeopardy 2014 GloVe 2017 Transformer 2018 BERT 2020 GPT3 Bag of words TF-IDF Word Embeddings 1999 LSA 2003 LDA – applied in ML 2013 Word2Vec 2009 GPUs in deep learning Shallow Representations Sequence to Sequence 2015 FastText Large Language Model Text summarization, Classification – Spam, Sentiment Analysis Cons: Semantic relations between words Analogies, next word prediction, recommendation engines Cons: Context in long sentences, transfer learning Q&A, Chatbots, Language Translation, Image captioning, Natural Language inference Cons: Bias, illogical outcomes 2012 Deep learning & image classification 2022 PaLM 2021 MT-NLG Algos 1952 Stochastic Gradient Descent 9
  10. 10. LTI Proprietary and Confidential ©Larsen & Toubro Infotech Ltd. Privileged and Confidential https://www.agilemumbai.com/ Tricks almost human - PaLM 10 PaLM: Scaling Language Modeling with Pathways, Aakanksha Chowdhery∗ Sharan Narang∗ Jacob Devlin∗ - Google Research Algos
  11. 11. LTI Proprietary and Confidential ©Larsen & Toubro Infotech Ltd. Privileged and Confidential https://www.agilemumbai.com/ Rise and rise of GPU 11 datapoints gpu time cpu time epochs batch_size 5000 6min 26s 10hr 5min 3 16 1000 1m 18s 1 hr 1 min 3 16 500 39.9s 55min 48s 3 16 100 9s 13 min 6s 3 16 datapoints gpu time cpu time epochs batch_size 5000 2min 42s 56min 1 8 1000 34s 10min 2s 1 8 500 19.7s 3min 57s 1 8 100 7.85s 35.8s 1 8 Sentence Transformer Finetuning Benchmark Sentence Transformer Embedding Benchmark datapoints gpu cpu 1000 1.95s 32.4s 2000 3.7s 1m 4s 5000 9.11s 2m 42s 10000 17.9s 5m 19s 20000 35.4s 13m 47s datapoints gpu cpu 1000 3.59s 1m 1s 2000 6.46s 1m 56s 5000 15.9s 4m 50s 10000 31.6s 10m 11s 20000 1m 3s 19m 59s Supervised dataset dbpedia_sample_abstract_20k_unprep dataset dbpedia_sample_abstract_20k_unprep Unsupervised GPU Machine: 11th Gen Intel i7 Win 10 NVIDIA RTX 3070, 8GB GPU CPU Machine: 11th Gen Intel(R) Core(TM) i7, 16GB Compute
  12. 12. LTI Proprietary and Confidential ©Larsen & Toubro Infotech Ltd. Privileged and Confidential https://www.agilemumbai.com/ There is a growing need for AI assisted software delivery acceleration 12 Tools & Automation Tools & Automation • Planning Tools • Collaboration Tools • Test Automation • DevSecOps Automation • Incident SOP Automation • Monitoring Tools Architecture Architecture • Cloud Native • Microservices • Low Code / No Code Platforms • Apification Ways of Working Ways of Working • Distributed Agile • DevSecOps • Site Reliability Engineering (SRE) • Hybrid Working Opportunities to deliver Step Improvement in Software Delivery • SDLC Digital Assets • Leverage of AI in Software Delivery • NLP algorithms sophistication • Insights driven decision making Insights Fueled Software Delivery Acceleration White Space – Our Innovation !! Industry Levers for software delivery acceleration
  13. 13. LTI Proprietary and Confidential ©Larsen & Toubro Infotech Ltd. Privileged and Confidential https://www.agilemumbai.com/ Value Delivery 13 15-20% Reduction in Defect Injection Rate >60% reduction in time spent on defect/ incident resolution 10X faster Change Impact & dependency Analysis 30-40% Optimize Regression Cycle by 20-30% Improvement in Application Availability >80% accuracy in Defect & Hotspot (Risk) prediction Empowering Software Personas to be Smarter, Faster, and Resilient…
  14. 14. LTI Proprietary and Confidential ©Larsen & Toubro Infotech Ltd. Privileged and Confidential https://www.agilemumbai.com/ API Marketplace Varied and Voluminous SDLC digital assets from the tools User Stories Logs Files Session Traces Config Files Business Req. Code Metadata Build Stats Test Cases Deployment Stats Defects Performance Stats Infra Telemetry Incidents Config Files ….. ……. ML/NLP Algorithms led correlated Knowledge Fabric Data Extraction and Ingestion Data Integration and Correlation Insights Derivation Data Delivery and Presentation Driving Insights Fueled Acceleration via API marketplace and Canvas Solutions CORRELATE COLLECT ACCELERATE 14 What are we doing ? Building E2E Knowledge Fabric leveraging differentiated correlation approach and state of the art ML/NLP algorithms Product Owner | Scrum Master | Developer | Quality Engineer | Site Reliability Engineer | Operations Engineers
  15. 15. LTI Proprietary and Confidential ©Larsen & Toubro Infotech Ltd. Privileged and Confidential https://www.agilemumbai.com/ Assisting Engineering Personas to Accelerate… 15 ✓ Evaluating the risk of introducing a change request ✓ Team onboarding & knowledge acquisition ✓ Identify cross-app dependencies I am Paul, a Product Owner & I want to accelerate ✓ Identifying code dependencies ✓ Understanding code hotspots and branch merge analysis ✓ Time taken for resolving defects I am Ritu, a Lead Developer & I want to accelerate ✓ Optimize regression test case suite ✓ Identifying defect-prone features ✓ Analyzing opportunities for automation I am Kristi, a Quality Engineer & I want to accelerate ✓ Time taken for sprint planning ✓ Measuring pod productivity ✓ Identifying dependencies between scrum teams I am Gary, a Scrum Master & I want to accelerate ✓ Orchestrating chaos attacks based on change introduced ✓ Time taken for triaging problems & issues ✓ Correlating infra telemetric & business transactions I am Steve, an SRE & I want to accelerate
  16. 16. LTI Proprietary and Confidential ©Larsen & Toubro Infotech Ltd. Privileged and Confidential https://www.agilemumbai.com/ Empowering the personas across the product engineering lifecycle with its Insight-led use cases 16 Assisted Resilience Engineering Defect Resolution Advisory AIOps - Intelligent Operations Situational Intelligence Change Impact Analysis Optimized Regression Testing Assisting Product Owner and Scrum Master with insights to analyze the risk of introducing a change Assisting Architects and Developers with embedded insights based on intended code change Assisting Developers and Quality Engineers with historical insights to accelerate defect resolution Assisting ITOPs managers and Support engineers for Faster issues resolution with AI- driven recommendation Assisting Quality Engineers with impact- based regression test suite design based on code change Assisting SRE, Architects and Developers with resilience hotspots and predictive insights on business resilience
  17. 17. LTI Proprietary and Confidential ©Larsen & Toubro Infotech Ltd. Privileged and Confidential Scaling DevOps via Insight Led Acceleration for a Wealth Management SaaS Provider in North America ✓ Impact of a code change on regression suite is unknown ✓ Selective regression is not possible ✓ Defect turnaround time is high impacting sprint velocity Lead Developer ✓ We are not able to do In-Sprint regression testing ✓ Regression bloat: each release, regression suite count is increasing by >5% Quality Engineer ✓ Regression issues are identified late in the cycle ✓ Increased risk of project delivery timeline ✓ SME dependency is High Business Sponsor ✓ End of day build pipeline doesn’t include any regressions testing ✓ Current Regression suite runs for more than 9 hours ✓ Recurring failures in daily build DevOps Engineer We started with listening to our client persona’s challenges !! Success Story #1 17
  18. 18. LTI Proprietary and Confidential ©Larsen & Toubro Infotech Ltd. Privileged and Confidential THEJOURNEY: Impact-based regression testing for each build integrated with the DevOps pipeline Success Story #1 18 Seamless integration of LTI Canvas with Client’s SDLC landscape – deployed connector for IBM RTC Correlated SDLC digital assets to create end to end knowledge Fabric, especially focusing on source code and application features (Test cases) Automated code trace created for 1000+ test cases across two different product suite Delivered optimized regression test suite for each build The Canvas Insights (Test Optimization feature) was implemented We are here Fully integrated DevOps pipeline with automated impact analysis across 3000+ test cases Savings in the order of 30-60% in regression test efforts and test cycle timelines – Regression duration reduced to less than 4 hours from an original execution time of around 9 hours Enabled smarter test case prioritization Enabled impact-based Regression testing based on code changes (In Sprint and Release Regression)
  19. 19. LTI Proprietary and Confidential ©Larsen & Toubro Infotech Ltd. Privileged and Confidential Critical CR Before Canvas ▪ No time to manually analyze or run Regression bed of 900+ test cases which takes 9+ hours Build Makes Incremental dev based on CR Testing Development ▪ Manual change impact analysis, SME intervention After Canvas ▪ Visibility on to the Impact of changes across user stories , test cases, code modules, defects ▪ Analyze potential for reuse ▪ Now only run Impacted list of test cases based on changes in the new release ▪ Execution time is now less than 4 hours depending on changes ▪ Higher cost of fixing issues later in the lifecycle ▪ Stretched timelines and budget ▪ Lower confidence in production Recurring Build failures, escalations from issues arising due defect leakage DevOps Engineer Business Sponsor Manual test of new functionality Investment product management (IPM) Release 3 Order Management (OM) Total Test Case count Client Management SDLC Data ingestion Application Modules in scope Case in Point 2-week sprint Skips in-sprint regression run, only weekend runs are done Build failure Knowledge Fabric Higher quality build, Mitigated defect leakage risk Effort/cost reduction and higher sprint velocity Saved 30-60% effort on regression execution THETRANSFORMATION: How Canvas Delivered Business Value for our client Success Story #1 >3000 Test cases Release 3 for IPM module. Critical CR delivery 19
  20. 20. LTI Proprietary and Confidential ©Larsen & Toubro Infotech Ltd. Privileged and Confidential https://www.agilemumbai.com/
  21. 21. LTI Proprietary and Confidential ©Larsen & Toubro Infotech Ltd. Privileged and Confidential https://www.agilemumbai.com/ 21 Delivery Centers 36 LTM Revenue $2.10 Bn Clients 486 Employees 45,000+ Meet LTI USA Canada India Denmark UK Singapore Japan South Africa UAE Germany France Australia Sales Office 53 Locations 33 Fortune 500 clients 73
  22. 22. LTI Proprietary and Confidential ©Larsen & Toubro Infotech Ltd. Privileged and Confidential Ready the organization for beyond-the-horizon capabilities in a deterministic, accelerated, fail-fast and fail-safe way. Purpose Designing the Future of Software Engineering LTI Canvas A Core Ecosystem of Our Clients and Partners The Grit Alliance Global Technology Office (GTO) 22 ©Larsen & Toubro Infotech Ltd. Privileged and Confidential

×