Starting your AI/ML project right (May 2020)

1. ©2019, Amazon Web Services, Inc. or its affiliates. All rights reserved Starting your AI/ML project right Julien Simon Global Technical Evangelist, AI & Machine Learning Amazon Web Services @julsimon

2. Does AI have a massive future? Sure! Please insert another coin. Do we (the builders) have a clear idea how to get there? Hmmmm.

3. « If you want to know the future, look at the past » Albert Einstein What’s our collective track record on understanding and implementing disruptive technologies?

4. You Your Web project Your competitor 2000 Universal Pictures

5. You Your E-commerce project Your competitor 2005 Universal Pictures

6. You Your M-commerce project Your competitor 2010 Universal Pictures

7. You Your Big Data project Your competitor 2015 Universal Pictures

8. The terrifying truth about tech projects Delusional stakeholders Business pressure Unprepared team Inadequate tools Improvised tactics Random acts of bravery Universal Pictures

9. « It’s different this time! The AI revolution is here! Blah blah blah » You know who

10. You Your AI / ML project Your competitor 2020 Universal Pictures

11. « Insanity is doing the same thing over and over again and expecting different results » Whoever said it first

12. Delusional stakeholders Business pressure Unprepared team Inadequate tools Improvised tactics Random acts of bravery Set expectations Define clear metrics Assess your skills Pick the best tool for the job Use best practices Iterate, iterate , iterate Tired of being shark food?

13. • What is the business question you’re trying to answer? – One sentence on the whiteboard – Must be quantifiable • Do you have (enough) data that could help? • Involve everyone and come to a common understanding – Business, IT, Data Engineering, Data Science, Ops, etc. « We want to see what this technology can do for us » « We have tons of relational data, surely we can do something with it » « I read this cool article about FooBar ML, we ought to try it » 1 - Set expectations 1-

14. 2 - Define clear metrics • What is the business metric showing success? • What’s the baseline (human and IT)? • What would be a significant and reasonable improvement? • What would be reasonable further improvements? « The confusion matrix for our support ticket classifier has significantly improved ». Huh? « P90 time-to-resolution is now under 24 hours ». Err…. « Misclassified emails have gone down 5.3% using the latest model ». So? « The latest survey shows that ‘very happy’ customers are up 9.2% ». Woohoo!

15. 3 - Assess needs (not wants) and skills • Building a data set describing the problem? • Cleaning and curating it? • Writing and tweaking ML algorithms? • Managing ML infrastructure? 100% DIY Fully managed ?

16. 4 - Pick the best tool for the job • Cost, time to market, accuracy: pick two • The least expensive and fastest option won’t probably be the most accurate. – Maybe enough to get started, and learn more about the problem. • Improving accuracy will take increasingly more time and money. – Diminishing returns! Know when to stop. • Keep an eye on actionable state of the art advances, ignore the rest – Transfer learning – AutoML Cost AccuracyTime

17. 5 - Use best practices • No, things are not different this time. • AI / ML is software engineering – Dev, test, QA, documentation, Agile, versioning, etc. – Involve all teams • Sandbox tests are nice, but truth is in production – Get there fast, as often as needed – CI / CD and automation are required – Devops for ML Universal Pictures

18. 6 - Iterate, iterate, iterate aka Boyd’s Law (1960) • Start small • Try the simple things first • Go to production quickly • Observe prediction errors • Act: fix data set? Add more data? Tweak the algo? Try another algo? • Repeat until accuracy gains become irrelevant • Move to the next project

19. 6 – Machine Learning *is* an iterative process

20. « Does this work? » Everyone in this room

21. Tens of thousand of active customers – all sizes, all verticals FINRA Expedia Group

22. The AWS ML Stack VISION SPEECH TEXT SEARCH CHATBOTS PERSONALIZATION FORECASTING FRAUD DEVELOPMENT CONTACT CENTERS Ground Truth ML Marketplace Neo Augmented AI Built-in algorithms Notebooks Experiments Processing & Model Evaluation Model training & tuning Debugger Autopilot Model hosting Model Monitor Deep Learning AMIs & Containers GPUs & CPUs Elastic Inference Inferentia FPGA Amazon Rekognition Amazon Polly Amazon Transcribe +Medical Amazon Comprehend +Medical Amazon Translate Amazon Lex Amazon Personalize Amazon Forecast Amazon Fraud Detector Amazon CodeGuru AI SERVICES ML SERVICES ML FRAMEWORKS & INFRASTRUCTURE Amazon Textract Amazon Kendra Contact Lens For Amazon Connect SageMaker Studio IDE Amazon SageMaker Deep Graph Library

23. M O D E R N I Z E Y O U R C O N TA C T C E N T E R T O I M P R O V E C U S T O M E R S E R V I C E Voice of the customer analytics | Automated service agents | Multi-lingual text support Workforce forecasting and agent analysis | Next best action recommendation POLLY TRANSCRIBE TRANSLATE COMPREHEND LEX PERSONALIZE

24. U S E A I S E RV I C E S TO S T R E N G T H E N S A F E T Y A N D S E C U R I T Y REKOGNITION IMAGE COMPREHENDREKOGNITION VIDEO Risk assessment | Threat detection | Identity verification | Alarm prioritization

25. A U T O M A T E M E D I A W O R K F L O W S T O R E D U C E C O S T S A N D M O N E T I Z E C O N T E N T Media metadata tagging | Highlight clipping | Subtitling and localization | Content moderation | Compliance | Contextual ad insertion REKOGNITION IMAGE REKOGNITION VIDEO COMPREHEND TRANSCRIBE TRANSLATE TEXTRACTSAGEMAKER

26. R E D U C E L O C A L I Z AT I O N C O S T S A N D I M P R O V E A C C U R A C Y POLLY TRANSCRIBE TRANSLATE COMPREHEND Website & document translation | Recorded call analysis | Video subtitling | Accessibility

27. U N D E R S TA N D T H E V O I C E O F Y O U R C U S TO M E R REKOGNITION IMAGE REKOGNITION VIDEO TRANSLATETRANSCRIBE COMPREHEND Problem detection | Sentiment analysis | Campaign targeting | Personalized service

28. P E R S O N A L I Z E C U S TO M E R E X P E R I E N C E S W I T H TA R G E T E D R E C O M M E N D AT I O N S Personalized recommendations | Personalized search | Personalized notifications PERSONALIZE

30. I N C R E A S E E F F I C I E N C Y W I T H A U TO M AT E D D O C U M E N T A N A LY S I S TEXTRACT COMPREHEND & COMPREHEND MEDICAL Fast archive search | Automated form processing | Systematic redaction

31. P R O T E C T U S E R S F R O M U N S A F E C O N T E N T UGC curation | Media compliance marking | Ad adjacency assurance REKOGNITION IMAGE TRANSCRIBE COMPREHEND REKOGNITION VIDEO

32. AWS Marketplace for Machine Learning Natural language processing Text-to-speech Object detection Speech recognition Grammar and parsingText generation Speaker identification Regression Text OCR Text classification Text clustering Computer vision 3D images Handwriting recognition Named entity recognition Anomaly detection Ranking Video classification Automatic labeling via machine learning IP protection Automated billing and metering SELLERS Broad selection of paid, free, and open-source algorithms and models Data protection Discoverable on your AWS bill BUYERS

33. AWS Marketplace Developer Challenge https://awsmarketplaceml.devpost.com/ https://www.youtube.com/watch?v=BRCS7Q3u-ck 1st place: Mobility Explorer https://devpost.com/software/mobility-explorer

34. Mobility Explorer - Architecture Amazon SageMaker Human Detector Bike Detector Model Model AWS Marketplace Vehicle Detector Model AWS CloudWatch AWS Lambda AWS CloudWatch AWS Lambda AWS CloudWatch AWS Lambda AWS Lambda AWS Lambda AWS Lambda AWS Lambda AWS Lambda AWS Lambda Amazon CloudFront Amazon Route 53 Amazon S3 Amazon Aurora Amazon API Gateway

35. ©2019, Amazon Web Services, Inc. or its affiliates. All rights reserved https://ml.aws Julien Simon Global Technical Evangelist, AI & Machine Learning Amazon Web Services @julsimon https://medium.com/@julsimon https://youtube.com/juliensimonfr

Starting your AI/ML project right (May 2020)

Julien SIMON

Starting your AI/ML project right (May 2020)