[DSC Europe 22] Bridging the gap between AI Research and Human-Centered Products - Andreja Ilic
•Télécharger en tant que PPTX, PDF•
0 j'aime•19 vues
AI creates a tone of value to business, society, and humanity. With fast-paced AI research, we see continues improvements in the benchmark data and stunning results which redefine possible. On the other hand, most of these systems are not widely deployed in real life. One of the challenges we face today in industry across domains is: how to move proof-of-concept AI to human-centered products. In this talk, I will share a perspective on this topic, including examples and learnings from my work in Microsoft when it comes to productization of SOTA AI that works for people.
Recommandé
Recommandé
Contenu connexe
Similaire à [DSC Europe 22] Bridging the gap between AI Research and Human-Centered Products - Andreja Ilic
Similaire à [DSC Europe 22] Bridging the gap between AI Research and Human-Centered Products - Andreja Ilic (20)
Plus de DataScienceConferenc1
Plus de DataScienceConferenc1 (20)
Dernier
Dernier (20)
[DSC Europe 22] Bridging the gap between AI Research and Human-Centered Products - Andreja Ilic
- 1. Bridging the gap between AI Research and Human-Centered Products Andreja Ilic, Principal Group Engineering Manager Microsoft Office AI, Product, and Engineering https://www.linkedin.com/in/andrejailic/
- 2. Traditional rule based & expert systems AI based systems Hybrid systems You are here There are two types of products: the ones they use AI, and the ones that will use AI.
- 3. AI Research or Proof of Concept (PoC) Human-Centered Products or Production at scale Bridging the gap between According to Gartner, 80% of AI projects fail to reach production, and only 60% of those that do are profitable
- 4. DL for chest radiograph diagnosis “achieved radiologist-level performance on 11 pathologies”
- 5. AI Research & PoC Human-Centered Products
- 6. Excel: Data from Picture
- 7. Microsoft Editor: Tone and Conciseness suggestions
- 8. Word & PowerPoint: Designer
- 9. ML Model(s) Model Explainability Responsible AI Benchmark data Small data & Data distribution Data representativeness Labeling process & scale Model Personalization Compliant re-training system Monitoring tools Hybrid Inferencing Trust & Why (Explainability) Variety of outputs and Ranking User in Control Feedback Loop Optimization and COGS Endpoints and Devices Seamless Experience Researcher's machine Metrics & Optimization function Data and Model version control Engineering System and Infra Data AI Models and System User Scenario and Experience AI Research Human-Centered Products … … Pre/post processing models … Data Analytics … AI view on Research and Product
- 10. AI Project Life Cycle Change Management Team Culture Non-Tech Challenges. Equally important for success.
- 12. (*) How AI Could Empower Any Business, Andrew Ng, TED (2022)
- 13. (*) Custom Vision | Azure Cognitive Service
- 14. Democratization of AI. AI Literacy.
- 15. Large-Scale AI models Input: The sun rose over the beach as the waves soothingly rolled … in and out. The sound of the waves was like music to my ears. I took a deep breath and smelled the salty air. I felt the sand between my toes and the sun on my skin. I was in heaven. Illustrated by DALL-E 2 Completed by GPT-3 Animated by NUWA-Infinity
- 17. Trend of sizes of state-of-the-art NLP models over time
- 18. Self-supervised learning & Transformers Raw data Training Large-Scale Model Adaption Task Q&A Image captioning Text Generation Object Detection Entity Reco … What's in a name? That which we call a rose, by any other word would ___?____ as sweet. ?
- 19. Self-supervised learning: The dark matter of intelligence
- 20. Summary
- 22. THANK YOU! Discussion and Q&A Andreja Ilic, Principal Group Engineering Manager Microsoft Office AI, Product, and Engineering https://www.linkedin.com/in/andrejailic/
Notes de l'éditeur
- Intro words. Topic of the talk & Title. AI and humanity in co-existence. Segue: start w/ why
- Technological revolutions. AI is the new electricity, catalyst of seismic changes, and tech platform of our generation. Traditional programming vs ML. Rules, Input, and Output. Automate and enhance workflows, or new unmet user needs. OCR for digitization and visual impairment. Trend that is clear – Algorithms are being replaced with ML/DNN models. Movie or product recommendation. AI Inflection Point today – technology has abstracted away a lot of layers so that wider base of people and product roles can try it. Example chatbot for customer service. Cloud first, Mobile first => AI first. Segue: OK, so what’s the problem
- Exciting breakthroughs and SOTA research, but despite all of this not much deployed in the real world. Gartner data. Goal of the talk: experience and perspectives for the future. Segue: start with examples
- Various papers, trusted and reviewed. Benchmark and model optimization function, precision. Pneumonia vs Hernia. Small data set. Explainability. Trust. OpenAI not in your day-to-day apps yet. Apps in your mobile not using them. Segue: Why is this?
- Why: it’s hard and expensive + descriptive + great responsibility. Beyond happy path. My intro. AI for productivity, quality, and cognitive load. Example from my work, to paint a picture of problems and insights.
- Analog content digitization, transformation, and augmentation. Scanner in your pocket. Tone of papers for ImageOfTable to Table. Great for demo, but sufficient? Generalization Robustness. Data is fixed and often preprocessed. Unseen scenarios and noise. Cow camel example, and lazy student. Correlation vs causation. GANs, Transfer Learning etc. Seamless user experience: Table auto-detection and confidence output per cell. Expert system with a lot of models. Hybrid Inferencing. Works on you Ferrari or Maserati, but should work on humble version. Model for client. Model distillation.
- Editor – Intelligent Writing Assistant, from s& to TP, summarization, etc. Tone for thoughtful decisions about messaging. Conciseness wordy to alternate clear versions. Data. Small data problem. Language. Responsibility AI. Machine amplifies, solution not to have better data. Toxic outputs – bias or offensiveness. System optimization. COGS & supply issues – there is not enough harder even if you have money. Scale of Editor – millions of request per second. Prediction model.
- Intelligent assistance tool that helps you create documents with a polished, professional, and consistent look and feel with no design skills required. Model (feature) metrics. Precision/Recall vs Kept. Quality of model from user behave. Engagement vs Task. Possession vs Love. Personalization. Subjectiveness, no ground truth data. Meaning, time to Wow – easy. Time to Value – hard. Ship initial model and personalize per user. Compliant re-training. When you ship it, it’s only the beginning. Eng system to process this and train.
- Hopefully lands message for iceberg Every box Research working on. Problems: a) isolated, and b) priority/funding. Segue: Up until now we talked about technical problems. What about non-tech ones?
- Project life cycle. AI different from software development. Iterations, start with ship, unknowns. Change. Transforms the work of people and meaning. AI Radiologist. Think about influence upfront. Culture. Success determined by team/culture. Working across roles. Learn it all, instead of know it all.
- From large companies to small business. Budget or hiring. One size fits all approach. Bakery example – collectively they are big, but each is different.
- Long tail problem. Sorted by value – customer problems on the right. Research correlated with this curve, but things changing.
- Low-code & no-code, provide data and hints. Tech savvy. Example of inventory process.
- AI – literacy and math. Society better with everybody reading, writing, and basic math. AI Next. AI in hands of research and business. Give it to hands of everybody. Democratization. Enabling all requires solving all the problems we talked about. Segue: Large scale models, reducing this gap.
- GPT-3, text completions on prompts. DALL:E image based on description. NUWA-Infinity animating – video from initial image frame. Humans adapt quickly. Mind-blowing. Segue: Cool, but what about the value.
- GPT3 Playground. Chat between client and bank agent. Mail covering details and answering questions. Speed-up the process for email follow-up.
- Chart linear, but log-scale. 86 billion neurons in the human brain. Know everything, understand nothing. This changed. Chess, Turing test, Elementary school – easy. Did not plateau. Bigger is better – monolingual to multilingual. Unimodal to multimodality. Segue: what is data.
- New paradigm in AI, learning from data. Extract representation and capture meaning and context. I will not go into Transformers, but this is current SOTA architecture for large-scale models. Example: text and masking some word, or frames and masking some frame. Model trained to predict.
- Supervised learning was a bottleneck. Generalist models. Learning as babies – observation or experimenting. Predicting future = common sense. Compression – understanding the essence of the topic. Creativity – biggest systemic error in thinking, models will contribute to knowledge. Don’t be racist. Foundational models – develop by large companies or governments.
- Not disappointed, not having magic list – start with why, humans at the center, importance of team, democratization of AI, research breakthroughs like large-scale models etc. One final thoughts – in 10 years, “remember the day”. Today.