DevDive_UnleashthFullPotentialofAutomationwithGenAI.pptx
Signaler
Partager
•1 j'aime•1,025 vues
![26
MeetingAI
How it works
20-30 minutes saved
in follow-up per meeting!
& Gen AI
Zoom
Recording
Upload Post-
meeting
Triggers
Automation
Meeting
Gen AI:
Extract action items
Summarize meeting
Microsoft Azure
OpenAI
Gen AI:
Compare action items
to list of existing
automations
Get
Documentation
Microsoft Azure
OpenAI
Get meeting
times
Get recording
link
Compose email
[Meeting Summary]
[Follow-up items]
[Action items]
Less time spent on admin
tasks after meetings
Send email
More time for account strategy,
market research, meeting preparation
List of Automations
supplied by robot
Pattern
1
&
2](https://image.slidesharecdn.com/devdiveunleashthfullpotentialofautomationwithgenai-230628174010-1062721a/75/devdiveunleashthfullpotentialofautomationwithgenaipptx-26-2048.jpg?cb=1687974383)
DevDive_UnleashthFullPotentialofAutomationwithGenAI.pptx
•1 j'aime•1,025 vues
Signaler
Partager
Télécharger pour lire hors ligne
dev dives
Recommandé
Contenu connexe
Tendances
Tendances(20)
Similaire à DevDive_UnleashthFullPotentialofAutomationwithGenAI.pptx
Similaire à DevDive_UnleashthFullPotentialofAutomationwithGenAI.pptx(20)
Plus de Rohit Radhakrishnan
Plus de Rohit Radhakrishnan(20)
Dernier
Dernier(20)
DevDive_UnleashthFullPotentialofAutomationwithGenAI.pptx
- 1. Dev Dives Go deeper, automate smarter
- 2. 2 Meet today’s team: Rohit Radhakrishnan Marketing Community Manager @UiPath Roxana Ivan Senior Product Marketing Manager @UiPath
- 3. 3 Meet today’s speakers: Dhruv Patel Senior Sales Solution Architect @UiPath Russel Alfeche UiPath MVP Technology Leader RPA @qBotica
- 4. 4 About today’s meeting • Enjoy the next 50-60 min packed with gen AI powered automations use cases, demos, and live Q&A. • Build your own Two Way Match Invoice Processing with UiPath+GenAI. You’ll receive the recording and the full pack of instructions to leverage the power of UiPath Document Understanding and GenAI to seamlessly process invoices. • Get answers to your questions and challenges. Please use the chat box for Qs during the presentation. Live Q&A session at the end. • You're encouraged to network and share your LinkedIn/Twitter in the chat. Have fun! Feedback is welcome.
- 5. Unleash the full potential of your automations with generative AI UiPath Dev Dives Webinar Series
- 6. 6 Poll time We’d love to know more about your experience with gen AI and automation! Take a quick poll by scanning the QR code or access it at https://www.menti.com/al1eexoy2cdr
- 7. 7 Agenda 01 02 03 UiPath + generative AI, AI-powered automation 3 patterns for gen AI powered automations Demo time!
- 8. 8 Explore Novel Use Cases Build Better Automations Faster UiPath Uses Gen AI in 3 Ways Generative AI Powered Automations Incorporating Gen AI directly into Business Automation Use Cases Supercharge Developer Productivity “Co-Pilot” Like experience bringing a user- friendly interface to building automations Product & Model Augmentation Add LLM capabilities to improve design time 1 2 3
- 9. 9 AI-powered automation Open | Flexible | Responsible Supported by UiPath Built with UiPath or BYO BYO Docs Screens Tasks Processes Solutions AI Infrastructure Integration Service – Validation Station – Active learning – Fine tuning – Guardrails – Auditing AI-powered automation Delivering enterprise automation with Specialized AI Generative AI Specialized AI Context HITL UI API Action People Comms Docs Data Processes
- 10. 10 Advantages of using UiPath with Gen AI for Automation More than calling a Generative AI Model Context Gathering Specialized Models Robots Do Work Human In The Loop Confidence in governance UiPath also has Specialized models to complement Gen AI interactions Gen AI needs context, UiPath can gather the context from all sources Gen AI is just a brain, Automation is the muscle that does the work Gen AI “hallucinates”. There are times you can not get things wrong Your business is governed with audit logs and controls while using Gen AI with UiPath
- 11. 3 Patterns for Gen AI Powered Automations A use case may use 1 or multiple patterns Pattern 1 Reader / Writer Pattern 2 Pattern 3 Analyst / Doer Assistant UiPath can execute processes as a result of LLM calls UiPath can gather context from multiple sources to generate and distribute personalized messages UiPath can add context and action to conversational assistants UiPath Advantage Source 1 Source 2 Source 3 Source 4 LLM Ingest Analyze Data Next Best Action System 1 System 2 System 3 System 4 Do LLM System 1 System 2 System 3 System 4 Do Conversational Interface Human Knowledge Response Action OR Source 1 Source 2 Source 3 Source 4 LLM Read Write Emails Summaries Content
- 12. 12 Pattern 1 - Reader / Writer Description: LLMs are great at taking context and generating personalized text, whether longer or shorter than the original context. UiPath is collecting all the relevant context and prompting the LLM for text. UiPath can keep a human in the loop to fine tune the output. Example Use Cases Cold Call Emails Gather context about your audience and generate email Customer Feedback Response Gather sentiment and customer history to generate tailored response Proposal Writer Combine multiple answers from your KB with additional context for a tailored answer to a proposal question Applicant Communications Combine feedback from interviewers, JD details, and applicant resume for tailored communications Customer Summary Summarize customer history, support ticket history, etc. for faster consumption by customer facing agents Email/PMO Summarizer Summarize information from PM tools, emails, other sources for faster executive overviews KYC Summarizer Gather and summarize materials from multiple sources for faster KYC review Compliance and ESG Reporting Monitor data and reports from multiple systems/source and generate consistent reporting Product Documentation Create and maintain product documentation summarizing information from feature tickets and marketing Fraud Communications Generate correspondence with customers collating information multiple systems Insurance Claims Communication Communicate to Customers around their claims request synthesizing information from multiple systems Healthcare Appeals Communication Tailor communications to customers about using information about the customer and the circumstance Human Input and Validation Human in the loop Source 1 Source 2 Source 3 Source 4 LLM Read Write Emails Summaries Content System 1 System 2 System 3 System 4 Distribute
- 13. 13 Pattern 2 – Analyst / Doer Description: LLMs can generate structured output (data tables, code, XML/JSON) from multiple unstructured sources when prompted well. UiPath is collecting the relevant sources and prompting the LLM to generate structured outputs. Critically, UiPath can validate the contents of the structured outputs vs. systems of record or humans. UiPath can further execute processes based on the output. Source 1 Source 2 Source 3 Source 4 LLM Ingest Analyze Data Next Best Action System 1 System 2 System 3 System 4 Do/Validate Human Input and Validation Human in the loop Example Use Cases Multi-Source Report Creating Gather reporting from different systems, documents, and emails and combine them into one set Structuring and Normalizing Data Normalize data from different sources into a common schema 2 Way Match (Generic Reconciliation) Normalize data from multiple systems and further reconcile the two noting differences for humans to validate Contact Center Next Best Action Gather context from sources, recommend an action from a list and execute the action After Call Work (Action Item Doer) Extract actions from call scripts for follow-up. Execute those automatable. Upsell / Cross-sell Assistant Gather customer history and needs, generate a recommendation for what to sell, and a script to sell it Contract Extractor Extract structured data out of contracts / amendments, validate against sources, and input into systems Company Filing Extractor Extract key figures out of company filings, validate with human, and use in processes Test Data Creator Generate test data for application testing and insert it into the system using UiPath Generic Classifier Take in unstructured sources and classify against a list of defined options. Use this data as input for processes Competition Analysis Monitor pricing, news, reviews of competitors and extract structured findings Vendor Selection Analyze proposals from multiple vendors, extract key differentiations, and recommend a vendor
- 14. 14 Pattern 3 - Assistant Description: The most common use case enterprises are building is a custom ChatGPT on their own knowledge sources. UiPath can augment and enrich these chat interfaces with more knowledge sources, either directly in the LLM or in the prompt. For assistant interactions that result in an action, UiPath be used for last-mile process execution. LLM System 1 System 2 System 3 System 4 Do Conversational Interface Human Knowledge Response Action OR Human in the loop Example Use Cases Knowledge Base Assistant Use UiPath to augment the knowledge accessible to the LLM. Vectorize databases or embed in the prompt. Support Escalation Assistant Variation of a Knowledge Base Assistant focused on customer support escalations. Learning Assistant UiPath can help find courses from different learning platforms and recommend one for a user Self Service Helpdesk LLMs will create enhanced assistant interfaces. Execute common tasks behind the scenes with UiPath. Employee Benefits Assistant Answer employee questions about benefits and automate benefit selection / changing Employee Travel Concierge UiPath can bring in context on travel policy and flight/hotel data to allow users to book compliant trips faster Legacy System Augmenter Legacy systems likely won’t have LLMs integrated. UiPath can be used to bring LLM experiences to old systems. Localization Assistant LLMs have become decent at localizing to different languages. UiPath can help validate the output against other tools Supply Chain Buyer Assistant A conversational interface help speed up inventory management and ordering across suppliers for buyers Personalized Assistant Use a conversational interface to perform more Reader/Writer and Analyzer/Doer actions Guided Form Entry A conversational interface collects & validates input from users. UiPath can input those answers into a system. Ask GPT (Document) Gather a specific document with automation and then allow users to ask questions about it
- 15. Demos UiPath + gen AI
- 16. 16 SAMPLE – Web Q&A Pattern 1 & 3
- 17. 17 SAMPLE –Video Summarization, Image Generation Pattern 1 & 3
- 18. 18 How does it work? Core Concept in NLP Embeddings Numerical Representation of the meanings of words or a group of words Used with semantic search to inject context nearest/relevant to the query into prompts (Context Injection)
- 19. 19 Knowledgebase or data source can be documents, web pages, video transcripts, etc Generate most reliable answers with LLM completions by leveraging the primed prompt Context Injection Generate Answer Inject Context Semantic Search Ask Question Generate Embeddings Fetch KB, Data Source Call embeddings API to calculate the transcript's embeddings and load into a vector database Use the question as an input for semantic search/similarity. Get only most relevant chunks of tokens from the embeddings using the query as an input. The distance between embeddings carries semantic meaning Inject the context gathered from the search, along with the specific query, into the prompt to prime it Claude GPT Vertex AI … Pattern 3
- 20. 20 2 Way Match Invoice Processing This process matches invoices to purchase orders to ensure accurate payment. This process ensures that only valid invoices with valid line items are paid, reducing the risk of fraud and errors; save time and money, and improve their relationships with business partners. Generative AI reduces the development complexity and enables developers to handle complex validation logic with natural language prompts! Increased Process Efficiency Improved Accuracy, Clear Compliance & Audit Trail Reduced Development Complexity Reduced Risk & Cost Savings Process: Invoice 2 Way Match & Gen AI Leverage the power of UiPath Document Understanding and Generative AI to seamlessly process invoices Natural language to prompt GenAI to handle the validation logic Automations handle processing, involving a human when needed Pattern 2
- 21. 21 Pharmacovigilance (Adverse Events) Pharmacovigilance is a crucial process for monitoring the safety of approved drugs by systematically collecting and analyzing data on adverse drug reactions (ADRs) and other drug-related problems. MedDRA, a standardized medical terminology, plays a vital role in this process. It provides a common language for classifying and coding adverse events and medical conditions, ensuring consistent reporting. Automation + GenAI reduce the burden on clinicians, analysts and medical professionals during this process to intake adverse events & encode them in MedDRA, WHO DD, etc. & Gen AI Pattern 2
- 22. 22 Date/Time Topic Status July 27, 10:30 AM EST / 3:30 PM GMT Empower your automation with Cloud Robots Register AMER Register EMEA & APAC August 31, 9:00 AM EST / 2:00 PM GMT Supercharge testing and RPA with coded automations Register AMER Register EMEA & APAC Next steps Download the automation template Two Way Match Invoice Processing: UiPath + GenAI > https://bit.ly/3JwLAnC Don't miss the next Dev Dives sessions. Save your seat > https://bit.ly/Dev-Dives_2023
- 23. 23 CATEGORY 1: AI POWERED AUTOMATION SOLUTIONS CATEGORY 2: INNOVATE THROUGH CORE BUSINESS AUTOMATION SOLUTIONS AWARDS/CATEGORY: 🥇 1ST place: 4500 $ 🥇 2nd place: 3500 $ 🥇 3rd place: 1500 $ http://bit.ly/UiPath_Hyper_Hack_2023
- 24. 24 Live Q&A
- 25. Thank you for your participation! Join us: community.uipath.com
- 26. 26 MeetingAI How it works 20-30 minutes saved in follow-up per meeting! & Gen AI Zoom Recording Upload Post- meeting Triggers Automation Meeting Gen AI: Extract action items Summarize meeting Microsoft Azure OpenAI Gen AI: Compare action items to list of existing automations Get Documentation Microsoft Azure OpenAI Get meeting times Get recording link Compose email [Meeting Summary] [Follow-up items] [Action items] Less time spent on admin tasks after meetings Send email More time for account strategy, market research, meeting preparation List of Automations supplied by robot Pattern 1 & 2
Notes de l'éditeur
- Left - you can do now Productivity / product augmentation – what we’re working on / on the roadmap Classify – GPT Labeling – 50/50 both specialized / gpt comparing results Extraction – specialized model, much higher accuracy level, Validation – higher confidence level – have two models working together
- Have the platform that combines the power of AI and Automation – In order to maximize the value of AI, need robots and be able to set the context – need to show a specific example…! Secure AI – UiPath Value Prop Context for Prompt HITL Brains and Muscle Specialized Models like DU and Comms Mining Orchestrating all of this together If you could make your average X as good as your best X – human doing something awesome and they are reading or writing Person to another Person
- Generating text-based responses
- How can you extend the generative capabilities of large language models through UiPath. Although chat plugins as well as multi-modal capabilities of ChatGPT are within the horizon, what if we can achieve these features now with UiPath? Because Unless you’re subscribed to chatgpt plus, you won’t have access to early preview of plugins which lets the model browse the web But essentially how does these work, it turns out it’s a pretty simple flow leveraging one of the core concept in NLP which is embeddings using embeddings Web QnA: Question about the linked url (url docloader + embeddings api + chat completion)
- ChatGPT and even ChatGPTPlus doesn’t support image generation yet but there are available APIs that can do that which you can easily integrate UiPath with. Image Generation: Image generation (chat completion to create prompt + stable diffusion api to generate image from prompt) Image Interpretation: Image interpreter (visual qna transformer models + chat completion) Another example is youtube summrizer, even if you’re subscribed to chatGPT plus, I don’t think there’s a plugin already yet, at least that I know of. But with UiPath you can do these by integrating with native youtube doc loaders, along with embeddings YT Vid Summary: Summarize youtube vid using URL (yt docloader + embeddings api + chat completion)
- What is the common denominator for these samples? Embeddings, vector database, semantic search or similarity, context injection 1 Let’s talk about embeddings which is a core concept in NLP 2 Words such as beautiful, pretty, awesome, they have meanings, and consequently, synonyms and if you put words together to make up sentences, it will create a thought which may or may not represent a whole new meaning. 3 Basically, embeddings are numerical representations of the meanings of words or a group of words or thoughts, which are also called vectors 4 So essentially if you plot the embeddings of words in a graph… words that have close meanings generate nearby vectors if you plot them in a multidimensional graph. 5 these can then be used for semantic search or similarity, so that given a question… A large language model is a type of artificial intelligence model that is designed to understand and generate human language. It is trained on vast amounts of text data, such as books, articles, and websites, and uses statistical techniques to learn the patterns and structures of language. The goal of a large language model is to be able to generate new, coherent sentences that are similar in style and content to those found in the training data. Embeddings are a key component of large language models. They are a way of representing words as numerical vectors that capture their meaning and context. Each word in the model's vocabulary is assigned a unique vector, and these vectors are learned during the training process. The vectors are designed to be similar for words that have similar meanings or are used in similar contexts, and dissimilar for words that are unrelated.Embeddings are used in large language models to help the model understand the meaning and context of words in a sentence. When the model is given a sentence to generate, it uses the embeddings of the words in the sentence to predict the most likely next word. By using embeddings, the model is able to capture the nuances of language and generate sentences that are both grammatically correct and semantically meaningful. A large language model is a type of neural network that is designed to process and understand natural language. These models are trained on massive amounts of text data, such as books, articles, and websites, in order to learn the patterns and relationships between words and phrases.One important aspect of large language models is the use of word embeddings. Word embeddings are a way of representing words as numerical vectors, which can be used as input to the neural network. The idea behind word embeddings is to map semantically similar words to vectors that are close together in the vector space, so that the neural network can better understand the meaning of the words.For example, the words "cat" and "dog" might be mapped to vectors that are close together, since they are both animals that are often kept as pets. On the other hand, the word "car" might be mapped to a vector that is farther away, since it is not semantically related to "cat" or "dog".By using word embeddings, large language models are able to better understand the meaning of natural language text, and can perform tasks such as language translation, question answering, and text generation. ===
- What is one of the things rpa is best at? Data gathering and consolidation How automation can help LLMs generate more relevant information and prevent hallucination through embeddings Generate Embeddings from knowledgebase > Ask a question > Use semantic similarity for the question asked vs the embeddings > Get most relevant chunks of tokens from the embeddings > Use the relevant information gathered along with the question into the prompt > Generate answer Context Injection The benefit of context injection is it primes the prompt with very specific information to use for the answer and always use up to date answer, you don’t have to retrain the entire model in case there are new information from the database Essentially the model doesn't have to be fine-tuned since the prompt is fed with very specific context based on the question, to generate the most appropriate answer.
- Created by Dhruv Patel