Generative AI, WiDS 2023.pptx

Colleen Farrelly
Colleen FarrellySenior Data Scientist/Author à Various/No Starch Press
GENERATIVE AI: AN INTRODUCTION Colleen M. Farrelly
WHO AM I?
• Data science
lead/advisor
• Author of The
Shape of Data
• AI artist
• Miami creative
MY PATH
• Told I was dumb and bad
at school
• Athlete/writer/inventor/
engineer outside of school
• Tried to drop out 1st
semester of university
• Dropped out of MD/PhD
program to enter AI
• 10+ years in R&D and AI
for social good
• Possibly PhD student in
the future?
WHAT IS
GENERATIVE
AI?
• Set of algorithms that
generate:
• Images
• Text samples
• Videos
• Audio content
• Guided by:
• Training sample
• User specifications
USE CASE 1: PUBLIC
HEALTH CAMPAIGNS
South Africa
AIDS education
example (2006-
2007)
COVID public
health
messaging
campaigns
USE CASE 2: CHATBOTS
Educational chatbot example
Built while working at Jenzabar
Automates student support
Ethical considerations of
chatbots
U SE C ASE 3: BETA
TESTIN G C ON TEN T
• New cartoon targeting
males ages 4-9 who
watch a lot of science
fiction:
• Generate lots of
potential content
• Choose promising
characters/scenes
• Do a pilot
• Create the show
OTHER USE CASES
• Video game character design (Tomb
Raider-type example)
• Public policy campaign content
generation
• Blog content
• Educational video generation
• Virtual therapist apps
• Personal assistant bots
• Many, many, many more!
HOW DO THESE ALGORITHMS WORK?
GPT
• Generative Pre-trained
Transformer 3
• Decoder-only transformer
network
• Gives sequence-to-sequence
decoder with long-range
memory
• Already blurring lines
between human
composition and AI
DALL-E 2
(OPEN AI)
• Capabilities:
• Can generate images from text
• Can insert new features or styles into that image
to modify it
• System pieces:
• Contrastive Language-Image Pre-training (CLIP)
• Prior model (build off existing repository)
• Decoder Diffusion model (inverse step)
STABLE
DIFFUSION
• Text-to-image technology
• Based on latent diffusion
models coupled to text
input
• Translation of text cues
• Markov chains wandering
around a latent space
• Denoise and renoise
images
• Leverages some inpainting
techniques to fill in gaps
PROMPT
ENGINEERING
• New field of engineering
unique to generative AI
• Leverage
• Knowledge of
technical architecture
• Good combinations of
elements to coax
system to desired
output
• Growing need for prompt
engineers in industry
• Many new career
opportunities
• Gaming systems
• Other content
generation
REPRESENTATION
• Languages with no or bad models
• Lingala
• Hausa
• Patwa
• Cultural contexts missing
• Burqa
• Subgroups
• Lack of worldwide access to some tools
• OpenAI and dozens of countries
OTHER
RESOURCES
• https://www.linkedin.com/in/c
olleenmfarrelly/
• https://stablediffusionweb.co
m/
• https://openai.com/
• https://hourone.ai/generative-
ai-video-des/
• https://midjourney.com/home/
• https://www.lesswrong.com/p
osts/fRzkDWewwyS6fHp4w/
palm-api-and-makersuite
• https://mathgpt.streamlit.app/
TOOLS
THAT
SHOULD
WORK IN
ANYWHERE
Speech generation:
• https://play.ht/text-to-speech-voices/egyptian-
arabic/
Text generation (OpenAI alternative, GPT-
2):
• https://huggingface.co/tasks/text-generation
Image generation:
• https://creator.nightcafe.studio/create
Hopefully OpenAI (and Google’s PaLM
API):
• DALL-E, ChatGPT, GPT-4, Whisper
1 sur 16

Contenu connexe

Tendances(20)

Large Language Models - Chat AI.pdfLarge Language Models - Chat AI.pdf
Large Language Models - Chat AI.pdf
David Rostcheck655 vues
The Creative Ai stormThe Creative Ai storm
The Creative Ai storm
Leandro Righini644 vues
Generative AI and law.pptxGenerative AI and law.pptx
Generative AI and law.pptx
Chris Marsden615 vues
Generative AI Risks & ConcernsGenerative AI Risks & Concerns
Generative AI Risks & Concerns
Ajitesh Kumar2.5K vues
AI FOR BUSINESS LEADERSAI FOR BUSINESS LEADERS
AI FOR BUSINESS LEADERS
Andre Muscat899 vues
Generative modelsGenerative models
Generative models
Birger Moell1.6K vues
Generative AIGenerative AI
Generative AI
All Things Open4.3K vues

Dernier(20)

Generative AI, WiDS 2023.pptx

  • 1. GENERATIVE AI: AN INTRODUCTION Colleen M. Farrelly
  • 2. WHO AM I? • Data science lead/advisor • Author of The Shape of Data • AI artist • Miami creative
  • 3. MY PATH • Told I was dumb and bad at school • Athlete/writer/inventor/ engineer outside of school • Tried to drop out 1st semester of university • Dropped out of MD/PhD program to enter AI • 10+ years in R&D and AI for social good • Possibly PhD student in the future?
  • 4. WHAT IS GENERATIVE AI? • Set of algorithms that generate: • Images • Text samples • Videos • Audio content • Guided by: • Training sample • User specifications
  • 5. USE CASE 1: PUBLIC HEALTH CAMPAIGNS South Africa AIDS education example (2006- 2007) COVID public health messaging campaigns
  • 6. USE CASE 2: CHATBOTS Educational chatbot example Built while working at Jenzabar Automates student support Ethical considerations of chatbots
  • 7. U SE C ASE 3: BETA TESTIN G C ON TEN T • New cartoon targeting males ages 4-9 who watch a lot of science fiction: • Generate lots of potential content • Choose promising characters/scenes • Do a pilot • Create the show
  • 8. OTHER USE CASES • Video game character design (Tomb Raider-type example) • Public policy campaign content generation • Blog content • Educational video generation • Virtual therapist apps • Personal assistant bots • Many, many, many more!
  • 9. HOW DO THESE ALGORITHMS WORK?
  • 10. GPT • Generative Pre-trained Transformer 3 • Decoder-only transformer network • Gives sequence-to-sequence decoder with long-range memory • Already blurring lines between human composition and AI
  • 11. DALL-E 2 (OPEN AI) • Capabilities: • Can generate images from text • Can insert new features or styles into that image to modify it • System pieces: • Contrastive Language-Image Pre-training (CLIP) • Prior model (build off existing repository) • Decoder Diffusion model (inverse step)
  • 12. STABLE DIFFUSION • Text-to-image technology • Based on latent diffusion models coupled to text input • Translation of text cues • Markov chains wandering around a latent space • Denoise and renoise images • Leverages some inpainting techniques to fill in gaps
  • 13. PROMPT ENGINEERING • New field of engineering unique to generative AI • Leverage • Knowledge of technical architecture • Good combinations of elements to coax system to desired output • Growing need for prompt engineers in industry • Many new career opportunities • Gaming systems • Other content generation
  • 14. REPRESENTATION • Languages with no or bad models • Lingala • Hausa • Patwa • Cultural contexts missing • Burqa • Subgroups • Lack of worldwide access to some tools • OpenAI and dozens of countries
  • 15. OTHER RESOURCES • https://www.linkedin.com/in/c olleenmfarrelly/ • https://stablediffusionweb.co m/ • https://openai.com/ • https://hourone.ai/generative- ai-video-des/ • https://midjourney.com/home/ • https://www.lesswrong.com/p osts/fRzkDWewwyS6fHp4w/ palm-api-and-makersuite • https://mathgpt.streamlit.app/
  • 16. TOOLS THAT SHOULD WORK IN ANYWHERE Speech generation: • https://play.ht/text-to-speech-voices/egyptian- arabic/ Text generation (OpenAI alternative, GPT- 2): • https://huggingface.co/tasks/text-generation Image generation: • https://creator.nightcafe.studio/create Hopefully OpenAI (and Google’s PaLM API): • DALL-E, ChatGPT, GPT-4, Whisper