SlideShare une entreprise Scribd logo
1  sur  26
Télécharger pour lire hors ligne
Olark - SupConf Fall 2017 - Live Chat Gender Experiment Results
Hi Samuel!
Hi Sam!
Hi Sarah!
What would you do?
Overall Chat CSAT: 4.52
Escalated Cases: 1
Blocked chats (trolls): 12
Overall Chat CSAT: 4.8
Escalated Cases: 12
Blocked chats (trolls): 32
Overall Chat CSAT: 4.8
Escalated Cases: 9
Blocked chats (trolls): ?
The thing is..
The thing is..
It’s all me
Only One Difference
● Same knowledge
● Same voice
● Same approach to
customers
What the customer saw
Why?
Olark - SupConf Fall 2017 - Live Chat Gender Experiment Results
Sexism is a real thing
● Anecdata isn’t enough
● With great data, comes great
responsibility
● With data, we can make bigger impacts
Creating an experiment
You can do this!
Hypotheses
● Sarah would take more
abuse on chat
● Samuel would have higher
CSAT
● Sarah would escalate
more cases
How I did it
1. One month as Sarah
2. One month as Samuel
3. One month as Sam
4. Changed avatar and
name in chat
5. Collected data
Sarah brought all the trolls to the yard
Sam ~3% of chats
Sarah ~7.5%of chats
Data Doesn’t Play Fair
● Overall team escalations plunged
● Components were fixed
● Resist temptation to elevate your
“anecdata”
CSAT
(aka how outsiders can make career decisions for
your team)
What would you do?
Overall Chat CSAT: 4.52
Escalated Cases: 1
Blocked chats (trolls): 12
Overall Chat CSAT: 4.8
Escalated Cases: 12
Blocked chats (trolls): 32
Overall Chat CSAT: 4.8
Escalated Cases: 9
Blocked chats (trolls): ?
A word of caution
Data doesn’t exist in a vacuum
We live in a data-obsessed time. Just having
data is not enough. We have to know what
it means, and we need to know the
implications of using it.
What are you measuring?
How might this data be used?
Goals v Impact
What I did not
measure
● Anything financial
● Race, age, ethnic,
religious, or cultural
biases
● Other support channels
Personally...
The sun emoji stays
No more abuse
tolerance
Higher self-
confidence
Easier than I expected
Who is scoring your team?
How’s that working for you?
Talk to ME!
Thank you!
@sarahinapickle
@sarahbetts

Contenu connexe

Dernier

Custom Speech Writing Services Online | Writing Sharks
Custom Speech Writing Services Online | Writing SharksCustom Speech Writing Services Online | Writing Sharks
Custom Speech Writing Services Online | Writing SharksWritingSharks
 
THIRTEEN ODDS DESIGN DOCKYARD PRIVATE LIMITED
THIRTEEN ODDS DESIGN DOCKYARD PRIVATE LIMITEDTHIRTEEN ODDS DESIGN DOCKYARD PRIVATE LIMITED
THIRTEEN ODDS DESIGN DOCKYARD PRIVATE LIMITEDAMRITMISHRA46
 
Touchstone Lofts – Loft conversions specialist
Touchstone Lofts – Loft conversions specialistTouchstone Lofts – Loft conversions specialist
Touchstone Lofts – Loft conversions specialistTouchstone Lofts
 
7 Ways To Select Best Corporate Gifts | Corporate Gifts Supplier
7 Ways To Select Best Corporate Gifts | Corporate Gifts Supplier7 Ways To Select Best Corporate Gifts | Corporate Gifts Supplier
7 Ways To Select Best Corporate Gifts | Corporate Gifts SupplierVerve Corporate Gifts
 
interior design company dubai | Sites DXB
interior design company dubai | Sites DXBinterior design company dubai | Sites DXB
interior design company dubai | Sites DXBSitesDXB
 
International Sustainable and Efficient Irrigation Solutions
International Sustainable and Efficient Irrigation SolutionsInternational Sustainable and Efficient Irrigation Solutions
International Sustainable and Efficient Irrigation SolutionsIrri Design Studio
 
one of the best by art prints online here
one of the best by art prints online hereone of the best by art prints online here
one of the best by art prints online heretoyism501
 
The Power of Quick Commerce Data Monitoring A Comprehensive Guide.pdf
The Power of Quick Commerce Data Monitoring A Comprehensive Guide.pdfThe Power of Quick Commerce Data Monitoring A Comprehensive Guide.pdf
The Power of Quick Commerce Data Monitoring A Comprehensive Guide.pdfjacklutz2
 
Retirement Planning - Transferring UK Pension to France.pptx
Retirement Planning - Transferring UK Pension to France.pptxRetirement Planning - Transferring UK Pension to France.pptx
Retirement Planning - Transferring UK Pension to France.pptxChaseBuchanan
 
What To Look For in Employee Onboarding Software?
What To Look For in Employee Onboarding Software?What To Look For in Employee Onboarding Software?
What To Look For in Employee Onboarding Software?eZaango HR
 
The Expertise of Our Web Development Company
The Expertise of Our Web Development CompanyThe Expertise of Our Web Development Company
The Expertise of Our Web Development CompanyNetqom Solutions
 
From A To Z Understanding The Full Spectrum Of Employment Background Checks
From A To Z Understanding The Full Spectrum Of Employment Background ChecksFrom A To Z Understanding The Full Spectrum Of Employment Background Checks
From A To Z Understanding The Full Spectrum Of Employment Background Checksaffordablebackgroundchecks
 
ONEUNIVERSE_PORTFOLIO_ENGLISH_202403.pdf
ONEUNIVERSE_PORTFOLIO_ENGLISH_202403.pdfONEUNIVERSE_PORTFOLIO_ENGLISH_202403.pdf
ONEUNIVERSE_PORTFOLIO_ENGLISH_202403.pdfLeonaNguyenThuyLinh
 
Here are five tips to make house packing quicker and faster.pptx
Here are five tips to make house packing quicker and faster.pptxHere are five tips to make house packing quicker and faster.pptx
Here are five tips to make house packing quicker and faster.pptxnidhimovee
 
Point of View Glazing - Expert Glazing Services
Point of View Glazing - Expert Glazing ServicesPoint of View Glazing - Expert Glazing Services
Point of View Glazing - Expert Glazing ServicesPoint of View Glazing
 
How Do Cleaners Prepare Before Commencing Builders Cleaning?
How Do Cleaners Prepare Before Commencing Builders Cleaning?How Do Cleaners Prepare Before Commencing Builders Cleaning?
How Do Cleaners Prepare Before Commencing Builders Cleaning?Renovation Cleaning
 
How Can Dumpster Rental Streamline Yard Waste Management In San Fernando Valley
How Can Dumpster Rental Streamline Yard Waste Management In San Fernando ValleyHow Can Dumpster Rental Streamline Yard Waste Management In San Fernando Valley
How Can Dumpster Rental Streamline Yard Waste Management In San Fernando ValleyBG’s Big Box Service
 

Dernier (19)

Custom Speech Writing Services Online | Writing Sharks
Custom Speech Writing Services Online | Writing SharksCustom Speech Writing Services Online | Writing Sharks
Custom Speech Writing Services Online | Writing Sharks
 
THIRTEEN ODDS DESIGN DOCKYARD PRIVATE LIMITED
THIRTEEN ODDS DESIGN DOCKYARD PRIVATE LIMITEDTHIRTEEN ODDS DESIGN DOCKYARD PRIVATE LIMITED
THIRTEEN ODDS DESIGN DOCKYARD PRIVATE LIMITED
 
Touchstone Lofts – Loft conversions specialist
Touchstone Lofts – Loft conversions specialistTouchstone Lofts – Loft conversions specialist
Touchstone Lofts – Loft conversions specialist
 
7 Ways To Select Best Corporate Gifts | Corporate Gifts Supplier
7 Ways To Select Best Corporate Gifts | Corporate Gifts Supplier7 Ways To Select Best Corporate Gifts | Corporate Gifts Supplier
7 Ways To Select Best Corporate Gifts | Corporate Gifts Supplier
 
interior design company dubai | Sites DXB
interior design company dubai | Sites DXBinterior design company dubai | Sites DXB
interior design company dubai | Sites DXB
 
International Sustainable and Efficient Irrigation Solutions
International Sustainable and Efficient Irrigation SolutionsInternational Sustainable and Efficient Irrigation Solutions
International Sustainable and Efficient Irrigation Solutions
 
one of the best by art prints online here
one of the best by art prints online hereone of the best by art prints online here
one of the best by art prints online here
 
QuickBooks Multi User Mode Not Working Solutions Steps.pdf
QuickBooks Multi User Mode Not Working Solutions Steps.pdfQuickBooks Multi User Mode Not Working Solutions Steps.pdf
QuickBooks Multi User Mode Not Working Solutions Steps.pdf
 
The Power of Quick Commerce Data Monitoring A Comprehensive Guide.pdf
The Power of Quick Commerce Data Monitoring A Comprehensive Guide.pdfThe Power of Quick Commerce Data Monitoring A Comprehensive Guide.pdf
The Power of Quick Commerce Data Monitoring A Comprehensive Guide.pdf
 
Retirement Planning - Transferring UK Pension to France.pptx
Retirement Planning - Transferring UK Pension to France.pptxRetirement Planning - Transferring UK Pension to France.pptx
Retirement Planning - Transferring UK Pension to France.pptx
 
What To Look For in Employee Onboarding Software?
What To Look For in Employee Onboarding Software?What To Look For in Employee Onboarding Software?
What To Look For in Employee Onboarding Software?
 
The Expertise of Our Web Development Company
The Expertise of Our Web Development CompanyThe Expertise of Our Web Development Company
The Expertise of Our Web Development Company
 
From A To Z Understanding The Full Spectrum Of Employment Background Checks
From A To Z Understanding The Full Spectrum Of Employment Background ChecksFrom A To Z Understanding The Full Spectrum Of Employment Background Checks
From A To Z Understanding The Full Spectrum Of Employment Background Checks
 
ONEUNIVERSE_PORTFOLIO_ENGLISH_202403.pdf
ONEUNIVERSE_PORTFOLIO_ENGLISH_202403.pdfONEUNIVERSE_PORTFOLIO_ENGLISH_202403.pdf
ONEUNIVERSE_PORTFOLIO_ENGLISH_202403.pdf
 
Here are five tips to make house packing quicker and faster.pptx
Here are five tips to make house packing quicker and faster.pptxHere are five tips to make house packing quicker and faster.pptx
Here are five tips to make house packing quicker and faster.pptx
 
Point of View Glazing - Expert Glazing Services
Point of View Glazing - Expert Glazing ServicesPoint of View Glazing - Expert Glazing Services
Point of View Glazing - Expert Glazing Services
 
A Guide to AI-powered Lead Generation Strategies
A Guide to AI-powered Lead Generation StrategiesA Guide to AI-powered Lead Generation Strategies
A Guide to AI-powered Lead Generation Strategies
 
How Do Cleaners Prepare Before Commencing Builders Cleaning?
How Do Cleaners Prepare Before Commencing Builders Cleaning?How Do Cleaners Prepare Before Commencing Builders Cleaning?
How Do Cleaners Prepare Before Commencing Builders Cleaning?
 
How Can Dumpster Rental Streamline Yard Waste Management In San Fernando Valley
How Can Dumpster Rental Streamline Yard Waste Management In San Fernando ValleyHow Can Dumpster Rental Streamline Yard Waste Management In San Fernando Valley
How Can Dumpster Rental Streamline Yard Waste Management In San Fernando Valley
 

En vedette

5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summarySpeakerHub
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next Tessa Mero
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentLily Ray
 
Time Management & Productivity - Best Practices
Time Management & Productivity -  Best PracticesTime Management & Productivity -  Best Practices
Time Management & Productivity - Best PracticesVit Horky
 
The six step guide to practical project management
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project managementMindGenius
 
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...RachelPearson36
 
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...Applitools
 
12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at Work12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at WorkGetSmarter
 
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...DevGAMM Conference
 
Barbie - Brand Strategy Presentation
Barbie - Brand Strategy PresentationBarbie - Brand Strategy Presentation
Barbie - Brand Strategy PresentationErica Santiago
 
Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them well
Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them wellGood Stuff Happens in 1:1 Meetings: Why you need them and how to do them well
Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them wellSaba Software
 
Introduction to C Programming Language
Introduction to C Programming LanguageIntroduction to C Programming Language
Introduction to C Programming LanguageSimplilearn
 
The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...
The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...
The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...Palo Alto Software
 
9 Tips for a Work-free Vacation
9 Tips for a Work-free Vacation9 Tips for a Work-free Vacation
9 Tips for a Work-free VacationWeekdone.com
 
I Rock Therefore I Am. 20 Legendary Quotes from Prince
I Rock Therefore I Am. 20 Legendary Quotes from PrinceI Rock Therefore I Am. 20 Legendary Quotes from Prince
I Rock Therefore I Am. 20 Legendary Quotes from PrinceEmpowered Presentations
 

En vedette (20)

5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
 
How to have difficult conversations
How to have difficult conversations How to have difficult conversations
How to have difficult conversations
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data Science
 
Time Management & Productivity - Best Practices
Time Management & Productivity -  Best PracticesTime Management & Productivity -  Best Practices
Time Management & Productivity - Best Practices
 
The six step guide to practical project management
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project management
 
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
 
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
 
12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at Work12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at Work
 
ChatGPT webinar slides
ChatGPT webinar slidesChatGPT webinar slides
ChatGPT webinar slides
 
More than Just Lines on a Map: Best Practices for U.S Bike Routes
More than Just Lines on a Map: Best Practices for U.S Bike RoutesMore than Just Lines on a Map: Best Practices for U.S Bike Routes
More than Just Lines on a Map: Best Practices for U.S Bike Routes
 
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
 
Barbie - Brand Strategy Presentation
Barbie - Brand Strategy PresentationBarbie - Brand Strategy Presentation
Barbie - Brand Strategy Presentation
 
Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them well
Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them wellGood Stuff Happens in 1:1 Meetings: Why you need them and how to do them well
Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them well
 
Introduction to C Programming Language
Introduction to C Programming LanguageIntroduction to C Programming Language
Introduction to C Programming Language
 
The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...
The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...
The Pixar Way: 37 Quotes on Developing and Maintaining a Creative Company (fr...
 
9 Tips for a Work-free Vacation
9 Tips for a Work-free Vacation9 Tips for a Work-free Vacation
9 Tips for a Work-free Vacation
 
I Rock Therefore I Am. 20 Legendary Quotes from Prince
I Rock Therefore I Am. 20 Legendary Quotes from PrinceI Rock Therefore I Am. 20 Legendary Quotes from Prince
I Rock Therefore I Am. 20 Legendary Quotes from Prince
 

Olark - SupConf Fall 2017 - Live Chat Gender Experiment Results

  • 5. What would you do? Overall Chat CSAT: 4.52 Escalated Cases: 1 Blocked chats (trolls): 12 Overall Chat CSAT: 4.8 Escalated Cases: 12 Blocked chats (trolls): 32 Overall Chat CSAT: 4.8 Escalated Cases: 9 Blocked chats (trolls): ?
  • 8. Only One Difference ● Same knowledge ● Same voice ● Same approach to customers What the customer saw
  • 11. Sexism is a real thing ● Anecdata isn’t enough ● With great data, comes great responsibility ● With data, we can make bigger impacts
  • 13. Hypotheses ● Sarah would take more abuse on chat ● Samuel would have higher CSAT ● Sarah would escalate more cases
  • 14. How I did it 1. One month as Sarah 2. One month as Samuel 3. One month as Sam 4. Changed avatar and name in chat 5. Collected data
  • 15. Sarah brought all the trolls to the yard Sam ~3% of chats Sarah ~7.5%of chats
  • 16. Data Doesn’t Play Fair ● Overall team escalations plunged ● Components were fixed ● Resist temptation to elevate your “anecdata”
  • 17. CSAT (aka how outsiders can make career decisions for your team)
  • 18. What would you do? Overall Chat CSAT: 4.52 Escalated Cases: 1 Blocked chats (trolls): 12 Overall Chat CSAT: 4.8 Escalated Cases: 12 Blocked chats (trolls): 32 Overall Chat CSAT: 4.8 Escalated Cases: 9 Blocked chats (trolls): ?
  • 19. A word of caution Data doesn’t exist in a vacuum
  • 20. We live in a data-obsessed time. Just having data is not enough. We have to know what it means, and we need to know the implications of using it.
  • 21. What are you measuring? How might this data be used? Goals v Impact
  • 22. What I did not measure ● Anything financial ● Race, age, ethnic, religious, or cultural biases ● Other support channels
  • 23. Personally... The sun emoji stays No more abuse tolerance Higher self- confidence Easier than I expected
  • 24. Who is scoring your team? How’s that working for you?

Notes de l'éditeur

  1. At Olark we <3 experiments. So much so we have an entire Jira filter for them, and a template for setting them up. I wanted take at some support issues, especially around how folks perform with CSAT, Escalations, and the like
  2. I enlisted some folks from my Olark team to help Let’s meet my co-conspirators!
  3. Stop--as a manager, how are you going to view these? I might want to work with Samuel on his communication and empathy. Sarah and Sam may need some help skill building to reduce load on the escalation team
  4. 5 years on frontline support, concern about burnout levels for unsupported teams Work in tech Flashy articles aren’t enough, wanted numbers NickO’s monthlong experiment was inspiring Because I wanted to know how my perceived gender affected my interactions with customers
  5. Also, seeing this every day gets quite old
  6. Sexism is very real and flashy stories aren’t enough But with data collection comes responsibility The flip side of that is with data, we have the power to rethink how we do things and make a change
  7. So let’s backtrack to how I set this up Because experiments are FUN and give you powerful information
  8. Any experiment starts first with a hypothesis. If you’re thinking an experiment starts with data, no Why did I choose these? What we’ve seen is Women are less respected, less valued. Treated poorly. I wanted to see if I could quantify that. And talked to our data ops, Ben Schultz who helped me find what we can collect and think through the experiment
  9. Then make a plan. Time, who is involved, resourcing, etc
  10. Trolls SUCK, Facebook example, proposition with apology Explain !block feature & prob density Again though, as a manager, I might worry about Sarah’s quick banhammer
  11. Data doesn’t play fair because life is real, yo Story of “I’ll have my team take a look, thanks!”
  12. Let’s talk about customer satisfaction ratings Overall, all 3 reps scored very well. 4.52-4.8/5
  13. Stop--as a manager, how are you going to view these? I might want to work with Samuel on his communication and empathy. Sarah and Sam may need some help skill building to reduce load on the escalation team
  14. So let’s have the responsibility talk
  15. We’re data obsessed. We think about responsibility with customer data, privacy acts, COPA, GDPR But it goes further. We USE data all the time, HOW?
  16. Determine what you want to know, then design the experiment not the other way round With piles of data at our fingertips, that just appears through our tools, do we know what the data really measures? If you measure ticket escalation, are you measuring knowledge? OR your customers willingness to listen to your team member?
  17. Because of this, I stopped short of measuring quite a few things No financial, cultural appropriation (personal comfort) or other channels
  18. My anecdata-ey takeaways
  19. In the coming month, I’ll be sharing some resources on how you can set up a similar experiment. I think we have a responsibility to collect data on issues we see and share that. With enough data, support can be a leader in fair practices, impacting the broader world of tech