SlideShare une entreprise Scribd logo
1  sur  26
Télécharger pour lire hors ligne
Interac(on	
  Mining:	
  the	
  new	
  
fron(er	
  of	
  Call	
  Center	
  Analy(cs	
  	
  
               Vincenzo	
  Pallo:a	
  	
  
               Rodolfo	
  Delmonte	
  	
  
               Lammert	
  Vrieling	
  	
  
                 David	
  Walker	
  
                          	
  

                    ©	
  2011	
  interAnaly(cs	
      1	
  
Outline	
  
•  Call	
  Center	
  Analy(cs	
  
•  Automa(c	
  Argumenta(ve	
  Analysis	
  for	
  
   Interac(on	
  Mining	
  
•  Experiments	
  with	
  Call	
  Center	
  Data	
  
•  Conclusions	
  




                         ©	
  2011	
  interAnaly(cs	
     2	
  
CALL	
  CENTER	
  ANALYTICS	
  


                 ©	
  2011	
  interAnaly(cs	
     3	
  
Call	
  Center	
  Analy(cs	
  
•  Call	
  centers	
  data	
  represent	
  a	
  valuable	
  asset	
  for	
  
   companies,	
  but	
  it	
  is	
  oOen	
  underexploited	
  for	
  
   business	
  purposes	
  because:	
  
    –  it	
  is	
  highly	
  dependent	
  on	
  quality	
  of	
  speech	
  recogni(on	
  
       technology	
  
    –  it	
  is	
  mostly	
  based	
  on	
  text-­‐based	
  content	
  analysis.	
  
•  Interac(on	
  Mining	
  as	
  a	
  viable	
  alterna(ve:	
  
    –  	
  more	
  robust	
  
    –  tailored	
  for	
  the	
  conversa(onal	
  domain	
  
    –  slanted	
  towards	
  pragma&c	
  and	
  discourse	
  analysis	
  	
  

                                    ©	
  2011	
  interAnaly(cs	
                       4	
  
Mainstream	
  Call	
  Center	
  Analy(cs	
  


                                  	
  re    al	
  
                           v  eil                  er	
  
                 ot  	
  un stom
      oe s	
  n           ut	
  cu
    D          	
  abo                n	
  
        hts sfac=o
     sig a=
   in         s
                       ©	
  2011	
  interAnaly(cs	
         5	
  
Call	
  Center	
  Analy(cs:	
  metrics	
  and	
  KPIs	
  
•  Agent	
  Performance	
  Sta(s(cs:	
  	
  
      –  Average	
  Speed	
  of	
  Answer,	
  Average	
  Hold	
  Time,	
  Call	
  Abandonment	
  Rate,	
  
         A<ained	
  Service	
  Level,	
  and	
  Average	
  Talk	
  Time.	
  	
  
      –  Quan(ta(ve	
  measurements	
  that	
  can	
  be	
  obtained	
  directly	
  through	
  ACD	
  
         (Automa(c	
  Call	
  Distribu(on),	
  Switch	
  Output	
  and	
  Network	
  Usage	
  Data.	
  
•  Peripheral	
  Performance	
  Data:	
  	
  	
  
      –  Cost	
  Per	
  Call,	
  First-­‐Call	
  Resolu&on	
  Rate,	
  Customer	
  Sa,sfac,on,	
  Account	
  
         Reten&on,	
  Staff	
  Turnover,	
  Actual	
  vs.	
  Budgeted	
  Costs,	
  and	
  Employee	
  Loyalty.	
  	
  
      –  Quan(ta(ve,	
  with	
  the	
  excep(on	
  of	
  Customer	
  Sa&sfac&on	
  that	
  is	
  usually	
  
         obtained	
  through	
  Customer	
  Surveys.	
  	
  
•  Performance	
  Observa(on:	
  	
  
    –  Call	
  Quality,	
  Accuracy	
  and	
  Efficiency,	
  Adherence	
  to	
  Script,	
  
       Communica,on	
  E,que;e,	
  and	
  Corporate	
  Image	
  Exemplifica,on.	
  	
  
      –  Qualita=ve	
  metrics	
  based	
  on	
  analysis	
  of	
  recorded	
  calls	
  and	
  session	
  monitoring	
  
         by	
  a	
  supervisor.	
  
                                                 ©	
  2011	
  interAnaly(cs	
                                           6	
  
Four	
  objec(ves	
  
1.  Iden(fy	
  Customer	
  Oriented	
  Behaviors,	
  	
  
    –  which	
  are	
  highly	
  correlated	
  to	
  posi(ve	
  customer	
  ra(ngs	
  
       (Rafaeli	
  et	
  al.	
  2007);	
  
2.  Iden(fy	
  Root	
  Cause	
  of	
  Problems	
  	
  
    –  by	
  looking	
  at	
  controversial	
  topics	
  and	
  how	
  agents	
  are	
  able	
  
       to	
  deal	
  with	
  them;	
  
3.  Iden(fy	
  customers	
  who	
  need	
  par(cular	
  a:en(on	
  	
  
    –  based	
  on	
  history	
  of	
  problema(c	
  interac(ons;	
  
4.  Learn	
  best	
  prac(ces	
  in	
  dealing	
  with	
  customers	
  	
  
    –  by	
  iden(fying	
  agents	
  able	
  to	
  carry	
  coopera(ve	
  
       conversa(ons.	
  	
  

                                       ©	
  2011	
  interAnaly(cs	
                            7	
  
ARGUMENTATIVE	
  ANALYSIS	
  FOR	
  
INTERACTION	
  MINING	
  	
  

               ©	
  2011	
  interAnaly(cs	
     8	
  
Argumenta(ve	
  Structure	
  of	
  
                          Conversa(ons	
  
DISCUSS(issue) <- PROPOSE(alternative)
1702.95 David: so - so my question is should we go ahead and get na- -
nine identical head mounted crown mikes ? {qy} 61a


      REJECT(alternative)
      1708.89 John: not before having one come here and have
      some people try it out . {s^arp^co} 61b.62a

           PROVIDE(justification)
           1714.09 B: because there's no point in doing that if it's
                     John: because there's no point in doing that if
           it's going to to be better . {s} {s} 61b+
           not not goingbe anyany better . 61b+

                   ACCEPT(justification)
                   1712.69 David: okay . {s^bk} 62b

    PROPOSE(alternative)
    1716.85 John: so why don't we get one of these with the crown with a different headset ? {qw^cs}
    63a

              PROVIDE(justification)
              1722.4 John: and - and see if that works . {s^cs} 63a+.64a 
              1723.53 Mark: and see if it's preferable and if it is then we'll get more . {s^cs^2} 64b
              1725.47 Mark: comfort . {s}


               ACCEPT(alternative)
               1721.56 David: yeah . {s^bk} 63b
               1726.05 Lucy: yeah . {b}    

               Why	
  was	
  David’s	
  proposal	
  on	
  microphones	
  rejected?	
  
               1727.34 John: yeah . {b}
                                                         ©	
  2011	
  interAnaly(cs	
                                                 9	
  
Automa(c	
  Argumenta(ve	
  Analysis	
  
•  Based	
  on	
  the	
  GETARUNS	
  system1.	
  
•  Clauses	
  in	
  Turns	
  are	
  labelled	
  with	
  Primi(ve	
  Discourse	
  
   Rela(ons:	
  	
  
       –  statement,	
  narra,on,	
  adverse,	
  result,	
  cause,	
  mo,va,on,	
  
          explana,on,	
  ques,on,	
  hypothesis,	
  elabora,on,	
  
          permission,	
  incep,on,	
  circumstance,	
  obliga,on,	
  
          evalua,on,	
  agreement,	
  contrast,	
  evidence,	
  hypoth,	
  
          seCng,	
  prohibi,on.	
  
•  And	
  then	
  Turns	
  are	
  labelled	
  with	
  Argumenta(ve	
  labels:	
  
       –  ACCEPT,	
  REJECT/DISAGREE,	
  PROPOSE/SUGGEST,	
  
          EXPLAIN/JUSTIFY,	
  	
  REQUEST	
  EXPLANATION/
          JUSTIFICATION.	
  
1	
  Delmonte	
  R.,	
  	
  Bistrot	
  A.,	
  Pallo:a	
  V.,Deep	
  Linguis(c	
  Processing	
  with	
  GETARUNS	
  for	
  spoken	
  dialogue	
  

Understanding.	
  Proceedings	
  LREC	
  2010	
  (P31	
  Dialogue	
  Corpora).	
  
                                                          ©	
  2011	
  interAnaly(cs	
                                                             10	
  
Evalua(on	
  
                                     ICSI	
  corpus	
  of	
  mee(ngs	
  (Janin	
  et	
  al.,	
  2003)	
  
                                           Precision:	
  81.26%	
  Recall:	
  97.53%	
  

                                                                                                   Total	
  
                                        Correct                      Incorrect                                                 Precision
                                                                                                  Found
          Accept                            662                             16                      678                             98%
           Reject                            64                             18                          82                          78%
         Propose                            321                             74                         395                          81%
         Request                            180                               1                        181                          99%
          Explain                           580                            312                         892                          65%
            Total                          1826                            421                        2247                      81.26%


Delmonte	
  R.,	
  	
  Bistrot	
  A.,	
  Pallo:a	
  V.,Deep	
  Linguis(c	
  Processing	
  with	
  GETARUNS	
  for	
  spoken	
  dialogue	
  
Understanding.	
  Proceedings	
  LREC	
  2010	
  (P31	
  Dialogue	
  Corpora).	
  
                                                              ©	
  2011	
  interAnaly(cs	
                                                    11	
  
EXPERIMENTS	
  WITH	
  CALL	
  CENTER	
  
DATA	
  

                 ©	
  2011	
  interAnaly(cs	
     12	
  
Ra(onale:	
  implement	
  the	
  four	
  
                objec(ves	
  
1.  Iden(fy	
  Customer	
  Oriented	
  Behaviors,	
  	
  
2.  Iden(fy	
  Root	
  Cause	
  of	
  Problems	
  	
  
3.  Iden(fy	
  customers	
  who	
  need	
  par(cular	
  
    a:en(on	
  	
  
4.  Learn	
  best	
  prac(ces	
  in	
  dealing	
  with	
  
    customers	
  	
  



                           ©	
  2011	
  interAnaly(cs	
      13	
  
The	
  Data	
  
•  Corpus	
  of	
  213	
  manually	
  transcribed	
  
   conversa(ons	
  of	
  a	
  help	
  desk	
  call	
  center	
  in	
  the	
  
   banking	
  domain.	
  	
  
•  Average	
  of	
  66	
  turns	
  per	
  conversa(on.	
  
•  Average	
  of	
  1.6	
  calls	
  per	
  agent.	
  	
  
•  Collected	
  for	
  a	
  study	
  aimed	
  at	
  iden(fying	
  
   customer	
  oriented	
  behaviors	
  that	
  could	
  favor	
  
   sa(sfactory	
  interac(on	
  with	
  customers	
  
   (Rafaeli	
  et	
  al.	
  2007).	
  	
  
                                ©	
  2011	
  interAnaly(cs	
                14	
  
Iden(fy	
  Customer	
  Oriented	
  Behaviors	
  
•  Based	
  on	
  the	
  work	
  of	
  Rafaeli	
  et	
  al.	
  2006.	
  
•  Customer	
  Oriented	
  Behaviors 	
  	
  
    –  an(cipa(ng	
  customers	
  requests 	
  22,45%	
  
    –  educa(ng	
  the	
  customer 	
  16,91%	
  
    –  offering	
  emo(onal	
  support 	
  21,57%	
  
    –  offering	
  explana(ons	
  /	
  jus(fica(ons	
  28,57%	
  
    –  personaliza(on	
  of	
  informa(on 	
  10,50%	
  


                                  ©	
  2011	
  interAnaly(cs	
             15	
  
Significant	
  correla(on	
  with	
  
   argumenta(ve	
  labels	
  




             ©	
  2011	
  interAnaly(cs	
     16	
  
Iden(fy	
  Root	
  Cause	
  of	
  Problems	
  
•  Coopera(veness	
  score	
  	
                      Argumenta=ve	
  Categories	

                 Coopera=veness
                                                                                                                 	

                                                      Accept	
  explana(on	

                               5	

    –  a	
  measure	
  obtained	
  by	
  
       averaging	
  the	
  score	
                    Suggest	

                                            4	


       obtained	
  by	
  mapping	
                    Propose	

                                            3	

       argumenta(ve	
  labels	
  of	
                 Provide	
  opinion	

                                 2	

       each	
  turn	
  in	
  the	
                    Provide	
  explana(on/jus(fica(on	

                   1	

       conversa(on	
  into	
  a	
  [-­‐5	
            Request	
  explana(on/jus(fica(on	

                   0	

       +5]	
  scale.	
  	
                            Ques(on	

                                           -­‐1	

                                                      Raise	
  issue	

                                    -­‐2	


•  Sen(ment	
  Analysis	
                             Provide	
  nega(ve	
  opinion	

                     -­‐3	


   module.	
                                          Disagree	

                                          -­‐4	

                                                      Reject	
  explana(on	
  or	
  jus(fica(on	

          -­‐5	



                                          ©	
  2011	
  interAnaly(cs	
                                               17	
  
Top	
  20	
  Controversial	
  Topics	
  with	
  average	
  
 coopera(veness	
  scores	
  and	
  sen(ment	
  




                       ©	
  2011	
  interAnaly(cs	
       18	
  
Coopera(veness	
  of	
  speakers	
  on	
  top	
  
        discussed	
  topics	
  




                   ©	
  2011	
  interAnaly(cs	
     19	
  
Iden(fy	
  problema(c	
  customers	
  




               ©	
  2011	
  interAnaly(cs	
     20	
  
Select	
  a	
  specific	
  customer	
  




              ©	
  2011	
  interAnaly(cs	
     21	
  
Visualize	
  a	
  selected	
  call	
  




             ©	
  2011	
  interAnaly(cs	
     22	
  
CONCLUSIONS	
  


              ©	
  2011	
  interAnaly(cs	
     23	
  
Conclusions	
  
•  New	
  Genera(on	
  Call	
  Center	
  Analy(cs	
  requires	
  
   Interac(on	
  Mining	
  
    –  Call	
  Center	
  Qualita(ve	
  metrics	
  and	
  KPIs	
  can	
  be	
  only	
  
       implemented	
  with	
  a	
  full	
  understanding	
  of	
  the	
  
       customer	
  interac(on	
  dynamics	
  
•  Argumenta(on	
  is	
  pervasive	
  in	
  conversa(ons.	
  
    –  In	
  order	
  to	
  recognize	
  argumenta(ve	
  acts,	
  advanced	
  
       Natural	
  Language	
  Understanding	
  is	
  necessary.	
  
•  Future	
  work:	
  
    –  Scalability:	
  need	
  to	
  process	
  millions	
  of	
  call	
  per	
  day!	
  
    –  Mul(-­‐language:	
  call	
  centers	
  all	
  over	
  the	
  world.	
  

                                      ©	
  2011	
  interAnaly(cs	
                          24	
  
The	
  Team	
  
Dr. Lammert Vrieling
(1968) - Chief Executive Officer
 ‣ 15 years in both profit and not-for-profit organizations as consultant, trainer/coach and as
    executive.
 ‣ Experience in the steel and aluminium industry, multimedia publishing and newspaper,
    financial services and in the not-for-profit sector.

David E. Walker
(1964) - Chief Operating Officer
 ‣ 25+ years in IT as Software engineer, developer, project manager, and architect.
 ‣ Senior Software Solutions Architect with extensive experience in designing, developing and
   delivering enterprise solutions for payment processing, human resource, healthcare,
   marketing, manufacturing and scientific research environments.

Prof. Dr. Rodolfo Delmonte
(1946) - Chief Science Officer
 ‣ Since 1993 head of Computational Linguistics Laboratory of the University of
   Venice, Italy
 ‣ 30+ years experience in computational linguistics
 ‣ From 1986 to 1992 he worked with the Department of Engineering of the
   University of Parma. From 1978 to 1986 worked with the Department of
Dr. Vincenzo Pallotta
(1966) - Chief Technology Officer
 ‣ 30 years in ICT
 ‣ 10 years in R&D
 ‣ Human-Language Technology, Digital Libraries, Artificial Intelligence,
    Ubiquitous Computing, Human-Computer Interaction, Usability Engineering,
    Information Retrieval, Web Search Engines, Semantic Web, Computational
    Logics, Training and Education, e-learning.                                                 23


                         ©	
  2011	
  interAnaly(cs	
                                                25	
  
…find	
  us	
  at	
  

                       www.interanaly=cs.ch	
  

Contenu connexe

Dernier

Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 

Dernier (20)

Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 

Interaction Mining: the new frontier of Call Center Analytics

  • 1. Interac(on  Mining:  the  new   fron(er  of  Call  Center  Analy(cs     Vincenzo  Pallo:a     Rodolfo  Delmonte     Lammert  Vrieling     David  Walker     ©  2011  interAnaly(cs   1  
  • 2. Outline   •  Call  Center  Analy(cs   •  Automa(c  Argumenta(ve  Analysis  for   Interac(on  Mining   •  Experiments  with  Call  Center  Data   •  Conclusions   ©  2011  interAnaly(cs   2  
  • 3. CALL  CENTER  ANALYTICS   ©  2011  interAnaly(cs   3  
  • 4. Call  Center  Analy(cs   •  Call  centers  data  represent  a  valuable  asset  for   companies,  but  it  is  oOen  underexploited  for   business  purposes  because:   –  it  is  highly  dependent  on  quality  of  speech  recogni(on   technology   –  it  is  mostly  based  on  text-­‐based  content  analysis.   •  Interac(on  Mining  as  a  viable  alterna(ve:   –   more  robust   –  tailored  for  the  conversa(onal  domain   –  slanted  towards  pragma&c  and  discourse  analysis     ©  2011  interAnaly(cs   4  
  • 5. Mainstream  Call  Center  Analy(cs    re al   v eil er   ot  un stom oe s  n ut  cu D  abo n   hts sfac=o sig a= in s ©  2011  interAnaly(cs   5  
  • 6. Call  Center  Analy(cs:  metrics  and  KPIs   •  Agent  Performance  Sta(s(cs:     –  Average  Speed  of  Answer,  Average  Hold  Time,  Call  Abandonment  Rate,   A<ained  Service  Level,  and  Average  Talk  Time.     –  Quan(ta(ve  measurements  that  can  be  obtained  directly  through  ACD   (Automa(c  Call  Distribu(on),  Switch  Output  and  Network  Usage  Data.   •  Peripheral  Performance  Data:       –  Cost  Per  Call,  First-­‐Call  Resolu&on  Rate,  Customer  Sa,sfac,on,  Account   Reten&on,  Staff  Turnover,  Actual  vs.  Budgeted  Costs,  and  Employee  Loyalty.     –  Quan(ta(ve,  with  the  excep(on  of  Customer  Sa&sfac&on  that  is  usually   obtained  through  Customer  Surveys.     •  Performance  Observa(on:     –  Call  Quality,  Accuracy  and  Efficiency,  Adherence  to  Script,   Communica,on  E,que;e,  and  Corporate  Image  Exemplifica,on.     –  Qualita=ve  metrics  based  on  analysis  of  recorded  calls  and  session  monitoring   by  a  supervisor.   ©  2011  interAnaly(cs   6  
  • 7. Four  objec(ves   1.  Iden(fy  Customer  Oriented  Behaviors,     –  which  are  highly  correlated  to  posi(ve  customer  ra(ngs   (Rafaeli  et  al.  2007);   2.  Iden(fy  Root  Cause  of  Problems     –  by  looking  at  controversial  topics  and  how  agents  are  able   to  deal  with  them;   3.  Iden(fy  customers  who  need  par(cular  a:en(on     –  based  on  history  of  problema(c  interac(ons;   4.  Learn  best  prac(ces  in  dealing  with  customers     –  by  iden(fying  agents  able  to  carry  coopera(ve   conversa(ons.     ©  2011  interAnaly(cs   7  
  • 8. ARGUMENTATIVE  ANALYSIS  FOR   INTERACTION  MINING     ©  2011  interAnaly(cs   8  
  • 9. Argumenta(ve  Structure  of   Conversa(ons   DISCUSS(issue) <- PROPOSE(alternative) 1702.95 David: so - so my question is should we go ahead and get na- - nine identical head mounted crown mikes ? {qy} 61a REJECT(alternative) 1708.89 John: not before having one come here and have some people try it out . {s^arp^co} 61b.62a PROVIDE(justification) 1714.09 B: because there's no point in doing that if it's John: because there's no point in doing that if it's going to to be better . {s} {s} 61b+ not not goingbe anyany better . 61b+ ACCEPT(justification) 1712.69 David: okay . {s^bk} 62b PROPOSE(alternative) 1716.85 John: so why don't we get one of these with the crown with a different headset ? {qw^cs} 63a PROVIDE(justification) 1722.4 John: and - and see if that works . {s^cs} 63a+.64a 1723.53 Mark: and see if it's preferable and if it is then we'll get more . {s^cs^2} 64b 1725.47 Mark: comfort . {s} ACCEPT(alternative) 1721.56 David: yeah . {s^bk} 63b 1726.05 Lucy: yeah . {b} Why  was  David’s  proposal  on  microphones  rejected?   1727.34 John: yeah . {b} ©  2011  interAnaly(cs   9  
  • 10. Automa(c  Argumenta(ve  Analysis   •  Based  on  the  GETARUNS  system1.   •  Clauses  in  Turns  are  labelled  with  Primi(ve  Discourse   Rela(ons:     –  statement,  narra,on,  adverse,  result,  cause,  mo,va,on,   explana,on,  ques,on,  hypothesis,  elabora,on,   permission,  incep,on,  circumstance,  obliga,on,   evalua,on,  agreement,  contrast,  evidence,  hypoth,   seCng,  prohibi,on.   •  And  then  Turns  are  labelled  with  Argumenta(ve  labels:   –  ACCEPT,  REJECT/DISAGREE,  PROPOSE/SUGGEST,   EXPLAIN/JUSTIFY,    REQUEST  EXPLANATION/ JUSTIFICATION.   1  Delmonte  R.,    Bistrot  A.,  Pallo:a  V.,Deep  Linguis(c  Processing  with  GETARUNS  for  spoken  dialogue   Understanding.  Proceedings  LREC  2010  (P31  Dialogue  Corpora).   ©  2011  interAnaly(cs   10  
  • 11. Evalua(on   ICSI  corpus  of  mee(ngs  (Janin  et  al.,  2003)   Precision:  81.26%  Recall:  97.53%   Total   Correct Incorrect Precision Found Accept 662 16 678 98% Reject 64 18 82 78% Propose 321 74 395 81% Request 180 1 181 99% Explain 580 312 892 65% Total 1826 421 2247 81.26% Delmonte  R.,    Bistrot  A.,  Pallo:a  V.,Deep  Linguis(c  Processing  with  GETARUNS  for  spoken  dialogue   Understanding.  Proceedings  LREC  2010  (P31  Dialogue  Corpora).   ©  2011  interAnaly(cs   11  
  • 12. EXPERIMENTS  WITH  CALL  CENTER   DATA   ©  2011  interAnaly(cs   12  
  • 13. Ra(onale:  implement  the  four   objec(ves   1.  Iden(fy  Customer  Oriented  Behaviors,     2.  Iden(fy  Root  Cause  of  Problems     3.  Iden(fy  customers  who  need  par(cular   a:en(on     4.  Learn  best  prac(ces  in  dealing  with   customers     ©  2011  interAnaly(cs   13  
  • 14. The  Data   •  Corpus  of  213  manually  transcribed   conversa(ons  of  a  help  desk  call  center  in  the   banking  domain.     •  Average  of  66  turns  per  conversa(on.   •  Average  of  1.6  calls  per  agent.     •  Collected  for  a  study  aimed  at  iden(fying   customer  oriented  behaviors  that  could  favor   sa(sfactory  interac(on  with  customers   (Rafaeli  et  al.  2007).     ©  2011  interAnaly(cs   14  
  • 15. Iden(fy  Customer  Oriented  Behaviors   •  Based  on  the  work  of  Rafaeli  et  al.  2006.   •  Customer  Oriented  Behaviors     –  an(cipa(ng  customers  requests  22,45%   –  educa(ng  the  customer  16,91%   –  offering  emo(onal  support  21,57%   –  offering  explana(ons  /  jus(fica(ons  28,57%   –  personaliza(on  of  informa(on  10,50%   ©  2011  interAnaly(cs   15  
  • 16. Significant  correla(on  with   argumenta(ve  labels   ©  2011  interAnaly(cs   16  
  • 17. Iden(fy  Root  Cause  of  Problems   •  Coopera(veness  score     Argumenta=ve  Categories Coopera=veness Accept  explana(on 5 –  a  measure  obtained  by   averaging  the  score   Suggest 4 obtained  by  mapping   Propose 3 argumenta(ve  labels  of   Provide  opinion 2 each  turn  in  the   Provide  explana(on/jus(fica(on 1 conversa(on  into  a  [-­‐5   Request  explana(on/jus(fica(on 0 +5]  scale.     Ques(on -­‐1 Raise  issue -­‐2 •  Sen(ment  Analysis   Provide  nega(ve  opinion -­‐3 module.   Disagree -­‐4 Reject  explana(on  or  jus(fica(on -­‐5 ©  2011  interAnaly(cs   17  
  • 18. Top  20  Controversial  Topics  with  average   coopera(veness  scores  and  sen(ment   ©  2011  interAnaly(cs   18  
  • 19. Coopera(veness  of  speakers  on  top   discussed  topics   ©  2011  interAnaly(cs   19  
  • 20. Iden(fy  problema(c  customers   ©  2011  interAnaly(cs   20  
  • 21. Select  a  specific  customer   ©  2011  interAnaly(cs   21  
  • 22. Visualize  a  selected  call   ©  2011  interAnaly(cs   22  
  • 23. CONCLUSIONS   ©  2011  interAnaly(cs   23  
  • 24. Conclusions   •  New  Genera(on  Call  Center  Analy(cs  requires   Interac(on  Mining   –  Call  Center  Qualita(ve  metrics  and  KPIs  can  be  only   implemented  with  a  full  understanding  of  the   customer  interac(on  dynamics   •  Argumenta(on  is  pervasive  in  conversa(ons.   –  In  order  to  recognize  argumenta(ve  acts,  advanced   Natural  Language  Understanding  is  necessary.   •  Future  work:   –  Scalability:  need  to  process  millions  of  call  per  day!   –  Mul(-­‐language:  call  centers  all  over  the  world.   ©  2011  interAnaly(cs   24  
  • 25. The  Team   Dr. Lammert Vrieling (1968) - Chief Executive Officer ‣ 15 years in both profit and not-for-profit organizations as consultant, trainer/coach and as executive. ‣ Experience in the steel and aluminium industry, multimedia publishing and newspaper, financial services and in the not-for-profit sector. David E. Walker (1964) - Chief Operating Officer ‣ 25+ years in IT as Software engineer, developer, project manager, and architect. ‣ Senior Software Solutions Architect with extensive experience in designing, developing and delivering enterprise solutions for payment processing, human resource, healthcare, marketing, manufacturing and scientific research environments. Prof. Dr. Rodolfo Delmonte (1946) - Chief Science Officer ‣ Since 1993 head of Computational Linguistics Laboratory of the University of Venice, Italy ‣ 30+ years experience in computational linguistics ‣ From 1986 to 1992 he worked with the Department of Engineering of the University of Parma. From 1978 to 1986 worked with the Department of Dr. Vincenzo Pallotta (1966) - Chief Technology Officer ‣ 30 years in ICT ‣ 10 years in R&D ‣ Human-Language Technology, Digital Libraries, Artificial Intelligence, Ubiquitous Computing, Human-Computer Interaction, Usability Engineering, Information Retrieval, Web Search Engines, Semantic Web, Computational Logics, Training and Education, e-learning. 23 ©  2011  interAnaly(cs   25  
  • 26. …find  us  at   www.interanaly=cs.ch