Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Review Analysis:
An Approach to Leveraging User-Generated Content in
the Context of Retail
Jennifer Prendki, Principal Dat...
Outline
• Business motivation
• Algorithm Pipeline
• Feature Space Computation
• Sentiment Capture
• Real-Life Examples an...
Business Motivation
3
 Business
Motivation
 Algorithm
 Feature Space
Computation
 Sentiment
Capture
 Real-Life
Exampl...
Business Motivation
4
 Business
Motivation
 Algorithm
 Feature Space
Computation
 Sentiment
Capture
 Real-Life
Exampl...
Business Motivation
5
 Business
Motivation
 Algorithm
 Feature Space
Computation
 Sentiment
Capture
 Real-Life
Exampl...
Business Motivation
6
 Business
Motivation
 Algorithm
 Feature Space
Computation
 Sentiment
Capture
 Real-Life
Exampl...
Business Motivation
“ I bought this for my daughter to
do her college work on, it's been
great, no problems so far. “
[Sup...
Review Analysis: A Current Landscape
• Sentiment analysis
• Best known use case: Social Media Analysis/Tweets
Why tweets? ...
Algorithm Pipeline
9
Product 𝛼
Product 𝛽
Product 𝛾
Review
Review
Review
Review
Review
Review
Fc
Category C
Feature
Space
C...
Algorithm Pipeline
10
Product 𝛼
Product 𝛽
Product 𝛾
Review
Review
Review
Review
Review
Review
Category C
F 𝛼
F 𝛽
F 𝛾
Senti...
Algorithm Pipeline
11
Product 𝛼
Product 𝛽
Product 𝛾
Review
Review
Review
Review
Review
Review
Category C
Sentiment
Sentime...
Feature Space Computation
• Textual reviews go through a careful process:
• TF/TF-Idf transform on documents
• Stop words ...
[0-1]
Intensity of negativity
in sentence
{'neg': 0.0, 'neu': 0.58, 'pos': 0.42, 'compound': 0.4404}
[0-1]
Intensity
of ne...
Sentiment Capture with Vader
14
”This computer is good deal.” “This computer is a bad deal.”
pos 0.42 0.0
neu 0.58 0.533
n...
Sentiment Capture with Vader
15
Product A
“The design and picture quality are amazing!”
I love it! Just perfect for people...
Sentiment Capture with Vader
16
Product A
design
Product B
design
picture
quality
picture
quality
design
picture quality
b...
Results Discussion: Real Life Example
17
Product BProduct A
 Business
Motivation
 Algorithm
 Feature Space
Computation
...
Results Discussion: Real Life Example
18
screen
design
quality
performance
NEG NEU POSscreen
design
performance
quality
sc...
Results Discussion: Real Life Example
19
This laptop exceeds my expectations. It's fast,
it's powerful, it's compact and g...
Results Discussion: Real Life Example
The design is what caught my eye. Everything
about this laptop is okay, except the b...
Conclusion and Future Work
Work in Progress
• Synonym computation: work in progress
• Observed bias in sentiment, needs pa...
References
[1] Gensim
https://radimrehurek.com/gensim/models/word2vec.html
[2] GloVe
http://nlp.stanford.edu/projects/glov...
Questions?
23
Back-Up Slides
Sentiment Capture with Vader
25
>>> sentence1 = ”This computer is a good deal."
{'neg':0.0,'neu':0.58,'pos':0.42,'compound...
Ratings vs Sentiment Analysis
26
Ratings (number of stars)
Average sentiment from
text review
negativeneutralpositive
 Bu...
Ratings vs Sentiment Analysis
27
Good reviews
Bad reviews
user bias =
𝑛 𝑝𝑜𝑠 − 𝑛(𝑛𝑒𝑔)
𝑛 𝑝𝑜𝑠 + 𝑛(𝑛𝑒𝑔)
where:
pos = number of...
Review Bias
28
• Where is subjectivity coming from?
• Language bias / gender bias / etc.
• Vader package biases due to dev...
Prochain SlideShare
Chargement dans…5
×

Jennifer Prendki, Principal Data Scientist, @WalmartLabs at MLconf SF 2016

782 vues

Publié le

Review Analysis: an Approach to Leveraging User-Generated Content in the Context of Retail: What are customers really thinking? What are they looking for specifically when shopping for a product? And if they are satisfied with their purchase, what is the main reason?

Today’s technology offers many different avenues for customers to express themselves, set their expectations in writing, and share their opinion, frustration or satisfaction regarding all kinds of products and services. ‘Leaving a review’ has become an integral part of the purchase process. Through reviews, customers are volunteering invaluable information that can be turned into insights that would help drive business decisions (if you are a retailer), or help you make a successful purchase (if you are a customer). Yet the amount of data available to make these decisions is oftentimes extremely large, and it might be difficult for a human to read and synthesize all that has been said about their product of interest.

Review analysis and opinion mining offer solutions to automate the analysis of customer feedback through large-scale machine learning, natural language processing and sentiment analysis, and allow retailers to better understand their customers… as well as their data.

In this talk, I will present the various ways in which machine learning techniques can be used to extract the most significant features for a given category of products. I will then dig into a process aiming at identifying the sentiments relative to these features, and a useful way to aggregate this information into insights that are both usable and readable by any user. I will end with mentions to some of the challenges met when trying to extract objective information from a data source likely tainted with human subjectivity in an ever-changing market.

Publié dans : Technologie
  • Soyez le premier à commenter

Jennifer Prendki, Principal Data Scientist, @WalmartLabs at MLconf SF 2016

  1. 1. Review Analysis: An Approach to Leveraging User-Generated Content in the Context of Retail Jennifer Prendki, Principal Data Scientist Walmart Global e-Commerce California, USA The Machine Learning Conference, San Francisco, CA 11/11/2016
  2. 2. Outline • Business motivation • Algorithm Pipeline • Feature Space Computation • Sentiment Capture • Real-Life Examples and Results • Future Work and Conclusions 2  Business Motivation  Algorithm  Feature Space Computation  Sentiment Capture  Real-Life Examples  Future Work and Conclusions
  3. 3. Business Motivation 3  Business Motivation  Algorithm  Feature Space Computation  Sentiment Capture  Real-Life Examples  Future Work and Conclusions
  4. 4. Business Motivation 4  Business Motivation  Algorithm  Feature Space Computation  Sentiment Capture  Real-Life Examples  Future Work and Conclusions
  5. 5. Business Motivation 5  Business Motivation  Algorithm  Feature Space Computation  Sentiment Capture  Real-Life Examples  Future Work and Conclusions
  6. 6. Business Motivation 6  Business Motivation  Algorithm  Feature Space Computation  Sentiment Capture  Real-Life Examples  Future Work and Conclusions
  7. 7. Business Motivation “ I bought this for my daughter to do her college work on, it's been great, no problems so far. “ [SuperMom72] “ Works like a charm, would definitely recommend to anyone on a budget. “ [Vamsy] “ Fast CPU but slow disk drive slows everything down. ” [TalonBay] “ I don't do gaming or downloading movies or music, so for those folks I can't speak to the performance. But for surfing the web, checking email, etc., this computer will save you time for watching the little ball spin!” [Anonymous] 7  Business Motivation  Algorithm  Feature Space Computation  Sentiment Capture  Real-Life Examples  Future Work and Conclusions
  8. 8. Review Analysis: A Current Landscape • Sentiment analysis • Best known use case: Social Media Analysis/Tweets Why tweets?  shorter, condensed, highly sentimental content • Movie review analysis: Kaggle: Analysis of the ‘Rotten Tomatoes’ Dataset • Regarding product review analysis • Little to no papers regarding product review analysis at commercial scale • Shortage of work regarding combination of topic modeling and sentiment analysis 8 Our research: Combine feature computation and sentiment analysis to summarize reviewers’ opinions about a specific product  Business Motivation  Algorithm  Feature Space Computation  Sentiment Capture  Real-Life Examples  Future Work and Conclusions
  9. 9. Algorithm Pipeline 9 Product 𝛼 Product 𝛽 Product 𝛾 Review Review Review Review Review Review Fc Category C Feature Space Computation F 𝛼 F 𝛽 F 𝛾 Feature Space Reduction  Business Motivation  Algorithm  Feature Space Computation  Sentiment Capture  Real-Life Examples  Future Work and Conclusions
  10. 10. Algorithm Pipeline 10 Product 𝛼 Product 𝛽 Product 𝛾 Review Review Review Review Review Review Category C F 𝛼 F 𝛽 F 𝛾 Sentiment Sentiment Sentiment Sentiment Sentiment Sentiment Sentiment Computation For Each Review … Sentiment Computed For Relevant Features  Business Motivation  Algorithm  Feature Space Computation  Sentiment Capture  Real-Life Examples  Future Work and Conclusions
  11. 11. Algorithm Pipeline 11 Product 𝛼 Product 𝛽 Product 𝛾 Review Review Review Review Review Review Category C Sentiment Sentiment Sentiment Sentiment Sentiment Sentiment 𝜎𝑡, 𝛼, 𝑓 𝜎𝑡, 𝛽, 𝑓 𝜎𝑡, 𝛾, 𝑓 ∀ 𝑡 ∈ 𝜏 ∀ 𝑓 ∈ F 𝛼 ∀ 𝑡 ∈ 𝜏 ∀ 𝑓 ∈ Fβ ∀ 𝑡 ∈ 𝜏 ∀ 𝑓 ∈ F 𝛾  Business Motivation  Algorithm  Feature Space Computation  Sentiment Capture  Real-Life Examples  Future Work and Conclusions Sentiment Computation For Each Review … Sentiment Computed For Relevant Features
  12. 12. Feature Space Computation • Textual reviews go through a careful process: • TF/TF-Idf transform on documents • Stop words removal, stemming, part-of-speech selection • Spell-checking • etc. • ‘Synonym’ computation • Can be done using Word Embedding (glove, word2vec) • Can be done building synonym graph using dictionary/Wikipedia • Is complex and tricky, context-sensitive, unsupervised 12 In short: Creating synonym sets is difficult, and challenging as an online algorithm In short: Preprocessing crucial to extracting relevant features  Business Motivation  Algorithm  Feature Space Computation  Sentiment Capture  Real-Life Examples  Future Work and Conclusions
  13. 13. [0-1] Intensity of negativity in sentence {'neg': 0.0, 'neu': 0.58, 'pos': 0.42, 'compound': 0.4404} [0-1] Intensity of neutrality in sentence [0-1] Intensity of positivity in sentence [-1,1] Combination of positive and negative sentiments. Allows positive and negative to ‘compensate’ one another Sentiment Capture with Vader VADER: Valence Aware Dictionary and sEntiment Reasoner • Is a Python sub-module found of the nltk module • Is a lexicon and rule-based sentiment analysis tool • Is specifically attuned to sentiments expressed in social media • Is fully open-sourced, developed and licensed by MIT 13 Sentiment is not boolean posneg neu Sentiment as a PDF  Business Motivation  Algorithm  Feature Space Computation  Sentiment Capture  Real-Life Examples  Future Work and Conclusions
  14. 14. Sentiment Capture with Vader 14 ”This computer is good deal.” “This computer is a bad deal.” pos 0.42 0.0 neu 0.58 0.533 neg 0.0 0.467 compound 0.4404 -0.5423 ”This computer is not powerful.” “This computer is not that powerful.” “This computer is not powerful, but I like it anyways.” “This computer is not that powerful, but I like it anyways.” pos 0.0 0.0 0.0 0.252 neu 0.632 0.682 0.618 0.619 neg 0.368 0.318 0.382 0.129 compound -0.3252 -0.3252 -0.5157 0.3786  Vader is sensitive to adverbs, punctuation, case, emoticons and nuances…  Business Motivation  Algorithm  Feature Space Computation  Sentiment Capture  Real-Life Examples  Future Work and Conclusions
  15. 15. Sentiment Capture with Vader 15 Product A “The design and picture quality are amazing!” I love it! Just perfect for people on a budget. And it is beautifully designed!! “Pretty good, but I am not a fan of the design.” “I don’t think it’s possible to find better for the price” design Product B “I just HATE the design!!” “Okay computer. Wish I read the other reviews first.” design picture quality picture quality + 0.39 + 0.39 + 0.56 ~ 0.48 - 0.62  Business Motivation  Algorithm  Feature Space Computation  Sentiment Capture  Real-Life Examples  Future Work and Conclusions
  16. 16. Sentiment Capture with Vader 16 Product A design Product B design picture quality picture quality design picture quality battery value design battery CPU processor 0.48 0.39 0.46 0.62 0.49 NA NA NA 0.75 0.87NANA 0.43 NA NA 3 1 4 2 1 0.39 0.60 NA NA 1 1 1 NA NA NA NA 0.62 ✍ Scraping Summarizing Sentiment Intensity + 0.39 + 0.56 ~ 0.49 - 0.62 + 0.39 Business Motivation  Algorithm  Feature Space Computation  Sentiment Capture  Real-Life Examples  Future Work and Conclusions
  17. 17. Results Discussion: Real Life Example 17 Product BProduct A  Business Motivation  Algorithm  Feature Space Computation  Sentiment Capture  Real-Life Examples  Future Work and Conclusions
  18. 18. Results Discussion: Real Life Example 18 screen design quality performance NEG NEU POSscreen design performance quality screen design quality performance Some dissatisfaction with overall quality Reviewers are rather happy with keyboard Weight is better for product A than for product B Customers satisfied with keyboard, display, screen, design, …Product B’s weakness is battery life The product’s features are well documented Product A Product B  Business Motivation  Algorithm  Feature Space Computation  Sentiment Capture  Real-Life Examples  Future Work and Conclusions
  19. 19. Results Discussion: Real Life Example 19 This laptop exceeds my expectations. It's fast, it's powerful, it's compact and great to travel with. “The screen is amazing and the keyboard too. the weight is so light, it's become my portable.” It’s durable, good keyboard, decent screen, and a good battery life. Plasticky build quality but holds up with my rough and tough handling. Is is surprisingly light. Keyboard is the best but it tales a bit of getting used to[…] Very light to carry and the carbon color gives an elegant finishing touch. Business Motivation  Algorithm  Feature Space Computation  Sentiment Capture  Real-Life Examples  Future Work and Conclusions Product A 11 reviews, ~ 53 words per review
  20. 20. Results Discussion: Real Life Example The design is what caught my eye. Everything about this laptop is okay, except the battery life. Overall a great laptop with good display and build quality, solid performance and sleek design the only major concern is battery life. The touch screen is absolutely first rate […], and the back-lit keyboard has just the right feel. This is the best computer I've ever owned. […]. I love the backlit keyboard, the easily adjustable resolution and the long battery life. Pros: great screen, keyboard feels nice, best touchpad, very fast, extremely light, built durable Cons: battery life is less than competitors […]. It's really light weight yet really durable. I love the keyboard and mouse pad. 28  Business Motivation  Algorithm  Feature Space Computation  Sentiment Capture  Real-Life Examples  Future Work and Conclusions Product B 40 reviews, ~ 83 words per review
  21. 21. Conclusion and Future Work Work in Progress • Synonym computation: work in progress • Observed bias in sentiment, needs particular attention • Alternative when no/little reviews exist? Potential future applications • Offer a snapshot of product reviews to customers • Assist customers in finding similar items with enhanced feature(s) • Process seller satisfaction information/rating • Customer email processing, determine subject of request automatically 21  Business Motivation  Algorithm  Feature Space Computation  Sentiment Capture  Real-Life Examples  Future Work and Conclusions
  22. 22. References [1] Gensim https://radimrehurek.com/gensim/models/word2vec.html [2] GloVe http://nlp.stanford.edu/projects/glove/ [3] Wordnet https://wordnet.princeton.edu/ [4] nltk.stem http://www.nltk.org/api/nltk.stem.html [5] nltk.vader Paper: VADER: A Parsimonious Rule-based Model for Sentiment Analysis of Social Media Text, C.J. Hutto, Eric Gilbert Code: http://www.nltk.org/_modules/nltk/sentiment/vader.html 22  Business Motivation  Algorithm  Feature Space Computation  Sentiment Capture  Real-Life Examples  Future Work and Conclusions
  23. 23. Questions? 23
  24. 24. Back-Up Slides
  25. 25. Sentiment Capture with Vader 25 >>> sentence1 = ”This computer is a good deal." {'neg':0.0,'neu':0.58,'pos':0.42,'compound':0.4404} >>> sentence2 = “This computer is a very good deal.” {'neg':0.0,'neu':0.61,'pos':0.39,'compound':0.4927} >>> sentence3 = “This computer is a very good deal!!” {'neg':0.0,'neu':0.57,'pos':0.43,'compound':0.5827} >>> sentence4 = “This computer is a very good deal!! :-)” {'neg':0.0,'neu':0.441,'pos':0.559,'compound':0.7462} >>> sentence5 = “This computer is a VERY good deal!! :-)” {'neg':0.0,'neu':0.393,'pos':0.607,'compound':0.8287} >>> sentence6 = “This computer is a very bad deal!! :-(” {'neg':0.588,'neu':0.412,'pos':0.0,'compound':-0.7987} Adverb addition Punctuation addition Emoticon addition Case enhancement Inverse polarity  Business Motivation  Algorithm  Feature Space Computation  Sentiment Capture  Real-Life Examples  Future Work and Conclusions
  26. 26. Ratings vs Sentiment Analysis 26 Ratings (number of stars) Average sentiment from text review negativeneutralpositive  Business Motivation  Algorithm  Feature Space Computation  Sentiment Capture  Real-Life Examples  Future Work and Conclusions
  27. 27. Ratings vs Sentiment Analysis 27 Good reviews Bad reviews user bias = 𝑛 𝑝𝑜𝑠 − 𝑛(𝑛𝑒𝑔) 𝑛 𝑝𝑜𝑠 + 𝑛(𝑛𝑒𝑔) where: pos = number of prior good reviews neg = number of prior bad reviews  Business Motivation  Algorithm  Feature Space Computation  Sentiment Capture  Real-Life Examples  Future Work and Conclusions
  28. 28. Review Bias 28 • Where is subjectivity coming from? • Language bias / gender bias / etc. • Vader package biases due to development specificity? (remember: originally developed for social media) • Incentivized customers/reviewers • Why is it important to correct for it? • Filtering/sorting with ratings doesn’t work as well as expected • Possible options • Filter reviews with large bias • Weight results • Re-center the output of Vader to fit our definition of ’neutrality’ In short: Biases in both ratings and textual sentiment, both need attention  Business Motivation  Algorithm  Feature Space Computation  Sentiment Capture  Real-Life Examples  Future Work and Conclusions

×