Through the development of electronic commerce, social media and collaborative media, the social commerce appeared. Social commerce, a subset of electronic commerce, is based on social interactions in order to buy and sell goods and services. Nowadays, before buying, people give more importance to the experience feedback they found on internet. However, it is difficult to get an overview of this experience feedback since it is scattered in many online resources, and buyers never have time to read many pages of comments. In this paper, we present an approach which grabs and analyzes experience feedback in order to publish a summary of opinions about a product. We develop this approach with a case study on smartphones and publish a dataset of thousands of comments on a wide range of smartphones. To summarize experience feedback, we use a linguistic appraisal model, based on appreciation, affect and judgement, and we set up an approach using methods and tools from the fields of natural language processing, opinion mining and sentiment analysis.

1. Opinion mining on experience
feedback
A case study on smartphones reviews
Laurent Brisson & Jean-Claude Torrel
IEEE RCIS 2015, May 13-15 2015, Athens

2. Opinion mining and sentiment analysis
Context
People give more importance to the experience feedback they found on
internet
Social interactions have a strong and immediate impact on purchase
behaviour
There is a huge amount of experience feedback available on internet
Tasks
Document level: “What’s the overall opinion about this organization in
latest news?’
Sentence level: “What are the positive tweets about this film?’
Aspect level: “My clients are they satisfied with the food and service?”
2/13 Laurent Brisson & Jean-Claude Torrel Opinion mining on experience feedback - RCIS 2015

3. Motivations
Objective:
To qualifiy experience feedback from customer reviews
Constraints:
Customer reviews could be about anything
The whole analysis process must be deployed in a short period:
• No ressource for annotating a learning dataset
• No supervised learning algorithms
3/13 Laurent Brisson & Jean-Claude Torrel Opinion mining on experience feedback - RCIS 2015

4. A 3-step approach to find relevant opinions
Knowledge
base
Sentence
pre-
processing
Textual data
Search strategies based on a
Linguistic framework for appraisal
Opinions
A knowledge base to define a relevant
perimeter
Sentence pre-processing to add
information
A linguistic framework to structure
search strategies
4/13 Laurent Brisson & Jean-Claude Torrel Opinion mining on experience feedback - RCIS 2015

5. First step: A knowledge base
Knowledge
base
Sentence
pre-
processing
Textual data
Search strategies based on a
Linguistic framework for appraisal
Opinions
The knowledge base is composed of:
An aspect-lexicon which defines the
relevant perimeter to extract opinions
Sentic* polarity lexicon to access word
polarity
*E. Cambria and A. Hussain. Sentic Computing: A
Common-Sense-Based Framework for Concept-Level
Sentiment Analysis. Cham, Switzerland: Springer (2015)
5/13 Laurent Brisson & Jean-Claude Torrel Opinion mining on experience feedback - RCIS 2015

6. Second step: Sentence pre-processing
Knowledge
base
Sentence
pre-
processing
Textual data
Search strategies based on a
Linguistic framework for appraisal
Opinions
Lexical analysis and transformations
Character cleaning
Full stop detection
Tokenization
Constituency-based syntactic analysis
S
VP
NP
NN
problem
JJ
only
DT
the
VBZ
is
NP
NN
camera
DT
The
6/13 Laurent Brisson & Jean-Claude Torrel Opinion mining on experience feedback - RCIS 2015

7. Third step: Search strategies
Knowledge
base
Sentence
pre-
processing
Textual data
Search strategies based on a
Linguistic framework for appraisal
Opinions
In order to extract opinions:
A breadth-first search is performed on the
parse tree
Automata detect appraisal patterns
defined in the linguistic framework
If a pattern is found, a specific rule is
applied to extract the opinion
7/13 Laurent Brisson & Jean-Claude Torrel Opinion mining on experience feedback - RCIS 2015

8. Linguistic framework for appraisal
The language of evaluation, James R. Martin and Peter R. R. White
Appraisal p. 34
attitude p. 45
affect p. 45
appreciation p. 56
judgement p. 52
engagement p. 97
graduation p. 137
Affect as:
• a mental process: “I like this phone”
• a mental state: “I’m very happy with
this phone”
Appreciation as:
• qualification: “The camera is the only
problem”
• presence/absence: “This phone hasn’t
got a camera”
• behavior: “The battery heats up a lot”
Judgement deals with attitudes towards
behaviour (social esteem or social
sanction)
8/13 Laurent Brisson & Jean-Claude Torrel Opinion mining on experience feedback - RCIS 2015

9. Search strategies: an example
“The camera is the only problem” (Appreciation by qualification)
S
VP
NP
NN
problem
JJ
only
DT
the
VBZ
is
NP
NN
camera
DT
The
γ 1
NP
camera
γ 0
γ 2
γ 3
MD
MD
RB
VB
VB
γ 0 γ 4 γ 5
only
γ 6
problem
γ 7
RB
DT
RB, DT NN
JJ
ADJP, ADVP
NN
NN
ADJP, ADVP
JJ
NN
γC
PP
NP
NP, PP
γB
VP
VP
9/13 Laurent Brisson & Jean-Claude Torrel Opinion mining on experience feedback - RCIS 2015

10. Experiment
Our dataset contains:
40,160 comments about 382 smartphones in 81,431 sentences for a total
size of 3.5 Mo.
368 technical specifications from 8 manufacturer websites.
A knowledge base with keyword for smartphone’s related concepts.
Our validation set contains :
708 opinions in 527 comments about 6 smartphones.
Annotations are a consensus between 3 reviewers.
They are available for download: http://goo.gl/YCC5We
10/13 Laurent Brisson & Jean-Claude Torrel Opinion mining on experience feedback - RCIS 2015

11. Results
Dataset
Appreciation Detection Detection with Polarity Eval.
Precision Recall F-Measure Precision Recall F-Measure
Galaxy S5 0.87 0.62 0.72 0.82 0.59 0.69
HTC Desire 310 0.90 0.50 0.64 0.82 0.48 0.60
HTC One (M8) 0.82 0.43 0.56 0.80 0.43 0.56
iPhone 6 0.83 0.50 0.62 0.77 0.47 0.59
Lumia 1320 0.90 0.68 0.78 0.86 0.68 0.76
XPERIA C 0.87 0.66 0.75 0.76 0.62 0.68
Average 0.87 0.57 0.68 0.81 0.55 0.65
Good average precision to find appreciation and qualify their polarity
Recall have to be improved to find more opinions:
• Affect extraction is not yet implemented
• Constituency-based syntactic analysis cannot handle complex sentences
11/13 Laurent Brisson & Jean-Claude Torrel Opinion mining on experience feedback - RCIS 2015

12. Future work
Knowledge
base
Sentence
pre-
processing
Textual data
Search strategies based on a
Linguistic framework for appraisal
Opinions
Manage context-based knowledge
Handle complex sentences with
dependency-based syntactic analysis
Find new rules fitted to affect,
judgement and engagement
12/13 Laurent Brisson & Jean-Claude Torrel Future work

13. Conclusion
Knowledge
base
Sentence
pre-
processing
Textual data
Search strategies based on a
Linguistic framework for appraisal
Opinions
Semantic ressources to define a
semantic relevant perimeter
NLP techniques for data preparation and
syntactic analysis
Use of a sound language framework for
appraisal in English
Search strategies to find relevant
opinions with their polarity
13/13 Laurent Brisson & Jean-Claude Torrel Conclusion