Prediction of Nash Bargaining Solution in Negotiation Dialogue [PRICAI '18]

1. Prediction of
Nash Bargaining Solution in
Negotiation Dialogue
Kosui Iwasa, Katsuhide Fujita
Tokyo University of Agriculture and Technology

2. Outline
Background
Related Works
Problem Definition
Proposed Method
Evaluation
Conclusion

3. Background

4. Importance of Negotiations
Generally, there are negotiations in …
business politics judicature
Negotiations are essential activities for people.

5. A Problem in Negotiations
Solutions
Pareto Front
Nash Bargaining Solution
Reward for !"
Rewardfor!#
com
prom
ise
Actual Agreement

6. Main Purpose of our Study
Utility for !"
Utilityfor!#
Solving the gap between Nash
bargaining solutions and actual
agreements
Nash Bargaining Solution
Actual Agreement
Predicting Nash Bargaining Solution
from negotiation dialogues

7. Contributions
Proposed Method
Predicting Nash bargaining solution by deep learning
from dialogues in natural language in multi-issue
negotiations
Experimental Result
In Social welfare and Nash product,
solutions predicted by the our method are superior to
solutions formed in human-human negotiations

8. Related Work

9. Automated Negotiation
Expected to support or act on human negotiations.
Agents can negotiate with each other to make
agreements by the predetermined negotiation
protocol (not in natural languages).
GENIUS: Automated Negotiation Platform

10. Comparison between agents and humans
in negotiations
In simple protocol
With well-defined
utility functions
Agents
In natural languages
With unclear preferences
Humans
The negotiation between humans have to
! define their own utility functions
! negotiate in simple protocols (not natural languages)

11. Human-Human
Negotiation dialogues
Where should we go on a
trip in our next vacation?
I want to go to Tokyo
and see a famous shrine.
Too bad. Prices in Tokyo
are too high. How about
Beijing?
A
B
C

12. Agent-Agent
Negotiation dialogues
OFFER: Tokyo
OFFER: Beijing
A
B
OFFER: Shanghai
C
ACCEPT
A

13. Agent-Agent Negotiations (SAOP)
in Multi-issue Negotiation Problems
OFFER:
Budget - $300
Term- 5 days
Destination – Tokyo
Transportation – Airplane
Hotel – Raymond Hotel
Day #1 – Go to Sensou-ji
A
… It is complicated to humans

14. The End-to-end Negotiator
in a Natural Language
Deal or No Deal? End-to-End Learning for Negotiation
Dialogues [Lewis et al. 2017]

15. Solution
Generator
The End-to-end Negotiator
in a Natural Language
Input
Dialogue
Encoder
<OTHER> I want hatsto
Speech
Generator
<ME> Is it ?
< OTHER > I want ?to
Hat: 2, Book: 1, Ball: 0

16. Negotiation Agents in Natural
Languages
!They could not outperform the results of humans
in both individual utility and social welfare.
!To act the agent in the user’s place in the real
world, their own utility functions should be
defined.

17. Problem Definition

18. Summary of
the Problem Definition
Two participants !", !$ exchange some items
Multi-issue negotiation
The utility of each agent is calculated as the
weighted average of option's score
There is no dependency between issues

19. An Example of
Issues and options
How to allocate fruitsDomain
ApplesIssues Bananas Oranges
Options
(# of items)
0 2… 0 5…
Every issue has options,
which is a integer and a limited range
0 3…

20. An Example of
A Solution
Domain
Issues
Options
(# of items)
One of A
Solution
1 2 2
How to allocate fruits
0 2… 0 5… 0 3…
Apples Bananas Oranges

21. How To Calculate Utility
For Each Agent?
Participant !"
Participant !#
Weights
Apple: 0.5
Banana: 0.3
Orange: 0.1
Weights
Apple: 0.1
Banana: 0.2
Orange: 0.7

22. How To Calculate Utility
For Each Agent?
Participant !"
Participant !#
Weights
Apple: 0.5
Banana: 0.3
Orange: 0.1
Weights
Apple: 0.1
Banana: 0.2
Orange: 0.7

23. How To Calculate Utility
For Each Agent?
Participant !"
Weights
Apple: 0.5
Banana: 0.3
Orange: 0.1
0.5 &
2
2
+ 0.3 &
3
5
+ 0.1 &
0
3
= 0.68
Utility for !" in the solution
Apple Banana Orange

24. Proposed Method

25. Outline of the proposed method
1. Predict the weights of each issue
for each participant from negotiation dialogues
in natural languages
2. Search for Nash bargaining solution
through exhaustive search
based on the predicted weights

26. Outline of the proposed method
1. Predict the weights of each issue
for each participant from negotiation dialogues
in natural languages
2. Search for Nash bargaining solution
through exhaustive search
based on the predicted weights

27. 1. Predict the weights
of each issue for each participant
I. Preprocessing

28. Input
1. Predict the weights
of each issue for each participant
II. Prediction with Bi-GRUs
Bi-GRUs
Encoder
Attention
Output
<TGT> I want <END>to
!
Apple: 0.4 Banana: 0.5 Orange: 0.1
Softmax

29. Summary of the proposed method
1. Predict the weights of each issue
for each participant from conversations
2. Search for Nash bargaining solution
through exhaustive search
based on the predicted weights

30. Evaluation

31. Experimental Settings
Dataset: provided by Facebook AI research
For the end-to-end negotiator (Lewis et al.)
Two humans negotiate in English and allocate books, hats, and
balls.
Hyperparameters
The gradient method: RMSProp
The number of GRU units: 256

32. Experiment #1
Prediction of Issue Weights
Evaluate the quality of prediction of issue weights
10-fold cross-validation to evaluate
Spearman's rank vs Ground truths: 61%
In prediction of the rank of item's importance
Accuracy: 70%
In prediction of the most important item

33. Experiment #2
Prediction of Nash bargaining solution
Evaluate the quality of predicted solutions by
comparing with agreements in human-human
negotiations
Metrics
Nash Product
The product of utilities in each participant
Social Welfare
The sum of utilities in each participant

34. Experiment #2
Result in Nash Product

35. Experiment #2
Result in Social Welfare

36. Conclusion

37. Conclusion
Proposed Method
Predict Nash bargaining solution from dialogues
by natural language in a multi-issue negotiation
using Bidirectional GRUs
Experimental Results
In Social welfare and Nash product,
the solutions predicted by our method are superior to
the solutions in human-human negotiations