Recommandé
Recommandé
Contenu connexe
En vedette
En vedette (12)
Plus de Prithviraj (Raj) Dasgupta
Dernier
Dernier (20)
Icec11slides uno
- 1. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta C-MANTIC Research Group Computer Science Department University of Nebraska at Omaha, USA ICEC 2011 1 / 37
- 2. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Outline Problem: Traders’ behavior in a prediction market and its impact on the prediction market’s behavior Solution: A multi-agent system that formalizes the strategic behavior and decision making by market’s participants based on a partially observable stochastic game Experimental validation: Comparison with other trading approaches 1 / 37
- 3. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Prediction Market A Prediction market is a market-based mechanism used to - combine the opinions on a future event from different people and - forecast the possible outcome of the event based on the aggregated opinion 2 / 37
- 4. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Prediction Market 3 / 37
- 5. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Prediction Market 4 / 37
- 6. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Prediction Market 5 / 37
- 7. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Prediction Market 6 / 37
- 8. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Prediction Market 7 / 37
- 9. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Prediction Market 8 / 37
- 10. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Prediction Market 9 / 37
- 11. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Prediction Market 10 / 37
- 12. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Prediction Market 11 / 37
- 13. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Prediction Market 12 / 37
- 14. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Prediction Market 13 / 37
- 15. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Prediction Market Main Features A prediction market is run for a real-life unknown event Each event has a finite duration Each event’s outcome has a security associated with it Traders buy and sell the securities based on their beliefs about the outcome of the event Traders’ beliefs are expressed as probabilities Market maker aggregates the probabilities from all the traders into a single probability, market price Market price of a security represents the probability of the outcome of an event associated with that security happening Traders get paid according to their reported beliefs 14 / 37
- 16. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work A Multi-Agent Prediction Market Software trading agents perform calculations and trade on behalf of human traders Provides testbed for modeling different strategic behaviors of traders through simulations with trading agents15 / 37
- 17. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Do Prediction Markets Work? Yes, evidence from real markets, laboratory experiments, and theory I.E.M. beat political polls 451/596 [Forsythe 1999, Berg 2001, Pennock 2002] HP market beat sales forecast 6/8 [Plott 2000] Sports betting markets provide accurate forecasts of game outcomes [Debnath 2003, Schmidt 2002] Market games work [Pennock 2001] Laboratory experiments confirm information aggregation [Forsythe 1990, Plott 1997, Chen 2001] Theory of Rational Expectations [Lucas 1972, Grossman 1981] and more... 16 / 37
- 18. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Research Problem Addressed Develop a formal, game-theoretic model of the trading agent behavior in prediction markets including - impact of information from external sources on trading agent decisions/behavior, - a solution concept for calculating the equilibrium strategies of the trading agents Research Questions: How does different traders’ behaviors affect market prices? What trading strategies give the highest utilities to the traders? How can prediction markets incentivize traders to participate and report their beliefs truthfully in order to achieve a higher prediction accuracy? 17 / 37
- 19. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Our Solution A partially observable stochastic game with information (POSGI)-based model of the trading agent behavior A correlated equilibrium (CE)-based solution to determine equilibrium strategy in the POSGI representation 18 / 37
- 20. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Partially Observable Stochastic Game with Information (POSGI) 19 / 37
- 21. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Partially Observable Stochastic Game with Information (POSGI) - Logarithmic Market Scoring Rule (LMSR) [Hanson 2007] gives formula to calculate aggregate market price from the outstanding quantity of a security 19 / 37
- 22. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Partially Observable Stochastic Game with Information (POSGI) - Belief state is updated using a Bayesian model of past beliefs, past actions and current observation 20 / 37
- 23. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Partially Observable Stochastic Game with Information (POSGI) - Information signal can be {−1, 0, 1} representing positive, neutral, or negative information 21 / 37
- 24. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Partially Observable Stochastic Game with Information (POSGI) 22 / 37
- 25. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Two Agent Trading Scenario with 1 security Example Actions = to buy, sell, or hold one security Market state is denoted by q C(q) is the cost function calculated by LMSR, that reflects the total money collected by the market maker Can construct a normal form game to capture the decision problem for each agent 23 / 37
- 26. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Correlated Equilibrium (CE) Set up Let N be the set of trading agents Let Ai be the set of actions for agent i ∈ N Let Φi be the set of mixed strategy profiles defined over Ai Joint strategy space is then Φ = × |N| i=1Φi Let φ ∈ Φ be a strategy profile and φi denote agent i’s component in φ Let ui(φ) be the utility of agent i from joint strategy φ 24 / 37
- 27. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Correlated Equilibrium (CE) Correlated Equilibrium is a probability distribution p on Φ such that for all agents i and all strategies φi, φi if all agents follow a strategy profile φ, agent i has no incentive to play another strategy φi instead The Chicken-Dare Game Assume there is a trusted third party that draws a strategy for each player and announces it to each player separately Players agree to follow the strategy suggested by the third party Players get higher payoffs when use CE than NE 25 / 37
- 28. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Correlated Equilibrium (CE) Algorithm CE algorithm applied to our POSGI model is based on the method proposed by Papadimitriou [2008] CE algorithm of the POSGI gives the action (buy or sell; quantity) for each agent at each time step Time complexity of our CE algorithm is O(|N| × |Φi|2) 26 / 37
- 29. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Correlated Equilibrium (CE) with Trading Agents’ Risk Preferences The beliefs and risk preferences of traders are directly correlated [Kadane 1988, Dimitrov 2008] Use constant relative risk averse (CRRA) utility function To find CE in the market with risk-averse trading agents: - Find the set of all Pareto optimal strategy profiles - Check whether a Pareto optimal strategy profile satisfies CE constraints 27 / 37
- 30. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Correlated Equilibrium (CE) with Trading Agents’ Risk Preferences A strategy profile φP is Pareto optimal if there does not exist another strategy profile φ such that ui(φ ) ≥ ui(φP )∀i ∈ N with at least one inequality strict Pareto optimal strategy is found by maximizing weighted utilities and solving the maximization problem using Lagrangian method Proposition If p is a correlated equilibrium and φP is a Pareto optimal strategy profile calculated by p in a prediction market with risk averse agents, then the strategy profile φP is incentive compatible, that is each agent is best off reporting truthfully 28 / 37
- 31. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Experimental Results Set up Default values for market related parameters came from Iowa Electronic Marketplace (IEM) movie market Assume there is one event with two outcomes Consider only disjoint events Report utilities and market price for the security corresponding to the outcome of the event being 1 29 / 37
- 32. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Experimental Results Set up We use the following strategies for comparison of the trading agents’ and market’s behavior 1 ZI (Zero Intelligence) - each agent submits randomly calculated quantity to buy or sell 2 ZIP (Zero Intelligence Plus) - each agent selects a quantity to buy or sell that satisfies a particular level of profit 3 CP (by Preist and Tol) - each agent adjusts its quantity to buy or sell based on past prices and tries to choose that quantity so that it is competitive among other agents 4 GD (by Gjerstad and Dickhaut) - each agent maintains a history of past transactions and chooses the quantity to buy or sell that maximizes its expected utility 5 DP (Dynamic Programming solution for POSG game) - each agent uses dynamic programming solution to find the best quantity to buy or sell [Hansen 2004] 30 / 37
- 33. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Experimental Results Risk-neutral Agents (a) DP strategy has the highest percentage of adoption of some CE (b) Prediction market with trading agents using CE strategy result in more accurate market prices (c) Trading agents using CE strategy obtain 38% more utility than the agents following the next best performing strategy DP 31 / 37
- 34. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Experimental Results Risk-neutral Agents (a) DP strategy has the highest percentage of adoption of some CE (b) Prediction market with trading agents using CE strategy result in more accurate market prices (c) Trading agents using CE strategy obtain 38% more utility than the agents following the next best performing strategy DP 31 / 37
- 35. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Experimental Results Risk-neutral Agents (a) DP strategy has the highest percentage of adoption of some CE (b) Prediction market with trading agents using CE strategy result in more accurate market prices (c) Trading agents using CE strategy obtain 38% more utility than the agents following the next best performing strategy DP 31 / 37
- 36. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Experimental Results Risk-neutral Agents (a) Trading agents using CE strategy obtain more utility than trading agents using ZIP strategy (b) Trading agents using CE strategy obtain more utility than trading agents using CP strategy 32 / 37
- 37. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Experimental Results Risk-neutral Agents (a) Trading agents using CE strategy obtain more utility than trading agents using ZIP strategy (b) Trading agents using CE strategy obtain more utility than trading agents using CP strategy 32 / 37
- 38. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Experimental Results Risk-neutral Agents (c) Trading agents using CE strategy obtain more utility than trading agents using GD strategy (d) Trading agents using CE strategy obtain more utility than trading agents using DP strategy 33 / 37
- 39. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Experimental Results Risk-neutral Agents (c) Trading agents using CE strategy obtain more utility than trading agents using GD strategy (d) Trading agents using CE strategy obtain more utility than trading agents using DP strategy 33 / 37
- 40. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Experimental Results Risk-averse Agents (a) DP strategy has the highest percentage of adoption of some CE (b) Prediction market with risk-averse trading agents using CE strategy result in more accurate market prices (c) Risk-averse trading agents using CE strategy obtain 41% more utility than the agents following the next best performing strategy DP 34 / 37
- 41. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Experimental Results Risk-averse Agents (a) DP strategy has the highest percentage of adoption of some CE (b) Prediction market with risk-averse trading agents using CE strategy result in more accurate market prices (c) Risk-averse trading agents using CE strategy obtain 41% more utility than the agents following the next best performing strategy DP 34 / 37
- 42. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Experimental Results Risk-averse Agents (a) DP strategy has the highest percentage of adoption of some CE (b) Prediction market with risk-averse trading agents using CE strategy result in more accurate market prices (c) Risk-averse trading agents using CE strategy obtain 41% more utility than the agents following the next best performing strategy DP 34 / 37
- 43. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Conclusions and Future Work In this work we have: - described a novel multi-agent based representation of the prediction market using POSG - developed a CE solution for solving POSG - empirically compared different agent behavior strategies in the prediction market - showed how CE can be obtained in the prediction market with risk averse agents 35 / 37
- 44. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Conclusions and Future Work In this work we have: - described a novel multi-agent based representation of the prediction market using POSG - developed a CE solution for solving POSG - empirically compared different agent behavior strategies in the prediction market - showed how CE can be obtained in the prediction market with risk averse agents In the future we plan to: Conduct experiments in an n-agent scenario using richer commercial data sets Investigate the dynamics evolving from multiple prediction markets that interact with each other 35 / 37
- 45. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work References 1 Y. Chen, D. Pennock. Utility Framework for Bounded-Loss Market Maker. Proc. of the 23rd Conference on Uncertainty in Artificial Intelligence (UAI 2007), pages 49-56, 2007. 2 E. Hansen, D. Bernstein, S. Zilberstein. Dynamic programming for partially observable stochastic games. In Proceedings of the 19th National Conference on Artificial Intelligence, pages 709-715, 2004. 3 R. Hanson. Logarithmic Market scoring rules for Modular Combinatorial Information Aggregation. Journal of Prediction Markets, 1(1):3-15, 2007. 4 Iowa Electronic Marketplace. URL: www.biz.uiowa.edu/iem/ 5 C. Papadimitriou, T. Roughgarden. Computing correlated equilibria in multi-player games. Journal of ACM, 55(3):1-29, 2008.36 / 37
- 46. A Multi-Agent Prediction Market based on Partially Observable Stochastic Game Janyl Jumadinova, Raj Dasgupta Outline Introduction Research Problem POSGI Trading Agents’ Strategy Experimental Results Future Work Thank You! Questions? jjumadinova@unomaha.edu C-MANTIC Research Group http://cmantic.unomaha.edu/ 37 / 37