RecSys 2010 Talk on Paper with the same name.

1. Thomas Sandholm, Hang Ung, Christina Aperjis, Bernardo Huberman Hewlett-Packard, HP Labs, Social Computing Lab RecSys, Barcelona September 27, 2010 Global budgets for local recommendations

2. Why Vote? How do we get more people to contribute their opinions?

4. 500 30 6 Users EXPERIMENTS LOCATIONS Paris Chicago Athens Palo Alto Mumbai Bangalore

5. Many people consume content – FEW LEAVE OPINIONS May 2008 300M+ April 2010 8M April 2010 27M Sample taken Users

6. RATING Budgets and RANKING Restaurants Restaurants $3 Pizza Bob:$10 $4 Pizza http://roundtable.com http://dominos.com Alice: $11 Restaurants $6 Burgers http://burgerking.com Restaurants $8 Diners http://applebees.com Top Channels Friend Channels $8 Applebee’s $6 Burger King $4 Dominos $3RoundTable $8 Diners $7 Pizza $6 Burgers $6 Burgers $4 Pizza John

7. RATING REWARDS Alice $5 http://dominos.com Bob $4 http://dominos.com A $3 http://roundtable.com B $3 http://roundtable.com http://roundtable.com $3 John Top Rewards (reward factor 10)$40Bob Dominos @ A $30AliceRoundTable @ B

8. CLICK TO RATING Ratio July 2010 4K May 2008 300M+ April 2010 8M April 2010 27M Sample taken Users

9. RECOMMENDATION SUCCESS Success = proportion of query sessions ending with clicks/ratings

10. System Coverage

11. Mechanical Turk Experiments Setup-5 Surveys -10 URLs -6 Locations -3 Continents -500 Users Surveys -1-5 Star -Budget -Star Bonus -Budget Bonus -Gloe

12. Mechanism Results Kendall Tau Rank Correlation (higher better) RMSE (lower better)

13. Bonus effect on participation Probability of signals Number of surveys taken

14. Lessons LEARNED Amazon Mechanical Turk Workers not random geographic sample Sensitive to task complexity Respond well to small incentives Budget Mechanism Higher quality recommendations with incentives Social/Economic/Status value extract more opinions Tuned based on usage, e.g. reward factor

15. FUTURE WORK Projects AfricaMap: map annotation in remote parts for disaster relief UNOSAT/Uni. Geneva Mobile print provider recommendations via HP ePrint Research GSP Auction for commercial bidding LMSR Market for recommendation arbitrage Enhance reward mechanism to both encourage and identify high quality contributions

16. Papers at www.hpl.hp.com/research/scl Live system at www.hpgloe.com Social Computing Lab THANK YOU

17. Backup

18. System Architecture

19. Scalability

20. HP Gloe: STATUS ~4k* users on Android, iPhone, BlackBerry, WebOS, Web… ~7m* recommendations at http://hpgloe.com *Sept 2010

21. Lessons LEARNED CONTINUED Gloe system Geohash location partitioning simple and efficient HTTP(S) GET/JSON(P) has served us well on all platforms MySQL & Sharded architecture flexible and fast