Yahoo! Answers is the largest Q&A service with hundreds of thousands of questions asked and answered every day. Traditional moderation systems failed to scale with the products growth. To address these challenges the team deployed a Community Moderation system which empowering trusted Answers members to report and remove abusive content automatically. One of the challenges to making the system work was to understand who to trust. The Answers team built a rich reputation model based on an analysis of over a dozen different system actions. Creating a successful community moderation system required changes in technology, community guidelines & policy, and user experience design.

4. Activity

5. Activity

11. Reputation System

12. Answers Moderation Model Hide Content Item? Content Reporter Author

13. Answers Moderation Model REPORTER Abuse Suspicion Reputation REPORTER Community Investment Reputation User’s Abusive Reputation Confirmed Abuse Reporter Reputation Abuse Reporter Bootstrap Reputation Abuse Reporter Reputation CONTENT ITEM Abuse Reputation AUTHOR Overall Asker Reputation AUTHOR Overall Answerer Reputation AUTHOR Overall Asker Reputation Question Quality Reputation AUTHOR Overall Answerer Reputation Answer Quality Reputation Hide Content Item?

14. the story of a report an abuse report comes in We need to decided: Who do we believe? Do we hide the content? A question is asked.. Author Reporter

19. How do we solve the cold start problem?

21. Compared to Illustration by Bryce Glass

22. Do we trust the author? How good are they at Asking questions? How good are they at Answering? Do they have a history of abuse? AUTHOR Overall Asker Reputation AUTHOR Overall Answerer Reputation AUTHOR Overall Asker Reputation Question Quality Reputation AUTHOR Overall Answerer Reputation Answer Quality Reputation Users Abusive Reputation

23. we make a decision an abuse report comes in A question is asked.. Reporter Author

26. decisions can be revised an abuse report comes in A question is asked.. Reporter Trust level is re-instated and temporary protection assigned. Trust level for Reporting is decreased. Author

28. So how did we do?

29. Speed

30. Accuracy

32. Questions? Bryce Glass Randy Farmer Ori Zaltzman http://buildingreputation.com / Micah Alpern [email_address] twitter:malpern