This document summarizes the 2008 International Workshop on Recommendation Systems for Software Engineering (RSSE). It introduces recommendation systems and their application to software engineering. Key topics discussed include long-standing questions about how recommendation systems work and emerging questions around their use for software development. The goals of the workshop are to identify new themes in the field, results ready for inclusion in recommendation systems, current shortcomings, and future directions for research.

1. Welcome to
RSSE 2008
Interna'onal Workshop on
Recommenda'on Systems
for So7ware Engineering
Mar2n Robillard  Rob Walker  Tom Zimmermann

2. Recommenda2on Systems...
• Emerged in the early 1990s
• To help users “make choice without sufficient
[…] experience” [Resnick and Varian 1997]
• For:
– Web resources (PHOAKS, Fab, Siteseer)
– Books, movies, music (e.g., Amazon)
– Hotels (e.g., TripAdvisor)
• A.k.a “Collabora2ve Filtering”

3. …for Soware Engineering
• A 21st‐century phenomenon?
• 2002
– CodeBroker [Ye and Fisher]
– Exper2se Browser [Mockus and Herbsleb]
• What’s happening?
– Masses of available data
– Increasingly rich programming domains

4. Long‐Standing Ques2ons
• What is a recommenda2on system?
• How do you “declare your interest” in specific
recommenda2ons
• What is the “cost structure” of choice?
• What is the incen2ve structure for
contribu2ng data?

5. Emerging Ques2ons
• What is special about recommenda2ons
systems for soware engineering?
• What are the different ways to integrate
recommenda2ons into developer workflows?
• How can we evaluate recommenda2ons?

6. Introduc2ons
Name + Affilia2on

7. Goals and Desired Outcomes
• New themes that have emerged
• Stable results we are ready to accept as part
of the recommenda2on system corpus
• Shortcomings of current state of the art
• Future direc2ons

8. Format
• Talks + Ques2ons + Discussions
• Special Poster Session with Blitz Talks
• 5‐minute Madness
• And another thing…
– Sign up on the signup sheet
– Consider pos2ng your slides
– Follow up might include a special issue

9. 5‐minute madness
• Gail:
– “The recommenda2on conundrum”: what do people need
help with, how to deal with compe2ng recommenders
– recommenda2ons work well when task‐first rather than
tool‐first
• Adrian:
– Interac2on effects with other recommenders, either
synchronously or sequen2al
• Rob:
– announcement of a data mining summer school
(rob.deline@microso.com)

10. WRAP‐UP

11. Current shortcomings
• Missing data impedes analyses (e.g., method
bindings)
• Too many (false‐posi2ve) recommenda2ons
• Ra2onales are oen absent
– Apparent tradeoff: usefulness/ease of provision
• User‐specific needs and staleness of those needs
• Lack of feedback capture
• Screen real estate needed

12. Themes
• Thoughts on what recommenda2on systems
generally involve
• Features of recommenda2on systems
– recommenda2on filtering
– ranking/ordering
– ra2onale
– means of presen2ng the results
• Cost model
• Level of granularity

13. Themes
• Modes of use (batch vs. in‐context)
• Tasks being supported
• Degree of collabora2on supported/required
– Privacy issues
• Evalua2on
– Evalua2on should not s2fle innova2on
– “Evalua2on drives innova2on”

14. Stable results
• Recommenda2ons can involve people,
ar2facts, or both
• Dimensions of recommenda2on systems
– popula2on set (what might be recommended)
– possibly, addi2onal data (analyzed to derive
results)
– result set
– ???
• ???

15. Future direc2ons
• Guidelines for what cons2tutes good eval. (rela. to maturity)
– user studies vs. simula2ve studies
• strengths/weaknesses
– benchmarks to avoid user studies, improve comparability
• via Wizard of Oz studies
• real user studies are necessary eventually, though
• What cons2tutes a good ra2onale
– Levels of abstrac2on approach suggested
• Understanding usage modes:
– When is it (not) appropriate to push informa2on, and how
prominently?
– Should systems only provide batch‐mode support or in‐context‐mode
support, or always both
• “Ungameable” recommenda2on systems?
• “Unified theory of RSSE”

18. Stay tuned …
• Workshop summary to go to Soware
Engineering Notes
• Future RSSE workshops
• Journal special issue on RSSE