Juho Kim, Haoqi Zhang, Paul André, Lydia B. Chilton, Wendy Mackay, Michel Beaudouin-Lafon, Robert C. Miller, and Steven P. Dow. Cobi: A Community-Informed Conference Scheduling Tool. UIST 2013.
http://dl.acm.org/citation.cfm?id=2502034
Eluru Call Girls Service ☎ ️93326-06886 ❤️🔥 Enjoy 24/7 Escort Service
Cobi: A Community-Informed Conference Scheduling Tool. UIST 2013 slides
1. Cobi: A Community-Informed
Conference Scheduling Tool
Juho Kim
Paul André
Wendy Mackay
Robert C. Miller
Haoqi Zhang
Lydia B. Chilton
Michel Beaudouin-Lafon
Steven P. Dow
1
3. Challenges for conference organizers
1. Lack data on paper affinities
2. Lack data on what people want
3. Lack software for resolving conflicts
101 102 103 104
4. Multi-dimensional requirements
1pm
?
?
Room size
Awarded papers
? good or bad?
Session length
?
Themes of the day
3pm
Similar sessions in nearby rooms
…
3
8. Authors, do you want to see the other paper?
Wikipedia Classroom
Experiment
A Pilot Study of Using
Crowds in the Classroom
✓
✓
✓
X
input mediation
constraint
✓
Encode if two or more
people express interest
in seeing both.
“papers of mutual interest should
not be in opposing sessions.”
8
9. constraint
conflict
1pm
“Wikipedia Classroom Experiment” and
“A Pilot Study of Using Crowds in the Classroom”
should not be in opposing sessions.
Room 1
Room 2
Wikipedia Classroom
Experiment
A Pilot Study of Using
Crowds in the Classroom
9
23. Committee Clustering
1722 paper affinities from
64 committee members
•
Paul André, Haoqi Zhang, Juho Kim, Lydia B. Chilton, Steven P. Dow, and Robert
C. Miller. Community clustering: Leveraging an academic crowd to form
coherent conference sessions. HCOMP 2013, to appear.
paper D
?
paper B
paper A
?
?
paper C
paper E
?
23
26. Authorsourcing
8651 preferences and constraints from
645 authors, covering 87% of accepted
submissions
my paper
fits with my paper
paper A
paper D
paper E
does not fit
paper B
paper C
paper Z
26
27. The value of community input
“Authors were asked for input, most gave it, we
tried hard to accommodate them, and almost
nobody complained.”
27
38. Mixed-initiative problem solving
“I was by and large driven by what Cobi was
suggesting. As you make progress you can
progressively integrate other criteria that are
not explicit in the system.”
38
43. Cobi is a community-informed
mixed-initiative system for a
large-scale planning problem.
Communitysourcing applications
Encoding constraints from community data
Mixed-initiative interface for conflict resolution
43
47. Different crowd, Different expertise
• Organizers: 3
– Session balance, theme of the day
• Committee members: 200+
– Affinities between papers in their subfield
• Authors: 1000+
– Fit with their own paper
• Attendees: 3000+
– Individual preferences for attending talks
50
48. Techniques for Affinity Creation
• Manual grouping from TP meeting
• Automatically generated affinity scores using
TF-IDF
• Committee-generated affinity scores
using community clustering
51
49. Fine-grained affinity data
• How relevant is this paper to yours?
– “should it be in the same session as your paper?”
• Is this paper interesting to you?
– “Would you like to see this paper’s presentation?”
52
50. Session-related
Constraint Type
author with papers in opposing
sessions
topics of interest to a persona in
opposing sessions
Data Source Severity Encoded
systemgenerated
high
systemgenerated medium
-
53
51. Paper-related
Constraint Type
papers of mutual interests in
opposing sessions
papers that do not fit well in the
same session
Data Source Severity Encoded
authorsourcing
Preference Type
papers good in the same session
Data Source Severity Total
authorsourcing N/A
805
high
923
authorsourcing medium
651
54
52. Chair-related
Constraint Type
chair's paper in own session
chair's paper in opposing sessions
chair interested in opposing
sessions
chair in a session with a bad fit
Preference Type
chair fits well in the session
Data Source Severity Encoded
systemgenerated
high
systemgenerated
high
authorsourcing medium
chairs
medium
243
-
Data Source Severity Total
chairs
N/A
55
53. Constraint Type
author with papers in opposing
sessions
topics of interest to a persona in
opposing sessions
papers of mutual interests in
opposing sessions
papers that do not fit well in the
same session
Related to
Session
Data Source
systemgenerated
systemgenerated
Paper
authorsourcing
Paper
authorsourcing medium
systemgenerated
high
systemgenerated
high
authorsourcing medium
chairs
medium
Session
chair's paper in own session
Chair
chair's paper in opposing sessions
chair interested in opposing sessions
chair in a session with a bad fit
Total violated
Chair
Chair
Chair
Preference Type
papers good in the same session
chair fits well in the session
Total satisfied
Paper
Chair
Severity Total Initial Final Change
high
-
31
0
-31
medium
-
6
4
-2
923
40
19
-21
651
129
42
-87
-
21
0
-21
243
-
6
5
0
238
0
4
1
70
-6
-1
1
-168
high
Data Source Severity Total Initial Final Change
authorsourcing N/A
805 268 272
4
chairs
N/A
90
78
-12
358 350
-8
56
54. System-defined conflicts over time
Conflict
Count
author in opposing sessions
persona in opposing sessions
30
Resolve author conflicts
20
Adjust session length
Balance awards
10
Switch rooms
0
0
200
400
600
800
Edit Number
57
55. Community-defined conflicts over time
papers that do not fit well in the same session
papers of mutual interests in opposing sessions
Conflict
Count
120
Make coherent sessions
Adjust session length
Balance awards
90
60
30
Switch rooms
0
0
200
400
Edit Number
600
800
58
56. chair's paper in own session
chair's paper in opposing sessions
chair interested in opposing sessions
chair in a session with a bad fit
Conflict
Count
30
20
Assign session chairs
10
0
270
420
570
720
Edit Number
59
Walking around the room with paper index cards to form sessions. Paper-based, manual, dependent on the people in the room.Organizers spend months to refine the program, to resolve conflicts and take into account dozens of other factors.
Paper affinities, which tell us if two papers have a good fit in a sessionIt’s hard to know what people want in advance: some rooms get over-populated while others don’tConsequences, manually checkingMore than just conflicts – purely automatic methods cannot take into account all the subjective, implicit requirements.
Collecting rich data from community members, asking people to tell us papers have a good fit, and what they want to see.Encoding community data in a way that the system can handleScheduling tool supports conflict resolution
Segway to conflicts – it’s about conflict resolution.
Now the system enters the move mode,Which displays the session being moved in the notification panel and highlights in yellow.The consequences of swapping this session with other sessions are shown in numbers.The lower the better, indicating that more conflicts will be resolved.The recommended edits that can resolve most conflicts are highlighted in green.Now let’s review one of the recommended options to see if swapping is a good idea.
Now the system enters the move mode,Which displays the session being moved in the notification panel and highlights in yellow.The consequences of swapping this session with other sessions are shown in numbers.The lower the better, indicating that more conflicts will be resolved.The recommended edits that can resolve most conflicts are highlighted in green.Now let’s review one of the recommended options to see if swapping is a good idea.
Collecting rich data from community members, asking people to tell us papers have a good fit, and what they want to see.Encoding community data in a way that the system can handleScheduling tool supports conflict resolution
Collecting rich data from community members, asking people to tell us papers have a good fit, and what they want to see.Encoding community data in a way that the system can handleScheduling tool supports conflict resolution
Collecting rich data from community members, asking people to tell us papers have a good fit, and what they want to see.Encoding community data in a way that the system can handleScheduling tool supports conflict resolution
Collecting rich data from community members, asking people to tell us papers have a good fit, and what they want to see.Encoding community data in a way that the system can handleScheduling tool supports conflict resolution
Collecting rich data from community members, asking people to tell us papers have a good fit, and what they want to see.Encoding community data in a way that the system can handleScheduling tool supports conflict resolution
Note that not all conflicts are removed. Mixed-initiative. Subjective requirements beyond conflicts.
If Steven had 10 papers, he’ll be able to go to all ten of his paper presentations even if he’s not presenting them.