Presentation of paper titled "Self-organization in online collaborative work settings", published by the SAGE/ACM Collective Intelligence journal, 2022. Link to publication: https://journals.sagepub.com/doi/pdf/10.1177/26339137221078005
2. HOW TO FORM SUCCESSFUL
TEAMS?
CRUCIAL
ACROSS
FIELDS
Innovation
Education
New forms of labor, such as
crowd work and macrotasking
THE WAY WE
WORK IS
CHANGING
More and more people work in
ad-hoc, project-based teams
Crowd labor attracts complex
tasks, which depend on
collaboration
TEAM
FORMATION:
THEN AND NOW
Then: Face-to-face settings,
manager decides group
members, based on knowledge
of skills or collaboration history
Now: Algorithms play central
role due to crowd work’s scale
2022 Self-organization in online collaborative work settings 2
3. ALGORITHMIC
TEAM
FORMATION
TYPICALLY TOP-DOWN
Using pre-existing information (e.g., skills,
demographics) to optimize certain business
objectives (e.g., deadline, cost)
No worker agency (e.g., feedback)
Fit for well-defined tasks (known-requirements, little
dependency on collaboration dynamics)
LACK OF AGENCY IN WHO
ONE WORKS WITH CAN
Stifle creativity and initiative-taking
Increase psychological discomfort
Result in less-than-optimal collaboration results
Especially when the task concerned is open-ended,
creative, and complex
2022 Self-organization in online collaborative work settings 3
4. HOW TO DESIGN
ALGORITHMS THAT GIVE
WORKERS CONTROL
WHILE MAINTAINING
QUALITY?
Self-organization in online collaborative work settings 4
2022
6. SOPs: CORE PRINCIPLES
SELF-ORGANIZATION
Crowd workers try out and evaluate
different collaborators
Gradually converge to preferred pairs
Algorithm facilitates rather than directing
EXPLORATION-
EXPLOITATION
Cycles of intra-pair collaboration and inter-pair
competition motivate workers to find the best
matches while progressing the task
Exploration of new collaborations, exploitation of
existing ones
Self-organization in online collaborative work settings 6
2022
7. SOPs METHOD: STEPS
COLLABORATION
Participants collaborate in dyads to progress a creative task, which for this study is
continuing a short fictional story
Self-organization in online collaborative work settings 7
2022
8. PEER EVALUATION
Then they evaluate their in-between collaboration
Self-organization in online collaborative work settings 8
2022
SOPs METHOD: STEPS
9. BEST STORY VOTING
Next, the pairs compete for the best story continuation through peer review. The result is
appended to the main story.
Self-organization in online collaborative work settings 9
2022
SOPs METHOD: STEPS
10. COLLABORATOR SELECTION
Finally, participants indicate which collaborators they want to work with in the next round
(process repeats for three rounds).
Self-organization in online collaborative work settings 10
2022
SOPs METHOD: STEPS
11. BEHIND THE SCENES: SOPs
ALGORITHM
Self-organization in online collaborative work settings 11
2022
12. EVALUATION: EXPERIMENTAL
CONDITIONS
SOPS
People indicate preferred
collaborators
Algorithm respects user
choices
PLACEBO
Illusion of agency
Same interface as SOPs
Algorithm does not take choices
into account, pairs users randomly
NO-AGENCY
Lack of agency, similar to
existing top-down team
formation methods
Randomly initialized pairs, fixed
throughout the process
2022 Self-organization in online collaborative work settings 12
13. EXPERIMENTAL SETUP
PARTICIPANT
S AND
OUTPUTS
140 people
18 experiments
18 final stories, 196 unique
story continuations
Random allocation to condition
Total time per experiment: 30
minutes
STORY
EVALUATION
External story evaluation,
10 different judges per
story (AMT)
STORY QUALITY
CRITERIA
Grammar: How grammatically and
syntactically correct is the story?
Interest: How interesting is the
story?
Originality: How original is the story?
Plot: How good is the story plot?
Overall impression: Overall, how
much did you like the story?
2022 Self-organization in online collaborative work settings 13
14. QUALITY OF THE
CREATIVE OUTCOME
SOPs wrote stories of
higher quality across all examined
axes
Self-organization in online collaborative work settings 14
2022
EVALUATION: RESULTS
15. QUALITY OF THE
COLLABORATION
People in the SOPs condition
evaluated one other higher in
three axes:
Collaboration
Helpfulness
Skillfulness
Self-organization in online collaborative work settings 15
2022
EVALUATION: RESULTS
16. COMPETENCIES IN
COMMON
SOP members were more
aligned in personal values,
skill similarity, and work
strategy
Self-organization in online collaborative work settings 16
2022
EVALUATION: RESULTS
17. IMPACT
2022 Self-organization in online collaborative work settings 17
SCIENTIFIC
Lay the ground for human-
centered, non-intrusive
collaborative crowd work
systems
Workers integral part of
algorithmic decision-making
INDUSTRIAL
Help platforms integrate
collaboration more
systematically into their
processes
SOCIETAL
Balanced coexistence between
human workers and collaboration
management algorithms
More respectful Future of Work
Better uses of human and
algorithmic capabilities
