Talk given by Sara-Jayne Farmer in November 2011.

1. Including the Human in
Humanitarian Big Data
Sara-Jayne Farmer
@bodaceacat

2. The UN’s Development Reversal Problem
2

3. Where we’d like to be
3

4. How bad is it? 2008 – 2011 information lags
4

5. 2008: Not knowing what, who, where
5

6. 2011: New Data Sources
6

7. But new data only gets us so far…
Humans don’t have all the info, machines don’t have all the smarts
7

8. Hunchworks: hypothesis management
» System = people + processes + tech
» Break down the siloes: Engage and connect the global network of
UN and international development field staff with analysts and
decision makers
» Use the machines better: use agents to detect weak signals
» Use the people better: Encourage „hunches‟ (hypotheses) about
anomalous or interesting data and observations
» Encourage cooperation: Rapidly collect information about emerging
situations
» Combine strengths: mix human insight with data analysis results
8

9. Hunch Mechanism
» Detect an interesting “weak signal”
» Hypothesize (make a hunch) about the nature of the signal
» Share the hunch with your social graph
» Attract related signals and evidence
» Verify hunch by engaging communities of practice
» Act
» Use confirmed hunch to refine trust within the system
9

10. Issues: Community, Complementary skills
10

11. Issues: Encouraging use, encouraging Trust
11

12. Issues: Encouraging Action
“is a hunch true enough to take action?”
12

13. Issues: Security, localisation, bandwidth
13

14. Issues: fun problems we’re working on
» Hunch set management
• Clustering: hunches are related
• Merging: hunches are the same thing
• Splitting: hunch has changed significantly
» Mixing human and bot users
• Autonomy: how to share information, decisions, responsibility
• Managing culture differences between humans and bots
» Managing credibility
• Rating hunches, users, data
• Overall and friends‟ ratings
» Hunch network as a reasoning system
• vs “a hunch is the discussion surrounding it”
14