A summary of research results from the Understanding Sources project of relevance to SwiftRiver and Ushahidi.

1. Sources, collaboration and verification
A summary of relevant findings for SwiftRiver from the Ushahidi Wikipedia
Understanding Sources project
Heather Ford.
SwiftRiver/Ushahidi ethnographer.

2. A reminder on the goals of the project
• Attempting to answer the following questions:
• How are Wikipedia editors choosing, verifying, editing sources?
• How are they collaborating to do this?
• What could we apply to Ushahidi/SwiftRiver?

3. Key differences between WP and Ushahidi/SwiftRiver
Wikipedia Ushahidi/SwiftRiver
Editors do not summarize what has
Editors make summarized statements
happened but provide other editorial
about what has happened linked /backed
functions like verifying statements and
up by sources
developing categories
Editors choose which sources to include All sources are generally included
Chosen sources must be what
Wikipedians define as “reliable” which Social media sources are often the primary
often does not include social media source of Ushahidi reports
sources
All statements must be “verified” to be Verified and unverified statements are both
included (usually) reflected

4. Key findings relevant to SwiftRiver
1. Relevance is the primary reason for choosing a source (not accuracy);
2. Verification is a two-step process of reading/watching the cited source and
then finding corroborating sources;
3. Important to authority and transparency is an understanding of who the
authors are, and what the policy/bar for entry is;
4. Talk pages are critical places to alert, discuss and collaborate;
5. There was incentive for collaboration around editing in breaking news stories.

5. 1. Relevance not accuracy
A number of factors are being weighed when deciding to choose a source beyond
the so-called “reliability” or accuracy of a source, including:
• the balance of secondary to primary sources in the article (Wikipedia policy calls for
editors to use a majority of secondary sources),
• whether the article is based on a breaking news story (in which case the majority of
sources might have to be primary, eyewitness sources), or an article on a person (in
which case their official Twitter feed, for example, may be considered a reliable
source on that person);
• whether the source is notable in the context of the article (misinformation can also
be relevant if it is widely reported and significant to the course of events as Judith
Miller’s NYT stories were for the Iraq War).
Thus relevance is more appropriate a factor in deciding whether to use a source
than accuracy. This becomes important when one considers how to filter results in
sources searches.

6. 2. The verification process
1.Verification is a two-step process of a) reading/watching the cited source in
order to check that it is being accurately reflected in the text, and b) finding
sources that corroborate or oppose the statements.
2.For example, the image to the right was used in
the article with the caption that reflected that
these were “police in civilian clothing beating a
protester” until one editor watched the footage
from where the screenshot originated and
asserted that there was nothing in the
commentary from Al Jazeera that backed up this
statement about the identity of those in the image.
3.Design that supports this special activity could enable editors/curators to:
highlight a questionable source/statement --> further investigate by curators
carefully reading/watching the original source + checking whether it is
accurately reflected in the story --> checking whether the information is
corroborated by other sources --> marking it as complete within a certain

7. 3. Who are the authors? What are their policies?
• Without an understanding of who edited a page and what the policies on
verification or source use are, there is no way for readers to accurately gauge
the veracity of the article;
• On Wikipedia, we know that editors are using the principles of
“verifiability, not truth”, “reliable sources” and “neutral point of
view” (NPOV) to curate and edit pages and although we could
know more about individual authorship, users have a general
idea of veracity because of the principles in force. SwiftRiver
and Ushahidi require curators to develop and reflect similar
principles to readers to answer questions such as:
• What are your principles for deciding which information you include/do not
include? What is your POV? How can I check up on the information provided
here? Are all sources going to be included? Only reliable ones? Only accurate
ones? How will rumor/misinformation be represented?

8. 4. Talk pages for collaboration
• Talk pages are critical places for collaboration between editors;
• It is here that they are able to reach consensus, provide space for leadership,
develop specific guidelines regarding style, format, etc for the article, alert
others as to breaking news, ask for help, show what they’ve done, verify
sources, provide corroboration, and provide moral support (amongst others);
• Could SwiftRiver/Ushahidi similarly provide a place for editors to discuss the
shape of the “story” they are telling and/or discuss the the verifiability of a
source?

9. 5. Collaboration around breaking news alerts
1.One of the most popular items offered on talk pages was for breaking news in which an
editor would bring a new item to the page to achieve one or more of the following goals:
a)
a way to alert other editors to recently-released information relevant to the
article (important especially since editors were working in different time zones and
the talk page would be one of the first places a collaborating editor would go when
they woke up or starting editing);
b)
an ask for others to add the source to the article for them if they didn’t have the
time or the access privileges, for example, if they were editing as an anonymous IP
during the periods when the page was semi-protected (some editors saw the
discovery of sources and posting to talk as their single task rather than editing);
c)
a place to “hold” sources while they were waiting for other sources or were
unsure as to the relevance of the article for inclusion and were asking for opinions
of other editors;
d) a way to ask other editors to find references for unsourced or incorrectly
sourced information on the article.

10. For example
Here, an editor brings a new article to the talk page, where s/he picks out the
relevant facts (this was new at the time because it had been unclear up until
then what the Egyptian army’s role was in the 2011 Revolution). S/he follows
it up with a corroborating source and then Lihaas edits the page and marks
the talk page as “done”.

11. Tasks
Tasks generally accorded with the following workflow:

12. design implications
• SwiftRiver could enable collaboration by providing a space for individuals to
bring breaking news to a common page for discussion regarding relevance/
summarizing/verification/corroboration etc.
• In this way, nuggets grow into larger stories that can then be connected with
one another. Rather than seeing verification as a separate process happening
after curatorship which is the case with the Standby Task Force, for example,
verification and curating/editing happen together in small teams of people
who adopt tasks, rather than predefined and continuous roles.

13. 3 key takeaways for Ushahidi/SwiftRiver
1. Use relevance rather than accuracy to filter results;
2. Distinguish between statements and sources/authors;
3. Decide what form/s a “report” will take.

14. 1. Use relevance rather than accuracy to filter results
• We were originally thinking of collaboration in terms of the same people doing
the same work and then for the algorithm to check whether there were
similarities/inconsistencies, but this requires scale and doesn’t accord with
the process by which people actually collaborate;
• People collaborated by checking up on cited sources, summarizing, finding
relevant news, and deciding where the news fitted.

15. For example:
Source 1
Statement Source 2
Source 3

16. For example:
Source 1
Aleppo_Alissa: “Bombs are going off all
around us in central Aleppo”
Statement Source 2
Source 3

17. For example:
Source 1
Aleppo_Alissa: “Bombs are going off all
around us in central Aleppo”
Statement Source 2
Reuters: “The Syrian government just
released a statement saying that they
were initiating a campaign to push
back the “terrorists” in Aleppo.”
Source 3

18. For example:
Source 1
Aleppo_Alissa: “Bombs are going off all
around us in central Aleppo”
Statement Source 2
Reuters: “The Syrian government just
released a statement saying that they
were initiating a campaign to push
back the “terrorists” in Aleppo.”
Source 3
Free_Syria: “Government forces just
bombed the Central Children’s Hospital
in Aleppo.”

19. For example:
Source 1
Aleppo_Alissa: “Bombs are going off all
around us in central Aleppo”
Statement Source 2
Reuters: “The Syrian government just
Witness Syria
released a statement saying that they
“Aleppo is under
were initiating a campaign to push
siege by government
back the “terrorists” in Aleppo.”
forces.”
Source 3
Free_Syria: “Government forces just
bombed the Central Children’s Hospital
in Aleppo.”

20. For example:
Source 1
(!)Aleppo_Alissa: “Bombs are going off all
around us in central Aleppo”
Statement Source 2
Reuters: “The Syrian government just
Witness Syria
released a statement saying that they
“Aleppo is under
were initiating a campaign to push
siege by government
back the “terrorists” in Aleppo.”
forces.”
Source 3
Free_Syria: “Government forces just
bombed the Central Children’s Hospital
in Aleppo.”

21. For example:
(!) Aleppo_Alissa has a history Source 1
of telling untruths. (!)Aleppo_Alissa: “Bombs are going off all
around us in central Aleppo”
Statement Source 2
Reuters: “The Syrian government just
Witness Syria
released a statement saying that they
“Aleppo is under
were initiating a campaign to push
siege by government
back the “terrorists” in Aleppo.”
forces.”
Source 3
Free_Syria: “Government forces just
bombed the Central Children’s Hospital
in Aleppo.”

22. For example:
(!) Aleppo_Alissa has a history Source 1
of telling untruths. (!)Aleppo_Alissa: “Bombs are going off all
around us in central Aleppo”
Statement Source 2
Reuters: “The Syrian government just
Witness Syria
released a statement saying that they
“Aleppo is under
were initiating a campaign to push
siege by government
back the “terrorists” in Aleppo.”
forces.”
Source 3
Free_Syria: “Government forces just
bombed the Central Children’s Hospital
in Aleppo.” (!)

23. For example:
(!) Aleppo_Alissa has a history Source 1
of telling untruths. (!)Aleppo_Alissa: “Bombs are going off all
around us in central Aleppo”
Statement Source 2
Reuters: “The Syrian government just
Witness Syria
released a statement saying that they
“Aleppo is under
were initiating a campaign to push
siege by government
back the “terrorists” in Aleppo.”
forces.”
Source 3
Free_Syria: “Government forces just
bombed the Central Children’s Hospital
in Aleppo.” (!)
Can we really take this as a
reliable source? Need more

24. For example:
(!) Aleppo_Alissa has a history Source 1
of telling untruths. (!)Aleppo_Alissa: “Bombs are going off all
around us in central Aleppo”
Statement Source 2
Reuters: “The Syrian government just
Witness Syria
released a statement saying that they
“Aleppo is under
were initiating a campaign to push
siege by government
back the “terrorists” in Aleppo.”
forces.” (!)
Source 3
Free_Syria: “Government forces just
bombed the Central Children’s Hospital
in Aleppo.” (!)
Can we really take this as a
reliable source? Need more

25. For example:
(!) Aleppo_Alissa has a history Source 1
of telling untruths. (!)Aleppo_Alissa: “Bombs are going off all
around us in central Aleppo”
Statement Source 2
Reuters: “The Syrian government just
Witness Syria
released a statement saying that they
“Aleppo is under
were initiating a campaign to push
siege by government
back the “terrorists” in Aleppo.”
forces.” (!)
Source 3
(!) Sources are unreliable -- it
isn’t clear who is doing the
Free_Syria: “Government forces just
shooting bombed the Central Children’s Hospital
in Aleppo.” (!)
Can we really take this as a
reliable source? Need more

26. Distinguishing between the author and the source/ref
Wikipedia:
Ushahidi:
Author (of statement) | Statement
A history of contribution that is Verified/Under dispute/Unverified
disputed/undisputed OR
reviewed/unreviewed (i.e. new) Unreviewed/Reviewed

27. 3. Decide what form a “report” will take
• Is it just a long list? Or a map? Or a textual summary? Or a live blog-style
report?
• This is important because this will determine what tasks people are
performing, how/whether sources should be linked into clusters, what are the
most important variables/categories for filtering results etc.
• If it’s a map, we might want to focus verification activities on making sure the
location (of the source/author and/or event) is accurate; if it’s a live blog, then
we want to enable users to provide feedback on the sources the editor has
used in the story, as well as to provide the editor with ways to characterise
her/his sources.

28. the end.