This was prepared for a presentation of http://swiftapp.org at InSTEDD. It gives an overview of the latest designs (which are changing quickly) and a backgrounder on how we got to this point.
Data Journalism Training @ Southern Metropolis Daily, Guangdong, China
Swift Update May 6
1. SWIFT
FOSS data triage platform
Crowdsourced Crisis Data Media Analytics
(Africa) (Middle East)
2. May 6, 2009 Team Update
Where we are today?
Reeling from an awesome conference.
Enjoying the good company of InSTEDD.
International coworkers in person for the first time.
Swift is going live after months of collaborative design work.
Numerous deployments are scheduled in very near term.
Currently iterating “live” for votereport.in *gulp :)
9. How might we imagine a better
crisis reporting tool?
By designing a better crisis listening tool.
10. why
quot;At present, early warning units within the UN ... use manual
labor to collect relevant information from online sources.
Most units employ full-time staff for this, often
meaning that 80% of an analyst’s time is actually used to
collect pertinent articles and reports, leaving only 20% of
the time for actual analysis, interpretation and policy
recommendations. We can do better. Analysts ought
to be spending 80% of their time analyzing.quot;
Patrick Meier, “Crimson Hexagon: Early Warning 2.0?” February 17, 2009
http://earlywarning.wordpress.com/2009/02/17/crimson-hexagon-early-warning-20
11. We need new listening tools.
Ability to report eyewitness stories
Ability to act on them
40000
30000
20000
10000
0
1910 1940 1980 2010
19. What can we build when we focus
on the user interface needs of citizen editors?
How does this changes our idea of an “analyst”?
Can data entry actually make an impact?
Could it actually be ... enjoyable to use?
20. What if we assume that citizen editors deserve the best?
21. What if we treat their data with the same respect?
24. Output: The Crisis API
The resulting crisis database is open, with distributed storage
and standards-based portability, based on:
ICAL / FOAF / Dublin Core / Etc.
via
RSS / CSV / JSON / SQL / Etc.
and all of it is handled by Freebase.com
25. Swift the method has evolved into a platform:
1. data gathering engine (aggregator)
2. data structuring tool (wiki)
3. most importantly, an API for crisis data
26. Swift is an aggregator
with entity extraction
By “roping together” relevant feeds, then parsing their
content, we can get a rich database of people, places
and organizations in real time.
We are working with
Freebase.com and Calais.
Swift is mostly a Rails app,
Twitter Vote Report.
30. Swift is designed for
Improving information findability in a crisis
Making it easier to find things that you didn't know you were looking for
Better understanding media from other parts of the world
Making urgent data more sharable (structured, published and accessible)
Making it more obvious what information is missing about an crisis
Promoting the work of eyewitnesses with prepared crisis editors
Expanding the grassroots reporting network
Preserving information across crises
44. From the Swift perspective, if you want to find
the new opportunities for change ...
Focus on speed and people.
We can work together faster than ever before.
45. Human-curated data can happen in realtime
with the help of entity recognition and reconciliation.
Really, they just help us work faster.
54. GEEK OUT A SEC
Every incoming is parsed into an object with
attributes:
1. URI.body (the text of the url)
2. URI.rating (anyone can rate through a web UI)
3. URI.submitters (anyone who linked to it)
4. URI.history (every revision preserved)
5. URI.tags (added by humans and machines)
61. Weather Related Disasters
Urban and Rural Fire
Earthquake
Economic Crisis
Homelessness
War
Possibilities with speed: Internal Displacement & Refugee
Environmental Crisis
Aviation Disasters
Highway Accidents
What to do in a ___ based on my location? Volcano
Report a ____ location Missing Person
Report a ___ accident. Structure Collapse
___ detection and reporting. Railroad Disasters
Health Disasters
“There was a ____. Are you ok?” alerts.
Power Outage
Neighborhood-level ____ warnings.
Explosions
Floods
AIDS
Famine
Landslide
Stampede
Foot and Mouth
Avalanche
Fatal Accident
Tsunami Disaster
Bird Flu
62. Swift’s realtime strengthens the Ushahidi
alert cycle
Subscribe to disaster keyword based
alerts about an emergency
eg: “san francisco, earthquake”
64. See Also
Crimson Hexagon state of the art NLP.
VRA's GeoMonitor, a natural language parser that reads the headlines of Reuters and AFP news
wires and codes to state quot;who did what, to who, where and when?quot; Rated quot;virtually identicalquot; to
human event summarizing.
JRC's European Media Monitor, which can parse thousands of different news sources but faces
limitations since analysts still need to read each article to understand the nature of the terrorist
event. http://emm.jrc.it/NewsBrief/clusteredition/en/latest.html
Tabari text parsing engine, quot;event data coder that has been used in at least five NSF-sponsored
projects and produced data used in a number of refereed articles in political science.quot;
FORECITE Forecasting of Crises and Instability Using Text-Based Events, developed by the US
Center for Army Analysis (CAA)
UN's HEWSweb A relatively new early warning system
Biowarn Textual analysis with infectious disease focus
Cewarn
FAST Comprehensive system by Swisspeace
65. A minimal implementation: an analogy
We’ll put a card table at the public library and
work on reports with highlighters, put
everything into a card catalog and leave it in
the library.