Institut de Seguretat Pública de Catalunya (fire brigades, police, civil protection). Get the book: http://bigcrisisdata.org/

1. BIG CRISIS DATA
Towards Participatory Data Mining
CARLOS CASTILLO
Director of Research for Data Science at Eurecat
Escola d'estiu: La seguretat en l'era de la informació.
Institut de Seguretat Pública de Catalunya — 6/July/2016

2. Topic of this short talk: CRISIS INFORMATICS
Patrick Meier
QCRI →
Muhammad Imran
QCRI
Irina Temnikova
QCRI
Aditi Gupta
IIIT Delhi →
P. K. Kumaraguru
IIIT Delhi
Alexandra Olteanu
EPFL →
Ji Lucas
QCRI
Ferda Ofli
QCRI
Hemant Purohit
Wright State George Mason→
Book (2016) from
Cambridge University PressWork done (2012-2015) by Crisis Computing team at QCRI
Sarah Vieweg
QCRI
Fernando Diaz
Microsoft

3. BIGCRISISDATA.ORG
Key messages
● Social media is a huge force during crises, emergencies, and disasters
– You cannot control the wind, but you can harness its power
●
Social media contains highly relevant information surrounded by tons of noise
– Mining is an industrial process, or at the very least one that requires special tools
●
Digital volunteers will help you if you let them
– Crisis mapping is the first of many paradigms for participatory data mining

4. Social Media is a huge
force during crises,
emergencies, and
disasters

5. 5
Two months ago (March 22, 2016)
Police asks the public:
1) to use social media, not phone;
2) to reduce video/audio streaming;
3) to avoid sharing real-time information
about police actions
Attacks in the airport and a metro station
in Brussels kill 35 and injure 340

6. 6
2,800-words Wikipedia article
In the first 8 hours after the attacks ...
Reddit post with 17,000 comments
700+ YouTube videos per hour
Facebook pages and Safety Check
Tweets and photos

7. 7 BIGCRISISDATA.ORG
A Common Pattern
Disaster or mass-convergence event
People have increased communication needs
People are familiar with social media
Internet is not bullet-proof but fairly resilient
Emergency agencies encourage social media usage
Intensive usage of social media by the public for
emergency communications

8. 8 BIGCRISISDATA.ORG
ExampleS from #SMEM papers
“OMG! The fire seems out of control: It’s running down the hills!”
Bush fire near Marseilles, France, in 2009 [Longueville et al. 2009]
“Red River at East Grand Forks is 48.70 feet, +20.7 feet of flood stage... #flood09”
Red River Valley floods in 2009 [Starbird et al. 2010]
“My moms backyard in Hatteras. That dock is usually about 3 feet above water [photo]”
Hurricane Sandy 2013 [Leavitt and Clark 2014]
“Sirens going off now!! Take cover...be safe!”
Moore Tornado 2013 [Blanford et al. 2014].
“There is shooting at Utøya, my little sister is there and just called home!”
2011 attacks in Norway [Perng et al. 2013]

9. 9 BIGCRISISDATA.ORG
Problem 1: Scale

10. 10 BIGCRISISDATA.ORG
Problem 2: Noise
●
The Internet is strong
because it is chaotic
●
As they say ...
DEAL WITH IT

11. Social media contains
relevant information
surrounded by noise

12. 12 BIGCRISISDATA.ORG
The tension: Familiar tools vs new uses
●
“In times of disaster and crisis, people tend to gravitate
towards the systems and networks that are most familiar
to them. They shun the specialized sites set up by aid
workers for the comfort of already-established systems
they know well from prior experience.”
●
“Familiar sites for sharing photos of weddings become
locations for sharing breaking news … Participants are using
social web tools in ways the designers of such systems had
neither anticipated nor considered. As a result, the vast
majority of today's technologies are woefully ill-equipped
for crisis situations”

13. 13 BIGCRISISDATA.ORG
mining is not this

14. 14 BIGCRISISDATA.ORG
mining is more like this

15. 15 BIGCRISISDATA.ORG
Data Mining is not this

16. 16 BIGCRISISDATA.ORG
Data Mining is more like this

17. 17 BIGCRISISDATA.ORG
Don't do it entirely by hand
● Many tools and systems, some of them free, are available for:
– Organizing your communications with the public through social media
– Monitoring the response of people to social media activity
– Monitoring social media activity
●
You probably don't need “big systems,” but most likely a collection of specialized tools
– Each tool does one job well
– Tools may overlap a bit, sometimes a lot
– If a tool ceases to be relevant, stop using it and find a new one
●
Big tightly-integrated systems bring some comfort but you pay a lot and loose flexibility

18. 18 BIGCRISISDATA.ORG
Data miner reflex: to classify and cluster
Caution &
Advice
Information
Sources
Damage &
Casualties
Donations
Gov
Eyewitness
Media
NGO
Outsider
...
...

19. 19 BIGCRISISDATA.ORG
A study of Twitter on 26 crises
Results from Olteanu et al. CSCW 2015. Data available at http://crisislex.org/

20. 20 BIGCRISISDATA.ORG
information sources
Results from Olteanu et al. CSCW 2015. Data available at http://crisislex.org/
-3m

21. 21 BIGCRISISDATA.ORG
Temporal progression
Peak
12 hr 24 hr 36 hr 48 hr ... several days
Caution and
advice
Sympathy and
support
Affected
individuals
Infrastructure
and utilities
Other specific
information
Donations and
volunteering
Results from Olteanu et al. CSCW 2015. Data available at http://crisislex.org/

22. 22
Information Extraction + donations matching
...
Classified
tweets
@TheNGO looking for blood donors at the
Riverside Stadium
@APerson do you know where can I donate
blood near Middlesbrough?
See Purohit et al. 2013 for automatic donations matching.

23. Digital volunteers
will help you, if you
let them

24. BIGCRISISDATA.ORG
Physical and digital convergence
● Disasters involve the convergence of people, resources, and information.
– Some of the people, some of the resources, and some of the information are actually
helpful in the disaster response [Fritz and Mathewson 1975]
●
Social media and mobile phone cameras evaporate traditional scene control [Crowe 2012],
yellow tape does not work so well online
– Note: rescue personnel, including firefighters, police officers, and others who
physically converge on the scene have responsibility and legal codes to respect. As the
public also enters the scene (physically or virtually), they must adhere to ethical
standards they may be unfamiliar with. Technology can be used to weaken or to
strengthen those standards.

25. 25Facebook: Visualizing Crisis Relief in Nepal (September 2015)..
The geography of an event 2015 Nepal earthquake

26. 26Patrick Meier – http://irevolution.net/
“What can speed humanitarian
response to tsunami-ravaged
coasts? Expose human rights
atrocities? Launch helicopters to
rescue earthquake victims?
Outwit corrupt regimes?
A map.”

27. 27 BIGCRISISDATA.ORG
Community-powered crisis mapping

28. 28 BIGCRISISDATA.ORG
Crowdsourced Mapping of wildlife in Namibia
See Ofli et al. 2016 for details.

29. 29 BIGCRISISDATA.ORG
Automatic mapping
Floods in Germany
De Albuquerque et al. 2015
Dengue in Brazil
Gomide et al. 2015
Earthquakes in Italy
Cresci et al. 2014

30. 30 BIGCRISISDATA.ORG
hybrid mapping: AIDR + MICROMAPPERS
Manual processing:
crowdsourcing
Automatic processing:
machine learning
See Imran et al. 2014 for details on AIDR. Find out more at http://aidr.qcri.org/

31. 31See Imran et al. 2014 for details on AIDR. Find out more at http://aidr.qcri.org/

32. 32 BIGCRISISDATA.ORG
Content Credibility
● Internet content is considered as less truthful or valid than other contents
– Reality is more nuanced, otherwise nobody would use Wikipedia or Google
– Are emergency managers used to work with perfect information?
●
Learn to deal with imperfect information, e.g., Admiralty code:
A Reliable
B Usually reliable
C Fairly reliable
D Not usually reliable
E Unreliable
F Cannot be judged
1 Confirmed
2 Probably true
3 Possibly true
4 Doubtfully true
5 Improbable
6 Cannot be judged

33. 33 BIGCRISISDATA.ORG
There are no easy answers, but tools can help a bit
http://twitdigest.iiitd.edu.in/TweetCred/

34. 34 BIGCRISISDATA.ORG
Crowdsourced tools can also help
One can create tools that
encourage a behavior, e.g.,
we don't want people to
upvote/downvote
content, but instead to
provide evidence.
https://veri.ly/

35. 35 BIGCRISISDATA.ORG
The future: Real-Time Crowdsourced Mining
● Mapping disaster-affected
areas using UAVs
● Is crowdsourced stream
mining possible?
See Patrick Meier's blog post from Nov. 2015 for details.

36. 36Visit the UAViators community for more information on video clickers.

37. 37 BIGCRISISDATA.ORG
To re-cap
● Social media is a huge force during crises, emergencies, and disasters
– You cannot control the wind, but you can harness its power
●
Social media contains highly relevant information surrounded by tons of noise
– Mining is an industrial process, or at the very least one that requires special tools
● Digital volunteers will help you if you let them
– Crisis mapping is the first of many paradigms for participatory data mining

38. Thank YOU!
Patrick Meier
QCRI →
Muhammad Imran
QCRI
Irina Temnikova
QCRI
Aditi Gupta
IIIT Delhi →
P. K. Kumaraguru
IIIT Delhi
Alexandra Olteanu
EPFL →
Ji Lucas
QCRI
Ferda Ofli
QCRI
Hemant Purohit
Wright State George Mason→
Sarah Vieweg
QCRI
Fernando Diaz
Microsoft
carlos.castillo@eurecat.org
BIGCRISISDATA.ORG

39. 39

40. 40