How Your Facebook Update Could Make You a Victim of Crime : On the Privacy Implications of Multimedia Retrieval

How Your Facebook Update Could
Make You a Victim of Crime
: On the Privacy Implications of Multimedia
Retrieval

International Computer Science Institute
Jaeyoung Choi
jaeyoung@icsi.berkeley.edu

UC Berkeley KGSA Talk & Seminar - Oct 16, 2012
Monday, October 29, 12

Phillip Markoff


Phillip Markoff
• Boston University medical
student


Phillip Markoff
• Boston University medical
student
• ‘Craigslist Killer’
- He found his victims through the ads
on Craigslist


Lonely Heart
Killers


Lonely Heart
Killers
1940s
Victims posted personal ads on
newspaper


• I don’t post personal ads to Craigslist or
newspaper... so I’m safe?


• I don’t post personal ads to Craigslist or
newspaper... so I’m safe?

• Well.. do you use Facebook? Twitter?


Facebook Friend or Foe?

http://www.sileo.com/facebook-status-update-leads-to-robbery/


Facebook Friend or Foe?

http://www.sileo.com/facebook-status-update-leads-to-robbery/

Facebook ‘friend’ robbed $11,000 off Keri McMullen’s house.

http://pleaserobme.com/

Gathers check-in infos (FourSquare,
Gowalla, Loopt) from your Twitter timeline

• I don’t use ‘check-in’ feature... so I’m safe?


http://icanstalku.com/

Mayhemic Labs:
“Are you aware that Tweets are geo-tagged?”, and can
tell you EXACTLY where you are?
http://icanstalku.com/

What is Geo-Tag?

Source: Wikipedia 9


Geo-Tagging : Beneﬁts

Allows easier clustering of photo and video
series as well as additional services.
10


• OK.. can you do real harm with Geo-tags??


Doing Real Harm with Geo-Tag

G. Friedland and R. Sommer: "Cybercasing the Joint: On the Privacy
Implications of Geotagging", Proceedings of the Fifth USENIX Workshop
on Hot Topics in Security (HotSec 10), Washington, D.C, August 2010. 12



• Cybercasing: Using online (location-based) data and
services to enable real-world attacks.




• Cybercasing: Using online (location-based) data and
services to enable real-world attacks.

• Three Case Studies:



Case Study 1: Twitter

13



•Pictures in Tweets can be geo-tagged

13



•From a technically-savvy celebrity we
found:

13



found:
–Home location (several pics)

13



found:
–Where the kids go to school

13



found:
–The place where he/she walks the dog

13



found:
–The place where he/she walks the dog
–“Secret” office

13


Celebs unaware of Geo-
Tagging

Source: ABC News 14


Celebs unaware of Geotagging

15


Google Maps shows Address...

16


Case Study 2: Craigslist

“For Sale” section of Bay Area Craigslist.com:

In 4 days:

• 68729 pictures total
• 1.3% geo-tagged

17


People are Unaware of Geo-
Tagging

18


Tagging

•Many ads with geo-location otherwise
anonymized

18


Tagging

anonymized
•Sometimes selling high-valued goods, e.g.
cars, diamonds

18


Tagging

anonymized
cars, diamonds
•Sometimes “call Sunday after 6pm”

18


Tagging

anonymized
cars, diamonds
•Sometimes “call Sunday after 6pm”
•Multiple photos allow interpolation of
coordinates for higher accuracy

18


Craigslist: Real Example

19


Geo-Tagging Resolution

iPhone 3G picture Google Street View
gure 1: 1: Photo a bike taken with an an iPhone 3G and corresponding Google Street View image based onon the stor
Figure Photo of of a bike taken with iPhone 3G and a a corresponding Google Street View image based the stored
ordinates. The accuracy of thethe camera location (marked) front of the garage is about +/−1 m. Many classified advertise
coordinates. The accuracy of camera location (marked) in in front of the garage is about +/−1 m. Many classified advert
Measured accuracy: +/- 1m
es contain photos describing objects forfor sale taken home that automatically contain geo-tagging.
sites contain photos describing objects sale taken at at home that automatically contain geo-tagging.

n would make it easy to to increase the confidence in the in in the second step identifying all other videos fr
tion would make it easy increase the confidence in the the second step identifying all other videos from
20
resultsOctober 29, 12
further.
ults further.
Monday,
corresponding users. 106 ofof these turned out to hav
corresponding users. 106 these turned out to have

What about Inference?

Valuable

Owner


Case Study 3: YouTube

22



Recall:

22



Recall:
• Once data is published, the Internet keeps
it (in potentially many copies).

22



Recall:
• APIs are easy to use and allow quick
retrieval of large amounts of data

22



Recall:
• APIs are easy to use and allow quick
retrieval of large amounts of data

Can we ﬁnd people on vacation in YouTube?

22


Cybercasing on YouTube
Experiment: Cybercasing using the YouTube
API (240 lines in Python)
Location
Radius Query
Keywords

Results

Users? Query
YouTube

Results
Time-Frame
Distance

Filter

Cybercasing
23
Candidates


Input parameters

24



Input parameters

Location: 37.869885,-122.270539
Radius: 100km
Keywords: kids
Distance: 1000km
Time-frame: this_week
24


Cybercasing on
YouTube

Output

25


Cybercasing on
YouTube

Output
Initial videos: 1000 (max_res)

25


Cybercasing on
YouTube

Output
➡User hull: ~50k videos

25


Cybercasing on
YouTube

Output
➡Vacation hits: 106

25


Cybercasing on
YouTube

Output
➡Vacation hits: 106
➡Cybercasing targets: >12

25


• What if I turn off geo-tagging feature?


Ongoing Work:

http://mmle.icsi.berkeley.edu
27


Multimodal Location Estimation

We infer location of a media (video/photo/
document) based on visual, audio, and tags:
•Use geo-tagged data as training data
•Allows faster search, inference, and
intelligence gathering even without GPS.

28


http://www.multimediaeval.org/

Mediaeval Placing Task
- An annual benchmark which provides standardized
datasets to the community of researchers for the
evaluation of new algorithms

Overview of Our Approach
{berkeley,
sathergate,
{berkeley,
haas}
campanile}

Edge:
Correlated
loca7ons

(e.g.
common
tag,
visual,

acous7c
feature) Node:
Geoloca7on
of

video
k p(xj |{tk })
p(xi |{ti }) j

p(xi , xj |{tk }
i {tk })
j
{campanile} {campanile,
haas}
Edge
Poten,al:
Strength
of
an
edge,
(e.g.
posterior

distribu5on
of
loca5ons
given
common
tags)
J. Choi, G. Friedland, V. Ekambaram, K. Ramchandran: "Multimodal Location Estimation of Consumer Media: 30
Dealing with Sparse Training Data," in Proceedings of IEEE ICME 2012, Melbourne, Australia, July 2012.

Results: MediaEval

J. Choi, G. Friedland, V. Ekambaram, K. Ramchandran: "The 2012 ICSI/Berkeley
Video Location Estimation System," in Proceedings of MediaEval 2012, Pisa, Italy,
October 2012.

YouTube Cybercasing
Revisited
Old Experiment No Geotags
Initial Videos 1000 (max) 107
User Hull ~50k ~2000
Potential Hits 106 112
Actual Targets >12 >12

Even without Geo-Tags, cybercasing on
YouTube video is readily possible
G. Friedland, and J. Choi, “Semantic Computing and Privacy: a Case Study Using
Inferred Geo-Location.” in Int. J. Semantic Computing, Vol. 5, Nr. 1 (2011) , p. 32
79-93.

• But... is this really about Geo-Tags?


• But... is this really about Geo-Tags?
• No, it’s about the privacy implications of
multimedia retrieval in general.


Example

34


Example

Idea: Can one link videos across acounts?

34


Example

Idea: Can one link videos across acounts?
(e.g. YouTube linked to Facebook vs
anonymized dating site)

34


Persona Linking using Internet
Videos

Speaker Recognition System
- Given a voice sample, it tells whether it’s from Howard, Gerald, Jae, etc..

City Identification System
- Modified from the traditional Speaker Recognition System
- Given an audio sample, it tells whether it’s from Seoul, San Francisco, Berlin, etc..


Experiment
- 4869 test videos from Flickr
- 500 users in training,
493 hits, 2289 non-hits in test

-Audio characteristics :
“wild” (70% heavy noise, 50%
speech )

H. Lei, J. Choi, A. Janin, and G. Friedland: “Persona Linking: Matching Uploaders of
Videos Accross Accounts”, at IEEE International Conference on Acoustic, Speech, and
Signal Processing (ICASSP), Prague, May 2011


User ID on Flickr videos

Even with a preliminary
experiment setting, the system
performs much better than
random.
(26.3% < 50% Equal Error Rate)


Linkage Attacks on SNS

• There are methods to link a user’s accounts only by
accessing publicly available data.

• Your anonymity across different social networking
services accounts can be compromised.


Linkage Attacks
•  Multiple accounts on social networks
•  Same or different purposes : reviewing ...
•  What about the aggregate trace they leave
?

2

Works and slides credit: Oana Goga (http://www-npa.lip6.fr/~goga/)

De-anonymization Model

?
im ilar
o ws
h

Targeted account
(YELP users are id’d)
Candidate list

7


Where a user is posting

- Twitter locations

- Yelp locations

5


When a user is posting

6


Performance of matching with
location profile
1
Yelp − Twitter 35% of Flickr Flickr and 60%
40% of and Yelp
Flickr − Twitter
accountsYelp accounts can
of can be matched
0.8 to a set of 250 Twitter set of
be matched to a

•x
accounts Twitter accounts
1000
CDF users

0.6

0.4

0.2

0
10 100 250 1000 10000
Rank of the ground truth user
11


Problems Reformulated


• Many applications are encouraging sharing data
heavily and users follow


• Multimedia isn’t only a lot of data, it’s also a lot
of information


of information
• Users and engineers often unaware of (hidden)
retrieval possibilities of shared (multimedia) data


of information
• Users and engineers often unaware of (hidden)
retrieval possibilities of shared (multimedia) data
• Local anonymization and privacy policies
ineffective against cross-site inference


Status Quo


Status Quo

• People will continue to want social
networks and location-based services


Status Quo

• Industry and research will continue to
improve retrieval techniques


Status Quo

• Industry and research will continue to
improve retrieval techniques
• Government will continue to do forensics
and intelligence gathering


What Now?

• Research might help to:
• quantify and qualify risk factors
• visualize and offer choices in UIs
• identify privacy breaking information


Conclusion
• We should continue to explore multimedia
retrieval
• At the same time we should:
• research methods to help mitigate risks and
offer choice
• develop privacy policies and APIs that take
into account multimedia retrieval
• educate users and engineers on privacy issues


Take Home Message


Take Home Message

• Be aware of the risks of revealing your
personal life online


Take Home Message

• Be aware of the risks of revealing your
personal life online
• Think TWICE before you post something
on Facebook/Twitter/...


Thank You!
Questions?
Work together with:
Robin Sommer, Oana Goga, Venkatesan Ekambaram, Kannan Ramchandran,
Luke Gottlieb, Howard Lei, Adam Janin, Oriol Vinyals, Trevor Darrell, Gerald
Friedland
and many others..

49


How Your Facebook Update Could Make You a Victim of Crime : On the Privacy Implications of Multimedia Retrieval

Recommandé

Recommandé

Contenu connexe

Similaire à How Your Facebook Update Could Make You a Victim of Crime : On the Privacy Implications of Multimedia Retrieval

Similaire à How Your Facebook Update Could Make You a Victim of Crime : On the Privacy Implications of Multimedia Retrieval (16)

How Your Facebook Update Could Make You a Victim of Crime : On the Privacy Implications of Multimedia Retrieval