A presentation from Museums and the Web 2009. Timothy Baldwin, University of Melbourne, Australia Lejoe Thomas Kuriakose, University of Melbourne, Australia In this project, we explored the deployment of RFID-based technologies to observe visitors’ behaviour in a museum to record what exhibits they visit and when. This data has tremendous potential to enhance museum visits through personalizing information via dynamic interfaces, or profiling the visitor to make recommendations for future activity inside or outside the museum. The outcomes of the project demonstrate the viability of (passive) RFID technologies for museum visitor tracking and provide empirical validation of the near-human tracking accuracy of the system in two different environments. Session: Location-Aware Services [Technology]

1. Cheap, Accurate RFID Tracking of
Museum Visitors for Personalized
Content Delivery
Timothy Baldwin and Lejoe Thomas Kuriakose

2. 1 RFID Tracking of Museum Visitors for Personalized Content Delivery
Kubadji Project Overview
• The Kubadji project is the combination of:
user modelling
language technology
in settings that: (a) are informationally rich; (b) are
occupied by users with different needs/backgrounds;
(c) have expert domain knowledge of user behaviour;
and (d) are associated with contexts and user
behaviours that are informative in understanding users.
MW 2009: 18 April, 2009

3. 2 RFID Tracking of Museum Visitors for Personalized Content Delivery
to . . . ?
–W
hat
’s t
ore his
is relate
em ?
ll m
Te
– does th
so far!?
story me
– The sa
the
of
ore
– How
M
––W
her
en
ext
?
Image: http://www.flickr.com/photos/paullew/2639507829/ MW 2009: 18 April, 2009

4. 3 RFID Tracking of Museum Visitors for Personalized Content Delivery
• Initial focus on:
virtual simulation of the museum environment
modelling based on historical data and incorporating
static (web documents, Wikipedia, curator data,
physical layout, ...) and dynamic (observation
of visitor behaviour, relevance feedback, free-text
interaction, ...) information
• Collaboration between University of Melbourne, Monash
University and Melbourne Museum
MW 2009: 18 April, 2009

5. 4 RFID Tracking of Museum Visitors for Personalized Content Delivery
– Path prediction
– Exhibit recommendation
– Prediction of exhibit relatedness
– Personalised summaries
MW 2009: 18 April, 2009

6. 4 RFID Tracking of Museum Visitors for Personalized Content Delivery
– Visitor tracking
– Path prediction
– Exhibit recommendation
– Prediction of exhibit relatedness
– Personalised summaries
MW 2009: 18 April, 2009

7. 4 RFID Tracking of Museum Visitors for Personalized Content Delivery
– Visitor tracking
– Path prediction
– Exhibit recommendation
– Prediction of exhibit relatedness
– Personalised summaries
MW 2009: 18 April, 2009

8. 5 RFID Tracking of Museum Visitors for Personalized Content Delivery
– Visitor tracking
– Path prediction
– Exhibit recommendation
– Prediction of exhibit relatedness
– Personalised summaries
MW 2009: 18 April, 2009

9. 6 RFID Tracking of Museum Visitors for Personalized Content Delivery
Museum Visitor Tracking
• Basic problem: track the
path of each museum
visitor
with no intrusion
at minimal cost
as accurately as possible
• Underlying questions: how accurate do we need to
be for “useful” personalisation? how do we measure
accuracy?
MW 2009: 18 April, 2009

10. 7 RFID Tracking of Museum Visitors for Personalized Content Delivery
Contributions of this Research
• Development of an evaluation framework for “discrete”
tracking, e.g. for museum purposes
• Comparison of tracking accuracy under different
modalities, and finding that proximity-based tracking
is a reasonable proxy for full-on gaze-based tracking
MW 2009: 18 April, 2009

11. 8 RFID Tracking of Museum Visitors for Personalized Content Delivery
Mainstream Tracking Options
• Passive RFID
• Active RFID
• Bluetooth
• Wireless LAN
• (Indoor) GPS
MW 2009: 18 April, 2009

12. 8 RFID Tracking of Museum Visitors for Personalized Content Delivery
Mainstream Tracking Options
• Passive RFID
• Active RFID
• Bluetooth
• Wireless LAN
• (Indoor) GPS
MW 2009: 18 April, 2009

13. 9 RFID Tracking of Museum Visitors for Personalized Content Delivery
Sensing vs. “Bookmarking” vs. Tracking
• Sensing = automatically sensing that a visitor is in the
proximity of a given exhibit (and acting accordingly)
• Bookmarking = visitor “swipes” a given exhibit (e.g. to
get more information post-visit)
• Tracking = tracing a given visitor’s path through the
museum space
MW 2009: 18 April, 2009

14. 10 RFID Tracking of Museum Visitors for Personalized Content Delivery
Continuous vs. Discrete Tracking
• Ultimately, we are interested in which exhibits a given
visitor has visited, when, and for how long; as such,
discrete tracking is (probably) sufficient for our purposes
Continuous: Discrete:
vs.
MW 2009: 18 April, 2009

15. 11 RFID Tracking of Museum Visitors for Personalized Content Delivery
Spin-off Benefits of Tracking
• “Heat mapping” of visitor pathways through the
museum
• Analysis of exhibit “clusters”
• Profiling of different visitor types
• Targeted marketing
MW 2009: 18 April, 2009

16. 12 RFID Tracking of Museum Visitors for Personalized Content Delivery
The Gory Details
MW 2009: 18 April, 2009

17. 13 RFID Tracking of Museum Visitors for Personalized Content Delivery
Hardware Set-up
• Visitors provided with passive RFID tags, housed in a
badge holder worn around the neck
• Exhibits/exhibit areas instrumented with RFID
antennae, connected to individual RFID readers
• RFID readers connected to a central server via USB
connections
• Tracker output takes the form of a sequence of
exhibiti, visitorj , timestart, timeend tuples
MW 2009: 18 April, 2009

18. 14 RFID Tracking of Museum Visitors for Personalized Content Delivery
Man vs. Machine
• The basic question we are asking is:
how accurate is automatic tracking?
• To answer this question, we compare RFID-based
tracking to human tracking over a series of visitors to
two separate mini-galleries, with the humans alternating
between the two “tracking modalities” of:
1. physical proximity-based exhibit “engagement”
2. visitor gaze-based exhibit “engagement”
MW 2009: 18 April, 2009

19. 15 RFID Tracking of Museum Visitors for Personalized Content Delivery
Tracking Experiment
• Two separate mini-galleries, each with three exhibits:
• Two human trackers (same tracking modality)
• 15 visitors in total, one at a time
MW 2009: 18 April, 2009

20. 16 RFID Tracking of Museum Visitors for Personalized Content Delivery
GeckoTracker
• The human trackers used the GeckoTracker
tracking software to record where a given visitor went
• (Same software used for large-scale full-visit tracking
experiment through Melbourne Museum)
MW 2009: 18 April, 2009

21. 17 RFID Tracking of Museum Visitors for Personalized Content Delivery
Evaluation
• Basic methodology, given an evaluation metric:
evaluate the “agreement” between the two human
trackers [gold-standard]
evaluate the “agreement” between the RFID output
and the intersection of the two human trackers
compare the two
MW 2009: 18 April, 2009

22. 18 RFID Tracking of Museum Visitors for Personalized Content Delivery
Evaluation Metric 1: Edit Distance
• Intuition: how many “tweaks” to the sequence of
exhibits returned by method A need to be made to
transform it into the sequence for method B? [ignore
time spent at exhibits]
d( E1, E3, E1 , E2, E1 ) = 2
d( E1, E3, E1 , E3, E1 ) = 1
• Use edit distance to calculate the distance between a
given pair of sequences
MW 2009: 18 April, 2009

23. 19 RFID Tracking of Museum Visitors for Personalized Content Delivery
Evaluation Metric 2: Paired t-test
• Intuition: what is the statistical similarity in the timed
sequence of exhibits returned by method A relative to
that for method B?
t( E1(0 : 10, 0 : 25), ... , E2(0 : 25, 0 : 37), ... ) = 0.7
t( E1(0 : 10, 0 : 25), ... , E1(0 : 10, 0 : 27), ... ) = 0.1
• Normalise the times, and calculate using the two-tailed
paired t-test
MW 2009: 18 April, 2009

24. 20 RFID Tracking of Museum Visitors for Personalized Content Delivery
Results: Human vs. Machine (Gallery 1)
Tracker Edit Paired
Modality
Pairing Distance t-test
Human vs. Human 0.50 0.41
Distance
Human vs. machine 0.75 0.71
Human vs. Human 0.75 0.43
Gaze
Human vs. machine 1.00 0.76
MW 2009: 18 April, 2009

25. 21 RFID Tracking of Museum Visitors for Personalized Content Delivery
Results: Human vs. Machine (Gallery 2)
Tracker Edit Paired
Modality
Pairing Distance t-test
Human vs. Human 1.25 0.17
Distance
Human vs. machine 2.25 0.36
Human vs. Human 1.25 0.30
Gaze
Human vs. machine 1.50 0.37
MW 2009: 18 April, 2009

26. 22 RFID Tracking of Museum Visitors for Personalized Content Delivery
Numbers, Numbers Everywhere ...
• RFID-based tracker always performs worse than human
performance
• Relative human vs. machine agreement is almost
unchanged between the distance and gaze modalities
• Open-plan galleries are more of a challenge for both
humans and machines
MW 2009: 18 April, 2009

27. 23 RFID Tracking of Museum Visitors for Personalized Content Delivery
Error Analysis
• False negatives the primary cause of errors for the RFID-
based tracker
• Signal dropout also contributed
MW 2009: 18 April, 2009

28. 24 RFID Tracking of Museum Visitors for Personalized Content Delivery
Summary
• Promising results for RFID-based tracking
• Proposal of a series of evaluation metrics for tracking
evaluation
• Solid platform to do all the
sexy research ...
MW 2009: 18 April, 2009