Here is my presentation of MULHER at AVI 2012.

1. Interactive Exploration of
Music Listening Histories
Ricardo Dias, Manuel J. Fonseca,
Daniel Gonçalves

2. Context & Motivation

3. Proliferation of lifelogging services

4. Tracking Listening Habits

5. Music Listening Histories
User 1
Artist 1, Song 1, Timestamp
Artist 2, Song 3, Timestamp
Artist 1, Song 2, Timestamp Time
Artist 3, Song 5, Timestamp
Artist 4, Song 4, Timestamp
…
User 2
…

6. Profile Generation

7. Recommendation

8. Pattern detection in user habits
zachstern@flickr

9. Good at recording data…

10. … and allow direct access to it!

11. But Visualization is also important!
ottonassar@flickr

12. Related work

13. StreamGraph [Byron and Wattenber 2008]

14. Tangle, Strings and Knots [Baur2009]

15. LastHistory [Baur2011]

16. Fans effort to visualize data
Scrobbling Timeline Arc diagrams
LastGraph

17. Problems

18. Main issues
o More concerned about design and aesthetics
o Static visualizations and only overviews of
listening histories
o Lack of interactive browsing and filtering
o Scalability issues, regarding the number of
songs to represent

19. Main issues
o More concerned about design and aesthetics
o Static visualizations and only overviews of
listening histories
o Lack of interactive browsing and filtering
o Scalability issues, regarding the number of
songs to represent

20. Main issues
o More concerned about design and aesthetics
o Static visualizations and only overviews of
listening histories
o Lack of interactive browsing and filtering
o Scalability issues, regarding the number of
songs to represent

21. Main issues
o More concerned about design and aesthetics
o Static visualizations and only overviews of
listening histories
o Lack of interactive browsing and filtering
o Scalability issues, regarding the number of
songs to represent

22. Our solution

23. Design principles
Rationale

24. Exploration
thegiantvermin@flickr

25. Overview
[Dias and Fonseca2010]

26. Interactivity
Ultrastart

27. Details on demand
m4tik@flickr

28. Inference
mterraza@sxc

29. Analysis

30. Music Listening History Explorer
MULHER

31. MULHER

32. Main Visualization: Stacked dot

33. Main Visualization: Timeline

34. Main Visualization: Background

35. Filtering
o Genre
o Artist name Single selection
o Free Text
Multiple Selection
o Time
Combining Filters

36. Statistics
General Statistics Daily and Hourly Visualizations
With Context Without Context

37. Brushing & Highlighting

38. Implementation
Regular Web Application Backend Server
HTML, CSS, Javascript
Protovis Java backend
JSON to represent data

39. Demo

40. Evaluation

41. Two Experiments
swamibu@flickr

42. First experiment
Common listening history
Evaluate:
– UI usability
– Users satisfaction
and experience

43. Second Experiment: Case Studies
o Users exploring their own listening histories
o Listening Patterns Detection

44. Experiments Details
Test Procedure Users
Tasks Listening Logs
= ≠

45. Setup Procedure
o 45 min for each test
o User’s personal computers
mytudut@flickr

46. Test Session Procedure
1. Quick introduction
2. Application description
3. Practice Time
4. Tasks execution
5. Satisfaction survey & Informal Interview

47. Tasks
9 tasks
2 Categories
1. Explore and browse the listening history
2. Pattern detection and trends on music listening
habits

48. Exploring Tasks: examples
Indicate the most played song of the artist X
over the last three months
Describe the trend on the previous identified
song

49. Exploring Tasks: examples
Indicate the most played song of the artist X
over the last three months
Describe the trend on the previous identified
song

50. Pattern Detection Tasks: examples
Describe and try to justify the listening changes
that occurred over the last three months
Describe the listening habits on the selected
period

51. Pattern Detection Tasks: examples
Describe and try to justify the listening changes
that occurred over the last three months
Describe the listening habits on the selected
period

52. First Experiment

53. Common Log
o Listening Histories from one of the authors
o 5.000 records (11/2009 to 07/2011)
alaasafei@sxc

54. Participants
8 2
N = 10
Ages between 20-50 years
Listen to music almost every day

55. Occupations
4 CS students
2undergraduate CS students
2 SW engineers
1 journalist
1 HS student

56. Statistical Evaluation
o Effectiveness
o Error rate
o Satisfaction
iamwahid@sxc

57. First Experience
Results

58. Effectiveness
100%

59. Overall Task Success Rate
97% 3%
Successfull Unsuccessfull

60. Problem of Task 3
# Users
10
6
0
T1 T2 T3 T4 T5 T6 T7 T8 T9

61. Overall Tasks Easiness
1 8 1
0 0
Very Difficult Normal Easy Very easy
difficult

62. User Experience Satisfaction
Fantastic Rewarding Stimulant Easy Flexible
9
8
7
6
5
4
3
2
1
Horrible Frustrating Boring Hard Rigid

63. Second Experiment

64. Users’ logs
o Personal listening histories
o From 3.000 to 30.000 records
alaasafei@sxc

65. Participants: Second Experience
#N = 4
Ages between 20-40 years
All listen to music every day
Active Last.fm account

66. Occupations
2 CS Assistant Teachers
1 Senior Software Engineer
1 CS PhD Student

67. Results

68. Identical Statistical Results
lumaxart@flickr

69. Pattern Inference – Users interviews
whitebeard@sxc

70. Life Past Events
User 1
“Here I was working on a scientific paper,
because I was listening only classical music, and
I like to hear that kind of music when I'm
writing, But then I skipped listening to music,
because I had some project discussions, and no
time to listen to music”

71. Profile Inference
User 2
"Well, do you know why this part of the
visualization contains mostly recent music, even
though I just prefer to listen old music?“

72. Profile Inference
User 2
"Well, do you know why this part of the
visualization contains mostly recent music, even
though I just prefer to listen old music?“
 one of his favorite artists just released a
new album after years of absence

73. Hidden Time Habits
User 4
"I did not realized that I was listening too much
music in late night, but now that I think of this, I
usually listen to more rhythmic music at that
time to stay awake a little longer, mostly when
I'm working"

74. Listening Trends
User 3
"Looks like that through a regular day I keep
changing the genre of music I listen to. I start
with something stronger in the morning and
then end the day with more relaxing songs!"

75. Discussion

76. Timeline
o Timeline-based mechanism proved to be a
major asset:
– Main browsing and filtering technique
– Effectiveness and flexibility validated by
experimental results

77. Context Information
o Considered to be an important aspect of the
solution
– Info about most played elements
– Acts as a visual clue to start the exploration

78. “Age of Songs”
o Can effectively convey information about the
listening habits
– Different profiles by direct color inspection

79. Knowledge analysis and inference
o Performed by combining insights from the
different techniques
o Possible main based on time, absence of
music listening and context information

80. Conclusions

81. Conclusions
Novel solution for exploring and filtering
listening histories
1. Combines a timeline-based visualization with a
set of synchronized-views to perform direct
exploration
2. Introduces a new feature: the “age” of songs
3. Allows listening pattern detection, only based on
time and some textual metadata

82. Future Work
o Data mining on listening histories data
– Discover new hidden listening patterns
– New pattern examples:
• Users that always seek the tops
• Others that enjoy mostly female voices
• Listen to classic music in the morning but rhythmic
music in the afternoon
o Map the new patterns to this and other
visualizations

83. Thank you
Questions?
ricardo.dias@ist.utl.pt
web.ist.utl.pt/~ricardo.dias