1. MindCanvas: Reimagining online
research
Rashmi Sinha
Uzanto
www.uzanto.com
www.themindcanvas.com
1

2. Duct-tape research!
(in the trenches - Information Architecture
Redesign of eBay.com)
2

3. Background to IA redesign
• eBay.com has grown in organic fashion –
inconsistent, fractured user experience
– Separate design / usability efforts for parts of eBay
• first effort to focus on user experience for entire site
• Basic eBay facts
– World’s largest online marketplace (79 million in US)
– Types of users
• Buyers & sellers
• New & experienced
• Individuals & corporations
• Different countries & cultures
3

4. Challenges
• New IA would need to
– Serve diverse user group
• varying goals / mental models
– Be future-oriented, allow for change & growth
– Changes needed to be evolutionary not revolutionary
• accommodate users familiar with current site
• Senior management at eBay needed proof
– Is there need for such drastic restructuring?
• Research to be completed in 6 weeks!
4

5. Goals of research
• Research has multiple goals
– Need successful design, not just good design
• Serves user
• Satisfies business goals
• Is accepted by management
• 3-stage research method
• Stages: Exploration  Understanding  Verification
• Stages complement & build upon each other
• Mix of qualitative & quantitative methods
• Qualitative: rich, open-ended understanding of users / domain
• Quantitative: recommendations grounded in data (help make case
to management)
5

6. 3-stage user research
Preliminary IA
3 stages
Scope of content & ebay.com
functionality
Stage 1: Explore
Stakeholders Interviews, free-listing Buy
Sub cat1
Sell
SubCat1
Community
SubCat1
xx SubCat2
SubCat2
el
xx SubCat3
SubCat3
ev es
SubCat4
xx
-l
op gori
SubCat4
Stage 2: Understand
SubCat4
xx
T e
SubCat5
t
Subcat2
Open card-sorting ca
SubCat6 SubCat5
Subcat3
xx
Subcat4
xxx
xxx
xxx
xxx
Stage 3: Verify / xxx
Refine Verify xx
User thinking / categorization xx
Large-sample closed
categorization card-sorting
xxx
6

7. Stage 1: Explore domain with Freelisting
• Goals
– Explore domain in open-ended fashion, map scope.
– Generate list of current & “Horizon Tasks” (planned for
future)
• Method
– Interviews & Freelisting
– With eBay users, stakeholders & designers
• Freelisting tasks (eBay is about tasks!)
– What are all the tasks users can do on eBay.com
• List an item, Pay for it
– What else will users be able to do on eBay.com 2-5
years that they can’t do today?
• Results: List of 100 representative tasks (25 Horizon)
7

8. Stage 2: Explore user categorizations
• Goals:
– Gain insight into user thinking about eBay
– Identify top-level categories for ALL site
content and functionality
• Method: Interviews & Open card
sorting
– List of 100 tasks (including 20 Horizon
Tasks).
– 35 participants (including sellers & buyers;
new and advanced users).
8

9. Identify top levels of hierarchy
• Hierarchical Cluster analysis to
generate aggregate user
categorizations. Dendrogram- User group !
Task N
Task Z
Task B
• Separate analysis for 4 user groups
Task X
Task P
Task Ge
Task K
– Identify inconsistencies in Task A
Task M
categorizations Task L
Task D
Task S
– Reconcile to create one scheme. Task R
Task F
Task O
Task U
Task Y
Save items
• Result: Preliminary IA Fn eBay internatLrn
Translate listing
View item desc. Dendogram
Fn proFn services
– 5 top-level categories accounting for all Browse items
Fn types of items
site content and functionality. Fn specific item
Fn items nearby
Fn BIN items
Search & browse
Fn items by seller
Fn accessories
Contact buyer
Contact seller
Get buying tools
EsTrk car payment 9
Offer for unsold item
Negotiate price
Reverse auction

10. Stage 3: Verify & refine scheme
• Goals:
– Verify 5-category scheme covers all of eBay.
• What tasks/concepts don’t fit? Where do users expect this info?
– Input from larger, more diverse set of users.
– Establish method for future verification of IA.
• and to generalize scheme to international eBays.
• Method: closed card sorting
– Large sample (~ 1,000) representing all eBay user types.
– Study conducted using online survey.
• Participants categorized tasks into one of categories (or selected
Other if it didn’t belong)
• Result
– Site-map showing the structure for all main parts of the
eBay.com
10

11. Sell - Before
11

12. Sell Hub - After
12

13. eBay redesigned successfully!
• Three-stage methodology Preliminary IA
– Understand business context
site.com
– Delineate user categorizations Top Top Top Top
Category1 Category 2 Category 3 Category 4
– Create IA blueprint Sub cat1
xx
SubCat1
SubCat2
SubCat1
SubCat1
SubCat2
SubCat2
xx SubCat3
SubCat3
SubCat3
xx SubCat4
SubCat4 xx
SubCat4
• Deliverable: site-map with structure
xx
SubCat5 xx
Subcat2
SubCat5 xx
SubCat6
Subcat3
xxx
for main parts of eBay.com.
xx
Subcat4 xx
xxx
xxx xx
xxx
xxx
– map used as blueprint of IA redesign xx
xxx
xx
(took 2 years).
SubCat4
xx
SubCat5
xxx
– Generalized to international eBays. SubCat6
xxx
xx
xx
xx
– Positive business implications (ROI)!
xx
xx
SubCat7
– Users did not protest!
13

14. Duct-tape research!
There has to be a better way - fast, cheap
(relatively) & online…
14

15. Why we built MindCanvas
• Qualitative research is great, takes a lot of time
• Online tools suck, get used anyway (users not engaged)
• Business stakeholders respond to quantitative analysis &
large sample sizes
• Deliverables do not cater to designers
• Research findings remain locked up with analyst
15

16. Inspiration
• Luis Von Ahn’s ESP Game
– Rely on games to label the web
• PopCap Games
– Quick, short, engaging games
• Visualization research
– bringing statistics to designers
16

17. 17

18. MindCanvas interpretation
back to the future!
18

19. What is MindCanvas?
Remote research Interactive
methods Visualizations
Statistical Analysis &
Data Mining
loosely coupled, use as qualitative
or quantitative
19

20. Game-like elicitation methods
• Surveys do not engage people. Games are fun!
– People engage in complex tasks willingly
• Do game-like methods work better?
– MindCanvas has game-like elements (its not a game)
• Interactive, visual, screen build up, fun
• For both qualitative & quantitative
– Remote moderated for qualitative
– Remote unmoderated for quantitative
20

21. Rich interactive visualizations
• Large datasets need statistical analysis & data mining
• Visualizations-to-go
• Self-contained files - embedded into PowerPoint, emailed, shared
• Interactive visualizations with slides & knobs!
• Easy to share with team
• Visualization posters
– Designers want printouts to hang on walls, and scribble on
21

22. MindCanvas research focuses on the mind
• Inspiration from
cognitive anthropology, Preferences
Language
psychology, market
research
Categori
• Semi-structured zations
methods
• Quantitative
aggregation &
visualization
22

23. Who we built for: designer persona
The busy designer / researcher who wants to
look at patterns visually
Does not want to
spend time
learning statistics
Needs
convenient
methods to do
research
Want to directly
interact with users,
does not have time
23

24. Who we built for: participant persona
The not-tech savvy person who occasionally
takes surveys
“It is very easy to
complete the task.
“This was the In fact, I had fun
most fun I have doing this.”
had in a while!”
“I think there are
too many cards. But
it was kind of fun.”
24

25. Where MindCanvas could help
• Product Development: Finding the right
feature-set for the right users (or personas)
• Information Architecture: Understanding how
people think for information design.
• Early visual design validation: Understanding
what people think & feel about a design when its
just an image.
25

26. MindCanvas in product development
• Initial understanding of domain
– What’s important to people? What words do they
use?
• Method: Freelisting
• Deliverable: ListMap
• Prioritizing features / outcomes users care about
– Method: Divide-the-dollar
– Deliverable:
• WeightMap
• ClusterBrowser
26

27. MindCanvas in Information Architecture
• How do people think about a domain?
– Method: OpenSort
– Deliverable
• Dendogram
• SimilarityBrowser
• VocabularyBrowser1, 2
• Is your information architecture effective? Is one scheme
more effective than another?
– Method: TreeSort
– Deliverable:
• ClusterBrowser
27

28. 28

29. Questions?
rashmi@uzanto.com
MindCanvas team is…
Delhi
Kapil Mohan
Kamalmeet Singh
Debasish Biswas
Sunil Gupta
Amit Ranjan
Mountain View
Jon Boutelle
Rashmi Sinha
29

30. Methods & Deliverables
• Current Methods
– OpenSort: Dendogram, VocabBrowser, SimilarityBrowser
– TreeSort: ClusterBrowser, SimilarityBrowser
– Divide-the-Dollar: WeightMap, ClusterBrowser
– Free Listing: ListMap
– Utility Questions: MultipleChoice, EssayType
• Upcoming Methods: MenuTrace, MindMontage,
Likert Sort
• Upcoming Designables: OnionMap
30

31. Collaborative Research
Process
• High quality research needs expertise
– research design + statistics
• Service should enable high quality
research
– Structured research process
• Templates for study design
• Help with data collection
31

32. What is MindCanvas
• An innovative research platform for high
quality research
– Game-like elicitation methods
– Interactive visualizations (designables-to-go)
– Research expertise
32