Increasing globalization and technological development has led companies and people across the globe to connect through the global internet community. However, people with different cultural backgrounds may perceive the same information in different ways. One of the hurdles to use websites efficiently is the indifferent structures of information on website, and their relation with the characteristics of intended users and the context of use for the websites. The purpose of this dissertation is to assist Human-Computer Interaction (HCI) practitioners and researchers with better design of website structures for user groups with different cultural backgrounds. This dissertation looks into issues related to website user experience (UX) and focus on how the structuring of information is seen from local users’ perspectives. In particular, it attempts to look into the alignment between websites’ information architecture (IA) and users’ views of website information structure, by applying a cross-cultural and context of use perspective on the UX of websites in three countries: Pakistan, Malaysia, and Denmark. The researcher investigates to what degree users’ cognitive styles and contexts of use are aligned with local websites’ information architecture, and how this (lack of) alignment shapes the resulting UX. This dissertation adopts a mixed-method approach to studying the UX association with users’ cognitive styles, contexts of use, and website structures. In this regard a critical literature review of the existing HCI research on website UX in Asia was performed. Four UX studies were conducted on local university websites in Denmark and Pakistan, and local e-commerce websites in Pakistan and Malaysia. Empirical data was collected through questionnaires, card-based brainstorming, card sorting, information retrieval tasks, and retrospective interviews with 108 university students across four studies. Users’ cognitive styles were captured through users´ information classification (card sorting) activities with website contents. These data were analyzed through edit distance measures and hierarchical clustering through dendrograms. The context of use was captured by collecting data on domains of websites, interviews about the use of languages, background questionnaires, and information retrieval tasks. These data were analyzed through quantitative analysis and coding of retrospective interviews. The structures of the websites were captured through analysis of website navigation and labels employed by each website. The literature review revealed that HCI research into website UX in Asia is growing, in particular examining academic, e-commerce websites, and tourism websites. The literature review indicated that the use of cultural theories and frameworks for studying website UX is limited. The results of the four UX studies indicated differences in cognitive style and context of use across the three locations, Denmark, Pakistan, and

1. By
Ather Nawaz
Department of IT Management, Copenhagen Business School (CBS)
18 Dec. 2013
Website User Experience:
A cross-cultural study of the relation between users’ cognitive style,
context of use, and the information architecture of local websites

2. Agenda
• Background
• Research questions
• Theoretical focus
• Philosophy of science
• Empirical studies
• Results
• Implications
Outline

3. Background
• Access to the information through websites
• Globalization practices vs. Localization practices
- Cultural markers
• Users access information in different ways
- Localize software through cultural-user interface (Yeo 1996)
Background

4. Overview of
webpages

5. Department of IT Management
• Cognitive style refers to the different ways in which
people think about and remember information.
• Information architecture (IA) refers to the structure of
the website or wireframes of websites.
• The context of use refers to the characteristics of
intended users, the tasks users perform, tools users use
and the environment in which users use the system.
• User Experience (UX) refers to a person’s perceptions
and responses that result from the use or anticipated use
of a product, system or service
Definitions
Definitions

6. To what degree are users’ cognitive styles and
contexts of use aligned with the information
architecture of a local website, and how does
the (lack of) alignment shape the UX?
Research Question

7. Research sub-questions
1. Which issues, related to websites and types of websites, are
prominent in website UX research done in Asia?
2. How do local users’ cognitive styles relate to their context of use
when using local websites?
3. How do local users’ cognitive styles and context of use relate to
the information architecture of local websites?
4. How do different card-sorting analyses influence the design of
the information architecture of a website?
Research sub-questions

8. Focus of main question and
sub research question 1
Focus of main question and
sub research question 2
Focus of main question and
sub research question 3
Focus of main question and
sub research question 4
Article
Website UX in Asia ‘from
Within’: An overview of a
decade of literature
Information classification on
university websites: A two-
country card sort study.
Information classification
and acquisition on websites:
users´ interaction on e-
commerce websites in two
Asian Countries
A comparison of card-sorting
analysis methods
Purpose
Which issues related to
websites and domain of
websites are important in
website usability research
done in Asia?
How users´ cognitive styles
relate to their context of use
when using local websites
How do local users´’
cognitive styles and context
of use relates to information
structure of local websites
Explaining the analysis and
choice of analysis for card
sorting methods
Results
reported in
Article 1 Article 2 Article 3 Article 4
Literature study
Study 1 and study 2 Study 3 and study 4 Card sorting analysis
Overview of research

9. Concept / Literature Map
INF Research Seminar
9
1st stage
literature
review

10. INF Research Seminar
10
Researchers
Choong 1997
Clemmensen 2009-2011
Hertzum 2010
Isa et a., 2007-2009
Kamppuri 2011
Vatrapu 2007
Yeo 1996-2002
HCI and Culture

11. Critical situationIdeal situation
Theoretical focus

12. Surface
Visual design
Skeleton
• Interaction Design
• Navigaiton Design
• Information Design
Structure
• Interaction Design
• Information Architecture
Scope
• Functional Space
• Content Requirementcture
Strategy
• Site Objecteives
Scope in IA

13. Adapted from Rauterberg (2000)
Realistic perspective conjectural realist
Ontological standpoint

14. Epistemological consideration
Adapted from Van de Ven (2007)
Epistemological consideration

15. Stages of UX studies
• Comparative research design
• Mix-methods approach
Techniques and tools
– Qualitative assessment of labels
– Qualitative assessment through ATLAS.ti
– W-cardsort analyser
– Similarity Matrix/ Dendrogram
– Edit-distance
Research Design

16. Data collection model

17. Studies design
Participants, Material, and tasks

18. Studies design
University websites in Pakistan and Denmark

19. E-commerce websites in Pakistan and Malaysia

20. • Card sorting
• Card-based brainstorming
• Information retrieval
• Usability evaluation
• Retrospective interview
Activities in studies

21. Research question
To what degree are users’ cognitive styles and contexts of
use aligned with the information architecture of a local
website, and how does the (lack of) alignment shape the
UX?
Users cognitive style and context of use
(Malaysia and Pakistan) is loosely aligned with
information architecture of local websites.
Results

22. Literature
Review
Study 1
PK1
Study 2
DK2
Study 3
PK3
Study 4
MY4
Card-sort
Analysis
Article 1 X
Article 2 X X
Article 3 X X
Article 4 X X
Main research question
Rq1 Rq2 Rq3 Rq4
Results

23. Literature study Study 1 and Study 2 Study 3 and Study 4 Card sorting analysis
Type
Literature review UX studies with 17+14
participants
UX studies with 33+41
participants
Evaluation of methods
Data
60 Articles between 2001-
2011
- ACM Digital library
- Scopus
- Web of Science (SSCI)
- Science Direct
- Brainstorming
- Card sorting
- Task exploration
- The retrospective
interviews
- Brainstorming
- Card sorting
- Task exploration
- The retrospective
interviews
Card sort data
38 Malaysian participants of
study 4
Tools: W Card Sort Analyzer
(tool for analyzing edit-
distance)
Optimal workshop (tool for
discovering test models)
Results /
- Limited use of
Hofstede’s cultural
model in UX studies
- Increase in UX studies in
last decade in Asia
- UX studies of website
information structure are
limited
- Information depth varies
- Variation in task
completion time
- Variation in taxonomical
classification and
thematical classification
- Not much difference in
users´ agreement in card
sorting activities
- Some differences in
language preferences and
language use
The choice of analysis
technique for card sorting
studies can impact on the
resulting information
structure for a website
Results

24. Literature study

25. Literature study Study 1 and Study 2 Study 3 and Study 4 Card sorting analysis
Type
Literature review UX studies with 17+14
participants
UX studies with 33+41
participants
Evaluation of methods
Data
60 Articles between 2001-
2011
- ACM Digital library
- Scopus
- Web of Science (SSCI)
- Science Direct
- Brainstorming
- Card sorting
- Task exploration
- The retrospective
interviews
- Brainstorming
- Card sorting
- Task exploration
- The retrospective
interviews
Card sort data
38 Malaysian participants of
study 4
Tools: W Card Sort Analyzer
(tool for analyzing edit-
distance)
Optimal workshop (tool for
discovering test models)
Results /
- Limited use of
Hofstede’s cultural
model in UX studies
- Increase in UX studies in
last decade in Asia
- UX studies of website
information structure are
limited
- Information depth varies
- Variation in task
completion time
- Variation in taxonomical
classification and
thematical classification
- Not much difference in
users´ agreement in card
sorting activities
- Some differences in
language preferences and
language use
The choice of analysis
technique for card sorting
studies can impact on the
resulting information
structure for a website
Results

26. Study 1 – Lahore, Pakistan

27. Card-based brainstorming data
Card-sorting data
Results study 1-2

28. Information Retrieval
Depth-success
Depth- time
Results study 1-2

29. UX assessment - study 1 in Pakistan
UX assessment (1.Very poor ... 5.Very good) (M + SD)
Ease of Use 2.4 + 0.51
Attractiveness of design/appearance 2.4 + 0.63
Ease of finding information services 2.5 + 0.52
Information that was clear and easy to understand 2.1 + 0.62
Accurate and up to date information 2.5 + 0.65
Usefulness of site search 2.2 + 0.58
UX assessment - study 2 in Denmark
UX assessment (1.Very poor ... 5.Very good) (M + SD)
Ease of use 3.2 + 1.12
Attractiveness of design/appearance 3.4 + 1.28
Ease of finding information services 2.7 + 1.14
Information that was clear and easy to understand 3.7 + 1.14
Accurate and up to date information 3.4 + 1.16
Usefulness of site search 3.4 + 1.01
Results study 1-2

30. Literature study Study 1 and Study 2 Study 3 and Study 4 Card sorting analysis
Type
Literature review UX studies with 17+14
participants
UX studies with 33+41
participants
Evaluation of methods
Data
60 Articles between 2001-
2011
- ACM Digital library
- Scopus
- Web of Science (SSCI)
- Science Direct
- Brainstorming
- Card sorting
- Task exploration
- The retrospective
interviews
- Brainstorming
- Card sorting
- Task exploration
- The retrospective
interviews
Card sort data
38 Malaysian participants of
study 4
Tools: W Card Sort Analyzer
(tool for analyzing edit-
distance)
Optimal workshop (tool for
discovering test models)
Results /
- Limited use of
Hofstede’s cultural
model in UX studies
- Increase in UX studies in
last decade in Asia
- UX studies of website
information structure are
limited
- Information depth varies
- Variation in task
completion time
- Variation in taxonomical
classification and
thematical classification
- Not much difference in
users´ agreement in card
sorting activities
- Some differences in
language preferences and
language use
The choice of analysis
technique for card sorting
studies can impact on the
resulting information
structure for a website
Results

31. Study 3 - Pakistan

32. Study 4, Malaysia

33. Card-sorting activity
• Participants made on average 6.31 first level
categories (SD + 2.80) and 1.59 (SD + 2.69) second
level categories.
• Less than half of the participants (43%) made
second level categories.
• Participants agreed 60% or more on card placement
for most of the cards (35 of 41, or 85%)
Results – study 3, Pakistan

34. Cluster analysis
Participants
agreed 60% or
more on card
placement for
most of the
cards (85% of
the cards)
Results – study 3, Pakistan

35. Language fluency
Browsing webpages in language(es)
Results – study 3, Pakistan

36. Card-sorting activity
• Participants made on average 6.03 first level
categories (SD + 2.40) and 4.76 (SD + 5.24) second
level categories.
• More than half of the participants (55%) made
second level categories.
• Participants agreed 60% or more on card placement
for most of the cards (33of 37, or 89%)
Results – Study 4, Malaysia

37. Cluster analysis
Participants
agreed 60% or
more on card
placement for
most of the
cards (89% of
the cards)
Results – Study 4, Malaysia

38. Language fluency Results – Study 4, Malaysia
Local language
websites
English language
websitesDo not Open it 2 (5%) 0 (0%)
Less than 1 minute 6 (15%) 3 (7.9%)
1-10 minutes 17 (44.8%) 7 (18.4%)
10-30 minutes or
more
13 (34%) 28 (73%)
Browsing webpages in language(es)

39. Literature study Study 1 and Study 2 Study 3 and Study 4 Card sorting analysis
Type
Literature review UX studies with 17+14
participants
UX studies with 33+41
participants
Evaluation of methods
Data
60 Articles between 2001-
2011
- ACM Digital library
- Scopus
- Web of Science (SSCI)
- Science Direct
- Brainstorming
- Card sorting
- Task exploration
- The retrospective
interviews
- Brainstorming
- Card sorting
- Task exploration
- The retrospective
interviews
Card sort data
38 Malaysian participants of
study 4
Tools: W Card Sort Analyzer
(tool for analyzing edit-
distance)
Optimal workshop (tool for
discovering test models)
Results /
- Limited use of
Hofstede’s cultural
model in UX studies
- Increase in UX studies in
last decade in Asia
- UX studies of website
information structure are
limited
- Information depth varies
- Variation in task
completion time
- Variation in taxonomical
classification and
thematical classification
- Not much difference in
users´ agreement in card
sorting activities
- Some differences in
language preferences and
language use
The choice of analysis
technique for card sorting
studies can impact on the
resulting information
structure for a website
Results

40. Best merger method Actual merger method
Card sorting Analysis

41. in Relation to my concept definition
• Cognitive styles and context of use is closely aligned,
and it varies largely in Pakistan and Malaysia
• The loose alignment between context of use and
cognitive style in Pakistan and Malaysia results in
lower UX
• The IA of websites should be localized due to loose
alignment between cognitive style and context of use
Finding

42. in Relation to literature in HCI and culture
• UX should be re-interpreted from local perspective (
Smith et al., 2007, Clemmensen 2009)
• Users cultural background effects on the usability of
system ( Yeo, 1998, Choong 1997)
• UX may take different meaning depending upon the
users cultural background (Hertzum, 2010)
• The IA of websites should be localized due to loose
alignment between cognitive style and context of use (
Isa et al., 2007-2009)
Finding

43. INF Research Seminar
43
• Genre of websites
• Quality measure of websites
• Limited use of e-commerce websites
• Amounts of produces and information
• Difficulty in group comparison
• Limitations in using Edit distance
Limitations

44. Thank you

45. INF Research Seminar
Drafts
Nawaz, A. & Clemmensen, T. Issue with comparative local cases for website structures [AIS Transactions
on Human-Computer Interaction]
Nawaz, A. A Comparison of Card-sorting Analysis Methods. Submitted to Asia- Pacific CHI (APCHI2012)
Nawaz, A., & Clemmensen, T. Website usability in Asia. Submitted to special issue on local and
indigenous HCI @ IJHCI
Nawaz, A. (2012). Is It Culture or Is It Learning? A Conceptual Framework of Information Literacy and
Flow in relation to Websites’ Information Architecture. Paper presented at 4th ACM International
Conference on Intercultural Collaboration, Bengaluru, India, March 21-23.
Nawaz, A. (2011). Evaluation of Information Classification on Websites and Impact of Culture: A Cross
Country Comparison of Information Classification. Paper presented at 13th IFIP TC 13 International
Conference ( INTERACT), Lisbon, Portugal, September 5-9.
Nawaz, A., Clemmensen, T., Hertzum, M. (2011). Information Classification on University Websites: A
Cross-Country Card Sort Study. Paper presented at 34th IRIS2011, Turku, Finland, August 16-19.
Nawaz, A. (2010). Information architecture of websites and the impact of ‘language and culture’ on the
design and structure websites. Paper presented at 3rd ACM International Conference on Intercultural
Collaboration, Copenhagen, Denmark, August 19-20.
Nawaz, A. and T. Clemmensen (2010). Using card sorting to explore collectivism in students’
approaches on a university website. Paper presented at 11th Danish HCI Research Symposium,
Copenhagen, Denmark, November 23
Short Paper
Short Paper
Full Paper
Full Paper
Short Paper
Full Paper
Short Paper
Full Paper

46. for researchers
- Focus of research should be on different variety of domains of
websites
- Website UX should be re-interpreted according to to users
cognitive style, and context of use
- Website IA do vary on the basis of users background
for practitioners
- The information structure of the websites should be localized to
accommodate users cognitive style and context of use
- Methodological issues in card-sorting studies
Implication