Part 1: Website quality evaluation 7Loci method

WEE-NET
Web Engineering Network
of Excellence
Summer School in La Plata
January 29 February 9 2007
Web Sites Quality Evaluation
Projects with the 7Loci method
Luisa Mich
etourism.economia.unitn.it

Schema
Goal of the talk: presents real websites
quality evaluation projects
Yet another method! The 7 Loci meta-model
and method
WEE-NET
of Excellence
Summer School in La Plata, Argentina
Luisa Mich – The challenges of the NOW Economy
2
model A first project, to illustrate the method
Comparative evaluation projects
A multi-steps evaluation project

Context
Web applications play a strategic role for
companies, organizations, communities:
some of them would not even exist without
Internet and their Web site
WEE-NET
of Excellence
3
The new (now) economy (society) imposes
new challenges for the Web developers and
Web engineers have to become aware of
their responsibility

Strategic role of Web sites
Given the high level of competition existing
on the Web, simply being present on-line
does not guarantee that a site’s sponsors
will reach their objectives for and through
the site. Not only, but often the creation of
WEE-NET
of Excellence
4
the Web site coincides with the birth of the
company whose business model does not
foresee a physical location nor an
articulated organisational structure
(consider for example, virtual banks)

Web engineering in the practice
“Web Engineering uses scientific,
engineering, and management principles
and systematic approaches to successfully
develop, deploy, and maintain high-quality
Web systems and applications (Murugesan
WEE-NET
of Excellence
5
et al., 1999).
Do Web developers adopt a Web
engineering approach?
Not always!

Web sites quality?
Examples from the Tourism sector:
Tour operators, hotel chains: OK
Tourist boards: some
WEE-NET
of Excellence
6
Small hotels: )-:

Our approach
7Loci as a meta-model and method
Quality: definition
Web sites: characteristics
WEE-NET
of Excellence
7
Assumptions
The ciceronian loci
The process: defining a “right weight”
evaluation model

Definition of quality
Quality is “the totality of
characteristics of an entity that
bear on its ability to satisfy
stated and implied needs” (ISO
WEE-NET
of Excellence
8
8402)

Entity - Web site
Embedded information technology:
constitutes part of the products and
services
Strategic role
WEE-NET
of Excellence
9
Components: SW, HW, interfaces,
networks, graphic design, skills,…

Presence of several diverse
components in a Web site
They require a multidimensional, systemic
approach and a multidisciplinary
development team: a successful Web site
will be the fruit not only of ICT experts, but
also of experts in business, marketing,
WEE-NET
of Excellence
10
creative design, and representatives from
the field or domain itself (professionals
from the tourism field, for example, for the
site of a tourist organization)

Stated and implied needs-
Requirements
Many actors/stakeholders
Different goals
Variety of factors
WEE-NET
of Excellence
11
Quality trade-offs

Variety of stakeholders
The design and use of a Web site involves a
wide spectrum of actors or “stakeholders”
both internal and external to the company
Web sites have a potentially wider and
more differentiated target consumer base:
WEE-NET
of Excellence
12
users can play different roles (employee, client,
consumer, company – as in B2B applications –
etc, developer);
users can have very different profiles,
characterized by different languages, cultures,
education, computer proficiency, etc.

Stakeholders
WEE-NET
of Excellence
13

To satisfy - Evaluation
to evaluate ¹ to measure
qualitative quantitative
subjective objective
WEE-NET
of Excellence
14
interpretation reproduceable
TRASPARENCY

“judged” sports vs. “measured” sports
diving, football,
figure skating, cyclism,
WEE-NET
of Excellence
15
gymnastics,… tennis,…
judges, scores,
parameters

There are many models!!!
Efficiency - automatic evaluation
Detail level - factors, metrics
Objective - e.g., competition
WEE-NET
of Excellence
16
Focus - 1 aspect or point of
view (e.g., SW, HCI)
Kind of site - domain, B2C, B2B
advantages and disadvantages

Models for evaluating software quality, e.g.,
ISO 9126, ISO 14598, IEEE Std 1061, …
Approaches focused on the usability
related to the area of HCI
WEE-NET
of Excellence
17
Models introduced specifically for evaluation
or design of web sites
see http://etourism.economia.unitn.it/bibliography_items/index/1

A very simple model
WEE-NET
of Excellence
18
La Repubblica (Il Venerdì)

A more complex model
(Olsina L., Università de La Pampa, Argentina)
WEE-NET
of Excellence
19
Luis Olsina, Daniela Godoy, Guillermo Lafuente, Gustavo Rossi: Assessing the quality of
academic websites: A case study. The New Review of Hypermedia and Multimedia 5: 81-103
(1999)

Quality management
We need a method that allows:
a multi-stakeholders approach
completeness
WEE-NET
of Excellence
20
resources/project management

Web sites quality evaluation
method
A method needs:
an “adequate” web sites model
an evaluation process: activities, input,
output, goals, techniques, roles, tasks,
WEE-NET
of Excellence
21
resources

The 7Loci Method
Models for the evaluation of quality of Web
sites are based on characteristics described
using diverse criteria
We have introduced the 7Loci meta-model,
WEE-NET
of Excellence
22
originally called 2QCV3Q, with the initials
coming from the 7 loci of Ciceronian
rhetoric on which the model was built

Requirements for the quality model
It must take into account the highly
interactive/communicative nature
of a web site, and
be domain independent, robust, user-
WEE-NET
of Excellence
23
user-friendly,
general purpose, complete
(robust), scalable
The 7 loci of Ciceronian rhetoric can be
interpreted as the 7 fundamental
dimensions of a Web site

Dimensions of the 7Loci meta-model
QVIS? (Who) Identity
QVID? (What?) Content
CVR? (Why)? Services
VBI? (Where?) Location
WEE-NET
of Excellence
24
QVANDO? (When?) Maintenance
QVOMODO? (How?) Usability
QVIBUS AVXILIIS? Feasibility
(With what means and devices?)

The 7 loci
Identity regards the image that the
organization projects and therefore all
elements that come together in defining the
identity of the “owner” of the site
WEE-NET
of Excellence
25
Content and Services refer, respectively, to
the information and the services available
for users on the site, and how this
information fulfils the stakeholders’ goals

The 7 loci, Cont'd
Location concerns the visibility of a site; it
also refers to the site’s ability to provide
users with a space where they can interact
with each other and with the organization
WEE-NET
of Excellence
26
Maintenance comprises all activities that
guarantee proper functioning and
operability of the site, in addition to
updating the technology and adapting the
site to new requirements

The 7 loci, Cont'd
Usability, the most thoroughly studied
dimension concerns any aspects which
enable a relatively undemanding use of the
site in terms of time or cognitive input
WEE-NET
of Excellence
27
Feasibility includes all aspects strictly
relating to project management, including
the limits on time and budget, the skills
required and the choice of architecture

The evaluation process
Site mission
Quality
Model
Evaluation
plan
Evaluation
results
WEE-NET
of Excellence
28
Evaluation purpose
Users requirements
Set-up Design Realization

Evaluation set-up
Requirements analysis to determine:
purpose of the evaluation
type and domain of the site (or sites) to
be evaluated
WEE-NET
of Excellence
29
development phase of the site
objectives of the owner of the site
users’ profiles

Evaluation Design
Instantiation of the quality model taken into
account of the requirements
To define the survey modalities: techniques
(inspective vs. user-based) and tools,
WEE-NET
of Excellence
30
number and role (competency) of the
evaluators, qualitative vs. qualitative
assessments, etc.

Realization of the evaluation
One or more visits to the site/s
Use of automatic tools
Statistical analysis
WEE-NET
of Excellence
31
Graphic representations
Reports

Choosing the right-weight
model
The problem: to establish the level at which
to evaluate the characteristics of the Web
sites
7Loci as theoretical reference framework or
WEE-NET
of Excellence
32
meta-model to instantiate quality models
used in the evaluation projects
Comparison of the results of the light-heavyweight
models: to validate the
approach

Web sites characterization
Quality in Web sites is determined by
several diverse factors: some of which are
general and therefore are considered for all
types of site and in all domains
WEE-NET
of Excellence
33
For example: the correct functioning of the
site, its conformity with standards of
language use or of accessibility (Web
Content Accessibility Guidelines of the W3C,
or standards introduced in Italy through the
Stanca law in 2004)

“Standard” models
Standard “questions”, criteria can be used:
when there are experts in both the
domain and web site quality
or when we do not need detailed
WEE-NET
of Excellence
34
information (e.g., when we have to
compare web sites)

High level analysis
LOCI CICERONIANI-DIMENSIONI SITO Punti di forza/Punti di debolezza
QVIS
(Persona: Who?)
WEE-NET
of Excellence
35
IDENTITÀ

A more detailed model
Peso Punteggio
Identità
Nome del sito - URL
Brand - Il sito ha una forte identità di marchio? 0.20
Immagine - Il sito trasmette una forte immagine dell'ente? 0.30
Design – La grafica del sito è attraente per gli utenti? 0.20
Personalizzazione - Il sito si adatta alle diverse categorie di utenti? 0.20
Totale
Contenuto
Copertura - Le informazioni rispondono agli obiettivi di ente e utenti? 0.30
Valore informazioni - Le informazioni ed i link sono utili? 0.30
Qualità informazioni - Le informazioni sono accurate e complete? 0.30
Autori, fonti contenuti – Sono citate le fonti delle informazioni? 0.10
Totale
Servizi
Funzioni ente - Le funzioni corrispondono agli obiettivi dell’ente? 0.30
Funzioni utenti - Le funzioni corrispondono agli obiettivi degli utenti? 0.30
WEE-NET
of Excellence
36
Correttezza - Le funzioni sono prive di errori? 0.20
Sicurezza, privacy - Le transazioni sono sicure e gli accessi controllati? 0.20
Totale
Localizzazione
URL - L'URL del sito è intuitivo e facile da ricordare? 0.40
Retrieval - Il sito compare nelle prime posizioni dei motori di ricerca? 0.40
Contatti – È possibile contattare l’ente e/o il web master? 0.10
Comunità – È possibile interagire con altri utenti del sito? 0.10
Totale
Gestione
Aggiornamento e revisioni - Le informazioni sono aggiornate? 0.40
Date - Appaiano le date delle ultime modifiche? 0.20
Check-up - Il sito ha una buona manutenzione? 0.20
Strumenti - Le tecnologie adottate sono aggiornate? 0.20
Totale
Usabilità
Requisiti HW e SW - L'uso del sito richiede HW e SW comune? 0.20
Accessibilità – Sono rispettate le norme per l’accesso di utenti disabili? 0.20
Navigazione - È facile navigare nel sito? C'è una mappa, un help? 0.20
Tempi di scaricamento - I tempi del download sono accettabili? 0.20
Lingue - È possibile scegliere più lingue? 0.20
Adapted from: Polillo - http://www.rpolillo.it
Terminologia – Il linguaggio e i simboli sono facili da capire? 0.20
Totale

“Specialized” models
When the evaluation must consider aspects
that have to do with the domain and the
specific aims of the site, we have to define
specialized models that can deal with the
unique semantic aspects of the site
WEE-NET
of Excellence
37

The quality model factory
Quality model factory
Web site evaluation
project
Quality
evaluation
Web sites
classification
7Loci
instantiation
Ontologies
WEE-NET
of Excellence
38
Requirement
elicitation
Quality
requirements
Validation
Quality evaluation
Modules
Quality models
repository
Requirements
repository

A domain specific model
1. Il sito contiene indicazioni sul tipo di albergo, ad esempio, il numero di stelle? Sì No
2. Il sito contiene immagini delle stanze? Sì No
3. Il sito contiene informazioni sui servizi presenti nelle stanze, ad es., TV, telefono, aria condizionata, ecc.? Sì No
4. È presente informazione sulle dotazioni per il fitness ed il wellness, ad es., palestra, sauna, piscina, ecc.? Sì No
WEE-NET
of Excellence
39
5. Ci sono informazioni sulla ristorazione offerta dall’albergo? Sì No
6. Vengono date spiegazioni su come raggiungere l’albergo: indirizzo e modalità per arrivarci con i vari mezzi? Sì No
7. Ci sono informazioni relative alle caratteristiche della regione, quali ad es. il paesaggio, prodotti tipici, folcklore, ecc.? Sì No
8. Vengono date informazioni sulle attività praticabili nella regione, ad es. attività sportive, passeggiate, ecc.? Sì No
9. Ci sono informazioni utili aggiuntive, ad es., le previsioni meteorologiche, bollettino del traffico/viabilità, ecc.? Sì No

Total vs. specialized questions
Specialised questions per dimension:
Identity, Content more than 80%
Services about 30.0%
WEE-NET
of Excellence
40
Location, Maintenance, Usability: from
0% to 15% ca.

Choosing a “weight” for the model
In the SW engineering field this implies
comparing and choosing between
lightweight and heavyweight methods
The right question is not so much which of
WEE-NET
of Excellence
41
the two approaches is the better, but rather
which method has the “right” weight for a
given project

7Loci as web sites quality
evaluation meta-model
To serve as a conceptual schema for the
quality criteria used in an evaluation project
must be
scalable, to allow for evaluations at
WEE-NET
of Excellence
42
varying degrees of detail
domain independent, applicable in
diverse domains
user-friendly, to support the involvement
of people with different skills

A comparison
The RTB project: study line of a triennial
project of the eTourism research group
Goals:
to compare the principal Web sites of the
WEE-NET
of Excellence
43
alpine regions, to identify possible
benchmarks
to identify guidelines or
recommendations for the (re) design of a
RTB Web site

A “standard” table as “lightweight”
model
CICERONIAN LOCI ATTRIBUTES
QVIS
(Persona: Who?)
IDENTITY
Identification
- Brand (organisation or company); Charisma (individual)
- Image
Characterisation
- Design
- Personalization
QVID
(Factum: What?)
222211117777((((11117777
Coverage
- Domain referred to owner’s and users’ goals
- Value of information and links
Accuracy
- Quality of information
- Source(s), author(s)
CVR
(Causa: Why?)
Functionalities
- Adequacy to owner’s goals
- Adequacy to users’ goals
WEE-NET
of Excellence
44
6666((((55559999,,,,((((6666
Control
- Correctness
- Security, ethics and privacy
VBI
(Locus: Where?)
////2222$$$$7777,,,,22221111
Reachibility
- Intuitive URL
- Retrieval
Interactivity
- Contact information
- Community building
QVANDO
(Quando: When?)
0000$$$$1111$$$$****((((0000((((11117777
Currency
- Update
- Dates
Maintenance
- Check-up
- Tools
QVOMODO
(Modus: How?)
88886666$$$$%%%%,,,,////,,,,7777
Accessibility
- Hardware and Software requirements
- People with disabilities
Navigation
- Structure, Orientation
- Download times
Comprehensibility
- Languages
- Level of terminology

A benchmark analysis of the RTB
Web sites
Identity Content Services Location Management Usability
Tirol 3,4 3,4 3,75 2,2 3,2 2,95
Ticino 2,3 3,25 3,25 2,2 3,15 3,1
WEE-NET
of Excellence
45
Valais 3,05 2,5 3,75 2,2 3,45 2,75
Zentralschweiz 2,35 2,65 2,25 1,8 2,35 2,5
Freiburgerland 3,65 3 2,75 1,2 2,15 2,95
Piemonte 1,6 1,4 1 1,4 2,85 2,5
Valle d’Aosta 2 2,85 2,5 1,6 3,25 2,6
Trentino 3,35 3,75 3,25 2,4 2,85 2,5
Alto Adige 3,05 3,1 2,5 1,8 2,85 1,7
Veneto 1,65 2,75 2 1,6 1,6 2,6

A “heavy weight” model
Iterative process
Brainstorming session with 3 senior and
3 junior researchers
- first version of the model
WEE-NET
of Excellence
46
Formulation of the elements of the
framework into Boolean questions
- reducing subjectivity

The evaluation table
WEE-NET
of Excellence
47

Characteristics of the two models
The “lightweight model” is:
general purpose
domain-independent
more subjective
WEE-NET
of Excellence
48
The “heavyweight model” is:
more detailed
require more time to set up,
it can become obsolete sooner
the use of Boolean questions reduces arbitrariness
but requires attention in defining the questions

The 7Loci as meta-model:
validation
Comparing the results of the lightweight
and heavyweight models:
the difference in scores using the two
models were modest
WEE-NET
of Excellence
49
they divide the sites in a largely coherent
and homogeneous manner

The 7Loci as meta-model:
validation (cont’d)
The 7Loci meta-model provides a common
conceptual framework that supports the evaluator in
finding the right weight quality model, doing so by
taking into account the goals of the project:
it is possible to adopt a relatively standard model
built around the dimensions of 7Loci – largely
WEE-NET
of Excellence
50
domain independent – to identify high performing
sites and to rank them
if the aim is to identify weaknesses in the site
and plan its eventual re-design, the 7Loci
dimensions can be used to define more detailed
models based on user needs, owner’s goals, etc.

Conclusions
The 7Loci meta-model provide as a common
conceptual framework that supports the evaluator in
finding the right weight quality model, doing so by
taking into account the goals of the project
it is possible to adopt a relatively standard model
built around the dimensions of 7Loci – largely
WEE-NET
of Excellence
51
domain independent – to identify high performing
sites and to rank them
if the aim is to identify weaknesses in the site
and plan its eventual re-design, the 7Loci
dimensions can be used to define a more detailed
model based on user needs

References
Mich L., Franch M., Gaio L. (2003) Evaluating and
Designing the Quality of Web Sites, IEEE Multimedia, 10
(1), 34-43
Mich L., Franch M., Novi Inverardi, P.L., Marzani P. (2003)
Choosing the rightweight model for Web site quality
evaluation. In J.M. Cueva Lovelle et al. (eds) Web
Engineering, Int. Conf. ICWE 2003, LNCS 2722, 334-337,
Springer
Mich L., Franch M. (2005) Instantiating Web Sites Quality
Models: an Ontologies driven Approach, 5th Int. Workshop
52
on Web Engineering IWWOST'05, Jun. 13., 2005, Porto, P
Mich L., Franch M., Martini U. (2005) A Modular Approach
to Quality Evaluation of Tourist Destination Web Sites: The
Quality Model Factory. In Frew A. J. (ed) ICT in Tourism
ENTER2005, Innsbruck, A, Jan. 26-28, LNCS, 555-565,
Springer
L. Mich, M. Franch (2002) Requirements for a tool to
support evaluation of web sites quality based on the
2QCV3Q model, in Proc. IADIS Int. Conf. WWW/Internet,
Lisbon: IADIS Press, 2002, 541-544

Part 1: Website quality evaluation 7Loci method

Recommandé

Recommandé

Contenu connexe

En vedette

En vedette (8)

Similaire à Part 1: Website quality evaluation 7Loci method

Similaire à Part 1: Website quality evaluation 7Loci method (20)

Plus de Luisa Mich

Plus de Luisa Mich (20)

Dernier

Dernier (20)

Part 1: Website quality evaluation 7Loci method