Where and how we’ll move
When analyzing web sites, especially when comparing more than one web application, it is necessary to
focus on industry.
We are taking into consideration two organizations belonging to the country military defense industry
(for this reason they are competitors); this particular context has special characteristics, and web sites
representing an Internet access point to such institutions, must meet some requirements.
Detecting these requirements makes easy to evaluate our web sites, if all conditions are met, then the
web applications we are analyzing provide a good service.
What services web sites representing country defense institutions should provide
Getting into the clothes of the user who is navigating through the web application in exam, the first
question we must answer to is:“What services do I expect to find?“. Another useful question is:“What
kind of users would access the web site?“. Although these two questions may appear as useless as point-
less, they help us to find out our requirements for the web sites we are going to examinate. So, let us
answer the questions.
User typologies
Web applications like the ones we are considering, are visited by internal personnel (for accessing private
areas, private services and so on), by newsmen (to get news and press releases by voicemen and official
sources), by citizens and users involved in the military and institutional fields (like politicians, military
personnel and so on). The most important point we must reach, is that the web sites in exam are con-
sulted by a very wide range of people belonging to various divisions:
• Politicians
• Military men
• Civilians
• Military or civil institutions
• Privates
Purposes and communication objectives: providing services (analysis on usability)
All these typologies individuate functionalities that the web application should provide; in fact the main
purpose of both web sites is to providing a set of services accessible to users. But we are not interested
in the services’content (for the moment); we want to analyze how the system provides access to these
elements and how it makes them usable for the user. For this reason we can answer to the other question
and detect the main services the two web sites should afford:
Context at a glance: Country
Military/Defense industry
Pg. 2

• A constantly updated news area where it is possible to get information about the most important is-
sues regarding country defense system and country threat present level.
• A specific area reserved for civilian and military leaders and heads of the country. This service should
provide access to the main hierarchy in order to detect immediatly people involved in the various
fields of the country defense departments.
• A dedicated area for past, present and future missions in homeland and foreign territories. This area is
supposed to provide information about soldiers dispatch and mission objectives and status.
• An area providing access to the main alliances between the country government and other govern-
ments in the world. This area should also provide information regarding international alliances (i.e.
NATO and similar).
• An area providing links to the main defense branches. This list is supposed to provide a connection to
the main military and non-military institutions involved in the country defense system just like Army,
Navy, Country Defense Headquarters, Auronautics and so on.
• An area dedicated for users who wants to join the defense system as member of the civilian person-
nel or the military personnel. This service should provide links to the main institutions ruling the
recruitment/employment procedures.
• A connection to the main judicial branches of the country. This connection is supposed to provide
information about the ordinary laws and the military laws regarding the most important national and
international issues about war and conflicts.
This collection is a first draft about the main services that both web applications should provide to the
user (and will be revised later in the next chapters).
When analyzing the two web sites, our work is simplified because we can focus on the services and then
make comparisons.
Usability criteria are many and it is easy to get lost among them, focusing by services is a way for setting
a pattern in our analysis, the list above will be a sort of guideline in our evaluation process.
Introduction to difesa.it and defense.gov + initial con-
siderarions
Both web sites act in the military/institutional fields and they provide a connection to the country de-
fense and homeland security departments. For this reason they are not just simple web applications navi-
gated and consulted by users; they provide services and information which are useful for a wide range of
kinds of people: civilians, military men, politicians and so on.
At a first glance they look very different in setting and in their graphic structure (we are not begginning
our analysis, we are just having a first look to the applications, without going deeper), but their differ-
ences rely in other aspects; for example, given the fact that they work for homeland security, both sites
use different main languages: American and Italian (we still don’t know if one of them also provides sup-
Pg. 3

port for different cultures). They also refer to different hierarchical systems: in Italy the homeland security
department is represented by the Ministero della Difesa, while for the U.S.A. we have the Department of
Defense, and it is important to underline that these two institutions have different rules and a different
internal organization.
All the differences don’t really rely on the website itself, it is a matter of institutional correlation. The
homeland defense department has a deep connection with the country political organization; for this
reason, services provided by the two web applications will be various and sometimes they won’t have
any meeting point. This fact will complicate our path based on services description (we have to detect
common services and try to generalize those services which both applications provide in a different
way), but, once finished, our analysis will not be affected by any external problem.
Websites usability contest now begins: US Defense Department vs IT Ministero della
Difesa
Now we can officially start our analysis by introducing the main actors: the websites we are going to
compare.
Ministero della Difesa della Repubblica Italiana: www.difesa.it
United States Department of Defense: www.defense.gov
Pg. 4

A system for assigning score
Through our analysis we have to take into consideration various characteristics for both the sites. In the
next chapters we are going to explain the path to follow for covering all elements inside the applications
(in order not to miss anything), but before doing that, it is necessary to specify which methodology we
are going to use for assigning a“score“ to every evaluated element.
Element: definition
Although we still have to define what we are going to evaluate (elements), we need an initial definition
of“element to evaluate”.
With the expression“element to evaluate“ or simply“element“ we mean any application’s characteristic
which it is possible to assign a score to, indicating the application’s attitude in implementing that aspect
(the higher is the score, the higher is the characteristic’s quality).
Element importance
We are going to examinate many elements, but not all of them have the same importance in the site
evaluation; meaning that although two elements are evaluated with the same“score“, they may not as-
sume the same value inside the global evaluation. In order to take into consideration this situation, we
need to tile another evaluative quantity (a second score) to our present score: this entity is called ele-
ment’s weight or element’s importance and it is a numerical quantity assuming these values:
• Irrelevant - 0.2: The considered element has not a sufficient importance in the global evaluation
process; if present, the element is just an available feature.
• Considerable - 0.5: The considered element is not needed in the global evaluation process.
• Important - 0.75: The considered element is important in the global evaluation process.
• Determinant - 1.0: The considered element is really important (necessary) in the global evaluation
process.
Element score
We can now define the score assigned to an element. We define score as a numerical quantity indicating
the quality of the element’s implementation proposed by the application:
• Very bad - 0.0: The considered element’s implementation is absolutely not good, doesn’t work as
expected at all, and doesn’t provide a useful service.
• Not good - 1.0: The considered element’s implementation provides a poor service not satisfying all
requirements and not working as expected (in some aspects).
Evaluation criteria and
examination path
Pg. 5

• Good - 2.0: The considered element’s implementation provides a good service satisfying many
requirements and working as expected (in almost all aspects); this level defines a minimal quality
threshold.
• Very good - 3.0: The considered element’s implementation provides a reliable service satisfying all
requirements and working as expected (in all aspects), this level defines the best quality threshold.
• Excellent - 4.0: The considered element’s implementation provides a reliable and robust service satis-
fying all requirements and providing access to other useful functionalities (even if not required); the
element works as expected (in every scenario).
Six evaluation dimensions
An element always represents an aspect of the application, we organize all aspects and characteristics
into six groups (dimensions) so that, finally, we can represent the global evaluation process result into
one single radar chart. These dimensions are:
• Architecture: Site structure and navigation system.
• Communication: Attainment of the communication objectives.
• Functionalities: Correctness of the site functionalities.
• Content: Quality of the informative content (information and data).
• Management: Site operativity (site works fine?).
• Accessibility: Site accessibility (is site accessible to all kinds of users?).
Totalizing dimensional score
We are going to group elements by dimension and assign to each one a score (concordently with the
assumpions made before), but in order to get a final score for each dimension, we need to find a math-
ematical procedure.
• Let D be the dimensional score (score referred to a particular dimension).
• Let n be the elements in the dimension.
• Let Pi be the element’s importance (weight).
• Let Ei be the element’s score.
Given these hypothesis we consider the following formula:
It formerly represents a weighted media where weights are element’s importances and scores are insert-
Pg. 6
1
1
1 1
1
n
i i n n
i
i i in
i i
i
i
P E
D P P E
P
−
=
= =
=
⋅
 
= = ⋅ ⋅ 
 
∑
∑ ∑
∑

ed inside the mathematical process.
For every dimension we get a dimensional score to use for creating the final radar chart.
Totalizing site evaluation score
We also provide a numerical quantity useful for resuming all aspects of the site, without considering
dimensions. This number can be obtained in this way.
• Let S be the final site score.
• Let n be all the examinated elements.
• Let Pi be the element’s importance (weight).
• Let Ei be the element’s score.
Given these hypothesis we consider the following formula:
Doing so, we will be able to absolutely determine which site is better than the other just by making a
numerical comparison using this quantity.
Our next steps
Once the scoring methodologies and formulas are defined, we are now able to begin our analysis. Focus-
ing on dimensions (in the order shown before), we are going to define elements to work on. For every
element we are going to assign a weight and examinate how both sites implement it, in order to assign a
score to each web site. Once all elements are finished, we finally get the dimensional score for every site.
This procedure will be performed for every dimension.
Now we can start examination process and evaluation.
Pg. 7
1
1
1 1
1
n
i i n n
i
i i in
i i
i
i
P E
S P P E
P
−
=
= =
=
⋅
 
= = ⋅ ⋅ 
 
∑
∑ ∑
∑

Site architecture dimension defines all those elements regarding the web application’s page relations,
links structure, user navigation system and site map. Basically we are now going to examinate how the
web application organizes its content and provides information to users.
Web site structure
Using the expression“site structure”we mean the site’s pages organization. Every web site uses its own
pages structure so that it is possible to indentify many known patterns, recognizing the site structure let
us refer it to a known pattern, and every known pattern has known defects and known esteems.
Element importance
The structure of a web site is crucial for a good usability level, the better is the structure the more con-
fident will be the user in navigating the web application and the better will be his use experience. We
assign to this element importance Determinant (1.0).
Getting site structure
To get the structure of a web application it is necessary to navigate the web site through its pages. Our
web sites have very different internal organizations, this makes different the navigation flow too (naviga-
tion flow will be described later). Starting from the home page and getting deeper in the structure we
can draw a simple diagram, we should not be (in this place) too much detailed, so it is possible to omit
secondary pages and focus on main areas.
Examinating structures
We can see below both web sites’structure, it is possible to notice many differences relying in general
setting and pages disposition in the diagram.
Beginning our analysis we can focus on setting. In the US Defense site we can notice a hierarchical order
starting from the home page, going to the six main areas (News, Press resources until Contact us) and
ending up in the second level pages (each one belonging to a specific area); basically the entire structure
is divided in two levels, the secondary level contains detailed elements shown in the first level.
The IT Defense site uses a different approach: all pages are, in fact, inserted in a common level so that it is
impossible to identify a hierachical scheme.
Comparing both diagrams we can find usability goals and failures. At a first look at the IT Defense site’s
structure, while its monolithic disposition may appear as dispersive and chaotic, it provides a single
access point to all application’s areas; this approach must not be underenstimated: in fact, sometimes (es-
pecially for this kind of web sites), users need a fast gateway for entering requested pages.
The completely different US Defense site’s structure defines, instead, a good organized disposition of
pages that are reachable by the user each time he enters a particular area; this approach compels the
user to navigate other pages in order to access the desired content (because of the hierarchy); although
Evaluation for Architecture
Pg. 8

this organization implies more effort by the user, it gives a good logical map to him; in fact, if on one
side we have this usability miss (apparently), it is demostrated (on the other hand) that users are able to
perceive page paths and patterns (if pages structure is very good) and locate autonomously the desired
page (even if it is the first time they navigate the web application).
Element score
Because of all considerations made before, we find a condition of parity in both diagrams, the score will
be the same for the web sites. However, notwithstanding the positive considerations made, we won’t as-
sign the max score because both sites are not absolutely simple to navigate: the US Defense site is cha-
otic in some pages, generating many loops inside the diagram we didn’t show for space needs, and the IT
Defense site uses different pages to organize correlate content (leading user to confusion).
SCORE FOR US DEFENSE SITE IS: VERY GOOD (3.0)
SCORE FOR IT DEFENSE SITE IS: VERY GOOD (3.0)
Pg. 9
Web site structure for defense.gov
Web site structure for difesa.it

Web site navigation (tools and menues)
While a standalone client application is generally utilized by the user, a web application is navigated by
him. This concept is important to understand because it focuses on a truth:“a web site is a complex col-
lection of views through which the user can access the provided functionalities and content defining, so,
several different browsing flows“.
Element importance
Site navigation is an expression indicating a web application’s ability to define a set of tools able to lead
the user through the site’s structure. We considered, a while ago, the structures of both web sites; site
navigation is a correlate element because a good structure is nothing without a good navigation system.
Navigation is in charge of defining the best patterns the user will follow to access the site’s pages.
Because of all these considerations, this element has a high importance level: Determinant (1.0).
Navigation tools
A web site generally implements a set of tools in order to help the user to navigate the site structure. One
of this features is of course the site map, but this is an element we’ll see separately (given its importance).
There are other controls provided to users for site’s pages browsing purposes, for esample menues and
navigation bars. Both web sites uses menues but with a different approach.
The US Defense site defines a top menu showing the six main areas located in the primary level of its site
structure (see the diagram in the previos paragraph); when entering a secondary level page (depending
on which first level page the user comes from) top menu may change (in graphic and content) showing
more options for the selected area, or stay the same.
The site also offers another menu localized in the bottom part of some secondary level pages (especially
in pages examinating top leaders and American defense department personnel), it shows external links
(always regarding the American defense institutions) or more options for the current page, sometimes it
also duplicates the links shown in the top menu of the same page.
The US Defense site finally uses a side menu showing inner levels’options. In a lot of pages, the side
menu organizaes a set of links (by argument) in a very good list structure, the user is able to select pages
regarding the same argument of the current page without losing its position, because the side menu is
always available in pages shown inside it.
The IT Defense site, given its one level structure (see previous paragraph), uses one side menu showing
the unsorted list of all areas in the site structure. In detail pages the web site offers also a second side
menu (on the right) showing links regarding the current page and sometimes having the same title of
links in the left menu, these links are not the same because they lead to different places; all links shown in
the right menu lead to indicated pages focusing on details correlate to the current page.
There is, finally, another navigation tool provided in many pages of the site: a site map path located in the
top area of the page showing the current user location in the sites structure; not all pages implement this
feature.
Examinating navigation
The US Defense web site, with its navigation features, provides to the user various tools for jumping to
desired pages; in this way the user is always surrounded with many links (in the top, side and bottom
Pg. 10

areas) and has a high interaction level with the application. We must also consider that it is the typical
American surrounding strategy the one used in this place; in fact, placing banners, links, animated gifs
and interactive tools all around the page’s main content, it is possible to create a perimetral interaction
region where user finds all the links he needs; Americans are accustomed to this template and for this
reason they are very familiar to it. That’s why we evaluate the US Defense navigation approach as very
good because it matches the user needs (American defense web site offers information mainly for Ameri-
can people).
The IT Defense site doesn’t offer a very useful navigation structure, because the main side menu (the left
one showing all areas) is chaotic and unsorted, there isn’t even a grouped scheme, links are all presented
as a list. It is demostrated that long lists are very tedious for the user to utilize because it is difficult to
search for a specific link.
Furthermore, the right side menu showing page context based links, is not so immediate to use; the user
is confused at the beginning, seeing two similar menues, and there is not a description explaining the
purpose of both navigation lists.
All considerations made up until now lead to an usability failure: in order to avoid confusion and link-
ing chaos, it is necessary to use one menu only and using, for inner pages, a tree structure; here we see
a navigation based on two unsorted lists, the user is really compelled to watch both sides of the page in
order to detect what he needs and this is a very difficult situation for him.
A positive factor is represented by the horizontal top site map path; users feel much comfort in seeing
such a navigation tool because it is like a map with a large red point with the text“You are here“; this is
sure an usability goal, even if we must take into consideration that not all pages implement this feature.
Element score
Given all considerations made before we can extract the final score for both web applications. US De-
fense site has a good navigation structure, user may encounter an initial difficulty in seeing so many links,
but we must keep in mind that this is an American commonly used graphic pattern that renders this
visualization a good way for navigating the web site.
IT Defense site has many usability misses that are not covered by the horizontal map path navigator
(which is just single white stone in a box of black ones).
SCORE FOR IT DEFENSE SITE IS: NOT GOOD (1.0)
Site map
In a web application the site map is a good feature that enables the user to look at our site just like if he
were to watch a city from an observatory. This is not a required feature, but sure is something simplifying
user navigation. We are treating site map as a separate element from navigation element, because we
think of it as an external tool not so strictly connected to site navigation (even if some connection points
are present).
Element importance
We already said that site map is not a required tool, this implies that importance level, for this element, is
lower than Determinant.
Pg. 11

In this place we consider site map as a useful tool, if a site implements it then usability level grows. Be-
cause of this reason we mark this element as Important (0.75).
Maps analysis
Both sites implement a site map but using completely different approaches.
The US Defense site has a very common graphic pattern used for site maps: it shows a grouped list of
links to the main areas of the site’s structure in the absolute bottom part of the page. When the user ac-
cesses the home page, after scrolling to the bottom region, he can see a collection of lists organized in
groups and divided in different columns (five columns). Links are grouped by area and, globally, it is pos-
sible to recognize a structure similar to a distributed tree. Even in this details we notice the presence of a
common American graphic pattern, so we can also imply that American people are accustomed to this
kind of maps.
The IT Defense site uses, insted, a common Italian graphic pattern for web application maps: that is an
entire page (reachable by a link located in the bottom part of every site’s page) dedicated to the map and
showing a list of the main pages grouped by areas.
Examinating maps
The US Defense site map is simple and has a very special particularity: depending on which page the user
is navigating, the map adapts itself showing more or less links in order not to show useless information
(just like uncorrelate pages or redountant links). By doing so, the user has always a sort of pocket guide
available, where all directions are showed depending on his position.
The IT Defense site has, instead, a very common structure regarding its map; that’s why we don’t go
deeper in our analysis being unable to describe special features or similar.
Element score
It is evident that US Defense site reaches a very good usability level (considering also that all its map is
viewable in a small part of the page while in IT Defense site the map uses all page’s space and user needs
to scroll). Regarding IT Defense site, we have just a simple service not showing particular features.
Given all considerations made up until now, we express these final scores.
SCORE FOR US DEFENSE SITE IS: EXCELLENT (4.0)
SCORE FOR IT DEFENSE SITE IS: GOOD (2.0)
External links and community
Although it might seem that external links are not part of a web site’s architecture, they provide a service,
for the user, to reach other web applications that are probably connected to the main one (for various
reasons). That’s why, in our evaluation process, we will consider external links as an element to assign a
score to.
Brief considerations on external links and community connections in web sites rep-
resenting institutions
Pg. 12

It is a common mistake considering that external links have the same value in all web applications. It is
not the truth because the relevance of external links is deeply connected with the web site’s context/
industry. In our case the institutional field is a very particular context. When a user is navigating the web
site of an institution, he expects to be able to access other institutions in order to get other services, or
also expects to find a connection to the seervices in some community engines. Today communities are a
widely diffused concept, almost everyone has an account with Facebook or Twitter and so on... An insti-
tution having also some instances of its services in a community web site surely is a very good service
provider.
Element importance
Considering what we said just two lines before, we consider this element to be important in a web site,
for this reason the importance level is set to: Important (0.75).
Analyzing external links structure
Both US Defense web site and IT Defense web site have external links. But even in this case it is possible
to notice many differences.
The US Defense site places community links (and a few link to other military institutions) in the home
page and in the contacts section on the left side of the screen. We can see a wide range of external areas
connected to the US Department of Defense just like: DoLive, Facebook, Flickr, Twitter and YouTube.
There are also links for subscribing to the defense.gov site newsletter: RSS Feeds, Podcasts and Widgets
are just an example.
In the DoD Websites area of the web site, the user can find all external links he needs thanks to a very
useful page showing all links in an alphabetically sorted table (where the user can choose a letter in order
to show all links having the first letter corresponding to the selected one).
The IT Defense site uses a different approach. As for the site map, links page has a completely dedicated
page. And also in this case, all links are presented in a list. This time the list is sorted by area generating a
structure similar to a tree.
In this case, the web site shows the list of institutional links only, there are no links to community engines.
We can say that US Defense site has a high usability level because all links are viewable without scrolling
the page and have icons next to them that helps the user to find out what he needs, even without read-
ing what’s written. This feature (readibility) is present also in the page for external links where navigation
by letter avoids long and endless lists very difficult to consult.
On the other hand, we also need to consider that IT Defense site makes all mistakes described before.
Element score
Given all considerations upon both sites’external links, we set these scores:
Getting dimensional score
Pg. 13

We can now get the final score for this dimension using the formula introduced in the first chapter.
As we can see US Defense site has a higher usability level regarding the Architecture dimension; the dif-
ference with the IT Defense site is considerable.
ARCHITECTURE DIMENSIONAL SCORE FOR US DEFENSE SITE IS: 3.40
ARCHITECTURE DIMENSIONAL SCORE FOR IT DEFENSE SITE IS: 2.00
In order to better see the situation, we can chart data and results evaluated up until now in a bar diagram
showing both scores and the mean between them.
Pg. 14
( ) ( )
( )
1 14 4
1 1 1 1
1
1,0 1,0 0,75 0,75 1,0 3,0 1,0 3,0 0,75 4,0 0,75 4,0
3,0 3,0 2,96 2,96 11,92
3,40
3,5 3,5
n n
i i i i i i
i i i i
D P P E P P E
− −
= = = =
−
   
= ⋅ ⋅ = ⋅ ⋅ =   
   
= + + + ⋅ ⋅ + ⋅ + ⋅ + ⋅ =
+ + +
= = =
∑ ∑ ∑ ∑
( ) ( )
1 14 4
1 1 1 1
1
1,0 1,0 0,75 0,75 1,0 3,0 1,0 1,0 0,75 2,0 0,75 2,0
3,0 1,0 1,5 1,5 7
2,0
3,5 3,5
n n
i i i i i i
i i i i
D P P E P P E
− −
= = = =
−
   
= ⋅ ⋅ = ⋅ ⋅ =   
   
= + + + ⋅ ⋅ + ⋅ + ⋅ + ⋅ =
+ + +
= = =
∑ ∑ ∑ ∑
Charting score for this dimension
0,0 0,5 1,0 1,5 2,0 2,5 3,0 3,5
Mean
IT Defense
US Defense

Every web site defines a communication level where the main objective is fulfilling the application pur-
poses. In our case, we have already defined both sites’purposes and communication objectives in the
first chapter; so we can start analyzing those elements connected to the communication level.
Home page
Home page is the first thing a user looks at entering a web site, it is calculated that the medium user ac-
cess and first navigation time for a web application consists in few seconds (10, 20 and in some cases also
more than 45 seconds) before he leaves. Home page, has for this reason, a very important target: com-
municating the largest possible quantity of information to the user, in order to allow him understanding
what the web application is about.
Element importance
Web sites’home page importance is recognized by all users in the world and all web developers and
programmers; it is useless to spend time for explaining why we give the highest importance level to this
element. Importance: Determinant (1.0).
Home page analysis
We are going to proceed by questions. The first one is:“Does the home page communicate directly and
in the fewest time possible, what the entire web site is about?“ and the others are:“Does the home page
use any payoff or tagline?“ and:“Does the home page provide any description or any special content to
inform the user about site’s details or information?“.
The US Defense site’s home page has several characteristics. It is 1000 pixels wide and screen centered, all
content is easily viewable without scrolling the page (the hidden part includes the site map only); there
is a top image title, the main navigation menu, a side menu for quick external links and a centrered news
spot area with animated content changing every time.
The American circular symbol with the eagle (in particular, the American eagle holding three arrows
refers to the American Defense) on the top area, helps the (American) user to recognize that the site he is
browsing is an institutional oriented application. The background color also underlines the institutional
aspect of the web site (blue is widely used in the U.S.A. for pointing out internal institutions, this is an
example of differences between cultures, in Italy this aspect connected to color is not as strong as in the
U.S.A.).
American defense home page doesn’t use payoffs or taglines, there is just one significant text:“United
States Department of Defense“ giving the whole idea about the site context. There are also no descrip-
tions or information about the web site.
In the end, images and animations showing soldiers, marines and military personnel at work (with a tex-
tual description for every item), renders impossible to misunderstand what the site is about.
Evaluation for Communication
Pg. 15

The IT Defense site’s home page has different characteristics. It is 750 pixels wide and screen centered;
the graphic structure defines a top image, a top menu bar and the main body divided in three columns
showing a side menu, a central content with a news image (static) and a banner/news side area.
The top icon showing the Italian Republic symbol and the Italian Flag’s colors give the user a good start-
ing point in order to understand what the web application is about. The user would probably have no
doubt about the site’s content after seeing, on the other side of the page on the top area, the flag of the
Ministry of Defense showing the four military forces and a central star.
Going down from the top region, the page is divided in three vertical parts. The user must scroll the page
in order to see all content, and the page is very long, too much long for a home page.
The news image in the central part of the page changes depending on the headlines but there are no
descriptions apart from the news title.
Performing a last examination about links provided and site’s areas access controls, we can see the US
Defense site shows a lot of links and connections to other related institutions, and all links are available
without scrolling the page with significant icons and images. From this point point of view, IT Defense
site does not provide the same usability level: there are some links refering to other Italian institutions
but they are reachable by scrolling the page only.
Element score
Given the results of our analysis, we can see that the American defense web site has a very good home
page, showing all information in the most reasonable way, and allowing the user to understand in few
seconds the site’s purposes and communication objectives.
A different result comes from Italian defense web site: the home page sure is able to communicate the
web application’s purposes but it is not as immediate to understand as the competitor’s home page
does; this is probably an usability failure caused by the poor graphic structure, especially regarding the
total page height: the home page is not enough wide and is too much long. There are also too many in-
formation that are not presented in a suitable content structure, in fact there are too many lists and even
they are too much long.
Brand image
Brand image is a particular feature provided by home pages, or other pages, in order to let the user un-
derstand what the web application is about in the fewest time possible.
Element importance
Given the sites context, brand image is not a required element, brand images are useful elements for web
sites selling products or similar. For this reason we give a low importance level to this element. Impor-
tance: Irrilevant (0.2).
Brand image analysis
Both sites don’t really implement a brand image, the only one image indetifiable as a brand is the news
image. So we cannot say much about this element.
Pg. 16

In particular, the US Defense’s home page has a good brand because news image changes everytime
showing significant pictures concerning the military/defense site context.
IT Defense’s home page has just a static image; pictures shown, depending on headlines, concerns often
political events, for this reason the brand level is low.
In other pages, the US Defense site shows often a wide picture related to the area the user is navigating.
The IT Defense site, instead, doesn’t use brands in pages different by the home.
Element score
Given the considerations made up until now, we can set a score for the brand image element.
Graphic structure
In the programmers/web developers & designers world there is a still open debate regarding graphic
vs textual content and which element is more important than the other. In this contest the question
is:“What’s the most important part in a web site to focus attention on: information, data and textual
content or graphics?“. As we said, the debate is still open, but there are few good answers; one of this
explains how, from usability point of view, the graphic structure of a web site is the only one element
providing the user information on what the web site is about without any need of reading any line of
text. For this reason graphics is important. In this paragraph we’ll analyze this element for both sites.
Element importance
Considering what we said few lines before, there is no need to explain why we choose the highest impor-
tance level for this element. Importance: Determinant (1.0).
Analysis on page layout
Every web site defines two sets of pages different from the graphic structure point of view: the home
page and all the other pages. So, apart from the home, all pages of a web site have the same graphic
layout.
The US Defense pages’ layout has the following characteristics:
• Pages are width fixed and all 1000 pixels wide. They are browser centered.
• Under the top image, there is the main menu with textual links. The main body is divided in three sec-
tions: a side submenu, the content area and the features/headlines panels. Almost all pages have this
structure so that the user acquires familiarity with links and controls disposition. In the bottom region
there is the area map, showing pages related to the current one.
• The content area is always filled with text and images; text has always the same structure with boxes
showing related data.
This is undoubtly a good page layout even if, given the large quantity of information, it would be better
Pg. 17

using pages that covers all browser width.
The IT Defense pages’ layout has the following characteristics:
• Pages are width fixed and all 750 pixels wide. They are browser centered.
• Pages are divided in four regions: a top region with page title and the main image showing the of-
ficial logo of the Italian Ministry of Defense, and the main region divided in three parts: a side menu, a
central content area and anothe side submenu.
Even in this case we think that page layout width is not suitable for this site, but in this case we don’t sug-
gest to adapt the width to the browser area, it is sufficient to change the size of the page to 950 or 1000
pixels.
Focusing on color schemes we find many differences between the web sites.
The American Defense site uses brilliant colors; the chromatic range varies from blue to white, but always
inside the cold frequencies (blue, light blue, dark blue, white). Text uses other colors: red, orange, black
and white with blue background (in case of links some colors changes).
Italian Defense site has a different color range, in particular the main difference with the competitor is
that this site uses light colors: light grey, light yellow, light blue and sometimes light violet. This scheme
is not recommended by web design guidelines, because the absence of strong gradients renders difficult
the user navigation.
Analyzing gestalt, both sites reach a good level but, even in this case, the American Defense site has a
better approach than Italian one. The reason relies in colors: in the U.S.A. we have already said that blue is
a color associated to the institutional field, American culture renders easy to identify web sites working
for/as institutions while in Italy this situation is not present.
There are still few aspects to analyze bofore setting a score, one of this is typography.
The US Defense site uses common font families for text (compatible with many operating systems). There
is a wide use of character styles just like bold, underline, background images for titles, side images and
colors. Clickable and dynamic text has a different style in order to be recognized by the user (especcially
for news abstacts), although there are various link styles, all links are underlined or show a little arrow at
the end of the text.
In IT Defense site, text has few styles in order to differ a paragraph title from the body. Links are recogniz-
able because of the underlined style applied to it, but sometimes clickable text uses other graphic tap-
perns (some links are underlined while others are bold or others are also boxed), this is not a good thing
because some links are not recognized by the user.
Focusing, in the end, on attractiveness, we find two different worlds. Although this kind of aspects is too
much subjective, a good way for evaluating this parameter is focusing on images and styles. US Defense
web site, from this point of view, is the more attractive one, because of the number of textual styles it
uses, the number and quality of images and the site color scheme; Italian Defense site has a bad color
scheme and there are not many textual styles and icons. If we were to set a judge about attractiveness,
we would probably say that US Defense reaches a very good level while Italian Defense gets a sufficient
level.
Element score
Pg. 18

There is not much to add to what we have said up until now, we can set the score.
Using the same procedure of the past chapter we can get the final dimensional score. As we can see US
The US Defense site has a higher usability level regarding the Communication dimension; the difference
with the IT Defense site is considerable.
In order to better see the situation, we can chart data and results evaluated up until now in a bar diagram
Pg. 19
( ) ( )
1 13 3
1 1 1 1
1
1,0 0,2 1,0 1,0 4,0 0,2 4,0 1,0 4,0
4,0 0,8 4,0 8,8
4,0
2,2 2,2
n n
i i i i i i
i i i i
D P P E P P E
− −
= = = =
−
   
= ⋅ ⋅ = ⋅ ⋅ =   
   
= + + ⋅ ⋅ + ⋅ + ⋅ =
+ +
= = =
∑ ∑ ∑ ∑
( ) ( )
1 13 3
1 1 1 1
1
1,0 0,2 1,0 1,0 2,0 0,2 2,0 1,0 2,0
2,0 0,4 2,0 4,4
2,0
2,2 2,2
n n
i i i i i i
i i i i
D P P E P P E
− −
= = = =
−
   
= ⋅ ⋅ = ⋅ ⋅ =   
   
= + + ⋅ ⋅ + ⋅ + ⋅ =
+ +
= = =
∑ ∑ ∑ ∑
0,0 0,5 1,0 1,5 2,0 2,5 3,0 3,5 4,0
Mean
IT Defense
US Defense

Services, functionalities and interactivity provided by web sites must satisfy user needs and must also be
easy to understand and to use. We are now approaching to study these aspects of both web applications
evaluating what functionalities they provide (apart from search function that will be described in a dedi-
cated section) and then examinating their quality from several points of view.
Implemented functionalities
Let us first detect what functionalities both sites provide to the user. Listing them, we’ll be able to make
comparisons and see which site has more services.
Element importance
User expects to have some services in the web site he is navigating, functions and services are essential
because they are the core of the application. That’s why we give the highest importance level to this ele-
ment: Determinant (1.0).
US Defense services
The American Department of Defense web site has a lot of functionalities that improve user interactivity
with the system, here we show those ones we believe to be the most important and useful:
• Threat advisory: A tools always available on the top right side of the page informing how much high
is the present American military and homeland defense emergency level. This functionality is very
useful and provides a constantly updated data about homeland security.
• External link disclaimer: This function enables the user to interact with the web site, allowing him to
notify links that does not concern the American defense
web site mission, in order to improve the web application
infrastructure. The user is requested to write a description
about those links he believes being inappropriate, and he
is also requested to submit those links he believes ought
to be included in the web application. Sure this is a good
service because, doing so, the system traces user prefer-
ences and it is also a good way for improving usability.
• Interactive survey: This is a tool available only for those
users navigating many pages of the web site. When
the application detects a user who never entered the
site before, a request for submitting a survey about the
American defense web site appears. Users can reject or
accept, in case they accept, the application shows them
Evaluation for Functionality
Pg. 20
defense.gov survey

a series of questions (in a new window) the users can answer to. At the end, a message tells the user
that his choices will be considered in order to improve the web site services for matching his prefer-
ences, among with all the other users’ones. This is a good functionality for the same reasons seen in
the previous point.
• RSS, Podcasts and e-mail subscriptions: The web site offers links for allowing the user to subscribe
to e-mail alerts and newsletter; there is also the possibility to access to RSS Feeds and Podcasts in
order to be always updated with the last news and information. This is a good functionality because it
is possible to access to the web site’s services, not only through an Internet browser, but also through
e-mail providers and mobile devices’s navigation software.
• Personnel biographies inspector: The web site provides a page dedicated to internal personnel
biographies. It is a very good table (five columns) showing, sorted by name and hierarchical position,
all most important people involved in defense and homeland security (civilians and military men).
Every cell of the table shows the picture (121x151 pixels) and, below, the name and position. This is a
good service because it is useful for lacating personnel and watching every biography by clicking the
image or the text below it.
IT Defense services
Pg. 21
defense.gov biographies inspector

Watching and browsing the Italian defense web site, the user finds fewer services than the American
website:
• Personnel hierarchy: The web site provides a useful service showing the internal hierarchy diagram
with the Minister of Defense at vertex and all the other charges under him. Sure it is a good service
but it lacks of details: for example there are no pictures and there are no names but just the charge’s
title and nothing more.
• Announcements: There is a page showing all Italian military divisions and providing access to the
corresponding area in order to let the user see all announcements regarding the chosen division (Es-
ercito Italiano, Marina Militare Italiana or Aeronautica Italiana). This functionality is very good because
it provides a fast link for other services like recruitment (a very important aspect in the context we are
examinating).
• Virtual Museum: It is a functionality activated one year ago. By installing a specific 3D web engine, it
is possible to download client side a VRML file showing a tridimensional museum with various items
belonging to the main Italian military divisions (Esercito Italiano, Marina Militare Italiana or Aeronau-
tica Italiana). It is a good interactive tool, but it is based on a not standard technology: the 3D player
is the VRML player and it is not a wide diffused software (some users may decide not to install this
component because they don’t trust it).
Element score
As we can see, the US Defense site has many useful services with high usability levels. The Italian Defense
site does not provide so many functionalities and, furthermore, some of them does not reach a good us-
ability level. Our evaluation process focuses the number of services and thei quality. For these reason, we
can set these scores.
Errors management
Even when encoutering errors, web sites should pay attention to how errors are shown and what infor-
mation the user is allowed to see.
Element importance
Errors management is a functionality the user does not pay attention to. It is demostrated that, when en-
countering errors, the user (whether an error page is shown with some description or not) does the most
common thing that everyone would do in such cases: clicking the go-back button of the browser and
going to the previous page. This does not mean that a good error management system is an irrilevant
element, especially for sites implementing complicated functionalities that can go wrong somewhere
(for example registration procedures or transactions). For this reason we give a medium level importance
to this element: Considerable (0.5).
Analyzing error management: trying to crack functionalities
Pg. 22

Examinating error management is difficult because we need to encounter an error. So our actions are just
trying to crack some functionality in the application (not doing what expected) and see what we get. Lt
us proceed by functionality:
• Registration form: It is possible to create, for US Defense site, a personal account indicating e-mail
address and other low level profile data; if data format is not the expected one, a client validator
marks incorrect fields not allowing the user to continue. The form does not perform a live email
check, so if the email inserted is not correct, the user doesn’t get any advise and will never find the
confirmation mail in order to complete registration.
The Italian defense site does not provide a registration service.
• Unavailable resource: The most common error is the user trying to access a page that does not exist
or that has been removed. So, just by typing an incorrect page, we can see how our web sites re-
spond. The US Defense and IT Defense both show an error page with a simple text indicating that the
requested page does not exist. However we must say that the American Defense site does not always
provide the error page, in fact the application, based on ASP.NET technology for some areas, doesn’t
specify the custom error page (a simple setting in the web.config file in ASP.NET architectures); if a
not existing page has an extension different from“.aspx“, the error page is shown. The Italian Defense
site has good settings and all calls to not existing resources make the application show the default
error page.
• Multimedial errors: The US Defense site has several videos (YouTube embeeded); when data are
not loaded correctly, very often because of connection problems, a text advises the user that player
is unable to show content due to connection problems. The IT Defense site does not have videos or
multimedia content.
Apart from every specific case, the default error pages do not provide a detailed description of the
encountered problem, maybe the American site provides more textual information regarding possible
causes. The Italian site provides a link, in the default error page, for returning back to the home page; the
US Defense site, instead, does not provide a ruturn-to-home link, but links for going to the previous page,
contacting the web site administrator and accessing the advanced site search.
Element score
There is a different approach used by both sites for showing and managing errors; in this element exami-
nation, maybe, we can see a better implementation in the Italian web site (we are referring to the setting
error in the US Defense site), even if the default error page in the American defense application provides
more links (more usability). For these reasons we set the same score to both sites (although the American
site provides videos and the registration form differently by the Italian site which is very poor from this
point of view).
SCORE FOR US DEFENSE SITE IS: GOOD (2.0)
Searching in the web site
Pg. 23

Today every web site provides a search function. Some of them also implement advanced search tools to
allow the user to find what he wants in the application.
Element importance
Searching in a web site having so many information and data (like US Defense and IT Defense) is essential
for the user, for this reason we assign to this element the highest importance level: Determinant (1.0).
US Defense site search tool
Search tool is available to the user, at the beginning, in the top right side area of the page as a textbox
with a button for submitting the search string. It is possible to perform a simple search or an advanced
one by clicking the provided link near the search textbox.
Adavced search has many parameters: it is possible to choose where to search (area), which words should
be included and which should not be, it is possible to sort results by relevance and select also how many
results showing per page; in the end there is also an option for narrowing specific elements in the results
list. After inserting the string and submitting it, the results are whown as a list in Google style. There is
also the possibility to perform another search process within the results using the provided link on the
top and the bottom of the page. Results are paginated.
IT Defense site search tool
As for the US Defense web site the search tool is available in the same position in the page showing
a textbox and a button. There are two search modes: simple and advanced search, but the advanced
search is available only when a simple search is performed.
The advanced search have not as many parameters as the American defense site; it is possible to type
which words to search, where to search, the content type and the search type (full text or title). After
clicking on the search button, the results are shown in a simple list where only the item title is available
(no abstract, no narrowed words and no description is provided). The results are paginated (no possibility
to set how many records show per page).
Element score
Both sites implement a search engine able to return good results basing on a string inserted by the user.
Main differences, usability goals and misses rely in results quality and search options. Regarding the
quality and quantity of results: both sites return a lot of elements all related to the search string; so, from
this point of view, we can’t still say which search tool is better. Analyzing usability, we can notice a higher
level in US Defense web site: there are more options for search process, the user can choose pagination
settings and also the possibility to narrow words in the list, while all these aspects are not implemented
by the Italian defense website. For this reason we can say that, for this element, both web applications
reach a good score, but the US Defense web site obtains the highest one:
Functions correctness
Websites not always implement working functions, sometimes it is possible to encounter some proce-
Pg. 24

dural errors or problems. In this final part of the functionality examination, we’ll focus on this aspect.
Element importance
Errors and inconsistance are aspects of an application that should be avoided (it is not a case that an ap-
plication is always debugged before it is published/released). There is no more explaination needed to
justify why we give the highest importance level to this element: Determinant (1.0).
Errors, correctness and malfunctions
In the Italian defense web site, the flash version, when using the search functionality, the final results
consist in a list of numbers; probably, in the xml loaded document (the common way for interfacing Flash
with a database), the programmers forgot to get from the item ID, the corresponding title and data. Fur-
thermore, when clicking on an item, the Flash player tries to open a popup window for showing content,
but all browsers today don’t allow popups.
Regarding the US Defense site, there are no bugs or errors to report.
Of course, this does not mean that our analysis has covered all pages of both applications in order to to
look for malfunctions, we focused on the main areas and tried to analyze as many pages as possible. Do-
ing so, this is what we got.
Element score
US Defense site passed this“user functionality test“ because we didn’t find any error. IT Defense site, in-
stead, showed some problems in the Flash animated version (this implies a reduction in score).
Using the same procedure of before it is possible to calculate the dimensional score for the US Defense
site and the IT Defense site:
Pg. 25
( ) ( )
1 14 4
1 1 1 1
1
1,0 0,5 1,0 1,0 1,0 4,0 0,5 2,0 1,0 4,0 1,0 4,0
4,0 1,0 4,0 4,0 13,0
3,71
3,5 3,5
n n
i i i i i i
i i i i
D P P E P P E
− −
= = = =
−
   
= ⋅ ⋅ = ⋅ ⋅ =   
   
= + + + ⋅ ⋅ + ⋅ + ⋅ + ⋅ =
+ + +
= = =
∑ ∑ ∑ ∑
( ) ( )
1 14 4
1 1 1 1
1
1,0 0,5 1,0 1,0 1,0 1,0 0,5 2,0 1,0 3,0 1,0 3,0
1,0 1,0 3,0 3,0 8,0
2,28
3,5 3,5
n n
i i i i i i
i i i i
D P P E P P E
− −
= = = =
−
   
= ⋅ ⋅ = ⋅ ⋅ =   
   
= + + + ⋅ ⋅ + ⋅ + ⋅ + ⋅ =
+ + +
= = =
∑ ∑ ∑ ∑

Defense site has a higher usability level regarding the Functionality dimension; the difference with the IT
Defense site is considerable.
For a better vision of the situation, we can chart data and results evaluated up until now in a bar diagram
Pg. 26
0,0 0,5 1,0 1,5 2,0 2,5 3,0 3,5 4,0
Mean
IT Defense
US Defense

When the user navigates the pages of a website, very often, he is looking for something. In many cases
desired data relies in a textual information, and if text has not a good organization, the user will not be
able to find what he needs so easily. It is obvious that content management is essential when we exami-
nate a web application from the point of view of usability.
Content organization/labelling
Organizing and labelling content is a very important operation, doing so it is possible to individuate
some areas where data and information are related. This operation let the user find what he is looking for
fastly and easily.
Element importance
We have already talked about importance of this element, there no more words needed to explain why
we assign to this element the highest importance level: Determinant (1.0).
US Defense content organization
The American defense web site has a good content organization. Starting from the home page to all
other areas and secondary level pages, it is always possible to see a content tree somewhere with titles
and brief descrition or abstracts.
For example, in the home page, on the right side area, it is shown the latest news box with the main news
categories in bold, and all subsections (for each category) shown in smaller characters and highlighted
with a blue bright color.
In the leaders page, on the left side, content is organized in a small but very useful tree, showing main
categories and subsections under them. On the left side it is also possible to see the recent speeches box
showing a list of the most recent speeches made by the most important US Defense leaders; speech title
is bold and a brief description is provided in smaller gray characters under it.
Multimedia content is also well organized. In the multimedia page it is possible to choose a section
showing many photos, with their description, in a sorted list (by date) with pagination controls available
on the top and bottom parts of the list.
Furthermore, in every page, the content is always organized in the same way with the same graphic
style; this is important because usability level increases: the user recognizes the organization and already
knows how to search for what he needs.
IT Defense content organization
The Italian defense site organizes content differently.
At the beginning, entering the home page, the user dosn’t find a very structured content organization.
On the left side of the page it is possible to use the content menu providing an unsorted list of the main
areas of the site. Sure this is one of the worst approach to content organization because there is a com-
Evaluation for Content
Pg. 27

plete lack of labelling: content is not organized in labels or groups, there is just a simple list where the
user must read every item in order to find what he is looking for (if we consider that the list is also unsort-
ed, searching becomes very difficult).
In the other pages it is often shown a right side panel listing most recent news, but also in this case, the
list is not easy to consult: every item consists in a variable length text where news title and abstract have
the same character style and color; this approach is wrong because, when listing something, making the
title character style different from the remeaning text, makes easier for the user to better distinguish the
news itself.
Element score
Summarizing the evaluation of this element, we see very different situations in both sites, meaning that,
in this place, the US Defense site reaches high usability levels while the IT Defense site is able to obtain,
very difficultly, just the minimun score. This great disparity relies in the lack of a good content manage-
ment in the Italian defense web application.
SCORE FOR IT DEFENSE SITE IS: VERY BAD (0.0)
Text styling and web standard styles compliance
Some aspects regarding web sites text styling have already been described in the previous chapters.
Here we want to examinate how both sites style textual elements.
We also want to see if our wab applications are compliant to standard styling technologies like CSS; so,
we are going to check W3C compliance.
Element importance
Text has a structure and an organization defined by the content manager; in order to support this struc-
ture, it is possible to use advanced or even basic text styles. The higher is the styling quality, the higher is
the usability level: in fact, a good text structure let the user find faster and easier what he needs. For this
reason we set a medium-high importance level: Important (0,75).
Text styles
From the point of view of text styling, we have many differences between both web sites; focusing on
general settings we can see different colors, different text fonts, different text sizes, different effects and
layouts. By downloading CSS stylesheets of both applications, we can see what styling policy is used by
the two competitors.
The US Defense site’s CSS stylesheet defines Arial based font families with variable dimensions in the
range 11..15 pixels. There are other elements in the stylesheet used for styling titles, subtitles, links and
other textual elements in the page. It is possible to state, in general, that we have a great variety of dif-
ferent textual definitions, this let the application have a good usability level because the user is able to
distinguish different textual elements by their style (and different styles means different semantic refere-
ments).
The IT Defense site’s CSS stylesheet defines Verdana, Helvetica ans Sans-Serif based font families with
70% font size. There are a lot of definitions in the style sheet but very few of them focus on text, in fact
Pg. 28

this is the evidence explaining why the web site has not many textual styles.
W3C compliance
For a site, being compliant to standards defined by W3C sure is a good thing, also from the point of view
of usability. When a site is compliant with standard technologies, it is correctly viewable by almost all
browsers, this is an usability goal without any doubt.
Unfortunaltely, both sites show several problems when validated using a W3C HTML validator.
• US Defense site - XHTML 1.0 Transitional: Validator finds 60 errors and 44 warnings in almost all
pages; web site doesn’t pass validation.
• US Defense site - XHTML 1.0 Strict: Validator finds 63 errors and 44 warnings in almost all pages;
web site doesn’t pass validation.
• IT Defense site - XHTML 1.0 Transitional: Validator finds 6 errors and 1 warning in almost all pages;
web site doesn’t pass validation.
• IT Defense site - XHTML 1.0 Strict: Validator finds 12 errors and 1 warning in almost all pages; web
site doesn’t pass validation.
Summarizing these results, we can say that Italian defense web site is much more compliant to CSS stan-
dards.
Element score
On one side, the US Defense site has a higher usability level because of text styling, but on the other
hand it fails CSS validation. The IT Defense site lives the opposite situation: CSS validation is good, but it
lacks of textual styles. These considerations should justify why we assign the same score to both sites.
Information quality
We have described how content should be provided and shown to the user, but what about content
itself? What about the quality of provided information? This is the aspect we are going to focus on in this
paragraph.
Element importance
A website can be evaluated examinating various aspects regarding usability, but one aspect of usability
is the quality of information. Let us consider an example: a tourism web site describes to the user how to
reach Grand Central Station starting from Central Park (North-East corner) in New York City (NY - USA). If
directions we got are written in bad English, with errors, or they are just not easy to understand, we find
it difficult to follow them and arriving to Grand Central, we would propably get lost and be compelled to
ask for directions to someone on the road. Although a good graphic style, the content quality must be
high in order to meet users’needs, for this reason we assign to this element the highest importance level:
Determinant (1.0).
Pg. 29

Evaluating content quality
Both American and Italian Defense sites have a very good information quality level. To better examinate
how information is provided by the two web sites, we analyze content by section:
• News and press resources: Both sites expose content in very clear language. It is obvious that this
type of information is written by journalists and reporters, for this reason, consistency and grammar
correctness levels are very high. Information provided are useful and they all regard the present mili-
tary situation of the considered country (USA and Italy); there are articles for every argument: from
politics to military contingents spread all over the hottest world territories.
• Leaders biographies: Biographies of Italian and American leaders expose content synthetically and
in the clearest way. It is possible to read events in chronological order and the user, for US leaders, can
also read speeches of the most important people.
• Proceedings: Descriptions for the most important proceedings (for example how to be recruited,
how to take part to special exams and so son) are explained in clear language in the common institu-
tional register. Information provided are very useful for almost all kinds of users.
• FAQ: Although Italian defense web site does not provide a direct link to a FAQ area (to access this
page it is necessary to search the word:“FAQ“), both web sites have good Frequently Asked Questions
pages. IT Defense site provides a list with some questions (bold text) and the corresponding answer
(plain text) regarding post military service employment; descriptions are not much detailed but in
some answer, a link to a detail page is provided.
US Defense site has a much better page dedicated for Questions. It is possible to access three areas,
where the user can browse questions and answers, or making new questions adding some comments
or even accessing to personal questions made before. Questions and answers are styled differently
and they are paginated, it is also possible to search for a question or search by argument too. Provid-
ed content is very self-explaining and interactive, clicking on one question, not only the answer, but
also other information and links to other questions related to the chosen one are shown.
Element score
There is not much disparity between the two sites, we just need to point out that the US Defense site
reaches a better usability level because of the FAQ section having more options and more information
and data.
Website localization: multilanguage support
There are websites, operating in some particular contextes, that must provide support for more than one
language. In our case we are operating inside the institutional/military context: does a web site operat-
ing here need to provide support for other languages? The answer is yes and no. Our web sites (US and
IT defense) provide information to Italian and American people about military contigents, military opera-
tions, politicians and so on, but all these data regard the home territory, and in the home territory there is
Pg. 30

no need to have a different language from the local one. So at a first look we would state that a different
language is useless to be provided, but it would not be correct; there are users accessing the web site
other than citizens or American/Italian people: for example journalists, researchers, foreign people and so
on; they would probably like to read information provided in both sites.
Element importance
It is important to focus that an additional language is not such a strong requirement, our sites must first
provide a service for local people (USA or Italian citizens), and then, eventually, provide additional ser-
vices for other languages. For this reason we assign to this element a medium importance level: Consider-
able (0.5).
Localization in IT Defense and US Defense sites
There is nothing much to say about the US Defense web site, because there are no other languages other
than American in every page of the application.
The Italian defense web site provides support for English, French and German but in some pages only, for
example, the most important leaders’biographies are available in four languages (including Italian).
Element score
This is the first element where IT Defense site obtains a higher score than US Defense.
SCORE FOR US DEFENSE SITE IS: VERY BAD (0.0)
Using the same procedure of before it is possible to calculate the dimensional score for the US Defense
site and the IT Defense site.
Defense site has a higher usability level regarding the Content dimension; the difference with the IT De-
fense site is considerable.
Note that in this dimension both sites obtain a very low score, this is because there are elements where
they got a 0.0 usability level.
Pg. 31
( ) ( )
1 14 4
1 1 1 1
1
1,0 0,75 1,0 0,5 1,0 4,0 0,75 2,0 1,0 4,0 0,5 0,0
4,0 1,5 4,0 0,0 9,50
2,92
3,25 3,25
n n
i i i i i i
i i i i
D P P E P P E
− −
= = = =
−
   
= ⋅ ⋅ = ⋅ ⋅ =   
   
= + + + ⋅ ⋅ + ⋅ + ⋅ + ⋅ =
+ + +
= = =
∑ ∑ ∑ ∑
( ) ( )
1 14 4
1 1 1 1
1
1,0 0,75 1,0 0,5 1,0 0,0 0,75 2,0 1,0 3,0 0,5 2,0
0,0 1,5 3,0 1,0 5,50
1,69
3,25 3,25
n n
i i i i i i
i i i i
D P P E P P E
− −
= = = =
−
   
= ⋅ ⋅ = ⋅ ⋅ =   
   
= + + + ⋅ ⋅ + ⋅ + ⋅ + ⋅ =
+ + +
= = =
∑ ∑ ∑ ∑

Charting results it is possible to see the global situation from a better point of view. The bar diagram
shows both scores and the mean between them.
Pg. 32
0,0 0,5 1,0 1,5 2,0 2,5 3,0
Mean
IT Defense
US Defense

A web site can also be evaluated considering how well it is managed by developers, content managers
and web masters. The better is managed a web site, the better will be the usability level.
Site availability
Users accessing a web site sometimes are not able to reach the entire application or some part of it due
to unavailability situations. When a web site is unavailable (or even some part of it), the user cannot ac-
cess its services, this is sure a usability failure.
Element importance
Keeping available a web application is essential and very important, the user should always access al ser-
vices, this depends by many factors, one is, for example: maintainance. Importance level is: Determinant
(1.0).
Availability analysis
It is much difficult to see if a web site is always available. Unavailability situations can be noticed when
a web application is navigated day by day; in our case, we need much more time. Not having this time,
we perform a different operation: looking into forums we questioned people about the US Defense web
site availability and the Italian defense site availability. This approach returned some results only for the
American site (good results: the site is always reachable); we could not retrieve any information about the
IT Defense site in forums and chats; for this reason we can rely only on our visits and, doing so, no prob-
lems were noticed.
Element score
Although this element could not be processed properly, we assign a score to it basing on data we could
retireve; given this situation we assign to both sites the highest score.
SCORE FOR IT DEFENSE SITE IS: EXCELLENT (4.0)
Site monitoring
This is another element very difficult to evaluate because we need to be part of the development team of
both sites. We obviously cannot meet this condition and, in order to assign a score, we can try to exami-
nate this element operating differently.
In this case we sent to both sites’webmasters an email, asking for information about site monitoring ex-
plaining our intentions (a site analysis for academic and research purposes). Although we got an answer
Evaluation for Site Management
Pg. 33

by the two webmasters, they didn’t provide us any information regarding sensible data.
Element score
We cannot assign a score because of what said before. When evaluating the dimensional score, we’ll not
take into consideration this element.
SCORE FOR US DEFENSE SITE IS: ??
SCORE FOR IT DEFENSE SITE IS: ??
Site updating
A constantly updated web site sure provides high quality services, especially when the web application
exposes pages and areas like news or press releases.
Element importance
Our applications provide a lot of content that needs to be updated daily, for this reason this element is
assigned to a high importance level: Determinant (1.0).
Evaluating updating status
Updating status is a term used for referring to the hightest updating frequency of a site, it is like answer-
ing to the question:“How many times per day do you update the web site?“. In our case we must try to
obtain this data by looking at the web sites and seeing what changes everyday.
Focusing on the US Defense website we can notice, navigating everyday the web application, a high
updating level: the highlight images change every three or four hours , the main news are shown in rota-
tor mode so that every time the user can read the title of a different news, even if the set of news doesn’t
change in the same day. This pattern is almost the same in the pages providing access to the main areas
of the site, a set of images is shown in these pages and they rotate thanks to an interactive tool (Flash
player), the next day it is possible to see a different set of images. Also the textual content changes
sometimes in the most important areas, but somewhere it is static (for example in the pages dedicated to
leaders biographies).
Moving to the IT Defense site, things are very different and much more static. The web site, in fact, is
updated only in the news area; even the home page is not changed, the only image (the central news im-
age) has remained the same for days.
Element score
It is easy to assign a score for this element given the high updating level of the US Defense site and the
low level of the Italian defense site.
Broken links
Pg. 34

Element importance
Broken links represent a very common situation the user must face when navigating a web site in the
Internet, when a web application expose links that refer to non reachable locations, the result is an error
page (if present) avising the encountered problem, the user so must go back and choose another link
(hoping that it works). Such situations should be avoided by web developers, for this reason the impor-
tance level for this element is high: Important (0.75).
Searching for broken connections
It is very strange to say and even unexpected but, examinating the main and the most important areas of
both sites, we were not able to detect any broken kink. All links work fine.
Element score
Given our considerations, there is no need to not assign the highest score to both web sites.
Contacts and communication with users
Giving to users the possibility to communicate with web site managers or masters is a good thing. This
let the user find some explainations in the most difficult procedures in the application, and anyway,
when the user can communicate with someone on the other side, he perceives a good feedback and his
opinion on the web site is postive.
Element importance
Communicating with users and letting them contact someone behind the web application is an impor-
tant usability aspect, that’s why the importance level for this element is the highest: Determinant (1.0).
How can I communicate with someone behind the web application?
When trying to communicate with someone inside the US Defense web site, things get very easy. It is
unexpected such a well organized communication structure, but just accessing the“Contact Us“ section
of the site, it is possible to see a lot of options. The first step is locating the right link, in fact, depending
on who you are and what you seek, there is the specific link with the right information you need to con-
tact someone. There are links for every kind of information the user may ever need: links for journalists or
normal users, links for asking information to Department of Defense personnel or Petagon personnel; it is
also possible contacting personnel in the US Army, Navy, Marine Corps and Air Force, and there are links
even for veterans and retired civilians. But it’s not finished here. When writing an email (using often a
form provided by the web site, rather than using your own email composer) to any provided contact, an
autogenerated message appears in you mailbox at once, telling you that your email has been processed
and that very soon an operator will send you a message trying to answer your questions or providing the
information you asked; and after few hours the promised email comes.
Moving to the Italian Defense site, everything changes. Unfortunately, there is not much to say: the web
site does not provide any informatic way for contacting someone behind the site. There is not an email
Pg. 35

or a commnunity web site, there are just links to other sites (and even there, the user finds few emails)
and lots of phone numbers. This represents a very bad situation becuse the user cannot contact anybody
utulizing his own computer.
Element score
For this element, the US Defense site undoubtly reaches the hight score, while the IT Defense site doesn’t
meet any requirement, for this reason, score is zero.
SCORE FOR IT DEFENSE SITE IS: VERY BAD (0.0)
Using the same procedure of before it is possible to calculate the dimensional score for both sites:
The American Defense site has a higher usability level regarding the Management dimension.
Pg. 36
( ) ( )
1 14 4
1 1 1 1
1
1,0 1,0 0,75 1,0 1,0 4,0 1,0 4,0 0,75 4,0 1,0 4,0
4,0 4,0 3,0 4,0 15,0
4
3,75 3,75
n n
i i i i i i
i i i i
D P P E P P E
− −
= = = =
−
   
= ⋅ ⋅ = ⋅ ⋅ =   
   
= + + + ⋅ ⋅ + ⋅ + ⋅ + ⋅ =
+ + +
= = =
∑ ∑ ∑ ∑
( ) ( )
1 14 4
1 1 1 1
1
1,0 1,0 0,75 1,0 1,0 4,0 1,0 1,0 0,75 4,0 1,0 0,0
4,0 1,0 3,0 0,0 8,0
2,46
3,25 3,25
n n
i i i i i i
i i i i
D P P E P P E
− −
= = = =
−
   
= ⋅ ⋅ = ⋅ ⋅ =   
   
= + + + ⋅ ⋅ + ⋅ + ⋅ + ⋅ =
+ + +
= = =
∑ ∑ ∑ ∑
0,0 0,5 1,0 1,5 2,0 2,5 3,0 3,5 4,0
Mean
IT Defense
US Defense

We are going to evaluate both applications from the point of view of accessibility in order to see how
much reachable are the web sites or if disabled people can navigate the applications.
Site reachability
When we look for a web site, if we don’t know the site path or complete name, we try to use a web search
engine in order to find it using some keywords. In our case these keywords would probably be: Ameri-
can defense, Italian defense, Ministro della difesa, Secretary of defense, defense, difesa, military defense,
difesa militare, homeland security, sicurezza nazionale and so on. Do the two sites appear among the first
six or ten entries of the final results of the most common and famous search engines?
Element importance
This element is very important because people who never visited both sites, can reach them without
even knowing their complete names; this represents sure a good usability aspect. Importance level is:
Determinant (1.0).
Searching for US Defense and IT Defense sites
To test reachability for both sites, let us take into consideration these search keywords:
• Keywords K1: American, Defense | Difesa, Italiana
• Keywords K2: USA, Homeland, Security | Sicurezza, Nazionale, Italiana
• Keywords K3: USA, Department, Defense | Ministero, Difesa, Italiana
• Keywords K4: USA, Secretary, Defense | Ministro, Difesa, Italiana
• Keywords K5: USA, Military, Defense | Difesa, Militare, Italiana
For every group of keyword (KX) we will perform a search using one of these engines: Google, Microsoft
Bing, AltaVista, Yahoo Search and InfoSeek Japan. Testing every group of words, we’ll annotate the posi-
tion where the web site appears in order to understant its reachability.
After doing as many searches as for the number of keyword groups (for every web search engine), we’ll
build a summarization table showing on rows the search engines, and on columns the keyword groups
using KX notation.
In every cell it is shown the positions where the web site appears, we can have one position or more de-
pending on how many entries the web site is visualized in the results list.
Let us first take into consideration the US Defense site:
Evaluation for Accessibility
Pg. 37

K1 K2 K3 K4 K5
GOOGLE Not found Not found 1st*; 3rd 1st*; 3rd; 13th* 1st*; 4th
MS BING Not found Not found 2nd*; 3rd; 7th* 1st; 4th*; 6th 5th*; 12th
ALTAVISTA Not found Not found 2nd; 3rd*; 5th* 2nd*; 3rd; 4th* 3rd; 4th*; 9th*
YAHOO Not found 33rd* 1st*; 2nd*; 3rd 3rd; 4th; 6th* 1st*; 4th; 5th
INFOSEEK (JP) Not found Not found 17th* Not found Not found
Note: for elements with * it means that shown link adresses the user to the subdomain www.defenselink.mil
used by www.defense.gov as a support domain.
Moving now to the Italian defense site:
K1 K2 K3 K4 K5
GOOGLE 1st; 2nd*; 3rd* Not found 1st; 2nd*; 3rd* 1st; 2nd*; 3rd*
1st*; 2nd*;
3rd*
MS BING 1st; 2nd*; 6th Not found 1st; 3rd*; 7th 1st; 3rd* 1st*; 2nd; 6th*
ALTAVISTA 1st; 2nd; 6th* 5th 1st; 3rd; 4th* 1st; 3rd 1st; 2nd; 3rd*
YAHOO 1st Not found 1st; 3rd* 1st 1st*; 5th*; 9th
INFOSEEK (JP) 1st Not found 1st; 2nd; 3rd* 1st
1st*; 2nd*;
6th*
Note: for elements with * it means that shown link adresses the user to a subdomain of www.difesa.it for ex-
ample: www.esercito.difesa.it or www.marina.difesa.it and so on.
Element score
This is an unexpected result: the Italian defense web site is more reachable (even in the Japanese area)
than the American one. American defense gets 14 matches while Italian defense gets 21 matches in all
search engines.
Browser compatibility
If someone questioned a web developer:“What’s the thing you are most worried about?“, the poor
developer would probably answer you:“Web browsers“. It is simple, having a web site compatible with
all browsers (meaning a site that can be viewed in the same identical way indipendently by the browser
utilized) is a great usability goal. All over the world there are more than 7.5 billions people and they sure
don’t use the same browser; letting a site be visualized almost in every browser is very difficult, but if this
requirement is satisfacted, usability level increases a lot.
Pg. 38

Element importance
This element is very important because people don’t use the same browsers. Given this situation, it is
necessary to build web sites trying to let different users see always the same graphic structure. For this
reason we assign the highe importance level: Determinant (1.0).
How are viewed our websites in the most famous browsers in the world?
We must try to be clear and examinate every site’s characteristic in some browsers. We choose these
ones:
• Microsoft Internet Explorer 8.0
• Mozilla FireFox 3.0
• Apple Safari 4.0
The characteristics we want to analyze regard the most common graphical aspects: tables, divs, fonts,
form controls and so on.
Let us start with the American defense site:
INTERNET EXPLORER FIREFOX SAFARI
TABLES MANAGEMENT Correct view Correct view Correct view
DIVS MANAGEMENT Correct view
Some divs overlap on
others hiding some
links and text
Some divs overlap on
others hiding some
links and text
FONTS AND TEXT Correct view
Text aligned to image
is rendered differently,
not respecting condi-
tions on margins
Correct view
LINKS Correct view Correct view Correct view
FORM CONTROLS Correct view Correct view Correct view
MULTIMEDIA Correct view Correct view Correct view
BACKGROUNDS Correct view Correct view Correct view
Now let us move to the Italian defense site:
TABLES MANAGEMENT Correct view Correct view Correct view
DIVS MANAGEMENT Correct view Correct view Correct view
Pg. 39

FONTS AND TEXT Correct view
Text margins and text
wrap are different. Bold
characters are greater.
Some fonts are ren-
dered differently.
Correct view
LINKS Correct view Correct view Correct view
FORM CONTROLS Correct view
Borders are rendered
differently (solid and no
3d effect like others)
Correct view
MULTIMEDIA Correct view Correct view Correct view
BACKGROUNDS Correct view Correct view Correct view
Element score
Well, from the point of view of browser compatibility, both sites have few problems with FireFox. Apple
and MSIE view the sites almost in the same way. For this reason we assign to both applications the same
score.
Accessibility
Here’s a very huge problem for many web sites. Today, providing site accessibility for disabled people has
become a very important requirement, but not all web sites provide services in this direction. There are
many ways for making a web application usable also for people that have not our same skills; in our case
we’ll check for our web applications to be compliant to some simple standards, in particular those ones:
• WAI/WCAG: They are guidelines defined by W3C and they are still working on it.
• Vocal synthesis guidelines: There are special programs that use vocal synthesis for speaking textual
content of Internet pages; in order to let these services work on web pages, it is necessary to pay
attention to few things, even taking care of small details it is possible to provide a better service, for
everyone, for real.
Element importance
We assign to this element the highest importance level: Determinant (1.0).
Accessibility validation
We want to evaluate both sites using some guidelines obtained by merging WAI and other important
guidelines:
• Color blindness: Some people cannot see colors in the same way like everyone because of many
Pg. 40

color deficiencies. It is possible to pay attention in web sites, and using good colors in order not to
confuse those people affected by this inability. In this place we’ll focus on two important known defi-
ciencies: Protanopia and Deuteranopia, two forms of Dichromacy, the inability of distinguishing two
close colors placed in the same chromatic range (red, green or blue). In order to avoid this problem
it is required not to place two similar colors togheter in the same page, for example the orange near
to red, if these colors are used to distinguish two different types of information, the user, affected by
Dichromacy, will not notice the difference.
• Partially sighted and blind people: There are people who cannot see and people who can see but
very difficultly; life for them is much difficult because they don’t have the most used sense: sight. In
order to let them use special programs based on vocal synthesis (text to sound), web sites should
respect web standards and following these rules:
• When listing items, use <ol> or <ul> and <li> tags.
• Use always CSS to format text (at every level: external file, page level or inline level).
• Use spans and divs, and in every image always remember to place the alt attribute.
• Do not use tables for defining pages layouts.
• When defining top bars or menus, use HTML horyzontally or vertically styled list tags.
• Use attribute alt wherever it is allowed.
• Do not use multimedial components (for example Flash player) for showing important infor-
mation.
• Do not use images for defining buttons or creative content.
• Provide every anchor <a> with the attributes rel and title correctly set.
These few rules can help building accessible web sites. We are going to check whether our web applica-
tions are accessible by all types of users, by following these guidelines.
Let us focus on the American and Italian defense web sites:
DEFENSE.GOV DIFESA.IT
COLORS COMBINATION
Blue background and white fore-
ground is a good combination but
the top bar combines different
shades of blue (it is not a great prob-
lem because the white text gener-
ates a very good constrast after all).
Background/Foreground combina-
tion is absolutely wrong because
these colors will be merged by
people affected by Dichromacy.
Generally there is a very bad choice
of colors, the entire page itself is flat-
tened when seen by people affected
by color blindness because of the
lack of contrast: there is a unique
color range: gray/light blue.
Pg. 41

DEFENSE.GOV DIFESA.IT
HTML LISTS OK OK
CSS STYLES OK OK
SPANS AND DIVS OK OK
TABLES
The top area (background top im-
age, logo and search textbox) is
structured using a table. This could
be avoided using divs.
OK
MENUES AND TOP
BARS
OK OK
IMAGES OK OK
MULTIMEDIA
In the home page there is an images
rotator, apparently it might seem a
Flash component, but it is a javas-
cript/html ASP.NET ASHX control: it
does not create any problem.
There are no multimedial compo-
nents.
BUTTONS OK OK
LINKS Rel attribute is not specified in links. Rel attribute is not specified in links.
Element score
Although final result were expected to be not very good, we must say the contrary. Both sites have a
good usability level for disabled people even if some rule is not respected, we reach good scores.
Using the same procedure of before it is possible to calculate the dimensional score for both sites:
The IT Defense site has a higher usability level (it is the first time) regarding the Accessibility dimension.
Pg. 42
( ) ( )
1 13 3
1 1 1 1
1
1,0 1,0 1,0 1,0 2,0 1,0 3,0 1,0 3,0
2,0 3,0 3,0 8,0
2,66
3,0 3,0
n n
i i i i i i
i i i i
D P P E P P E
− −
= = = =
−
   
= ⋅ ⋅ = ⋅ ⋅ =   
   
= + + ⋅ ⋅ + ⋅ + ⋅ =
+ +
= = =
∑ ∑ ∑ ∑
( ) ( )
1 13 3
1 1 1 1
1
1,0 1,0 1,0 1,0 4,0 1,0 3,0 1,0 3,0
4,0 3,0 3,0 10,0
3,33
3,0 3,0
n n
i i i i i i
i i i i
D P P E P P E
− −
= = = =
−
   
= ⋅ ⋅ = ⋅ ⋅ =   
   
= + + ⋅ ⋅ + ⋅ + ⋅ =
+ +
= = =
∑ ∑ ∑ ∑

This is the first time that the Italian defense site gets a higher score than the American one.
Pg. 43
0,0 0,5 1,0 1,5 2,0 2,5 3,0 3,5
Mean
IT Defense
US Defense

We are now ready to resume every dimension analyzed up until now in a single chart and in a single
number.
Global evaluation
Global diagram for US Defense web site Global diagram for IT Defense web site
The chart on the left shows a
global vision (all dimensions) for
the US Defense site.
Almost all dimensions have good
levels, but we can see that Local-
ization is the Achilles’heel of this
application.
Regarding everything else, there
are no problems at all.
The chart on the right shows a
global vision (all dimensions) for
the IT Defense site. If compared to
the US defense chart, we can see a
totally different condition. Almost
all dimensions have low levels,
there are just two dimensions that
reach the highest level. Generally
speaking, usability is not good.
Pg. 44
0,5
1,0
1,5
2,0
2,5
3,0
3,5
4,0
Accessibility
Management
Content
Functionality
Communication
Architecture
0,5
1,0
1,5
2,0
2,5
3,0
3,5
Accessibility
Management
Content
Functionality
Communication
Architecture
0,5
1,0
1,5
2,0
2,5
3,0
3,5
4,0
Accessibility
Compatibility
Reachability
Contacts
Broken links
Updating
Availability
Localization
Info quality
Text styles
Labelling
Correctness
Search
Errors
Functions
Graphics
Brand image
Home page
External links
Site map
Navigation
Site structure
0,5
1,0
1,5
2,0
2,5
3,0
3,5
4,0
Accessibility
Compatibility
Reachability
Contacts
Broken links
Updating
Availability
Localization
Info quality
Text styles
Labelling
Correctness
Search
Errors
Functions
Graphics
Brand image
Home page
External links
Site map
Navigation
Site structure

Comparative analysis on E-Gov web sites

Comparative analysis on E-Gov web sites

Recommandé

Recommandé

Contenu connexe

En vedette

En vedette (17)

Similaire à Comparative analysis on E-Gov web sites

Similaire à Comparative analysis on E-Gov web sites (20)

Plus de Andrea Tino

Plus de Andrea Tino (20)

Dernier

Dernier (20)

Comparative analysis on E-Gov web sites