Testing Web Based Applications[1]

Testing
QUICK LOOK
s Why errors in Web-based

Testing
applications are hard to reproduce
s Analyzing these errors in a way
that saves testing time

Web-based
Applications
Analyzing and reproducing errors in a Web environment
by Hung Q. Nguyen

he testing of Web-based done … especially in Web environ-

T 1
applications has much in ments. Web environments are dense
common with the testing with error-prone technology vari-
of desktop systems: You ables. Here are five fundamental con-
need to test the usual func- siderations of Web-application test- What are we really
tionality, configuration, ing:
and compatibility, as well as perform- seeing–an error
ing all the standard test types. But 1. When we see an error on the
Web application testing is more diffi- client side, we are seeing the or a symptom?
cult because complexities are multi- symptom of an error—not the er-
plied by all the distributed system ror itself. Without diagnosing the environment,
components that interact with the ap- we can’t be certain what causes a
plication. When we see an error in a 2. Errors may be environment-de- symptom to appear. If one of the en-
Web environment, it’s often difficult pendent and may not appear in vironment-specific variables from ei-
to pinpoint where the error occurs, different environments. ther the client side or the server side
and, because the behavior we see or is removed or altered, we might not
the error message we receive may be be able to reproduce the problem.
3. Errors may be in the code or in
the result of errors happening on dif- Here is an example. I’m testing a
the configuration.
ferent parts of the Web system, the er- Web-based defect tracking applica-
ror may be difficult to reproduce. So tion, and going through the process
how do we analyze errors within a 4. Errors may reside in any of sever- of creating a new bug report. When I
Web-based system, and what consid- al layers. select the NEW button, I receive an
erations should be made for reproduc- error message:
ing such errors? 5. Examining the two classes of op-
When we have an understanding erating environments — static Microsoft OLE DB Provider for ODBC
of the underlying technology, we are versus dynamic—demands dif- Drivers error '80040e14'
better able to maximize testing effi- ferent approaches.
ciencies—writing more reproducible After spending some time investigat-
bug reports and finding more errors Now let’s take a look at each of these ing my browser environment, I dis-
in less time. This is easier said than five considerations in more detail: cover that JavaScript is disabled in
23
May/June 2000 Software Testing & Quality Engineering www.stqemagazine.com

the browser preferences dialog box. operating environment. More com- server configuration issue. However,
Enabling JavaScript eliminates the monly, we refer to environment-in- if the installation program failed to
error. (Whether or not this is a bug is dependent errors as functionality- programmatically configure the Web
not part of this discussion.) The idea specific errors. server according to specification,
here is that if I add additional infor- then this is a software error. If a
mation regarding the JavaScript set- system administrator fails to proper-
ting to the bug report, I can save our ly configure the Web server accord-
team some time in analyzing this
problem. Furthermore, “disabling
JavaScript” is added to my test suite
3 ing to specification, this then be-
comes a user error.

from this point on; it will be applied
to all areas of the application so that
Is it a coding error Application directory has not been con-
figured properly to execute scripts A
all potentially related errors can be
uncovered.
or a configuration typical application-server directory
contains scripts to be executed when
problem? they are called by a Web server on
the behalf of a client. For security
Errors (or the symptoms of sup- reasons, a Web server can be config-

2
To reproduce an environment-dependent error
Is the error
environment- we have to perfectly replicate both the exact sequence
dependent? of activities and the environment conditions.
To reproduce an environment-de-
pendent error we have to perfectly posed errors) may be resolved with ured to allow or disallow scripts to
replicate both the exact sequence of code fixes (assuming the errors are be executed within certain directo-
activities and the environment con- in fact real) or system reconfigura- ries. If your application-server direc-
ditions (operating system, browser tion (client, ser ver, or network). tory is designed to contain scripts
version, add-on components, data- Don’t jump too quickly to the conclu- that will be executed—but the Web
base server, Web server, third-party sion that it’s a bug! server is configured to disable script
components, ser ver/client re- execution in that directory—the ap-
sources, network bandwidth and Microsoft OLE DB Provider for ODBC plication will not work. Is this a soft-
traffic, etc.) in which the application Drivers error '80004005' ware error or a configuration
operates. For example, when you try problem?
to log into your Web application Here is an example illustrating the
while using a 28.8 kbps dial-up con- challenge of identifying possible Default Web page has not been set up
nection, you experience login fail- configuration problems as opposed properly The issue is similar to the
ures due to timeout in the authenti- to actual software errors. It shows problem above.
cation process—but the same login an error message caused by a
steps will authenticate successfully “failed login” that has been generat- SQL Server is not running The appli-
if you are on a T-1 connection at ed by a Web application. By simply cation server needs to connect to the
1.54 mbps. In this case, you have looking at this error message, it is backend database living on the SQL
an environment-dependent error impossible to determine whether server in order to execute queries,
where the dependency is in the this error is the result of a software store procedures, and access data. If
bandwidth. bug, a server-side configuration is- the SQL server process itself is not
Environment-independent er- sue, a compatibility issue, a browser running, then obviously the applica-
rors, on the other hand, are relatively configuration issue, or all of the tion will not work.
easier to reproduce—it’s not neces- above.
sary to replicate the operating envi- After further analyzing the fail- DLL/COM objects are missing or were
ronment. With environment-inde- ure, I discover several possible con- unsuccessfully registered Perhaps the
pendent errors, all that need be ditions that might generate this error installation program failed to copy
replicated is the steps that reveal the message: all the DLLs used by the application
error. For example, if the company server during setup. If any DLL need-
name is misspelled on all of the prod- IIS (Web server) virtual directory has not ed by the application server is miss-
uct’s online pages as WebTessting. been set up properly When the virtual ing, the application will not work.
Con, you will always see this error— directory is not properly configured, Perhaps the installation program
independent of the hardware, soft- the requested files, scripts, or data correctly copied all the needed mod-
ware, and resource variables in your will not be found. Typically, this is a ules, but failed to register one or
24
www.stqemagazine.com Software Testing & Quality Engineering May/June 2000

more of them. For example, with OLE- ty issues that are similar to PC envi-
based objects such as COM or DCOM,
their class ID (CLSID) must be regis-
4 ronments, where all components are
in one box. Issues multiply within
tered in the Registry Database before client/server systems, however, be-
they can be used. If an application Which layer cause there may be many clients and
tries to access a COM object that was ser vers connected on a network.
not registered successfully, the appli- really causes Typical client/server configuration
cation will not work. and compatibility issues involve the
This problem is often caused by the problem? hardware and operating system mix
errors in the installation procedures. (UNIX-based boxes versus Windows-
If, on the other hand, the compo- Errors in Web systems are often dif- based boxes, for example) and the
nents must be manually registered ficult to consistently reproduce be- software mix on the server side (Web
then this becomes a configuration cause of the many variables intro- server packages, database server
issue. duced by the distributed nature of packages, firewalls, COM objects,
client/server architecture (i.e., serv- CORBA objects, etc.). Issues may also
Browser-side JavaScript setting has er, client, and networking compo- involve the software mix on the
been disabled This is a browser-side nents). There are at least three usu- client side (TCP/IP stacks, dialer
configuration problem since the ap- al suspects in a Web environment: software, helper components, brows-
plication requires the browser to The client, the server, and the net- er brands, and browser versions).
have JavaScript enabled. Is this a work. Additionally, browser settings, such
software error, a configuration prob- Both the client and the server as general settings, connection set-
EDIT PASS 05/21/99

lem, or a technical support issue? carry configuration and compatibili- tings, security settings (including Ac-

Making Your
Web Application Test Report
More Reproducible
s Check if the client operating system, versions, and patches s Check for proper registration of components (COMs,
meet system requirements Java, etc.)

s Check if the correct version of the browser is installed on s Check to ensure that DNS is properly configured
the client machine
s Check if firewall configuration is causing packets to drop
s Check if the browser is properly installed on the machine or blocking access
(for example, the JVM is also successfully installed)
s Check if a slow connection is causing the application to
s Check the browser settings time-out

s Try the same set of steps with different browsers (e.g., s Check for potential race or time-related conditions
Netscape Navigator versus Internet Explorer)
s Check for potential network inaccessibility issues on the
s Try the same set of steps with different supported versions client machines
of the same browsers (e.g., 3.1, 3.2, 4.2, 4.3, etc.)
s Check for potential network inaccessibility issues on the
s Check to ensure that all servers are running server machines

s Check to ensure that all service-based components have s Check if the server operating system version and patches
been started meet system requirements

s Check to ensure that application access privileges are s Check if the proper versions of the server software such
CREDIT CREDIT CREDIT

properly set up as Web server, SQL database, and other middle-ware
packages are installed
s Check for missing components on the server (DLLs,
scripts, etc.) s Check server configurations for proper settings

25

Dynamic Environments (i.e., resource
Physical Server and time-related errors) in which
Client Client
Database otherwise compatible components
Server may exhibit errors due to memory-
Web related errors and latency condi-
Server tions. (We’ll discuss dynamic envi-
Application ronments in more detail later in this
Server section.)

Static Operating
Ethernet
Environment: Configuration
and Compatibility Variables
Configuration and compatibility issues
FIGURE 1 Web server, application server, and database server in one box
may occur at any point within a Web
system: client, server, or network.
Physical Server 1 Configuration issues involve various
Client Client server software and hardware set-ups,
browser settings, network connec-
tions, and TCP/IP stack set-ups. The
Database
Server browser setting/JavaScript example
discussed earlier illustrated one type
of configuration issue. A different type
of configuration issue is shown in Fig-
ures 1 and 2, with two possible physi-
Ethernet cal server configurations: one-box and
two-box configurations.
Our sample application under
test has some charting capabilities
Web
Server that enable a user to generate met-
rics reports, such as bar charts and
line charts. When a user requests a
Application metrics report, the application server
Server
pseudo code runs as follows:
Physical Server 2
1. Connect to the database server
and run the query.
FIGURE 2 Web server and application server in one box; database server in
another box 2. Write the query result to a file
named c:tempchart.val

tiveX controls, plug-ins, Java, script-
ing, downloads, user authentication, 5 3. Execute the Chart JavaApplet.
Read from c:tempchart.val and
etc.), content settings, program set- use the data to draw a graph.
tings, and other advanced settings
(including browsing options, multi-
Static and 4. Send the JavaApplet to the
media options, Java VM options,
printing options, and HTTP options)
dynamic operating browser.

introduce a multitude of variables
that should be tested and included in
environments During testing for this application, I
discovered that the charting feature
analyses.
The network offers another set
are different. worked on one of the above configu-
rations, but not the other. After I in-
of variables. The network affects the In general, there are two classes of vestigated further, I learned that the
Web application in several ways, in- operating environments—each with problem only occurred in the two-
cluding timing-related issues (race its own unique testing implications: box configuration. After examining
conditions, performance, time-outs, the code, I realized that the problem
etc.) due to bandwidth and latency, Static Environments (i.e., configura- is in steps 2 and 3. In step 2, the
potential configuration and compati- tion and compatibility errors) in quer y result is written to
bility issues due to hardware devices which incompatibility issues may ex- c:tempchart.val of the database serv-
ANNIE BISSETT

such as gateways and routers, and ist regardless of variable conditions er local drive. In step 3, the Chart
side effects related to security imple- such as processing speed and avail- JavaApplet is running on the applica-
mentations. able memory. tion server that is not in the same
26

I am not suggesting that we read the code ment cannot be replicated (due to
its dynamic nature), the error be-
comes irreproducible or hard-to-re-
every time we come across an error...I merely produce.
By the way, this is the reason
want to point out that it is essential to that memory-related errors are often
hard to reproduce. When a memory-
identify which server configurations are problematic, overwrite error exists in the code,
for example, it will always cause a
memory-overwritten problem. How-
and include such information in bug reports. ever, from a black-box testing per-
spective, we will never have a chance
I would also run a cursory suite of test cases to see the symptom of this error until
the specific overwritten byte(s) of
code or data is executed or read. In
on all distributed configurations that are supported this example, the set of steps repre-
sents the exact set of black-box ac-
by the application server under test. tivities. The memory-overwrite error
represents the actual error in the
code. The condition in which the
box with the database server. When each time a test procedure is execut- overwritten byte is executed or read
it attempts to open the file ed, it causes the operating environ- represents the dynamic operating en-
c:tempchart.val on the application ment to become dynamic. The vironment or condition needed to re-
ser ver local drive, the file is not attribute can be anything from reveal (reproduce) the error.
there. source-specific (available RAM, disk Here is a Web application exam-
In this case, I am not suggesting space, etc.) to timing-specific (net- ple of a dynamic environment-related
that we read the code every time we work latency, the order of user trans- error in which we will examine a
come across an error; I leave the de- actions being submitted, etc.). time-related error. The specification
bugging work for the developers. I When a test case depends on requires that:
merely want to point out that it is es- the exact replication of both the set
sential to identify which server con- of steps and the operating envi- s Project names within the system must be
figurations are problematic, and in- ronment but the operating environ- unique
clude such information in bug
reports. I would also run a cursory
suite of test cases on all distributed The home directory path for the Web server on the host myserver is mapped to:
configurations that are supported by C:INETPUBWWWROOT
the application server under test.
Compatibility issues are also When a page is requested from http://myserver/ data will be pulled from:
important in static operating envi- C:INETPUBWWWROOT
ronments. As an example, in Figure 3
we see a compatibility difference be- A filename (mychart.jar) is stored at C:INETPUBWWWROOTMYAPPBIN.
tween Netscape Navigator and Inter-
net Explorer. The application session path (relative path) is pointing to
This is not to say that Internet C:INETPUBWWWROOTMYAPPBIN, and a file is requested from .LIB.
Explorer is better than Netscape
If I use Internet Explorer version 3.x, the Web server looks for the file in
Navigator; it simply means that there
C:INETPUBWWWROOTMYAPPBINLIB because the browser relies on the relative
are incompatibility issues between
paths. This is the intended behavior and the file will be found; this tells me that my
browsers—and that the code should
application will work as expected using Internet Explorer 3.x.
not assume that relative paths work
for all browsers. More importantly, it If instead I use Netscape Navigator version 3.x (a browser that doesn’t like .), the Web
suggests that when you experience server defaults to C:INETPUBWWWROOTLIB and tries to look for mychart.jar from
an error in one environment, the there instead. This is a problem for this particular application because the file
same error may not appear in a dif- (mychart.jar) will not be found there—so I know this feature will not work using Netscape
ferent environment if it’s an environ- 3.x.
ment-dependent error.
When I brought up the Java Console, I saw the following, which confirmed my finding:
Dynamic Operating #Unable to load archive
Environment: Things http://myserver/lib/mychart.jar:java.io.IOException:<null>
Don’t Stay the Same
When the value of a specific environ-
ment attribute does not stay constant FIGURE 3 Compatibility issue between browsers
27

s Error detection and handling for potential
duplication be performed on the client- ...
side using JavaScript <td width=quot;80quot; bgcolor=#00CCCC>  </td>
<td width=quot;80quot; bgcolor=#00CCCC align=quot;leftquot; height=quot;9quot;>
s Users will be able to add or delete project
<font size=1 face=quot;Arialquot; color=quot;#400040quot;>
names by requesting the Setting Up Proj-
ects page Project:<br></font>
<select name=quot;namelistquot; size=quot;9quot; OnChange=quot;ListSelected()quot;>
s When a user creates a new project name, <option value=quot;Anotherquot;>Another</option>
a browser-side JavaScript checks the in- <option value=quot;NewProjquot;>NewProj</option>
put name against the select list embed-
</select></td>
ded in the HTML page (as illustrated in
Figure 4) <td width=quot;100quot; bgcolor=#00CCCC>  </td>
...
Take a look at the time-related error
illustrated in Figure 5. These before
and after screenshots of the Setting FIGURE 4 Browser-side JavaScript checks the input name against the values in
Up Projects page illustrate that the the namelist
application failed to detect the dupli-
cate name “Doomed.” Figure 4 walks
BEFORE

you through the explanation of this
time-related error that involves two
users adding new project names to
the same database.
As illustrated in Table 1, User A
and User B create new projects si-
multaneously, but without knowl-
edge of each other’s actions. In step
3, User A adds a project named An-
other. Since that project name
already exists, his browser’s
JavaScript displays a message
prompting him for a different proj-
ect name.
User B adds a project named
Doomed. Her browser’s JavaScript
does not detect Doomed as a preexist-
ing project name and so adds it to
both the database and the returned
AFTER

list. The updated project name list is
sent back to User B.
User A subsequently adds the
same name, Doomed, to the project
list. His browser’s JavaScript does
not detect the name on the HTML list,
so it adds the name Doomed to the
database again—as well as to the re-
turned list. The updated project
name list is sent back to User A with
two Doomed entries included.
This result fails to meet the
product’s specification. Unless this
situation happens to be a well-de-
signed test case, accidentally discov-
ering this error and attempting to re-
produce it is not a simple task. In
this example, the actual error is in
the failure of the application to
check for ser ver-side duplicate
names (in addition to client-side FIGURE 5 TOP : Before the project name “Doomed” has been entered by the user;
checking). The steps include User A’s BOTTOM : After the application failed to detect the duplicate project named “Doomed”

28

activities. The dynamic operating en- fect your ability to replicate errors. are investigating, but also the undis-
vironment is created by User B’s ac- With the application of some of the covered errors that are related to
tivities—which are hidden or un- skills covered in this article, I hope them. STQE
known to User A. that your Web testing experience will
be less frustrating and more enjoy- Hung Q. Nguyen (hungn@logigear.
able. com) is the president and CEO of
In Conclusion Remember that nothing will re-
place your testing skills—your ability
LogiGear Corporation, a full-ser-
vice consulting firm offering out-
To be effective in analyzing and re- to come up with good test cases, ask sourced testing, QA training, and
producing errors in a Web environ- relevant what-if questions, keep TRACKGEAR™ (a Web-based de-
ment, you need to have a command careful notes, and methodically in- fect tracking solution). He is co-au-
over the operating environment. You vestigate hard-to-reproduce errors. It thor of Testing Computer Software
also need to understand how envi- is these skills that will assist you in and author of the soon-to-be-pub-
ronment-specific variables may af- finding not only the errors that you lished Testing Web Applications.

STEP REQUEST HTML LIST BEFORE HTML LIST AFTER D ATA B A S E B E F O R E D ATA B A S E A F T E R

1 USER A gets the Another Another Another
Setting Up Project NewProj NewProj NewProj
Page
2 USER B gets the Another Another Another
Setting Up Project NewProj NewProj NewProj
Page
3 USER A adds a Another Another Another Another
new project named NewProj NewProj NewProj NewProj
quot;Anotherquot;
4 USER B adds a Another Another Another Another
quot;Doomedquot; Doomed Doomed
5 USER A adds a Another Another Another Another
quot;Doomedquot; Doomed Doomed Doomed
Doomed Doomed

TABLE 1 User A and User B activities

29

Testing Web Based Applications[1]

Recommandé

Recommandé

Contenu connexe

En vedette

En vedette (9)

Similaire à Testing Web Based Applications[1]

Similaire à Testing Web Based Applications[1] (20)

Plus de MBA_Community

Plus de MBA_Community (9)

Dernier

Dernier (20)

Testing Web Based Applications[1]