1. Article Critique 17 Nov 2008
EDS 502 (RESEARCH METHODS IN EDUCATION)
Comparing EFL Learners'
Reading Comprehension Between
Hypertext and Printed Text
Tseng, M. (2008). Comparing EFL learners' reading comprehension between hypertext and
printed text. CALL-EJ Online, 9(2)
1

2. Critique in a Few Words
Summary of the Article
1.
Critique (+/- points)
2.
Recommendation
3.
Implications of the study
4.
Summary
5.
2

3. Before summary:
3

4. Summary of the Article
What are the effects of reading printed text and hypertext on
RQ:
the EFL learners' reading comprehension?
Quasi-experimental article: matching only post-test only
Design:
control group design
Forty-six Taiwanese students equally divided into two
Participants
groups: the Experimental Group and the Control Group.
:
Instruments
Four reading texts and a comprehension test and a
:
questionnaire
The students who read from printed text were more
successful, apart from one skill: guessing a word’s meaning
Results:
from context
4

5. Summary of the Article (Cont.)
(1) to give students some related vocabulary, and pictures
to predict the articles,
Procedure:
(2) to give them four articles to read
(3) to test their skimming skill, scanning skill, and
vocabulary knowledge through reading comprehension
questions
5

6. Summary of the Article (Cont.)
There is a difference between reading from a printed text
Hypothesis: and reading from a computer screen
Independent Variable: Reading printed text and hypertext
Variables:
Dependent Variable: EFL learners' reading
comprehension
6

7. 2. Critique of the Article
The possible confounding variables or their
 Threats to internal possible effects on the outcome of the
research were not mentioned
validity:
Implementation:
 No information about the course instructors
 What were the major difficulties of reading hypertext?
(Tseng, 2008)
Subject Characteristics :
 No random sampling and random assignment (JOINED?)
 No detailed information to control confounding variables
(age, gender, reading ability, socio-economic status, etc…)
Location:
 Experimental group: Each student has a computer
Control group: no computers
Instrumentation:
 Instrument Decay: 9-page reading test.
Attitude of Subjects:
Control group: no computers (John Henry effect)
7

8. 2. Critique of the Article (Cont.)
 Validity and reliability of test
scores:• No statistical evidence on validity and
reliability
• Only raw scores were provided
8

9. 2. Critique of the Article (Cont.)
Questionnaire Results
 Internal validity threats to the outcome of the study
 No statistics, did not mention validity or reliability
9

10. 2. Critique of the Article (Cont.)
 External Validity
 No population and ecological generalizability
 Did not mention target and accessible
populations
 Quasi-experimental design
 No random assignment
 No pre-test
 Conclusion: Interpretation from the raw data
 Contradiction with the literature
review
Literature review: Advantages of reading from the computer
screen. (non linear reading)
Result: This study also confirmed that students made poor
performance when they read hypertext (Tseng, 2008).
10

11. 3. Recommendation
1. State confounding variables
2. Supply more information about the individuals who
implemented the study
3. Try to use random sampling and if possible random
assignment
4. Supply more information about the subjects
5. Try to keep locations constant, remove such variables as
uncomfortable seating
6. Support the instruments with reliability and validity data
7. Not to affect attitudes of the subjects provide similar
opportunities
8. Interpretation: Give statistical data analysis
9. Mention generalizibility (external validity)
10. Choose a stronger design (one of the randomized designs)
11. Literature review should present more research findings
rather than definitions
11

12. 4. Implications of the study
The pedagogical implications:
The importance of selecting web pages for students
The importance of setting up computer screens and web pages
The importance of teaching students how to read hypertext.
Only three questions without statistical data analysis helped such interpretation:
1. What factors affected reading via computer screens? About 26.4% of
participants chose the size of the font, 24.5% of them chose the background color
of the web pages, and 47.3% chose the downloading speed.
2. In the background color of web pages, about 35% of participants chose dark
background with white words, 60% of them chose white background with black
words and 5% chose either ways. It might be speculated that students extended
their reading inclination to computer screens because most of the paper was
white and printing color was black.
3. Participants were asked whether they preferred to read texts through computer
screens or paper. In the Control Group, 18% of them chose screens and 82% of
them chose paper whereas in the Experimental Group, 17% of them chose
screens and 83% of them chose paper. Entirely, 31% of the entire class chose
screen and 69 of them chose paper. For participants, they still preferred to read
texts on paper instead of computer screens.
12

13. 5. Summary
While the study has merit, the methods need to be re-evaluated.
The power of the study needs to be increased by obtaining a
larger sample size.
The numerous potential threats to internal and external validity
need to be addressed and minimized where possible.
It would also be helpful to be given data analysis regarding the
validity and reliability of the scores of the instruments.
Without these, it is impossible to evaluate the potential
meaningfulness of this study.
13