questions in textbooks

1. Comprehension Questions in Bahasa Indonesia and English Textbooks
Asf_suarman@yahoo.co.id
SMPN 1 Cinangka
Questions in their while or post reading activities are mostly used to check the students’ comprehension about texts. They ought
to challenge all levels of the students’ comprehension: literal, inferential and evaluative. They are mostly written after texts on
textbooks. However, not all textbooks take those levels of comprehension into account. This study investigated what levels of
comprehension were challenged in the Bahasa Indonesia and English textbooks for Junior High School students and how teachers
treat the questions in text books. It was found that in the Bahasa Indonesia textbook, 19.9 % of questions challenge literal
comprehension level, 45.9% check inferential level and 34% exercise evaluative one. In the English textbooks, 62.9 % challenge
the students’ literal comprehension level and 32.7 train inferential and only 4.2% exercise evaluative ones. It was also found that
most teachers treated the questions as they were. They scarcely modified them. These findings show in the textbooks the
comprehension levels are not evenly challenged. The fact that literal level was not much tested In Bahasa Indonesia might lead
students guess the content of the texts without reading. Meanwhile in English textbooks, neglecting evaluative comprehension
level might lead students to be uncritical. Therefore, it is recommended that textbook writers as well as teachers should be more
cautious in generating questions so that all levels of comprehension are well exercised.
In many language lessons, textbooks help teachers teach intensive reading. This lesson is aimed to help
students to comprehend texts (Aebersold and Field 1997 in Hedgcock and Ferris, 2009). They work
simultaneously with the same texts and activities like re-reading the texts by answering comprehension
questions (Hedgcock and Ferris 2009 p. 172). The questions function to check students’ understanding
(Nunan and Lamb, 1996 in Yang 2010). The students as readers are more likely to understand what they
read when they are asked comprehension questions about the texts and receive immediate feedback about
their answers (Bassel and Rasinki, 2008 p. 20).
Comprehension questions are expected not only to facilitate but also to challenge at least three levels of
comprehension: literal, inferential and evaluative or critical.
Literal comprehension is level of understanding of a text wherein a reader has access and can recognize
and recall details in the text (Brasel and Rasinski, 2008, p. 87). It requires recognition and recall of ideas,
information and happening explicitly stated in the reading selection (Clymer, 1968 in Hudson, 2007 p.
85). The examples of them are finding date of flight; who is …; what/where did she/he …; what happened
when or during …; find out the differences between … and …?
Inferential or interpretative comprehension is the level of understanding wherein readers can read
meanings which are not directly stated on the texts (Brasel and Rasinski, 2008, p. 17; Alexander, 1989;
Burnes & Glenda, 1985 in Setiadi, 2010 p. 92; Briskin, 2005). This level requires the orchestration and
the manipulation of information from the text as well as within the readers (Brasel and Rasinski, 2008, p.
17) The questions are, for example, how did she converse with …; what was the weather like; do you
think …; what will happen next etc.
Evaluative or critical comprehension refers to the understanding of a text which requires readers to use an
adequately developed knowledge base (Carr and Thompson, 1996) and new information (Briskin, 2005)
including prior knowledge, intuition, and imagination to make hypotheses (Pennel, 2002); to draw
conclusions, to make reasonable predictions, connections between conclusions and critical judgments
about what texts (Keene and Zimmerman, 1997 in Pennel, 2002; Clymer 1968 in Hudson, 2007 pp. 84-5;
Brasell & Rasinski, 2008 p. 17). The examples of the questions are what strange ideas did … have? why
was … true?; how do you feel about this character?; what do you think …’s attitude?

2. Based on the categories above, the questions on Bahasa Indonesia textbooks from one publisher and
English textbooks from two publishers for grade 7, 8 and 9 junior high schools were categorized. Only the
questions written under/after the texts were analyzed. Four teachers were interviewed.
From Bahasa Indonesia textbooks, it was found that the questions of inferential comprehension
dominated. In grade 7’s textbook, among 90 questions from 19 texts, there were 21 questions (23%)
drilling literal comprehension, 38 questions (42%) challenging inferential comprehension and 31
questions (34%) exercising evaluative comprehension level. In grade 8’s, among 98 questions in 17 texts,
12 questions (12%) challenging literal comprehension, 46 (47%) drilling inferential and 40 questions
(41%) exercising evaluative comprehension level. In grade 9’s, among 123 questions in 19 texts, 29
questions (24%) exercising literal comprehension, 59 questions (48%) practicing inferential
comprehension and 35 (28 %) questions challenging evaluative comprehension. To get the picture of
overall distribution, see the table below.
The Comprehension Questions In Bahasa Indonesia Textbooks
50.00
40.00
30.00
Literal
20.00
Inferential
10.00
Evaluative
Ina Book
7
Ina Book
8
Ina Book
9
Overall
This finding shows that inferential questions are the most frequently exercised. This means that the
textbooks trained students to be good at inferring which is important for reading (Mickulecky, 1990).
However, the literal comprehension was seldom challenged. This could be risky since the students may
guess the answer without active reading. They might be reluctant to read the text and rely their answer on
their prediction, instead.
On the contrary, from the two English textbooks, it was found that the questions about literal
comprehension dominated in all grades. In grade 7’s, among 104 questions in 21 texts, there were 81
questions (78%) exercising literal, 21 questions (20%) challenging inferential and 2 (2%) questions
checking evaluative comprehension level. In grade 8’s, among 323 questions in 58 texts, there are 182
questions (56%) challenging literal, 121 questions (37%) examining inferential and 9 questions (6 %)
training evaluative comprehension. Finally, in grade 9’s, among 193 questions in 50 texts, 193 questions
(65%) challenging literal comprehension, 95 questions (32%) training inferential and 9 questions (3 %)
exercising evaluative comprehension level. The table below the ratio

3. The Comprehension Questions in English Textbooks
80.00
60.00
40.00
20.00
-
Literal
Inferential
Evaluative
These findings show that the questions in the textbooks mostly challenge literal comprehension. This
means that the students were mostly exercised to recognize and recall of ideas, information and happening
explicitly stated in the texts (Clymer, 1968 iin Hudson, 2007 p. 85; Briskin, 2005;). These are in line with
what Myhil et. al. (2006) claims that factual questions are the most common ones. These also confirm the
statement of Muhamad (1999) that teachers tend to ask their students mainly literal comprehension
questions.
This implies that the students were not challenged to have better comprehension. Their evaluative
comprehension is not frequently exercised. This is parallel to the claim that the emphasis of lesson was
reading and doing exercises leading to rote-learning, which could have negative impact on the students’
meaningful learning, and critical thinking has never been the object of learning (Mok, J. 2010). When the
students’ critical thinking is not exercised, they do not have the ability to take control of their conscious
thought processes. With this, they risk being controlled by the ideas of others.
In addition, based on the interview, it was found that teachers tend to utilize the questions as they were
written. They modify or add more questions only when the students do not understand the questions,
when the time for teaching the texts were still long or when other language skills, speaking or writing, are
trained. This confirms the claim that most class activities were dominated by textbooks: it was a textbook
which determines what the lecturer and students do (Zohrabi, M. 2011).
In summary, this study concludes that the questions in Bahasa Indonesia and English textbooks different
emphasis in challenging students’ comprehension levels. The former contained 19.9% literal, 45.9%
inferential and 34% evaluative comprehension questions. The latter contained 62.9 % literal, 32.7% and
only 4.2% evaluative comprehension questions. The fact that literal level was not much tested In Bahasa
Indonesia might lead students guess the content of the texts without reading. Meanwhile in English
textbooks, neglecting evaluative comprehension might lead students to be uncritical. It was also found
that most teachers utilized the questions as written and scarcely modified them. So, the questions on the
textbooks determined what was exercised in many classrooms.
Therefore, it is recommended that both textbook writers and teachers should be more cautious in
generating questions, to include all levels of comprehension evenly. Reading teacher should modify or
add questions to challenge all the comprehension levels.

4. BIBLIOGRAPHY
Aebersold, J.A., and Field, M.L. (1997). From reader to reading teacher: Issues and strategies for second language
classroom. Cambridge, England: Cambridge University Press. in Hedgcock and Ferris, 2009
Alexander , E. (1989). Teaching Reading Illinois. London: Scott, Foresman and Company.
Brasell, D. & Rasinski, T. (2008). Comprehension that Works: Taking Students Beyond Ordinary Understanding to
Deep Comprehension. Huntington Beach: Shell Education.
Briskin, L. (2005). A Guide To Active Reading And Asking Questions: A Handout For Students. Online article.
Available at www.yorku.ca/laps/sosc/Foundations/documents/ActiveReading.pdf . Accessed on February
15. 2012.
Burnes, D & Page, Glenda. (1985). Insight and Strategies for Teaching Reading. Sydney: Harcourt Brace
Jovanovich. In Setiadi, R. 2010. Self-Efficacy in Indonesian Literacy Teaching Context: A Theoretical and
Empirical Perspective. Bandung: Rizki Press.
Carr, S. C. and Thompson, B. (1996). The Effects of prior knowledge and schema activation strategies on the
inferential reading comprehension of children with and without learning disabilities. Learning Disability
Quarterly. 19, 48 - 61.
Clymer, T. (1968). ‘What is “reading”?’: some current concepts. 67th Yearbook of National Society for the Study of
Education, 1968, 7-29. In Hudson, T. (2007). Teaching Second Language Reading. Oxford: Oxford
University Press.
Hudson, T. (2007). Teaching Second Language Reading. Oxford: Oxford University Press.
Hedgecock, J.S. and Ferris, D. R. (2009). Teaching Readers of English: Students, Texts and Contexts. New York:
Routledge.
Keene, E. O., & Zimmerman, S. (1997) Mosaic of thought: Teaching reading comprehension in a reader’s
workshop. Portsmouth, NH: Heinemann. In Pennel, D. (2002). Explicit Instruction for Implicit Meaning:
Strategies for Teaching Inferential Reading Comprehension. Available at education.wm.edu accessed on
November 12, 2012.
Mikulecky, B.S. (1990). A Short Course in Teaching Reading Skills. USA: Addison-Wesley Publishing Company
Inc.
Mok, J. 2010 The New Role of English Language Teachers: Developing Students’ Critical Thinking in Hongkong
Secondary School Classroom. ASIAN EFL Journal. 12 (2). P. 262-287
Muhamad, A. (1999). What Do We Test When We Test Reading Comprehension? The Internet TESL Journal, Vol.
V, No. 12, December 1999 available at http://iteslj.org/Techniques/Nunn-Interacting.html accessed on
February 5th, 2012.
Myhil, D. Jones, S & Hopper, R. 2006. Talking, Listening, Learning: Effective Talk in Primary Classrooms.
Maidenhead, UK: Open university Press. in Yang, C. C. R. (2010). Teacher questions in Second Language
Classrooms: An Investigation of three case studies. Asian EFL Journal. 12 (1) pp. 181-201)
Nunan, D & Lamb, C. 1996. Self-Directed Teacher. Cambridge: Cambridge University Press. In Yang, C. C. R,
2010. Teacher questions in Second Language Classrooms: An Investigation of three case studies. Asian
EFL Journal. 12 (1) pp. 181-201

5. Pennel, D. (2002). Explicit Instruction for Implicit Meaning: Strategies for Teaching Inferential Reading
Comprehension. Available at education.wm.edu accessed on November 12, 2012.
Setiadi, R. (2010). Self-Efficacy in Indonesian Literacy Teaching Context: A Theoretical and Empirical Perspective.
Bandung: Rizki Press.
Yang, C. C. R, 2010. Teacher questions in Second Language Classrooms: An Investigation of three case studies.
Asian EFL Journal. 12 (1) pp. 181-201.
Zohrabi, M. 2011. An Evaluation of Class Activities and Exercises in ELT Classroom for General Purpose Course.
Canadian Center of Science and Education. 4 (1). 141-151. In Proquest Documents: English Language
Teaching. 84.