EAC 786 presentation at NC State University. Adams, M. J. D. (17 November 2010). Online course evaluations: Nonresponse and ClassEval in Fall 2009. Presentation to EAC 786 (Teaching in College) class at North Carolina State University, Raleigh, NC.

1. Online Course Evaluations: Nonresponse
and ClassEval in Fall 2009

2. COURSE EVALUATIONS
Paper versus Online
Ratings
Can
they?
Why?
How?

3. METHODS OF ANALYSIS
• Study measured likelihood of response
• Hierarchical modeling techniques
• 30 variables
– Students (demographics, housing, number of surveys
students completed)
– Course (if department of course and student’s major
matched; grade earned)

4. RESULTS
Statistically significant
characteristics
associated with
nonrepsonse:
– Gender (Male)
– Ethnicity (African-American,
Asian)
– Housing (off-
campus/commuters)
– Athletes
– Grades (Ds, Fs, ungraded
students/courses)
– Age (traditional age)
– Nontransfers
– Class rank (sophomores &
juniors)
– Students with more than
10 SETs to complete

5. 48% response rate
(overall)

6. RESULTS (continued)
• Using Holland’s six major academic types
– Realistic majors were more likely to respond than social,
artistic, conventional, enterprising, and investigative
disciplines.
– Social majors were less likely to respond than all other
students if the course was in the same department as the
student’s major.
• Most variables were no longer statistically significant
when the course was in the same department as
their major.

8. DISCUSSION
• Mostly aligned with previous research and theories
of survey participation*
– Exceptions = Environment of major/course, class rank
• Survey Fatigue – Are we oversurveying?
• Introduced new potential influences on participation
– Campus housing, athletes, transfer status,
* Avery et al., 2006; Cohen, 1981; Clarksberg, et al., 2008; Dey, 1997; Dillman et al., 2002, 2009; Fidelman, 2007; Groves et
al., 2004, 2009; Johnson et al., 2002; Jones, 2009; Kaplowitz et al., 2004; Lepkowski & Couper, 2002; Marsh, 2007; Moore &
Tarnai, 2002; Porter et al., 2004; Porter & Umbach, 2006a; Porter & Whitcomb, 2005; Sax et al., 2003, 2008

9. TO INCREASE RESPONSE RATES
• Faculty attitudes and efforts are key to obtaining
higher response rates*
• The instrument and administration of it
• Target students unlikely to respond
• Evaluate the environment
– Are course evaluations online accessible?
– Is the instructor encouraging response? Is the discipline?
* (Ballantyne, 2003; Dillman, 1978; Dillman et al., 2002, 2009; Dommeyer et al., 2004; Groves et al., 1992)

10. ADDITIONAL RECOMMENDATIONS
• Obtain more information about the course, the
faculty, class size, etc.

11. FOR MORE INFORMATION
meredith_adams@ncsu.edu
Full study available online at
http://repository.lib.ncsu.edu/ir/handle/1840.16/6364