1. NEERO 45th Annual Confernece
Symposium 2.2 | Thursday, 10:15-11:45am
Symposium Title:
Student Evaluations of Higher and Basic Education Instructors:
Three Case Studies
Presenters:
Tom
Neuschafer
Nichol
Murrray
Diane
Onorato

2. Objectives of Presentation
Secondary Heading
1. To demonstrate that high school students can recognize
good teaching and can collaborate to build an assessment
tool which parallels professionally tested instruments
2. To demonstrate evidence that college students must have
buy-in and confidence that their opinions are valued and
to share strategies for increasing the response rates of
evaluations.
3. To demonstrate a process by which an online or blended
course can be improved by student evaluations using an
instrument which correlates to the course design and
management

3. Student-Designed Evaluations
The Evaluation
• The need for an evaluation
• Wright’s article
• The evaluation
• What they were told / what they weren’t told
• Questions on parenthesis
• Grading system
• Explanation sections

4. Student-Designed Evaluations
The Evaluation
Let’s take a look at the actual evaluation!
Sections of the evaluation
• About the instructor
• About the course
• About the program
Now let’s hear some of their reasons, in their own
words!

5. Student-Designed Evaluations
Student Reasons

6. Top Complaints of Faculty
General
• Instructors arriving late or unprepared
• Instructors that lecture for an entire class period
• Instructors taking too long to respond to messages and
provide feedback on work
• Instructors who do not provide enough examples or case
studies
• Instructors who are boring
• Instructors with poor speaking skills/Cannot understand
instructor
• When an instructor thinks that he or she is better and
above the students

7. Top Complaints of Faculty
Online Courses
• Instructions that are not clear
• Waiting too long for grades/feedback
• Missing or disengaged instructors (lack
of support)
• Poor course design – navigation is
confusing, hard to find assignments

8. Top Praises of Faculty
• Professionalism – dress, punctuality, behavior
• A detailed, organized syllabus that clearly conveys
expectations
• Flexibility – when an instructor is willing to make
changes to the schedule and content when
appropriate
• Interesting and relevant course material
• Varied methods of instruction
• When an instructor shows a passion for teaching
• Ample interaction with students
• Respect and fairness
Source: Butler County Community College Office of Academic Affairs

9. Student Evaluations of Higher Education Faculty
Concerns
Students are being
selective in who
they choose to
evaluate.
Evaluations are
now
administered
online.
Ineffective
Feedback
Students are
not taking the
evaluations
seriously.
Instructors do not
receive results until
after the semester
is over.

10. Online Course Evaluation link
Actions Taken
•
•
•
Discussed concerns with VP of Academic Affairs
Looked at the current evaluation instrument for alignment
with my teaching
Added additional questions to the evaluation instrument:
Please rate the following statement as it applies to you:
1. I feel that my instructor cares about my success in this
course.
2. My instructor responds to my questions in a timely
manner.
3. My instructor takes time to include lessons that are
purposeful and can be applied to real-life situations.
4. My instructor values me as an individual.

11. Case Studies
Fall 2012
Spring 2013
• 4 course sections (81
students)
• 4 different
approaches to
encourage
participation
• Administered
ongoing course
feedback survey
• 2 course sections (27
students)
• Reflective writing
activity regarding
attitudes toward
instructor
evaluations
• Class discussion
about evaluations

12. Results – Fall 2012
Response Rate
6/25 = 24%
Section L01 – Announced that the evaluation window
has opened
9/22 = 41%
Section L02 – Announced that the evaluation
window opened; sent weekly e-mail reminders
9/17 = 53%
Section C51– Announced that the evaluation window
opened; sent students a link to the online evaluation site
(through their personal e-mail)
13/17 = 76%
Section B01– Class discussion on instructor
evaluations; gave students opportunity to
complete evaluations during class time in the
computer lab

13. Respose Rates

14. Ongoing Feedback Survey
Administered 2/3 into semester
Important to share and discuss comments with
students
1. My instructor is available to me in person, by phone, and
via e-mail.
2. Participation in this class is valued.
3. I feel comfortable in this class.
4. Which of the following were the most interest and helpful
methods of instruction for you? (methods listed)
5. Comment on any suggestions you may have to improve
your experience in this course.

15. Spring 2013 – Reflective Writing Activity
Survey on Faculty and Course Evaluations
 How seriously do you approach faculty evaluations?
 When the time comes to complete evaluations, do you complete them
for all of your instructors, some of them, or none?
 Do you think the instructors read and take your comments into
consideration?
 Think of any negative experiences you have had with a
course/instructor. What made it a negative experience?
 How would you describe your ideal instructor?
 If you could add one question to a faculty evaluation, what would it be?
 If you were the President of the college, what methods would you use
to evaluate instructors?

16. Findings from Survey
The majority stated that they complete evaluations
for the instructors that they either had a very positive
experience with or one that they thought was very
bad.
“I complete them if I had a bad experience with a teacher.”
“If they don’t ask, I don’t care. If they do ask, then I’ll do it.”
“Students do not learn or care about classes that the
instructor does not care about.”

17. Findings from Survey
When describing an ideal instructor:
Respectful, on time, easy-to-reach, interesting, varies
instruction, posts grades online
“One who applies it to real-life rather than just making me
memorize”
“I like professors who make a lasting impression.”
Spring 2013:
Results will be
available after
May 7.

18. Strategies to Increase Awareness of Faculty Evaluations
• Take evaluation questions into consideration when
developing your course syllabus, objectives,
content, and assessments.
• Provide opportunities during class time to
complete evaluations.
• Reflective writing activities
• In-class discussions
• Mid-semester surveys
• Share findings with students

19. Evaluation of Blended College
Courses Using a Community of
Inquiry Survey: Opportunity for
Improvement
Diane Z. Onorato
Indiana University of PA
Mercyhurst University, North East, PA
Survey retrieved from
http://communitiesofinquiry.com/model

20. Babson Survey Research Group and the College Board’s 2011 report on
the status of online learning in the US,
1. 6.1 million college students took an online course in Fall 2010
2. 31 percent of all college students were taking at least one online course
in 2011
3. Online education had 10% growth rate; traditional education, a 2% growth rate
4. 65 percent of all reporting tertiary level schools include online
education as part of their strategic plan.
5. According to the most recent distance and online learning report from
Converge (2011), in five years, if the percentages of increase stay the
same online students will outnumber traditional students.
6. 1.8 million online basic education student in 2009-10 (Converge, 2011, p.
5)

21. Theoretical
framework of a
Community of
Inquiry (CoI)
“collaborativeconstructivist…
learning experience
through the
development of three
interdependent
elements –
social, cognitive and
teaching presence.”

22. Definition of Blended Learning
Blended learning is the organic integration of thoughtfully
selected and complementary face-to-face and online
approaches and technologies.
Blended learning is seen as an opportunity to fundamentally
redesign how we approach teaching and learning in ways that
higher education institutions may benefit from increased
effectiveness, convenience and efficiency.
(Garrison & Vaughan, 2008)

23. Teaching Presence is “the design,
facilitation, and direction of
cognitive and social processes for
the purpose of realizing personally
meaningful and educationally
worthwhile learning outcomes”
(Anderson, Rourke, Garrison, &
Archer, 2001, p. 5).
Categories
Examples of Indicators
Facilitating Discourse
Defining, focusing, motivating with knowledge of students’
efficacy; Clarifying ideas, shared experiences; Managing
netiquette; Providing feedback and directions
Direct Instruction
Discussion; Questioning, Injection of new knowledge
Instructional Design &
Organization
Setting curriculum, standards, targets, scaffolding; Calendar;
Designing methods, activities; Use of medium; Offering
direction for technical support

24. Teaching Presence: Survey Items
Design & Organization
1. Clearly communicated important course goals
2. Clearly communicated important course topics
3. Provided clear instructions on how to participate
4. Clearly communicated important due dates/time frames
Facilitation
5. Identifying areas of agreement and disagreement …to help me to learn
6. Guiding class towards understanding course topics …to clarify my thinking
7. Keeping participants engaged and participating in productive dialogue
8. Keeping the course participants on task
9. Encouraged course participants to explore new concepts
10. Reinforced the development of a sense of community
Direct Instruction
11. Focus discussion on relevant issues in a way that helped me to learn
12. Feedback that helped me understand my strengths and weaknesses
Retrieved from
http://communitiesofinquiry.com/methodology
13. Feedback in a timely fashion

25. Social presence is “the
ability of participants to
identify with the community
(e.g., course of
study), communicate
purposefully in a trusting
environment, and develop
inter-personal relationships(Garrison, 2007; van Shie,
2008, retrieved from
by way of projecting their http://communitiesofinqu
iry.com/sites/community
individual personalities” ofinquiry.com/files/conce
pt-map.gif)
(Garrison, 2009, p. 352).
Categories
Examples of Indicators—sense of self and audience
Emotional Expression Using humor, self-disclosure, emoticons; Expressions of feelings; a sense
of safety to speak without unfair treatment
Open
Communication
Continuing threads; Quoting and referring to each others’ statements;
Complimenting and agreeing; Risk-free Expression
Group Cohesion
Encouraging and including others; Using names and greetings

26. Social Presence: Survey Items
Retrieved from
http://communitiesofinquiry.com
/methodology
Affective expression
14. Getting to know other[s] gave me a sense of belonging
15. Able to form distinct impressions of [others]
16. Online . . . is an excellent medium for social interaction
Open communication
17. Comfortable conversing
18. Comfortable participating in . . . discussions
19. Comfortable interacting with other[s]
Group cohesion
20. Comfortable disagreeing . . . while still maintaining sense of trust
21. My point of view was acknowledged by other[s]
22. Develop a sense of collaboration

27. Cognitive Presence is “the
extent to which learners are
able to construct and confirm
meaning through sustained
reflection and discourse”
(Garrison, Anderson, & Archer,
2001, p. 11).
Category
Examples of Indicator–reflection and interaction
Triggering Event
Sense of puzzlement about a topic; Recognizing the problem but not yet
beginning an activity on the topic
Exploration
Information exchange through discussion; Recognition of divergent views;
Offering of suggestions; Brainstorming
Integration
Connecting ideas; Finding points of convergence; Offering tentative
conclusions or solutions
Resolution
Test and defend solutions; Apply new ideas

28. Cognitive Presence: Survey Items
Triggering event
23. Problems posed increased my interest
24. Activities piqued my curiosity
25. Felt motivated to explore content related questions.
Exploration
26. Utilized a variety of information sources
27. Brainstorming and finding relevant information helped
me resolve content-related questions
28. Discussions help[ed] me appreciate different
perspectives.

29. Cognitive Presence: Survey Items (continued)
Integration
29. Combining new information helped me answer questions
30. Activities helped [with]. . . explanations/ solutions
31. Reflection . . . helped me understand fundamental
concepts
Resolution
32. Can describe ways to test and apply [course ideas]
33. Have developed [practical] solutions
34. Can apply the knowledge created in this course to my
work or . . . other activities

30. Results from CoI Survey
Indicator
Class Class Class Class Class Class Class Class Class
#1
#2
#3
#4
#5
#6
#7
#8
#9
TP design & organization
8%
0%
0%
4%
10%
11%
10%
10%
15%
TP facilitation
5%
25%
4%
6%
17%
12%
8%
13%
17%
TP direct instruction
13%
17%
7%
13%
13%
13%
9%
13%
20%
SP affective expression
3%
17%
4%
8%
3%
11%
7%
10%
27%
SP open communication
0%
0%
4%
8%
10%
11%
9%
13%
20%
SP group cohesion
0%
0%
7%
9%
13%
13%
11%
10%
27%
CP triggering event
17%
0%
7%
12%
10%
11%
11%
12%
13%
CP exploration
17%
0%
4%
13%
20%
14%
9%
13%
20%
CP integration
0%
0%
0%
11%
10%
9%
7%
8%
20%
CP resolution
10%
0%
7%
10%
13%
15%
11%
13%
33%

31. Findings & Implications
•
•
•
•
Scaffolding
Time for reflection
Clear expectations, models
Active and collaborative
learning
• Manage the learning
environment; summarize the
conversation/new direction
• Provide feedback beyond
grades
• Students’ self-efficacy

32. Findings & Implications
Terry Ann Morris (2011) 65% completion for online CC
and 72% for traditional.
Survey says,
1. Enjoyed discussion activities (multiple days)
2. Instructor feedback was the high point (24 hrs.)
3. Don’t like chat rooms, social networking with no
instructional purpose
4. Self-efficacy is determining factor in successful
completion
5. Varied favorite activities underscore the diversity of
the population

33. To enhance social
& cognitive
presence, use a
Wiki using a
scaffold to create a
pro/con debate

34. Cognitive Presence—Arbaugh (2007) most difficult
Elder and Paul’s Model of Critical Thinking (2010)
Carefully design word tasks to promote critical thinking.
They developed a model of questions based on
standards that improve learner’s thinking skills and help
them question ideas before they decide if they should
accept them.
Questions include clarification, accuracy, precision,
relevance, logic, significance, fairness.

35. To keep learners engaged, Survey (Bonk)

36. Teaching presence
Welcome email
Scavenger hunt
Wiki introductions
Once upon a time
Guest lectures
Templates
Scaffolding
Multiple discussion forums
Pride Wall

37. Importance of Teaching Presence
“The body of evidence is growing rapidly attesting to the
importance of teaching presence for successful online
learning …
“The consensus is that teaching presence is a significant
determinate of student satisfaction, perceived learning, and
sense of community.”
Dr. Marti Cleveland-Innes, Athabasca University
Dr. Randy Garrison, University of Calgary (2007)
13th Annual Sloan-C Conference

38. Students must feel a sense of participation
• Students know and can articulate what good
teaching, learning, and collaboration is at the high
school level.
• If there is no student evaluation procedure, a high
school class can collaborate to produce a suitable
instrument.
• Students must have buy-in or an accessible, secure
means of participation in course evaluation in
order for significant numbers to provide usable
feedback.

39. The evaluation must measure class procedures to
allow for improvement
The evaluation instrument should measure what
happens in class and be a standard across similarly
constructed classes so all faculty can be evaluated
fairly.
Results of a course evaluation by a student should
provide data for teachers to improve their courses
with respect to increased perceived learning and
student satisfaction.

40. NEERO 45th Annual Confernece
Symposium 2.2 | Thursday, 10:15-11:45am
Symposium Title:
Student Evaluations of Higher and Basic Education Instructors:
Three Case Studies
Thank you for your attention!
Tom
Neuschafer
Nichol
Murrray
Diane
Onorato