This presentation focuses on the assessment of a Writing Intensive course, from both the administrative and teaching points of view.

1. TYCA Northeast
Annual Conference 2010
Capitol Improvements: The Two-Year College as an Agency of Change

2. Collaborative
Assessment:
Working Together Toward Institutional Change
Kelly Bender and Elizabeth Nesius
Passaic County Community College

3. This presentation can be found on
SlideShare

4. This presentation looks at methods of collaboration for
course-and program-level assessment and data-sharing to
help both faculty and program administrators. Instructors
can improve the WAC elements of their courses without
sacrificing content, while still furthering the program
mission of helping students to write well and think critically
in any discipline. Writing program administrators, on the
other hand, can improve both the writing program
implementation and the training of its faculty to teach WAC
courses. Students will benefit by experiencing WAC courses
that are clearly defined, engaging, and optimized for their
learning.

5. PCCC Writing Initiative
 Title V Grant: $2.5 million over 5 years
 Develop 20 Distinct, Writing Intensive
Courses (with multiple sections, across 3
campuses)
 Develop and provide online resources for
technology and writing
 Support students and faculty with an on-
ground writing center and online tutoring
 Students required to take 2 WI courses (1
if getting AAS degree)

6. Writing Intensive Courses
 19 Distinct WI courses currently
developed or in development across the
curriculum
 Departments include: Computer
Information Systems, English,
Humanities, Math, and Science
 Courses designed to contain writing,
critical thinking, and information literacy
elements

7. What We Assess
 Students
◦ Assessed in writing, critical thinking, and information
literacy skills
◦ Assessed in the course by instructors
 Courses
◦ Course materials created collaboratively in Faculty
Institutes
◦ Course materials checked and approved by WI
Administrators
 Program
◦ Students assessed across courses by WI administrators
in the 3 areas
◦ Students and instructors self-assess by means of a
survey

8. How We Assess
 Students
◦ Instructors: student grades
◦ Administrators: student performance on CWE
 Courses
◦ Course materials collected at the beginning of each
semester
◦ Evidence of materials being used by student use of
ePortfolios
 Program
◦ Student writing assessed with 3 rubrics: writing,
critical thinking, information literacy
◦ Student and faculty surveys

9. Assessing Writing
 First we had to determine the criteria for
writing assessment:
◦ Appropriateness
◦ Unity & Focus
◦ Development
◦ Organization
◦ Sentence Structure
◦ Usage
◦ Mechanical Conventions
 Additional criteria for course writing are
left up to individual instructors

10. Writing Rubric Student __________________________________ Assignment ______________________________________
Beginning Developing Competent Accomplished
Vocabulary/Language
appropriate to this course
Words used incorrectly;
displays fundamental errors
in vocabulary usage; lacks
variety.
Generally correct usage but
little variety; some variety
but words used incorrectly.
The majority correctly used
with some variety in
vocabulary.
Varied vocabulary; correct
and sophisticated use of
language.
Conventions
mechanics, spelling, sentence structure,
word forms
(problems noted below)
Many serious errors which
persistently interfere with
meaning.
Some serious errors that may
interfere with meaning.
A few errors which do not
interfere with meaning.
No serious errors; few
minor errors.
Organization
includes: appropriate length, order
(logical, chronological etc.) and
transitions
No clear structure;
organizational requirements
not met.
Some structure and order
evident; minimally, an
introduction, body &
conclusion present.
Generally well-organized;
clear and logical flow of
ideas. Fulfills the
assignment.
Well-organized with
appropriate structure and
smooth transitions.
Content
includes addressing the assigned topic
Lacks thesis; incomplete
development of evidence
and/or minimal relevance to
topic.
May lack thesis. Some
evidence underdeveloped or
off-topic.
Has supporting evidence
and examples for thesis.
Strong, convincing evidence
to support thesis and
argument.
 Tense formation
 subject-verb agreement
 pronoun usage/agreement
 word choice/meaning
 proper modifiers
 Variety of formations
 correct construction
 Spelling
 Capitalization
 punctuation
COMMENTS:
WI v10.1
Assessing Writing

11. EN 205 – Introduction to
Literature Grading Rubric
Student __________________________________ Assignment ______________________________________
Accomplished Competent Developing Beginning
Vocabulary/Language
appropriate to this course
Varied vocabulary; correct
and sophisticated use of
language.
The majority correctly
used with some variety
in vocabulary.
Generally correct usage but
little variety.
Words used incorrectly;
displays fundamental errors
in vocabulary usage; lacks
variety.
Conventions
(problems noted below)
No fundamental errors; few
minor errors.
A few errors which do
not interfere with
meaning.
Some fundamental errors
that may interfere with
meaning.
Many fundamental errors
which persistently interfere
with meaning.
Organization
includes: appropriate length, order
(logical, chronological etc.) and
transitions
Well-organized with
appropriate structure and
smooth transitions.
Generally well-
organized; follows basic
assignment guidelines;
clear and logical flow of
ideas.
Some structure and order
evident, attempt at following
assignment guidelines;
minimally, introduction,
body & conclusion.
No clear structure; does not
follow assignment guidelines;
incoherent.
Content
includes addressing the assigned topic
Strong, convincing evidence
to support thesis and
argument.
Has supporting evidence
and examples for thesis.
May lack thesis or
conclusion. Evidence
underdeveloped or off-topic.
Lacks thesis and/or
conclusion. Minimal
relevance to topic.
Citation of sources
Uses citations when
appropriate and demonstrates
mastery of formatting; goes
beyond the amount of citation
or sources required.
Uses required amount of
citations when
appropriate and uses
proper formatting most
of the time.
Uses citations infrequently
and shows inconsistent
formatting; understands
some elements of citation
but not others.
Uses little or no citation and
displays little knowledge of
formatting; clearly cannot
differentiate between most of
the elements of citation.
USAGE
 Tense formation
 subject-verb agreement
 pronoun usage/agreement
 word choice/meaning
 proper modifiers
SENTENCE CONSTRUCTION
 Variety of formations
 correct construction
MECHANICS
 Spelling
 Capitalization
 punctuation
COMMENTS:
Revised 11/1/2010

12. Assessing Writing
 Faculty use rubric with students for
formative writing assessment
 Extra space to add content or other
elements for assessment based on
assignment
 WI Administrators use rubric for
summative assessment of writing (without
additional elements)

13. TOOLS: The Writing Center
The Writing Center
provides support for
students at all stages
of the writing process,
whether it is
brainstorming a topic,
structuring an essay,
or editing and
revising.
It also offers support
to students for their
Writing Intensive
courses in the forms
of face-to-face and
small group
tutoring/instruction.

14. The Writing Center
While the Writing Center made the
assessment of student writing “easier,” it
was also helpful in terms of assessing the
course.
Attendance at the Writing Center was made
mandatory. This created a direct
correlation between the outcome of
student writing and the effectiveness of a
support tool.

15. Assessing Critical Thinking
 Criteria for critical thinking assessment
◦ Identifying arguments and counter-arguments
◦ Assessing validity of both sides
◦ Drawing conclusions based on those
assessments
 Faculty use rubric with students for
formative critical thinking assessment
 Administrators assess critical thinking
across courses (in writing assignments
that require critical thinking) using same
rubric

16. Assessing Critical Thinking
Critical Thinking Rubric STUDENT _________________________________________ Assignment ______________________________
Beginning Developing Competent Accomplished
Evidence
 Biased interpretations
 Insufficient search for
evidence
 or inappropriate
evidence choices
 Misinterprets some
evidence
 Limited
information search
 limited source
evaluation or
attribution
 Accurately interprets the
majority of evidence
 Sufficient search for
evidence
 Shows some evaluation of
sources
 Appropriate evidence
 Accurately interprets the
evidence presented
Arguments
 Fails to identify
argument
 or counter-arguments
 Has argument but
inadequate
identification of
relevant counter-
arguments.
 Identifies relevant
argument(s)
 including counter-
arguments
 Identifies pertinent
arguments, pro and con
 Rebuts counter arguments
Conclusion(s)
Does not reach any conclusion
justified by evidence.
Conclusion based on
incorrect, irrelevant , or
insufficient evidence
Conclusion(s) are reasonably
justified or reasonably explained.
Draws thoroughly justified
conclusions by appropriate
interpretations of evidence
Comments
V4 June 2010

17. Assess Information Literacy
 Criteria for information literacy
assessment
◦ Defines and articulates need for information
◦ Information retrieval
◦ Citation of sources
◦ Evaluation of Sources
◦ Uses information effectively
 Information literacy required in at least
one assignment
 Assessment by faculty and administrators
using the same rubric

18. Assessing Information Literacy
Information Literacy Rubric Student ________________________ Reader ____________________ Score _____________
Beginning Developing Competent Accomplished
Defines and
articulates the need
for information
No clear statement of purpose;
does not associate information
need with the assignment
Shows purpose but not clearly
articulated; associates
information need with the
assignment but does not
articulate a thesis statement
Purpose is clearly
articulated and information
need is clearly recognized
Purpose is highly focused and/or
nuanced; thesis statement or research
question takes multiple viewpoints into
account
Information
retrieval
Retrieves little or none of the
information required for the
assignment; or, information
retrieved clearly does not meet
the information need
Retrieves some of the
information required for the
assignment; some of the
information retrieved may not
meet the information need
Retrieves the required
amount of information for
the assignment and the
majority of it meets the
information need
Retrieves much more than the
required amount to complete the
assignment; information retrieved
meets the information need; multiple
viewpoints are considered
Citation of sources Uses little or no citation and
displays little knowledge of
formatting; clearly cannot
differentiate between most of
the elements of citation
Uses citations infrequently and
shows inconsistent formatting;
understands some elements of
citation but not others
Uses citations when
appropriate and uses
proper formatting most of
the time
Uses citations when appropriate and
demonstrates mastery of formatting;
goes beyond the amount of citation or
sources required
Evaluation of
Sources
Little or no recognition of the
relevance and quality (authority,
currency, objectivity) of
information sources
Recognizes the relevance of
information sources but has
difficulty evaluating their quality
Recognizes both the quality
and relevance of
information sources
Information sources are highly relevant
and evaluation of sources is highly
nuanced
Uses information
effectively in their
writing
assignments
Uses little or no useful
information from outside
sources
Attempts to incorporate
information from outside
sources; sometimes off-topic
Uses information from
outside sources to support
the topic; shows some
synthesis with own ideas
Use of information from outside
sources is highly refined, articulated,
and synthesized with own ideas
Comments:
Revised 6/6/08

19. Assessing the Course
Course assessment is essential in
determining:
 Both the quality and quantity of learning for
students
 An appropriate response to their personal
development needs
 Whether students have acquired the
knowledge, skills and competencies
consistent with the aims of the course
 The effectiveness of the curriculum and
faculty development

20. Methods of Course Assessment
 By Instructors
◦ Faculty institutes
◦ ePortfolios
◦ Course Survey
 By Administrators
◦ Course materials
◦ ePortfolios

21. Course Assessment: Faculty
Institutes
Faculty Administrators
 Institute Part 1
◦ Overview of Initiative
◦ Overview of WI course
components
◦ Look at course as it
stands for writing, critical
thinking and information
literacy elements
◦ Begin development of
new materials with
rubrics and templates
 Institute Part 1
◦ Collect current course
materials (syllabus and
assignments)
◦ Assist in development
of new materials
◦ Collect and file new
materials
◦ Survey faculty on
success of workshop
components

22. Faculty Institutes
Faculty Administrators
 Institute Part 2
◦ Some materials tested
in regular section of
courses
◦ Materials refined based
on in-course testing
◦ More materials
developed
◦ Integration of media
and other resources
 Institute Part 2
◦ Discuss piloting of
materials from part 1
◦ Assist in refining
materials and
development of new
materials
◦ Assist in incorporating
media, other resources
and support services

23. Assessing the Initiative
 The program is assessed for
accomplishment of its goals
◦ For faculty development
◦ For student development
 Methods of Program Assessment
◦ ePortfolios
◦ Student Survey
◦ Faculty Survey
◦ Focus Groups

24. ePortfolio Information
 Electronic portfolios offer three distinct benefits:
1) They are a creative, lively form in which to
show a student’s best work;
2) They allow students to gain an awareness of a
larger audience (a portfolio can be published on an
intranet or the Internet or saved to a CD and given
to college representatives, employers, or anyone a
student wants to impress);
3) They can evolve and grow over a student’s entire
academic career.
"From Worn-Out to Web - Based : Better Student Portfolios." Phi Delta Kappan 85.10
(June 2004): 792-794. Academic Search Premier. EBSCO. [Library name], [City],
[State abbreviation]. 27 Aug. 2009
<http://search.ebscohost.com/login.aspx?direct=true&db=aph&AN=13292501&site
=ehost-live>.

25. ePortfolio Information
 The portfolio not only offers a tool for authentic
assessment but also a means for students to be
reflective practitioners, emphasizing the how and
why as much as the what. Time spent in portfolio
assessment is not time taken away from teaching
or academics, but time refocused and redefined,
with the portfolio viewed as a natural
complement to learning.
Lombardi, Judy. "To Portfolio or not to Portfolio: Helpful or Hyped?." College Teaching
56.1 (Winter2008 2008): 7-10. Academic Search Premier. EBSCO. [Library name],
[City], [State abbreviation]. 27 Aug. 2009
<http://search.ebscohost.com/login.aspx?direct=true&db=aph&AN=31161341&site
=ehost-live>.

26. ePortfolio Information
 Assessment findings derived from students, tutors,
and teachers indicate that electronic portfolios have
had several positive effects on student learning.
These portfolios vividly record writing as a process,
providing students and teachers with an effective
means of assessing the development of that process
over a semester. In addition, they effectively display
the final results of the semester, including a student’s
self-assessments in the form of written reflections.
Click, Ben A., and Sarah C. Magruder.. "Implementing Electronic Portfolios for
Performance Assessment:A Pilot Program Involving a College Writing Center."
Assessment Update 16.4 (July 2004): 1-15. Academic Search Premier. EBSCO.
[Library name], [City], [State abbreviation]. 27 Aug. 2009
<http://search.ebscohost.com/login.aspx?direct=true&db=aph&AN=13998617&site
=ehost-live>.

27. Assessment via ePortfolios
Faculty Administrators
 Used for student and
course assessment
 Students upload all formal
assignments to portfolios
 Allows instructors to look
at student progress
throughout the course
 Students write reflections
on various aspects of the
course, or on the course
as a whole
 Used for course and
program assessment
 Random selection of
portfolios to review across
courses
 10% of portfolios
assessed
 Look at assignments for
writing, critical thinking,
and information literacy
elements

28. Sample ePortfolios
 www.stevecuello.v2efolioworld.mnscu.edu
 www.kristycancel.v2efolioworld.mnscu.ed
u
 www.luzcampusano.v2efolioworld.mnscu.
edu

29. Course Survey
 Used by instructor for course assessment
 Provides the instructor with feedback from
his or her students
 Through narrowly focused questions,
specific feedback is received on:
◦ Exact assignment
◦ Support systems in place (i.e. The Writing
Center, eTutoring.org)
◦ Text (book and reading selections)
◦ Instructor
◦ Grading Rubric

30. Student Survey & Results
1. When you registered for this course, were you
aware that it was a Writing Intensive course?
68% Yes
2. What is your overall opinion of the material
covered in the course?
3. Would you take another WI course? 93% Yes
4. What is your experience with eTutoring.org?
5. What is your experience with the Writing
Center?
6. What did you like best about this course?
7. What, if anything, would you
change/add/remove from the course?

31. Student Testimonials
 “I like to read more because of this class
and Professor Bender.”
 “I never thought I was going to have so
much fun in a Literature course or on an
intensive writing course. I’m glad it was a
great experience.”
 I definitely improved my writing skills. I
think if this was just an intro to lit class it
would be boring, the writing is what made
it fun. I feel so much more confident in my
writing.”

32. WI Student Survey
 Used by WI Administrators for program
assessment
 Conducted across all WI courses
 Neither students nor courses identified
 Allows students to report on various
aspects of the WI experience
◦ Workload
◦ Support services
◦ Self-assessment on improvement in writing,
critical thinking and information literacy

33. WI Student Survey Results
 Were you aware that you registered for a WI
course?
◦ 57% Yes
◦ 43% No
 Rating of services
◦ eTutoring
 Excellent or Good 68.2%
 Fair or Poor 20.5%
 Did not use 11.4%
◦ Writing Center
 Excellent or Good 62.2%
 Fair or Poor 6%
 Did not use 31.8%

34. WI Student Survey Results
 Rating of services
◦ Libguides
 Excellent or Good 50.4%
 Fair or Poor 13.5%
 Did not use 36.1%
◦ ePortfolios
 Excellent or Good 44.7%
 Fair or Poor 24.2%
 Did not use 31.1%
 Rating of skills
◦ Writing
 Improved 71.5%
 About the same 27.7%

35. WI Student Survey Results
 Rating of skills
◦ Information literacy
 Improved 63.4%
 About the same 31.3%
◦ Critical Thinking
 Improved 66.4%
 About the same 29.1%
◦ Technology
 Improved 50.7%
 About the same 42.5%

36. Faculty Survey
 All WI Faculty surveyed
 Survey administered online, anonymous
 Allows faculty to report on services offered in
the Writing Initiative
◦ Institutes
◦ Professional development opportunities
◦ Support services
 Faculty report on observations of student
improvement in the three areas
 Report on own improvement in technology
and media use and use of support services

37. Faculty Survey Results
 Familiar with Initiative goals for students
◦ Use the writing process: 100%
◦ Support of claims with specific evidence: 89%
◦ Edit writing according to the rules of standard
academic English: 100%
◦ Evaluate sources for credibility and academic
appropriateness: 89%
◦ Employ techniques for integrating information:
67%
◦ Exhibit the ability to think critically: 100%
◦ Cite sources using an appropriate
documentation style: 89%

38. Faculty Survey Results
 100% found the goals helpful for improving
student learning
 67% used rubrics for evaluation before
teaching a WI course
 87% used rubrics for evaluation in WI
courses
 33% made use of available tutoring
services for their classes before teaching
WI courses

39. Faculty Survey Results
 100% incorporated tutoring into their WI
courses
 67% felt their students were more
comfortable doing and integrating
research after taking a WI course
 78% felt their students were more aware
of critical thinking after taking a WI
course
 89% felt that WI components had value
for their other courses

40. Focus Groups
 Conducted by Institutional Research
 Recorded discussion of strengths and
weaknesses of program
 WI Administrators and WI Faculty met
separately
 IR wrote report analyzing the results for
each group and making recommendations

41. Focus Group Results
Faculty Administrators
 Felt that there were too
many technological
elements required in
the course
 Wanted an increase in
communication among
WI faculty. They felt
well-supported by the
staff and often
communicated with
them, however.
 Desire more help from
the college community
as a whole to overcome
obstacles to program
success
 See communication,
especially among
faculty and program
administrators, as key
to program success

42. For the Future
Intro. to Literature Writing Initiative
 Work more closely with
counselors/registrar/
advisors to ensure that
students are aware of the
course they are taking
 Possibly incorporate more
use of the Writing Center
 Increase the amount of
available tech support to
allow for easier ePortfolio
use
 Better marketing of WI
courses to students
 Better marketing of the
Writing Center to students
 Better marketing of
professional development
opportunities to WI
faculty
 Allow faculty to use either
paper or online portfolios
for their classes

43. Thank you!
Kelly Bender: kbender@pccc.edu
Elizabeth Nesius: enesius@pccc.edu

44. Portfolio Results
 Writing
◦ Minimum pass rate or higher: 77%
◦ Below passing: 23%
 Critical Thinking
◦ Minimum pass rate or higher: 25%
◦ Below passing (or no evidence of CT): 75%
 Information Literacy
◦ Minimum pass rate or higher: 50%
◦ Below passing in one or more areas: 50%

45. Sample ePortfolios

46. Sample ePortfolios

47. Sample ePortfolios