Position Paper on Effectiveness Study of Blended Learning Model - 2007-2009

1. Daria Valentini, PhD (Stonehill College) and Mark Lewis, PhD (Regis College
and University of Massachusetts Boston)
Immersion and Continuity: A Blended Model for College Level
Introductory Language Courses
Italian 101-102: 2007-2009
Introduction
The principal investigators for this research project are:
Daria Valentini, who has taught Introductory Italian for many years at
Stonehill College, where blended learning environments have yet to be
implemented on a broad scale
Mark Lewis, who participated in a grant to develop and implement blended
learning across different sectors of the liberal arts and sciences curriculum at
Regis College from 2003-2008
The project idea evolved from the opportunity to compare students in the blended
course with those in the traditional face-to-face course, in order to ascertain the
level of effectiveness of the blended learning model.
Results include survey data from students in the blended sections, as well as a
comparison of student grades on multiple assignments. Also discussed will be the
decision by Professor Valentini to adopt e-learning strategies and resources at
Stonehill College. Specifically, she has created a web-enhanced Introductory Italian
course that continues to take full advantage of the three weekly face-to-face
meetings, while expanding students’ exposure to the language outside class.
Context: Blended Course Design and Instructional Objectives
Students in first and second semester Italian demonstrated their knowledge using a
variety of methods. In class, they worked for a portion of each class period with a
partner to a) act out a dialogue or b) transform sentences using newly introduced
vocabulary and focusing on newly introduced grammar topics. Similarly, outside of
class each student completed between eight and ten sets of computer-based online
exercises and submitted them to the publisher’s course management system for
review by the instructor. Spread out over the 15 weeks of the semester, these
assignments counted for a total of 20% of their grade for the course. Each week
students received a homework grade, which allowed them to monitor their own
progress and performance. While opportunities for students to practice their
1

2. Daria Valentini, PhD (Stonehill College) and Mark Lewis, PhD (Regis College
and University of Massachusetts Boston)
spoken Italian were frequent within these sets of exercises, assessment and
improvement of their spoken Italian occurred only in the face-to-face classroom. 1
Furthermore, four chapter tests were taken in class to assess comprehensive
knowledge. Like the online exercises, these tests emphasized grammar, vocabulary,
culture, and communication pertinent to everyday situations. Also like the online
exercises, the tests assessed students’ skill development in listening, reading, and
writing. In addition to the chapter tests, three timed quizzes were administered
online using the learning management system’s quiz generator.
In addition to quizzes, the functions of the LMS most heavily relied upon for
instruction were the online gradebook, announcements, links to Internet resources,
and the uploading of instructor generated text and image resources as a supplement
to instruction. Such resources helped to personalize the online learning
environment, and they were often spontaneously added, accompanied by an
announcement as well as an email message alerting students to the new content.
Timed release of content in the LMS helped student stay focused on each week’s
lesson, although content from previous weeks was continuously available.
Our primary objectives in this redesign of Introductory Italian were as follows:
Enable students to succeed with 33% reduction in seat time
Make effective use of f2f class time to improve communicative competence,
and shift a larger portion of comprehension and review of structure to online
environment
Increase the frequency of interactive assignment due dates, and provide rich
and rapid feedback on each one
Create a preferred overall language learning experience
Comparison of Blended Model with F2F Model
Our research compares the blended format for Introductory Italian (50 mins.
twice weekly f2f + unlimited asynchronous online) with traditional format
(50 mins. thrice weekly with limited web-enhanced component)
Course design was streamlined by assigning dynamically released interactive
exercises twice weekly. This content provided a concentrated amount of
interactive learning and practice using “Quia”, the online lab manual to
accompany a major textbook, Prego! 7th edition (McGraw Hill, 2007)
1 In an online course with a virtual classroom, students could record themselves
using widely available tools for digital audio, and instructors could provide
individualized feedback either in written or oral form at several points throughout
the course.
2

3. Daria Valentini, PhD (Stonehill College) and Mark Lewis, PhD (Regis College
and University of Massachusetts Boston)
Variables that were consistent between the two delivery models include:
same course duration and approximate number of students, same textbook,
same material covered in the same amount of time, same quizzes and exams
administered
Two different instructors taught these courses at two different private
Boston area colleges (Regis College and Stonehill College), working to ensure
consistency
Four semesters of teaching were studied
Methods & Results
A Student Evaluation of Blended Course Design survey was conducted using the
TLTGroup’s Flashlight Online survey generator, version 1.0, and was administered
for three populations of IT 101-102 students in three different semesters between
2008 and 2009. The survey contained 27 items.2
Student Evaluation of Blended Course Design results:
Percentage of students who agreed with the following:
86% - I learned the same as (43%) or more than (43%) in a traditional format.
71% - I was better able to understand ideas and concepts in this course.
71% - I gained confidence in my ability to learn difficult subject matter.
78% - I learned at my own pace and was encouraged to spend more time on task.
78% - I was better able to visualize ideas and concepts taught in this course.
100% - The instructor gave clear explanations of what was expected in a blended
course.
In addition, the standard paper-based course evaluation was administered. Both
instructors received positive feedback about their respective courses and teaching
abilities, including accessibility outside of class.
Finally, grades for these sections of IT 101-102 were compared with grades earned
by students in traditional IT 101-102 courses that did not use technology-assisted
learning and were not taught as blended courses.
Blended student grades (ave. over two years)
o Homework – 90.2%
o Tests – 91.0%
2Some of the wording from the original items in Flashlight 1.0 has been either
adapted or abbreviated.
3

4. Daria Valentini, PhD (Stonehill College) and Mark Lewis, PhD (Regis College
and University of Massachusetts Boston)
o Course – 90.7%
Face-to-face (ave. over two years)
o Homework – 89.9%
o Tests – 89.9%
o Course – 87.4%
4