1. Analysis of Self-Directed Mastery
Learning of Honors Physics
Capstone Defense
University of Florida
June 1, 2011
by Wendy Athens
Good morning. My capstone project on self-directed mastery learning in a physics classroom
has been a great learning experience. It was my first formal study of my teaching practices
and it was enlightening. I couldn’t have reached this point without the insight and support of
my committee members, and I thank you very much.
2. Benjamin Bloom
Why mastery
learning?
LFM
•Every student can master concepts given
sufficient time and support
•Formative testing with feedback &
correctives
•~ 1 sigma learning gains
• Attitudinal gains
Enrichment Activities
Unit 2
Formative
Unit 1 Formative
Assessment A Correctives Assessment B
The origin began two summers ago while reading Benjamin Bloom’s paper called “Learning
for Mastery”, published in 1968. This journal article resonated with me and gave me hope
that I could improve learning outcomes in my physics classroom by flexing time and
providing more 1-to-1 tutoring. Bloom’s fundamental belief was that time, not aptitude, was
the variant of learning given proper support. The basic structure is a formative testing -
feedback -correctives loop with enrichment activities for advanced students.
3. Why self-directed? SDL
• Important life skill
• Social constructivist
• Engaging - appeals to interests &
learning styles
• Frees instructor for one-to-one
tutoring
Providing 1-to-1 time with students drove me to self-directed learning, because I had keep the other students
productively occupied while I met with one. This led me to the SDL literature, and I was grateful because in a
knowledge-driven society, SDL is key to survival and contribution, and much recent attention has been focused on
building these skills earlier in life. The term “self-directed” might mislead - SDL is hardly independent learning
but social constructivist, requiring feedback from instructor and peers to progress. I came to find LFM and SDL to
be mutually reinforcing / symbiotic.
4. Research Design
Three types of support:
expert/peer/computer
tutor (masteringphysics®)
VOLOs
Google Apps infrastructure
Time Tickets
Last summer, I constructed the technology infrastructure to support the self-directed mastery learning
environment. I used Google Apps-based infrastructure that supported more 2-way communication between
teacher-student and student-student. I embedded self-paced learning contracts called VOLOs, which provided
students with a limited choice of learning activities. This image shows three key supports available to students in
blue: the instructor, peers, and the computer tutor (primarily masteringphysics). The course was 25% whole-
class/traditional and 75% self-directed. I used Google Forms to create Time Tickets to track students’ use of time
on a daily basis.
5. VOLO
Power of choice
Learning contract
Animations & simulations
Clear rubric
!
The self-direction was accomplished mainly through the VOLOs. You can see the VOLO for the capstone learning
unit in this slide. It’s this light blue tic-tac-toe box providing links to all activities and worksheets. Students were
required to complete three learning activities, and could choose two more. The simple rubric for VOLOs was 5
boxes completed => 100% for the VOLO grade. The students would highlight the 5 they chose at the outset of the
unit, then mark “DONE” when completed. I gave students the ability to edit the class website and maintain a
personal web page like this one.
6. Context
• Honors Physics - 2 sections
• Private high school
• N = 24
• 50%/50% male/female
• 20% minority
• Pilot study
• Year-long effort
The context for this study were two sections of honors physics at a private high school in Fort Myers for a total of
24 students, split evenly male/female. I kicked off the year using the self-directed mastery learning framework
and during the pilot worked out many of the glitches. The capstone study began in Semester 2 and ran for 4
weeks. The focus of the learning unit was conservation of energy.
7. RQ #1: RQ #2: RQ #3:
What SDL activities How many students Did successful and
did honors physics achieved mastery unsuccessful
students choose in and how did they students differ in
their self-directed spend their time? their perceptions of
mastery learning the self-directed
environment? mastery learning
environment?
Three research questions
The three research questions pursued in this study are shown here. It was interesting to see what learning
activities students chose and how they spent their time. It was even more interesting to compare students who
were successful vs. unsuccessful. Success was defined as clearing the unit test in four tries. Mastery learning is
based on formative testing and these students had a pre-test and four similar tests (not the same). They received
individual feedback from me after each test. I was surprised that all of my students did not clear the test quickly,
but I’m getting ahead of myself...
8. RQ #1: Data
What SDL activities • VOLO activities
did honors physics
• Time Tickets
students choose in
their self-directed Data Analysis
mastery learning
• # SDL activities completed
environment?
• Minutes per SDL activity
• Likert rating of educational
& enjoyment value of each
SDL activity
To answer the first research question I simply had to look at the personal page of each student and see which
activities they chose. The Time Ticket data detailed their time usage. In the post survey, students were also
asked to rate the educational value and enjoyment value of these activities.
9. Data
RQ #2:
• Unit test scores
How many students
achieved mastery • VOLO activities
and how did they • Time Tickets
spend their time?
Data Analysis
• # students who cleared unit
test given four tries
• SDL activities completed
• Minutes per SDL activity
The second research question split my students into two groups: either they were or were not successful in
clearing the unit test within the timeframe of the learning unit. From these two groups comparisons were made
with respect to time allocation, activity choices, and perceptions.
10. Data
RQ #3: • Survey results
Did successful and
unsuccessful Data Analysis
students differ in
•Compare Likert scores of
their perceptions of
successful/unsuccessful students
the self-directed
mastery learning •Mann-Whitney U Test for
environment? individual questions and categories
• Comparison of favorite resources
3 categories:
SDL readiness
Problem solving & physics thinking
SDL-LFM environment
Students were administered a post survey containing 45 questions coming from three validated sources in
addition to custom questions about the self-directed mastery learning environment. I used Guglielmino’s Self-
Directed Learning Readiness Scale to extract questions about SDL readiness, Halloun’s Views About Sciences
Survey (VASS) and the Colorado Learning Attitudes About Sciences Survey (CLASS) to extract questions about
learning physics and gaining a sense of concept mastery. By comparing responses of successful vs. unsuccessful
students, I gained a sense of where student readiness and viewpoints differed.
11. Results
The results of this capstone study surprised me. I really thought when I wrote the unit tests that most students
would clear it by the second try and that everyone would love choosing their learning activities and controlling
their time.
12. Success
!
7 out of 24 successful
Total time: Successful
633 min, unsuccessful
500 min
Successful spent 25%
more time in learning
activities
Successful spent triple
the time in retesting
!
What I found instead was that students struggled to clear the unit test and only 7 of 24 students were successful
within the timeframe of the study. This happened even though I was spending more 1-to-1 time than usual
tutoring them and providing individualized feedback (in fact it turned out that I spent an average of 16 min per
student per class period versus the 4 min that is normally possible). This data points to a limitation of the study,
which was the accuracy of the Time Tickets, however, it was my sense that rather than lecturing I was providing
more targeted 1-to-1 instruction for students.
When students were split successful/unsuccessful, a very interesting pattern of time emerged. Successful
students consistently invested more time in learning activities. In fact the only learning activity that they did not
spend more time in was the “most enjoyable” one, Roller Coaster Tycoon. Overall, successful students invested
25% more time in all learning activities, triple the time in retesting and 44% more time with the instructor.
13. Problem solving
ability and
SDL readiness: alignment with
Perceptions
expert thinking
about the
U = 23, about physics:
self-directed
p = 0.0366
mastery learning
U = 25,
Source: SDLRS p = 0.0121 environment:
Source: VASS, U = 25, p = 0.0121
CLASS
Survey results
Perceptions of successful vs. unsuccessful students showed statistically
significant differences using Mann-Whitney U Test (alpha = 0.05)
Post survey results were reviewed on a per question basis and also in three categories. The three categories were
perceptions about SDL, problem-solving stance and alignment with expert thinking about physics, and the self-
directed mastery learning environment. I used the Mann Whitney U Test to determine statistical differences in the
distribution of the Likert ratings of the successful vs. unsuccessful students. I chose the Mann Whitney U Test
because it was deemed more appropriate for small sample sizes that are non-normally distributed and for data
that is categorical (Huck, 2008).
14. Individual survey prompts that were
statistically different
#14 Understanding what I read is a problem for me.
U = 23, p = 0.0366; average Likert: 3.4 | 4.2
#30 I am satisfied with my grade for this unit.
U = 16, p = 0.0076; average Likert: 4.0 | 2.2
#27 I prefer traditional lecture with weekly lab
compared to self-paced VOLO work.
U = 90, p = 0.0232; average Likert: 1.7 | 3.0
#8 I cannot learn physics if the teacher
does not explain things well in class.
U = 91, p = 0.0209; average Likert: 2.0 |2.9
All individual survey responses are available for your reference at the end of the slides. Differences in Likert
ratings are apparent throughout. This slide shows the four standalone survey prompts that were highlighted to
have statistically different responses between successful and unsuccessful students. The first is such a surprise
for a junior honors physics student at a private high school. Are we dealing with some literacy issues? The
second is understandable since all of these students care about grades -- some more so than learning, as we will
see. #27 was expected because based on learning styles and feedback from the pilot, I knew some of my
students preferred a more traditional classroom, and in fact I had increased the traditional content to 25% from
something lower based on their feedback. The final prompt about needing the teacher gets to the heart of SDL,
and students taking ownership for their own learning. Of course all students want their teacher to explain things
well, but to say “I CANNOT learn physics...” points to self-efficacy differences between students.
15. Epilogue
!
Feedback on unit
performance followed by 5
more days to improve
scores...
51% gain in test scores!
Class average jumped from
57% to 79%!
Before we draw conclusions, it is important to reveal an interesting epilogue to the capstone story because this
event lends credence to the conclusions. As already stated, only 7 of 24 students passed the unit test within 4
tries and within 4 weeks. The average unit test score was a very poor 57% and the overall grades were as shown
here. This data was revealed to students, then they were given 5 more days to improve their scores and retake
the test. We also reviewed the version 4 test. Suddenly everything changed! Now 8 more students cleared the
unit test and the class average moved to 79%. Again, this was a similar, not the same, test, and I didn’t view it as
being any more/less difficult than the previous.
16. Conclusions
Bottom line: Only a minority of these honors
physics students were ready for SDL.
The bottom line of this study is that only a minority (29%) of these students flourished in the self-directed mastery
learning environment and achieved concept mastery.
17. Time on task
Successful students invested 25%
more time in learning activities and
triple the time in retesting.
Consistent with Abar & Loken, 2010.
Time, not aptitude, is the variant of
learning. - Bloom, 1968
Conclusion: Time on task
contributed to success; SDL
instructor must raise student “Time plus energy equals learning.
awareness to manage their own There is no substitute for time on task.”
-Chickering & Gamson, 1991, p. 66
learning (Idros, 2010; Taylor, 1995)
One conclusion from this study is that time invested in learning yields concept mastery. Time on task IS
important, and this quote from Chickering and Gamson expresses the idea so clearly. Since time on task
contributes to success, the SDL instructor must raise students awareness of the importance of their time
management and personal learning goal-setting skills.
18. SDL Readiness Differences
SDL is dependent on the learner, not on the
instructional strategy (Bolhuis, 2003)
Tolerance for uncertainty, motivation,
knowledge base, goal-orientation, and self-
management are factors
Modeling of SDL strategies and expert
thinking is recommended (Pape, 2003;
Verschaffel, 1999)
Conclusion: Gradual transition to SDL needed
with modeling (Abdullah, 2001; Bolhuis,
2003; Breslow, 2003; Gibbons, 2002)
Another conclusion of this study is that capstone students needed SDL coaching, or “modeling”, as recommended
by Pape and Verschaffel. Modeling with discourse is promoted in the literature as a great starting point to build
SDL skills. Instructors model how they categorize problems, how they eliminate certain approaches and adopt
others, and how they solve the problem. Students should practice what the instructor demonstrates to really lock
it in. SDL readiness differences are attributed to differences in tolerance of uncertainty, motivation, knowledge-
base, goal-orientation, and self-management. When viewed from the lens that these are the skills necessary for
success in college, the workplace, and life in general, educators must invest effort in building SDL skills in
students in parallel to content instruction. Instructors need to incorporate emotional support through positive
feedback and encouragement as well as providing Learning to Learn strategies suggested by Pape and Verschaffel.
19. Perceptions about Physics
Successful students showed greater
confidence in problem solving and closer
alignment with expert thinking.
Successful students showed intention in the
selection of SDL activities and fit new
information into cognitive framework.
Conclusion: Metacognition is an important
aspect of SDL and relates to concept
mastery (Bransford, 2000; Gagne, 1998;
Halloun & Hestenes, 1998)
The third conclusion of the study was that successful students were much more intentional in
why they picked certain learning activities and they were aware of what they gleaned from
each activity and how it fit into their cognitive framework. Considering this was iteration #2
following the pilot, these students had already adopted “best practices” that made them
successful including particular resources. The pattern recognition of experts is what I am
trying to represent with this chess board: experts see patterns and can quickly categorize
problems, which frees their cognitive resources for thinking. Novices remain mired in details.
According to post survey prompts, successful students had already developed a conceptual
framework around which they constructed new knowledge. The recommendation to
instructors therefore becomes, how can I organize information to make it more intelligible
and connected for the student? Especially in SDL settings, concept maps, whole-class “big
ideas” lectures, and peer critique help tie together concepts and create “sense-making”.
20. Learning Environment Perceptions
The majority of students were positive about the SDL-LFM learning
environment (86% | 76%).
Peer interaction was the most time-consuming SDL activity.
Formative testing and masteringphysics® were praised.
Conclusions: Positive responses were an outcome of the power of choice
(Knowles, 1975; Tomlinson, 2003); frequent and individualized feedback
(Bloom, 1984; Chickering & Gamson, 1991); adequate support (Kirschner,
2006); lack of competition (Bloom, 1968); peer interactions must be more
purposeful (Crouch & Mazur, 2001; Merrill & Gilbert 2008).
Technology infrastructure was effective in the background and supported the
learning process.
The third conclusion of the study was that students were positive about the self-directed
mastery learning environment, more so for successful students. The literature supports this
outcome for several reasons. Giving students a choice of learning activity is motivating in
itself, besides the fact that they can follow their interests. Regarding feedback, feedback is
the fuel of learning and students flourish with frequent and individualized feedback.
Regarding support, SDL students must have adequate support to be successful and capstone
students used all 3 forms of support and expressed they were adequately supported.
Formative testing was praised because it gave students a chance to learn from their mistakes
in a targeted way and also removed competitive pressure. Finally, peer interaction was
important and was the most significant use of time, however it could be more refined. The
technology infrastructure ran appropriately and effectively in the background.
21. Profile of a Self-Directed Honors Physics Student
A profile of a self-directed honors physics student emerged from this study.
22. SDL readiness
Actively constructed knowledge
into cognitive framework
Prior knowledge through discourse and feedback
Prioritized time Enjoyed the learning process
Metacognitive intention Persisted
Used many resources Motivated
Profile of a Self-Directed Honors Physics Student
The attributes that seemed important for students to be successful in the capstone setting
are shown here. Self-direction contributed to concept mastery. For the four week
timeframe they were given, successful students prioritized their time well and used many
resources, including human resources of the instructor and peers. They persisted in
formative testing and doing homework. They were motivated to work and they were
intentional about how they used their time. They filled in gaps of knowledge through
discourse and self-study.
23. Recommendation
Content- and Process-Oriented Instruction (Bolhuis, 2003)
Because self-directed learning skills facilitate concept mastery and are deemed important life
skills in our society, an instructor needs to balance teaching content with teaching self-
direction. Bolhuis (2003) expresses this balancing act as process-oriented instruction, which
focuses on four principles.
24. Constructive friction
Recommendation
Content- and Process-Oriented Instruction (Bolhuis, 2003)
The first is to maintain healthy “constructive friction” and move gradually to SDL because
learners differ in readiness.
25. Constructive friction
Content
Recommendation
Content- and Process-Oriented Instruction (Bolhuis, 2003)
The second is to focus on building content knowledge, especially by making the learning
more meaningful through real-world, problem-based learning.
26. Constructive friction
Content Emotional support
Recommendation
Content- and Process-Oriented Instruction (Bolhuis, 2003)
The third is to pay attention to the emotional aspects of learning through positive feedback,
helping students embrace uncertainty, fostering motivation, and modeling the self-
gratification of solving a hard problem.
27. Social context
Constructive friction
Content Emotional support
Recommendation
Content- and Process-Oriented Instruction (Bolhuis, 2003)
The fourth principle is to treat learning as a social phenomenon and facilitate social skills and
effective cooperative learning so learners can more effectively construct their knowledge
through discourse with the instructor and peers.
29. Credits
Slide #1: Tutor: math4allages.wordpress.com
Benjamin Bloom link
SDL images: http://www.selfdirectedlearning.com
Hands with time image link
Tutor image link
Chess board image link
Students image link
Balancing act link
