The University of Idaho Extension has organized a Sixth Grade Forestry Tour in Clearwater County, Idaho for the last 51 years. During 2012, the program was evaluated to examine knowledge gained and retained by students in Clearwater County and track changes in behavior and attitudes about natural resources, and general satisfaction with the program’s logistics, staff, and facilities. All sixth grade students at Clearwater County schools were given a pretest in the spring consisting of questions about their knowledge of natural resources and their attitudes and behaviors towards natural resources. They were also asked to draw a picture of themselves in the forest. Students who attended the three-day tour were given the same test on the last day of the tour. A delayed posttest was given in the fall to all previously tested students, now beginning the seventh grade. Clearwater County students who did not attend the three-day tour served as the control group. The pre/posttest experimental design approach was used to determine the beginning knowledge of students, the knowledge gained as well as retained, and the attitude/behavioral changes which could be attributed to program attendance as opposed to maturity.
#nexc2012
Clearwater County Sixth Grade Forestry Tour: Program Evaluation Methods and Tools
1. Clearwater County SGFT Program Evaluation
Jillian Marotz Moroney
University of Idaho
Bioregional Planning and Community Design
2. Roadmap
• Program History
• 2012 Program Evaluation
• Methods/Findings
• Recommendations
• Tools for Assessment of Participants
• Knowledge
• Behaviors/Attitudes
• Perceptions
• Lessons learned and suggestions
3. Program History
Program History
• Established 1961
• Completely free to participants
• Three days, two nights
• Teacher continuing education
certification through University of Idaho
• Covers topics including:
• Forestry
• Fish and wildlife
• Range
• Water Quality
• Geology
• Search and Rescue
• Fire
4. 2012 Forestry Tour
• 32 students
• 15 boys/17 girls
• 11-12 years old
• Have parents
who work in the
logging
industry: 9
• Born outside of
Idaho: 19
6. Participant Assessment
of Program : Day 1
Wetlands
July 10, 2012 Event Assessment
Standard
Tour Event Mean NezPerce Median Mode
Fisheries
Deviation
Pierce Pool • Heat/Sun
3.97 0.16 4 4
Empire Lumber
• Instructors
3.74 0.44 4 4
• Activities
Outdoor Skills 3.44 0.75 4 4
Tree ID
Resource Careers 3.28 0.76 3 3
Dwarshak Fisheries 3.21 0.73 3 3
Wetlands 3.18 0.88 3 4
Rodeo
Nez Perce Fisheries 3.13 0.70
Grounds 3 3
Tree ID 3.05 0.94 3 4
Rodeo Grounds 3 0.83 3 4
(1=Poor, 2=Fair, 3=Good, 4=Excellent) N=32
7. Participant Assessment
of Program : Day 2
July 11, 2012 Event Assessment
Standard
Tour Event Mean Median Mode
Deviation
Fire Control and Airplane Drop 3.95 0.22 4 4
Mining Demo • 95% Said 3.92 they learned
that 0.27 4 4
Search and Rescue something this day 0.53
3.68 4 4
ID Dept. of Lands 3.51 0.61 4 4
Pierce Museum • Events with hands on
3.42 0.67 3 3
Jaype Mill Pond activities scored highest
3.39 0.68 3.5 4
Potlatch Forestry Management 3.39 0.69 3.5 4
Forestry Consulting 3.32 0.70 3 4
Group Activity 3.22 0.76 3 3
NRCS 2.89 0.77 3 3
(1=Poor, 2=Fair, 3=Good, 4=Excellent) N=32
8. Participant Assessment
of Program : Day 3
64% of participants said that the
July 12, 2012 Event Assessment
lookout was their favorite part of
Thursday.
Tour Event Mean Standard Deviation Median Mode
Bertha Hill Lookout 3.92 0.28 4 4
Outdoor Challenge 3.57 0.61 4 4
Natural Resource
3.51 0.61 4 4
Activity
Group Activity 3.4 0.65 3 4
Woody Biomass 3.2 0.80 3 3
Logging Operations 3.06 1.01 3 4
Tree ID 2.94 1.01 3 4
(1=Poor, 2=Fair, 3=Good, 4=Excellent) The Tree ID quiz scored the lowest out ofN=32
any
activity from the whole tour.
http://idahoptv.org/outdoors/shows/eyesoftheforest/berthaHillLookout.cfm
9. Participant Assessment
of Program : Day 3
• Staff
• Adults
• Teens
• Guest Speakers
Staff Assessment
Standard
Staff Member Mean Median Mode
Deviation
Adults 3.75 0.82 4 4
Guest Speakers 3.74 0.61 4 4
Teens 3.57 0.65 4 4
(1=Poor, 2=Fair, 3=Good, 4=Excellent) N=32
11. Recommendations
• Keep time of day in mind when planning the
schedule
• Hands on activities with lectures
• More training/screening of teen leaders
• More free time in the evenings
12. Recommendations Cont.
• Communicate program themes to the presenters
• Connect students to Idaho natural resources
• Increase student interest in natural resources
• Communicate what impact humans have on
nature and natural resources
• Increasing the comfort and connection students
have with the outdoors
14. Pre-Test Post-Test Delayed Post-Test
Administered Administered at the Administered fall
spring 2012 to all end of the three day 2012 to all seventh
sixth grade Forestry Tour to all grade students in
students in participants. Clearwater County,
Clearwater County, Idaho.
Idaho.
15. Knowledge Gain and Retention
• 20 Questions
• Questions submitted by presenters, program
organizer, state and federal curriculum
• Example:
What is woody biomass used for?
a. Generating electricity
b. Heating homes and other buildings
What are plants that c. Making liquid fuel
annually lose their d. All of the above
leaves called?
a. Deciduous
b. Nocturnal Fish have a very good sense
c. Naked of smell.
d. Coniferous a. True
b. False
16. Pre and Post-Test Averages
Pre-Test Post-Test
Control Average: 11.9 Control Average: 11.8
Participant Average: 12.2 Participant Average: 13.8
Control N= 32 Participant N= 25
18. Participant Averages by Gender
Pre-Test Post-Test Delayed Post-Test
Male Mean: 12.8 Male Mean: 13.7 Male Mean: 14.1
Female Mean: 11.7 Female Mean: 13.9 Female Mean: 13.5
N Males= 12 N Females= 13
19. Attitudes/Behaviors
• 10 Statements
• Statements from extensive literature review,
adapted to age group
• Example:
STRONGLY DON’T STRONGLY
DISAGREE AGREE
DISAGREE KNOW AGREE
I am interested in forestry
1 2 3 4 5
and natural resources
I talk about natural
1 2 3 4 5
resources with my friends
I feel good when I am outside
1 2 3 4 5
in nature
I feel a strong connection to
1 2 3 4 5
the forests in Idaho
20. Change in Behaviors and Attitudes
5
4
3
Rating
2
1
0
1 2 3 4 5 6 7 8 9 10
Question Number
21. 5
Pre-Test
Control v Participants Behaviors and Attitudes
4
Rating
3
2
1
0
1 2 3 4 5 6 7 8 9 10
Question Number
Post-Test
5
4
Rating
3
2
1
0
1 2 3 4 5 6 7 8 9 10
Question Number
22. Forest Perceptions
• “Draw yourself in a forest.”
• Looked for before and after differences in
drawings
• Tree number/type
• Natural features
23. “Draw yourself in a forest.”
• 18/25 (72%) students who attended the tour
had more in depth drawings
• 13/32 (41%) who took only pre and delayed post
test showed more in depth drawings
Pre-Test Post-Test Delayed Post-Test
26. Lessons and Suggestions
• 2012 Program Evaluation
• Methods
• Follow up questions to yes/no
• Communicate with presenters program goals
and themes
• Consider weather, age group, activities…
• Tools for Assessment of Participants
• Many tools (knowledge, behaviors, attitudes,
perceptions, demographics….)
• More tools means more data- pros and cons
27. Jillian Marotz Moroney
University of Idaho
maro2762@vandals.uidaho.edu
Thank you!
Thesis Committee
Dr. Tamara Laninga
Dr. Randy Brooks
Dr. James Connors