Call Girls Electronic City Just Call 7001305949 Top Class Call Girl Service A...
DGHD Program Manager Retreat Evaluation Presentation
1. IMPROVING OUTCOME
EVALUATION
DGHD Program Managers Retreat -July 25 2013
Christopher Fink, PhD
Ohio Wesleyan University
Dept. of Health & Human Kinetics
Thursday, July 25, 13
2. OVERVIEW
• Importance of Evaluation
• Improving Evaluation through
Planning (Logic model)
• Evaluation Considerations and
Methods
• Application to DGHD
Priorities
• Discussion
Thursday, July 25, 13
3. TEN ESSENTIAL SERVICES
OF PUBLIC HEALTH
1. Monitor health status to identify
community health problems.
2. Diagnose and investigate health
problems and health hazards in the
community.
3. Inform, educate, and empower people
about health issues.
4. Mobilize community partnerships to
identify and solve health problems.
5. Develop policies and plans that
support individual and community
health efforts.
6. Enforce laws and regulations that
protect health and ensure safety.
7. Link people to needed personal health
services and assure the provision of
health care when otherwise
unavailable.
8. Assure a competent public health and
personal healthcare workforce.
9. Evaluate effectiveness, accessibility, and
quality of personal and population-
based health services.
10.Research for new insights and
innovative solutions to health
problems.
http://www.apha.org/programs/standards/performancestandardsprogram/
resexxentialservices.htm
Thursday, July 25, 13
4. TEN ESSENTIAL SERVICES
OF PUBLIC HEALTH
1. Monitor health status to identify
community health problems.
2. Diagnose and investigate health
problems and health hazards in the
community.
3. Inform, educate, and empower people
about health issues.
4. Mobilize community partnerships to
identify and solve health problems.
5. Develop policies and plans that
support individual and community
health efforts.
6. Enforce laws and regulations that
protect health and ensure safety.
7. Link people to needed personal health
services and assure the provision of
health care when otherwise
unavailable.
8. Assure a competent public health and
personal healthcare workforce.
9.Evaluate effectiveness,
accessibility, and quality of
personal and population-
based health services.
10.Research for new insights and
innovative solutions to health
problems.
http://www.apha.org/programs/standards/performancestandardsprogram/
resexxentialservices.htm
Thursday, July 25, 13
5. "It does not take much strength to do things, but it
requires a great deal of strength to decide what to do."
Elbert Hubbard
http://www.pickthebrain.com/blog/the-strength-to-overcome/
Thursday, July 25, 13
42. DETERMINANTS:ASSESSING
IMPORTANCE & CHANGEABILITY
Determinants Importance Changeability
Evidence for
Importance
Knowledge + +++
Precondition for
personal attitude
Risk perception + +
Precondition for
personal relevance
Self-efficacy +++ + r=0.71, p<0.01
Personal norm +++ + r=0.78, p<0.01
Subjective norm + + r=0.29, p<0.01
Skills ++ +
Precondition for self-
efficacy improvement
Thursday, July 25, 13
43. EVALUATION &
THEORETICAL FRAMEWORK:
LOGIC MODEL
Change
Objectives
Personal
Determinants
Performance
Objectives
Behavioral
Outcomes
Health
QOL
Improvement
Change
Objectives
Performance
Objectives
Environmental
Outcomes
Personal
Determinants
At-Risk Group
Environmental Agent
{
Logic of Change
{Outcomes
Thursday, July 25, 13
44. MATRIX FOR CHILDREN IN
SPF PROGRAM
Behavior: Wear Sunscreen and Protective Clothing When OutdoorsBehavior: Wear Sunscreen and Protective Clothing When OutdoorsBehavior: Wear Sunscreen and Protective Clothing When OutdoorsBehavior: Wear Sunscreen and Protective Clothing When OutdoorsBehavior: Wear Sunscreen and Protective Clothing When Outdoors
Personal DeterminantsPersonal DeterminantsPersonal DeterminantsPersonal DeterminantsPersonal Determinants
Performance
Objectives
(Children)
Attitudes Skills/Self-Efficacy Knowledge
Outcome
Expectations
PO 1. Cooperate
with sun
protection
practices by
parent or
preschool staff
A.1 Express
positive feeling
toward being
protected from
the sun
OE 1. Describe how
cooperating will keep
skin healthy
PO 1.1 Stand still
for application
A. 1.1 Express
positive attitude
toward being a
helper
K 1.1 Explain
how standing still
allows sunscreen
to be put on
evenly
Thursday, July 25, 13
45. MATRIX FOR ENVIRONMENT
IN SPF PROGRAM
Organizational Environmental Change in SPF ProgramOrganizational Environmental Change in SPF ProgramOrganizational Environmental Change in SPF ProgramOrganizational Environmental Change in SPF ProgramOrganizational Environmental Change in SPF Program
Personal DeterminantsPersonal DeterminantsPersonal DeterminantsPersonal DeterminantsPersonal Determinants
Performance
Objectives
(Agents)
Perceived
Norms
Attitudes Skills/Self-Efficacy
Outcome
Expectations /
Perceived
Susceptibility
PO 1. Preschool
directors decide
to prioritize
increasing amt. of
shade areas used
by preschoolers
PN 1. Recognize
sun protection
and shade
adequacy as
concerns for
preschool
administration
A 1. Describe
assuring shade as
positive
OEPS 1. a Describe
children as
susceptible to skin
cancer caused by
lack of shade
OEPS 1. b.Argue
that increasing shade
will decrease risk
Thursday, July 25, 13
48. LOGIC MODEL LINKTO
EVALUATION
• Behavioral / Environmental Outcomes = Impact of Activities/Program
• Performance Objectives = Effectiveness of Activities/Program
• Specificity of objectives is key
• Uncover key factors for behavioral/environmental success
• Often help to define the behavior or environmental condition (improving
evaluation)
Thursday, July 25, 13
49. LOGIC MODEL LINKTO
EVALUATION
• Behavioral / Environmental Outcomes = Impact of Activities/Program
• Performance Objectives = Effectiveness of Activities/Program
• Specificity of objectives is key
• Uncover key factors for behavioral/environmental success
• Often help to define the behavior or environmental condition (improving
evaluation)
• Change Objectives = Immediate impact
Thursday, July 25, 13
50. What we observe is not nature itself, but nature
exposed to our method of questioning.
-Werner Heisenberg
http://resources0.news.com.au/images/2012/11/06/1226511/766824-solar-eclipse-glasses.jpg
Thursday, July 25, 13
55. PROCESS EVALUATION
• Reach: The proportion of the intended audience to whom
the program is actually delivered
Thursday, July 25, 13
56. PROCESS EVALUATION
• Reach: The proportion of the intended audience to whom
the program is actually delivered
• Dose Delivered:The amount of intended units of each
program component that is delivered
Thursday, July 25, 13
57. PROCESS EVALUATION
• Reach: The proportion of the intended audience to whom
the program is actually delivered
• Dose Delivered:The amount of intended units of each
program component that is delivered
• Dose Received:The extent to which the participants
engage with the program
Thursday, July 25, 13
58. PROCESS EVALUATION
• Reach: The proportion of the intended audience to whom
the program is actually delivered
• Dose Delivered:The amount of intended units of each
program component that is delivered
• Dose Received:The extent to which the participants
engage with the program
• Fidelity:The extent to which the program was delivered as
intended
Thursday, July 25, 13
66. • Interviews with key informants
QUALITATIVE + QUANTITATIVE
MEASUREMENT
Qualitative Quantitative
Thursday, July 25, 13
67. • Interviews with key informants
• Observations of program activities
QUALITATIVE + QUANTITATIVE
MEASUREMENT
Qualitative Quantitative
Thursday, July 25, 13
68. • Interviews with key informants
• Observations of program activities
• Focus groups
QUALITATIVE + QUANTITATIVE
MEASUREMENT
Qualitative Quantitative
Thursday, July 25, 13
69. • Interviews with key informants
• Observations of program activities
• Focus groups
• Thematic analysis
QUALITATIVE + QUANTITATIVE
MEASUREMENT
Qualitative Quantitative
Thursday, July 25, 13
70. • Interviews with key informants
• Observations of program activities
• Focus groups
• Thematic analysis
QUALITATIVE + QUANTITATIVE
MEASUREMENT
Qualitative Quantitative
Thursday, July 25, 13
71. • Interviews with key informants
• Observations of program activities
• Focus groups
• Thematic analysis
QUALITATIVE + QUANTITATIVE
MEASUREMENT
Qualitative Quantitative
Thursday, July 25, 13
72. • Interviews with key informants
• Observations of program activities
• Focus groups
• Thematic analysis
• Demographics
QUALITATIVE + QUANTITATIVE
MEASUREMENT
Qualitative Quantitative
Thursday, July 25, 13
73. • Interviews with key informants
• Observations of program activities
• Focus groups
• Thematic analysis
• Demographics
• Pre-post test scores
QUALITATIVE + QUANTITATIVE
MEASUREMENT
Qualitative Quantitative
Thursday, July 25, 13
74. • Interviews with key informants
• Observations of program activities
• Focus groups
• Thematic analysis
• Demographics
• Pre-post test scores
• Ratings of attitude and self-efficacy
QUALITATIVE + QUANTITATIVE
MEASUREMENT
Qualitative Quantitative
Thursday, July 25, 13
75. • Interviews with key informants
• Observations of program activities
• Focus groups
• Thematic analysis
• Demographics
• Pre-post test scores
• Ratings of attitude and self-efficacy
• Existing numerical data
QUALITATIVE + QUANTITATIVE
MEASUREMENT
Qualitative Quantitative
Thursday, July 25, 13
76. • Interviews with key informants
• Observations of program activities
• Focus groups
• Thematic analysis
• Demographics
• Pre-post test scores
• Ratings of attitude and self-efficacy
• Existing numerical data
• Statistical analysis
QUALITATIVE + QUANTITATIVE
MEASUREMENT
Qualitative Quantitative
Thursday, July 25, 13
78. EXAMPLES OF QUALITATIVE +
QUANTITATIVE DATA USE
What is being
measured?
Quantitative Data Qualitative Data
Awareness
Written instrument (y/n) to
measure awareness of PSA on
healthy eating
Interviews at local supermarkets
to measure awareness of healthy
eating campaign
Knowledge
Written/oral knowledge
questionnaire (pre-post, likert) to
measure knowledge gained at
nutrition seminar
Interviews to measure
knowledge about nutritional
value of f/v, focus groups w/
children about same
Attitudes
Written instrument (pre-post,
likert) to measure attitudes and
preferences of children for f/v
Interviews with mothers of small
children to ID attitudes toward
feeding kids f/v
Behavior
Self-report w/ scaled responses
to measure freq. + type of
healthy eating, obs. counts
Interviews with individuals to
gather info on eating behaviors,
focus groups, obs.
Thursday, July 25, 13
79. EXAMPLES OF QUALITATIVE +
QUANTITATIVE DATA USE
What is being
measured?
Quantitative Data Qualitative Data
Policy changes
Measurement of number of
children who consume f/v before
and after school policy requiring
salad bars in school
Interviews with children in
school about what they like/
dislike on salad bar since policy
implemented
Organizational Changes
Records documenting # and
demographics of people who go
to food bank for food + now get
info on preparing nutritious food
and food stamp program
Interviews w/ food bank staff to
gather opinions about whether
food bank’s new nutrition ed and
food stamp outreach has positive
effect on clients
Environmental Changes
Survey of farmers’ markets in the
county to document the size of
the markets, types of produce
provided, cost of produce
Observations of and interviews
with people attending farmers’
markets to identify satisfaction
with market
Thursday, July 25, 13
80. EVALUATION:
OTHERTHOUGHTS
• Link between objective, outcome, and measurement is key
• Follow SMART objective guidelines to make evaluation easier
and more effective
• Use theory and logic model (where possible) to guide the
process to make change more likely, evaluation more effective
• But, don’t get stuck there!
• Be sure to have process, impact, outcome represented in plan
Thursday, July 25, 13
81. DGHD PRIORITIES:
THOUGHTS
• A healthy community, a chance to
climb even higher
• Great efforts, great people, great
community buy-in
• Excellent priorities and processes
• Where to improve?
Thursday, July 25, 13
82. DGHD PRIORITIES:
SOME EXAMPLES
OBJECTIVE /
Tactic
TIMEFRAME OUTCOME
MEASUREMENT
a. unit b.collection method
Determine sources
and then monitor,
evaluate and
disseminate county
wide and community
specific obesity data.
June 1, 2012 -
December 31, 2012
for determination of
data sources, then
annually each January
Determine obesity
trends for Delaware
County.
a. % of county residents that are
obese, evaluate change over time
and disseminate information to
community via agency website
b. review of standardized list of
data
Goal 1.1
Thursday, July 25, 13
83. DGHD PRIORITIES:
SOME EXAMPLES
OBJECTIVE /
Tactic
TIMEFRAME OUTCOME
MEASUREMENT
a. unit b.collection method
100% of the clients
requesting birth
certificates will be
given a safe food
handling brochure.
Ongoing
Improved knowledge
of safe food handling.
a. Number of birth certificates
requested and count of those
receiving food safety information
b. Number that receive food safety
information
Goal 3.7
Thursday, July 25, 13
84. DGHD PRIORITIES:
SOME EXAMPLES
OBJECTIVE /
Tactic
TIMEFRAME OUTCOME
MEASUREMENT
a. unit b.collection method
Utilize Wii game
system and “Impaired
Driving Fatal Vision
Mario Cart” at least 10
times, with targeted
populations, annually.
July, 2010 (with
borrowed system), and
ongoing
Raise awareness of
impaired youth driving
in a safe environment.
a. The number of events at which
an interactive impaired driving
program is utilized with high school
students
b. Tracking by staff
Goal 4.2.1
Thursday, July 25, 13
85. SUMMARY
• Importance of evaluation
• Importance of planning and
objectives in evaluation
• Consistency and using a
framework
• Method based on needs, goals
• Asking the right questions is
key!
Thursday, July 25, 13
86. Map 0
Full Generic Map
nothing is perfect. life is messy. relationships are
complex. outcomes are uncertain. people are irrational.
hugh mackay
Thursday, July 25, 13
87. If anyone on the verge of action should judge himself
according to the outcome, he would never begin."
Søren Kierkegaard
Thursday, July 25, 13
88. Thanks!
Presentation PDF available at:
http://slideshare.net/ChristopherFink
Harvard Campaign & Public Communication Evaluation Example
http://db.tt/pufWl30z
Healthy Maine Healthy Weight Awareness Campaign Evaluation
http://db.tt/zVqSQ7hP
WHO ImmunizationTracking & Data Management Info
http://db.tt/OLu3wEuE
Thursday, July 25, 13
