Using program theory-driven evaluation in primary care units

1. Using program theory-driven
evaluation in primary care units
An integrated evaluation perspective on
a learning project providing equitable care in Sweden
The 7th Nordic Research Conference in Health Promotion Promoting Health in Everyday
Settings: Opportunities and Challenges, 17-19 June 2013, Vestfold University College in
Norway
Osman Aytar, Sharareh Akhavan, Marianne Granath, Hanna
Wallin,
Lennart Bogg, Anne Söderlund and Per Tillgren

2. School of Health, Care
and Social Welfare
School of Health, Care
and Social Welfare
● To present the program theory for evaluation of
Lärandeprojekt - Vård på lika villkor (Learning Project -
Care on equal conditions) which is a national project
from the Swedish Association of Local Authorities and
Regions (SALAR) in cooperation with the Swedish
Government.
●In this project SALAR will, together nine primary care
units and the research group from MDH, identify and
develop methods and procedures that facilitate a more
setting based equitable care.
● The evaluation is conducted by a research group at
Mälardalen University (MDH).
2
Purpose of the
presentation

3. School of Health, Care
and Social Welfare
Costs
Perspective:
- Patients
- Caregivers
- Healthcare payers
Financial and non-
financial
Focus on caregivers
Short-, medium- and
long-term
Direct and indirect
Intended and
unintended
Identifying and
testing practices and
methods for a more
setting based
equitable care
Identified and tested
practices and
methods are
available to all
counties and regions
Initiation,
coordination and
research support
Learning seminars
Teamwork at care
units
Agreements
between:
- The government
and SALAR
- SALAR and care
units
- SALAR and MDH
Resources
(personnel, budget
etc.)
Goals and
resources
OutcomesOutputsActivities
External factors
Central, regional and local
Stakeholders
Patients, caregivers, healthcare payers, SALAR, the goverment, MDH
Program theory for Learning Project - Care on equal conditions
(Lärandeprojekt - Vård på lika villkor)
© MDH and SALAR (Revised version, 2013-04-29)

4. School of Health, Care
and Social Welfare
School of Health, Care
and Social Welfare
● The program theory model which has been developed
early in the project includes three types of evaluation:
● process evaluation
● performance evaluation
● health economic evaluation
● Focus on stakeholders and external factors.
● Focus on outcomes from different perspectives.
● This model is developed for the project at the national
level, but can be used as a source of inspiration even by
the care units involved in the project.
4
The aims of using
program theory

5. School of Health, Care
and Social Welfare
School of Health, Care
and Social Welfare
● Initiation, coordination and research support:
● Invitation from SALAR to care units
● Applications of interest from care units
● Meeting notes
● Interviews with some selected persons
● A self-assessment by the research team at MDH
● Learning seminars:
● Activities at seminars
● Participant surveys
● Teamwork at care units
● Team surveys
● Interviews with some selected tutors and participants
5
Process evaluation

6. School of Health, Care
and Social Welfare
School of Health, Care
and Social Welfare
● A web Questionnaire in the beginning and in the end of the project
Results shows responses to close-ended questions regarding;
● Factors that are important for access to health care on equal terms
● Organizational / structural criteria to provide care on equal terms
● Operations-factors to provide care on equal terms
● Factors affecting improvement and change
● Three important resources to effect change
Results from the open-ended questions shows responses to issues as;
● The meaning of "care on equal terms”
● Patient/client groups who find it difficult to be heard
● Patient /client groups that are more likely to get attention
● Factors (in addition to those as already mentioned in close-ended
questions) that affect clients/patients access to care on equal terms
● Factors which may influence health care facilities to offer health care
on equal terms
6
Performance evaluation

7. School of Health, Care
and Social Welfare
School of Health, Care
and Social Welfare
● The health economic evaluation involves collecting
data on the time used by participants for learning and
for developing and implementing interventions to
improve equity in access and utilization of first line
care
● Data is also collected on the provider payment systems
which are in place in the respective county councils in
order to assess the financial incentives and support for
equity oriented initiatives
7
Health economic
evaluation

8. School of Health, Care
and Social Welfare
School of Health, Care
and Social Welfare
● Cultural and historical
descriptions
● Socio-economic
background:
● Sickness
● Financial help
● Income distribution
● Lifestyles and health:
● Smoking
● Alcohol consumption
● Exercise habits
● Organizational activities
● Mental illness and
insomnia
● Self-perceived health
● Overweight
● Care consumption
● Drug consumption
8
Area descriptions
of care units
● Descriptions of
participant care units:
● Economy
● Staff
● Quality indicators
● Listed patients

9. School of Health, Care
and Social Welfare
School of Health, Care
and Social Welfare
● Based on the process and the results so far:
● Program theory is a fruitful perspective for improvement
and development of the learning project.
● Program theory gives structure to different types of
evaluation in relation to each other.
● Opportunity to revise the model during the project is also
another important possibility about ability to respond to
the emerging problems and/or needs on the way.
9
Usefulness of
program theory

10. School of Health, Care
and Social Welfare
School of Health, Care
and Social Welfare
For more information:
●Osman Aytar: osman.aytar@mdh.se
●Sharareh Akhavan: sharareh.akhavan@mdh.se
Thanks for your attention!
10