Methodologies and approaches to evaluation in third sector organisations / Jean Ellis
•Télécharger en tant que PPT, PDF•
0 j'aime•11,021 vues
![The policy waves ,[object Object],[object Object],[object Object],[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (20)
Similaire à Methodologies and approaches to evaluation in third sector organisations / Jean Ellis
Similaire à Methodologies and approaches to evaluation in third sector organisations / Jean Ellis (20)
Dernier
Dernier (20)
Methodologies and approaches to evaluation in third sector organisations / Jean Ellis
- 1. Methodologies and approaches to evaluation in the third sector International conference on evaluation, Barcelona 3 November 2011 Dr Jean Ellis Charities Evaluation Services
- 3. Some definitions Monitoring The routine, systematic collection of information for the purpose of checking progress against project plans Evaluation Using monitoring and other information to make judgements and give value
- 8. Steps to self-evaluation 1- Clarify aims and outcomes, objectives and outputs 2- Set performance indicators; choose tools 5-Use findings 3- Collect information 4- Analyse and interpret FIVE STEP PLAN
- 9. CES planning triangle © Overall aim Objectives This is what we do Specific aims This is why we do it Outputs Outcomes
- 10. The Family Centre To provide opportunities for play, drama and dancing To provide workshops, information and advice on parenting To organise outings for children and their parents To improve the lives of the children who use the centre To increase children’s self-esteem To enable parents to give each other practical support Objectives Specific aims Overall aim To run a drop-in centre with a play area To improve the parenting knowledge and skills of the parents who use the centre
- 12. More definitions Outputs are an organisation’s detailed activities, services and products – what you do Outcomes are all the changes, benefits, learning or other effects that happen as a result of an organisation’s activities – what you change
- 14. Intermediate outcomes There are often smaller changes that need to happen before the final, desired outcome can be reached. These steps along the way are known as intermediate outcomes. Families engage with befriending service Greater awareness within the family of later life conditions Families feel supported, less isolated Increased communication and coping skills Improved family relationships and dynamics
- 15. The Outcomes Star © Triangle Consulting 2011
- 16. Underpinned by a model of change © Triangle Consulting 2011
- 24. Challenges and opportunities Managing data Getting skilled Finding meaning Methods Innovation Using findings New technology
Notes de l'éditeur
- Good for users: An intervention in itself, Keep staff focused on outcomes, Opportunity to shape services Respond to external pressures: Charity commission (Hallmarks of an effective charity’, SORP, Summary Information Return) , Funding lever, General drive to accountability, Payment by results Good for staff: Feedback on performance, Involvement in assessment and shaping services, Can challenge/ confirm views, Role clarity, Save time in long run Make your organisation better: Better information systems, Effective use of resources, Demonstrates results, Transparency , Responsive to needs Tension between accountability and learning. Jean’s research 2008 found that much monitoring and self-evaluation done for accountability. Accountability paradox, from accountability and learning: ‘ The collection of data for compliance purposes alone, even with a focus on benefits for users , may not in itself provide a means of improving value.’ ‘ Paradoxically, in the context of an overall effectiveness agenda, the external reporting focus limits the use of findings for improved management and wider learning – to increase effectiveness.’
- Some of you may have seen before – this is a planning tool – won’t have time to do today
- 2.50 Mention hard and soft outcomes Outcome areas Example outcomes Organisational strength More secure funding; Improved management structures Service provision Greater lobbying voice; New organisations established to deliver services Policy Change in funding policy; Change in national policy Sharing good practice More joint ventures between member organisations
- The Age UK befriending scheme
- Impact arts wanted to set up this new type of project. Fab pad offers arts, design and practical skills training to vulnerable young people, when they get a new tenancy, to help them turn their house into a home. Their funders considered it risky, and didn’t quite understand the role of arts in social change Impact wanted to demonstrate the different made in terms their funders would understand As a result of their SROI, Fab Pad have gained follow-on funding support for the programme, developed a higher profile with their main partners and funders, and have begun applying SROI to other aspects of their business as a strategic management tool. Are planning to do SROIs on their different Fab Pad projects and compare them Remember the ratio doesn’t represent an actual social return; it is a sum of how much all the stakeholders value the service outcomes What we need is a standard approach to measuring and combining proxies that have been calculated differently, rather than measure everything in same way
- Reporting and dissemination should be part of the plan from the outset – to increased engagement and utilisation Know your audience: Before you start your evaluation, answer the following questions: Who are your primary evaluation users? What do they need to know? What do they not need to know? When do they need to know it? Why do they need to know it? What methods will they think are credible? What does this mean for your evaluation? Range of methods to capture different audiences Analysis continuum: Raw data: Familiarise, edit, filter Description: Develop a framework, code and describe Interpretation: What do the findings mean? Recommendations: What are the practical consequences? Findings should answer Evaluation Qs – use these to keep you focused. Recommendations must link to findings – common mistake
- Here are some challenges we can discuss at the end One that might be worrying you is funding ie, cost of external vs self-evaluation How about also better understanding needed or the role of evaluation in organisational learning, not just accountability Also an issue of reconciling and choosing between different tools and approaches