Ce diaporama a bien été signalé.
Le téléchargement de votre SlideShare est en cours. ×

Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities

Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Prochain SlideShare
Outcome mapping DPH Day 2012
Outcome mapping DPH Day 2012
Chargement dans…3
×

Consultez-les par la suite

1 sur 38 Publicité

Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities

Télécharger pour lire hors ligne

The benefits and challenges of using Outcome Harvesting to evaluate a short-term intervention are explored using the example of an 18 month social change project supported by the UK Department for International Development in Tanzania. The project was that was highly ambitious: it sought to influence changes in gender attitudes and behaviour of the general public in Tanzania. Challenges included the lack of outcome indications in project document and the lack of knowledge of outcomes among project personnel. Outcome Harvesting was adapted to allow the harvesting of outcomes using focus groups of those the project sought to influence directly. The concept of ‘proto-outcome’ was used for suggestions of attitude changes that may lead ultimately to behaviour changes. Substantiation of outcomes involved not only third parties but direct observation. The resulting descriptions of outcomes and the evaluation findings proved valuable for learning in the organisation, Search for Common Ground.

The benefits and challenges of using Outcome Harvesting to evaluate a short-term intervention are explored using the example of an 18 month social change project supported by the UK Department for International Development in Tanzania. The project was that was highly ambitious: it sought to influence changes in gender attitudes and behaviour of the general public in Tanzania. Challenges included the lack of outcome indications in project document and the lack of knowledge of outcomes among project personnel. Outcome Harvesting was adapted to allow the harvesting of outcomes using focus groups of those the project sought to influence directly. The concept of ‘proto-outcome’ was used for suggestions of attitude changes that may lead ultimately to behaviour changes. Substantiation of outcomes involved not only third parties but direct observation. The resulting descriptions of outcomes and the evaluation findings proved valuable for learning in the organisation, Search for Common Ground.

Publicité
Publicité

Plus De Contenu Connexe

Diaporamas pour vous (20)

Similaire à Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities (20)

Publicité

Plus par John Mauremootoo (20)

Plus récents (20)

Publicité

Searching for outcomes in rural Tanzania: Harvesting directly from those influenced: Implications for evaluating behavioural change initiatives with communities

  1. 1. Searching for outcomes in rural Tanzania: Harvesting directly from those influenced Implications for evaluating behavioural change initiatives with communities John Mauremootoo, Richard Smith, Dunstan Kishekya Presentation given at the American Evaluation Association on 12th November 2015 in the session entitled: The strengths and challenges of Outcome Harvesting for Evaluating in Complex Situations. Experiences from around the World.
  2. 2. The Project A vehicle for tackling polarizing social issues
  3. 3. The Project The Team Tanzania (The Team) one of 15 completed or ongoing projects implemented in Africa and Asia by Search for Common Ground (SFCG) using The Team concept – The Team is a vehicle for tackling polarizing societal issues by stimulating learning in a persuasive, but non- confrontational manner. For Tanzania the issue chosen was gender equality Using an ‘edutainment’ approach (TV and radio soap operas) the project planned to reach and influence a wide audience.
  4. 4. Goal To contribute to strengthening the implementation and enforcement of gender-sensitive legislation in Tanzania by influencing changes in gender attitudes and behaviour of the general public
  5. 5. Priority gender equality themes targeted in the 18 month project
  6. 6. 1. Inheritance & women’s consideration in inheritance issues
  7. 7. 2. Women’s leadership
  8. 8. 3. Gender-based violence
  9. 9. 4. Retention of girls in secondary school
  10. 10. 5. Rape
  11. 11. The Project Theory of Change Reach Response Resonance Numbers who watch and listen to the show The attitude and knowledge changes at the individual and institutional levels The actions that have been triggered by the programmes
  12. 12. Evaluation Method Identification, description and interpretation of outcomes through Outcome Harvesting (using an OM-inspired definition of Outcome as changes in actions, relationships, policies or practices of one or more social actors influenced by the intervention). With especial emphasis on harvesting outcomes of social actors the project had been seeking to influence by mobile cinema screenings and follow up discussions.
  13. 13. Evaluation challenges The “Blank Slate of: • Poor understanding of outcome concept among project implementation team • Little or no knowledge of outcomes among the project implementation team • No documentation of outcomes in reports Short duration of assignment given the fact that all outcome data had to be collected from scratch
  14. 14. Data collection via focus group discussion
  15. 15. Focus Group Discussion Guiding questions explained issues such as: • The definition of outcomes • The definition of contribution • How to determine how importance of The Team’s contribution • Sources of substantiation • Similar outcomes • Negative outcomes Data collection via focus group discussion
  16. 16. “Treatment” “Control” Harvesting outcomes through focus group discussions with social actors the project had been seeking to influence by mobile cinema screenings and follow up discussions Harvesting outcomes from ‘control groups’ – target group representatives who had not participated in the mobile screenings and discussions
  17. 17. Follow up Follow-up work to finalise and substantiate results descriptions
  18. 18. Results: The Team Contribution 54 outcomes and 10 “proto-outcomes” 0 10 20 30 40 50 Useful (25) Very important (25) Important (14) percentage Strengthened and / or brought forward changes that may have happened to some extent anyway Either essential to the change or greatly accelerated it Strengthened and / or brought forward changes that may have happened to some extent anyway Helped realise a change that may not otherwise have happened or would have happened very much more slowly
  19. 19. Results: Outcomes directly related to priority issues
  20. 20. Inheritance & women’s consideration in inheritance issues Results relating to shared ownership of goods and property
  21. 21. Women’s leadership Women taking more responsibility in work, home and in political life
  22. 22. Gender-based violence Men stopping beating their wives and others encouraging men to stop beating their wives
  23. 23. Retention of girls in secondary school Parents prioritising girls education and others encouraging parents to prioritise girls education
  24. 24. Rape No results provided
  25. 25. Results: Other Outcomes Financial implications of gender equality Participatory decision making
  26. 26. Implications for evaluating behavioural change initiatives with communities
  27. 27. 1. Outcome Mapping Concepts OM concepts of outcomes as behavioural change and contribution and the differences between knowledge, attitude and behaviour were easy and intuitive to grasp for the participating communities.
  28. 28. 2. Knowledge & Attitude Changes Claimed attitude changes even without behavioural change are important and can be readily assessed using OH These “proto-outcomes” helped: • To understand the claimed process of change even where there have been no tangible changes in behaviour • To compare the quality and process of change between “treatment” and “control” • Inform the change agent
  29. 29. 3. Utility of Control Groups In situations where there is a well-defined theory of change the control group provides a counter-factual
  30. 30. 4. Focus Group Format The focus group format is effective for discussing less sensitive topics but may not be the best way of evaluating results concerning sensitive issues and totally inappropriate for the most sensitive such as rape
  31. 31. 5. The value of Follow up The importance of an iterative follow-up process to enrich the result description and enhance its credibility.
  32. 32. 6. The Timing of the Evaluation a) It was surprising there were outcomes from a very ambitious project when its 18 months had not yet finished, b) It was a snap shot & provided credible evidence of change c) Follow up was needed, e.g. after say 2 years d) It ideally should really have been a formative evaluation.
  33. 33. Now What? If we consider the evaluation as an intervention then both its findings and the process need to be built on to help maximise positive change. This was not formally done for a variety of reasons which in our opinion was a lost opportunity as is often the case for evaluations.
  34. 34. • Ansila Marandu (evaluation assistant) • Paul Glick (Search for Common Ground) • Cornelia Wamba (Search for Common Ground) • Stella Msami (Search for Common Ground) • VanessaCorlazzoli (Search for Common Ground) • The Mvomero Organizations Coalition • Women and Girls Fight illiteracy and Poverty Organisation Save the Children Tanzania • Kate Dyer (AcT and KPMG) • Layla Ghaid (AcT and KPMG) Acknowledgements
  35. 35. Images Screenshots from The Team Tanzania on YouTube
  36. 36. Images Stock Images from Shutterstock © All rights reserved
  37. 37. Images Public domain
  38. 38. All Other Images John Mauremootoo (Creative Commons Attribution: CC-BY)

Notes de l'éditeur

  • 4 results

    Example
    Outcome Description
    Following The Team Mairo Keria Mugesa has involved his wife in decision-making and joint ownership of their business. Their dairy cow is now under joint ownership, they bought 200 chicks together in July 2013, constructed a well jointly from July 2013 and have managed to build an improved home.

    Contribution Description
    After The Team (specifically the incidences when the female teacher being ridiculed by male teachers and even male students) and the FGD in which women's right to own property was discussed, Mairo could see the potential socio-­economic benefits of involving his wife in decision-­‐making and ownership.
  • 7 results

    Example
    Outcome Description
    From June 2013 Tatu Mbonde the leader of the Mvomero widow's group has initiated a participatory planning process for the group to help them to earn income from agriculture and save to help members in trouble.

    Contribution Description
    Before watching The Team and being involved in the focus group discussions Tatu and her group had no idea of planning together at all. From watching The Team and the discussions that followed, Tatu learned that teamwork among community members brings about efficiency.
  • 6 results

    Example
    Outcome Description
    Following The Team and up to October 2013 Marwa Weibina had stopped beating his wife.

    Contribution Description
    After The Team, Marwa Mirumbe and his friends and advised Marwa Weibina against beating his wife. This followed the Team episodes and the focus group discussions in which the negative effects of GBV were highlighted. Alternative gentle approaches to reconciliation such as consultations with elders were discussed.
  • 4 results

    Example
    Outcome Description
    In April 2013, Amina Daudi's parents allowed her 14 year old younger sister to attend year one of secondary school.

    Contribution Description
    Watching the aspect of The Team TV programme of women trying to pull girls from school for marriage and the follow-up discussions on girls’ identity helped Amina to realise that girls have unique opportunities for schooling. This motivated her to fight for her sister's right to attend school.
  • Other Outcomes (in line with objectives but not clearly related to pre-defined priority themes)
    20 results related to contributions to gender equality with financial implications
    6 related to participatory approaches to decision making and planning

    Financial related
    Example
    Outcome Description
    Following The Team Dhamira Athumani who was jobless young lady began to grow vegetables which she is now selling. She has also joined a theatre and performance art group where she is learning drama.

    Contribution Description
    Dhamira Athumani was one of the women that Twalah Mbwangali (the 47 year old male Chairperson of Miembe Miwili sub--‐village) counselled to use different strategies to fight poverty. This action was the result of watching The Team and seeing how Miss Wito used different strategies to achieve her goal.

    Participatory approaches
    Example
    Outcome Description
    Following The Team Zainabu Said Zuberi, a 33 year old female farmer changed her approach to dealing with conflict in the family and with neighbours to a participatory approach that involves being humble.

    Contribution Description
    By adopting this approach she was able to help resolve a misunderstanding between her neighbours. Through The Team Zainabu saw Miss Wito’s consultative approach to leadership.
  • 5. Value of focus group discussions for capturing the essence of a result and the importance of an iterative follow-up process to enrich the result description and enhance its credibility.

    To be robust, it is arguable that each new piece of information needs triangulating. A tedious process if followed literally and probably only possible for a limited number of outcomes. But if some uncertainty that the final outcome / contribution descriptions are fully accepted by all informants, then it is an enriching and probably practical approach.
  • Evaluation use:
    If we believe the Simultaneity Principle (from Appreciative Inquiry) that inquiry is an intervention then an evaluation such as this has considerable potential as a means for accelerating the change process through evaluation use – both of its findings and the process.
    The use of findings is obvious but there is also the use of the process, e.g. the focus group discussions informed others in the group about outcomes and this may have affected their attitudes and subsequent behaviour. The partner organisations became more interested. So we are missing a trick if we do not go on to know, show and grow the bright spots. The evaluator can contribute to this take off process by having a thorough follow-up session with the change agent and partners… sadly this often does not happen for all sorts of reasons.

×