To train the health professionals on health system research proposal development, To acquaint the participants with health research process, and To train basic managerial skills required to manage proposed health research.

1. Supervision, Monitoring and Evaluation
Ashok Pandey
NHRC
6/6/19 1

2. Background
Research requires:
• Maintaining standards in the procedures
• Acquiring quality data
• Timely completion of data acquisition, analysis and dissemination
“Supervision, Monitoring and Evaluation plan” is required during proposal
development to research implementation.
Supervision and monitoring is a continuous process
6/6/19 2

3. MONITORING is the on-going process by which information is gathered
concerning the implementation and evolution of the research project.
6/6/19 3

4. During monitoring sessions you will review:
The resources needed for the project, including staff, equipment, supplies, logistical
support and funds, to assess if they are available when needed and being
appropriately used;
• The activities of each team member and their relations to the project as a whole, to
assess if the work plan is being carried out as planned and what delays or
difficulties, if any, have emerged that need to be addressed;
• The flow and quality of the data that are being collected; and to what extent they
meet the objectives or answer the research questions; and
• The research team’s communication and co-ordination with the study population,
other collaborating groups, and funding authorities.
6/6/19 4

5. Contd ….
Monitoring will usually take place at team meetings during field activities. If
there is a gap in the fieldwork, it may be necessary to convene a special
meeting.
It is advisable to keep close track of changes in the work plan and problems
encountered and solved (or not solved) so that you can inform your
facilitator and superiors, and include this information in your preliminary
report.
6/6/19 5

6. Monitoring in health research
Monitoring: A process of measuring, recording, collecting, and analyzing
progress of research work and communicating it to the investigators (PI, Co-
PI) so that any deviation from the planned operations are detected, causes of
the deviations are diagnosed and suitable corrective actions are taken.
Generally it is carried out during data collection phase. However, it is
required at every phase of research
6/6/19 6

7. Monitoring can be of..
• Timeline of the planned activities
• Resources used
• Quality
6/6/19 7

8. Research monitoring tools
Customized monitoring checklists or standard checklist
– Based on the determined frequency of reporting
Gantt Chart: A popular work plan tool is used for monitoring as well. Gantt chart
has the plan for research activities divided for different time intervals. So, the chart
can be used for monitoring whether the assigned tasks are completed in their
specified time intervals.
The proposal itself can also serve as monitoring tool for health research.
6/6/19 8

9. Mechanisms for monitoring
• Results from researcher
• Reports from independent agency (DSMB)
• Review of Adverse Events reports
• Random inspection of research site, data and consent documentation
• Interview of research participants
6/6/19 9

10. Data and Safety Monitoring Boards (DSMBs)
DSMB are required for the appropriate oversight and monitoring of the conduct of
clinical trials to ensure the safety of participants and the validity and integrity of the
data.
• Data and safety monitoring is required for all types of clinical trials, including
• Physiologic, Toxicity, And Dose-Finding Studies (Phase I);
• Efficacy Studies (Phase II);
• Efficacy, Effectiveness And Comparative Trials (Phase III); Etc.
Monitoring should be commensurate with risks -- The method and degree of
monitoring needed is related to the degree of risk involved.
6/6/19 10

11. Example: Plan for supervision and monitoring
Title: Randomised controlled trial of topical chlorhexidine 0.2% versus topical natamycin 5% for
fungal keratitis in Nepal
The study will be monitored by a DSMB.
The DSMB charter is also provided as an additional document.
Aims: To safeguard the interests of trial participants, monitor the main outcome measures including
safety and efficacy, and monitor the overall conduct of the trial.
Terms of reference: The DSMB should receive and review information on the progress and accruing
data of this trial and provide advice on the conduct of the trial to the Principle Investigator (PI)
Specific roles of the DSMB: Interim review of the trial’s progress including updated figures on
recruitment, data quality, adherence to protocol treatment and follow-up, and main outcomes and
safety data.
6/6/19 11

12. Contd..
Specifically, these roles include to:
• monitor evidence for any differences in the outcome measures by treatment arm
• monitor evidence for treatment harm (e.g. SAEs and SARs, deaths)
• assess the impact and relevance of external evidence (eg results from other trials or studies
involving CHX or NATA)
• decide whether to recommend that the trial continues to recruit participants or whether
recruitment should be terminated either for everyone or for some treatment groups and/or
some participant subgroups
• decide whether trial follow-up should be stopped earlier
• assess data quality, including completeness (and by so doing encourage collection of high
quality data)
• maintain confidentiality of all trial information that is not in the public domain 6/6/19 12

13. Contd..
Specifically, these roles include to:
• monitor recruitment figures and losses to follow-up
• monitor compliance with the protocol, including adherence to drugs, by
participants and investigators
• suggest additional data analyses if necessary
• advise on protocol modifications proposed by investigators or sponsors (e.g. to
inclusion criteria, trial endpoints, or sample size)
• monitor compliance with previous DSMB recommendations
The members of the DSMB will include: 1. Experts/specialist 2. Subject experts 3.
Biostatistician 4. NHRC member - NHRC to kindly advise a suitable candidate
6/6/19 13

14. Title: Pattern of Psychiatric Morbidity among
Adults in Butwal Metropolitan City, Rupandehi.
Orientation will be done thoroughly to the enumerators and the research
team by the principal investigator.
Data will be rechecked by principle investigator and other research team
members as peer review to minimize error and maintain data quality.
After collection of data, checking, compiling will be done manually on the
same day of data collection.
6/6/19 14

15. 6/6/19 15

16. Supervision in context of health research
Supervision is a supportive assistance and guidance given to the research team (such as
enumerators) by senior research team for improving performance.
Supervisory styles:
– Observation of enumerator’s work
– Observation of and interaction with enumerator
– Interaction only etc.
Determining
– Who will supervise and whom to supervise?
– Supervisor enumerator ratio
6/6/19 16

17. Things to focus while supervising
• Selection of the appropriate respondent,
• Compliance with the interviewing rules and guidelines set forth in the
training,
• Performance of non-interview tasks e.g. Observation, specimen collection
etc.,
• Identification of interviewer-questionnaire interface problems.
6/6/19 17

18. Sample supervision checklist for a enumerator
SN Criteria Yes No Feedback given Remarks
1Study subject enrolled as per sampling plan
2Informed consent taken
3Objective explained to participants
4Unnecessary probing
5Too much leading questions
6Good interviewing skill
Overall Impression Name and signature of supervisor
6/6/19 18

19. Other issues to be planned
• Re-contacting 10% of the respondents who completed the interview
• Plan in case of any falsified interview e.g. verifying all of that
interviewer’s work or retaking the interview?
• Clinical verification of certain percent of interview result by medical
doctors
6/6/19 19

20. Writing monitoring and supervision plan in a proposal
E.g. Monitoring of field staff will be carried out by central level research
team by the use of monitoring checklist.
The checklist will comprise of defined indicators, reporting frequency and
target.
Similarly, either PI or Co-PI will visit the field twice a month during data
collection period and supervise the field staff.
6/6/19 20

21. • Growing demand for transparency and accountability
Researchers developed a comprehensive list of evaluation
tools and techniques and
Explained when each might be most useful and why.
6/6/19 21

22. Evaluation
6/6/19 22

23. Thank you !
6/6/19 23