This sample answer sheet corresponds with the first webinar in the Online Journal Club series, “Can the use of text message reminders improve vaccination rates?” To access the presentation slides in English click here: http://www.slideshare.net/NCCMT/online-journal-club The National Collaborating Centre for Methods and Tools is funded by the Public Health Agency of Canada and affiliated with McMaster University. The views expressed herein do not necessarily represent the views of the Public Health Agency of Canada. NCCMT is one of six National Collaborating Centres (NCCs) for Public Health. The Centres promote and improve the use of scientific research and other knowledge to strengthen public health practices and policies in Canada.

1. Critical Review of Intervention Studies Worksheet CAIS
NCCMT On-Line Journal Club Oct 13, 2016 / Nov 17, 2016
Article: Stockwell, M. S., Westhoff, C., Kharbanda, E. O., Vargas, C. Y., Camargo, S., Vawdrey, D. K., & Castaño, P. M.
(2014). Influenza Vaccine Text Message Reminders for Urban, Low-Income Pregnant Women: A Randomized
Controlled Trial. American Journal of Public Health, 104(Suppl 1), e7–e12. http://doi.org/10.2105/AJPH.2013.301620
SAMPLE Comments
I. Are the Results Valid?
1. Did the trial address a
clearly focused issue?
Yes
P – low income, urban obstetric patients
I – 5 weekly text messages re vaccines + 2 appointment reminder text messages, in
addition to phone calls as in control
C – usual phone reminders re appointments
O – influenza vaccination
2. Was the assignment of
patients to treatments
randomized?
Yes- 1:1 allocation , randomization through SPSS, concealed to researcher
3. Were patients, health
workers and study
personnel ‘blinded’?
No - researchers blinded. Not possible to blind patients or health care personnel.
4. Were the groups similar at
the start of the trial?
See Table 1
Yes – except for gestational age, which was accounted for in the adjusted analysis.
5. Aside from the
experimental intervention
were the groups treated
equally?
Can’t tell. No note of other treatment, but cannot always assume no other difference
in treatment.
6. Were all of the patients who
entered the trial properly
accounted for?
Yes – Figure 1. Accounts for all dropouts.
However, no specific mention of Intention to Treat analysis
II. What are the results?
7. How large was the
treatment effect?
See Table 2 – main results – absolute difference and relative risk, and adjusted
Odds ratios.
No statistically significant differences in Relative Risk (CIs all cross 1); no
statistically significant difference in Absolute Risk (CIs all cross 0), Some small
differences in adjusted Odds Ratios.
Some interesting findings in sub analyses (table 3) – that women at 28-33 weeks
GA at beginning of trial more likely to be vaccinated.

2. Critical Review of Intervention Studies Worksheet CAIS
NCCMT On-Line Journal Club Oct 13, 2016 / Nov 17, 2016
Article: Stockwell, M. S., Westhoff, C., Kharbanda, E. O., Vargas, C. Y., Camargo, S., Vawdrey, D. K., & Castaño, P. M.
(2014). Influenza Vaccine Text Message Reminders for Urban, Low-Income Pregnant Women: A Randomized
Controlled Trial. American Journal of Public Health, 104(Suppl 1), e7–e12. http://doi.org/10.2105/AJPH.2013.301620
Based on Critical Skills Appraisal Programme. (2013). 11 questions to help you make sense of a
trial. Retrieved from http://media.wix.com/ugd/dded87_40b9ff0bf53840478331915a8ed8b2fb.pdf
8. How precise was the
treatment effect?
For statistically significant Odds Ratios – eg AORb
By Dec 31, AOR (95% CI) is 1.30 (1.003, 1.69). Not very precise (wide confidence
interval).
Also, at lower end of CI (1.003) effect is very small
III. Will the results help
locally?
9. Can the results be applied
in your context?
Maybe – if technology already exists or does not cost too much to implement.
10. Were all clinically important
outcomes considered?
Were participants previously vaccinated?
11. Are the benefits worth the
harms and costs?
Would look to see if there is a review, other trials, or wait until more evidence.