ANATOMY AND PHYSIOLOGY OF REPRODUCTIVE SYSTEM.pptx
Core Outcome Measures in Effectiveness Trials
1. Core Outcome Measures in Effectiveness Trials
www.comet-initiative.org
Dr Aoife Waters
MRC North West Hub for Trials Methodology Research
University of Liverpool, UK
2. Outline
• Background
– What is the problem?
– Why do we need a more scientific approach to
outcomes?
• Core Outcome Sets (COS)
• The COMET Initiative
3. Why are outcomes important?
• Interventions are compared in RCTs by measuring
differences in patient outcomes between the groups
• Selection of appropriate outcomes is therefore crucial in
order to assess the effectiveness of an intervention
• “Clinical trials are only as credible as their outcomes”
(Tugwell 1993)
4. What is the problem?
• Several tens of thousands of research studies are underway and 500+ are
published every week
• Identifying effectiveness of interventions can be challenging because studies in
the same clinical area or of comparable interventions describing findings in
different ways
• It’s also difficult for trials to affect policy and healthcare decision making unless
they are designed in ways that reveal meaningful answers for patients
• Measurement and reporting of outcomes needs to be tidied up if research is to
achieve its aim of helping practitioners and patients to improve health care and
health
5. Cosmetic Outcomes Systematic Review
Aspects of cosmesis assessed
Potter et al. Assessment of Cosmesis After Breast Reconstruction Surgery: a Systematic Review. Ann Surg Oncol (2011) 18:813–823
6. Problems with outcomes in six
important Cochrane reviews
5 most accessed in 2009*:
• Preventing childhood obesity
• Treating childhood obesity
• Promoting activity in children
• Preventing falls in the elderly
• Preventing Type 2 DM
Most cited in 2009*:
• Nicotine replacement therapy
*Tovey D. Impact of Cochrane Reviews [editorial]. The Cochrane Library 2010 (7 July)
7. “The studies … varied greatly in intervention design, outcome
measurements and methodological quality.” (Preventing childhood obesity)
“Appropriate short- and long-term outcomes need to be defined for
children and youth at various weight levels, rather than using conventional
or adult-oriented outcomes.” (Treating childhood obesity)
“The studies were heterogeneous in terms of study design, quality, target
population, theoretical underpinning, and outcome measures, making it
impossible to combine study findings using statistical methods..”
(Promoting activity in children)
“No study reported relevant data on diabetes and cardiovascular related
morbidity, mortality and quality of life..” (Preventing Type 2 DM)
“We sought data for rate of falls, number of people falling, and number of
people sustaining a fracture. However, few studies provided fracture data.”
(Preventing falls in the elderly)
“Definitions of abstinence varied considerably ... In five studies it was unclear
exactly how abstinence was defined.” (Nicotine
replacement therapy)
8. Outcome reporting bias
-the selection of a subset of the original recorded outcomes in a study,
selected on the basis of the results, for inclusion in publication
• ORB suspected in at least one trial in 34% of 283 reviews (Kirkham et al,
BMJ 2010)
•42 significant meta-analyses
– 8 (19%) would not have remained significant
– 11 (26%) would have overestimated the treatment effect by > 20%
• Outcomes that are statistically significant are more likely to be fully
reported
OR 2.2 to 4.7 (Dwan et al, PLoS ONE 2008)
9. Core outcome sets
– an agreed standardised set of outcomes that should be
measured and reported, as a minimum, in all clinical
trials in specific areas of health or health care
•Disease/condition specific
•All treatment types or a particular intervention
•Should consider both benefits and harms
•The minimum – expect others to be collected
•Focus of effectiveness trials
•Involve stakeholders in their development
•Relevant within routine clinical practice
10. Advantages of COS
•Increases consistency across trials
• Maximise potential for trials to contribute to meta-
analyses of these key outcomes
•Reduction in selective reporting
• Much more likely to measure appropriate outcomes due
to early involvement of stakeholders
•More likely to reach a conclusion about an interventions
effectiveness more quickly
11. The COMET (Core Outcome Measures
in Effectiveness Trials) Initiative
• Brings together people interested in the development of COS
• Facilitate and promote development and application of COS
• Liverpool, 2010; Bristol, 2011
- Trialists, systematic reviewers, health service users, clinical
teams, journal editors, trial funders, policy makers, regulators
12. The COMET (Core Outcome Measures
in Effectiveness Trials) Initiative
• ‘What’ to measure
– Guidance on methods for developing core outcome sets,
including patient involvement and funding applications
– Providing a reporting standard for COS
• ‘How’ to measure (validity, reliability, feasibility)
– PROMIS
– COSMIN
– TREAT-NMD
13. COMET website and database
• COMET website makes it easier to find existing research,
helping
• funders who may wish to fund work in this area avoid
duplication
• researchers who want to know what developments
are underway
www.comet-
initiative.org/studies/search
14. COMET website and database
• Work is ongoing to identify, collate and maintain
relevant resources in an online searchable database
• 130 completed projects in various areas of health
• COS development is planned or ongoing in 25 clinical
areas, with a further 32 in discussion
www.comet-
initiative.org/studies/search
16. Impact
• In the months since the launch of the COMET website and
database (August 2011)
– 1335 searches have been undertaken
– 3874 individuals visited (6679 visits, 27791 page views)
– 91 countries visiting the site
17. Development of core outcome sets:
issues to consider (submitted)
• Scope
• Identifying existing knowledge
• Stakeholder involvement
• Consensus methods
• Achieving global consensus
• Implementation
• Regular review, feedback, updating
18. What next?
• SR and survey
– Identify COS (and other relevant work)
– Describe methods used for COS development
– Survey COS developers
• Cochrane CRG survey
• Funding applications
• COMET III
• PPI meeting
19. Stakeholder buy-in: NIHR HTA
‘Where established Core Outcomes exist they should be
included amongst the list of outcomes unless there is
good reason to do otherwise. Please see The COMET
Initiative website at www.comet-initiative.org to
identify whether Core Outcomes have been
established.'
20. Professor Hywel Williams, Chair of the NIHR HTA
Commissioning Board: ‘Patients and professionals making
decisions about health care need access to reliable
evidence. The new COMET database will help researchers
across the NIHR family and beyond when choosing the
outcomes to include in the studies that will establish this
evidence base'.
21. In conclusion
• Currently little consistency
– Unacceptable waste of data
– Need to make things better
• It is vital to collect important outcomes in all trials
– Especially outcomes important to patients
– There should be a more scientific approach to outcomes
• Growing activity in development of core outcomes and
support for COMET
• Improving the quality of evidence to support informed
healthcare decision and inform policy
22. www.comet-initiative.org
Elizabeth Gargon
e.gargon@liv.ac.uk
Twitter: @COMETinitiative
Acknowledgments
COMET Management Group: Doug Altman, Jane Blazeby, Elizabeth
Gargon, Mike Clarke, Paula Williamson
Funding: MRC HTMR Network, now MRC
Collaborators: Peter Tugwell, Maarten Boers, Caroline Terwee, Holger
Schunemann, Michael Rose, Sunita Vohra, Roberto D’Amico, Lorenzo
Moja
Notes de l'éditeur
However, not only is there variation in which outcomes are measured, there is also a big variation in how these are measured and this was well demonstrated by a SR conducted by Shelley Potter in Bristol in which she looked at studies reporting on cosmetic outcome in women who have had reconstructive surgery following mastectomy for breast cancer. She identified 194 studies, and found that cosmetic outcome was measured in 16 different ways and that the most frequently measured outcome wasn’t even reported in half of the total studies. There is a clear lack of consistency but one way of addressing this is through the development of COS.
In six very important Cochrane reviews, the authors clearly describe problems with outcome, selection, measurement and reporting. problems with outcomes were explicitly identified in these reviews.
For those of you who may not be entirely familiar with the concept of outcome reporting bias, it has been defined as the selection of a subset of the original recorded outcomes in a study, selected on the basis of the results, for inclusion in publication. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported.
COS can apply to a specific disease/condition Not ‘one size fits all’
Remit is ‘what’ not HOW Ultimately we want to work towards providing a reporting standard for COS development e.g. If use delphi- how many stakeholders, rounds etc. Links to these on web-site Established a more formal link with these groups in work going forward, work packages
Mention how many of the 130 are consensus COS – 42 consensus 14/25 ongoing are consensus Currently resource rather than a guideline Issues with backing ‘COS’ needs updated
Projects looking at core outcome sets and their development Linked research indicated i.e. SR-DELPHI for same project
Scope of the core outcome set- in my example of head and neck cancer we wondered about disease sub-site, stage, treatment modality, stratifying by HPV status. Identifying existing knowledge- systematic review Consensus methods- many methods employed by COS developers- Delphi, nominal group technique Should be developed scientifically guidance Uptake should be monitored and feedback sought to assess impact (OMERACT evaluation ongoing) to inform possible future update
Identify COS work will enable us to describe methods Together should help to inform guidance around developing COS negotiation stage with EU FP7 grant (but don't say 'awarded')
FUNDING APPLICATION FORM FOR RCTs
The National Institute of Health Research, the largest funder of healthcare research in the UK, highlighted the importance of core outcome sets, the work of COMET and the COMET database on its home page in late 2011.
Little consistency in outcome measurement and reporting