The document summarizes a webinar presented by the Patient-Centered Outcomes Research Institute's (PCORI) Methodology Committee on their report setting standards for patient-centeredness and research prioritization. The webinar agenda included introductions, a focus on patient engagement, an overview of the Methodology Committee's mission and report, and a discussion of patient-centeredness and research prioritization. The report proposed standards in key domains to guide high-quality, patient-centered outcomes research.
Patient-Centered Outcomes Research Institute Sets Standards
1. Patient-Centered Outcomes Research Institute
PCORI Methodology Committee
Report
Setting Standards for Patient-Centeredness and
Research Prioritization
August 14, 2012
2. Presenters
Ethan Basch, MD, MSc David Meltzer, MD, PhD
Member, Methodology Committee Member, Methodology Committee
Associate Attending Physician & Chief of the Section of Hospital
Outcomes Scientist Medicine
Memorial Sloan-Kettering Cancer Center The University of Chicago
Gail Hunt Lori Frank, PhD
Member, PCORI Board of Governors Director
President and CEO of the National Engagement Research
Alliance for Caregiving PCORI
2
3. Webinar Agenda
1. Introduction to PCORI 1:00pm – 1:05pm ET
2. A Unique Focus on Patient Engagement 1:05pm – 1:10pm ET
3. Methodology Committee Mission & Report 1:10pm – 1:20pm ET
4. Patient-Centeredness and Research 1:20pm – 1:30pm ET
Prioritization
5. Questions and Answers 1:30pm – 2:00pm ET
Please submit questions for the Q&A portion of today’s webinar to
methodswebinar@pcori.org
Formal public comments can be submitted at
pcori.org/survey/methodology-report/
3
4. Poll Questions 1 - 4
1. Are you familiar with the contents of the Methodology Committee Report?
(Y/N)
2. For this webinar, which stakeholder community best describes your
interest in engaging PCORI?
3. Rate your understanding of the process the Methodology Committee used
to generate standards:
a) I do not understand the process the Methodology Committee used to generate
standards
b) I understand the process somewhat
c) I have good understanding of the process the Methodology Committee used to
generate standards
4. Which response most closely matches your opinion of the Standards in
the draft Report?
a) The Standards largely cover the main areas important to patient-centered
outcomes research
b) Several important areas are not covered and additional Standards should be
considered
4 c) Don’t know/Not sure
5. Webinar Agenda
1. Introduction to PCORI 1:00pm – 1:05pm ET
2. A Unique Focus on Patient Engagement 1:05pm – 1:10pm ET
3. Methodology Committee Mission & Report 1:10pm – 1:15pm ET
4. Patient-Centeredness and Research 1:15pm – 1:30pm ET
Prioritization
5. Questions and Answers 1:30pm – 2:00pm ET
Please submit questions for the Q&A portion of today’s webinar to
methodswebinar@pcori.org
Formal public comments can be submitted at
pcori.org/survey/methodology-report/
5
6. About PCORI
• An independent, non-profit organization authorized by
Congress.
• Committed to continuously seeking input from patients
and a broad range of stakeholders to guide its work.
• Mission − To help people make informed health care
decisions and improve health care delivery and
outcomes by:
Producing and promoting high integrity, evidence-
based information that comes from research
guided by patients, caregivers and the
broader health care community.
6
7. Defining Patient-Centered Outcomes Research
(PCOR)
Helps people and their caregivers communicate and make informed health care
decisions, allowing their voices to be heard in assessing the value of health
care options. This research answers patient-centered questions such as:
Expectations Options Outcomes Decisions
“Given my “What are my “What can I do “How can
personal options and what to improve the clinicians and the
characteristics, are the potential outcomes that care delivery
conditions and benefits and are most systems help me
preferences, harms of those important to make the best
what should I options?” me?” decisions about
expect will my health and
happen to me?” healthcare?”
7
8. Commitment to Patient-Engagement
• Bringing patients’ and caregivers’ voices to research is one of
PCORI’s core values.
• The development and widespread adoption of standards for patient-
centeredness and engagement in research is critical for helping patients
and caregivers make more informed health decisions.
• The draft Report’s “patient-centeredness” standards were informed, in
part, by patients and caregiver interviews and focus groups throughout
the country.
• PCORI looks forward to continually tapping into the energy and wisdom of
the patient, caregiver and other stakeholder communities to create a new
model for research.
8
10. Patient-Centeredness and the draft Methodology Report
The standards include specific calls for patient involvement in all phases of
patient-centered outcomes research (PCOR) including:
– Formulating research questions
– Defining essential characteristics of study participants, comparators,
and outcomes
– Monitoring study conduct and progress
– Disseminating results
• The proposed standards also highlight the importance of patient
participation in the process of prioritizing which research proposals are
funded.
10
11. Webinar Agenda
1. Introduction to PCORI 1:00pm – 1:05pm ET
2. A Unique Focus on Patient Engagement 1:05pm – 1:10pm ET
3. Methodology Committee Mission & 1:10pm – 1:20pm ET
Report
4. Patient-Centeredness and Research 1:20pm – 1:30pm ET
Prioritization
5. Questions and Answers 1:30pm – 2:00pm ET
Please submit questions for the Q&A portion of today’s webinar to
methodswebinar@pcori.org
Formal public comments can be submitted at
11 pcori.org/survey/methodology-report/
12. PCORI Methodology Committee
MEMBER TITLE
Sherine Gabriel, MD, MSc Professor of Medicine and of Epidemiology , William J. and Charles H. Mayo Professor at Mayo Clinic
(Chair)
Sharon-Lise Normand, MSc, Professor of Health Care Policy (Biostatistics) in the Department of Health Care Policy at Harvard Medical
PhD (Vice Chair) School and Professor in the Department of Biostatistics at the Harvard School of Public Health
Naomi Aronson, PhD Executive Director of the Blue Cross and Blue Shield Association Technology Evaluation Center
Ethan Basch, MD, MSc Associate Attending Physician and Outcomes Scientist at Memorial Sloan-Kettering Cancer Center
Alfred Berg, MD, MPH Professor in the Department of Family Medicine at the University of Washington in Seattle
David Flum, MD, MPH Professor in the Department of Surgery and Adjunct Professor in Health Services and Pharmacy at the University
of Washington Schools of Medicine, Public Health and Pharmacy
Steven Goodman, MD, PhD Associate Dean for Clinical and Translational Research, School of Medicine , Stanford University
Mark Helfand, MD, MS, MPH Professor of Medicine and Professor of Medical Informatics and Clinical Epidemiology at the Oregon Health &
Science University
John Ioannidis, MD, DSc C.F. Rehnborg Chair in Disease Prevention, Professor of Medicine, Professor of Health Research and Policy, and
Director of the Stanford Prevention Research Center at Stanford University
Michael Lauer, MD Director of the Division of Cardiovascular Sciences at the National Heart, Lung, and Blood Institute
David Meltzer, MD, PhD Chief of the Section of Hospital Medicine, The University of Chicago
Brian Mittman, PhD Director, VA Center for Implementation Practice and Research Support, Department of Veterans Affairs Greater
Los Angeles VA Healthcare System
Robin Newhouse, PhD, RN Chair and Professor, Organizational Systems and Adult Health at University of Maryland School of Nursing
Sebastian Schneeweiss, MD, Associate Professor of Medicine and Epidemiology at Harvard Medical School and Vice Chief of the Division of
ScD Pharmacoepidemiology and Pharmacoeconomics at the Brigham and Women’s Hospital
Jean Slutsky, PA, MSPH Director of the Center for Outcomes and Evidence , Agency for Healthcare Research and Quality
Mary Tinetti, MD Gladdys Phillips Crofoot Professor of Medicine, Epidemiology, and Public Health in the Division of Geriatrics at Yale
University School of Medicine
Clyde Yancy, MD, MSc Chief, Cardiology, Northwestern University Feinberg School of Medicine
13. Methodology Report
• The mandate for PCORI’s Methodology Committee is to define
methodological standards and a translation table to guide
health care stakeholders towards the best methods for patient-
centered outcomes research (PCOR).
• Rigorous methods are essential to building trust in research
findings.
• The report is the necessary catalyst for scientifically rigorous,
patient-centered outcomes research that can inform decision-
making.
• Once the Report is revised and accepted by the PCORI Board of
Governors, future PCORI funding applicants will be expected to
reference the Standards in their applications and use the
Standards in their PCORI funded research.
13
14. Methodology Report – What is a Standard?
Building on the work of the Institute of Medicine*,
the Methodology Committee defined a standard as…
• A process, action, or procedure for performing PCOR that is deemed essential
to producing scientifically valid, transparent, and reproducible results; a
standard may be supported by scientific evidence, reasonable expectation
that the standard helps achieve the anticipated level of quality in PCOR, or by
broad acceptance of the practice in PCOR
• The recommendation is actionable, feasible, and implementable
• Proposed standards are intended for use by the PCORI Board, in PCORI
policies and procedures, and by PCORI researchers
*Reference: National Research Council. Find What Works in Health Care: Standards for Systematic Reviews.
14 Washington, DC: The National Academies Press; 2011.
15. Methodology Report – What Questions
Should our Standards Address?
The MC sought to address selected topics in 4 broad phases of
activities in the first Methodology Report:
What study How do we How do we
What should designs carry out and enable people
we study? should we govern the to apply the
use? study? study results?
15
16. Methodology Report Development
1 Methods • Working groups identified and prioritized major research methods
Selection questions to be addressed
2 • Researchers contracted to address selected topics
• Contractors developed research materials (e.g., reports, summary
Committee Expertise
Information templates for proposed standard)
Gathering • MC solicited for external feedback on the translation table (RFI)
• Workshops held to discuss contractor findings, with invited experts
in attendance
3 • MC conducted in-depth internal review of materials developed by
contractors, and support staff
• MC independently submitted preliminary votes on proposed
Internal Review standards
• MC deliberated to reach consensus on recommendations to be
endorsed in the report
4 Report • Refined recommendations and report content per committee
evaluations and discussions
Generation
16
17. Methodology Report – Internal Review
The MC deliberated and agreed upon standards based on the following:
Patient- Respect for and responsiveness to individual
Centeredness patient preferences, needs, and values
Objectivity, minimizing bias, improving
Scientific Rigor reproducibility, complete reporting
Explicit methods, consistent application, public
Transparency review
Empirical/ Information upon which a proposed standard is
Theoretical Basis based
Other Practicality, feasibility, barriers to implementation,
Considerations and cost
17
18. Methodology Report
Submitted to the PCORI
Board of Governors on May
10, 2012
Accepted by the PCORI
Board of Governors on May
21, 2012
A public comment period
on the draft report:
Through September 14
2012
Revised Report goes to the
Board of Governors
November 2012
18
19. Webinar Agenda
1. Introduction to PCORI 1:00pm – 1:05pm ET
2. A Unique Focus on Patient Engagement 1:05pm – 1:10pm ET
3. Methodology Committee 1:10pm – 1:20pm ET
Mission & Report
4. Patient-Centeredness and Research 1:20pm – 1:35pm
Prioritization
5. Questions and Answers 1:35pm – 2:00pm ET
Please submit questions for the Q&A portion of today’s webinar to
methodswebinar@pcori.org
Formal public comments can be submitted at
19 pcori.org/survey/methodology-report/
20. Methodology Report – Research Domains
Patient- Patient Research
Centeredness Engagement Prioritization
Heterogeneity
Causal General and Missing
of Treatment
Inference Crosscutting Data
Effects
Data Adaptive Data Diagnostic
Networks Trials Registries Testing
20
21. What Makes a Study Patient-Centered?
• Patient-centered outcomes research starts from the
perspective of individual facing health decisions.
• Every phase of patient-centered outcomes research should be
directed towards informing health decisions that affect
outcomes meaningful to patients.
• Patient-centered outcomes research helps people make
informed health care decisions.
21
22. Patient Engagement
Source: 1Curtis, P, Slaughter-Mason, S,
Thielke, A, Gordon, C, Pettinari, C, Ryan, K,
Church, B, King, V(2012). PCORI Expert
Interviews Project: Final report. Portland, OR:
Center for Evidence-based Policy, Oregon
Health & Science University 22
23. Standards for Patient-Centeredness
and Engagement
3.1.2 Identify Specific Populations and Health Decision(s) Affected by the Research
3.1.5 Measure Outcomes that People in the Population of Interest Notice and Care
About
4.1.1 Engage Patient Informants, Persons Representative of the Population of
Interest, in All Phases of Patient-centered Outcomes Research (PCOR)
4.1.2 Identify, Select, Recruit, and Retain Study Participants Representative of the
Spectrum of the Population of Interest Facing the Health Decision of Interest and
Ensure that Data Are Collected Thoroughly and Systematically from All Study
Participants
4.1.3 Use Patient-Reported Outcomes When Patients or People at Risk of a
Condition Are the Best Source of Information
4.1.4 Develop and Implement a Dissemination Assessment to Achieve Broad
Awareness of Study Results
23
24. Research Prioritization
• Need to select from among all possible research topics
– Methodology Committee Process
• Consider Prioritization Factors
• Develop Framework for Establishing Priorities
• Created Standards for selected components of Framework
• Standards must align with overall PCORI approach
– Promote patient-centeredness and engagement
24
25. Prioritization Factors
• Disease or condition incidence, prevalence and burden
• Patient needs, outcomes and preferences
• Gaps in evidence
• Relevance to informed health decisions
• Potential for improvement based on new evidence
• Efficient use of PCORI research resources
• Priorities developed by other organizations
25
26. Methods to Assist in Prioritization
• Topic Generation
• Identify questions that could be studied
• Gap Analysis in Systematic Review
• Reviewing what has already been studied and figuring out
what questions research has not answered yet
• Value of Information Analysis
• A conceptually-driven framework for estimating the impact
that new information from research could have
• Peer/Stakeholder Review
• Involving patients and other decision-makers in deciding
what to study
26
27. Framework for Prioritization
Value of Information
Gap Analysis in Value of
Peer/Stakeholder
Topic Generation Systematic Information
Review
Review Analysis
27
28. Standards for Research Prioritization
• 5.1.1 Use Systematic Reviews to Identify Gaps in Evidence
• 5.1.2 Protect Independence in Peer Review of Research Funding
Proposals
• 5.1.3 Ensure Adequate Representation of Minorities and
Disadvantaged Segments of the Population in Peer Review of
Research Funding Proposals
28
29. Webinar Agenda
1. Introduction to PCORI 1:00pm – 1:05pm ET
2. A Unique Focus on Patient Engagement 1:05pm – 1:10pm ET
3. Methodology Committee Mission & Report 1:10pm – 1:20pm ET
4. Patient-Centeredness and 1:20pm – 1:30pm ET
Research Prioritization
5. Questions and Answers 1:30pm – 2:00pm ET
Please submit questions for the Q&A portion of today’s webinar to
methodswebinar@pcori.org
Formal public comments can be submitted at
pcori.org/survey/methodology-report/
29
30. Questions and Answers
Please submit questions for the Q&A portion of today’s webinar to
methodswebinar@pcori.org
Formal public comments can be submitted at
pcori.org/survey/methodology-report/
30
31. Poll Questions 5 - 8
5. Have you submitted an application for funding to PCORI in the past?
(Y/N)
6. Do you plan to submit an application for funding to PCORI in the future?
(Y/N)
7. Rate your understanding of the process the Methodology Committee used
to generate standards:
a) I do not understand the process the Methodology Committee used to
generate standards
b) I understand the process somewhat
c) I have good understanding of the process the Methodology Committee
used to generate standards
8. Do you plan to submit comments on the Report through the PCORI
website? (Y/N)
31
32. We look forward to your comments on the Draft
Methodology Report
Visit us at www.pcori.org
(today’s webinar will be
archived there)
Subscribe to PCORI updates
at pcori.org/subscribe
Follow @PCORI on Twitter
Watch our YouTube channel
PCORINews
32
Notes de l'éditeur
Lori to introduce webinar and welcome listeners.Please comment on title to use.
Lori to describe agenda for the hour, with explanation of how to submit questions and first of many encouragements for people to submit comments through website. Lori to explain that webinar is being recorded and will be available on the PCORI website, and that the Aug 3 webinar is already available, the one in which RM standards were discussed.
Lori to introduce first poll and give instructions for completing. Poll questions:1. Are you familiar with the contents of the Methodology Committee Report? (Y/N)2. Are you a researcher? (Y/N)3. Rate your understanding of the process the Methodology Committee used to generate standards: a) I do not understand the process the Methodology Committee used to generate standards. b) I understand the process somewhat. c) I have good understanding of the process the Methodology Committee used to generate standards. 4. Which response most closely matches your opinion of the Standards in the draft Report? a) The Standards largely cover the main areas important to patient-centered outcomes research. (Agree/Disagree) b) Several important areas are not covered and additional Standards should be considered. (Agree/Disagree)
LoriThank you. Now we’ll hear from Gail Hunt…(introduce Gail to talk about PCORI)
Gailto speak to this slide.Before we get to the specifics about the Methodology Report, I want provide some background on the mission of PCORI.PCORI is an independent, non-profit organization committed to seeking input from patients and other stakeholders on all aspects of our work. Why was PCORI created?For all of the research conducted; for all of the therapies created; for all of the advances in care that have been made – Patients and clinicians do not always have the information they need to make choices that reflect their unique situation, priorities or personal preferences. In some areas, research has yet to ask some of the most important questions patients have. PCORI is committed to helping people make informed health decisions b producing and promoting high quality evidence-based information to guide patients, caregivers, and clinicians.
Gail’s slideHere is a definition of patient-centered outcomes research that guides the work PCORI does.
Gail
GailIn the course of creating the MC report, PCORI, through contractors, spoke with a lot of people all over the country. This map shows some of the parts of the country where researchers spoke with patients, caregivers, and other stakeholders in patient-centered outcomes research. In addition to input from patients, caregivers and other stakeholders, input was also received from 17 groups representing 100 individuals on research teams, 15 invited experts that provided input on the standards at workshops in March 2012, 24 submissions to provide input on the translation table after a Request for Information, and 57 stakeholders were interviewed to understand CER use in health records and informatics.
GailSpeak to bullets on slide and hand back over to Lori.
Lori reminds people to submit questions for Q&A portion of today’s session.Lori: Next we’ll hear from two members of the Methodology Committee to provide some background on the process the MC used and then they’ll review specific standards.Lori turns it over to Ethan
Ethan:Gail Hunt introduced you to the overall mission of PCORI, and Gail Hunt explained how PCORI’s focus on patient-centeredness makes it unique.I will now provide some background on the Methodology Committee and the draft standards.The Methodology Report is now posted for public comment with the comment period open until September 14.The Methodology Committee of the Patient-Centered Outcomes Research Institute (PCORI) was appointed by the Comptroller General of the U.S. Government Accountability Office in January 2011, after appointment of the PCORI Board in September 2010. The MC has a specific function outlined in the statute.To achieve its goals, The Methodology Committee consists of 17 experts: You can see the diversity of scientific expertise on this slide, which includes:Health services researchClinical researchComparative clinical effectiveness researchBiostatistics Genomics Research methodologies.This slide is included as reference and will be available in the archived slideset, and details about the membership of the Committee is available in the Report.
Ethan:The mandate for the Methodology Committee of PCORI is to define methodological standards and develop a translation table to guide health care stakeholders towards the best methods for patient-centered outcomes research (PCOR). The MC has four specific roles:Provide guidance about the appropriate use of methods in patient-centered outcomes research Establish priorities to address gaps in research methods or their application Recommend actions to support standards Map research methods to specific research questions via Translation TableThe methodology report is the first deliverable to address PCOR methods. The report includes the first set of methodological standards and a translation table to guide health care stakeholders towards the best methods for patient-centered outcomes research (PCOR). Use of these standards will promote Rigorous methods that are essential to building trust in research findings. Using scientifically rigorous methods will produce patient-centered outcomes research results that can inform decision-making. Once the Report is revised and accepted by the PCORI Board of Governors, future PCORI funding applicants will be expected to reference the Standards in their applications and use the Standards in their PCORI funded research.Ethan turns it over to David.
David:The methodology report includes the recommended standards for PCOR. Building on the work of the IOM*, the MC defined a standard as…A process, action, or procedure for performing PCOR that is deemed essential to producing scientifically valid, transparent, and reproducible results. Standards can be supported by scientific evidence, a reasonable expectation that the standard helps achieve the anticipated level of quality in PCOR, or by broad acceptance of the practice in PCORThe recommendation is actionable, feasible, and implementableProposed standards are intended for use by the PCORI Board, in PCORI policies and procedures, and by PCORI researchers
David:For the first report, the methodology committee sought to address selected topics in four broad phases of activities. Those four phases were:What should we study? (research priorities)What study design should we use, and how do we carry out and govern the study? (research designs and methods)How do we enable people to apply the study results? (patient-centerdness)
DavidThe process for developing the first methodology report over our first year together as the methodology committee involved four phases: method selection, information gathering, internal review, and report generation.Method selection involve working in groups to prioritize the major research methods questions to be addressed.Second, we entered anInformation gathering phase in which request for proposals were publicly solicited for selected topics, contractors were selected and completed their reports with summary templates of proposed standards. In addition to methodology committee solicited externalfeedback on the translation table through a request for information. Workshops were held to discuss contractor findings with invited experts in attendance.Third, the methodology committee began a phase of internal review. The committee conducted an in-depth internal review of the materials that were developed by the contractors and support staff. We then independently submitted are pulmonary votes on the proposed standards and then met in person to deliberate and reach consensus on the recommendations to be endorsed in the report.The report was then generated and refined based on committee of evaluations and discussions.
David
DavidThe first methodology report was specifically required by the legislation that created PCORI, as we stated earlier. The MC submitted a draft report to the Board of Governors in May. The board accepted the draft and we are now asking for public comment. We want and need input….and based on that as well as ongoing review by the Methods Committee the report will be revised and a final version will be submitted to the Board in November.
David turns it back to Ethan
EthanAs we mentioned earlier, today’s webinar covers standards related to patient-centeredness and patient engagement, and research prioritization, this top row in green. The webinar on August 3 addressed standards relating to research methods, represented by all the blue boxes, the two bottom rows.
EthanThis slide summarizes the main features of PCOR. PCOR is focused on helping people make the best health decisions possible. The patient’s perspective must be a part of the research for it to be patient-centered. The patient perspective should be a part of all phases of research. There are many ways to incorporate the patient perspective into research.
EthanTo be truly patient-centered, research needs to engage patients at all steps. This image represents engagement at all steps. The MC workgroup charged with developing these standards worked with experts who reviewed the literature and spoke with patients, caregivers, and other stakeholders, through interviews and through focus groups, in order to provide information relevant for the development of the standards.
Ethan[Explain numbers correspond to chapter location.]The Standards focus on includingDiverse patientsOutcomes that are important to patientsAnd then focusing on getting the results to the people who need them.Ethan turns it back to David.
David
DavidThese are the factors the committee considered as it worked to develop a framework and selected standardsThe first two are linked to PCORI’s goals and mission…..research topics should address patient needs and help with decisions.Condition or disease incidence, prevalence and burden are all measure of how much impact a disease has on people. it is about how many new cases there are, how many total people have the condition and the impact the disease has on people’s livesExample of other organizations include the IOM or government agencies or private groups like foundations and or professional associations, or patient advocacy groups.
David
DavidThis framework lays out the steps.
DavidThe standards address only 2 of the 4 areas in the Framework….One standard is about making sure we use existing evidence to identify what needs to be studied.---Gap analysis of systematic reviews should be used as part of the process of identifyingand prioritizing research gaps to establish funding priorities by PCORI.Two standards are about the peer review process.One is about assuring its independence…Adopted methods of peer review should aim to safeguard independence betweenreviewers and those being reviewed.The other is about making peer review more inclusive.Approaches to topic generation in PCOR should involve both consultative andcollaborative functions.David turns it back to Lori.
LoriThank you allWe have been collecting questions as those listening in have been submitting them so now we’ll read some questions and I’ll ask Ethan and David and Gail to address them.
This slide provides the address to submit your questions to – we will answer as many questions as time permit. (Reminder to submit public comment to website location.)
1:58: Thank you again for your time today. We also would like to draw your attention to another poll we would like to you to complete…1. Have you submitted an application for funding to PCORI in the past? (Y/N)2. Do you plan to submit an application for funding to PCORI in the future? (Y/N)3. Rate your understanding of the process the Methodology Committee used to generate standards: a) I do not understand the process the Methodology Committee used to generate standards. b) I understand the process somewhat. c) I have good understanding of the process the Methodology Committee used to generate standards. 4. Do you plan to submit comments on the Report through the PCORI website? (Y/N)
Lori2:00pm ETThank you for your time today and thank you for your interest in the work of the MC and PCORI. A copy of these slides will be available on the PCORI website, and you can submit all comments on the MC Report through the website as well. This concludes our webinar today. Thank you again.