The use of ‘colloquial evidence’ in HTA: the experience of NICE
•Télécharger en tant que PPT, PDF•
1 j'aime•2,496 vues
The use of ‘colloquial evidence’ in HTA: the experience of NICE
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à The use of ‘colloquial evidence’ in HTA: the experience of NICE
Similaire à The use of ‘colloquial evidence’ in HTA: the experience of NICE (20)
Plus de HTAi Bilbao 2012
Plus de HTAi Bilbao 2012 (20)
Dernier
Dernier (20)
The use of ‘colloquial evidence’ in HTA: the experience of NICE
- 1. The use of ‘colloquial evidence’ in HTA: the experience of NICE Health Technology Assessment international (HTAi) Annual Meeting 26 June 2012: 17:30-19:00 (Abstract ID: 381) Presenter: Thomas Wilkinson Authors: Tarang Sharma, Bindweep Kaur, Moni Choudhury, Bhash Naidoo, Sarah Garner, Lucy Connor (NICE)
- 2. Evidence for developing guidance • NICE produces guidance on best available evidence. • Evidence: – “findings from research and other knowledge that may serve as a useful basis for decision making in public health and health care.” (Health evidence network, WHO in Lomas et al, 2005)
- 3. Colloquial evidence • Definition: “anything that establishes a fact or gives reason for believing something.” (Oxford Dictionary, 1980) • Provides context and complements scientific evidence • Increasingly used in HTA Adapted from Lomas et • Controversial al (2005)
- 4. Aims and objectives • Aim: • To explore the use of colloquial evidence in the process of developing a guidance across all guidance-producing centres at NICE. • Objectives: – To understand the context of colloquial evidence from the available literature. – To identify consistencies and variations in the use of colloquial evidence across all centres at NICE. – To map the use of colloquial evidence at NICE to the conceptual framework outlined by Lomas et al 2005.
- 5. Methods
- 6. Results: Mapping colloquial evidence at NICE • All guidance-producing centres utilise colloquial evidence with scientific evidence, as per Lomas et al 2005.
- 7. Results: Colloquial evidence is utilised at all stages of guidance production Grey Literature Expert/ Patient/Carer input Stakeholder views
- 8. E.g. CG105 Motor Neurone Disease - Information and support needs of patients with motor neurone disease and their families and carers • Systematic review searches identified 11 eligible studies and a set of interviews from health talk online. • All were critically appraised via qualitative studies checklist and analysed through a ‘Thematic-Conceptual Meta Matrix’ (Miles and Rosenblum 1987). • Example of one of the evidence profiles looking at one outcome (timing, level of information and ways of communication). NICE clinical guideline 105 – Non-invasive ventilation for motor neurone disease: http://publications.nice.org.uk/motor-neurone-disease-cg105/other-versions-of-this-guideline#quick-reference-guide
- 9. Discussion and Conclusion • Colloquial Evidence is utilised alongside scientific evidence across NICE. • Methodology for integrating all kinds of evidence: – Deliberative process: “an integration of technical analysis and stakeholder and lay public deliberation, contrasting with the traditional ‘top- down’ or ‘bureaucratic rationalistic’ policy orientation” (Culyer et al, 2006) – Lack of methodology? – MCDA approach?
- 10. Next Steps • No formal tools for critically appraising colloquial evidence utilised across NICE: – Proposing SART Model • Source • Accuracy • Relevance • Timeliness • Following changes may impact the way Colloquial Evidence is utilised currently at NICE: – Methods and Process Manuals Update – Changing NICE’s remit – introduction of social care guidance – Value based pricing • Developing further methods of identifying and capturing colloquial evidence.
- 11. References • Culyer AJ, Lomas J (2006) ‘Deliberative processes and evidence informed decision making in healthcare: do they work and how might we know?’ Evidence and Policy Vol. 2(3) pp 357-71. • Lomas J, Culyer T, McCutcheon C et al. (2005) Conceptualizing and combining evidence for health system guidance: final report. Ottawa: Canadian Health Services Research Foundation • Reay CA, Colechin ES, Bousfield DR et al (2011) ‘Review of published literature relating to methods for identifying, synthesis and integration of colloquial evidence’. Newcastle upon Tyne Hospitals NHS foundation trust • 20 National Institute for Health and Clinical Excellence (2009). Centre for Clinical Practice. The guidelines manual 2009 Available at: http://www.nice.org.uk/aboutnice/howwework/developingniceclinicalguidelines/clinicalguidelinedevelopm • National Institute for Health and Clinical Excellence (2009) Centre for Public Health Excellence. Methods for development of NICE public health guidance 2009 Available at: http://www.nice.org.uk/aboutnice/howwework/developingnicepublichealthguidance/publichealth guidanceprocessandmethodguides/public_health_guidance_process_and_method_guides.jsp? domedia=1&mid=F6A97CF4-19B9-E0B5-D42B4018AE84DD51 • National Institute for Health and Clinical Excellence (2008) Implementation support process manual Available at: http://www.nice.org.uk/usingguidance/niceimplementationprogramme/nice_implementation_pro gramme.jsp?domedia=1&mid=656BA97E-19B9-E0B5-D437A317C324122F • National Institute for Health and Clinical Excellence (2011) Citizen’s Council http://www.nice.org.uk/aboutnice/howwework/citizenscouncil/citizens_council.jsp
- 12. Links/Contact details Links: Research and Development, NICE: http://www.nice.org.uk/aboutnice/howwework/researchanddevelopment/about.jsp Authors/Contributors: • Ms Tarang Sharma, Senior Analyst • Ms Bindweep Kaur, Analyst • Miss Moni Choudhury, Analyst • Dr Bhash Naidoo, Associate Director • Dr Sarah Garner, Associate Director • Ms Lucy Connor, Programme Manager Contact: • Sarah Garner Tel: 44 (0)207 045 2097; • Email: Sarah.garner@nice.org.uk
Notes de l'éditeur
- NICE produces guidance based on best available evidence which includes both scientific as well as non-scientific or colloquial. Lomas et al have divided the scientific evidence into context free and context specific evidence. Context free evidence produces universal truths of what works and is usually available from RCT. And context specific evidence is more about what works in specific circumstances and for specific populations which is usually available from epidemiological studies and observational studies. However, while producing NICE guidance we are very commonly faced with issues of limited evidence and a need for filling gaps in the evidence. To overcome these issues, Lomas et al suggests utilising colloquial evidence or ‘Other knowledge’ which is usually in the form of grey literature or expert opinion.
- Colloquial Evidence has been defined by the oxford dictionary as ‘anything that establishes a fact or gives reason for believing something’ and it goes beyond the research evidence that is obtained from rigorous RCTs or epidemiological studies. It includes evidence from experts/ professionals/ lobbyists and pressure groups as well as from pragmatics. It also includes evidence on social values which should underpin the guidance, population’s habits and traditions (which should be taken into consideration while developing a guidance) and political judgements that might influence the guidance. Colloquial evidence helps to provide context for scientific evidence as well as fills gaps in the scientific evidence and is being increasingly used in Health technology assessments. For example, new medical devices and diagnostics do not have a big evidence pool to suggest their clinical or cost-effectiveness and in situations such as this, colloquial evidence can be extremely useful. However, it’s use is highly controversial and is increasingly being debated. Some researchers do not regard it as evidence but ‘colloquial information’, although guidance developers regard it as an important piece of evidence on which they can base their recommendations.
- The use of colloquial evidence in the process of guidance development was identified as a methodological priority by the internal methods advisory group at NICE and the R&D team within NICE took on this project. The aim of the project was to explore the use of colloquial evidence in the process of developing a guidance across all guidance producing centres at NICE. The project had 3 objectives: To understand the context of colloquial evidence from the available literature To identify the consistencies and variations in the use of colloquial evidence across all centres at NICE To map the use of colloquial evidence at NICE to the conceptual framework outlined by Lomas et al (mentioned in the previous slides)
- Literature Review: A rapid scoping literature review of key background papers in the field (focusing on summarising information from recent systematic reviews and reports) was undertaken. The scoping review informed the setting out of the context and background for the mapping study of the use of colloquial evidence at NICE. Data extraction: The methods and processes manuals of all guidance producing centres at NICE were considered for extraction and the data was extracted from manuals of clinical guidelines, public health, technology appraisals (single technology assessment [STA] and multiple technology assessment [MTA]), interventional procedures [IP], medical technologies evaluation programme [MTEP], quality standards (using the interim process guide) and the diagnostics assessment programme [DAP]. All text referring to colloquial evidence was extracted into data extraction forms by 2 reviewers and was quality assured by an independent 3 rd reviewer. Data analysis: The data was analysed by utilising thematic analysis. All the data in the data extraction forms was divided into various themes and was reviewed as per the conceptual framework provided by Lomas et al. Data analysis:
- The focused literature review provided some key papers which helped us to lay out the context of colloquial evidence. The literature also provided us with some critical appraisal tools which can be used to critically appraise colloquial evidence. Data analysis clearly reflected that all guidance producing centres were utilising colloquial evidence , in addition to scientific evidence, for their guidance. It was either used more or less often, depending on the nature of the guidance being produced (public health, DAP used more) and in the type of question being considered by the guidance developers. Both context free and context specific evidence was utilised (in the form of RCTs and epidemiological studies) to estimate the effect and the ICER. In addition, expert and patient/carer input was utilised as evidence in the form of expert testimonies and also during the deliberative process. Comments from stakeholder consultations were utilised across the whole process of guidance development. Grey literature in the form of conference abstracts and websites was utilised. The public health guidance specifically utilised field work to map current practice. Colloquial evidence is also utilised at NICE in the form of citizen’s council which is a board of 30 lay members of the public. The council discusses social and moral issues raised by NICE guidance and provides NICE with advice that reflects the public’s perspectives. The citizen’s council report feed into the methods and processes guide of all centres and although do not influence the recommendations directly, make sure that the public’s views underpin each guidance.
- Colloquial evidence is utilised at all stages of guidance production from topic selection to final guidance and implementation tools. Topic selection and Scoping: Expert and patient/carer views are solicited informally through discussions or formally through a stakeholder workshops. Hand searching of websites at the scoping stage of guidance is undertaken to identify any relevant reports and guidance relevant to the topic. The draft scope undergoes a process of consultation, whereby patient groups, professionals, manufacturers etc can comment, and their views are considered to finalise the scope. Evidence review and deliberative process: Expert and patient/carer views can be used as evidence in the form of expert testimonies or commentaries or through conducting surveys and/ or audits. It can also be used to populate the parameters of the economic model to fill in the gaps in the literature. Reviews to map local practice and of current policy and practice are undertaken for Public Health guidance. Hand searching for conference abstracts and reports during guidance development can also be undertaken, to supplement the evidence in the published peer-reviewed literature. Data from online registers and surveys are also be used. Healthcare professionals and patient/ carers express and share their views and experiences which are considered as evidence by the independent committees or groups. The members of independent committees as well as co-opted experts themselves also share their views on the evidence during a deliberative process at these formal meetings. Economic Modelling: Expert input in the form of which parameters need to be added into the economic model Consultation: Comments from stakeholders such as patient groups, professionals, manufacturers, peer reviewers and experts on the draft guidance are taken into consideration for final guidance publication. Implementation Tools: Once the guidance has been finalised for publication, different implementation support tools such as audit support. Educational tools, costing support and implementation advice are produced with the help of experts.
- This a figure to illustrate how different types of evidence can be considered together such that a recommendation can be made for a particular outcome. In this example for a question identifying the information and support needs of patients & families and carers with MND – a systematic review identified 11 published studies from the literature. But additionally a set of relevant interviews from the healthtalk website hosted by University of Oxford, were also considered alongside the published literature as part of the overall evidence to be considered by the guideline development group for making a recommendation for practice. All studies and the interviews from the website were critically appraised and analysed together using rigorous qualitative research method techniques. However, this is not done systematically across different pieces of guidance and across the different programmes at NICE.
- All guidance producing programmes at NICE use the best available evidence to inform their decisions of clinical and cost effectiveness, but more often it requires making judgements because of the lack of good quality or complete evidence including scientific and colloquial. More used with Public health, MTEP, DAP (lack of evidence, complementing evidence) Colloquial evidence is therefore utilised along side scientific evidence across the guidance development process and across all guidance producing centres at NICE. However, there are still questions about methodology for integrating colloquial evidence with scientific evidence. The deliberative process has been considered to be the “ careful, deliberate consideration and discussion of the advantages and disadvantages of various options ”. Therefore this is a sense of “ weighing up of the evidence ” (but this can be done from a range of explicit algorithmic formal quantitative methods to a very informal discursive manner) and can both “ elicit and combine evidence ” 9 . This has been argued to be a form of multi criteria decision making (MCDA) model, where various forms and types of evidence are considered together to provide the final recommendations for practice and where individual components may also be derived through a MCDA process (e.g. economic modelling)
- Although literature has identified critical appraisal tools, these are not currently used by any of the centres at NICE. We are proposing a SART model One first needs to examine the source of the colloquial evidence to determine its authority (to see by whom was it written and its purpose, whether one could trust the authors, or website; does the evidence appear to be objective or are there any potential biases present etc). It is worth considering how accurate the information could be (how plausible are the claims; are the aims and information clear and if sources are identified, are they available, are any inconsistencies present in the content etc) It is worth seeing how relevant the evidence is with respect to the scope of the guidance, i.e. how applicable the information really is to the question at hand, (is all the information there, is it complete or discussing only part of the equation etc). One should also consider the timeliness of the information, how current is it – i.e. when was it written or the last date when the website was updated etc. Methods and Process Manuals Update may impact the way CE is utilised in the decision making process Changing NICE’s remit: quality standards and social care Value based pricing: more emphasis on innovation and broader outputs. More work is needed on further developing the methods of identifying and capturing colloquial evidence