This document discusses the need for rigorous evaluation of e-health and mhealth technologies. It notes that while such technologies are widely used, their actual impacts on productivity and health outcomes are unclear. Evaluations of healthcare IT have found unintended consequences like increased clinician workload and new types of medical errors. The document calls for investment in e-health to be based on evidence from well-designed trials and evaluations. It identifies key questions around measuring impacts, designing technologies to improve health, and supporting organizational changes. Finally, it presents guidelines for good evaluation practices in health informatics.
Call Girls Guntur Just Call 8250077686 Top Class Call Girl Service Available
Fostering research in effective e-Health and m-Health evaluation, Dr Simon Poon
1. Fostering Research in e-Health and mHealth Evaluation
FACULTY OF
ENGINEERING &
INFORMATION
TECHNOLOGIES
CPC e-Health showcase 6th Nov 2013
Dr. Simon Poon
2. The Problem
› Evaluation of Technological Impacts:
› The Productivity Paradox Debate
- “You can see the computer age everywhere but in the productivity statistics”
(Robert Solow 1987)
- “Despite differences in methodology and data sources, a consensus is building
that the remarkable behaviour of IT prices provides the key to the surge in
economic growth”
(Dale Jorgenson, 2001)
› Evaluations in Healthcare IT:
(Seldling et al. 2013)
2
3. Unintended Consequences
› Hinders clinical practice (The Telegraph UK, 14 December 2009)
› is not working as it should in UK (BBC News, 18 May 2011)
› “provide little support for the cognitive tasks of the clinicians or the
workflow that can increase the chance of error” (National Research Council
Report on Healthcare 2009, USA)
Types of Unintended Consequences
of EMR
More work for clinicians
Workflow Issues
Never ending system demands
Paper persistence
Change in communication
Emotions
New kinds of errors
Changes in power structure
Overdependence on technology
TOTAL
% Frequency
N= 324 (Campbell et al 2006)
Sample size
(Ash et al 2007a)
19.8
17.6
14.8
10.8
10.1
7.7
7.1
6.8
5.2
100.0
125
149
143
N/A
146
140
82
61
138
984
Rahadhan & Poon (2012)
3
4. Evaluation Research in e-Health
› Investing in e-Health on the basis on its evidence, not on its potential
- Urquhart (2009): found limited evidence of effective interventions… more well
designed trials are needed
- Shojania (2011): evaluated the effects of on-screen computer reminders…
median improvement of 4%
- Bright et al. (2012): reviewed 148 randomised trials… 86% assessed process
measures, 20% assessed outcomes and 15% accessed costs
› Key questions:
- What is valid and reliable measures of e-Health solutions?
- How to design and configure e-Health to improve health outcomes?
- How to identify complementary organisational changes and channel resource for
training?
4
5. Guideline for good evaluation practice in health
informatics (GEP-HI)
Nykänen et al. (2011)
5
Historically focused on the “Technology” not on the “Informatics”Many studies reported potential benefits of the CPOE system. One major challenge of CPOE systems is the insufficient reduction of medication errors – errors are unrelated to order structure or standardisation, but result from a lack of information of knowledge of the prescriber…E-health solutions however may play an important role both at the level of distinct sub-steps of the drug treatment process.Moreover, they may bridge interfaces, facilitate communication between persons involved and may thus help standardizing and structuring the overall treatment process.
Types of unintended consequences spread from workflow issues, communication problems to the occurrence of “new errors”.Unintended consequences were allegedly caused by inability of EMR/EHR to correspond to work organization and usual behaviours (Koppel et al 2005).Unintended consequences reflects HCI problems (Socio-technical Problem) (Redwood et al 2011)
Cochrane Report on ICT (Urquhart 2009)We found some limited evidence of effects on practice attributable to changes in [nursing] record systems.There is very limited evidence on effective interventions promoting the adoption of ICTs by healthcare professionals… more well designed trials are needed.Cochrane Report on Reminders (Shojania 2011)This review found 28 studies that evaluated the effects of different on-screen computer reminders.The review found small to moderate benefits. The reminders improved physician practices by a median of 4%. In eight of the studies, patients' healthimproved by a median of 3%.Bright et al (Annals of Int Med 2012)148 randomized trials reviewed Most (86%) assessed process measures; only 20% assessed outcomes and 15% costsMore evidence from commercially developed systems
STARE-HI - Statement on Reporting of Evaluation Studies in Health Informatics