Publicité

Contenu connexe

Similaire à Measuring the Effectiveness of eHealth Initiatives in Hospitals(20)

Publicité

Measuring the Effectiveness of eHealth Initiatives in Hospitals

  1. Measuring the effectiveness of e-health initiatives in hospitals Prof Johanna Westbrook Health Informatics Research & Evaluation Unit The University of Sydney
  2. Health Informatics Research & Evaluation Unit • 17 research staff most funded by grants • Aims: – Develop and test rigorous and innovative evaluation tools & approaches. – Produce research evidence about impact of ICT on health care delivery, professionals’ work and patient outcomes. – Disseminate evidence to inform policy, system design, integration and effective use of ICT in health care.
  3. Research Questions • Do pathology order entry systems deliver more efficient care? • Do electronic medication management systems make health care safer? • Do clinical systems make clinical work more efficient and release clinicians to spend more time with patients? • What is the role of mobile technologies in supporting clinical work in hospitals? • Approaches, results to date, methodological challenges
  4. Is care delivery more efficient? • Few studies all specialised units all reported improved turnaround times. Computerised test ordering Turnaround time = Time from receipt of specimen in laboratory to report of result
  5. AIMS 1. Do turnaround times decrease in the first 12 months following system introduction and are improvements sustained? 3. What is the impact on pathology staff?
  6. Methods 650 teaching hospital Measurement of TAT pre & post CPOE -Cerner Millennium PowerChart Periods Jul – Aug 2003 Jul – Aug 2004 (post 1) Jul – Aug 2005 (post 2) Westbrook JI, et al. (2006) Computerised pathology test order-entry reduces laboratory turnaround times and influences tests ordered by hospital clinicians: A controlled before and after study. Journal of Clinical Pathology, 59, 533-536.
  7. Test turnaround time significantly declined Year 1 by 18.6% , Year 2 by 12.6% Period No. tests Mean in minutes (95% C.I.) 2003 97851 35.35 (35.11,35.59) 2004 113752 28.77 (28.59,28.95) All tests 2005 131022 25.14 (24.99,25.29) • Average number of tests per patient did not change: 92.5 assays/pt vs 103.2 (P=0.23)
  8. Changes in TAT post CPOE in four hospitals 2005 Before 2006 After 2007 After Kruskal- Wallis Hospital A - Median TAT 77 68 66 P<0.001 % tests using CPOE 75% 80% Hospital B - Median TAT 145 129 108 P<0.001 % tests using CPOE 0-44% 57% Hospital C- Median TAT 138 135 113 P<0.001 % tests using CPOE 29-38% 53% Hospital D- Median TAT 141 139 128 P<0.001 % tests using CPOE 56-71% 74%
  9. Effectiveness – Does a reduction in TAT really matter? Is there a relationship between TATs and lengths of stays in an emergency department prior to CPOE? Regression analyses - TAT was a significant factor contributing to patients’ length of stay in ED (p<0.0001). Westbrook JI, et al (2009) Does computerised provider order entry reduce test turnaround times?: a before and after study at four hospitals. Stud Technol Inform; 150: 527-531.
  10. Qualitative studies to assess the impact pathology work Focus groups & interviews with management, pathology, clinical and IT department staff Observational video study of pathology staff over several months
  11. “…I don’t have figures to prove this, but in my estimation it has made the turnaround time longer.” (Senior scientist, 2004)
  12. Implementing Systems Changes in roles & responsibilities Elimination of some tasks but creation of new tasks Failure of one group to use the system as expected impacts upon the work of others These elements of system impact are as important as quantitative indicators!
  13. Benefits realisation framework Efficiency Effectiveness Quality Test costs Redundant test rates Turn around times Work practices Patient safety Compliance with guidelines Patient management Length of stay Test volumes Communication Georgiou A, et al (2007) The impact of computerised physician order entry systems on pathology services: a systematic review. Intern J Med Informatics 76 (7), 514-529. Georgiou et al. (2008) Electronic test management systems and hospital pathology services – a framework for investigating their impact. Encyclopaedia of Healthcare Information Systems
  14. Will electronic medication management systems make our health services safer?
  15. Do e-prescribing systems reduce prescribing errors in hospital inpatients? 13 papers (US 6, UK 4, Europe 2, Israel 1) – 9 showed significant decrease – 2 decrease in some categories – 2 an increase in errors • Limitations in study designs, eg only specific drugs • Only 5 studies examined severity of errors – 2 defined their scales • Very limited evidence of effectiveness to reduce serious errors Reckmann, Westbrook et al (2009) Does computerized order entry reduce prescribing errors for hospital inpatients? A systematic review. Journal of American Medical Informatics Association. 16 (5) 613-623.
  16. Controlled Before & After study 2 Hospitals 2 Systems 6 wards
  17. Methods Prospective medication chart review pre & post. – Inter-rater reliability, kappa = 0.82-0.84 Classification of: – error types – severity – 5 point scale – Clinical – Documentation – System-related • 2006 – pre 2008/9 - post Prescribing error types Wrong medication Wrong dose / volume Wrong rate /frequency Wrong route Wrong timing Wrong strength Wrong formulation Wrong patient Not prescribed Drug-drug interaction Duplicated drug therapy Patient Allergic Drug not indicated Inadequate monitoring Unclear order Incomplete order Unsigned order Legal/Procedural System related
  18. Do electronic medication administration records reduce errors? Few studies – all flawed methods – Perceptions of staff – Examination of voluntary incident reports
  19. Observational Medication Administration Error Study • Observe nurses as they prepare & administer medications • Record interruptions & multi-tasking • Compare observed data with patients’ charts to identify errors
  20. Study Methods 6 wards at 2 hospitals Information sessions to recruit nurses – - 82% response rate (n=98 nurses pre) • Researchers arrived on the wards at peak medication times (7:00-19:30) • Approx 8 administrations/observation Hr • Inter-rater reliability – Kappa score 0.94-0.96 • Serious error protocol used 10 times
  21. How does system use impact upon patterns of work? Will these systems save time? Do drs & nurses spend more time with patients?
  22. Aim: To develop a reliable method for observing and recording time spent by clinicians in different work tasks Work Observation Method By Activity Timing (WOMBAT) Westbrook JI, Ampt A (2009) Design, application and testing of the Work Observation Method by Activity Timing (WOMBAT) to measure clinicians’ patterns of work and communication. International Journal of Medical Informatics. 78S, S25-S33.
  23. PDA data collection tool What task? With whom? With what? Interruptions Multi-tasking
  24. Controlled before and after study nurses and doctors 4 wards at baseline 1 or 2 intervention wards 2 control wards post Completion date Dec 2009
  25. Proportions of observed time in tasks across four wards (Before) Task Nurses N=52 Ward Drs N=19 ED Drs N=40 Hours of observation 250 hrs 151 hrs 210 hrs Direct Care 24% 15% 29% Professional Communication 22% 33% 24% Medication tasks 17% 7% 5% Indirect Care 13% 18% 26% Social Activities 13% 17% 6% In transit 9% 6% 3% Documentation 8% 14% 16% Supervision/education 3% 7% 2% Administration 3% 2% 2% Answering Pager 1% 1% <1%
  26. Time with patients & interruptions (Baseline data) Nurses = 34.5%, interrupted 1/49mins, 12% multi-tasking Ward Drs = 15.0%, interrupted 1/21mins, 20% multi-tasking • On average nurses spend 8.4 mins/shift talking with a Dr. Westbrook JI, et al (2008) Medical Journal of Australia. 188(9): 506-509.
  27. Distribution of doctors’ tasks over the day 2006 0% 5% 10% 15% 20% 25% Time Proportion of task Direct care Indirect care Medication tasks All documentation Prof Communication In transit
  28. Distribution of doctors’ tasks over the day including social tasks 2006 0% 5% 10% 15% 20% 25% Time Proportion of task Direct care Indirect care Medication tasks All documentation Prof Communication In transit Social
  29. Data Analysis • Changes in – distribution of time across tasks – average time for each task – frequency of each task – times of the day when tasks completed – with whom tasks are completed • A lot more to come …….
  30. Challenges of integrating the use of technology into everyday work practices
  31. Poor mobility workarounds may result in less safe practices
  32. Paper is a highly mobile technology!
  33. Capturing what happens on a ward Structured observations Video observations Talking to staff 80 hours observation, 2 wards Aim: To measure which devices nurses and doctors select Andersen P, Lindgaard A, Prgomet M, Creswick N, Westbrook JI (2009) Is selection of hardware device related to clinical task?: A multi-method study of mobile and fixed computer use by doctors and nurses on hospital wards. J Medical Internet Research. 11(3) Available devices on each ward: Two forms of COWs (n=5 & 6) Two forms of tablets – (Motion computing C5 & LE1700) (n=2/ward) Fixed PCs (n=7)
  34. Computers on wheels 82% of nurses’ tasks 3% of nurses’ work tasks
  35. Doctors’ on ward rounds 57% of tasks completed using a generic COW 36% of tasks completed using a tablet Only 3% of tasks completed at the patient’s bedside
  36. Conclusions Recognise the limitations of existing evidence-base Use explicit indicators & measure them Engagement of academics/clinicians/ vendors Feedback impact data to staff Create a market for evidence of impact Share & compare between systems, organisations
  37. Acknowledgements HIREU Team • Andrew Georgiou • Joanne Callen • Amanda Woods • Margaret Reckmann • Connie Lo • Yvonne Koh • Fiona Ray • Nerida Creswick • Marilyn Rob • Mirela Prgomet • Antonia Hordern • Fiona McWhinney • Pia Andersen • Anne-Mette Lingaard Funding Bodies • Australian Research Council • NH & MRC • NSW Health • HCF Research Foundation • SSWAHS Hospital staff at our study sites Publications available at: www.fhs.usyd.edu.au/hireu/ J.Westbrook@usyd.edu.au
Publicité