Recommandé
Contenu connexe
En vedette
En vedette (13)
Similaire à 2015 july06 psc frances healey ps data or ps intelligence 30 mins
Similaire à 2015 july06 psc frances healey ps data or ps intelligence 30 mins (20)
Dernier
Dernier (20)
2015 july06 psc frances healey ps data or ps intelligence 30 mins
- 1. www.england.nhs.uk Patient safety data or patient safety intelligence? Dr Frances Healey RGN, RMN, PhD Head of Patient Safety Insight NHS England 6 July 2015
- 2. www.england.nhs.uk 1st rule of #statisticsclub Qualitative data are at least equally important, and probably much more important, than quantitative data….. 2 ….. but it’s quantitative data that have the pitfalls & perils, so that is my focus today
- 5. www.england.nhs.uk 5 Dammit Jim, I’m a doctor, not a rocket scientist Dammit Frances, I’m a nurse, not statistician
- 6. www.england.nhs.uk 2nd rule of #statisticsclub You are probably much less logical than you think you are 6
- 8. www.england.nhs.uk 8www.england.nhs.uk “The results at that stage showed a slight numerical advantage for those who had been treated at home. It was of course completely insignificant statistically. “I rather wickedly compiled two reports, one reversing the numbers of deaths on the two sides of the trial. As we were going into committee, in the anteroom, I showed some cardiologists the results……..
- 9. www.england.nhs.uk 9 “……they were vociferous in their abuse: `Archie’, they said, `we always thought you were unethical. You must stop the trial at once…’ “I let them have their say for some time and then apologised and gave them the true results, challenging them to say, as vehemently, that coronary care units should be stopped immediately. “There was dead silence and I felt rather sick because they were, after all, my medical colleagues.” Professor Archibald Cochrane & Max Blythe One Man's Medicine (1989) p.211
- 10. www.england.nhs.uk Cognitive dissonance 10 • We have a strong need for our personal beliefs and our personal actions to chime • The discomfort we feel when they don’t is ‘cognitive dissonance’ http://britishgeriatricssociety. wordpress.com/2013/05/16/ all-down-to-numbers/ • If we believe we are part of effective, motivated, caring teams, it is very hard to also simultaneously believe: o We haven’t achieved real improvements in safety o We might be less safe than peers
- 11. www.england.nhs.uk 11 http://www.health.org.uk/multimedia/slideshow/hard-data-soft-intelligence/ Data (mis)used for reassurance
- 12. www.england.nhs.uk Which is not to say we shouldn’t use data for motivation “The consistent delivery of well-executed safe care under typically difficult circumstances tends to go unrecognised"
- 14. www.england.nhs.uk 3rd rule of #statisticsclub One size does not fit all - there is no such thing as a good indicator, or a good data source, just one that is good in particular situation for a particular purpose 14 0 50000 100000 150000 200000 250000 300000 350000 400000 450000 500000 Oct - Dec 2003 Jul - Sep 2004 Apr - Jun 2005 Jan - Mar 2006 Oct - Dec 2006 Jul - Sep 2007 Apr - Jun 2008 Jan - Mar 2009 Oct - Dec 2009 Jul - Sep 2010 Apr - Jun 2011 Jan - Mar 2012 Oct - Dec 2012 Jul - Sep 2013 Apr - Jun 2014 IncidentsSubmitted
- 15. www.england.nhs.uk 15 CULTURE indicators STRUCTURE indicators PROCESS indicators planning process indicators delivery process indicators OUTCOME indicators Are we safe today? Types of indicator
- 16. www.england.nhs.uk Our high tech interactive facility 16
- 17. www.england.nhs.uk What type of indicator? 17 1. CULTURE indicator 2. STRUCTURE indicator 3. PROCESS - planning process indicator 4. PROCESS - delivery process indicator 5. OUTCOME indicator 97% of patients who need a pressure reliving mattress received it within four hours
- 18. www.england.nhs.uk What type of indicator? 18 1. CULTURE indicator 2. STRUCTURE indicator 3. PROCESS - planning process indicator 4. PROCESS - delivery process indicator 5. OUTCOME indicator 86% of nurses agree that most pressure ulcers can be prevented
- 19. www.england.nhs.uk What type of indicator? 19 1. CULTURE indicator 2. STRUCTURE indicator 3. PROCESS - planning process indicator 4. PROCESS - delivery process indicator 5. OUTCOME indicator We have 42 pressure relieving mattresses per 100 beds
- 20. www.england.nhs.uk 4th rule of #statisticsclub We don’t do structural measurement nearly often enough 20 30% 9% 26% 35% on all wards on most wards on one or some wards not on any wards “This [MH unit for older people] has no physio input. Balance and strength assessments never get done” “We cannot put walking frames within reach as there is no room left once you have a chair beside the bed” Royal College of Physicians 2012 Report of the 2011 inpatient falls pilot audit www.rcplondon.ac.uk Weekend access to mobility aids for new patients
- 21. www.england.nhs.uk Purposes of safety measurement 21 One often used model: measurement for research measurement for judgement measurement for improvement Alternative less prone to misunderstanding measurement to understand priorities measurement to see how we compare to others measurement to see if we’re getting better (or worse)
- 22. www.england.nhs.uk 0% 2% 4% 6% 8% 10% 12% 14% 16% 18% 0-4 5-9 10-14 15-19 20-24 25-29 30-34 35-39 40-44 45-49 50-54 55-59 60-64 65-69 70-74 75-79 80-84 85-89 90-94 95-99 100+ % of all reported acute falls Age group Breakdown by age of falls in acute clusters 0% 1% 2% 3% 4% 5% 6% 00 (12 AM - Midnight) 01 (1 AM)02 (2 AM)03 (3 AM)04 (4 AM)05 (5 AM)06 (6 AM)07 (7 AM)08 (8 AM)09 (9 AM)10 (10 AM)11 (11 AM)12 (12 PM - Midday) 13 (1 PM)14 (2 PM)15 (3 PM)16 (4 PM)17 (5 PM)18 (6 PM)19 (7 PM)20 (8 PM)21 (9 PM)22 (10 PM)23 (11 PM) % of all reported acute falls Hour Falls incidents by hour of occurrence, for acute clusters Understanding priority areas 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Acute hospitals Community hospitals Mental health units Location of incident Percentofsample Apparently unwitnessed by staff Witnessed by staff 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Acute hospitals Community hospitals Mental health units Location of incident Percentofsample Apparently unwitnessed by staff Witnessed by staff 0% 5% 10% 15% 20% 25% 30% 35% Acute hospitals Community hospitals Mental health units 0% 5% 10% 15% 20% 25% 30% 35% Acute hospitals Community hospitals Mental health units
- 23. www.england.nhs.uk 5th rule of #statisticsclub Measurement to see how we compare to others: when it comes to comparing outcomes, case mix really matters 23 (but case mix shouldn't be a problem for well-designed process measures)
- 24. www.england.nhs.uk Safety outcomes & case mix 0% 5% 10% 15% 20% 25% 0-4 5-9 10-14 15-19 20-24 25-29 30-34 35-39 40-44 45-49 50-54 55-59 60-64 65-69 70-74 75-79 80-84 85-89 90-94 95-99 100+ Age of patient (years) 0% 5% 10% 15% 20% 25% 0-4 5-9 10-14 15-19 20-24 25-29 30-34 35-39 40-44 45-49 50-54 55-59 60-64 65-69 70-74 75-79 80-84 85-89 90-94 95-99 100+ Age of patient (years) 85 years + Deandra S et al. Arch Gerontol Geriatr 56 (2013) 407–415 NPSA Slips trips and falls in hospital data update NPSA 2010 Risk factors for hospital falls Odds Ratio History of falls 2.85 (1.14–7.15) Cognitive impairment 1.52 (1.18–1.94)
- 25. www.england.nhs.uk Older people are not evenly distributed 25 Therefore unadjusted higher/lower rates of falls compared to other trusts with very different age profiles are highly unlikely to be useful indicators of relative safety All ages falls rate 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 R ETIR EM ENT TO W N B R ETIR EM ENT TO W N D R ETIR EM ENT TO W N A R ETIR EM ENT TO W N C U RBAN TEACH IN G B U RBAN TEACH IN G C U RBAN TEACH IN G D U RBAN TEACH IN G A fallsper1,000beddays
- 26. www.england.nhs.uk 50-fold differences between wards 26 Royal College of Physicians 2011 The FallSafe Quality Improvement project: report for the Health Foundation Therefore higher/lower rates of falls compared to other wards in the same trust highly unlikely to be useful local indicators of safety
- 27. www.england.nhs.uk Comparable and non-comparable safety indicators
- 28. www.england.nhs.uk 28 http://britishgeriatricssociety.wordpress.com/2013/12/19/fallsafe-are-culture- clashes-good-for-us/ 0 5000 10000 15000 20000 25000 6th rule of #statisticsclub Numbers don’t know if they are in research study or a QI project
- 29. www.england.nhs.uk 7th rule of #statisticsclub When it comes to measuring if we have got better (or worse) size matters (and we usually have plentiful processes to measure but far fewer outcomes)
- 31. www.england.nhs.uk Sample safety outcome indicators: scaled to ward-level* IF these safety outcomes were distributed evenly across acute wards, an average ward would have around: • One case of c difficile per year • One MRSA bloodstream infection per decade • One new pressure ulcer per quarter • One fall with minor injury per month • One fall with hip fracture every five years * Approximations based on c. 5,000 acute/rehabilitation hospital wards in England, PHE trust attributed/trust-assigned HCAI data, NRLS reported falls, assumption that acute ‘new’ p ulcer prevalence as measured by ST represents about 4 x acute p ulcer incidence
- 32. www.england.nhs.uk What scale and time would give you a reasonable chance of being able to distinguish a 25% improvement from natural variation ? 32 For falls with injury 1. One ward two years 2. Ten wards two years 3. One medium sized hospital two years 4. Five hospitals two years 5. Fifty hospitals two years
- 33. www.england.nhs.uk What scale and time would give you a reasonable chance of being able to distinguish a 25% improvement from natural variation ? 33 For hospital-associated MRSA? 1. One ward two years 2. Ten wards two years 3. One medium sized hospital two years 4. Five hospitals two years 5. Fifty hospitals two years
- 34. www.england.nhs.uk 34 Collaboration is not just good for learning, it increases sample size
- 35. www.england.nhs.uk 8th rule of #statisticsclub Your data don’t have to be perfect to be good enough – but you do need to know how imperfect they are 35
- 36. www.england.nhs.uk 0 2 4 6 8 10 12 14 16 60% certain last fall was reported 77% certain last fall was reported 0 2 4 6 8 10 12 14 16 0 2 4 6 8 10 12 14 16
- 37. www.england.nhs.uk 9th rule of #statisticsclub If it looks too good to be true, it probably is! 37 0 10 20 30 40 50 60 70 80 Jan Mar May Aug Nov Oct Aug Oct Dec Feb Apr Jun Aug Oct Dec 2008 2009 2010 2011
- 38. www.england.nhs.uk 10th rule of #statisticsclub Don’t try to measure too much 38
- 42. www.england.nhs.uk 85% 15% time spent improving time spent measuring
- 43. www.england.nhs.uk Thank you for listening frances.healey@nhs.net @FrancesHealey Please tweet further #statisticsclub rules of your own