Development and Validation of a NSTI Mortality Risk Calculator Using NSQIP
•
2 j'aime•1,201 vues
Our team's presentation from the 2012 WSA meeting
Recommandé
Contenu connexe
Similaire à Development and Validation of a NSTI Mortality Risk Calculator Using NSQIP
Similaire à Development and Validation of a NSTI Mortality Risk Calculator Using NSQIP (20)
Plus de Amalia Cochran
Plus de Amalia Cochran (17)
Dernier
Dernier (20)
Development and Validation of a NSTI Mortality Risk Calculator Using NSQIP
- 1. Development and Validation of a Necrotizing Soft-Tissue Infection Mortality Risk Calculator Using NSQIP Iris Faraklas, RN, BSN, Gregory J. Stoddard, MPH, Leigh Neumayer MD, FACS, Jeffrey Saffle, MD, FACS, Amalia Cochran, MD, FACS University of Utah
- 2. Disclosure Information • Nothing to disclose
- 3. Necrotizing Soft Tissue Infections (NSTI) • Aggressive infections requiring prompt surgical debridement and systemic support – May be mono- or polymicrobial • Incidence in US: 500-1,500 cases per year • Previous work from our group showed a mortality rate of 12%
- 4. Identifying Risk Factors • Focus on correctly identifying NSTI • Anaya et al (2009) created a mortality risk calculator – 2 centers – 350 patients in 9 years
- 5. National Surgical Quality Improvement Program (NSQIP) • Created by the VA – Managed by ACS • Prospective, multicenter database >500 hospitals • Patient data – Pre-, intra-, and postoperative variables – 30-day postoperative mortality and morbidity outcomes
- 6. Receiver Operating Characteristic (ROC) • Risk estimate to 1 discriminate cases from non-cases .8 – Non-survivors from Sensitivity .6 survivors • ROC=1.0 if .4 discrimination is perfect .2 • ROC=0.50 if 0.50 1.0 ROC area = 0.85 discrimination is no 0 0 .2 .4 .6 .8 1 better than chance 1 - Specificity Low…..False Positives….High
- 7. Rule of Thumb 1 .8 .6 .4 .2 ROC area = 0.80 0.70 0.85 0 0 .2 .4 .6 .8 1 1 - Specificity
- 8. Objective To develop and validate a 30-day postoperative mortality risk calculator for NSTI patients using NSQIP
- 9. Methods: Dataset • IRB approval • 2005-2010 NSQIP Participant Use Data Files- HIPAA Compliant • Diagnosis (using ICD-9-CM) – Necrotizing fasciitis (728.86) – Fournier’s gangrene (608.83) – Gas gangrene (040.0)
- 10. Methods: Dataset • Pre- and intraoperative variables included: – Demographics & lifestyle variables – Comorbidities & previous surgeries – ASA classification – Presence of septic shock per NSQIP criteria • Laboratory variables within 2 days preoperative – Missing laboratory values were excluded
- 11. Methods: Dataset • Primary outcome variable: Mortality • NSQIP tracks outcomes for 30 days postoperatively
- 12. Methods: Statistical Analysis • Univariate exploratory analysis on each variable compared to mortality • All variables showing significance (p<0.05) were included in stepwise multiple logistic regression • Preoperative laboratory values >400 observations were included: – Hct, BUN, Crt, Plt • The bootstrap method was used to validate the model
- 13. Results • 1,392 patients identified – 82% (1,142) Necrotizing fasciitis – 8.6% (119) Gas gangrene – 9.4% (131) Fournier’s gangrene Number of NSTI cases vs. Total NSQIP Cases per Year Year NSTI Cases Total NSQIP Cases 2005 47 34,099 2006 140 118,391 2007 212 211,407 2008 281 271,368 2009 352 336,190 2010 360 363,431
- 14. Results: Demographics n=1,392 • Median Age: 55 (IQR: 46-63) • 58% Male • Median BMI: 32 (IQR: 26-40) • 51% Were either partially or totally dependent • 49% Diabetic • 54% Hypertension requiring medication
- 15. Results: Demographics n=1,392 • 71% Admitted from home • 43% ASA Class ≥4 • 25% Septic shock • 31% OR in past 30 days • 62% Surgery considered emergent
- 16. Results: Outcomes n=1,392 • Median length of stay: – 16 days (IQR: 9-30) – 9 patients locked (>120 days) • 30-day mortality: 13% (n=181)
- 17. Multivariable Logistic Regression Model for 30-Day Postoperative Mortality (n=1,329) Variables Odds 95% Confidence P value Ratio Interval >60 years old 2.47 1.72-3.55 <0.001 Dependence Level Partial Dependent 1.61 0.95-2.69 0.072 Completely Dependent 2.33 1.43-3.80 0.001 Dialysis Prior to OR 1.89 1.15-3.10 0.012 ASA Class ≥ 4 3.55 2.25-5.59 <0.001 Emergent OR 1.56 1.03-2.34 0.035 Preoperative Septic Shock 2.35 1.55-3.56 <0.001 Platelet Count Platelet count 3.48 1.65-7.37 0.001 <50,000/mm3 Platelet count 1.86 1.21-2.87 0.005 <150,000/mm3 but >50,000/mm3
- 18. Results: Outcomes • ROC of 0.85 (CI:0.82- 1 0.87) .8 – Strong predictive model • Bootstrap validation Sensitivity .6 showed a ROC of 0.83 .4 (CI:0.81-0.86) – Represents the model .2 in future patients ROC area = 0.85 0 0 .2 .4 .6 .8 1 1 - Specificity
- 20. Limitations • Relatively small number of patients • Lack of microbial picture and surgical management • Variability of NSTI diagnosis • Selection bias for tertiary facilities
- 21. Conclusions • Strong predictive model • Variables are available early in hospital course • Risk models should not dictate management • Helpful communication tool
- 22. Thank you
Notes de l'éditeur
- Necrotizing soft-tissue infections (NSTI) are a group of uncommon, aggressive infections, requiring prompt surgical debridement and systemic support ; these infections may be monomicrobial or polymicrobial. Varying studies have reported the incidence in the US ranges from 500-1,500 cases per year. A previous large patient series from our group showed a mortality rate of 12%.
- In the past decade investigators have focused on correctly identifying NSTI cases. Some studies have tried to delineate specific risk mortality factors . To date Anaya (An I yah) et al created the only mortality risk scoring system. Shown in this table are their 6 variables. It should be noted that this scoring system was created from only 2 centers with a total of 350 patients over 9 years.
- Large national databases with validated data obtained by trained collectors have emerged as valuable sources of high-quality data, creating a unique opportunity to evaluate mortality risk factors for NSTI. The National Surgical Quality Improvement Program (NSQIP) is perhaps the best of these databases. The VA had the foresight to create NSQIP- currently it is managed by the American College of Surgeons. nsqip is a prospective, multicenter database with greater than 500 hospitals participating. Over a 130 data points are collected, including pre, intra and post operative variables, 30-day mortality and morbidity outcomes are also collected for patients undergoing surgical procedures. With this large program we were able to develop and validate a mortality risk calculator for NSTI patients.
- Before diving into our study…a little statistical back story A Receiver Operating Characteristic (or the ROC) is a plot of the true positive against the false positive rate at different points in a diagnostic test. It is the ability of the risk estimate to discriminate cases from noncases (or in our study: nonsurvivors from survivors).15, 16 The closer the curve follows the left-hand border and then the top border, the more accurate the test. The ROC area equals 1 if discrimination is perfect <<click>>and .5 if it is no better than chance.
- Among statisticians a rule of thumb is that a ROC of >.7 shows acceptable discrimination in clinical studies , while a ROC >.8 is considered excellent discrimination.
- The objective of this study was to develop and validate a 30-day postoperative mortality risk calculator for NSTI patients using the NSQIP database
- After receiving IRB approval, data were extracted from 2005 through 2010. The Participant Use Data Files do not identify hospitals, health care providers or patients. The files are HIPAA compliant. Patients discharged with an ICD-9 diagnosis of necrotizing fasciitis; Fournier’s gangrene, or gas gangrene were included in the analysis.
- Due to our time constraint The full definition of all nsqip variables are delineated elsewhere. 23 Some of the pre- and intraoperative variables included were: Demographic and lifestyle variables, comorbidities and previous surgeries Other factors considered were the patient’s ASA classification and presence of septic shock. If available within 2 days prior to surgery, preoperative laboratory variables were reported as number of abnormal per reported. Missing laboratory variables were not assumed to be normal and missing values were excluded from regression.
- The primary outcome variable was surgical mortality. Outcomes were tracked for 30 days postoperatively. If the patient was still inpatient at 30 days they were tracked until discharge or until NSQIP locked the data file at 120 days after their first surgery.
- Univariate exploratory analysis was performed on each preoperative variable compared to mortality. All variables showing significance were included in the stepwise multiple logistic regression analysis. Only the following preoperative laboratory values had > 400 observations and were included in the regression: hematocrit, BUN, creatinine and platelet count. Bootstrap method was used to validate this model
- A total of 1,392 patients were identified from 2005 through 2010, 82% had a diagnosis of necrotizing fasciitis , 8.6% were diagnosed with gas gangrene and the remaining 9.4% were diagnosed with Fournier’s gangrene. As shown in this Table , the number of NSTI admissions reported to NSQIP increased progressively from 47 patients in 2005 to 360 in 2010. Please note that the total number NSQIP cases reported increased more than 10X over this same period as centers were added to the program
- This is a much abbreviated general demographics slide. Patients were mostly male in their 50s, and obese. More than half were considered partially or totally dependent, and required medication to treat their diabetes and or hypertension.
- The majority were admitted from home. Over 40% were ASA classified as a 4 or greater; a quarter were in septic shock, while a third had had previous surgery in the past month and the majority of the surgeries were considered emergent in nature meaning they were within 12 hours of admission
- Median length of stay was 16 days; however there were 9 patients that were still in the hospital at time that NSQIP locked their data file (>120 days from first surgery). And the Thirty-day mortality was 13%.
- Multivariable logistic regression identified the following 7 independent variables that affected mortality: patients older than 60, totally dependent functional status, requiring dialysis prior to surgery, having a ASA class 4 or greater, considered as an emergent surgery (meaning <12 hours post admission), septic shock and low platelet count stratified between those <50,000 and those (<150,000/mm3 but more than 50,000
- The ROC for our model was .85 which indicates a strong predictive model. Using boot-strap validation the ROC curve area was .83, which represents how the model will behave in future patients.
- The model was used to develop an interactive risk calculator. Once the model was validated, an interactive spreadsheet was created that uses the demonstrated risk factors from a specific patient to return a probability of mortality expressed as a percent. This calculator is available upon request. It calculates a patient’s estimated risk of mortality, after clinicians enter patient data based on the variable definitions that are listed on spreadsheet. Definitions are the same as NSQIP definitions and if the user hovers over the variable the specific definition pops up.
- As with any study, limitations exist: due to the rarity of disease the number of patients included in this analysis is relatively small. Although NSQIP is probably one of the best databases available- it doesn’t include the microbial picture or which patients required serial debridements. Nor are we able to control for diagnosis variablity that might occur at different facilities which is ultimately only as good as the ICD-9 coding. Also tertiary facilities might participate in NSQIP more frequently thus a selection bias might exist.
- As stated earlier an ROC greater than .8 is considered an excellent predictive model this calculator is based on a strong predictive model (with an ROC of .85). The variables included in the model are available early in the hospital course. However risk models should not dictate management but are just one added tool in the clinican’s toolbox. This simply provides extra information or a helpful communication tool to help patients and their families make informed decisions.