Comparative prognostic accuracy of sepsis scores for hospital mortality in adults with suspected infection in non-ICU and ICU at an academic public hospital
Christopher P. Kovach aff001; Grant S. Fletcher aff002; Kristina E. Rudd aff001; Rosemary M. Grant aff004; David J. Carlbom aff001
Authors place of work:
Division of Pulmonary, Critical Care, and Sleep Medicine, Department of Medicine, University of Washington, Seattle, Washington, United States of America
aff001; Division of Hospital Medicine, Department of Medicine, University of Washington, Seattle, Washington, United States of America
aff002; Clinical Research, Investigation, and Systems Modeling of Acute Illness (CRISMA) Center, Department of Critical Care Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania, United States of America
aff003; Professional Development and Nursing Excellence, Harborview Medical Center, Seattle, Washington, United States of America
Published in the journal:
PLoS ONE 14(9)
Sepsis is a global healthcare challenge and reliable tools are needed to identify patients and stratify their risk. Here we compare the prognostic accuracy of the sepsis-related organ failure assessment (SOFA), quick SOFA (qSOFA), systemic inflammatory response syndrome (SIRS), and national early warning system (NEWS) scores for hospital mortality and other outcomes amongst patients with suspected infection at an academic public hospital.
Measurements and main results
10,981 adult patients with suspected infection hospitalized at a U.S. academic public hospital between 2011–2017 were retrospectively identified. Primary exposures were the maximum SIRS, qSOFA, SOFA, and NEWS scores upon inclusion. Comparative prognostic accuracy for the primary outcome of hospital mortality was assessed using the area under the receiver operating characteristic curve (AUROC). Secondary outcomes included mortality in ICU versus non-ICU settings, ICU transfer, ICU length of stay (LOS) >3 days, and hospital LOS >7 days. Adjusted analyses were performed using a model of baseline risk for hospital mortality. 774 patients (7.1%) died in hospital. Discrimination for hospital mortality was highest for SOFA (AUROC 0.90 [95% CI, 0.89–0.91]), followed by NEWS (AUROC 0.85 [95% CI, 0.84–0.86]), qSOFA (AUROC 0.84 [95% CI, 0.83–0.85]), and SIRS (AUROC 0.79 [95% CI, 0.78–0.81]; p<0.001 for all comparisons). NEWS (AUROC 0.94 [95% CI, 0.93–0.95]) outperformed other scores in predicting ICU transfer (qSOFA AUROC 0.89 [95% CI, 0.87–0.91]; SOFA AUROC, 0.84 [95% CI, 0.82–0.87]; SIRS AUROC 0.81 [95% CI, 0.79–0.83]; p<0.001 for all comparisons). NEWS (AUROC 0.86 [95% CI, 0.85–0.86]) was also superior to other scores in predicting ICU LOS >3 days (SOFA AUROC 0.84 [95% CI, 0.83–0.85; qSOFA AUROC, 0.83 [95% CI, 0.83–0.84]; SIRS AUROC, 0.75 [95% CI, 0.74–0.76]; p<0.002 for all comparisons).
Multivariate prediction scores, such as SOFA and NEWS, had greater prognostic accuracy than qSOFA or SIRS for hospital mortality, ICU transfer, and ICU length of stay. Complex sepsis scores may offer enhanced prognostic performance as compared to simple sepsis scores in inpatient hospital settings where more complex scores can be readily calculated.
Medicine and health sciences – Diagnostic medicine – Signs and symptoms – Sepsis – Systemic inflammatory response syndrome – Pathology and laboratory medicine – Health care – Health care facilities – Hospitals – Intensive care units – Patients – Inpatients – Critical care and emergency medicine – Biology and life sciences – Population biology – Population metrics – Death rates – Physical sciences – Chemistry – Chemical elements – Oxygen
Sepsis is a major healthcare challenge in the United States and globally, and is associated with profound mortality, morbidity, and healthcare costs [1–7]. Early recognition and treatment of sepsis improves outcomes; reliable tools are needed to identify patients at increased risk of developing sepsis and to prognosticate their mortality and other complications [8, 9]. Sepsis-3 authors recommend the quick sepsis-related organ failure assessment (qSOFA) score to identify patients at high risk of developing sepsis outside the intensive care unit (ICU) and the SOFA score for patients in the ICU [10, 11]. Internal and external validation studies have demonstrated the superiority of the qSOFA and SOFA scores for the identification and mortality prognostication of sepsis patients, when compared to the systemic inflammatory response syndrome (SIRS) criteria [10, 12–15]. While the Sepsis-3 authors initially proposed the qSOFA and SOFA scores as tools to identify patients with organ dysfunction among those with suspected infection, there is widespread interest in using these and other scores in prognosticating patient outcomes secondary to sepsis [16–21]. A recent meta-analysis comparing the qSOFA score with SIRS criteria concluded that the qSOFA score was more predictive of hospital mortality but SIRS was superior for sepsis diagnosis . However, other studies have shown that alternative scores, such as the national early warning score (NEWS) may be superior [20, 23–26]. The ideal sepsis identification and outcome prognostication scoring system remains uncertain. We compared the prognostic accuracy of sepsis scores for hospital mortality among patients with suspected infection presenting to the emergency department (ED) and then admitted to either the acute care service or ICU of an academic public hospital. We hypothesized that there were important differences between scores which may impact score performance among different hospitalized populations.
Study design and population
A retrospective cohort study was performed using all patients ≥18 years of age with suspected infection who presented to the ED and were admitted to Harborview Medical Center, a tertiary academic public hospital in Seattle, WA with 413 beds, between January 2011 and March 2017. Patients were identified on the basis of suspected infection given that there is no gold standard for the diagnosis of sepsis . Suspected infection was defined as (1) any blood, urine, or sputum culture order followed by clinician order of an intravenous (IV) antibiotic within 72 hours, or (2) clinician order of an IV antibiotic followed by a culture order within 24 hours. This method was chosen for consistency with recent major sepsis studies [7, 10, 14, 15, 23]. All patients in the ED, acute care service, or ICU were eligible for inclusion in the study. The time at which a patient met the definition of suspected infection was used as the time of study inclusion. Primary exposures were the maximum SIRS, qSOFA, SOFA, and NEWS scores upon inclusion. Patients who were directly admitted to the hospital without being evaluated in the ED, those transferred from the ED or inpatient wards of another hospital, those who were evaluated in the ED and then discharged, and those admitted to inpatient psychiatric or rehabilitation services were excluded. The study was approved by the University of Washington Institutional Review Board (IRB #00002870).
Patient demographic data, vital signs, laboratory values, orders (e.g., medications, cultures, oxygen therapy, vasopressors), hospital mortality data, and ICU and hospital length of stay (LOS) were extracted from the electronic health record, de-identified, and made available on a secure server for analysis. Each patient’s qSOFA, SOFA, SIRS, and NEWS scores were calculated at time of inclusion in the study using the most deranged physiologic and laboratory parameters recorded within the 24 hours preceding and the 24 hours following time of inclusion [11, 12, 25, 27]. Standard criteria for score positivity were applied, using a threshold of 2 or more points for each of SIRS, qSOFA, and SOFA, and a threshold of 5 or more points for NEWS. Glasgow coma scale (GCS) ≤14 was used to define altered mental status [10, 28]. No contribution was made to the total score if an individual component of the score was missing. Patients in whom all components of any score were missing were excluded from analysis.
The primary outcome was prognostic accuracy of individual scores for hospital mortality. We did not compare the ability of scores to identify patients with sepsis because of the limitations of the retrospective study design and the lack of a gold standard for comparison. Secondary outcomes included hospital mortality stratified by non-ICU and ICU setting at time of inclusion, transfer to the ICU from a non-ICU setting, ICU LOS >3 days, and overall hospital LOS >7 days following study inclusion. ICU transfer was further defined as patient transfer from a non-ICU to ICU setting within the 24 hours preceding and the 24 hours following time of study inclusion with subsequent ICU duration of at least 24 hours or death within 24 hours of transfer. Secondary outcomes were chosen to reflect clinical events significant to both individual patients and more broadly to hospitals and health systems.
All analyses were performed using Stata version 15 (StataCorp, College Station, TX). Patient characteristics are presented as number (%), mean ± standard deviation (SD) for quantification of normally distributed variables, or median and interquartile range (IQR) for non-normally distributed variables. For comparison of continuous variables, Student’s t-test was used. For comparison of dichotomous variables, chi-square test was applied. Comparative prognostic accuracy for the primary and secondary outcomes was assessed using the area under the receiver operating characteristic curve (AUROC) for each score individually (crude analysis) and in conjunction with a baseline risk model (adjusted analysis) to demonstrate the additional prognostic value of sepsis scores beyond potentially confounding demographic factors. Age, sex, and race were used to calculate a baseline level of risk for mortality and other outcomes based on sepsis data from the United States demonstrating disparities according to these factors [29–31]. Adjusted risk ratios for outcomes comparing positive vs. negative scores (e.g., SOFA >2 vs. SOFA <2) were assessed. A 2-sided p-value of <0.01 was used to indicate statistical significance and ensure a robust analysis based on the Bonferroni correction for multiple comparisons .
There were 125,431 patient encounters during the study period, of which 10,981 met study inclusion criteria. Thirty-nine patients had incomplete records precluding the calculation of at least one of the scores and were omitted from analysis; 10,942 patients were included in the study (S1 Fig). Median age was 52 years (IQR, 29–75 years), 70% were male (n = 7,645), and 30% were female (n = 3,297) (Table 1). 61% were Caucasian (n = 6,710), 18% African-American (n = 1,922), and 21% other racial/ethnic groups (n = 2,349). 7,193 patients (66%) were in a non-ICU setting at the time they met inclusion criteria, compared to 3,749 (34%) in the ICU. 774 patients (7.1%) died in the hospital; of these, 116 (15%) deaths occurred in a non-ICU setting and 658 (85%) occurred in the ICU. 313 patients were transferred from a non-ICU setting to the ICU. Of those patients who were admitted to the ICU at any point during their hospitalization, 1,817 patients (48%) had an ICU LOS > 3 days. Hospital LOS >7 days occurred in 3,428 patients (31%).
Within the study cohort, 8,534 (78%) had a SIRS score ≥2; 4,864 (44%) had a qSOFA score ≥2; 9,746 (89%) had a NEWS score ≥5; and 6,219 (57%) had a SOFA score of ≥2. The full distributions of scores and their relationship with hospital mortality are presented in Figs 1 and 2. The incidence of missing score components in the study cohort was low (S1 Table). However, the PaO2/FiO2 ratio and serum bilirubin level were missing for 77% (n = 8,489) and 36% (n = 3,927) of patients, respectively.
After adjusting for baseline risk factors for death, discrimination of hospital mortality was significantly higher for SOFA (AUROC 0.90 [95% CI, 0.89–0.91]) than NEWS (AUROC 0.85 [95% CI, 0.84–0.86], p<0.001), qSOFA (AUROC 0.84 [95% CI, 0.83–0.85], p<0.001), or SIRS (AUROC 0.79 [95% CI, 0.78–0.81], p<0.001) (Table 2, S2 Fig). Similarly, SOFA (AUROC 0.82 [95% CI, 0.80–0.83]) outperformed all other scores in discrimination of hospital mortality in an ICU setting (NEWS AUROC 0.70 [95% CI, 0.68–0.72]; qSOFA AUROC 0.68 [95% CI, 0.66–0.70]; SIRS AUROC 0.68 [95% CI, 0.66–0.70]; all p<0.001) (S3 Fig). For hospital mortality in a non-ICU setting SOFA (AUROC 0.86 [95% CI, 0.83–0.89]) and NEWS (AUROC 0.84 [95% CI, 0.81–0.88]) had similar prognostic accuracy (p = 0.38) and were no different from qSOFA (AUROC 0.84 [95% CI, 0.80–0.87]; qSOFA vs. NEWS p = 0.26; qSOFA vs. SOFA p = 0.129) but were more predictive than SIRS (AUROC 0.81 [95% CI, 0.78–0.84]; SIRS vs. NEWS p = 0.012; SIRS vs. SOFA p = 0.004) (S4 Fig).
NEWS (AUROC 0.94 [95% CI, 0.93–0.95]) outperformed qSOFA (AUROC 0.89 [95% CI, 0.87–0.91], p<0.001), SOFA (AUROC 0.84, [95% CI, 0.82–0.87], p<0.001), and SIRS (AUROC 0.81 [95% CI, 0.79–0.83], p<0.001) in predicting transfer from a non-ICU to an ICU setting (Table 3; S5 Fig). Similarly, NEWS (AUROC 0.86 [95% CI, 0.85–0.86]) was superior to SOFA (AUROC 0.84 [95% CI, 0.83–0.85], p = 0.002), qSOFA (AUROC 0.83 [95% CI, 0.83–0.84], p<0.001), and SIRS (AUROC 0.75 [95% CI, 0.74–0.76], p<0.001) in predicting ICU LOS >3 days (S6 Fig). For hospital LOS >7 days qSOFA (AUROC 0.64 [95% CI, 0.63–0.65]), NEWS (AUROC 0.65 [95% CI, 0.64–0.66]), and SOFA (AUROC 0.64 [95% CI, 0.63–0.65]) were all more predictive than SIRS (SIRS vs. qSOFA p = 0.002; SIRS vs. NEWS p<0.001; SIRS vs. SOFA p = 0.01) but were not statistically different from each other (S7 Fig).
Analyses using crude data without adjustment for baseline risk of mortality are reported in the supplement and resulted in similar estimates of AUROC for outcomes by each scoring system (S2 Table, S3 Table, S8–S13 Figs).
Assessment of adjusted risk ratios for outcomes comparing positive vs. negative scores in a binary fashion (e.g., qSOFA ≥2 vs. qSOFA <2) are also reported in the supplement (S14 Fig). Relative risk of hospital mortality (RR 47.0 [95% CI, 11.7–188.8]), non-ICU mortality (RR 10.0 [95% CI, 2.5–40.6]), ICU transfer (RR 51.0 [95% CI, 31.1–83.5]), and ICU LOS >3 days (RR 67.4 [95% CI, 25.2–180.1]) were highest with NEWS ≥5. SOFA ≥2 (RR 10.9 [95% CI, 5.4–22.2]) had the highest relative risk of ICU mortality. There was no appreciable difference in the relative risk of hospital LOS >7 days between scores.
In this study comparing the prognostic accuracy of SOFA, qSOFA, SIRS, and NEWS scores for clinically relevant outcomes in a large population of adult patients with suspected infection at an academic public hospital, more detailed multivariate prediction scores outperformed simpler scores. SOFA, a complex score combining physiologic and laboratory data, demonstrated superior prognostic accuracy for overall hospital mortality and ICU mortality compared to all other scores. NEWS, which utilizes physiologic data only, outperformed all other scores in predicting transfer to the ICU and for ICU LOS >3 days. There was no significant difference between qSOFA and SIRS in predicting the primary outcome of hospital mortality. However, qSOFA was superior to SIRS in predicting transfer to the ICU, ICU LOS >3 days, and hospital LOS >7 days in the study cohort.
This study’s finding that SIRS is a poor predictor of mortality is consistent with prior studies [33, 34]. In the sepsis consensus definition paper, the ability of SOFA, qSOFA, and SIRS to predict hospital mortality was determined in a mixed cohort of ICU and non-ICU encounters . Seymour and colleagues found that 1) the predictive validity of qSOFA for hospital mortality was statistically greater than SOFA or SIRS for non-ICU encounters, and 2) the predictive validity of SOFA for hospital mortality was superior to qSOFA or SIRS for ICU encounters . Freund et al. conducted an international prospective cohort study that showed the superiority of qSOFA in prognosticating hospital mortality as compared to SIRS in patients presenting to the emergency department with suspected infection . Raith et al. performed a retrospective cohort analysis on a large cohort of patients in Australian and New Zealand ICUs and found that SOFA had greater prognostic accuracy for hospital mortality compared to SIRS or qSOFA . Moreover, qSOFA was found to have superior discrimination of mortality as compared to SIRS in adult patients with suspected infection hospitalized in low- and middle-income countries . A recent meta-analysis of 38 studies comparing the prognostic accuracy of qSOFA and SIRS for hospital mortality among patients with suspected infection reported that qSOFA was more predictive of mortality but SIRS was superior for sepsis diagnosis . However, these findings are inconclusive given that the included studies varied significantly in the studied patient population (e.g., ED vs. acute care vs. ICU), outcome measures (hospital mortality vs. 28-day mortality), and on the definition of “suspected infection,” with only 10 of the 38 studies using a standardized approach incorporating antibiotic treatment or initiation of body fluid cultures.
This study found that SOFA outperformed qSOFA, SIRS, and NEWS in predicting overall hospital mortality and ICU mortality. Nonetheless, the observed event rate of non-ICU fatalities was low in this cohort. Given that the majority of deaths occurred in the ICU, we found few significant differences between scores in prognosticating non-ICU mortality. This study further confirms the superiority of SOFA to discriminate ICU mortality, as previously reported by others [14, 16]. In this cohort qSOFA was superior to SIRS in predicting overall hospital mortality, transfer to the ICU, ICU LOS >3 days, and hospital LOS >7 days. This finding is consistent with the meta-analysis by Fernando et al. in which qSOFA was found to be superior to SIRS in predicting hospital mortality . However, we found that NEWS had better discriminative value than qSOFA for hospital mortality and was superior to all other scores in predicting ICU transfer and ICU LOS >3 days. This finding strongly supports mounting evidence that qSOFA should not replace general early-warning scores in risk-stratifying patients with suspected infection in high-resource hospitalized settings [20, 23, 24, 26]. However, qSOFA may have utility in environments where calculation of complex scores is a challenge, such as outpatient clinics, emergency departments, or low-resource settings.
This study has several strengths. The analysis is based on a large dataset encompassing patients in non-ICU and ICU settings and includes both medical and surgical patient cohorts. The dataset was designed to address the study question and was further strengthened by the low incidence of missing data elements. The data have excellent external validity as evidenced by a hospital mortality rate of 1.7% in the non-ICU cohort and 16.4% in the ICU cohort that is highly consistent with other published reports [10, 14]. This work benefitted from its use of a reproducible identification schema for suspected infection and use of similar methodology to the consensus paper and other major studies [7, 10, 14, 15, 23]. Hospital mortality was the primary outcome, but the study also measured the ability of scores to prognosticate other clinically relevant outcomes pertinent to individual patients, hospitals, and healthcare systems that have not been previously reported in other large studies of this kind.
Data were retrospectively collected for this analysis and were limited to a single academic public hospital in the United States. The majority of patients in this cohort were male and/or Caucasian. While this is consistent with the hospital’s overall patient demographics and previously published studies from this center, the generalizability of the results to other hospitals and healthcare settings with different patient demographics is unclear . Calculation of scores was based upon the most deranged physiologic and biochemical score components within the 24 hours preceding and 24 hours following inclusion in the study, consistent with other studies in the field [7, 10, 14, 15, 23]. Thus, these data may bias towards higher scores. Additionally, missing data regarding the respiratory and hepatic components of the SOFA score may have limited its prognostic accuracy for mortality outcomes in this analysis. While adjusted analyses were performed to demonstrate the additive power of sepsis scores to predict outcomes beyond baseline risk, the variables used to generate this model were limited in scope due to lack of administrative data pertaining to relevant comorbid conditions on admission. This study reports on the performance of scores in ICU and non-ICU settings but does not compare score performance across the ED, acute care service, and ICU due to its retrospective nature and limitations of the available administrative data. Finally, the majority of deaths in this study cohort occurred in the ICU and thus the present work may be underpowered to determine which score performs best in non-ICU environments.
In this large single-center retrospective cohort study of adult medical and surgical inpatients with suspected infection, multivariate prediction scores such as SOFA and NEWS demonstrated superior prognostic accuracy for hospital mortality, ICU transfer, and ICU LOS as compared to qSOFA and SIRS. These findings suggest that complex scores may such as SOFA and NEWS may offer enhanced prognostic performance over simple sepsis scores such as qSOFA and SIRS in inpatient hospital settings where more complex scores can be readily calculated.
S2 Table [docx]
Crude AUROCs and comparisons for prediction of hospital mortality outcomes.
S3 Table [docx]
Crude AUROCs and comparisons for prediction of ICU transfer and length of stay outcomes.
S1 Fig [tiff]
Flow diagram of eligible patient population and exclusion criteria.
S2 Fig [tif]
Adjusted area under the receiver operating characteristic curves (AUROCs) of prognostic accuracy for in-hospital mortality for SIRS, qSOFA, NEWS, and SOFA scores.
S3 Fig [tif]
Adjusted area under the receiver operating characteristic curves (AUROCs) of prognostic accuracy for ICU mortality for SIRS, qSOFA, NEWS, and SOFA scores.
S4 Fig [tif]
Adjusted area under the receiver operating characteristic curves (AUROCs) of prognostic accuracy for non-ICU mortality for SIRS, qSOFA, NEWS, and SOFA scores.
S5 Fig [tif]
Adjusted area under the receiver operating characteristic curves (AUROCs) of prognostic accuracy for ICU transfer for SIRS, qSOFA, NEWS, and SOFA scores.
S6 Fig [tif]
Adjusted area under the receiver operating characteristic curves (AUROCs) of prognostic accuracy for ICU length of stay (LOS) >3 days for SIRS, qSOFA, NEWS, and SOFA scores.
S7 Fig [tif]
Adjusted area under the receiver operating characteristic curves (AUROCs) of prognostic accuracy for hospital length of stay (LOS) >7 days for SIRS, qSOFA, NEWS, and SOFA scores.
S8 Fig [tif]
Crude area under the receiver operating characteristic curves (AUROCs) of prognostic accuracy for hospital mortality for SIRS, qSOFA, NEWS, and SOFA scores.
S9 Fig [tif]
Crude area under the receiver operating characteristic curves (AUROCs) of prognostic accuracy for ICU mortality for SIRS, qSOFA, NEWS, and SOFA scores.
S10 Fig [tif]
Crude area under the receiver operating characteristic curves (AUROCs) of prognostic accuracy for non-ICU mortality for SIRS, qSOFA, NEWS, and SOFA scores.
S11 Fig [tif]
Crude area under the receiver operating characteristic curves (AUROCs) of prognostic accuracy for ICU transfer for SIRS, qSOFA, NEWS, and SOFA scores.
S12 Fig [tif]
Crude area under the receiver operating characteristic curves (AUROCs) of prognostic accuracy for ICU length of stay (LOS) >3 days for SIRS, qSOFA, NEWS, and SOFA scores.
S13 Fig [tif]
Crude area under the receiver operating characteristic curves (AUROCs) of prognostic accuracy for hospital length of stay (LOS) >7 days for SIRS, qSOFA, NEWS, and SOFA scores.
S14 Fig [tiff]
Adjusted risk ratios for outcomes comparing positive vs. negative SIRS, qSOFA, NEWS, and SOFA scores.
1. Paoli CJ, Reynolds MA, Sinha M, Gitlin M, Crouser E. Epidemiology and Costs of Sepsis in the United States-An Analysis Based on Timing of Diagnosis and Severity Level. Crit Care Med. 2018;46(12):1889–97. doi: 10.1097/CCM.0000000000003342 30048332
2. Liu V, Escobar GJ, Greene JD, Soule J, Whippy A, Angus DC, et al. Hospital deaths in patients with sepsis from 2 independent cohorts. Jama. 2014;312(1):90–2. doi: 10.1001/jama.2014.5804 24838355
3. Martin GS, Mannino DM, Eaton S, Moss M. The epidemiology of sepsis in the United States from 1979 through 2000. N Engl J Med. 2003;348(16):1546–54. doi: 10.1056/NEJMoa022139 12700374
4. Rudd KE, Delaney A, Finfer S. Counting Sepsis, an Imprecise but Improving Science. Jama. 2017;318(13):1228–9. doi: 10.1001/jama.2017.13697 28903164
5. Fleischmann C, Scherag A, Adhikari NK, Hartog CS, Tsaganos T, Schlattmann P, et al. Assessment of Global Incidence and Mortality of Hospital-treated Sepsis. Current Estimates and Limitations. Am J Respir Crit Care Med. 2016;193(3):259–72. doi: 10.1164/rccm.201504-0781OC 26414292
6. Hatfield KM, Dantes RB, Baggs J, Sapiano MRP, Fiore AE, Jernigan JA, et al. Assessing Variability in Hospital-Level Mortality Among U.S. Medicare Beneficiaries With Hospitalizations for Severe Sepsis and Septic Shock. Crit Care Med. 2018;46(11):1753–60. doi: 10.1097/CCM.0000000000003324 30024430
7. Rhee C, Dantes R, Epstein L, Murphy DJ, Seymour CW, Iwashyna TJ, et al. Incidence and Trends of Sepsis in US Hospitals Using Clinical vs Claims Data, 2009–2014. Jama. 2017;318(13):1241–9. doi: 10.1001/jama.2017.13836 28903154
8. Yealy DM, Huang DT, Delaney A, Knight M, Randolph AG, Daniels R, et al. Recognizing and managing sepsis: what needs to be done? BMC Med. 2015;13(98):015–0335.
9. Seymour CW, Gesten F, Prescott HC, Friedrich ME, Iwashyna TJ, Phillips GS, et al. Time to Treatment and Mortality during Mandated Emergency Care for Sepsis. N Engl J Med. 2017;376(23):2235–44. doi: 10.1056/NEJMoa1703058 28528569
10. Seymour CW, Liu VX, Iwashyna TJ, Brunkhorst FM, Rea TD, Scherag A, et al. Assessment of Clinical Criteria for Sepsis: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). Jama. 2016;315(8):762–74. doi: 10.1001/jama.2016.0288 26903335
11. Vincent JL, de Mendonca A, Cantraine F, Moreno R, Takala J, Suter PM, et al. Use of the SOFA score to assess the incidence of organ dysfunction/failure in intensive care units: results of a multicenter, prospective study. Working group on "sepsis-related problems" of the European Society of Intensive Care Medicine. Crit Care Med. 1998;26(11):1793–800. doi: 10.1097/00003246-199811000-00016 9824069
12. Bone RC, Balk RA, Cerra FB, Dellinger RP, Fein AM, Knaus WA, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest. 1992;101(6):1644–55. doi: 10.1378/chest.101.6.1644 1303622
13. Freund Y, Lemachatti N, Krastinova E, Van Laer M, Claessens YE, Avondo A, et al. Prognostic Accuracy of Sepsis-3 Criteria for In-Hospital Mortality Among Patients With Suspected Infection Presenting to the Emergency Department. Jama. 2017;317(3):301–8. doi: 10.1001/jama.2016.20329 28114554
14. Raith EP, Udy AA, Bailey M, McGloughlin S, MacIsaac C, Bellomo R, et al. Prognostic Accuracy of the SOFA Score, SIRS Criteria, and qSOFA Score for In-Hospital Mortality Among Adults With Suspected Infection Admitted to the Intensive Care Unit. Jama. 2017;317(3):290–300. doi: 10.1001/jama.2016.20328 28114553
15. Rudd KE, Seymour CW, Aluisio AR, Augustin ME, Bagenda DS, Beane A, et al. Association of the Quick Sequential (Sepsis-Related) Organ Failure Assessment (qSOFA) Score With Excess Hospital Mortality in Adults With Suspected Infection in Low- and Middle-Income Countries. Jama. 2018;319(21):2202–11. doi: 10.1001/jama.2018.6229 29800114
16. Minne L, Abu-Hanna A, de Jonge E. Evaluation of SOFA-based models for predicting mortality in the ICU: A systematic review. Crit Care. 2008;12(6):17.
17. Keegan MT, Gajic O, Afessa B. Severity of illness scoring systems in the intensive care unit. Crit Care Med. 2011;39(1):163–9. doi: 10.1097/CCM.0b013e3181f96f81 20838329
18. Kollef MH, Schuster DP. Predicting intensive care unit outcome with scoring systems. Underlying concepts and principles. Crit Care Clin. 1994;10(1):1–18. 8118722
19. Sbiti-Rohr D, Kutz A, Christ-Crain M, Thomann R, Zimmerli W, Hoess C, et al. The National Early Warning Score (NEWS) for outcome prediction in emergency department patients with community-acquired pneumonia: results from a 6-year prospective cohort study. BMJ Open. 2016;6(9):2015–011021.
20. Goulden R, Hoyle MC, Monis J, Railton D, Riley V, Martin P, et al. qSOFA, SIRS and NEWS for predicting inhospital mortality and ICU admission in emergency admissions treated as sepsis. Emerg Med J. 2018;35(6):345–9. doi: 10.1136/emermed-2017-207120 29467173
21. Churpek MM, Snyder A, Sokol S, Pettit NN, Edelson DP. Investigating the Impact of Different Suspicion of Infection Criteria on the Accuracy of Quick Sepsis-Related Organ Failure Assessment, Systemic Inflammatory Response Syndrome, and Early Warning Scores. Crit Care Med. 2017;45(11):1805–12. doi: 10.1097/CCM.0000000000002648 28737573
22. Fernando SM, Tran A, Taljaard M, Cheng W, Rochwerg B, Seely AJE, et al. Prognostic Accuracy of the Quick Sequential Organ Failure Assessment for Mortality in Patients With Suspected Infection: A Systematic Review and Meta-analysis. Ann Intern Med. 2018;168(4):266–75. doi: 10.7326/M17-2820 29404582
23. Churpek MM, Snyder A, Han X, Sokol S, Pettit N, Howell MD, et al. Quick Sepsis-related Organ Failure Assessment, Systemic Inflammatory Response Syndrome, and Early Warning Scores for Detecting Clinical Deterioration in Infected Patients outside the Intensive Care Unit. Am J Respir Crit Care Med. 2017;195(7):906–11. doi: 10.1164/rccm.201604-0854OC 27649072
24. Szakmany T, Pugh R, Kopczynska M, Lundin RM, Sharif B, Morgan P, et al. Defining sepsis on the wards: results of a multi-centre point-prevalence study comparing two sepsis definitions. Anaesthesia. 2018;73(2):195–204. doi: 10.1111/anae.14062 29150856
25. Prytherch DR, Smith GB, Schmidt PE, Featherstone PI. ViEWS—Towards a national early warning score for detecting adult inpatient deterioration. Resuscitation. 2010;81(8):932–7. doi: 10.1016/j.resuscitation.2010.04.014 20637974
26. Redfern OC, Smith GB, Prytherch DR, Meredith P, Inada-Kim M, Schmidt PE. A Comparison of the Quick Sequential (Sepsis-Related) Organ Failure Assessment Score and the National Early Warning Score in Non-ICU Patients With/Without Infection. Crit Care Med. 2018;46(12):1923–33. doi: 10.1097/CCM.0000000000003359 30130262
27. Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). Jama. 2016;315(8):801–10. doi: 10.1001/jama.2016.0287 26903338
28. Teasdale G, Jennett B. Assessment of coma and impaired consciousness. A practical scale. Lancet. 1974;2(7872):81–4. doi: 10.1016/s0140-6736(74)91639-0 4136544
29. Nasa P, Juneja D, Singh O, Dang R, Arora V. Severe sepsis and its impact on outcome in elderly and very elderly patients admitted in intensive care unit. J Intensive Care Med. 2012;27(3):179–83. doi: 10.1177/0885066610397116 21436163
30. Pietropaoli AP, Glance LG, Oakes D, Fisher SG. Gender differences in mortality in patients with severe sepsis or septic shock. Gend Med. 2010;7(5):422–37. doi: 10.1016/j.genm.2010.09.005 21056869
31. Chaudhary NS, Donnelly JP, Wang HE. Racial Differences in Sepsis Mortality at U.S. Academic Medical Center-Affiliated Hospitals. Crit Care Med. 2018;46(6):878–83. doi: 10.1097/CCM.0000000000003020 29438109
32. Bland JM, Altman DG. Multiple significance tests: the Bonferroni method. Bmj. 1995;310(6973):170. doi: 10.1136/bmj.310.6973.170 7833759
33. Kaukonen KM, Bailey M, Pilcher D, Cooper DJ, Bellomo R. Systemic inflammatory response syndrome criteria in defining severe sepsis. N Engl J Med. 2015;372(17):1629–38. doi: 10.1056/NEJMoa1415236 25776936
34. Liao MM, Lezotte D, Lowenstein SR, Howard K, Finley Z, Feng Z, et al. Sensitivity of systemic inflammatory response syndrome for critical illness among ED patients. Am J Emerg Med. 2014;32(11):1319–25. doi: 10.1016/j.ajem.2014.07.035 25205616
35. Rubenfeld GD, Caldwell E, Peabody E, Weaver J, Martin DP, Neff M, et al. Incidence and outcomes of acute lung injury. N Engl J Med. 2005;353(16):1685–93. doi: 10.1056/NEJMoa050333 16236739