Comparison of REMS, NEWS, qSOFA and SIRS criteria scales for sepsis prediction in patients with diagnosis “SARS-CoV-2, virus unidentified”: a retrospective observational study

Background. Despite ample research on the coronavirus infection sequence and therapy, the incidence of adverse outcomes remains very high. Sepsis stands among the major factors greatly complicating treatment and increasing the risk of death. A timely identification of highrisk sepsis patients is a cornerstone of effective sepsis prevention.
Objectives. A comparative prognostic power assessment between the quick Sequential Organ Failure Assessment (qSOFA) scale, National Early Warning Score (NEWS), Initial Prehospital Rapid Emergency Medicine Score (REMS) and the Systemic Inflammatory Response Syndrome (SIRS) criteria for sepsis detection in anaesthetic intensive care patients with a diagnosis: SARS-CoV-2, virus unidentified.
Methods. A retrospective observational study included 166 patients over 18-year age with unconfirmed infection (ICD-10 code U07.2). The qSOFA, NEWS, REMS and SIRS point estimates were obtained from each patient. The patients were retrospectively divided in two cohorts by sepsis presence (Sepsis-3 criteria) to determine the express scales power in evaluating the risk of sepsis (estimated as area under ROC curve, AUROC).
Results. Data on 102 patients were included in the final analysis. Fifty-eight (57%) patients were terminal, and 55 (54%) developed sepsis. The estimates are as follows: NEWS — AUROC 0.848 [95% confidence interval (CI) 0.764–0.912], sensitivity 76.36% [95% CI 63.0–86.8], specificity 82.98% [95% CI 69.2–92.4], optimal cut-off threshold >5 points; qSOFA — AUROC 0.700 [95% CI 0.602–0.787], sensitivity 76.36% [95% CI 63.0–86.8], specificity 61.70% [95% CI 46.4–75.5], optimal cut-off threshold >0 points; REMS — AUROC 0.739 [95% CI 0.643–0.821], sensitivity 69.09% [95% CI 55.2–80.9], specificity 65.96% [95% CI 50.7–79.1], optimal cut-off threshold >5 points; SIRS criteria — AUROC 0.723 [95% CI 0.626–0.807], sensitivity 98.18% [95% CI 90.3–100.0], specificity 31.91% [95% CI 19.1–47.1], optimal cut-off threshold >0 points.
Conclusion. The NEWS scale revealed a good prognostic power to estimate the risk of sepsis in patients with suspected COVID-19 disease. The qSOFA, REMS scales and SIRS criteria possess a good calibration capacity, albeit insufficient resolution, which limits their prognostic value.

1. Huang C., Wang Y., Li X., Ren L., Zhao J., Hu Y., Zhang L., Fan G., Xu J., Gu X., Cheng Z., Yu T., Xia J., Wei Y., Wu W., Xie X., Yin W., Li H., Liu M., Xiao Y., Gao H., Guo L., Xie J., Wang G., Jiang R., Gao Z., Jin Q., Wang J., Cao B. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. Lancet. 2020; 395(10223): 497–506. DOI: 10.1016/S0140-6736(20)30183-5

2. Lithander F.E., Neumann S., Tenison E., Lloyd K., Welsh T.J., Rodrigues J.C.L., Higgins J.P.T., Scourfield L., Christensen H., Haunton V.J., Henderson E.J. COVID-19 in older people: a rapid clinical review. Age. Ageing. 2020; 49(4): 501–515. DOI: 10.1093/ageing/afaa093

3. Singer M., Deutschman C.S., Seymour C.W., Shankar-Hari M., Annane D., Bauer M., Bellomo R., Bernard G.R., Chiche J.D., Coopersmith C.M., Hotchkiss R.S., Levy M.M., Marshall J.C., Martin G.S., Opal S.M., Rubenfeld G.D., van der Poll T., Vincent J.L., Angus D.C. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016; 315(8): 801–810. DOI: 10.1001/jama.2016.0287

4. Fleischmann C., Scherag A., Adhikari N.K., Hartog C.S., Tsaganos T., Schlattmann P., Angus D.C., Reinhart K.; International Forum of Acute Care Trialists. Assessment of Global Incidence and Mortality of Hospital-treated Sepsis. Current Estimates and Limitations. Am. J. Respir. Crit. Care. Med. 2016; 193(3): 259–272. DOI: 10.1164/rccm.201504-0781OC

5. Khwannimit B., Bhurayanontachai R. The epidemiology of, and risk factors for, mortality from severe sepsis and septic shock in a tertiary-care university hospital setting. Epidemiol. Infect. 2009; 137(9): 1333–1341. DOI: 10.1017/S0950268809002027

6. Khwannimit B., Bhurayanontachai R., Vattanavanit V. Comparison of the accuracy of three early warning scores with SOFA score for predicting mortality in adult sepsis and septic shock patients admitted to intensive care unit. Heart. Lung. 2019; 48(3): 240–244. DOI: 10.1016/j.hrtlng.2019.02.005

7. Rudd K.E., Seymour C.W., Aluisio A.R., Augustin M.E., Bagenda D.S., Beane A., Byiringiro J.C., Chang C.H., Colas L.N., Day N.P.J., De Silva A.P., Dondorp A.M., Dünser M.W., Faiz M.A., Grant D.S., Haniffa R., Van Hao N., Kennedy J.N., Levine A.C., Limmathurotsakul D., Mohanty S., Nosten F., Papali A., Patterson A.J., Schieffelin J.S., Shaffer J.G., Thuy D.B., Thwaites C.L., Urayeneza O., White N.J., West T.E., Angus D.C.; Sepsis Assessment and Identification in Low Resource Settings (SAILORS) Collaboration. Association of the Quick Sequential (Sepsis-Related) Organ Failure Assessment (qSOFA) Score With Excess Hospital Mortality in Adults With Suspected Infection in Low- and Middle-Income Countries. JAMA. 2018; 319(21): 2202–2211. DOI: 10.1001/jama.2018.6229

8. Rudnov V.А., Kulabukhov V.V. Sepsis-3: updated main definitions, potential problems and next practical steps. Messenger of Anesthesiology and Resuscitation. 2016; 13(4): 4–11 (In Russ., English abstract). DOI: 10.21292/2078-5658-2016-13-4-4-11

9. Guan W.J., Ni Z.Y., Hu Y., Liang W.H., Ou C.Q., He J.X., Liu L., Shan H., Lei C.L., Hui D.S.C., Du B., Li L.J, Zeng G., Yuen K.Y., Chen R.C., Tang C.L., Wang T., Chen P.Y., Xiang J., Li S.Y., Wang J.L., Liang Z.J., Peng Y.X., Wei L., Liu Y., Hu Y.H., Peng P., Wang J.M., Liu J.Y., Chen Z., Li G., Zheng Z.J., Qiu S.Q., Luo J., Ye C.J., Zhu S.Y., Zhong N.S.; China Medical Treatment Expert Group for Covid-19. Clinical Characteristics of Coronavirus Disease 2019 in China. N. Engl. J. Med. 2020; 382(18): 1708–1720. DOI: 10.1056/NEJMoa2002032

10. Zhou F., Yu T., Du R., Fan G., Liu Y., Liu Z., Xiang J., Wang Y., Song B., Gu X., Guan L., Wei Y., Li H., Wu X., Xu J., Tu S., Zhang Y., Chen H., Cao B. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. Lancet. 2020; 395(10229): 1054–1062. DOI: 10.1016/S0140-6736(20)30566-3

11. Wu Z., McGoogan J.M. Characteristics of and Important Lessons From the Coronavirus Disease 2019 (COVID-19) Outbreak in China: Summary of a Report of 72 314 Cases From the Chinese Center for Disease Control and Prevention. JAMA. 2020; 323(13): 1239– 1242. DOI: 10.1001/jama.2020.2648

12. Olsson T., Terent A., Lind L. Rapid Emergency Medicine score: a new prognostic tool for in-hospital mortality in nonsurgical emergency department patients. J. Intern. Med. 2004; 255(5): 579–587. DOI: 10.1111/j.1365-2796.2004.01321.x

13. Kuo S.H., Tsai C.F., Li C.R., Tsai S.J., Chao W.N., Chan K.S., Lee Y.T., Wong R.H., Chen C.C., Chen S.C. Rapid Emergency Medicine Score as a main predictor of mortality in Vibrio vulnificus-related patients. Am. J. Emerg. Med. 2013; 31(7): 1037–1041. DOI: 10.1016/j.ajem.2013.03.030

14. Tanne J.H., Hayasaki E., Zastrow M., Pulla P., Smith P., Rada A.G. Covid-19: how doctors and healthcare systems are tackling coronavirus worldwide. BMJ. 2020; 368:m1090. DOI: 10.1136/bmj.m1090

15. Zabolotskikh I.B., Kirov M. Yu., Lebedinskii K.M., Protsenko D.N., Avdeev S.N., Andreenko A.A., Arsentyev L.V., Afonchikov V.S., Afukov I.I., Belkin A.A., Boeva E.A., Bulanov A. Yu., Vasiliev Ya.I., Vlasenko A.V., Gorbachev V.I., Grigoryev E.V., Grigoryev S.V., Gritsan A.I., Eremenko A.A., Ershov E.N., Zamyatin M.N., Ivanova G.E., Kuzovlev A.N., Kulikov A.V., Lakhin R.E., Leiderman I.N., Lenkin A.I., Mazurok V.A., Musaeva T.S., Nikolaenko E.M., Orlov Yu.P., Petrikov S.S., Roitman E.V., Ronenson A.M., Smetkin A.A., Sokolov A.A., Stepanenko S.M., Subbotin V.V., Ushakova N.D., Khoronenko V.E., Tsarenko S.V., Shifman E.M., Shukevich D.L., Shchegolev A.V., Yaroshetskiy A.I., Yarustovsky M.B. Anesthesia and intensive care for patients with COVID-19. Russian Federation of anesthesiologists and reanimatologists guidelines. Annals of Critical Care. 2022; 1: 5–140 (In Russ., English abstract). DOI: 10.21320/1818-474X-2022-1-5-140

16. Goulden R., Hoyle M.C., Monis J., Railton D., Riley V., Martin P., Martina R., Nsutebu E. qSOFA, SIRS and NEWS for predicting inhospital mortality and ICU admission in emergency admissions treated as sepsis. Emerg. Med. J. 2018; 35(6): 345–349. DOI: 10.1136/emermed-2017-207120

17. Musaeva T.S., Mirtskhulava N.G., Zybin K.D., Sinkov S.V., Astakhova E.O., Lobur E.I., Noskov A.A., Zabolotskikh I.B. Validity of qsofa, news and rems express-scales in patients with COVID-19 (virus not identified): a retrospective observational study. Russian Journal of Anaesthesiology and Reanimatology. 2020; 6-2: 27–32 (In Russ., English abstract). DOI: 10.17116/anaesthesiology202006227

18. Zabolotskikh I.B., Zybin K.D., Noskov A.A., Kuznetsova M.P., Musaeva T.S., Sinkov S.V. Comparison of REMS, NEWS and qSOFA scales in mortality prediction for patients with confirmed SARS-CoV-2 infection upon admission to ICU: a retrospective observational trial. Russian Journal of Anaesthesiology and Reanimatology. 2020; 6-2: 20–26 (In Russ., English abstract). DOI: 10.17116/anaesthesiology202006220

19. Zybin K.D., Noskov А.А., Musaeva T.S., Kuznetsova М.P., Goncharenko S.I., Vysotsky O.V., Danilyuk P.I., Petrushenko E.S., Veselenko M.I., Potapova A.A., Sinkov S.V., Durleshter V.M.. Comparison of REMS, NEWS, qSOFA scales and SIRS criteria in sepsis prediction for patients with confirmed SARS-CoV-2 infection: retrospective observational study. Annals of Critical Care. 2021; 1: 48–56 (In Russ., English abstract). DOI: 10.21320/1818-474X-2021-1-48-56

20. Ruangsomboon O., Boonmee P., Limsuwat C., Chakorn T., Monsomboon A. The utility of the rapid emergency medicine score (REMS) compared with SIRS, qSOFA and NEWS for Predicting in-hospital Mortality among Patients with suspicion of Sepsis in an emergency department. BMC Emerg. Med. 2021; 21(1): 2. DOI: 10.1186/s12873-020-00396-x

21. Brink A., Alsma J., Verdonschot R.J.C.G., Rood P.P.M., Zietse R., Lingsma H.F., Schuit S.C.E. Predicting mortality in patients with suspected sepsis at the Emergency Department; A retrospective cohort study comparing qSOFA, SIRS and National Early Warning Score. PLoS One. 2019; 14(1): e0211133. DOI: 10.1371/journal.pone.0211133

22. Wattanasit P., Khwannimit B. Comparison the accuracy of early warning scores with qSOFA and SIRS for predicting sepsis in the emergency department. Am. J. Emerg. Med. 2021; 46: 284–288. DOI: 10.1016/j.ajem.2020.07

23. Pairattanakorn P., Angkasekwinai N., Sirijatuphat R., Wangchinda W., Tancharoen L., Thamlikitkul V. Diagnostic and prognostic utility compared among different sepsis scoring systems in adult patients with sepsis in thailand: a prospective cohort study. Open. Forum. Infect. Dis. 2020; 8(1): ofaa573. DOI: 10.1093/ofid/ofaa573

24. Myrstad M., Ihle-Hansen H., Tveita A.A., Andersen E.L., Nygård S., Tveit A., Berge T. National Early Warning Score 2 (NEWS2) on admission predicts severe disease and in-hospital mortality from Covid-19 — a prospective cohort study. Scand. J. Trauma. Resusc. Emerg. Med. 2020; 28(1): 66. DOI: 10.1186/s13049-020-00764-3

25. Wallgren U.M., Sjölin J., Järnbert-Pettersson H., Kurland L. Performance of NEWS2, RETTS, clinical judgment and the Predict Sepsis screening tools with respect to identification of sepsis among ambulance patients with suspected infection: a prospective cohort study. Scand. J. Trauma. Resusc. Emerg. Med. 2021; 29(1): 144. DOI: 10.1186/s13049-021-00958-3

26. Richardson S., Hirsch J.S., Narasimhan M., Crawford J.M., McGinn T., Davidson K.W.; the Northwell COVID-19 Research Consortium, Barnaby D.P., Becker L.B., Chelico J.D., Cohen S.L., Cookingham J., Coppa K., Diefenbach M.A., Dominello A.J., Duer-Hefele J., Falzon L., Gitlin J., Hajizadeh N., Harvin T.G., Hirschwerk D.A., Kim E.J., Kozel Z.M., Marrast L.M., Mogavero J.N., Osorio G.A., Qiu M., Zanos T.P. Presenting characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the New York city area. JAMA. 2020; 323(20): 2052–2059. DOI: 10.1001/jama.2020.6775

27. Wu C., Chen X., Cai Y., Xia J., Zhou X., Xu S., Huang H., Zhang L., Zhou X., Du C., Zhang Y., Song J., Wang S., Chao Y., Yang Z., Xu J., Zhou X., Chen D., Xiong W., Xu L., Zhou F., Jiang J., Bai C., Zheng J., Song Y. Risk factors associated with acute respiratory distress syndrome and death in patients with coronavirus disease 2019 pneumonia in Wuhan, China. JAMA Intern. Med. 2020; 180(7): 934–943. DOI: 10.1001/jamainternmed.2020.0994