Forcing dichotomous disease classification from reference standards leads to bias in diagnostic accuracy estimates: A simulation study

Host biomarkers and combinatorial scores for the detection of serious and invasive bacterial infection in pediatric patients with fever without source

BACKGROUND

Improved tools are required to detect bacterial infection in children with fever without source (FWS), especially when younger than 3 years old. The aim of the present study was to investigate the diagnostic accuracy of a host signature combining for the first time two viral-induced biomarkers, tumor necrosis factor-related apoptosis-inducing ligand (TRAIL) and interferon γ-induced protein-10 (IP-10), with a bacterial-induced one, C-reactive protein (CRP), to reliably predict bacterial infection in children with fever without source (FWS) and to compare its performance to routine individual biomarkers (CRP, procalcitonin (PCT), white blood cell and absolute neutrophil counts, TRAIL, and IP-10) and to the Labscore.

METHODS

This was a prospective diagnostic accuracy study conducted in a single tertiary center in children aged less than 3 years old presenting with FWS. Reference standard etiology (bacterial or viral) was assigned by a panel of three independent experts. Diagnostic accuracy (AUC, sensitivity, specificity) of host individual biomarkers and combinatorial scores was evaluated in comparison to reference standard outcomes (expert panel adjudication and microbiological diagnosis).

RESULTS

241 patients were included. 68 of them (28%) were diagnosed with a bacterial infection and 5 (2%) with invasive bacterial infection (IBI). Labscore, ImmunoXpert, and CRP attained the highest AUC values for the detection of bacterial infection, respectively 0.854 (0.804-0.905), 0.827 (0.764-0.890), and 0.807 (0.744-0.869). Labscore and ImmunoXpert outperformed the other single biomarkers with higher sensitivity and/or specificity and showed comparable performance to one another although slightly reduced sensitivity in children < 90 days of age.

CONCLUSION

Labscore and ImmunoXpert demonstrate high diagnostic accuracy for safely discriminating bacterial infection in children with FWS aged under and over 90 days, supporting their adoption in the assessment of febrile patients.

Biomarkers as Prognostic Predictors and Therapeutic Guide in Critically Ill Patients: Clinical Evidence

A biomarker is a molecule that can be measured in a biological sample in an objective, systematic, and precise way, whose levels indicate whether a process is normal or pathological. Knowing the most important biomarkers and their characteristics is the key to precision medicine in intensive and perioperative care. Biomarkers can be used to diagnose, in assessment of disease severity, to stratify risk, to predict and guide clinical decisions, and to guide treatments and response to them. In this review, we will analyze what characteristics a biomarker should have and how to ensure its usefulness, and we will review the biomarkers that in our opinion can make their knowledge more useful to the reader in their clinical practice, with a future perspective. These biomarkers, in our opinion, are lactate, C-Reactive Protein, Troponins T and I, Brain Natriuretic Peptides, Procalcitonin, MR-ProAdrenomedullin and BioAdrenomedullin, Neutrophil/lymphocyte ratio and lymphopenia, Proenkephalin, NefroCheck, Neutrophil gelatinase-associated lipocalin (NGAL), Interleukin 6, Urokinase-type soluble plasminogen activator receptor (suPAR), Presepsin, Pancreatic Stone Protein (PSP), and Dipeptidyl peptidase 3 (DPP3). Finally, we propose an approach to the perioperative evaluation of high-risk patients and critically ill patients in the Intensive Care Unit (ICU) based on biomarkers.

Machine learning prediction of postoperative major adverse cardiovascular events in geriatric patients: a prospective cohort study

Background

Postoperative major adverse cardiovascular events (MACEs) account for more than one-third of perioperative deaths. Geriatric patients are more vulnerable to postoperative MACEs than younger patients. Identifying high-risk patients in advance can help with clinical decision making and improve prognosis. This study aimed to develop a machine learning model for the preoperative prediction of postoperative MACEs in geriatric patients.

Methods

We collected patients’ clinical data and laboratory tests prospectively. All patients over 65 years who underwent surgeries in West China Hospital of Sichuan University from June 25, 2019 to June 29, 2020 were included. Models based on extreme gradient boosting (XGB), gradient boosting machine, random forest, support vector machine, and Elastic Net logistic regression were trained. The models’ performance was compared according to area under the precision-recall curve (AUPRC), area under the receiver operating characteristic curve (AUROC) and Brier score. To minimize the influence of clinical intervention, we trained the model based on undersampling set. Variables with little contribution were excluded to simplify the model for ensuring the ease of use in clinical settings.

Results

We enrolled 5705 geriatric patients into the final dataset. Of those patients, 171 (3.0%) developed postoperative MACEs within 30 days after surgery. The XGB model outperformed other machine learning models with AUPRC of 0.404(95% confidence interval [CI]: 0.219–0.589), AUROC of 0.870(95%CI: 0.786–0.938) and Brier score of 0.024(95% CI: 0.016–0.032). Model trained on undersampling set showed improved performance with AUPRC of 0.511(95% CI: 0.344–0.667, p < 0.001), AUROC of 0.912(95% CI: 0.847–0.962, p < 0.001) and Brier score of 0.020 (95% CI: 0.013–0.028, p < 0.001). After removing variables with little contribution, the undersampling model showed comparable predictive accuracy with AUPRC of 0.507(95% CI: 0.338–0.669, p = 0.36), AUROC of 0.896(95%CI: 0.826–0.953, p < 0.001) and Brier score of 0.020(95% CI: 0.013–0.028, p = 0.20).

Conclusions

In this prospective study, we developed machine learning models for preoperative prediction of postoperative MACEs in geriatric patients. The XGB model showed the best performance. Undersampling method achieved further improvement of model performance.

Trial registration

The protocol of this study was registered at www.chictr.org.cn (15/08/2019, ChiCTR1900025160)

Supplementary Information

The online version contains supplementary material available at 10.1186/s12871-022-01827-x.

Evaluation of Biomarkers in Critical Care and Perioperative Medicine: A Clinician’s Overview of Traditional Statistical Methods and Machine Learning Algorithms

Interest in developing and using novel biomarkers in critical care and perioperative medicine is increasing. Biomarkers studies are often presented with flaws in the statistical analysis that preclude them from providing a scientifically valid and clinically relevant message for clinicians. To improve scientific rigor, the proper application and reporting of traditional and emerging statistical methods (e.g., machine learning) of biomarker studies is required. This Readers' Toolbox article aims to be a starting point to nonexpert readers and investigators to understand traditional and emerging research methods to assess biomarkers in critical care and perioperative medicine.

Development of a clinical prediction rule for sepsis in primary care: protocol for the TeSD-IT study

BACKGROUND

Early recognition and treatment of sepsis is crucial to prevent detrimental outcomes. General practitioners (GPs) are often the first healthcare providers to encounter seriously ill patients. The aim of this study is to assess the value of clinical information and additional tests to develop a clinical prediction rule to support early diagnosis and management of sepsis by GPs.

METHODS

We will perform a diagnostic study in the setting of out-of-hours home visits in four GP cooperatives in the Netherlands. Acutely ill adult patients suspected of a serious infection will be screened for eligibility by the GP. The following candidate predictors will be prospectively recorded: (1) age, (2) body temperature, (3) systolic blood pressure, (4) heart rate, (5) respiratory rate, (6) peripheral oxygen saturation, (7) mental status, (8) history of rigors, and (9) rate of progression. After clinical assessment by the GP, blood samples will be collected in all patients to measure C-reactive protein, lactate, and procalcitonin. All patients will receive care as usual. The primary outcome is the presence or absence of sepsis within 72 h after inclusion, according to an expert panel. The need for hospital treatment for any indication will be assessed by the expert panel as a secondary outcome. Multivariable logistic regression will be used to design an optimal prediction model first and subsequently derive a simplified clinical prediction rule that enhances feasibility of using the model in daily clinical practice. Bootstrapping will be performed for internal validation of both the optimal model and simplified prediction rule. Performance of both models will be compared to existing clinical prediction rules for sepsis.

DISCUSSION

This study will enable us to develop a clinical prediction rule for the recognition of sepsis in a high-risk primary care setting to aid in the decision which patients have to be immediately referred to a hospital and who can be safely treated at home. As clinical signs and blood samples will be obtained prospectively, near-complete data will be available for analyses. External validation will be needed before implementation in routine care and to determine in which pre-hospital settings care can be improved using the prediction rule.

TRIAL REGISTRATION

The study is registered in the Netherlands Trial Registry (registration number NTR7026).