The presence of radiological features on chest radiographs: how well do clinicians agree?

A Deep Learning Model for the Diagnosis and Discrimination of Gram-Positive and Gram-Negative Bacterial Pneumonia for Children Using Chest Radiography Images and Clinical Information

Purpose

This study aimed to develop a deep learning model based on chest radiography (CXR) images and clinical data to accurately classify gram-positive and gram-negative bacterial pneumonia in children to guide the use of antibiotics.

Methods

We retrospectively collected CXR images along with clinical information for gram-positive (n=447) and gram-negative (n=395) bacterial pneumonia in children from January 1, 2016, to June 30, 2021. Four types of machine learning models based on clinical data and six types of deep learning algorithm models based on image data were constructed, and multi-modal decision fusion was performed.

Results

In the machine learning models, CatBoost, which only used clinical data, had the best performance; its area under the receiver operating characteristic curve (AUC) was significantly higher than that of the other models (P<0.05). The incorporation of clinical information improved the performance of deep learning models that relied solely on image-based classification. Consequently, AUC and F1 increased by 5.6% and 10.2% on average, respectively. The best quality was achieved with ResNet101 (model accuracy: 0.75, recall rate: 0.84, AUC: 0.803, F1: 0.782).

Conclusion

Our study established a pediatric bacterial pneumonia model that utilizes CXR and clinical data to accurately classify cases of gram-negative and gram-positive bacterial pneumonia. The results confirmed that the addition of image data to the convolutional neural network model significantly improved its performance. While the CatBoost-based classifier had greater advantages owing to a smaller dataset, the quality of the Resnet101 model trained using multi-modal data was comparable to that of the CatBoost model, even with a limited number of samples.

Seeking diagnostic and prognostic biomarkers for childhood bacterial pneumonia in sub-Saharan Africa: study protocol for an observational study

INTRODUCTION

Clinically diagnosed pneumonia in children is a leading cause of paediatric hospitalisation and mortality. The aetiology is usually bacterial or viral, but malaria can cause a syndrome indistinguishable from clinical pneumonia. There is no method with high sensitivity to detect a bacterial infection in these patients and, as result, antibiotics are frequently overprescribed. Conversely, unrecognised concomitant bacterial infection in patients with malarial infections occur with omission of antibiotic therapy from patients with bacterial infections. Previously, we identified two combinations of blood proteins with 96% sensitivity and 86% specificity for detecting bacterial disease. The current project aimed to validate and improve these combinations by evaluating additional biomarkers in paediatric patients with clinical pneumonia. Our goal was to describe combinations of a limited number of proteins with high sensitivity and specificity for bacterial infection to be incorporated in future point-of-care tests. Furthermore, we seek to explore signatures to prognosticate clinical pneumonia.

METHODS AND ANALYSIS

Patients (n=900) aged 2-59 months presenting with clinical pneumonia at two Gambian hospitals will be enrolled and classified according to criteria for definitive bacterial aetiology (based on microbiological tests and chest radiographs). We will measure proteins at admission using Luminex-based immunoassays in 90 children with definitive and 160 with probable bacterial aetiology, and 160 children classified according to the prognosis of their disease. Previously identified diagnostic signatures will be assessed through accuracy measures. Moreover, we will seek new diagnostic and prognostic signatures through machine learning methods, including support vector machine, penalised regression and classification trees.

ETHICS AND DISSEMINATION

Ethics approval has been obtained from the Gambia Government/Medical Research Council Unit The Gambia Joint Ethics Committee (protocol 1616) and the institutional review board of Boston University Medical Centre (STUDY00000958). Study results will be disseminated to the staff of the study hospitals, in scientific seminars and meetings, and in publications.

TRIAL REGISTRATION NUMBER

H-38462.

Reliability of prehospital patient classification in helicopter emergency medical service missions

Background

Several scores and codes are used in prehospital clinical quality registries but little is known of their reliability. The aim of this study is to evaluate the inter-rater reliability of the American Society of Anesthesiologists physical status (ASA-PS) classification system, HEMS benefit score (HBS), International Classification of Primary Care, second edition (ICPC-2) and Eastern Cooperative Oncology Group (ECOG) performance status in a helicopter emergency medical service (HEMS) clinical quality registry (CQR).

Methods

All physicians and paramedics working in HEMS in Finland and responsible for patient registration were asked to participate in this study. The participants entered data of six written fictional missions in the national CQR. The inter-rater reliability of the ASA-PS, HBS, ICPC-2 and ECOG were evaluated using an overall agreement and free-marginal multi-rater kappa (Κ_free).

Results

All 59 Finnish HEMS physicians and paramedics were invited to participate in this study, of which 43 responded and 16 did not answer. One participant was excluded due to unfinished data entering. ASA-PS had an overall agreement of 40.2% and Κ_free of 0.28 in this study. HBS had an overall agreement of 44.7% and Κ_free of 0.39. ICPC-2 coding had an overall agreement of 51.5% and Κ_free of 0.47. ECOG had an overall agreement of 49.6% and Κ_free of 0.40.

Conclusion

This study suggests a marked inter-rater unreliability in prehospital patient scoring and coding even in a relatively uniform group of practitioners working in a highly focused environment. This indicates that the scores and codes should be specifically designed or adapted for prehospital use, and the users should be provided with clear and thorough instructions on how to use them.

The reliability and accuracy of operational system data in a nationwide helicopter emergency medical services mission database

AIM

The aim of this study was to evaluate the reliability and accuracy of documentation in FinnHEMS database, which is a nationwide helicopter emergency service (HEMS) clinical quality registry.

METHODS

This is a nationwide study based on written fictional clinical scenarios. Study subjects were HEMS physicians and paramedics, who filled in the clinical quality registry based on the clinical scenarios. The inter-rater -reliability of the collected data was analyzed with percent agreement and free-marginal multi-rater kappa.

RESULTS

Dispatch coding had a percent agreement of 91% and free-marginal multi-rater kappa value of 0.83. Coding for transportation or mission cancellation resulted in an agreement of 84% and free-marginal kappa value of 0.68. An agreement of 82% and a kappa value of 0.73 for dispatcher coding was found. Mission end, arrival at hospital and HEMS unit dispatch -times had agreements from 80 to 85% and kappa values from 0.61 to 0.73. The emergency call to dispatch centre time had an agreement of 71% and kappa value of 0.56. The documentation of pain had an agreement of 73% on both the first and second measurements. All other vital parameters had less than 70% agreement and 0.40 kappa value in the first measurement. The documentation of secondary vital parameter measurements resulted in agreements from 72 to 91% and kappa values from 0.43 to 0.64.

CONCLUSION

Data from HEMS operations can be gathered reliably in a national clinical quality registry. This study revealed some inaccuracies in data registration and data quality, which are important to detect to improve the overall reliability and validity of the HEMS clinical quality register.

Lung ultrasound for the diagnosis of pneumonia in children with acute bronchiolitis

BACKGROUND

Guidelines currently do not recommend the routine use of chest x-ray (CXR) in bronchiolitis. However, CXR is still performed in a high percentage of cases, mainly to diagnose or rule out pneumonia. The inappropriate use of CXR results in children exposure to ionizing radiations and increased medical costs. Lung Ultrasound (LUS) has become an emerging diagnostic tool for diagnosing pneumonia in the last decades. The purpose of this study was to assess the diagnostic accuracy and reliability of LUS for the detection of pneumonia in hospitalized children with bronchiolitis and to evaluate the agreement between LUS and CXR in diagnosing pneumonia in these patients.

METHODS

We enrolled children admitted to our hospital in 2016-2017 with a diagnosis of bronchiolitis and undergone CXR because of clinical suspicion of concomitant pneumonia. LUS was performed in each child by a pediatrician blinded to the patient's clinical, laboratory and CXR findings. An exploratory analysis was done in the first 30 patients to evaluate the inter-observer agreement between a pediatrician and a radiologist who independently performed LUS. The diagnosis of pneumonia was established by an expert clinician based on the recommendations of the British Thoracic Society guidelines.

RESULTS

Eighty seven children with bronchiolitis were investigated. A final diagnosis of concomitant pneumonia was made in 25 patients. Sensitivity and specificity of LUS for the diagnosis of pneumonia were 100% and 83.9% respectively, with an area under-the-curve of 0.92, while CXR showed a sensitivity of 96% and specificity of 87.1%. When only consolidation > 1 cm was considered consistent with pneumonia, the specificity of LUS increased to 98.4% and the sensitivity decreased to 80.0%, with an area under-the-curve of 0.89. Cohen's kappa between pediatrician and radiologist sonologists in the first 30 patients showed an almost perfect agreement in diagnosing pneumonia by LUS (K 0.93).

CONCLUSIONS

This study shows the good accuracy of LUS in diagnosing pneumonia in children with clinical bronchiolitis. When including only consolidation size > 1 cm, specificity of LUS was higher than CXR, avoiding the need to perform CXR in these patients. Added benefit of LUS included high inter-observer agreement.

TRIAL REGISTRATION

Identifier: NCT03280732 . Registered 12 September 2017 (retrospectively registered).

An empirical method to cluster objective nebulizer adherence data among adults with cystic fibrosis

BACKGROUND

The purpose of using preventative inhaled treatments in cystic fibrosis is to improve health outcomes. Therefore, understanding the relationship between adherence to treatment and health outcome is crucial. Temporal variability, as well as absolute magnitude of adherence affects health outcomes, and there is likely to be a threshold effect in the relationship between adherence and outcomes. We therefore propose a pragmatic algorithm-based clustering method of objective nebulizer adherence data to better understand this relationship, and potentially, to guide clinical decisions.

METHODS TO CLUSTER ADHERENCE DATA

This clustering method consists of three related steps. The first step is to split adherence data for the previous 12 months into four 3-monthly sections. The second step is to calculate mean adherence for each section and to score the section based on mean adherence. The third step is to aggregate the individual scores to determine the final cluster ("cluster 1" = very low adherence; "cluster 2" = low adherence; "cluster 3" = moderate adherence; "cluster 4" = high adherence), and taking into account adherence trend as represented by sequential individual scores. The individual scores should be displayed along with the final cluster for clinicians to fully understand the adherence data.

THREE ILLUSTRATIVE CASES

We present three cases to illustrate the use of the proposed clustering method.

CONCLUSION

This pragmatic clustering method can deal with adherence data of variable duration (ie, can be used even if 12 months' worth of data are unavailable) and can cluster adherence data in real time. Empirical support for some of the clustering parameters is not yet available, but the suggested classifications provide a structure to investigate parameters in future prospective datasets in which there are accurate measurements of nebulizer adherence and health outcomes.

Lung ultrasound for the diagnosis of community-acquired pneumonia in children

Ultrasound (US) has been proposed as an alternative first-line imaging modality to diagnose community-acquired pneumonia in children. Lung US has the potential benefits over chest radiography of being radiation free, subject to fewer regulatory requirements, relatively lower cost and with immediate bedside availability of results. However, the uptake of lung US into clinical practice has been slow and it is not yet included in clinical guidelines for community-acquired pneumonia in children. The aim of this review is to give an overview of the equipment and techniques used to perform lung US in children with suspected pneumonia and the interpretation of relevant sonographic findings. We also summarise the current evidence of diagnostic accuracy and reliability of lung US compared to alternative imaging modalities in children and critically consider the strengths and limitations of lung US for use in children presenting with suspected community-acquired pneumonia.

Responses to Bacteria, Virus, and Malaria Distinguish the Etiology of Pediatric Clinical Pneumonia

RATIONALE

Plasma-detectable biomarkers that rapidly and accurately diagnose bacterial infections in children with suspected pneumonia could reduce the morbidity of respiratory disease and decrease the unnecessary use of antibiotic therapy.

OBJECTIVES

Using 56 markers measured in a multiplexed immunoassay, we sought to identify proteins and protein combinations that could discriminate bacterial from viral or malarial diagnoses.

METHODS

We selected 80 patients with clinically diagnosed pneumonia (as defined by the World Health Organization) who also met criteria for bacterial, viral, or malarial infection based on clinical, radiographic, and laboratory results. Ten healthy community control subjects were enrolled to assess marker reliability. Patients were subdivided into two sets: one for identifying potential markers and another for validating them.

MEASUREMENTS AND MAIN RESULTS

Three proteins (haptoglobin, tumor necrosis factor receptor 2 or IL-10, and tissue inhibitor of metalloproteinases 1) were identified that, when combined through a classification tree signature, accurately classified patients into bacterial, malarial, and viral etiologies and misclassified only one patient with bacterial pneumonia from the validation set. The overall sensitivity and specificity of this signature for the bacterial diagnosis were 96 and 86%, respectively. Alternative combinations of markers with comparable accuracy were selected by support vector machine and regression models and included haptoglobin, IL-10, and creatine kinase-MB.

CONCLUSIONS

Combinations of plasma proteins accurately identified children with a respiratory syndrome who were likely to have bacterial infections and who would benefit from antibiotic therapy. When used in conjunction with malaria diagnostic tests, they may improve diagnostic specificity and simplify treatment decisions for clinicians.

A chest radiograph scoring system in patients with severe acute respiratory infection: a validation study

BACKGROUND

The term severe acute respiratory infection (SARI) encompasses a heterogeneous group of respiratory illnesses. Grading the severity of SARI is currently reliant on indirect disease severity measures such as respiratory and heart rate, and the need for oxygen or intensive care. With the lungs being the primary organ system involved in SARI, chest radiographs (CXRs) are potentially useful for describing disease severity. Our objective was to develop and validate a SARI CXR severity scoring system.

METHODS

We completed validation within an active SARI surveillance project, with SARI defined using the World Health Organization case definition of an acute respiratory infection with a history of fever, or measured fever of ≥ 38 °C; and cough; and with onset within the last 10 days; and requiring hospital admission. We randomly selected 250 SARI cases. Admission CXR findings were categorized as: 1 = normal; 2 = patchy atelectasis and/or hyperinflation and/or bronchial wall thickening; 3 = focal consolidation; 4 = multifocal consolidation; and 5 = diffuse alveolar changes. Initially, four radiologists scored CXRs independently. Subsequently, a pediatrician, physician, two residents, two medical students, and a research nurse independently scored CXR reports. Inter-observer reliability was determined using a weighted Kappa (κ) for comparisons between radiologists; radiologists and clinicians; and clinicians. Agreement was defined as moderate (κ > 0.4-0.6), good (κ > 0.6-0.8) and very good (κ > 0.8-1.0).

RESULTS

Agreement between the two pediatric radiologists was very good (κ = 0.83, 95% CI 0.65-1.00) and between the two adult radiologists was good (κ = 0.75, 95% CI 0.57-0. 93). Agreement of the clinicians with the radiologists was moderate-to-good (pediatrician:κ = 0.65; pediatric resident:κ = 0.69; physician:κ = 0.68; resident:κ = 0.67; research nurse:κ = 0.49, medical students: κ = 0.53 and κ = 0.56). Agreement between clinicians was good-to-very good (pediatrician vs. physician:κ = 0.85; vs. pediatric resident:κ = 0.81; vs. medicine resident:κ = 0.76; vs. research nurse:κ = 0.75; vs. medical students:κ = 0.63 and 0.66). Following review of discrepant CXR report scores by clinician pairs, κ values for radiologist-clinician agreement ranged from 0.59 to 0.70 and for clinician-clinician agreement from 0.97 to 0.99.

CONCLUSIONS

This five-point CXR scoring tool, suitable for use in poorly- and well-resourced settings and by clinicians of varying experience levels, reliably describes SARI severity. The resulting numerical data enables epidemiological comparisons of SARI severity between different countries and settings.

Why some children hospitalized for pneumonia do not consult with a general practitioner before the day of hospitalization

BACKGROUND

Early consultation in primary care may provide an opportunity for early intervention in children developing pneumonia, but little is known about why some children do not consult a general practitioner (GP) before hospitalization.

OBJECTIVES

To identify differences between children who consulted a GP and children who did not consult a GP before the day of hospital presentation with pneumonia or empyema.

METHODS

Carers of children aged six months to 16 years presenting to hospital with pneumonia or empyema completed a questionnaire, with a subset participating in an interview to identify physical, organizational and psychological barriers to consultation. Responses from those who had consulted a GP before the day of hospital presentation were compared with those who had not on a range of medical, social and environmental variables.

RESULTS

Fifty seven (38%) of 151 participants had not consulted a GP before the day of hospital presentation. On multivariate analysis, illness duration ≥ 3 days (odds ratio [OR] 4.36, 95% confidence interval [CI]: 1.67-11.39), prior antibiotic use (OR: 10.35, 95% CI: 2.16-49.55) and home ownership (OR: 3.17, 95% CI: 1.07-9.37) were significantly associated with early GP consultation (P < 0.05). Interviews with 28 carers whose children had not seen a GP before the day of presentation revealed that most had not considered it and/or did not think their child's initial symptoms were serious or unusual; 11 (39.3%) had considered consulting a GP but reported barriers to access.

CONCLUSION

Lack of early GP consultation was strongly associated with rapid evolution of pneumonia.