Inter-rater reliability in quality assurance (QA) of pediatric chest X-rays

Generative Artificial Intelligence for Chest Radiograph Interpretation in the Emergency Department

Importance

Multimodal generative artificial intelligence (AI) methodologies have the potential to optimize emergency department care by producing draft radiology reports from input images.

Objective

To evaluate the accuracy and quality of AI-generated chest radiograph interpretations in the emergency department setting.

Design, Setting, and Participants

This was a retrospective diagnostic study of 500 randomly sampled emergency department encounters at a tertiary care institution including chest radiographs interpreted by both a teleradiology service and on-site attending radiologist from January 2022 to January 2023. An AI interpretation was generated for each radiograph. The 3 radiograph interpretations were each rated in duplicate by 6 emergency department physicians using a 5-point Likert scale.

Main Outcomes and Measures

The primary outcome was any difference in Likert scores between radiologist, AI, and teleradiology reports, using a cumulative link mixed model. Secondary analyses compared the probability of each report type containing no clinically significant discrepancy with further stratification by finding presence, using a logistic mixed-effects model. Physician comments on discrepancies were recorded.

Results

A total of 500 ED studies were included from 500 unique patients with a mean (SD) age of 53.3 (21.6) years; 282 patients (56.4%) were female. There was a significant association of report type with ratings, with post hoc tests revealing significantly greater scores for AI (mean [SE] score, 3.22 [0.34]; P < .001) and radiologist (mean [SE] score, 3.34 [0.34]; P < .001) reports compared with teleradiology (mean [SE] score, 2.74 [0.34]) reports. AI and radiologist reports were not significantly different. On secondary analysis, there was no difference in the probability of no clinically significant discrepancy between the 3 report types. Further stratification of reports by presence of cardiomegaly, pulmonary edema, pleural effusion, infiltrate, pneumothorax, and support devices also yielded no difference in the probability of containing no clinically significant discrepancy between the report types.

Conclusions and Relevance

In a representative sample of emergency department chest radiographs, results suggest that the generative AI model produced reports of similar clinical accuracy and textual quality to radiologist reports while providing higher textual quality than teleradiologist reports. Implementation of the model in the clinical workflow could enable timely alerts to life-threatening pathology while aiding imaging interpretation and documentation.

Patient rotation chest X-rays and the consequences of misinterpretation in paediatric radiology

PURPOSE

We aimed to demonstrate the consequences of rotation on neonatal chest radiographs and how it affects diagnosis. In addition, we demonstrate methods for determining the presence and direction of rotation.

BACKGROUND

Patient rotation is common in chest X-rays of neonates. Rotation is present in over half of chest X-rays from the ICU, contributed to by unwillingness of technologists to reposition new-borns for fear of dislodging lines and tubes. There are six main effects of rotation on supine paediatric chest X-rays: 1) unilateral hyperlucency of the side that the patient is rotated towards; 2) the side 'up' appears larger; 3) apparent deviation of the cardiomediastinal shadow in the direction that the chest is rotated towards; 4) apparent cardiomegaly; 5) distorted cardio-mediastinal configuration; and 6) reversed position of the tips of the umbilical artery and vein catheters with rotation to the left. These effects can cause diagnostic errors due to misinterpretation, including air-trapping, atelectasis, cardiomegaly, and pleural effusions, or disease may be masked. We demonstrate the methods of evaluating rotation with examples, including a 3D model of the bony thorax as a guide. In addition, multiple examples of the effects of rotation are provided including examples where disease was misinterpreted, underestimated or masked.

CONCLUSION

Rotation is often unavoidable in neonatal chest X-rays, especially in the ICU. It is therefore important for physicians to recognise rotation and its effects, and to be aware that it can mimic or mask disease.

Comparison of chest radiograph findings in ambulatory and hospitalized children with pulmonary tuberculosis

BACKGROUND

The diagnosis of childhood tuberculosis (TB) is, in many instances, solely reliant on chest radiographs (CXRs), as they are often the only diagnostic tool available, especially in TB-endemic areas. Accuracy and reliability of CXRs for detecting TB lymphadenopathy may vary between groups depending on severity of presentation and presence of parenchymal disease, which may obscure visualization.

OBJECTIVE

To compare CXR findings in ambulatory versus hospitalized children with laboratory confirmed pulmonary TB versus other lower respiratory tract infections (LRTI) and test inter-rater agreement for these findings.

MATERIALS AND METHODS

Retrospective review, by two pediatric radiologists, of CXRs performed on children < 12 years old referred for evaluation of LRTI with clinical suspicion of pulmonary TB in inpatient and outpatient settings. Each radiologist commented on imaging findings of parenchymal changes, lymphadenopathy, airway compression and pleural effusion. Frequency of imaging findings was compared between patients based on location and diagnosis and inter-rater agreement was determined. Accuracy of radiographic diagnosis was compared to laboratory testing which served as the gold standard.

RESULTS

The number of enrolled patients was 181 (54% males); 69 (38%) were ambulatory and 112 (62%) were hospitalized. Of those enrolled, 87 (48%) were confirmed to have pulmonary TB, while 94 (52%) were other LRTI controls. Lymphadenopathy and airway compression were more common in TB patients than other LRTI controls, regardless of patient location. Parenchymal changes and pleural effusion were more common in hospitalized than ambulatory patients, regardless of patient diagnosis. Agreement for parenchymal changes was higher in the hospitalized group (kappa [κ] = 0.75), while agreement for lymphadenopathy (κ = 0.65) and airway compression (κ = 0.68) was higher in the ambulatory group. The specificity of CXRs for TB diagnosis (> 75%) was higher than the sensitivity (< 50%) for both ambulatory and hospitalized groups.

CONCLUSION

Higher frequency of parenchymal changes among hospitalized children may conceal specific imaging findings of TB such as lymphadenopathy, contributing to the poor reliability of CXRs. Despite this, the high specificity of CXRs shown in our results is encouraging for continued use of radiographs for TB diagnosis in both settings.

Biases associated with database structure for COVID-19 detection in X-ray images

Several artificial intelligence algorithms have been developed for COVID-19-related topics. One that has been common is the COVID-19 diagnosis using chest X-rays, where the eagerness to obtain early results has triggered the construction of a series of datasets where bias management has not been thorough from the point of view of patient information, capture conditions, class imbalance, and careless mixtures of multiple datasets. This paper analyses 19 datasets of COVID-19 chest X-ray images, identifying potential biases. Moreover, computational experiments were conducted using one of the most popular datasets in this domain, which obtains a 96.19% of classification accuracy on the complete dataset. Nevertheless, when evaluated with the ethical tool Aequitas, it fails on all the metrics. Ethical tools enhanced with some distribution and image quality considerations are the keys to developing or choosing a dataset with fewer bias issues. We aim to provide broad research on dataset problems, tools, and suggestions for future dataset developments and COVID-19 applications using chest X-ray images.

Quality assurance of paediatric lateral chest radiographs

INTRODUCTION

Lateral chest radiographs aid in paediatric clinical practice in countries where the diagnosis of primary pulmonary tuberculosis (PTB) still relies heavily on the chest radiograph. This study aimed to create a validated quality assurance (QA) tool investigating the diagnostic performance of this projection by applying this to a database of lateral chest radiographs in children with suspected PTB.

METHOD

The QA tool was built to include a compilation of criteria from the different sources, accompanied by graphic representations and objective measurements where appropriate. Each defined criterion (radiographic error) was evaluated by implementing the QA tool on 300 radiographs, scored by three readers. The sample was subjected to two separate sets of data analysis, based on averages, and on majority decision methodology.

RESULTS

The QA tool was based on existing published criteria, as well as under-collimation and under-inspiration, two de novo criteria. For the total 900 reads, errors were categorized as patient-related in 681 (75.7%) and radiographer-related in 421 (46.8%) and 122 (13.6%) had no errors. The average number of errors per radiograph ranged from 0.9 to 4.7 errors out of the 11 quality factors reviewed. When considering the majority decision, the median errors per radiograph was 1 (IQR 1-2) (range 0-5). Inter-rater agreement varied for different criteria.

CONCLUSION

A novel QA tool for evaluating lateral chest radiographs was developed which requires further efforts of refinement regarding criteria such as exposure, field of view: under-collimation, and motion artifact, which remain subjective. The designed QA tool will allow comparison of radiograph quality before and after interventions. Furthermore, the tool can be used in tackling childhood PTB in low- and middle-income countries (LMICs) since the hallmark of the disease is lymphadenopathy, which is often depicted best on lateral chest radiographs.

Magnetic resonance urography: a practical approach to preparation, protocol and interpretation

Magnetic resonance urography (MRU) is an important MRI application that provides noninvasive comprehensive morphological and functional evaluation of the kidneys and urinary tract. It can be used to assess congenital anomalies of the kidney and urinary tract, which often present as urinary tract dilation. In children, MRU allows for high tissue contrast and high spatial resolution without requiring ionizing radiation. Magnetic resonance urography requires patient preparation in the form of pre-examination intravenous hydration, placement of a urinary catheter, and the administration of diuretics at the time of the exam. The imaging protocol is based on T2-weighted images for anatomical assessment and dynamic post-contrast images for functional evaluation. These images are then used to generate quantitative and graphic results including contrast transit and excretion time as well as to calculate differential renal function. This review focuses on a simple approach to pediatric MRU acquisition and interpretation based on clinical cases and the authors' experience.