Artificial intelligence for radiological paediatric fracture assessment: a systematic review

Artificial intelligence improves resident detection of pediatric and young adult upper extremity fractures

PURPOSE

We wished to evaluate if an open-source artificial intelligence (AI) algorithm ( https://www.childfx.com ) could improve performance of (1) subspecialized musculoskeletal radiologists, (2) radiology residents, and (3) pediatric residents in detecting pediatric and young adult upper extremity fractures.

MATERIALS AND METHODS

A set of evaluation radiographs drawn from throughout the upper extremity (elbow, hand/finger, humerus/shoulder/clavicle, wrist/forearm, and clavicle) from 240 unique patients at a single hospital was constructed (mean age 11.3 years, range 0-22 years, 37.9% female). Two fellowship-trained musculoskeletal radiologists, three radiology residents, and two pediatric residents were recruited as readers. Each reader interpreted each case initially without and then subsequently 3-4 weeks later with AI assistance and recorded if/where fracture was present.

RESULTS

Access to AI significantly improved area under the receiver operator curve (AUC) of radiology residents (0.768 [0.730-0.806] without AI to 0.876 [0.845-0.908] with AI, P < 0.001) and pediatric residents (0.706 [0.659-0.753] without AI to 0.844 [0.805-0.883] with AI, P < 0.001) in identifying fracture, respectively. There was no evidence of improvement for subspecialized musculoskeletal radiology attendings in identifying fracture (AUC 0.867 [0.832-0.902] to 0.890 [0.856-0.924], P = 0.093). There was no evidence of difference between overall resident AUC with AI and subspecialist AUC without AI (resident with AI 0.863, attending without AI AUC 0.867, P = 0.856). Overall physician radiograph interpretation time was significantly lower with AI (38.9 s with AI vs. 52.1 s without AI, P = 0.030).

CONCLUSION

An openly accessible AI model significantly improved radiology and pediatric resident accuracy in detecting pediatric upper extremity fractures.

"How I would like AI used for my imaging": children and young persons' perspectives

OBJECTIVES

Artificial intelligence (AI) tools are becoming more available in modern healthcare, particularly in radiology, although less attention has been paid to applications for children and young people. In the development of these, it is critical their views are heard.

MATERIALS AND METHODS

A national, online survey was publicised to UK schools, universities and charity partners encouraging any child or young adult to participate. The survey was "live" for one year (June 2022 to 2023). Questions about views of AI in general, and in specific circumstances (e.g. bone fractures) were asked.

RESULTS

One hundred and seventy-one eligible responses were received, with a mean age of 19 years (6-23 years) with representation across all 4 UK nations. Most respondents agreed or strongly agreed they wanted to know the accuracy of an AI tool that was being used (122/171, 71.3%), that accuracy was more important than speed (113/171, 66.1%), and that AI should be used with human oversight (110/171, 64.3%). Many respondents (73/171, 42.7%) felt AI would be more accurate at finding problems on bone X-rays than humans, with almost all respondents who had sustained a missed fracture strongly agreeing with that sentiment (12/14, 85.7%).

CONCLUSIONS

Children and young people in our survey had positive views regarding AI, and felt it should be integrated into modern healthcare, but expressed a preference for a "medical professional in the loop" and accuracy of findings over speed. Key themes regarding information on AI performance and governance were raised and should be considered prior to future AI implementation for paediatric healthcare.

CLINICAL RELEVANCE STATEMENT

Artificial intelligence (AI) integration into clinical practice must consider all stakeholders, especially paediatric patients who have largely been ignored. Children and young people favour AI involvement with human oversight, seek assurances for safety, accuracy, and clear accountability in case of failures.

KEY POINTS

Paediatric patient's needs and voices are often overlooked in AI tool design and deployment. Children and young people approved of AI, if paired with human oversight and reliability. Children and young people are stakeholders for developing and deploying AI tools in paediatrics.

Evaluating the impact of artificial intelligence-assisted image analysis on the diagnostic accuracy of front-line clinicians in detecting fractures on plain X-rays (FRACT-AI): protocol for a prospective observational study

INTRODUCTION

Missed fractures are the most frequent diagnostic error attributed to clinicians in UK emergency departments and a significant cause of patient morbidity. Recently, advances in computer vision have led to artificial intelligence (AI)-enhanced model developments, which can support clinicians in the detection of fractures. Previous research has shown these models to have promising effects on diagnostic performance, but their impact on the diagnostic accuracy of clinicians in the National Health Service (NHS) setting has not yet been fully evaluated.

METHODS AND ANALYSIS

A dataset of 500 plain radiographs derived from Oxford University Hospitals (OUH) NHS Foundation Trust will be collated to include all bones except the skull, facial bones and cervical spine. The dataset will be split evenly between radiographs showing one or more fractures and those without. The reference ground truth for each image will be established through independent review by two senior musculoskeletal radiologists. A third senior radiologist will resolve disagreements between two primary radiologists. The dataset will be analysed by a commercially available AI tool, BoneView (Gleamer, Paris, France), and its accuracy for detecting fractures will be determined with reference to the ground truth diagnosis. We will undertake a multiple case multiple reader study in which clinicians interpret all images without AI support, then repeat the process with access to AI algorithm output following a 4-week washout. 18 clinicians will be recruited as readers from four hospitals in England, from six distinct clinical groups, each with three levels of seniority (early-stage, mid-stage and later-stage career). Changes in the accuracy, confidence and speed of reporting will be compared with and without AI support. Readers will use a secure web-based DICOM (Digital Imaging and Communications in Medicine) viewer (www.raiqc.com), allowing radiograph viewing and abnormality identification. Pooled analyses will be reported for overall reader performance as well as for subgroups including clinical role, level of seniority, pathological finding and difficulty of image.

ETHICS AND DISSEMINATION

The study has been approved by the UK Healthcare Research Authority (IRAS 310995, approved on 13 December 2022). The use of anonymised retrospective radiographs has been authorised by OUH NHS Foundation Trust. The results will be presented at relevant conferences and published in a peer-reviewed journal.

TRIAL REGISTRATION NUMBERS

This study is registered with ISRCTN (ISRCTN19562541) and ClinicalTrials.gov (NCT06130397). The paper reports the results of a substudy of STEDI2 (Simulation Training for Emergency Department Imaging Phase 2).

Accuracy of wrist fracture detection on radiographs by artificial intelligence compared to human clinicians. A systematic review and meta-analysis

PURPOSE

The aim of the study is to perform a systematic review and meta-analysis comparing the diagnostic performance of artificial intelligence (AI) and human readers in the detection of wrist fractures.

METHOD

This study conducted a systematic review following PRISMA guidelines. Medline and Embase databases were searched for relevant articles published up to August 14, 2023. All included studies reported the diagnostic performance of AI to detect wrist fractures, with or without comparison to human readers. A meta-analysis was performed to calculate the pooled sensitivity and specificity of AI and human experts in detecting distal radius, and scaphoid fractures respectively.

RESULTS

Of 213 identified records, 20 studies were included after abstract screening and full-text review. Nine articles examined distal radius fractures, while eight studies examined scaphoid fractures. One study included distal radius and scaphoid fractures, and two studies examined paediatric distal radius fractures. The pooled sensitivity and specificity for AI in detecting distal radius fractures were 0.92 (95% CI 0.88-0.95) and 0.89 (0.84-0.92), respectively. The corresponding values for human readers were 0.95 (0.91-0.97) and 0.94 (0.91-0.96). For scaphoid fractures, pooled sensitivity and specificity for AI were 0.85 (0.73-0.92) and 0.83 (0.76-0.89), while human experts exhibited 0.71 (0.66-0.76) and 0.93 (0.90-0.95), respectively.

CONCLUSION

The results indicate comparable diagnostic accuracy between AI and human readers, especially for distal radius fractures. For the detection of scaphoid fractures, the human readers were similarly sensitive but more specific. These findings underscore the potential of AI to enhance fracture detection accuracy and improve clinical workflow, rather than to replace human intelligence.

Diagnostic performance of an AI algorithm for the detection of appendicular bone fractures in pediatric patients

PURPOSE

To evaluate the diagnostic performance of an Artificial Intelligence (AI) algorithm, previously trained using both adult and pediatric patients, for the detection of acute appendicular fractures in the pediatric population on conventional X-ray radiography (CXR).

MATERIALS AND METHODS

In this retrospective study, anonymized extremities CXRs of pediatric patients (age <17 years), with or without fractures, were included. Six hundred CXRs (maintaining the positive-for-fracture and negative-for-fracture balance) were included, grouping them per body part (shoulder/clavicle, elbow/upper arm, hand/wrist, leg/knee, foot/ankle). Follow-up CXRs and/or second-level imaging were considered as reference standard. A deep learning algorithm interpreted CXRs for fracture detection on a per-patient, per-radiograph, and per-location level, and its diagnostic performance values were compared with the reference standard. AI diagnostic performance was computed by using cross-tables, and 95 % confidence intervals [CIs] were obtained by bootstrapping.

RESULTS

The final cohort included 312 male and 288 female with a mean age of 8.9±4.5 years. Three undred CXRs (50 %) were positive for fractures, according to the reference standard. For all fractures, the AI tool showed a per-patient 91.3% (95%CIs = 87.6-94.3) sensitivity, 76.7% (71.5-81.3) specificity, and 84% (82.1-86.0) accuracy. In the per-radiograph analysis the AI tool showed 85% (81.9-87.8) sensitivity, 88.5% (86.3-90.4) specificity, and 87.2% (85.7-89.6) accuracy. In the per-location analysis, the AI tool identified 606 bounding boxes: 472 (77.9 %) were correct, 110 (18.1 %) incorrect, and 24 (4.0 %) were not-overlapping.

CONCLUSION

The AI algorithm provides good overall diagnostic performance for detecting appendicular fractures in pediatric patients.

Detecting pediatric appendicular fractures using artificial intelligence

OBJECTIVE

The primary objective was to assess the diagnostic accuracy of a deep learning-based artificial intelligence model for the detection of acute appendicular fractures in pediatric patients presenting with a recent history of trauma to the emergency department. The secondary goal was to examine the effect of assistive support on the emergency doctor's ability to detect fractures.

METHODS

The dataset was 5,150 radiographs of which 850 showed fractures, while 4,300 radiographs did not show any fractures. The process utilized 4,532 (88%) radiographs, inclusive of both fractured and non-fractured radiographs, in the training phase. Subsequently, 412 (8%) radiographs were appraised during validation, and 206 (4%) were set apart for the testing phase. With and without artificial intelligence assistance, the emergency doctor reviewed another set of 2,000 radiographs (400 fractures and 600 non-fractures each) for labeling in the second test.

RESULTS

The artificial intelligence model showed a mean average precision 50 of 89%, a specificity of 92%, a sensitivity of 90%, and an F1 score of 90%. The confusion matrix revealed that the model trained with artificial intelligence achieved accuracies of 93 and 95% in detecting fractures, respectively. Artificial intelligence assistance improved the reading sensitivity from 93.7% (without assistance) to 97.0% (with assistance) and the reading accuracy from 88% (without assistance) to 94.9% (with assistance).

CONCLUSION

A deep learning-based artificial intelligence model has proven to be highly effective in detecting fractures in pediatric patients, enhancing the diagnostic capabilities of emergency doctors through assistive support.

Artificial Intelligence Application in Skull Bone Fracture with Segmentation Approach

This study aims to evaluate an AI model designed to automatically classify skull fractures and visualize segmentation on emergent CT scans. The model's goal is to boost diagnostic accuracy, alleviate radiologists' workload, and hasten diagnosis, thereby enhancing patient outcomes. Unique to this research, both pediatric and post-operative patients were not excluded, and diagnostic durations were analyzed. Our testing dataset for the observer studies involved 671 patients, with a mean age of 58.88 years and fairly balanced gender representation. Model 1 of our AI algorithm, trained with 1499 fracture-positive cases, showed a sensitivity of 0.94 and specificity of 0.87, with a DICE score of 0.65. Implementing post-processing rules (specifically Rule B) improved the model's performance, resulting in a sensitivity of 0.94, specificity of 0.99, and a DICE score of 0.63. AI-assisted diagnosis resulted in significantly enhanced performance for all participants, with sensitivity almost doubling for junior radiology residents and other specialists. Additionally, diagnostic durations were significantly reduced (p < 0.01) with AI assistance across all participant categories. Our skull fracture detection model, employing a segmentation approach, demonstrated high performance, enhancing diagnostic accuracy and efficiency for radiologists and clinical physicians. This underlines the potential of AI integration in medical imaging analysis to improve patient care.

Diagnostic Accuracy of Artificial Intelligence-Based Algorithms in Automated Detection of Neck of Femur Fracture on a Plain Radiograph: A Systematic Review and Meta-analysis

Objectives

To evaluate the diagnostic accuracy of artificial intelligence-based algorithms in identifying neck of femur fracture on a plain radiograph.

Design

Systematic review and meta-analysis.

Data sources

PubMed, Web of science, Scopus, IEEE, and the Science direct databases were searched from inception to 30 July 2023.

Eligibility criteria for study selection

Eligible article types were descriptive, analytical, or trial studies published in the English language providing data on the utility of artificial intelligence (AI) based algorithms in the detection of the neck of the femur (NOF) fracture on plain X-ray.

Main outcome measures

The prespecified primary outcome was to calculate the sensitivity, specificity, accuracy, Youden index, and positive and negative likelihood ratios. Two teams of reviewers (each consisting of two members) extracted the data from available information in each study. The risk of bias was assessed using a mix of the CLAIM (the Checklist for AI in Medical Imaging) and QUADAS-2 (A Revised Tool for the Quality Assessment of Diagnostic Accuracy Studies) criteria.

Results

Of the 437 articles retrieved, five were eligible for inclusion, and the pooled sensitivity of AIs in diagnosing the fracture NOF was 85%, with a specificity of 87%. For all studies, the pooled Youden index (YI) was 0.73. The average positive likelihood ratio (PLR) was 19.88, whereas the negative likelihood ratio (NLR) was 0.17. The random effects model showed an overall odds of 1.16 (0.84-1.61) in the forest plot, comparing the AI system with those of human diagnosis. The overall heterogeneity of the studies was marginal (I2 = 51%). The CLAIM criteria for risk of bias assessment had an overall >70% score.

Conclusion

Artificial intelligence (AI)-based algorithms can be used as a diagnostic adjunct, benefiting clinicians by taking less time and effort in neck of the femur (NOF) fracture diagnosis.

Study registration

PROSPERO CRD42022375449.

Supplementary Information

The online version contains supplementary material available at 10.1007/s43465-024-01130-6.

Bowing fractures of the forearm in children: pathophysiology, diagnosis and management

Bowing fractures of the forearm are characterised by numerous micro-fractures on the concave surface of the affected bone(s), usually caused by a fall on an outstretched arm. Children are more susceptible to this type of injury than adults because their long bones have more elasticity. Bowing fractures of the forearm are challenging to diagnose because there are no obvious cortical defects, which can lead to inappropriate management and associated complications, including loss of movement range and loss of function. This article discusses bowing fractures of the forearm in children, including their pathophysiology, diagnosis and management. It aims to enhance emergency nurses' awareness and knowledge of this type of injury in children and of the challenges surrounding diagnosis and management.

A review on artificial intelligence for the diagnosis of fractures in facial trauma imaging

Patients with facial trauma may suffer from injuries such as broken bones, bleeding, swelling, bruising, lacerations, burns, and deformity in the face. Common causes of facial-bone fractures are the results of road accidents, violence, and sports injuries. Surgery is needed if the trauma patient would be deprived of normal functioning or subject to facial deformity based on findings from radiology. Although the image reading by radiologists is useful for evaluating suspected facial fractures, there are certain challenges in human-based diagnostics. Artificial intelligence (AI) is making a quantum leap in radiology, producing significant improvements of reports and workflows. Here, an updated literature review is presented on the impact of AI in facial trauma with a special reference to fracture detection in radiology. The purpose is to gain insights into the current development and demand for future research in facial trauma. This review also discusses limitations to be overcome and current important issues for investigation in order to make AI applications to the trauma more effective and realistic in practical settings. The publications selected for review were based on their clinical significance, journal metrics, and journal indexing.

Artificial intelligence-based detection of paediatric appendicular skeletal fractures: performance and limitations for common fracture types and locations

BACKGROUND

Research into artificial intelligence (AI)-based fracture detection in children is scarce and has disregarded the detection of indirect fracture signs and dislocations.

OBJECTIVE

To assess the diagnostic accuracy of an existing AI-tool for the detection of fractures, indirect fracture signs, and dislocations.

MATERIALS AND METHODS

An AI software, BoneView (Gleamer, Paris, France), was assessed for diagnostic accuracy of fracture detection using paediatric radiology consensus diagnoses as reference. Radiographs from a single emergency department were enrolled retrospectively going back from December 2021, limited to 1,000 radiographs per body part. Enrolment criteria were as follows: suspected fractures of the forearm, lower leg, or elbow; age 0-18 years; and radiographs in at least two projections.

RESULTS

Lower leg radiographs showed 607 fractures. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were high (87.5%, 87.5%, 98.3%, 98.3%, respectively). Detection rate was low for toddler's fractures, trampoline fractures, and proximal tibial Salter-Harris-II fractures. Forearm radiographs showed 1,137 fractures. Sensitivity, specificity, PPV, and NPV were high (92.9%, 98.1%, 98.4%, 91.7%, respectively). Radial and ulnar bowing fractures were not reliably detected (one out of 11 radial bowing fractures and zero out of seven ulnar bowing fractures were correctly detected). Detection rate was low for styloid process avulsions, proximal radial buckle, and complete olecranon fractures. Elbow radiographs showed 517 fractures. Sensitivity and NPV were moderate (80.5%, 84.7%, respectively). Specificity and PPV were high (94.9%, 93.3%, respectively). For joint effusion, sensitivity, specificity, PPV, and NPV were moderate (85.1%, 85.7%, 89.5%, 80%, respectively). For elbow dislocations, sensitivity and PPV were low (65.8%, 50%, respectively). Specificity and NPV were high (97.7%, 98.8%, respectively).

CONCLUSIONS

The diagnostic performance of BoneView is promising for forearm and lower leg fractures. However, improvement is mandatory before clinicians can rely solely on AI-based paediatric fracture detection using this software.

Commercially available artificial intelligence tools for fracture detection: the evidence

Missed fractures are a costly healthcare issue, not only negatively impacting patient lives, leading to potential long-term disability and time off work, but also responsible for high medicolegal disbursements that could otherwise be used to improve other healthcare services. When fractures are overlooked in children, they are particularly concerning as opportunities for safeguarding may be missed. Assistance from artificial intelligence (AI) in interpreting medical images may offer a possible solution for improving patient care, and several commercial AI tools are now available for radiology workflow implementation. However, information regarding their development, evidence for performance and validation as well as the intended target population is not always clear, but vital when evaluating a potential AI solution for implementation. In this article, we review the range of available products utilizing AI for fracture detection (in both adults and children) and summarize the evidence, or lack thereof, behind their performance. This will allow others to make better informed decisions when deciding which product to procure for their specific clinical requirements.

Artificial intelligence to identify fractures on pediatric and young adult upper extremity radiographs

BACKGROUND

Pediatric fractures are challenging to identify given the different response of the pediatric skeleton to injury compared to adults, and most artificial intelligence (AI) fracture detection work has focused on adults.

OBJECTIVE

Develop and transparently share an AI model capable of detecting a range of pediatric upper extremity fractures.

MATERIALS AND METHODS

In total, 58,846 upper extremity radiographs (finger/hand, wrist/forearm, elbow, humerus, shoulder/clavicle) from 14,873 pediatric and young adult patients were divided into train (n = 12,232 patients), tune (n = 1,307), internal test (n = 819), and external test (n = 515) splits. Fracture was determined by manual inspection of all test radiographs and the subset of train/tune radiographs whose reports were classified fracture-positive by a rule-based natural language processing (NLP) algorithm. We trained an object detection model (Faster Region-based Convolutional Neural Network [R-CNN]; "strongly-supervised") and an image classification model (EfficientNetV2-Small; "weakly-supervised") to detect fractures using train/tune data and evaluate on test data. AI fracture detection accuracy was compared with accuracy of on-call residents on cases they preliminarily interpreted overnight.

RESULTS

A strongly-supervised fracture detection AI model achieved overall test area under the receiver operating characteristic curve (AUC) of 0.96 (95% CI 0.95-0.97), accuracy 89.7% (95% CI 88.0-91.3%), sensitivity 90.8% (95% CI 88.5-93.1%), and specificity 88.7% (95% CI 86.4-91.0%), and outperformed a weakly-supervised model (AUC 0.93, 95% CI 0.92-0.94, P < 0.0001). AI accuracy on cases preliminary interpreted overnight was higher than resident accuracy (AI 89.4% vs. 85.1%, 95% CI 87.3-91.5% vs. 82.7-87.5%, P = 0.01).

CONCLUSION

An object detection AI model identified pediatric upper extremity fractures with high accuracy.

Developing an artificial intelligence diagnostic tool for paediatric distal radius fractures, a proof of concept study

INTRODUCTION

In the UK, 1 in 50 children sustain a fractured bone yearly, yet studies have shown that 34% of children sustaining an injury do not have a visible fracture on initial radiographs. Wrist fractures are particularly difficult to identify because the growth plate poses diagnostic challenges when interpreting radiographs.

METHODS

We developed Convolutional Neural Network (CNN) image recognition software to detect fractures in radiographs from children. A consecutive data set of 5,000 radiographs of the distal radius in children aged less than 19 years from 2014 to 2019 was used to train the CNN. In addition, transfer learning from a VGG16 CNN pretrained on non-radiological images was applied to improve generalisation of the network and the classification of radiographs. Hyperparameter tuning techniques were used to compare the model with the radiology reports that accompanied the original images to determine diagnostic test accuracy.

RESULTS

The training set consisted of 2,881 radiographs with a fracture and 1,571 without; 548 radiographs were outliers. With additional augmentation, the final data set consisted of 15,498 images. The data set was randomly split into three subsets: training (70%), validation (10%) and test (20%). After training for 20 epochs, the diagnostic test accuracy was 85%.

DISCUSSION

A CNN model is feasible in diagnosing paediatric wrist fractures. We demonstrated that this application could be utilised as a tool for improving diagnostic accuracy. Future work would involve developing automated treatment pathways for diagnosis, reducing unnecessary hospital visits and allowing staff redeployment to other areas.

Elbow trauma in children: development and evaluation of radiological artificial intelligence models

Rationale and Objectives

To develop a model using artificial intelligence (A.I.) able to detect post-traumatic injuries on pediatric elbow X-rays then to evaluate its performances in silico and its impact on radiologists' interpretation in clinical practice.

Material and Methods

A total of 1956 pediatric elbow radiographs performed following a trauma were retrospectively collected from 935 patients aged between 0 and 18 years. Deep convolutional neural networks were trained on these X-rays. The two best models were selected then evaluated on an external test set involving 120 patients, whose X-rays were performed on a different radiological equipment in another time period. Eight radiologists interpreted this external test set without then with the help of the A.I. models .

Results

Two models stood out: model 1 had an accuracy of 95.8% and an AUROC of 0.983 and model 2 had an accuracy of 90.5% and an AUROC of 0.975. On the external test set, model 1 kept a good accuracy of 82.5% and AUROC of 0.916 while model 2 had a loss of accuracy down to 69.2% and of AUROC to 0.793. Model 1 significantly improved radiologist's sensitivity (0.82 to 0.88, P = 0.016) and accuracy (0.86 to 0.88, P = 0,047) while model 2 significantly decreased specificity of readers (0.86 to 0.83, P = 0.031).

Conclusion

End-to-end development of a deep learning model to assess post-traumatic injuries on elbow X-ray in children was feasible and showed that models with close metrics in silico can unpredictably lead radiologists to either improve or lower their performances in clinical settings.

Transfer learning-based ensemble convolutional neural network for accelerated diagnosis of foot fractures

The complex shape of the foot, consisting of 26 bones, variable ligaments, tendons, and muscles leads to misdiagnosis of foot fractures. Despite the introduction of artificial intelligence (AI) to diagnose fractures, the accuracy of foot fracture diagnosis is lower than that of conventional methods. We developed an AI assistant system that assists with consistent diagnosis and helps interns or non-experts improve their diagnosis of foot fractures, and compared the effectiveness of the AI assistance on various groups with different proficiency. Contrast-limited adaptive histogram equalization was used to improve the visibility of original radiographs and data augmentation was applied to prevent overfitting. Preprocessed radiographs were fed to an ensemble model of a transfer learning-based convolutional neural network (CNN) that was developed for foot fracture detection with three models: InceptionResNetV2, MobilenetV1, and ResNet152V2. After training the model, score class activation mapping was applied to visualize the fracture based on the model prediction. The prediction result was evaluated by the receiver operating characteristic (ROC) curve and its area under the curve (AUC), and the F1-Score. Regarding the test set, the ensemble model exhibited better classification ability (F1-Score: 0.837, AUC: 0.95, Accuracy: 86.1%) than other single models that showed an accuracy of 82.4%. With AI assistance for the orthopedic fellow, resident, intern, and student group, the accuracy of each group improved by 3.75%, 7.25%, 6.25%, and 7% respectively and diagnosis time was reduced by 21.9%, 14.7%, 24.4%, and 34.6% respectively.

Assessment of a combined musculoskeletal and chest deep learning-based detection solution in an emergency setting

Rationale and objectives

Triage and diagnostic deep learning-based support solutions have started to take hold in everyday emergency radiology practice with the hope of alleviating workflows. Although previous works had proven that artificial intelligence (AI) may increase radiologist and/or emergency physician reading performances, they were restricted to finding, bodypart and/or age subgroups, without evaluating a routine emergency workflow composed of chest and musculoskeletal adult and pediatric cases. We aimed at evaluating a multiple musculoskeletal and chest radiographic findings deep learning-based commercial solution on an adult and pediatric emergency workflow, focusing on discrepancies between emergency and radiology physicians.

Material and methods

This retrospective, monocentric and observational study included 1772 patients who underwent an emergency radiograph between July and October 2020, excluding spine, skull and plain abdomen procedures. Emergency and radiology reports, obtained without AI as part of the clinical workflow, were collected and discordant cases were reviewed to obtain the radiology reference standard. Case-level AI outputs and emergency reports were compared to the reference standard. DeLong and Wald tests were used to compare ROC-AUC and Sensitivity/Specificity, respectively.

Results

Results showed an overall AI ROC-AUC of 0.954 with no difference across age or body part subgroups. Real-life emergency physicians' sensitivity was 93.7 %, not significantly different to the AI model (P = 0.105), however in 172/1772 (9.7 %) cases misdiagnosed by emergency physicians. In this subset, AI accuracy was 90.1 %.

Conclusion

This study highlighted that multiple findings AI solution for emergency radiographs is efficient and complementary to emergency physicians, and could help reduce misdiagnosis in the absence of immediate radiological expertize.

Assessment of an artificial intelligence aid for the detection of appendicular skeletal fractures in children and young adults by senior and junior radiologists

BACKGROUND

As the number of conventional radiographic examinations in pediatric emergency departments increases, so, too, does the number of reading errors by radiologists.

OBJECTIVE

The aim of this study is to investigate the ability of artificial intelligence (AI) to improve the detection of fractures by radiologists in children and young adults.

MATERIALS AND METHODS

A cohort of 300 anonymized radiographs performed for the detection of appendicular fractures in patients ages 2 to 21 years was collected retrospectively. The ground truth for each examination was established after an independent review by two radiologists with expertise in musculoskeletal imaging. Discrepancies were resolved by consensus with a third radiologist. Half of the 300 examinations showed at least 1 fracture. Radiographs were read by three senior pediatric radiologists and five radiology residents in the usual manner and then read again immediately after with the help of AI.

RESULTS

The mean sensitivity for all groups was 73.3% (110/150) without AI; it increased significantly by almost 10% (P<0.001) to 82.8% (125/150) with AI. For junior radiologists, it increased by 10.3% (P<0.001) and for senior radiologists by 8.2% (P=0.08). On average, there was no significant change in specificity (from 89.6% to 90.3% [+0.7%, P=0.28]); for junior radiologists, specificity increased from 86.2% to 87.6% (+1.4%, P=0.42) and for senior radiologists, it decreased from 95.1% to 94.9% (-0.2%, P=0.23). The stand-alone sensitivity and specificity of the AI were, respectively, 91% and 90%.

CONCLUSION

With the help of AI, sensitivity increased by an average of 10% without significantly decreasing specificity in fracture detection in a predominantly pediatric population.