Interobserver Variability of Hip Dysplasia Indices on Sweep Ultrasound for Novices, Experts, and Artificial Intelligence

BACKGROUND

Ultrasound for developmental dysplasia of the hip (DDH) is challenging for nonexperts to perform and interpret. Recording "sweep" images allows more complete hip assessment, suitable for automation by artificial intelligence (AI), but reliability has not been established. We assessed agreement between readers of varying experience and a commercial AI algorithm, in DDH detection from infant hip ultrasound sweeps.

METHODS

We selected a full spectrum of poor-to-excellent quality images and normal to severe dysplasia, in 240 hips (120 single 2-dimensional images, 120 sweeps). For 12 readers (radiologists, sonographers, clinicians and researchers; 3 were DDH subspecialists), and a ultrasound-FDA-cleared AI software package (Medo Hip), we calculated interobserver reliability for alpha angle measurements by intraclass correlation coefficient (ICC2,1) and for DDH classification by Randolph Kappa.

RESULTS

Alpha angle reliability was high for AI versus subspecialists (ICC=0.87 for sweeps, 0.90 for single images). For DDH diagnosis from sweeps, agreement was high between subspecialists (kappa=0.72), and moderate for nonsubspecialists (0.54) and AI (0.47). Agreement was higher for single images (kappa=0.80, 0.66, 0.49). AI reliability deteriorated more than human readers for the poorest-quality images. The agreement of radiologists and clinicians with the accepted standard, while still high, was significantly poorer for sweeps than 2D images (P<0.05).

CONCLUSIONS

In a challenging exercise representing the wide spectrum of image quality and reader experience seen in real-world hip ultrasound, agreement on DDH diagnosis from easily obtained sweeps was only slightly lower than from single images, likely because of the additional step of selecting the best image. AI performed similarly to a nonsubspecialist human reader but was more affected by low-quality images.

Surveillance Practice for Sonographic Detection of Intracranial Abnormalities in Premature Neonates: A Snapshot of Current Neonatal Cranial Ultrasound Practice in Australia

There are no publications reporting on scan duration and Doppler use during neonatal cranial ultrasound scans. We investigated current practice of neonatal cranial ultrasound at four large tertiary neonatal intensive care units in Australia. Cranial scans were prospectively recorded between March 2015 and November 2016. Variables, including total number of scans, scan duration and frequency and duration of colour and spectral Doppler mode, were extracted. A total of 196 scans formed the final cohort. The median (range) number of scans for each neonate was 1 (1-12). The median (range) overall total scan duration was 309 (119-801) s. Colour mode with or without spectral Doppler mode was used in approximately half of the cohort (106/196, 54%). Our findings comport with our hypotheses. Operators performing neonatal cranial scans in Australia have low overall scan durations. Although the use of Doppler mode during neonatal cranial scans is not standard practice in all neonatal intensive care units, it is used widely irrespective of the degree of prematurity or the presence of brain pathology. Further efforts are required to incorporate recommendations on scan duration and the routine use of Doppler mode during neonatal cranial scans. This is especially imperative given that the most vulnerable neonates with the greater neural tissue sensitivity are likely to be scanned more often.

Knowledge of Safety, Training, and Practice of Neonatal Cranial Ultrasound: A Survey of Operators

OBJECTIVES

Ultrasound can lead to thermal and mechanical effects in interrogated tissues. This possibility suggests a potential risk during neonatal cranial ultrasound examinations. The aim of this study was to explore safety knowledge and training of neonatal cranial ultrasound among Australian operators who routinely perform these scans.

METHODS

An online survey was administered on biosafety and training in neonatal cranial ultrasound, targeting all relevant professionals who can perform neonatal cranial ultrasound examinations in Australia: namely, radiologists, neonatologists, sonographers, and pediatricians. The survey was conducted between November 2013 and May 2014.

RESULTS

A total of 282 responses were received. Twenty of 208 (10%) answered all ultrasound biosafety questions correctly, and 49 of 169 (29%) correctly defined the thermal index. Two-thirds (134 of 214 [63%]) of respondents failed to recognize that reducing the overall scanning time is the most effective method of reducing the total power exposure. Only 13% (31 of 237) indicated that a predetermined fixed period of training or that a specified minimum number of supervised scans was used during training. The reported number of supervised scans during training was highly variable. Almost half of the participants (82 of 181 [45%]) stated that they had received supervision for 10 to 50 scans (median, 20 scans).

CONCLUSIONS

There is a need to educate operators on biosafety issues and approaches to minimize power outputs and reduce the overall duration of cranial ultrasound scans. Development of standardized training requirements may be warranted.