Intraobserver and interobserver reliability in assessment of neonatal cranial ultrasounds

Interobserver reliability between pediatric radiologists and residents in ultrasound evaluation of intraventricular hemorrhage in premature infants

BACKGROUND

Germinal matrix intraventricular hemorrhage (IVH) may contribute to significant morbidity and mortality in premature infants. Timely identification and grading of IVH affect decision-making and clinical outcomes. There is possibility of misinterpretation of the ultrasound appearances, and the interobserver variability has not been investigated between radiology resident and board-certified radiologist.

AIM

To assess interobserver reliability between senior radiology residents performing bedside cranial ultrasound during on-call hours and pediatric radiologists.

METHODS

From June 2018 to June 2020, neonatal cranial ultrasound examinations were performed in neonatal intensive care unit. Ultrasound findings were recorded by the residents performing the ultrasound and the pediatric attending radiologists.

RESULTS

In total, 200 neonates were included in the study, with a mean gestational age of 30.9 wk. Interobserver agreement for higher grade (Grade III & IV) IVH was excellent. There was substantial agreement for lower grade (Grade I & II) IVH.

CONCLUSION

There is strong agreement between radiology residents and pediatric radiologists, which is higher for high grade IVHs.

Additional value of advanced neurosonography and magnetic resonance imaging in fetuses at risk for brain damage

OBJECTIVE

To assess the additional value of fetal multiplanar (axial, coronal and sagittal) neurosonography and magnetic resonance imaging (MRI) to that of the standard axial ultrasound planes in diagnosing brain damage in fetuses at high risk.

METHODS

This was a prospective, multicenter, observational study. Women were eligible for participation if their fetus was at risk for acquired brain anomalies. Risk factors were congenital infection, alloimmune thrombocytopenia, fetal growth restriction, trauma during pregnancy, fetal hydrops, monochorionic twins and prior ultrasound finding suggestive of an acquired brain anomaly. Examinations of the fetal brain before birth comprised axial ultrasound and advanced neurosonography biweekly and MRI once. After birth, neonatal cranial ultrasound was performed at < 24 h and at term-equivalent age. Neonatal brain MRI was performed once at term-equivalent age. An expert panel blinded to medical information, including imaging findings by the other methods, evaluated the presence of periventricular echogenicity (PVE) changes, peri- and intraventricular hemorrhage (IVH) and changes in basal ganglia and/or thalami echogenicity (BGTE) on ultrasound, and the equivalent signal intensity (SI) changes on MRI. Conclusions on imaging findings were generated by consensus. The children were followed up with examinations for psychomotor development at 1 year of age, using the Touwen examination and Alberta Infant Motor Scale, and at 2 years of age using Bayley Scale of Infant Development-III (BSID-III) and behavioral, sensory profile and linguistic questionnaires; scores > 1 SD below the mean were considered suspicious for neurodevelopmental sequelae.

RESULTS

Fifty-six fetuses were examined, and in 39/56 fetuses, all fetal-imaging modalities were available. PVE/SI changes were observed in 6/39, 21/39 and 2/39 fetuses on axial ultrasound planes, multiplanar neurosonography and MRI, respectively. IVH was found in 3/39, 11/39 and 1/39 fetuses, and BGTE/SI changes in 0/39, 12/39 and 0/39 fetuses, respectively. Outcome was suspicious for neurodevelopmental sequelae in 13/46 infants at 1 year, and at 2 years, 41/41 children had scores within 1 SD of the mean on BSID-III and 20 had scores > 1 SD below the mean on the behavioral (5/38), sensory profile (17/37) and/or linguistic (6/39) questionnaires.

CONCLUSIONS

In this cohort of fetuses at risk for brain damage, the severity of acquired brain anomalies was limited. Nevertheless, multiplanar neurosonography detected more fetal PVE changes, IVH and/or BGTE changes compared to the standard axial ultrasound planes and MRI. Fetal MRI did not demonstrate any anomalies that were not seen on neurosonography. Neurodevelopmental outcome at 2 years of age showed no or mild impairment in most cases. © 2019 Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of International Society of Ultrasound in Obstetrics and Gynecology.

The most useful cranial ultrasound predictor of neurodevelopmental outcome at 2 years for preterm infants

AIM

To determine the most important cranial ultrasound predictors of abnormality associated with neurodevelopmental outcome at 2 years of age in preterm infants.

MATERIALS AND METHODS

A total of 343 preterm infants born between 2005 and 2010 and cared for in KK Women's and Children's Hospital, a tertiary paediatric hospital, with birth weight ≤1,250 g were assessed in this retrospective study. Serial cranial ultrasound examinations were examined for intraventricular haemorrhage and cystic periventricular leukomalacia. Ventricular-brain ratio on term equivalent cranial ultrasound was measured. Neurodevelopmental outcome was assessed by the performance on Bayley Scales of Infant and Toddler Development, 3^rd edition (Bayley-III) at 2 years corrected age. Mental delay was defined as having a combined Bayley-III score (the average of cognitive and language scores) <80.

RESULTS

The mean cognitive, language, and motor scores on Bayley-III in this cohort were 93±15, 83±18, and 92±15, respectively. Twenty-six percent of the preterm infants had mental delay and 4% had cerebral palsy. Ventricular-brain ratio >0.35 was the most significant factor associated with mental delay (odds ratio 5.28, 95% CI: 1.49-18.71, p=0.01). Other significant risk factors for mental delay were male gender, postnatal steroids, and necrotising enterocolitis, whereas maternal tertiary education was a protective factor against adverse outcome.

CONCLUSION

Ventricular-brain ratio >0.35 on term-equivalent cranial ultrasound in preterm infants is the strongest predictor for mental delay on Bayley score at 2 years of age.

Added value of double reading in diagnostic radiology,a systematic review

Objectives

Double reading in diagnostic radiology can find discrepancies in the original report, but a systematic program of double reading is resource consuming. There are conflicting opinions on the value of double reading. The purpose of the current study was to perform a systematic review on the value of double reading.

Methods

A systematic review was performed to find studies calculating the rate of misses and overcalls with the aim of establishing the added value of double reading by human observers.

Results

The literature search resulted in 1610 hits. After abstract and full-text reading, 46 articles were selected for analysis. The rate of discrepancy varied from 0.4 to 22% depending on study setting. Double reading by a sub-specialist, in general, led to high rates of changed reports.

Conclusions

The systematic review found rather low discrepancy rates. The benefit of double reading must be balanced by the considerable number of working hours a systematic double-reading scheme requires. A more profitable scheme might be to use systematic double reading for selected, high-risk examination types. A second conclusion is that there seems to be a value of sub-specialisation for increased report quality. A consequent implementation of this would have far-reaching organisational effects.

Key Points

• In double reading, two or more radiologists read the same images.

• A systematic literature review was performed.

• The discrepancy rates varied from 0.4 to 22% in various studies.

• Double reading by sub-specialists found high discrepancy rates.

Electronic supplementary material

The online version of this article (10.1007/s13244-018-0599-0) contains supplementary material, which is available to authorised users.

Severe intraventricular hemorrhage and withdrawal of support in preterm infants

OBJECTIVE

The objective of the study was to determine whether withdrawal of support in severe 'intraventricular hemorrhage' (IVH), that is, IVH grade 3 and periventricular hemorrhagic infarction (PVHI), has decreased after publication of studies that show improved prognosis and to examine cranial ultrasonograms, including PVHI territories defined by Bassan.

STUDY DESIGN

Retrospective cohort of preterm infants from 23 0/7 to 28 6/7 weeks' gestation in 1993 to 2013.

RESULTS

Among the 1755 infants, 1494 had no bleed, germinal matrix hemorrhage (GMH) or IVH grade 2, 137 had grade 3 IVH and 124 had PVHI. The odds of withdrawal of support, adjusted for severity of GMH-IVH and baseline variables, did not decrease after publications showing better prognosis. Among 82 patients who died with PVHI, 76 had life support withdrawn, including 34 without another contributing cause of death. The median number of PVHI territories involved was three.

CONCLUSION

Withdrawal of support adjusted for severity of GMH-IVH did not significantly change after publications showing better prognosis.

Ultrasound measurements of the lateral ventricles in neonates: why, how and when? A systematic review

Germinal matrix-intraventricular haemorrhage and subsequent post-haemorrhagic ventricular dilatation (PHVD) are frequently encountered complications in preterm neonates. As progressive dilatation of the lateral ventricles may be associated with elevated intracranial pressure, ultrasound measurements of ventricular size play a major role in the evaluation of neonates at risk of ventricular dilatation as well as in assessing the effect of intervention for PHVD. A systematic search was carried out in Medline and Embase to identify neonatal and foetal ultrasound studies on lateral ventricular size. This review presents an overview of the available data concerning neonatal reference values for lateral ventricular size, the influence of gender, ventricular asymmetry and the effect of the mode of delivery on the phenomenon of ventricular reopening following birth.

CONCLUSION

Serial cranial ultrasound measurements of the lateral ventricles play a key role in the early recognition and therapeutic evaluation of post-haemorrhagic ventricular dilation and can be of prognostic value in neonates with ventricular dilatation.

Neuroimaging and neurodevelopmental outcomes in preterm infants

Imaging of the preterm infant brain has advanced dramatically beyond the earliest era of transillumination. Computed tomography (CT), a crucial innovation during the early 1970s, allowed noninvasive visualization of intracerebral lesions, particularly hemorrhage. The capability to document brain injury in the preterm infant led to better clarification of links to developmental outcomes. With the development of cranial ultrasound (CUS), and more recently, magnetic resonance imaging (MRI), CT is used rarely for imaging the brain of preterm infants. Despite extensive experience with neonatal neuroimaging, significant questions still remain. Substantial controversies exist pertaining to when and how neuroimaging should be performed and how images should be interpreted.

Interobserver reliability and accuracy of cranial ultrasound scanning interpretation in premature infants

OBJECTIVE

To assess interobserver reliability between 2 central readers of cranial ultrasound scanning (CUS) and accuracy of local, compared with central, interpretations.

STUDY DESIGN

The study was a retrospective analysis of CUS data from the National Institute of Child Health and Human Development (NICHD) trial of inhaled nitric oxide for premature infants. Interobserver reliability of 2 central readers was assessed with kappa or weighted kappa. Accuracy of local, compared with central, interpretations was assessed by using sensitivity and specificity.

RESULTS

CUS from 326 infants had both central reader and local interpretations. Central reader agreement for grade 3/4 intraventricular hemorrhage (IVH), grade 3/4 IVH or periventricular leukomalacia (PVL), grade of IVH, and degree of ventriculomegaly was very good (kappa = 0.84, 0.81, 0.79, and 0.75, respectively). Agreement was poor for lower grade IVH and for PVL alone. Local interpretations were highly accurate for grade 3/4 IVH or PVL (sensitivity, 87%-90%; specificity, 92%-93%), but sensitivity was poor-to-fair for grade 1/2 IVH (48%-68%) and PVL (20%-44%).

CONCLUSIONS

Our findings demonstrate reliability and accuracy of highly unfavorable CUS findings, but suggest caution when interpreting mild to moderate IVH or white matter injury.

Adverse neurodevelopmental outcomes among extremely low birth weight infants with a normal head ultrasound: prevalence and antecedents

OBJECTIVE

Severe abnormalities of the head ultrasound (HUS) are important predictors of cerebral palsy (CP) and mental retardation, and a normal HUS usually ensures the absence of major impairments. With the increasing survival of extremely low birth weight (ELBW) infants (birth weight <1000 g), the prognostic significance of a normal HUS may differ. This study examined the prevalence of and risk factors for CP and impaired mental development among ELBW infants with a normal HUS.

METHODS

Study infants were ELBW infants who were cared for in Neonatal Research Network centers in the years 1995-1999, had a normal early and late HUS, survived to discharge, and returned for follow-up assessments at 18 to 22 months' corrected age. The outcomes of interest were a score <70 on the Bayley Scales of Infant Development-II Mental Developmental Index (MDI) and CP. Risk factors included maternal demographics; infant characteristics; and interventions or morbidities related to the lung, infection, and nutrition. Logistic regression was used to estimate odds ratios (ORs) and 95% confidence intervals (CIs). A time-oriented approach was used to select variables for inclusion in logistic models.

RESULTS

Of 1749 infants with a normal early and late HUS (performed at a mean age of 6 and 47 days, respectively), 1473 (84%) returned for follow-up assessment. Infants had a birth weight of 792 +/- 134 g (mean +/- SD) and gestational age of 26 +/- 2 weeks. Rates of CP and MDI <70 were 9.4% and 25.3%, respectively, and 29.2% of infants had either CP or MDI <70. In multivariate analyses, factors associated with CP were male gender (OR: 1.8; 95% CI: 1.2-2.6), multiple birth, (OR: 1.6; 95% CI: 1.1-2.5), decreasing birth weight (OR: 1.3 for each 100-g decrease; 95% CI: 1.1-1.5), pneumothorax (OR: 2.3; 95% CI: 1.2-4.4), and days of conventional ventilation (OR: 1.2 for each additional 10 days; 95% CI: 1.1-1.3). With the exception of pneumothorax, these same factors were associated with MDI <70, in addition to less maternal education (OR: 1.4; 95% CI: 1.0-1.9) and Medicaid or lack of coverage for maternal insurance (OR: 1.7; 95% CI: 1.2-2.4).

CONCLUSIONS

Nearly 30% of ELBW infants with a normal HUS had either CP or a low MDI. Risk factors that are associated with this high rate of adverse outcomes include pneumothorax, prolonged exposure to mechanical ventilation, and educational and economic disadvantage. Improvements in pulmonary care to reduce duration of ventilation and avoid air leaks might improve neurodevelopmental outcome for ELBW infants.

Towards safer diagnosis in clinical practice - understanding and using diagnostic tests in the neonatal unit more appropriately

The diagnostic process is a complex task that is more often than not done inherently by clinicians. However, it is in fact based around quantitative risk assessment and, as a result, when done intuitively is open to a significant risk of bias. By adopting a more structured and quantitative approach to diagnosis, clinicians might be in a position to make better diagnostic decisions. To achieve this, explicit recognition about the uncertainty surrounding diagnosis and knowledge about the basic properties of diagnostic tests, including disease incidence and predictive values, is necessary, as well as some consideration of newer concepts such as 'action thresholds'. Examples from everyday neonatal practice illustrate the potential clinical risks associated with the inappropriate use and interpretation of diagnostic tests and the potential benefits of approaching diagnosis in a more robust manner. A number of tools are now readily available to help clinicians move towards more 'evidence-based' diagnosis.

Observer variability in interpretation of abdominal radiographs of infants with suspected necrotizing enterocolitis

We examined (1) the observer variability (both interobserver and intraobserver) in interpretation of abdominal radiographs of infants with suspected necrotizing enterocolitis (NEC), (2) the interobserver variability for individual radiologic signs used to diagnose NEC, and (3) the influence of experience in determining the extent of observer variability. Our hypotheses were (1) there would be considerable observer variability in interpretation of abdominal radiographs of infants with suspected NEC; (2) the extent of observer variability would differ for individual radiologic signs of NEC; and (3) the extent of observer variability would be determined by the observer's experience. The participants included 12 observers: two pediatric radiologists, four attending neonatologists, three neonatal fellows, and three pediatric residents. The participating observers under similar interpretation conditions, twice independently, interpreted the same 40 pairs of abdominal radiographs from infants with suspected NEC. The interval between the two interpretations was 3 to 6 months. Intraobserver and interobserver variability was assessed by applying the Kappa statistic to the radiologic signs of NEC for the two separate interpretations. The observers were blinded to patient's identity and the clinical course. Each observer recorded the absence, suspicion, or presence of (1) intestinal distention, (2) air fluid levels, (3) bowel wall thickening, (4) pneumatosis intestinalis, (5) portal venous gas, (6) pneumoperitoneum, and (7) NEC. We found low intraobserver and interobserver agreements. There was considerable variation in observer variability for individual radiologic signs. Trained observers performed better than intraining observers. We conclude that the radiologic signs in isolation should not be considered reliable. We recommend studies to formulate more objective criteria for many of the radiographic features of NEC. Standardization and periodic enforcement of these criteria among observers could reduce observer variability. We suggest that, to decrease both false-negative and false-positive interpretation, an experienced observer should always review the radiographs of infants with suspected NEC.

3-D sonographic volume measurement of the cerebral ventricular system: in vitro validation

This in vitro study evaluates the accuracy and observer dependency of a freehand three-dimensional (3-D) ultrasound system for measuring cerebral ventricular volume in infants. A sphere, a cylinder, and three cerebral ventricle phantoms were made of agarose and embedded in an echogenic matrix after measuring true volumes by water displacement. Volumes of the models were calculated by 3-D software after manual contour marking on ultrasound cross-sections. Mean +/- SD sonographic volume was 94.6%+/-7.3% (n = 130) of true volumes. Intraobserver variation (n = 10) was 2.3%-5.3% for complete investigation (scanning and marking), and 1.8%-4.1% for marking alone. Difference of means (n = 10) between two observers was 7.6% to 10.8% for complete investigation, and 0.6% to 1.6% for the marking process. We conclude that 3-D freehand ultrasonography may be useful for examining ventricle dilatation in infancy.

Correlation of ultrasound with postmortem neuropathologic studies in neonates

We examined the correlation between cranial sonogram and postmortem examination neurodiagnoses in 51 infants (30 boys and 21 girls) who underwent cranial sonogram before death. The mean gestation of the infants at birth was 32 weeks 6 days; the mean birth weight, 1,992 gm; and the mean survival, 27 days. Most infants had several postmortem examination neurodiagnoses; therefore the postmortem examination diagnosis considered to be most significant in terms of clinical management and long-term neurologic function was chosen for each infant and designated as the primary diagnosis. The ability of the cranial sonogram to diagnose the primary diagnosis was then evaluated. The accuracy of the cranial sonogram in defining primary diagnoses was 59%. There were 21 cases (41%) in which cranial sonogram failed to define the primary diagnosis; nine of these could be explained on the basis of the timing of the ultrasound in relation to the age of the lesion or to the microscopic nature of the lesion. In 12 patients the factors causing the failure of cranial sonogram to define primary postmortem examination diagnoses could not be identified. The positive predictive value of a cranial sonogram diagnosis was 77%; the negative predictive value was 19%.