Evaluating bias in validity studies of developmental/behavioral screening tests

Can Dynamic Assessment Identify Language Disorder in Multilingual Children? Clinical Applications From a Systematic Review

PURPOSE

Multilingual children are disproportionately represented on speech pathology caseloads, in part due to the limited ability of traditional language assessments to accurately capture multilingual children's language abilities. This systematic review evaluates the evidence for identification of language disorder in multilingual children using dynamic assessment and considers clinical applications of the evidence.

METHOD

A systematic search of the Cumulative Index to Nursing and Allied Health Literature, Education Resources Information Centre, Education Source, Google Scholar, Linguistics, Medline, and PsycINFO databases produced 10 articles that met the inclusion criteria: between-groups comparison studies that used dynamic assessment to identify language disorder in children under 12 years old that spoke a different language at home to the majority society language. Articles were critically appraised using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) protocol.

RESULTS

Nine of the 10 studies reported that their dynamic assessment identified language disorder in multilingual children. However, small sample sizes, limited language pairs, variability in the reference standard, and design deficiencies resulted in poor ratings for all studies on QUADAS-2.

CONCLUSIONS

The studies in this review reflected an emergent area of research. Preliminary guidelines for clinical application indicate that dynamic assessment may be a suitable and time-efficient complementary method of diagnosis of language disorder in multilingual children. Further recommendations about age of use, language of instruction, and relevant scores are included.

Elaboration of Screening Scales for Mental Development Problems Detection in Russian Preschool Children: Psychometric Approach

Background: computer-based screenings are usually used for early detection of a child’s mental development problems. However, there are no such screenings in Russia yet. This study aimed to elaborate scales for rapid monitoring of mental development of 3-year-olds. Methods: 863 children took part in the study, among them 814 children of the group Norm, 49 children with developmental delay (DD), including 23 children with symptoms of autistic spectrum disorder (ASD). The multifactor study of mental development tool was used as a part of a software complex for longitudinal research for data collection. This study used a set of 233 tasks that were adequate for 3-year-olds. Exploratory and confirmatory factor analysis was used for the elaboration and factor validation of the scales. The structure of the relationship between scales and age was refined using structural equation modeling. Results: as a result of the research, screening scales were elaborated: “Logical reasoning”, “Motor skills”, “General awareness”, “Executive functions”. The factor validity and reliability of scales were proved. The high discriminability of the scales in distinguishing the “Norm” and “DD” samples was revealed. The developed test norms take into account the child’s age in days and allow identifying a “risk group” with an expected forecast accuracy of at least 90%. The obtained scales meet psychometric requirements for their application and allow creating an online screening system for wide application.

The Structured Observation of Motor Performance in Infants can detect cerebral palsy early in neonatal intensive care recipients

BACKGROUND

The detection of motor problems in infancy requires a detailed assessment method that measures both the infants' level of motor development and movement quality.

AIMS

To evaluate the ability of the Structured Observation of Motor Performance in Infants (SOMP-I) to detect cerebral palsy (CP) in neonatal intensive care recipients.

STUDY DESIGN

Prospective cohort study analyzed retrospectively.

SUBJECTS

212 (girls: 96) neonatal intensive care recipients (mean gestational age 34weeks, range: 23-43). Twenty infants were diagnosed with CP.

OUTCOME MEASURES

The infants were assessed using SOMP-I at 2, 4, 6 and 10months' corrected age. Accuracy measures were calculated for level of motor development, quality of motor performance and a combination of the two to detect CP at single and repeated assessments.

RESULTS

At 2months, 17 of 20 infants with CP were detected, giving a sensitivity of 85% (95% CI 62-97%) and a specificity of 48% (95% CI 40-55%), while the negative likelihood ratio was 0.3 (95% CI 0.1-0.9) and the positive likelihood ratio was 1.6 (95% CI 1.3-2.0). At 6months all infants with CP were detected using SOMP-I, and all infants had repeatedly been assessed outside the cut-offs. Specificity was generally lower for all assessment ages, however, for repeated assessments sensitivity reached 90% (95% CI 68-99%) and specificity 85% (95% CI 79-90%).

CONCLUSIONS

SOMP-I is sensitive for detecting CP early, but using the chosen cut-off can lead to false positives for CP. Assessing level and quality in combination and at repeated assessments improved predictive ability.

Concurrent validity of ages and stages questionnaires in preterm infants

BACKGROUND

Although preterm infants born at 29 to 36 gestational weeks (GW) are at risk for developmental delay, they do not always benefit from systematic follow-up. Primary care physicians are then responsible for their developmental surveillance and need effective screening tests. This study aimed to determine whether the Ages and Stages Questionnaires (ASQ) at 12 and 24 months' corrected age (CA) identify developmental delay in preterm infants.

METHODS

With a cross-sectional design involving 2 observations at 12 and 24 months' CA, 124 and 112 preterm infants were assessed. Infants were born between May 2004 and April 2006 at 29 to 36 GW. The ASQ and the Bayley Scales of Infant Development were used. Concurrent validity was calculated by using κ coefficient, sensitivity, and specificity.

RESULTS

At 12 months' CA, the ASQ did not perform well in identifying infants with mental delay (κ = 0.08-0.19; sensitivity = 0.20-0.60; specificity = 0.68-0.88). Agreement (κ = 0.28-0.44) and specificity (0.90-0.97) were better for the psychomotor scale, but the sensitivity remained insufficient (0.25-0.52). At 24 months, the ASQ had good sensitivity (0.75-0.92) and specificity (0.55-0.78) for detecting mental delays (κ = 0.45). Results remained unsatisfactory for detecting motor delays (sensitivity = 0.31-0.50; specificity = 0.73-0.92).

CONCLUSIONS

Preterm infants with developmental delays at 12 months' CA are not adequately identified with the ASQ. At 24 months' CA, the ASQ identifies mental delays but not psychomotor delays. Additional measures should be used to increase yield of detecting at-risk preterm infants.

Identification of developmental-behavioral problems in primary care: a systematic review

CONTEXT

Recent mandates and recommendations for formal screening programs are based on the claim that pediatric care providers underidentify children with developmental-behavioral disorders, yet the research to support this claim has not been systematically reviewed.

OBJECTIVE

To review research literature for studies regarding pediatric primary care providers' identification of developmental-behavioral problems in children.

METHODS

On the basis of a Medline search conducted on September 22, 2010, using relevant key words, we identified 539 articles for review. We included studies that (1) were conducted in the United States, (2) were published in peer-reviewed journals, (3) included data that addressed pediatric care providers' identification of developmental-behavioral problems in individual patients, (4) included an independent assessment of patients' developmental-behavioral problems, such as diagnostic interviews or validated screening instruments, and (5) reported data sufficient to calculate sensitivity and specificity. Studies were not limited by sample size. Eleven articles met these criteria. We used Quality Assessment of Diagnostic Accuracy Studies (QUADAS) criteria to evaluate study quality. Although the studies were similar in many ways, heterogeneous methodology precluded a meta-analysis.

RESULTS

Sensitivities for pediatric care providers ranged from 14% to 54%, and specificities ranged from 69% to 100%. The authors of 1 outlier study reported a sensitivity of 85% and a specificity of 61%.

CONCLUSIONS

Pediatricians are often the first point of entry into developmental and mental health systems. Knowing their accuracy in identifying children with developmental-behavioral disabilities is essential for implementing optimal evaluation programs and achieving timely identification. Moreover, these statistics are important to consider when planning large-scale screening programs.