Strengths and Difficulties Questionnaire: internal validity and reliability for New Zealand preschoolers

Investigating the relationship between prenatal alcohol exposure and children's behavioural and emotional development: analysis of the Growing Up in New Zealand study

AIMS

To examine the relationship between prenatal alcohol exposure (PAE) and children's behavioural and emotional development in a large generalizable sample of women and their children in Aotearoa New Zealand.

METHODS

Using data from the Growing Up in New Zealand longitudinal cohort, we investigated the relationship between maternal PAE and behavioural and emotional development in 8-year-old children. We explored secondary outcomes including measures of language, executive function, academic achievement, and adaptive behaviour.

RESULTS

We found no significant differences in the measures of behavioural and emotional development in children 8 years old based on alcohol consumption. No significant differences in behavioural and emotional development were found based on amount of PAE and when PAE occurred, despite controlling for a range of potential confounding factors, such as neighbourhood deprivation and maternal health measures. PAE was associated with significantly higher scores for parent-rated oral language indicating better oral language. In Māori mothers, PAE was significantly associated with an increased risk of higher scores on two of the Strengths and Difficulties Questionnaire subscales.

CONCLUSIONS

We did not find an association between PAE and behavioural and emotional development in children aged 8 years. PAE and behavioural and emotional development are difficult to measure accurately, and the moderating variables between them are complex. Future analyses will require larger cohorts of mothers and their children using precise measures of PAE and outcomes to enable more precise estimates of association.

Screening Children in India: Translation and Psychometric Evaluation of the Parents' Evaluation of Developmental Status and the Strength and Difficulties Questionnaire

Timely screening and surveillance of children for developmental delay and social-emotional learning difficulties are essential in Low- and Middle-Income Countries like India. Screening measures like the Parents' Evaluation of Developmental Status (PEDS) and Strength and Difficulties Questionnaire (SDQ) are considered suitable for India due to their low cost, easy accessibility, and no training requirement for administration. However, India lacks validated screening measures, and the PEDS and SDQ have yet to be validated for children in India. The study aimed to translate the PEDS and SDQ from English to Hindi and psychometrically evaluate the same measures on children aged 4-8 years in India. The original PEDS and SDQ forms and their translations were pilot tested on 55 participants and evaluated using data from 407 children with typical development (TD) and 59 children with developmental disability (DD). Parents and teachers reported no meaningful discrepancy between the original and translated (Hindi) questionnaires. Internal consistency for the PEDS was acceptable, but unacceptable for most subscales on the SDQ, for both TD and DD samples. Test-retest reliability was poor for the PEDS but adequate for the SDQ. Results from known-group validity testing showed that the PEDS scores could be used to distinguish between the TD and DD samples. The results from this study provide further support for the use of the PEDS and SDQ in developing countries like India.

Identifying neurodevelopmental disabilities from nationalised preschool health check

OBJECTIVE

Models of psychometric screening to identify individuals with neurodevelopmental disabilities (NDDs) have had limited success. In Aotearoa/New Zealand, routine developmental surveillance of preschool children is undertaken using the Before School Check (B4SC), which includes psychometric and physical health screening instruments. This study aimed to determine whether combining multiple screening measures could improve the prediction of NDDs.

METHODS

Linked administrative health data were used to identify NDDs, including attention deficit hyperactivity disorder, autism spectrum disorder and intellectual disability, within a multi-year national cohort of children who undertook the B4SC. Cox proportional hazards models, with different combinations of potential predictors, were used to predict onset of a NDD. Harrell's c-statistic for composite models were compared with a model representing recommended cutoff psychometric scores for referral in New Zealand.

RESULTS

Data were examined for 287,754 children, and NDDs were identified in 10,953 (3.8%). The best-performing composite model combining the Strengths and Difficulties Questionnaire, the Parental Evaluation of Developmental Status, vision screening and biological sex had 'excellent' predictive power (C-statistic: 0.83) compared with existing referral pathways which had 'poor' predictive power (C-statistic: 0.68). In addition, the composite model was able to improve the sensitivity of NDD diagnosis detection by 13% without any reduction in specificity.

CONCLUSIONS

Combination of B4SC screening measures using composite modelling could lead to significantly improved identification of preschool children with NDDs when compared with surveillance that rely on individual psychometric test results alone. This may optimise access to academic, personal and family support for children with NDDs.

Psychometric properties of the Social Behavior Questionnaire (SBQ) in a longitudinal population-based sample

We assessed the psychometric properties of the Social Behavior Questionnaire (SBQ), a 30-item questionnaire evaluating social (e.g., disruptive behaviors, bullying) and emotional problems (e.g., anxiety, depression) among children aged 3.5–12 years. Children ( n = 1,950, 50.21% boys) were drawn from the Quebec Longitudinal Study of Child Development. Mothers reported the frequency with which children presented social and emotional behaviors from 3.5 to 8 years of age, and teachers from 6 to 12 years. We assessed internal structure using Confirmatory Factor Analysis, reliability using Cronbach’s alpha, and convergent and discriminant validity using a multitrait-multimethod (MTMM) approach. The six-factor (emotional distress, withdrawal, impulsive/hyperactive/inattentive, disruptive behaviors, prosocial behaviors, and peer relationships difficulties) structure of the SBQ showed good fit from ages 3.5 to 12 years. Reliability estimates were good to excellent (alphas > .7), and MTMM showed good convergent and discriminant validity. Overall, the SBQ presented good psychometric properties with a large population-based sample aged 3.5–12 years. Further studies should assess its screening potential by investigating its convergent validity with diagnostic information.

Factor structure of the SDQ and longitudinal associations from pre-school to pre-teen in New Zealand

Objective

The objective of this study was to assess the validity of the Strengths and Difficulties Questionnaire in a cohort of New Zealand children followed from birth to the age of eleven. The study also aimed to assess the stability of the child data in relation to behavioural outcomes during this period.

Methods

Children in the Auckland Birthweight Collaborative Study were assessed at approximately 3½, 7 and 11 years of age. At all time-points parents completed the parent version of the Strengths and Difficulties Questionnaire, and the children themselves completed the self-report version at 11 years of age. The validity and internal consistency were assessed using exploratory factor analysis, Cronbach’s alpha, and McDonald’s Omega. Cross tabulations and Chi-square statistics were used to determine whether Total Difficulty scores, as per accepted cut-offs, remained stable over time (between normal and abnormal/borderline categories).

Results

The factor structure remained relatively consistent across all three time-points, though several questions did not load as per the originally published factor analysis at the earliest age. The internal consistency of the Strengths and Difficulties Questionnaire was good at all time-points and for parent- and child-completed versions. There was low agreement in the total scores between time points.

Conclusions

The factor analysis shows that the Strengths and Difficulties Questionnaire has a similar factor structure, particularly in older ages, to that previously published and shows good internal consistency. At the pre-school follow up, a larger than expected proportion of children were identified with high scores, particularly in the conduct sub-scale. Children’s behaviour changes over time, with only poor to moderate agreement between those identified as abnormal or borderline over the longitudinal follow up.

Setting and maintaining standards for patient-reported outcome measures: can we rely on the COSMIN checklists?

As test-developers we have often been troubled by published reviews of patient-reported outcome measures (PROMs). Too often minor issues are judged important while other reviews exclude the best measures available. Perhaps this led several groups to make recommendations for evaluating the quality of PROMs. The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist is the latest set of recommendations. While reviewing the COSMIN literature and reviews conducted using their recommendations several concerns became apparent. The checklist is not evidence-based, relying on the opinion of researchers experienced in health-related quality of life. PROMs measuring other types of outcomes are inadequately covered by the checklist. COSMIN choose to focus on Classical Test Theory and the checklists are not appropriate for use with PROMs developed using modern measurement. Such an approach only obstructs progress in the field of outcome measurement. The retrospective nature of the evaluations also penalizes new PROMs. While the checklists imply that composite, ordinal level measurement is acceptable, crucial aspects of instrument development and quality are excluded. Reviews based on the COSMIN checklist produce contradictory conclusions and fail to provide evidence to support the recommendations. These problems suggest that the checklists themselves lack reliability and validity. It is also clear that several reviewers lack the expertise to apply the checklists. Researchers require a good grounding in instrument development and psychometrics to produce quality reviews. The science of modern PROM development is still in an early phase. Few available PROMs have sufficient quality, limiting the need for complex reviews. Standards need to be agreed for high quality outcome measurement. The issue is who should set these standards? Most published reviews merely scratch the surface and lack essential detail. All reviews of PROMs should be treated with caution, irrespective of whether the COSMIN checklist was employed.

Maternal prenatal stress and postnatal depressive symptoms: discrepancy between mother and teacher reports of toddler psychological problems

PURPOSE

Maternal prenatal stress and postnatal depression are reported to increase the risk for early offspring psychological problems. We examined whether these two stressors predicted toddler emotional or behavioral problems based on the mother and teacher reports, respectively.

METHODS

A longitudinal study within the Odense Child Cohort (OCC). Prenatal stress was assessed (gestation week 28) using Cohen's Perceived Stress Scale (PSS). Depressive symptoms were assessed (3 months after birth) using the Edinburgh Postnatal Depression Scale (EPDS). Behavioral and emotional problems were assessed by mothers using the preschool version of Child Behaviour Checklist (CBCL) and by teachers using the caregiver-teacher report form (CTR-F).

RESULTS

N = 1302 mother-child dyads were included. CBCL (N = 1302) was collected at 29 months (SD 5.3) and C-TRF (N = 989) at 32.6 months (SD 6.9). N = 70 mothers (5.4%) were at high risk for postnatal depression (EPDS score > 12). Generalized additive models showed that prenatal stress (increase of + 1 on PSS-10 total score) predicted an increase in CBCL (+ 0.011) and C-TRF (+ 0.015) total scores. Postnatal depressive symptoms (increase of + 1 on EPDS total score) only predicted an increase in CBCL total score (+ 0.026).

CONCLUSION

Prenatal maternal stress was a significant predictor of both mother and teacher reported toddler emotional and behavioral problems, although effect sizes were small. Postnatal depressive symptoms were associated with increased maternal (but not teacher) reporting of toddler problems. Mothers reported more toddler psychological problems than teachers, and the mother-teacher discrepancy was positively correlated to maternal postnatal depressive symptoms.

Detecting mental health problems after paediatric acquired brain injury: A pilot Rasch analysis of the strengths and difficulties questionnaire

The parent-reported Strengths and Difficulties Questionnaire (SDQ-P) is commonly used to assess for mental health problems, but its psychometric properties have not been studied in the paediatric Acquired Brain Injury (ABI) population. This study investigated the properties of the SDQ-P and its subscales in this population using Rasch analysis. One hundred and forty-three SDQ-Ps and 123 Impact Supplements were analyzed. Sixty-nine percent of SDQ-Ps were completed by female carers, 59% of young people were male, and 58% had Traumatic Brain Injury (TBI). In this population the SDQ-P Total Difficulties Scale and the Conduct Problems subscale showed questionable construct validity. The individual subscales and Impact Supplement did not meet the criteria for reliability. Two items had disordered thresholds. The individual subscales showed mistargeting and 13-24% person misfit. Two items were significantly underdiscriminating. There was differential item functioning with age and time post-injury, and local dependence between subscale items. The Total Difficulties scale was multidimensional. The most easily endorsed items were in keeping with common symptoms of brain injury. These findings suggest the SDQ-P in its current form may not be a reliable and valid assessment measure for mental health difficulties in the paediatric ABI population and requires further investigation.