A comparison of seminar and computer based training on the accuracy and reliability of raters using the Children's Global Assessment Scale (CGAS)

A novel evaluation of two related and two independent algorithms for eye movement classification during reading

Nystrӧm and Holmqvist have published a method for the classification of eye movements during reading (ONH) (Nyström & Holmqvist, 2010). When we applied this algorithm to our data, the results were not satisfactory, so we modified the algorithm (now the MNH) to better classify our data. The changes included: (1) reducing the amount of signal filtering, (2) excluding a new type of noise, (3) removing several adaptive thresholds and replacing them with fixed thresholds, (4) changing the way that the start and end of each saccade was determined, (5) employing a new algorithm for detecting PSOs, and (6) allowing a fixation period to either begin or end with noise. A new method for the evaluation of classification algorithms is presented. It was designed to provide comprehensive feedback to an algorithm developer, in a time-efficient manner, about the types and numbers of classification errors that an algorithm produces. This evaluation was conducted by three expert raters independently, across 20 randomly chosen recordings, each classified by both algorithms. The MNH made many fewer errors in determining when saccades start and end, and it also detected some fixations and saccades that the ONH did not. The MNH fails to detect very small saccades. We also evaluated two additional algorithms: the EyeLink Parser and a more current, machine-learning-based algorithm. The EyeLink Parser tended to find more saccades that ended too early than did the other methods, and we found numerous problems with the output of the machine-learning-based algorithm.

Is human classification by experienced untrained observers a gold standard in fixation detection?

Manual classification is still a common method to evaluate event detection algorithms. The procedure is often as follows: Two or three human coders and the algorithm classify a significant quantity of data. In the gold standard approach, deviations from the human classifications are considered to be due to mistakes of the algorithm. However, little is known about human classification in eye tracking. To what extent do the classifications from a larger group of human coders agree? Twelve experienced but untrained human coders classified fixations in 6 min of adult and infant eye-tracking data. When using the sample-based Cohen's kappa, the classifications of the humans agreed near perfectly. However, we found substantial differences between the classifications when we examined fixation duration and number of fixations. We hypothesized that the human coders applied different (implicit) thresholds and selection rules. Indeed, when spatially close fixations were merged, most of the classification differences disappeared. On the basis of the nature of these intercoder differences, we concluded that fixation classification by experienced untrained human coders is not a gold standard. To bridge the gap between agreement measures (e.g., Cohen's kappa) and eye movement parameters (fixation duration, number of fixations), we suggest the use of the event-based F1 score and two new measures: the relative timing offset (RTO) and the relative timing deviation (RTD).

Childhood Symptoms of ADHD Overrule Comorbidity in Relation to Psychosocial Outcome at Age 15: A Longitudinal Study

Objective

Neurodevelopmental problems (NDPs) may influence the transition from childhood to adolescence. Our aim was to study long-term psychosocial outcomes of NDPs, focusing on ADHD.

Method

Data was collected through a telephone interview with parents of twins at ages 9 or 12 years. NDP screen-positive children were clinically assessed at age 15; N = 450. Psychosocial outcome concerning peers, school, internalizing problems, antisocial behavior, alcohol misuse, drug misuse, and impaired daily functioning was examined.

Results

Even after controlling for other NDP comorbidity, screen-positivity for ADHD doubled or tripled the odds of later psychosocial problems. When controlling for parental education level, the significant effect of ADHD remained only for antisocial behavior and impaired daily functioning.

Conclusions

Signs of NDPs as well as other psychiatric diagnoses at ages 9 or 12 years are associated with a more problematic adolescence. However, despite the presence of comorbidity, early ADHD symptoms stand out as the most important risk factor for later antisocial development and impaired daily functioning.

Brief report: "Quick and (not so) dirty" assessment of change in autism: cross-cultural reliability of the Developmental Disabilities CGAS and the OSU autism CGI

There are few evaluated economic tools to assess change in autism. This study examined the inter-rater reliability of the Developmental Disabilities Children's Global Assessment Scale (DD-CGAS), and the OSU Autism Clinical Global Impression (OSU Autism CGI) in a European setting. Using these scales, 16 clinicians with multidisciplinary background and varying experience independently rated eight vignettes of autism spectrum disorder for severity and general psychosocial functioning at referral and discharge. Intraclass correlation coefficient (ICCs) for experienced clinicians were .75 for the DD-CGAS and .72 for the OSU Autism CGI. In inexperienced clinicians these ICCs were .58 and .59. Results confirm previous North American studies, and further extents the reliability of the instruments to untrained, less experienced clinicians with different professions.

Maternal pre-pregnancy body mass index and offspring attention deficit hyperactivity disorder: a population-based cohort study using a sibling-comparison design

BACKGROUND

High maternal pre-pregnancy body mass index (BMI) is associated with increased risk of offspring attention deficit hyperactivity disorder (ADHD). However, the role of unmeasured familial confounding for this association remains unclear.

METHODS

We conducted a population-based cohort study via linkage of Swedish national and regional registers to investigate maternal pre-pregnancy BMI (underweight: BMI <18.5; overweight: 25≤ BMI <30; obesity: BMI ≥30) in relation to offspring ADHD. We followed 673 632 individuals born in Sweden between 1992 and 2000, with prospectively collected information on maternal pre-pregnancy BMI, until they received an ADHD diagnosis or ADHD medication, death, emigration or 31 December 2009. Hazard ratios (HRs) were estimated by Cox proportional hazards models. Stratified Cox proportional hazards models were applied to data on full siblings to control for unmeasured familial confounding.

RESULTS

At the population level, pre-pregnancy overweight/obesity was associated with increased risk of offspring ADHD (HR(overweight) = 1.23, 95% CI = 1.18-1.27, P = 0.01; HR(obesity) = 1.64, 95% CI = 1.57-1.73, P = 0.01), after adjustment for measured covariates. In full sibling comparisons, however, previously observed associations no longer remained (HR(overweight) = 0.98, 95% CI = 0.83-1.16, P = 0.82; HR(obesity) = 1.15, 95% CI = 0.85-1.56, P = 0.38).

CONCLUSIONS

The results suggested that the association between maternal pre-pregnancy overweight/obesity and offspring ADHD could be ascribed to unmeasured familial confounding.

Outcomes of child psychiatric treatment

OBJECTIVE

The aim of this study was to investigate outcomes of child psychiatric outpatient treatment as usual and to identify outcome predictors, with special regard to attention-deficit/hyperactivity disorder (ADHD), mood disorder, obsessive-compulsive disorder and conduct disorder.

METHOD

Routinely collected data from 12 613 outpatients between July 2006 and January 2010 in Stockholm, Sweden were analysed. The outcome measure was change in Children's Global Assessment Scale (CGAS) ratings between first visit and case closure (∆CGAS).

RESULTS

CGAS improved during the course of treatment across all diagnostic groups, ranging from a mean change of 4 (mental retardation) to 16 (suicide attempts). ∆CGAS was two times higher in the mood disorder group compared with the ADHD group. In the mood disorder group, several psychotherapies were associated with better outcome but not medication. In the ADHD group, psychotherapeutic interventions were also associated with better outcome, but those who received treatment with central stimulants received less non-medical interventions.

CONCLUSION

Whereas the functional impairment and the level of improvement in mood disorder corresponded to previous efficacy studies, the ADHD patients were more impaired and improved less after treatment. This should prompt a critical discussion as to whether ADHD patients receive the best available treatment in CAMHS in Stockholm and elsewhere.