Diagnostic Utility of WISC-IV General Abilities Index and Cognitive Proficiency Index Difference Scores Among Children With ADHD

Measurement Invariance of the WISC-V across a Clinical Sample of Children and Adolescents with ADHD and a Matched Control Group

Measurement invariance of the Wechsler Intelligence Scale for Children, Fifth Edition (WISC-V) 10-primary subtest battery was analyzed across a group of children and adolescents with ADHD (n = 91) and a control group (n = 91) matched by sex, age, migration background, and parental education or type of school. First, confirmatory factor analyses (CFAs) were performed to establish the model fit for the WISC-V second-order five-factor model in each group. A sufficiently good fit of the model was found for the data in both groups. Subsequently, multigroup confirmatory factor analyses (MGCFAs) were conducted to test for measurement invariance across the ADHD and control group. Results of these analyses indicated configural and metric invariance but did not support full scalar invariance. However, after relaxing equality constraints on the Vocabulary (VC), Digit Span (DS), Coding (CD), Symbol Search (SS), and Picture Span (PS) subtest intercepts as well as on the intercepts of the first-order factors Working Memory (WM) and Processing Speed (PS), partial scalar invariance could be obtained. Furthermore, model-based reliability coefficients indicated that the WISC-V provides a more precise measurement of general intelligence (e.g., represented by the Full-Scale IQ, FSIQ) than it does for cognitive subdomains (e.g., represented by the WISC-V indexes). Group comparisons revealed that the ADHD group scored significantly lower than the control group on four primary subtests, thus achieving significantly lower scores on the corresponding primary indexes and the FSIQ. Given that measurement invariance across the ADHD and the control group could not be fully confirmed for the German WISC-V, clinical interpretations based on the WISC-V primary indexes are limited and should only be made with great caution regarding the cognitive profiles of children and adolescents with ADHD.

Association of the Comprehensive Attention Test and the Korean Wechsler Intelligence Scale for Children-Fourth Edition in Children and Adolescents With Attention-Deficit/Hyperactivity Disorder

Objectives

This study aimed to investigate the correlation between the Comprehensive Attention Test, Korean-Wechsler Intelligence Scale for Children-Fourth Edition, and Attention-Deficit/Hyperactivity Disorder (ADHD) Rating Scale-IV scores in children and adolescents with ADHD.

Methods

Fifty-five children and adolescents diagnosed with ADHD and not taking psychiatric medications were included in this retrospective study. A correlation analysis was performed.

Results

Although simple visual and auditory selective attention have diagnostic value in traditional continuous performance tests, this study revealed that inhibition-sustained attention and interference-selective attention are also effective in evaluating ADHD. Furthermore, the correlation between the attention and intelligence test scores varied depending on the use of visual or auditory stimuli.

Conclusion

The findings of this study contribute to clarifying our understanding of the cognitive characteristics of children and adolescents with ADHD and can be used in future research.

The structure, profile, and diagnostic significance of intelligence in children with ADHD are impressively similar to those of children with a specific learning disorder

This study examines the structure, profile, and diagnostic significance of intelligence in a group of 948 children diagnosed with attention deficit/hyperactivity disorder (ADHD) assessed with the WISC-IV and compared with children with specific learning disorders (SLDs) and with typically developing children. Based on four indexes, the WISC-IV configuration found in TD resulted applicable to ADHD, but with generally lower loadings on g. The Perceptual Reasoning and Verbal Comprehension indexes not only had higher loadings compared to the other two indexes but also represented the relative strengths of children with ADHD, as previously observed for children with SLD. In fact, the WISC pattern could be successfully used for discriminating between ADHD and TD, but not between ADHD and SLD. The latter result was not due to a co-occurrence of a learning disorder because the presence or absence of an associated diagnosis of SLD negligibly affected the pattern observed in ADHD. We concluded that the characteristics of intelligence in children with ADHD can be relevant for assessing this disorder, and that ADHD and SLDs share largely similar underlying cognitive features.

Ahead of the (ROC) Curve: A Statistical Approach to Utilizing Ex-Gaussian Parameters of Reaction Time in Diagnosing ADHD Across Three Developmental Periods

INTRODUCTION

Performance on executive function (EF) tasks is only modestly predictive of a diagnosis of Attention Deficit Hyperactivity Disorder (ADHD), despite the common assumption that EF deficits are ubiquitous to the disorder. The current study sought to determine whether ex-Gaussian parameters of simple reaction time are better able to discriminate between children and adults with and without ADHD, compared with traditional measures of inhibitory control.

METHODS

Receiver Operating Characteristic (ROC) analyses and the area under the curve (AUC) were used to examine the ability of performance on two commonly used tasks of inhibitory control (i.e. stop signal reaction time (SSRT) and go-no-go tasks) to predict ADHD status in preschool (N = 108), middle childhood (N = 309), and young adulthood (N = 133).

RESULTS

Across all samples, SSRT, go-no-go percentage of failed inhibits, and standard deviation of reaction (SDRT) time to "go" trials, all successfully discriminated between individuals with and without ADHD. Ex-Gaussian decomposition of the RT distribution indicated that both larger tau and larger sigma drove findings for SDRT. Contrary to predictions, traditional measures of inhibitory control were equal if not better predictors of ADHD status than ex-Gaussian parameters.

CONCLUSIONS

Findings support ongoing work to quantify the separate contributions of cognitive subprocesses that drive task performance, which in turn is critical to developing and improving process-based approaches in clinical assessment.

Effects of Bacopa monnieri (CDRI 08®) in a population of males exhibiting inattention and hyperactivity aged 6 to 14 years: A randomized, double-blind, placebo-controlled trial

The current study investigated the efficacy of extract of Bacopa monnieri (BM; CDRI 08®) in reducing levels of inattention and hyperactivity in young children. BM has demonstrated improvements in cognitive outcomes in adults, yet little research is available on its effects in younger populations. A 14-week randomized, double-blind, placebo-controlled clinical trial, with placebo run-in and run-out phases, investigated the effects of BM on behavioural, cognitive, mood, and sleep effects in male children aged 6 to 14 years against placebo. One-hundred and twelve participants were recruited into the trial, with 93 datasets available for analysis. No significant behavioural differences were noted between treatment groups. Cognitive outcomes indicated decreased error-making in children taking CDRI 08® (p = .04) and increased speed of reaction time in those taking placebo (p = .04) at study end. Improvements in cognitive flexibility (p = .01), executive functioning (p = .04), interpersonal problems (p = .02), and sleep routine (p = .04) were noted in those consuming CDRI 08® over placebo. CDRI 08® did not improve behavioural outcomes, but may have cognitive, mood and sleep benefits in children aged 6 to 14 years. Further study is required to support the findings presented here.

Children and adolescents with neurodevelopmental disorders show cognitive heterogeneity and require a person-centered approach

We aimed to identify patterns of cognitive differences and characterize subgroups of Mexican children and adolescents with three neurodevelopmental disorders (NDD): intellectual disability (ID), autism spectrum disorders (ASD) and attention deficit/hyperactivity disorder (ADHD). The sample included 74 children and adolescents 6–15 years; 34% had ID, ASD or ADHD, 47% had ID in comorbidity with ASD, ADHD or both, 11% had ASD + ADHD, 8% were children without NDD. We applied WISC-IV, Autism Diagnostic Interview-Revised, Mini-International Neuropsychiatric Structured Interview, Child Behavior Checklist, and UNICEF Child Functioning Module. We evaluated the normality of the WISC-IV sub-scales using the Shapiro-Francia test, then conducted a latent class analysis and assessed inter-class differences in terms of household, parent and child characteristics. The following four-class solution best fit the data: “Lower Cognitive Profile” (LCP), “Lower Working Memory” (LWM), “Higher Working Memory” (HWM), “Higher Cognitive Profile” (HCP). LCP included most of the children with ID, who had a low Working Memory (WM) index score. LWM included mainly children with ASD or ID + ADHD; their Perceptual Reasoning (PR) and Processing Speed (PS) index scores were much higher than those for Verbal Comprehension (VC) and WM. HWM included children with ASD or ADHD; their scores for PR, PS and VC were high with lower WM (although higher than for LWM). HCP included children without NDD and with ASD or ADHD or both and had the highest scores on all indices. Children with NDD show cognitive heterogeneity and thus require individualized treatment plans.

Cognitive Profiles in the WISC-V of Children with ADHD and Specific Learning Disorders

Attention Deficit Hyperactivity Disorder (ADHD) has a high comorbidity with specific learning disorders (SLD). Children with ADHD and children with SLD show specific cognitive deficits. This study aims to examine similarities and differences between cognitive profiles of children with ADHD + SLD, children with SLD only, and a control group to find out whether specific or shared deficits can be identified for the groups. We compared the WISC-V profiles of 62 children with ADHD and SLD (19 girls, M-age = 10.44; SD = 2.44), 35 children with SLD (13 girls, M-age = 10.21; SD = 2.11) and 62 control children without ADHD or SLD (19 girls, M-age = 10.42; SD = 2.39). The ADHD + SLD group performed worse than the control group in the WISC-V indices WMI, PSI, FSIQ, AWMI, CPI and worse than the SLD group in these indices and the VCI, NVI and GAI. Therefore, compared to children with SLD, children with ADHD + SLD did not show specific impairments in any particular cognitive domain but rather non-specific impairment in almost all indices. Hence, the WISC-V is suited to depict the cognitive strength and weaknesses of an individual child as a basis for targeted intervention.

Cognitive and personality differences between adolescents with and without attention deficit/hyperactivity disorder

BACKGROUND

The relationships between cognitive and personality profile in teenagers with and without ADHD were assessed.

METHOD

Two groups of teenagers, one with ADHD (N = 135; mean age = 13.93) and another group without ADHD (N = 199, mean age = 14.29) were evaluated using the K-BIT and 16PF-APQ tests.

RESULTS

In cognitive variables, the results revealed that the ADHD group returned higher scores in the Matrices subtest and the IQ test. In personality variables, the group with ADHD exhibited higher scores in Tough-Mindedness and lower scores in Self-Control than the group without the disorder. The canonical correlation analysis applied to each group revealed a differing pattern of interrelationships between the cognitive-personality variables in the two groups. In adolescents with ADHD, we observed that higher scores in cognitive variables were associated with a more extroverted personality and less self-control, while in adolescents without ADHD, higher scores in cognitive variables were associated with less tough-mindedness and lower levels of self-control.

CONCLUSIONS

The cognitive and personality variables of adolescents with and without ADHD differ. These results will be useful for establishing a cognitive and personality profile for this section of the population. The educational implications of the study are under discussion.

ADHD Symptomatology, Executive Function and Cognitive Performance Differences between Family Foster Care and Control Group in ADHD-Diagnosed Children

Children in foster care have a high prevalence of attention deficit and hyperactivity disorder (ADHD) diagnosis, together with other difficulties in inattentive/hyperactive behaviors, executive and cognitive processes. Early exposure to adversity is a risk factor for developing ADHD via neurodevelopmental pathways. The goal of this research is (a) to study the cognitive and executive performance and inattentive/hyperactive behavior of ADHD-diagnosed children living in foster families in Spain, and (b) to analyze the role of placement variables in their performance. The sample was composed of 102 ADHD-diagnosed children aged 6- to 12-years-old, divided into two groups: 59 children living with non-relative foster families and 43 children not involved with protection services. Children’s executive function–inhibition, working memory, flexibility, attention, intellectual capacity, verbal comprehension, perceptive reasoning, working memory and processing speed were assessed using objective testing measures. At the same time, parents and teachers reported on children’s inattentive, hyperactive and impulsive behaviors. Children in foster care obtained lower scores in the general ability index than the control group after controlling the age at assessment. However, no differences were found in executive processes. Regarding placement factors, children with shorter exposure to adversities in their birth families and more time in foster care showed better executive performance. Professionals should consider the placement history of children in foster care and its influence on their symptomatology and cognitive capacities.

Clinical presentations of attention-deficit/hyperactivity disorder (ADHD) in children and adolescents: comparison of neurocognitive performance

This study aimed to compare performance in working memory (WM), processing speed (PRS), and attention measures in children and adolescents with typical development (TD) and with Attention deficit hyperactivity disorder (inattentive [ADHD-I] or combined [ADHD-C]) presentations, and to determine the predictive ability of the measures mentioned to discriminate between ADHD presentations and TD. 260 children and adolescents, 138 with ADHD (70 ADHD-I and 68 ADHD-C) and 122 TD in two age cohorts (8-12 years; 13-16 years), were assessed with WM and PRS indexes of Wechsler Intelligence Scale for Children (WISC-IV) and the d2 attention test. Significant differences between ADHD and TD groups in the WISC-IV GAI scores were found in children but not in adolescents. Children and adolescents with both ADHD presentations performed poorly on the PRS index, while on the WM index only children exhibited difficulties. In the attention test, children with ADHD-C showed more impulsivity and more difficulties for processing speed, concentration and accuracy than ADHD-I and TD. In addition, both ADHD presentations had higher inattention scores than TD. ADHD adolescents performed worse than TD in processing speed, concentration and accuracy. ADHD groups showed more impulsivity and inattention than TD. Digit Span and Symbol Search (WISC-IV) and processing speed and accuracy (d2) successfully classified ADHD and TD in children, but in adolescents, only coding (WISC-IV) and accuracy (d2) successfully classified ADHD presentations and TD. The WISC-IV and d2 yield neuropsychological profiles which reflect age-related cognitive changes and may allow the adaptation of more tailored early interventions for ADHD.

Theory of Mind in ADHD. A Proposal to Improve Working Memory through the Stimulation of the Theory of Mind

The aim of this study was to investigate the relationships between Theory of Mind (ToM), Working Memory (WM), and Verbal Comprehension (VC). Performance of these variables was evaluated in 44 elementary students (6-12 years) diagnosed with ADHD. Their performance in all variables was collected through the Neuropsychological Battery (NEPSY-II) and the Wechsler Intelligence Scale for Children IV. The results showed that fifty percent of the participants were below the 25th percentile in ToM and that this low performance was not related to age. In addition, analyses showed statistically significant relationships between WM, VC, and ToM. Analysis of the effect of WM and VC on ToM showed that only WM explained the variance in participant performance in ToM. These results led us to raise the need to include ToM among the skills to be stimulated in programs for the treatment of ADHD, accompanying other skills related to social adaptation that are usually included in such programs. Likewise, considering that ToM implies putting into practice skills such as considering different points of view, attending to relevant aspects of the context, making decisions, inferring mental states, and predicting behaviors, we believe that through the stimulation of ToM, WM would also be stimulated.

Detection of Executive Performance Profiles Using the ENFEN Battery in Children Diagnosed With Attention-Deficit Hyperactivity Disorder

Attention-deficit hyperactivity disorder (ADHD) is one of the most common neurodevelopmental disorders in children and adolescents. People who have this disorder are characterized by presenting difficulties in the processes of sustained attention, being very active, and having poor control of their impulses. Despite the high prevalence of this disorder and the existence of various tests used for its diagnosis, few data are available regarding the usefulness and diagnostic validity of these tools. Given the difficulties that these subjects present in executive functions, the aim of this study was to evaluate whether the Neuropsychological Assessment of Executive Functions battery for Children (ENFEN, for its acronym in Spanish, Portellano et al., 2009) allows to establish specific profiles of executive performance for people with attention-deficit hyperactivity disorder (ADHD). The sample was made up of 197 participants of both sexes, aged between 6 and 12 years age (134 with a clinical diagnosis and 63 without pathology). A nonexperimental design was followed, using a comparative descriptive study method. The results indicated that the scales of phonological fluency, color path, rings, and interference are the most associated with the diagnosis of ADHD, providing data on inhibition, mental flexibility, sustained and selective attention, planning, verbal fluency, and working memory, among others. The practical implication of these results is in line with providing support in the clinical diagnosis that is carried out in children's mental health units. In addition, the ENFEN tool can be valued as a suitable psychometric instrument in the psychoeducational field, helping professionals in a school environment to be more aware of the areas of cognitive development in which a student diagnosed with ADHD will have more difficulties and, in doing so, providing more adjusted and effective psychopedagogical measures when it comes to supporting students in their adaptation to the school environment.

ADHD subtype-specific cognitive correlates and association with self-esteem: a quantitative difference

BACKGROUND

Attention-deficit hyperactivity disorder (ADHD) is a major neurodevelopmental disorder with heterogeneous symptoms, subtypes, and cognitive deficits. Cognitive deficits are central to ADHD pathophysiology and one potential source of heterogeneity in ADHD. Subtype-specific cognitive correlates are not, however, well-studied. We explored cognitive correlates of ADHD subtypes based on the Wechsler Intelligence Scale for Children (WISC-IV) scores. We also assessed subtype-specific self-esteem rating in ADHD subtypes and explored its association with cognitive correlates.

METHODS

One hundred thirty-nine children with ADHD (80.6% boy, 19.4% girl) were categorized into the predominantly "hyperactive (ADHD-H)", "inattentive (ADHD-I)" and "combined (ADHD-C)" subtype based on their symptoms and scores on the Kiddie Schedule for Affective Disorders and Schizophrenia (K-SADS-PL) and Conners Parent-Rating Scale (CPRS-RS). They were then individually administrated the WISC-IV and completed a self-esteem inventory. Group differences in the WISC-IV indices and their predictability in discriminating ADHD subtypes were analyzed.

RESULTS

We found a quantitative differentiation of cognitive abilities among ADHD subtypes with "working memory" as the most compromised cognitive domain. ADHD-I had the poorest cognitive profile while ADHD-H scored highest in all cognitive domains. Importantly, cognitive abilities were negatively correlated with inattention and positively correlated with hyperactive symptoms. Moreover, self-esteem ratings were positively correlated with the cognitive domains and were rated differently based on the subtypes. ADHD-H, with the highest cognitive strength, reported the highest level of self-esteem among all subtypes.

CONCLUSIONS

ADHD subtype-specific symptoms, cognitive deficits, and self-esteem problems should be considered for precise diagnosis and effective and personalized treatment in ADHD in light of further supporting evidence and assessments. Cognitive interventions might be more compatible with and effective in inattentive and combined subtypes of ADHD. Working memory improving-based interventions can benefit all ADHD subtypes. A supportive educational system in school and providing adjunct supportive interventions should be considered for children with ADHD as well.

WISC-IV Intellectual Profiles in Korean Children and Adolescents with Attention Deficit/Hyperactivity Disorder

OBJECTIVE

This study aimed to compare the Wechsler Intelligence Scale for Children, Fourth Edition (WISC-IV) profiles of children with attention deficit/hyperactivity disorder (ADHD) and typically-developing children (TC) in Korea.

METHODS

The Korean version of the WISC-IV and the Advanced Test of Attention (ATA) were administered to 377 children and adolescents: 224 with ADHD (age 8.2±2.1 years, 182 boys) and 153 TC (age 8.7±2.4 years, 68 boys). Partial correlation and an analysis of covariance were used to investigate the relationship between the scores of the WISC-IV and the ATA.

RESULTS

The mean score of the full-scale intelligence quotient was lower in ADHD children than in TC (p<0.001). In analyses controlling for gender and with the full-scale intelligence quotient as a covariate, the working memory index (WMI) (p<0.001) and values of the Digit span subtest (p=0.001) of the WISC-IV were lower in the ADHD group than in TC. The WMI (r=-0.26, p<0.001) and its subtest Arithmetic scores (r=-0.25, p<0.001) were negatively correlated with Commission errors on the auditory ATA.

CONCLUSION

Children with ADHD have significantly lower WMI scores, which were clinically correlated with Commission errors on the auditory task of the ATA. Thus, the WMI is an indicator of attention deficit in children with ADHD.

Examining the Cognitive Proficiency Index in rehabilitation patients

This study examined the Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV) Cognitive Proficiency Index (CPI) in relation to other WAIS-IV indices, overall test battery mean (OTBM), and impairment (IMP) in an outpatient rehabilitation setting. Participants (N = 329) were 35% female and 88% Caucasian with average age and education of 42.9 (SD = 13.5) and 13.6 (SD = 2.4) years, respectively. Participants were grouped by diagnosis and validity: traumatic brain injury (TBI; n = 176; 39% mild), cerebrovascular accident (CVA; n = 52), other neurologic and psychiatric conditions (OTH; n = 49), and questionable performance validity (QPV; n = 52). OTBM was calculated from non-WAIS-IV tests; IMP was dichotomously defined as four or more non-WAIS-IV scores below cutoff (≤35 T). Significant group differences were observed on CPI, WAIS-IV indices, OTBM, and IMP. CPI significantly contributed (β = .51) to a linear regression model predicting OTBM (R² = .63) with education and GAI as covariates. A logistic regression model with IMP as the outcome and education, GAI, and CPI as predictors correctly classified 80% of cases with area under the curve of .86. A previously identified cutoff (CPI < 84) correctly classified 65-78% of clinical groups categorized by IMP. A novel cutoff (CPI ≤ 80) differentiated clinical participants with history of mild TBI from the QPV group with sensitivity of 44.2% and specificity of 89.7%. CPI showed incremental validity in predicting OTBM and IMP and warrants further study as a useful clinical addition to other WAIS-IV indices.

WISC-III Cognitive Profiles in Children with ADHD: Specific Cognitive Impairments and Diagnostic Utility

This study aimed to investigate the presence of specific cognitive impairments and the diagnostic utility of the WISC-III in children with ADHD. Ninety-eight children with ADHD and 81 children without ADHD matched by age and gender (control group), between the ages of 6 and 12 years, participated in the study. Children with ADHD revealed the most pronounced deficits in the subtests tapping working memory and processing speed. Freedom from Distractibility was the cognitive profile most impaired and that showed the highest diagnostic accuracy to discriminate children with ADHD. The optimal cutoff scores of the most common WISC-III cognitive profiles revealed greater diagnostic accuracy than the traditional approach of full or partial profiles. Taken together, these results suggested that in the context of a comprehensive psychological assessment, the WISC may provide knowledge about the specific cognitive strengths and weaknesses that characterize this disorder and may be useful in the decision-making process relative to ADHD diagnosis.

Utility of the General Ability Index (GAI) and Cognitive Proficiency Index (CPI) with survivors of pediatric brain tumors: Comparison to Full Scale IQ and premorbid IQ estimates

INTRODUCTION

Pediatric brain tumor survivors are at risk for working memory and processing speed impairment. The General Ability Index (GAI) provides an estimate of intellectual functioning that is less influenced by working memory and processing speed than a Full Scale IQ (FSIQ). The Cognitive Proficiency Index (CPI) provides a measure of efficient information processing derived from working memory and processing speed tasks. We examined the utility of the GAI and CPI to quantify neurocognitive outcomes in a sample of pediatric brain tumor survivors.

METHOD

GAI, CPI, and FSIQ scores from the Wechsler Intelligence Scale for Children-Fourth Edition (WISC-IV) were examined for 57 pediatric brain tumor survivors (ages 6-16 years) treated with cranial radiation therapy (RT).

RESULTS

GAI scores were higher than FSIQ and CPI scores, both p < .001. Lower CPI scores were associated with history of craniospinal irradiation and time since RT. Lower FSIQ and GAI scores were associated with higher RT dose and time since RT. The rate of clinically significant GAI-FSIQ discrepancies in our sample was greater than that observed in the WISC-IV standardization sample, p < .001. Estimated premorbid IQ scores were higher than GAI, p < .01, and FSIQ scores, p < .001.

CONCLUSIONS

Pediatric brain tumor survivors exhibit weaker cognitive proficiency than that expected for age, while general reasoning ability remains relatively spared. The GAI may be useful to quantify the intellectual potential of a survivor when appropriate accommodations are in place for relative cognitive proficiency weaknesses. The CPI may be a particularly sensitive outcome measure of treatment-related cognitive change in this population.

Construct Validity and Diagnostic Utility of the Cognitive Assessment System for ADHD

OBJECTIVE

The Das-Naglieri Cognitive Assessment System (CAS) is a test of cognitive abilities based on the Planning, Attention, Simultaneous, and Successive Theory (PASS). Studies of CAS performance by children with ADHD typically show lowest performance on Planning and deficits on Attention, but normal Simultaneous and Successive processing. Such distinct group differences studies support construct validity and are necessary, but not sufficient, for establishing diagnostic utility.

METHOD

Students meeting Diagnostic and Statistical Manual of Mental Disorders (4th ed., text rev.; DSM-IV-TR) criteria for ADHD (n = 20) and matched controls (n = 20) were assessed with the CAS to examine distinct group differences and diagnostic utility of CAS in correctly classifying the individuals from both groups.

RESULTS

CAS PASS scores were significantly related to ADHD characteristics demonstrating both distinct group differences (with medium to large effect sizes) and diagnostic utility (with medium effect sizes).

CONCLUSION

Support was observed for CAS PASS score characteristics previously observed among students with ADHD, and this is the first study to report on the diagnostic utility of CAS PASS scores. Given the small sample, additional large-scale studies and cross-validation is needed. (J. of Att. Dis. 2016; 20(6) 519-529).