The effect of achievement test selection on identification of learning disabilities within a patterns of strengths and weaknesses framework

Identifying students with dyslexia: exploration of current assessment methods

Early identification plays a crucial role in providing timely support to students with learning disabilities, such as dyslexia, in order to overcome their reading difficulties. However, there is significant variability in the methods used for identifying dyslexia. This study aimed to explore and understand the practices of dyslexia identification in the UK. A survey was conducted among 274 dyslexia professionals, including educational psychologists and dyslexia specialists, to investigate the types of assessments they employ, their approach to utilizing assessment data, their decision-making processes, and their conceptualization of dyslexia. Additionally, the study examined whether these professionals held any misconceptions or myths associated with dyslexia. Analysis of the survey data revealed substantial variability in how professionals conceptualize dyslexia, as well as variations in assessment methods. Furthermore, a significant proportion of the survey respondents subscribed to one or more misconceptions regarding dyslexia; the most common misconception identified among professionals was the belief that children with dyslexia read letters in reverse order. The findings highlight the need for standardized approaches to dyslexia identification and debunking prevailing misconceptions. The implications of these findings are discussed, emphasizing the importance of informed policy and practice in supporting students with dyslexia. Recommendations are provided to enhance consistency and accuracy in dyslexia identification, with the aim of facilitating early intervention and support for affected students.

Assessing Beliefs About Intrinsic and Extrinsic Determinants of SLD: Evaluating the Factor Structure of a Novel Instrument

Schools conduct comprehensive psychoeducational evaluations to identify students with specific learning disabilities (SLDs) and determine whether they qualify for special education services. This decision-making process is complex and research has documented many factors influencing SLD identification decisions. One such factor may be decision-makers' beliefs about the underlying causes of SLD, including intrinsic and extrinsic factors. However, no studies to date have examined the underlying factor structure of the responses to prompts about the causes of SLD from intrinsic and extrinsic perspectives. This study was conducted with a sample of 521 school psychologists as part of a larger study examining decision-making during SLD identification. Using confirmatory factor analyses (CFA) to compare two theoretically plausible models, results suggested that a single latent factor best captured variability in responses to these prompts. Implications for assessing beliefs and how they impact the psychoeducational assessment process to identify SLDs are discussed, along with areas for future research.

Are students with learning disabilities correctly diagnosed in Turkey?

There has been an ongoing debate on how to diagnose learning disability (LD), with experts proposing different approaches. Very limited research is available on the diagnosis of LD in Turkey. This study aims to analyze the problems related to the diagnosis of LD in Turkey. For this purpose, the intelligence test results of the diagnosed students, the opinions of the teachers who referred them for the diagnosis, and the views of the child psychiatrists and psychological counselors involved in the diagnosis were examined. The results showed that the IQ-Achievement Discrepancy approach is implemented in Turkey. It was also revealed that the students' Wechsler Intelligence Scale for Children (WISC-R) average score was 80.31, and teachers mostly reported the weaknesses of the students during the diagnosis process. Psychological counselors and psychiatrists pointed out the outdatedness of the tools used in the diagnosis, the limited number of measurement instruments to be used in the diagnosis, referral of too many students, and the lack of specialists to work in the diagnosis. The results are discussed on the basis of diagnostic approaches.

Assessment of Specific Learning Disabilities and Intellectual Disabilities

We review literature related to the assessment and identification of Specific Learning Disabilities (SLD) and Intellectual Disabilities (ID). SLD and ID are the only two disorders requiring psychometric test performance for identification within the group of neurodevelopmental disorders in Diagnostic and Statistical Manual - 5. SLD and ID are considered exclusionary of one another, but the processes for assessment and identification of each disorder vary. There is controversy about the identification and assessment methods for SLD, with little consensus. Unlike ID, SLD is weakly related to full-scale IQ, and there is insufficient evidence that the routine assessment of IQ or cognitive skills adds value to SLD identification and treatment. We have proposed a hybrid method based on the assessment of low achievement with norm-referenced tests, instructional response, and other disorders and contextual factors that may be comorbid or contraindicative of SLD. In contrast to SLD, there is strong consensus for a three-prong definition for the identification and assessment of ID: (a) significantly subaverage IQ, (b) adaptive behavior deficits that interfere with independent living in the community, and (c) age of onset in the developmental period. For both SLD and ID, we identify areas of controversy and best practices for identification and assessment.

Internalizing Symptoms and Reading Difficulties Among Early Elementary School Students

While the field of learning disabilities has grown substantially over the past several decades (Grigorenko et al. in Am Psychol 75:37, 2020) little work has explored the role of internalizing symptoms among struggling students. The present study compared struggling and typical readers on several child reported internalizing measures at both the beginning and end of a school year during which time they received either classroom-as-usual or research-team provided intensive intervention. Struggling readers who did and did not meet reading benchmarks were also compared at year-end. While minimal differences were present at the beginning of the year, numerous differences were observed at the end, with students exhibiting persistent reading struggles reporting significantly greater distress. Bi-directional associations emerged with beginning of year group status predicting internalizing symptoms and beginning of year internalizing symptoms predicting end of year intervention response group status. Findings are discussed in terms of future directions for enhancing intervention studies of struggling readers.

Executive Function: Association with Multiple Reading Skills

Executive function (EF) is related to reading. However, there is a lack of clarity around (a) the relative contribution of different components of EF to different reading components (word reading, fluency, comprehension), and (b) how EF operates in the context of known strong language predictors (e.g., components of the Simple View of Reading or SVR), and other skills theoretically related to reading (e.g., vocabulary, processing speed) and/or to EF (e.g., short-term memory, motor function). In a large sample of 3rd to 5th graders oversampled for struggling readers, this paper evaluates the impact of EF derived from a bifactor model (Cirino, Ahmed, Miciak, Taylor, Gerst, & Barnes, 2018) in the context of well-known covariates and demographics. Beyond common EF, five specific factors (two related to working memory, and factors of fluency, self-regulated learning, and behavioral inattention/metacognition) were addressed. EF consistently showed a unique contribution to already-strong predictive models for all reading outcomes; for reading comprehension, EF interacted with SVR indices (word reading and listening comprehension). The findings extend and refine our understanding of the contribution of EF to reading skill.

Cognitive, Intervention, and Neuroimaging Perspectives on Executive Function in Children With Reading Disabilities

The role of executive function (EF) in the reading process, and in those with reading difficulties, remains unclear. As members of the Texas Center for Learning Disabilities, we review multiple perspectives regarding EF in reading and then summarize some of our recent studies of struggling and typical readers in grades 3-5. Study 1a found that a bi-factor structure best represented a comprehensive assessment of EF. Study 1b found that cognitive and behavioral measures of EF related independently to math and reading. Study 1c found that EF related to reading, above and beyond other variables, but Study 1d found no evidence that adding an EF training component improved intervention response. Study 1e found that pretest EF abilities did not relate to intervention response. Neuroimaging studies examined EF-related brain activity during both reading and nonlexical EF tasks. In Study 2a, the EF task evoked control activity, but generated no differences between struggling and typical readers. The reading task, however, had group differences in both EF and reading regions. In Study 2b, EF activity during reading at pretest was related to intervention response. Across studies, EF appears involved in the reading process. There is less evidence for general EF predicting or improving intervention outcomes.

A Misuse of IQ Scores: Using the Dual Discrepancy/Consistency Model for Identifying Specific Learning Disabilities

The purpose of this article is to describe the origins of patterns of strengths and weaknesses (PSW) methods for identifying specific learning disabilities (SLD) and to provide a comprehensive review of the assumptions and evidence supporting the most commonly-used PSW method in the United States: Dual Discrepancy/Consistency (DD/C). Given their use in determining whether students have access to special education and related services, it is important that any method used to identify SLD have supporting evidence. A review of the DD/C evidence indicates it cannot currently be classified as an evidence-based method for identifying individuals with a SLD. We show that the DD/C method is unsound for three major reasons: (a) it requires test scores have properties that they fundamentally lack, (b) lack of experimental utility evidence supporting its use, and (c) evidence supporting the inability of the method to identify SLD accurately.

Reading Profiles in Multi-Site Data With Missingness

Children with reading disability exhibit varied deficits in reading and cognitive abilities that contribute to their reading comprehension problems. Some children exhibit primary deficits in phonological processing, while others can exhibit deficits in oral language and executive functions that affect comprehension. This behavioral heterogeneity is problematic when missing data prevent the characterization of different reading profiles, which often occurs in retrospective data sharing initiatives without coordinated data collection. Here we show that reading profiles can be reliably identified based on Random Forest classification of incomplete behavioral datasets, after the missForest method is used to multiply impute missing values. Results from simulation analyses showed that reading profiles could be accurately classified across degrees of missingness (e.g., ∼5% classification error for 30% missingness across the sample). The application of missForest to a real multi-site dataset with missingness (n = 924) showed that reading disability profiles significantly and consistently differed in reading and cognitive abilities for cases with and without missing data. The results of validation analyses indicated that the reading profiles (cases with and without missing data) exhibited significant differences for an independent set of behavioral variables that were not used to classify reading profiles. Together, the results show how multiple imputation can be applied to the classification of cases with missing data and can increase the integrity of results from multi-site open access datasets.

Are WISC IQ scores in children with mathematical learning disabilities underestimated? The influence of a specialized intervention on test performance

BACKGROUND

Intelligence measures play a pivotal role in the diagnosis of mathematical learning disabilities (MLD). Probably as a result of math-related material in IQ tests, children with MLD often display reduced IQ scores. However, it remains unclear whether the effects of math remediation extend to IQ scores.

AIMS

The present study investigated the impact of a special remediation program compared to a control group receiving private tutoring (PT) on the WISC IQ scores of children with MLD.

METHODS

We included N=45 MLD children (7-12 years) in a study with a pre- and post-test control group design. Children received remediation for two years on average.

RESULTS

The analyses revealed significantly greater improvements in the experimental group on the Full-Scale IQ, and the Verbal Comprehension, Perceptual Reasoning, and Working Memory indices, but not Processing Speed, compared to the PT group. Children in the experimental group showed an average WISC IQ gain of more than ten points.

CONCLUSION

Results indicate that the WISC IQ scores of MLD children might be underestimated and that an effective math intervention can improve WISC IQ test performance. Taking limitations into account, we discuss the use of IQ measures more generally for defining MLD in research and practice.

Adoption Costs Associated With Processing Strengths and Weaknesses Methods for Learning Disabilities Identification

There is controversy regarding the relative merits of cognitive assessment for the identification of learning disabilities. Proponents of cognitive assessment have suggested that multitiered systems of support (MTSS) should be supplemented with routine, systematic assessment of cognitive processes following a determination of inadequate response to evidence-based interventions in order to document a pattern of processing strengths and weaknesses (PSW methods) as an inclusionary criterion for learning disabilities. However, the financial costs incurred by this addition to MTSS are not well known. In the present study, we present a systematic case study to estimate the costs associated with adopting routine assessment of cognitive processing for students referred for special education evaluation. We estimate that implementation within a district would cost between $1,960 and $2,400 per student, assuming no existing infrastructure. These expenses are discussed in relation to evidence for the educational value of such assessments and inherent trade-offs between assessment and intervention.

Simulation of LD Identification Accuracy Using a Pattern of Processing Strengths and Weaknesses Method With Multiple Measures

We investigated the classification accuracy of learning disability (LD) identification methods premised on the identification of an intraindividual pattern of processing strengths and weaknesses (PSW) method using multiple indicators for all latent constructs. Known LD status was derived from latent scores; values at the observed level identified LD status for individual cases according to the concordance/discordance method. Agreement with latent status was evaluated using (a) a single indicator, (b) two indicators as part of a test-retest "confirmation" model, and (c) a mean score. Specificity and negative predictive value (NPV) were generally high for single indicators (median specificity = 98.8%, range = 93.4%-99.7%; median NPV = 94.2%, range = 85.6%-98.7%), but low for sensitivity (median sensitivity = 49.1%, range = 20.3%-77.1%) and positive predictive value (PPV; median PPV = 48.8%, range = 23.5%-69.6%). A test-retest procedure produced inconsistent and small improvements in classification accuracy, primarily in "not LD" decisions. Use of a mean score produced small improvements in classifications (mean improvement = 2.0%, range = 0.3%-2.8%). The modest gains in agreement do not justify the additional testing burdens associated with incorporating multiple tests of all constructs.

Cognitive discrepancy models for specific learning disabilities identification: Simulations of psychometric limitations

Few studies have investigated specific learning disabilities (SLD) identification methods based on the identification of patterns of processing strengths and weaknesses (PSW). We investigated the reliability of SLD identification decisions emanating from different achievement test batteries for 1 method to operationalize the PSW approach: the concordance/discordance model (C/DM; Hale & Fiorello, 2004). Two studies examined the level of agreement for SLD identification decisions between 2 different simulated, highly correlated achievement test batteries. Study 1 simulated achievement and cognitive data across a wide range of potential latent correlations between an achievement deficit, a cognitive strength and a cognitive weakness. Latent correlations permitted simulation of case-level data at specified reliabilities for cognitive abilities and 2 achievement observations. C/DM criteria were applied and resulting SLD classifications from the 2 achievement test batteries were compared for agreement. Overall agreement and negative agreement were high, but positive agreement was low (0.33-0.59) across all conditions. Study 2 isolated the effects of reduced test reliability on agreement for SLD identification decisions resulting from different test batteries. Reductions in reliability of the 2 achievement tests resulted in average decreases in positive agreement of 0.13. Conversely, reductions in reliability of cognitive measures resulted in small average increases in positive agreement (0.0-0.06). Findings from both studies are consistent with prior research demonstrating the inherent instability of classifications based on C/DM criteria. Within complex ipsative SLD identification models like the C/DM, small variations in test selection can have deleterious effects on classification reliability. (PsycINFO Database Record

Do Processing Patterns of Strengths and Weaknesses Predict Differential Treatment Response?

OBJECTIVE

No previous empirical study has investigated whether the LD identification decisions of proposed methods to operationalize processing strengths and weaknesses (PSW) approaches for LD identification are associated with differential treatment response. We investigated whether the identification decisions of the concordance/discordance model (C/DM; Hale & Fiorello, 2004) and Cross Battery Assessment approach (XBA method; Flanagan, Ortiz, & Alfonso, 2007) were consistent and whether they predicted intervention response beyond that accounted for by pretest performance on measures of reading.

METHOD

Psychoeducational assessments were administered at pretest to 203 4^th graders with low reading comprehension and individual results were utilized to identify students who met LD criteria according to the C/DM and XBA methods and students who did not. Resulting group status permitted an investigation of agreement for identification methods and whether group status at pretest (LD or not LD) was associated with differential treatment response to an intensive reading intervention.

RESULTS

The LD identification decisions of the XBA and C/DM demonstrated poor agreement with one another (κ = -.10). Comparisons of posttest performance for students who met LD criteria and those who did not meet were largely null, with small effect sizes across all measures.

CONCLUSIONS

LD status, as identified through the C/DM and XBA approaches, was not associated with differential treatment response and did not contribute educationally meaningful information about how students would respond to intensive reading intervention. These results do not support the value of cognitive assessment utilized in this way as part of the LD identification process.