1
|
Rohde A, Worrall L, Godecke E, O'Halloran R, Farrell A, Massey M. Diagnosis of aphasia in stroke populations: A systematic review of language tests. PLoS One 2018; 13:e0194143. [PMID: 29566043 PMCID: PMC5863973 DOI: 10.1371/journal.pone.0194143] [Citation(s) in RCA: 37] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2017] [Accepted: 02/26/2018] [Indexed: 11/24/2022] Open
Abstract
Background and purpose Accurate aphasia diagnosis is important in stroke care. A wide range of language tests are available and include informal assessments, tests developed by healthcare institutions and commercially published tests available for purchase in pre-packaged kits. The psychometrics of these tests are often reported online or within the purchased test manuals, not the peer-reviewed literature, therefore the diagnostic capabilities of these measures have not been systematically evaluated. This review aimed to identify both commercial and non-commercial language tests and tests used in stroke care and to examine the diagnostic capabilities of all identified measures in diagnosing aphasia in stroke populations. Methods Language tests were identified through a systematic search of 161 publisher databases, professional and resource websites and language tests reported to be used in stroke care. Two independent reviewers evaluated test manuals or associated resources for cohort or cross-sectional studies reporting the tests’ diagnostic capabilities (sensitivity, specificity, likelihood ratios or diagnostic odds ratios) in differentiating aphasic and non-aphasic stroke populations. Results Fifty-six tests met the study eligibility criteria. Six “non-specialist” brief screening tests reported sensitivity and specificity information, however none of these measures reported to meet the specific diagnostic needs of speech pathologists. The 50 remaining measures either did not report validity data (n = 7); did not compare patient test performance with a comparison group (n = 17); included non-stroke participants within their samples (n = 23) or did not compare stroke patient performance against a language reference standard (n = 3). Diagnostic sensitivity analysis was completed for six speech pathology measures (WAB, PICA, CADL-2, ASHA-FACS, Adult FAVRES and EFA-4), however all studies compared aphasic performance with that of non-stroke healthy controls and were consequently excluded from the review. Conclusions No speech pathology test was found which reported diagnostic data for identifying aphasia in stroke populations. A diagnostically validated post-stroke aphasia test is needed.
Collapse
Affiliation(s)
- Alexia Rohde
- School of Health and Rehabilitation Sciences, The University of Queensland, Brisbane, Queensland, Australia
| | - Linda Worrall
- School of Health and Rehabilitation Sciences, The University of Queensland, Brisbane, Queensland, Australia
| | - Erin Godecke
- School of Medical and Health Sciences, Edith Cowan University, Joondalup, Western Australia, Australia
| | - Robyn O'Halloran
- Department of Community and Clinical Allied Health, La Trobe University, Melbourne, Victoria, Australia
| | - Anna Farrell
- Department of Speech Pathology, Royal Brisbane and Women's Hospital, Brisbane, Queensland, Australia
| | - Margaret Massey
- School of Health and Rehabilitation Sciences, The University of Queensland, Brisbane, Queensland, Australia
| |
Collapse
|
2
|
Brackenbury T, Zickar MJ, Munson B, Storkel HL. Applying Item Response Theory to the Development of a Screening Adaptation of the Goldman-Fristoe Test of Articulation-Second Edition. JOURNAL OF SPEECH, LANGUAGE, AND HEARING RESEARCH : JSLHR 2017; 60:2672-2679. [PMID: 28859203 PMCID: PMC5831625 DOI: 10.1044/2017_jslhr-l-16-0392] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/07/2016] [Accepted: 04/11/2017] [Indexed: 05/13/2023]
Abstract
PURPOSE Item response theory (IRT) is a psychometric approach to measurement that uses latent trait abilities (e.g., speech sound production skills) to model performance on individual items that vary by difficulty and discrimination. An IRT analysis was applied to preschoolers' productions of the words on the Goldman-Fristoe Test of Articulation-Second Edition (GFTA-2) to identify candidates for a screening measure of speech sound production skills. METHOD The phoneme accuracies from 154 preschoolers, with speech skills on the GFTA-2 ranging from the 1st to above the 90th percentile, were analyzed with a 2-parameter logistic model. RESULTS A total of 108 of the 232 phonemes from stimuli in the sounds-in-words subtest fit the IRT model. These phonemes, and subgroups of the most difficult of these phonemes, correlated significantly with the children's overall percentile scores on the GFTA-2. Regression equations calculated for the 5 and 10 most difficult phonemes predicted overall percentile score at levels commensurate with other screening measures. CONCLUSIONS These results suggest that speech production accuracy can be screened effectively with a small number of sounds. They motivate further research toward the development of a screening measure of children's speech sound production skills whose stimuli consist of a limited number of difficult phonemes.
Collapse
|
3
|
Hula WD, Kellough S, Fergadiotis G. Development and Simulation Testing of a Computerized Adaptive Version of the Philadelphia Naming Test. JOURNAL OF SPEECH, LANGUAGE, AND HEARING RESEARCH : JSLHR 2015; 58:878-890. [PMID: 25813632 DOI: 10.1044/2015_jslhr-l-14-0297] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/22/2014] [Accepted: 02/26/2015] [Indexed: 06/04/2023]
Abstract
PURPOSE The purpose of this study was to develop a computerized adaptive test (CAT) version of the Philadelphia Naming Test (PNT; Roach, Schwartz, Martin, Grewal, & Brecher, 1996), to reduce test length while maximizing measurement precision. This article is a direct extension of a companion article (Fergadiotis, Kellough, & Hula, 2015), in which we fitted the PNT to a 1-parameter logistic item-response-theory model and examined the validity and precision of the resulting item parameter and ability score estimates. METHOD Using archival data collected from participants with aphasia, we simulated two PNT-CAT versions and two previously published static PNT short forms, and compared the resulting ability score estimates to estimates obtained from the full 175-item PNT. We used a jackknife procedure to maintain independence of the samples used for item estimation and CAT simulation. RESULTS The PNT-CAT recovered full PNT scores with equal or better accuracy than the static short forms. Measurement precision was also greater for the PNT-CAT than the static short forms, though comparison of adaptive and static nonoverlapping alternate forms showed minimal differences between the two approaches. CONCLUSION These results suggest that CAT assessment of naming in aphasia has the potential to reduce test burden while maximizing the accuracy and precision of score estimates.
Collapse
|
4
|
Ivanova MV, Hallowell B. A tutorial on aphasia test development in any language: Key substantive and psychometric considerations. APHASIOLOGY 2013; 27:891-920. [PMID: 23976813 PMCID: PMC3747007 DOI: 10.1080/02687038.2013.805728] [Citation(s) in RCA: 20] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/31/2023]
Abstract
BACKGROUND There are a limited number of aphasia language tests in the majority of the world's commonly spoken languages. Furthermore, few aphasia tests in languages other than English have been standardized and normed, and few have supportive psychometric data pertaining to reliability and validity. The lack of standardized assessment tools across many of the world's languages poses serious challenges to clinical practice and research in aphasia. AIMS The current review addresses this lack of assessment tools by providing conceptual and statistical guidance for the development of aphasia assessment tools and establishment of their psychometric properties. MAIN CONTRIBUTION A list of aphasia tests in the 20 most widely spoken languages is included. The pitfalls of translating an existing test into a new language versus creating a new test are outlined. Factors to consider in determining test content are discussed. Further, a description of test items corresponding to different language functions is provided, with special emphasis on implementing important controls in test design. Next, a broad review of principal psychometric properties relevant to aphasia tests is presented, with specific statistical guidance for establishing psychometric properties of standardized assessment tools. CONCLUSIONS This article may be used to help guide future work on developing, standardizing and validating aphasia language tests. The considerations discussed are also applicable to the development of standardized tests of other cognitive functions.
Collapse
Affiliation(s)
- Maria V. Ivanova
- Neurolinguistics Laboratory, Faculty of Philology, National Research University Higher School of Economics, Ul. Myasnickaya, d. 20, Moscow, Russia, 101000
| | - Brooke Hallowell
- Communication Sciences and Disorders, Ohio University Grover Center, W 218, Athens, OH, USA, 45701
| |
Collapse
|
5
|
Paula JJD, Bertola L, Nicolato R, Moraes END, Malloy-Diniz LF. Evaluating language comprehension in Alzheimer's disease: the use of the Token test. ARQUIVOS DE NEURO-PSIQUIATRIA 2012; 70:435-40. [PMID: 22699541 DOI: 10.1590/s0004-282x2012000600010] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/20/2011] [Accepted: 01/19/2012] [Indexed: 11/21/2022]
Abstract
OBJECTIVE To analyze the psychometric properties of the Token test (TT), a verbal comprehension test, and its applicability to the diagnosis of mild Alzheimer's disease (AD). METHODS One hundred and sixty participants (80 AD and 80 controls) performed the TT and a short battery of neuropsychological tests designed to evaluate general cognitive status, working memory and executive functions. Internal consistency, factor structure, correlation with other measures and group comparisons were evaluated. RESULTS The test evinced good internal consistency and a bi-factorial structure (related to comprehension and attention). Differences between AD and controls were significant, however the TT presented only moderate sensitivity and specificity for the AD diagnosis. CONCLUSION The TT showed evidence of good psychometric properties and adequacy for characterizing comprehension deficits in AD, but it was not an appropriate test for the AD detection and diagnosis.
Collapse
Affiliation(s)
- Jonas Jardim de Paula
- Faculdade de Filosofia e Ciências Humanas da Universidade Federal de Minas Gerais (UFMG), Belo Horizonte MG, Brazil.
| | | | | | | | | |
Collapse
|
6
|
Hula WD, Fergadiotis G, Martin N. Model choice and sample size in item response theory analysis of aphasia tests. AMERICAN JOURNAL OF SPEECH-LANGUAGE PATHOLOGY 2012; 21:S38-S50. [PMID: 22230175 DOI: 10.1044/1058-0360(2011/11-0090)] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/31/2023]
Abstract
PURPOSE The purpose of this study was to identify the most appropriate item response theory (IRT) measurement model for aphasia tests requiring 2-choice responses and to determine whether small samples are adequate for estimating such models. METHOD Pyramids and Palm Trees (Howard & Patterson, 1992) test data that had been collected from individuals with aphasia were analyzed, and the resulting item and person estimates were used to develop simulated test data for 3 sample size conditions. The simulated data were analyzed using a standard 1-parameter logistic (1-PL) model and 3 models that accounted for the influence of guessing: augmented 1-PL and 2-PL models and a 3-PL model. The model estimates obtained from the simulated data were compared to their known true values. RESULTS With small and medium sample sizes, an augmented 1-PL model was the most accurate at recovering the known item and person parameters; however, no model performed well at any sample size. Follow-up simulations confirmed that the large influence of guessing and the extreme easiness of the items contributed substantially to the poor estimation of item difficulty and person ability. CONCLUSION Incorporating the assumption of guessing into IRT models improves parameter estimation accuracy, even for small samples. However, caution should be exercised in interpreting scores obtained from easy 2-choice tests, regardless of whether IRT modeling or percentage correct scoring is used.
Collapse
Affiliation(s)
- William D Hula
- VA Pittsburgh Healthcare System and University of Pittsburgh, PA, USA.
| | | | | |
Collapse
|
7
|
Edmonds LA, Donovan NJ. Item-level psychometrics and predictors of performance for Spanish/English bilingual speakers on an object and action naming battery. JOURNAL OF SPEECH, LANGUAGE, AND HEARING RESEARCH : JSLHR 2012; 55:359-81. [PMID: 22215032 DOI: 10.1044/1092-4388(2011/10-0307)] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/24/2023]
Abstract
PURPOSE There is a pressing need for psychometrically sound naming materials for Spanish/English bilingual adults. To address this need, in this study the authors examined the psychometric properties of An Object and Action Naming Battery (An O&A Battery; Druks & Masterson, 2000) in bilingual speakers. METHOD Ninety-one Spanish/English bilinguals named O&A Battery items in English and Spanish. Responses underwent a Rasch analysis. Using correlation and regression analyses, the authors evaluated the effect of psycholinguistic (e.g., imageability) and participant (e.g., proficiency ratings) variables on accuracy. RESULTS Rasch analysis determined unidimensionality across English and Spanish nouns and verbs and robust item-level psychometric properties, evidence for content validity. Few items did not fit the model, there were no ceiling or floor effects after uninformative and misfit items were removed, and items reflected a range of difficulty. Reliability coefficients were high, and the number of statistically different ability levels provided indices of sensitivity. Regression analyses revealed significant correlations between psycholinguistic variables and accuracy, providing preliminary construct validity. The participant variables that contributed most to accuracy were proficiency ratings and time of language use. CONCLUSIONS Results suggest adequate content and construct validity of O&A items retained in the analysis for Spanish/English bilingual adults and support future efforts to evaluate naming in older bilinguals and persons with bilingual aphasia.
Collapse
|
8
|
Gutman R, DeDe G, Caplan D, Liu JS. Rasch Model and Its Extensions for Analysis of Aphasic Deficits in Syntactic Comprehension. J Am Stat Assoc 2011. [DOI: 10.1198/jasa.2011.ap10017] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
|
9
|
Gallardo G, Guàrdia J, Villaseñor T, McNeil MR. Psychometric data for the revised token test in normally developing Mexican children ages 4-12 years. Arch Clin Neuropsychol 2011; 26:225-34. [PMID: 21441259 DOI: 10.1093/arclin/acr018] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022] Open
Abstract
Language comprehension is vital to social and educational development but few pediatric tests are available for its assessment. To approach this problem, two versions of the Token Test (TT), "TT short form" (DeRenzi & Faglioni, 1978) and "Revised Token Test" (RTT), were first compared. Using a sample of 88 normally developing Spanish-speaking children, the tests were compared on their: (a) established psychometric development and (b) internal consistency. The RTT was judged to be superior and was selected for additional experimentation. The RTT was compared with a developmental measure of lexical knowledge on a cross-sectional sample of 250 4-12-year-old normally developing Spanish-speaking children. A significant positive and high correlation supports its concurrent validity. Significant differences across the age groups, along with a principal component analysis that yielded a three-factor structure, support its construct validity. Preliminary normative data across the nine age groups are provided.
Collapse
Affiliation(s)
- Geisa Gallardo
- Departamento de Neurociencias, Centro Universitario de Ciencias de la Salud, Universidad de Guadalajara, Mexico
| | | | | | | |
Collapse
|
10
|
Gutman R, DeDe G, Michaud J, Liu JS, Caplan D. Rasch models of aphasic performance on syntactic comprehension tests. Cogn Neuropsychol 2011; 27:230-44. [PMID: 21058076 DOI: 10.1080/02643294.2010.512285] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Abstract
Responses of 42 people with aphasia to 11 sentence types in enactment and sentence-picture matching tasks were characterized using Rasch models that varied in the inclusion of the factors of task, sentence type, and patient group. The best fitting models required the factors of task and patient group but not sentence type. The results provide evidence that aphasic syntactic comprehension is best accounted for by models that include different estimates of patient ability in different tasks and different difficulty of all sentences in different groups of patients, but that do not include different estimates of patient ability for different types of sentences.
Collapse
Affiliation(s)
- Roee Gutman
- Department of Statistics, Harvard College, Cambridge, MA, USA
| | | | | | | | | |
Collapse
|
11
|
Baylor CR, Yorkston KM, Eadie TL, Miller RM, Amtmann D. Developing the communicative participation item bank: Rasch analysis results from a spasmodic dysphonia sample. JOURNAL OF SPEECH, LANGUAGE, AND HEARING RESEARCH : JSLHR 2009; 52:1302-20. [PMID: 19717652 PMCID: PMC3074579 DOI: 10.1044/1092-4388(2009/07-0275)] [Citation(s) in RCA: 46] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/15/2023]
Abstract
PURPOSE The purpose of this study was to conduct the initial psychometric analyses of the Communicative Participation Item Bank-a new self-report instrument designed to measure the extent to which communication disorders interfere with communicative participation. This item bank is intended for community-dwelling adults across a range of communication disorders. METHOD A set of 141 candidate items was administered to 208 adults with spasmodic dysphonia. Participants rated the extent to which their condition interfered with participation in various speaking communication situations. Questionnaires were administered online or in a paper version per participant preference. Participants also completed the Voice Handicap Index (B. H. Jacobson et al., 1997) and a demographic questionnaire. Rasch analyses were conducted using Winsteps software (J. M. Linacre, 1991). RESULTS The results show that items functioned better when the 5-category response format was recoded to a 4-category format. After removing 8 items that did not fit the Rasch model, the remaining 133 items demonstrated strong evidence of sufficient unidimensionality, with the model accounting for 89.3% of variance. Item location values ranged from -2.73 to 2.20 logits. CONCLUSIONS Preliminary Rasch analyses of the Communicative Participation Item Bank show strong psychometric properties. Further testing in populations with other communication disorders is needed.
Collapse
Affiliation(s)
- Carolyn R Baylor
- Department of Rehabilitation Medicine, University of Washington, Seattle, WA, USA.
| | | | | | | | | |
Collapse
|
12
|
Milman LH, Dickey MW, Thompson CK. A psychometric analysis of functional category production in English agrammatic narratives. BRAIN AND LANGUAGE 2008; 105:18-31. [PMID: 18255135 PMCID: PMC2926308 DOI: 10.1016/j.bandl.2007.12.008] [Citation(s) in RCA: 10] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/19/2007] [Revised: 12/11/2007] [Accepted: 12/12/2007] [Indexed: 05/25/2023]
Abstract
Hierarchical models of agrammatism propose that sentence production deficits can be accounted for in terms of clausal syntactic structure [Friedmann, N., & Grodzinsky, Y. (1997). Tense and agreement in agrammatic production: Pruning the syntactic tree. Brain and Language, 56, 397-425; Hagiwara, H. (1995). The breakdown of functional categories and the economy of derivation. Brain and Language, 50, 92-116]. Such theories predict that morpho-syntactic elements associated with higher nodes in the syntactic tree (complementizers and verb inflections) will be more impaired than elements associated with lower structural positions (negation markers and aspectual verb forms). While this hypothesis has been supported by the results of several studies [Benedet, M. J., Christiansen, J. A., & Goodglass, H. (1998). A cross-linguistic study of grammatical morphology in Spanish- and English-speaking agrammatic patients. Cortex, 34, 309-336; Friedmann, N. (2001). Agrammatism and the psychological reality of the syntactic tree. Journal of Psycholinguistic Research, 30, 71-88; Friedmann, N. (2002). Question production in agrammatism: The tree pruning hypothesis. Brain and Language, 80, 160-187], it has also been challenged on several grounds [Burchert, F., Swoboda-Moll, M., & De Bleser, R. (2005a). Tense and agreement dissociations in German agrammatic speakers: Underspecification vs. hierarchy. Brain and Language, 94, 188-199; Lee, M. (2003). Dissociations among functional categories in Korean agrammatism. Brain and Language, 84, 170-188; Lee, J., Milman, L. H., & Thompson, C. K. (2005). Functional category production in agrammatic speech. Brain and Language, 95, 123-124]. In this paper the question of hierarchical structure was re-examined within the framework of Item Response Theory [IRT, Rasch, G. (1980). Probabilistic models for some intelligence and attainment tests (Expanded ed.). Chicago: University of Chicago Press]. IRT is a probabilistic model widely used in the field of psychometrics to model behavioral constructs as numeric variables. In this study we examined production of functional categories (complementizers, verb inflections, negation markers, and aspectual verb forms) in narrative samples elicited from 18 individuals diagnosed with nonfluent aphasia and 18 matched controls. Data from the aphasic participants were entered into an IRT analysis to test (1) whether production of clausal functional categories can be represented as a variable on a numeric scale; and (2) whether production patterns were consistent with hierarchical syntactic structure. Pearson r correlation coefficients were also computed to determine whether there was a relation between functional category production and other indices of language performance. Results indicate that functional category production can be modeled as a numeric variable using IRT. Furthermore, although variability was observed across individuals, consistent patterns were evident when the data were interpreted within a probabilistic framework. Although functional category production was moderately correlated with a second measure of clausal structure (clause length), it was not correlated with more distant language constructs (noun/verb ratio and WAB A.Q.). These results suggest that functional category production is related to some, but not all, measures of agrammatic language performance.
Collapse
Affiliation(s)
- Lisa H Milman
- Department of Speech and Hearing Science, Ohio State University, 1070 Carmack Road, Columbus, OH 43210, USA.
| | | | | |
Collapse
|
13
|
Milman LH, Holland A, Kaszniak AW, D'Agostino J, Garrett M, Rapcsak S. Initial validity and reliability of the SCCAN: using tailored testing to assess adult cognition and communication. JOURNAL OF SPEECH, LANGUAGE, AND HEARING RESEARCH : JSLHR 2008; 51:49-69. [PMID: 18230855 DOI: 10.1044/1092-4388(2008/004)] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/25/2023]
Abstract
PURPOSE The Scales of Cognitive and Communicative Ability for Neurorehabilitation (SCCAN; L. Milman & A. Holland, 2007) was developed in the hospital setting to address changes in assessment practice. The SCCAN was designed to provide an overview of impairment and activity limitations across 8 cognitive scales (Speech Comprehension, Oral Expression, Reading, Writing, Orientation, Attention, Memory, and Problem Solving). The scales were developed using item response theory so that tailored testing could be implemented to reduce test administration time. This research investigated the validity and reliability of the SCCAN. METHOD A battery of neuropsychological tests was administered to 40 neurologically healthy control participants and 51 participants diagnosed with left-hemisphere pathology, right-hemisphere pathology, or probable Alzheimer's disease. Analyses were performed to assess test sensitivity and specificity, construct validity, administration time, and reliability. RESULTS The test accurately classified 95% of the control participants and 98% of the participants diagnosed with neurological disorders. Results indicate that the test also differentiated the performance profiles of the 3 clinical populations. In addition, test scores correlated significantly with external measures of the same cognitive areas. Mean administration time was 34 min. Test-retest stability (r = .96, p < .001) and internal consistency (r = .99, p < .001) coefficients were both significant, indicating that tailored testing procedures generated reliable test scores.
Collapse
Affiliation(s)
- Lisa H Milman
- Department of Speech and Hearing Science, Ohio State University, 110 Pressey Hall, Columbus, OH 43210, USA.
| | | | | | | | | | | |
Collapse
|