1
|
Brooks BL, MacAllister WS, Fay-McClymont TB, Mish S, Vasserman M, Sherman EMS. Brief memory assessment in children: can the ChAMP Screening Index detect memory impairment? Child Neuropsychol 2024; 30:673-683. [PMID: 37782220 DOI: 10.1080/09297049.2023.2259540] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2023] [Accepted: 08/31/2023] [Indexed: 10/03/2023]
Abstract
Abbreviated memory batteries play a role in some clinical and research assessments, but their validity and accuracy need to be well supported. The purpose of this study was to examine the accuracy of the ChAMP Screening Index for detecting memory impairment. The sample included N = 804 youths (ages 5-21 years) with medical and neurological diagnoses who were presented for a clinical neuropsychological assessment. All completed the full Child and Adolescent Memory Profile and had valid data. The ChAMP Screening Index contains the first two subtests of the battery (Lists and Objects) and takes about 10 min to administer (full ChAMP is about 35 min). Analyses to examine the accuracy of the ChAMP Screening Index with both the Total Memory Index and Delayed Memory Index included Intraclass correlations, Cohen's Kappa coefficients, receiver operating characteristics, and classification metrics (e.g., sensitivity, specificity, positive predictive values [PPV], and negative predictive values [NPV]). Very strong correlations, minimal mean difference scores, substantial agreement on kappa coefficients, and outstanding receiver operating characteristics all support the Screening Index accuracy. A cutoff score on the Screening Index of 70 provides a good balance between a high PPV (.91) and a high NPV (.96) for the Total Memory Index. When detecting impairment on the Delayed Memory Index, a Screening Index cutoff score of 65 provides a balance between a high PPV (.92) and a high NPV (.94). This study supports the accuracy, validity, and utility of the 10-min ChAMP Screening Index in those clinical and research situations where a brief evaluation of memory is desired.
Collapse
Affiliation(s)
- Brian L Brooks
- Neurosciences Program, Alberta Children's Hospital, Calgary, AB, Canada
- Departments of Pediatrics, Clinical Neurosciences, and Psychology, University of Calgary, Calgary, AB, Canada
- Child Brain and Mental Health program, Alberta Children's Hospital Research Institute, Calgary, AB, Canada
| | - William S MacAllister
- Neurosciences Program, Alberta Children's Hospital, Calgary, AB, Canada
- Child Brain and Mental Health program, Alberta Children's Hospital Research Institute, Calgary, AB, Canada
- Department of Pediatrics, University of Calgary, Calgary, AB, Canada
| | - Taryn B Fay-McClymont
- Neurosciences Program, Alberta Children's Hospital, Calgary, AB, Canada
- Child Brain and Mental Health program, Alberta Children's Hospital Research Institute, Calgary, AB, Canada
- Department of Pediatrics, University of Calgary, Calgary, AB, Canada
- Department of Psychology, University of British Columbia Okanagan, Kelowna, BC, Canada
- Private Practice, Kelowna, BC, Canada
| | - Sandra Mish
- Neurosciences Program, Alberta Children's Hospital, Calgary, AB, Canada
| | - Marsha Vasserman
- Neurosciences Program, Alberta Children's Hospital, Calgary, AB, Canada
- Child Brain and Mental Health program, Alberta Children's Hospital Research Institute, Calgary, AB, Canada
- Department of Pediatrics, University of Calgary, Calgary, AB, Canada
| | | |
Collapse
|
2
|
Brooks BL, Fay-McClymont TB, MacAllister WS, Vasserman M, Mish S, Sherman EMS. New Child and Adolescent Memory Profile Embedded Performance Validity Test. ARCHIVES OF CLINICAL NEUROPSYCHOLOGY : THE OFFICIAL JOURNAL OF THE NATIONAL ACADEMY OF NEUROPSYCHOLOGISTS 2023:6972889. [PMID: 36617240 DOI: 10.1093/arclin/acac110] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 12/14/2022] [Indexed: 01/09/2023]
Abstract
OBJECTIVE It is essential to interpret performance validity tests (PVTs) that are well-established and have strong psychometrics. This study evaluated the Child and Adolescent Memory Profile (ChAMP) Validity Indicator (VI) using a pediatric sample with traumatic brain injury (TBI). METHOD A cross-sectional sample of N = 110 youth (mean age = 15.1 years, standard deviation [SD] = 2.4 range = 8-18) on average 32.7 weeks (SD = 40.9) post TBI (71.8% mild/concussion; 3.6% complicated mild; 24.6% moderate-to-severe) were administered the ChAMP and two stand-alone PVTs. Criterion for valid performance was scores above cutoffs on both PVTs; criterion for invalid performance was scores below cutoffs on both PVTs. Classification statistics were used to evaluate the existing ChAMP VI and establish a new VI cutoff score if needed. RESULTS There were no significant differences in demographics or time since injury between those deemed valid (n = 96) or invalid (n = 14), but all ChAMP scores were significantly lower in those deemed invalid. The original ChAMP VI cutoff score was highly specific (no false positives) but also highly insensitive (sensitivity [SN] = .07, specificity [SP] = 1.0). Based on area under the curve (AUC) analysis (0.94), a new cutoff score was established using the sum of scaled scores (VI-SS). A ChAMP VI-SS score of 32 or lower achieved strong SN (86%) and SP (92%). Using a 15% base rate, positive predictive value was 64% and negative predictive value was 97%. CONCLUSIONS The originally proposed ChAMP VI has insufficient SN in pediatric TBI. However, this study yields a promising new ChAMP VI-SS, with classification metrics that exceed any other current embedded PVT in pediatrics.
Collapse
Affiliation(s)
- Brian L Brooks
- Neurosciences Program, Alberta Children's Hospital, Calgary, Alberta T3B 6A8, Canada.,Departments of Pediatrics, Clinical Neurosciences, and Psychology, University of Calgary, Calgary, Alberta T2N 1N4, Canada.,Child Brain and Mental Health Section, Alberta Children's Hospital Research Institute, University of Calgary, Calgary, Alberta T2N 1N4, Canada
| | - Taryn B Fay-McClymont
- Neurosciences Program, Alberta Children's Hospital, Calgary, Alberta T3B 6A8, Canada.,Child Brain and Mental Health Section, Alberta Children's Hospital Research Institute, University of Calgary, Calgary, Alberta T2N 1N4, Canada.,Department of Pediatrics, University of Calgary, Calgary, Alberta T2N 1N4, Canada.,Department of Psychology, University of British Columbia Okanagan, Kelowna, British Columbia V1V 1V7, Canada
| | - William S MacAllister
- Neurosciences Program, Alberta Children's Hospital, Calgary, Alberta T3B 6A8, Canada.,Child Brain and Mental Health Section, Alberta Children's Hospital Research Institute, University of Calgary, Calgary, Alberta T2N 1N4, Canada.,Department of Pediatrics, University of Calgary, Calgary, Alberta T2N 1N4, Canada
| | - Marsha Vasserman
- Neurosciences Program, Alberta Children's Hospital, Calgary, Alberta T3B 6A8, Canada.,Child Brain and Mental Health Section, Alberta Children's Hospital Research Institute, University of Calgary, Calgary, Alberta T2N 1N4, Canada.,Department of Pediatrics, University of Calgary, Calgary, Alberta T2N 1N4, Canada
| | - Sandra Mish
- Neurosciences Program, Alberta Children's Hospital, Calgary, Alberta T3B 6A8, Canada
| | | |
Collapse
|
3
|
Nayar K, Ventura LM, DeDios-Stern S, Oh A, Soble JR. The Impact of Learning and Memory on Performance Validity Tests in a Mixed Clinical Pediatric Population. Arch Clin Neuropsychol 2021; 37:50-62. [PMID: 34050354 DOI: 10.1093/arclin/acab040] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/04/2021] [Indexed: 11/13/2022] Open
Abstract
OBJECTIVE This study examined the degree to which verbal and visuospatial memory abilities influence performance validity test (PVT) performance in a mixed clinical pediatric sample. METHOD Data from 252 consecutive clinical pediatric cases (Mage=11.23 years, SD=4.02; 61.9% male) seen for outpatient neuropsychological assessment were collected. Measures of learning and memory (e.g., The California Verbal Learning Test-Children's Version; Child and Adolescent Memory Profile [ChAMP]), performance validity (Test of Memory Malingering Trial 1 [TOMM T1]; Wechsler Intelligence Scale for Children-Fifth Edition [WISC-V] or Wechsler Adult Intelligence Scale-Fourth Edition Digit Span indices; ChAMP Overall Validity Index), and intellectual abilities (e.g., WISC-V) were included. RESULTS Learning/memory abilities were not significantly correlated with TOMM T1 and accounted for relatively little variance in overall TOMM T1 performance (i.e., ≤6%). Conversely, ChAMP Validity Index scores were significantly correlated with verbal and visual learning/memory abilities, and learning/memory accounted for significant variance in PVT performance (12%-26%). Verbal learning/memory performance accounted for 5%-16% of the variance across the Digit Span PVTs. No significant differences in TOMM T1 and Digit Span PVT scores emerged between verbal/visual learning/memory impairment groups. ChAMP validity scores were lower for the visual learning/memory impairment group relative to the nonimpaired group. CONCLUSIONS Findings highlight the utility of including PVTs as standard practice for pediatric populations, particularly when memory is a concern. Consistent with the adult literature, TOMM T1 outperformed other PVTs in its utility even among the diverse clinical sample with/without learning/memory impairment. In contrast, use of Digit Span indices appear to be best suited in the presence of visuospatial (but not verbal) learning/memory concerns. Finally, the ChAMP's embedded validity measure was most strongly impacted by learning/memory performance.
Collapse
Affiliation(s)
- Kritika Nayar
- Department of Psychiatry and Behavioral Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
| | - Lea M Ventura
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA.,Department of Pediatrics, University of Illinois College of Medicine, Chicago, IL, USA
| | - Samantha DeDios-Stern
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA
| | - Alison Oh
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA
| | - Jason R Soble
- Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA.,Department of Neurology, University of Illinois College of Medicine, Chicago, IL, USA
| |
Collapse
|
4
|
Kirk JW, Baker DA, Kirk JJ, MacAllister WS. A review of performance and symptom validity testing with pediatric populations. APPLIED NEUROPSYCHOLOGY-CHILD 2020; 9:292-306. [DOI: 10.1080/21622965.2020.1750118] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/11/2022]
Affiliation(s)
- John W. Kirk
- Kirk Neurobehavioral Health, Louisville, CO, USA
| | - David A. Baker
- Department of Physical Medicine & Rehabilitation, Children’s Hospital Colorado, University of Colorado School of Medicine, Aurora, CO, USA
| | | | - William S. MacAllister
- Alberta Children’s Hospital Research Institute and Alberta Health Sciences, Alberta, Canada
| |
Collapse
|