Determining Whether Or When To Adopt New Versions Of Psychological And Neuropsychological Tests: Ethical And Professional Considerations

New Versions of the MMPI and Rorschach: How Have Training Programs Responded?

In this brief article, we update the training of newer versions of the Minnesota Multiphasic Personality Inventory (MMPI) and Rorschach and compare to a 2015 assessment training survey of American Psychological Association accredited clinical psychology doctoral programs. The survey sample sizes for 2015, 2021, and 2022 were 83, 81, and 88, respectively. By 2015, of the programs teaching any adult MMPI version, almost all (94%) were still teaching the MMPI-2, and 68% had started teaching the MMPI-2-RF. In 2021 and 2022, respectively, almost all programs (96% and 94%) had started teaching the MMPI-2-RF or MMPI-3, although most were still teaching the MMPI-2 (77% and 66%). By 2015, of the programs teaching the Rorschach, 85% were still teaching the Comprehensive System (CS) and 60% had started teaching the Rorschach Performance Assessment System (R-PAS). In 2021 and 2022, respectively, most programs had started teaching R-PAS (77% and 77%) although many (65% and 50%) were still teaching the CS. Therefore, doctoral programs are indeed switching to newer versions of the MMPI and Rorschach, although more slowly than one might expect. We recommend that APA provide more guidance in selecting test versions for training programs, practitioners, and researchers.

Neuropsychological Deficits in Disordered Screen Use Behaviours: A Systematic Review and Meta-analysis

Over the last few decades, excessive and disordered screen use has become more prevalent, prompting investigations into its associated consequences. The extent to which disordered screen use behaviours impact neuropsychological functioning has been reportedly mixed and at times inconsistent. This review sought to synthesise the literature and estimate the magnitude of overall cognitive impairment across a wide range of disordered screen use behaviours. We also sought to determine the cognitive domains most impacted, and whether the observed impairments were moderated by the classification of screen-related behaviours (i.e., Internet or gaming) or the format of cognitive test administration (i.e., paper-and-pencil or computerised). A systematic search of databases (Embase, PsycINFO, MEDLINE) identified 43 cross-sectional articles that assessed neuropsychological performance in disordered screen use populations, 34 of which were included in the meta-analysis. A random-effects meta-analysis revealed significant small/medium (g = .38) cognitive deficits for individuals with disordered screen use behaviours relative to controls. The most affected cognitive domain with a significant medium effect size (g = .50) was attention and focus followed by a significant reduction in executive functioning (g = .31). The classification of disordered screen use behaviours into Internet or gaming categories or the format of cognitive testing did not moderate these deficits. Additionally, excluding disordered social media use in an exploratory analysis had little effect on the observed outcomes. This study highlights a number of methodological considerations that may have contributed to disparate findings and shows that disordered screen use can significantly impact cognitive performance. Recommendations for future research are also discussed. Data for this study can be found at https://osf.io/upeha/ .

Optimal MoCA cutoffs for detecting biologically-defined patients with MCI and early dementia

OBJECTIVE

In this phase II psychometric study on the Montreal cognitive assessment (MoCA), we tested the clinicometric properties of Italian norms for patients with mild cognitive impairment (PwMCI) and early dementia (PwD) and provided optimal cutoffs for diagnostic purposes.

METHODS

Retrospective data collection was performed for consecutive patients with clinically and biologically defined MCI and early dementia. Forty-five patients (24 PwMCI and 21 PwD) and 25 healthy controls were included. Raw MoCA scores were adjusted according to the conventional 1-point correction (Nasreddine) and Italian norms (Conti, Santangelo, Aiello). The diagnostic properties of the original cutoff (< 26) and normative cutoffs, namely, the upper limits (uLs) of equivalent scores (ES) 1, 2, and 3, were evaluated. ROC curve analysis was performed to obtain optimal cutoffs.

RESULTS

The original cutoff demonstrated high sensitivity (0.93 [95% CI 0.84-0.98]) but low specificity (0.44 [0.32-0.56]) in discriminating between patients and controls. Nominal normative cutoffs (ES0 uLs) showed excellent specificity (SP range = 0.96-1.00 [0.88-1.00]) but poor sensitivity (SE range = 0.09-0.24 [0.04-0.36]). The optimal cutoff for Nasreddine's method was 23.50 (SE = 0.82 [0.71-0.90]; SP = 0.72 [0.60-0.82]). Optimal cutoffs were 20.97, 22.85, and 22.29 (SE range = 0.69-0.73 [0.57-0.83], SP range = 0.88-0.92 [0.77-0.97]) for Conti's, Santangelo's, and Aiello's methods, respectively.

CONCLUSION

Using the 1-point correction, combined with a cutoff of 23.50, might be useful in ambulatory settings with a large turnout. Our optimal cutoffs can offset the poor sensitivity of Italian cutoffs.

Guidelines for the revision and use of revised psychological tests: A systematic review study

Tests are updated and revised periodically in order to remain current, valid and reliable in a competitive psychological testing industry. Despite the prevalence of test revisions, especially in recent years, a number of authors have commented on the lack of comprehensive guidelines for test revision. Whilst some guideline documents from test associations have mentioned test revision, these guidelines tend to be focussed on test user responsibility, with limited guidance for practitioners embarking on a test revision project. Test revision is expensive and time consuming, leaving little scope for experimentation or trial-and-error. Test revision deserves a comprehensive document that addresses aspects such as what the different types of revision are, when to embark on a revision, what process to follow and how test users should use revised tests. The current study developed a comprehensive and practical set of 23 guidelines across ten phases of a revision project to assist revision teams, test users and publishers. These guidelines were peer-reviewed and refined.

The Montreal Cognitive Assessment (MoCA): updated norms and psychometric insights into adaptive testing from healthy individuals in Northern Italy

Background

The availability of fine-grained, culture-specific psychometric outcomes can favor the interpretation of scores of the Montreal Cognitive Assessment (MoCA), the most frequently used instrument to screen for mild cognitive dysfunctions in both instrumental and non-instrumental domains. This study thus aimed at providing: (i) updated, region-specific norms for the Italian MoCA, by also (ii) comparing them to pre-existing ones with higher geographical coverage; (iii) information on sensitivity and discriminative capability at the item level.

Methods

Five hundred and seventy nine healthy individuals from Northern Italy (208 males, 371 females; age: 63.4 ± 15, 21–96; education: 11.3 ± 4.6, 1–25) were administered the MoCA. Item Response Theory (IRT) was adopted to assess item difficulty and discrimination. Normative values were derived by means of the Equivalent Scores (ESs) method, applied to the MoCA and its sub-scales. Average ESs were also computed. Agreement with previous ESs classification was assessed via Cohen’s k.

Results

Age and education significantly predicted all MoCA measures except for Orientation, which was related to age only. No sex differences were detected when tested along with age and education. Substantial disagreements with previous ESs classifications were detected. Several items proved to be scarcely sensitive, especially the place item from Orientation and the letter detection task. Memory items showed high discriminative capability, along with certain items assessing executive functions and orientation.

Discussion

Item-level information herewith provided for the Italian MoCA can help interpret its scores by Italian practitioners. Italian practitioners should consider an adaptive use of region-specific norms for the MoCA.

Investigating the compatibility, interchangeability, and clinical usability of BOT2-BF and BOT2-SF in primary school-aged children with and without developmental dyslexia

The aim of our study was to determine the sensitivity and specificity levels of BOT2-BF and BOT2-SF tests applied to children with developmental dyslexia and to examine their clinical usability. This study was designed as a case-controlled study. 82 (study group 41, control group 41) 7-12 year children were included. The study occurred three stages. In the first place, the similarities and differences of BOT2-BF and BOT2-SF tests observed in clinical practices were examined by taking the feedback of experts and test correlations. In the second step, BOT2-BF and BOT2-SF tests were examined with reference to the Core Outcome Set (COS) evaluation criteria. In the last step, in order to examine the sensitivity and specificity levels of the tests, assessments were performed in two groups with 1-week intervals. According to the COS results, the BOT2-BF test was examined better than the BOT2-SF test under cross-cultural validity title. The sensitivity and specificity of BOT2-SF were 0.92 and 0.51, and for BOT2-BF were 0.92 and 0.41, respectively. This study will help clinicians working on children with developmental dyslexia to decide the type of test they will use to evaluate and easy access to materials motor proficiency.

Investigating the interchangeability and clinical utility of MVPT-3 and MVPT-4 for 7-10 year children with and without specific learning disabilities

The aim of our study was to determine the sensitivity and specificity levels of MVPT-3 and MVPT-4 in 7-10 year children with specific learning disabilities (SLD) and to compare the clinical usability of these assessment tools. This study was designed as a case-controlled study. 96 ( study group = 48, control group = 48) 7-10 year children were included. The study occurred three stages. In the first place, MVPT-3 and MVPT-4 tests were examined with reference to the Core Outcome Set(COS) evaluation criteria. In the second step, the similarities and differences of MVPT-3 and MVPT-4 tests observed in clinical practices were examined by taking the feedback of experts. In the last step, in order to examine the sensitivity and specificity levels of the tests, assessments were performed in two groups with 1-week intervals. According to the COS results; MVPT-3 test was examined better than MVPT-4 test under cross-cultural validity title. The sensitivity and specificity of the MVPT-3 test were 0.68 and 0.77, and the sensitivity and specificity of MVPT-4 test were 0.50 and 0.87 respectively. This study will help clinicians working on children aged 7-10 to decide the type of test they will use to evaluate visual perception processes.

Neuropsychological assessment of traumatic brain injury: Current ethical challenges and recommendations for future practice

Numerous ethical challenges may arise over the course of neuropsychological assessment. This paper highlights the ethical considerations associated with neuropsychological assessment of individuals with traumatic brain injury. Issues regarding professional competency, providing and obtaining informed consent, neuropsychological test selection and administration, effectively communicating assessment results, and working as part of a multidisciplinary team are discussed with practical recommendations. Ultimately, a comprehensive understanding of these issues as well as an integration of resources to guide clinical practice will contribute to ethical decision-making and strong professional practice.

Reliable Change on Neuropsychological Tests in the Uniform Data Set

Longitudinal normative data obtained from a robust elderly sample (i.e., believed to be free from neurodegenerative disease) are sparse. The purpose of the present study was to develop reliable change indices (RCIs) that can assist with interpretation of test score changes relative to a healthy sample of older adults (ages 50+). Participants were 4217 individuals who completed at least three annual evaluations at one of 34 past and present Alzheimer's Disease Centers throughout the United States. All participants were diagnosed as cognitively normal at every study visit, which ranged from three to nine approximately annual evaluations. One-year RCIs were calculated for 11 neuropsychological variables in the Uniform Data Set by regressing follow-up test scores onto baseline test scores, age, education, visit number, post-baseline assessment interval, race, and sex in a linear mixed effects regression framework. In addition, the cumulative frequency distributions of raw score changes were examined to describe the base rates of test score changes. Baseline test score, age, education, and race were robust predictors of follow-up test scores across most tests. The effects of maturation (aging) were more pronounced on tests related to attention and executive functioning, whereas practice effects were more pronounced on tests of episodic and semantic memory. Interpretation of longitudinal changes on 11 cognitive test variables can be facilitated through the use of reliable change intervals and base rates of score changes in this robust sample of older adults. A Web-based calculator is provided to assist neuropsychologists with interpretation of longitudinal change.

Investigating the interchangeability and diagnostic utility of the PPVT-III and PPVT-IV for children with and without SLI

The receptive vocabulary performance of pre-school children with specific language impairment (SLI) and typically-developing (TD) controls was compared on the Peabody Picture Vocabulary Test third and fourth edition (PPVT-III and PPVT-IV) to determine consistency in performance and the effect of test revision on identification of impairment. Participants included 40 pre-school children with SLI and 40 controls. Tests were administered in counterbalanced order. Despite a strong relationship between performance on these two tests (p < 0.001), 35% of children performed differently between the two test versions. Children with SLI performed significantly worse than TD children on both tests (p < 0.001). Discriminate analyses identified an optimal standard score cut-off of 103 for both tests. Using this cut-off, sensitivity remained consistent at 80% (95% CI = 0.64-0.90), while specificity was 75% (95% CI = 0.59-0.87) on the PPVT-III and 70% (95% CI = 0.53-0.83) on the PPVT-IV. The results suggest that the two tests do not appear to be interchangeable for more than 1/3rd of children. The findings also highlight the misperception that newer test versions are superior to older in identifying presence or absence of language impairment. Children with SLI are unlikely to score low on these commonly used receptive vocabulary tests, despite known deficits of children with SLI in the area of vocabulary acquisition. Possible explanations for why children with SLI score well on these types of tests will be discussed.

Spanish multicenter normative studies (Neuronorma project): norms for the abbreviated Barcelona Test

The abbreviated Barcelona Test (a-BT) is an instrument widely used in Spain and Latin American countries for general neuropsychological assessment. The purpose of the present study was to provide new norms for the a-BT as part of the Neuronorma project. The sample consisted of 346 healthy controls. Overlapping cell procedure and midpoint techniques were applied to develop the normative data. Age, education, and sex influences were studied. Results indicated that although age and education affected the score on this test, sex did not. Raw scores were transformed to age-adjusted scaled scores (SS(A)) based on percentile ranks. These SS(A) were also converted into age-education scaled scores using a linear regression model. Norms were presented on age-education scaled scores. Also, the a-BT cognitive profile was presented and should prove to be clinically useful for interpretation. These co-normed data will allow clinicians to compare scores from a-BT with all the tests included in the Neuronorma project.