Verbal and non-verbal recognition memory assessment: validation of a computerized version of the Recognition Memory Test

BACKGROUND

The use of computerized devices for neuropsychological assessment (CNADs) as an effective alternative to the traditional pencil-and-paper modality has recently increased exponentially, both in clinical practice and research, especially due to the pandemic. However, several authors underline that the computerized modality requires the same psychometric validity as "in-presence" tests. The current study aimed at building and validating a computerized version of the verbal and non-verbal recognition memory test (RMT) for words, unknown faces and buildings.

METHODS

Seventy-two healthy Italian participants, with medium-high education and ability to proficiently use computerized systems, were enrolled. The sample was subdivided into six groups, one for each age decade. Twelve neurological patients with mixed aetiology, age and educational level were also recruited. Both the computerized and the paper-and-pencil versions of the RMT were administered in two separate sessions.

RESULTS

In healthy participants, the computerized and the paper-and-pencil versions of the RMT showed statistical equivalence for words, unknown faces and buildings. In the neurological patients, no statistical difference was found between the performance at the two versions of the RMT. A moderate-to-good inter-rater reliability between the two versions was also found in both samples. Finally, the computerized version of the RMT was perceived as acceptable by both healthy participants and neurological patients at System Usability Scale (SUS).

CONCLUSION

The computerized version of the RMT can be used as a reliable alternative to the traditional version.

Performance Validity Test Failure in the Clinical Population: A Systematic Review and Meta-Analysis of Prevalence Rates

Performance validity tests (PVTs) are used to measure the validity of the obtained neuropsychological test data. However, when an individual fails a PVT, the likelihood that failure truly reflects invalid performance (i.e., the positive predictive value) depends on the base rate in the context in which the assessment takes place. Therefore, accurate base rate information is needed to guide interpretation of PVT performance. This systematic review and meta-analysis examined the base rate of PVT failure in the clinical population (PROSPERO number: CRD42020164128). PubMed/MEDLINE, Web of Science, and PsychINFO were searched to identify articles published up to November 5, 2021. Main eligibility criteria were a clinical evaluation context and utilization of stand-alone and well-validated PVTs. Of the 457 articles scrutinized for eligibility, 47 were selected for systematic review and meta-analyses. Pooled base rate of PVT failure for all included studies was 16%, 95% CI [14, 19]. High heterogeneity existed among these studies (Cochran's Q = 697.97, p < .001; I2 = 91%; τ2 = 0.08). Subgroup analysis indicated that pooled PVT failure rates varied across clinical context, presence of external incentives, clinical diagnosis, and utilized PVT. Our findings can be used for calculating clinically applied statistics (i.e., positive and negative predictive values, and likelihood ratios) to increase the diagnostic accuracy of performance validity determination in clinical evaluation. Future research is necessary with more detailed recruitment procedures and sample descriptions to further improve the accuracy of the base rate of PVT failure in clinical practice.

Lowered cutoffs to reduce false positives on the Word Memory Test

OBJECTIVE

To adjust the decision criterion for the Word Memory Test (WMT, Green, 2003) to minimize the frequency of false positives.

METHOD

Archival data were combined into a database (n = 3,210) to examine the best cut score for the WMT. We compared results based on the original scoring rules and those based on adjusted scoring rules using a criterion based on 16 performance validity tests (PVTs) exclusive of the WMT. Cutoffs based on peer-reviewed publications and test manuals were used. The resulting PVT composite was considered the best estimate of validity status. We focused on a specificity of .90 with a false-positive rate of less than .10 across multiple samples.

RESULTS

Each examinee was administered the WMT, as well as on average 5.5 (SD = 2.5) other PVTs. Based on the original scoring rules of the WMT, 31.8% of examinees failed. Using a single failure on the criterion PVT (C-PVT), the base rate of failure was 45.9%. When requiring two or more failures on the C-PVT, the failure rate dropped to 22.8%. Applying a contingency analysis (i.e., X2) to the two failures model on the C-PVT measure and using the original rules for the WMT resulted in only 65.3% agreement. However, using our adjusted rules for the WMT, which consisted of relying on only the IR and DR WMT subtest scores with a cutoff of 77.5%, agreement between the adjusted and the C-PVT criterion equaled 80.8%, for an improvement of 12.1% identified. The adjustmeny resulted in a 49.2% reduction in false positives while preserving a sensitivity of 53.6%. The specificity for the new rules was 88.8%, for a false positive rate of 11.2%.

CONCLUSIONS

Results supported lowering of the cut score for correct responding from 82.5% to 77.5% correct. We also recommend discontinuing the use of the Consistency subtest score in the determination of WMT failure.

Assessing time-based prospective memory online: A comparison study between laboratory-based and web-based testing

Prospective memory (PM, i.e., the ability to remember and perform future intentions) is assessed mainly within laboratory settings; however, in the last two decades, several studies have started testing PM online. Most part of those studies focused on event-based PM (EBPM), and only a few assessed time-based PM (TBPM), possibly because time keeping is difficult to control or standardise without experimental control. Thus, it is still unclear whether time monitoring patterns in online studies replicate typical patterns obtained in laboratory tasks. In this study, we therefore aimed to investigate whether the behavioural outcome measures obtained from the traditional TBPM paradigm in the laboratory-accuracy and time monitoring-are comparable with an online version in a sample of 101 younger adults. Results showed no significant difference in TBPM performance in the laboratory versus online setting, as well as no difference in time monitoring. However, we found that participants were somewhat faster and more accurate at the ongoing task during the laboratory assessment, but those differences were not related to holding an intention in mind. The findings suggest that, although participants seemed generally more distracted when tested remotely, online assessment yielded similar results in key temporal characteristics and behavioural performance as for the laboratory assessment. The results are discussed in terms of possible conceptual and methodological implications for online testing.

Reliability and validity of the electronic version of the Hopkins verbal learning test-revised in middle-aged and elderly Chinese people

Background

The aging population is increasing, making it essential to have a standardized, convenient, and valid electronic memory test that can be accessed online for older people and caregivers. The electronic version of the Hopkins Verbal Learning Test-Revised (HVLT-R) as a test with these advantages and its reliability and validity has not yet been tested. Thus, this study examined the reliability and validity of the electronic version of the HVLT-R in middle-aged and elderly Chinese people to provide a scientific basis for its future dissemination and use.

Methods

We included 1,925 healthy participants aged over 40, among whom 38 were retested after 3-6 months. In addition, 65 participants completed both the pad and paper-and-pencil versions of the HVLT-R (PAP-HVLT-R). We also recruited 42 Alzheimer's disease (AD) patients, and 45 amnestic mild cognitive impairment (aMCI) patients. All participants completed the Pad-HVLT-R, the Hong Kong Brief Cognitive Test (HKBC), the Brief Visual Memory Test-Revised (BVMT-R), and the Logical Memory Test (LM).

Results

(1) Reliability: the Cronbach's α value was 0.94, the split-half reliability was 0.96. The test-retest correlation coefficients were moderate, ranging from 0.38 to 0.65 for direct variables and 0.16 to 0.52 for derived variables; (2) Concurrent validity: the Pad-HVLT-R showed a moderate correlation with the HKBC and BVMT-R, with correlation coefficients between total recall of 0.41 and 0.54, and between long-delayed recall of 0.42 and 0.59, respectively. It also showed a high correlation with the LM, with correlation coefficients of 0.72 for total recall and 0.62 for long-delayed recall; (3) Convergent validity: the Pad-HVLT-R was moderately correlated with the PAP version, with correlation coefficients ranging from 0.29 to 0.53 for direct variables and 0.15 to 0.43 for derived variables; (4) Discriminant capacity: the Pad-HVLT-R was effective in differentiating AD patients, as demonstrated by the ROC analysis with AUC values of 0.834 and 0.934 for total recall and long-delayed recall, respectively.

Conclusion

(1) The electronic version of HVLT-R has good reliability and validity in middle-aged and elderly Chinese people; (2) The electronic version of HVLT-R can be used as an effective tool to distinguish AD patients from healthy people.

The limitations and challenges in the assessment of executive dysfunction associated with real-world functioning: The opportunity of serious games

Nowadays, there is a broad range of methods for detecting and evaluating executive dysfunction ranging from clinical interview to neuropsychological evaluation. Nevertheless, a critical issue of these assessments is the lack of correspondence of the neuropsychological test's results with real-world functioning. This paper proposes serious games as a new framework to improve the neuropsychological assessment of real-world functioning. We briefly discuss the contribution and limitations of current methods of evaluation of executive dysfunction (paper-and-pencil tests, naturalistic observation methods, and Information and Communications Technologies) to inform on daily life functioning. Then, we analyze what are the limitations of these methods to predict real-world performance: (1) A lack of appropriate instruments to investigate the complexity of real-world functioning, (2) the vast majority of neuropsychological tests assess well-structured tasks, and (3) measurement of behaviors are based on simplistic data collection and statistical analysis. This work shows how serious games offer an opportunity to develop more efficient tools to detect executive dysfunction in everyday life contexts. Serious games provide meaningful narrative stories and virtual or real environments that immerse the user in natural and social environments with social interactions. In those highly interactive game environments, the player needs to adapt his/her behavioral performance to novel and ill-structured tasks which are suited for collecting user interaction evidence. Serious games offer a novel opportunity to develop better tools to improve diagnosis of the executive dysfunction in everyday life contexts. However, more research is still needed to implement serious games in everyday clinical practice.

Device-based physical activity and late-life depressive symptoms: An analysis of influential factors using share data

BACKGROUND

Regular physical activity is effective for preventing and managing depression; however, only a few studies have assessed physical activity using device-based measures. We aimed to examine the association between device-based total physical activity and late-life depressive symptoms and explore which factors may explain this relationship.

METHODS

Data from 10 European countries who participated in wave 8 of the Survey of Health, Ageing and Retirement in Europe (SHARE) were analyzed. Triaxial accelerometers (Axivity AX3; Axivity, Newcastle UK) were used to assess total physical activity. Depressive symptoms were assessed through the EURO-D scale. Possion regression models and mediation analysis were performed.

RESULTS

The final sample included 785 older adults (70 participants had missing data in one or more variables and were excluded from the analysis) [mean (SD): age 68.6 (8.8) years; 59.2 % female]. After adjusting for several confounders, a 10 % increase in the mean acceleration was associated with lower depressive symptoms (B = -0.0228; 95%CI = -0.0395, -0.0061). This association was partly explained by limitations in activities of daily living and quality of life.

LIMITATIONS

Cross-sectional design, convenience sample, bi-directionality, lack of cutoff points for classification of the Axivity AX3 placed in the thigh, into intensities.

CONCLUSIONS

Higher device-based total physical activity was linked with lower depressive symptoms in older European adults. Future studies employing device-based measures that allow to better capture important physical activity dimensions (i.e., intensity) will better inform about dose-response relationships.

Using Technology for the Efficient and Precise Assessment of Cognitive Skills in Countries with Limited Standardized Assessment Instruments: A Report on the Case of Saudi Arabia

In Saudi Arabia, the country’s progress toward appropriate and inclusive education programs for children with disabilities is still evolving. A crucial aspect of this evolution has been the development of a comprehensive assessment battery that covers a broad range of cognitive factors for the diagnosis of neurodevelopment disorders and other types of intellectual atypicalities, including giftedness. The Alif–Ya Assessment Battery consists of 47 subtests based on various theories of intelligence. Alif–Ya capitalizes on advanced technologies to enable its delivery either in person or remotely. Moreover, over half of Alif–Ya’s subtests are adaptive; items are selected for the test takers based on their previous responses. In this paper, we provide an overview of the Alif–Ya Assessment Battery, describe how it was designed to make the best use of the latest and best features of technology for the appropriate and accurate assessment of children and adolescents in the Kingdom of Saudi Arabia via remote or in-person administration, and present initial data collected with the battery.

A New Remote Guided Method for Supervised Web-Based Cognitive Testing to Ensure High-Quality Data: Development and Usability Study

BACKGROUND

The global COVID-19 pandemic has triggered a fundamental reexamination of how human psychological research can be conducted safely and robustly in a new era of digital working and physical distancing. Online web-based testing has risen to the forefront as a promising solution for the rapid mass collection of cognitive data without requiring human contact. However, a long-standing debate exists over the data quality and validity of web-based studies. This study examines the opportunities and challenges afforded by the societal shift toward web-based testing and highlights an urgent need to establish a standard data quality assurance framework for online studies.

OBJECTIVE

This study aims to develop and validate a new supervised online testing methodology, remote guided testing (RGT).

METHODS

A total of 85 healthy young adults were tested on 10 cognitive tasks assessing executive functioning (flexibility, memory, and inhibition) and learning. Tasks were administered either face-to-face in the laboratory (n=41) or online using remote guided testing (n=44) and delivered using identical web-based platforms (Cambridge Neuropsychological Test Automated Battery, Inquisit, and i-ABC). Data quality was assessed using detailed trial-level measures (missed trials, outlying and excluded responses, and response times) and overall task performance measures.

RESULTS

The results indicated that, across all data quality and performance measures, RGT data was statistically-equivalent to in-person data collected in the lab (P>.40 for all comparisons). Moreover, RGT participants out-performed the lab group on measured verbal intelligence (P<.001), which could reflect test environment differences, including possible effects of mask-wearing on communication.

CONCLUSIONS

These data suggest that the RGT methodology could help ameliorate concerns regarding online data quality-particularly for studies involving high-risk or rare cohorts-and offer an alternative for collecting high-quality human cognitive data without requiring in-person physical attendance.

Performance validity test failure in clinical populations-a systematic review

Performance validity tests (PVTs) are widely used in attempts to quantify effort and/or detect negative response bias during neuropsychological testing. However, it can be challenging to interpret the meaning of poor PVT performance in a clinical context. Compensation-seeking populations predominate in the PVT literature. We aimed to establish base rates of PVT failure in clinical populations without known external motivation to underperform. We searched MEDLINE, EMBASE and PsycINFO for studies reporting PVT failure rates in adults with defined clinical diagnoses, excluding studies of active or veteran military personnel, forensic populations or studies of participants known to be litigating or seeking disability benefits. Results were summarised by diagnostic group and implications discussed. Our review identified 69 studies, and 45 different PVTs or indices, in clinical populations with intellectual disability, degenerative brain disease, brain injury, psychiatric disorders, functional disorders and epilepsy. Various pass/fail cut-off scores were described. PVT failure was common in all clinical groups described, with failure rates for some groups and tests exceeding 25%. PVT failure is common across a range of clinical conditions, even in the absence of obvious incentive to underperform. Failure rates are no higher in functional disorders than in other clinical conditions. As PVT failure indicates invalidity of other attempted neuropsychological tests, the finding of frequent and unexpected failure in a range of clinical conditions raises important questions about the degree of objectivity afforded to neuropsychological tests in clinical practice and research.

Comparing Web-Based and Lab-Based Cognitive Assessment Using the Cambridge Neuropsychological Test Automated Battery: A Within-Subjects Counterbalanced Study

Background

Computerized assessments are already used to derive accurate and reliable measures of cognitive function. Web-based cognitive assessment could improve the accessibility and flexibility of research and clinical assessment, widen participation, and promote research recruitment while simultaneously reducing costs. However, differences in context may influence task performance.

Objective

This study aims to determine the comparability of an unsupervised, web-based administration of the Cambridge Neuropsychological Test Automated Battery (CANTAB) against a typical in-person lab-based assessment, using a within-subjects counterbalanced design. The study aims to test (1) reliability, quantifying the relationship between measurements across settings using correlational approaches; (2) equivalence, the extent to which test results in different settings produce similar overall results; and (3) agreement, by quantifying acceptable limits to bias and differences between measurement environments.

Methods

A total of 51 healthy adults (32 women and 19 men; mean age 36.8, SD 15.6 years) completed 2 testing sessions, which were completed on average 1 week apart (SD 4.5 days). Assessments included equivalent tests of emotion recognition (emotion recognition task [ERT]), visual recognition (pattern recognition memory [PRM]), episodic memory (paired associate learning [PAL]), working memory and spatial planning (spatial working memory [SWM] and one touch stockings of Cambridge), and sustained attention (rapid visual information processing [RVP]). Participants were randomly allocated to one of the two groups, either assessed in-person in the laboratory first (n=33) or with unsupervised web-based assessments on their personal computing systems first (n=18). Performance indices (errors, correct trials, and response sensitivity) and median reaction times were extracted. Intraclass and bivariate correlations examined intersetting reliability, linear mixed models and Bayesian paired sample t tests tested for equivalence, and Bland-Altman plots examined agreement.

Results

Intraclass correlation (ICC) coefficients ranged from ρ=0.23-0.67, with high correlations in 3 performance indices (from PAL, SWM, and RVP tasks; ρ≥0.60). High ICC values were also seen for reaction time measures from 2 tasks (PRM and ERT tasks; ρ≥0.60). However, reaction times were slower during web-based assessments, which undermined both equivalence and agreement for reaction time measures. Performance indices did not differ between assessment settings and generally showed satisfactory agreement.

Conclusions

Our findings support the comparability of CANTAB performance indices (errors, correct trials, and response sensitivity) in unsupervised, web-based assessments with in-person and laboratory tests. Reaction times are not as easily translatable from in-person to web-based testing, likely due to variations in computer hardware. The results underline the importance of examining more than one index to ascertain comparability, as high correlations can present in the context of systematic differences, which are a product of differences between measurement environments. Further work is now needed to examine web-based assessments in clinical populations and in larger samples to improve sensitivity for detecting subtler differences between test settings.

Body Mass and Physical Activity Uniquely Predict Change in Cognition for Aging Adults

BACKGROUND

Physical activity and body mass predict cognition in the elderly. However, mixed evidence suggests that obesity is associated with poorer cognition, while also protecting against cognitive decline in older age.

PURPOSE

We investigated whether body mass independently predicted cognition in older age and whether these associations changed over time.

METHODS

A latent curve structural equation modeling approach was used to analyze data from a sample of aging adults (N = 8442) split into two independent subsamples, collected over 6 years.

RESULTS

Lower baseline Body Mass Index (BMI) and higher physical activity independently predicted greater baseline cognition (p < 0.001). Decreases in BMI and physical activity independently predicted greater decline in the slope of cognition (p < 0.001).

CONCLUSIONS

Our results support the obesity paradox in cognitive aging, with lower baseline body mass predicting better cognition, but less decline over time protecting against cognitive decline. We discuss how weight loss in the elderly may serve as a useful indicator of co-occurring cognitive decline, and we discuss implications for health care professionals.

[Formula: see text]Determination of the smoking gun of intent: significance testing of forced choice results in social security claimants

OBJECTIVE

Significantly below-chance findings on forced choice tests have been described as revealing "the smoking gun of intent" that proved malingering. The issues of probability levels, one-tailed vs. two-tailed tests, and the combining of PVT scores on significantly below-chance findings were addressed in a previous study, with a recommendation of a probability level of .20 to test the significance of below-chance results. The purpose of the present study was to determine the rate of below-chance findings in a Social Security Disability claimant sample using the previous recommendations.

METHOD

We compared the frequency of below-chance results on forced choice performance validity tests (PVTs) at two levels of significance, .05 and .20, and when using significance testing on individual subtests of the PVTs compared with total scores in claimants for Social Security Disability in order to determine the rate of the expected increase.

RESULTS

The frequency of significant results increased with the higher level of significance for each subtest of the PVT and when combining individual test sections to increase the number of test items, with up to 20% of claimants showing significantly below-chance results at the higher p-value.

CONCLUSIONS

These findings are discussed in light of Social Security Administration policy, showing an impact on policy issues concerning child abuse and neglect, and the importance of using these techniques in evaluations for Social Security Disability.

Body mass and cognitive decline are indirectly associated via inflammation among aging adults

Inflammatory models of neurodegeneration suggest that higher circulating levels of inflammation can lead to cognitive decline. Despite established independent associations between greater body mass, increased inflammation, and cognitive decline, no prior research has explored whether markers of systemic inflammation might mediate the association between body mass and changes in cognitive functioning. To test such a model, we used two longitudinal subsamples (ns=9066; 12,561) of aging adults from the English Longitudinal Study of Ageing (ELSA) study, which included two cognitive measures components of memory and executive functioning, as well as measurements of body mass and systemic inflammation, assessed via C-reactive protein (CRP). Greater body mass was indirectly associated with declines in memory and executive functioning over 6years via relatively higher levels of CRP. Our results suggest that systemic inflammation is one biologically plausible mechanism through which differences in body mass might influence changes in cognitive functioning among aging adults.

Social participation predicts cognitive functioning in aging adults over time: comparisons with physical health, depression, and physical activity

OBJECTIVES

Several risk and protective factors are associated with changes in cognitive functioning in aging adults - including physical health, depression, physical activity, and social activities - though the findings for participation in social activities are mixed. This study investigated the longitudinal association between social participation and two domains of cognitive functioning, memory and executive function. A primary goal of our analyses was to determine whether social participation predicted cognitive functioning over-and-above physical health, depression, and physical activity in a sample with adequate power to detect unique effects.

METHOD

The sample included aging adults (N = 19,832) who participated in a large, multi-national study and provided data across six years; split into two random subsamples. Unique associations between the predictors of interest and cognitive functioning over time and within occasion were assessed in a latent curve growth model.

RESULTS

Social participation predicted both domains of cognitive functioning at each occasion, and the relative magnitude of this effect was comparable to physical health, depression, and physical activity level. In addition, social participation at the first time point predicted change in cognitive functioning over time. The substantive results in the initial sample were replicated in the second independent subsample.

CONCLUSION

Overall, the magnitude of the association of social participation is comparable to other well-established predictors of cognitive functioning, providing evidence that social participation plays an important role in cognitive functioning and successful aging.

The use of virtual reality in the military's assessment of service members with traumatic brain injury: recent developments and emerging opportunities

Traumatic brain injury (TBI) is a common event in the current extended conflicts by American service members, with estimates that as many as 300,000 have sustained combat-related concussions during Operation Iraqi Freedom and Operation Enduring Freedom. The limited ecological validity of traditional neuropsychological assessment measures presents a challenge to effective postconcussion evaluation of service members in relation to fitness-for-duty decisions or rehabilitation needs. Virtual reality (VR) technology offers a promising opportunity to advance the field of functional assessment for TBI. This article reviews the current professional literature on VR applications for TBI assessment, with special emphasis on those that are particularly relevant to U.S. service members. VR affords several advantages for clinical use. These include assessment of complex sets of cognitive and behavioral functions rather than the isolated components assessed by traditional measures; more precise control over the standardized presentation of task stimuli and the recording of response data; and enhanced ecological validity that can lead to more useful assessment data in the applied contexts faced by the U.S. military.