Tele-neuropsychological assessment tools in Italy: a systematic review on psychometric properties and usability

Methodological quality of 100 recent systematic reviews of health-related outcome measurement instruments: an overview of reviews

PURPOSE

Systematic reviews evaluating and comparing the measurement properties of outcome measurement instruments (OMIs) play an important role in OMI selection. Earlier overviews of review quality (2007, 2014) evidenced substantial concerns with regards to alignment to scientific standards. This overview aimed to investigate whether the quality of recent systematic reviews of OMIs lives up to the current scientific standards.

METHODS

One hundred systematic reviews of OMIs published from June 1, 2021 onwards were randomly selected through a systematic literature search performed on March 17, 2022 in MEDLINE and EMBASE. The quality of systematic reviews was appraised by two independent reviewers. An updated data extraction form was informed by the earlier studies, and results were compared to these earlier studies' findings.

RESULTS

A quarter of the reviews had an unclear research question or aim, and in 22% of the reviews the search strategy did not match the aim. Half of the reviews had an incomprehensive search strategy, because relevant search terms were not included. In 63% of the reviews (compared to 41% in 2014 and 30% in 2007) a risk of bias assessment was conducted. In 73% of the reviews (some) measurement properties were evaluated (58% in 2014 and 55% in 2007). In 60% of the reviews the data were (partly) synthesized (42% in 2014 and 7% in 2007); evaluation of measurement properties and data syntheses was not conducted separately for subscales in the majority. Certainty assessments of the quality of the total body of evidence were conducted in only 33% of reviews (not assessed in 2014 and 2007). The majority (58%) did not make any recommendations on which OMI (not) to use.

CONCLUSION

Despite clear improvements in risk of bias assessments, measurement property evaluation and data synthesis, specifying the research question, conducting the search strategy and performing a certainty assessment remain poor. To ensure that systematic reviews of OMIs meet current scientific standards, more consistent conduct and reporting of systematic reviews of OMIs is needed.

A digital teleneuropsychology platform for the diagnosis of mild cognitive impairment: from concept to certification as a medical device

BACKGROUND

Innovative digital solutions are shaping a new concept of dementia care, opening additional venues for prevention, diagnosis, monitoring and treatment. Hereby, we report the development of a tablet-based teleneuropsychology platform (Tenèpsia®), from concept to certification as Medical Device (MD) Class IIA, as per new MD regulation 745/2017.

METHODS

The platform was designed for the remote cognitive evaluation and created thanks to the effort of a collaborative working group including experts from three Italian scientific societies and Biogen Italia S.r.l. (hereafter "Biogen"), and developers from Xenia Reply and Inside AI. The development strategy was guided by converting traditional paper-and-pencil tests into digital versions while maintaining comparable neuropsychological features and optimizing patient accessibility and user experience. The experts focused on the choice and adaptation of traditional neuropsychology measures for a 45-min teleneuropsychology assessment.

RESULTS

The developers created a web and a mobile interface, respectively, for the professional (neuropsychologist) and non-professional (patient and caregiver) use. Recording of voice, drawing and typing information was enabled. Instant dashboards provide a quick overview of the patient's condition. Simulation activities were performed to obtain MD certification, valid across Europe.

CONCLUSION

Neuropsychology services will benefit from the implementation in clinics of harmonized digital tools with adequate scientific and technological standards. The use of digital cognitive testing for the diagnosis of mild cognitive impairment is expected to enhance patient and clinician outcomes through simplified, digital objective data collection, sparing of time and resources, with a positive impact on healthcare costs and access to treatments, reducing inequalities and delays in diagnosis and cure.

The Telephone Language Screener (TLS): standardization of a novel telephone-based screening test for language impairment

BACKGROUND

This study aimed at developing and standardizing the Telephone Language Screener (TLS), a novel, disease-nonspecific, telephone-based screening test for language disorders.

METHODS

The TLS was developed in strict pursuance to the current psycholinguistic standards. It comprises nine tasks assessing phonological, lexical-semantic and morpho-syntactic components, as well as an extra Backward Digit Span task. The TLS was administered to 480 healthy participants (HPs), along with the Telephone-based Semantic Verbal Fluency (t-SVF) test and a Telephone-based Composite Language Index (TBCLI), as well as to 37 cerebrovascular/neurodegenerative patients-who also underwent the language subscale of the Telephone Interview for Cognitive Status (TICS-L). An HP subsample was also administered an in-person language battery. Construct validity, factorial structure, internal consistency, test-retest and inter-rater reliability were tested. Norms were derived via Equivalent Scores. The capability of the TLS to discriminate patients from HPs and to identify, among the patient cohort, those with a defective TICS-L, was also examined.

RESULTS

The TLS was underpinned by a mono-component structure and converged with the t-SVF (p < .001), the TBCLI (p < .001) and the in-person language battery (p = .002). It was internally consistent (McDonald's ω = 0.67) and reliable between raters (ICC = 0.99) and at retest (ICC = 0.83). Age and education, but not sex, were predictors of TLS scores. The TLS optimally discriminated patients from HPs (AUC = 0.80) and successfully identified patients with an impaired TICS-L (AUC = 0.92). In patients, the TLS converged with TICS-L scores (p = 0.016).

DISCUSSION

The TLS is a valid, reliable, normed and clinically feasible telephone-based screener for language impairment.

Psychometrics and diagnostics of the Italian version of the Alternate Verbal Fluency Battery (AVFB) in non-demented Parkinson's disease patients

BACKGROUND

Verbal fluency (VF) tasks are known as suitable for detecting cognitive impairment (CI) in Parkinson's disease (PD). This study thus aimed to evaluate the psychometrics and diagnostics of the Alternate Verbal Fluency Battery (AVFB) by Costa et al. (2014) in an Italian cohort of non-demented PD patients, as well as to derive disease-specific cut-offs for it.

METHODS

N = 192 non-demented PD patients were screened with the Montreal Cognitive Assessment (MoCA) and underwent the AVFB-which includes phonemic, semantic and alternate VF tests (PVF; SVF; AVF), as well as a Composite Shifting Index (CSI) reflecting the "cost" of shifting from a single- to a double-cued VF task. Construct validity and diagnostics were assessed for each AVFB measure against the MoCA. Internal reliability and factorial validity were also tested.

RESULTS

The MoCA proved to be strongly associated with PVF, SVF and AVF scores, whilst moderately with the CSI. The AVFB was internally consistent and underpinned by a single component; however, an improvement in both internal reliability and fit to its factorial structure was observed when dropping the CSI. Demographically adjusted scores on PVF, SVF and AVF tests were diagnostically sound in detecting MoCA-defined cognitive impairment, whilst this was not true for the CSI. Disease-specific cut-offs for PVF, SVF and AVF tests were derived.

DISCUSSION

In conclusion, PVF, SVF and AVF tests are reliable, valid and diagnostically sound instruments to detect cognitive impairment in non-demented PD patients and are therefore recommended for use in clinical practice and research.

Neuropsychological Profile of Hospitalized Patients Due to COVID-19: Clinical and Inflammatory Correlates

OBJECTIVES

In the present study, we investigated the pattern of cognitive difficulties in hospitalized patients due to COVID-19 and its relation with the clinical features of the disease.

METHOD

Forty hospitalized patients with COVID-19 [mean age: 46.98 years (SD = 9.30); mean years of education: 13.65 (SD = 2.07) and 40 sex-, age- and education-matched healthy controls completed a set of neuropsychological measures administered by telephone. Participants' premorbid intellectual skills and patients' anxiety and depressive symptoms were also evaluated. The association of COVID-19-related biomarkers [oxygen saturation (SpO2), C-reactive protein (CRP), D-dimer and ferritin levels] with neuropsychological performances was examined with a series of hierarchical multiple linear regression analyses, after controlling for demographic and clinical characteristics, psychological distress and premorbid intellectual skills.

RESULTS

Patients performed worse than healthy participants on measures of verbal memory, attention and working memory. SpO2 levels were associated with patients' performance on verbal and working memory, whereas CRP levels were associated with performance on verbal memory, abstract reasoning and verbal fluency, after controlling for demographic and clinical characteristics. Ferritin levels predicted performance on the verbal fluency test, whereas D-dimer levels did not predict any of the neuropsychological measures.

CONCLUSIONS

Cognitive difficulties in verbal memory, attention and working memory were noted in patients with COVID-19. Markers of hyperinflammation predicted patients' performance above and beyond demographic characteristics, duration of symptoms, length of hospitalization and psychological distress.

Tele-Global Examination of Mental State (Tele-GEMS): an open tool for the remote neuropsychological screening

Tele-neuropsychology, i.e., the application of remote audio-visual technologies to neuropsychological evaluation or rehabilitation, has become increasingly popular and widespread during and after the COVID-19 pandemic. New tools with updated normative data and appropriate methodological developments are necessary. We present Tele-GEMS, a telephone-based cognitive screening developed on N = 601 Italian participants. It yields a global score tapping on orientation, memory, spatial representation, language, and pragmatic abilities. Its administration lasts about 10 min. Clinical cut-offs are provided, accounting for demographic variables (age, education, and sex) and also for a comprehensive index taking into account cognitively stimulating life experiences that can build up a cognitive reserve. Tele-GEMS shows good internal consistency and a good inter-rater agreement. The test includes the thresholds for estimating a significant change after repeated measurements. Tele-GEMS has a good construct validity as assessed with MoCA and a suitable criterion validity assessed with its in-person version (GEMS). All the materials and the instructions, including scripts and an online Application for the automatic calculation of cut-offs, are accessible on OSF at https://osf.io/t3bma/ under a Creative Commons license.

Tele-Assessment of Functional Capacity: Validity, Intra- and Inter-rater Reliability

BACKGROUND

Functional capacity evaluation is a standardized tool that assesses work-related skills. Although there are different test batteries, the most frequently used one is Work Well Systems. This study aims to determine the validity and inter- and intra-rater reliability of remote implementation of functional capacity tests (repetitive reaching, lifting object overhead, and working overhead) in asymptomatic individuals.

METHODS

A total of 51 asymptomatic individuals were included in the study. Participants completed all tests both face-to-face and remotely. Remote assessment videos were rewatched by the same researcher and different researchers for intra- and inter-rater reliability. All processes were scored by two independent researchers.

RESULTS

Remotely performing repetitive reaching (intraclass correlation coefficient [ICC]: 0.85-0.92, p < .001), lifting object overhead (ICC: 0.98, p < .001), and working overhead (ICC: 0.88 p < .001) tests are valid and reliable.

DISCUSSION

Repetitive reaching, lifting an object overhead, and sustained overhead work tests in the Work Well Systems-Functional Capacity Evaluation test battery can be performed remotely through videoconferencing. Remotely evaluating these tests, which are especially important in work-related situations, may be important in pandemic conditions and hybrid working conitions.

Neuropsychological Assessment in Patients with Traumatic Brain Injury: A Comprehensive Review with Clinical Recommendations

Traumatic brain injury is damage to the brain occurring after birth, often resulting in the deterioration of cognitive, behavioural, and emotional functions. Neuropsychological evaluation can assist clinicians to better assess the patient's clinical condition, reach differential diagnoses, and develop interventional strategies. However, considering the multiple rating scales available, it is not easy to establish which tool is most suitable for the different brain injury conditions. The aim of this review is to investigate and describe the most used neurocognitive assessment tools in patients with traumatic brain injury to provide clinicians with clear indications on their use in clinical practice. Indeed, during the acute phase, after the head trauma, alertness and wakefulness of the patients affected by a disorder of consciousness can be assessed using different scales, such as the Coma Recovery Scale-Revised. In both postacute and chronic phases after traumatic brain injury, general cognitive assessment tools (such as the Mini Mental State Examination) or more specific cognitive tests (e.g., Wisconsin Card Sorting Test and Trail Making Test) could be administered according to the patient's functional status. In this way, clinicians may be aware of the patient's neuropsychological and cognitive level, so they can guarantee a personalized and tailored rehabilitation approach in this frail patient population.

Teleneuropsychology for vascular cognitive impairment: Which tools do we have?

The halt of clinical activities imposed during the COVID-19 pandemic forced clinicians to find alternative strategies to provide continuity of care and services, and led to a renewed interest in use of teleneuropsychology (TNP) to remotely assess patients. Recent TNP guidelines recommend maximizing the reproduction of standard in-person assessment, particularly through videoconferences. However, consistency of the adaptations of usual cognitive tests to videoconference needs further elucidation. This review aims at critical reviewing which cognitive tests could be recommended for a remote evaluation of patients with vascular cognitive impairment (VCI) among those widely recognized as reference standards. Current evidence supports the use of global cognitive efficiency (MMSE and MoCA), verbal memory (Revised Hopkins Verbal Learning Test), and language tests (phonemic and semantic verbal fluencies, Boston Naming Test), while there is a lack of strong validity support for measures of visuospatial functions (Rey-Osterreith Complex Figure), and executive functioning and processing speed (Trail making Test, and Digit symbol or Symbol digit tests). This represents a major limitation in the evaluation of VCI because its cognitive profile in often characterized by attention and executive deficits. At present, a videoconference TNP visit appears useful for a brief evaluation of global cognitive efficiency, and to 'triage' patients towards a second level in person evaluation. In future, hybrid models of TNP based on data collected across multiple modalities, incorporating both adaptation of usual cognitive tools and new computerized tools in the supervised videoconference setting, are likely to become the best option for a comprehensive remote cognitive assessment.

Validity and reliability of a Chinese version of the self-evaluation of negative symptoms

The negative symptoms of schizophrenia can be present at any clinical stage, but evaluating the negative symptoms always remains challenging. To screen the negative symptoms effectively, self-evaluation should be introduced. To date, professional psychiatrists used almost all of the scales available to screen the negative symptoms but could not obtain an accurate outcome. At the same time, an advanced self-assessment scale is needed to accompany the patients' self-feeling-based treatment strategies to understand their feelings about their symptoms. Hence, Chinese self-evaluation of negative symptoms (SNS) should be introduced in China. This study aims to examine the validity and reliability of the Chinese version of SNS. Two hundred patients with schizophrenia were included in this study and were evaluated entirely with the self-assessed negative symptoms by the Chinese version. The correlation analysis was performed between SNS and the Scale for Assessment of Negative Symptoms (SANS) to assess the criterion validity of SNS for screening negative symptoms. Exploratory factor analysis was used to determine the constructive validity of the SNS. Two senior professional psychiatrists were involved in this assessment based on their clinical experience and capability to define the severity of the negative symptoms. Receiver operating characteristic curve (ROC) analysis was performed to assess the cutoff point of SNS. Cronbach's alpha coefficient and intraclass correlation (ICC) coefficient were used to determine the reliability of SNS. We have the following findings: The Chinese version of SNS demonstrated a significant correlation with the SANS (r = .774, p < .05). Exploratory factor analysis demonstrated that the factor loading varies from .442 to .788. ROC analysis demonstrated that at SNS ≥ 8, the patients demonstrated a mild severity of negative symptoms, and at SNS ≥ 15, the patients demonstrated a severe severity of negative symptoms. Subsequently, 9 < SNS < 14 was defined as a moderate severity of negative symptoms. The Cronbach's alpha and ICC coefficients of the Chinese version SNS were .877 and .774, respectively. Our results showed that the acceptable validity and reliability of the Chinese version of SNS confirmed that SNS is an ideal tool for self-assessment of the negative symptoms in patients with schizophrenia.

Teleneuropsychology: normative data for the assessment of memory in online settings

INTRODUCTION

The COVID-19 pandemic has forced significant changes in clinical practice. Psychologists and neuropsychologists had to modify their settings to assess patients' abilities, switching from an in-person modality to a remote setting by using video calling platforms. Consequently, this change brought about the need for new normative data tailored to remote settings.

AIM AND METHODS

The study aimed to develop normative data for the online assessment of neuropsychological memory tests and to compare it with the published norms obtained in standard settings. Two hundred and four healthy Italian volunteers performed three verbal memory tests through the Google Meet platform: the Digit Span (Backward and Forward), the Rey Auditory Verbal Learning, and the Verbal Paired Associated Learning Test.

RESULTS

This research provides specific norms that consider the influence of demographic characteristics. Their comparison with published norms shows a medium to high agreement between systems. The present study provides a reference for the clinical use of neuropsychological instruments to assess verbal memory in a remote setting and offers specific recommendations.

The Current Status of Telemedicine Technology Use Across the World Health Organization European Region: An Overview of Systematic Reviews

Background

Several systematic reviews evaluating the use of telemedicine by clinicians, patients, and health authorities to improve the delivery of care in the 53 member states of the World Health Organization (WHO) European Region have been conducted in recent years. However, a study summarizing the findings of these reviews has not been conducted.

Objective

This overview of systematic reviews aimed to summarize findings regarding the use of telemedicine across the 53 member states and identify the medical fields and levels of care in and at which the effectiveness, feasibility, and applicability of telemedicine have been demonstrated. The barriers to and facilitators of telemedicine use were also evaluated and collated to help with the design and implementation of telemedicine interventions.

Methods

Through a comprehensive systematic evaluation of the published and unpublished literature, we extracted clinical, epidemiological, and technology-related data from each review included in the study. We focused on evaluating the barriers to and facilitators of the use of telemedicine apps across the 53 member states considered. We rated the methodological quality of each of the included reviews based on A Measurement Tool to Assess Systematic Review 2 approach and judged the overall certainty of evidence by using the Grading of Recommendations, Assessment, Development, and Evaluations methodology. The entire process was performed by 2 independent authors.

Results

This overview drew on data from >2239 primary studies, with >20,000 enrolled patients in total, within the WHO European Region. On the basis of data from randomized trials, observational studies, and economic evaluations from several countries, the results show a clear benefit of telemedicine technologies in the screening, diagnosis, management, treatment, and long-term follow-up of a series of chronic diseases. However, we were unable to pool the results into a reliable numeric parameter because of the high heterogeneity of intervention methodologies, scheduling, primary study design discrepancies, settings, and geographical locations. In addition to the clinical outcomes of the interventions, the social and economic outcomes are highlighted.

Conclusions

The application of telemedicine is well established across countries in the WHO European Region; however, some countries could still benefit from the many uses of these digital solutions. Barriers related to users, technology, and infrastructure were the largest. Conversely, the provision of health services using technological devices was found to significantly enhance patients’ clinical outcomes, improve the long-term follow-up of patients by medical professionals, and offer logistical benefits for both patients and health workers.

Trial Registration

PROSPERO (International Prospective Register of Systematic Reviews) CRD42022309375; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=309375

The Italian telephone-based Verbal Fluency Battery (t-VFB): standardization and preliminary clinical usability evidence

Background

This study aimed at standardizing and providing preliminary evidence on the clinical usability of the Italian telephone-based Verbal Fluency Battery (t-VFB), which includes phonemic (t-PVF), semantic (t-SVF) and alternate (t-AVF) verbal fluency tasks.

Methods

Three-hundred and thirty-five Italian healthy participants (HPs; 140 males; age range = 18-96 years; education range = 4-23 years) and 27 individuals with neurodegenerative or cerebrovascular diseases were administered the t-VFB. Switch number and cluster size were computed via latent semantic analyses. HPs underwent the telephone-based Mental State Examination (MMSE) and Backward Digit Span (BDS). Construct validity, factorial structure, internal consistency, test-retest and inter-rater reliability and equivalence with the in-person Verbal Fluency tasks were assessed. Norms were derived via Equivalent Scores. Diagnostic accuracy against clinical populations was assessed.

Results

The majority of t-VFB scores correlated among each other and with the BDS, but not with the MMSE. Switch number correlated with t-PVF, t-SVF, t-AVF scores, whilst cluster size with the t-SVF and t-AVF scores only. The t-VFB was underpinned by a mono-component structure and was internally consistent (Cronbach's α = 0.91). Test-retest (ICC = 0.69-0.95) and inter-rater reliability (ICC = 0.98-1) were optimal. Each t-VFB test was statistically equivalent to its in-person version (equivalence bounds yielding a p < 0.05). Education predicted all t-VFB scores, whereas age t-SVF and t-AVF scores and sex only some t-SVF scores. Diagnostic accuracy against clinical samples was optimal (AUC = 0.81-0.86).

Discussion

The t-VFB is a valid, reliable and normed telephone-based assessment tool for language and executive functioning, equivalent to the in-person version; results show promising evidence of its diagnostic accuracy in neurological populations.

Telephone-based Frontal Assessment Battery (t-FAB): standardization for the Italian population and clinical usability in neurological diseases

Background

Despite the relevance of telephone-based cognitive screening tests in clinical practice and research, no specific test assessing executive functioning is available. The present study aimed at standardizing and providing evidence of clinical usability for the Italian telephone-based Frontal Assessment Battery (t-FAB).

Methods

The t-FAB (ranging 0–12), comprising two subtests, has two versions: one requiring motor responses (t-FAB-M) and the other verbal responses (t-FAB-V). Three hundred and forty-six Italian healthy adults (HPs; 143 males; age range = 18–96 years; education range = 4–23 years) and 40 participants with neurological diseases were recruited. To HPs, the t-FAB was administered along with a set of telephone-based tests: MMSE, verbal fluency (VF), backward digit span (BDS). The in-person version of the FAB was administered to both HPs and clinical groups. Factorial structure, construct validity, inter-rater and test–retest reliability, t-FAB-M vs. t-FAB-V equivalence and diagnostic accuracy were assessed. Norms were derived via Equivalent Scores.

Results

In HPs, t-FAB measures yielded high inter-rater/test–retest reliability (ICC = .78–.94), were internally related (p ≤ .005) and underpinned by a single component, converging with the telephone-based MMSE, VF, BDS (p ≤ .0013). The two t-FAB versions were statistically equivalent in clinical groups (ps of both equivalence bounds < .001). Education predicted all t-FAB scores (p < .001), whereas age only the t-FAB-M score (p ≤ .004). t-FAB scores converge with the in-person FAB in HPs and clinical groups (r_s = .43–.78). Both t-FAB versions were accurate in discriminating HPs from the clinical cohort (AUC = .73-.76).

Discussion

The t-FAB is a normed, valid, reliable and clinically usable telephone-based cognitive screening test to adopt in both clinical and research practice.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40520-022-02155-3.

Italian telephone-based Mini-Mental State Examination (Itel-MMSE): item-level psychometric properties

Background

The Italian telephone-based Mini-Mental State Examination (Itel-MMSE), despite being psychometrically sound, has shown relevant ceiling effects, which may negatively impact the interpretation of its scores. In address to overcome such an issue, this study aimed at providing item-level insights on the Itel-MMSE through Item Response Theory (IRT) analyses.

Methods

Five-hundred and sixty-seven healthy Italian adults (227 males, 340 females; mean age: 51 ± 17 years, range 18–96; mean education: 13.31 ± 4.3 years). A two-parameter logistic IRT model was implemented to assess item discrimination and difficulty of the Itel-MMSE. Construct unidimensionality, statistical independence of items, and model and item fit were tested. Informativity levels were also assessed graphically.

Results

With respect to the Itel-MMSE total score, ceiling effects were found in 92.7% of participants. Unidimensionality was violated; both model and item fit were poor; a few items showed statistical dependence. Both the whole test and its items proved to be scarcely informative, especially for medium-to-high levels of ability, except for attention and spatial orientation subtests, which consistently yielded the highest discriminative capability.

Discussion

The Itel-MMSE appears to be most informative in low-performing healthy individuals. However, the present findings should not lead practitioners to aprioristically equate ceiling effects/low informativity to clinical uselessness. Items assessing attention and, to a lesser extent, spatial orientation appear to be the most informative.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40520-021-02041-4.

Metal substitutions incarbonic anhydrase: a halide ion probe study

Monitoring of the mental health status of the population and assessment of its determinants are 2 of the most relevant pillars of public mental health, and data from population health surveys could be instrumental to support them. Although these surveys could be an important and suitable resource for these purposes, due to different limitations and challenges, they are often relegated to the background behind other data sources, such as electronic health records. These limitations and challenges include those related to measurement properties and cross-cultural validity of the tools used for the assessment of mental disorders, their degree of representativeness, and possible difficulties in the linkage with other data sources. Successfully addressing these limitations could significantly increase the potential of health surveys in the monitoring of mental disorders and ultimately maximize the impact of the relevant policies to reduce their burden at the population level. The widespread use of data from population health surveys, ideally linked to electronic health records data, would enhance the quality of the information available for research, public mental health decision-making, and ultimately addressing the growing burden of mental disorders.