Equivalency of In-Person Versus Remote Assessment: WISC-V and KTEA-3 Performance in Clinically Referred Children and Adolescents

CLINICAL AND COGNITIVE PHENOTYPING OF COPY NUMBER VARIANTS PATHOGENIC FOR NEURODEVELOPMENTAL DISORDERS FROM A MULTI-ANCESTRY BIOBANK

Background

Rare copy number variants (CNVs) are pathogenic for neurodevelopmental disorders (NDDs) and effect neurocognitive impairment. In aggregate, NDD CNVs may present in up to 2% of population cohorts with implications for neuropsychiatric disease risk and cognitive health. However, analyses of NDD CNVs in biobanks or population cohorts have been hindered by limited clinical or cognitive phenotypes, and a lack of ancestral diversity. In the current proof-of-concept study, NDD CNV carriers were recontacted from BioMe, a multi-ancestry biobank derived from the Mount Sinai healthcare system, to enable 'deep phenotyping' beyond electronic health record outcomes.

Methods

From BioMe biobank, 892 adult participants were recontacted, including 335 harboring NDD CNVs, 217 with schizophrenia and 340 neurotypical controls as comparators. Clinical and cognitive assessments were administered to each recruited participant.

Results

Seventy-three participants completed study assessments (mean age=48.8 years; 66% female; 36% African, 26% European, 34% Hispanic), or 8% of the recontacted subset, including 30 NDD CNV carriers across 15 loci. Among NDD CNV carriers, assessments indicated 40% with mood and anxiety disorders, 30% with learning disorders, and 13% with a history of special education. NDD CNV carriers were significantly cognitively impaired compared to controls on digit span backwards (Beta=-1.76, FDR=0.04) and digit span sequencing (Beta=-2.01, FDR=0.04).

Conclusions

Feasibility of "recall-by-genotype" from a multi-ancestry biobank was established for NDD CNV carriers, along with comparator groups. The current study corroborated past reports of NDD CNVs effects of cognitive impairment, while elucidating clinical phenotypes for recalled individuals. Future "recall-by-genotype" studies may further facilitate clinical characterization of disease-relevant genomic variants.

Validity of remote live stream video evaluation of the North Star Ambulatory Assessment in patients with Duchenne muscular dystrophy

Conducting functional assessments remotely can help alleviate the burden of in-person assessment on patients with Duchenne muscular dystrophy and their caregivers. The objective of this study was to evaluate whether scores from remote functional assessment of patients with Duchenne muscular dystrophy correspond to in-person scores on the same functional assessments. Remote live stream versus in-person scores on the North Star Ambulatory Assessment (including time [seconds] to complete the 10-meter walk/run and time to rise from the floor [supine to stand]) were assessed using statistical analyses, including intraclass correlation coefficient, and Pearson, Spearman, and Bland-Altman analyses. The remote and in-clinic assessments had to occur within 2 weeks of one another to be considered for this analysis. This analysis included patients with Duchenne muscular dystrophy, aged 4 to 7 years. Participants in this analysis received delandistrogene moxeparvovec (as part of SRP-9001-101 [Study 101; NCT03375164] or SRP-9001-102 [Study 102; NCT03769116]) or were randomized to receive placebo (in Part 1 of Study 102). This study evaluates score reproducibility between live stream remote scoring versus in-person functional assessments as determined by intraclass correlation coefficient, and Pearson, Spearman, and Bland-Altman analyses. The results showed that scores from remote functional assessment of patients with Duchenne muscular dystrophy strongly correlated with those obtained in person. These findings demonstrate congruence between live stream remote and in-person functional assessment and suggest that remote assessment has the potential to reduce the burden on a family by supplementing in-clinic visits.

Diagnostic certainty during in-person and telehealth autism evaluations

Background

Many diagnostic evaluations abruptly shifted to telehealth during the COVID-19 pandemic; however, little is known about the impact on diagnosis patterns for children evaluated for autism spectrum disorder (ASD). The purpose of this clinical research study was to examine (1) the frequency of diagnoses evaluated beyond ASD; (2) the frequency of diagnoses made, including ASD; and (3) clinician diagnostic certainty for all diagnoses evaluated for children who received an evaluation due to primary concerns about ASD via telehealth during the pandemic compared to those evaluated in person before the pandemic at an ASD specialty clinic.

Methods

The sample included 2192 children, 1-17 years (M = 6.5 years; SD = 3.9), evaluated by a physician/psychologist at an ASD specialty center. A total of 649 children were evaluated in-person September 1, 2019-March 13, 2020 (pre-pandemic) and 1543 were evaluated via telehealth March 14, 2020-July 26, 2021 (during pandemic). Upon completion of each evaluation, clinicians provided a final diagnostic determination (i.e., "Yes," "No," "Possible," or "Not Assessed") for the following DSM-5 conditions: ASD, attention-deficit/hyperactivity disorder (ADHD), intellectual developmental disorder (IDD), anxiety (ANX), depression (DEP), and behavioral disorder (BD). "Possible" indicated lower certainty and the diagnosis was not provided. "Not Assessed" indicated the disorder was not evaluated.

Results

Diagnostic certainty for ASD and ADHD was lower and clinicians evaluated for and made diagnoses of IDD less often during evaluations that occurred via telehealth during the pandemic versus in person before the pandemic. DEP and BD were diagnosed more frequently, diagnostic certainty of DEP was lower, and no differences in the frequency of ANX diagnoses emerged during evaluations conducted via telehealth during the pandemic compared to those conducted in person before the pandemic.

Conclusions

Differences emerged in the frequency of diagnoses evaluated and made and diagnostic certainty for evaluations conducted via telehealth during the pandemic compared to in person before the pandemic, which likely impacted patients and reflect real-word challenges. Future work should examine whether these patterns are generalizable and the mechanisms that contribute to these differences.

Tele-Neuropsychology: From Science to Policy to Practice

OBJECTIVE

The primary aim of this paper is to accelerate the number of randomized experimental studies of the reliability and validity in-home tele-neuropsychological testing (tele-np-t).

METHOD

We conducted a critical review of the tele-neuropsychology literature. We discuss this research in the context of the United States' public and private healthcare payer systems, including the Centers for Medicare & Medicaid Services (CMS) and Current Procedural Terminology (CPT) coding system's telehealth lists, and existing disparities in healthcare access.

RESULTS

The number of tele-np publications has been stagnant since the onset of the COVID-19 pandemic. There are less published experimental studies of tele-neuropsychology (tele-np), and particularly in-home tele-np-t, than other tele-np publications. There is strong foundational evidence of the acceptability, feasibility, and reliability of tele-np-t, but relatively few studies of the reliability and validity of in-home tele-np-t using randomization methodology.

CONCLUSIONS

More studies of the reliability and validity of in-home tele-np-t using randomization methodology are necessary to support inclusion of tele-np-t codes on the CMS and CPT telehealth lists, and subsequently, the integration and delivery of in-home tele-np-t services across providers and institutions. These actions are needed to maintain equitable reimbursement of in-home tele-np-t services and address the widespread disparities in healthcare access.

The Use of Telepractice to Administer Norm-Referenced Communication and Cognition Assessments in Children With Hearing Loss: A Rapid Review

PURPOSE

Telepractice administration of norm-referenced assessments of communication and cognition is relatively new, and evidence to support this practice for children with hearing loss is limited. This rapid review examines the validity, reliability, feasibility, and common features of telepractice-administered norm-referenced assessments of communication and cognition for children with hearing loss to determine whether results via telepractice-administration are usable.

METHOD

This rapid review was conducted in accordance with the Cochrane Collaboration Rapid Reviews Methods Group recommendations. Rayyan software was used for initial and full-text screening, and the Quality Assessment of Diagnostic Accuracy Studies was used to measure study quality.

RESULTS

Electronic databases searches identified two studies that met the eligibility criteria. The findings of this rapid review provide some evidence to indicate that results of norm-referenced assessments do not differ depending on test administration conditions or hearing group status. Although both studies were of sufficient quality, replicability of these studies is uncertain due to the limited description of telepractice administration procedures. Participants indicated that sound levels provided by inbuilt computer speakers were adequate for them to be able to participate in telepractice assessment.

CONCLUSION

Telepractice administration of the norm-referenced assessments in the included studies may be valid, reliable, and feasible; however, the generalizability of these findings to other norm-referenced assessments is uncertain due to the limited amount of research.

Comparing remote versus in-person assessment of learning skills in children with specific learning disabilities

Background

Interactive telemedicine applications have been progressively introduced in the assessment of cognitive and literacy skills. However, there is still a lack of research focusing on the validity of this methodology for the neuropsychological assessment of children with Specific Learning Disorder (SLD).

Methods

Seventy-nine children including 40 typically developing children (18 males, age 11.5 ± 1.06) and 39 children with SLD (24 males, age 12.3 ± 1.28) were recruited. Each participant underwent the same neuropsychological battery assessing reading accuracy, speed, and comprehension, writing, numerical processing, computation, and semantic numerical sense, twice (once during an in-person session (I) and once during a remote (R) home-based videoconference session). Four groups were subsequently defined based on the administration order. Repeated-measure-ANOVAs with assessment type (R vs. I testing) as within-subject factor and diagnosis (SLD vs. TR) and administration order (R-I vs. I-R) as between-subject factors, and between-group t-tests comparing the two assessment types within each time of administration, were run.

Results

No differences emerged between I and R assessments of reading accuracy and speed, numerical processing, and computation; on the contrary, potential biases against R assessment emerged when evaluating skills in writing, reading comprehension, and semantic numerical sense. However, regardless of the assessment type, the scores obtained with I and R assessments within the same administration time point overlapped.

Discussion

These results partially support the validity and reliability of the assessment of children's learning skills via a remote home-based videoconferencing system. Implementing telemedicine as an assessment tool may increase timely access to primary health care and to support research activity.

Computerized Cognitive Test Batteries for Children and Adolescents-A Scoping Review of Tools For Lab- and Web-Based Settings From 2000 to 2021

OBJECTIVE

Cognitive functioning is essential to well-being. Since cognitive difficulties are common in many disorders, their early identification is critical, notably during childhood and adolescence. This scoping review aims to provide a comprehensive literature overview of computerized cognitive test batteries (CCTB) that have been developed and used in children and adolescents over the past 22 years and to evaluate their psychometric properties.

METHOD

Among 3192 records identified from three databases (PubMed, PsycNET, and Web of Science) between 2000 and 2021, 564 peer-reviewed articles conducted in children and adolescents aged 3 to 18 years met inclusion criteria. Twenty main CCTBs were identified and further reviewed following PRISMA guidelines. Relevant study details (sample information, topic, location, setting, norms, and psychometrics) were extracted, as well as administration and instrument characteristics for the main CCTBs.

RESULTS

Findings suggest that CCTB use varies according to age, location, and topic, with eight tools accounting for 85% of studies, and the Cambridge Neuropsychological Test Automated Battery (CANTAB) being most frequently used. Few instruments were applied in web-based settings or include social cognition tasks. Only 13% of studies reported psychometric properties.

CONCLUSIONS

Over the past two decades, a high number of computerized cognitive batteries have been developed. Among these, more validation studies are needed, particularly across diverse cultural contexts. This review offers a comprehensive synthesis of CCTBs to aid both researchers and clinicians to conduct cognitive assessments in children in either a lab- or web-based setting.

Social cognition and behavioral outcomes in congenital heart disease: profiles and neuropsychiatric comorbidities

Autism spectrum disorders are more prevalent in children with congenital heart disease (CHD) than in the general population. Children with CHD without diagnosed autism are also at increased risk for neurodevelopmental and psychiatric impairments. We characterized social and behavioral outcomes in children with CHD and examined neurodevelopmental and psychiatric comorbidities. Children without diagnosed autism who underwent infant open-heart surgery were eligible. Parent-reports assessed social communication, unusual behaviors, self-regulation, anxiety, and executive function (EF). Neuropsychological tests assessing theory of mind (ToM), working memory, and verbal comprehension were administered. Outcomes were compared to normative data. Linear regressions were estimated with parent-reported scores and ToM abilities as outcomes. Predictors were anxiety symptoms, parent-reported EF, and working memory scores. Covariates were age, parental education, ADHD diagnosis, and verbal comprehension. Clinically relevant comorbidities were identified (N children scoring ≥1SD below the norm). Fifty-six children (10.8 ± 1.8 years) participated virtually. Compared to norms, children with CHD had impaired ToM, more unusual behaviors (p = .002), and less self-regulation (p = .018), but better social communication (p = .014). "Autism-like" traits were positively associated with anxiety symptoms (ß(95% CI) = 0.28(0.08-0.49), p = .008) and worse working memory (ß(95% CI) = -0.36(-0.59-0.13), p = .003). Twenty-one out of 22 children who displayed clinically relevant social and behavioral scores also showed anxiety symptoms (n = 4), impaired EF (n = 7), or both (n = 10). Children with CHD without diagnosed autism have elevated unusual behaviors, lower self-regulation, and impaired ToM. There is a high risk of co-existing anxiety and impaired EF which may increase disease burden. Targeted therapeutic interventions are needed to reduce long-term psychosocial risks in these children.AbbreviationAttention deficit/hyperactivity disorder (ADHD), Autism Spectrum Rating Scale (ASRS), Behavior Rating Inventory of Executive Functions for school-aged children, 2nd Edition (BRIEF-2), cardiopulmonary bypass (CPB), congenital heart disease (CHD), Empathy/Systematizing Quotient Child Version (ESQ-C), Multidimensional Anxiety Scale for Children, 2nd Edition (MASC-2), Social Responsiveness Scale (School-age form), 2nd Edition (SRS-2), theory of mind (ToM), Theory of Mind Task Battery (ToM-TB), Wechsler Intelligence Scale for Children, 5th edition (WISC-V).

Remote Assessment: Origins, Benefits, and Concerns

Although guidelines surrounding COVID-19 have relaxed and school-aged students are no longer required to wear masks and social distance in schools, we have become, as a nation and as a society, more comfortable working from home, learning online, and using technology as a platform to communicate ubiquitously across ecological environments. In the school psychology community, we have also become more familiar with assessing students virtually, but at what cost? While there is research suggesting score equivalency between virtual and in-person assessment, score equivalency alone is not sufficient to validate a measure or an adaptation thereof. Furthermore, the majority of psychological measures on the market are normed for in-person administration. In this paper, we will not only review the pitfalls of reliability and validity but will also unpack the ethics of remote assessment as an equitable practice.

Best Practices and Methodological Strategies for Addressing Generalizability in Neuropsychological Assessment

Generalizability considerations are widely discussed and a core foundation for understanding when and why treatment effects will replicate across sample demographics. However, guidelines on assessing and reporting generalizability-related factors differ across fields and are inconsistently applied. This paper synthesizes obstacles and best practices to apply recent work on measurement and sample diversity. We present a brief history of how knowledge in psychology has been constructed, with implications for who has been historically prioritized in research. We then review how generalizability remains a contemporary threat to neuropsychological assessment and outline best practices for researchers and clinical neuropsychologists. In doing so, we provide concrete tools to evaluate whether a given assessment is generalizable across populations and assist researchers in effectively testing and reporting treatment differences across sample demographics.

Can We Expand the Pool of Youth Who Receive Telehealth Assessments for ADHD? Covariates of Service Utilization

BACKGROUND

During the COVID-19 pandemic, telehealth became widely utilized for healthcare, including psychological evaluations. However, whether telehealth has reduced or exacerbated healthcare disparities for children with Attention-Deficit/Hyperactivity Disorder (ADHD) remains unclear.

METHODS

Data (race, ethnicity, age, insurance type, ADHD presentation, comorbidities, and distance to clinic) for youth with ADHD (Mage = 10.97, SDage = 3.42; 63.71% male; 51.62% White) were extracted from the medical record at an urban academic medical center. Three naturally occurring groups were compared: those evaluated in person prior to COVID-19 (n =780), in person during COVID-19 (n = 839), and via telehealth during COVID-19 (n = 638).

RESULTS

Children seen via telehealth were significantly more likely to be older, White, have fewer comorbid conditions, and live farther from the clinic than those seen in person.

CONCLUSIONS

The current study suggests that telehealth has not eliminated barriers to care for disadvantaged populations. Providers and institutions must take action to encourage telehealth use among these groups.

Does Assessment Format Matter? A Comparison of In-Person Versus Teletesting Scores for Youth with ADHD

OBJECTIVE

This study examined test score equivalency between traditional in-person assessment and teletesting among youth diagnosed with Attention-Deficit/Hyperactivity Disorder (ADHD).

METHOD

In all, 896 youth with ADHD, ages 5-21 years, were administered cognitive, academic achievement, and verbal fluency measures via either teletesting (n = 448) or traditional in-person assessment (n = 448). The teletesting and in-person groups were matched on age, sex, and insurance type (as a proxy for income).

RESULTS

Results indicated no significant differences in test scores obtained via in-person and teletesting evaluations across all examined measures.

CONCLUSION

Clinically referred youth with ADHD perform similarly on measures of cognitive functioning, academic achievement, and verbal fluency, regardless of whether these measures are administered in-person or via teletesting. While additional evidence for equivalent psychometric properties of neuropsychological instruments administered remotely is needed, this study offers support for the validity of remote administration among youth with ADHD.

Tele-assessment of cognitive functions in children: a systematic review

Cognitive Tele-Assessment approach (CTA) has been widely used in adults for clinical, research, and screening purposes. In the last decades, it has been considered a useful tool for evaluating child development in both clinical and educational settings and new instruments for CTA in children have been developed. In comparison to In Person Assessment (IPA), CTA can have several advantages, such as increasing accessibility, cutting waiting lists, reducing time and travel costs, and assisting with infection control by minimizing face-to-face contact in times of pandemic. Nevertheless, several issues related to the feasibility and reliability of using CTA to evaluate cognitive development are still open. The present systematic review has a twofold aim: 1. to describe the cognitive functions that are most frequently measured by CTA in children, the procedures used, and the characteristics of the samples investigated; 2. to investigate the agreement between CTA and IPA scores in children.In the present systematic review, 23 studies using CTA in children, with typical or atypical development, have been selected and analyzed. Results support the similarities in performance scores between IPA and CTA and good compliance by children and their families in participating in CTA. Nonetheless, most studies suggest that several methodological precautions must be taken to manage technical and procedural characteristics that may represent challenges for CTA of children. Suggestions for a correct use of CTA, factors affecting the validity of the results and directions for future research are discussed.