Real-life ankle submovements and computer mouse use reflect patient-reported function in adult ataxias

Novel disease-modifying therapies are being evaluated in spinocerebellar ataxias and multiple system atrophy. Clinician-performed disease rating scales are relatively insensitive for measuring disease change over time, resulting in large and long clinical trials. We tested the hypothesis that sensors worn continuously at home during natural behaviour and a web-based computer mouse task performed at home could produce interpretable, meaningful and reliable motor measures for potential use in clinical trials. Thirty-four individuals with degenerative ataxias (spinocerebellar ataxia types 1, 2, 3 and 6 and multiple system atrophy of the cerebellar type) and eight age-matched controls completed the cross-sectional study. Participants wore an ankle and wrist sensor continuously at home for 1 week and completed the Hevelius computer mouse task eight times over 4 weeks. We examined properties of motor primitives called 'submovements' derived from the continuous wearable sensors and properties of computer mouse clicks and trajectories in relationship to patient-reported measures of function (Patient-Reported Outcome Measure of Ataxia) and ataxia rating scales (Scale for the Assessment and Rating of Ataxia and the Brief Ataxia Rating Scale). The test-retest reliability of digital measures and differences between ataxia and control participants were evaluated. Individuals with ataxia had smaller, slower and less powerful ankle submovements during natural behaviour at home. A composite measure based on ankle submovements strongly correlated with ataxia rating scale scores (Pearson's r = 0.82-0.88), strongly correlated with self-reported function (r = 0.81), had high test-retest reliability (intraclass correlation coefficient = 0.95) and distinguished ataxia and control participants, including preataxic individuals (n = 4) from controls. A composite measure based on computer mouse movements and clicks strongly correlated with ataxia rating scale total (r = 0.86-0.88) and arm scores (r = 0.65-0.75), correlated well with self-reported function (r = 0.72-0.73) and had high test-retest reliability (intraclass correlation coefficient = 0.99). These data indicate that interpretable, meaningful and highly reliable motor measures can be obtained from continuous measurement of natural movement, particularly at the ankle location, and from computer mouse movements during a simple point-and-click task performed at home. This study supports the use of these two inexpensive and easy-to-use technologies in longitudinal natural history studies in spinocerebellar ataxias and multiple system atrophy of the cerebellar type and shows promise as potential motor outcome measures in interventional trials.

Accuracy and Reliability of At-home Quantification of Motor Impairments Using a Computer-based Pointing Task with Children with Ataxia-Telangiectasia

Methods for obtaining accurate quantitative assessments of motor impairments are essential in accessibility research, design of adaptive ability-based assistive technologies, as well as in clinical care and medical research. Currently, such assessments are typically performed in controlled laboratory or clinical settings under professional supervision. Emerging approaches for collecting data in unsupervised settings have been shown to produce valid data when aggregated over large populations, but it is not yet established if in unsupervised settings measures of research or clinical significance can be collected accurately and reliably for individuals. We conducted a study with 13 children with ataxia-telangiectasia and 9 healthy children to analyze the validity, test-retest reliability, and acceptability of at-home use of a recent active digital phenotyping system, called Hevelius. Hevelius produces 32 measures derived from the movement trajectories of the mouse cursor, and it produces a quantitative estimate of motor impairment in the dominant arm using the dominant arm component of the Brief Ataxia Rating Scale (BARS). The severity score estimates generated by Hevelius from single at-home sessions deviated from clinician-assigned BARS scores more than the severity score estimates generated from single sessions conducted under researcher supervision. However, taking a median of as few as 2 consecutive sessions produced severity score estimates that were as accurate or better than the estimates produced from single supervised sessions. Further, aggregating as few as 2 consecutive sessions resulted in good test-retest reliability (ICC = 0.81 for A-T participants). This work demonstrated the feasibility of performing accurate and reliable quantitative assessments of individual motor impairments in the dominant arm through tasks performed at home without supervision by the researchers. Further work is needed, however, to assess how broadly these results generalize.

Sex and age differences in "theory of mind" across 57 countries using the English version of the "Reading the Mind in the Eyes" Test

The "Reading the Mind in the Eyes" Test (Eyes Test) is a widely used assessment of "theory of mind." The NIMH Research Domain Criteria recommends it as one of two tests for "understanding mental states." Previous studies have demonstrated an on-average female advantage on the Eyes Test. However, it is unknown whether this female advantage exists across the lifespan and across a large number of countries. Thus, we tested sex and age differences using the English version of the Eyes Test in adolescents and adults across 57 countries. We also tested for associations with sociodemographic and cognitive/personality factors. We leveraged one discovery dataset (N = 305,726) and three validation datasets (Ns = 642; 5,284; and 1,087). The results show that: i) there is a replicable on-average female advantage in performance on the Eyes Test; ii) performance increases through adolescence and shallowly declines across adulthood; iii) the on-average female advantage is evident across the lifespan; iv) there is a significant on-average female advantage in 36 out of 57 countries; v) there is a significant on-average female advantage on translated (non-English) versions of the Eyes Test in 12 out of 16 countries, as confirmed by a systematic review; vi) D-scores, or empathizing-systemizing, predict Eyes Test performance above and beyond sex differences; and vii) the female advantage is negatively linked to "prosperity" and "autonomy," and positively linked to "collectivism," as confirmed by exploratory country-level analyses. We conclude that the on-average female advantage on the Eyes Test is observed across ages and most countries.

Barriers and Facilitators to the Implementation of Family-Centered Technology in Complex Care: Feasibility Study

Background

Care coordination is challenging but crucial for children with medical complexity (CMC). Technology-based solutions are increasingly prevalent but little is known about how to successfully deploy them in the care of CMC.

Objective

The aim of this study was to assess the feasibility and acceptability of GoalKeeper (GK), an internet-based system for eliciting and monitoring family-centered goals for CMC, and to identify barriers and facilitators to implementation.

Methods

We used the Consolidated Framework for Implementation Research (CFIR) to explore the barriers and facilitators to the implementation of GK as part of a clinical trial of GK in ambulatory clinics at a children’s hospital (NCT03620071). The study was conducted in 3 phases: preimplementation, implementation (trial), and postimplementation. For the trial, we recruited providers at participating clinics and English-speaking parents of CMC<12 years of age with home internet access. All participants used GK during an initial clinic visit and for 3 months after. We conducted preimplementation focus groups and postimplementation semistructured exit interviews using the CFIR interview guide. Participant exit surveys assessed GK feasibility and acceptability on a 5-point Likert scale. For each interview, 3 independent coders used content analysis and serial coding reviews based on the CFIR qualitative analytic plan and assigned quantitative ratings to each CFIR construct (–2 strong barrier to +2 strong facilitator).

Results

Preimplementation focus groups included 2 parents (1 male participant and 1 female participant) and 3 providers (1 in complex care, 1 in clinical informatics, and 1 in neurology). From focus groups, we developed 3 implementation strategies: education (parents: 5-minute demo; providers: 30-minute tutorial and 5-minute video on use in a clinic visit; both: instructional manual), tech support (in-person, virtual), and automated email reminders for parents. For implementation (April 1, 2019, to December 21, 2020), we enrolled 11 providers (7 female participants, 5 in complex care) and 35 parents (mean age 38.3, SD 7.8 years; n=28, 80% female; n=17, 49% Caucasian; n=16, 46% Hispanic; and n=30, 86% at least some college). One parent-provider pair did not use GK in the clinic visit, and few used GK after the visit. In 18 parent and 9 provider exit interviews, the key facilitators were shared goal setting, GK’s internet accessibility and email reminders (parents), and GK’s ability to set long-term goals and use at the end of visits (providers). A key barrier was GK’s lack of integration into the electronic health record or patient portal. Most parents (13/19) and providers (6/9) would recommend GK to their peers.

Conclusions

Family-centered technologies like GK are feasible and acceptable for the care of CMC, but sustained use depends on integration into electronic health records.

Trial Registration

ClinicalTrials.gov NCT03620071; https://clinicaltrials.gov/ct2/show/NCT03620071

Outcomes from Returning Individual versus Only Study-Wide Biomonitoring Results in an Environmental Exposure Study Using the Digital Exposure Report-Back Interface (DERBI)

BACKGROUND

Study participants want to receive their biomonitoring results for environmental chemicals, and ethics guidelines encourage reporting back. However, few studies have quantitively assessed participants' responses to individual exposure reports, and digital methods have not been evaluated.

OBJECTIVES

We isolated effects of receiving personal results vs. only study-wide findings and investigated whether effects differed for Black participants.

METHODS

We randomly assigned a subset of 295 women from the Child Health and Development Studies, half of whom were Black, to receive a report with personal environmental chemical results or only study-wide (aggregate) findings. Reports included results for 42 chemicals and lipids and were prepared using the Digital Exposure Report-Back Interface (DERBI). Women were interviewed before and after viewing their report. We analyzed differences in website activity, emotional responses, and intentions to participate in future research by report type and race using Wilcoxon rank sum tests, Wilcoxon-Pratt signed ranks tests, and multiple regression.

RESULTS

The personal report group spent approximately twice as much time on their reports as the aggregate group before the post-report-back interview. Among personal-report participants (n=93), 84% (78) viewed chemical group information for at least one personal result highlighted on their home page; among aggregate-report participants (n=94), 66% (62) viewed any chemical group page. Both groups reported strong positive feelings (curious, informed, interested, respected) about receiving results before and after report-back and mild negative feelings (helpless, scared, worried). Although most participants remained unworried after report-back, worry increased by a small amount in both groups. Among Black participants, higher post report-back worry was associated with having high levels of chemicals.

CONCLUSIONS

Participants were motivated by their personal results to access online information about chemical sources and potential health effects. Report-back was associated with a small increase in worry, which could motivate appropriate action. Personal report-back increased engagement with exposure reports among Black participants. https://doi.org/10.1289/EHP9072.

Free-Living Motor Activity Monitoring in Ataxia-Telangiectasia

With disease-modifying approaches under evaluation in ataxia-telangiectasia and other ataxias, there is a need for objective and reliable biomarkers of free-living motor function. In this study, we test the hypothesis that metrics derived from a single wrist sensor worn at home provide accurate, reliable, and interpretable information about neurological disease severity in children with A-T.

A total of 15 children with A-T and 15 age- and sex-matched controls wore a sensor with a triaxial accelerometer on their dominant wrist for 1 week at home. Activity intensity measures, derived from the sensor data, were compared with in-person neurological evaluation on the Brief Ataxia Rating Scale (BARS) and performance on a validated computer mouse task.

Children with A-T were inactive the same proportion of each day as controls but produced more low intensity movements (p < 0.01; Cohen’s d = 1.48) and fewer high intensity movements (p < 0.001; Cohen’s d = 1.71). The range of activity intensities was markedly reduced in A-T compared to controls (p < 0.0001; Cohen’s d = 2.72). The activity metrics correlated strongly with arm, gait, and total clinical severity (r: 0.71–0.87; p < 0.0001), correlated with specific computer task motor features (r: 0.67–0.92; p < 0.01), demonstrated high reliability (r: 0.86–0.93; p < 0.00001), and were not significantly influenced by age in the healthy control group.

Motor activity metrics from a single, inexpensive wrist sensor during free-living behavior provide accurate and reliable information about diagnosis, neurological disease severity, and motor performance. These low-burden measurements are applicable independent of ambulatory status and are potential digital behavioral biomarkers in A-T.

How machine-learning recommendations influence clinician treatment selections: the example of the antidepressant selection

Decision support systems embodying machine learning models offer the promise of an improved standard of care for major depressive disorder, but little is known about how clinicians' treatment decisions will be influenced by machine learning recommendations and explanations. We used a within-subject factorial experiment to present 220 clinicians with patient vignettes, each with or without a machine-learning (ML) recommendation and one of the multiple forms of explanation. We found that interacting with ML recommendations did not significantly improve clinicians' treatment selection accuracy, assessed as concordance with expert psychopharmacologist consensus, compared to baseline scenarios in which clinicians made treatment decisions independently. Interacting with incorrect recommendations paired with explanations that included limited but easily interpretable information did lead to a significant reduction in treatment selection accuracy compared to baseline questions. These results suggest that incorrect ML recommendations may adversely impact clinician treatment selections and that explanations are insufficient for addressing overreliance on imperfect ML algorithms. More generally, our findings challenge the common assumption that clinicians interacting with ML tools will perform better than either clinicians or ML algorithms individually.

Visualization of Electronic Health Record Data for Decision-Making in Diabetes and Congestive Heart Failure

Objective The aim of this study is to study the impact of graphical representation of health record data on physician decision-making to inform the design of health information technology.

Materials and Methods We conducted a within participants crossover design study using a simulated electronic health record (EHR) in which we presented cases with and without visualized data designed to highlight important clinical trends or relationships, followed by assessment of the impact on decision-making about next steps for patients with chronic diseases. We then asked whether trends were observed and about usability and satisfaction using validated usability questions and asked open-ended questions as well. Time to answer questions was also collected.

Results Twenty-one primary care providers participated in the study, including five for testing only and sixteen for the full study. Questions about clinical assessment or next actions were answered correctly 55% of the time. Regarding objective trends in the data, participants described noticing the trends 85% of the time. Differences in noticing trends or difficulty level of questions were not statistically significant. Satisfaction with the tool was high and participants agreed strongly that it helped them make better decisions without adding to the time it took.

Discussion The simulation allowed us to test the impact of a visualization on clinician practice in a realistic setting. Designers of EHRs should consider the ways information presentation can affect decision-making.

Conclusion Testing visualization tools can be done in a clinically realistic context. Providers desire visualizations and believe that they help them make better and faster decisions.

Conducting online virtual environment experiments with uncompensated, unsupervised samples

Web-based experimentation with uncompensated and unsupervised samples allows for a larger and more diverse sample population, more generalizable results, and faster theory to experiment cycle. Given that participants are unsupervised, it is still unknown whether the data collected in such settings would be of sufficiently high quality to support robust conclusions. Therefore, we investigated the feasibility of conducting such experiments online using virtual environment technologies. We conducted a conceptual replication of two prior experiments that have been conducted in virtual environments. Our results replicate findings previously obtained in conventional laboratory settings. These results hold across different device types of participants (ranging from desktop, through mobile devices to immersive virtual reality headsets), suggesting that experiments can be conducted online with uncompensated samples in virtual environments.

Computer mouse use captures ataxia and parkinsonism, enabling accurate measurement and detection

Background

Objective assessments of movement impairment are needed to support clinical trials and facilitate diagnosis. The objective of the current study was to determine if a rapid web‐based computer mouse test (Hevelius) could detect and accurately measure ataxia and parkinsonism.

Methods

Ninety‐five ataxia, 46 parkinsonism, and 29 control participants and 229,017 online participants completed Hevelius. We trained machine‐learning models on age‐normalized Hevelius features to (1) measure severity and disease progression and (2) distinguish phenotypes from controls and from each other.

Results

Regression model estimates correlated strongly with clinical scores (from r = 0.66 for UPDRS dominant arm total to r = 0.83 for the Brief Ataxia Rating Scale). A disease change model identified ataxia progression with high sensitivity. Classification models distinguished ataxia or parkinsonism from healthy controls with high sensitivity (≥0.91) and specificity (≥0.90).

Conclusions

Hevelius produces a granular and accurate motor assessment in a few minutes of mouse use and may be useful as an outcome measure and screening tool. © 2019 The Authors. Movement Disorders published by Wiley Periodicals, Inc. on behalf of International Parkinson and Movement Disorder Society.

DERBI: A Digital Method to Help Researchers Offer "Right-to-Know" Personal Exposure Results

Researchers and clinicians in environmental health and medicine increasingly show respect for participants and patients by involving them in decision-making. In this context, the return of personal results to study participants is becoming ethical best practice, and many participants now expect to see their data. However, researchers often lack the time and expertise required for report-back, especially as studies measure greater numbers of analytes, including many without clear health guidelines. In this article, our goal is to demonstrate how a prototype digital method, the Digital Exposure Report-Back Interface (DERBI), can reduce practical barriers to high-quality report-back. DERBI uses decision rules to automate the production of personalized summaries of notable results and generates graphs of individual results with comparisons to the study group and benchmark populations. Reports discuss potential sources of chemical exposure, what is known and unknown about health effects, strategies for exposure reduction, and study-wide findings. Researcher tools promote discovery by drawing attention to patterns of high exposure and offer novel ways to increase participant engagement. DERBI reports have been field tested in two studies. Digital methods like DERBI reduce practical barriers to report-back thus enabling researchers to meet their ethical obligations and participants to get knowledge they can use to make informed choices.

Evaluation of filesystem provenance visualization tools

Having effective visualizations of filesystem provenance data is valuable for understanding its complex hierarchical structure. The most common visual representation of provenance data is the node-link diagram. While effective for understanding local activity, the node-link diagram fails to offer a high-level summary of activity and inter-relationships within the data. We present a new tool, InProv, which displays filesystem provenance with an interactive radial-based tree layout. The tool also utilizes a new time-based hierarchical node grouping method for filesystem provenance data we developed to match the user's mental model and make data exploration more intuitive. We compared InProv to a conventional node-link based tool, Orbiter, in a quantitative evaluation with real users of filesystem provenance data including provenance data experts, IT professionals, and computational scientists. We also compared in the evaluation our new node grouping method to a conventional method. The results demonstrate that InProv results in higher accuracy in identifying system activity than Orbiter with large complex data sets. The results also show that our new time-based hierarchical node grouping method improves performance in both tools, and participants found both tools significantly easier to use with the new time-based node grouping method. Subjective measures show that participants found InProv to require less mental activity, less physical activity, less work, and is less stressful to use. Our study also reveals one of the first cases of gender differences in visualization; both genders had comparable performance with InProv, but women had a significantly lower average accuracy (56%) compared to men (70%) with Orbiter.

Personalized dynamic accessibility

In this forum we celebrate research that helps to successfully bring the benefits of computing technologies to children, older adults, people with disabilities, and other populations that are often ignored in the design of mass-marketed products. Juan Pablo Hourcade, Editor

Evaluation of artery visualizations for heart disease diagnosis

Heart disease is the number one killer in the United States, and finding indicators of the disease at an early stage is critical for treatment and prevention. In this paper we evaluate visualization techniques that enable the diagnosis of coronary artery disease. A key physical quantity of medical interest is endothelial shear stress (ESS). Low ESS has been associated with sites of lesion formation and rapid progression of disease in the coronary arteries. Having effective visualizations of a patient's ESS data is vital for the quick and thorough non-invasive evaluation by a cardiologist. We present a task taxonomy for hemodynamics based on a formative user study with domain experts. Based on the results of this study we developed HemoVis, an interactive visualization application for heart disease diagnosis that uses a novel 2D tree diagram representation of coronary artery trees. We present the results of a formal quantitative user study with domain experts that evaluates the effect of 2D versus 3D artery representations and of color maps on identifying regions of low ESS. We show statistically significant results demonstrating that our 2D visualizations are more accurate and efficient than 3D representations, and that a perceptually appropriate color map leads to fewer diagnostic mistakes than a rainbow color map.

Ability-Based Design

Current approaches to accessible computing share a common goal of making technology accessible to users with disabilities. Perhaps because of this goal, they may also share a tendency to centralize disability rather than ability. We present a refinement to these approaches called ability-based design that consists of focusing on ability throughout the design process in an effort to create systems that leverage the full range of human potential. Just as user-centered design shifted the focus of interactive system design from systems to users, ability-based design attempts to shift the focus of accessible design from disability to ability. Although prior approaches to accessible computing may consider users’ abilities to some extent, ability-based design makes ability its central focus. We offer seven ability-based design principles and describe the projects that inspired their formulation. We also present a research agenda for ability-based design.

Design Space and Evaluation Challenges of Adaptive Graphical User Interfaces

Adaptive graphical user interfaces (GUIs) have the potential to improve performance and user satisfaction by automatically tailoring the presentation of functionality to each individual user. In practice, however, many challenges exist and evaluation results of adaptive GUIs have been mixed. To guide researchers and designers in developing effective adaptive GUIs, we outline a design space and discuss three important aspects to consider when conducting user evaluations of these types of interfaces: the control and reporting of adaptive algorithm characteristics, the impact of task choice and user characteristics on the overall effectiveness of a design, and evaluation measures that are appropriate for adaptive interaction.

Goal Crossing with Mice and Trackballs for People with Motor Impairments

Prior research shows that people with motor impairments face considerable challenges when using conventional mice and trackballs. One challenge is positioning the mouse cursor within confined target areas; another is executing a precise click without slipping. These problems can make mouse pointing in graphical user interfaces very difficult for some people. This article explores goal crossing as an alternative strategy for more accessible target acquisition. In goal crossing, targets are boundaries that are simply crossed by the mouse cursor. Thus, goal crossing avoids the two aforementioned problems. To date, however, researchers have not examined the feasibility of goal crossing for people with motor difficulties. We therefore present a study comparing area pointing and goal crossing. Our performance results indicate that although Fitts' throughput for able-bodied users is higher for area pointing than for goal crossing (4.72 vs. 3.61 bits/s), the opposite is true for users with motor impairments (2.34 vs. 2.88 bits/s). However, error rates are higher for goal crossing than for area pointing under a strict definition of crossing errors (6.23% vs. 1.94%). We also present path analyses and an examination of submovement velocity, acceleration, and jerk (the change in acceleration over time). These results show marked differences between crossing and pointing and almost categorically favor crossing. An important finding is that crossing reduces jerk for both participant groups, indicating more fluid, stable motion. To help realize the potential of goal crossing for computer access, we offer design concepts for crossing widgets that address the occlusion problem, which occurs when one crossing goal obscures another in persistent mouse-cursor interfaces. This work provides the motivation and initial steps for further exploration of goal crossing on the desktop, and may help researchers and designers to radically reshape user interfaces to provide accessible goal crossing, thereby lowering barriers to access.