A computerized tablet with visual feedback of hand position for functional magnetic resonance imaging

Perceptual-Cognitive Integration for Goal-Directed Action in Naturalistic Environments

Real-world actions require one to simultaneously perceive, think, and act on the surrounding world, requiring the integration of (bottom-up) sensory information and (top-down) cognitive and motor signals. Studying these processes involves the intellectual challenge of cutting across traditional neuroscience silos, and the technical challenge of recording data in uncontrolled natural environments. However, recent advances in techniques, such as neuroimaging, virtual reality, and motion tracking, allow one to address these issues in naturalistic environments for both healthy participants and clinical populations. In this review, we survey six topics in which naturalistic approaches have advanced both our fundamental understanding of brain function and how neurologic deficits influence goal-directed, coordinated action in naturalistic environments. The first part conveys fundamental neuroscience mechanisms related to visuospatial coding for action, adaptive eye-hand coordination, and visuomotor integration for manual interception. The second part discusses applications of such knowledge to neurologic deficits, specifically, steering in the presence of cortical blindness, impact of stroke on visual-proprioceptive integration, and impact of visual search and working memory deficits. This translational approach-extending knowledge from lab to rehab-provides new insights into the complex interplay between perceptual, motor, and cognitive control in naturalistic tasks that are relevant for both basic and clinical research.

An fMRI meta-analysis of the role of the striatum in everyday-life vs laboratory-developed habits

The dorsolateral striatum plays a critical role in the acquisition and expression of stimulus-response habits that are learned in experimental laboratories. Here, we use meta-analytic procedures to contrast the neural circuits activated by laboratory-acquired habits with those activated by stimulus-response behaviours acquired in everyday-life. We confirmed that newly learned habits rely more on the anterior putamen with activation extending into caudate and nucleus accumbens. Motor and associative components of everyday-life habits were identified. We found that motor-dominant stimulus-response associations developed outside the laboratory primarily engaged posterior dorsal putamen, supplementary motor area (SMA) and cerebellum. Importantly, associative components were also represented in the posterior putamen. Thus, common neural representations for both naturalistic and laboratory-based habits were found in the left posterior and right anterior putamen. These findings suggest a partial common striatal substrate for habitual actions that are performed predominantly by stimulus-response associations represented in the posterior striatum. The overlapping neural substrates for laboratory and everyday-life habits supports the use of both methods for the analysis of habitual behaviour.

Protracted Neural Development of Dorsal Motor Systems During Handwriting and the Relation to Early Literacy Skills

Handwriting is a complex visual-motor skill that affects early reading development. A large body of work has demonstrated that handwriting is supported by a widespread neural system comprising ventral-temporal, parietal, and frontal motor regions in adults. Recent work has demonstrated that this neural system is largely established by 8 years of age, suggesting that the development of this system occurs in young children who are still learning to read and write. We made use of a novel MRI-compatible writing tablet that allowed us to measure brain activation in 5-8-year-old children during handwriting. We compared activation during handwriting in children and adults to provide information concerning the developmental trajectory of the neural system that supports handwriting. We found that parietal and frontal motor involvement during handwriting in children is different from adults, suggesting that the neural system that supports handwriting changes over the course of development. Furthermore, we found that parietal and frontal motor activation correlated with a literacy composite score in our child sample, suggesting that the individual differences in the dorsal response during handwriting are related to individual differences in emerging literacy skills. Our results suggest that components of the widespread neural system supporting handwriting develop at different rates and provide insight into the mechanisms underlying the contributions of handwriting to early literacy development.

Trail Making Test Performance Using a Touch-Sensitive Tablet: Behavioral Kinematics and Electroencephalography

The Trail Making Test (TMT) is widely used to probe brain function and is performed with pen and paper, involving Parts A (linking numbers) and B (alternating between linking numbers and letters). The relationship between TMT performance and the underlying brain activity remains to be characterized in detail. Accordingly, sixteen healthy young adults performed the TMT using a touch-sensitive tablet to capture enhanced performance metrics, such as the speed of linking movements, during simultaneous electroencephalography (EEG). Linking and non-linking periods were derived as estimates of the time spent executing and preparing movements, respectively. The seconds per link (SPL) was also used to quantify TMT performance. A strong effect of TMT Part A and B was observed on the SPL value as expected (Part B showing increased SPL value); whereas the EEG results indicated robust effects of linking and non-linking periods in multiple frequency bands, and effects consistent with the underlying cognitive demands of the test.

Tablet Technology for Writing and Drawing during Functional Magnetic Resonance Imaging: A Review

Functional magnetic resonance imaging (fMRI) is a powerful modality to study brain activity. To approximate naturalistic writing and drawing behaviours inside the scanner, many fMRI-compatible tablet technologies have been developed. The digitizing feature of the tablets also allows examination of behavioural kinematics with greater detail than using paper. With enhanced ecological validity, tablet devices have advanced the fields of neuropsychological tests, neurosurgery, and neurolinguistics. Specifically, tablet devices have been used to adopt many traditional paper-based writing and drawing neuropsychological tests for fMRI. In functional neurosurgery, tablet technologies have enabled intra-operative brain mapping during awake craniotomy in brain tumour patients, as well as quantitative tremor assessment for treatment outcome monitoring. Tablet devices also play an important role in identifying the neural correlates of writing in the healthy and diseased brain. The fMRI-compatible tablets provide an excellent platform to support naturalistic motor responses and examine detailed behavioural kinematics.

Ecological validity of experimental set-up affects parietal involvement during letter production

Studies of symbol production using fMRI often use techniques that introduce an artificial pairing between motor production and visual perception. These techniques allow participants to see their own output by recording their pen trajectories using a touchscreen-only tablet and displaying these productions on a mirror placed above their head. We recently developed an MR-safe writing tablet with video display that allows participants to see their own hand and their own productions while producing symbols in real time on the surface where they are producing them-allowing for more ecologically valid fMRI studies of production. We conducted a study to determine whether the participation of posterior parietal cortex during symbol production was affected by the pairing of motor production and visual feedback associated with the two types of tablets. We performed ROI analyses in intraparietal sulcus while adult participants produced letters to dictation using either a touchscreen-only tablet (no visual guidance of the hand) (n = 14) or using a touchscreen-and-video-display tablet (visual guidance of the hand) (n = 14). We found that left posterior intraparietal sulcus was more active during production with the touchscreen-only tablet than during production with the touchscreen-and-video-display tablet. These results suggest that posterior parietal involvement during production tasks is associated with the somewhat artificial visual-motor pairing that is introduced by the techniques used in some studies of symbol production.

Functional magnetic resonance imaging of the trail-making test in older adults

The trail-making test (TMT) is a popular neuropsychological test, which is used extensively to measure cognitive impairment associated with neurodegenerative disorders in older adults. Behavioural performance on the TMT has been investigated in older populations, but there is limited research on task-related brain activity in older adults. The current study administered a naturalistic version of the TMT to a healthy older-aged population in an MRI environment using a novel, MRI-compatible tablet. Functional MRI was conducted during task completion, allowing characterization of the brain activity associated with the TMT. Performance on the TMT was evaluated using number of errors and seconds per completion of each link. Results are reported for 36 cognitively healthy older adults between the ages of 52 and 85. Task-related activation was observed in extensive regions of the bilateral frontal, parietal, temporal and occipital lobes as well as key motor areas. Increased age was associated with reduced brain activity and worse task performance. Specifically, older age was correlated with decreased task-related activity in the bilateral occipital, temporal and parietal lobes. These results suggest that healthy older aging significantly affects brain function during the TMT, which consequently may result in performance decrements. The current study reveals the brain activation patterns underlying TMT performance in a healthy older aging population, which functions as an important, clinically-relevant control to compare to pathological aging in future investigations.

Functional MRI of Letter Cancellation Task Performance in Older Adults

The Letter Cancellation Task (LCT) is a widely used pen-and-paper probe of attention in clinical and research settings. Despite its popularity, the neural correlates of the task are not well understood. The present study uses functional magnetic resonance imaging (fMRI) and specialized tablet technology to identify the neural correlates of the LCT in 32 healthy older adults between 50-85 years of age, and further investigates the effect of healthy aging on performance. Subjects performed the LCT in its standard pen-and-paper administration and with the tablet during fMRI. Performance on the tablet was significantly slower than on pen-and-paper, with both response modes showing slower performance as a function of age. Across all ages, bilateral brain activation was observed in the cerebellum, superior temporal lobe, precentral gyrus, frontal gyrus, and occipital and parietal areas. Increasing age correlated with reduced brain activity in the supplementary motor area, middle occipital gyrus, medial and inferior frontal gyrus, cerebellum and putamen. Better LCT performance was correlated with increased activity in the middle frontal gyrus, and reduced activity in the cerebellum. The brain regions activated are associated with visuospatial attention and motor control, and are consistent with the neural correlates of LCT performance previously identified in lesion studies.

The Neural Correlates of the Clock-Drawing Test in Healthy Aging

Importance: The clock-drawing test (CDT) is an important neurocognitive assessment tool, widely used as a screening test for dementia. Behavioral performance on the test has been studied extensively, but there is scant literature on the underlying neural correlates.

Purpose: To administer the CDT naturalistically to a healthy older aging population in an MRI environment, and characterize the brain activity associated with test completion.

Main Outcome and Measure: Blood-oxygen-level dependent (BOLD) functional MRI was conducted as participants completed the CDT using novel tablet technology. Brain activity during CDT performance was contrasted to rest periods of visual fixation. Performance on the CDT was evaluated using a standardized scoring system (Rouleau score) and time to test completion. To assess convergent validity, performance during fMRI was compared to performance on a standard paper version of the task, administered in a psychometric testing room.

Results: Study findings are reported for 33 cognitively healthy older participants aged 52–85. Activation was observed in the bilateral frontal, occipital and parietal lobes as well as the supplementary motor area and precentral gyri. Increased age was significantly correlated with Rouleau scores on the clock number drawing (R2) component (rho = -0.55, p < 0.001); the clock hand drawing (R3) component (rho = -0.50, p < 0.005); and the total clock (rho = -0.62, p < 0.001). Increased age was also associated with decreased activity in the bilateral parietal and occipital lobes as well as the right temporal lobe and right motor areas.

Conclusion and Relevance: This imaging study characterizes the brain activity underlying performance of the CDT in a healthy older aging population using the most naturalistic version of the task to date. The results suggest that the functions of the occipital and parietal lobe are significantly altered by the normal aging process, which may lead to performance decrements.

The MRItab: A MR-compatible touchscreen with video-display

BACKGROUND

A touchscreen interface permits rich user interactions for research in many fields, but is rarely found within a Magnetic Resonance Imaging (MRI) environment due to difficulties adapting conventional technologies to the strong electromagnetic fields. Conventional MR-compatible video display technology uses either large-screen displays that are placed outside of the bore of the MRI itself, or projectors located beyond the participant's reach, making touch interfaces impossible.

NEW METHOD

Here, we describe the MR-compatibility of the 'MRItab' in terms of MR safety and image quality. The MRItab adapts inexpensive off-the-shelf components with special signal-driver circuitry and shielding to bring the touchscreen interface into the MR environment, without adversely affecting MRI image quality, thereby making touch interfaces possible.

RESULTS

Our testing demonstrated that the functioning of the MRItab was not affected by the functioning of the MRI scanner and that the MRItab did not adversely affect the image data acquired. Participants were able to interact naturally with the MRItab during MRI scanning.

COMPARISON WITH OTHER METHOD (S)

The MRItab is the first MR-compatible touchscreen device with video-display screen capabilities designed for use in the MRI environment. This interactive digital device is the first to allow participants to see their hands directly as they interact with a touch-sensitive display screen, resulting in high ecological validity.

CONCLUSIONS

The MRItab provides a methodological advantage for research in many fields, given the realistic human-computer interaction it supports.

Functional MRI of Handwriting Tasks: A Study of Healthy Young Adults Interacting with a Novel Touch-Sensitive Tablet

Handwriting is a complex human activity that engages a blend of cognitive and visual motor skills. Current understanding of the neural correlates of handwriting has largely come from lesion studies of patients with impaired handwriting. Task-based fMRI studies would be useful to supplement this work. To address concerns over ecological validity, previously we developed a fMRI-compatible, computerized tablet system for writing and drawing including visual feedback of hand position and an augmented reality display. The purpose of the present work is to use the tablet system in proof-of-concept to characterize brain activity associated with clinically relevant handwriting tasks, originally developed to characterize handwriting impairments in Alzheimer’s disease patients. As a prelude to undertaking fMRI studies of patients, imaging was performed of twelve young healthy subjects who copied sentences, phone numbers, and grocery lists using the fMRI-compatible tablet. Activation maps for all handwriting tasks consisted of a distributed network of regions in reasonable agreement with previous studies of handwriting performance. In addition, differences in brain activity were observed between the test subcomponents consistent with different demands of neural processing for successful task performance, as identified by investigating three quantitative behavioral metrics (writing speed, stylus contact force and stylus in air time). This study provides baseline behavioral and brain activity results for fMRI studies that adopt this handwriting test to characterize patients with brain impairments.

Tablet-Based Functional MRI of the Trail Making Test: Effect of Tablet Interaction Mode

The Trail Making Test (TMT) is widely used for assessing executive function, frontal lobe abilities, and visual motor skills. Part A of this pen-and-paper test (TMT-A) involves linking numbers randomly distributed in space, in ascending order. Part B (TMT-B) alternates between linking numbers and letters. TMT-B is more demanding than TMT-A, but the mental processing that supports the performance of this test remains incompletely understood. Functional MRI (fMRI) may help to clarify the relationship between TMT performance and brain activity, but providing an environment that supports real-world pen-and-paper interactions during fMRI is challenging. Previously, an fMRI-compatible tablet system was developed for writing and drawing with two modes of interaction: the original cursor-based, proprioceptive approach, and a new mode involving augmented reality to provide visual feedback of hand position (VFHP) for enhanced user interaction. This study characterizes the use of the tablet during fMRI of young healthy adults (n = 22), with half of the subjects performing TMT with VFHP and the other half performing TMT without VFHP. Activation maps for both TMT-A and TMT-B performance showed considerable overlap between the two tablet modes, and no statistically differences in brain activity were detected when contrasting TMT-B vs. TMT-A for the two tablet modes. Behavioral results also showed no statistically different interaction effects for TMT-B vs. TMT-A for the two tablet modes. Tablet-based TMT scores showed reasonable convergent validity with those obtained by administering the standard pen-and-paper TMT to the same subjects. Overall, the results suggest that despite the slightly different mechanisms involved for the two modes of tablet interaction, both are suitable for use in fMRI studies involving TMT performance. This study provides information for using tablet-based TMT methods appropriately in future fMRI studies involving patients and healthy individuals.

Evaluation of Handwriting Movement Kinematics: From an Ecological to a Magnetic Resonance Environment

Writing is a means of communication which requires complex motor, perceptual, and cognitive skills. If one of these abilities gets lost following traumatic events or due to neurological diseases, handwriting could deteriorate. Occupational therapy practitioners provide rehabilitation services for people with impaired handwriting. However, to determine the effectiveness of handwriting interventions no studies assessed whether the proposed treatments improved the kinematics of writing movement or had an effect at the level of the central nervous system. There is need to find new quantitative methodologies able to describe the behavioral and the neural outcomes of the rehabilitative interventions for handwriting. In the present study we proposed a combined approach that allowed evaluating the kinematic parameters of handwriting movements, acquired by means of a magnetic resonance-compatible tablet, and their neural correlates obtained simultaneously from a functional magnetic resonance imaging examination. Results showed that the system was reliable in term of reproducibility of the kinematic data during a test/re-test procedure. Further, despite the modifications with respect to an ecological writing movement condition, the kinematic parameters acquired inside the MR-environment were descriptive of individuals’ movement features. At last, the imaging protocol succeeded to show the activation of the cerebral regions associated with the production of writing movement in healthy people. From these findings, this methodology seems to be promising to evaluate the handwriting movement deficits and the potential alterations in the neural activity in those individuals who have handwriting difficulties. Finally, it would provide a mean to quantitatively assess the effect of a rehabilitative treatment.