Attrition from Web-Based Cognitive Testing: A Repeated Measures Comparison of Gamification Techniques

How do surrogates make treatment decisions for patients with dementia: An experimental survey study

Despite the growing need for surrogate decision-making for older adults, little is known about how surrogates make decisions and whether advance directives would change decision-making. We conducted a nationally representative experimental survey that cross-randomized cognitive impairment, gender, and characteristics of advance care planning among hospitalized older adults through a series of vignettes. Our study yielded three main findings: first, respondents were much less likely to recommend life-sustaining treatments for patients with dementia, especially after personal exposure. Second, respondents were more likely to ignore patient preferences for life-extending treatment when the patient had dementia, and choose unwanted life-extending treatments for patients without dementia. Third, in scenarios where the patient's wishes were unclear, respondents were more likely to choose treatments that matched their own preferences. These findings underscore the need for improved communication and decision-making processes for patients with cognitive impairment and highlight the importance of choosing a surrogate decision-maker with similar treatment preferences.

Measuring the Reliability of a Gamified Stroop Task: Quantitative Experiment

BACKGROUND

Few gamified cognitive tasks are subjected to rigorous examination of psychometric properties, despite their use in experimental and clinical settings. Even small manipulations to cognitive tasks require extensive research to understand their effects.

OBJECTIVE

This study aims to investigate how game elements can affect the reliability of scores on a Stroop task. We specifically investigated performance consistency within and across sessions.

METHODS

We created 2 versions of the Stroop task, with and without game elements, and then tested each task with participants at 2 time points. The gamified task used points and feedback as game elements. In this paper, we report on the reliability of the gamified Stroop task in terms of internal consistency and test-retest reliability, compared with the control task. We used a permutation approach to evaluate internal consistency. For test-retest reliability, we calculated the Pearson correlation and intraclass correlation coefficients between each time point. We also descriptively compared the reliability of scores on a trial-by-trial basis, considering the different trial types.

RESULTS

At the first time point, the Stroop effect was reduced in the game condition, indicating an increase in performance. Participants in the game condition had faster reaction times (P=.005) and lower error rates (P=.04) than those in the basic task condition. Furthermore, the game condition led to higher measures of internal consistency at both time points for reaction times and error rates, which indicates a more consistent response pattern. For reaction time in the basic task condition, at time 1, rSpearman-Brown=0.78, 95% CI 0.64-0.89. At time 2, rSpearman-Brown=0.64, 95% CI 0.40-0.81. For reaction time, in the game condition, at time 1, rSpearman-Brown=0.83, 95% CI 0.71-0.91. At time 2, rSpearman-Brown=0.76, 95% CI 0.60-0.88. Similarly, for error rates in the basic task condition, at time 1, rSpearman-Brown=0.76, 95% CI 0.62-0.87. At time 2, rSpearman-Brown=0.74, 95% CI 0.58-0.86. For error rates in the game condition, at time 1, rSpearman-Brown=0.76, 95% CI 0.62-0.87. At time 2, rSpearman-Brown=0.74, 95% CI 0.58-0.86. Test-retest reliability analysis revealed a distinctive performance pattern depending on the trial type, which may be reflective of motivational differences between task versions. In short, especially in the incongruent trials where cognitive conflict occurs, performance in the game condition reaches peak consistency after 100 trials, whereas performance consistency drops after 50 trials for the basic version and only catches up to the game after 250 trials.

CONCLUSIONS

Even subtle gamification can impact task performance albeit not only in terms of a direct difference in performance between conditions. People playing the game reach peak performance sooner, and their performance is more consistent within and across sessions. We advocate for a closer examination of the impact of game elements on performance.

A Smartphone-Gamified Virtual Reality Exposure Therapy Augmented With Biofeedback for Ailurophobia: Development and Evaluation Study

BACKGROUND

To the best of our knowledge, no specialized research has been conducted to address ailurophobia (fear of cats) in Iran or globally. This has driven our project, along with the prevalence of ailurophobia and the absence of a gamified virtual reality exposure therapy (VRET) that incorporates affordable and easily accessible biofeedback (BF) tools. We hypothesize that a gamified VRET augmented with BF will yield more positive effects than a similar device lacking BF.

OBJECTIVE

This study primarily focuses on the development and preliminary evaluation of a smartphone-gamified VRET integrated with BF, targeting animal phobia, with a specific case study on ailurophobia. The secondary objectives are using affordable and readily available BF found in devices such as smart bands and smartwatches and creating a mobile virtual reality gamified app to improve patients' adherence to treatments while simultaneously enhancing the app's accessibility, scalability, and outreach.

METHODS

Evaluations encompassed 3 methods. First, we identified the tool's potential positive effects on phobia interventions, exploring 4 effects: intrinsic motivation, simulation of fearful situations, management of stressful circumstances without therapists' presence and mitigation of catastrophic thoughts, and preliminary effects on ailurophobia treatment. Participants were divided into BF and non-BF groups. Second, we gathered user preferences and opinions about the treatment. Third, we conducted heuristic evaluations using 44 heuristics from existing system usability scales assessing user interfaces, virtual reality platforms, and video games' playability. To interpret the data, mean scores; ANOVA, single factor; and ANOVA, 2-factor with replication were used. A total of 29 individuals were identified, of which 10 met the eligibility criteria or were accessible.

RESULTS

The smartphone-gamified VRET augmented with BF exhibited better results on the identified effects compared with the non-BF version and contributed to normalizing encounters with cats. Moreover, 41 of the 44 heuristics achieved a percentage above 62%, indicating its potential as a therapeutic product and its ability to enhance patient adherence to treatments. Patient preferences on the treatment and its strengths and weaknesses were provided for further improvement.

CONCLUSIONS

The tool has the potential to evolve into a comprehensive solution by incorporating various types of cats and their behaviors, simulating environments in which they are commonly found, and enhancing its appeal through an increased sense of adventure without inducing unrealistic fears. By adapting fear elements, the game can be tailored to treat various animal phobias. Phobia-focused games should avoid action and combat scenarios to prevent reinforcement of fear responses. After rigorous evaluation, further exploration is required to provide remote use beyond clinical settings.

Effects of the egg protein hydrolysate NWT-03 on cognitive function in men and women with the metabolic syndrome: a randomized, double-blind, placebo-controlled study

Objectives: The metabolic syndrome is associated with cardiovascular diseases and cognitive decline. The egg protein hydrolysate NWT-03 has shown to improve cardiovascular risk factors in humans. This study investigated whether NWT-03 also has an effect on cognitive function.Methods: Men and women with the metabolic syndrome (n = 76) with a mean age of 60 ± 10 years participated in this randomized, double-blind, placebo-controlled, cross-over trial with an intervention (5 g/day NWT-03) and control period (5 g/day maltodextrin) of 4 weeks separated by a wash-out period of 2-8 weeks. Cognitive function was assessed with the anti-cue reaction time test (impulse control) and psychomotor vigilance test (sustained attention) at day 0, 2, and 27 of both periods. Serum brain-derived neurotrophic factor (BDNF) concentrations were measured at the start and end of both periods.Results: NWT-03 consumption significantly improved the change (day 27 - day 0) in response times of the anti-cue reaction time test compared with the control period (P < 0.001), but not of the psychomotor vigilance test (P = 0.487). Serum BDNF concentrations of all subjects did not significantly change (P = 0.241).Conclusion: NWT-03 has the ability to improve cognitive function within the executive function domain. The underlying mechanism warrants further research and could either be indirect via inhibition of dipeptidyl peptidase 4 (DPP4) or direct via passage of small peptides over the blood-brain barrier inducing local effects.Trial registration: ClinicalTrials.gov identifier: NCT02561663.

A proof-of-concept study exploring the effects of impulsivity on a gamified version of the stop-signal task in children

This proof-of-concept study provides an appraisal of a remotely administered gamified Stop-Signal Task (gSST) for future use in studies using child sample. Performance on the standard Stop-Signal (SST) task has been shown previously to differentiate attention-deficit-hyperactivity-disorder groups from controls. As is the case with the SST, it was envisaged that those with greater impulsivity would perform worse than those with lower levels of impulsivity in the gSST. The potential advantage of the gSST is that it could be perceived as less monotonous than the original SST and has the potential to provide higher data quality in child samples, however future research will need to be conducted to determine this. The gSST was administered remotely via video chat to 30 child participants within a community sample aged 8-12 to investigate the effect of ADHD symptoms and intrinsic motivation on gSST performance. Qualitative data was collected based on feedback from participants to gain insight into how the gSST was received by participants. A positive correlation was observed between impulsive/hyperactivity and gSST performance, however there was insufficient evidence to suggest that impulsivity predicted performance. With regards to accuracy, results suggested that impulsivity level significantly predicted the rate of go-omission errors. No relationships were observed between intrinsic motivation inventory (IMI) subscales and performance or IMI and impulsivity. Nevertheless, mean IMI scores were overarchingly high for each of the IMI subscales, suggesting that regardless of performance and/or level of impulsive behaviour, the child sample obtained in this study demonstrated high levels of intrinsic motivation, which was supported by the predominantly positive subjective feedback provided by the child participants. The present study provides some evidence based on quantitative and qualitative results for the efficacy of gSST for use with children. Future research with a larger sample of children is warranted to examine how performance on the SST and gSST compare/differ.

The Design Preferences for a Serious Game-based Cognitive Assessment of Older People in Prison: Thematic Analysis (Preprint)

BACKGROUND

Serious games have the potential to transform the field of cognitive assessment. The use of serious game-based cognitive assessments in prison environments is particularly exciting. This is because interventions are urgently needed to address the rapid increase in the number of currently incarcerated older adults globally and because of the heightened risks of dementia and cognitive decline present in this population. Game-based assessments are assumed to be fun, engaging, and suitable alternatives to traditional cognitive testing, but these assumptions remain mostly untested in older adults. This is especially true for older adults in prison, whose preferences and needs are seldom heard and may deviate from those previously captured in studies on cognition and serious games.

OBJECTIVE

This study aimed to understand the design preferences of older adults in prison for a game-based cognitive assessment.

METHODS

This study used reflexive thematic analysis, underpinned by critical realism, and applied the technique of abduction. Overall, 4 focus groups with a total of 20 participants were conducted with older adults (aged ≥50 years; aged ≥45 years for Aboriginal and Torres Strait Islander people) across 3 distinct prison environments in Australia.

RESULTS

Self-determination theory was used as a theoretical foundation to interpret the results. Overall, 3 themes were generated: Goldilocks-getting gameplay difficulty just right through optimal challenge (the first theme emphasizes the participants' collective desire for an individualized optimal level of difficulty in serious gameplay), Avoiding Childish Graphics-gimmicky gameplay can be condescending (the second theme raises the importance of avoiding immature and childlike gameplay features, as some older end users in prison felt that these can be condescending), and A Balanced Diet-meaningful choice and variety keeps game-based assessments fun (the third theme highlights the strong user preference for meaningful choice and variety in any serious game-based cognitive assessment to maximize in-game autonomy).

CONCLUSIONS

The collection of these themes provides novel insights into key game design preferences of marginalized older adults.

Considerations on the Obstacles That Lead to Slow Recruitment in a Pain Management Clinical Trial: Experiences from the Belgian PELICAN (PrEgabalin Lidocaine Capsaicin Neuropathic Pain) Pragmatic Study

Background

A qualitative evaluation study of the prematurely terminated PrEgabalin Lidocaine Capsaicin Neuropathic Pain (PELICAN) study was performed. The PELICAN study aimed to examine pain management for localized neuropathic pain (LNP), as epidemiological figures have shown a high percentage of LNP patients in Belgium. The study compared systemic and topical medications according to pain relief, adverse effects, and several measures of quality of life.

Objective

Achieving better study patient recruitment through qualitative research. To investigate and determine the causes of the observed recruitment problems in the PELICAN study, pain centers involved in the study as well as nonrecruiting pain centers were included. Furthermore, it aimed to highlight the positive and negative lessons learned from the conducted study and the number of obstacles the team had to overcome.

Methods

A qualitative study, using a mixed methods approach, was performed. Multiple pain centers in Belgium completed an online survey, after which a structured interview was conducted to elaborate the responses in more detail. The broad topics of these meetings were feedback about the study, reviewing survey answers, and actions undertaken to enhance recruitment.

Results

Different factors contributed to the low recruitment rate in the PELICAN study, such as limited and late referral from the general practitioners to the Belgian pain centers, insufficient internal referrals from nonpain specialists, lack of specific expertise on LNP in some centers, scarcity of staff, limited reimbursement to administer complex analgesic schemes, overestimation of the patient population, and the reluctance of patients to participate in pain research. Additionally, shortcomings in the implemented study design and the need for more logistical investments were identified.

Conclusion

The findings of the qualitative study demonstrate the need for further, more varied LNP research in Belgium, not limited to pharmacological studies. It also sheds important light on the recruitment obstacles that may be faced during these studies. Future studies could support this research by offering better proposals for feasibility and recruitment, for instance, by designing and conducting a compelling pilot study or applying social media during the recruitment phase. Clinical Trials. This trial is registered with NCT03348735. EUDRACT number 2018-003617-17.

The association between trait anxiety and the fear of being laughed at in college: A preliminary study with a Brazilian sample

AIM

To evaluate how gelotophobia correlates with trait anxiety in a sample of Brazilian college students.

METHODS

We evaluated the association of GELOPH &lt; 15 &gt; scores with both self-reported experiences of bullying victimization and trait anxiety measures assessed by the State-Trait Anxiety Inventory (STAI). The study consisted of a sample of 65 adult volunteers (M = 21.48, SD = 2.54 years, 38 females), recruited through social media or flyer distribution, and submitted to online versions of the gelotophobia assessment instrument (GELOPH &lt; 15 &gt;) and the STAI.

RESULTS

Most participants (N = 56, 86.15%) had an STAI-T score indicative of high trait anxiety. The average GELOPH &lt; 15 &gt; score of the sample was 2.69 (0.65) and 39 of the subjects (60%) were considered gelotophobes. There was a strong positive correlation between the GELOPH &lt; 15 &gt; and STAI-T scores but no correlation between bullying and either the STAI-T and GELOPH &lt; 15 &gt; scores. However, the great majority of subjects with gelotophobia reported been previously bullied.

CONCLUSION

In our sample, all gelotophobes had trait anxiety, but only a fraction of anxious subjects had gelotophobia. These preliminary findings expand on previous reports underscoring the high prevalence of mental health problems afflicting higher education students in Brazil.

Impact of Visual Game-Like Features on Cognitive Performance in a Virtual Reality Working Memory Task: Within-Subjects Experiment

BACKGROUND

Although the pursuit of improved cognitive function through working memory training has been the subject of decades of research, the recent growth in commercial adaptations of classic working memory tasks in the form of gamified apps warrants additional scrutiny. In particular, the emergence of virtual reality as a platform for cognitive training presents opportunities for the use of novel visual features.

OBJECTIVE

This study aimed to add to the body of knowledge regarding the use of game-like visual design elements by specifically examining the application of two particular visual features common to virtual reality environments: immersive, colorful backgrounds and the use of 3D depth. In addition, electroencephalography (EEG) data were collected to identify potential neural correlates of any observed changes in performance.

METHODS

A simple visual working memory task was presented to participants in several game-like adaptations, including the use of colorful, immersive backgrounds and 3D depth. The impact of each adaptation was separately assessed using both EEG and performance assessment outcomes and compared with an unmodified version of the task.

RESULTS

Results suggest that although accuracy and reaction time may be slightly affected by the introduction of such game elements, the effects were small and not statistically significant. Changes in EEG power, particularly in the beta and theta rhythms, were significant but failed to correlate with any corresponding changes in performance. Therefore, they may only reflect cognitive changes at the perceptual level.

CONCLUSIONS

Overall, the data suggest that the addition of these specific visual features to simple cognitive tasks does not appear to significantly affect performance or task-dependent cognitive load.

Teacher Training Effectiveness in Self-Regulation in Virtual Environments

Higher education in the 21st century faces the challenge of changing the way in which knowledge is conveyed and how teachers and students interact in the teaching-learning process. The current pandemic caused by SARS-CoV-2 has hastened the need to face up to this challenge and has furthered the need to approach the issue from the perspective of digitalisation. To achieve this, it is necessary to design training programmes geared towards teaching staff and which address both the use of technology and instructional design aimed at promoting the development of self-regulated learning (SRL) and automatic feedback systems. In this study, work was carried out with 23 teachers (8 inexperienced and 15 experienced teachers) in a training programme conducted through Moodle. The aims were: (1) to test whether there were any significant differences between the behaviour patterns of new teachers compared to experienced teachers, (2) to determine whether clusters of behaviour patterns corresponded to the type of teacher and (3) to ascertain whether the level of teacher satisfaction with the training activity in digital teaching will depend on the type of teacher. A quantitative as well as a qualitative design was applied. Differences were found in the behaviour patterns in the training activities for the development of rubrics and use of learning analytics systems in virtual learning environments. It was also found that the type of teacher did not correspond exactly to the behaviour cluster in the learning platform. In addition, no significant differences were found in the level of satisfaction between the two kinds of teacher. The main contribution this study makes is to provide a detailed description of the training stage as well as the materials required for its repetition. Further analytical studies are required on teacher perception of training programmes in digital teaching in order to provide personalised training proposals that lead to an effective use of teaching in digital environments.

Web-Based Cognitive Testing in Psychiatric Research: Validation and Usability Study

Background

Cognitive impairments are features of many psychiatric disorders and affect functioning. A barrier to cognitive research on psychiatric disorders is the lack of large cross-disorder data sets. However, the collection of cognitive data can be logistically challenging and expensive. Web-based collection may be an alternative; however, little is known about who does and does not complete web-based cognitive assessments for psychiatric research.

Objective

The aims of this study are to develop a web-based cognitive battery for use in psychiatric research, validate the battery against the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) Consensus Cognitive Battery, and compare the characteristics of the participants who chose to take part with those of the individuals who did not participate.

Methods

Tasks were developed by The Many Brains Project and selected to measure the domains specified by the MATRICS initiative. We undertook a cross-validation study of 65 participants with schizophrenia, bipolar disorder, depression, or no history of psychiatric disorders to compare the web-based tasks with the MATRICS Consensus Cognitive Battery. Following validation, we invited participants from 2 large ongoing genetic studies, which recruited participants with psychiatric disorders to complete the battery and evaluated the demographic and clinical characteristics of those who took part.

Results

Correlations between web-based and MATRICS tasks ranged between 0.26 and 0.73. Of the 961 participants, 887 (92.3%) completed at least one web-based task, and 644 (67%) completed all tasks, indicating adequate completion rates. Predictors of web-based participation included being female (odds ratio [OR] 1.3, 95% CI 1.07-1.58), ethnicity other than White European (OR 0.66, 95% CI 0.46-0.96), higher levels of education (OR 1.19, 95% CI 1.11-1.29), diagnosis of an eating disorder (OR 2.17, 95% CI 1.17-4) or depression and anxiety (OR 5.12, 95% CI 3.38-7.83), and absence of a diagnosis of schizophrenia (OR 0.59, 95% CI 0.35-0.94). Lower performance on the battery was associated with poorer functioning (B=−1.76, SE 0.26; P<.001).

Conclusions

Our findings offer valuable insights into the advantages and disadvantages of testing cognitive function remotely for mental health research.

Conceptual Ambiguity Surrounding Gamification and Serious Games in Health Care: Literature Review and Development of Game-Based Intervention Reporting Guidelines (GAMING)

BACKGROUND

In health care, the use of game-based interventions to increase motivation, engagement, and overall sustainability of health behaviors is steadily becoming more common. The most prevalent types of game-based interventions in health care research are gamification and serious games. Various researchers have discussed substantial conceptual differences between these 2 concepts, supported by empirical studies showing differences in the effects on specific health behaviors. However, researchers also frequently report cases in which terms related to these 2 concepts are used ambiguously or even interchangeably. It remains unclear to what extent existing health care research explicitly distinguishes between gamification and serious games and whether it draws on existing conceptual considerations to do so.

OBJECTIVE

This study aims to address this lack of knowledge by capturing the current state of conceptualizations of gamification and serious games in health care research. Furthermore, we aim to provide tools for researchers to disambiguate the reporting of game-based interventions.

METHODS

We used a 2-step research approach. First, we conducted a systematic literature review of 206 studies, published in the Journal of Medical Internet Research and its sister journals, containing terms related to gamification, serious games, or both. We analyzed their conceptualizations of gamification and serious games, as well as the distinctions between the two concepts. Second, based on the literature review findings, we developed a set of guidelines for researchers reporting on game-based interventions and evaluated them with a group of 9 experts from the field.

RESULTS

Our results show that less than half of the concept mentions are accompanied by an explicit definition. To distinguish between the 2 concepts, we identified four common approaches: implicit distinction, synonymous use of terms, serious games as a type of gamified system, and distinction based on the full game dimension. Our Game-Based Intervention Reporting Guidelines (GAMING) consist of 25 items grouped into four topics: conceptual focus, contribution, mindfulness about related concepts, and individual concept definitions.

CONCLUSIONS

Conceptualizations of gamification and serious games in health care literature are strongly heterogeneous, leading to conceptual ambiguity. Following the GAMING can support authors in rigorous reporting on study results of game-based interventions.

The Making and Evaluation of Digital Games Used for the Assessment of Attention: Systematic Review

Background

Serious games are now widely used in many contexts, including psychological research and clinical use. One area of growing interest is that of cognitive assessment, which seeks to measure different cognitive functions such as memory, attention, and perception. Measuring these functions at both the population and individual levels can inform research and indicate health issues. Attention is an important function to assess, as an accurate measure of attention can help diagnose many common disorders, such as attention-deficit/hyperactivity disorder and dementia. However, using games to assess attention poses unique problems, as games inherently manipulate attention through elements such as sound effects, graphics, and rewards, and research on adding game elements to assessments (ie, gamification) has shown mixed results. The process for developing cognitive tasks is robust, with high psychometric standards that must be met before these tasks are used for assessment. Although games offer more diverse approaches for assessment, there is no standard for how they should be developed or evaluated.

Objective

To better understand the field and provide guidance to interdisciplinary researchers, we aim to answer the question: How are digital games used for the cognitive assessment of attention made and measured?

Methods

We searched several databases for papers that described a digital game used to assess attention that could be deployed remotely without specialized hardware. We used Rayyan, a systematic review software, to screen the records before conducting a systematic review.

Results

The initial database search returned 49,365 papers. Our screening process resulted in a total of 74 papers that used a digital game to measure cognitive functions related to attention. Across the studies in our review, we found three approaches to making assessment games: gamifying cognitive tasks, creating custom games based on theories of cognition, and exploring potential assessment properties of commercial games. With regard to measuring the assessment properties of these games (eg, how accurately they assess attention), we found three approaches: comparison to a traditional cognitive task, comparison to a clinical diagnosis, and comparison to knowledge of cognition; however, most studies in our review did not evaluate the game’s properties (eg, if participants enjoyed the game).

Conclusions

Our review provides an overview of how games used for the assessment of attention are developed and evaluated. We further identified three barriers to advancing the field: reliance on assumptions, lack of evaluation, and lack of integration and standardization. We then recommend the best practices to address these barriers. Our review can act as a resource to help guide the field toward more standardized approaches and rigorous evaluation required for the widespread adoption of assessment games.

Turn on, Tune in, and Drop out: Predictors of Attrition in a Prospective Observational Cohort Study on Psychedelic Use

Background

The resurgence of research and public interest in the positive psychological effects of psychedelics, together with advancements in digital data collection techniques, have brought forth a new type of research design, which involves prospectively gathering large-scale naturalistic data from psychedelic users; that is, before and after the use of a psychedelic compound. A methodological limitation of such studies is their high attrition rate, particularly owing to participants who stop responding after initial study enrollment. Importantly, study dropout can introduce systematic biases that may affect the interpretability of results.

Objective

Based on a previously collected sample (baseline n=654), here we investigated potential determinants of study attrition in web-based prospective studies on psychedelic use.

Methods

Logistic regression models were used to examine demographic, psychological trait and state, and psychedelic-specific predictors of dropout. Predictors were assessed 1 week before, 1 day after, and 2 weeks after psychedelic use, with attrition being defined as noncompletion of the key endpoint 4 weeks post experience.

Results

Predictors of attrition were found among demographic variables including age (β=0.024; P=.007) and educational levels, as well as personality traits, specifically conscientiousness (β=–0.079; P=.02) and extraversion (β=0.082; P=.01). Contrary to prior hypotheses, neither baseline attitudes toward psychedelics nor the intensity of acute challenging experiences were predictive of dropout.

Conclusions

The baseline predictors of attrition identified here are consistent with those reported in longitudinal studies in other scientific disciplines, suggesting their transdisciplinary relevance. Moreover, the lack of an association between attrition and psychedelic advocacy or negative drug experiences in our sample contextualizes concerns about problematic biases in these and related data.

A Gamification Framework for Cognitive Assessment and Cognitive Training: Qualitative Study

Background

Cognitive tasks designed to measure or train cognition are often repetitive and presented in a monotonous manner, features that lead to participant boredom and disengagement. In this situation, participants do not put forth their best effort to do these tasks well. As a result, neuropsychologists cannot draw accurate conclusions about the data collected, and intervention effects are reduced. It is assumed that greater engagement and motivation will manifest as improved data quality. Gamification, the use of game elements in nongame settings, has been heralded as a potential mechanism for increasing participant engagement in cognitive tasks. Some studies have reported a positive effect of gamification on participant performance, although most studies have shown mixed results. One reason for these contrasting findings is that most studies have applied poor and heterogeneous design techniques to gamify cognitive tasks. Therefore, an appropriate gamification design framework is needed in these tasks.

Objective

This study aimed to propose a framework to guide the design of gamification in cognitive tasks.

Methods

We employed a design science research (DSR) approach to provide a framework for gamifying cognitive assessments and training by synthesizing current gamification design frameworks and gamification works in cognitive assessment and training, as well as incorporating field experiences. The prototypes of the framework were iteratively evaluated with 17 relevant experts.

Results

We proposed a framework consisting of 7 phases: (1) preparation; (2) knowing users; (3) exploring existing tools for assessing or training a targeted cognitive context and determining the suitability of game-up and mapping techniques; (4) ideation; (5) prototyping using the Objects, Mechanics, Dynamics, Emotions (OMDE) design guideline; (6) development; and (7) disseminating and monitoring.

Conclusions

We found that (1) an intermediate design framework is needed to gamify cognitive tasks, which means that game elements should be selected by considering current cognitive assessment or training context characteristics since game elements may impose an irrelevant cognitive load that, in turn, can jeopardize data quality; (2) in addition to developing a new gamified cognitive task from scratch, 2 gamification techniques are widely used (first, adding game elements to an existing cognitive task and second, mapping an existing game to a cognitive function or impairment to assess or train it); and (3) further research is required to investigate the interplay of cognitive processes and game mechanics.

Preserved Inhibitory Control Deficits of Overweight Participants in a Gamified Stop-Signal Task: Experimental Study of Validity

Background

Gamification in mental health could increase training adherence, motivation, and transfer effects, but the external validity of gamified tasks is unclear. This study documents that gamified task variants can show preserved associations between markers of behavioral deficits and health-related variables. We draw on the inhibitory control deficit in overweight populations to investigate effects of gamification on performance measures in a web-based experimental task.

Objective

This study tested whether associations between inhibitory control and overweight were preserved in a gamified stop-signal task (SST).

Methods

Two versions of an adaptive SST were developed and tested in an online experiment. Participants (n=111) were randomized to 1 of the 2 task variants and completed a series of questionnaires along with either the gamified SST or a conventional SST. To maximize its possible effects on participants’ inhibitory control, the gamified SST included multiple game elements in addition to the task itself and the stimuli. Both variants drew on the identical core mechanics, but the gamified variant included an additional narrative, graphical theme, scoring system with visual and emotional feedback, and the presence of a companion character. In both tasks, food and neutral low-poly stimuli were classified based on their color tone (go trials), but responses were withheld in 25% of the trials (stop trials). Mean go reaction times and stop-signal reaction times (SSRT) were analyzed as measures of performance and inhibitory control.

Results

Participants in the gamified SST had longer reaction times (803 [SD 179] ms vs 607 [SD 90] ms) and worse inhibitory control (SSRT 383 [SD 109] ms vs 297 [SD 45] ms). The association of BMI with inhibitory control was relatively small (r=.155, 95% CI .013-.290). Overweight participants had longer reaction times (752 [SD 217] ms vs 672 [SD 137] ms) and SSRTs (363 [SD 116] ms vs 326 [SD 77] ms). Gamification did not interact with the effect of overweight on mean performance or inhibitory control. There were no effects of gamification on mood and user experience, despite a negative effect on perceived efficiency.

Conclusions

The detrimental effects of heightened BMI on inhibitory control were preserved in a gamified version of the SST. Overall, the effects of overweight were smaller than in previously published web-based and laboratory studies. Gamification elements can impact behavioral performance, but gamified tasks can still assess inhibitory control deficits. Although our results are promising, according validations may differ for other types of behavior, gamification, and health variables.

Effective Gamification of the Stop-Signal Task: Two Controlled Laboratory Experiments

Background

A lack of ability to inhibit prepotent responses, or more generally a lack of impulse control, is associated with several disorders such as attention-deficit/hyperactivity disorder and schizophrenia as well as general damage to the prefrontal cortex. A stop-signal task (SST) is a reliable and established measure of response inhibition. However, using the SST as an objective assessment in diagnostic or research-focused settings places significant stress on participants as the task itself requires concentration and cognitive effort and is not particularly engaging. This can lead to decreased motivation to follow task instructions and poor data quality, which can affect assessment efficacy and might increase drop-out rates. Gamification—the application of game-based elements in nongame settings—has shown to improve engaged attention to a cognitive task, thus increasing participant motivation and data quality.

Objective

This study aims to design a gamified SST that improves participants’ engagement and validate this gamified SST against a standard SST.

Methods

We described the design of our gamified SST and reported on 2 separate studies that aim to validate the gamified SST relative to a standard SST. In study 1, a within-subject design was used to compare the performance of the SST and a stop-signal game (SSG). In study 2, we added eye tracking to the procedure to determine if overt attention was affected and aimed to replicate the findings from study 1 in a between-subjects design. Furthermore, in both studies, flow and motivational experiences were measured.

Results

In contrast, the behavioral performance was comparable between the tasks (P<.87; BF01=2.87), and the experience of flow and intrinsic motivation were rated higher in the SSG group, although this difference was not significant.

Conclusions

Overall, our findings provide evidence that the gamification of SST is possible and that the SSG is enjoyed more. Thus, when participant engagement is critical, we recommend using the SSG instead of the SST.

Gorilla in our midst: An online behavioral experiment builder

Behavioral researchers are increasingly conducting their studies online, to gain access to large and diverse samples that would be difficult to get in a laboratory environment. However, there are technical access barriers to building experiments online, and web browsers can present problems for consistent timing-an important issue with reaction-time-sensitive measures. For example, to ensure accuracy and test-retest reliability in presentation and response recording, experimenters need a working knowledge of programming languages such as JavaScript. We review some of the previous and current tools for online behavioral research, as well as how well they address the issues of usability and timing. We then present the Gorilla Experiment Builder (gorilla.sc), a fully tooled experiment authoring and deployment platform, designed to resolve many timing issues and make reliable online experimentation open and accessible to a wider range of technical abilities. To demonstrate the platform's aptitude for accessible, reliable, and scalable research, we administered a task with a range of participant groups (primary school children and adults), settings (without supervision, at home, and under supervision, in both schools and public engagement events), equipment (participant's own computer, computer supplied by the researcher), and connection types (personal internet connection, mobile phone 3G/4G). We used a simplified flanker task taken from the attentional network task (Rueda, Posner, & Rothbart, 2004). We replicated the "conflict network" effect in all these populations, demonstrating the platform's capability to run reaction-time-sensitive experiments. Unresolved limitations of running experiments online are then discussed, along with potential solutions and some future features of the platform.

A Mobile App Delivering a Gamified Battery of Cognitive Tests Designed for Repeated Play (OU Brainwave): App Design and Cohort Study

BACKGROUND

Mobile phone and tablet apps are an increasingly common platform for collecting data. A key challenge for researchers has been participant "buy-in" and attrition for designs requiring repeated testing.

OBJECTIVE

The objective of this study was to develop and assess the utility of 1-2 minute versions of both classic and novel cognitive tasks using a user-focused and user-driven mobile phone and tablet app designed to encourage repeated play.

METHODS

A large sample of app users (N=13,979 at first data collection) participated in multiple, self-paced sessions of classic working memory (N-back), spatial cognition (mental rotation), sustained attentional focus (persistent vigilance task), and split attention (multiple object tracking) tasks, along with the implementation of a comparatively novel action-learning task. The "OU Brainwave" app was designed to measure time-of-day variation in cognitive performance and did not offer any training program or promise any cognitive enhancement. To record participants' chronotype, a full Morningness-Eveningness questionnaire was also included, which measures whether a person's circadian rhythm produces peak alertness in the morning, in the evening, or in between. Data were collected during an 18-month period. While the app prompted re-engagement at set intervals, participants were free to complete each task as many times as they wished.

RESULTS

We found a significant relationship between morningness and age (r=.298, n=12,755, P<.001), with no effect of gender (t_13,539=-1.036, P=.30). We report good task adherence, with ~4000 participants repeatedly playing each game >4 times each-our minimum engagement level for analysis. Repeated plays of these games allowed us to replicate commonly reported gender effects in gamified spatial cognition (F_1,4216=154.861, P<.001, η²_ρ=.035), split attention (F_1,4185=11.047, P=.001, η²_ρ=.003), and sustained attentional focus (F_1,4238=15.993, P<.001, η²_ρ=.004) tasks. We also report evidence of a small gender effect in an action-learning task (F_1,3988=90.59, P<.001, η²_ρ=.022). Finally, we found a strong negative effect of self-reported age on performance, when controlling for number of plays, in sustained attentional focus (n=1596, F_6,1595=30.23, P<.001, η²=.102), working memory (n=1627, F_6,1626=19.78, P<.001, η²=.068), spatial cognition (n=1640, F_6,1639=23.74, P<.001, η²=.080), and split attention tasks (n=1616, F_6,1615=2.48, P=.02, η²=.009).

CONCLUSIONS

Using extremely short testing periods and permitting participants to decide their level of engagement-both in terms of which gamified task they played and how many sessions they completed-we were able to collect a substantial and valid dataset. We suggest that the success of OU Brainwave should inform future research oriented apps-particularly in issues of balancing participant engagement with data fidelity.

If we build it, will they come? Issues of engagement with digital health interventions for trauma recovery

Exposure to traumatic events is extremely common with nearly 75% reported to have experienced one or more traumatic events worldwide. A significant number of those exposed will develop posttraumatic stress disorder (PTSD) along with depression, anxiety, and substance use disorders. Globally, trauma-related mental health disorders are the leading cause of global disability burden, and many of these disorders are caused, or worsened, by exposure to wars, natural and human-caused disasters, and other traumatic events. Significant barriers to treatment exist including logistical, geographical, financial, stigma, and other attitudinal challenges. One opportune approach to overcoming these barriers is the provision of mental health interventions via technology that can be readily standardized for wide dissemination of evidence-based care. However, engagement with technology-based interventions is a concern and limited participation and high attrition rates are common. This may be especially true for trauma survivors who often experience symptoms of avoidance and hyperarousal. Engagement is regarded as an essential component of intervention efficacy and has been demonstrated to be associated with more positive clinical outcomes, yet theoretically based research in this area is sparse. This review focuses on the complex issue of engagement with digital health interventions (DHIs). Specifically, we review the conceptualization and measurement of engagement, predictors of engagement, and importantly, the relationship of engagement with intervention effectiveness. Finally, a theoretically based model of engagement is proposed that considers the unique challenges of trauma recovery. This review is not intended to provide a systematic or exhaustive set of recommendations, rather it is intended to highlight the challenges of engagement research including its definition, measurement, and modeling. Future engagement research that includes valid and reliable measures of engagement will enable consistent exploration of engagement predictors that can then inform methods for increasing engagement and, ultimately, intervention effectiveness.