Exploring the effectiveness of commercial and custom-built games for cognitive training

Can we enhance working memory? Bias and effectiveness in cognitive training studies

Meta-analyses have found that working memory (WM) can be improved with cognitive training; however, some authors have suggested that these improvements are mostly driven by biases in the measurement of WM, especially the use of similar tasks for assessment and training. In the present meta-analysis, we investigated whether WM, fluid intelligence, executive functions, and short-term memory can be improved by cognitive training and evaluated the impact of possible sources of bias. We performed a risk of bias assessment of the included studies and took special care in controlling for practice effects. Data from 52 independent comparisons were analyzed, including cognitive training aimed at different cognitive functions. Our results show small improvements in WM after training (SMD = 0.18). Much larger effects were observed when the analysis was restricted to assessment tasks similar to those used for training (SMD = 1.15). Fluid intelligence was not found to improve as a result of training, and improvements in WM were not related to changes in fluid intelligence. Our analyses did however indicate that cognitive training can improve specific executive functions. Contrary to expectations, a set of meta-regressions indicated that characteristics of the training programme, such as dosage and type of training, do not have an impact on the effectiveness of training. The risk of bias assessment revealed some concerns in the randomization process and possible selective reporting among studies. Overall, our results identified various potential sources of bias, with the most significant being the choice of assessment tasks.

Cognitive Enhancement via Network-Targeted Cortico-cortical Associative Brain Stimulation

Fluid intelligence (gf) represents a crucial component of human cognition, as it correlates with academic achievement, successful aging, and longevity. However, it has strong resilience against enhancement interventions, making the identification of gf enhancement approaches a key unmet goal of cognitive neuroscience. Here, we applied a spike-timing-dependent plasticity (STDP)-inducing brain stimulation protocol, named cortico-cortical paired associative stimulation (cc-PAS), to modulate gf in 29 healthy young subjects (13 females-mean ± standard deviation, 25.43 years ± 3.69), based on dual-coil transcranial magnetic stimulation (TMS). Pairs of neuronavigated TMS pulses (10-ms interval) were delivered over two frontoparietal nodes of the gf network, based on individual functional magnetic resonance imaging data and in accordance with cognitive models of information processing across the prefrontal and parietal lobe. cc-PAS enhanced accuracy at gf tasks, with parieto-frontal and fronto-parietal stimulation significantly increasing logical and relational reasoning, respectively. Results suggest the possibility of using SPTD-inducing TMS protocols to causally validate cognitive models by selectively engaging relevant networks and manipulating inter-regional temporal dynamics supporting specific cognitive functions.

Quantifying the Difference between Active and Passive Control Groups in Cognitive Interventions Using two Meta-Analytical Approaches

Despite promising reports of broad cognitive benefit in studies of cognitive training, it has been argued that the reliance of many studies on no-intervention control groups (passive controls) make these reports difficult to interpret because placebo effects cannot be ruled out. Although researchers have recently been trying to incorporate more active controls, in which participants engage in an alternate intervention, previous work has been contentious as to whether this actually yields meaningfully different results. To better understand the influence of passive and active control groups on cognitive interventions, we conducted two meta-analyses to estimate their relative effect sizes. While the first one broadly surveyed the literature by compiling data from 34 meta-analyses, the second one synthesized data from 42 empirical studies that simultaneously employed both types of controls. Both analyses showed no meaningful performance difference between passive and active controls, suggesting that current active control placebo paradigms might not be appropriately designed to reliably capture these non-specific effects or that these effects are minimal in this literature.

Divergent Research Methods Limit Understanding of Working Memory Training

Working memory training has been a hot topic over the last decade. Although studies show benefits in trained and untrained tasks as a function of training, there is an ongoing debate on the efficacy of working memory training. There have been numerous meta-analyses put forth to the field, some finding overall broad transfer effects while others do not. However, discussion of this research typically overlooks specific qualities of the training and transfer tasks. As such, there has been next to no discussion in the literature on what training and transfer tasks features are likely to mediate training outcomes. To address this gap, here, we characterized the broad diversity of features employed in N-back training tasks and outcome measures in published working memory training studies. Extant meta-analyses have not taken into account the diversity of methodology at this level, primarily because there are too few studies using common methods to allow for a robust meta-analysis. We suggest that these limitations preclude strong conclusions from published data. In order to advance research on working memory training, and in particular, N-back training, more studies are needed that systematically compare training features and use common outcome measures to assess transfer effects.

Working memory training revisited: A multi-level meta-analysis of n-back training studies

The efficacy of working memory (WM) training has been a controversial and hotly debated issue during the past years. Despite a large number of training studies and several meta-analyses, the matter has not yet been solved. We conducted a multi-level meta-analysis on the cognitive transfer effects in healthy adults who have been administered WM updating training with n-back tasks, the most common experimental WM training paradigm. Thanks to this methodological approach that has not been employed in previous meta-analyses in this field, we were able to include effect sizes from all relevant tasks used in the original studies. Altogether 203 effect sizes were derived from 33 published, randomized, controlled trials. In contrast to earlier meta-analyses, we separated task-specific transfer (here untrained n-back tasks) from other WM transfer tasks. Two additional cognitive domains of transfer that we analyzed consisted of fluid intelligence (Gf) and cognitive control tasks. A medium-sized transfer effect was observed to untrained n-back tasks. For other WM tasks, Gf, and cognitive control, the effect sizes were of similar size and very small. Moderator analyses showed no effects of age, training dose, training type (single vs. dual), or WM and Gf transfer task contents (verbal vs. visuospatial). We conclude that a substantial part of transfer following WM training with the n-back task is task-specific and discuss the implications of the results to WM training research.

Reevaluating the effectiveness of n-back training on transfer through the Bayesian lens: Support for the null

A recent meta-analysis by Au et al. Psychonomic Bulletin & Review, 22, 366-377, (2015) reviewed the n-back training paradigm for working memory (WM) and evaluated whether (when aggregating across existing studies) there was evidence that gains obtained for training tasks transferred to gains in fluid intelligence (Gf). Their results revealed an overall effect size of g = 0.24 for the effect of n-back training on Gf. We reexamine the data through a Bayesian lens, to evaluate the relative strength of the evidence for the alternative versus null hypotheses, contingent on the type of control condition used. We find that studies using a noncontact (passive) control group strongly favor the alternative hypothesis that training leads to transfer but that studies using active-control groups show modest evidence in favor of the null. We discuss these findings in the context of placebo effects.

Working Memory Training Does Not Improve Performance on Measures of Intelligence or Other Measures of "Far Transfer": Evidence From a Meta-Analytic Review

It has been claimed that working memory training programs produce diverse beneficial effects. This article presents a meta-analysis of working memory training studies (with a pretest-posttest design and a control group) that have examined transfer to other measures (nonverbal ability, verbal ability, word decoding, reading comprehension, or arithmetic; 87 publications with 145 experimental comparisons). Immediately following training there were reliable improvements on measures of intermediate transfer (verbal and visuospatial working memory). For measures of far transfer (nonverbal ability, verbal ability, word decoding, reading comprehension, arithmetic) there was no convincing evidence of any reliable improvements when working memory training was compared with a treated control condition. Furthermore, mediation analyses indicated that across studies, the degree of improvement on working memory measures was not related to the magnitude of far-transfer effects found. Finally, analysis of publication bias shows that there is no evidential value from the studies of working memory training using treated controls. The authors conclude that working memory training programs appear to produce short-term, specific training effects that do not generalize to measures of "real-world" cognitive skills. These results seriously question the practical and theoretical importance of current computerized working memory programs as methods of training working memory skills.

There is no convincing evidence that working memory training is effective: A reply to Au et al. (2014) and Karbach and Verhaeghen (2014)

The possible cognitive benefits of working memory training programs have been the subject of intense interest and controversy. Recently two meta-analyses have claimed that working memory training can be effective in enhancing cognitive skills in adulthood (Au et al. Behavioural Brain Research 228:(1) 107-115, 2014) and stemming cognitive decline in old age (Karbach & Verhaeghen Psychological Science 25:2027-2037, 2014). The current article critically evaluates these claims. We argue that these meta-analyses produce misleading results because of (1) biases in the studies included, (2) a failure to take account of baseline differences when calculating effect sizes, and (3) a failure to emphasize the difference between studies with treated versus untreated control groups. We present new meta-analyses and conclude that there is no convincing evidence that working memory training produces general cognitive benefits.

There is no convincing evidence that working memory training is NOT effective: A reply to Melby-Lervåg and Hulme (2015)

Our recent meta-analysis concluded that training on working memory can improve performance on tests of fluid intelligence (Au et al., Psychon Bull Rev, 22(2), 366-377, 2015). Melby-Lervåg and Hulme (Psychon Bull Rev, doi: 10.3758/s13423-015-0862-z ) challenge this conclusion on the grounds that it did not take into consideration baseline differences on a by-study level and that the effects were primarily driven by purportedly less rigorous studies that did not include active control groups. Their re-analysis shows that accounting for baseline differences produces a statistically significant, but considerably smaller, overall effect size (g = 0.13 vs g = 0.24 in Au et al.), which loses significance after excluding studies without active controls. The present report demonstrates that evidence of impact variation by the active/passive nature of control groups is ambiguous and also reveals important discrepancies between Melby-Lervåg and Hulme's analysis and our original meta-analysis in terms of the coding and organization of data that account for the discrepant effect sizes. We demonstrate that there is in fact no evidence that the type of control group per se moderates the effects of working memory training on measures of fluid intelligence and reaffirm the original conclusions in Au et al., which are robust to multiple methods of calculating effect size, including the one proposed by Melby-Lervåg and Hulme.

High User Control in Game Design Elements Increases Compliance and In-game Performance in a Memory Training Game

Computer games are increasingly being used for training cognitive functions like working memory and attention among the growing population of older adults. While cognitive training games often include elements like difficulty adaptation, rewards, and visual themes to make the games more enjoyable and effective, the effect of different degrees of afforded user control in manipulating these elements has not been systematically studied. To address this issue, two distinct implementations of the three aforementioned game elements were tested among healthy older adults (N = 21, 69.9 ± 6.4 years old) playing a game-like version of the n-back task on a tablet at home for 3 weeks. Two modes were considered, differentiated by the afforded degree of user control of the three elements: user control of difficulty vs. automatic difficulty adaptation, difficulty-dependent rewards vs. automatic feedback messages, and user choice of visual theme vs. no choice. The two modes ("USER-CONTROL" and "AUTO") were compared for frequency of play, duration of play, and in-game performance. Participants were free to play the game whenever and for however long they wished. Participants in USER-CONTROL exhibited significantly higher frequency of playing, total play duration, and in-game performance than participants in AUTO. The results of the present study demonstrate the efficacy of providing user control in the three game elements, while validating a home-based study design in which participants were not bound by any training regimen, and could play the game whenever they wished. The results have implications for designing cognitive training games that elicit higher compliance and better in-game performance, with an emphasis on home-based training.

Transfer after Working Memory Updating Training

During the past decade, working memory training has attracted much interest. However, the training outcomes have varied between studies and methodological problems have hampered the interpretation of results. The current study examined transfer after working memory updating training by employing an extensive battery of pre-post cognitive measures with a focus on near transfer. Thirty-one healthy Finnish young adults were randomized into either a working memory training group or an active control group. The working memory training group practiced with three working memory tasks, while the control group trained with three commercial computer games with a low working memory load. The participants trained thrice a week for five weeks, with one training session lasting about 45 minutes. Compared to the control group, the working memory training group showed strongest transfer to an n-back task, followed by working memory updating, which in turn was followed by active working memory capacity. Our results support the view that working memory training produces near transfer effects, and that the degree of transfer depends on the cognitive overlap between the training and transfer measures.

Vigilance on the move: video game-based measurement of sustained attention

Vigilance represents the capacity to sustain attention to any environmental source of information over prolonged periods on watch. Most stimuli used in vigilance research over the previous six decades have been relatively simple and often purport to represent important aspects of detection and discrimination tasks in real-world settings. Such displays are most frequently composed of single stimulus presentations in discrete trials against a uniform, often uncluttered background. The present experiment establishes a dynamic, first-person perspective vigilance task in motion using a video-game environment. 'Vigilance on the move' is thus a new paradigm for the study of sustained attention. We conclude that the stress of vigilance extends to the new paradigm, but whether the performance decrement emerges depends upon specific task parameters. The development of the task, the issues to be resolved and the pattern of performance, perceived workload and stress associated with performing such dynamic vigilance are reported.

PRACTITIONER SUMMARY

The present experiment establishes a dynamic, first-person perspective movement-based vigilance task using a video-game environment. 'Vigilance on the move' is thus a new paradigm for the evaluation of sustained attention in operational environments in which individuals move as they monitor their environment. Issues addressed in task development are described.