Tracing the trajectory of skill learning with a very large sample of online game players

Using games to understand the mind

Board, card or video games have been played by virtually every individual in the world. Games are popular because they are intuitive and fun. These distinctive qualities of games also make them ideal for studying the mind. By being intuitive, games provide a unique vantage point for understanding the inductive biases that support behaviour in more complex, ecological settings than traditional laboratory experiments. By being fun, games allow researchers to study new questions in cognition such as the meaning of 'play' and intrinsic motivation, while also supporting more extensive and diverse data collection by attracting many more participants. We describe the advantages and drawbacks of using games relative to standard laboratory-based experiments and lay out a set of recommendations on how to gain the most from using games to study cognition. We hope this Perspective will lead to a wider use of games as experimental paradigms, elevating the ecological validity, scale and robustness of research on the mind.

Large-scale citizen science reveals predictors of sensorimotor adaptation

Sensorimotor adaptation is essential for keeping our movements well calibrated in response to changes in the body and environment. For over a century, researchers have studied sensorimotor adaptation in laboratory settings that typically involve small sample sizes. While this approach has proved useful for characterizing different learning processes, laboratory studies are not well suited for exploring the myriad of factors that may modulate human performance. Here, using a citizen science website, we collected over 2,000 sessions of data on a visuomotor rotation task. This unique dataset has allowed us to replicate, reconcile and challenge classic findings in the learning and memory literature, as well as discover unappreciated demographic constraints associated with implicit and explicit processes that support sensorimotor adaptation. More generally, this study exemplifies how a large-scale exploratory approach can complement traditional hypothesis-driven laboratory research in advancing sensorimotor neuroscience.

A theory of the skill-performance relationship

The skill-performance relationship is a cornerstone of a meritocratic society. People are selected for schools, colleges and jobs based on the premise that more skillful individuals perform better. Scientific understanding of the skill-performance relationship demands that the effect of skill on performance is objectively assessed without subjective, social, and political considerations. One of the best areas for this analysis is sports. In many sports settings, the skill-performance relationship can objectively be examined at the technical, behavioral, psychological, and neurological levels. This examination reveals that skill and performance are inextricably intertwined. While skill affects performance, performance in turn defines and affects skill. To disentangle the previously confusing and interchangeable use of these key constructs, the paper presents a theoretical model specifying that ability and effort have their own direct effects on performance, as well as indirect effects on performance through skill possession and skill execution in cognitive and physical domains of human performance. Thus, ability and skill are not the same. Although skill is a key determinant of performance, recent theory and research suggests that successful performers are successful not just because of their skills per se, but because they take advantage of their skills by creating more occurrences of momentum, making them last longer, and using them to bounce back faster from streaks of unsuccessful performance. Thus, momentum is an important mediator of the effects of skill on performance.

Play in video games

Video game play is remarkably ubiquitous in today's society given its recent emergence only in the late 1950s. While this fast evolution could exemplify the power of play, video games exploit but also extend other types of play. Here, we review a classification of the ecosystem of video games useful in the emerging field of the cognitive neuroscience of video games. We then discuss how video games may leverage different play types, considering first locomotor-rotational, object, and social play before highlighting the importance of role, rule, and pretend play in video games. With an eye toward comparative studies of the neural bases of play across species, we discuss whether video games may fulfil the five criteria from Burghardt (2005) to identify play. Finally, in line with play's possible preparatory role for adulthood, we review the positive impact on cognition and future learning of action-like video games. Highlighting that not all video games have this impact, we note more granular hypotheses about the biological functions of play are to be encouraged.

Kinematic markers of skill in first-person shooter video games

Video games present a unique opportunity to study motor skill. First-person shooter (FPS) games have particular utility because they require visually guided hand movements that are similar to widely studied planar reaching tasks. However, there is a need to ensure the tasks are equivalent if FPS games are to yield their potential as a powerful scientific tool for investigating sensorimotor control. Specifically, research is needed to ensure that differences in visual feedback of a movement do not affect motor learning between the two contexts. In traditional tasks, a movement will translate a cursor across a static background, whereas FPS games use movements to pan and tilt the view of the environment. To this end, we designed an online experiment where participants used their mouse or trackpad to shoot targets in both visual contexts. Kinematic analysis showed player movements were nearly identical between contexts, with highly correlated spatial and temporal metrics. This similarity suggests a shared internal model based on comparing predicted and observed displacement vectors rather than primary sensory feedback. A second experiment, modeled on FPS-style aim-trainer games, found movements exhibited classic invariant features described within the sensorimotor literature. We found the spatial metrics tested were significant predictors of overall task performance. More broadly, these results show that FPS games offer a novel, engaging, and compelling environment to study sensorimotor skill, providing the same precise kinematic metrics as traditional planar reaching tasks.

Initial motor skill performance predicts future performance, but not learning

People show vast variability in skill performance and learning. What determines a person's individual performance and learning ability? In this study we explored the possibility to predict participants' future performance and learning, based on their behavior during initial skill acquisition. We recruited a large online multi-session sample of participants performing a sequential tapping skill learning task. We used machine learning to predict future performance and learning from raw data acquired during initial skill acquisition, and from engineered features calculated from the raw data. Strong correlations were observed between initial and final performance, and individual learning was not predicted. While canonical experimental tasks developed and selected to detect average effects may constrain insights regarding individual variability, development of novel tasks may shed light on the underlying mechanism of individual skill learning, relevant for real-life scenarios.

Conversation dynamics in a multiplayer video game with knowledge asymmetry

Despite the challenges associated with virtually mediated communication, remote collaboration is a defining characteristic of online multiplayer gaming communities. Inspired by the teamwork exhibited by players in first-person shooter games, this study investigated the verbal and behavioral coordination of four-player teams playing a cooperative online video game. The game, Desert Herding, involved teams consisting of three ground players and one drone operator tasked to locate, corral, and contain evasive robot agents scattered across a large desert environment. Ground players could move throughout the environment, while the drone operator’s role was akin to that of a “spectator” with a bird’s-eye view, with access to veridical information of the locations of teammates and the to-be-corralled agents. Categorical recurrence quantification analysis (catRQA) was used to measure the communication dynamics of teams as they completed the task. Demands on coordination were manipulated by varying the ground players’ ability to observe the environment with the use of game “fog.” Results show that catRQA was sensitive to changes to task visibility, with reductions in task visibility reorganizing how participants conversed during the game to maintain team situation awareness. The results are discussed in the context of future work that can address how team coordination can be augmented with the inclusion of artificial agents, as synthetic teammates.

Mind the gap: Distributed practice enhances performance in a MOBA game

Understanding how humans master complex skills has the potential for wide-reaching societal benefit. Research has shown that one important aspect of effective skill learning is the temporal distribution of practice episodes (i.e., distributed practice). Using a large observational sample of players (n = 162,417) drawn from a competitive and popular online game (League of Legends), we analysed the relationship between practice distribution and performance through time. We compared groups of players who exhibited different play schedules using data slicing and machine learning techniques, to show that players who cluster gameplay into shorter time frames ultimately achieve lower performance levels than those who space their games across longer time windows. Additionally, we found that the timing of intensive play periods does not affect final performance-it is the overall amount of spacing that matters. These results extend some of the key findings in the literature on practice and learning to an ecologically valid environment with huge n. We discuss our work in relation to recent studies that have examined practice effects using Big Data and suggest solutions for salient confounds.

Maximizing the Potential of Digital Games for Understanding Skill Acquisition

Gaming is a domain of profound skill development. Players’ digital traces create data that track the development of skill from novice to expert levels. We argue that existing work, although promising, has yet to take advantage of the potential of game data for understanding skill acquisition, and that to realize this potential, future studies can use the fit of formal learning curves to individual data as a theoretical anchor. Learning-curve analysis allows learning rate, initial performance, and asymptotic performance to be separated out, and so can serve as a tool for reconciling the multiple factors that may affect learning. We review existing research on skill development using data from digital games, showing how such work can confirm, challenge, and extend existing claims about the psychology of expertise. Learning-curve analysis provides the foundation for direct experiments on the factors that affect skill development, which are necessary for a cross-domain cognitive theory of skill. We conclude by making recommendations for, and noting obstacles to, experimental studies of skill development in digital games.

Long-Term Motor Learning in the "Wild" With High Volume Video Game Data

Motor learning occurs over long periods of practice during which motor acuity, the ability to execute actions more accurately, precisely, and in less time, improves. Laboratory-based studies of motor learning are typically limited to a small number of participants and a time frame of minutes to several hours per participant. There is a need to assess the generalizability of theories and findings from lab-based motor learning studies on larger samples and time scales. In addition, laboratory-based studies of motor learning use relatively simple motor tasks which participants are unlikely to be intrinsically motivated to learn, limiting the interpretation of their findings in more ecologically valid settings ("in the wild"). We studied the acquisition and longitudinal refinement of a complex sensorimotor skill embodied in a first-person shooter video game scenario, with a large sample size (N = 7174, 682,564 repeats of the 60 s game) over a period of months. Participants voluntarily practiced the gaming scenario for up to several hours per day up to 100 days. We found improvement in performance accuracy (quantified as hit rate) was modest over time but motor acuity (quantified as hits per second) improved considerably, with 40-60% retention from 1 day to the next. We observed steady improvements in motor acuity across multiple days of video game practice, unlike most motor learning tasks studied in the lab that hit a performance ceiling rather quickly. Learning rate was a non-linear function of baseline performance level, amount of daily practice, and to a lesser extent, number of days between practice sessions. In addition, we found that the benefit of additional practice on any given day was non-monotonic; the greatest improvements in motor acuity were evident with about an hour of practice and 90% of the learning benefit was achieved by practicing 30 min per day. Taken together, these results provide a proof-of-concept in studying motor skill acquisition outside the confines of the traditional laboratory, in the presence of unmeasured confounds, and provide new insights into how a complex motor skill is acquired in an ecologically valid setting and refined across much longer time scales than typically explored.

Effects of practice on a mechanical horse with an online feedback on performing a sitting postural coordination

The present research aims at quantifying the impact of practicing a new coordination pattern with an online visual feedback on the postural coordination performed on a mechanical horse. Forty-four voluntary participants were recruited in this study. They were randomly assigned to four practice groups based on i) with or without feedback (i.e., group 1, control, did not receive the feedback; group 2, 3 and 4 received an online feedback during practice) and ii) the specific trunk/horse coordination to target during practice (group 1, target coordination = 180° (without feedback); group 2, target coordination = 0°; group 3, target coordination = 90°; group 4, target coordination = 180°). All participants performed pre-, practice, post- and retention sessions. The pre-, post- and retention sessions consisted of four trials, with one trial corresponding to one specific target coordination to maintain between their own oscillations and the horse oscillations (spontaneous, 0°, 90°, and 180°). The practice phase was composed of three different sessions during which participants received an online feedback about the coordination between their own oscillations and the horse oscillations. Results showed a significant change with practice in the trunk/horse coordination patterns which persisted even after one month (retention-test). However, all the groups did not show the same nature of change, evidenced by a high postural variability during post-test for 0° and 90° target coordination groups, in opposition to the 180° and spontaneous groups who showed a decrease in coordination variability for the 180° group. The coordination in anti-phase was characterized as spontaneously adopted by participants on the mechanical horse, explaining the ease of performing this coordination (compared to the 0° and 90° target coordination). The effect of online visual feedback appeared not only on the coordination pattern itself, but most importantly on its variability during practice, including concerning initially stable coordination patterns.

Mapping the outer reaches of the learning curve: Complex intellectual skill performance after decades of extensive practice

Little is known about the outer reaches of learning curves for very complex cognitive skills exercised over decades. Can skill performance improve as long as practice lasts or do all learners ultimately plateau? Furthermore, does natural talent set widely varying performance limits or do all learning curves eventually converge? Chess skill learning curves were examined for 333 players, mostly grandmasters, who, over a median 20 years, played at least 1500 internationally-rated games. Curves of nine of the 333 participants who played more than 3050 games, and one of the 333 and two others who played more than 4250 games over more than 30 years, also were examined. Players on average reached an approximate plateau by around 1200 games. This asymptotic value has changed little from the 1990s despite massive changes in the chess environment. Out to more than 3050 or 4250 games, players stayed at an approximate plateau for many games and years and then performance eventually declined. Chess skill learning curves do not rise forever. Over extensive practice lasting decades, they typically plateau for a long time and eventually decline. The more talented tend to plateau later and curves of the greater and lesser talented do not converge.

Cognitive effects of video games in older adults and their moderators: a systematic review with meta-analysis and meta-regression

The growing interest in preserving cognition through video games in the ageing population is evidenced by its wealth of literature. However, there has been a lack of consensus on video games efficacy and even limited understanding on the factors moderating its effectiveness. The present review sought to evaluate video game effects, relative to no-game control condition, on cognitive functions in older adults. The functions included processing speed, attention, delayed memory, reasoning, and executive functions (inhibition, shifting and updating memory). Personal and methodological moderators were explored to explain the variability in cognitive effects using meta-regression. Through a systematic literature search of online databases, 27 intervention studies were eligible, with a total of 1126 participants, for analysis. Using random-effect models, small but significant training effects were found on updating memory but none were detected in other cognitive functions. The heterogeneity across studies was in general high for all cognitive functions and was partially accounted for, in all cognitive functions except for inhibition and updating, by different moderators such as the proportion of females, outcome measure parameters, training intensity, and game type. The findings highlight limited effects of video games on cognitive functions that were largely influenced by factors related to study design.

Exploring and Exploiting Uncertainty: Statistical Learning Ability Affects How We Learn to Process Language Along Multiple Dimensions of Experience

While the effects of pattern learning on language processing are well known, the way in which pattern learning shapes exploratory behavior has long gone unnoticed. We report on the way in which individual differences in statistical pattern learning affect performance in the domain of language along multiple dimensions. Analyzing data from healthy monolingual adults' performance on a serial reaction time task and a self-paced reading task, we show how individual differences in statistical pattern learning are reflected in readers' knowledge of linguistic co-occurrence patterns and in their exploration and exploitation of content-specific and task-general information. First, we investigated the extent to which an individual's pattern learning correlates with his or her sensitivity to systematic morphological and syntactic co-occurrences, as evidenced while reading authentic sentences. We found that the stream of morphological and syntactic information has a more pronounced effect on the reading speed of, as we will label them, content-sensitive learners in that the more probable the co-occurrence pattern, the faster their reading of that pattern will be. Next, we investigated how differences in pattern learning are reflected in the ways in which individuals approach the reading task itself and adapt to it. Casting this relation in terms of exploration/exploitation strategies, known from Reinforcement Learning, we conclude that content-sensitive learners are also more likely to initially probe (explore) a wider range of directly relevant patterns, which they can later use (exploit) to optimize their reading performance further. By affecting exploratory behavior, pattern learning influences the information that is gathered and becomes available for exploitation, thereby increasing the effect pattern learning has on language cognition.

Self-regulated spacing in a massive open online course is related to better learning

In this study, we examined students' natural studying behaviors in massive, open, online course (MOOC) on introductory psychology. We found that, overall, distributing study across multiple sessions-increasing spacing-was related to increased performance on end-of-unit quizzes, even when comparing the same student across different time-points in the course. Moreover, we found important variation on who is more likely to engage in spaced study and benefit from it. Students with higher ability and students who were more likely to complete course activities were more likely to space their study. Spacing benefits, however, were largest for the lower-ability students and for those students who were less likely to complete activities. These results suggest that spaced study might work as a buffer, improving performance for low ability students and those who do not engage in active practices. This study highlights the positive impact of spacing in real-world learning situations, but more importantly, the role of self-regulated learning decisions in shaping the impact of spaced practice.

Video games as rich environments to foster brain plasticity

This chapter highlights the key role of two main factors, attentional control and reward processing, in unlocking brain plasticity. We first review the evidence for the role that each of these mechanisms plays in neuroplasticity, and then make the case that tools and technologies that combine these two are likely to result in maximal and broad, generalized benefits. In this context, we review the evidence concerning the impact of video game play on brain plasticity, with an eye toward plasticity-driving methods such as the seamless integration of neurofeedback into the video game platforms.

Working memory affects anticipatory behavior during implicit pattern learning

We investigated the relation between implicit sequence learning and individual differences in working memory (WM) capacity. Participants performed an oculomotor version of the serial reaction time (SRT) task and three computerized WM tasks. Implicit learning was measured using anticipation measures only, as they represent strong indicators of learning. Our results demonstrate that anticipatory behavior in the SRT task changes as a function of WM capacity, such that it increases with decreased WM capacity. On the other hand, WM capacity did not affect the overall number of correct anticipations in the task. In addition, we report a positive relation between WM capacity and the number of consecutive correct anticipations (or chunks), and a negative relation between WM capacity and the overall number of errors, indicating different learning strategies during implicit sequence learning. The results of the current study are theoretically important, because they demonstrate that individual differences in WM capacity could account for differences in learning processes, and ultimately change individuals' anticipatory behavior, even when learning is implicit, without intention and awareness.

A large-scale analysis of task switching practice effects across the lifespan

In modern life, we engage with many sources of information concurrently. To do this, we must continuously switch between different tasks, but this comes at a cost to performance, especially in older adults. Using a large dataset from an online cognitive-training platform, we develop a computational model of task switching that defines distinct latent measures of activating the relevant task, deactivating the irrelevant task, and making a decision. This model shows that, although task practice can improve task-switching performance, persistent costs remain even after extensive practice, and more so in older adults. We show that, with extensive task practice, older people can become functionally similar to less-practiced younger people.

An important feature of human cognition is the ability to flexibly and efficiently adapt behavior in response to continuously changing contextual demands. We leverage a large-scale dataset from Lumosity, an online cognitive-training platform, to investigate how cognitive processes involved in cued switching between tasks are affected by level of task practice across the adult lifespan. We develop a computational account of task switching that specifies the temporal dynamics of activating task-relevant representations and inhibiting task-irrelevant representations and how they vary with extended task practice across a number of age groups. Practice modulates the level of activation of the task-relevant representation and improves the rate at which this information becomes available, but has little effect on the task-irrelevant representation. While long-term practice improves performance across all age groups, it has a greater effect on older adults. Indeed, extensive task practice can make older individuals functionally similar to less-practiced younger individuals, especially for cognitive measures that focus on the rate at which task-relevant information becomes available.

Classic motor chunking theory fails to account for behavioural diversity and speed in a complex naturalistic task

In tasks that demand rapid performance, actions must be executed as efficiently as possible. Theories of expert motor performance such as the motor chunking framework suggest that efficiency is supported by automatization, where many serial actions are automatized into smaller chunks, or groups of commonly co-occuring actions. We use the fast-paced, professional eSport StarCraft 2 as a test case of the explanatory power of the motor chunking framework and assess the importance of chunks in explaining expert performance. To do so, we test three predictions motivated by a simple motor chunking framework. (1) StarCraft 2 players should exhibit an increasing number of chunks with expertise. (2) The proportion of actions falling within a chunk should increase with skill. (3) Chunks should be faster than non-chunks containing the same atomic behaviours. Although our findings support the existence of chunks, they also highlight two problems for existing accounts of rapid motor execution and expert performance. First, while better players do use more chunks, the proportion of actions within a chunks is stable across expertise and expert sequences are generally more varied (the diversity problem). Secondly, chunks, which are supposed to enjoy the most extreme automatization, appear to save little or no time overall (the time savings problem). Instead, the most parsimonious description of our latency analysis is that players become faster overall regardless of chunking.

The Impact of Domain-Specific Experience on Chess Skill: Reanalysis of a Key Study

How important are training and other forms of domain-relevant experience in predicting individual differences in expertise? To answer this question, we used structural equation modeling to reanalyze data from a study of chess by Charness, Tuffiash, Krampe, Reingold, and Vasyukova (2005). Latent variables reflecting serious chess activity and formal instruction, along with a manifest variable indexing serious starting age, accounted for 63% of the variance in peak rating. Serious starting age had a significant negative effect on peak rating (β = –.15), even after we controlled for domain-specific experience, indicating an advantage for starting earlier. We also tested the prediction that formal instruction increases the effectiveness of serious study (Ericsson & Charness, 1994) using moderated regression. This claim was not supported. Overall, the results affirm that serious study and other forms of domain-specific experience are important pieces of the expertise puzzle, but other factors must matter too.

Supplemental materials are available at https://www.press.uillinois.edu/journals/ajp/media/chess_skill

Faithfulness-boost effect: Loyal teammate selection correlates with skill acquisition improvement in online games

The problem of skill acquisition is ubiquitous and fundamental to life. Most tasks in modern society involve the cooperation with other subjects. Notwithstanding its fundamental importance, teammate selection is commonly overlooked when studying learning. We exploit the virtually infinite repository of human behavior available in Internet to study a relevant topic in anthropological science: how grouping strategies may affect learning. We analyze the impact of team play strategies in skill acquisition using a turn-based game where players can participate individually or in teams. We unveil a subtle but strong effect in skill acquisition based on the way teams are formed and maintained during time. "Faithfulness-boost effect" provides a skill boost during the first games that would only be acquired after thousands of games. The tendency to play games in teams is associated with a long-run skill improvement while playing loyally with the same teammate significantly accelerates short-run skill acquisition.

A Mobile App Delivering a Gamified Battery of Cognitive Tests Designed for Repeated Play (OU Brainwave): App Design and Cohort Study

BACKGROUND

Mobile phone and tablet apps are an increasingly common platform for collecting data. A key challenge for researchers has been participant "buy-in" and attrition for designs requiring repeated testing.

OBJECTIVE

The objective of this study was to develop and assess the utility of 1-2 minute versions of both classic and novel cognitive tasks using a user-focused and user-driven mobile phone and tablet app designed to encourage repeated play.

METHODS

A large sample of app users (N=13,979 at first data collection) participated in multiple, self-paced sessions of classic working memory (N-back), spatial cognition (mental rotation), sustained attentional focus (persistent vigilance task), and split attention (multiple object tracking) tasks, along with the implementation of a comparatively novel action-learning task. The "OU Brainwave" app was designed to measure time-of-day variation in cognitive performance and did not offer any training program or promise any cognitive enhancement. To record participants' chronotype, a full Morningness-Eveningness questionnaire was also included, which measures whether a person's circadian rhythm produces peak alertness in the morning, in the evening, or in between. Data were collected during an 18-month period. While the app prompted re-engagement at set intervals, participants were free to complete each task as many times as they wished.

RESULTS

We found a significant relationship between morningness and age (r=.298, n=12,755, P<.001), with no effect of gender (t_13,539=-1.036, P=.30). We report good task adherence, with ~4000 participants repeatedly playing each game >4 times each-our minimum engagement level for analysis. Repeated plays of these games allowed us to replicate commonly reported gender effects in gamified spatial cognition (F_1,4216=154.861, P<.001, η²_ρ=.035), split attention (F_1,4185=11.047, P=.001, η²_ρ=.003), and sustained attentional focus (F_1,4238=15.993, P<.001, η²_ρ=.004) tasks. We also report evidence of a small gender effect in an action-learning task (F_1,3988=90.59, P<.001, η²_ρ=.022). Finally, we found a strong negative effect of self-reported age on performance, when controlling for number of plays, in sustained attentional focus (n=1596, F_6,1595=30.23, P<.001, η²=.102), working memory (n=1627, F_6,1626=19.78, P<.001, η²=.068), spatial cognition (n=1640, F_6,1639=23.74, P<.001, η²=.080), and split attention tasks (n=1616, F_6,1615=2.48, P=.02, η²=.009).

CONCLUSIONS

Using extremely short testing periods and permitting participants to decide their level of engagement-both in terms of which gamified task they played and how many sessions they completed-we were able to collect a substantial and valid dataset. We suggest that the success of OU Brainwave should inform future research oriented apps-particularly in issues of balancing participant engagement with data fidelity.

Chess databases as a research vehicle in psychology: Modeling large data

The game of chess has often been used for psychological investigations, particularly in cognitive science. The clear-cut rules and well-defined environment of chess provide a model for investigations of basic cognitive processes, such as perception, memory, and problem solving, while the precise rating system for the measurement of skill has enabled investigations of individual differences and expertise-related effects. In the present study, we focus on another appealing feature of chess—namely, the large archive databases associated with the game. The German national chess database presented in this study represents a fruitful ground for the investigation of multiple longitudinal research questions, since it collects the data of over 130,000 players and spans over 25 years. The German chess database collects the data of all players, including hobby players, and all tournaments played. This results in a rich and complete collection of the skill, age, and activity of the whole population of chess players in Germany. The database therefore complements the commonly used expertise approach in cognitive science by opening up new possibilities for the investigation of multiple factors that underlie expertise and skill acquisition. Since large datasets are not common in psychology, their introduction also raises the question of optimal and efficient statistical analysis. We offer the database for download and illustrate how it can be used by providing concrete examples and a step-by-step tutorial using different statistical analyses on a range of topics, including skill development over the lifetime, birth cohort effects, effects of activity and inactivity on skill, and gender differences.

Evaluating the Theoretic Adequacy and Applied Potential of Computational Models of the Spacing Effect

The spacing effect is among the most widely replicated empirical phenomena in the learning sciences, and its relevance to education and training is readily apparent. Yet successful applications of spacing effect research to education and training is rare. Computational modeling can provide the crucial link between a century of accumulated experimental data on the spacing effect and the emerging interest in using that research to enable adaptive instruction. In this paper, we review relevant literature and identify 10 criteria for rigorously evaluating computational models of the spacing effect. Five relate to evaluating the theoretic adequacy of a model, and five relate to evaluating its application potential. We use these criteria to evaluate a novel computational model of the spacing effect called the Predictive Performance Equation (PPE). Predictive Performance Equation combines elements of earlier models of learning and memory including the General Performance Equation, Adaptive Control of Thought-Rational, and the New Theory of Disuse, giving rise to a novel computational account of the spacing effect that performs favorably across the complete sets of theoretic and applied criteria. We implemented two other previously published computational models of the spacing effect and compare them to PPE using the theoretic and applied criteria as guides.

Predicting Airport Screening Officers' Visual Search Competency With a Rapid Assessment

Objective The study's objective was to assess a new personnel selection and assessment tool for aviation security screeners. A mobile app was modified to create a tool, and the question was whether it could predict professional screeners' on-job performance. Background A variety of professions (airport security, radiology, the military, etc.) rely on visual search performance-being able to detect targets. Given the importance of such professions, it is necessary to maximize performance, and one means to do so is to select individuals who excel at visual search. A critical question is whether it is possible to predict search competency within a professional search environment. Method Professional searchers from the USA Transportation Security Administration (TSA) completed a rapid assessment on a tablet-based X-ray simulator (XRAY Screener, derived from the mobile technology app Airport Scanner; Kedlin Company). The assessment contained 72 trials that were simulated X-ray images of bags. Participants searched for prohibited items and tapped on them with their finger. Results Performance on the assessment significantly related to on-job performance measures for the TSA officers such that those who were better XRAY Screener performers were both more accurate and faster at the actual airport checkpoint. Conclusion XRAY Screener successfully predicted on-job performance for professional aviation security officers. While questions remain about the underlying cognitive mechanisms, this quick assessment was found to significantly predict on-job success for a task that relies on visual search performance. Application It may be possible to quickly assess an individual's visual search competency, which could help organizations select new hires and assess their current workforce.

Female Chess Players Outperform Expectations When Playing Men

Stereotype threat has been offered as a potential explanation of differential performance between men and women in some cognitive domains. Questions remain about the reliability and generality of the phenomenon. Previous studies have found that stereotype threat is activated in female chess players when they are matched against male players. I used data from over 5.5 million games of international tournament chess and found no evidence of a stereotype-threat effect. In fact, female players outperform expectations when playing men. Further analysis showed no influence of degree of challenge, player age, nor prevalence of female role models in national chess leagues on differences in performance when women play men versus when they play women. Though this analysis contradicts one specific mechanism of influence of gender stereotypes, the persistent differences between male and female players suggest that systematic factors do exist and remain to be uncovered.

Digital Traces of Behaviour Within Addiction: Response to Griffiths (2017)

Griffiths' (2017) response to the recent commentary piece by Ryding and Kaye (2017) on "Internet Addiction: A conceptual minefield" provided a useful critique and extension of some key issues. We take this opportunity to further build upon on one of these issues to provide some further insight into how the field of "internet addiction" (IA) or technological addictions more generally, may benefit from capitalising on behavioural data. As such, this response extends Griffiths' (2007) points surrounding the efficacy of behavioural data previously used in studies on problematic gambling, to consider its merit for future research on IA or associated topics such as Internet Gaming Disorder (IGD) or "Smartphone addiction". Within this, we highlight the challenges associated with utilising behavioural data but provide some practical solutions which may support researchers and practitioners in this field. These recent developments could, in turn, advance our understanding and potentially validate such concepts by establishing behavioural correlates, conditions and contexts. Indeed, corroborating behavioural metrics alongside self-report measures presents a key opportunity if scholars and practitioners are to move the field forward.

Exploring the relationship between video game expertise and fluid intelligence

Hundreds of millions of people play intellectually-demanding video games every day. What does individual performance on these games tell us about cognition? Here, we describe two studies that examine the potential link between intelligence and performance in one of the most popular video games genres in the world (Multiplayer Online Battle Arenas: MOBAs). In the first study, we show that performance in the popular MOBA League of Legends’ correlates with fluid intelligence as measured under controlled laboratory conditions. In the second study, we also show that the age profile of performance in the two most widely-played MOBAs (League of Legends and DOTA II) matches that of raw fluid intelligence. We discuss and extend previous videogame literature on intelligence and videogames and suggest that commercial video games can be useful as 'proxy' tests of cognitive performance at a global population level.

Finding the traces of behavioral and cognitive processes in big data and naturally occurring datasets

Today, people generate and store more data than ever before as they interact with both real and virtual environments. These digital traces of behavior and cognition offer cognitive scientists and psychologists an unprecedented opportunity to test theories outside the laboratory. Despite general excitement about big data and naturally occurring datasets among researchers, three "gaps" stand in the way of their wider adoption in theory-driven research: the imagination gap, the skills gap, and the culture gap. We outline an approach to bridging these three gaps while respecting our responsibilities to the public as participants in and consumers of the resulting research. To that end, we introduce Data on the Mind ( http://www.dataonthemind.org ), a community-focused initiative aimed at meeting the unprecedented challenges and opportunities of theory-driven research with big data and naturally occurring datasets. We argue that big data and naturally occurring datasets are most powerfully used to supplement-not supplant-traditional experimental paradigms in order to understand human behavior and cognition, and we highlight emerging ethical issues related to the collection, sharing, and use of these powerful datasets.

Visual Search: You Are Who You Are (+ A Learning Curve)

Not everyone is equally well suited for every endeavor-individuals differ in their strengths and weaknesses, which makes some people better at performing some tasks than others. As such, it might be possible to predict individuals' peak competence (i.e., ultimate level of success) on a given task based on their early performance in that task. The current study leveraged "big data" from the mobile game, Airport Scanner (Kedlin Company), to assess the possibility of predicting individuals' ultimate visual search competency using the minimum possible unit of data: response time on a single visual search trial. Those who started out poorly were likely to stay relatively poor and those who started out strong were likely to remain top performers. This effect was apparent at the level of a single trial (in fact, the first trial), making it possible to use raw response time to predict later levels of success.

Spacing Repetitions Over Long Timescales: A Review and a Reconsolidation Explanation

Recent accounts of the spacing effect have proposed molecular explanations that explain spacing over short, but not long timescales. In the first half of this paper, we review research on the spacing effect that has employed spaces of 24 h or more across skill-related tasks, language-related tasks and generalization for adults and children. Throughout this review, we distinguish between learning and retention by defining learning (or acquisition) as performance at the end of training and retention as performance after a delay period. Using this distinction, we find age- and task-related differences in the manifestation of the spacing effect over long timescales. In the second half of this paper, we discuss a reconsolidation account of the spacing effect. In particular, we review the evidence that suggests the spacing of repetitions influences the subsequent consolidation and reconsolidation processes; we explain how a reconsolidation account may explain the findings for learning; the inverted-U curve for retention; and compare the reconsolidation account with previous consolidation accounts of the spacing effect.

Master Maker: Understanding Gaming Skill Through Practice and Habit From Gameplay Behavior

The study of expertise is difficult to do in a laboratory environment due to the challenge of finding people at different skill levels and the lack of time for participants to acquire mastery. In this paper, we report on two studies that analyze naturalistic gameplay data using cohort analysis to better understand how skill relates to practice and habit. Two cohorts are analyzed, each from two different games (Halo Reach and StarCraft 2). Our work follows skill progression through 7 months of Halo matches for a holistic perspective, but also explores low-level in-game habits when controlling game units in StarCraft 2. Players who played moderately frequently without long breaks were able to gain skill the most efficiently. What set the highest performers apart was their ability to gain skill more rapidly and without dips compared to other players. At the beginning of matches, top players habitually warmed up by selecting and re-selecting groups of units repeatedly in a meaningless cycle. They exhibited unique routines during their play that aided them when under pressure.

A large scale test of the gaming-enhancement hypothesis

A growing research literature suggests that regular electronic game play and game-based training programs may confer practically significant benefits to cognitive functioning. Most evidence supporting this idea, the gaming-enhancement hypothesis, has been collected in small-scale studies of university students and older adults. This research investigated the hypothesis in a general way with a large sample of 1,847 school-aged children. Our aim was to examine the relations between young people’s gaming experiences and an objective test of reasoning performance. Using a Bayesian hypothesis testing approach, evidence for the gaming-enhancement and null hypotheses were compared. Results provided no substantive evidence supporting the idea that having preference for or regularly playing commercially available games was positively associated with reasoning ability. Evidence ranged from equivocal to very strong in support for the null hypothesis over what was predicted. The discussion focuses on the value of Bayesian hypothesis testing for investigating electronic gaming effects, the importance of open science practices, and pre-registered designs to improve the quality of future work.

Improving training for sensory augmentation using the science of expertise

Sensory substitution and augmentation devices (SSADs) allow users to perceive information about their environment that is usually beyond their sensory capabilities. Despite an extensive history, SSADs are arguably not used to their fullest, both as assistive technology for people with sensory impairment or as research tools in the psychology and neuroscience of sensory perception. Studies of the non-use of other assistive technologies suggest one factor is the balance of benefits gained against the costs incurred. We argue that improving the learning experience would improve this balance, suggest three ways in which it can be improved by leveraging existing cognitive science findings on expertise and skill development, and acknowledge limitations and relevant concerns. We encourage the systematic evaluation of learning programs, and suggest that a more effective learning process for SSADs could reduce the barrier to uptake and allow users to reach higher levels of overall capacity.

Brief Online Training Enhances Competitive Performance: Findings of the BBC Lab UK Psychological Skills Intervention Study

In conjunction with BBC Lab UK, the present study developed 12 brief psychological skill interventions for online delivery. A protocol was designed that captured data via self-report measures, used video recordings to deliver interventions, involved a competitive concentration task against an individually matched computer opponent, and provided feedback on the effects of the interventions. Three psychological skills were used; imagery, self-talk, and if-then planning, with each skill directed to one of four different foci: outcome goal, process goal, instruction, or arousal-control. This resulted in 12 different intervention participant groups (randomly assigned) with a 13th group acting as a control. Participants (n = 44,742) completed a competitive task four times-practice, baseline, following an intervention, and again after repeating the intervention. Results revealed performance improved following practice with incremental effects for imagery-outcome, imagery-process, and self-talk-outcome and self-talk-process over the control group, with the same interventions increasing the intensity of effort invested, arousal and pleasant emotion. Arousal-control interventions associated with pleasant emotions, low arousal, and low effort invested in performance. Instructional interventions were not effective. Results offer support for the utility of online interventions in teaching psychological skills and suggest brief interventions that focus on increasing motivation, increased arousal, effort invested, and pleasant emotions were the most effective.

Privileging Naturals Over Strivers

A preference for “naturals” over “strivers” in performance judgments was investigated to test whether the effect is generalizable across domains, as well as to ascertain any costs imposed on decision quality by favoring naturals. Despite being presented with entrepreneurs equal in achievement, participants judged the natural and his business proposal to be superior to the striver and his proposal on multiple dimensions of performance and success (Study 1a and Study 1b). These findings were extended in Study 2, which quantified the costs of the naturalness bias using conjoint analysis to measure specific decision tradeoffs. Together, these three studies show that people tend to pass over better-qualified individuals in favor of apparent naturals.

The Impacts of Video Games on Cognition (and How the Government Can Guide the Industry)

Video game play has become a pervasive part of American culture. The dramatic increase in the popularity of video games has resulted in significant interest in the effects that video gaming may have on the brain and behavior. The scientific research to date indicates that some, but not all, commercial video games do indeed have the potential to cause large-scale changes in a wide variety of aspects of human behavior, including the focus of this review—cognitive abilities. More recent years have seen the rise of a separate form of video games, the so-called “brain games,” or games designed with the explicit goal of enhancing cognitive abilities. Although research on such brain games is still in its infancy, and the results have definitely not been uniformly positive, there is nonetheless reason for continued optimism that custom games can be developed that make a lasting and positive impact on human cognitive skills. Here, we discuss the current state of the scientific literature surrounding video games and human cognition with an emphasis on points critically related to public policy.

Conducting perception research over the internet: a tutorial review

This article provides an overview of the recent literature on the use of internet-based testing to address important questions in perception research. Our goal is to provide a starting point for the perception researcher who is keen on assessing this tool for their own research goals. Internet-based testing has several advantages over in-lab research, including the ability to reach a relatively broad set of participants and to quickly and inexpensively collect large amounts of empirical data, via services such as Amazon’s Mechanical Turk or Prolific Academic. In many cases, the quality of online data appears to match that collected in lab research. Generally-speaking, online participants tend to be more representative of the population at large than those recruited for lab based research. There are, though, some important caveats, when it comes to collecting data online. It is obviously much more difficult to control the exact parameters of stimulus presentation (such as display characteristics) with online research. There are also some thorny ethical elements that need to be considered by experimenters. Strengths and weaknesses of the online approach, relative to others, are highlighted, and recommendations made for those researchers who might be thinking about conducting their own studies using this increasingly-popular approach to research in the psychological sciences.

Positive Imagery-Based Cognitive Bias Modification as a Web-Based Treatment Tool for Depressed Adults: A Randomized Controlled Trial

Depression is a global health problem requiring treatment innovation. Targeting neglected cognitive aspects may provide a useful route. We tested a cognitive-training paradigm using positive mental imagery (imagery cognitive bias modification, imagery CBM), developed via experimental psychopathology studies, in a randomized controlled trial. Training was delivered via the Internet to 150 individuals with current major depression. Unexpectedly, there was no significant advantage for imagery CBM compared with a closely matched control for depression symptoms as a whole in the full sample. In exploratory analyses, compared with the control, imagery CBM significantly improved anhedonia over the intervention and improved depression symptoms as a whole for those participants with fewer than five episodes of depression and those who engaged to a threshold level of imagery. Results suggest avenues for improving imagery CBM to inform low-intensity treatment tools for depression. Anhedonia may be a useful treatment target for future work.