Modality effects and the structure of short-term verbal memory

Investigating mental workload across task modalities: a multimodal analysis using pupillometry

Understanding mental workload is challenging due to its multidimensional nature and varying sensitivities of its primary measures: task performance, perceived workload, and physiological responses. This study investigates the effects of task load on performance, perceived workload, and pupil measures across different information modalities. A within-subjects experiment involving three tasks (digit span, matrix span, and dual n-back) was conducted with three task load levels. Workload measures included accuracy/sensitivity, reaction time, NASA-TLX, peak pupil diameter, and peak pupil latency. Consistent patterns of associations between task load and these workload measures were found across the three modalities. Workload measures revealed a nuanced interplay between task modality and task load. Robustness of peak pupil latency, accuracy, and NASA-TLX was highlighted across verbal and visual modalities, while peak pupil diameter showed a weaker impact with differences between modalities. The findings encourage multivariate assessment of mental workload to account for different task modalities.

The rising entropy of English in the attention economy

We present evidence that the word entropy of American English has been rising steadily since around 1900. We also find differences in word entropy between media categories, with short-form media such as news and magazines having higher entropy than long-form media, and social media feeds having higher entropy still. To explain these results we develop an ecological model of the attention economy that combines ideas from Zipf's law and information foraging. In this model, media consumers maximize information utility rate taking into account the costs of information search, while media producers adapt to technologies that reduce search costs, driving them to generate higher entropy content in increasingly shorter formats.

Screen Time Exposure and Domain-Specific Working Memory in Young Adults

Introduction With technology advancing across all fields, the utility of digital screens is increasing among all age groups for various purposes. Research indicates that while digital technology presents clear advantages, prolonged exposure can have detrimental effects on various aspects of health, behavior, emotions, and cognitive functions like attention and working memory. A crucial cognitive process for learning and information processing which is working memory, can be affected by factors including screen time. Studies have found that the impact of screen time on working memory can be negative, positive, or show no discernible relationship. However, earlier investigations are limited to smartphone use as screen time exposure and further to only active screen time. As there is a dearth of studies in the Indian context and young adults are more exposed to screen time, it is important to investigate along these lines. Hence, the present study aimed to investigate the impact of active and passive screen time exposure on modality-specific working memory in young adults. Methods Seventy-seven neurotypical individuals aged between 18 and 22 years were recruited. The study utilized auditory and visual reverse digit span tasks and the Corsi-backward task to measure working memory span. Screen time data of the participants were collected through a self-administered 18-item questionnaire covering active and background screen time domains. Results and discussion The present study concluded that only active screen time has a significant effect on visual reverse digit span and supports the notion of the visual superiority effect against an auditory superior effect as suggested by earlier findings. The preliminary findings of correlation observed exclusively within the visual domain in this study could be attributed to the potential impact of screen time exposure (active screen time and textual content). Screen usage demands effective switching between various visual stimuli and ongoing updates of information in memory. Nonetheless, interpreting this explanation and generalization requires caution, given the low ecological validity of the task employed in the study. Future investigations should aim to collect screen time exposure data more objectively, perhaps through online tracking techniques. Furthermore, it would be prudent to expand the correlation analysis to include other cognitive processes and populations.

Conversational technology and reactions to withheld information

People frequently face decisions that require making inferences about withheld information. The advent of large language models coupled with conversational technology, e.g., Alexa, Siri, Cortana, and the Google Assistant, is changing the mode in which people make these inferences. We demonstrate that conversational modes of information provision, relative to traditional digital media, result in more critical responses to withheld information, including: (1) a reduction in evaluations of a product or service for which information is withheld and (2) an increased likelihood of recalling that information was withheld. These effects are robust across multiple conversational modes: a recorded phone conversation, an unfolding chat conversation, and a conversation script. We provide further evidence that these effects hold for conversations with the Google Assistant, a prominent conversational technology. The experimental results point to participants' intuitions about why the information was withheld as the driver of the effect.

Comparing auditory and visual aspects of multisensory working memory using bimodally matched feature patterns

Working memory (WM) reflects the transient maintenance of information in the absence of external input, which can be attained via multiple senses separately or simultaneously. Pertaining to WM, the prevailing literature suggests the dominance of vision over other sensory systems. However, this imbalance may be stemming from challenges in finding comparable stimuli across modalities. Here, we addressed this problem by using a balanced multisensory retro-cue WM design, which employed combinations of auditory (ripple sounds) and visuospatial (Gabor patches) patterns, adjusted relative to each participant's discrimination ability. In three separate experiments, the participant was asked to determine whether the (retro-cued) auditory and/or visual items maintained in WM matched or mismatched the subsequent probe stimulus. In Experiment 1, all stimuli were audiovisual, and the probes were either fully mismatching, only partially mismatching, or fully matching the memorized item. Experiment 2 was otherwise same as Experiment 1, but the probes were unimodal. In Experiment 3, the participant was cued to maintain only the auditory or visual aspect of an audiovisual item pair. In two of the three experiments, the participant matching performance was significantly more accurate for the auditory than visual attributes of probes. When the perceptual and task demands are bimodally equated, auditory attributes can be matched to multisensory items in WM at least as accurately as, if not more precisely than, their visual counterparts.

Memory and creativity: A meta-analytic examination of the relationship between memory systems and creative cognition

Increasing evidence suggests that specific memory systems (e.g., semantic vs. episodic) may support specific creative thought processes. However, there are a number of inconsistencies in the literature regarding the strength, direction, and influence of different memory (semantic, episodic, working, and short-term) and creativity (divergent and convergent thinking) types, as well as the influence of external factors (age, stimuli modality) on this purported relationship. In this meta-analysis, we examined 525 correlations from 79 published studies and unpublished datasets, representing data from 12,846 individual participants. We found a small but significant (r = .19) correlation between memory and creative cognition. Among semantic, episodic, working, and short-term memory, all correlations were significant, but semantic memory - particularly verbal fluency, the ability to strategically retrieve information from long-term memory - was found to drive this relationship. Further, working memory capacity was found to be more strongly related to convergent than divergent creative thinking. We also found that within visual creativity, the relationship with visual memory was greater than that of verbal memory, but within verbal creativity, the relationship with verbal memory was greater than that of visual memory. Finally, the memory-creativity correlation was larger for children compared to young adults despite no impact of age on the overall effect size. These results yield three key conclusions: (1) semantic memory supports both verbal and nonverbal creative thinking, (2) working memory supports convergent creative thinking, and (3) the cognitive control of memory is central to performance on creative thinking tasks.

Online reviews, customer Q&As, and product sales: A PVAR approach

Online reviews and customer Q&As have emerged as two vital forms of electronic word-of-mouth (eWOM) that significantly influence consumer decisions in e-commerce. Yet, a comprehensive understanding of the individual and combined roles of these eWOM types in shaping market dynamics remains elusive. This study addresses this research gap by tracking and analyzing three months of eWOM and sales data for 120 laptops on Amazon, comprising 7,205 online reviews, 6,365 customer Q&A questions, and 7,419 answers. Leveraging the Panel Vector Autoregression (PVAR) model and STATA16.0 software, we unravel the intricate dynamics between online reviews, customer Q&As, and laptop sales. The empirical results reveal distinctive influence mechanisms of online reviews and customer Q&As on product sales, with review volume and answer valence positively affecting sales. Importantly, answer volume was found to stimulate online reviews and enhance their valence. Our study elucidates the interplay among online reviews, customer Q&As, and product sales, underscoring the need for future research on multi-type eWOM. Further, the insights gleaned offer valuable guidance for online platforms and retailers to strategize their eWOM management.

A cognitive account of the puzzle of ideography

We posit a cognitive account of the puzzle of ideography, which complements the standardization account of Morin. Efficient standardization of spoken language is phenomenologically attributed to a modality effect coupled with chunking of cognitive representations, further aided by multisensory integration and the serialized nature of attention. These mechanisms explain why languages dominate graphic codes for general-purpose communication.

Can synchronised tones facilitate immediate memory for printed lists?

In verbal list recall, adding features redundant with the ones to be recalled theoretically could assist recall, by providing additional retrieval cues, or it could impede recall, by draining attention away from the features to be recalled. We examined young adults' immediate memory of lists of printed digits when these lists were sometimes accompanied by synchronised, concurrent tones, one per digit. Unlike most previous irrelevant-sound effects, the tones were not asynchronous with the printed items, which can corrupt the episodic record, and did not repeat within a list. Memory of the melody might bring to mind the associated digits like lyrics in a song. Sometimes there were instructions to sing the digits covertly in the tone pitches. In three experiments, there was no evidence that these methods enhanced memory. Instead, there appeared to be a distraction effect from the synchronised tones, as in the irrelevant sound effect with asynchronised tones.

The effects of diurnal variability and modality on false memories formation

The main objective of the current study was to investigate the effect of time of day on visual and auditory, short-term memory (STM), and long-term memory (LTM) distortions using a hybrid Deese-Roediger-McDermott procedure. In Experiment 1, we used semantically related words, whereas in Experiment 2 - words were characterized by phonological similarity. The results showed a relationship between modality and types of stimuli. In STM, more semantic errors were found in the evening for items presented visually and more errors following auditory presentation for phonologically similar words. In LTM, the analysis revealed a higher rate of semantic distortions in the evening hours for auditorily presented words. For words with phonological similarity, we observed more errors in the evening without the effect of modality. The results support the hypothesis that more reliance is placed on elaborative processing in the evening and more on maintenance processing in the morning; however, this is not modality invariant.

Wakeful Rest Benefits Recall, but Not Recognition, of Incidentally Encoded Memory Stimuli in Younger and Older Adults

Older adults exhibit deficits in episodic memory tasks, which have often been attributed to encoding or retrieval deficits, with little attention to consolidation mechanisms. More recently, researchers have attempted to measure consolidation in the context of a behavioral experiment using the wakeful rest paradigm (i.e., a brief, quiet period of minimal stimulation, which facilitates memory performance, compared to a distractor task). Critically, older adults might not produce this effect, given established age differences in other episodic memory processes and mind-wandering. In three experiments, we directly compared younger and older adults in modified versions of the wakeful rest paradigm. Critically, we utilized incidental encoding procedures (all experiments) and abstract shape stimuli (in Experiment 3) to limit the possibility of retrieval practice or maintenance rehearsal as potential confounding mechanisms in producing the wakeful rest effect. Wakeful rest reliably and equally benefited recall of incidentally encoded words in both younger and older adults. In contrast, wakeful rest had no benefit for standard accuracy measures of recognition performance in verbal stimuli, although there was an effect in response latencies for non-verbal stimuli. Overall, these results suggest that the benefits of wakeful rest on episodic retrieval are preserved across age groups, and hence support age-independence in potential consolidation mechanisms as measured by wakeful rest. Further, these benefits do not appear to be dependent on the intentionality of encoding or variations in distractor task types. Finally, the lack of wakeful rest benefits on recognition performance might be driven by theoretical constraints on the effect or methodological limitations of recognition memory testing in the current paradigm.

Working memory development: A 50-year assessment of research and underlying theories

The author has thought about working memory, not always by that name, since 1969 and has conducted research on its infant and child development since the same year that the seminal work of Baddeley and Hitch (1974) was published. The present article assesses how the field of working memory development has been influenced since those years by major theoretical perspectives: empiricism (along with behaviorism), nativism (along with modularity), cognitivism (along with constructivism), and dynamic systems theory. The field has not fully discussed the point that these theoretical perspectives have helped to shape different kinds of proposed working memory systems, which in turn have deeply influenced what is researched and how it is researched. Here I discuss that mapping of theoretical viewpoints onto assumptions about working memory and trace the influence of this mapping on the field of working memory development. I illustrate where these influences have led in my own developmental research program over the years.

The effect of visual multimedia instructions against fake news spread: A quasi-experimental study with Nigerian students

This study examined the effect of visual multimedia instructions (guided by literacy concepts) as an intervention strategy for improving fake news knowledge, detection skills and curtailing the tendency to share fake news. We used the inoculation theory, message interpretation process (MIP) theory and cognitive theory of multimedia learning to provide a useful explanation for the interventions of literacy concept. Our study made use of 470 participants divided into two groups, comprising the control group, n  = 235 and the treatment group, n  = 235. After the experiment, we found that participants in the visual multimedia experimental group demonstrated a higher knowledge of fake news, better ability to detect fake news and shared more accurate news articles, compared to their counterparts who were instructed in a non-multimedia setting. We focused only on university students from one institution in Nigeria. Thus, we encourage future studies to extend beyond the student population.

Modality Effects Examined by Means of an Online Sentence-Picture Comparison Task

Are oral sentences accompanied by pictures easier to understand than written sentences accompanied by the same pictures? This question-intensely discussed for more than two decades in educational, psychological, and psycholinguistic research in terms of modality effect in multimedia learning, split-attention effect, or visuospatial load effect-was examined by means of an online sentence-picture comparison task, with participants reading or listening to short sentences accompanied by pictures. Sentences and pictures referred to a conceptually and linguistically basic universe that was constructed from a limited number of familiar objects and characteristics. Mixed repeated-measures ANOVAs were calculated separately for simultaneously and sequentially presented sentence-picture versions, with modality (oral, written), picture complexity (complex, simple), and sentence-picture compatibility (compatible, incompatible) as within-subject factors, and age (6th-graders, adults) as between-subjects factor. The experiment was based on reaction time and acuity measures. The online sentence-picture comparison task requires participants to take note of both components (verbal and pictorial information). The presence of two age groups made it possible to examine modality effects from a developmental point of view by suggesting that learners' written comprehension and monitoring skills at some point equal and then surpass their oral comprehension and monitoring skills. The experiment showed the necessity to interpret modality main effects in the context of their two-way and three-way interactions with other variables since modality effects taken alone do not tell the whole story. The concepts of split attention, temporal contiguity, and element interactivity were discussed in this context.

The Production Effect Interacts With Serial Positions

Abstract: Reading some words aloud during presentation, that is, producing them, and reading other words silently generate a large memory advantage for words that are produced. This robust within-list production effect is in contrast with the between-lists condition in which all words are read aloud or silently. In a between-lists condition, produced items are better recognized, but not better recalled. The lack of a between-lists production effect with recall tasks has often been presented as one of its defining characteristics and as a benchmark for evaluating models. Recently, Cyr et al. (2021) showed that this occurs because item production interacts with serial positions: Produced items are less well recalled on the first serial positions than silently read items, while the reverse pattern is observed for the recency portion of the curve. However, this pattern was observed with a repeated-measures design, and it may be a by-product of compensatory processes under the control of participants. Here, using a between-participants design, we observed the predicted interaction between production and serial positions. The results further support the Revised Feature Model (RFM) suggesting that produced items are encoded with more modality-dependent distinctive features, therefore benefiting recall. However, the production of the additional distinctive features would disrupt rehearsal.

Recall, response bias and recognition are differentially impacted by social anxiety irrespective of feedback modality

BACKGROUND AND OBJECTIVES

This study replicates and extends Houle-Johnson et al.'s (2019) findings to better understand the role of feedback modality, ambiguity and social anxiety in the recognition and recall of self-relevant feedback.

METHODS

Participants gave a speech and were provided with positive, negative, and ambiguous feedback via written text, (n = 33) or recorded sentences (n = 31) and later completed a recognition and recall task for the feedback.

RESULTS

Recognition (p = .80, η_p² = 0) and recall (p = .09, η_p² = 0.08) did not differ between written or recorded feedback. All participants demonstrated a negative response bias (p < .001, η_p² = 0.22) and recalled more negative than positive feedback (p = .02, η_p² = 0.10) but were no more accurate in recognizing negative compared to positive feedback (p = .08, η_p² = 0). Although social anxiety did not impact recognition accuracy (p = .94, η_p² = 0), participants with high social anxiety demonstrated a more pronounced negative response bias (p < .01, η_p² = 0.11) and negative recall bias (p = .02, SE = 1.12) than low social anxiety participants. Moreover, the more negatively ambiguous items were perceived, the more likely they were identified old in the high social anxiety group, whereas the opposite was true for the low social anxiety group (B = .13, p < .10).

LIMITATIONS

Task believability was relatively low across all participants.

CONCLUSIONS

Our findings suggest that modality does not influence memory for feedback. Moreover, social anxiety might be characterized by a negative bias in recall and response bias, but not necessarily increased accuracy in recognition of negative feedback.

The effects of stimulus modality, task complexity, and cuing on working memory and the relationship with speech recognition in older cochlear implant users

INTRODUCTION

The role of working memory (WM) in speech recognition of older cochlear implant (CI) users remains unclear. This study 1) examined the effects of aging and CI on WM performance across different modalities (auditory vs. visual) and cuing conditions, and 2) assessed how specific WM measures relate to sentence and word recognition in noise.

METHOD

Fourteen Older CI users, 12 Older acoustic-hearing (AH) listeners with age-appropriate hearing loss, and 15 Young normal-hearing (NH) listeners were tested. Participants completed two simple span tasks (auditory digit and visual letter span), two complex WM tasks (reading span and cued-modality WM with simultaneously presented auditory digits and visual letters), and two speech recognition tasks (sentence and word recognition in speech-babble noise).

RESULTS

The groups showed similar simple span performance, except that Older CI users had lower auditory digit span than Young NH listeners. Both older groups had similar reading span performance, but scored significantly lower than Young NH listeners, indicating age-related declines in attentional and phonological processing. A similar group effect was observed in the cued-modality WM task. All groups showed higher recall for auditory digits than for visual letters and the advantage was most evident without modality cuing. All groups displayed greater cuing benefits for visual recall than for auditory recall, suggesting that participants consistently allocated more attention to auditory stimuli regardless of cuing. For Older CI users, after controlling for the previously reported spectral resolution, auditory-uncued WM performance was significantly correlated with word recognition but not sentence recognition.

CONCLUSIONS

Complex WM was significantly affected by aging but not by CI. Neither aging nor CI significantly affected modality cuing benefits in the WM task. For Older CI users, complex auditory WM with attentional control may better reflect the cognitive load of speech recognition in noise than simple span or complex visual WM.

Spatial and Temporal Context Jointly Modulate the Sensory Response within the Ventral Visual Stream

Both spatial and temporal context play an important role in visual perception and behavior. Humans can extract statistical regularities from both forms of context to help process the present and to construct expectations about the future. Numerous studies have found reduced neural responses to expected stimuli compared with unexpected stimuli, for both spatial and temporal regularities. However, it is largely unclear whether and how these forms of context interact. In the current fMRI study, 33 human volunteers were exposed to pairs of object stimuli that could be expected or surprising in terms of their spatial and temporal context. We found reliable independent contributions of both spatial and temporal context in modulating the neural response. Specifically, neural responses to stimuli in expected compared with unexpected contexts were suppressed throughout the ventral visual stream. These results suggest that both spatial and temporal context may aid sensory processing in a similar fashion, providing evidence on how different types of context jointly modulate perceptual processing.

The influence of time structure on prediction motion in visual and auditory modalities

Usually people can estimate the correct position of a moving object even when it temporarily moves behind an occlusion. Studies have been performed on this type of occluded motion with prediction motion (PM) tasks in the laboratory. Previous publications have emphasized that people could use mental imagery or apply an oculomotor system to estimate the arrival of a moving stimulus at the target place. Nevertheless, these two ways cannot account for the performance difference under a different set of conditions. Our study tested the role of time structure in a time-to-collision (TTC) task using visual and auditory modalities. In the visual condition, the moving red bar travelled from left to right and was invisible during the entire course but flashed at the initial and the occluded points. The auditory condition and visual condition were alike, except that the flashes in the visual condition were changed to clicks at the initial and the occluded points. The results illustrated that participants' performance was better in the equal time structure condition. The comparison between the two sense modalities demonstrated a similar tendency, which suggested there could be common cognitive processes between visual and auditory modalities when participants took advantage of temporal cues to judge TTC.

Boys-Specific Text-Comprehension Enhancement With Dual Visual-Auditory Text Presentation Among 12-14 Years-Old Students

Quality of language comprehension determines performance in all kinds of activities including academics. Processing of words initially develops as auditory, and gradually extends to visual as children learn to read. School failure is highly related to listening and reading comprehension problems. In this study we analyzed sex-differences in comprehension of texts in Spanish (standardized reading test PROLEC-R) in three modalities (visual, auditory, and both simultaneously: dual-modality) presented to 12-14-years old students, native in Spanish. We controlled relevant cognitive variables such as attention (d2), phonological and semantic fluency (FAS) and speed of processing (WISC subtest Coding). Girls' comprehension was similar in the three modalities of presentation, however boys were importantly benefited by dual-modality as compared to boys exposed only to visual or auditory text presentation. With respect to the relation of text comprehension and school performance, students with low grades in Spanish showed low auditory comprehension. Interestingly, visual and dual modalities preserved comprehension levels in these low skilled students. Our results suggest that the use of visual-text support during auditory language presentation could be beneficial for low school performance students, especially boys, and encourage future research to evaluate the implementation in classes of the rapidly developing technology of simultaneous speech transcription, that could be, in addition, beneficial to non-native students, especially those recently incorporated into school or newly arrived in a country from abroad.

Meaningful Versus Meaningless Sounds and Words

The current study assessed memory performance for perceptually similar environmental sounds and speech-based material after short and long delays. In two studies, we demonstrated a similar pattern of memory performance for sounds and words in short-term memory, yet in long-term memory, the performance patterns differed. Experiment 1 examined the effects of two different types of sounds: meaningful (MFUL) and meaningless (MLESS), whereas Experiment 2 assessed memory performance for words and nonwords. We utilized a modified version of the classical Deese-Roediger-McDermott (Deese, 1959; Roediger & McDermott, 1995) procedure and adjusted it to test the effects of acoustic similarities between auditorily presented stimuli. Our findings revealed no difference in memory performance between MFUL and MLESS sounds, and between words and nonwords after short delays. However, following long delays, greater reliance on meaning was noticed for MFUL sounds than MLESS sounds, while performance for linguistic material did not differ between words and nonwords. Importantly, participants' memory performance for words and nonwords was accompanied by a more lenient response strategy. The results are discussed in terms of perceptual and semantic similarities between MLESS and MFUL sounds, as well as between words and nonwords.

Bootstrapping the visuospatial bootstrapping effect and testing its spatialisation

When sequences of digits are visually presented within a numerical keypad on a screen, memory span increases, this effect was named visuospatial bootstrapping. The aim of the first experiment was to know if this effect could emerge without presenting a keypad on the screen. For this purpose, a three-phase experiment was designed. During phase 1, the immediate serial recall of two groups of participants was compared (pre-training): the first group saw sequences of one-digit numbers displayed on a screen within a keypad (the keypad group) whereas the second group heard the (same) sequences (the auditive group). During phase 2, all participants underwent a training session to help them visualise in their mind a keypad. Finally, in the third phase, participants were tested again with an immediate serial recall task (post-training). Results showed that both groups had comparable performance in post-training indicating that the visuospatial bootstrapping could be obtained without displaying a numerical keypad. The second experiment also involved a keypad group and an auditive group and was designed to investigate their spatial representation. Results showed that both groups spatialised the digits following the keypad spatial configuration: digits 1-4-7 were associated to left, 2-5-8 to middle and 3-6-9 to right.

Attention effects in working memory that are asymmetric across sensory modalities

A key unanswered question about working memory is the nature of interference between items. At one extreme of existing theories, interference occurs between any two items because of a general capacity limit. At another extreme, interference depends on the similarity between particular features of different items. We examine this question in three experiments by presenting two sets of items on each trial, comprising tones or colors, with three levels of similarity between the two sets: cross-modal, unimodal with different marking features (two different musical instruments or shapes), and unimodal with the same marking feature. Another question is the extent to which the entry of presented items into working memory is obligatory or optional, which we examined by requiring retention of the first, the second, or both sets of stimuli for a recognition test shortly after the presentation of the two sets. The combination of the set similarity and attention manipulations allows us to draw conclusions about the nature of working-memory storage. The findings were not entirely in accord with any pre-existing theory. The effects of feature similarity were present in both modalities but more pronounced for sounds, whereas the detrimental effects of attention to both sets for retention occurred only for visual stimuli. Based on the findings we suggest a new, hybrid conception of working memory storage.

The Auditory English Lexicon Project: A multi-talker, multi-region psycholinguistic database of 10,170 spoken words and nonwords

The Auditory English Lexicon Project (AELP) is a multi-talker, multi-region psycholinguistic database of 10,170 spoken words and 10,170 spoken nonwords. Six tokens of each stimulus were recorded as 44.1-kHz, 16-bit, mono WAV files by native speakers of American, British, and Singapore English, with one from each gender. Intelligibility norms, as determined by average identification scores and confidence ratings from between 15 and 20 responses per token, were obtained from 561 participants. Auditory lexical decision accuracies and latencies, with between 25 and 36 responses per token, were obtained from 438 participants. The database also includes a variety of lexico-semantic variables and structural indices for the words and nonwords, as well as participants' individual difference measures such as age, gender, language background, and proficiency. Taken together, there are a total of 122,040 sound files and over 4 million behavioral data points in the AELP. We describe some of the characteristics of this database. This resource is freely available from a website ( https://inetapps.nus.edu.sg/aelp/ ) hosted by the Department of Psychology at the National University of Singapore.

Comparing verbal working memory load in auditory and visual modalities using functional near-infrared spectroscopy

The verbal identity n-back task is commonly used to assess verbal working memory (VWM) capacity. Only three studies have compared brain activation during the n-back when using auditory and visual stimuli. The earliest study, a positron emission tomography study of the 3-back, found no differences in VWM-related brain activation between n-back modalities. In contrast, two subsequent functional magnetic resonance imaging (fMRI) studies of the 2-back found that auditory VWM was associated with greater left dorsolateral prefrontal cortex (DL-PFC) activation than visual VWM, perhaps suggesting that auditory VWM requires more cognitive effort than its visual counterpart. The current study aimed to assess whether DL-PFC activation (i.e., cognitive effort) differs by VWM modality. To do this, 16 younger adults completed an auditory and visual n-back, both at four levels of VWM load. Concurrently, activation of the PFC was measured using functional near-infrared spectroscopy (fNIRS), a silent neuroimaging method. We found that DL-PFC activation increased with VWM load, but it was not affected by VWM modality or the interaction between load and modality. This supports the view that both VWM modalities require similar cognitive effort, and perhaps that previous fMRI results were an artefact of scanner noise. We also found that, across conditions, DL-PFC activation was positively correlated with reaction time. This may further support DL-PFC activation as an index of cognitive effort, and fNIRS as a method to measure it.

Visual and Auditory Verbal Memory in Older Adults: Comparing Postlingually Deaf Cochlear Implant Users to Normal-Hearing Controls

Purpose The aim of this study was to test whether a group of older postlingually deafened cochlear implant users (OCIs) use similar verbal memory strategies to those used by older normal-hearing adults (ONHs). Verbal memory functioning was assessed in the visual and auditory modalities separately, enabling us to eliminate possible modality-based biases. Method Participants performed two separate visual and auditory verbal memory tasks. In each task, the visually or aurally presented study words were learned by vocal production (saying aloud) or by no production (reading silently or listening), followed by a free recall test. Twenty-seven older adults (> 60 years) participated (OCI = 13, ONH = 14), all of whom demonstrated intact cognitive abilities. All OCIs showed good open-set speech perception results in quiet. Results Both ONHs and OCIs showed production benefits (higher recall rates for vocalized than nonvocalized words) in the visual and auditory tasks. The ONHs showed similar production benefits in the visual and auditory tasks. The OCIs demonstrated a smaller production effect in the auditory task. Conclusions These results may indicate that different modality-specific memory strategies were used by the ONHs and the OCIs. The group differences in memory performance suggest that, even when deafness occurs after the completion of language acquisition, the reduced and distorted external auditory stimulation leads to a deterioration in the phonological representation of sounds. Possibly, this deterioration leads to a less efficient auditory long-term verbal memory.

Effects of Visual and Aural Presentation Styles and Rhythm on Working Memory as Measured by Monosyllabic Sequential Digit Recall

Although information is frequently paired with music to enhance recall, there is a lack of basic research investigating how aspects of recorded music, as well as how it is presented, facilitate working memory. Therefore, the purpose of this study was to determine the effects of visual and aural presentation styles, rhythm, and participant major on working memory as measured by sequential monosyllabic digit recall performance. We isolated visual and aural presentation styles and rhythm conditions during six different treatment stimuli presented on a computer screen in the study: (a) Visual Rhythm; (b) Visual No Rhythm; (c) Aural Rhythm; (d) Aural No Rhythm; (e) Visual + Aural Rhythm; (f) Visual + Aural No Rhythm. Participants' (N = 60; 30 nonmusic majors and 30 music majors) task was to immediately recall the information paired with music within each condition. Analyses of variance indicated a significant difference between the visual and visual + aural presentation style conditions with the visual + aural condition having more accurate recall. While descriptive data indicated that rhythm tended to facilitate recall, there was no significant difference between rhythm and no rhythm conditions. Nonmusic major participants tended to have slightly more accurate recall than music major participants, although this difference was not significant. Participants tended to have higher recall accuracy during primacy and recency serial positions. As participants had most accurate recall during the visual + aural presentation style conditions, it seems that the multi-sensory presentation modes can be effective for teaching information to be immediately recalled as long as they do not contain too much information and overload the limited storage capacity of working memory. Implications for clinical practice, limitations, and suggestions for future research are provided.

Cognitive functioning, sleep quality, and work performance in non-clinical burnout: The role of working memory

We investigated the relationship between cognitive functioning, work performance, and sleep in non-clinical burnout. In a working population, an online survey was conducted with additional online neuropsychological tests of varying complexity, measuring attention and different components of working memory, of which the coordinating subcomponent the ‘Central Executive’ is thought to be the most vulnerable to stress. Results indicate that non-clinical burnout is associated with more—though not severe—sleep problems, more depressive complaints, impaired work performance, and with both subjective and objective cognitive impairments. Compared with healthy respondents (N = 107), people with non-clinical burnout (N = 17) had a significantly poorer performance on the tests of the visuospatial sketchpad and the Central Executive of the working memory. Our study also indicates that more complex tests may be more sensitive in detecting cognitive dysfunction in non-clinical burnout. Furthermore, a relationship was found between dual-task performance and work performance. Regarding to sleep quality, in our sample of people with non-clinical burnout, there were no severe sleep problems. In the entire sample, however, insomnia was significantly related to subjective, but not objective, cognitive functioning, and also not to work performance.

The role of iconicity and simultaneity for efficient communication: The case of Italian Sign Language (LIS)

A fundamental assumption about language is that, regardless of language modality, it faces the linearization problem, i.e., an event that occurs simultaneously in the world has to be split in language to be organized on a temporal scale. However, the visual modality of signed languages allows its users not only to express meaning in a linear manner but also to use iconicity and multiple articulators together to encode information simultaneously. Accordingly, in cases when it is necessary to encode informatively rich events, signers can take advantage of simultaneous encoding in order to represent information about different referents and their actions simultaneously. This in turn would lead to more iconic and direct representation. Up to now, there has been no experimental study focusing on simultaneous encoding of information in signed languages and its possible advantage for efficient communication. In the present study, we assessed how many information units can be encoded simultaneously in Italian Sign Language (LIS) and whether the amount of simultaneously encoded information varies based on the amount of information that is required to be expressed. Twenty-three deaf adults participated in a director-matcher game in which they described 30 images of events that varied in amount of information they contained. Results revealed that as the information that had to be encoded increased, signers also increased use of multiple articulators to encode different information (i.e., kinematic simultaneity) and density of simultaneously encoded information in their production. Present findings show how the fundamental properties of signed languages, i.e., iconicity and simultaneity, are used for the purpose of efficient information encoding in Italian Sign Language (LIS).

The Specific Impact of Apolipoprotein E Epsilon 2 on Cognition and Brain Function in Cognitively Normal Elders and Mild Cognitive Impairment Patients

Variants in the apolipoprotein E (APOE) gene play an important role in the development of Alzheimer’s disease (AD). Specifically, the APOE ε4 allele is an established genetic risk factor for AD, while the APOE ε2 allele is a protective factor against AD. However, the mechanism underlying this impact of APOE genotype on the pathogenesis of AD remain unclear. This study sought to investigate the influence of APOE genotype on cognition and neuroimaging features in cognitively normal (CN) elderly individuals and patients with mild cognitive impairment (MCI). A total of 177 participants were selected from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database, including 101 MCI patients and 76 CN individuals. A 2 × 3 (consisting of two groups and three APOE genotypes) analysis of covariance was carried out to measure the influences of diagnosis and APOE genotype on cognition and brain features, assessed based on global functional connectivity density (gFCD) and hippocampal volume. In addition, a mediation analysis was carried out to investigate the indirect influence of neuroimaging features on the relationship between APOE genotype and cognitive performance in the MCI group. This analysis revealed that APOE genotype had an influence on brain function in the bilateral precentral gyrus, right thalamus, and posterior cingulate cortex (PCC). In addition, an interactive influence between diagnosis and APOE genotype was found on general cognition, immediate memory, executive function, hippocampal volume, and gFCD in the right dorsolateral prefrontal cortex and medial prefrontal cortex (MPFC). Finally, this mediation analysis revealed that hippocampal volume and gFCD in the thalamus may mediate the relationship between APOE genotype and cognitive performance in the MCI group. Taken together, our findings provide novel insights into the neural mechanisms underlying the genetically guided pathogenic mechanisms of AD.

False recognition modality effects in short-term memory: Reversing the auditory advantage

The auditory advantage in short-term false recognition - reduced false memories for auditory compared to visually presented words (Olszewska, Reuter-lorenz, Munier, & Bendler, 2015), has been attributed to greater item distinctiveness in auditory compared to visual memory traces. If so, varying auditory trace distinctiveness should influence false recognition rates. Phonologically and semantically related words were presented visually or aurally. The auditory advantage for semantic lists was replicated but a reversal was observed for phonological lists. Reducing modality-specific acoustic and phonological distinctiveness by increasing phonological similarity led to increased false memory. The findings are consistent with a framework positing the generation of input-dependent memory traces and the role of relative distinctiveness in influencing short-term memory.

Short-term false memories vary as a function of list type

False memories have primarily been investigated at long-term delays in the Deese-Roediger-McDermott (DRM) procedure, but a few studies have reported meaning-based false memories at delays as short as 1-4 s. The current study further investigated the processes that contribute to short-term false memories with semantic and phonological lists (Experiment 1) and hybrid lists containing items of each type (Experiment 2). In Experiment 1, more false memories were found for phonological than for semantic lists. In Experiment 2, an asymmetrical hyper-additive effect was found such that including one or two phonological associates in pure semantic lists yielded a robust increase in false alarms, whereas including semantic associates in pure phonological lists did not affect false alarms. These results are more consistent with the activation-monitoring account of false memory creation than with fuzzy trace theory that has not typically been referenced when describing phonological false memories.

Predicting Speech Recognition Using the Speech Intelligibility Index and Other Variables for Cochlear Implant Users

Purpose Although the speech intelligibility index (SII) has been widely applied in the field of audiology and other related areas, application of this metric to cochlear implants (CIs) has yet to be investigated. In this study, SIIs for CI users were calculated to investigate whether the SII could be an effective tool for predicting speech perception performance in a population with CI. Method Fifteen pre- and postlingually deafened adults with CI participated. Speech recognition scores were measured using the AzBio sentence lists. CI users also completed questionnaires and performed psychoacoustic (spectral and temporal resolution) and cognitive function (digit span) tests. Obtained SIIs were compared with predicted SIIs using a transfer function curve. Correlation and regression analyses were conducted on perceptual and demographic predictor variables to investigate the association between these factors and speech perception performance. Result Because of the considerably poor hearing and large individual variability in performance, the SII did not predict speech performance for this CI group using the traditional calculation. However, new SII models were developed incorporating predictive factors, which improved the accuracy of SII predictions in listeners with CI. Conclusion Conventional SII models are not appropriate for predicting speech perception scores for CI users. Demographic variables (aided audibility and duration of deafness) and perceptual-cognitive skills (gap detection and auditory digit span outcomes) are needed to improve the use of the SII for listeners with CI. Future studies are needed to improve our CI-corrected SII model by considering additional predictive factors. Supplemental Material https://doi.org/10.23641/asha.8057003.

Specific conditions for a selective deficit in memory for order in children with dyslexia

Short-term memory (STM) models distinguish between item and order memorization. The present study aims to explore how item and order STM are affected by the nature of the stimuli, the sequential versus simultaneous mode of presentation, the visual versus auditory presentation modality, the possibility of verbal recoding. A total of 20 children with dyslexia were matched one-by-one with 20 typically reading children on sex, age (8-14 years), and grade. Computerized STM tasks were administered while manipulating type (item vs. order), stimuli (letters vs. colors), sequentiality, input and output modality, as well as the presence/absence of articulatory suppression and distractors. General Linear Model analyses were conducted on accuracy scores for item and order STM. Both item and order recall scores were lower for children with dyslexia. Although order STM in the visual input condition turned out to be more impaired than item STM in the dyslexic group, both item and order memory impairments become evident when verbal recoding is prevented through articulatory suppression. Moreover, dyslexic children, unlike typical readers, were not facilitated by the linguistic nature of the stimuli to be remembered. The present findings suggest that the often-reported selective impairment of serial memory in dyslexia is restricted to stimuli that are verbal in nature or can be verbally recoded, whereas both item and order memory impairments become evident when verbal recoding is prevented through articulatory suppression. The presence of distractors is particularly detrimental to STM in the dyslexic group. The sensitivity to distractors, suppression, and stimuli in STM is predictive of reading performance.

How much is remembered as a function of presentation modality?

According to a widespread claim used for teaching recommendations, students remember 10% of what they read, 20% of what they hear, 30% of what they see, and 50% of what they see and hear. Clearly, the percentages cannot be correct, and there is no empirical evidence for the ordering. To investigate the ordering, in a navigation paradigm, subjects were given messages instructing them to move in a grid of four stacked matrices by clicking a computer mouse. Three modalities were compared presented either once, see (visual arrows), hear (auditory words), read (visual words); twice in succession, see-see, hear-hear, read-read; or in two different successive modalities, see-hear, hear-see, see-read, read-see, hear-read, read-hear. Better performance was found for messages presented twice than once, but messages in the two modalities were not always better than twice in one modality. For the twice-presented messages, performance varied as a function of the second modality, with see best and read worst. However, the ordering for the first modality was not reliable and was inconsistent with the widespread claim. Thus, the widespread claim is clearly wrong, not only in its percentages, but also because of its lack of generality.

Can Music Foster Learning - Effects of Different Text Modalities on Learning and Information Retrieval

This study investigates the possibilities of fostering learning based on differences in recall and comprehension after learning with texts which were presented in one of three modalities: either in a spoken, written, or sung version. All three texts differ regarding their processing, especially when considering working memory. Overall, we assume the best recall performance after learning with the written text and the best comprehension performance after learning with the sung text, respectively, compared to both other text modalities. We also analyzed whether the melody of the sung material functions as a mnemonic aid for the learners in the sung text condition. If melody and text of the sung version are closely linked, presentation of the melody during the post-test phase could foster text retrieval. 108 students either learned from a sung text performed by a professional singer, a printed text, or the same text read out loud. Half of the participants worked on the post-test while listening to the melody used for the musical learning material and the other half did not listen to a melody. The written learning modality led to significantly better recall than with the spoken (d = 0.97) or sung text (d = 0.78). However, comprehension after learning with the sung modality was significantly superior compared to when learning with the written learning modality (d = 0.40). Reading leads to more focus on details, which is required to answer recall questions, while listening fosters a general understanding of the text, leading to higher levels of comprehension. Listening to the melody during the post-test phase negatively affected comprehension, irrespective of the modality during the learning phase. This can be explained by the seductive detail effect, as listening to the melody during the post-test phase may distract learners from their main task. In closing, theoretical and practical implications are discussed.

The Word-Length Effect in Probed and Serial Recall

The word-length effect in immediate serial recall has been explained as the possible consequence of rehearsal processes or of output processes. In the first experiment adult subjects heard lists of five long or short words while engaging in articulatory suppression during presentation. Full serial recall or probed recall for a single item followed the list either immediately or after a 5-second delay to encourage rehearsal. The word-length effect was not influenced by recall delay, but was much smaller in probed than in serial recall. Examination of the serial position curves suggested that this might be due to a recency component operating in probed recall. Experiment 2 confirmed a word-length-insensitive recency effect in probed recall and showed that this was resistant to an auditory suffix, unlike the small recency effect found in serial recall. Experiment 3 used visual presentation without concurrent articulation. Under these conditions there was no recency effect for either recall method, but the word-length effect was again much smaller in probed than in serial recall. This was confirmed in Experiment 4, in which the presentation of serial and probed recall was randomized across trials, showing that the differences between recall methods could not be due to encoding strategies. We conclude that for visual presentation, at least part of the word-length effect originates in output processes. For auditory presentation the position is less clear, as serial and probed recall appear to draw on different resources. The nature of the output processes that may give rise to word-length effects is discussed.

Common modality effects in immediate free recall and immediate serial recall

In 2 experiments, participants were presented with lists of between 2 and 12 words for either immediate free recall (IFR) or immediate serial recall (ISR). Auditory recall advantages at the end of the list (modality effects) and visual recall advantages early in the list (inverse modality effects) were observed in both tasks and the extent and magnitude of these effects were dependent upon list length. Both tasks displayed modality effects with short lists that were large in magnitude but limited to the final serial position, consistent with those observed in the typically short lists used in ISR, and both tasks displayed modality effects with longer lists that were small in magnitude and more extended across multiple end-of-list positions, consistent with those observed in the typically longer lists used in IFR. Inverse modality effects were also observed in both tasks at early list positions on longer lengths. Presentation modality did not affect where recall was initiated, but modality effects were greatest on trials where participants initiated recall with the first item. We argue for a unified account of IFR and ISR. We also assume that the presentation modality affects the encoding of all list items, and that modality effects emerge due to the greater resistance of auditory items to output interference. (PsycINFO Database Record

Nonword repetition in adults who stutter: The effects of stimuli stress and auditory-orthographic cues

Purpose

Adults who stutter (AWS) are less accurate in their immediate repetition of novel phonological sequences compared to adults who do not stutter (AWNS). The present study examined whether manipulation of the following two aspects of traditional nonword repetition tasks unmask distinct weaknesses in phonological working memory in AWS: (1) presentation of stimuli with less-frequent stress patterns, and (2) removal of auditory-orthographic cues immediately prior to response.

Method

Fifty-two participants (26 AWS, 26 AWNS) produced 12 bisyllabic nonwords in the presence of corresponding auditory-orthographic cues (i.e., immediate repetition task), and the absence of auditory-orthographic cues (i.e., short-term recall task). Half of each cohort (13 AWS, 13 AWNS) were exposed to the stimuli with high-frequency trochaic stress, and half (13 AWS, 13 AWNS) were exposed to identical stimuli with lower-frequency iambic stress.

Results

No differences in immediate repetition accuracy for trochaic or iambic nonwords were observed for either group. However, AWS were less accurate when recalling iambic nonwords than trochaic nonwords in the absence of auditory-orthographic cues.

Conclusions

Manipulation of two factors which may minimize phonological demand during standard nonword repetition tasks increased the number of errors in AWS compared to AWNS. These findings suggest greater vulnerability in phonological working memory in AWS, even when producing nonwords as short as two syllables.

Differences in Verbal and Visuospatial Forward and Backward Order Recall: A Review of the Literature

How sequential, verbal and visuospatial stimuli are encoded and stored in memory is not clear in cognitive psychology. Studies with order recall tasks, such as the digit, and Corsi span, indicate that order of presentation is a crucial element for verbal memory, but not for visuospatial memory. This seems to be due to the different effects of forward and backward recall in verbal and visuospatial tasks. In verbal span tasks, performance is worse when recalling things in backward sequence rather than the original forward sequence. In contrast, when it comes to visuospatial tasks, performance is not always worse for a modified backward sequence. However, worse performance in backward visuospatial recall is evident in individuals with weak visuospatial abilities; such individuals perform worse in the backward version of visuospatial tasks than in the forward version. The main aim of the present review is to summarize findings on order recall in verbal and visuospatial materials by considering both cognitive and neural correlates. The results of this review will be considered in the light of the current models of WM, and will be used to make recommendations for future studies.

Impact of Evidence Type and Judicial Warning on Juror Perceptions of Global and Specific Witness Evidence

The Court of Appeal in England and Wales held (R. v. Sardar, 2012) there had been no exceptional circumstances that justified a jury retiring with a transcript of the complainant's interview. This paper reports an investigation into the impact multiple evidence forms and use of a judicial warning has on juror evaluations of a witness. The warning focuses juror attention on placing disproportionate weight on the evidence as opposed to their general impression of it. Sixty jury-eligible participants were presented with witness evidence in transcript, video, or transcript plus video format. Half the participants in each condition received the warning. All mock jurors completed a questionnaire which assessed perceptions of witness and task. Outcomes showed that transcript plus video evidence, when accompanied by a warning, did impact on mock jurors' global assessments of the witness. The warning made the task less clear for jurors and, in the video condition, led to higher ratings of how satisfactory and reliable the witness was. Findings support the provision of a judicial warning to jurors and show some initial support for judiciary opposition to the provision of an additional transcript only when jurors are asked to make the more usual global witness assessments.

Have We Forgotten Auditory Sensory Memory? Retention Intervals in Studies of Nonverbal Auditory Working Memory

Researchers have shown increased interest in mechanisms of working memory for nonverbal sounds such as music and environmental sounds. These studies often have used two-stimulus comparison tasks: two sounds separated by a brief retention interval (often 3-5 s) are compared, and a "same" or "different" judgment is recorded. Researchers seem to have assumed that sensory memory has a negligible impact on performance in auditory two-stimulus comparison tasks. This assumption is examined in detail in this comment. According to seminal texts and recent research reports, sensory memory persists in parallel with working memory for a period of time following hearing a stimulus and can influence behavioral responses on memory tasks. Unlike verbal working memory studies that use serial recall tasks, research paradigms for exploring nonverbal working memory-especially two-stimulus comparison tasks-may not be differentiating working memory from sensory memory processes in analyses of behavioral responses, because retention interval durations have not excluded the possibility that the sensory memory trace drives task performance. This conflation of different constructs may be one contributor to discrepant research findings and the resulting proliferation of theoretical conjectures regarding mechanisms of working memory for nonverbal sounds.