Unraveling the linguistic nature of specific autobiographical memories using a computerized classification algorithm

A natural language model to automate scoring of autobiographical memories

Biases in the retrieval of personal, autobiographical memories are a core feature of multiple mental health disorders, and are associated with poor clinical prognosis. However, current assessments of memory bias are either reliant on human scoring, restricting their administration in clinical settings, or when computerized, are only able to identify one memory type. Here, we developed a natural language model able to classify text-based memories as one of five different autobiographical memory types (specific, categoric, extended, semantic associate, omission), allowing easy assessment of a wider range of memory biases, including reduced memory specificity and impaired memory flexibility. Our model was trained on 17,632 text-based, human-scored memories obtained from individuals with and without experience of memory bias and mental health challenges, which was then tested on a dataset of 5880 memories. We used 20-fold cross-validation setup, and the model was fine-tuned over BERT. Relative to benchmarking and an existing support vector model, our model achieved high accuracy (95.7%) and precision (91.0%). We provide an open-source version of the model which is able to be used without further coding, by those with no coding experience, to facilitate the assessment of autobiographical memory bias in clinical settings, and aid implementation of memory-based interventions within treatment services.

Transdiagnostic and transtherapeutic strategies for optimising autobiographical memory

Our memories for past personally experienced autobiographical events play an important role in therapy, irrespective of presenting issue, diagnoses or therapeutic modality. Here, we summarise evidence for how autobiographical memory abilities can influence our mental health and the relevance of this for the treatment of mental health problems. We then guide the reader through principles and strategies for optimising autobiographical memory within treatment. We ground these recommendations within research for stand-alone interventions for improving autobiographical memory and from studies of how to support the formation and retrieval of therapeutic memories. Options are given for clinicians to guide clients in improving retrieval of autobiographical memories within treatment, for improving autobiographical memory for the therapeutic experience itself, and for creating improvements in autobiographical memory that endure post-treatment. We also provide worksheets for clinicians to use within treatment.

Improving usual care outcomes in major depression in youth by targeting memory specificity: A randomized controlled trial of adjunct computerized memory specificity training (c-MeST)

OBJECTIVE

Extending on previous findings that computerized Memory Specificity Training (c-MeST) improves memory specificity and depressive symptoms in Major Depressive Disorder (MDD) in adults, this study aimed to assess the effects of c-MeST in youth with MDD on memory specificity and depression in addition to other treatment.

METHODS

Participants aged 15-25 (N = 359, 76 % female; M age = 19.2, SD = 3.1), receiving predominantly psychological therapy or counseling (85 %) and/or antidepressants (52 %) were randomized to usual care and c-MeST or usual care. Cognitive and clinical outcomes were assessed at baseline and at one, three, and six-month follow-ups.

RESULTS

The usual care and c-MeST group reported higher memory specificity at one-month (d = 0.42, p = .022), but not at three or six months (d's < 0.15, p's > 0.05). The rate of MDE was numerically lower in the c-MeST group at each follow-up time-point, but group was not a statistically significant predictor at one month (64 % usual care and c-MeST vs. 68 % usual care, OR = 0.81, p = .606), three months (67 % usual care and c-MeST vs. 72 % usual care, OR = 0.64, p = .327) or six months (55 % usual care and c-MeST vs. 68 % usual care, OR = 0.56, p = .266). The usual care and c-MeST group did report lower depressive symptoms at one month (d = 0.42, p = .023) and six-months (d = 0.84, p = .001), but not three-months (d = 0.13, p > .05).

CONCLUSIONS

c-MeST may reduce symptoms in youth with MDD when provided alongside other treatments. However, there are significant limitations to this inference, including high attrition in the study and a need for more data on the acceptability of the intervention.

Automated scoring of the autobiographical interview with natural language processing

The autobiographical interview has been used in more than 200 studies to assess the content of autobiographical memories. In a typical experiment, participants recall memories, which are then scored manually for internal details (episodic details from the central event) and external details (largely non-episodic details). Scoring these narratives requires a significant amount of time. As a result, large studies with this procedure are often impractical, and even conducting small studies is time-consuming. To reduce scoring burden and enable larger studies, we developed an approach to automatically score responses with natural language processing. We fine-tuned an existing language model (distilBERT) to identify the amount of internal and external content in each sentence. These predictions were aggregated to obtain internal and external content estimates for each narrative. We evaluated our model by comparing manual scores with automated scores in five datasets. We found that our model performed well across datasets. In four datasets, we found a strong correlation between internal detail counts and the amount of predicted internal content. In these datasets, manual and automated external scores were also strongly correlated, and we found minimal misclassification of content. In a fifth dataset, our model performed well after additional preprocessing. To make automated scoring available to other researchers, we provide a Colab notebook that is intended to be used without additional coding.

Reducing the stickiness of negative memory retrieval through positive memory training in adolescents

BACKGROUND AND OBJECTIVES

Individuals at risk for depression exhibit a decreased ability to disengage from negative memory retrieval during times of mental distress, partly because they have difficulty retrieving positive memories to repair sad mood. In this study, we tested whether this persistent tendency for negative memory retrieval could be reduced in adolescents through repeated practice to retrieve positive autobiographical memories, namely Positive Memory Specificity Training (PMST). Further, we examined the impact of this intervention on secondary outcomes, including depressive symptoms, emotion regulation strategies, and fear of positive emotions.

METHODS

Adolescents (n = 68) between 16 and 18 years old were randomly allocated to either PMST or bogus control training. Persistent negative memory retrieval was assessed following the training using a behavioral decision-making task (Emotional Reversal Learning Task). Additionally, participants completed self-report measurements (e.g., depressive symptoms) before and two weeks after the training.

RESULTS

We found preliminary supportive evidence for a significant training effect such that adolescents following PMST showed less persistence in negative memory retrieval compared to those in the control group. Only for anhedonia a significant training effect was found, indicating a possible adverse effect of the intervention.

LIMITATIONS

The primary outcome was assessed only at post-intervention to prevent a potential learning effect due to repeated measurements. We cannot exclude the possibility that baseline individual differences contaminated our results. To examine possible adverse effects of PMST, larger sample are needed.

CONCLUSIONS

PMST may help to reduce persistent negative memory retrieval in adolescents. Recommendations for future studies are addressed.

Autobiographical memory and psychopathology: Is memory specificity as important as we make it seem?

Several decades of research have established reduced autobiographical memory specificity, or overgeneral memory, as an important cognitive factor associated with the risk for and maintenance of a range of psychiatric diagnoses. In measuring this construct, experimenters code autobiographical memories for the presence or absence of a single temporal detail that indicates that the remembered event took place on a single, specific, day (Last Thursday when I rode bikes with my son), or multiple days (When I rode bikes with my son). Studies indicate that the specificity of memories and the amount of other episodic detail that they include (e.g., who, what, and where) are related and may rely on the same neural processes to elicit their retrieval. However, specificity and detailedness are nonetheless separable constructs: imperfectly correlated and differentially associated with current and future depressive symptoms and other associated intrapersonal (e.g., rumination) and interpersonal (e.g., social support) outcomes. The ways in which the details of our memories align with narrative themes (i.e., agency, communion, identity) and the coherence with which these details are presented, are also emerging as important factors associated with psychopathology. The temporal specificity of autobiographical memories may be important, but other memory constructs warrant further attention in research and theory, especially given the associations, and dependencies, between each of these constructs. Researchers in this area must consider carefully whether their research questions necessitate a focus on autobiographical memory specificity or whether a more inclusive analysis of other autobiographical memory features is necessary and more fruitful. This article is categorized under: Psychology > Memory.

Machine learning to detect invalid text responses: Validation and comparison to existing detection methods

A crucial step in analysing text data is the detection and removal of invalid texts (e.g., texts with meaningless or irrelevant content). To date, research topics that rely heavily on analysis of text data, such as autobiographical memory, have lacked methods of detecting invalid texts that are both effective and practical. Although researchers have suggested many data quality indicators that might identify invalid responses (e.g., response time, character/word count), few of these methods have been empirically validated with text responses. In the current study, we propose and implement a supervised machine learning approach that can mimic the accuracy of human coding, but without the need to hand-code entire text datasets. Our approach (a) trains, validates, and tests on a subset of texts manually labelled as valid or invalid, (b) calculates performance metrics to help select the best model, and (c) predicts whether unlabelled texts are valid or invalid based on the text alone. Model validation and evaluation using autobiographical memory texts indicated that machine learning accurately detected invalid texts with performance near human coding, significantly outperforming existing data quality indicators. Our openly available code and instructions enable new methods of improving data quality for researchers using text as data.

Natural language processing in clinical neuroscience and psychiatry: A review

Natural language processing (NLP) is rapidly becoming an important topic in the medical community. The ability to automatically analyze any type of medical document could be the key factor to fully exploit the data it contains. Cutting-edge artificial intelligence (AI) architectures, particularly machine learning and deep learning, have begun to be applied to this topic and have yielded promising results. We conducted a literature search for 1,024 papers that used NLP technology in neuroscience and psychiatry from 2010 to early 2022. After a selection process, 115 papers were evaluated. Each publication was classified into one of three categories: information extraction, classification, and data inference. Automated understanding of clinical reports in electronic health records has the potential to improve healthcare delivery. Overall, the performance of NLP applications is high, with an average F1-score and AUC above 85%. We also derived a composite measure in the form of Z-scores to better compare the performance of NLP models and their different classes as a whole. No statistical differences were found in the unbiased comparison. Strong asymmetry between English and non-English models, difficulty in obtaining high-quality annotated data, and train biases causing low generalizability are the main limitations. This review suggests that NLP could be an effective tool to help clinicians gain insights from medical reports, clinical research forms, and more, making NLP an effective tool to improve the quality of healthcare services.

Machine Learning and Natural Language Processing in Mental Health: Systematic Review

BACKGROUND

Machine learning systems are part of the field of artificial intelligence that automatically learn models from data to make better decisions. Natural language processing (NLP), by using corpora and learning approaches, provides good performance in statistical tasks, such as text classification or sentiment mining.

OBJECTIVE

The primary aim of this systematic review was to summarize and characterize, in methodological and technical terms, studies that used machine learning and NLP techniques for mental health. The secondary aim was to consider the potential use of these methods in mental health clinical practice.

METHODS

This systematic review follows the PRISMA (Preferred Reporting Items for Systematic Review and Meta-analysis) guidelines and is registered with PROSPERO (Prospective Register of Systematic Reviews; number CRD42019107376). The search was conducted using 4 medical databases (PubMed, Scopus, ScienceDirect, and PsycINFO) with the following keywords: machine learning, data mining, psychiatry, mental health, and mental disorder. The exclusion criteria were as follows: languages other than English, anonymization process, case studies, conference papers, and reviews. No limitations on publication dates were imposed.

RESULTS

A total of 327 articles were identified, of which 269 (82.3%) were excluded and 58 (17.7%) were included in the review. The results were organized through a qualitative perspective. Although studies had heterogeneous topics and methods, some themes emerged. Population studies could be grouped into 3 categories: patients included in medical databases, patients who came to the emergency room, and social media users. The main objectives were to extract symptoms, classify severity of illness, compare therapy effectiveness, provide psychopathological clues, and challenge the current nosography. Medical records and social media were the 2 major data sources. With regard to the methods used, preprocessing used the standard methods of NLP and unique identifier extraction dedicated to medical texts. Efficient classifiers were preferred rather than transparent functioning classifiers. Python was the most frequently used platform.

CONCLUSIONS

Machine learning and NLP models have been highly topical issues in medicine in recent years and may be considered a new paradigm in medical research. However, these processes tend to confirm clinical hypotheses rather than developing entirely new information, and only one major category of the population (ie, social media users) is an imprecise cohort. Moreover, some language-specific features can improve the performance of NLP methods, and their extension to other languages should be more closely investigated. However, machine learning and NLP techniques provide useful information from unexplored data (ie, patients' daily habits that are usually inaccessible to care providers). Before considering It as an additional tool of mental health care, ethical issues remain and should be discussed in a timely manner. Machine learning and NLP methods may offer multiple perspectives in mental health research but should also be considered as tools to support clinical practice.

The current state of memory Specificity Training (MeST) for emotional disorders

Memory Specificity Training (MeST) is an intervention developed from basic science that has found clinical utility. MeST uses cued recall exercises to target the difficulty that some people with emotional disorders have in recalling personally experienced events. MeST is simple enough to be delivered alongside traditional interventions or online by artificial intelligence. Currently, research indicates MeST's effects are immediate but short-lived, and there is limited research indicating its superiority over established interventions. Future investigations must establish the dosage and specific components of MeST that are necessary for clinically significant effects. Further, it must establish the secondary processes (e.g., problem-solving) that mediate between MeST-driven improvements in memory and symptoms. Similar interventions that build upon the idea of training autobiographical memory specificity are also emerging and warrant further investigation.

Remediating reduced memory specificity in bipolar disorder: A case study using a Computerized Memory Specificity Training

OBJECTIVES

Reduced autobiographical memory specificity (rAMS) is a vulnerability factor found across unipolar depression (UD), posttraumatic stress disorder (PTSD), eating disorder, schizophrenia, and bipolar disorder (BD). A group delivered psychological therapy training called Memory Specificity Training (MeST) remediates rAMS in UD and PTSD, with additional downstream effects on related psychological processes and symptoms. Its impact in BD is unknown. In this case study, we examined the impact of a computerized version of MeST (c-MeST) on improving AMS and related symptoms and processes in participant with rapid cycling type I BD.

METHOD

An experimental case study with an ABA design was used. During baseline (14 days, Phase A), the training phase (nine sessions across 17 days, Phase B), and a 1-month follow-up (Phase A), memory specificity, depressive symptoms, and related processes and symptoms were repeatedly measured.

RESULTS

Memory specificity increased significantly after the participant completed c-MeST. Session-to-session scores indicated that AMS improved most from the in-person baseline assessment to the first online session. All other measures of processes and symptoms deteriorated during the training phase but regressed to baseline during follow-up.

CONCLUSION

Memory specificity was improved as indicated by increased AMS from pre-intervention measurement to 1-month follow-up. Other improvements in symptoms were not observed. Rather, some related maladaptive psychological processes and symptoms worsened during the training phase and regressed to baseline during follow-up.

Efficacy of online Memory Specificity Training in adults with a history of depression, using a multiple baseline across participants design

BACKGROUND AND OBJECTIVES

Memory Specificity Training (MeST), a group training protocol, is effective in improving autobiographical memory specificity (AMS), and in so doing, reducing emotional disorder symptoms amongst clinical groups. We examined MeST's effectiveness when the core component (memory specificity trials) is offered online and individually (c-MeST).

METHODS

A multiple-baseline across-participants design with a randomization-to-baseline length (14 to 33 days) was used. Participants were twenty adults (16 female; M age  = 50, SD = 12) experiencing reduced AMS, at least one lifetime depressive episode and who currently reported at least minimal depressive symptoms. During baseline, the training phase (nine sessions across 17 days) and a three-month follow-up assessment, AMS, depressive symptoms and related processes were measured.

RESULTS

AMS improved significantly by three months follow-up. Session-to-session scores indicated that AMS improved most from baseline to the first online session, with no further improvement thereafter. In contrast to studies with clinical participants, no significant change in symptoms or secondary processes such as rumination was found.

CONCLUSIONS

Translating MeST into an online, individual version is a feasible, low-cost intervention for reduced AMS. Future research should examine c-MeST's potential for preventing increases in symptoms in at-risk samples with longer follow-ups as well as its potential for reducing symptoms in clinical groups.

Remediating Reduced Autobiographical Memory in Healthy Older Adults With Computerized Memory Specificity Training (c-MeST): An Observational Before-After Study

Background

The ability to retrieve specific autobiographical memories decreases with cognitive aging. This decline is clinically relevant due to its association with impairments in problem solving, daily functioning, and depression. A therapist-delivered group training protocol, Memory Specificity Training (MeST), has been shown to enhance the retrieval of specific memories while ameliorating the impairments and negative outcomes associated with reduced specificity. The therapist-delivered nature of this intervention means it is relatively expensive to deliver and difficult for people with mobility impairments, such as older people, to receive.

Objective

The objective of this study was to test if a novel, Web-based computerized version of a group training protocol called Memory Specificity Training, has the potential to increase autobiographical memory specificity and impact associated secondary psychological processes.

Methods

A total of 21 participants (13 female; mean age 67.05, SD 6.55) who experienced a deficit in retrieving specific autobiographical memory were trained with c-MeST. We assessed memory specificity at preintervention and postintervention, as well as secondary processes such as depressive symptoms, rumination, and problem-solving skills.

Results

Memory specificity increased significantly after participants completed c-MeST (r=.57). Session-to-session scores indicated that autobiographical memory specificity improved most from the online baseline assessment to the first Web-based session. Symptoms or secondary processes such as problem-solving skills did not change significantly.

Conclusions

A Web-based automated individual version of MeST is a feasible, low-cost intervention for reduced memory specificity in healthy older adults. Future studies should clarify the preventive impact of c-MeST in other at-risk sample populations with longer follow-up times.

Computerised memory specificity training (c-MeST) for the treatment of major depression: a study protocol for a randomised controlled trial

INTRODUCTION

Major depression is a prevalent and debilitating disorder, but many sufferers do not receive support or respond to current treatments. The development of easily accessible and low-intensity treatments that have clear cognitive mechanisms of change is indicated. Memory specificity training (MeST) is an intervention for depression that targets deficits in recalling detailed memories of past experiences through repeated practice of autobiographical memory retrieval. This randomised controlled trial will assess the efficacy of an online, computerised version of MeST (c-MeST).

METHODS AND ANALYSIS

Adults aged 18 and over with a current major depressive episode (MDE) will be recruited and randomised to have access to the seven session, online c-MeST programme for 2 weeks, or to a wait-list control group. The primary outcomes will be diagnostic status of MDE and self-reported depressive symptoms at postintervention. One-month and three-month follow-ups will be collected. Increases in autobiographical memory specificity will be assessed as a mediator of change, as well as other variables thought to contribute to reduced memory specificity, such as rumination and cognitive avoidance.

ETHICS AND DISSEMINATION

Ethics approval has been granted by the Deakin University Human Research Ethics Committee to conduct the study (ID: 2017_168). The findings will be disseminated through scholarly publications and workshops and will inform future trials, such as with an active comparator or as an adjunct treatment.

TRIAL REGISTRATION NUMBER

ACTRN12618000257268; Pre-results.

Autobiographical memory coherence and specificity: Examining their reciprocal relation and their associations with internalizing symptoms and rumination

Autobiographical memories consist of different features that have been shown to relate to psychological well-being and psychopathology. Two such characteristics show quite some overlap, namely memory coherence and memory specificity, although their association has never been investigated before. In this study, we examined the association between memory coherence and memory specificity in a sample of first-year psychology students. Additionally, to gain more insight into the relation between memory coherence and psychopathology, we investigated the association with known correlates of memory specificity, namely internalizing symptoms and rumination. We found that narrating about personal experiences in a coherent manner is related to retrieving more specific memories. However, the association between memory coherence and memory specificity was rather weak. Furthermore, we found that memory coherence was negatively associated with the level of depressive symptoms and could predict these symptoms even after controlling for memory specificity and rumination. Given the potential clinical importance of these findings, future research should focus on examining the specific circumstances in which memory coherence is related to psychopathology, and on mechanisms that could explain this association.

The transportability of Memory Specificity Training (MeST): adapting an intervention derived from experimental psychology to routine clinical practices

Background

Accumulating evidence shows that a cognitive factor associated with a worsening of depressive symptoms amongst people with and without diagnoses of depression – reduced Autobiographical Memory (rAMS) – can be ameliorated by a group cognitive training protocol referred to as Memory Specificity Training (MeST). When transporting interventions such as MeST from research to routine clinical practices (RCPs), modifications are inevitable, with potentially a decrease in effectiveness, so called voltage drop. We examined the transportability of MeST to RCPs as an add-on to treatment as usual with depressed in- and out- patients.

Methods

We examined whether 1) MeST was adaptable to local needs of RCPs by implementing MeST in a joint decision-making process in seven Belgian RCPs 2) without losing its effect on rAMS. The effectiveness of MeST was measured by pre- and post- intervention measurements of memory specificity.

Results

Adaptations were made to the MeST protocol to optimize the fit with RCPs. Local needs of RCPs were met by dismantling MeST into different subparts. By dismantling it in this way, we were able to address several challenges raised by clinicians. In particular, multidisciplinary teams could divide the workload across different team members and, for the open version of MeST, the intervention could be offered continuously with tailored dosing per patient. Both closed and open versions of MeST, with or without peripheral components, and delivered by health professionals with different backgrounds, resulted in a significant increase in memory specificity for depressed in- and out- patients in RCPs.

Conclusions

MeST is shown to be a transportable and adaptable add-on intervention which effectively maintains its core mechanism when delivered in RCPs.

Trial registration

ISRCTN registry, IDISRCTN10144349, registered on January 22, 2019. Retrospectively registered.

Electronic supplementary material

The online version of this article (10.1186/s40359-019-0279-y) contains supplementary material, which is available to authorized users.

The computerized scoring algorithm for the autobiographical memory test: updates and extensions for analyzing memories of English-speaking adults

The Autobiographical Memory Test (AMT) has been central in psychopathological studies of memory dysfunctions, as reduced memory specificity or overgeneralised autobiographical memory has been recognised as a hallmark vulnerability for depression. In the AMT, participants are asked to generate specific memories in response to emotional cue words, and their responses are scored by human experts. Because the manual coding takes some time, particularly when analysing a large dataset, recent studies have proposed computerised scoring algorithms. These algorithms have been shown to reliably discriminate between specific and non-specific memories of English-speaking children and Dutch- and Japanese-speaking adults. The key limitation is that the algorithm is not developed for English-speaking adult memories, which may cover a wider range of vocabulary that the existing algorithm for English-speaking child memories cannot process correctly. In the present study, we trained a new support vector machine to score memories of English-speaking adults. In a performance test (predicting memory specificity against human expert coding), the adult-memory algorithm outperformed the child-memory variant. In another independent performance test, the adult-memory algorithm showed robust performances to score memories that were generated in response to a different set of cues. These results suggest that the adult-memory algorithm reliably scores memory specificity.

Clinical Natural Language Processing in languages other than English: opportunities and challenges

Background

Natural language processing applied to clinical text or aimed at a clinical outcome has been thriving in recent years. This paper offers the first broad overview of clinical Natural Language Processing (NLP) for languages other than English. Recent studies are summarized to offer insights and outline opportunities in this area.

Main Body

We envision three groups of intended readers: (1) NLP researchers leveraging experience gained in other languages, (2) NLP researchers faced with establishing clinical text processing in a language other than English, and (3) clinical informatics researchers and practitioners looking for resources in their languages in order to apply NLP techniques and tools to clinical practice and/or investigation. We review work in clinical NLP in languages other than English. We classify these studies into three groups: (i) studies describing the development of new NLP systems or components de novo, (ii) studies describing the adaptation of NLP architectures developed for English to another language, and (iii) studies focusing on a particular clinical application.

Conclusion

We show the advantages and drawbacks of each method, and highlight the appropriate application context. Finally, we identify major challenges and opportunities that will affect the impact of NLP on clinical practice and public health studies in a context that encompasses English as well as other languages.

Making Sense of Big Textual Data for Health Care: Findings from the Section on Clinical Natural Language Processing

Objectives: To summarize recent research and present a selection of the best papers published in 2016 in the field of clinical Natural Language Processing (NLP). Method: A survey of the literature was performed by the two section editors of the IMIA Yearbook NLP section. Bibliographic databases were searched for papers with a focus on NLP efforts applied to clinical texts or aimed at a clinical outcome. Papers were automatically ranked and then manually reviewed based on titles and abstracts. A shortlist of candidate best papers was first selected by the section editors before being peer-reviewed by independent external reviewers. Results: The five clinical NLP best papers provide a contribution that ranges from emerging original foundational methods to transitioning solid established research results to a practical clinical setting. They offer a framework for abbreviation disambiguation and coreference resolution, a classification method to identify clinically useful sentences, an analysis of counseling conversations to improve support to patients with mental disorder and grounding of gradable adjectives. Conclusions: Clinical NLP continued to thrive in 2016, with an increasing number of contributions towards applications compared to fundamental methods. Fundamental work addresses increasingly complex problems such as lexical semantics, coreference resolution, and discourse analysis. Research results translate into freely available tools, mainly for English.