Comparisons of Self-Report With Objective Measurements Suggest Faster Responding but Little Change in Response Quality Over Time in Ecological Momentary Assessment Studies

Response times (RTs) to ecological momentary assessment (EMA) items often decrease after repeated EMA administration, but whether this is accompanied by lower response quality requires investigation. We examined the relationship between EMA item RTs and EMA response quality. In one data set, declining response quality was operationalized as decreasing correspondence over time between subjective and objective measures of blood glucose taken at the same time. In a second EMA study data set, declining response quality was operationalized as decreasing correspondence between subjective ratings of memory test performance and objective memory test scores. We assumed that measurement error in the objective measures did not increase across time, meaning that decreasing correspondence across days within a person could be attributed to lower response quality. RTs to EMA items decreased across study days, while no decrements in the mean response quality were observed. Decreasing EMA item RTs across study days did not appear problematic overall.

A meta-cognitive Wisconsin Card Sorting Test in people with schizophrenia and bipolar disorder: Self-assessment of sorting performance

People with serious mental illness have challenged self-awareness, including momentary monitoring of performance. A core feature of this challenge is in the domain of using external information to guide behavior, an ability that is measured very well by certain problem-solving tasks such as the Wisconsin Card Sorting Test (WCST) . We used a modified WCST to examine correct sorts and accuracy decisions regarding the correctness of sort. Participants with schizophrenia (n = 99) or bipolar disorder (n = 76) sorted 64 cards and then made judgments regarding correctness of each sort prior to feedback. Time series analyses examined the course of correct sorts and correct accuracy decisions by examining the momentary correlation and lagged correlation on the next sort. People with schizophrenia had fewer correct sorts, fewer categories, and fewer correct accuracy decisions (all p<.001). Positive response biases were seen in both groups. After an incorrect sort or accuracy decision, the groups were equally likely to be incorrect on the next sort or accuracy decision. Following correct accuracy decisions, participants with bipolar disorder were significantly (p=.003) more likely to produce a correct sort or accuracy decision. These data are consistent with previous studies implicating failures to consider external feedback for decision making. Interventions aimed at increasing consideration of external information during decision making have been developed and interventions targeting use of feedback during cognitive test performance are in development.

Confidence, accuracy judgments and feedback in schizophrenia and bipolar disorder: a time series network analysis

BACKGROUND

Inaccurate self-assessment of performance is common among people with serious mental illness, and it is associated with poor functional outcomes independent from ability. However, the temporal interdependencies between judgments of performance, confidence in accuracy, and feedback about performance are not well understood.

METHODS

We evaluated two tasks: the Wisconsin Card Sorting Test (WCST) and the Penn Emotion recognition task (ER40). These tasks were modified to include item-by-item confidence and accuracy judgments, along with feedback on accuracy. We evaluated these tasks as time series and applied network modeling to understand the temporal relationships between momentary confidence, accuracy judgments, and feedback. The sample constituted participants with schizophrenia (SZ; N = 144), bipolar disorder (BD; N = 140), and healthy controls (HC; N = 39).

RESULTS

Network models for both WCST and ER40 revealed denser and lagged connections between confidence and accuracy judgments in SZ and, to a lesser extent in BD, that were not evidenced in HC. However, associations between feedback regarding accuracy with subsequent accuracy judgments and confidence were weaker in SZ and BD. In each of these comparisons, the BD group was intermediate between HC and SZ. In analyses of the WCST, wherein incorporating feedback is crucial for success, higher confidence predicted worse subsequent performance in SZ but not in HC or BD.

CONCLUSIONS

While network models are exploratory, the results suggest some potential mechanisms by which challenges in self-assessment may impede performance, perhaps through hyperfocus on self-generated judgments at the expense of incorporation of feedback.

Childhood trauma and real-world social experiences in psychosis

BACKGROUND

Childhood trauma is associated with a variety of negative outcomes in psychosis, but it is unclear clear if childhood trauma affects day-to-day social experiences. We aimed to examine the association between childhood trauma and functional and structural characteristics of real-world social relationships in psychosis.

METHODS

Participants with psychotic disorders or affective disorders with psychosis completed ecological momentary assessments (EMAs) over ten days (N = 209). Childhood trauma was assessed retrospectively using the Childhood Trauma Questionnaire. Associations between childhood trauma and EMA-assessed social behavior and perceptions were examined using linear mixed models. Analyses were adjusted for sociodemographic characteristics and psychotic and depressive symptom severity.

RESULTS

Higher levels of childhood trauma were associated with more perceived threat (B = -0.19, 95 % CI [-0.33, -0.04]) and negative self-perception (B = -0.18, 95 % CI [-0.34, -0.01]) during recent social interactions, as well as reduced social motivation (B = -0.29, 95 % CI [-0.47, -0.10]), higher desire for social avoidance (B = 0.34, 95 % CI [0.14, 0.55]), and lower sense of belongingness (B = -0.24, 95 % CI [-0.42, -0.06]). These negative social perceptions were mainly linked with emotional abuse and emotional neglect. In addition, paranoia was more strongly associated with negative social perceptions in individuals with high versus low levels of trauma. Childhood trauma was not associated with frequency (i.e., time spent alone) or type of social interactions.

CONCLUSION

Childhood trauma - particularly emotional abuse and neglect - is associated with negative social perceptions but not frequency of real-world social interactions. Our findings suggest that childhood trauma may affect day-to-day social experiences beyond its association with psychosis.

Dynamics of task-based confidence in schizophrenia using seasonal decomposition approach

Objective

Introspective Accuracy (IA) is a metacognitive construct that refers to alignment of self-generated accuracy judgments, confidence, and objective information regarding performance. IA not only refers to accuracy and confidence during tasks, but also predicts functional outcomes. The consistency and magnitude of IA deficits suggest a sustained disconnect between self-assessments and actual performance. The cognitive origins of IA are unclear and are not simply due to poor performance. We tried to capture task and diagnosis-related differences through examining confidence as a timeseries.

Method

This relatively large sample (N = 171; Bipolar = 71, Schizophrenia = 100) study used item by item confidence judgments for tasks including the Wisconsin Card Sorting Task (WCST) and the Emotion Recognition task (ER-40). Using a seasonal decomposition approach and AutoRegressive, Integrative and Moving Averages (ARIMA) time-series analyses we tested for the presence of randomness and perseveration.

Results

For the WCST, comparisons across participants with schizophrenia and bipolar disorder found similar trends and residuals, thus excluding perseverative or random responding. However, seasonal components were weaker in participants with schizophrenia, reflecting a reduced impact of feedback on confidence. In contrast, for the ER40, which does not require identification of a sustained construct, seasonal, trend, and residual analyses were highly comparable.

Conclusion

Seasonal analysis revealed that confidence judgments in participants with schizophrenia on tasks requiring responses to feedback reflected diminished incorporation of external information, not random or preservative responding. These analyses highlight how time series analyses can specify potential faulty processes for future intervention.

Automated Analysis of Drawing Process to Estimate Global Cognition in Older Adults: Preliminary International Validation on the US and Japan Data Sets

BACKGROUND

With the aging of populations worldwide, early detection of cognitive impairments has become a research and clinical priority, particularly to enable preventive intervention for dementia. Automated analysis of the drawing process has been studied as a promising means for lightweight, self-administered cognitive assessment. However, this approach has not been sufficiently tested for its applicability across populations.

OBJECTIVE

The aim of this study was to evaluate the applicability of automated analysis of the drawing process for estimating global cognition in community-dwelling older adults across populations in different nations.

METHODS

We collected drawing data with a digital tablet, along with Montreal Cognitive Assessment (MoCA) scores for assessment of global cognition, from 92 community-dwelling older adults in the United States and Japan. We automatically extracted 6 drawing features that characterize the drawing process in terms of the drawing speed, pauses between drawings, pen pressure, and pen inclinations. We then investigated the association between the drawing features and MoCA scores through correlation and machine learning-based regression analyses.

RESULTS

We found that, with low MoCA scores, there tended to be higher variability in the drawing speed, a higher pause:drawing duration ratio, and lower variability in the pen's horizontal inclination in both the US and Japan data sets. A machine learning model that used drawing features to estimate MoCA scores demonstrated its capability to generalize from the US dataset to the Japan dataset (R²=0.35; permutation test, P<.001).

CONCLUSIONS

This study presents initial empirical evidence of the capability of automated analysis of the drawing process as an estimator of global cognition that is applicable across populations. Our results suggest that such automated analysis may enable the development of a practical tool for international use in self-administered, automated cognitive assessment.

Dynamics of Loneliness Among Older Adults During the COVID-19 Pandemic: Pilot Study of Ecological Momentary Assessment With Network Analysis

OBJECTIVE

The COVID-19 pandemic has had potentially severe psychological implications for older adults, including those in retirement communities, due to restricted social interactions, but the day-to-day experience of loneliness has received limited study. We sought to investigate sequential association, if any, between loneliness, activity, and affect.

METHODS

We used ecological momentary assessment (EMA) with dynamic network analysis to investigate the affective and behavioral concomitants of loneliness in 22 residents of an independent living sector of a continuing care retirement community (mean age 80.2; range 68-93 years).

RESULTS

Participants completed mean 83.9% of EMA surveys (SD = 16.1%). EMA ratings of loneliness were moderately correlated with UCLA loneliness scale scores. Network models showed that loneliness was contemporaneously associated with negative affect (worried, anxious, restless, irritable). Negative (but not happy or positive) mood tended to be followed by loneliness and then by exercise or outdoor physical activity. Negative affect had significant and high inertia (stability).

CONCLUSIONS

The data suggest that EMA is feasible and acceptable to older adults. EMA-assessed loneliness was moderately associated with scale-assessed loneliness. Network models in these independent living older adults indicated strong links between negative affect and loneliness, but feelings of loneliness were followed by outdoor activity, suggesting adaptive behavior among relatively healthy adults.

Text mining for modeling of protein complexes enhanced by machine learning

MOTIVATION

Procedures for structural modeling of protein-protein complexes (protein docking) produce a number of models which need to be further analyzed and scored. Scoring can be based on independently determined constraints on the structure of the complex, such as knowledge of amino acids essential for the protein interaction. Previously, we showed that text mining of residues in freely available PubMed abstracts of papers on studies of protein-protein interactions may generate such constraints. However, absence of post-processing of the spotted residues reduced usability of the constraints, as a significant number of the residues were not relevant for the binding of the specific proteins.

RESULTS

We explored filtering of the irrelevant residues by two machine learning approaches, Deep Recursive Neural Network (DRNN) and Support Vector Machine (SVM) models with different training/testing schemes. The results showed that the DRNN model is superior to the SVM model when training is performed on the PMC-OA full-text articles and applied to classification (interface or non-interface) of the residues spotted in the PubMed abstracts. When both training and testing is performed on full-text articles or on abstracts, the performance of these models is similar. Thus, in such cases, there is no need to utilize computationally demanding DRNN approach, which is computationally expensive especially at the training stage. The reason is that SVM success is often determined by the similarity in data/text patterns in the training and the testing sets, whereas the sentence structures in the abstracts are, in general, different from those in the full text articles.

AVAILABILITYAND IMPLEMENTATION

The code and the datasets generated in this study are available at https://gitlab.ku.edu/vakser-lab-public/text-mining/-/tree/2020-09-04.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Do Words Matter? Detecting Social Isolation and Loneliness in Older Adults Using Natural Language Processing

Introduction: Social isolation and loneliness (SI/L) are growing problems with serious health implications for older adults, especially in light of the COVID-19 pandemic. We examined transcripts from semi-structured interviews with 97 older adults (mean age 83 years) to identify linguistic features of SI/L. Methods: Natural Language Processing (NLP) methods were used to identify relevant interview segments (responses to specific questions), extract the type and number of social contacts and linguistic features such as sentiment, parts-of-speech, and syntactic complexity. We examined: (1) associations of NLP-derived assessments of social relationships and linguistic features with validated self-report assessments of social support and loneliness; and (2) important linguistic features for detecting individuals with higher level of SI/L by using machine learning (ML) models. Results: NLP-derived assessments of social relationships were associated with self-reported assessments of social support and loneliness, though these associations were stronger in women than in men. Usage of first-person plural pronouns was negatively associated with loneliness in women and positively associated with emotional support in men. ML analysis using leave-one-out methodology showed good performance (F1 = 0.73, AUC = 0.75, specificity = 0.76, and sensitivity = 0.69) of the binary classification models in detecting individuals with higher level of SI/L. Comparable performance were also observed when classifying social and emotional support measures. Using ML models, we identified several linguistic features (including use of first-person plural pronouns, sentiment, sentence complexity, and sentence similarity) that most strongly predicted scores on scales for loneliness and social support. Discussion: Linguistic data can provide unique insights into SI/L among older adults beyond scale-based assessments, though there are consistent gender differences. Future research studies that incorporate diverse linguistic features as well as other behavioral data-streams may be better able to capture the complexity of social functioning in older adults and identification of target subpopulations for future interventions. Given the novelty, use of NLP should include prospective consideration of bias, fairness, accountability, and related ethical and social implications.

The Gut Microbiome, Aging, and Longevity: A Systematic Review

Aging is determined by complex interactions among genetic and environmental factors. Increasing evidence suggests that the gut microbiome lies at the core of many age-associated changes, including immune system dysregulation and susceptibility to diseases. The gut microbiota undergoes extensive changes across the lifespan, and age-related processes may influence the gut microbiota and its related metabolic alterations. The aim of this systematic review was to summarize the current literature on aging-associated alterations in diversity, composition, and functional features of the gut microbiota. We identified 27 empirical human studies of normal and successful aging suitable for inclusion. Alpha diversity of microbial taxa, functional pathways, and metabolites was higher in older adults, particularly among the oldest-old adults, compared to younger individuals. Beta diversity distances significantly differed across various developmental stages and were different even between oldest-old and younger-old adults. Differences in taxonomic composition and functional potential varied across studies, but Akkermansia was most consistently reported to be relatively more abundant with aging, whereas Faecalibacterium, Bacteroidaceae, and Lachnospiraceae were relatively reduced. Older adults have reduced pathways related to carbohydrate metabolism and amino acid synthesis; however, oldest-old adults exhibited functional differences that distinguished their microbiota from that of young-old adults, such as greater potential for short-chain fatty acid production and increased butyrate derivatives. Although a definitive interpretation is limited by the cross-sectional design of published reports, we integrated findings of microbial composition and downstream functional pathways and metabolites, offering possible explanations regarding age-related processes.

Prediction of Loneliness in Older Adults Using Natural Language Processing: Exploring Sex Differences in Speech

OBJECTIVE

The growing pandemic of loneliness has great relevance to aging populations, though assessments are limited by self-report approaches. This paper explores the use of artificial intelligence (AI) technology to evaluate interviews on loneliness, notably, employing natural language processing (NLP) to quantify sentiment and features that indicate loneliness in transcribed speech text of older adults.

DESIGN

Participants completed semi-structured qualitative interviews regarding the experience of loneliness and a quantitative self-report scale (University of California Los Angeles or UCLA Loneliness scale) to assess loneliness. Lonely and non-lonely participants (based on qualitative and quantitative assessments) were compared.

SETTING

Independent living sector of a senior housing community in San Diego County.

PARTICIPANTS

Eighty English-speaking older adults with age range 66-94 (mean 83 years).

MEASUREMENTS

Interviews were audiotaped and manually transcribed. Transcripts were examined using NLP approaches to quantify sentiment and expressed emotions.

RESULTS

Lonely individuals (by qualitative assessments) had longer responses with greater expression of sadness to direct questions about loneliness. Women were more likely to endorse feeling lonely during the qualitative interview. Men used more fearful and joyful words in their responses. Using linguistic features, machine learning models could predict qualitative loneliness with 94% precision (sensitivity = 0.90, specificity = 1.00) and quantitative loneliness with 76% precision (sensitivity = 0.57, specificity = 0.89).

CONCLUSIONS

AI (e.g., NLP and machine learning approaches) can provide unique insights into how linguistic features of transcribed speech data may reflect loneliness. Eventually linguistic features could be used to assess loneliness of individuals, despite limitations of commercially developed natural language understanding programs.

Blind prediction of homo- and hetero-protein complexes: The CASP13-CAPRI experiment

We present the results for CAPRI Round 46, the third joint CASP-CAPRI protein assembly prediction challenge. The Round comprised a total of 20 targets including 14 homo-oligomers and 6 heterocomplexes. Eight of the homo-oligomer targets and one heterodimer comprised proteins that could be readily modeled using templates from the Protein Data Bank, often available for the full assembly. The remaining 11 targets comprised 5 homodimers, 3 heterodimers, and two higher-order assemblies. These were more difficult to model, as their prediction mainly involved "ab-initio" docking of subunit models derived from distantly related templates. A total of ~30 CAPRI groups, including 9 automatic servers, submitted on average ~2000 models per target. About 17 groups participated in the CAPRI scoring rounds, offered for most targets, submitting ~170 models per target. The prediction performance, measured by the fraction of models of acceptable quality or higher submitted across all predictors groups, was very good to excellent for the nine easy targets. Poorer performance was achieved by predictors for the 11 difficult targets, with medium and high quality models submitted for only 3 of these targets. A similar performance "gap" was displayed by scorer groups, highlighting yet again the unmet challenge of modeling the conformational changes of the protein components that occur upon binding or that must be accounted for in template-based modeling. Our analysis also indicates that residues in binding interfaces were less well predicted in this set of targets than in previous Rounds, providing useful insights for directions of future improvements.

Natural language processing in text mining for structural modeling of protein complexes

Background

Structural modeling of protein-protein interactions produces a large number of putative configurations of the protein complexes. Identification of the near-native models among them is a serious challenge. Publicly available results of biomedical research may provide constraints on the binding mode, which can be essential for the docking. Our text-mining (TM) tool, which extracts binding site residues from the PubMed abstracts, was successfully applied to protein docking (Badal et al., PLoS Comput Biol, 2015; 11: e1004630). Still, many extracted residues were not relevant to the docking.

Results

We present an extension of the TM tool, which utilizes natural language processing (NLP) for analyzing the context of the residue occurrence. The procedure was tested using generic and specialized dictionaries. The results showed that the keyword dictionaries designed for identification of protein interactions are not adequate for the TM prediction of the binding mode. However, our dictionary designed to distinguish keywords relevant to the protein binding sites led to considerable improvement in the TM performance. We investigated the utility of several methods of context analysis, based on dissection of the sentence parse trees. The machine learning-based NLP filtered the pool of the mined residues significantly more efficiently than the rule-based NLP. Constraints generated by NLP were tested in docking of unbound proteins from the DOCKGROUND X-ray benchmark set 4. The output of the global low-resolution docking scan was post-processed, separately, by constraints from the basic TM, constraints re-ranked by NLP, and the reference constraints. The quality of a match was assessed by the interface root-mean-square deviation. The results showed significant improvement of the docking output when using the constraints generated by the advanced TM with NLP.

Conclusions

The basic TM procedure for extracting protein-protein binding site residues from the PubMed abstracts was significantly advanced by the deep parsing (NLP techniques for contextual analysis) in purging of the initial pool of the extracted residues. Benchmarking showed a substantial increase of the docking success rate based on the constraints generated by the advanced TM with NLP.

Electronic supplementary material

The online version of this article (10.1186/s12859-018-2079-4) contains supplementary material, which is available to authorized users.

Modeling CAPRI targets 110-120 by template-based and free docking using contact potential and combined scoring function

The paper presents analysis of our template-based and free docking predictions in the joint CASP12/CAPRI37 round. A new scoring function for template-based docking was developed, benchmarked on the Dockground resource, and applied to the targets. The results showed that the function successfully discriminates the incorrect docking predictions. In correctly predicted targets, the scoring function was complemented by other considerations, such as consistency of the oligomeric states among templates, similarity of the biological functions, biological interface relevance, etc. The scoring function still does not distinguish well biological from crystal packing interfaces, and needs further development for the docking of bundles of α-helices. In the case of the trimeric targets, sequence-based methods did not find common templates, despite similarity of the structures, suggesting complementary use of structure- and sequence-based alignments in comparative docking. The results showed that if a good docking template is found, an accurate model of the interface can be built even from largely inaccurate models of individual subunits. Free docking however is very sensitive to the quality of the individual models. However, our newly developed contact potential detected approximate locations of the binding sites.

Text Mining for Protein Docking

The rapidly growing amount of publicly available information from biomedical research is readily accessible on the Internet, providing a powerful resource for predictive biomolecular modeling. The accumulated data on experimentally determined structures transformed structure prediction of proteins and protein complexes. Instead of exploring the enormous search space, predictive tools can simply proceed to the solution based on similarity to the existing, previously determined structures. A similar major paradigm shift is emerging due to the rapidly expanding amount of information, other than experimentally determined structures, which still can be used as constraints in biomolecular structure prediction. Automated text mining has been widely used in recreating protein interaction networks, as well as in detecting small ligand binding sites on protein structures. Combining and expanding these two well-developed areas of research, we applied the text mining to structural modeling of protein-protein complexes (protein docking). Protein docking can be significantly improved when constraints on the docking mode are available. We developed a procedure that retrieves published abstracts on a specific protein-protein interaction and extracts information relevant to docking. The procedure was assessed on protein complexes from Dockground (http://dockground.compbio.ku.edu). The results show that correct information on binding residues can be extracted for about half of the complexes. The amount of irrelevant information was reduced by conceptual analysis of a subset of the retrieved abstracts, based on the bag-of-words (features) approach. Support Vector Machine models were trained and validated on the subset. The remaining abstracts were filtered by the best-performing models, which decreased the irrelevant information for ~ 25% complexes in the dataset. The extracted constraints were incorporated in the docking protocol and tested on the Dockground unbound benchmark set, significantly increasing the docking success rate.

Protein interactions are central for many cellular processes. Physical characterization of these interactions is essential for understanding of life processes and applications in biology and medicine. Because of the inherent limitations of experimental techniques and rapid development of computational power and methodology, computer modeling is a tool of choice in many studies. Publicly available information from biomedical research is readily accessible on the Internet, providing a powerful resource for modeling of proteins and protein complexes. A major paradigm shift in modeling of protein complexes is emerging due to the rapidly expanding amount of such information, which can be used as modeling constraints. Text mining has been widely used in recreating networks of protein interactions, as well as in detecting small molecule binding sites on proteins. Combining and expanding these two well-developed areas of research, we applied the text mining to physical modeling of protein complexes (protein docking). Our procedure retrieves published abstracts on a protein-protein interaction and extracts the relevant information. The results show that correct information on binding can be obtained for about half of protein complexes. The extracted constraints were incorporated in a modeling procedure, significantly improving its performance.