Comparing Themes Extracted via Topic Modeling and Manual Content Analysis: Korean-Language Discussions of Dementia on Twitter

We randomly examined Korean-language Tweets mentioning dementia/Alzheimer's disease (n= 12,413) posted from November 28 to December 9, 2020, without limiting geographical locations. We independently applied Latent Dirichlet Allocation (LDA) topic modeling and qualitative content analysis to the texts of the Tweets. We compared the themes extracted by LDA topic modeling to those identified via manual coding methods. A total of 16 themes were detected from manual coding, with inter-rater reliability (Cohen's kappa) of 0.842. The proportions of the most prominent themes were: burdens of family caregiving (48.50%), reports of wandering/missing family members with dementia (18.12%), stigma (13.64%), prevention strategies (5.07%), risk factors (4.91%), healthcare policy (3.26%), and elder abuse/safety issues (1.75%). Seven themes whose contents were similar to themes derived from manual coding were extracted from the LDA topic modeling results (perplexity: -6.39, coherence score: 0.45). Our findings suggest that applying LDA topic modeling can be fairly effective at extracting themes from Korean Twitter discussions, in a manner analogous to qualitative coding, to gain insights regarding caregiving for family members with dementia, and our approach can be applied to other languages.

Comparing Emotional Valence Scores of Twitter Posts from Manual Coding and Machine Learning Algorithms to Gain Insights to Refine Interventions for Family Caregivers of Persons with Dementia

We randomly extracted Korean-language Tweets mentioning dementia/Alzheimer's disease (n= 12,413) from November 28 to December 9, 2020. We independently applied three machine learning algorithms (Afinn, Syuzhet, and Bing) using natural language processing (NLP) techniques and qualitative manual scoring to assign emotional valence scores to Tweets. We then compared the means and distributions of the four emotional valence scores. Visual examination of the graphs produced indicated that each method exhibited unique patterns. The aggregated mean emotional valence scores from the NLP methods were mostly neutral, vs. slightly negative for manual coding (Afinn 0.029, 95% CI [-0.019, 0.077]; Syuzhet 0.266, [0.236, 0.295]; Bing -0.271, [-0.289, -0.252]; manual coding -1.601, [-1.632, -1.569]). One-way analysis of variance (ANOVA) showed no statistically significant differences among the four means after normalization. These findings suggest that the application of NLP can be fairly effective in extracting emotional valence scores from Korean-language Twitter content to gain insights regarding family caregiving for a person with dementia.