Use of a Machine Learning Program to Correctly Triage Incoming Text Messaging Replies From a Cardiovascular Text-Based Secondary Prevention Program: Feasibility Study

“Not by Our Feeling, But by Other's Seeing”: Sentiment Analysis Technique in Cardiology—An Exploratory Review

Sentiment Analysis (SA) is a novel branch of Natural Language Processing (NLP) that measures emotions or attitudes behind a written text. First applications of SA in healthcare were the detection of disease-related emotional polarities in social media. Now it is possible to extract more complex attitudes (rank attitudes from 1 to 5, assign appraisal values, apply multiple text classifiers) or feelings through NLP techniques, with clear benefits in cardiology; as emotions were proved to be veritable risk factors for the development of cardiovascular diseases (CVD). Our narrative review aimed to summarize the current directions of SA in cardiology and raise the awareness of cardiologists about the potentiality of this novel domain. This paper introduces the readers to basic concepts surrounding medical SA and the need for SA in cardiovascular healthcare. Our synthesis of the current literature proved SA's clinical potential in CVD. However, many other clinical utilities, such as the assessment of emotional consequences of illness, patient-physician relationship, physician intuitions in CVD are not yet explored. These issues constitute future research directions, along with proposing detailed regulations, popularizing health social media among elders, developing insightful definitions of emotional polarity, and investing research into the development of powerful SA algorithms.

Primary care physicians' electronic health record proficiency and efficiency behaviors and time interacting with electronic health records: a quantile regression analysis

OBJECTIVE

This study aimed to understand the association between primary care physician (PCP) proficiency with the electronic health record (EHR) system and time spent interacting with the EHR.

MATERIALS AND METHODS

We examined the use of EHR proficiency tools among PCPs at one large academic health system using EHR-derived measures of clinician EHR proficiency and efficiency. Our main predictors were the use of EHR proficiency tools and our outcomes focused on 4 measures assessing time spent in the EHR: (1) total time spent interacting with the EHR, (2) time spent outside scheduled clinical hours, (3) time spent documenting, and (4) time spent on inbox management. We conducted multivariable quantile regression models with fixed effects for physician-level factors and time in order to identify factors that were independently associated with time spent in the EHR.

RESULTS

Across 441 primary care physicians, we found mixed associations between certain EHR proficiency behaviors and time spent in the EHR. Across EHR activities studied, QuickActions, SmartPhrases, and documentation length were positively associated with increased time spent in the EHR. Models also showed a greater amount of help from team members in note writing was associated with less time spent in the EHR and documenting.

DISCUSSION

Examining the prevalence of EHR proficiency behaviors may suggest targeted areas for initial and ongoing EHR training. Although documentation behaviors are key areas for training, team-based models for documentation and inbox management require further study.

CONCLUSIONS

A nuanced association exists between physician EHR proficiency and time spent in the EHR.

Text Message Analysis Using Machine Learning to Assess Predictors of Engagement With Mobile Health Chronic Disease Prevention Programs: Content Analysis

BACKGROUND

SMS text messages as a form of mobile health are increasingly being used to support individuals with chronic diseases in novel ways that leverage the mobility and capabilities of mobile phones. However, there are knowledge gaps in mobile health, including how to maximize engagement.

OBJECTIVE

This study aims to categorize program SMS text messages and participant replies using machine learning (ML) and to examine whether message characteristics are associated with premature program stopping and engagement.

METHODS

We assessed communication logs from SMS text message-based chronic disease prevention studies that encouraged 1-way (SupportMe/ITM) and 2-way (TEXTMEDS [Text Messages to Improve Medication Adherence and Secondary Prevention]) communication. Outgoing messages were manually categorized into 5 message intents (informative, instructional, motivational, supportive, and notification) and replies into 7 groups (stop, thanks, questions, reporting healthy, reporting struggle, general comment, and other). Grid search with 10-fold cross-validation was implemented to identify the best-performing ML models and evaluated using nested cross-validation. Regression models with interaction terms were used to compare the association of message intent with premature program stopping and engagement (replied at least 3 times and did not prematurely stop) in SupportMe/ITM and TEXTMEDS.

RESULTS

We analyzed 1550 messages and 4071 participant replies. Approximately 5.49% (145/2642) of participants responded with stop, and 11.7% (309/2642) of participants were engaged. Our optimal ML model correctly classified program message intent with 76.6% (95% CI 63.5%-89.8%) and replies with 77.8% (95% CI 74.1%-81.4%) balanced accuracy (average area under the curve was 0.95 and 0.96, respectively). Overall, supportive (odds ratio [OR] 0.53, 95% CI 0.35-0.81) messages were associated with reduced chance of stopping, as were informative messages in SupportMe/ITM (OR 0.35, 95% CI 0.20-0.60) but not in TEXTMEDS (for interaction, P<.001). Notification messages were associated with a higher chance of stopping in SupportMe/ITM (OR 5.76, 95% CI 3.66-9.06) but not TEXTMEDS (for interaction, P=.01). Overall, informative (OR 1.76, 95% CI 1.46-2.12) and instructional (OR 1.47, 95% CI 1.21-1.80) messages were associated with higher engagement but not motivational messages (OR 1.18, 95% CI 0.82-1.70; P=.37). For supportive messages, the association with engagement was opposite with SupportMe/ITM (OR 1.77, 95% CI 1.21-2.58) compared with TEXTMEDS (OR 0.77, 95% CI 0.60-0.98; for interaction, P<.001). Notification messages were associated with reduced engagement in SupportMe/ITM (OR 0.07, 95% CI 0.05-0.10) and TEXTMEDS (OR 0.28, 95% CI 0.20-0.39); however, the strength of the association was greater in SupportMe/ITM (for interaction P<.001).

CONCLUSIONS

ML models enable monitoring and detailed characterization of program messages and participant replies. Outgoing message intent may influence premature program stopping and engagement, although the strength and direction of association appear to vary by program type. Future studies will need to examine whether modifying message characteristics can optimize engagement and whether this leads to behavior change.