Rosenthal S, Das S, Hsueh PYS, Barker K, Chen CH. Efficient goal attainment and engagement in a care manager system using unstructured notes.
JAMIA Open 2020;
3:ooaa001. [PMID:
32142137 PMCID:
PMC7309242 DOI:
10.1093/jamiaopen/ooaa001]
[Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2019] [Revised: 11/22/2019] [Accepted: 01/28/2020] [Indexed: 11/14/2022] Open
Abstract
OBJECTIVE
To improve efficient goal attainment of patients by analyzing the unstructured text in care manager (CM) notes (CMNs). Our task is to determine whether the goal assigned by the CM can be achieved in a timely manner.
MATERIALS AND METHODS
Our data consists of CM structured and unstructured records from a private firm in Orlando, FL. The CM data is based on phone interactions between the CM and the patient. A portion of the data has been manually annotated to indicate engagement. We present 2 machine learning classifiers: an engagement model and a goal attainment model.
RESULTS
We can successfully distinguish automatically between engagement and lack of engagement. Subsequently, incorporating engagement and features from textual information from the unstructured notes significantly improves goal attainment classification.
DISCUSSION
Two key challenges in this task were the time-consuming annotation effort for engagement classification and the limited amount of data for the more difficult goal attainment class (specifically, for people who take a long time to achieve their goals). We successfully explore domain adaptation and transfer learning techniques to improve performance on the under-represented classes. We also explore the value of using features from unstructured notes to improve the model and interpretability.
CONCLUSIONS
Unstructured CMNs can be used to improve accuracy of our classification models for predicting patient self-management goal attainment. This work can be used to help identify patients who may require special attention from CMs to improve engagement in self-management.
Collapse