Increase in Cancer Center Staff Effort Related to Electronic Patient Portal Use

The Growing Role of Digital Health Tools in the Care of Patients with Cancer: Current Use, Future Opportunities, and Barriers to Effective Implementation

PURPOSE OF REVIEW

This article aims to describe the ways in which digital health technologies are currently being used to improve the delivery of cancer care, highlight opportunities to expand their use, and discuss barriers to effective and equitable implementation.

RECENT FINDINGS

The utilization of digital health tools and development of novel care delivery models that leverage such tools is expanding. Recent studies have shown feasibility and increased implementation in the setting of oncologic care. With technological advances and key policy changes, utilization of digital health tools has greatly increased over the past two decades and transformed how cancer care is delivered. As digital health tools are expanded and refined, there is potential for improved access to and quality and efficiency of cancer care. However, careful consideration should be given to key barriers of digital health tool adoption, such as infrastructural, patient-level, and health systems-level challenges, to ensure equitable access to care and improvement in health outcomes.

Taking It to the States: Adapting Information Blocking Legislation to Oncology

Oncologists have successfully pushed for new state laws on how we release life-changing test results

"You've Got Mail"-Receiving Bad News Through a Patient Portal

This Viewpoint discusses the impersonalization of delivering bad news to patients through a patient portal rather than in person.

Understanding the Cures Act Information Blocking Rule in cancer care: a mixed methods exploration of patient and clinician perspectives and recommendations for policy makers

BACKGROUND

The 21^st Century Cures Act Interoperability and Information Blocking Rule was created to increase patient access to health information. This federally mandated policy has been met with praise and concern. However, little is known about patient and clinician opinions of this policy within cancer care.

METHODS

We conducted a convergent parallel mixed methods study to understand patient and clinician reactions to the Information Blocking Rule in cancer care and what they would like policy makers to consider. Twenty-nine patients and 29 clinicians completed interviews and surveys. Inductive thematic analysis was used to analyze the interviews. Interview and survey data were analyzed separately, then linked to generate a full interpretation of the results.

RESULTS

Overall, patients felt more positive about the policy than clinicians. Patients wanted policy makers to understand that patients are unique, and they want to individualize their preferences for receiving health information with their clinicians. Clinicians highlighted the uniqueness of cancer care, due to the highly sensitive information that is shared. Both patients and clinicians were concerned about the impact on clinician workload and stress. Both expressed an urgent need for tailoring implementation of the policy to avoid unintended harm and distress for patients.

CONCLUSIONS

Our findings provide suggestions for optimizing the implementation of this policy in cancer care. Dissemination strategies to better inform the public about the policy and improve clinician understanding and support are recommended. Patients who have serious illness or diagnoses such as cancer and their clinicians should be included when developing and enacting policies that could have a significant impact on their well-being. Patients with cancer and their cancer care teams want the ability to tailor information release based on individual preferences and goals. Understanding how to tailor implementation of the Information Blocking Rule is essential for retaining its benefits and minimizing unintended harm for patients with cancer.

Quantifying Patient Portal Use: Systematic Review of Utilization Metrics

BACKGROUND

Use of patient portals has been associated with positive outcomes in patient engagement and satisfaction. Portal studies have also connected portal use, as well as the nature of users' interactions with portals, and the contents of their generated data to meaningful cost and quality outcomes. Incentive programs in the United States have encouraged uptake of health information technology, including patient portals, by setting standards for meaningful use of such technology. However, despite widespread interest in patient portal use and adoption, studies on patient portals differ in actual metrics used to operationalize and track utilization, leading to unsystematic and incommensurable characterizations of use. No known review has systematically assessed the measurements used to investigate patient portal utilization.

OBJECTIVE

The objective of this study was to apply systematic review criteria to identify and compare methods for quantifying and reporting patient portal use.

METHODS

Original studies with quantifiable metrics of portal use published in English between 2014 and the search date of October 17, 2018, were obtained from PubMed using the Medical Subject Heading term "Patient Portals" and related keyword searches. The first search round included full text review of all results to confirm a priori data charting elements of interest and suggest additional categories inductively; this round was supplemented by the retrieval of works cited in systematic reviews (based on title screening of all citations). An additional search round included broader keywords identified during the full-text review of the first round. Second round results were screened at abstract level for inclusion and confirmed by at least two raters. Included studies were analyzed for metrics related to basic use/adoption, frequency of use, duration metrics, intensity of use, and stratification of users into "super user" or high utilizers. Additional categories related to provider (including care team/administrative) use of the portal were identified inductively. Additional analyses included metrics aligned with meaningful use stage 2 (MU-2) categories employed by the US Centers for Medicare and Medicaid Services and the association between the number of portal metrics examined and the number of citations and the journal impact factor.

RESULTS

Of 315 distinct search results, 87 met the inclusion criteria. Of the a priori metrics, plus provider use, most studies included either three (26 studies, 30%) or four (23 studies, 26%) metrics. Nine studies (10%) only reported the patient use/adoption metric and only one study (1%) reported all six metrics. Of the US-based studies (n=76), 18 (24%) were explicitly motivated by MU-2 compliance; 40 studies (53%) at least mentioned these incentives, but only 6 studies (8%) presented metrics from which compliance rates could be inferred. Finally, the number of metrics examined was not associated with either the number of citations or the publishing journal's impact factor.

CONCLUSIONS

Portal utilization measures in the research literature can fall below established standards for "meaningful" or they can substantively exceed those standards in the type and number of utilization properties measured. Understanding how patient portal use has been defined and operationalized may encourage more consistent, well-defined, and perhaps more meaningful standards for utilization, informing future portal development.

Participants' views and experiences from setting up a shared patient portal for primary and specialist health services- a qualitative study

Background

Recently, there has been an increasing focus among healthcare organisations on implementing patient portals. Previous studies have mainly focussed on the experiences of patient portal use. Few have investigated the processes of deciding what content and features to make available, in particular for shared portals across healthcare domains. The aim of the study was to investigate views on content and experiences from the configuration process among participants involved in setting up a shared patient portal for primary and specialist health services.

Methods

A qualitative study including 15 semi-structured interviews with persons participating in patient portal configuration was conducted from October 2019 to June 2020.

Results

Whether a shared patient portal for all the health services in the region should be established was not questioned by any of the informants. It was experienced as a good thing to have numerous participants present in the discussions on configuration, but it also was said to increase the complexity of the work. The informants considered a patient portal to be of great value for patient care, among other things because it would lead to improvements in patient follow-up and increased patient empowerment. Nevertheless, some informants advocated caution as they thought the patient portal possibly could lead to an increase in healthcare providers’ workloads and to anxiety and worries, as well as to inequality in access to health care among patients. The findings were categorized into the themes ‘A tool for increased patient involvement’, ‘Which information should be available for the patient’, ‘Concerns about increased workload’, ‘Too complex to use versus not interesting enough’, ‘Involving all services’ and ‘Patient involvement’.

Conclusions

Establishing a shared patient portal for primary and specialist health services was considered unproblematic. There was, however, variation in opinions on which content and features to include. This variation was related to concerns about increasing the workload for health care providers, causing anxiety and inequality among patients, and ensuring that the solution would be interesting enough to adopt.

The impact of a comprehensive electronic patient portal on the health service use: an interrupted time-series analysis

BACKGROUND

There is little empirical research on the potential benefit that electronic patient portals (EPP) can have on the care quality and health outcomes of diverse multi-ethnic international populations. The purpose of this study is to determine the extent to which an EPP was associated with improvements in health service use.

METHODS

Using a quasi-experimental interrupted time-series approach, we assessed health service use before (April 2012-September 2015) and after (October 2015-December 2016) the implementation of a comprehensive EPP at four hospitals in Madrid, Spain. Primary outcomes were number of outpatient visits, any hospital admission, any 30-day all-cause readmission and any emergency department visit.

RESULTS

Implementation of the EPP was associated with a significant decline in readmissions. Among patients with chronic heart failure, EPP implementation was associated with a significant decline for all outcome measures, and among patients with COPD, a decline in all outcomes except readmissions. Among patients diagnosed with malignant hematological diseases, no significant changes were identified.

CONCLUSIONS

EPPs hold promise for reducing hospital readmissions. Certain patient populations with chronic conditions may differentially benefit from portal use depending on their needs for communication with their providers.

A Retrospective Analysis of Provider-to-Patient Secure Messages: How Much Are They Increasing, Who Is Doing the Work, and Is the Work Happening After Hours?

BACKGROUND

Patient portal registration and the use of secure messaging are increasing. However, little is known about how the work of responding to and initiating patient messages is distributed among care team members and how these messages may affect work after hours.

OBJECTIVE

This study aimed to examine the growth of secure messages and determine how the work of provider responses to patient-initiated secure messages and provider-initiated secure messages is distributed across care teams and across work and after-work hours.

METHODS

We collected secure messages sent from providers from January 1, 2013, to March 15, 2018, at Mayo Clinic, Rochester, Minnesota, both in response to patient secure messages and provider-initiated secure messages. We examined counts of messages over time, how the work of responding to messages and initiating messages was distributed among health care workers, messages sent per provider, messages per unique patient, and when the work was completed (proportion of messages sent after standard work hours).

RESULTS

Portal registration for patients having clinic visits increased from 33% to 62%, and increasingly more patients and providers were engaged in messaging. Provider message responses to individual patients increased significantly in both primary care and specialty practices. Message responses per specialty physician provider increased from 15 responses per provider per year to 53 responses per provider per year from 2013 to 2018, resulting in a 253% increase. Primary care physician message responses increased from 153 per provider per year to 322 from 2013 to 2018, resulting in a 110% increase. Physicians, nurse practitioners, physician assistants, and registered nurses, all contributed to the substantial increases in the number of messages sent.

CONCLUSIONS

Provider-sent secure messages at a large health care institution have increased substantially since implementation of secure messaging between patients and providers. The effort of responding to and initiating messages to patients was distributed across multiple provider categories. The percentage of message responses occurring after hours showed little substantial change over time compared with the overall increase in message volume.

Full Radiology Report through Patient Web Portal: A Literature Review

The aim of this study discusses the gap between the patient web portal and providing a full radiology report. A literature review was conducted to examine radiologists, physicians, and patients’ opinions and preferences of providing patients with online access radiology reports. The databases searched were Pubmed and Google Scholar and the initial search included 927 studies. After review, 47 studies were included in the study. We identified several themes, including patients’ understanding of radiology reports and radiological images, as well as the need for decreasing the turnaround time for reports availability. The existing radiology reports written for physicians are not suited for patients. Further studies are needed to guide and inform the design of patient friendly radiology reports. One of the ways that can be used to fill the gap between patients and radiology reports is using social media sites.

Patient and provider perspectives on delivery of oral cancer therapies

OBJECTIVE

The introduction of oral cancer therapies presents new challenges to delivery of quality cancer care. Little is known about how patients and providers address and overcome these challenges. We conducted a qualitative study exploring the range of patient and provider perspectives on oral cancer therapies.

METHODS

We conducted semi-structured interviews with patients and providers at a tertiary referral center and county safety-net hospital in Dallas, TX. Interviews probed perspectives on differences between parenteral chemotherapy and oral therapies, adherence, communication, and cost/insurance. Interview transcripts were analyzed thematically using a deductively-driven coding scheme corresponding to the interview guide.

RESULTS

We conducted 22 patient (13 at tertiary referral center, 9 at safety-net hospital) and 10 provider (7 oncologists, 2 nurses, 1 pharmacist) interviews. Key themes from interviews included: (1) differences in parenteral chemotherapy vs. oral therapy; (2) adherence and dosing; and (3) experiences related to cost and communication.

CONCLUSIONS

Nearly all providers described challenges engaging with and educating patients about oral cancer therapies. Despite our initial hypothesis, safety-net patients encountered few barriers accessing oral therapies compared to patients receiving care in the tertiary referral center.

PRACTICE IMPLICATIONS

Our findings will guide future interventions to monitor and support cancer patients receiving oral therapies.

Identifying How Patient Portals Impact Communication in Oncology

Patient portals are becoming widespread throughout health-care systems. Initial research has demonstrated that they positively impact patient-provider communication and patients' health knowledge, but little is known about the impact of patient portals in the cancer setting, where highly complex and uncertain medical data are available for patients to view. To better understand communicative behaviors and perceptions of the patient portal and how it is utilized in oncology, in-depth, semi-structured interviews were conducted with 48 participants: 35 patients and 13 oncologists. Thematic analysis identified that portals help to enhance participation during in-person consultations, increase patients' self-advocacy, and build rapport with providers. However, patients' comfort level with reviewing information via the portal depended upon the severity of the test. Oncologists worried about patient anxiety and widening health disparities, but acknowledged that the portal can motivate them to expedite communication about laboratory and scan results. As patient portals become more widely used in all medical settings, oncologists should become more engaged with how patients are viewing their medical information and consider the portal within the framework of patient-centered care by valuing patients' communication preferences.

Developing a patient portal for haematology patients requires involvement of all stakeholders and a customised design, tailored to the individual needs

Background

Electronic patient portals are increasingly being implemented, also in (haemato) oncology. However, portal usage is low and depends on user and provider engagement. We explored wishes, expectations and thoughts of patients with a haematologic malignancy and their physicians with regard to the electronic patient portal.

Methods

Based on insights from literature and a focus group discussion we built a 44-item questionnaire. This questionnaire was spread amongst patients with a haematologic malignancy at the outpatient clinic that was not yet exposed to patient portal facilities. Haematologists completed a questionnaire based on literature.

Results

Patients were interested in many different types of access to information and portal functionalities. However, their opinions varied about the provision of access to the portal to other people, the role of the physician, possibilities for communication via the portal and timing of access. The physicians acknowledged the relevance of the electronic patient portal, but had some worries about the patients’ autonomous information handling, organizational and technical issues. Patients frequently expressed to be open about the potential of the patient portal to orchestrate their care. Nevertheless, most physicians appreciated their supporting role towards the patient.

Conclusions

Patients and physicians appreciated the electronic patient portal. Both groups need to be involved in further portal development to improve engagement by meeting patients’ wishes, taking into account organizational and professional issues and managing expectations for both parties. To fit various patient profiles, portal design should be flexible and individualized. Further research should focus on the perceived added value and the impact on patient related outcome measures of portals.

Electronic supplementary material

The online version of this article (10.1186/s12911-019-0868-y) contains supplementary material, which is available to authorized users.

Secure Messaging Through PositiveLinks: Examination of Electronic Communication in a Clinic-Affiliated Smartphone App for Patients Living with HIV

Purpose: Secure messaging between patients and their health care team can facilitate chronic care management. PositiveLinks^® (PL) is a clinic-affiliated smartphone application designed for patients living with HIV that includes a secure messaging feature for patients, PL staff, and clinic providers to communicate. Our aim was to examine the content and function of messaging within PL.

Methodology: We examined messages exchanged through PL from November 2017 through January 2018. Qualitative analysis included categorization of topics as: related to the app, medical care, or social needs. Messaging functions were categorized as information exchange or rapport building.

Results: Of the 1,474 PL messages analyzed, 44% were sent by PL staff, 38% by patients, and 18% by providers, whereas 61% were received by patients, 22% by providers, and 17% by PL staff. Message topics included app-related (57.6%), medical care (34.3%), and social concerns (12.4%). App-related messages addressed technical difficulties, software updates, or coordinating phone payments. Medical messages included medical information, medications, appointments, outreach, and care coordination for physical and mental health. Social messages related to insurance, transportation, housing, food, utilities, disability, finances, and work absences. Message function coding showed that 87.3% of messages contained information exchange and 33.8% contained rapport building. Messages sent by providers were most likely to contain rapport building at 54.8%.

Conclusion: PL messaging was used to handle medical and social needs with potential impact on patients' health and offers an opportunity to strengthen patient–provider relationships through responsiveness and rapport building. Secure messaging through a clinic-affiliated smartphone app could enhance patient-centered care between clinical visits.

OpenNotes in oncology: oncologists' perceptions and a baseline of the content and style of their clinician notes

Patients' ability to access their provider's clinical notes (OpenNotes) has been well received and has led to greater transparency in health systems. However, the majority of this research has occurred in primary care, and little is known about how patients' access to notes is used in oncology. This study aims to understand oncologists' perceptions of OpenNotes, while also establishing a baseline of the linguistic characteristics and patterns used in notes. Data from 13 in-depth, semistructured interviews with oncologists were thematically analyzed. In addition, the Linguistic Inquiry and Word Count (LIWC) program evaluated over 200 clinician notes, measuring variables encompassing emotions, thinking styles, social concerns, and parts of speech. Analysis from LIWC revealed that notes contained negative emotional tone, low authenticity, high clout, and high analytical writing. Oncologists' use of stigmatized and sensitive words, such as "obese" and "distress," was mainly absent. Themes from interviews revealed that oncologists were uncertain about patients' access to their notes and may edit their notes to avoid problematic terminology. Despite their reluctance to embrace OpenNotes, they envisioned opportunities for an improved patient-provider relationship due to patients initiating interactions from viewing notes. Oncologists believe notes are not intended for patients and altering their content may compromise the integrity of the note. This study established a baseline for further study to compare notes pre-implementation to post-implementation. Further analysis will clarify whether oncologists are altering the style and content of their notes and determine the presence of patient-centered language.

Mobile Device Applications for Electronic Patient Portals in Oncology

PURPOSE

Mobile devices provide individuals with rapid and frequent access to electronic patient portals. We investigated how oncology patients use this technology to review test results and communicate with providers.

PATIENTS AND METHODS

We performed a retrospective study of patients enrolled in the MyChart electronic health portal associated with the Epic electronic medical record at the Harold C. Simmons Comprehensive Cancer Center from 2012 to 2017. We recorded type of portal access according to year and patient characteristics. Associations among patient characteristics and types of portal access were tested using Mann-Whitney U test, χ2 test, and linear Gaussian regression models.

RESULTS

Since the availability of a mobile device application in 2012, 2,524 patients with cancer accessed MyChart from a mobile device at least once, which accounted for 291,526 mobile log-ins. The number of patients with MyChart mobile application log-ins increased from 4% in 2012 to 13% in 2017 ( P = .004). Among these patients, the median proportion of log-ins that occurred through mobile device use increased from 22% to 72% during this time period ( P < .001). Mobile access occurred more frequently among younger ( P < .001), black ( P = .002), and Hispanic ( P = .004) patients. Since 2012, total portal log-in frequency increased approximately 110% among patients who used the mobile application compared with 25% among those who did not use the mobile application ( P < .001).

CONCLUSION

Mobile access to electronic health portals has increased patient portal use, particularly among traditionally underserved populations. How this widely and immediately available technology affects patient expectations and experiences warrants additional study.

It Takes Two to Tango: Engaging Patients and Providers With Portals

Patient portals are designed to be tools to more fully engage patients in their health care and help enable them to better manage their own health information. As the U.S. health care system rapidly adopted electronic health records (EHRs) over the past decade, many with linked patient portals, enthusiasm and expectations for this new technology as a means to engage and empower patients grew. Most patient portals have a set of core features designed to facilitate health care transactions, information tracking, and communication with care team members. The evidence supporting the anticipated benefits of patient portals on patient outcomes, however, remains mixed and incomplete. Moreover, a paradox exists in that, despite a high consumer interest in patient portals, widespread adoption remains relatively low. Potential reasons include the need for greater provider endorsement, examination and adaptation of clinical workflows, and the recognition of patient engagement as a reciprocal process.

Optimizing the User Experience: Identifying Opportunities to Improve Use of an Inpatient Portal

BACKGROUND

Patient portals specifically designed for the inpatient setting have significant potential to improve patient care. However, little is known about how the users of this technology, the patients, may interact with the inpatient portals. As a result, hospitals have limited ability to design approaches that support patient use of the portal.

OBJECTIVES

This study aims to evaluate the user experience associated with an inpatient portal.

METHODS

We used a Think-Aloud protocol to study user interactions with a commercially available inpatient portal-MyChart Bedside (MCB). Study participants included 19 English-speaking adults over the age of 18 years. In one-on-one sessions, participants narrated their experience using the MCB application and completing eight specific tasks. Recordings were transcribed and coded into three dimensions of the user experience: physical, cognitive, and sociobehavioral.

RESULTS

Our analysis of the physical experience highlighted the navigational errors and technical challenges associated with the use of MCB. We also found that issues associated with the cognitive experience included comprehension problems that spurred anxiety and uncertainty. Analysis of the sociobehavioral experience suggested that users have different learning styles and preferences for learning including self-guided, handouts, and in-person training.

CONCLUSION

Inpatient portals may be an effective tool to improve the patient experience in the hospital. Moreover, making this technology available to inpatients may help to foster ongoing use of technology across the care continuum. However, deriving the benefits from the technology requires appropriate support. We identified multiple opportunities for hospital management to intervene. In particular, teaching patients to use the application by making a variety of instructional materials available could help to reduce several identified barriers to use. Additionally, hospitals should be prepared to manage patient anxiety and increased questioning arising from the availability of information in the inpatient portal application.

A Review of Web Portal Use by Oncology Patients

BACKGROUND

Patient portals and other Internet-based technologies have been increasingly used to improve cancer care coordination. Patient portals may introduce special considerations in oncology populations where longitudinal outpatient care is often more intensive than in most other specialties.

METHODS

This article, which is based upon bibliographic searches in PubMed, reviews the literature on web portal use by cancer patients. Articles published in English from 2000 to August 2018 were identified using the following MeSH search terms and Boolean algebra commands: web portal AND cancer. Information obtained from bibliographic searches (title and topic of article, information in abstract, and keywords) was used to determine whether to retain each article identified in this way.

RESULTS

A total of 263 article citations were identified in the bibliographic searches. Of these, 10 met the eligibility criteria. A variety of study designs were used including focus groups, usability testing, in-person interviews, questionnaire surveys, retrospective cohort, and non-randomized trial. Cancer patients had reached modest levels of portal use. Increased portal use has been associated with younger age, white race, and higher socioeconomic status. Most cancer patients used portals to look up testing results and provide notes, but had difficulty in interpreting the results appropriately.

CONCLUSIONS

Our study adds to the growing evidence that patient portals play a significant role in promoting self-management in cancer survivors. Additional studies are needed to determine factors influencing portal use, so effective interventions can be developed to enhance portal use.

Cancer Center Clinic and Research Team Perceptions of Identity and Interactions

PURPOSE

Conduct of cancer clinical trials requires coordination and cooperation among research and clinic teams. Diffusion of and confusion about responsibility may occur if team members' perceptions of roles and objectives do not align. These factors are critical to the success of cancer centers but are poorly studied.

METHODS

We developed a survey adapting components of the Adapted Team Climate Inventory, Measure of Team Identification, and Measure of In-Group Bias. Surveys were administered to research and clinic staff at a National Cancer Institute-designated comprehensive cancer center. Data were analyzed using descriptive statistics, t tests, and analyses of variance.

RESULTS

Responses were received from 105 staff (clinic, n = 55; research, n = 50; 61% response rate). Compared with clinic staff, research staff identified more strongly with their own group ( P < .01) but less strongly with the overall cancer center ( P = .02). Both clinic staff and research staff viewed their own group's goals as clearer than those of the other group ( P < .01) and felt that members of their groups interacted and shared information within ( P < .01) and across ( P < .01) groups more than the other group did. Research staff perceived daily outcomes as more important than did clinic staff ( P = .05), specifically research-related outcomes ( P = .07).

CONCLUSION

Although there are many similarities between clinic and research teams, we also identified key differences, including perceptions of goal clarity and sharing, understanding and alignment with cancer center goals, and importance of outcomes. Future studies should examine how variation in perceptions and group dynamics between clinic and research teams may impact function and processes of cancer care.

A comparison of rule-based and machine learning approaches for classifying patient portal messages

OBJECTIVE

Secure messaging through patient portals is an increasingly popular way that consumers interact with healthcare providers. The increasing burden of secure messaging can affect clinic staffing and workflows. Manual management of portal messages is costly and time consuming. Automated classification of portal messages could potentially expedite message triage and delivery of care.

MATERIALS AND METHODS

We developed automated patient portal message classifiers with rule-based and machine learning techniques using bag of words and natural language processing (NLP) approaches. To evaluate classifier performance, we used a gold standard of 3253 portal messages manually categorized using a taxonomy of communication types (i.e., main categories of informational, medical, logistical, social, and other communications, and subcategories including prescriptions, appointments, problems, tests, follow-up, contact information, and acknowledgement). We evaluated our classifiers' accuracies in identifying individual communication types within portal messages with area under the receiver-operator curve (AUC). Portal messages often contain more than one type of communication. To predict all communication types within single messages, we used the Jaccard Index. We extracted the variables of importance for the random forest classifiers.

RESULTS

The best performing approaches to classification for the major communication types were: logistic regression for medical communications (AUC: 0.899); basic (rule-based) for informational communications (AUC: 0.842); and random forests for social communications and logistical communications (AUCs: 0.875 and 0.925, respectively). The best performing classification approach of classifiers for individual communication subtypes was random forests for Logistical-Contact Information (AUC: 0.963). The Jaccard Indices by approach were: basic classifier, Jaccard Index: 0.674; Naïve Bayes, Jaccard Index: 0.799; random forests, Jaccard Index: 0.859; and logistic regression, Jaccard Index: 0.861. For medical communications, the most predictive variables were NLP concepts (e.g., Temporal_Concept, which maps to 'morning', 'evening' and Idea_or_Concept which maps to 'appointment' and 'refill'). For logistical communications, the most predictive variables contained similar numbers of NLP variables and words (e.g., Telephone mapping to 'phone', 'insurance'). For social and informational communications, the most predictive variables were words (e.g., social: 'thanks', 'much', informational: 'question', 'mean').

CONCLUSIONS

This study applies automated classification methods to the content of patient portal messages and evaluates the application of NLP techniques on consumer communications in patient portal messages. We demonstrated that random forest and logistic regression approaches accurately classified the content of portal messages, although the best approach to classification varied by communication type. Words were the most predictive variables for classification of most communication types, although NLP variables were most predictive for medical communication types. As adoption of patient portals increases, automated techniques could assist in understanding and managing growing volumes of messages. Further work is needed to improve classification performance to potentially support message triage and answering.