New insights about community benefit evaluation: Using the Community Health Implementation Evaluation Framework to assess what hospitals are measuring

BACKGROUND

Nonprofit hospitals are required to conduct community health needs assessments (CHNA) every 3 years and develop corresponding implementation plans. Implemented strategies must address the identified community needs and be evaluated for impact.

PURPOSE

Using the Community Health Implementation Evaluation Framework (CHIEF), we assessed whether and how nonprofit hospitals are evaluating the impact of their CHNA-informed community benefit initiatives.

METHODOLOGY

We conducted a content analysis of 83 hospital CHNAs that reported evaluation outcomes drawn from a previously identified 20% random sample ( n = 613) of nonprofit hospitals in the United States. Through qualitative review guided by the CHIEF, we identified and categorized the most common evaluation outcomes reported.

RESULTS

A total of 485 strategies were identified from the 83 hospitals' CHNAs. Evaluated strategies most frequently targeted behavioral health ( n = 124, 26%), access ( n = 83, 17%), and obesity/nutrition/inactivity ( n = 68, 14%). The most common type of evaluation outcomes reported by CHIEF category included system utilization ( n = 342, 71%), system implementation ( n = 170, 35%), project management ( n = 164, 34%), and social outcomes ( n = 163, 34%).

PRACTICE IMPLICATIONS

CHNA evaluation strategies focus on utilization (the number of individuals served), with few focusing on social or health outcomes. This represents a missed opportunity to (a) assess the social and health impacts across individual strategies and (b) provide insight that can be used to inform the allocation of limited resources to maximize the impact of community benefit strategies.

Tele-Oncology Use During the COVID-19 Pandemic: Patient Experiences and Communication Behaviors with Clinicians

Background: Tele-oncology became a widely used tool during the COVID-19 pandemic, but there was limited understanding of how patient-clinician communication occurred using the technology. Our goal was to identify how communication transpired during tele-oncology consultations compared with in-person appointments. Methods: A convergent parallel mixed-method design was utilized for the web-based survey, and follow-up interviews were conducted with cancer patients from March to December 2020. Participants were recruited from the University of Florida Health Cancer Center and two national cancer organizations. During the survey, participants rated their clinician's patient-centered communication behaviors. Open-ended survey responses and interview data were combined and analyzed thematically using the constant comparative method. Results: A total of 158 participants completed the survey, and 33 completed an interview. Ages ranged from 19 to 88 years (mean = 64.2; standard deviation = 13.0); 53.2% identified as female and 44.9% as male. The majority of respondents (76%) considered communication in tele-oncology equal to in-person visits. Preferences for tele-oncology included the ability to get information from the clinician, with 13.5% rating tele-oncology as better than in-person appointments. Tele-oncology was considered worse than in-person appointments for eye contact (n = 21, 12.4%) and virtual waiting room times (n = 50, 29.4%). The following qualitative themes corresponded with several quantitative variables: (1) commensurate to in-person appointments, (2) uncertainty with the digital platform, (3) lack of a personal connection, and (4) enhanced patient experience. Conclusion: Patient-centered communication behaviors were mostly viewed as equally prevalent during tele-oncology and in-person appointments. Addressing the challenges of tele-oncology is necessary to improve the patient experience.

Health-related social needs information in the emergency department: clinician and patient perspectives on availability and use

BACKGROUND

Patient health-related social needs (HRSN) complicate care and drive poor outcomes in emergency department (ED) settings. This study sought to understand what HRSN information is available to ED physicians and staff, and how HRSN-related clinical actions may or may not align with patient expectations.

METHODS

We conducted a qualitative study using in-depth semi-structured interviews guided by HRSN literature, the 5 Rights of Clinical Decision Support (CDS) framework, and the Contextual Information Model. We asked ED providers, ED staff, and ED patients from one health system in the mid-Western United Stated about HRSN information availability during an ED encounter, HRSN data collection, and HRSN data use. Interviews were recorded, transcribed, and analyzed using modified thematic approach.

RESULTS

We conducted 24 interviews (8 per group: ED providers, ED staff, and ED patients) from December 2022 to May 2023. We identified three themes: (1) Availability: ED providers and staff reported that HRSNs information is inconsistently available. The availability of HRSN data is influenced by patient willingness to disclose it during an encounter. (2) Collection: ED providers and staff preferred and predominantly utilized direct conversation with patients to collect HRSNs, despite other methods being available to them (e.g., chart review, screening questionnaires). Patients' disclosure preferences were based on modality and team member. (3) Use: Patients wanted to be connected to relevant resources to address their HRSNs. Providers and staff altered clinical care to account for or accommodate HRSNs. System-level challenges (e.g., limited resources) limited provider and staff ability to address patients HRSNs.

CONCLUSIONS

In the ED, HRSNs information was inconsistently available, collected, or disclosed. Patients and ED providers and staff differed in their perspectives on how HSRNs should be collected and acted upon. Accounting for such difference in clinical and administrative decisions will be critical for patient acceptance and effective usage of HSRN information.

Management of Fall Risk Among Older Adults in Diverse Primary Care Settings

OBJECTIVES

Falls are persistent among community-dwelling older adults despite existing prevention guidelines. We described how urban and rural primary care staff and older adults manage fall risk and factors important to integration of computerized clinical decision support (CCDS).

METHODS

Interviews, contextual inquiries, and workflow observations were analyzed using content analysis and synthesized into a journey map. Sociotechnical and PRISM domains were applied to identify workflow factors important to sustainable CCDS integration.

RESULTS

Participants valued fall prevention and described similar approaches. Available resources differed between rural and urban locations. Participants wanted evidence-based guidance integrated into workflows to bridge skills gaps.

DISCUSSION

Sites described similar clinical approaches with differences in resource availability. This implies that a single intervention would need to be flexible to environments with differing resources. Electronic Health Record's inherent ability to provide tailored CCDS is limited. However, CCDS middleware could integrate into different settings and increase evidence use.

Examining primary care provider experiences with using a clinical decision support tool for pain management

Objective

To evaluate primary care provider (PCP) experiences using a clinical decision support (CDS) tool over 16 months following a user-centered design process and implementation.

Materials and Methods

We conducted a qualitative evaluation of the Chronic Pain OneSheet (OneSheet), a chronic pain CDS tool. OneSheet provides pain- and opioid-related risks, benefits, and treatment information for patients with chronic pain to PCPs. Using the 5 Rights of CDS framework, we conducted and analyzed semi-structured interviews with 19 PCPs across 2 academic health systems.

Results

PCPs stated that OneSheet mostly contained the right information required to treat patients with chronic pain and was correctly located in the electronic health record. PCPs used OneSheet for distinct subgroups of patients with chronic pain, including patients prescribed opioids, with poorly controlled pain, or new to a provider or clinic. PCPs reported variable workflow integration and selective use of certain OneSheet features driven by their preferences and patient population. PCPs recommended broadening OneSheet access to clinical staff and patients for data entry to address clinician time constraints.

Discussion

Differences in patient subpopulations and workflow preferences had an outsized effect on CDS tool use even when the CDS contained the right information identified in a user-centered design process.

Conclusions

To increase adoption and use, CDS design and implementation processes may benefit from increased tailoring that accommodates variation and dynamics among patients, visits, and providers.

Generalizability and portability of natural language processing system to extract individual social risk factors

OBJECTIVE

The objective of this study is to validate and report on portability and generalizability of a Natural Language Processing (NLP) method to extract individual social factors from clinical notes, which was originally developed at a different institution.

MATERIALS AND METHODS

A rule-based deterministic state machine NLP model was developed to extract financial insecurity and housing instability using notes from one institution and was applied on all notes written during 6 months at another institution. 10% of positively-classified notes by NLP and the same number of negatively-classified notes were manually annotated. The NLP model was adjusted to accommodate notes at the new site. Accuracy, positive predictive value, sensitivity, and specificity were calculated.

RESULTS

More than 6 million notes were processed at the receiving site by the NLP model, which resulted in about 13,000 and 19,000 classified as positive for financial insecurity and housing instability, respectively. The NLP model showed excellent performance on the validation dataset with all measures over 0.87 for both social factors.

DISCUSSION

Our study illustrated the need to accommodate institution-specific note-writing templates as well as clinical terminology of emergent diseases when applying NLP model for social factors. A state machine is relatively simple to port effectively across institutions. Our study. showed superior performance to similar generalizability studies for extracting social factors.

CONCLUSION

Rule-based NLP model to extract social factors from clinical notes showed strong portability and generalizability across organizationally and geographically distinct institutions. With only relatively simple modifications, we obtained promising performance from an NLP-based model.

Enhancing an enterprise data warehouse for research with data extracted using natural language processing

Objective

This study aims to develop a generalizable architecture for enhancing an enterprise data warehouse for research (EDW4R) with results from a natural language processing (NLP) model, which allows discrete data derived from clinical notes to be made broadly available for research use without need for NLP expertise. The study also quantifies the additional value that information extracted from clinical narratives brings to EDW4R.

Materials and methods

Clinical notes written during one month at an academic health center were used to evaluate the performance of an existing NLP model and to quantify its value added to the structured data. Manual review was utilized for performance analysis. The architecture for enhancing the EDW4R is described in detail to enable reproducibility.

Results

Two weeks were needed to enhance EDW4R with data from 250 million clinical notes. NLP generated 16 and 39% increase in data availability for two variables.

Discussion

Our architecture is highly generalizable to a new NLP model. The positive predictive value obtained by an independent team showed only slightly lower NLP performance than the values reported by the NLP developers. The NLP showed significant value added to data already available in structured format.

Conclusion

Given the value added by data extracted using NLP, it is important to enhance EDW4R with these data to enable research teams without NLP expertise to benefit from value added by NLP models.

Usability Testing of an Interoperable Computerized Clinical Decision Support Tool for Fall Risk Management in Primary Care

BACKGROUND

Falls are a widespread and persistent problem for community-dwelling older adults. Use of fall prevention guidelines in the primary care setting has been suboptimal. Interoperable computerized clinical decision support systems have the potential to increase engagement with fall risk management at scale. To support fall risk management across organizations, our team developed the ASPIRE tool for use in differing primary care clinics using interoperable standards.

OBJECTIVES

Usability testing of ASPIRE was conducted to measure ease of access, overall usability, learnability, and acceptability prior to pilot .

METHODS

Participants were recruited using purposive sampling from two sites with different electronic health records and different clinical organizations. Formative testing rooted in user-centered design was followed by summative testing using a simulation approach. During summative testing participants used ASPIRE across two clinical scenarios and were randomized to determine which scenario they saw first. Single Ease Question and System Usability Scale were used in addition to analysis of recorded sessions in NVivo.

RESULTS

All 14 participants rated the usability of ASPIRE as above average based on usability benchmarks for the System Usability Scale metric. Time on task decreased significantly between the first and second scenarios indicating good learnability. However, acceptability data were more mixed with some recommendations being consistently accepted while others were adopted less frequently.

CONCLUSION

This study described the usability testing of the ASPIRE system within two different organizations using different electronic health records. Overall, the system was rated well, and further pilot testing should be done to validate that these positive results translate into clinical practice. Due to its interoperable design, ASPIRE could be integrated into diverse organizations allowing a tailored implementation without the need to build a new system for each organization. This distinction makes ASPIRE well positioned to impact the challenge of falls at scale.

Accuracy of Electronic Health Record Food Insecurity, Housing Instability, and Financial Strain Screening in Adult Primary Care

This study assesses the accuracy of electronic health record–based screening questionnaires about social risk factors using external single-domain questionnaires as a comparator.

A large language model for electronic health records

There is an increasing interest in developing artificial intelligence (AI) systems to process and interpret electronic health records (EHRs). Natural language processing (NLP) powered by pretrained language models is the key technology for medical AI systems utilizing clinical narratives. However, there are few clinical language models, the largest of which trained in the clinical domain is comparatively small at 110 million parameters (compared with billions of parameters in the general domain). It is not clear how large clinical language models with billions of parameters can help medical AI systems utilize unstructured EHRs. In this study, we develop from scratch a large clinical language model-GatorTron-using >90 billion words of text (including >82 billion words of de-identified clinical text) and systematically evaluate it on five clinical NLP tasks including clinical concept extraction, medical relation extraction, semantic textual similarity, natural language inference (NLI), and medical question answering (MQA). We examine how (1) scaling up the number of parameters and (2) scaling up the size of the training data could benefit these NLP tasks. GatorTron models scale up the clinical language model from 110 million to 8.9 billion parameters and improve five clinical NLP tasks (e.g., 9.6% and 9.5% improvement in accuracy for NLI and MQA), which can be applied to medical AI systems to improve healthcare delivery. The GatorTron models are publicly available at: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/clara/models/gatortron_og .

Evaluation of federated learning variations for COVID-19 diagnosis using chest radiographs from 42 US and European hospitals

OBJECTIVE

Federated learning (FL) allows multiple distributed data holders to collaboratively learn a shared model without data sharing. However, individual health system data are heterogeneous. "Personalized" FL variations have been developed to counter data heterogeneity, but few have been evaluated using real-world healthcare data. The purpose of this study is to investigate the performance of a single-site versus a 3-client federated model using a previously described Coronavirus Disease 19 (COVID-19) diagnostic model. Additionally, to investigate the effect of system heterogeneity, we evaluate the performance of 4 FL variations.

MATERIALS AND METHODS

We leverage a FL healthcare collaborative including data from 5 international healthcare systems (US and Europe) encompassing 42 hospitals. We implemented a COVID-19 computer vision diagnosis system using the Federated Averaging (FedAvg) algorithm implemented on Clara Train SDK 4.0. To study the effect of data heterogeneity, training data was pooled from 3 systems locally and federation was simulated. We compared a centralized/pooled model, versus FedAvg, and 3 personalized FL variations (FedProx, FedBN, and FedAMP).

RESULTS

We observed comparable model performance with respect to internal validation (local model: AUROC 0.94 vs FedAvg: 0.95, P = .5) and improved model generalizability with the FedAvg model (P < .05). When investigating the effects of model heterogeneity, we observed poor performance with FedAvg on internal validation as compared to personalized FL algorithms. FedAvg did have improved generalizability compared to personalized FL algorithms. On average, FedBN had the best rank performance on internal and external validation.

CONCLUSION

FedAvg can significantly improve the generalization of the model compared to other personalization FL algorithms; however, at the cost of poor internal validity. Personalized FL may offer an opportunity to develop both internal and externally validated algorithms.

Recent tapering from long-term opioid therapy and odds of opioid-related hospital use

PURPOSE

The number of patients tapered from long-term opioid therapy (LTOT) has increased in recent years in the United States. Some patients tapered from LTOT report improved quality of life, while others face increased risks of opioid-related hospital use. Research has not yet established how the risk of opioid-related hospital use changes across LTOT dose and subsequent tapering. Our objective was to examine associations between recent tapering from LTOT with odds of opioid-related hospital use.

METHODS

Case-crossover design using 2014-2018 health information exchange data from Indiana. We defined opioid-related hospital use as hospitalizations, and emergency department (ED) visits for a drug overdose, opioid abuse, and dependence. We defined tapering as a 15% or greater dose reduction following at least 3 months of continuous opioid therapy of 50 morphine milligram equivalents (MME)/day or more. We used conditional logistic regression to estimate odds ratios (OR) with 95% confidence intervals (CI).

RESULTS

Recent tapering from LTOT was associated with increased odds of opioid-related hospital use (OR: 1.50, 95%CI: 1.34-1.63), ED visit (OR: 1.52; 95%CI: 1.35-1.72), and inpatient hospitalization (OR: 1.40; 95%CI: 1.20-1.65). We found no evidence of heterogeneity of the effect of tapering on opioid-related hospital use by gender, age, and race. Recent tapering among patients on a high baseline dose (>300 MME) was associated with increased odds of opioid-related hospital use (OR: 2.95, 95% CI: 2.12-4.11, p < 0.001) compared to patients on a lower baseline doses.

CONCLUSIONS

Recent tapering from LTOT is associated with increased odds of opioid-related hospital use.

A novel method for evaluating physician communication: A pilot study testing the feasibility of parent-assisted audio recordings via Zoom

OBJECTIVE

Quality of physician consultations are best assessed via direct observation, but require intensive in-clinic research staffing. To evaluate physician consultation quality remotely, we pilot tested the feasibility of parents using their personal mobile phones to facilitate audio recordings of pediatric visits.

METHODS

Across four academic pediatric primary care clinics, we invited all physicians with a patient panel (n=20). For participating physicians, we identified scheduled patients from medical records. We invited parents to participate via text message and phone calls. During their adolescent's appointment, parents used their mobile phone to connect to Zoom for remote research staff to audio record.

RESULTS

In Spring 2021, five of 20 (25%) physicians participated. During a nine-week period, we invited parents of all 54 patients seen by participating physicians of which 15 (28%) completed adult consent and adolescent assent and 10 (19%) participated. For 9 recordings, at least 45% of the conversation was audible.

CONCLUSIONS

It was feasible and acceptable to directly observe physician consultations virtually with Zoom, although participation rates and potentially audio quality were lower.

INNOVATION

Patients used their cellular phone calling features to connect to Zoom where research staff audio-recorded their physician consultation to evaluate communication quality.

Use of Acute Mental Health Care in U.S. Children's Hospitals Before and After Statewide COVID-19 School Closure Orders

OBJECTIVE

This study aimed to examine changes in child emergency department (ED) discharges and hospitalizations for primary general medical (GM) and primary psychiatric disorders; prevalence of psychiatric disorders among acute care encounters; and change in acute mental health (MH) care encounters by disorder type and, within these categories, by child sociodemographic characteristics before and after statewide COVID-19–related school closure orders.

METHODS

This retrospective, cross-sectional cohort study used the Pediatric Health Information System database to assess percent changes in ED discharges and hospitalizations (N=2,658,474 total encounters) among children ages 3–17 years in 44 U.S. children’s hospitals in 2020 compared with 2019, by using matched data for 36- and 12-calendar-week intervals.

RESULTS

Decline in MH ED discharges accounted for about half of the decline in ED discharges and hospitalizations for primary GM disorders (−24.8% vs. −49.1%), and MH hospitalizations declined 3.4 times less (−8.0% vs. −26.8%) in 2020. Suicide attempt or self-injury and depressive disorders accounted for &gt;50% of acute MH care encounters before and after the statewide school closures. The increase in both ED discharges and hospitalizations for suicide attempt or self-injury was 5.1 percentage points (p&lt;0.001). By fall 2020, MH hospitalizations for suicide attempt or self-injury rose by 41.7%, with a 43.8% and 49.2% rise among adolescents and girls, respectively.

CONCLUSIONS

Suicide or self-injury and depressive disorders drove acute MH care encounters in 44 U.S. children’s hospitals after COVID-19–related school closures. Research is needed to identify continuing risk indicators (e.g., sociodemographic characteristics, psychiatric disorder types, and social determinants of health) of acute child MH care.

A framework for a consistent and reproducible evaluation of manual review for patient matching algorithms

Healthcare systems are hampered by incomplete and fragmented patient health records. Record linkage is widely accepted as a solution to improve the quality and completeness of patient records. However, there does not exist a systematic approach for manually reviewing patient records to create gold standard record linkage data sets. We propose a robust framework for creating and evaluating manually reviewed gold standard data sets for measuring the performance of patient matching algorithms. Our 8-point approach covers data preprocessing, blocking, record adjudication, linkage evaluation, and reviewer characteristics. This framework can help record linkage method developers provide necessary transparency when creating and validating gold standard reference matching data sets. In turn, this transparency will support both the internal and external validity of recording linkage studies and improve the robustness of new record linkage strategies.

Assessing the use of a clinical decision support tool for pain management in primary care

Objective

Given time constraints, poorly organized information, and complex patients, primary care providers (PCPs) can benefit from clinical decision support (CDS) tools that aggregate and synthesize problem-specific patient information. First, this article describes the design and functionality of a CDS tool for chronic noncancer pain in primary care. Second, we report on the retrospective analysis of real-world usage of the tool in the context of a pragmatic trial.

Materials and methods

The tool known as OneSheet was developed using user-centered principles and built in the Epic electronic health record (EHR) of 2 health systems. For each relevant patient, OneSheet presents pertinent information in a single EHR view to assist PCPs in completing guideline-recommended opioid risk mitigation tasks, review previous and current patient treatments, view patient-reported pain, physical function, and pain-related goals.

Results

Overall, 69 PCPs accessed OneSheet 2411 times (since November 2020). PCP use of OneSheet varied significantly by provider and was highly skewed (site 1: median accesses per provider: 17 [interquartile range (IQR) 9-32]; site 2: median: 8 [IQR 5-16]). Seven "power users" accounted for 70% of the overall access instances across both sites. OneSheet has been accessed an average of 20 times weekly between the 2 sites.

Discussion

Modest OneSheet use was observed relative to the number of eligible patients seen with chronic pain.

Conclusions

Organizations implementing CDS tools are likely to see considerable provider-level variation in usage, suggesting that CDS tools may vary in their utility across PCPs, even for the same condition, because of differences in provider and care team workflows.

Study protocol for a type III hybrid effectiveness-implementation trial to evaluate scaling interoperable clinical decision support for patient-centered chronic pain management in primary care

Background

The US continues to face public health crises related to both chronic pain and opioid overdoses. Thirty percent of Americans suffer from chronic noncancer pain at an estimated yearly cost of over $600 billion. Most patients with chronic pain turn to primary care clinicians who must choose from myriad treatment options based on relative risks and benefits, patient history, available resources, symptoms, and goals. Recently, with attention to opioid-related risks, prescribing has declined. However, clinical experts have countered with concerns that some patients for whom opioid-related benefits outweigh risks may be inappropriately discontinued from opioids. Unfortunately, primary care clinicians lack usable tools to help them partner with their patients in choosing pain treatment options that best balance risks and benefits in the context of patient history, resources, symptoms, and goals. Thus, primary care clinicians and patients would benefit from patient-centered clinical decision support (CDS) for this shared decision-making process.

Methods

The objective of this 3-year project is to study the adaptation and implementation of an existing interoperable CDS tool for pain treatment shared decision making, with tailored implementation support, in new clinical settings in the OneFlorida Clinical Research Consortium. Our central hypothesis is that tailored implementation support will increase CDS adoption and shared decision making. We further hypothesize that increases in shared decision making will lead to improved patient outcomes, specifically pain and physical function. The CDS implementation will be guided by the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. The evaluation will be organized by the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework. We will adapt and tailor PainManager, an open source interoperable CDS tool, for implementation in primary care clinics affiliated with the OneFlorida Clinical Research Consortium. We will evaluate the effect of tailored implementation support on PainManager’s adoption for pain treatment shared decision making. This evaluation will establish the feasibility and obtain preliminary data in preparation for a multi-site pragmatic trial targeting the effectiveness of PainManager and tailored implementation support on shared decision making and patient-reported pain and physical function.

Discussion

This research will generate evidence on strategies for implementing interoperable CDS in new clinical settings across different types of electronic health records (EHRs). The study will also inform tailored implementation strategies to be further tested in a subsequent hybrid effectiveness-implementation trial. Together, these efforts will lead to important new technology and evidence that patients, clinicians, and health systems can use to improve care for millions of Americans who suffer from pain and other chronic conditions.

Trial registration

ClinicalTrials.gov, NCT05256394, Registered 25 February 2022.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-022-01217-4.

Secure Messaging and COVID-19: A Content Analysis of Patient-Clinician Communication During the Pandemic

Background: Coronavirus disease 2019 (COVID-19) immediately impacted patient-clinician communication, particularly in the oncology setting. Relatedly, secure messaging (SM) usage greatly increased, yet it is unknown what was discussed and whether the technology was utilized to disseminate information. Aims: This study aimed at identifying the most frequently discussed topics using SM as well as at understanding how the communication process transpired during the early stages of the pandemic. Materials and Methods: A mixed-methods design was utilized, consisting of a content analysis of more than 4,200 secure messages, aggregated into 1,454 patient-clinician discussions. Data were collected from February 2020 to May 2020. Discussions were from various oncology departments and included physicians, physician assistants, and nurses. Based on the identified categories, a thematic analysis was conducted to understand the nuances occurring within discussions. Results: Out of the 1,454 discussions, 26% (n = 373) related to COVID-19. Of the COVID-19 discussion, the most frequently coded category was "changes, adjustments, and re-arranging care" (65%, n = 241), followed by "risk for COVID-19" (24%, n = 90), "precautions inside the hospital" (18%, n = 66), and "precautions outside the hospital" (14%, n = 52). Natural language processing techniques were used to confirm the validity of the results. Thematic analysis revealed that patients were proactive in rescheduling appointments, expressed anxiety about being immunocompromised, and clinicians were uncertain about providing recommendations related to COVID-19. Conclusions: The COVID-19 outbreak revealed the need for responsive and effective public health communication. The SM can disseminate information from trusted sources, clinicians, but can be better utilized to deliver tailored information for specific patient populations.

Dosing profiles of concurrent opioid and benzodiazepine use associated with overdose risk among US Medicare beneficiaries: group-based multi-trajectory models

BACKGROUND AND AIMS

One-third of opioid (OPI) overdose deaths involve concurrent benzodiazepine (BZD) use. Little is known about concurrent opioid and benzodiazepine use (OPI-BZD) most associated with overdose risk. We aimed to examine associations between OPI-BZD dose and duration trajectories, and subsequent OPI or BZD overdose in US Medicare.

DESIGN

Retrospective cohort study.

SETTING

US Medicare.

PARTICIPANTS

Using a 5% national Medicare data sample (2013-16) of fee-for-service beneficiaries without cancer initiating OPI prescriptions, we identified 37 879 beneficiaries (age ≥ 65 = 59.3%, female = 71.9%, white = 87.6%, having OPI overdose = 0.3%).

MEASUREMENTS

During the 6 months following OPI initiation (i.e. trajectory period), we identified OPI-BZD dose and duration patterns using group-based multi-trajectory models, based on average daily morphine milligram equivalents (MME) for OPIs and diazepam milligram equivalents (DME) for BZDs. To label dose levels in each trajectory, we defined OPI use as very low (< 25 MME), low (25-50 MME), moderate (51-90 MME), high (91-150 MME) and very high (>150 MME) dose. Similarly, we defined BZD use as very low (< 10 DME), low (10-20 DME), moderate (21-40 DME), high (41-60 DME) and very high (> 60 DME) dose. Our primary analysis was to estimate the risk of time to first hospital or emergency department visit for OPI overdose within 6 months following the trajectory period using inverse probability of treatment-weighted Cox proportional hazards models.

FINDINGS

We identified nine distinct OPI-BZD trajectories: group A: very low OPI (early discontinuation)-very low declining BZD (n = 10 598; 28.0% of the cohort); B: very low OPI (early discontinuation)-very low stable BZD (n = 4923; 13.0%); C: very low OPI (early discontinuation)-medium BZD (n = 4997; 13.2%); D: low OPI-low BZD (n = 5083; 13.4%); E: low OPI-high BZD (n = 3906; 10.3%); F: medium OPI-low BZD (n = 3948; 10.4%); G: very high OPI-high BZD (n = 1371; 3.6%); H: very high OPI-very high BZD (n = 957; 2.5%); and I: very high OPI-low BZD (n = 2096; 5.5%). Compared with group A, five trajectories (32.3% of the study cohort) were associated with increased 6-month OPI overdose risks: E: low OPI-high BZD [hazard ratio (HR) = 3.27, 95% confidence interval (CI) = 1.61-6.63]; F: medium OPI-low BZD (HR = 4.04, 95% CI = 2.06-7.95); G: very high OPI-high BZD (HR = 6.98, 95% CI = 3.11-15.64); H: very high OPI-very high BZD (HR = 4.41, 95% CI = 1.51-12.85); and I: very high OPI-low BZD (HR = 6.50, 95% CI = 3.15-13.42).

CONCLUSIONS

Patterns of concurrent opioid and benzodiazepine use most associated with overdose risk among fee-for-service US Medicare beneficiaries initiating opioid prescriptions include very high-dose opioid use (MME > 150), high-dose benzodiazepine use (DME > 40) or medium-dose opioid with low-dose benzodiazepine use.

Inpatient opioid receipt and care experiences for vaginal delivery

OBJECTIVES

To examine the relationship between care experiences and inpatient opioid receipt during and after delivery for women hospitalized for vaginal delivery (VD).

STUDY DESIGN

We used a pooled cross-sectional design with inverse probability weighting to examine the association between inpatient opioid receipt and care experiences of women hospitalized for VD at a single health care system in a Midwestern state. We used 4 Hospital Consumer Assessment of Healthcare Providers and Systems scores (2 pain care items and 2 global items) as measures of care experiences of women hospitalized for VD.

METHODS

We used 4 inverse probability-weighted logit regressions to estimate the relationship between inpatient opioid receipt and each patient care experience measure. In supplementary analyses, we used the same inverse probability-weighted methods to estimate the relationship between receipt of opioids and patient care experience measures in 3 patient subgroups based on mean patient-reported pain score during hospitalization (no pain, mild pain, moderate pain).

RESULTS

We found no relationship between inpatient opioid receipt and inpatient pain care experiences. As an exception, we found that women hospitalized for VD were 5 (95% CI, 2-8) percentage points more likely to rate the hospital as 10 ("the best hospital possible") during hospitalizations in which an opioid was received. We also found higher overall ratings of the hospital among hospitalized women who reported mild pain if they received an opioid (marginal effects = 0.05; 95% CI, 2-8 percentage points).

CONCLUSIONS

Receipt of opioids may not be a significant determinant of the pain-specific patient care experiences of women hospitalized for VD.

Evaluating a Prototype Clinical Decision Support Tool for Chronic Pain Treatment in Primary Care

OBJECTIVES

 The Chronic Pain Treatment Tracker (Tx Tracker) is a prototype decision support tool to aid primary care clinicians when caring for patients with chronic noncancer pain. This study evaluated clinicians' perceived utility of Tx Tracker in meeting information needs and identifying treatment options, and preferences for visual design.

METHODS

 We conducted 12 semi-structured interviews with primary care clinicians from four health systems in Indiana. The interviews were conducted in two waves, with prototype and interview guide revisions after the first six interviews. The interviews included exploration of Tx Tracker using a think-aloud approach and a clinical scenario. Clinicians were presented with a patient scenario and asked to use Tx Tracker to make a treatment recommendation. Last, participants answered several evaluation questions. Detailed field notes were collected, coded, and thematically analyzed by four analysts.

RESULTS

 We identified several themes: the need for clinicians to be presented with a comprehensive patient history, the usefulness of Tx Tracker in patient discussions about treatment planning, potential usefulness of Tx Tracker for patients with high uncertainty or risk, potential usefulness of Tx Tracker in aggregating scattered information, variability in expectations about workflows, skepticism about underlying electronic health record data quality, interest in using Tx Tracker to annotate or update information, interest in using Tx Tracker to translate information to clinical action, desire for interface with visual cues for risks, warnings, or treatment options, and desire for interactive functionality.

CONCLUSION

 Tools like Tx Tracker, by aggregating key information about past, current, and potential future treatments, may help clinicians collaborate with their patients in choosing the best pain treatments. Still, the use and usefulness of Tx Tracker likely relies on continued improvement of its functionality, accurate and complete underlying data, and tailored integration with varying workflows, care team roles, and user preferences.

Evaluation of electronic recruitment efforts of primary care providers as research subjects during the COVID-19 pandemic

BACKGROUND

Recruiting healthcare providers as research subjects often rely on in-person recruitment strategies. Little is known about recruiting provider participants via electronic recruitment methods. In this study, conducted during the COVID-19 pandemic, we describe and evaluate a primarily electronic approach to recruiting primary care providers (PCPs) as subjects in a pragmatic randomized controlled trial (RCT) of a decision support intervention.

METHODS

We adapted an existing framework for healthcare provider research recruitment, employing an electronic consent form and a mix of brief synchronous video presentations, email, and phone calls to recruit PCPs into the RCT. To evaluate the success of each electronic strategy, we estimated the number of consented PCPs associated with each strategy, the number of days to recruit each PCP and recruitment costs.

RESULTS

We recruited 45 of 63 eligible PCPs practicing at ten primary care clinic locations over 55 days. On average, it took 17 business days to recruit a PCP (range 0-48) and required three attempts (range 1-7). Email communication from the clinic leaders led to the most successful recruitments, followed by brief synchronous video presentations at regularly scheduled clinic meetings. We spent approximately $89 per recruited PCP. We faced challenges of low email responsiveness and limited opportunities to forge relationships.

CONCLUSION

PCPs can be efficiently recruited at low costs as research subjects using primarily electronic communications, even during a time of high workload and stress. Electronic peer leader outreach and synchronous video presentations may be particularly useful recruitment strategies.

TRIAL REGISTRATION

ClinicalTrials.gov , NCT04295135 . Registered 04 March 2020.

Unraveling COVID-19: A Large-Scale Characterization of 4.5 Million COVID-19 Cases Using CHARYBDIS

Purpose

Routinely collected real world data (RWD) have great utility in aiding the novel coronavirus disease (COVID-19) pandemic response. Here we present the international Observational Health Data Sciences and Informatics (OHDSI) Characterizing Health Associated Risks and Your Baseline Disease In SARS-COV-2 (CHARYBDIS) framework for standardisation and analysis of COVID-19 RWD.

Patients and Methods

We conducted a descriptive retrospective database study using a federated network of data partners in the United States, Europe (the Netherlands, Spain, the UK, Germany, France and Italy) and Asia (South Korea and China). The study protocol and analytical package were released on 11th June 2020 and are iteratively updated via GitHub. We identified three non-mutually exclusive cohorts of 4,537,153 individuals with a clinical COVID-19 diagnosis or positive test, 886,193 hospitalized with COVID-19, and 113,627 hospitalized with COVID-19 requiring intensive services.

Results

We aggregated over 22,000 unique characteristics describing patients with COVID-19. All comorbidities, symptoms, medications, and outcomes are described by cohort in aggregate counts and are readily available online. Globally, we observed similarities in the USA and Europe: more women diagnosed than men but more men hospitalized than women, most diagnosed cases between 25 and 60 years of age versus most hospitalized cases between 60 and 80 years of age. South Korea differed with more women than men hospitalized. Common comorbidities included type 2 diabetes, hypertension, chronic kidney disease and heart disease. Common presenting symptoms were dyspnea, cough and fever. Symptom data availability was more common in hospitalized cohorts than diagnosed.

Conclusion

We constructed a global, multi-centre view to describe trends in COVID-19 progression, management and evolution over time. By characterising baseline variability in patients and geography, our work provides critical context that may otherwise be misconstrued as data quality issues. This is important as we perform studies on adverse events of special interest in COVID-19 vaccine surveillance.

Student perceptions of a teaching electronic medical record in Health Administration education

Given the ubiquity of electronic health records (EHR), health administrators should be formally trained on the use and evaluation of EHR data for common operational tasks and managerial decision-making. A teaching electronic medical record (tEMR) is a fully operational electronic medical record that uses de-identified electronic patient data and provides a framework for students to familiarize themselves with the data, features, and functionality of an EHR. Although purported to be of value in health administration programs, specific benefits of using a tEMR in health administration education is unknown. We sought to examine Master of Health Administration (MHA) students' perceptions of the use, challenges, and benefits of a tEMR. We analyzed qualitative data collected from a focus group session with students who were exposed to the tEMR during a semester MHA course. We also administered pre- and post-survey questions on students' self-efficacy and perceptions of the ease of use, usefulness, and intention to use health care data analysis in their future jobs. We found several MHA students valued their exposure to the tEMR, as this provided them a realistic environment to explore de-identified patient data. Scores for students' perceived ease of using healthcare data analysis in their future job significantly increased following use of the tEMR (pre-test score M=3.31, SD=0.21; post-test score M=3.71, SD=0.18; p=0.01). Student exposure and use of a tEMR may positively affect perceptions of using EHR data for strategic and managerial tasks typical of health administrators.

State-level regulations and opioid-related health outcomes

INTRODUCTION

Due to the ongoing opioid use disorder crisis, improved access to opioid treatment programs (OTPs) is needed. However, OTPs operate in a complex regulatory environment which may limit their ability to positively affect health outcomes. The objective of this study was to examine how the number and type of state OTP regulations are associated with opioid-related deaths, hospitalizations, and emergency department visits.

METHODS

Cross-sectional data capturing information about OTP state-level regulations collected by Jackson et al. was combined with other secondary sources. OTP regulations were categorized based on the nature of their focus. Analyses include bivariate and multivariable regressions that controlled for region and other state laws that can affect opioid outcomes.

RESULTS

In bivariate analysis, a greater number of OTP regulations was positively correlated with both deaths and emergency visits. Moreover, a greater number of regulations in the Physical Facilities Management category (e.g., rules related to restrooms, lighting, and signage) was positively correlated with both deaths and hospitalizations. The number of regulations in the Staffing Requirement category was positively associated with emergency visits. In adjusted analysis, the number of regulations in the Physical Facilities Management category was positively associated with opioid-related deaths.

CONCLUSIONS

States with a higher number of regulations had poorer opioid-related outcomes. Additional research is needed to support policy decisions that can improve access to OTPs and reduce avoidable deaths, hospitalizations, and emergency visits.

Mental Health-Related Emergency Department Visits Among Children During The Early COVID-19 Pandemic

Objective

To measure univariate and covariate-adjusted trends in children's mental health-related emergency department (MH-ED) use across geographically diverse areas of the U.S. during the first wave of the Coronavirus-2019 (COVID-19) pandemic.

Method

This is a retrospective, cross-sectional cohort study using electronic health records from four academic health systems, comparing percent volume change and adjusted risk of child MH-ED visits among children aged 3-17 years, matched on 36-week (3/18/19-11/25/19 vs. 3/16/20-11/22/20) and 12-week seasonal time intervals. Adjusted incidence rate ratios (IRR) were calculated using multivariate Poisson regression.

Results

Visits declined during spring-fall 2020 (n = 3892 vs. n = 5228, -25.5%) and during spring (n = 1051 vs. n = 1839, -42.8%), summer (n = 1430 vs. n = 1469, -2.6%), and fall (n = 1411 vs. n = 1920, -26.5%), compared with 2019. There were greater declines among males (28.2% vs. females -22.9%), children 6-12-year (-28.6% vs. -25.9% for 3-5 years and -22.9% for 13-17 years), and Black children (-34.8% vs. -17.7% to -24.9%). Visits also declined for developmental disorders (-17.0%) and childhood-onset disorders (e.g., attention deficit and hyperactivity disorders; -18.0%). During summer-fall 2020, suicide-related visits rose (summer +29.8%, fall +20.4%), but were not significantly elevated from 2019 when controlling for demographic shifts. In contrast, MH-ED use during spring-fall 2020 was significantly reduced for intellectual disabilities (IRR 0.62 [95% CI 0.47-0.86]), developmental disorders (IRR 0.71 [0.54-0.92]), and childhood-onset disorders (IRR 0.74 [0.56-0.97]).

Conclusions

The early pandemic brought overall declines in child MH-ED use alongside co-occurring demographic and diagnostic shifts. Children vulnerable to missed detection during instructional disruptions experienced disproportionate declines, suggesting need for future longitudinal research in this population.

Assessment of structured data elements for social risk factors

OBJECTIVES

Computable social risk factor phenotypes derived from routinely collected structured electronic health record (EHR) or health information exchange (HIE) data may represent a feasible and robust approach to measuring social factors. This study convened an expert panel to identify and assess the quality of individual EHR and HIE structured data elements that could be used as components in future computable social risk factor phenotypes.

STUDY DESIGN

Technical expert panel.

METHODS

A 2-round Delphi technique included 17 experts with an in-depth knowledge of available EHR and/or HIE data. The first-round identification sessions followed a nominal group approach to generate candidate data elements that may relate to socioeconomics, cultural context, social relationships, and community context. In the second-round survey, panelists rated each data element according to overall data quality and likelihood of systematic differences in quality across populations (ie, bias).

RESULTS

Panelists identified a total of 89 structured data elements. About half of the data elements (n = 45) were related to socioeconomic characteristics. The panelists identified a diverse set of data elements. Elements used in reimbursement-related processes were generally rated as higher quality. Panelists noted that several data elements may be subject to implicit bias or reflect biased systems of care, which may limit their utility in measuring social factors.

CONCLUSIONS

Routinely collected structured data within EHR and HIE systems may reflect patient social risk factors. Identifying and assessing available data elements serves as a foundational step toward developing future computable social factor phenotypes.

Understanding Health Care Administrators’ Data and Information Needs for Decision Making during the COVID-19 Pandemic: A Qualitative Study at an Academic Health System

Objective. The COVID-19 pandemic created an unprecedented strain on the health care system, and administrators had to make many critical decisions to respond appropriately. This study sought to understand how health care administrators used data and information for decision making during the first 6 mo of the COVID-19 pandemic. Materials and Methods. We conducted semistructured interviews with administrators across University of Florida (UF) Health. We performed an inductive thematic analysis of the transcripts. Results. Four themes emerged from the interviews: 1) common types of health systems or hospital operations data; 2) public health and other external data sources; 3) data interaction, integration, and exchange; and 4) novelty and evolution in data, information, or tools used over time. Participants illustrated the organizational, public health, and regional information they considered essential (e.g., hospital census, community positivity rate, etc.). Participants named specific challenges they faced due to data quality and timeliness. Participants elaborated on the necessity of data integration, validation, and coordination across different boundaries (e.g., different hospital systems in the same metro areas, public health agencies at the local, state, and federal level, etc.). Participants indicated that even within the first 6 mo of the COVID-19 pandemic, the data and tools used for making critical decisions changed. Discussion. While existing medical informatics infrastructure can facilitate decision making in pandemic response, data may not always be readily available in a usable format. Interoperable infrastructure and data standardization across multiple health systems would help provide more reliable and timely information for decision making. Conclusion. Our findings contribute to future discussions of improving data infrastructure and developing harmonized data standards needed to facilitate critical decisions at multiple health care system levels.

How opioid prescribing policies influence primary care clinicians' treatment decisions and conversations with patients with chronic pain

OBJECTIVE

The 2016 Centers for Disease Control and Prevention guideline for prescribing opioids for chronic pain aimed to assist primary care clinicians in safely and effectively prescribing opioids for chronic noncancer pain. Individual states, payers, and health systems issued similar policies imposing various regulations around opioid prescribing for patients with chronic pain. Experts argued that healthcare organizations and clinicians may be misapplying the federal guideline and subsequent opioid prescribing policies, leading to an inadequate pain management. The objective of this study was to understand how primary care clinicians involve opioid prescribing policies in their treatment decisions and in their conversations with patients with chronic pain.

DESIGN

We conducted a secondary qualitative analysis of data from 64 unique primary care visits and 87 post-visit interviews across 20 clinicians from three healthcare systems in the Midwestern United States. Using a multistep process and thematic analysis, we systematically analyzed data excerpts addressing opioid prescribing policies.

RESULTS

Opioid prescribing policies influenced clinicians' treatment decisions to not initiate opioids, prescribe fewer opioids overall (theme #1), and begin tapering and discontinuation of opioids (theme #2) for most patients with chronic pain. Clinical precautions, described in the opioid prescribing policies to monitor use, were directly invoked during visits for patients with chronic pain (theme #3).

CONCLUSIONS

Opioid prescribing policies have multidimensional influence on clinician treatment decisions for patients with chronic pain. Our findings may inform future studies to explore mechanisms for aligning pressures around opioid prescribing, stemming from various opioid prescribing policies, with the need to deliver individualized pain care.

Primary care physicians' electronic health record proficiency and efficiency behaviors and time interacting with electronic health records: a quantile regression analysis

OBJECTIVE

This study aimed to understand the association between primary care physician (PCP) proficiency with the electronic health record (EHR) system and time spent interacting with the EHR.

MATERIALS AND METHODS

We examined the use of EHR proficiency tools among PCPs at one large academic health system using EHR-derived measures of clinician EHR proficiency and efficiency. Our main predictors were the use of EHR proficiency tools and our outcomes focused on 4 measures assessing time spent in the EHR: (1) total time spent interacting with the EHR, (2) time spent outside scheduled clinical hours, (3) time spent documenting, and (4) time spent on inbox management. We conducted multivariable quantile regression models with fixed effects for physician-level factors and time in order to identify factors that were independently associated with time spent in the EHR.

RESULTS

Across 441 primary care physicians, we found mixed associations between certain EHR proficiency behaviors and time spent in the EHR. Across EHR activities studied, QuickActions, SmartPhrases, and documentation length were positively associated with increased time spent in the EHR. Models also showed a greater amount of help from team members in note writing was associated with less time spent in the EHR and documenting.

DISCUSSION

Examining the prevalence of EHR proficiency behaviors may suggest targeted areas for initial and ongoing EHR training. Although documentation behaviors are key areas for training, team-based models for documentation and inbox management require further study.

CONCLUSIONS

A nuanced association exists between physician EHR proficiency and time spent in the EHR.

Long-term opioid therapy tapering: Trends from 2014 to 2018 in a Midwestern State

BACKGROUND

The 2016 Centers for Disease Control and Prevention guideline for prescribing opioids for chronic pain (Guideline hereafter) emphasized tapering patients from long-term opioid therapy (LTOT) when the harms outweigh the benefits.

METHODS

To examine tapering from LTOT before and after the Guideline release, we conducted a retrospective cohort study of adults with high-dose LTOT (mean of >50 Morphine Milligram Equivalents [MME]/day) from 2014 to 2018 from one Midwest state's Health Information Exchange. We identified tapering (dose reductions in mean MME/day greater than 15%, 30%, 50%) and rapid discontinuation episodes (reduction to zero MME/day) over a 6-month follow-up period relative to a 3-month baseline period. We used segmented regressions to estimate outcomes adjusted for time trends and relevant state laws limiting opioid prescribing.

RESULTS

The Guideline release was associated with statistically significant immediate increase in the patient likelihood of experiencing tapering (15%: 1.8% point [95% confidence interval (CI): 1.2-2.6; 30%: 1.4% point, 95% CI: 0.7-2.2; 50%: 0.8% point, 95% CI: 0.2-1.4) and rapid discontinuation episodes (0.006% point, 95% CI: 0.001-0.01). After the Guideline release, the patient likelihood of tapering increased over time (15%: 0.4% point/month, 95% CI: 0.3-0.5; 30%: 0.3% point/month, 95% CI:0.2-0.4; 50%: 0.3% point/month, 95% CI: 0.2-0.3; rapid discontinuation: 0.01% point/month, 95% CI: 0.007-0.01). Tapering and rapid discontinuation trends was similar among gender and race categories.

CONCLUSION

The Guideline may be a useful tool in altering opioid prescribing practices, particularly for patients on shorter durations of LTOT.

Disparities in Pediatric Patient Portal Activation and Feature Use

Objective

Disparities in adult patient portal adoption are well-documented; however, less is known about disparities in portal adoption in pediatrics. This study examines the prevalence and factors associated with patient portal activation and the use of specific portal features in general pediatrics.

Materials and methods

We analyzed electronic health record data from 2012 to 2020 in a large academic medical center that offers both parent and adolescent portals. We summarized portal activation and use of select portal features (messaging, records access and management, appointment management, visit/admissions summaries, and interactive feature use). We used logistic regression to model factors associated with patient portal activation among all patients along with feature use and frequent feature use among ever users (ie, ≥1 portal use).

Results

Among 52 713 unique patients, 39% had activated the patient portal, including 36% of patients aged 0–11, 41% of patients aged 12–17, and 62% of patients aged 18–21 years. Among activated accounts, ever use of specific features ranged from 28% for visit/admission summaries to 92% for records access and management. Adjusted analyses showed patients with activated accounts were more likely to be adolescents or young adults, white, female, privately insured, and less socioeconomically vulnerable. Individual feature use among ever users generally followed the same pattern.

Conclusions

Our findings demonstrate that important disparities persist in portal adoption in pediatric populations, highlighting the need for strategies to promote equitable access to patient portals.

Consumer Perspectives on Maternal and Infant Health Apps: Qualitative Content Analysis

BACKGROUND

Despite the popularity of maternal and infant health mobile apps, ongoing consumer engagement and sustained app use remain barriers. Few studies have examined user experiences or perceived benefits of maternal and infant health app use from consumer perspectives.

OBJECTIVE

This study aims to assess users' self-reported experiences with maternal and infant health apps, perceived benefits, and general feedback by analyzing publicly available user reviews on two popular app stores-Apple App Store and Google Play Store.

METHODS

We conducted a qualitative assessment of publicly available user reviews (N=2422) sampled from 75 maternal and infant health apps designed to provide health education or decision-making support to pregnant women or parents and caregivers of infants. The reviews were coded and analyzed using a general inductive qualitative content analysis approach.

RESULTS

The three major themes included the following: app functionality, where users discussed app features and functions; technical aspects, where users talked about technology-based aspects of an app; and app content, where users specifically focused on the app content and the information it provides. The six minor themes included the following: patterns of use, where users highlighted the frequency and type of use; social support, where users talked about receiving social support from friends, family and community of other users; app cost, where users talked about the cost of an app within the context of being cost-effective or a potential waste of money; app comparisons, where users compared one app with others available in app stores; assistance in health care, where users specifically highlighted the role of an app in offering clinical assistance; and customer care support, where users specifically talked about their interaction with the app customer care support team.

CONCLUSIONS

Users generally tend to value apps that are of low cost and preferably free, with high-quality content, superior features, enhanced technical aspects, and user-friendly interfaces. Users also find app developer responsiveness to be integral, as it offers them an opportunity to engage in the app development and delivery process. These findings may be beneficial for app developers in designing better apps, as no best practice guidelines currently exist for the app environment.

Implementation context for addressing social needs in a learning health system: a qualitative study

INTRODUCTION

Unmet social needs contribute to growing health disparities and rising health care costs. Strategies to collect and integrate information on social needs into patients' electronic health records (EHRs) show promise for connecting patients with community resources. However, gaps remain in understanding the contextual factors that impact implementing these interventions in clinical settings.

METHODS

We conducted qualitative interviews with patients and focus groups with providers (January-September 2020) in two primary care clinics to inform the implementation of a module that collects and integrates patient-reported social needs information into the EHR. Questions addressed constructs within the Theoretical Framework for Acceptability and the Consolidated Framework for Implementation Research. Data were coded deductively using team-based framework analysis, followed by inductive coding and matrix analyses.

RESULTS

Forty patients participated in interviews, with 20 recruited at the clinics and 20 from home. Two focus groups were conducted with a total of 12 providers. Factors salient to acceptability and feasibility included patients' discomfort answering sensitive questions, concerns about privacy, difficulty reading/understanding module content, and technological literacy. Rapport with providers was a facilitator for patients to discuss social needs. Providers stressed that limited time with patients would be a barrier, and expressed concerns about the lack of available community resources.

CONCLUSION

Findings highlight the need for flexible approaches to assessing and discussing social needs with patients. Feasibility of the intervention is contingent upon support from the health system to facilitate social needs assessment and discussion. Further study of availability of community resources is needed to ensure intervention effectiveness.

Content analysis of behavior change techniques in maternal and infant health apps

Maternal and infant health (MIH) mobile applications (apps) are increasingly popular and frequently used for health education and decision making. Interventions grounded in theory-based behavior change techniques (BCTs) are shown to be effective in promoting healthy behavior changes. MIH apps have the potential to be useful tools, yet the extent to which they incorporate BCTs is still unknown. The objective of this study was to assess the presence of BCTs in popular MIH apps available in the Apple App and Google Play stores. Twenty-nine popular MIH apps were coded for the presence of 16 BCTs using the mHealth app taxonomy. Popular MIH apps whose purpose was to provide health education or decision-making support to pregnant women or parents/caregivers of infants were included in the final sample. On an average, the reviewed apps included seven BCTs (range 2-16). Techniques such as personalization, review of general or specific goals, macro tailoring, self-monitoring of goals, and health behavior linkages were most frequently present. No differences in the presence of BCTs between paid and free apps were observed. Popular MIH apps typically included only a minority of BCTs found to be useful for health promotion. However, apps developed by healthcare developers incorporated a higher number of BCTs within the app content. Therefore, app developers and policymakers may consider strategies to increase health expert involvement in app design and content delivery.

Practice and market factors associated with provider volume of health information exchange

OBJECTIVE

To assess the practice- and market-level factors associated with the amount of provider health information exchange (HIE) use.

MATERIALS AND METHODS

Provider and practice-level data was drawn from the Meaningful Use Stage 2 Public Use Files from the Centers for Medicare and Medicaid Services, the Physician Compare National Downloadable File, and the Compendium of US Health Systems, among other sources. We analyzed the relationship between provider HIE use and practice and market factors using multivariable linear regression and compared primary care providers (PCPs) to non-PCPs. Provider volume of HIE use is measured as the percentage of referrals sent with electronic summaries of care (eSCR) reported by eligible providers attesting to the Meaningful Use electronic health record (EHR) incentive program in 2016.

RESULTS

Providers used HIE in 49% of referrals; PCPs used HIE in fewer referrals (43%) than non-PCPs (57%). Provider use of products from EHR vendors was negatively related to HIE use, while use of Athenahealth and Greenway Health products were positively related to HIE use. Providers treating, on average, older patients and greater proportions of patients with diabetes used HIE for more referrals. Health system membership, market concentration, and state HIE consent policy were unrelated to provider HIE use.

DISCUSSION

HIE use during referrals is low among office-based providers with the capability for exchange, especially PCPs. Practice-level factors were more commonly associated with greater levels of HIE use than market-level factors.

CONCLUSION

This furthers the understanding that market forces, like competition, may be related to HIE adoption decisions but are less important for use once adoption has occurred.

Assessing variation in state opioid tapering laws: Comparing state laws with the CDC guideline

OBJECTIVE

In 2016, the Center for Disease Control and Prevention released an opioid prescribing guideline for primary care in response to opioid overdose deaths. Despite efforts to encourage safer prescribing practices, experts and federal agencies suspect prescribing guidelines may be misapplied in clinical practice, resulting in abrupt tapering from opioid therapy. Although state laws likely influence prescriber behavior, little is known about state tapering laws. Thus, we examined the scope and variation of state tapering laws compared with federal opioid guidelines.

METHODS

We conducted a comprehensive review of state laws through December 31, 2019 using keyword searches in LexisNexus. Identified laws were coded based on the inclusion of attributes derived from federal opioid guidelines and an expert consensus panel report. We examined whether law attributes were associated with state characteristics, including region, population, governor's political affiliation, opioid prescribing rates, and opioid overdose rates.

RESULTS

We found 27 states and one federal district had law(s) mentioning tapering. Most laws were authored by medical boards or workers' compensation groups (65.6%) while some laws included a penalty (32.8%). Approximately half of guideline attributes (54.2%) were included in state laws; however, only two state's laws cautioned against abrupt tapering. States with higher overdose death rates were more likely to enact a tapering law (p<.001) and have a penalty (p=.007).

CONCLUSIONS

State tapering laws incorporate some federal guideline attributes but most lack attributes deemed critical by experts. Without clear instruction, patients risk inappropriate tapering and discontinuation of opioid therapy. Given these findings, policymakers should consider addressing this gap.

Increasing awareness and uptake of the MenB vaccine on a large university campus

Objective: At a large public university, we aimed to evaluate an intervention designed to increase serogroup B meningococcal (MenB) vaccine uptake and awareness.Methods: Using a pretest-posttest design with a double posttest, we evaluated an intervention conducted by a local foundation and the Florida Department of Health that distributed MenB vaccine on campus and conducted an educational campaign. Prior to intervention activities, we recruited students to complete a survey about their MenB knowledge and attitudes. For survey participants who provided contact information, we sent two follow-up surveys and assessed MenB vaccine records. We used chi-square tests, adjusted for nonindependence, to compare preintervention to postintervention (three-month and one-year) vaccination and attitudes.Results: Among the 686 students with accessible vaccine records, MenB vaccine initiation increased 9% (from 24% to 33%) and completion increased 8% (from 13% to 21%) from before the intervention to one year after the intervention. When restricting to students who completed the relevant follow-up surveys, the percentage of students who heard of the MenB vaccine increased by 15% (p > .001) from before the intervention to three months after (n = 188 students) and maintained a 10% increase (p > .001) one year after the intervention (n = 261 students). Among students that heard of the MenB vaccine, the percentage of students who thought they needed the MenB vaccine even though they received the MenACWY increased 14% (p = .03) by the three-month postintervention survey and up to 18% by the one-year follow-up (p = .002).Conclusions: A university-wide, on-campus vaccination and educational campaign increased college students' MenB vaccine initiation, completion, and knowledge.Clinicaltrials.gov ID: NCT02975596.

Assessing the practice of data quality evaluation in a national clinical data research network through a systematic scoping review in the era of real-world data

Objective

To synthesize data quality (DQ) dimensions and assessment methods of real-world data, especially electronic health records, through a systematic scoping review and to assess the practice of DQ assessment in the national Patient-centered Clinical Research Network (PCORnet).

Materials and Methods

We started with 3 widely cited DQ literature—2 reviews from Chan et al (2010) and Weiskopf et al (2013a) and 1 DQ framework from Kahn et al (2016)—and expanded our review systematically to cover relevant articles published up to February 2020. We extracted DQ dimensions and assessment methods from these studies, mapped their relationships, and organized a synthesized summarization of existing DQ dimensions and assessment methods. We reviewed the data checks employed by the PCORnet and mapped them to the synthesized DQ dimensions and methods.

Results

We analyzed a total of 3 reviews, 20 DQ frameworks, and 226 DQ studies and extracted 14 DQ dimensions and 10 assessment methods. We found that completeness, concordance, and correctness/accuracy were commonly assessed. Element presence, validity check, and conformance were commonly used DQ assessment methods and were the main focuses of the PCORnet data checks.

Discussion

Definitions of DQ dimensions and methods were not consistent in the literature, and the DQ assessment practice was not evenly distributed (eg, usability and ease-of-use were rarely discussed). Challenges in DQ assessments, given the complex and heterogeneous nature of real-world data, exist.

Conclusion

The practice of DQ assessment is still limited in scope. Future work is warranted to generate understandable, executable, and reusable DQ measures.

Research informatics and the COVID-19 pandemic: Challenges, innovations, lessons learned, and recommendations

The recipients of NIH's Clinical and Translational Science Awards (CTSA) have worked for over a decade to build informatics infrastructure in support of clinical and translational research. This infrastructure has proved invaluable for supporting responses to the current COVID-19 pandemic through direct patient care, clinical decision support, training researchers and practitioners, as well as public health surveillance and clinical research to levels that could not have been accomplished without the years of ground-laying work by the CTSAs. In this paper, we provide a perspective on our COVID-19 work and present relevant results of a survey of CTSA sites to broaden our understanding of the key features of their informatics programs, the informatics-related challenges they have experienced under COVID-19, and some of the innovations and solutions they developed in response to the pandemic. Responses demonstrated increased reliance by healthcare providers and researchers on access to electronic health record (EHR) data, both for local needs and for sharing with other institutions and national consortia. The initial work of the CTSAs on data capture, standards, interchange, and sharing policies all contributed to solutions, best illustrated by the creation, in record time, of a national clinical data repository in the National COVID-19 Cohort Collaborative (N3C). The survey data support seven recommendations for areas of informatics and public health investment and further study to support clinical and translational research in the post-COVID-19 era.

Perspectives of Pregnant and Breastfeeding Women on Participating in Longitudinal Mother-Baby Studies Involving Electronic Health Records: Qualitative Study

BACKGROUND

Electronic health records (EHRs) hold great potential for longitudinal mother-baby studies, ranging from assessing study feasibility to facilitating patient recruitment to streamlining study visits and data collection. Existing studies on the perspectives of pregnant and breastfeeding women on EHR use have been limited to the use of EHRs to engage in health care rather than to participate in research.

OBJECTIVE

The aim of this study is to explore the perspectives of pregnant and breastfeeding women on releasing their own and their infants' EHR data for longitudinal research to identify factors affecting their willingness to participate in research.

METHODS

We conducted semistructured interviews with pregnant or breastfeeding women from Alachua County, Florida. Participants were asked about their familiarity with EHRs and EHR patient portals, their comfort with releasing maternal and infant EHR data to researchers, the length of time of the data release, and whether individual research test results should be included in the EHR. The interviews were transcribed verbatim. Transcripts were organized and coded using the NVivo 12 software (QSR International), and coded data were thematically analyzed using constant comparison.

RESULTS

Participants included 29 pregnant or breastfeeding women aged between 22 and 39 years. More than half of the sample had at least an associate degree or higher. Nearly all participants (27/29, 93%) were familiar with EHRs and had experience accessing an EHR patient portal. Less than half of the participants (12/29, 41%) were willing to make EHR data available to researchers for the duration of a study or longer. Participants' concerns about sharing EHRs for research purposes emerged in 3 thematic domains: privacy and confidentiality, transparency by the research team, and surrogate decision-making on behalf of infants. The potential release of sensitive or stigmatizing information, such as mental or sexual health history, was considered in the decisions to release EHRs. Some participants viewed the simultaneous use of their EHRs for both health care and research as potentially beneficial, whereas others expressed concerns about mixing their health care with research.

CONCLUSIONS

This exploratory study indicates that pregnant and breastfeeding women may be willing to release EHR data to researchers if researchers adequately address their concerns regarding the study design, communication, and data management. Pregnant and breastfeeding women should be included in EHR-based research as long as researchers are prepared to address their concerns.

Developing and Validating a Computable Phenotype for the Identification of Transgender and Gender Nonconforming Individuals and Subgroups

Transgender and gender nonconforming (TGNC) individuals face significant marginalization, stigma, and discrimination. Under-reporting of TGNC individuals is common since they are often unwilling to self-identify. Meanwhile, the rapid adoption of electronic health record (EHR) systems has made large-scale, longitudinal real-world clinical data available to research and provided a unique opportunity to identify TGNC individuals using their EHRs, contributing to a promising routine health surveillance approach. Built upon existing work, we developed and validated a computable phenotype (CP) algorithm for identifying TGNC individuals and their natal sex (i.e., male-to-female or female-to-male) using both structured EHR data and unstructured clinical notes. Our CP algorithm achieved a 0.955 F1-score on the training data and a perfect F1-score on the independent testing data. Consistent with the literature, we observed an increasing percentage of TGNC individuals and a disproportionate burden of adverse health outcomes, especially sexually transmitted infections and mental health distress, in this population.

Receipt of opioids and patient care experiences among nonsurgical hospitalized adults

OBJECTIVE

To examine the association between receipt of opioids and patient care experiences among nonsurgical hospitalized adults.

DATA SOURCES

A total of 17 691 patient-level responses to the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) patient care experience survey linked to medical records from nonsurgical hospitalizations in an 11-hospital health care system in a Midwestern state, years 2011-2016.

STUDY DESIGN

We conducted a pooled cross-sectional study that used propensity score matching analyses and logistic regression to estimate the relationship between patients' care experience measures (overall and pain-specific) and their receipt of opioids while hospitalized. In supplementary analyses, we used the same propensity score matching methods to estimate the relationship between patient care experience measures and receipt of opioids in four patient subgroups based on average patient-reported pain during hospitalization (no pain; mild pain; moderate pain; and severe pain).

PRINCIPAL FINDINGS

Receipt of opioids was not associated with patient care experience measures in our main analysis. In our supplementary analysis, we found lower ratings for pain control among hospitalizations for patients who reported moderate pain (Marginal Effects = -4.5 percent; P value = .015).

CONCLUSIONS

Counter to some previous studies, we observed that receipt of opioids was not associated with patient care experience measures for nonsurgical hospitalized adults. These findings may be due to different pain experiences of adults hospitalized for nonsurgical versus surgical reasons.

An Electronic Tool to Support Patient-Centered Broad Consent: A Multi-Arm Randomized Clinical Trial in Family Medicine

PURPOSE

Patients are frequently asked to share their personal health information. The objective of this study was to compare the effects on patient experiences of 3 electronic consent (e-consent) versions asking patients to share their health records for research.

METHODS

A multi-arm randomized controlled trial was conducted from November 2017 through November 2018. Adult patients (n = 734) were recruited from 4 family medicine clinics in Florida. Using a tablet computer, participants were randomized to (1) a standard e-consent (standard), (2) an e-consent containing standard information plus hyperlinks to additional interactive details (interactive), or (3) an e-consent containing standard information, interactive hyperlinks, and factual messages about data protections and researcher training (trust-enhanced). Satisfaction (1 to 5), subjective understanding (0 to 100), and other outcomes were measured immediately, at 1 week, and at 6 months.

RESULTS

A majority of participants (94%) consented to future uses of their health record information for research. No differences in study outcomes between versions were observed at immediate or 1-week follow-up. At 6-month follow-up, compared with the standard e-consent, participants who used the interactive e-consent reported greater satisfaction (B = 0.43; SE = 0.09; P <.001) and subjective understanding (B = 18.04; SE = 2.58; P <.001). At 6-month follow-up, compared with the interactive e-consent, participants who used the trust-enhanced e-consent reported greater satisfaction (B = 0.9; SE = 1.0; P <.001) and subjective understanding (B = 32.2; SE = 2.6, P <.001).

CONCLUSIONS

Patients who used e-consents with interactive research details and trust-enhancing messages reported higher satisfaction and understanding at 6-month follow-up. Research institutions should consider developing and further validating e-consents that interactively deliver information beyond that required by federal regulations, including facts that may enhance patient trust in research.

Does an interactive trust-enhanced electronic consent improve patient experiences when asked to share their health records for research? A randomized trial

OBJECTIVE

In the context of patient broad consent for future research uses of their identifiable health record data, we compare the effectiveness of interactive trust-enhanced e-consent, interactive-only e-consent, and standard e-consent (no interactivity, no trust enhancement).

MATERIALS AND METHODS

A randomized trial was conducted involving adult participants making a scheduled primary care visit. Participants were randomized into 1 of the 3 e-consent conditions. Primary outcomes were patient-reported satisfaction with and subjective understanding of the e-consent. Secondary outcomes were objective knowledge, perceived voluntariness, trust in medical researchers, consent decision, and time spent using the application. Outcomes were assessed immediately after use of the e-consent and at 1-week follow-up.

RESULTS

Across all conditions, participants (N = 734) reported moderate-to-high satisfaction with consent (mean 4.3 of 5) and subjective understanding (79.1 of 100). Over 94% agreed to share their health record data. No statistically significant differences in outcomes were observed between conditions. Irrespective of condition, black participants and those with lower education reported lower satisfaction, subjective understanding, knowledge, perceived voluntariness, and trust in medical researchers, as well as spent more time consenting.

CONCLUSIONS

A large majority of patients were willing to share their identifiable health records for research, and they reported positive consent experiences. However, incorporating optional additional information and messages designed to enhance trust in the research process did not improve consent experiences. To improve poorer consent experiences of racial and ethnic minority participants and those with lower education, other novel consent technologies and processes may be valuable. (An Interactive Patient-Centered Consent for Research Using Medical Records; NCT03063268).

Characterizing variability in state-level regulations governing opioid treatment programs

INTRODUCTION

The opioid use crisis has left nearly 1 million people in need of treatment. States have focused primarily on policies aimed at decreasing the prevalence of opioid use disorder. However, opioid treatment programs (OTPs), an evidence-based modality which can prevent and decrease opioid-related mortality and morbidity, remain highly complex with variation in treatment by state. Evidence-based state-level regulation of OTPs can be a powerful tool and may help improve the unmet need for treatment. This study characterized the variability in state laws that regulate OTPs and examines how this variability is associated with state characteristics. Our data provides an opportunity for policymakers to consider regulations that increase access to care and retention in OTPs, which could improve population health.

MATERIALS AND METHODS

Utilizing policy mapping techniques, we identified all regulations governing OTPs in effect on January 1, 2017 and determined whether the most common regulations were consistent with best practices. We then examined how the number and type of regulations were associated with state characteristics. All policy mapping research was conducted between November 2017 and March 2019.

RESULTS

We identified 89 different regulations, the most common of which exists in fewer than half of all states; and most exist in <25% of states. Eighteen of the 30 most common regulations were inconsistent with best practice recommendations. Overall, variability in the number and type of OTP regulations was related to geographic location as opposed to state size.

CONCLUSIONS

Wide-ranging variability exists in the regulations of OTPs across the U.S. Most state OTP regulations are not congruent with best practices.

Clinical perspectives on hospitals' role in the opioid epidemic

BACKGROUND

Policymakers, legislators, and clinicians have raised concerns that hospital-based clinicians may be incentivized to inappropriately prescribe and administer opioids when addressing pain care needs of their patients, thus potentially contributing to the ongoing opioid epidemic in the United States. Given the need to involve all healthcare settings, including hospitals, in joint efforts to curb the opioid epidemic, it is essential to understand if clinicians perceive hospitals as contributors to the problem. Therefore, we examined clinical perspectives on the role of hospitals in the opioid epidemic.

METHODS

We conducted individual semi-structured interviews with 23 clinicians from 6 different acute care hospitals that are part of a single healthcare system in the Midwestern United States. Our participants were hospitalists (N = 12), inpatient registered nurses (N = 9), and inpatient adult nurse practitioners (N = 2). In the interviews, we asked clinicians whether hospitals play a role in the opioid epidemic, and if so, how hospitals may contribute to the epidemic. We used a qualitative thematic analysis approach to analyze coded text for patterns and themes and examined potential differences in themes by respondent type using Dedoose software.

RESULTS

The majority of clinicians believed hospitals contribute to the opioid epidemic. Multiple clinicians cited Center for Medicare and Medicaid Services' (CMS) reimbursement policy and the Joint Commission's report as drivers of inappropriate opioid prescribing in hospitals. Furthermore, numerous clinicians stated that opioids are inappropriately administered in the emergency department (ED), potentially as a mechanism to facilitate discharge and prevent re-admission. Many clinicians also described how overreliance on pre-populated pain care orders for surgical (orthopedic) patients, may be contributing to inappropriate opioid use in the hospital. Finally, clinicians suggested the following initiatives for hospitals to help address the crisis: 1) educating patients about negative consequences of using opioids long-term and setting realistic pain expectations; 2) educating medical staff about appropriate opioid prescribing practices, particularly for patients with complex chronic conditions (chronic pain; opioid use disorder (OUD)); and 3) strengthening the hospital leadership efforts to decrease inappropriate opioid use.

CONCLUSIONS

Our findings can inform efforts at decreasing inappropriate opioid use in hospitals.

Factors Related to User Ratings and User Downloads of Mobile Apps for Maternal and Infant Health: Cross-Sectional Study

BACKGROUND

Mobile health apps related to maternal and infant health (MIH) are prevalent and frequently used. Some of these apps are extremely popular and have been downloaded over 5 million times. However, the understanding of user behavior and user adoption of these apps based on consumer preferences for different app features and categories is limited.

OBJECTIVE

This study aimed to examine the relationship between MIH app characteristics and users' perceived satisfaction and intent to use.

METHODS

The associations between app characteristics, ratings, and downloads were assessed in a sample of MIH apps designed to provide health education or decision-making support to pregnant women or parents and caregivers of infants. Multivariable linear regression was used to assess the relationship between app characteristics and user ratings, and ordinal logistic regression was used to assess the relationship between app characteristics and user downloads.

RESULTS

The analyses of user ratings and downloads included 421 and 213 apps, respectively. The average user rating was 3.79 out of 5. Compared with the Apple App Store, the Google Play Store was associated with high user ratings (beta=.33; P=.005). Apps with higher standardized user ratings (beta=.80; P<.001), in-app purchases (beta=1.12; P=.002), and in-app advertisements (beta=.64; P=.02) were more frequently downloaded. Having a health care organization developer as part of the development team was neither associated with user ratings (beta=-.20; P=.06) nor downloads (beta=-.14; P=.63).

CONCLUSIONS

A majority of MIH apps are developed by non-health care organizations, which could raise concern about the accuracy and trustworthiness of in-app information. These findings could benefit app developers in designing better apps and could help inform marketing and development strategies. Further work is needed to evaluate the clinical accuracy of information provided within the apps.