Patient reported outcomes - experiences with implementation in a University Health Care setting

Linking QuickDASH and PROMIS Upper-Extremity Computer-Adaptive Test Scores in Hand Surgery: A Crosswalk Study

PURPOSE

Assessment of patient-reported outcome measures (PROMs) for hand and upper-extremity surgery patients using measures such as the Quick Disabilities of the Arm, Shoulder, and Hand (qDASH), as well as general measures including the Patient-Reported Outcomes Measurement Information System Upper Extremity Physical Function domain via a Computer-Adaptive Test (PROMIS UE CAT), has become commonplace. The aim of this study was to link, for crosswalking, the qDASH measure to both versions of the PROMIS UE CAT (v1.2 and v2.0).

METHODS

We included 18,944 hand and upper-extremity patients who completed both versions of the PROMIS UE CAT and the qDASH at the same clinical encounter. Shoulder pathology was excluded. Score linkage was performed using the R package equate, and multiple equating models (linear regression, identity, mean, linear, equipercentile, and circle-arc models) were used to establish crosswalk tables.

RESULTS

Mean qDASH and PROMIS UE CAT v1.2 scores were 38.2 (SD = 23.1) and 36.6 (SD = 9.8), respectively. Mean qDASH and PROMIS UE CAT v2.0 scores were 37.3 (SD = 21.8) and 38.3 (SD = 10.4), respectively. Pearson correlations had very strong linear relationships between the qDASH and the PROMIS UE CAT v1.2 and PROMIS UE CAT v2.0 (r = -0.83 [-0.84, -0.92] and r = -0.80 [-0.81, -0.80], respectively). For the equipercentile equating models, the intraclass correlation coefficient (ICC) had very strong positive relationships to linking measures with ICC = 0.85 (0.84, 0.86) for the qDASH-UE CAT v1.2 crosswalk and ICC = 0.83 (0.82, 0.84) for the qDASH-UE CAT v2.0 crosswalk.

CONCLUSIONS

The linkages establish crosswalk tables using equipercentile equating models to convert the PROMIS UE CAT v1.2 and v2.0 scores to the qDASH and vice versa.

CLINICAL RELEVANCE

This study provides crosswalk tables for commonly collected PROMs in hand surgery, increasing the comparability of results between centers using different PROMs to study the same conditions or treatments.

Sex Differences in Patient-Reported Depression Following Vascular Surgery Procedures

INTRODUCTION

Female patients frequently experience worse clinical outcomes than male patients after undergoing vascular surgery procedures. However, it is unclear whether these sex-based disparities also impact mental health outcomes. This study was designed to investigate sex differences in patient-reported outcome measures of depression for patients undergoing vascular surgery.

METHODS

We retrospectively analyzed 107 patients (73 males and 34 females) who underwent vascular surgery procedures between January 2016 and April 2023. These patients completed a Patient-Reported Outcome Measurement Information System (PROMIS) Item Bank v1.0-Depression assessment 90 d before surgery and at least once after surgery. After stratifying patients by sex, we analyzed changes in PROMIS depression scores using a multiple mixed-effects linear regression model. Then, logistic regression was used to compare the proportion of patients who achieved a clinically meaningful difference in PROMIS depression score within 15 mo after surgery.

RESULTS

There was no significant difference between female and male patients among rates of complications, length of hospital stay, or rates of nonhome discharge. However, female sex was associated with significantly improved PROMIS depression scores after surgery compared to male sex (P = 0.034). Furthermore, female patients were over 3-fold more likely than male patients to reach the minimal clinically important difference threshold for improvement in PROMIS depression scores (odds ratio 4.66, 95% confidence interval 1.39-15.61).

CONCLUSIONS

These results suggest that female sex is associated with improved patient-reported measures of depression after undergoing vascular surgery. Clinicians should consider these mental health benefits when evaluating female patients for vascular interventions.

Comprehensive analysis of same day discharge after atrial fibrillation ablation: Clinical, cost, and patient reported outcomes

BACKGROUND

Same day discharge (SDD) following atrial fibrillation (AF) ablation procedure has emerged as routine practice, and primarily driven by operator discretion. However, the impacts of SDD on clinical outcomes, healthcare system costs, and patient reported outcomes (PROs) have not been systematically studied.

METHODS

We retrospectively analyzed patients undergoing routine AF ablation procedures with SDD versus overnight observation (NSDD). After propensity adjustment we compared postprocedure adverse events (AEs), healthcare system costs, and changes in PROs.

RESULTS

We identified 310 cases, with 159 undergoing SDD and 151 staying at least one midnight in the hospital (NSDD). Compared with NSDD, SDD patients were similar age (mean 64 vs. 66, p = 0.3), sex (26% female vs. 27%, p = 0.8), and with lower mean CHADS2-VA2Sc scores (2.0 vs. 2.7; p < 0.011). The primary outcome of AEs was noninferior in SDD versus NSDD patients (odds ratio 0.45, 95% confidence interval 0.21-0.99; noninferiority margin of 10%). There were also no differences in overall cost to the healthcare system between SDD and NSDD (p = 0.11). PROs numerically favored SDD (p = NS for all scores).

CONCLUSIONS

Physician selection for SDD appears at least as safe as NSDD with respect to clinical outcomes and SDD is not significantly less costly to the health system. There is a trend towards more favorable, general PROs among SDD patients. Routine SDD should be strongly considered for patients undergoing routine AF ablation procedures.

Recommendations to address respondent burden associated with patient-reported outcome assessment

Patient-reported outcomes (PROs) are increasingly used in healthcare research to provide evidence of the benefits and risks of interventions from the patient perspective and to inform regulatory decisions and health policy. The use of PROs in clinical practice can facilitate symptom monitoring, tailor care to individual needs, aid clinical decision-making and inform value-based healthcare initiatives. Despite their benefits, there are concerns that the potential burden on respondents may reduce their willingness to complete PROs, with potential impact on the completeness and quality of the data for decision-making. We therefore conducted an initial literature review to generate a list of candidate recommendations aimed at reducing respondent burden. This was followed by a two-stage Delphi survey by an international multi-stakeholder group. A consensus meeting was held to finalize the recommendations. The final consensus statement includes 19 recommendations to address PRO respondent burden in healthcare research and clinical practice. If implemented, these recommendations may reduce PRO respondent burden.

Seamless Integration of Computer-Adaptive Patient Reported Outcomes into an Electronic Health Record

BACKGROUND

Patient-reported outcome (PRO) measures have become an essential component of quality measurement, quality improvement, and capturing the voice of the patient in clinical care. In 2004, the National Institutes of Health endorsed the importance of PROs by initiating the Patient-Reported Outcomes Measurement Information System (PROMIS), which leverages computer-adaptive tests (CATs) to reduce patient burden while maintaining measurement precision. Historically, PROMIS CATs have been used in a large number of research studies outside the electronic health record (EHR), but growing demand for clinical use of PROs requires creative information technology solutions for integration into the EHR.

OBJECTIVES

This paper describes the introduction of PROMIS CATs into the Epic Systems EHR at a large academic medical center using a tight integration; we describe the process of creating a secure, automatic connection between the application programming interface (API) which scores and selects CAT items and Epic.

METHODS

The overarching strategy was to make CATs appear indistinguishable from conventional measures to clinical users, patients, and the EHR software itself. We implemented CATs in Epic without compromising patient data security by creating custom middleware software within the organization's existing middleware framework. This software communicated between the Assessment Center API for item selection and scoring and Epic for item presentation and results. The middleware software seamlessly administered CATs alongside fixed-length, conventional PROs while maintaining the display characteristics and functions of other Epic measures, including automatic display of PROMIS scores in the patient's chart. Pilot implementation revealed differing workflows for clinicians using the software.

RESULTS

The middleware software was adopted in 27 clinics across the hospital system. In the first 2 years of hospital-wide implementation, 793 providers collected 70,446 PROs from patients using this system.

CONCLUSION

This project demonstrated the importance of regular communication across interdisciplinary teams in the design and development of clinical software. It also demonstrated that implementation relies on buy-in from clinical partners as they integrate new tools into their existing clinical workflow.

Applying the Patient Empowerment, Engagement, and Activation Survey to Improve Patient Outcomes

This article focuses on applying the patient empowerment, engagement, and activation (PEEA) model as an implementation strategy to achieve patient- or person-centered care. There is substantial evidence linking PEEA to improved patient care outcomes and reduced healthcare costs. Interrelationships among the 3 concepts are discussed and how the PEEA survey can be used to evaluate patients' activation for self-care postdischarge and their perceptions of nurses' empowering and engaging them during the care encounter.

Development and Psychometric Validation of Patient-Reported Outcome Measures (PROMs BCC-20) for Assessing Comfort during Chemotherapy in Breast Cancer Patients

OBJECTIVE

The study aims to develop and psychometric validate Patient-Reported Outcomes Measures for Assessing Comfort during Chemotherapy in Breast Cancer Patients (PROMs BCC-20).

METHODS

This study was conducted in two phases: 1) items were developed from the literature review and in-depth interviews, and 2) Exploratory factor analysis (EFA), Confirmatory factor analysis (CFA), and concurrent validity were performed to evaluate construct validity. The participants were cancer stage I-IIIC, adult females, performance status was assessed by Eastern Cooperative Oncology Group (ECOG) score ≤ 2 after receiving the second cycle of adjuvant chemotherapy and selected by purposive sampling method. For each group of EFA and CFA was 250 participants.

RESULT

Five hundred breast cancer patients during adjuvant chemotherapy were recruited from three tertiary cancer centers. A succession of EFA using principal axis factoring with Promax rotation revealed four dimensions yielded a seven factors solution, explaining a 60.07 percent variance. CFA contains 20 items with five factors; 1) social function, four items; 2) digestive function, three items; 3) emotional function, six items; 4) environmental function, three items; and 5) sleep quality, four items via maximum likelihood with bootstrapping indicated a good fit model (SRMR = 0.045, RMSEA = 0.040, CFI = 0.947, and TLI = 0.935). The Cronbach's alpha of 0.86 demonstrated strong internal consistency reliability. Pearson's correlation coefficient showed acceptable criterion validity.

CONCLUSION

The PROMS BCC-20 provides good psychometric properties and practical patients' direct reports of comfort in breast cancer patients during chemotherapy. The PROMs BCC-20 should be standardized for comfort measurement and tailor-made nursing care to provide patient satisfaction and good nursing outcomes.

Measuring What Matters: A Qualitative Study of the Relevance and Clinical Utility of PROMIS Surveys in Systemic Lupus Erythematosus

OBJECTIVE

To evaluate the relevance and clinical utility of the Patient-Reported Outcomes Measurement Information System (PROMIS) surveys in patients with systemic lupus erythematosus (SLE).

METHODS

Adults with SLE receiving routine outpatient care at a tertiary care academic medical center participated in a qualitative study. Patients completed PROMIS computerized adaptive tests (CATs) in 12 selected domains and rated the relevance of each domain to their experience with SLE. Focus groups and interviews were conducted to elucidate the relevance of the PROMIS surveys, identify additional domains of importance, and explore the utility of the surveys in clinical care. Focus group and interview transcripts were coded, and a thematic analysis was performed using an iterative inductive process.

RESULTS

Twenty-eight women and 4 men participated in 4 focus groups and 4 interviews, respectively. Participants endorsed the relevance and comprehensiveness of the selected PROMIS domains in capturing the effect of SLE on their lives. They ranked fatigue, pain interference, sleep disturbance, physical function, and applied cognition abilities as the most salient health-related quality of life (HRQOL) domains. They suggested that the disease-agnostic PROMIS questions holistically captured their lived experience of SLE and its common comorbidities. Participants were enthusiastic about using PROMIS surveys in clinical care and described potential benefits in enabling disease monitoring and management, facilitating communication, and empowering patients.

CONCLUSION

PROMIS includes the HRQOL domains that are of most importance to individuals with SLE. Patients suggest that these universal tools can holistically capture the impact of SLE and enhance routine clinical care.

PROMs and PREMs in routine perinatal care: mixed methods evaluation of their implementation into integrated obstetric care networks

BACKGROUND

In the transition towards value-based healthcare, patient-reported outcome and experience measures (PROM and PREM) are recommended by international collaborations and government programs to guide clinical practice and quality improvement. For many conditions, using PROM/PREM over the complete continuum of care requires implementation across care organizations and disciplines. Along PROM/PREM implementation in obstetric care networks (OCN), we aimed to evaluate implementation outcomes and the processes influencing these outcomes in the complex context of care networks across the continuum of perinatal care.

METHODS

Three OCN in the Netherlands implemented PROM/PREM in routine practice, using an internationally developed outcomes set with care professionals and patient advocates. Their aim was to use PROM/PREM results individually to guide patient-specific care decisions and at group-level to improve quality of care. The implementation process was designed following the principles of action research: iteratively planning implementation, action, data generation and reflection to refine subsequent actions, involving both researchers and care professionals. During the one-year implementation period in each OCN, implementation outcomes and processes were evaluated in this mixed-methods study. Data generation (including observation, surveys and focus groups) and analysis were guided by two theoretical implementation frameworks: the Normalization Process Theory and Proctor's taxonomy for implementation outcomes. Qualitative findings were supplemented with survey data to solidify findings in a broader group of care professionals.

RESULTS

Care professionals in OCN found the use of PROM/PREM acceptable and appropriate, recognized their benefits and felt facilitated in their patient-centered goals and vision. However, feasibility for daily practice was low, mainly due to IT issues and time constraints. Hence PROM/PREM implementation did not sustain, but strategies for future PROM/PREM implementation were formulated in all OCN. Processes contributing positively to implementation outcomes were internalization (understand the value) and initiation (driven by key-participants), whereas challenges in relational integration (maintain confidence) and reconfiguration (refine activities) affected implementation negatively.

CONCLUSION

Although implementation did not sustain, network-broad PROM/PREM use in clinic and quality improvement matched professionals' motivation. This study provides recommendations to implement PROM/PREM meaningfully in practice in ways that support professionals in their drive towards patient-centered care. In order for PROM/PREM to fulfill their potential for value-based healthcare, our work highlights the need for sustainable IT infrastructures, as well as an iterative approach to refine their complex implementation into local contexts.

Using co-production to implement patient reported outcome measures in third sector organisations: a mixed methods study

Background

Third sector organisations such as charities and community groups are using Patient Reported Outcome Measures (PROMs) at an aggregated service level to demonstrate their impact to commissioners to generate or retain funding. Despite this motivation, organisations can struggle with implementing PROMs. Previous studies have identified facilitators including organisations using an appropriate measure, co-producing the PROMs process with staff, and investing resources to support the use of measures. However, to date no studies have applied this learning to third sector organisations to evaluate whether taking an evidence-informed implementation approach improves the use of PROMs.

Methods

A Community-Based Participatory Research approach was used which involved university-based researchers supporting two third sector organisations to implement PROMs. The researchers provided evidence-informed advice and training. The organisations were responsible for implementing PROMs. The researchers evaluated implementation through a mixed methods approach including five key informant interviews, four evaluation groups and analysis of collected PROMs data (n = 313).

Results

Both third sector organisations faced considerable constraints in incorporating known facilitators and addressing barriers. The organisations involved staff in choosing an acceptable measure. However, competing priorities including external pressures to use specific PROMs, busy workloads and staff opinions created challenges to using measures. Investment of time and energy into developing an outcomes-based organisational culture was key to enable the prioritisation of PROMs. For example, discussing PROMs in supervision so that they were viewed as part of people’s job roles. Organisations found that implementation took several years and was disrupted by other pressures.

Conclusions

Whilst organisations were motivated to implement PROMs to obtain or retain funding, they faced considerable practical and ideological challenges. Consequently, some stakeholders felt that alternative methods to measuring impact could potentially be more feasible than PROMs.

Supplementary Information

The online version contains supplementary material available at 10.1186/s41687-022-00485-4.

Ejection Fraction Improvement Does Not Reflect Changes in Quality of Life Following Cardiac Resynchronization Therapy

OBJECTIVE

To determine if health-related quality of life (HRQoL) improvement after cardiac resynchronization therapy (CRT) correlates with improved left-ventricular ejection fraction (EF).

BACKGROUND

CRT was reported to improve EF and HRQoL in clinical trials of heart failure with reduced EF (HFrEF). It is unknown if improvements in HRQoL reflect EF response to CRT.

METHODS

We included HFrEF patients who underwent CRT and had both pre- and post-CRT HRQoL assessment. EF response was categorized as absent (0% change or decrease), modest (0%-19% increase), or significant ( > 20% increase). We examined the associations between EF response and generic (PROMIS) and HF-specific (KCCQ-12) HRQoL.

RESULTS

The group included 115 patients with mean age of 65 years and baseline EF of 31%; 39% were female (n = 45). Nineteen percent (n = 22) had significant, 57% (n = 66) modest, and 23% (n = 27) absent EF responses. AF burden across significant (8.9%), modest (4.8%), and absent EF responders (1.4%) was similar ( P = 0.20). Significant improvements in KCCQ-12 (43.4-57.5, P = 0.003), current health visual analog scale (49.1-55.9, P = 0.042), PROMIS fatigue (58.9-55.1, P = 0.026), and PROMIS satisfaction (42.7-46.4, P = 0.020) resulted following CRT across all groups. There was no association between significant EF improvement and HRQoL by KCCQ-12 (nonresponse, 44.4%; modest response, 33.3%; and significant response, 22.2%) at 1 year ( P = 0.52 across all groups).

CONCLUSION

CRT was associated with a modest to significant EF response in a majority of patients. However, EF response did not significantly correlate with generic or HF-specific HRQoL measures. Further investigations are warranted into determinants of improved HRQoL following CRT.

Key considerations to reduce or address respondent burden in patient-reported outcome (PRO) data collection

Patient-reported outcomes (PROs) are used in clinical trials to provide evidence of the benefits and risks of interventions from a patient perspective and to inform regulatory decisions and health policy. The collection of PROs in routine practice can facilitate monitoring of patient symptoms; identification of unmet needs; prioritisation and/or tailoring of treatment to the needs of individual patients and inform value-based healthcare initiatives. However, respondent burden needs to be carefully considered and addressed to avoid high rates of missing data and poor reporting of PRO results, which may lead to poor quality data for regulatory decision making and/or clinical care.

The collection of patient-reported outcomes (PROs) may capture patients’ assessments of their health status. Here authors highlight PRO-specific issues that should be considered to minimise respondent burden in clinical trials and routine care.

Significant Factors of High Performance Outcomes for Tibial Plafond Fractures

BACKGROUND

Limited research exists about high performance postsurgical tibial plafond fractures. This study aimed to identify aspects of the plafond fracture injury and care associated with "high performance" based on Patient Reported Outcome Measurement Information System (PROMIS) Physical Function (PF) scores.

METHODS

Tibial plafond fracture patients with minimum 12-month follow-up treated at a level 1 trauma center from 2006 to 2019 were categorized into high (top 25%) vs average-low (AL) (bottom 75%) performers based on PROMIS PF scores. Demographics and fracture characteristics of high and AL performers were compared. Variables with a P value less than .1 were used in stepwise logistic regressions.

RESULTS

The final cohort of 198 patients was divided into high (n=51) and AL (n=147) performers based on PF scores. The mean PF scores for the high and AL groups were 58 (SD=5.3) and 41.9 (SD=6.5), respectively. The mean PROMIS Pain Interference scores for the high performers and AL were 43.3 (SD=4.9) and 56.8 (SD=8.6) (P < .001), respectively. FAAM activities of daily living (high 95.4 [SD=5.5] vs AL 70.4 [SD=19.8], P < .001) and FAAM sports (high 76.8 [SD=21.3] vs AL 27.3 [SD=28.9], P < .001) subscale scores were significantly greater in the high-performing group. Sex, age, marital status, and diabetes were nonsignificant factors in univariate analysis. Significant demographic factors associated with high performance from univariate analysis were Caucasian race, private insurance, no tobacco use, lower body mass index (BMI), and mechanism of injury. Patients with fracture characteristics of OTA 43C (complete articular) fractures, bone loss, open fractures, or nonexcellent reduction were significantly less likely to be high performers. Additionally, less than excellent reductions were more common in more complex fracture patterns. Multivariable regression modeling showed that higher BMI, OTA/AO 43-C (complete articular) fracture classification, and open fracture were significant independent risk factors for reducing likelihood of high performance.

CONCLUSION

Significant independent factors associated with high performance after tibial plafond fracture are lower BMI, closed fracture, and OTA/AO 43-B (partial articular) fracture.

Capturing patient-reported outcomes: paper versus electronic survey administration

Objective: To compare the capture rates and costs of paper patient-reported outcomes (pPRO) administered in-clinic and electronic PROs (ePRO) collected through emails and texts. Design: Retrospective review. Setting: Level 1 trauma center. Patients/Participants: The pPRO program enrolled 2164 patients for postsurgical follow-up in 4 fracture types: ankle, distal radius, proximal humerus, and implant removal from 2012 to 2017. The ePRO program enrolled 3096 patients in 13 fracture types from 2018 to 2020. Among the patients enrolled in the ePRO program, 1296 patients were matched to the 4 original fracture types and time points. Main Outcome Measures: PRO capture rates in 4 fracture types by matched time point and estimated cost of each program per enrolled patient. Results: At first follow-up, pPRO provided a higher capture rate than ePRO for 3 of 4 fracture types except for implant removal (P < 0.05). However, at 6-month and 1-year follow-ups, ePRO demonstrated statistically significant higher capture rates when compared with pPRO for all applicable modules (P < 0.05). The average cost for the pPRO program was $171 per patient versus $56 per patient in the ePRO program. Patients were 1.19 times more likely to complete ePRO compared with pPRO (P = 0.007) after controlling for age, sex, fracture type, and time point. Conclusion: The electronic PRO service has improved long-term capture rates compared with paper PROs, while minimizing cost. A combined program that includes both in-clinic and out of clinic effort may be the ideal model for collection of PROs. Level of Evidence: Level 3.

Sociodemographic Factors Are Associated with Patient-Reported Outcome Measure Completion in Orthopaedic Surgery: An Analysis of Completion Rates and Determinants Among New Patients

Patient-reported outcome measures (PROMs) and, specifically, the Patient-Reported Outcomes Measurement Information System (PROMIS), are increasingly utilized for clinical research, clinical care, and health-care policy. However, completion of these outcome measures can be inconsistent and challenging. We hypothesized that sociodemographic variables are associated with the completion of PROM questionnaires. The purposes of the present study were to calculate the completion rate of assigned PROM forms and to identify sociodemographic and other variables associated with completion to help guide improved collection efforts.

Methods

All new orthopaedic patients at a single academic medical center were identified from 2016 to 2020. On the basis of subspecialty and presenting condition, patients were assigned certain PROMIS forms and legacy PROMs. Demographic and clinical information was abstracted from the electronic medical record. Bivariate analyses were performed to compare characteristics among those who completed assigned PROMs and those who did not. A multivariable logistic regression model was created to determine which variables were associated with successful completion of assigned PROMs.

Results

Of the 219,891 new patients, 88,052 (40%) completed all assigned PROMs. Patients who did not activate their internet-based patient portal had a 62% increased likelihood of not completing assigned PROMs (odds ratio [OR], 1.62; 95% confidence interval [CI], 1.58 to 1.66; p < 0.001). Non-English-speaking patients had a 90% (OR, 1.90; 95% CI, 1.82 to 2.00; p < 0.001) increased likelihood of not completing assigned PROMs at presentation. Older patients (≥65 years of age) and patients of Black race had a 23% (OR, 1.23; 95% CI, 1.19 to 1.27; p < 0.001) and 24% (OR, 1.24; 95% CI, 1.19 to 1.30; p < 0.001) increased likelihood of not completing assigned PROMs, respectively.

Conclusions

The rate of completion of PROMs varies according to sociodemographic variables. This variability could bias clinical outcomes research in orthopaedic surgery. The present study highlights the need to uniformly increase completion rates so that outcomes research incorporates truly representative cohorts of patients treated. Furthermore, the use of these PROMs to guide health-care policy decisions necessitates a representative patient distribution to avoid bias in the health-care system.

Level of Evidence

Prognostic Level III. See Instructions for Authors for a complete description of levels of evidence.

Deep Learning for Cancer Symptoms Monitoring on the Basis of Electronic Health Record Unstructured Clinical Notes

PURPOSE

Symptoms are vital outcomes for cancer clinical trials, observational research, and population-level surveillance. Patient-reported outcomes (PROs) are valuable for monitoring symptoms, yet there are many challenges to collecting PROs at scale. We sought to develop, test, and externally validate a deep learning model to extract symptoms from unstructured clinical notes in the electronic health record.

METHODS

We randomly selected 1,225 outpatient progress notes from among patients treated at the Dana-Farber Cancer Institute between January 2016 and December 2019 and used 1,125 notes as our training/validation data set and 100 notes as our test data set. We evaluated the performance of 10 deep learning models for detecting 80 symptoms included in the National Cancer Institute's Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) framework. Model performance as compared with manual chart abstraction was assessed using standard metrics, and the highest performer was externally validated on a sample of 100 physician notes from a different clinical context.

RESULTS

In our training and test data sets, 75 of the 80 candidate symptoms were identified. The ELECTRA-small model had the highest performance for symptom identification at the token level (ie, at the individual symptom level), with an F1 of 0.87 and a processing time of 3.95 seconds per note. For the 10 most common symptoms in the test data set, the F1 score ranged from 0.98 for anxious to 0.86 for fatigue. For external validation of the same symptoms, the note-level performance ranged from F1 = 0.97 for diarrhea and dizziness to F1 = 0.73 for swelling.

CONCLUSION

Training a deep learning model to identify a wide range of electronic health record-documented symptoms relevant to cancer care is feasible. This approach could be used at the health system scale to complement to electronic PROs.

Patient reported outcomes in oncology: changing perspectives-a systematic review

In public health context, oncology is associated with severe negative impact on patients and on their relatives’ quality of life. Over the last decades, survival has remained at 50% worldwide for some tumor locations. Patient reported outcomes (PROs) assessment and, the corresponding use in clinical practice, help establishing patient individualized profiling involving caregivers. The purpose of this systematic review was to examine critical success factors for PROs assessment in daily clinical oncology practice. Additionally, we investigated how PROs collection can change oncology perspectives for patients and caregivers. According to PRISMA guidelines, 83 studies were included in this systematic review, whether related with implementation in daily clinical practice or associated with its use in oncology. PROs assessment gathers multi-professional teams, biomedical and clinical expertise, patients, families and caregivers. Institutional involvement, first line for caregiver’s adherence, team continuous formation, encompassing training and support, design of clear workflows, continuous monitoring, and data analysis are crucial for implementation. PROs measures are decisive in oncology. Several items were improved, including caregiver–patient–physician communication, patient risk groups identification, unmet problems and needs detection, disease course and treatment tracking, prognostic markers, cost-effectiveness measurement and comfort/support provision for both patients and caregivers. Routine assessment and implementation of PROs in clinical practice are a major challenge and a paradigm transformation for future.

Quantifying the Impact of Atrial Fibrillation on Heart Failure-Related Patient-Reported Outcomes in the Utah mEVAL Program

BACKGROUND

Atrial fibrillation (AF) frequently complicates heart failure (HF), and each is associated with lower overall health-related quality of life. We aimed to quantify the incremental burden of AF on the health-related quality of life of patients with HF in clinical practice.

METHODS AND RESULTS

We used data from the Utah mEVAL program to analyze patient-reported outcomes (PROs) among patients with HF with and without AF. The primary outcome was the HF-specific Kansas City Cardiomyopathy Questionnaire, with generic PROs as secondary outcomes. Among 1707 patients with HF, 36% had AF (n = 616). Those with HF and AF were older (mean age 69 years vs 58 years, P < .001), more likely to have prior stroke (29% vs 17%, P < .001) and ischemic cardiomyopathy (28% vs 23%, P = .01), but had similar ejection fractions (mean 44% each, P = .6). After adjustment, and compared with HF alone, HF with AF was associated with worse Kansas City Cardiomyopathy Questionnaire scores (adjusted mean difference -3.45, 95% confidence interval [CI] -6.24 to -0.65), and worse Patient-Reported Outcomes Measurement Information System physical function scores (adjusted mean difference -1.63, 95% CI -2.59 to -0.67). The difference in visual analog scale general health was borderline (adjusted mean difference -2.01, 95% CI -4.51 to 0.49), and Patient-Reported Outcomes Measurement Information System depression scores were similar (adjusted mean difference 0.54, 95% CI -0.48 to 1.57).

CONCLUSIONS

AF complicates nearly one-third of HF cases, and patients with HF and AF are substantially older and sicker. After adjustment, AF was independently associated with worse disease-specific and overall health-related quality of life than HF alone. Whether maintaining sinus rhythm can improve the HF-related health status of patients with HF in clinical practice should be explored further.

Clinicians' perspectives on the integration of electronic patient-reported outcomes into dermatology clinics: a qualitative study

BACKGROUND

Skin conditions can have profound negative symptomatic and psychological effects. Failure to address these effects can lead to poor treatment adherence and/or patient dissatisfaction. Despite patient-reported outcome (PRO) use being highly recommended, real-world adoption has been slow.

OBJECTIVES

To assess clinicians' perceived facilitators and barriers to using PROs in daily practice.

METHODS

We conducted in-person semi-structured interviews with 19 clinicians and thematic analysis of transcripts.

RESULTS

Three main themes emerged: (1) clinicians' attitudes about the value of Skindex-16 in daily practice, (2) patient attitudes influencing clinicians' use of Skindex-16, and (3) clinicians' perceptions of their ability to use PROs successfully for clinical care. Clinicians recognized benefits to using Skindex-16, such as revealing patients' hidden concerns and highlighting discrepancies with the clinician's severity assessments. Conversely, clinicians also identified limitations, such as time constraints and lack of relevance for some skin conditions. Patient complaints about PRO relevance have influenced clinicians' use of Skindex-16 negatively. Finally, some clinicians recognized the need for more training in score interpretation and implementation strategies for optimal clinical flow.

CONCLUSIONS

While most clinicians believed PROs like Skindex-16 can be useful for patient care, barriers need to be addressed to make PROs more practical for routine clinical care.

Enhancing the use of EHR systems for pragmatic embedded research: lessons from the NIH Health Care Systems Research Collaboratory

OBJECTIVE

We identified challenges and solutions to using electronic health record (EHR) systems for the design and conduct of pragmatic research.

MATERIALS AND METHODS

Since 2012, the Health Care Systems Research Collaboratory has served as the resource coordinating center for 21 pragmatic clinical trial demonstration projects. The EHR Core working group invited these demonstration projects to complete a written semistructured survey and used an inductive approach to review responses and identify EHR-related challenges and suggested EHR enhancements.

RESULTS

We received survey responses from 20 projects and identified 21 challenges that fell into 6 broad themes: (1) inadequate collection of patient-reported outcome data, (2) lack of structured data collection, (3) data standardization, (4) resources to support customization of EHRs, (5) difficulties aggregating data across sites, and (6) accessing EHR data.

DISCUSSION

Based on these findings, we formulated 6 prerequisites for PCTs that would enable the conduct of pragmatic research: (1) integrate the collection of patient-centered data into EHR systems, (2) facilitate structured research data collection by leveraging standard EHR functions, usable interfaces, and standard workflows, (3) support the creation of high-quality research data by using standards, (4) ensure adequate IT staff to support embedded research, (5) create aggregate, multidata type resources for multisite trials, and (6) create re-usable and automated queries.

CONCLUSION

We are hopeful our collection of specific EHR challenges and research needs will drive health system leaders, policymakers, and EHR designers to support these suggestions to improve our national capacity for generating real-world evidence.

A learning health systems approach to integrating electronic patient-reported outcomes across the health care organization

INTRODUCTION

Foundational to a learning health system (LHS) is the presence of a data infrastructure that can support continuous learning and improve patient outcomes. To advance their capacity to drive patient-centered care, health systems are increasingly looking to expand the electronic capture of patient data, such as electronic patient-reported outcome (ePRO) measures. Yet ePROs bring unique considerations around workflow, measurement, and technology that health systems may not be poised to navigate. We report on our effort to develop generalizable learnings that can support the integration of ePROs into clinical practice within an LHS framework.

METHODS

Guided by action research methodology, we engaged in iterative cycles of planning, acting, observing, and reflecting around ePRO use with two primary goals: (1) mobilize an ePRO community of practice to facilitate knowledge sharing, and (2) establish guidelines for ePRO use in the context of LHS practice. Multiple, emergent data collection activities generated generalizable guidelines that document the tangible best practices for ePRO use in clinical care. We organized guidelines around thematic areas that reflect LHS structures and stakeholders.

RESULTS

Three core thematic areas (and 24 guidelines) emerged. The theme of governance reflects the importance of leadership, knowledge management, and facilitating organizational learning around best practice models for ePRO use. The theme of integration considers the intersection of workflow, technology, and human factors for ePROs across areas of care delivery. Lastly, the theme of reporting reflects critical considerations for curating data and information, designing system functions and interactions, and presentation of ePRO data to support the translation of knowledge to action.

CONCLUSIONS

The guidelines produced from this work highlight the complex, multidisciplinary nature of implementing change within LHS contexts, and the value of action research approaches to enable rapid, iterative learning that leverages the knowledge and experience of communities of practice.

Implementation of a cloud-based electronic patient-reported outcome (ePRO) platform in patients with advanced cancer

Background

Patient reported outcomes (PROs) have been associated with improved symptom management and quality of life in patients with cancer. However, the implementation of PROs in an academic clinical practice has not been thoroughly described. Here we report on the execution, feasibility and healthcare utilization outcomes of an electronic PRO (ePRO) application for cancer patients at an academic medical center.

Methods

We conducted a randomized trial comparing an experimental ePRO arm to standard of care in patients with advanced cancer in the thoracic, gastrointestinal, and genitourinary oncology groups at Stanford Cancer Center from March 2018 to November 2019. We describe the pre-implementation, implementation, and post-implementation phases of the ePRO arm, technological barriers, electronic health record (EHR) integration, clinician burden, and patient data privacy and security. Feasibility was pre-specified to be at least 70% completion of all questionnaires. Acceptability was based on patient and clinician feedback. Ambulatory healthcare utilization was assessed by reviewing numbers of phone messages, electronic portal messages, and referrals for supportive care.

Results

Of 617 ePRO questionnaires sent to 72 patients, 445 (72%) were completed. Most clinicians (87.5%) and patients (93%) felt neutral or positive about the ePRO tool’s ease of use. Exposure to ePRO did not cause a measurable change in ambulatory healthcare utilization, with a median of less than two phone messages and supportive care referrals, and 5–6 portal messages.

Conclusions

Web-based ePRO tools for patients with advanced cancer are feasible and acceptable without increasing clinical burden. Key lessons include the importance of pilot testing, engagement of stakeholders at all levels, and the need for customization by disease group. Future directions for this work include completion of EHR integration, expansion to other centers, and development of integrated workflows for routine clinical practice.

Supplementary Information

The online version contains supplementary material available at 10.1186/s41687-021-00358-2.

The Minimal Clinically Important Difference for PROMIS Physical Function in Patients With Thumb Carpometacarpal Arthritis

Background: This study was performed to determine the minimal clinically important difference (MCID) of Patient-Reported Outcomes Measurement Information System (PROMIS) Physical Function (PF) computer adaptive test (CAT) for patients with thumb carpometacarpal (CMC) arthritis. Methods: This study retrospectively analyzed data from 152 adults receiving surgical and nonsurgical care for unilateral thumb CMC arthritis at a single institution between January 2016 and January 2018. Patients completed PROMIS PF v1.2/2.0 CAT at each visit. At follow-up, patients also completed two 6-item anchor questions assessing the degree of perceived improvement. Statistical testing analyzed the ability of the clinical anchor to discriminate levels of improvement. An anchor-based MCID estimate was calculated as the mean PROMIS PF change score in the mild improvement group. The anchor-based MCID value was examined for the influence of patient age, initial and final PROMIS scores, and follow-up interval. A distribution-based MCID value was calculated incorporating the standard error of measurement and effect size. Results: The change in PROMIS PF scores was significantly different between encounters where patients reported no change, mild improvement, and much improvement. The anchor-based MCID estimate for PROMIS PF was 3.9 (95% confidence interval, 3.3-4.7). Individual MCID values were weakly correlated with the final absolute PROMIS PF score but did not correlate with patient age, time between visits, or the initial absolute PROMIS PF score. The distribution-based MCID value was 3.5 (95% confidence interval, 3.1-3.9). Conclusions: The estimated range of MCID values for PROMIS PF is 3.5 to 3.9 points in patients treated for thumb CMC arthritis.

Can PROMIS measures be used to create subgroups for patients seeking orthopaedic care?

Aims

Patient-reported outcome measures have become an important part of routine care. The aim of this study was to determine if Patient-Reported Outcomes Measurement Information System (PROMIS) measures can be used to create patient subgroups for individuals seeking orthopaedic care.

Methods

This was a cross-sectional study of patients from Duke University Department of Orthopaedic Surgery clinics (14 ambulatory and four hospital-based). There were two separate cohorts recruited by convenience sampling (i.e. patients were included in the analysis only if they completed PROMIS measures during a new patient visit). Cohort #1 (n = 12,141; December 2017 to December 2018,) included PROMIS short forms for eight domains (Physical Function, Pain Interference, Pain Intensity, Depression, Anxiety, Sleep Quality, Participation in Social Roles, and Fatigue) and Cohort #2 (n = 4,638; January 2019 to August 2019) included PROMIS Computer Adaptive Testing instruments for four domains (Physical Function, Pain Interference, Depression, and Sleep Quality). Cluster analysis (K-means method) empirically derived subgroups and subgroup differences in clinical and sociodemographic factors were identified with one-way analysis of variance.

Results

Cluster analysis yielded four subgroups with similar clinical characteristics in Cohort #1 and #2. The subgroups were: 1) Normal Function: within normal limits in Physical Function, Pain Interference, Depression, and Sleep Quality; 2) Mild Impairment: mild deficits in Physical Function, Pain Interference, and Sleep Quality but with Depression within normal limits; 3) Impaired Function, Not Distressed: moderate deficits in Physical Function and Pain Interference, but within normal limits for Depression and Sleep Quality; and 4) Impaired Function, Distressed: moderate (Physical Function, Pain Interference, and Sleep Quality) and mild (Depression) deficits.

Conclusion

These findings suggest orthopaedic patient subgroups differing in physical function, pain, and psychosocial distress can be created from as few as four different PROMIS measures. Longitudinal research is necessary to determine whether these subgroups have prognostic validity.

Cite this article: Bone Jt Open 2021;2(7):493–502.

Electronic health record-integrated approach for collection of patient-reported outcome measures: a retrospective evaluation

Background

The integration of Patient Reported Outcome Measures (PROMs) into clinical care presents many challenges for health systems. PROMs provide quantitative data regarding patient-reported health status. However, the most effective model for collecting PROMs has not been established. Therefore the purpose of this study is to report the development and preliminary evaluation of the standardized collection of PROMs within a department of orthopedic surgery at a large academic health center.

Methods

We utilized the Users’ Guide to Integrating Patient-Reported Outcomes in Electronic Health Records by Gensheimer et al., 2018 as a framework to describe the development of PROMs collection initiative. We framed our initiative by operationalizing the three aspects of PROM collection development: Planning, Selection, and Engagement. Next, we performed a preliminary evaluation of our initiative by assessing the response rate of patients completing PROMs (no. of PROMs completed/no. of PROMs administered) across the entire department (18 clinics), ambulatory clinics only (14 clinics), and hospital-based clinics only (4 clinics). Lastly, we reported on the mean response rates for the top 5 and bottom 5 orthopaedic providers to describe the variability across providers.

Results

We described the development of a fully-integrated, population health based implementation strategy leveraging the existing resources of our local EHR to maximize clinical utility of PROMs and routine collection. We collected a large volume of PROMs over a 13 month period (n = 10,951) across 18 clinical sites, 7 clinical specialties and over 100 providers. The response rates varied across the department, ranging from 29 to 42%, depending on active status for the portal to the electronic health record (MyChart). The highest single provider mean response rate was 52%, and the lowest provider rate was 13%. Rates were similar between hospital-based (26%) and ambulatory clinics (29%).

Conclusions

We found that our standardized PROMs collection initiative, informed by Gensheimer et al., achieved scope and scale, but faced challenges in achieving a high response rate commensurate with existing literature. However, most studies reported a targeted recruitment strategy within a narrow clinical population. Further research is needed to elucidate the trade-off between scalability and response rates in PROM collection initiatives.

Systematic collection of patient-reported outcomes in atrial fibrillation: feasibility and initial results of the Utah mEVAL AF programme

AIMS

Incorporating patient-reported outcomes (PROs) into routine care of atrial fibrillation (AF) enables direct integration of symptoms, function, and health-related quality of life (HRQoL) into practice. We report our initial experience with a system-wide PRO initiative among AF patients.

METHODS AND RESULTS

All patients with AF in our practice undergo PRO assessment with the Toronto AF Severity Scale (AFSS), and generic PROs, prior to electrophysiology clinic visits. We describe the implementation, feasibility, and results of clinic-based, electronic AF PRO collection, and compare AF-specific and generic HRQoL assessments. From October 2016 to February 2019, 1586 unique AF patients initiated 2379 PRO assessments, 2145 of which had all PRO measures completed (90%). The median completion time for all PRO measures per visit was 7.3 min (1st, 3rd quartiles: 6, 10). Overall, 38% of patients were female (n = 589), mean age was 68 (SD 12) years, and mean CHA2DS2-VASc score was 3.8 (SD 2.0). The mean AFSS symptom score was 8.6 (SD 6.6, 1st, 3rd quartiles: 3, 13), and the full range of values was observed (0, 35). Generic PROs of physical function, general health, and depression were impacted at the most severe quartiles of AF symptom score (P < 0.0001 for each vs. AFSS quartile).

CONCLUSION

Routine clinic-based, PRO collection for AF is feasible in clinical practice and patient time investment was acceptable. Disease-specific AF PROs add value to generic HRQoL instruments. Further research into the relationship between PROs, heart rhythm, and AF burden, as well as PRO-guided management, is necessary to optimize PRO utilization.

Quality of Life Assessed Using Skindex-16 Scores Among Patients With Acne Receiving Isotretinoin Treatment

Importance

Acne is a common dermatologic condition and significantly affects psychosocial health and quality of life. An international task force recommended routine use of quality-of-life measures for clinic visits associated with acne management, but this has yet to translate into clinical practice.

Objective

To assess mean Skindex-16 scores over time among patients with moderate to severe acne receiving isotretinoin treatment.

Design, Setting, and Participants

A longitudinal, retrospective case series study of Skindex-16 data collected at monthly visits from 57 consecutive patients with acne receiving isotretinoin; data were collected and evaluated between November 23, 2016, and January 22, 2019. Continuous variables were compared using quantile regression. Multivariable linear mixed models evaluated mean (95% CI) score trajectory over time.

Main Outcomes and Measure

Skindex-16 scores, including normalized scores for the emotional, symptomatic, and functional aspects of having skin disease as well as an overall score.

Results

Fifty-seven patients (31 [54.4 %] males, with median [interquartile range] age of 17.2 [15.9-18.1] years) in this case series study completed the Skindex-16 at baseline and at least once during follow-up. Baseline Skindex-16 scores were similar by sex but worse with increasing age. Emotional impact was more bothersome to patients with acne requiring isotretinoin treatment than either symptoms or functioning. Improvements of greater than 50% in overall and Emotional domain scores were seen by month 2 of receiving isotretinoin treatment (eg, overall scores decreased from 39.4 to 17.5 by month 2; a decrease of 22.0; P < .001). Qualitatively, Skindex-16 scores reached their nadir between months 3 and 5; at month 4, overall Skindex-16 scores showed a 4.4-fold improvement (from 39.4 at baseline to 8.9; P < .001) and Emotional domain scores showed a 4.8-fold improvement (from 57.7 at baseline to 11.9; P < .001).

Conclusions and Relevance

The findings of this case series suggest that patients receiving isotretinoin treatment achieve greater than a 50% improvement in quality of life by month 2 and can expect approximately 4-fold to 5-fold improvements from baseline with a full course of isotretinoin. This study shows the potential of routine administration of quality of life measures to assess patient care in dermatology.

Prospective Collection of PROMIS Physical Function Measure Demonstrates Significant Improvement After Bariatric Surgery

BACKGROUND

We evaluated quality of life among bariatric surgery patients using patient-reported outcomes (PROs). We hypothesized that physical function would improve after bariatric surgery.

METHODS

We prospectively collected PROs beginning in December 2015. We used the validated Patient-Reported Outcomes Measurement Information System (PROMIS) instruments because of their broad applicability and ability to use computer-adapted technology to survey. Measures are repeated at clinic visits, both pre- and postoperatively. Data were reviewed through February 2018. Data were analyzed comparing pre- and postop physical function PRO (PF PRO) by procedure: laparoscopic Roux-en-Y gastric bypass (LRYGB) or sleeve gastrectomy (LSG). Additional variables were included in an adjusted linear mixed-effects regression model in order to isolate the effect of surgery on PF PRO over time.

RESULTS

This cohort included 279 bariatric surgery patients. The mean follow-up time was 1.5 years after surgery. The procedure groups were similar in terms of age and race but differed by gender and preoperative BMI. The PF-PRO measure showed significant improvement following surgery for both procedures.

CONCLUSION

Patient-reported physical function improved significantly after bariatric surgery. There was no significant difference between procedures.

Distress Screening Through Patient-Reported Outcomes Measurement Information System (PROMIS) at an Academic Cancer Center and Network Site: Implementation of a Hybrid Model

PURPOSE

Cancer care guidelines recommend regular distress screening of patients, with approximately one in three patients with cancer experiencing significant distress. However, the implementation of such programs is variable and inconsistent. We sought to assess the feasibility of implementing a hybrid electronic and paper screening tool for distress in all patients coming to a large academic cancer center and an associated integrated network site.

METHODS

Patients at an academic cancer center (Stanford Cancer Center) and its associated integrated network site received either an electronic or on-paper modified Patient-Reported Outcomes Measurement Information System-Global Health questionnaire, to assess overall health and distress. We used the Reach, Effectiveness, Adoption, Implementation, and Maintenance implementation framework to test and report on the feasibility of using this questionnaire. Iterative workflow changes were made to implement the questionnaire throughout the healthcare system, including processes to integrate with existing electronic health records.

RESULTS

From June 2015 to December 2017, 53,954 questionnaires representing 26,242 patients were collected. Approximately 30% of the questionnaires were completed before the visit on an electronic patient portal. The number of patients meeting the positive screen threshold remained around 40% throughout the study period. Following assessment, there were 3,763 referrals to cancer supportive services. Of note, those with a positive screen were more likely to have a referral to supportive care (odds ratio, 6.4; 95% CI, 5.8 to 6.9; P < .0001).

CONCLUSION

The hybrid electronic and on-paper use of a commonly available patient-reported outcome tool, Patient-Reported Outcomes Measurement Information System-Global Health, as a large-scale distress screening method, is feasible at a large integrated cancer center.

The impact of patient-reported outcomes on loss to follow-up care after bariatric surgery

BACKGROUND

High rates of attrition to post-bariatric surgical care continue to be common, despite recommendations for lifelong follow-up. There is little available work focusing on the etiology of attrition to post-bariatric surgical follow-up. Patient-reported outcomes (PROs) are metrics of patients' perceptions of their own health and have been used for their predictive value in other specialties. The relationships between PROs and loss to follow-up have not been explored.

METHODS

PRO data from patients who met the Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (MBSAQIP) definition of loss to follow-up at 1-year postoperatively were reviewed and compared to patients who were compliant with 1-year follow-up. Patient-reported outcomes measurement information system (PROMIS) measures are routinely collected pre- and postoperatively at our institution using a series of validated computer-adaptive tests that assess depression, satisfaction with social roles, pain interference, and physical function. A series of univariate logistic regressions tested whether baseline PROs or change in PROs from baseline to 6-month postoperatively predicted loss to follow-up at 1 year.

RESULTS

Neither baseline PROs nor change in depression, satisfaction with social roles, pain interference, or physical function were significant predictors of loss to follow-up. Similarly, patient state of residence, Charlson Comorbidity Index, BMI, and percent excess weight loss were not significant predictors of follow-up attrition.

CONCLUSION

The PROs in this study were not significant predictors of loss to follow-up at 1-year postoperatively. The rate of bariatric procedures continues to increase nationally, so does the potential for late post-surgical complications. Given the potential impact of loss to follow-up on adverse late post-surgical outcomes, there is a need to facilitate long-term post-surgical follow-up and more investigation is needed to identify and intervene on underlying causes of bariatric patient follow-up attrition.

Implementation of a standard outcome set in perinatal care: a qualitative analysis of barriers and facilitators from all stakeholder perspectives

BACKGROUND

To improve their quality, healthcare systems are increasingly focused on value delivered to patients. For perinatal care, the International Consortium for Health Outcomes Measurement (ICHOM) proposed a patient-centred outcome set with both clinical and patient-reported measures for pregnancy and childbirth (PCB set). This study aimed to identify factors that affect the implementation of the PCB set at the pre-implementation stage, using the consolidated framework for implementation research (CFIR).

METHODS

In this qualitative study, we conducted semi-structured interviews amongst a purposive sample of key stakeholders within an obstetric care network (OCN): 1) patients, 2) perinatal care professionals involved in the full cycle of perinatal care, and 3) policy makers, including hospital managers, administrative staff and health care insurers. While the CFIR guided data capture and structuring, thematic analysis revealed overarching themes that best reflected the barriers and facilitators from different stakeholder perspectives. Within these overarching themes, the CFIR constructs were maintained.

RESULTS

Interviews were conducted with 6 patients, 16 professionals and 5 policy makers. Thematic analysis supported by the CFIR framework identified four main themes: the instrument and its implementation process, use in individual patient care, use in quality improvement, and the context of the OCN. Important barriers included professional workload, data reliability, and interprofessional and interorganizational collaboration. Potential facilitators were the PCB set's direct value in individual care, interprofessional feedback and education, and aligning with existing systems. Prominent variations between stakeholder groups included the expected patient burden, the level of use, transparency of outcomes and the degree of integrated care.

CONCLUSIONS

This study clarified critical factors that affect successful implementation of the PCB set in perinatal care. Practice recommendations, suggested at multiple levels, can enable structural patient-centred care improvement and may unite stakeholders towards integrated birth care.

Patient-reported outcomes and subsequent management in atrial fibrillation clinical practice: Results from the Utah mEVAL AF program

BACKGROUND

Atrial fibrillation (AF) significantly reduces health-related quality of life (HRQoL), previously measured in clinical trials using patient-reported outcomes (PROs). We examined AF PROs in clinical practice and their association with subsequent clinical management.

METHODS

The Utah My Evaluation (mEVAL) program collects the Toronto AF Symptom Severity Scale (AFSS) in AF outpatients at the University of Utah. Baseline factors associated with worse AF symptom score (range 0-35, higher is worse) were identified in univariate and multivariable analyses. Secondary outcomes included AF burden and AF healthcare utilization. We also compared subsequent clinical management at 6 months between patients with better versus worse AF HRQoL.

RESULTS

Overall, 1338 patients completed the AFSS symptom score, which varied by sex (mean 7.26 for males vs. 10.27 for females; p < .001), age (<65, 9.73; 65-74, 7.66; ≥75, 7.58; p < .001), heart failure (9.39 with HF vs. 7.67 without; p < .001), and prior ablation (7.28 with prior ablation vs. 8.84; p < .001). In multivariable analysis, younger age (mean difference 2.92 for <65 vs. ≥75; p < .001), female sex (mean difference 2.57; p < .001), pulmonary disease (mean difference 1.88; p < .001), and depression (mean difference 2.46; p < .001) were associated with higher scores. At 6-months, worse baseline symptom score was associated with the use of rhythm control (37.1% vs. 24.5%; p < .001). Similar cofactors and results were associated with increased AF burden and health care utilization scores.

CONCLUSIONS

AF PROs in clinical practice identify highly-symptomatic patients, corroborating findings in more controlled, clinical trials. Increased AFSS score correlates with more aggressive clinical management, supporting the utility of disease-specific PROs guiding clinical practice.

Remote symptom monitoring integrated into electronic health records: A systematic review

OBJECTIVE

People with long-term conditions require serial clinical assessments. Digital patient-reported symptoms collected between visits can inform these, especially if integrated into electronic health records (EHRs) and clinical workflows. This systematic review identified and summarized EHR-integrated systems to remotely collect patient-reported symptoms and examined their anticipated and realized benefits in long-term conditions.

MATERIALS AND METHODS

We searched Medline, Web of Science, and Embase. Inclusion criteria were symptom reporting systems in adults with long-term conditions; data integrated into the EHR; data collection outside of clinic; data used in clinical care. We synthesized data thematically. Benefits were assessed against a list of outcome indicators. We critically appraised studies using the Mixed Methods Appraisal Tool.

RESULTS

We included 12 studies representing 10 systems. Seven were in oncology. Systems were technically and functionally heterogeneous, with the majority being fully integrated (data viewable in the EHR). Half of the systems enabled regular symptom tracking between visits. We identified 3 symptom report-guided clinical workflows: Consultation-only (data used during consultation, n = 5), alert-based (real-time alerts for providers, n = 4) and patient-initiated visits (n = 1). Few author-described anticipated benefits, primarily to improve communication and resultant health outcomes, were realized based on the study results, and were only supported by evidence from early-stage qualitative studies. Studies were primarily feasibility and pilot studies of acceptable quality.

DISCUSSION AND CONCLUSIONS

EHR-integrated remote symptom monitoring is possible, but there are few published efforts to inform development of these systems. Currently there is limited evidence that this improves care and outcomes, warranting future robust, quantitative studies of efficacy and effectiveness.

Comparison of International Classification of Diseases and Related Health Problems, Tenth Revision Codes With Electronic Medical Records Among Patients With Symptoms of Coronavirus Disease 2019

IMPORTANCE

International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) codes are used to characterize coronavirus disease 2019 (COVID-19)-related symptoms. Their accuracy is unknown, which could affect downstream analyses.

OBJECTIVE

To compare the performance of fever-, cough-, and dyspnea-specific ICD-10 codes with medical record review among patients tested for COVID-19.

DESIGN, SETTING, AND PARTICIPANTS

This cohort study included patients who underwent quantitative reverse transcriptase-polymerase chain reaction testing for severe acute respiratory syndrome coronavirus 2 at University of Utah Health from March 10 to April 6, 2020. Data analysis was performed in April 2020.

MAIN OUTCOMES AND MEASURES

The sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of ICD-10 codes for fever (R50*), cough (R05*), and dyspnea (R06.0*) were compared with manual medical record review. Performance was calculated overall and stratified by COVID-19 test result, sex, age group (<50, 50-64, and >64 years), and inpatient status. Bootstrapping was used to generate 95% CIs, and Pearson χ2 tests were used to compare different subgroups.

RESULTS

Among 2201 patients tested for COVD-19, the mean (SD) age was 42 (17) years; 1201 (55%) were female, 1569 (71%) were White, and 282 (13%) were Hispanic or Latino. The prevalence of fever was 66% (1444 patients), that of cough was 88% (1930 patients), and that of dyspnea was 64% (1399 patients). For fever, the sensitivity of ICD-10 codes was 0.26 (95% CI, 0.24-0.29), specificity was 0.98 (95% CI, 0.96-0.99), PPV was 0.96 (95% CI, 0.93-0.97), and NPV was 0.41 (95% CI, 0.39-0.43). For cough, the sensitivity of ICD-10 codes was 0.44 (95% CI, 0.42-0.46), specificity was 0.88 (95% CI, 0.84-0.92), PPV was 0.96 (95% CI, 0.95-0.97), and NPV was 0.18 (95% CI, 0.16-0.20). For dyspnea, the sensitivity of ICD-10 codes was 0.24 (95% CI, 0.22-0.26), specificity was 0.97 (95% CI, 0.96-0.98), PPV was 0.93 (95% CI, 0.90-0.96), and NPV was 0.42 (95% CI, 0.40-0.44). ICD-10 code performance was better for inpatients than for outpatients for fever (χ2 = 41.30; P < .001) and dyspnea (χ2 = 14.25; P = .003) but not for cough (χ2 = 5.13; P = .16).

CONCLUSIONS AND RELEVANCE

These findings suggest that ICD-10 codes lack sensitivity and have poor NPV for symptoms associated with COVID-19. This inaccuracy has implications for any downstream data model, scientific discovery, or surveillance that relies on these codes.

Improving Quality of Care in Rheumatoid Arthritis Through Mobile Patient-Reported Outcome Measurement: Focus Group Study

BACKGROUND

Patient-reported outcomes (PROs) for chronic disease management can be integrated into the routine workflow by leveraging mobile technology.

OBJECTIVE

The objective of our study was to describe the process of our quality improvement (QI) efforts using tablets for PRO collection in a busy, academic rheumatology practice to support a treat-to-target (TTT) approach for rheumatoid arthritis (RA) management.

METHODS

Our QI team designed a process for routine collection of PROs for RA patients at the Arthritis Center, employing information technology and an electronic medical record (EMR) system. Patients received a tablet at the clinic check-in desk to complete the Routine Assessment of Patient Index Data 3 (RAPID3) survey, a validated RA PRO. RAPID3 scores were uploaded to the EMR in real time and available for use in shared decision making during routine office visits. Weekly data were collected on RAPID3 completion rates and shared with front desk staff and medical assistants to drive improvement. Patients in our patient family advisory council and focus groups provided informal feedback on the process.

RESULTS

From May 1, 2017, to January 31, 2019, a total of 4233 RAPID3 surveys were completed by 1691 patients. The mean age of patients was 63 (SD 14) years; 84.00% (1420/1691) of the patients were female, and 83.00% (1403/1691) of the patients were white. The rates of RAPID3 completion increased from 14.3% (58/405) in May 2017 to 68.00% (254/376) in September 2017 and were sustained over time through January 2019. Informal feedback from patients was positive and negative, relating to the usability of the tablet and the way rheumatologists used and explained the RAPID3 data in shared decision making during the office visit.

CONCLUSIONS

We designed a sustainable and reliable process for collecting PROs from patients with RA in the waiting room and integrated these data through the EMR during office visits.

Accuracy of Patient Identification of Electrocardiogram-Verified Atrial Arrhythmias

This cross-sectional study describes the sensitivity and specificity of patient self-assessment for atrial arrhythmia compared with 12-lead electrocardiogram and describes the association of patient perception of arrhythmia with symptom burden.

Integration of Patient-reported Outcomes in a Total Joint Arthroplasty Program at a High-volume Academic Medical Center

INTRODUCTION

Despite widely appreciated barriers to successful clinical implementation, the literature regarding how to operationalize electronic health record-integrated patient-reported outcomes (PROs) remains sparse. We offer a detailed summary of the implementation of PROs into the standard of care at a major tertiary academic medical center.

METHODS

Collection of four Patient-Reported Outcomes Measurement Information System computer adaptive tests was piloted in a large academic orthopaedic surgery ambulatory clinic starting in October 2016. The Patient-Reported Outcomes Measurement Information System computer adaptive tests (Physical Function, Pain Intensity, Pain Interference, and Ability to Return to Social Roles and Activities) were initially implemented as manual order sets to be administered before surgery through 2 years after surgery. Completion rate over time, mean time to completion for all PRO domains, and the overall distribution of symptom severity were used to evaluate the success of the pilot. A subsequent optimization and redesign of the pilot was conducted using tablets, automation of questionnaire deployment, and improved results review to address obstacles encountered during the pilot phase.

RESULTS

Two thousand nine distinct joint arthroplasty patients (mean age = 65) completed at least one set of PRO assessments, with overall completion rates reaching 68% and mean completion time of 3 minutes. Focal points during the implementation process included engagement and training of staff, selection of an appropriate patient population and outcome measures, and user friendly data displays for patients and providers.

CONCLUSION

Our pilot program successfully demonstrated that PROs can be administered, scored, and made immediately available within the electronic health record to patients and their providers with minimal disruption of clinical workflows. Although considerable operational and technological challenges remain, we found that the implementation of PROs in clinical care within an ambulatory practice at an academic medical center can be achieved through a constellation of several key factors.

Planning for patient-reported outcome implementation: Development of decision tools and practical experience across four clinics

INTRODUCTION

Many institutions are attempting to implement patient-reported outcome (PRO) measures. Because PROs often change clinical workflows significantly for patients and providers, implementation choices can have major impact. While various implementation guides exist, a stepwise list of decision points covering the full implementation process and drawing explicitly on a sociotechnical conceptual framework does not exist.

METHODS

To facilitate real-world implementation of PROs in electronic health records (EHRs) for use in clinical practice, members of the EHR Access to Seamless Integration of Patient-Reported Outcomes Measurement Information System (PROMIS) Consortium developed structured PRO implementation planning tools. Each institution pilot tested the tools. Joint meetings led to the identification of critical sociotechnical success factors.

RESULTS

Three tools were developed and tested: (1) a PRO Planning Guide summarizes the empirical knowledge and guidance about PRO implementation in routine clinical care; (2) a Decision Log allows decision tracking; and (3) an Implementation Plan Template simplifies creation of a sharable implementation plan. Seven lessons learned during implementation underscore the iterative nature of planning and the importance of the clinician champion, as well as the need to understand aims, manage implementation barriers, minimize disruption, provide ample discussion time, and continuously engage key stakeholders.

CONCLUSIONS

Highly structured planning tools, informed by a sociotechnical perspective, enabled the construction of clear, clinic-specific plans. By developing and testing three reusable tools (freely available for immediate use), our project addressed the need for consolidated guidance and created new materials for PRO implementation planning. We identified seven important lessons that, while common to technology implementation, are especially critical in PRO implementation.

Amand D, del Carmen M, Ferris T, Heng M, Licurse A, Meyer G, Sequist TD. Factors Associated With Increased Collection of Patient-Reported Outcomes Within a Large Health Care System

IMPORTANCE

The collection of patient-reported outcomes (PROs) has garnered intense interest, but dissemination of PRO programs has been limited, as have analyses of the factors associated with successful programs.

OBJECTIVE

To identify factors associated with improving PRO collection rates within a large health care system using a centralized PRO infrastructure.

DESIGN, SETTING, AND PARTICIPANTS

This cohort study included 205 medical and surgical clinics in the Partners Healthcare system in Massachusetts that implemented a PRO program between March 15, 2014, and December 31, 2018, using a standardized centralized infrastructure. Data were analyzed from March to April 2019.

EXPOSURES

Relevant clinical characteristics were recorded for each clinic launching a PRO program.

MAIN OUTCOMES AND MEASURES

The primary outcome was the mean PRO collection rate during each clinic's most recent 6 months of collection prior to January 2019. Data were analyzed using a linear regression model with the 6-month PRO collection rate as the dependent variable and clinic characteristics as independent variables. Secondary analysis used a logistic regression model to assess clinical factors associated with successful clinics, defined as those that collected PROs at a rate greater than 50%.

RESULTS

Between March 2014 and December 2018, 205 Partners Healthcare clinics were available for analysis, and 4 061 205 PRO measures from 745 028 encounters were collected. Among these, 103 clinics (50.2%) collected at a rate greater than 50%. Increased collection rates were associated with more than 50% of physicians in a clinic trained on PROs (change, 19.6% [95% CI, 9.9%-29.4%]; P < .001), routine administrative oversight of collection rates (change, 16.0% [95% CI, 6.6%-25.5%]; P = .001), previous collection of PROs on paper (change, 12.5% [95% CI, 4.7%-20.3%]; P = .002), presence of a clinical champion (change, 11.2% [95% CI, 2.5%-20.0%]; P = .01) and payer incentive (change, 10.5% [95% CI, 2.0%-18.9%]; P = .02).

CONCLUSIONS AND RELEVANCE

These findings suggest that training physicians on the use of PROs, administrative surveillance of collection rates, and the presence of a local clinical champion may be promising interventions for increasing PRO collection. Clinics that have previously collected PROs may have greater success in increasing collections. Payer incentive for collection was associated with improved collections, but not associated with successful programs.

Cross-cultural adaptation and validation of the RhinAsthma Patient Perspective (RAPP) in the Polish population

Introduction

The RhinAsthma Patient Perspective (RAPP) was developed in Italian to assess the Health Related Quality of Life (HRQoL) impairment in patients with asthma and allergic rhinitis (AR) in daily practice.

Aim

To cross-culturally validate the Polish version.

Material and methods

The Polish version was administered to patients suffering from asthma and rhinitis in a prospective observational study. Polish RAPP, along with SF-12, ACT, and a Symptomatologic VAS was filled in twice, with a 4-week interval between visits. At visit 2, a Global Rating Scale (GRS) was completed to assess any change in health status. Internal consistency, validity, reliability, discriminant ability and responsiveness to change as well as Minimal Important Difference were determined.

Results

The factor and confirmatory analysis revealed a unidimensional structure of RAPP. Internal consistency was satisfactory with Cronbach’s α (visit 1 = 0.85, visit 2 = 0.89). High reliability (ICC = 0.89 and a CCC = 0.94) was found. Validity analyses showed good correlations of the Polish RAPP with Physical and Mental Component Scores of SF-12. In addition, RAPP adequately discriminated patients on the basis of the asthma control level and rhinitis severity (p < 0.03 for all the analyses), and demonstrated to be sensitive to change. MID value was 1 point.

Conclusions

The study confirmed the reliability and validity of the Polish version of RAPP demonstrating that it is a useful tool in the assessment of HRQoL in patients with asthma and comorbid allergic rhinitis, in clinical practice.

Development and validation of an interpretive guide for PROMIS scores

BACKGROUND

Accurate score interpretation is required for the appropriate use of patient-reported outcome measures in clinical practice.

OBJECTIVE

To create and evaluate figures (T-score Maps) to facilitate the interpretation of scores on Patient-Reported Outcome Measurement Information System (PROMIS) measures.

METHODS

For 21 PROMIS® short forms, item-level information was used to predict the most probable responses to items for the range of possible scores on each short form. Predicted responses were then "mapped" graphically along the range of possible scores. In a previously conducted longitudinal study, 1594 adult participants with chronic conditions (e.g., multiple sclerosis) responded to four items each of a subset of these PROMIS short forms. Participants' responses to these items were compared to those predicted by the T-score Maps. Difference scores were calculated between observed and predicted scores, and Spearman correlations were calculated.

RESULTS

We constructed T-score Maps for 21 PROMIS short forms for adults and pediatric self- and parent-proxy report. For the clinical population, participants' actual responses were strongly correlated with their predicted responses (r = 0.762 to 0.950). The majority of predicted responses exactly matched observed responses (range 69.5% to 85.3%).

CONCLUSION

Results support the validity of the predicted responses used to construct T-score Maps. T-score Maps are ready to be tested as interpretation aids in a variety of applications.

Routinely measuring symptom burden and health-related quality of life in dialysis patients: first results from the Dutch registry of patient-reported outcome measures

Background

The use of patient-reported outcome measures (PROMs) is becoming increasingly important in healthcare. However, incorporation of PROMs into routine nephrological care is challenging. This study describes the first experience with PROMs in Dutch routine dialysis care.

Methods

A pilot study was conducted in dialysis patients in 16 centres. Patients were invited to complete PROMs at baseline and 3 and 6 months. PROMs consisted of the 12-item short-form and Dialysis Symptom Index to assess health-related quality of life (HRQoL) and symptom burden. Response rates, HRQoL and symptom burden scores were analysed. Qualitative research methods were used to gain insight into patients’ views on using PROMs in clinical practice.

Results

In total, 512 patients (36%) completed 908 PROMs (24%) across three time points. Response rates varied from 6 to 70% among centres. Mean scores for physical and mental HRQoL were 35.6 [standard deviation (SD) 10.2] and 47.7 (SD 10.6), respectively. Patients experienced on average 10.8 (SD 6.1) symptoms with a symptom burden score of 30.7 (SD 22.0). Only 1–3% of the variation in PROM scores can be explained by differences between centres. Patients perceived discussing their HRQoL and symptom scores as insightful and valuable. Individual feedback on results was considered crucial.

Conclusions

The first results show low average response rates with high variability among centres. Dialysis patients experienced a high symptom burden and poor HRQoL. Using PROMs at the individual patient level is suitable and may improve patient–professional communication and shared decision making. Further research is needed to investigate how the collection and the use of PROMs can be successfully integrated into routine care to improve healthcare quality and outcomes.

Capturing the patient voice: implementing patient-reported outcomes across the health system

PURPOSE

Supporting the capture and use of patient-reported outcomes (PROs) at the point-of-care enriches information about important clinical and quality of life outcomes. Yet the ability to scale PROs across healthcare systems has been limited by knowledge gaps around how to manage the diversity of PRO uses and leverage health information technology. In this study, we report learnings and practice insights from UW Medicine's practice transformation efforts to incorporate patient voice into multiple areas of care.

METHODS

Using a participatory, action research approach, we engaged with UW Medicine clinical and administrative stakeholders experienced with PRO implementation to inventory PRO implementations across the health system, characterize common clinical uses for PROs, and develop recommendations for system-wide governance and implementation of PROs.

RESULTS

We identified a wide breadth of PRO implementations (n = 14) in practice and found that nearly half (47%) of employed PRO measures captured shared clinical domains (e.g., depression). We developed three vignettes (use cases) that illustrate how users interact with PROs, characterize common ways PRO implementations support clinical care across the health system (1) Preventive care, (2) Chronic/Specialty care, and (3) Surgical/Interventional care), and elucidate opportunities to enhance efficient PRO implementations through system-level standards and governance.

CONCLUSIONS

Practice transformation efforts increasingly require integration of the patient voice into clinical care, often through the use of PROs. Learnings from our work highlight the importance of proactively considering how PROs will be used across the layers of healthcare organizations to optimize the design and governance of PROs.

SMART Markers: collecting patient-generated health data as a standardized property of health information technology

A patient-centered health system needs precise computable measurements to derive value. While validated patient-reported outcomes (PROs) are increasingly used in trials, their adoption in care remains limited and generally separated from the medical record. Further, absence of systematic processes for patient-led data submission excludes valuable data from digital devices that can potentially aid in contextualizing health status. With prior experience in developing apps for the Patient-Reported Outcomes Measurement Information System (PROMIS), we sought to make collecting patient-generated health data (PGHD) a fundamental property of health information technology at scale, and in an interoperable, standards-compliant fashion. We build upon the open SMART on FHIR (Fast Health Interoperability Resources) specification to create SMART Markers-a mobile device software framework encapsulating functionality needed for rapid deployment of both patient- and practitioner-facing PGHD apps. We refactored previously developed PROMIS apps to use SMART Markers for handling PGHD-request creation, on-device administration, and generation of a variety of PGHD types and submission of results to a FHIR server. Validation and conformance tests were performed on the generated output and app-reusability was demonstrated across two demo servers. App developers can import SMART Markers into their existing or new apps to readily leverage an interoperable PGHD capturing functionality out of the box, without having to reinvent the wheel. Our approach enables the creation of health system integrated, context-specific experiences for both patients and practitioners.