Promoting effective use of patient-reported outcomes in clinical practice: themes from a "Methods Tool kit" paper series

Evaluating a web-based personalized decision report for total knee or hip replacement: Lessons learned from patients

RATIONALE

Patient-reported outcomes (PROs) are increasingly used in the context of clinical care, but evaluation of patients' perspectives of PRO-based applications in routine care remains limited.

AIMS AND OBJECTIVES

This paper investigates patients' acceptability of a personalized web-based decision report for total knee or hip replacement and identifies opportunities to refine the report.

METHOD

This qualitative evaluation was embedded in a pragmatic cluster randomized trial of the report. We interviewed 25 patients with knee and hip osteoarthritis about their experiences using the personalized decision report in the context of a surgical consultation. The web-based report contained current descriptive PRO scores of pain, function and general physical health; tailored predicted postoperative PRO scores (i.e., personalized likely outcomes based on actual knee or hip replacement outcomes of similar patients in a national registry); and information about alternative nonoperative treatments. Two trained researchers analysed the interview data qualitatively using a combination of inductive and deductive coding.

RESULTS

We identified three major categories for evaluation: content of report, presentation of data in report and engagement with report. Patients generally liked the report overall but specifically valued different pages of the report based on where they were in the surgical decision-making process. Patients identified areas of confusion in data presentation related to graph orientation, terminology and interpretation of T-scores. Patients also highlighted support needs to meaningfully engage with the information in the report.

CONCLUSION

Our findings highlight areas of opportunity to further refine this personalized web-based decision report and similar patient-facing PRO applications for routine clinical care. Specific examples include additional tailoring of reports via filterable web-based dashboards and scalable educational supports to facilitate more independent patient understanding and use.

Patient-reported outcome (PRO) measurements in chronic and malignant diseases: ten years' experience with PRO-algorithm-based patient-clinician interaction (telePRO) in AmbuFlex

BACKGROUND

Patient-reported Outcome (PRO) measures may be used as the basis for out-patient follow-up instead of fixed appointments. The patients attend follow-up from home by filling in questionnaires developed for that specific aim and patient group (telePRO). The questionnaires are handled in real time by a specific algorithm, which assigns an outcome color reflecting clinical need. The specific questionnaires and algorithms (named solutions) are constructed in a consensus process with clinicians. We aimed to describe AmbuFlex' telePRO solutions and the algorithm outcomes and variation between patient groups, and to discuss possible applications and challenges.

METHODS

TelePRO solutions with more than 100 processed questionnaires were included in the analysis. Data were retrieved together with data from national registers. Characteristics of patients, questionnaires and outcomes were tabulated for each solution. Graphs were constructed depicting the overall and within-patient distribution of algorithm outcomes for each solution.

RESULTS

From 2011 to 2021, 29 specific telePRO solutions were implemented within 24 different ICD-10 groups. A total of 42,015 patients were referred and answered 171,268 questionnaires. An existing applicable instrument with cut-off values was available for four solutions, whereas items were selected or developed ad hoc for the other solutions. Mean age ranged from 10.7 (Pain in children) to 73.3 years (chronic kidney disease). Mortality among referred patients varied between 0 (obesity, asthma, endometriosis and pain in children) and 528 per 1000 patient years (Lung cancer). There was substantial variation in algorithm outcome across patient groups while different solutions within the same patient group varied little.

DISCUSSION

TelePRO can be applied in diseases where PRO can reflect clinical status and needs. Questionnaires and algorithms should be adapted for the specific patient groups and clinical aims. When PRO is used as replacement for clinical contact, special carefulness should be observed with respect to patient safety.

Reliability and validity of the Shoulder Pain and Disability Index in a sample of patients with frozen shoulder

BACKGROUND

The Shoulder and Pain Disability Index (SPADI) is a widely used outcome measure. The aim of this study is to explore the reliability and validity of SPADI in a sample of patients with idiopathic frozen shoulder.

METHODS

The SPADI was administered to 124 patients with idiopathic frozen shoulder. A sub-group of 29 patients were retested after 7 days. SPADI scores were correlated with other outcome measures (i.e., Disabilities of the Arm, Shoulder and Hand Questionnaire - DASH; Numerical Pain Rating Scale-NPRS; and 36-item Short Form Health Survey-SF-36) to examine construct validity. Structural validity was assessed by a Two-Factors Confirmatory Factor Analysis (CFA). Internal consistency, test-retest reliability, and measurement error were also analyzed.

RESULTS

The construct validity was satisfactory as seven out of eight of the expected correlations formulated (≥ 75%) for the subscales were satisfied. The CFA showed good values of all indicators for both Pain and Disability subscales (Comparative Fit Index = 0.999; Tucker-Lewis Index = 0.997; Root Mean Square Error of Approximation = 0.030). Internal consistency was good for pain (α = 0.859) and disability (α = 0.895) subscales. High test-retest reliability (Intraclass correlation coefficient [ICC]) was found for pain (ICC = 0.989 [95% Confidence Interval (CI = 0.975-0.995]) and disability (ICC = 0.990 [95% CI = 0.988-0.998]). Standard Error of Measurement values of 2.27 and 2.32 and Minimal Detectable Change values of 6.27 and 6.25 were calculated for pain and disability subscales, respectively.

CONCLUSION

The SPADI demonstrated satisfactory reliability and validity properties in a sample of patients with idiopathic frozen shoulder.

A RE-AIM Evaluation of a Visualization-Based Electronic Patient-Reported Outcome System

OBJECTIVES

Health care systems are primarily collecting patient-reported outcomes (PROs) for research and clinical care using proprietary, institution- and disease-specific tools for remote assessment. The purpose of this study was to conduct a Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) evaluation of a scalable electronic PRO (ePRO) reporting and visualization system in a single-arm study.

METHODS

The "mi.symptoms" ePRO system was designed using gerontechnological design principles to ensure high usability among older adults. The system enables longitudinal reporting of disease-agnostic ePROs and includes patient-facing PRO visualizations. We conducted an evaluation of the implementation of the system guided by the RE-AIM framework. Quantitative data were analyzed using basic descriptive statistics, and qualitative data were analyzed using directed content analysis.

RESULTS

Reach-the total reach of the study was 70 participants (median age: 69, 31% female, 17% Black or African American, 27% reported not having enough financial resources). Effectiveness-half (51%) of participants completed the 2-week follow-up survey and 36% completed all follow-up surveys. Adoption-the desire for increased self-knowledge, the value of tracking symptoms, and altruism motivated participants to adopt the tool. Implementation-the predisposing factor was access to, and comfort with, computers. Three enabling factors were incorporation into routines, multimodal nudges, and ease of use. Maintenance-reinforcing factors were perceived usefulness of viewing symptom reports with the tool and understanding the value of sustained symptom tracking in general.

CONCLUSION

Challenges in ePRO reporting, particularly sustained patient engagement, remain. Nonetheless, freely available, scalable, disease-agnostic systems may pave the road toward inclusion of a more diverse range of health systems and patients in ePRO collection and use.

Collection and Utilization of Patient-Reported Outcome Measures in a Colorectal Surgery Clinic

INTRODUCTION

The routine collection of patient-reported outcome measures (PROMs) promises to improve patient care. However, in colorectal surgery, PROMs are uncommonly collected outside of clinical research studies and rarely used in clinical care. We designed and implemented a quality improvement project with the goals of routinely collecting PROMs and increasing the frequency that PROMs are utilized by colorectal surgeons in clinical practice.

METHODS

This mixed-methods, quality improvement project was conducted in the colorectal surgery clinic of a tertiary academic medical center. Patients were administered up to five PROMs before each appointment. PROM completion rates were measured. Additionally, we performed two educational interventions to increase utilization of our electronic health record's PROM dashboard by colorectal surgeons. Utilization rates and attitudes toward the PROM dashboard were measured.

RESULTS

Overall, patients completed 3600 of 3977 (90.9%) administered PROMs during the study period. At baseline, colorectal surgeons reviewed 6.7% of completed PROMs. After two educational interventions, this increased to 39.3% (P = 0.004). Colorectal surgeons also felt that the PROM dashboard was easier to use. Barriers to greater PROM dashboard utilization included poor user interface/user experience and a perceived lack of knowledge, time, and relevance.

CONCLUSIONS

The collection of PROMs in colorectal surgery clinics is feasible and can result in high PROM completion rates. Educational interventions can improve the utilization of PROMs by colorectal surgeons in clinical practice. Our experience collecting PROMs through this quality improvement initiative can serve as a template for other colorectal surgery clinics interested in collecting and utilizing data from PROMs.

Examining the Utility of the HIV Disability Questionnaire (HDQ) in Clinical Practice: Perspectives of People Living with HIV and Healthcare Providers

Our aim was to examine the utility of the HIV Disability Questionnaire (HDQ), a patient-reported outcome measure for use in clinical practice from the perspectives of people living with HIV (PLWH) and healthcare providers. We conducted a qualitative descriptive study. Fifteen PLWH and five healthcare providers participated in an interview, of which ten PLWH participated in a follow-up focus group discussion. The HDQ has value in clinical practice, including its role in assessing disability, facilitating communication, tailoring treatments, and guiding referrals. Strengths of the HDQ included its comprehensiveness, relevance of domains, and importance of specific items. Concerns related to length of the HDQ, the potential for some items to trigger emotional response, and negative connotations with the term ‘disability.’ Recommendations for HDQ implementation included the importance of score interpretability, shortening the questionnaire, and tailoring administration to the individual. Results suggest the HDQ possesses clinical utility with PLWH and healthcare providers.

Using Autoregressive Integrated Moving Average (ARIMA) Modelling to Forecast Symptom Complexity in an Ambulatory Oncology Clinic: Harnessing Predictive Analytics and Patient-Reported Outcomes

An increasing incidence of cancer has led to high patient volumes and time challenges in ambulatory oncology clinics. By knowing how many patients are experiencing complex care needs in advance, clinic scheduling and staff allocation adjustments could be made to provide patients with longer or shorter timeslots to address symptom complexity. In this study, we used predictive analytics to forecast the percentage of patients with high symptom complexity in one clinic population in a given time period. Autoregressive integrated moving average (ARIMA) modelling was utilized with patient-reported outcome (PRO) data and patient demographic information collected over 24 weeks. Eight additional weeks of symptom complexity data were collected and compared to assess the accuracy of the forecasting model. The predicted symptom complexity levels were compared with observation data and a mean absolute predicting error of 5.9% was determined, indicating the model’s satisfactory accuracy for forecasting symptom complexity levels among patients in this clinic population. By using a larger sample and additional predictors, this model could be applied to other clinics to allow for tailored scheduling and staff allocation based on symptom complexity forecasting and inform system level models of care to improve outcomes and provide higher quality patient care.

Quality of life assessments in clinical practice using either the EORTC-QLQ-C30 or the SEIOQL-DW: a randomized study

Background

Effective patient-physician communication can improve patient understanding, agreement on treatment and adherence. This may, in turn, impact on clinical outcomes and patient quality of life (QoL). One way to improve communication is by using patient-reported outcome measures (PROMs). Heretofore, studies of the impact of using PROMs in clinical practice have mostly evaluated the use of standardized PROMs. However, there is reason to believe that individualized instruments may be more appropriate for this purpose.

The aim of this study is to compare the effectiveness of the standardized QoL-instrument, the European Organization for Research and Treatment of Cancer Quality of Life C-30 (EORTC-QOL-C30) and the individualized QoL instrument, the Schedule for the Evaluation of Individual Quality of Life-Direct Weighting (SEIQoL-DW), in clinical practice.

Methods

In a prospective, open-label, controlled intervention study at two hospital out-patient clinics, 390 patients with gastrointestinal cancer were randomly assigned either to complete the EORTC-QOL-C30 or the SEIQoL-DW immediately before the consultation, with their responses being shared with their physician. This was repeated in 3–5 consultations over a period of 4–6 months. The primary outcome measure was patients’ health-related QoL, as measured by FACIT-G. Patients’ satisfaction with the consultation and survival were secondary outcomes.

Results

There was no significant difference between the groups with regard to study outcomes. Neither intervention instrument resulted in any significant changes in health-related QoL, or in any of the secondary outcomes, over time. This may reflect either a genuine lack of effect or sub-optimization of the intervention. Since there was no comparison to standard care an effect in terms of lack of deterioration over time cannot be excluded.

Conclusions

Future studies should focus on the implementation process, including the training of physicians to use the instruments and their motivation for doing so. The effects of situational use of standardized or individualized instruments should also be explored. The effectiveness of the different approaches may depend on contextual factors including physician and patient preferences.

Outcomes after kidney transplantation, let's focus on the patients' perspectives

Graft function and patient survival are traditionally the most used parameters to assess the objective benefits of kidney transplantation. Monitoring graft function, along with therapeutic drug concentrations and transplant complications, comprises the essence of outpatient management in kidney transplant recipients (KTRs). However, the patient's perspective is not always included in this process. Patients' perspectives on their health after kidney transplantation, albeit subjective, are increasingly acknowledged as valuable healthcare outcomes and should be considered in order to provide patient-centred healthcare. Such outcomes are known as patient-reported outcomes (PROs; e.g. health-related quality of life and symptom burden) and are captured using PRO measures (PROMs). So far, PROMs have not been routinely used in clinical care for KTRs. In this review we will introduce PROMs and their potential application and value in the field of kidney transplantation, describe commonly used PROMs in KTRs and discuss structural PROMs implementation into kidney transplantation care.

Development of a Patient Reported Measure of Experimental Transplants with HIV and Ethics in the United States (PROMETHEUS)

BACKGROUND

Transplantation of HIV-positive (HIV+) donor organs for HIV+ recipients (HIV D+/R+) is now being performed as research in the United States, but raises ethical concerns. While patient-reported outcome measures are increasingly used to evaluate clinical interventions, there is no published measure to aptly capture patients' experiences in the unique context of experimental HIV D+/R+ transplantation. Therefore, we developed PROMETHEUS (patient-reported measure of experimental transplants with HIV and ethics in the United States). To do so, we created a conceptual framework, drafted a pilot battery using existing and new measures related to this context, and refined it based on cognitive and pilot testing. PROMETHEUS was administered 6-months post-transplant in a clinical trial evaluating these transplants. We analyzed data from the first 20 patient-participants for reliability and validity by calculating Cronbach's alpha and reviewing item performance characteristics.

RESULTS

PROMETHEUS 1.0 consisted of 29 items with 5 putative subscales: Emotions; Trust; Decision Making; Transplant; and Decision Satisfaction. Overall, responses were positive. Cronbach's alpha was > 0.8 for all subscales except Transplant, which was 0.38. Two Transplant subscale items were removed due to poor reliability and construct validity.

CONCLUSIONS

We developed PROMETHEUS to systematically capture patient-reported experiences with this novel experimental transplantation program, nested it in an actual clinical trial, and obtained preliminary data regarding its performance.

A New Procedure to Assess When Estimates from the Cumulative Link Model Can Be Interpreted as Differences for Ordinal Scales in Quality of Life Studies

Purpose

Assessing the clinical importance of an exposure effect on a quality of life (QoL) score often requires quantifying the effect in terms of a difference in scores. Using the linear regression model (LRM) for this purpose assumes the ordinal score is a proxy for an underlying continuous variable, but the analysis offers no assessment for the validity of the assumption. We propose an approach that assesses the proxy assumption and estimates the exposure effect by using the cumulative link model (CLM).

Patients and methods

CLM is a well-established regression model that assumes an ordinal score is an ordered category generated from applying thresholds to a latent continuous variable. Our approach assesses the proxy assumption by testing whether these thresholds are equidistant. We compared the performance of CLM and LRM using simulated ordinal data and illustrated their application to the effect of time since diagnosis on five subscales of fatigue among breast cancer survivors measured using the Multidimensional Fatigue Inventory.

Results

CLM had good performance in estimating the difference in means with simulated ordinal data satisfying the proxy assumption, even when the outcome had only a few categories. When the proxy assumption was inadequate, both the CLM and LRM had biased estimates with poor coverage. The proxy assumption was appropriate for four of the five subscales in our real data application to fatigue scores, which highlighted the importance of assessing the proxy assumption to avoid reporting invalid estimates in terms of the difference in scores.

Conclusion

The proxy assumption is critical to the interpretation of the exposure effect on the difference in mean QoL scores. CLM offers a valid test for the presence of an association, a method for assessing the proxy assumption, and when the assumption is adequate, an assessment for clinical significance using the difference in means.

Nondietary Cardiovascular Health Metrics With Patient Experience and Loss of Productivity Among US Adults Without Cardiovascular Disease: The Medical Expenditure Panel Survey 2006 to 2015

Background

The American Heart Association 2020 Impact Goals aimed to promote population health through emphasis on cardiovascular health (CVH). We examined the association between nondietary CVH metrics and patient‐reported outcomes among a nationally representative sample of US adults without cardiovascular disease.

Methods and Results

We included adults aged ≥18 years who participated in the Medical Expenditure Panel Survey between 2006 and 2015. CVH metrics were scored 1 point for each of the following: not smoking, being physically active, normal body mass index, no hypertension, no diabetes mellitus, and no dyslipidemia, or 0 points if otherwise. Diet was not assessed in Medical Expenditure Panel Survey. Patient‐reported outcomes were obtained by telephone survey and included questions pertaining to patient experience and health‐related quality of life. Regression models were used to compare patient‐reported outcomes based on CVH, adjusting for sociodemographic factors and comorbidities. There were 177 421 Medical Expenditure Panel Survey participants (mean age, 45 [17] years) representing ~187 million US adults without cardiovascular disease. About 12% (~21 million US adults) had poor CVH. Compared with individuals with optimal CVH, those with poor CVH had higher odds of reporting poor patient‐provider communication (odds ratio, 1.14; 95% CI, 1.05–1.24), poor healthcare satisfaction (odds ratio, 1.15; 95% CI, 1.08–1.22), poor perception of health (odds ratio, 5.89; 95% CI, 5.35–6.49), at least 2 disability days off work (odds ratio, 1.39; 95% CI, 1.30–1.48), and lower health‐related quality of life scores.

Conclusions

Among US adults without cardiovascular disease, meeting a lower number of ideal CVH metrics is associated with poor patient‐reported healthcare experience, poor perception of health, and lower health‐related quality of life. Preventive measures aimed at optimizing ideal CVH metrics may improve patient‐reported outcomes among this population.