Should English healthcare providers be penalised for failing to collect patient-reported outcome measures? A retrospective analysis

Metabolic syndrome increases the length of stay and medical complications after hip and knee arthroplasty: results from a prospective cohort study of 2,901 patients

BACKGROUND AND PURPOSE

 Metabolic syndrome (MetS) affects more than 60% of the patients having a hip or knee arthroplasty due to osteoarthritis. As it is debated whether metabolic syndrome increases the risk of complications, we aimed to investigate the length of stay (LOS) and risk of readmission at 30 and 90 days after surgery, including causes of readmission.

METHODS

 We conducted a prospective cohort study of 2,901 patients undergoing hip and knee arthroplasty from May 2017 to November 2019. Physical examination, blood samples, and medical history from national registries determined the diagnosis of metabolic syndrome from the International Diabetes Federation definition. We used multivariate linear regression to investigate differences in LOS according to MetS, and binary regression to investigate the risk and causes of readmission within 30 and 90 days, including 95% confidence intervals (CI) and P values.

RESULTS

 Patients with MetS showed a slightly longer LOS (0.20 days, CI 0.10-0.29) and had an increased risk of readmission within 90 days (adjusted relative risk [RR] 1.2, CI 1.0-1.4; P = 0.02), but not within 30 days (adjusted RR 1.1, CI 0.9-1.4; P = 0.3) after surgery. Cardiovascular disease was the dominant cause of readmission.

CONCLUSION

Although patients with MetS do not experience a clinically relevant longer LOS after hip and knee arthroplasty, they have an increased risk of 90-day readmission mainly due to cardiovascular complications, which should be considered when planning surgical care in this group of patients.

Can prediction models for hospital readmission be improved by incorporating patient-reported outcome measures? A systematic review and narrative synthesis

PURPOSE

To investigate the roles, challenges, and implications of using patient-reported outcome measures (PROMs) in predicting the risk of hospital readmissions.

METHODS

We systematically searched four bibliometric databases for peer-reviewed studies published in English between 1 January 2000 and 15 June 2023 and used validated PROMs to predict readmission risks for adult populations. Reported studies were analysed and narratively synthesised in accordance with the CHARMS and PRISMA guidelines.

RESULTS

Of the 2858 abstracts reviewed, 23 studies met predefined eligibility criteria, representing diverse geographic regions and medical specialties. Among those, 19 identified the positive contributions of PROMs in predicting readmission risks. Seven studies utilised generic PROMs exclusively, eleven used generic and condition-specific PROMs, while 5 focussed solely on condition-specific PROMs. Logistic regression was the most used modelling approach, with 13 studies aiming at predicting 30-day all-cause readmission risks. The c-statistic, ranging from 0.54 to 0.84, was reported in 22/23 studies as a measure of model discrimination. Nine studies reported model calibration in addition to c-statistic. Thirteen studies detailed their approaches to dealing with missing data.

CONCLUSION

Our study highlights the potential of PROMs to enhance predictive accuracy in readmission models, while acknowledging the diversity in data collection methods, readmission definitions, and model evaluation approaches. Recognizing that PROMs serve various purposes beyond readmission reduction, our study supports routine data collection and strategic integration of PROMs in healthcare practices to improve patient outcomes. To facilitate comparative analysis and broaden the use of PROMs in the prediction framework, it is imperative to consider the methodological aspects involved.

Benchmarking community/primary care musculoskeletal services: A narrative review and recommendation

INTRODUCTION

High quality data on service performance is essential in healthcare to evidence efficacy, efficiency, and value. There remains a paucity of publicly reported data in community and primary care musculoskeletal (MSK) services. There is also a lack of guidance on which metrics MSK services should be collecting and reporting, and how this data could be used to directly improve patient outcomes, experiences, and value.

METHOD

A narrative review of the evidence around benchmarking MSK services was undertaken with a focus on how to develop routine data collection within community/primary care settings, and how to develop benchmarking capabilities for the future, looking towards a national MSK audit. This evidence was triangulated with the findings from recent MSK data studies undertaken by the authors and emerging UK policy and guidance in this area.

RECOMMENDATIONS

To enable MSK benchmarking services need to collect consistent, standardised outcomes and, therefore, we have developed a recommendation on a minimum MSK 'core outcome set' of Patient Reported Outcome Measures (PROMs) and Patient Reported Experience Measures (PREMs) (PROMs: MSK-HQ, NPRS, WPAI; PREMs: National MSK PREM). In addition, we make recommendations on the use of a standardised evidence-based method for case-mix adjustment and outlier identification (using the following baseline demographics and clinical factors; age, sex, ethnicity, pain site, comorbidities, duration of symptoms, previous surgery, previous pain episodes), alongside considerations on how this data should be integrated and reported within NHS systems.

CONCLUSIONS

Capturing high quality MSK data in a standardised, consistent, and sustainable way is a significant challenge. Policy holders, commissioners, managers, and clinicians need to be realistic with expectations, and take time to explore barriers to implementation including, funding, digital infrastructure/intra-operability, data sharing/governance, digital literacy, and local/national leadership. Next steps include developing a national MSK audit programme to provide a benchmarking model to support continuous improvements in care quality for patients living with MSK conditions.

Using Patient-Reported Outcome Measures to Evaluate Care for Patients With Inflammatory Chronic Rheumatic Disease

OBJECTIVES

Few countries integrate patient-reported outcome measures (PROMs) in routine performance assessment and those that do focus on elective surgery. This study addresses the challenges of using PROMs to evaluate care in chronic conditions. We set out a modeling strategy to assess the extent to which changes over time in self-reported health status by patients with inflammatory chronic rheumatic disease are related to their biological drug therapy and rheumatology center primarily responsible for their care.

METHODS

Using data from the Portuguese Register of Rheumatic Diseases, we assess health status using the Health Assessment Questionnaire-Disability Index for rheumatic patients receiving biological drugs between 2000 and 2017. We specify a fixed-effects model using the least squares dummy variables estimator.

RESULTS

Patients receiving infliximab or rituximab report lower health status than those on etanercept (the most common therapy) and patients in 4 of the 26 rheumatology centers report higher health status than those at other centers.

CONCLUSIONS

PROMs can be used for those with chronic conditions to provide the patient's perspective about the impact on their health status of the choice of drug therapy and care provider. Care for chronic patients might be improved if healthcare organizations monitor PROMs and engage in performance assessment initiatives on a routine basis.

Using EQ-5D Data to Measure Hospital Performance: Are General Population Values Distorting Patients' Choices?

Background. The English National Health Service publishes hospital performance indicators based on average postoperative EQ-5D index scores after hip replacement surgery to inform prospective patients' choices of hospital. Unidimensional index scores are derived from multidimensional health-related quality-of-life data using preference weights estimated from a sample of the UK general population. This raises normative concerns if general population preferences differ from those of the patients who are to be informed. This study explores how the source of valuation affects hospital performance estimates. Methods. Four different value sets reflecting source of valuation (general population v. patients), valuation technique (visual analog scale [VAS] v. time tradeoff [TTO]), and experience with health states (currently experienced vs. experimentally estimated) were used to derive and compare performance estimates for 243 hospitals. Two value sets were newly estimated from EQ-5D-3L data on 122,921 hip replacement patients and 3381 members of the UK general public. Changes in hospital ranking (nationally) and performance outlier status (nationally; among patients' 5 closest hospitals) were compared across valuations. Results. National rankings were stable under different valuations (rank correlations >0.92). Twenty-three (9.5%) hospitals changed outlier status when using patient VAS valuations instead of general population TTO valuations, the current approach. Outlier status also changed substantially at the local level. This was explained mostly by the valuation technique, not the source of valuations or experience with the health states. Limitations. No patient TTO valuations were available. The effect of value set characteristics could be established only through indirect comparisons. Conclusion. Different value sets may lead to prospective patients choosing different hospitals. Normative concerns about the use of general population valuations are not supported by empirical evidence based on VAS valuations.

Patient-Reported Outcomes: Understanding Surgical Efficacy and Quality from the Patient's Perspective

In surgery, quality assessment encourages improved care delivery, better outcomes, and helps determine surgical efficacy. Quality is important from a patient, provider, payer, and policy maker standpoint. However, given the growth of outpatient procedures, expansion of surgical indications to enhance function, and the decline of perioperative morbidity and mortality, many traditional quality metrics, such as mortality, readmissions, and complications, may not fully capture quality. As such, patient-reported outcomes (PROs) can be used to complement the established clinical outcomes and describe surgical efficacy and quality from the patient's point of view. Generic and disease-specific PRO measures capture health-related quality of life, functional status, and pain. These measures permit a more holistic understanding of how surgery affects different aspects of a patient's health, augment other clinical outcomes, and are commonly used to determine efficacy in clinical trials. Moreover, our national reimbursement structure is currently evolving to include PROs for certain surgical conditions in measures of quality and with direct linkage to payments. Even so, there continues to be challenges in the implementation of PRO measures in everyday surgical practice, with questions of optimal administration and how to integrate these measures into provider work flow. Despite these challenges, PROs provide vital information regarding surgical efficacy and quality and are critical in the delivery of patient-centered care.

Bias through selective inclusion and attrition: Representativeness when comparing provider performance with routine outcome monitoring data

Background

Observational research based on routine outcome monitoring is prone to missing data, and outcomes can be biased due to selective inclusion at baseline or selective attrition at posttest. As patients with complete data may not be representative of all patients of a provider, missing data may bias results, especially when missingness is not random but systematic.

Methods

The present study establishes clinical and demographic patient variables relevant for representativeness of the outcome information. It applies strategies to estimate sample selection bias (weighting by inclusion propensity) and selective attrition bias (multiple imputation based on multilevel regression analysis) and estimates the extent of their impact on an index of provider performance. The association between estimated bias and response rate is also investigated.

Results

Provider‐based analyses showed that in current practice, the effect of selective inclusion was minimal, but attrition had a more substantial effect, biasing results in both directions: overstating and understating performance. For 22% of the providers, attrition bias was estimated to be in excess of 0.05 ES. Bias was associated with overall response rate (r = .50). When selective inclusion and attrition bring providers' response below 50%, it is more likely that selection bias increased beyond a critical level, and conclusions on the comparative performance of such providers may be misleading.

Conclusions

Estimates of provider performance were biased by selection, especially by missing data at posttest. Results on the extent and direction of bias and minimal requirements for response rates to arrive at unbiased performance indicators are discussed.

Multidimensional performance assessment of public sector organisations using dominance criteria

Public sector organisations pursue multiple objectives and serve a number of stakeholders. But stakeholders are rarely explicit about the valuations they attach to different objectives, nor are these valuations likely to be identical. This complicates the assessment of their performance because no single set of weights can be chosen legitimately to aggregate outputs into unidimensional composite scores. We propose the use of dominance criteria in a multidimensional performance assessment framework to identify best practice and poor performance under relatively weak assumptions about stakeholders' preferences. We use as an example providers of hip replacement surgery in the English National Health Service and estimate multivariate multilevel models to study their performance in terms of length of stay, readmission rates, post-operative patient-reported health status and waiting time. We find substantial correlation between objectives and demonstrate that ignoring the correlation can lead to incorrect assessments of performance.

Chronic Urticaria Patient Perspective (CUPP): The First Validated Tool for Assessing Quality of Life in Clinical Practice

BACKGROUND

There is a need for validated tools to assess health-related quality of life (HRQoL) in routine clinical practice.

OBJECTIVE

The aim of this study was to validate the Chronic Urticaria Patient Perspective (CUPP) for assessment of patients with chronic urticaria (CU) in clinical practice.

METHODS

A provisional CUPP was developed from candidate items identified by following an iterative process in a retrospective analysis of 249 Chronic Urticaria Quality of Life Questionnaire questionnaires. The psychometric properties of the CUPP were then tested on a sample of patients enrolled in 13 Italian centers.

RESULTS

The study population in the validation phase comprised 152 patients. The 10-item version of the CUPP showed satisfactory internal consistency (Cronbach's alpha values of 0.76 at visit 1 and 0.90 at visit 2), good criteria, and discriminative and convergent validity. Reliability was assessed in 34 patients with no changes in health (Global Rating Scale = 0 at visit 2) and was satisfactory (CCC [concordance correlation coefficient] = 0.9). Changes in CUPP scores were significantly associated with changes in Urticaria Activity Score (UAS)-Hive count (r = 0.36, P < .001), UAS-Itch severity (r = 0.48, P < .001), and UAS-Total score (r = 0.342, P < .001), all of which indicated good responsiveness. The minimal important difference was 1.5.

CONCLUSIONS

CUPP is a simple 10-question tool with good psychometric properties that provides a valid, reliable, and standardized measurement of HRQoL in patients with CU.

Use of large-scale HRQoL datasets to generate individualised predictions and inform patients about the likely benefit of surgery

Purpose

The English NHS has mandated the routine collection of health-related quality of life (HRQoL) data before and after surgery, giving prospective patient information about the likely benefit of surgery. Yet, the information is difficult to access and interpret because it is not presented in a lay-friendly format and does not reflect patients’ individual circumstances. We set out a methodology to generate personalised information to help patients make informed decisions.

Methods

We used anonymised, pre- and postoperative EuroQol-5D-3L (EQ-5D) data for over 490,000 English NHS patients who underwent primary hip or knee replacement surgery or groin hernia repair between April 2009 and March 2016. We estimated linear regression models to relate changes in EQ-5D utility scores to patients’ own assessment of the success of surgery, and calculated from that minimally important differences for health improvements/deteriorations. Classification tree analysis was used to develop algorithms that sort patients into homogeneous groups that best predict postoperative EQ-5D utility scores.

Results

Patients were classified into between 55 (hip replacement) to 60 (hernia repair) homogeneous groups. The classifications explained between 14 and 27% of variation in postoperative EQ-5D utility score.

Conclusions

Patients are heterogeneous in their expected benefit from surgery, and decision aids should reflect this. Large administrative datasets on HRQoL can be used to generate the required individualised predictions to inform patients.

Elective hospital admissions: secondary data analysis and modelling with an emphasis on policies to moderate growth

Background

The English NHS faces financial pressures that may render the growth rates of elective admissions seen between 2001/2 and 2011/12 unsustainable. A better understanding of admissions growth, and the influence of policy, are needed to minimise the impact on health gain for patients.

Objectives

This project had several objectives: (1) to better understand the determinants of elective activity and policy to moderate growth at minimum health loss for patients; (2) to build a rich data set integrating health, practice and local area data to study general practitioner (GP) referrals and resulting admissions; (3) to predict patients whose treatment is unlikely to be cost-effective using patient-reported outcomes and to examine variation in provider performance; and (4) to study how policies that aim to reduce elective admissions may change demand for emergency care. The main drivers of elective admissions growth have increased either supply of or demand for care, and could include, for example, technical innovations or increased awareness of treatment benefits. Of the factors studied, neither system reform nor population ageing appears to be a key driver. The introduction of the prospective payment tariff ‘Payment by Results’ appears to have led to primary care trusts (PCTs) having increasingly similar lengths of stay. In deprived areas, increasing GP supply appears to moderate elective admissions. Reducing the incidence of single-handed practices tends to reduce referrals and admissions. Policies to reduce referrals are likely to reduce admissions but treatments may be particularly reduced in the lowest referring practices, in which resulting health loss may be greatest. In this model, per full-time equivalent, female and highly experienced GPs identify more patients admitted by specialists.

Results

It appears from our studies that some patient characteristics are associated with not achieving sufficient patient gain to warrant cost-effective treatment. The introduction of independent sector treatment centres is estimated to have caused an increase in emergency activity rates at local PCTs. The explanations offered for increasing elective admissions indicate that they are manageable by health policy.

Conclusions

Further work is required to understand some of the results identified, such as whether or not high-volume Clinical Commissioning Groups are fulfilling unmet need; why some practices refer at low rates relative to admissions; why the period effect, which results from factors that equally affect all in the study at a point in time, dominates in the age–period–cohort analysis; and exactly how the emergency and elective sections of hospital treatment interact. This project relies on the analysis of secondary data. This type of research does not easily facilitate the important input of clinical experts or service users. It would be beneficial if other methods, including surveys and consultation with key stakeholders, could be incorporated into future research now that we have uncovered important questions.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

A Review of Databases Used in Orthopaedic Surgery Research and an Analysis of Database Use in Arthroscopy: The Journal of Arthroscopic and Related Surgery

PURPOSE

The purpose of this study was to evaluate how database use has changed over time in Arthroscopy: The Journal of Arthroscopic and Related Surgery and to inform readers about available databases used in orthopaedic literature.

METHODS

An extensive literature search was conducted to identify databases used in Arthroscopy and other orthopaedic literature. All articles published in Arthroscopy between January 1, 2006, and December 31, 2015, were reviewed. A database was defined as a national, widely available set of individual patient encounters, applicable to multiple patient populations, used in orthopaedic research in a peer-reviewed journal, not restricted by encounter setting or visit duration, and with information available in English.

RESULTS

Databases used in Arthroscopy included PearlDiver, the American College of Surgeons National Surgical Quality Improvement Program, the Danish Common Orthopaedic Database, the Swedish National Knee Ligament Register, the Hospital Episodes Statistics database, and the National Inpatient Sample. Database use increased significantly from 4 articles in 2013 to 11 articles in 2015 (P = .012), with no database use between January 1, 2006, and December 31, 2012.

CONCLUSIONS

Database use increased significantly between January 1, 2006, and December 31, 2015, in Arthroscopy.

LEVEL OF EVIDENCE

Level IV, systematic review of Level II through IV studies.

Choice of hospital: Which type of quality matters?

The implications of hospital quality competition depend on what type of quality affects choice of hospital. Previous studies of quality and choice of hospitals have used crude measures of quality such as mortality and readmission rates rather than measures of the health gain from specific treatments. We estimate multinomial logit models of hospital choice by patients undergoing hip replacement surgery in the English NHS to test whether hospital demand responds to quality as measured by detailed patient reports of health before and after hip replacement. We find that a one standard deviation increase in average health gain increases demand by up to 10%. The more traditional measures of hospital quality are less important in determining hospital choice.

Addressing Missing Data in Patient-Reported Outcome Measures (PROMS): Implications for the Use of PROMS for Comparing Provider Performance

Patient-reported outcome measures (PROMs) are now routinely collected in the English National Health Service and used to compare and reward hospital performance within a high-powered pay-for-performance scheme. However, PROMs are prone to missing data. For example, hospitals often fail to administer the pre-operative questionnaire at hospital admission, or patients may refuse to participate or fail to return their post-operative questionnaire. A key concern with missing PROMs is that the individuals with complete information tend to be an unrepresentative sample of patients within each provider and inferences based on the complete cases will be misleading. This study proposes a strategy for addressing missing data in the English PROM survey using multiple imputation techniques and investigates its impact on assessing provider performance. We find that inferences about relative provider performance are sensitive to the assumptions made about the reasons for the missing data.