History Bias, Study Design, and the Unfulfilled Promise of Pay-for-Performance Policies in Health Care

Levosimendan's effect on venoarterial extracorporeal membrane oxygenation weaning

BACKGROUND

Veno-arterial extracorporeal membrane oxygenation (V-A ECMO) provides temporary haemodynamic support in refractory cardiogenic shock. Recent retrospective studies on levosimendan on V-A ECMO weaning had conflicting results. This study aimed to determine the association between levosimendan on V-A ECMO weaning success in a tertiary centre in Hong Kong.

METHODS

This retrospective study was conducted in an intensive care unit in Hong Kong. All adult patients requiring V-A ECMO from January 2016 to September 2020 were included. Patients who were given levosimendan were compared to patients who were not, on rates of successful V-A ECMO weaning. The groups were also compared after propensity matching based on covariates closely associated with the use of levosimendan.

RESULTS

A total of 119 patients were included in the study, with 38 in the levosimendan group and 81 in the non-levosimendan group. Patients treated with levosimendan trended towards improved weaning success, but the difference was not statistically significant (63% vs 53%, p = 0.404). In the propensity-matched groups, there was no difference in weaning success (odds ratio 1.00, 95% CI 0.23-8.00). The levosimendan group was associated with lower vasopressor requirement, lower lactate levels, and more significant drop in lactate in the first 2 days of V-A ECMO. The levosimendan group had longer ECMO duration. There was no difference in other secondary outcomes including mortality, length of stay in ICU and hospital and duration of mechanical ventilation. There was no difference in the rate of ventricular arrhythmias.

CONCLUSION

Levosimendan did not improve V-A ECMO weaning success in our cohort with refractory cardiogenic shock.

Association between discontinuing chronic opioid therapy and newly diagnosed substance use disorders, accidents, self-inflicted injuries and drug overdoses within the prescribers' health care system: a retrospective cohort study

BACKGROUND AND AIM

Prescribers are commonly confronted with discontinuing opioid therapy among patients prescribed chronic opioid therapy (COT). This study aimed to measure the association between discontinuing COT and diagnoses of substance use disorders (SUDs) and opioid-related adverse outcomes (AOs).

DESIGN

Retrospective cohort study.

SETTING

United States Veterans Healthcare Administration.

PARTICIPANTS

Veterans with chronic pain on COT who discontinued opioid therapy were compared with those continuing COT using data from fiscal years 2009 to 2015.

MEASUREMENTS

Newly diagnosed substance use disorders (SUD composite; individual types: opioid, non-opioid drug and alcohol use disorders) and opioid-related adverse outcomes (AO composite; individual types: accidents resulting in wounds/injuries, opioid-related accidents/overdoses, alcohol and non-opioid medication-related accidents/overdoses, self-inflicted injuries and violence-related injuries) were evaluated. Primary analyses were conducted using 1:1 matching of discontinuers with those continuing COT based on propensity score and index date (±180-day window). Sensitivity analyses were conducted using logistic regressions with stabilized inverse probability of treatment weighting (SIPTW) and instrumental variable (IV) models.

FINDINGS

A total of 15 695 (75.4%) and 17 337 (76.6%) discontinuers were matched with those continuing COT among the cohorts testing SUD and AO development respectively. In the primary propensity score matched analyses, the composite SUD outcome was not different between discontinuers and those continuing COT (OR = 0.932, 95% CI = 0.850, 1.022). The composite AO outcome was lower among discontinuers (OR = 0.660, 95% CI = 0.623, 0.699) compared with those continuing COT. SIPTW analyses found lower SUD (OR = 0.789, 95% CI = 0.743, 0.837), and AO (OR = 0.660, 95% CI = 0.623, 0.699) rates among discontinuers. IV models found mixed and sometimes contradictory results.

CONCLUSIONS

Discontinuing patients from chronic opioid therapy appears to be associated with decreased diagnoses for opioid-related adverse outcomes. The association with substance use disorders appears to be inconclusive.

Impact of transitioning from long-term to intermittent opioid therapy on the development of opioid-related adverse outcomes: A retrospective cohort study

BACKGROUND

Increasing pressures exist to reduce or discontinue opioid use among patients currently on long-term opioid therapy (LTOT). It is essential to understand the potential effects of opioid reduction.

METHODS

This retrospective cohort study was conducted among veterans with chronic pain and on LTOT. Using 1:1 propensity score-matched samples of veterans switching to intermittent opioid therapy and those continuing LTOT, we examined the development of subsequent substance use disorders (SUD composite; individual SUD types: opioid, non-opioid drug, and alcohol use disorders) and opioid-related adverse outcomes (ORAO composite; individual ORAO types: accidents resulting in wounds/injuries, opioid-related and alcohol/non-opioid medication-related accidents and overdoses, self-inflicted and violence-related injuries). Sensitivity analyses were conducted using logistic regression with stabilized inverse probability of treatment weighting (SIPTW) and instrumental variable (IV) models.

RESULTS

A total of 29,293 veterans switching to intermittent therapy were matched to veterans continuing LTOT. With matched samples, no differences were found in composite SUDs and ORAOs between the groups. With SIPTW, veterans switching to intermittent opioid therapy had higher odds of composite SUDs and ORAOs (SUDs aOR=1.12, 95%CI: 1.07,1.17; ORAOs aOR=1.05, 95%CI:1.00,1.09). IV models found lower risks for composite SUDs and ORAOs among veterans switching to intermittent opioid therapy (SUDs: β = -0.38, 95%CI:-0.63,-0.13; ORAOs: β = -0.27, 95%CI:-0.50,-0.04).

CONCLUSIONS

There were no consistent associations between transitioning patients from LTOT to intermittent opioid therapy and the risk of SUDs and ORAOs.

Association Between Pain Intensity and Discontinuing Opioid Therapy or Transitioning to Intermittent Opioid Therapy After Initial Long-Term Opioid Therapy: A Retrospective Cohort Study

The purpose of this study was to evaluate changes in pain intensity among Veterans transitioning from long-term opioid therapy (LTOT) to either intermittent therapy or discontinuation compared to continued LTOT. Pain intensity was assessed using the Numeric Rating Scale in 90-day increments starting in the 90-day period prior to potential opioid transitions and the two ensuing 90-day periods after transition. Primary analyses used a 1:1 greedy propensity matched sample. A total of 29,293 Veterans switching to intermittent opioids and 5,972 discontinuing opioids were matched to Veterans continuing LTOT. Covariates were well balanced after matching except minor differences in baseline mean pain scores. Pain scores were lower in the follow up periods for those switching to intermittent opioids and discontinuing opioids compared to those continuing LTOT (0-90 days: Intermittent: 3.79, 95%CI: 3.76, 3.82; LTOT: 4.09, 95%CI: 4.06, 4.12, P < .0001; Discontinuation: 3.06, 95%CI: 2.99, 3.13; LTOT: 3.86, 95%CI: 3.79, 3.94, P = <.0001; 91-180 days: Intermittent: 3.76, 95%CI: 3.73, 3.79; LTOT: 3.99, 95%CI: 3.96, 4.02, P < .0001; Discontinuation: 3.01, 95%CI: 2.94, 3.09; LTOT: 3.80, 95%CI: 3.73, 3.87, P = <.0001). Sensitivity analyses found similar results. Discontinuing opioid therapy or switching to intermittent opioid therapy was not associated with increased pain intensity. PERSPECTIVE: This article evaluates the association of switching to intermittent opioid therapy or discontinuing opioids with pain intensity after using opioids long-term. Pain intensity decreased after switching to intermittent therapy or discontinuing opioids, but remained relatively stable for those continuing long-term opioid therapy. Switching to intermittent opioids or discontinuing opioids was not associated with increased pain intensity.

The use of segmented regression for evaluation of an interrupted time series study involving complex intervention: the CaPSAI project experience

An interrupted time series with a parallel control group (ITS-CG) design is a powerful quasi-experimental design commonly used to evaluate the effectiveness of an intervention, on accelerating uptake of useful public health products, and can be used in the presence of regularly collected data. This paper illustrates how a segmented Poisson model that utilizes general estimating equations (GEE) can be used for the ITS-CG study design to evaluate the effectiveness of a complex social accountability intervention on the level and rate of uptake of modern contraception. The intervention was gradually rolled-out over time to targeted intervention communities in Ghana and Tanzania, with control communities receiving standard of care, as per national guidelines. Two ITS GEE segmented regression models are proposed for evaluating of the uptake. The first, a two-segmented model, fits the data collected during pre-intervention and post-intervention excluding that collected during intervention roll-out. The second, a three-segmented model, fits all data including that collected during the roll-out. A much simpler difference-in-difference (DID) GEE Poisson regression model is also illustrated. Mathematical formulation of both ITS-segmented Poisson models and that of the DID Poisson model, interpretation and significance of resulting regression parameters, and accounting for different sources of variation and lags in intervention effect are respectively discussed. Strengths and limitations of these models are highlighted. Segmented ITS modelling remains valuable for studying the effect of intervention interruptions whether gradual changes, over time, in the level or trend in uptake of public health practices are attributed by the introduced intervention.

Trial Registration: The Australian New Zealand Clinical Trials registry.

Trial registration number: ACTRN12619000378123.

Trial Registration date: 11-March-2019.

Hospital performance and payment: impact of integrating pay-for-performance on healthcare effectiveness in Lebanon

Background: In 2014 the Lebanese Ministry of Public Health integrated pay-for-performance into setting hospital reimbursement tiers, to provide hospitalization service coverage for the majority of the Lebanese population. This policy was intended to improve effectiveness by decreasing unnecessary hospitalizations, and improve fairness by including risk-adjustment in setting hospital performance scores. Methods: We applied a systematic approach to assess the impact of the new policy on hospital performance. The main impact measure was a national casemix index, calculated across 2011-2016 using medical discharge and surgical procedure codes. A single-group interrupted time series analysis model with Newey ordinary least squares regression was estimated, including adjustment for seasonality, and stratified by case type. Code-level analysis was used to attribute and explain changes in casemix index due to specific diagnoses and procedures. Results: Our final model included 1,353,025 cases across 146 hospitals with a post-intervention lag-time of two months and seasonality adjustment. Among medical cases the intervention resulted in a positive casemix index trend of 0.11% per month (coefficient 0.002, CI 0.001-0.003), and a level increase of 2.25% (coefficient 0.022, CI 0.005-0.039). Trend changes were attributed to decreased cases of diarrhea and gastroenteritis, abdominal and pelvic pain, essential hypertension and fever of unknown origin. A shift from medium to short-stay cases for specific diagnoses was also detected. Level changes were attributed to improved coding practices, particularly for breast cancer, leukemia and chemotherapy. No impact on surgical casemix index was found. Conclusions: The 2014 policy resulted in increased healthcare effectiveness, by increasing the casemix index of hospitals contracted by the Ministry. This increase was mainly attributed to decreased unnecessary hospitalizations and was accompanied by improved medical discharge coding practices. Integration of pay-for-performance within a healthcare system may contribute to improving effectiveness. Effective hospital regulation can be achieved through systematic collection and analysis of routine data.

Association Between a National Insurer's Pay-for-Performance Program for Oncology and Changes in Prescribing of Evidence-Based Cancer Drugs and Spending

PURPOSE

Cancer drug prescribing by medical oncologists accounts for the greatest variation in practice and the largest portion of spending on cancer care. We evaluated the association between a national commercial insurer's ongoing pay-for-performance (P4P) program for oncology and changes in the prescribing of evidence-based cancer drugs and spending.

METHODS

We conducted an observational difference-in-differences study using administrative claims data covering 6.7% of US adults. We leveraged the geographically staggered, time-varying rollout of the P4P program to simulate a stepped-wedge study design. We included patients age 18 years or older with breast, colon, or lung cancer who were prescribed cancer drug regimens by 1,867 participating oncologists between 2013 and 2017. The exposure was a time-varying dichotomous variable equal to 1 for patients who were prescribed a cancer drug regimen after the P4P program was offered. The primary outcome was whether a patient's drug regimen was a program-endorsed, evidence-based regimen. We also evaluated spending over a 6-month episode period.

RESULTS

The P4P program was associated with an increase in evidence-based regimen prescribing from 57.1% of patients in the preintervention period to 62.2% in the intervention period, for a difference of +5.1 percentage point (95% CI, 3.0 percentage points to 7.2 percentage points; P < .001). The P4P program was also associated with a differential $3,339 (95% CI, $1,121 to $5,557; P = .003) increase in cancer drug spending and a differential $253 (95% CI, $100 to $406; P = .001) increase in patient out-of-pocket spending, but no significant changes in total health care spending ($2,772; 95% CI, -$181 to $5,725; P = .07) over the 6-month episode period.

CONCLUSION

P4P programs may be effective in increasing evidence-based cancer drug prescribing, but may not yield cost savings.

Hospital performance and payment: impact of integrating pay-for-performance on healthcare effectiveness in Lebanon

Background: In 2014 the Lebanese Ministry of Public Health integrated pay-for-performance into setting hospital reimbursement tiers, to provide hospitalization service coverage for the majority of the Lebanese population. This policy was intended to improve effectiveness by decreasing unnecessary hospitalizations, and improve fairness by including risk-adjustment in setting hospital performance scores. Methods: We applied a systematic approach to assess the impact of the new policy on hospital performance. The main impact measure was a national casemix index, calculated across 2011-2016 using medical discharge and surgical procedure codes. A single-group interrupted time series analysis model with Newey ordinary least squares regression was estimated, including adjustment for seasonality, and stratified by case type. Code-level analysis was used to attribute and explain changes in casemix index due to specific diagnoses and procedures. Results: Our final model included 1,353,025 cases across 146 hospitals with a post-intervention lag-time of two months and seasonality adjustment. Among medical cases the intervention resulted in a positive casemix index trend of 0.11% per month (coefficient 0.002, CI 0.001-0.003), and a level increase of 2.25% (coefficient 0.022, CI 0.005-0.039). Trend changes were attributed to decreased cases of diarrhea and gastroenteritis, abdominal and pelvic pain, essential hypertension and fever of unknown origin. A shift from medium to short-stay cases for specific diagnoses was also detected. Level changes were attributed to improved coding practices, particularly for breast cancer, leukemia and chemotherapy. No impact on surgical casemix index was found. Conclusions: The 2014 policy resulted in increased healthcare effectiveness, by increasing the casemix index of hospitals contracted by the Ministry. This increase was mainly attributed to decreased unnecessary hospitalizations and was accompanied by improved medical discharge coding practices. Integration of pay-for-performance within a healthcare system may contribute to improving effectiveness. Effective hospital regulation can be achieved through systematic collection and analysis of routine data.

Health system reconstitution syndrome: an often misunderstood phenomenon in global health practice

The beginning of the 21st century was marked by the new definition and framework of health systems strengthening (HSS). The global movement to improve access to high-quality care garnered new resources to design and implement comprehensive HSS programs. In this effort, billions of dollars flowed from novel mechanisms such as The Global Fund to Fight AIDS, Tuberculosis and Malaria; Gavi, the Vaccine Alliance; and several bilateral funders. However, poor health outcomes, particularly in low-income countries, raise questions about the effectiveness of HSS program implementation. While several evaluation projects focus on the ultimate impact of HSS programs, little is known about the short- and mid-term reactions occurring throughout the active implementation of HSS interventions. Using the well-documented WHO framework of six HSS building blocks, we describe the evolution and phases of health system reconstitution syndrome (HSRS), including: (1) quiescent phase, (2) reactive phase, (3) restorative phase and (4) stability phase. We also discuss the implications of HSRS on global health funding, implementation, policy and research. Recognizing signs of HSRS could improve the rigour of HSS program design and minimize premature decisions regarding the progress of HSS interventions.

Bias in Before-After Studies: Narrative Overview for Anesthesiologists

Before-after study designs are effective research tools and in some cases, have changed practice. These designs, however, are inherently susceptible to bias (ie, systematic errors) that are sometimes subtle but can invalidate their conclusions. This overview provides examples of before-after studies relevant to anesthesiologists to illustrate potential sources of bias, including selection/assignment, history, regression to the mean, test-retest, maturation, observer, retrospective, Hawthorne, instrumentation, attrition, and reporting/publication bias. Mitigating strategies include using a control group, blinding, matching before and after cohorts, minimizing the time lag between cohorts, using prospective data collection with consistent measuring/reporting criteria, time series data collection, and/or alternative study designs, when possible. Improved reporting with enforcement of the Enhancing Quality and Transparency of Health Research (EQUATOR) checklists will serve to increase transparency and aid in interpretation. By highlighting the potential types of bias and strategies to improve transparency and mitigate flaws, this overview aims to better equip anesthesiologists in designing and/or critically appraising before-after studies.

Provider Perspectives on Quality Payment Programs Targeting Diabetes in Primary Care Settings

Public and private insurers increasingly use quality payment programs as a tool to improve quality of care in primary care settings. However, little is known about primary care providers' perspectives on whether and how quality payment programs improve diabetes quality of care. In this qualitative study, the authors conducted semi-structured interviews and focus groups with 23 providers from March to June 2015. Transcripts were analyzed to identify key themes using the immersion-crystallization method. Almost all of the providers believed that insurers play a meaningful role in improving quality of care for diabetes patients. Most thought that insurers' efforts are more effective when channeled through providers and delivery systems rather than directed at patients. Providers generally believed that quality payment programs have had a positive impact on quality of diabetes care, although provider views were not evidence based. Providers in practices in which quality payment programs were believed to have had a positive impact stated that the programs provided financial incentives and resources for improved population health management systems and additional staff. Conversely, most providers did not believe that quality payment programs have had any impact via direct financial incentives to individual physicians. A few providers were skeptical about the impact of quality payment programs and noted negative consequences that they had observed. Providers recommended strategies to improve quality payment programs (eg, refine quality measures, provide regular feedback on quality and costs) and additional strategies that insurers could consider to address provider- and patient-level barriers to high-quality diabetes care.

Impact of an Advanced Imaging Utilization Review Program on Downstream Health Care Utilization and Costs for Low Back Pain

BACKGROUND

Early magnetic resonance imaging (MRI) for acute low back pain (LBP) has been associated with increased costs, greater health care utilization, and longer disability duration in workers' compensation claimants.

OBJECTIVES

To assess the impact of a state policy implemented in June 2010 that required prospective utilization review (UR) for early MRI among workers' compensation claimants with LBP.

RESEARCH DESIGN

Interrupted time series.

SUBJECTS

In total, 76,119 Washington State workers' compensation claimants with LBP between 2006 and 2014.

MEASURES

Proportion of workers receiving imaging per month (MRI, computed tomography, radiographs) and lumbosacral injections and surgery; mean total health care costs per worker; mean duration of disability per worker. Measures were aggregated monthly and attributed to injury month.

RESULTS

After accounting for secular trends, decreases in early MRI [level change: -5.27 (95% confidence interval, -4.22 to -6.31); trend change: -0.06 (-0.01 to -0.12)], any MRI [-4.34 (-3.01 to -5.67); -0.10 (-0.04 to -0.17)], and injection [trend change: -0.12 (-0.06 to -0.18)] utilization were associated with the policy. Radiograph utilization increased in parallel [level change: 2.46 (1.24-3.67)]. In addition, the policy resulted in significant decreasing changes in mean costs per claim, mean disability duration, and proportion of workers who received disability benefits. The policy had no effect on computed tomography or surgery utilization.

CONCLUSIONS

The UR policy had discernable effects on health care utilization, costs, and disability. Integrating evidence-based guidelines with UR can improve quality of care and patient outcomes, while reducing use of low-value health services.

Counter-Point: Staying Honest When Policy Changes Backfire

Despite the good intentions of the Food and Drug Administration (FDA), many drug warnings are ineffective or have unintended consequences, particularly if the media exaggerates the messages and scares the public. The controversial 2003 to 2004 FDA warnings on youth suicidality associated with antidepressant use are a case in point. In a 10-year interrupted time series (ITS) analysis in 11 health plans, we found that the warnings and hyped media coverage led to substantial reductions in antidepressant use (declines in antidepressant use and overall care corroborated in several studies), and small, visible increases in emergency room and inpatient poisonings with psychotropic drugs. In a gross misunderstanding of the method, Dr Stone calls ITS, "an intuition based upon false analogies, fallacious assumptions and analytical error." We demonstrate visually using published studies that the ITS method is one of the oldest (hundreds of years) and strongest quasi-experimental study designs, and that the alternative data analyses proposed by Dr Stone do not have rates (denominators), nor baselines, so the measures of change are invalid.

Selecting and Improving Quasi-Experimental Designs in Effectiveness and Implementation Research

Interventional researchers face many design challenges when assessing intervention implementation in real-world settings. Intervention implementation requires holding fast on internal validity needs while incorporating external validity considerations (such as uptake by diverse subpopulations, acceptability, cost, and sustainability). Quasi-experimental designs (QEDs) are increasingly employed to achieve a balance between internal and external validity. Although these designs are often referred to and summarized in terms of logistical benefits, there is still uncertainty about (a) selecting from among various QEDs and (b) developing strategies to strengthen the internal and external validity of QEDs. We focus here on commonly used QEDs (prepost designs with nonequivalent control groups, interrupted time series, and stepped-wedge designs) and discuss several variants that maximize internal and external validity at the design, execution and implementation, and analysis stages.

Association between the 2012 Health and Social Care Act and specialist visits and hospitalisations in England: A controlled interrupted time series analysis

BACKGROUND

The 2012 Health and Social Care Act (HSCA) in England led to among the largest healthcare reforms in the history of the National Health Service (NHS). It gave control of £67 billion of the NHS budget for secondary care to general practitioner (GP) led Clinical Commissioning Groups (CCGs). An expected outcome was that patient care would shift away from expensive hospital and specialist settings, towards less expensive community-based models. However, there is little evidence for the effectiveness of this approach. In this study, we aimed to assess the association between the NHS reforms and hospital admissions and outpatient specialist visits.

METHODS AND FINDINGS

We conducted a controlled interrupted time series analysis to examine rates of outpatient specialist visits and inpatient hospitalisations before and after the implementation of the HSCA. We used national routine hospital administrative data (Hospital Episode Statistics) on all NHS outpatient specialist visits and inpatient hospital admissions in England between 2007 and 2015 (with a mean of 26.8 million new outpatient visits and 14.9 million inpatient admissions per year). As a control series, we used equivalent data on hospital attendances in Scotland. Primary outcomes were: total, elective, and emergency hospitalisations, and total and GP-referred specialist visits. Both countries had stable trends in all outcomes at baseline. In England, after the policy, there was a 1.1% (95% CI 0.7%-1.5%; p < 0.001) increase in total specialist visits per quarter and a 1.6% increase in GP-referred specialist visits (95% CI 1.2%-2.0%; p < 0.001) per quarter, equivalent to 12.7% (647,000 over the 5,105,000 expected) and 19.1% (507,000 over the 2,658,000 expected) more visits per quarter by the end of 2015, respectively. In Scotland, there was no change in specialist visits. Neither country experienced a change in trends in hospitalisations: change in slope for total, elective, and emergency hospitalisations were -0.2% (95% CI -0.6%-0.2%; p = 0.257), -0.2% (95% CI -0.6%-0.1%; p = 0.235), and 0.0% (95% CI -0.5%-0.4%; p = 0.866) per quarter in England. We are unable to exclude confounding due to other events occurring around the time of the policy. However, we limited the likelihood of such confounding by including relevant control series, in which no changes were seen.

CONCLUSIONS

Our findings suggest that giving control of healthcare budgets to GP-led CCGs was not associated with a reduction in overall hospitalisations and was associated with an increase in specialist visits.