Forgetting the learning curve for a moment: how much performance is unrelated to own experience?

Are Medical Device Characteristics Included in HTA Methods Guidelines and Reports? A Brief Review

It is well accepted that medical devices (MDs) and procedures have several unique characteristics compared with pharmaceuticals, such as learning curve (LC), incremental innovation (II), dynamic pricing (DP), and organizational impact (OI). The objective of this study was to determine the extent to which these MD characteristics are routinely assessed by health technology assessment (HTA) agencies and incorporated in their guidelines and reports. Three approaches were taken. First, a review of the most recent HTA methods guidelines from 14 selected HTA agencies and 5 HTA networks was undertaken. Next, HTA reports from these agencies were reviewed for inclusion of MD-specific characteristics for 16 selected MDs. Finally, a narrative literature review on this topic was conducted. A total of 13 of the included HTA organizations, and some HTA networks (2/5), have published either general or MD-specific method guidelines, whilst several addressed MD-specific characteristics. NICE included all four MD characteristics in their guidelines, but this did not equate to their inclusion in published HTA evaluations. European Network HTA (EUnetHTA) described the inclusion of LC (within patient safety) and OI within their guidance. The results highlight a lack of consistency. For the narrative review, 10/149 articles identified were reviewed. Most provided recommendations on challenges faced by HTAs, proposed steps to address uncertainties around MD characteristics and reported a lack of methodological guidance for evaluating MDs. A lack of inclusion of MD characteristics in HTA is a complex interplay of several important factors. For these characteristics to become a formal part of HTA of MDs in the future, clear guidance and frameworks are required to enable manufacturers to develop appropriate evidence, and HTA practitioners to assess their impact more broadly.

Worth the trip? The effect of hospital clinic closures for patients undergoing scheduled surgery

Recent decades' hospital closures and consolidations have been rationalized with reference to arguments of efficiency and quality returns to scale and scope. However, closures are met with public outcry from patients living in areas affected by such closures if accompanying increases in travel time are not offset by a higher quality of care. It is broadly established that increases in patients' travel time to acute care lower the probability of survival, but in non-acute and scheduled care we lack knowledge about the quality of care that patients living in closure-affected areas receive. In the non-acute setting of scheduled breast cancer surgery, this study examines how hospital clinic closures affect the quality of care that closure-affected patients receive. The effects are identified using closures of breast cancer clinics in Denmark from 2000 to 2011, during which time the number of clinics was more than halved. Using event study designs on population-wide Danish register data from 1996 to 2014, this study examine changes in surgical outcomes for 9790 patients living in municipalities where the nearest clinic has been closed. The results show that closures have reduced the number of hospitalization days and shifted surgical procedures to state-of-the-art breast-conserving techniques without generating adverse health effects and without causing crowding in non-closing clinics. An examination of the mechanisms suggests that added volume returns at non-closing clinics were of less importance than simply reallocating patients to higher-quality clinics. Closures of clinics performing scheduled surgery may be an effective policy instrument if the goal is to reduce variation in the delivery of hospital care. Increased access to state-of-the-art care may counterbalance patients' concerns of losing their local clinic. However, if the clinics to be closed are small compared to non-closing clinics then there is no potential for added economies of scale or scope in non-closing clinics.

Care Levels for Fetal Therapy Centers

Fetal therapies undertaken to improve fetal outcome or to optimize transition to neonate life often entail some level of maternal, fetal, or neonatal risk. A fetal therapy center needs access to resources to carry out such therapies and to manage maternal, fetal, and neonatal complications that might arise, either related to the therapy per se or as part of the underlying fetal or maternal condition. Accordingly, a fetal therapy center requires a dedicated operational infrastructure and necessary resources to allow for appropriate oversight and monitoring of clinical performance and to facilitate multidisciplinary collaboration between the relevant specialties. Three care levels for fetal therapy centers are proposed to match the anticipated care complexity, with appropriate resources to achieve an optimal outcome at an institutional and regional level. A level I fetal therapy center should be capable of offering fetal interventions that may be associated with obstetric risks of preterm birth or membrane rupture but that would be very unlikely to require maternal medical subspecialty or intensive care, with neonatal risks not exceeding those of moderate prematurity. A level II center should have the incremental capacity to provide maternal intensive care and to manage extreme neonatal prematurity. A level III therapy center should offer the full range of fetal interventions (including open fetal surgery) and be able manage any of the associated maternal complications and comorbidities, as well as have access to neonatal and pediatric surgical intervention including indicated surgery for neonates with congenital anomalies.

Forgetting-by-not-doing: The case of surgeons and cesarean sections

This paper provides new evidence on the link between patient outcome and physician experience. Using birth certificates data from a large hospital in Italy, I analyze whether cesarean section surgeons who have performed more procedures in the recent past observe an improvement in performance. By using data from the Italian health care system, where patients are not allowed to choose their physician, I lower concerns of potential reverse causality (selective referral). I find evidence indicating a strong learning-by-doing effect: for emergent cases, a one standard deviation increase in recent experience reduces the likelihood of neonatal intensive care unit admission by nearly 3.2 percentage points (13.8%) and of being born with a low Apgar Score by about 1.9 percentage points (13.2%), all else equal. This effect is not present for the case of elective C-sections.

Volume-Outcome Relationships in Extracorporeal Membrane Oxygenation: Retrospective Analysis of Administrative Data From Pennsylvania, 2007-2015

This article seeks to understand whether a volume-outcome relationship exists in adult extracorporeal membrane oxygenation (ECMO). We examined primary administrative discharge data from the Pennsylvanian Health Care Cost Containment Council for all 2,948 consecutive adults treated with ECMO in Pennsylvania between January 1, 2007, and December 31, 2015. We used a well-fitting backwards stepwise logistic regression to obtain patient-level predicted mortality. Number of cases and risk-adjusted mortality was aggregated by calendar quarter and by hospital. Graphical and correlation analysis was used to understand the volume-outcome relationship, focusing separately on the impact of a hospital's current scale (annual volume of adult ECMO) and a hospital's cumulative experience (total cumulative volume of adult ECMO since starting operations). We found that more than 9 years, there was essentially no reduction in overall risk-adjusted mortality in the state. For individual hospitals, once institutions had as few as 50 cases' total experience or performed just 10 cases annually, there was no clear improvement in outcomes. Patients in hospitals with fewer than 50 cases ever performed appeared objectively less sick compared with patients in hospital with more experience. We conclude that there is little evidence of volume-outcome relationships in adult ECMO in this state, although we acknowledge that the absence of clinical chart data limits these conclusions.

Does my high blood pressure improve your survival? Overall and subgroup learning curves in health

Learning curves in health are of interest for a wide range of medical disciplines, healthcare providers, and policy makers. In this paper, we distinguish between three types of learning when identifying overall learning curves: economies of scale, learning from cumulative experience, and human capital depreciation. In addition, we approach the question of how treating more patients with specific characteristics predicts provider performance. To soften collinearity problems, we explore the use of least absolute shrinkage and selection operator regression as a variable selection method and Theil-Goldberger mixed estimation to augment the available information. We use data from the Belgian Transcatheter Aorta Valve Implantation (TAVI) registry, containing information on the first 860 TAVI procedures in Belgium. We find that treating an additional TAVI patient is associated with an increase in the probability of 2-year survival by about 0.16%-points. For adverse events like renal failure and stroke, we find that an extra day between procedures is associated with an increase in the probability for these events by 0.12%-points and 0.07%-points, respectively. Furthermore, we find evidence for positive learning effects from physicians' experience with defibrillation, treating patients with hypertension, and the use of certain types of replacement valves during the TAVI procedure.

Fetal Treatment 2017: The Evolution of Fetal Therapy Centers - A Joint Opinion from the International Fetal Medicine and Surgical Society (IFMSS) and the North American Fetal Therapy Network (NAFTNet)

More than 3 decades ago, a small group of physicians and other practitioners active in what they called "fetal treatment" authored an opinion piece outlining the current status and future challenges anticipated in the field. Many advances in maternal, neonatal, and perinatal care and diagnostic and therapeutic modalities have been made in the intervening years, yet a thoughtful reassessment of the basic tenets put forth in 1982 has not been published. The present effort will aim to provide a framework for contemporary redefinition of the field of fetal treatment, with a brief discussion of the necessary minimum expertise and systems base for the provision of different types of interventions for both the mother and fetus. Our goal will be to present an opinion that encourages the advancement of thoughtful practice, ensuring that current and future patients have realistic access to centers with a range of fetal therapies with appropriate expertise, experience, and subspecialty and institutional support while remaining focused on excellence in care, collaborative scientific discovery, and maternal autonomy and safety.

The Role of Learning in Health Technology Assessments: An Empirical Assessment of Endovascular Aneurysm Repairs in German Hospitals

Changes in performance due to learning may dynamically influence the results of a technology evaluation through the change in effectiveness and costs. In this study, we estimate the effect of learning using the example of two minimally invasive treatments of abdominal aortic aneurysms: endovascular aneurysm repair (EVAR) and fenestrated EVAR (fEVAR). The analysis is based on the administrative data of over 40,000 patients admitted with unruptured abdominal aortic aneurysm to more than 500 different hospitals over the years 2006 to 2013. We examine two patient outcomes, namely, in-hospital mortality and length of stay using hierarchical regression models with random effects at the hospital level. The estimated models control for patient and hospital characteristics and take learning interdependency between EVAR and fEVAR into account. In case of EVAR, we observe a significant decrease both in the in-hospital mortality and length of stay with experience accumulated at the hospital level; however, the learning curve for fEVAR in both outcomes is effectively flat. To foster the consideration of learning in health technology assessments of medical devices, a general framework for estimating learning effects is derived from the analysis. © 2017 The Authors. Health Economics published by John Wiley & Sons, Ltd.

Learning curves for cardiothoracic and vascular surgical procedures--a systematic review

OBJECTIVES

The aim of this systematic review is to evaluate the learning curve (LC) literature and identify the LC of cardiothoracic and vascular surgical procedures.

SUMMARY AND BACKGROUND

The LC describes an observation that a learner's performance improves over time during acquisition of new motor skills. Measuring the LC of surgical procedures has important implications for surgical innovation, education, and patient safety. Numerous studies have investigated LCs of isolated operations in cardiothoracic and vascular surgeries, but a lack of uniformity in the methods and variables used to measure LCs has led to a lack of systematic reviews.

METHODS

The MEDLINE®, EMBASE™, and PsycINFO® databases were systematically searched until July 2013. Articles describing LCs for cardiothoracic and vascular procedures were included. The type of procedure, statistical analysis, number of participants, procedure setting, level of participants, outcomes, and LCs were reviewed.

RESULTS

A total of 48 studies investigated LCs in cardiothoracic and vascular surgeries. Based on operating time, the LC for coronary artery bypass surgery ranged between 15 and 100 cases; for endoscopic vessel harvesting and other cardiac vessel surgery between 7 and 35 cases; for valvular surgery, which included repair and replacement, between 20 and 135 cases; for video-assisted thoracoscopic surgery, between 15 and 35 cases; for vascular neurosurgical procedures between 100 and 500 cases, based on complications; for endovascular vessel repairs between 5 and 40 cases; and for ablation procedures between 25 and 60 cases. However there was a distinct lack of standardization in the variables/outcome measures used, case selection, prior experience, and supervision of participating surgeons and a range of statistical analyses to compute LCs was noted.

CONCLUSION

LCs in cardiothoracic and vascular procedures are hugely variable depending on the procedure type, outcome measures, level of prior experience, and methods/statistics used. Uniformity in methods, variables, and statistical analysis is needed to derive meaningful comparisons of LCs. Acknowledgment and application of learning processes other than those reliant on volume-outcomes relationship will benefit LC research and training of surgeons.

Association between physicians' experience after training and maternal obstetrical outcomes: cohort study

OBJECTIVE

To assess the association between obstetricians' years of experience after training and the maternal complications of their patients during their first 40 years of post-residency practice.

DESIGN

Retrospective cohort analysis.

SETTING

Obstetrical discharges from acute care hospitals in Florida and New York between academic years 1992 and 2009.

POPULATION

6,704,311 deliveries performed by 5175 obstetricians.

MAIN OUTCOME MEASURE

Three composite measures of maternal complication rates per physician year from vaginal and cesarean births separately and combined, adjusted for secular trends.

RESULTS

Obstetricians' maternal complication rates declined during the first three decades after completion of residency. The improvement was largest in the first decade and diminished thereafter. For all deliveries, the change was -0.21 (95% confidence interval -0.23 to -0.19) percentage points per year in the first decade, -0.11 (-0.13 to -0.09) percentage points per year in the second decade, and -0.05 (-0.08 to -0.01) percentage points in the third decade (P<0.001 for second to first decade comparison; P=0.001 for third to second decade comparison). The patterns were comparable for cesarean deliveries and vaginal deliveries and across several sensitivity checks.

CONCLUSIONS

Among obstetricians practicing in Florida and New York, those with more years of experience had fewer maternal complications. This association persisted over the first three decades of practice but diminished in magnitude.

Impact of hospital case volume on quality of end-of-life care in terminal cancer patients

BACKGROUND

Quality of end-of-life (EOL) care is gaining increasing attention. However, the relationship between hospital case volume and performance of benchmark quality indicators is not well characterized. The aim of this study was to determine whether hospital case volume affects EOL care for terminal cancer patients.

METHODS

We conducted a retrospective cross-sectional study using claims data of patients who died of cancer at acute-care hospitals in Kyoto prefecture, Japan, between March 2009 and May 2010. Hospitals were grouped into tertiles based on the number of terminal cancer cases. We used multilevel logistic regression models to examine the association of the following quality indicators with the tertiles: opioid use during the last 2 months of life (indicating good quality of care), provision of intensive care unit (ICU) service or life-sustaining treatments during the last month of life (poor quality), and chemotherapy during the last month of life (poor quality).

RESULTS

The final sample for analysis consisted of 3294 decedents from 88 hospitals. Significant associations between hospital case volume and quality of EOL care were identified after adjusting for patient and hospital characteristics. Small- and medium-volume hospitals were found to be less likely to administer opioids, and medium-volume hospitals were more likely to provide ICU service or life-sustaining treatments when compared with large-volume hospitals. No significant association between chemotherapy use and case volume was observed.

CONCLUSIONS

The results showed that the case volume of terminally ill cancer patients was associated with several aspects of quality of EOL care.

Surgeon and hospital volume as quality indicators for CABG in Taiwan: examining hazard to mortality and accounting for unobserved heterogeneity

OBJECTIVE

To investigate whether provider volume has an impact on the hazard of mortality for coronary artery bypass grafting (CABG) patients in Taiwan.

DATA SOURCES/STUDY SETTING

Multiple sources of linked data from the National Health Insurance Program in Taiwan.

STUDY DESIGN

The linked data were used to identify 27,463 patients who underwent CABG without concomitant angioplasty or valve procedures and the surgeon and hospital volumes. Generalized estimating equations and hazard models were estimated to assess the impact of volume on mortality. The hazard modeling technique used accounts for bias stemming from unobserved heterogeneity.

PRINCIPAL FINDINGS

Both surgeon and hospital volume quartiles are inversely related to the hazard of mortality after CABG. Patients whose surgeon is in the three higher volume quartiles have lower 1-, 3-, 6-, and 12-month mortality after CABG, while only those having their procedure performed at the highest quartile of volume hospitals have lower mortality outcomes.

CONCLUSIONS

Mortality outcomes are related to provider CABG volume in Taiwan. Unobserved heterogeneity is a concern in the volume-outcome relationship; after accounting for it, surgeon volume effects on short-term mortality are large. Using models controlling for unobserved heterogeneity and examining longer term mortality may still differentiate provider quality by volume.

Learning by doing, scale effects, or neither? Cardiac surgeons after residency

OBJECTIVE

To examine impacts of operating surgeon scale and cumulative experience on postoperative outcomes for patients treated with coronary artery bypass grafts (CABG) by "new" surgeons. Pooled linear, fixed effects panel, and instrumented regressions were estimated.

DATA SOURCES

The administrative data included comorbidities, procedures, and outcomes for 19,978 adult CABG patients in Florida in 1998-2006, and public data on 57 cardiac surgeons who completed residencies after 1997.

STUDY DESIGN

Analysis was at the patient level. Controls for risk, hospital scale and scope, and operating surgeon characteristics were made. Patient choice model instruments were constructed. Experience was estimated allowing for "forgetting" effects.

PRINCIPAL FINDINGS

Panel regressions with surgeon fixed effects showed neither surgeon scale nor cumulative volumes significantly impacted mortality nor consistently impacted morbidity. Estimation of "forgetting" suggests that almost all prior experience is depreciated from one quarter to the next. Instruments were strong, but exogeneity of volume was not rejected.

CONCLUSIONS

In postresidency surgeons, no persuasive evidence is found for learning by doing, scale, or selection effects. More research is needed to support the cautious view that, for these "new" cardiac surgeons, patient volume could be redistributed based on realized outcomes without disruption.