Quality Improvement in the Era of Big Data

A joint physics and radiobiology DREAM team vision - Towards better response prediction models to advance radiotherapy

Radiotherapy developed empirically through experience balancing tumour control and normal tissue toxicities. Early simple mathematical models formalized this practical knowledge and enabled effective cancer treatment to date. Remarkable advances in technology, computing, and experimental biology now create opportunities to incorporate this knowledge into enhanced computational models. The ESTRO DREAM (Dose Response, Experiment, Analysis, Modelling) workshop brought together experts across disciplines to pursue the vision of personalized radiotherapy for optimal outcomes through advanced modelling. The ultimate vision is leveraging quantitative models dynamically during therapy to ultimately achieve truly adaptive and biologically guided radiotherapy at the population as well as individual patient-based levels. This requires the generation of models that inform response-based adaptations, individually optimized delivery and enable biological monitoring to provide decision support to clinicians. The goal is expanding to models that can drive the realization of personalized therapy for optimal outcomes. This position paper provides their propositions that describe how innovations in biology, physics, mathematics, and data science including AI could inform models and improve predictions. It consolidates the DREAM team's consensus on scientific priorities and organizational requirements. Scientifically, it stresses the need for rigorous, multifaceted model development, comprehensive validation and clinical applicability and significance. Organizationally, it reinforces the prerequisites of interdisciplinary research and collaboration between physicians, medical physicists, radiobiologists, and computational scientists throughout model development. Solely by a shared understanding of clinical needs, biological mechanisms, and computational methods, more informed models can be created. Future research environment and support must facilitate this integrative method of operation across multiple disciplines.

Eugene Nicholas Myers' Lecture on Head and Neck Cancer, 2020: The Surgeon as a Prognostic Factor in Head and Neck Cancer Patients Undergoing Surgery

This paper is a transcript of the 29 th Eugene N. Myers, MD International Lecture on Head and Neck Cancer presented at the American Academy of Otolaryngology-Head and Neck Surgery (AAO-HNS) in 2020. By the end of the 19 th century, the survival rate in treated patients was 10%. With the improvements in surgical techniques, currently, about two thirds of patients survive for > 5 years. Teamwork and progress in surgical reconstruction have led to advancements in ablative surgery; the associated adjuvant treatments have further improved the prognosis in the last 30 years. However, prospective trials are lacking; most of the accumulated knowledge is based on retrospective series and some real-world data analyses. Current knowledge on prognostic factors plays a central role in an efficient treatment decision-making process. Although the influence of most tumor- and patient-related prognostic factors in head and neck cancer cannot be changed by medical interventions, some environmental factors-including treatment, decision-making, and quality-can be modified. Ideally, treatment strategy decisions should be taken in dedicated multidisciplinary team meetings. However, evidence suggests that surgeons and hospital volume and specialization play major roles in patient survival after initial or salvage head and neck cancer treatment. The metrics of surgical quality assurance (surgical margins and nodal yield) in neck dissection have a significant impact on survival in head and neck cancer patients and can be influenced by the surgeon's expertise. Strategies proposed to improve surgical quality include continuous performance measurement, feedback, and dissemination of best practice measures.

Evaluation of Reliability and Correlations of Quality Measures in Cancer Care

IMPORTANCE

Measurement of the quality of care is important for alternative payment models in oncology, yet the ability to distinguish high-quality from low-quality care across oncology practices remains uncertain.

OBJECTIVE

To assess the reliability of cancer care quality measures across oncology practices using registry and claims-based measures of process, utilization, end-of-life (EOL) care, and survival, and to assess the correlations of practice-level performance across measure and cancer types.

DESIGN, SETTING, AND PARTICIPANTS

This cross-sectional study used the Surveillance, Epidemiology, and End Results (SEER) Program registry linked to Medicare administrative data to identify individuals with lung cancer, breast cancer, or colorectal cancer (CRC) that was newly diagnosed between January 1, 2011, and December 31, 2015, and who were treated in oncology practices with 20 or more patients. Data were analyzed from January 2018 to December 2020.

MAIN OUTCOMES AND MEASURES

Receipt of guideline-recommended treatment and surveillance, hospitalizations or emergency department visits during 6-month chemotherapy episodes, care intensity in the last month of life, and 12-month survival were measured. Summary measures for each domain in each cohort were calculated. Practice-level rates for each measure were estimated from hierarchical linear models with practice-level random effects; practice-level reliability (reproducibility) for each measure based on the between-measure variance, within-measure variance, and distribution of patients treated in each practice; and correlations of measures across measure and cancer types.

RESULTS

In this study of SEER registry data linked to Medicare administrative data from 49 715 patients with lung cancer treated in 502 oncology practices, 21 692 with CRC treated in 347 practices, and 52 901 with breast cancer treated in 492 practices, few practices had 20 or more patients who were eligible for most process measures during the 5-year study period. Patients were 65 years or older; approximately 50% of the patients with lung cancer and CRC and all of the patients with breast cancer were women. Most measures had limited variability across practices. Among process measures, 0 of 6 for lung cancer, 0 of 6 for CRC, and 3 of 11 for breast cancer had a practice-level reliability of 0.75 or higher for the median-sized practice. No utilization, EOL care, or survival measure had reliability across practices of 0.75 or higher. Correlations across measure types were low (r ≤ 0.20 for all) except for a correlation between the CRC process and 1-year survival summary measures (r = 0.35; P < .001). Summary process measures had limited or no correlation across lung cancer, breast cancer, and CRC (r ≤ 0.16 for all).

CONCLUSIONS AND RELEVANCE

This study found that quality measures were limited by the small numbers of Medicare patients with newly diagnosed cancer treated in oncology practices, even after pooling 5 years of data. Measures had low reliability and had limited to no correlation across measure and cancer types, suggesting the need for research to identify reliable quality measures for practice-level quality assessments.

Effect of Case-Mix and Random Variation on Breast Cancer Care Quality Indicators and Their Rankability

OBJECTIVES

Hospital comparisons to improve quality of care require valid and reliable quality indicators. We aimed to test the validity and reliability of 6 breast cancer indicators by quantifying the influence of case-mix and random variation.

METHODS

The nationwide population-based database included 79 690 patients with breast cancer from 91 Dutch hospitals between 2011 and 2016. The indicator-scores calculated were: (1) irradical breast-conserving surgery (BCS) for invasive disease, (2) irradical BCS for ductal carcinoma-in-situ, (3) breast contour-preserving treatment, (4) magnetic resonance imaging (MRI) before neo-adjuvant chemotherapy, (5) radiotherapy for locally advanced disease, and (6) surgery within 5 weeks from diagnosis. Case-mix and random variation adjustments were performed by multivariable fixed and random effect logistic regression models. Rankability quantified the between-hospital variation, representing unexplained differences that might be the result of the level of quality of care, as low (<50%), moderate (50%-75%), or high (>75%).

RESULTS

All of the indicators showed between-hospital variation with wide (interquartile) ranges. Case-mix adjustment reduced variation in indicators 1 and 3 to 5. Random variation adjustment (further) reduced the variation for all indicators. Case-mix and random variation adjustments influenced the indicator-scores of individual hospitals and their ranking. Rankability was poor for indicator 1, 2, and 5, and moderate for 3, 4, and 6.

CONCLUSIONS

The 6 indicators lacked validity and/or reliability to a certain extent. Although measuring quality indicators may stimulate quality improvement in general, comparisons and judgments of individual hospital performance should be made with caution if based on indicators that have not been tested or adjusted for validity and reliability, especially in benchmarking.

From Multiple Quality Indicators of Breast Cancer Care Toward Hospital Variation of a Summary Measure

OBJECTIVES

To improve quality in breast cancer care, large numbers of quality indicators are collected per hospital, but benchmarking remains complex. We aimed to assess the validity of indicators, develop a textbook outcome summary measure, and compare case-mix adjusted hospital performance.

METHODS

From a nationwide population-based registry, all 79 690 nonmetastatic breast cancer patients surgically treated between 2011 and 2016 in 91 hospitals in The Netherlands were included. Twenty-one indicators were calculated and their construct validity tested by Spearman's rho. Between-hospital variation was expressed by interquartile range (IQR), and all valid indicators were included in the summary measure. Standardized scores (observed/expected based on case mix) were calculated as above (>100) or below (<100) expected. The textbook outcome was presented as a continuous and all-or-none score.

RESULTS

The size of between-hospital variation varied between indicators. Sixteen (76%) of 21 quality indicators showed construct validity, and 13 were included in the summary measure after excluding redundant indicators that showed collinearity with others owing to strong construct validity. The median all-or-none textbook outcome score was 49% (IQR 42%-54%) before and 49% (IQR 48%-51%) after case-mix adjustment. From the total of 91 hospitals, 3 hospitals were positive (3%) and 9 (10%) were negative outliers.

CONCLUSIONS

The textbook outcome summary measure showed discriminative ability when hospital performance was presented as an all-or-none score. Although indicator scores and outlier hospitals should always be interpreted cautiously, the summary measure presented here has the potential to improve Dutch breast cancer quality indicator efforts and could be implemented to further test its validity, feasibility, and usefulness.

The Value of Repeated Breast Surgery as a Quality Indicator in Breast Cancer Care

Breast-conserving surgery, a major achievement in surgical oncology, has allowed an increasing number of breast cancer patients to avoid the mutilation of mastectomy. However, mastectomy still is performed in certain circumstances although breast-conserving surgery would be equally safe. Many reasons, including patients' and surgeons' personal motivations, influence the decision-making process before the final choice between breast preservation and mastectomy. The importance of quality measurement and reporting in medicine is increasingly recognized, and breast surgery is no exception. The substantial variability of re-excision rates for positive surgical margins after a first attempt at breast-conserving surgery suggests that improvement is possible. Therefore, the re-excision rate has been proposed as a quality metric for assessing and comparing the performance of different institutions. Indeed, re-excision rates can be reduced by actionable factors such as accurate preoperative local staging, localization of occult lesions, and intraoperative assessment of the oriented specimen. However, equally important non-actionable risk factors pertaining the biology, detectability, and resectability of the tumor also should be taken into account. Therefore, if the re-excision rate has to be used as a performance indicator of breast surgical care, critical interpretation of results with accurate case-mix adjustment are mandatory, and reasonable targets must be appropriately set so that surgeons treating patients at higher risk of positive margins are not unduly penalized.

Ten-year patient journey of stage III non-small cell lung cancer patients: A single-center, observational, retrospective study in Korea (Realtime autOmatically updated data warehOuse in healTh care; UNIVERSE-ROOT study)

INTRODUCTION

Until the recent approval of immunotherapy after completing concurrent chemoradiotherapy (CCRT), there has been little progress in treating unresectable stage III non-small cell lung cancer (NSCLC). This prompted us to search real-world data (RWD) to better understand diagnosis and treatment patterns, and outcomes.

METHODS

This non-interventional observational study used a unique, novel algorithm for big data analysis to collect and assess anonymized patient electronic medical records from a clinical data warehouse (CDW) over a 10-year period to capture real-world patterns of diagnosis, treatment, and outcomes of stage III NSCLC patients. We describe real-world patterns of diagnosis and treatment of patients with newly-diagnosed stage III NSCLC, and patients' characteristics, and assessment of treatment outcomes.

RESULTS

We analyzed clinical variables from 23,735 NSCLC patients. Stage III patients (N = 4138, 18.2 %) were diagnosed as IIIA (N = 2,547, 11.2 %) or IIIB (N = 1,591. 7.0 %). Treated stage III patients (N = 2530, 61.1 %) had a median age of 64.2 years, were mostly male (78.5 %) and had an ECOG performance status of 1 (65.2 %). Treatment comprised curative-intent surgery (N = 1,254, 49.6 %) with 705 receiving neoadjuvant therapy; definitive CRT (N = 648, 25.6 %); palliative CT (N = 270, 10.7 %), or thoracic RT (N = 170, 6.7 %). Median OS (range) for neoadjuvant, surgery, CRT, palliative chemotherapy, lung RT alone, and supportive care was 49.2 (42.0-56.5), 52.5 (43.1-61.9), 30.3 (26.6-34.0), 14.7 (13.0-16.4), 8.8 (6.2-11.3), and 2.0 (1.0-3.0) months, respectively.

CONCLUSIONS

This unique in-house algorithm enabled a rapid and comprehensive analysis of big data through a CDW, with daily automatic updates that documented real-world PFS and OS consistent with the published literature, and real-world treatment patterns and clinical outcomes in stage III NSCLC patients.

Real-world evidence research based on big data: Motivation-challenges-success factors

Background

In recent years there has been an increasing, partially also critical interest in understanding the potential benefits of generating real-world evidence (RWE) in medicine.

Objectives

The benefits and limitations of RWE in the context of randomized controlled trials (RCTs) are described along with a view on how they may complement each other as partners in the generation of evidence for clinical oncology. Moreover, challenges and success factors in building an effective RWE network of cooperating cancer centers are analyzed and discussed.

Material and methods

This article is based on a selective literature search (predominantly 2015–2017) combined with our practical experience to date in establishing European oncology RWE networks.

Results

RWE studies can be highly valuable and complementary to RCTs due to their high external validity. If cancer centers successfully address the various challenges in the establishment of an effective RWE study network and in the consequent execution of studies, they may efficiently generate high-quality research findings on treatment effectiveness and safety. Concerns pertaining to data privacy are of utmost importance and discussed accordingly. Securing data completeness, accuracy, and a common data structure on routinely collected disease and treatment-related data of patients with cancer is a challenging task that requires high engagement of all participants in the process.

Conclusion

Based on the discussed prerequisites, the analysis of comprehensive and complex real-world data in the context of a RWE study network represents an important and promising complementary partner to RCTs. This enables research into the general quality of cancer care and can permit comparative effectiveness studies across partner centers. Moreover, it will provide insights into a broader optimization of cancer care, refined therapeutic strategies for patient subgroups as well as avenues for further research in oncology.