Improving the Delivery of High-Quality Cancer Care in Medically Underserved Communities: A Formative Evaluation Method

BACKGROUND AND OBJECTIVES

In 2015, the American Society of Clinical Oncology launched a new program: Improving Quality of Care in Underserved Communities with the overarching aim of serving patients with cancer who have traditionally had difficulty accessing the care they need. Cancer care requires intense coordination of complex services to provide safe, effective, timely, and equitable care. If chemotherapy and/or radiation is needed, patients must navigate a complex system of care many times, a formidable challenge for many disadvantaged patients. Many practices believe that these patients face such significant issues that it is almost impossible to provide high-quality care. A grant from the Stavros Niarchos Foundation allowed us to select 4 oncology practices serving high proportions of racial minorities and persons of low socioeconomic status to participate in the new American Society of Clinical Oncology program. The program had 2 objectives: (1) to improve the capacity and capability of the participating practices to provide evidence-based, high-quality care; and (2) to identify and disseminate lessons learned for improving quality of care among oncology practices serving underserved patients.

METHODS

The program leveraged existing programs including the Quality Oncology Practice Initiative, which is a national performance measurement and improvement program that collects data about processes of care provided in the outpatient medical oncology setting, and the American Society of Clinical Oncology Quality Training Program, which provides training in how to apply the tools and methods of quality improvement in routine care settings. Training was provided in face-to-face and virtual meetings and participants were provided mentors throughout the program. At the conclusion, a formative evaluation method was used to assess whether the goals had been achieved. Objectives, activities, and desired outcomes were identified for each of the goals and thus became the framework for the evaluation.

RESULTS

The program met the stated goals and objectives. The evaluation revealed many successes, some surprises, and a list of improvements that were incorporated in the next iteration of this program. Based on data from the evaluation, the Niarchos Foundation provided funds for an additional 10 practices to participate in a similar program in 2020.

CONCLUSION

This article outlines the evaluation of a new program demonstrating that medical oncology practices can make improvements in the care of their underserved populations if provided with the proper tools, methods, and coaching. The use of formative evaluation methodology also identified opportunities for improvement and ultimately resulted in additional funding for more practices to participate in the program.

ASCO Quality Training Program: A Five-Year Review

PURPOSE

ASCO introduced the Quality Training Program (QTP) in 2013 with the aim to train oncology professionals to design, implement, and lead successful quality improvement (QI) activities and assume leadership positions to champion culture change in their practices.

METHODS

The QTP is a formal 6-month program taught by QI faculty and mentored by QI coaches over 5 days of in-person learning across 3 sessions and hands-on learning at the participants' practices. Sessions include seminars, case examples, and small-group exercises. Participants attend in multidisciplinary teams and focus on a problem they wish to solve in their practice. Scheduled conference calls with QI coaches are held between sessions. Participants complete pre- and post-QTP surveys (10-point Likert scale, with 1 = no knowledge/competence and 10 = complete knowledge/competence) and provide direct written feedback.

RESULTS

Since its inception, QTP has had 15 courses (10 domestic and 5 international) with 120 teams and 544 total participants. QTP is led by an 8-member steering group with 16 faculty and coaches. All postsurvey items showed an increase in knowledge and competence. Each item's score was calculated as the mean difference between before and after scores. Participants stated an increase of 46%-84% (overall mean increase: knowledge, 38%; competence, 37%). The greatest increases were in methodology and practical tools to make changes in practice (writing an aim statement, implementing rapid improvement, using process analysis tools, flowcharting the process). The most common suggestion for improvement was allowing more time for the project. Participants are encouraged to write articles and present work in poster and plenary sessions. QTP courses have led to 7 manuscripts and 21 abstract presentations to national meetings. Six QTP alumni are now QI coaches and faculty.

CONCLUSION

The QTP is a successful QI course for oncology professionals who need to measure performance, investigate quality and safety issues, and implement change. It is the only oncology-focused QI training, as all faculty and coaches are providers and QI specialists with oncology experience, which makes this a unique opportunity. The success will provide further momentum to offer QTP domestically and around the world.

Toward a Common Goal: Improving Safety of Oral Chemotherapy Prescribing Practices at a Jurisdictional Level

PURPOSE

Extending the safety agenda from parenteral to oral chemotherapy was identified as a provincial improvement priority in the 2014-2019 Cancer Care Ontario (CCO) Systemic Treatment Provincial Plan. Elimination of handwritten prescriptions for oral chemotherapy was one of the specific goals and led to a provincial quality improvement (QI) initiative involving systemic treatment facilities across 14 regional cancer programs.

METHODS

The initiative was centrally organized by CCO but locally implemented by the regional partners. CCO provided templates and tools, such as preprinted orders (PPOs), project charters, and an evaluation plan, and facilitated cross-jurisdictional knowledge sharing and exchange. Regions had flexibility in determining their local implementation strategies and were responsible for conducting chart audits to evaluate implementation success. Each participating hospital completed 3 audits-at baseline, immediately after implementation (audit 1), and 1 year later (audit 2)-using either a clinic-based or an outpatient pharmacy-based assessment.

RESULTS

Thirty-five facilities providing systemic treatment participated. At baseline, the provincial average for the use of computerized physician order entry (CPOE) or PPOs for prescribing oral chemotherapy was 71%. After implementation of the QI initiative, the provincial average for the use of CPOE or PPO increased to 91% at audit 1 and 95% at audit 2.

CONCLUSION

Although not all facilities met the goal of 100% CPOE or PPO compliance, the QI initiative led to improvement in safe prescribing practices for oral chemotherapy. A coordinated QI approach between a central decision maker and local partners can be an effective strategy to encourage high-quality cancer care and promote a culture of safety across a jurisdiction.

Measuring the Impact of Academic Cancer Network Development on Clinical Integration, Quality of Care, and Patient Satisfaction

PURPOSE

Many US academic centers have acquired community practices to expand their clinical care and research footprint. The objective of this assessment was to determine whether the acquisition and integration of community oncology practices by Yale/Smilow Cancer Hospital improved outcomes in quality of care, disease team integration, clinical trial accrual, and patient satisfaction at network practice sites.

METHODS

We evaluated quality of care by testing the hypothesis that core Quality Oncology Practice Initiative measures at network sites that were acquired in 2012 were significantly different after their 2016 integration into the network. Clinical and research integration were measured using the number of tumor board case presentations and total accruals in clinical trials. We used Press-Ganey scores to measure patient satisfaction pre- and postintegration.

RESULTS

Mean Quality Oncology Practice Initiative scores at Smilow Care Centers were significantly higher in 2016 than in 2012 for core measures related to improvement in tumor staging ( z = 1.33; P < .05), signed consent and documentation plans for antineoplastic treatment ( z = 2.69; P < .01; and z = 2.36; P < .05, respectively), and appropriately quantifying and addressing pain during office visits ( z = 2.95; P < .05; and z = 3.1; P < .01, respectively). A total of 493 cases were presented by care center physicians at the tumor board in 2017 compared with 45 presented in 2013. Compared with 2012, Smilow Care Center clinical trial accrual increased from 25 to 170 patients in 2017. Last, patient satisfaction has remained at greater than the 90th percentile pre- and postintegration.

CONCLUSION

The process of integration facilitates the ability to standardize cancer practice and provides a platform for quality improvement.

Quality Improvement Training in a Variety of Cancer Care Delivery Settings: Experiences From a Comprehensive Cancer Center, an Academic Medical Center, and Community Practices

Purpose:

Training clinical and supportive staff in quality improvement (QI) theory and use of QI tools has the potential to improve oncology care delivery. We report our combined experience of providing training to oncologists in a variety of local settings and assess the effect of the training on individual participants and for institutions.

Methods:

Multidisciplinary oncology teams at a comprehensive cancer center, an academic medical center, and community practices were led through experiential QI training that spanned several months. The curriculum included didactic training sessions that attendees applied to their local project-based work and that required plan-do-study-act cycles. The curriculum was adapted to the smaller practice setting through use of a workbook and a reduced focus on quantitative methods. All teams were supported by coaches and provided final presentations to leadership. The self-rated abilities of trainees to use 15 QI tools were assessed with a pre/post training survey that had five response categories (information, skill, knowledge, understanding, and wisdom). Local institutional and external project presentations were tracked.

Results:

During 7 years, 129 trainees participated in 56 QI projects. All of the 15 QI tools had 80% of trainees rate themselves in the top three categories (knowledge, understanding, and wisdom) after the training; none met this threshold before. Multiple projects were presented in institutional and external settings. Most projects targeted three of the four domains of the ASCO Quality Oncology Practice Initiative certification program standards.

Conclusions:

We implemented and sustained QI training programs in a variety of cancer delivery settings. The flexible training model should be easily adoptable by others.

Reduction of Inappropriate Prophylactic Pegylated Granulocyte Colony-Stimulating Factor Use for Patients With Non–Small-Cell Lung Cancer Who Receive Chemotherapy: An ASCO Quality Training Program Project of the Cleveland Clinic Taussig Cancer Institute

Purpose:

Routine prophylactic pegylated granulocyte colony-stimulating factor (pGCSF) administration for patients receiving chemotherapy regimens associated with low risk (< 10%) for neutropenic fever (LRNF) is not recommended. Inappropriate use of pGCSF increases patient morbidity and health care costs.

Methods:

A multidisciplinary team reviewed the charts of patients with non–small-cell lung cancer (NSCLC) at the Taussig Cancer Institute in whom a new chemotherapy regimen was initiated from April through November 2013. pGCSF use was identified and deemed appropriate if prescribed for chemotherapy associated with high risk of neutropenic fever (> 20%) or intermediate risk (10% to 20%) if other risk factors for neutropenic fever were present. Use with LRNF chemotherapy was recorded as inappropriate.

Results:

One hundred eighty patients with NSCLC received a new chemotherapy regimen during the specified time period. Thirty-four of 119 patients (28%) treated with LRNF chemotherapy received pGCSF. Each patient received an average of 2.6 doses of pGCSF (total, 89 doses). We implemented three plan-do-study-act cycles: education of providers, development of Taussig Cancer Institute consensus guidelines for pGCSF in NSCLC, and removal of standing pGCSF orders from LRNF chemotherapy in the electronic medical record. Analysis during the change period revealed 4% of patients with NSCLC treated with LRNF chemotherapy received pGCSF. Cost analysis showed an 84% decrease in billed charges per month. No increase in neutropenic fever admissions was found.

Conclusion:

pGCSF was excessively prescribed for patients with NSCLC. Factors contributing to inappropriate use included provider lack of familiarity with guidelines and knowledge with regard to the risk of neutropenic fever for individual chemotherapy regimens, and electronic medical record chemotherapy templates that contain standing GCSF orders. Interventions to address these gaps quickly produced improved compliance with guidelines and led to significant cost savings.

Aiming for ideal care: a proposed framework for cancer quality improvement

The authors review the advances made in oncology over the past 50 years and describe methods for closing the gap in care quality.

ReCAP: Feasibility and Effectiveness of a Pilot Program to Facilitate Quality Improvement Learning in Oncology: Experience of the American Society of Clinical Oncology Quality Training Program

PURPOSE

Studies have demonstrated that structured training programs can improve health professionals' skills in performing clinical care or research. We sought to develop and test a novel quality training program (QTP) tailored to oncology clinicians.

METHODS

The American Society of Clinical Oncology QTP consisted of three in-person learning sessions and four phases: prework, planning, implementation, and sustain and spread. We measured two primary outcomes: program feasibility and effectiveness. Feasibility was evaluated by recording participation. Effectiveness was measured using the Kirkpatrick model, which evaluates four outcomes: reaction, learning, behavior, and results. We collected qualitative feedback through a focus group of participants and mixed quantitative–qualitative results from a 6-month follow-up evaluation survey. Results are presented using descriptive statistics.

RESULTS

We received feedback from of 80% of participants who took part in 92% of in-person program days. QTP deliverables were completed by 100% of teams; none withdrew from the program. Regarding reaction, 100% of respondents expressed interest in actively contributing to future QTP courses. For learning, most teams continued to use the core methodology tools (eg, project charter, aims statements) after the program. Regarding behavior, when asked about intention to serve as a local quality improvement leader, a majority said they “definitely will” serve as: team leader on a specific project (75%), project champion or sponsor (75%), or teacher or trainer for others (64%). In evaluating outcomes, 50% reported applying learned methodology to new projects at their local institution.

CONCLUSION

We demonstrate one of the first feasible and effective training programs to facilitate quality improvement learning for oncology clinicians.

Bridging the Medical Education and Quality Cancer Care Divide: A Call to Action

ASCO Leadership Development Program members were tasked by the ASCO Board of Directors to explore how to optimize integration of key aspects of oncology care in order to facilitate continuous quality improvement for the practicing oncologist. This Perspective summarizes their findings.