External audit of clinical practice and medical decision making in a new Asian oncology center: Results and implications for both developing and developed nations

Mediators of change in healthcare organisations subject to external assessment: a systematic review with narrative synthesis

OBJECTIVES

External inspections are widely used to improve the quality of care. The effects of inspections remain unclear and little is known about how they may work. We conducted a narrative synthesis of research literature to identify mediators of change in healthcare organisations subject to external inspections.

METHODS

We performed a literature search (1980-January 2020) to identify empirical studies addressing change in healthcare organisations subject to external inspection. Guided by the Consolidated Framework for Implementation Research, we performed a narrative synthesis to identify mediators of change.

RESULTS

We included 95 studies. Accreditation was the most frequent type of inspection (n=68), followed by statutory inspections (n=19), and external peer review (n=9). Our findings suggest that the regulatory context in which the inspections take place affect how they are acted on by those being inspected. The way inspections are conducted seem to be critical for how the inspection findings are perceived and followed up. Inspections can engage and involve staff, facilitate leader engagement, improve communication and enable the creation of new networks for reflection on clinical practice. Inspections can contribute to creating an awareness of the inspected organisation's current practice and performance gaps, and a commitment to change. Moreover, they can contribute to facilitating the planning and implementation of change, as well as self-evaluation and the use of data to evaluate performance.

CONCLUSIONS

External inspections can affect different mediators of organisational change. The way and to what extent they do depend on a range of factors related to the outer setting, the way inspections are conducted and how they are perceived and acted on by the inspected organisation. To improve the quality of care, the organisational change processes need to involve and impact the way care is delivered to the patients.

Feasibility and impact of national peer reviewed clinical audits in radiotherapy departments

PURPOSE/OBJECTIVE

A national incentive brought about the instauration of systematic clinical audits of all Belgian radiotherapy departments (n = 25) from 2011 to 2015 using the International Atomic Energy Agency QUATRO (Quality Improvement Quality Assurance Team for Radiation Oncology) methodology. The impact of these audits was evaluated and the emitted recommendations originating from the audit reports were analysed to identify areas of weakness on a national basis.

METHOD

The QUATRO audits performed in each radiotherapy department gave rise to reports in which each department received a list of recommendations that it is free to implement. These audit reports were analyzed to identify common areas for which improvements were recommended. Moreover, questionnaires were sent to all departments in order to evaluate the overall usefulness of the recommendations as well as the relevancy and the actual impact of each individual recommendation.

RESULTS

Of the 381 emitted recommendations, 34% concerned process optimization of which a quarter involved process improvement and protocol development. Twenty-seven percent of the recommendations concerned infrastructure of which one-third was related to the quality of the equipment or facility. Nineteen and 20% of recommendations addressed department organisational and staff issues respectively. When analysing the departments' feedback questionnaires, 54% of the departments evaluated the audits' recommendations as being very useful. Furthermore, 42.7% of the recommendations were found to be very relevant and 23.5% were deemed to have an important impact.

CONCLUSION

This first round of audits in Belgium allowed for the identification of common areas for improvements of practice in radiation oncology departments, with a focus on process optimization and infrastructure elements. Similarly, the audits' emitted recommendations were globally deemed very relevant. Encouraged, by this analysis, a second cycle of audits has started in Belgium with a modified version of the QUATRO document (B-QUATRO).

Recommendations for the treatment of children with radiotherapy in low- and middle-income countries (LMIC): A position paper from the Pediatric Radiation Oncology Society (PROS-LMIC) and Pediatric Oncology in Developing Countries (PODC) working groups of the International Society of Pediatric Oncology (SIOP)

Pediatric radiotherapy is a critical part of pediatric oncology protocols and the quality of the radiotherapy may determine the future quality of life for long-term survivors. Multidisciplinary team decision making provides the basis for high-quality care. However, delivery of high-quality radiotherapy is dependent on resources. This article provides guidelines for delivery of good quality radiation therapy in resource-limited countries based on rational procurement and maintenance planning, protocol development, three-dimensional planning, quality assurance, and adequate staff numbers and training.

Automatic detection of patient identification and positioning errors in radiation therapy treatment using 3-dimensional setup images

PURPOSE

To develop an automated system that detects patient identification and positioning errors between 3-dimensional computed tomography (CT) and kilovoltage CT planning images.

METHODS AND MATERIALS

Planning kilovoltage CT images were collected for head and neck (H&N), pelvis, and spine treatments with corresponding 3-dimensional cone beam CT and megavoltage CT setup images from TrueBeam and TomoTherapy units, respectively. Patient identification errors were simulated by registering setup and planning images from different patients. For positioning errors, setup and planning images were misaligned by 1 to 5 cm in the 6 anatomical directions for H&N and pelvis patients. Spinal misalignments were simulated by misaligning to adjacent vertebral bodies. Image pairs were assessed using commonly used image similarity metrics as well as custom-designed metrics. Linear discriminant analysis classification models were trained and tested on the imaging datasets, and misclassification error (MCE), sensitivity, and specificity parameters were estimated using 10-fold cross-validation.

RESULTS

For patient identification, our workflow produced MCE estimates of 0.66%, 1.67%, and 0% for H&N, pelvis, and spine TomoTherapy images, respectively. Sensitivity and specificity ranged from 97.5% to 100%. MCEs of 3.5%, 2.3%, and 2.1% were obtained for TrueBeam images of the above sites, respectively, with sensitivity and specificity estimates between 95.4% and 97.7%. MCEs for 1-cm H&N/pelvis misalignments were 1.3%/5.1% and 9.1%/8.6% for TomoTherapy and TrueBeam images, respectively. Two-centimeter MCE estimates were 0.4%/1.6% and 3.1/3.2%, respectively. MCEs for vertebral body misalignments were 4.8% and 3.6% for TomoTherapy and TrueBeam images, respectively.

CONCLUSIONS

Patient identification and gross misalignment errors can be robustly and automatically detected using 3-dimensional setup images of different energies across 3 commonly treated anatomical sites.

Radiotherapy Monte Carlo simulation using cloud computing technology

Cloud computing allows for vast computational resources to be leveraged quickly and easily in bursts as and when required. Here we describe a technique that allows for Monte Carlo radiotherapy dose calculations to be performed using GEANT4 and executed in the cloud, with relative simulation cost and completion time evaluated as a function of machine count. As expected, simulation completion time decreases as 1/n for n parallel machines, and relative simulation cost is found to be optimal where n is a factor of the total simulation time in hours. Using the technique, we demonstrate the potential usefulness of cloud computing as a solution for rapid Monte Carlo simulation for radiotherapy dose calculation without the need for dedicated local computer hardware as a proof of principal.

A cost simulation for mammography examinations taking into account equipment failures and resource utilization characteristics

OBJECTIVE

This work develops a cost analysis estimation for a mammography clinic, taking into account resource utilization and equipment failure rates.

MATERIALS AND METHODS

Two standard clinic models were simulated, the first with one mammography equipment, two technicians and one doctor, and the second (based on an actually functioning clinic) with two equipments, three technicians and one doctor. Cost data and model parameters were obtained by direct measurements, literature reviews and other hospital data. A discrete-event simulation model was developed, in order to estimate the unit cost (total costs/number of examinations in a defined period) of mammography examinations at those clinics. The cost analysis considered simulated changes in resource utilization rates and in examination failure probabilities (failures on the image acquisition system). In addition, a sensitivity analysis was performed, taking into account changes in the probabilities of equipment failure types.

RESULTS

For the two clinic configurations, the estimated mammography unit costs were, respectively, US$ 41.31 and US$ 53.46 in the absence of examination failures. As the examination failures increased up to 10% of total examinations, unit costs approached US$ 54.53 and US$ 53.95, respectively. The sensitivity analysis showed that type 3 (the most serious) failure increases had a very large impact on the patient attendance, up to the point of actually making attendance unfeasible.

CONCLUSIONS

Discrete-event simulation allowed for the definition of the more efficient clinic, contingent on the expected prevalence of resource utilization and equipment failures.

Radiation Oncology Safety Information System (ROSIS)--profiles of participants and the first 1074 incident reports

BACKGROUND AND PURPOSE

The Radiation Oncology Safety Information System (ROSIS) was established in 2001. The aim of ROSIS is to collate and share information on incidents and near-incidents in radiotherapy, and to learn from these incidents in the context of departmental infrastructure and procedures.

MATERIALS AND METHODS

A voluntary web-based cross-organisational and international reporting and learning system was developed (cf. the www.rosis.info website). Data is collected via online Department Description and Incident Report Forms. A total of 101 departments, and 1074 incident reports are reviewed.

RESULTS

The ROSIS departments represent about 150,000 patients, 343 megavoltage (MV) units, and 114 brachytherapy units. On average, there are 437 patients per MV unit, 281 per radiation oncologist, 387 per physicist and 353 per radiation therapy technologist (RT/RTT). Only 14 departments have a completely networked system of electronic data transfer, while 10 departments have no electronic data transfer. On average seven quality assurance (QA) or quality control (QC) methods are used at each department. A total of 1074 ROSIS reports are analysed; 97.7% relate to external beam radiation treatment and 50% resulted in incorrect irradiation. Many incidents arise during pre-treatment but are not detected until later in the treatment process. Where an incident is not detected prior to treatment, an average of 22% of the prescribed treatment fractions were delivered incorrectly. The most commonly reported detection methods were "found at time of patient treatment" and during "chart-check".

CONCLUSION

While the majority of the incidents that reported to this international cross-organisational reporting system are of minor dosimetric consequence, they affect on average more than 20% of the patient's treatment fractions. Nonetheless, defence-in-depth is apparent in departments registered with ROSIS. This indicates a need for further evaluation of the effectiveness of quality controls.

Cancer control in developing countries: using health data and health services research to measure and improve access, quality and efficiency

Background

Cancer is a rapidly increasing problem in developing countries. Access, quality and efficiency of cancer services in developing countries must be understood to advance effective cancer control programs. Health services research can provide insights into these areas.

Discussion

This article provides an overview of oncology health services in developing countries. We use selected examples from peer-reviewed literature in health services research and relevant publicly available documents. In spite of significant limitations in the available data, it is clear there are substantial barriers to access to cancer control in developing countries. This includes prevention, early detection, diagnosis/treatment and palliation. There are also substantial limitations in the quality of cancer control and a great need to improve economic efficiency. We describe how the application of health data may assist in optimizing (1) Structure: strengthening planning, collaboration, transparency, research development, education and capacity building. (2) Process: enabling follow-up, knowledge translation, patient safety and quality assurance. (3) Outcome: facilitating evaluation, monitoring and improvement of national cancer control efforts. There is currently limited data and capacity to use this data in developing countries for these purposes.

Summary

There is an urgent need to improve health services for cancer control in developing countries. Current resources and much-needed investments must be optimally managed. To achieve this, we would recommend investment in four key priorities: (1) Capacity building in oncology health services research, policy and planning relevant to developing countries. (2) Development of high-quality health data sources. (3) More oncology-related economic evaluations in developing countries. (4) Exploration of high-quality models of cancer control in developing countries. Meeting these needs will require national, regional and international collaboration as well as political leadership. Horizontal integration with programs for other diseases will be important.

Is rural radiation oncology practice quality as good as the big smoke? Results of the Australian radiotherapy single machine unit trial

Radiotherapy utilization rates in rural Australia are suboptimal, with one solution being the building of single machine units (SMUs). One concern raised with such an approach is the quality of care delivered in SMUs. The Australian and Victorian governments have established two SMUs in the state of Victoria, with each SMU operated as a satellite service of a major 'hub' site. We report on the planned evaluation of practice quality. Radiation oncologist (RO) clinical practice was externally audited using the Royal Australian and New Zealand College of Radiologists Peer Review Audit instrument. This tool splits RO clinical practice into documentation/quality assurance (QA) criteria and decision-making criteria. Over the four sites, 130 patients were randomly selected for audit. At hub sites, 79.6% of all criteria audited were adequate, compared with 84.4% of criteria audited at SMUs (P = 0.0002). This difference was largely because of better adherence to documentation/QA criteria at the SMU sites. RO decision-making and protocol adherence were routinely very high and consistent with other clinical practice audits. There were no significant differences between hubs and SMUs for adherence to decision-making criteria; however, the few potential deficiencies in patient care identified occurred only at the hub sites. In at least one of these cases, potential suboptimal management was as a direct result of inadequate documentation. This audit found that SMUs provide as high a standard of radiotherapeutic care as larger hub departments. The findings also emphasize the need for all departments to target clinical documentation.

Efficacy of an integrated continuing medical education (CME) and quality improvement (QI) program on radiation oncologist (RO) clinical practice

PURPOSE

There has been little radiation oncologist (RO)-specific research in continuing medical education (CME) or quality improvement (QI) program efficacy. Our aim was to evaluate a CME/QI program for changes in RO behavior, performance, and adherence to department protocols/studies over the first 12 months of the program.

METHODS AND MATERIALS

The CME/QI program combined chart audit with feedback (C-AWF), simulation review AWF (SR-AWF), reminder checklists, and targeted CME tutorials. Between April 2003 and March 2004, management of 75 patients was evaluated by chart audit with feedback (C-AWF) and 178 patients via simulation review audit (SR-AWF) using a validated instrument. Scores were presented, and case management was discussed with individualized educational feedback. RO behavior and performance was compared over the first year of the program.

RESULTS

Comparing the first and second 6 months, there was a significant improvement in mean behavior (12.7-13.6 of 14, p = 0.0005) and RO performance (7.6-7.9 of 8, p = 0.018) scores. Protocol/study adherence significantly improved from 90.3% to 96.6% (p = 0.005). A total of 50 actions were generated, including the identification of learning needs to direct CME tutorials, the systematic change of suboptimal RO practice, and the alteration of deficient management of 3% of patients audited during the program.

CONCLUSION

An integrated CME/QI program combining C-AWF, SR-AWF, QI reminders, and targeted CME tutorials effectively improved targeted RO behavior and performance over a 12-month period. There was a corresponding increase in departmental protocol and study adherence.