Precision Metrics: A Narrative Review on Unlocking the Power of KPIs in Radiology for Enhanced Precision Medicine

(Background) Over the years, there has been increasing interest in adopting a quality approach in radiology, leading to the strategic pursuit of specific and key performance indicators (KPIs). These indicators in radiology can have significant impacts ranging from radiation protection to integration into digital healthcare. (Purpose) This study aimed to conduct a narrative review on the integration of key performance indicators (KPIs) in radiology with specific key questions. (Methods) This review utilized a standardized checklist for narrative reviews, including the ANDJ Narrative Checklist, to ensure thoroughness and consistency. Searches were performed on PubMed, Scopus, and Google Scholar using a combination of keywords related to radiology and KPIs, with Boolean logic to refine results. From an initial yield of 211 studies, 127 were excluded due to a lack of focus on KPIs. The remaining 84 studies were assessed for clarity, design, and methodology, with 26 ultimately selected for detailed review. The evaluation process involved multiple assessors to minimize bias and ensure a rigorous analysis. (Results and Discussion) This overview highlights the following: KPIs are crucial for advancing radiology by supporting the evolution of imaging technologies (e.g., CT, MRI) and integrating emerging technologies like AI and AR/VR. They ensure high standards in diagnostic accuracy, image quality, and operational efficiency, enhancing diagnostic capabilities and streamlining workflows. KPIs are vital for radiological safety, measuring adherence to protocols that minimize radiation exposure and protect patients. The effective integration of KPIs into healthcare systems requires systematic development, validation, and standardization, supported by national and international initiatives. Addressing challenges like CAD-CAM technology and home-based radiology is essential. Developing specialized KPIs for new technologies will be key to continuous improvement in patient care and radiological practices. (Conclusions) In conclusion, KPIs are essential for advancing radiology, while future research should focus on improving data access and developing specialized KPIs to address emerging challenges. Future research should focus on expanding documentation sources, improving web search methods, and establishing direct connections with scientific associations.

Developing, Purchasing, Implementing and Monitoring AI Tools in Radiology: Practical Considerations. A Multi-Society Statement From the ACR, CAR, ESR, RANZCR & RSNA

Artificial intelligence (AI) carries the potential for unprecedented disruption in radiology, with possible positive and negative consequences. The integration of AI in radiology holds the potential to revolutionize healthcare practices by advancing diagnosis, quantification, and management of multiple medical conditions. Nevertheless, the ever-growing availability of AI tools in radiology highlights an increasing need to critically evaluate claims for its utility and to differentiate safe product offerings from potentially harmful, or fundamentally unhelpful ones. This multi-society paper, presenting the views of Radiology Societies in the USA, Canada, Europe, Australia, and New Zealand, defines the potential practical problems and ethical issues surrounding the incorporation of AI into radiological practice. In addition to delineating the main points of concern that developers, regulators, and purchasers of AI tools should consider prior to their introduction into clinical practice, this statement also suggests methods to monitor their stability and safety in clinical use, and their suitability for possible autonomous function. This statement is intended to serve as a useful summary of the practical issues which should be considered by all parties involved in the development of radiology AI resources, and their implementation as clinical tools. KEY POINTS.

Developing, Purchasing, Implementing and Monitoring AI Tools in Radiology: Practical Considerations. A Multi-Society Statement From the ACR, CAR, ESR, RANZCR & RSNA

Artificial Intelligence (AI) carries the potential for unprecedented disruption in radiology, with possible positive and negative consequences. The integration of AI in radiology holds the potential to revolutionize healthcare practices by advancing diagnosis, quantification, and management of multiple medical conditions. Nevertheless, the ever‑growing availability of AI tools in radiology highlights an increasing need to critically evaluate claims for its utility and to differentiate safe product offerings from potentially harmful, or fundamentally unhelpful ones. This multi‑society paper, presenting the views of Radiology Societies in the USA, Canada, Europe, Australia, and New Zealand, defines the potential practical problems and ethical issues surrounding the incorporation of AI into radiological practice. In addition to delineating the main points of concern that developers, regulators, and purchasers of AI tools should consider prior to their introduction into clinical practice, this statement also suggests methods to monitor their stability and safety in clinical use, and their suitability for possible autonomous function. This statement is intended to serve as a useful summary of the practical issues which should be considered by all parties involved in the development of radiology AI resources, and their implementation as clinical tools.

MRI log file analysis for workflow improvement

Magnetic Resonance Imaging (MRI) is an important diagnostic scanning tool for the detection and monitoring of specific diseases and conditions. However, the equipment cost, maintenance and specialty training of the technologists make the examination expensive. Consequently, unnecessary scanner time caused by poor scheduling, repeated sequences, aborted sequences, scanner idleness, or capture of non-diagnostic or low-value sequences is an opportunity to reduce costs and increase efficiency. This paper analyzes data collected from log files on 29 scanners over several years. 'Wasted' time is defined and key performance indicators (KPIs) are identified. A decrease in exam duration results when actively modifying and monitoring the number of sequences that comprise the exam card for a protocol.

Developing, purchasing, implementing and monitoring AI tools in radiology: Practical considerations. A multi-society statement from the ACR, CAR, ESR, RANZCR & RSNA

Artificial Intelligence (AI) carries the potential for unprecedented disruption in radiology, with possible positive and negative consequences. The integration of AI in radiology holds the potential to revolutionize healthcare practices by advancing diagnosis, quantification, and management of multiple medical conditions. Nevertheless, the ever-growing availability of AI tools in radiology highlights an increasing need to critically evaluate claims for its utility and to differentiate safe product offerings from potentially harmful, or fundamentally unhelpful ones. This multi-society paper, presenting the views of Radiology Societies in the USA, Canada, Europe, Australia, and New Zealand, defines the potential practical problems and ethical issues surrounding the incorporation of AI into radiological practice. In addition to delineating the main points of concern that developers, regulators, and purchasers of AI tools should consider prior to their introduction into clinical practice, this statement also suggests methods to monitor their stability and safety in clinical use, and their suitability for possible autonomous function. This statement is intended to serve as a useful summary of the practical issues which should be considered by all parties involved in the development of radiology AI resources, and their implementation as clinical tools.

Developing, purchasing, implementing and monitoring AI tools in radiology: practical considerations. A multi-society statement from the ACR, CAR, ESR, RANZCR & RSNA

Artificial Intelligence (AI) carries the potential for unprecedented disruption in radiology, with possible positive and negative consequences. The integration of AI in radiology holds the potential to revolutionize healthcare practices by advancing diagnosis, quantification, and management of multiple medical conditions. Nevertheless, the ever-growing availability of AI tools in radiology highlights an increasing need to critically evaluate claims for its utility and to differentiate safe product offerings from potentially harmful, or fundamentally unhelpful ones.This multi-society paper, presenting the views of Radiology Societies in the USA, Canada, Europe, Australia, and New Zealand, defines the potential practical problems and ethical issues surrounding the incorporation of AI into radiological practice. In addition to delineating the main points of concern that developers, regulators, and purchasers of AI tools should consider prior to their introduction into clinical practice, this statement also suggests methods to monitor their stability and safety in clinical use, and their suitability for possible autonomous function. This statement is intended to serve as a useful summary of the practical issues which should be considered by all parties involved in the development of radiology AI resources, and their implementation as clinical tools.Key points • The incorporation of artificial intelligence (AI) in radiological practice demands increased monitoring of its utility and safety.• Cooperation between developers, clinicians, and regulators will allow all involved to address ethical issues and monitor AI performance.• AI can fulfil its promise to advance patient well-being if all steps from development to integration in healthcare are rigorously evaluated.

Developing, Purchasing, Implementing and Monitoring AI Tools in Radiology: Practical Considerations. A Multi-Society Statement from the ACR, CAR, ESR, RANZCR and RSNA

Artificial Intelligence (AI) carries the potential for unprecedented disruption in radiology, with possible positive and negative consequences. The integration of AI in radiology holds the potential to revolutionize healthcare practices by advancing diagnosis, quantification, and management of multiple medical conditions. Nevertheless, the ever-growing availability of AI tools in radiology highlights an increasing need to critically evaluate claims for its utility and to differentiate safe product offerings from potentially harmful, or fundamentally unhelpful ones. This multi-society paper, presenting the views of Radiology Societies in the USA, Canada, Europe, Australia, and New Zealand, defines the potential practical problems and ethical issues surrounding the incorporation of AI into radiological practice. In addition to delineating the main points of concern that developers, regulators, and purchasers of AI tools should consider prior to their introduction into clinical practice, this statement also suggests methods to monitor their stability and safety in clinical use, and their suitability for possible autonomous function. This statement is intended to serve as a useful summary of the practical issues which should be considered by all parties involved in the development of radiology AI resources, and their implementation as clinical tools. This article is simultaneously published in Insights into Imaging (DOI 10.1186/s13244-023-01541-3), Journal of Medical Imaging and Radiation Oncology (DOI 10.1111/1754-9485.13612), Canadian Association of Radiologists Journal (DOI 10.1177/08465371231222229), Journal of the American College of Radiology (DOI 10.1016/j.jacr.2023.12.005), and Radiology: Artificial Intelligence (DOI 10.1148/ryai.230513). Keywords: Artificial Intelligence, Radiology, Automation, Machine Learning Published under a CC BY 4.0 license. ©The Author(s) 2024. Editor's Note: The RSNA Board of Directors has endorsed this article. It has not undergone review or editing by this journal.

Development of Quality Indicators for the Ultrasound Department through a Modified Delphi Method

This study aims to establish precise quality indicators for evaluating and enhancing ultrasound performance, employing a methodology based on a comprehensive review of the literature, expert insights, and practical application experiences. We conducted a thorough review of both the domestic and international literature on ultrasound quality control to identify potential indicators. A dedicated team was formed to oversee the complete indicator development process. Utilizing a three-round modified Delphi method, we sought expert opinions through personalized email correspondence. Subsequently, data from diverse hospital indicators were collected to validate and assess feasibility. A novel set of seven indicators was compiled initially, followed by the convening of a 36-member nationally representative expert panel. After three rounds of meticulous revisions, consensus was reached on 13 indicators across three domains. These finalized indicators underwent application in various hospital settings, demonstrating their initial validity and feasibility. The development of thirteen ultrasound quality indicators represents a significant milestone in evaluating ultrasound performance. These indicators empower hospitals to monitor changes in quality effectively, fostering efficient quality management practices.

Convolutional Neural Network Model for Segmentation and Classification of Clear Cell Renal Cell Carcinoma Based on Multiphase CT Images

(1) Background: Computed tomography (CT) imaging challenges in diagnosing renal cell carcinoma (RCC) include distinguishing malignant from benign tissues and determining the likely subtype. The goal is to show the algorithm's ability to improve renal cell carcinoma identification and treatment, improving patient outcomes. (2) Methods: This study uses the European Deep-Health toolkit's Convolutional Neural Network with ECVL, (European Computer Vision Library), and EDDL, (European Distributed Deep Learning Library). Image segmentation utilized U-net architecture and classification with resnet101. The model's clinical efficiency was assessed utilizing kidney, tumor, Dice score, and renal cell carcinoma categorization quality. (3) Results: The raw dataset contains 457 healthy right kidneys, 456 healthy left kidneys, 76 pathological right kidneys, and 84 pathological left kidneys. Preparing raw data for analysis was crucial to algorithm implementation. Kidney segmentation performance was 0.84, and tumor segmentation mean Dice score was 0.675 for the suggested model. Renal cell carcinoma classification was 0.885 accurate. (4) Conclusion and key findings: The present study focused on analyzing data from both healthy patients and diseased renal patients, with a particular emphasis on data processing. The method achieved a kidney segmentation accuracy of 0.84 and mean Dice scores of 0.675 for tumor segmentation. The system performed well in classifying renal cell carcinoma, achieving an accuracy of 0.885, results which indicates that the technique has the potential to improve the diagnosis of kidney pathology.

A National Quality Improvement Program on Ultrasound Department in China: A Controlled Cohort Study of 1297 Public Hospitals

Providing high-quality medical services is of great importance in the imaging department, as there is a growing focus on personal health, and high-quality services can lead to improved patient outcomes. Many quality improvement (QI) programs with good guidance and fine measurement for improvement have been reported to be effective. In order to improve the quality of ultrasound departments in China, we conducted this study of a national quality improvement program. A total of 1297 public hospitals were included in this QI program on ultrasound departments in China from 2017 to 2019. The effect of this QI program was investigated, and potential factors, including hospital level and local economic development, were considered. The outcome indicators, the positive rate and diagnostic accuracy, were improved significantly between the two phases (positive rate, 2017 vs. 2019: 66.21% vs. 73.91%, p < 0.001; diagnostic accuracy, 2017 vs. 2019: 85.37% vs. 89.74%; p < 0.001). Additionally, they were improved in secondary and tertiary hospitals, with the improvement in secondary hospitals being greater. Notably, the enhancement of diagnostic accuracy in low-GDP provinces was almost 20%, which was more significant than the enhancement in high-GDP provinces. However, the important structural indicator, the doctor-to-patient ratio, decreased from 1.05:10,000 to 0.96:10,000 (p = 0.026). This study suggests that the national ultrasound QI program improved the outcome indicators, with secondary-level hospitals improving more than tertiary hospitals and low-GDP provinces improving more than high-GDP regions. Additionally, as there is a growing need for ultrasound examinations, more ultrasound doctors are needed in China.

Value-Based Healthcare and Radiology: How can Value be Measured?

The concept of value-based healthcare (VBH) emerges as a response to traditional models of healthcare system management. More specifically, in radiology, the transition from volume to value has been discussed by its main associations, having as the main concern regarding the role of the specialty in a more integrated healthcare context.

Through a qualitative study, this work aims to analyse and evaluate how this new concept can be implemented in radiology by identifying obstacles and mapping the technical and procedural improvements necessary for its correct implementation in the national context of healthcare provision. Through interviews with different elements of the healthcare sector (from doctors to industry partners and researchers), it was possible to draw a set of metrics for measuring the value of radiology, alongside the implementation of a VBH strategy.

As the main conclusion, the implementation of a strategic agenda for the creation of value in radiology at the national level should be based on the reduction of variability and the identification of best practices in terms of adequacy, quality, safety and efficiency, aiming to satisfy the needs of requesting doctors and patients.

An introduction to key performance indicators for medical physicists

Qualified medical physicists (QMPs) are in a unique position to influence the creation and application of key performance indicators (KPIs) across diverse practices in health care. Developing KPIs requires the involvement of stakeholders in the area of interest. Fundamentally, KPIs should provide actionable information for the stakeholders using or viewing them. During development, it is important to strongly consider the underlying data collection for the KPI, making it automatic whenever possible. Once the KPI has been validated, it is important to setup a review cycle and be prepared to adjust the underlying data or action levels if the KPI is not performing as intended. Examples of specific KPIs for QMPs of common scopes of practice are provided to act as models to aid in implementation. KPIs are a useful tool for QMPs, regardless of the scope of practice or practice environment, to enhance the safety and quality of care being delivered.

Key Performance Indicators for Quality Imaging Practice: Why, What, and How

A common trend across health care organizations is the development of key performance indicators (KPIs) for characterizing quality, identifying areas in need of change, and quantifying the impact of change. This article outlines a list of KPIs that can be used to quantify, target, and optimize value and value delivery in medical imaging practice. Of particular focus here is the aspect of practice that should be overseen and informed by the work of medical physicists, along the trajectory and expectations of a Medical Physics 3.0 model. The authors offer a framework for developing site-specific KPIs and several demonstrative clinical examples.

UTERINE ARTERY EMBOLISATION: CONTINUOUS QUALITY IMPROVEMENT REDUCES RADIATION DOSE WHILE MAINTAINING IMAGE QUALITY

The purpose of this study was to introduce a continuous quality improvement (CQI) program for radiation dose optimisation during uterine artery embolisation (UAE) and assess its impact on dose reduction and image quality. The CQI program investigated the effects of optimising radiation dose parameters on the kerma-area product (KAP) and image quality when comparing a 'CQI intervention' group (n = 50) and 'Control' group (n = 50). Visual grading characteristics (VGC) analysis was used to assess image quality, using the 'Control' group as a reference. A significant reduction in KAP by 17% (P = 0.041, d = 0.2) and reference air kerma (Ka, r) by 20% (P = 0.027, d = 0.2) was shown between the two groups. The VGC analysis resulted in an area under the VGC curve (AUCVGC) of 0.54, indicating no significant difference in image quality between the two groups (P = 0.670). The implementation of the CQI program and optimisation of radiation dose parameters improved the UAE radiation dose practices at our centre. The dose reduction demonstrated no detrimental effects on image quality.

Availability of a final abdominopelvic CT report before emergency department disposition: risk-adjusted outcomes in patients with abdominal pain

OBJECTIVE

To determine whether availability of a final radiologist report versus an experienced senior resident preliminary report prior to disposition affects major care outcomes in emergency department (ED) patient presenting with abdominal pain undergoing abdominopelvic CT.

MATERIALS AND METHODS

This single-institution, IRB-approved, HIPAA-compliant retrospective cohort study included 5019 ED patients with abdominal pain undergoing abdominopelvic CT from October 2015 to April 2019. Patients were categorized as being dispositioned after either an experienced senior resident preliminary report (i.e., overnight model) or the final attending radiologist interpretation (i.e., daytime model) of the CT was available. Multivariable regression models were built accounting for demographic data, clinical factors (vital signs, ED triage score, laboratory data), and disposition timing to analyze the impact on four important patient outcomes: inpatient admission (primary outcome), readmission (within 30 days), second operation within 30 days, and death.

RESULTS

In the setting of an available experienced senior resident preliminary report, timing of the final radiologist report (before vs. after disposition) was not a significant multivariable predictor of inpatient admission (p = 0.63), readmission within 30 days (p = 0.66), second operation within 30 days (p = 0.09), or death (p = 0.63). Unadjusted event rates for overnight vs daytime reports, respectively, were 37.2% vs. 38.0% (inpatient admission), 15.9% vs. 16.5% (30-day readmission), 0.65% vs. 0.3% (second operation within 30 days), and 0.85% vs. 1.3% (death).

CONCLUSION

Given the presence of an experienced senior resident preliminary report, availability of a final radiology report prior to ED disposition did not affect four major clinical care outcomes of patients with abdominal pain undergoing abdominopelvic CT.

An International Survey of Quality and Safety Programs in Radiology

PURPOSE

The aim of this study was to determine the status of radiology quality improvement programs in a variety of selected nations worldwide.

METHODS

A survey was developed by select members of the International Economics Committee of the American College of Radiology on quality programs and was distributed to committee members. Members responded on behalf of their country. The 51-question survey asked about 12 different quality initiatives which were grouped into 4 themes: departments, users, equipment, and outcomes. Respondents reported whether a designated type of quality initiative was used in their country and answered subsequent questions further characterizing it.

RESULTS

The response rate was 100% and represented Australia, Canada, China, England, France, Germany, India, Israel, Japan, the Netherlands, Russia, and the United States. The most frequently reported quality initiatives were imaging appropriateness (91.7%) and disease registries (91.7%), followed by key performance indicators (83.3%) and morbidity and mortality rounds (83.3%). Peer review, equipment accreditation, radiation dose monitoring, and structured reporting were reported by 75.0% of respondents, followed by 58.3% of respondents for quality audits and critical incident reporting. The least frequently reported initiatives included Lean/Kaizen exercises and physician performance assessments, implemented by 25.0% of respondents.

CONCLUSION

There is considerable diversity in the quality programs used throughout the world, despite some influence by national and international organizations, from whom further guidance could increase uniformity and optimize patient care in radiology.

Promises of artificial intelligence in neuroradiology: a systematic technographic review

PURPOSE

To conduct a systematic review of the possibilities of artificial intelligence (AI) in neuroradiology by performing an objective, systematic assessment of available applications. To analyse the potential impacts of AI applications on the work of neuroradiologists.

METHODS

We identified AI applications offered on the market during the period 2017-2019. We systematically collected and structured information in a relational database and coded for the characteristics of the applications, their functionalities for the radiology workflow and their potential impacts in terms of 'supporting', 'extending' and 'replacing' radiology tasks.

RESULTS

We identified 37 AI applications in the domain of neuroradiology from 27 vendors, together offering 111 functionalities. The majority of functionalities 'support' radiologists, especially for the detection and interpretation of image findings. The second-largest group of functionalities 'extends' the possibilities of radiologists by providing quantitative information about pathological findings. A small but noticeable portion of functionalities seek to 'replace' certain radiology tasks.

CONCLUSION

Artificial intelligence in neuroradiology is not only in the stage of development and testing but also available for clinical practice. The majority of functionalities support radiologists or extend their tasks. None of the applications can replace the entire radiology profession, but a few applications can do so for a limited set of tasks. Scientific validation of the AI products is more limited than the regulatory approval.

Identification of quality improvement areas in pediatric MRI from analysis of patient safety reports

BACKGROUND

Analysis of safety reports has been utilized to guide practice improvement efforts in adult magnetic resonance imaging (MRI). Data specific to pediatric MRI could help target areas of improvement in this population.

OBJECTIVE

To estimate the incidence of safety reports in pediatric MRI and to determine associated risk factors.

MATERIALS AND METHODS

In a retrospective HIPAA-compliant, institutional review board-approved study, a single-institution Radiology Information System was queried to identify MRI studies performed in pediatric patients (0-18 years old) from 1/1/2010 to 12/31/2015. The safety report database was queried for events matching the same demographic and dates. Data on patient age, gender, location (inpatient, outpatient, emergency room [ER]), and the use of sedation/general anesthesia were recorded. Safety reports were grouped into categories based on the cause and their severity. Descriptive statistics were used to summarize continuous variables. Chi-square analyses were performed for univariate determination of statistical significance of variables associated with safety report rates. A multivariate logistic regression was used to control for possible confounding effects.

RESULTS

A total of 16,749 pediatric MRI studies and 88 safety reports were analyzed, yielding a rate of 0.52%. There were significant differences in the rate of safety reports between patients younger than 6 years (0.89%) and those older (0.41%) (P<0.01), sedated (0.8%) and awake children (0.45%) (P<0.01), and inpatients (1.1%) and outpatients (0.4%) (P<0.01). The use of sedation/general anesthesia is an independent risk factor for a safety report (P=0.02). The most common causes for safety reports were service coordination (34%), drug reactions (19%), and diagnostic test and ordering errors (11%).

CONCLUSION

The overall rate of safety reports in pediatric MRI is 0.52%. Interventions should focus on vulnerable populations, such as younger patients, those requiring sedation, and those in need of acute medical attention.

A Comprehensive Approach Towards Quality and Safety in Diagnostic Imaging Services: Our Experience at a Rural Tertiary Health Care Center

INTRODUCTION

An organization's transformation from imple-mentation of small, distinct Quality Improvement (QI) efforts to complete incorporation of Quality Improvement Program (QIP) into its culture occurs through a process of churning the foundational elements over time.

AIM

To develop a quality culture across the employees, identify measurable indicators and various tools to impart effective quality care and develop a learning culture for continuous quality improvement in the field of imaging services.

MATERIALS AND METHODS

To establish a QIP, the bare minimum requirement started with forming a quality committee. The committee identified the areas of improvement and ascertaining the core principle of Quality Management System (QMS) by having a Quality Manual, Standard Operating Procedures (SOP's), work-instructions, identification and monitoring of quality indicators and a training calendar. Appropriate tools like formatted daily registers, periodic check lists, run charts etc., were developed to collect the data followed by multiple PDSA cycles (Plan, Do, Study and Act) which helped identify the process bottlenecks, followed by implementing solutions and reanalysis.

RESULTS

A total of 17 measurable key performance indicators were identified from the four major quality tasks namely Safety, Process Improvement, Professional Outcome and Satisfaction, to assess the performance measures and targets of QIP.

CONCLUSION

Diagnostic services should evaluate how to choose the most appropriate method and develop a comprehensive QIP to meet the needs of the staff and the end users, thus, creating a working environment, where people constitutes the intrinsic value in attaining the ultimate quality and safety.

Key Performance Indicators in the Evaluation of the Quality of Radiation Safety Programs

Beaumont is a multiple hospital health care system with a centralized radiation safety department. The health system operates under a broad scope Nuclear Regulatory Commission license but also maintains several other limited use NRC licenses in off-site facilities and clinics. The hospital-based program is expansive including diagnostic radiology and nuclear medicine (molecular imaging), interventional radiology, a comprehensive cardiovascular program, multiple forms of radiation therapy (low dose rate brachytherapy, high dose rate brachytherapy, external beam radiotherapy, and gamma knife), and the Research Institute (including basic bench top, human and animal). Each year, in the annual report, data is analyzed and then tracked and trended. While any summary report will, by nature, include items such as the number of pieces of equipment, inspections performed, staff monitored and educated and other similar parameters, not all include an objective review of the quality and effectiveness of the program. Through objective numerical data Beaumont adopted seven key performance indicators. The assertion made is that key performance indicators can be used to establish benchmarks for evaluation and comparison of the effectiveness and quality of radiation safety programs. Based on over a decade of data collection, and adoption of key performance indicators, this paper demonstrates one way to establish objective benchmarking for radiation safety programs in the health care environment.