Measuring and Managing Radiologist Productivity, Part 2: Beyond the Clinical Numbers

Mandating Limits on Workload, Duty, and Speed in Radiology

Research has not yet quantified the effects of workload or duty hours on the accuracy of radiologists. With the exception of a brief reduction in imaging studies during the 2020 peak of the COVID-19 pandemic, the workload of radiologists in the United States has seen relentless growth in recent years. One concern is that this increased demand could lead to reduced accuracy. Behavioral studies in species ranging from insects to humans have shown that decision speed is inversely correlated to decision accuracy. A potential solution is to institute workload and duty limits to optimize radiologist performance and patient safety. The concern, however, is that any prescribed mandated limits would be arbitrary and thus no more advantageous than allowing radiologists to self-regulate. Specific studies have been proposed to determine whether limits reduce error, and if so, to provide a principled basis for such limits. This could determine the precise susceptibility of individual radiologists to medical error as a function of speed during image viewing, the maximum number of studies that could be read during a work shift, and the appropriate shift duration as a function of time of day. Before principled recommendations for restrictions are made, however, it is important to understand how radiologists function both optimally and at the margins of adequate performance. This study examines the relationship between interpretation speed and error rates in radiology, the potential influence of artificial intelligence on reading speed and error rates, and the possible outcomes of imposed limits on both caseload and duty hours. This review concludes that the scientific evidence needed to make meaningful rules is lacking and notes that regulating workloads without scientific principles can be more harmful than not regulating at all.

A Metrics-Based Research Salary Award System and Its 9-Year Impact on Publication Productivity

RATIONALE AND OBJECTIVES

Although metrics-based systems may incentivize academic output, no prior studies have evaluated the impact on publication metrics in academic radiology. This study presents a metrics-based system of awarding research protected time, and retrospectively evaluates its 9-year impact on publication productivity and impact factor.

MATERIALS AND METHODS

Based on a metrics-based algorithm to award department funded Research Protected Time (RPT), metrics pre-RPT (2003-2009) and during the RPT period (2010-2018) from an academic radiology department were retrospectively analyzed to test the hypothesis that the RPT program resulted in higher publication productivity and journal impact factor at the departmental level and for faculty members receiving the award. Comparison was made between (1) pre-RPT and RPT periods and (2) during the RPT period, between RPT and non-RPT faculty members, for annual publication productivity normalized to faculty count (Student's t test) and median impact factor (Wilcoxon rank sum test).

RESULTS

For the evaluation period of 2003-2018, 724 unique publications were identified: 15% (107/724) pre-RPT period and 85% (617/724) RPT period. Normalized annual publication productivity was higher during the RPT period compared to the Pre-RPT period (1.2 vs. 0.3, p = 0.002), and within the RPT period, higher among faculty who received RPT vs. non-RPT faculty (3.5 vs. 0.4, p = 0.002). Median impact factor was higher during the RPT period compared to pre-RPT period (2.843 vs. 2.322, p = 0.044), and within the RPT period, higher in RPT vs. non-RPT faculty (3.016 vs. 2.346, p < 0.001).

CONCLUSION

The implementation of a metrics-based system of funded, research protected time, was associated with increased publication productivity and increased impact factor.

Clinician-Scientists: Can They Survive in the Modern Era?

Clinician-scientists are commonly characterized as health care professionals who are proficient in both research and clinical practice. Their dual expertise positions them to play a vital role in translating research outcomes to clinical practice. However, economic changes in the past few decades are threatening their very survival. The purposes of this article are to review some of the economic forces that pose the greatest risks to clinician-scientists in the modern era and to glean lessons from the business world in overcoming these challenges. Health care consolidation and decreasing reimbursements are putting increasing financial pressure on academic institutions, leaving them more inclined to cut back on departmental research support. Innovative companies commit a certain percentage of their revenue to research and discovery. Academic institutions should similarly view their research budget as research and discovery that will sustain the future growth of radiology. They should quantify and define expectations for academic productivity, focus on return on investment, and bolster the infrastructure to foster commercial partnerships that can provide additional revenue to support the research mission. Success in academics does not occur by accident. It requires more than individual talent and hard work. It also requires institutional leaders who are committed to developing future academic leaders and supporting innovation.

Productivity evaluation of radiologists interpreting computed tomography scans using statistical process control charts

Radiology service managers search for efficient ways to monitor productivity and improve capacity. One way to assess radiologists' productivity is by measuring their time to complete reports. Radiology reporting times (RRTs) may be monitored using statistical tools, such as process control charts (CCs). This study was carried out in the radiology sector of a University-based general hospital with 850 inward beds. Productivity was monitored using CCs. The selected control variable was RRTs, and process capability was calculated using C_p and C_pk indices. Only chest computed tomography scans were analyzed, totaling 2862 exams over a 6-month period. Our objective was to develop a simple tool to monitor radiologist performance, as given by RRT, over time. For that, we constructed CCs using data from 10 radiologists to monitor the stability of their RRTs. Only 3 radiologists presented mean times below the group average; 6 displayed a trend in RRTs that characterized performance improvement, while 4 displayed the opposite trend. Capability measures for the group indicated a process that is not capable. We demonstrate that CCs may be a useful tool for monitoring radiologists' performances in CT scans interpretation. Results demonstrated that in the individual CT reporting process, common cause variability is the type of variability most frequently observed, being most likely related to natural variations in features of the images analyzed. Lastly, CCs may also assist in decision making in the sector, such as establishing minimum productivity goals based on historical performance.

Leveraging the electronic health record to evaluate the validity of the current RVU system for radiologists

BACKGROUND

Relative value units (RVUs) are utilized to evaluate physician productivity in many fields, including radiology. The goal of this paper is to use the electronic medical record (EMR) to evaluate whether the current RVU system allows for fair comparison between radiologists' time effort.

MATERIALS AND METHODS

The study was approved by the local Institutional Review Board (IRB). Over 600,000 radiology studies with unique current procedural terminology (CPT) codes were evaluated, and after exclusion of studies interpreted in conjunction with trainees or interpreted using other software systems, a total of 241,627 studies remained. The median 25%ile, 50%ile, 65%ile, 75%ile and modal study ascribable times (SATs) for each CPT code was calculated across all radiologists. To evaluate the potential bias incurred using the current RVU system, the number of days required to achieve the Association of Administrators in Academic Radiology AAARAD 65%ile were calculated.

RESULTS

RVU values were positively correlated with SATs (r = 0.69-0.71, p < 0.001). The variability in the radiologists' time to achieve the AAARAD 65%ile benchmark was highest for musculoskeletal imaging, and lowest for thoracic imaging. The discrepancy in the number of days of work required to achieve the AAARAD 65%ile benchmark was 141.1% (197.7 days) for musculoskeletal imaging, 107.5% (161.9 days) for neuroimaging, 89.6% (185.9 days) for body imaging, and 72.2% (84.0 days) for thoracic imaging.

CONCLUSION

The current RVU system is not strongly correlated with radiologist effort measured by radiologists' time. A time-based metric is more representative of radiologist work. However, there is no perfect method to measure radiologists' work.

Emerging Challenges and Opportunities in the Evolution of Teleradiology

OBJECTIVE. In recent decades, teleradiology has expanded considerably, and many radiology practices now engage in intraorganizational or extraorganizational teleradiology. In this era of patient primacy, optimizing patient care and care delivery is paramount. This article provides an update on recent changes, current challenges, and future opportunities centered around the ability of teleradiology to improve temporal and geographic imaging access. We review licensing and regulations and discuss teleradiology in providing services to rural areas and assisting with disaster response, including the response to the coronavirus disease (COVID-19) pandemic. CONCLUSION. Teleradiology can help increase imaging efficiency and mitigate both geographic and temporal discrepancies in imaging care. Technologic limitations and regulatory hurdles hinder the optimal practice of teleradiology, and future attention to these issues may help ensure broader patient access to high-quality imaging across the United States.

Estimating the productivity of radiologists in Brazil: the search for a benchmark

Objective

To determine the average productivity of radiologists, as measured by number of reports issued per 6-h shift, evaluating variables that could affect the results.

Materials and Methods

This was a study utilizing an online questionnaire sent to radiologists affiliated with the Brazilian College of Radiology and Diagnostic Imaging. The questions were related to the demographic profile and professional practice characteristics (form of remuneration, primary imaging method employed, and subspecialty) of the radiologists, as well as their individual productivity (average personal productivity) and the productivity considered reasonable in a 6-h shift. The association between productivity and the practice characteristics of the radiologists was determined by using Poisson regression to calculate the prevalence ratio.

Results

A total of 510 radiologists completed the questionnaire. The great majority of the respondents (84%) reported that their remuneration is directly related to their productivity. The productivity varied according to the subspecialty, work environment, and remuneration model.

Conclusion

We demonstrated that the productivity of radiologists is associated with the characteristics of their employment. We hope that this study will encourage other studies aimed at evaluating the productive capacity of the radiologists in Brazil, addressing the various functions they perform in their daily routine, including activities other than issuing reports.

Characteristics of Radiologists' Clinical Practice Patterns by Career Stage

PURPOSE

To assess characteristics of radiologists' clinical practice patterns by career stage.

METHODS

Radiologists' 2016 billed services were extracted from the Medicare Physician and Other Supplier Public Use File. Billed clinical work was weighted using work relative value units. Medical school graduation years were obtained from Medicare Physician Compare. Practice patterns were summarized by decades after residency.

RESULTS

Among 28,463 included radiologists, 32.7% were ≤10 years postresidency, 29.3% 11-20 years, 25.0% 21-30 years, 10.5% 31-40 years, 2.4% 41-50 years, 0.1% ≥51 years. Billed clinical work (normalized to a mean of 1.00 among all radiologists) ranged 0.92-1.07 from 1 to 40 years, decreasing to 0.64 for 41-50 years and 0.43 for ≥51 years. Computed tomography represented 34.7%-38.6% of billed clinical work from 1 to 30 years, decreasing slightly to 31.5% for 31-40 years. Magnetic resonance imaging represented 13.9%-14.3% from 1 to 30 years, decreasing slightly to 11.2% for 31-40 years. Ultrasonography represented 6.2%-11.6% across career stages. Nuclear medicine increased steadily from 1.7% for ≤10 years to 7.0% for 41-50 years. Mammography represented 9.9%-12.9% from 1 to 50 years. Radiography/fluoroscopy represented 15.1%-29.8% from 1 to 50 years, but 65.9% for ≥51 years.

CONCLUSION

The national radiologist workforce declines abruptly by more than half approximately 30 years after residency. Radiologists still working at 31-40 years, however, contribute similar billed clinical work, both overall and across modalities, as earlier career radiologists. Strategies to retain later-career radiologists in the workforce could help the specialty meet growing clinical demands, mitigate burnout in earlier career colleagues, and expand robust patient access to both basic and advanced imaging services.

In comparison with other abdominal imaging modalities, which radiologists interpret abdominal MRI?

PURPOSE

To assess subspecialty mix and case volumes of general and abdominal subspecialty radiologists interpreting abdominal MRI.

METHODS

The 2016 CMS Physician/Supplier Procedure Summary Master File was used to obtain billed counts of radiologist-interpreted abdominal fluoroscopy, US, CT, and MRI examinations. The CMS Physician and Other Supplier Public Use File was used to assess the subspecialty mix and case volume of the radiologists interpreting those examinations.

RESULTS

The fraction of all abdominal imaging examinations interpreted by generalists and abdominal subspecialty radiologists was 70.7% and 16.5% for fluoroscopy; 68.7% and 21.0% for US; 71.4% and 19.2% for CT; and 41.9% and 52.5% for MRI. In 2016, the fraction of general and abdominal radiologists interpreting > 50 fluoroscopy examinations on Medicare fee-for-service beneficiaries was 15.1% and 16.2%. For > 50 US examinations, the fraction was 61.5% and 60.5%; for > 50 CT examinations, 91.2% and 79.6%; and for > 50 MRI examinations, 4.0% and 28.5%. The fraction of abdominal imaging examinations interpreted overall by low-volume providers (those interpreting ≤ 50 examinations in 2016) was 59.5% for fluoroscopy, 17.5% for US, 6.3% for CT, and 50.6% for MRI.

CONCLUSION

Nationally, most abdominal fluoroscopy, US, and CT examinations are interpreted by general radiologists, who have similar annual volumes of these examinations as abdominal subspecialty radiologists. In contrast, most abdominal MRI examinations are interpreted by abdominal subspecialty radiologists, who attain considerably higher volumes. These findings have implications for workforce planning and abdominal imaging fellowship design to ensure their graduates are optimally prepared to contribute to their future practices.

Speed Versus Interpretation Accuracy: Current Thoughts and Literature Review

OBJECTIVE. Whether there is a precise relationship between reading speed and diagnostic accuracy has been an elusive and much debated issue. We discuss the literature and include practical considerations and relevant experience. CONCLUSION. To our knowledge, no credible relationship has been established between the speed of diagnostic image interpretation and accuracy. Furthermore, no nationally recognized guidelines address these factors, and it would be irresponsible to attribute widespread credibility to anecdotal studies. A variety of factors influence diagnostic accuracy, and length of interpretation time is not an established one.

Radiologists' Variation of Time to Read Across Different Procedure Types

The workload of US radiologists has increased over the past two decades as measured through total annual relative value units (RVUs). This increase in RVUs generated suggests that radiologists' productivity has increased. However, true productivity (output unit per input unit; RVU per time) is at large unknown since actual time required to interpret and report a case is rarely recorded. In this study, we analyzed how the time to read a case varies between radiologists over a set of different procedure types by retrospectively extracting reading times from PACS usage logs. Specifically, we tested two hypotheses that; i) relative variation in time to read per procedure type increases as the median time to read a procedure type increases, and ii) relative rankings in terms of median reading speed for individual radiologists are consistent across different procedure types. The results that, i) a correlation of -0.25 between the coefficient of variation and median time to read and ii) that only 12 out of 46 radiologists had consistent rankings in terms of time to read across different procedure types, show both hypotheses to be without support. The results show that workload distribution will not follow any general rule for a radiologist across all procedures or a general rule for a specific procedure across many readers. Rather the findings suggest that improved overall practice efficiency can be achieved only by taking into account radiologists' individual productivity per procedure type when distributing unread cases.

Inpatient Complexity in Radiology-a Practical Application of the Case Mix Index Metric

With ongoing healthcare payment reforms in the USA, radiology is moving from its current state of a revenue generating department to a new reality of a cost-center. Under bundled payment methods, radiology does not get reimbursed for each and every inpatient procedure, but rather, the hospital gets reimbursed for the entire hospital stay under an applicable diagnosis-related group code. The hospital case mix index (CMI) metric, as defined by the Centers for Medicare and Medicaid Services, has a significant impact on how much hospitals get reimbursed for an inpatient stay. Oftentimes, patients with the highest disease acuity are treated in tertiary care radiology departments. Therefore, the average hospital CMI based on the entire inpatient population may not be adequate to determine department-level resource utilization, such as the number of technologists and nurses, as case length and staffing intensity gets quite high for sicker patients. In this study, we determine CMI for the overall radiology department in a tertiary care setting based on inpatients undergoing radiology procedures. Between April and September 2015, CMI for radiology was 1.93. With an average of 2.81, interventional neuroradiology had the highest CMI out of the ten radiology sections. CMI was consistently higher across seven of the radiology sections than the average hospital CMI of 1.81. Our results suggest that inpatients undergoing radiology procedures were on average more complex in this hospital setting during the time period considered. This finding is relevant for accurate calculation of labor analytics and other predictive resource utilization tools.

Metrix Matrix: A Cloud-Based System for Tracking Non-Relative Value Unit Value-Added Work Metrics

PURPOSE

In the era of value-based medicine, it will become increasingly important for radiologists to provide metrics that demonstrate their value beyond clinical productivity. In this article the authors describe their institution's development of an easy-to-use system for tracking value-added but non-relative value unit (RVU)-based activities.

METHODS

Metrix Matrix is an efficient cloud-based system for tracking value-added work. A password-protected home page contains links to web-based forms created using Google Forms, with collected data populating Google Sheets spreadsheets. Value-added work metrics selected for tracking included interdisciplinary conferences, hospital committee meetings, consulting on nonbilled outside studies, and practice-based quality improvement. Over a period of 4 months, value-added work data were collected for all clinical attending faculty members in a university-based radiology department (n = 39). Time required for data entry was analyzed for 2 faculty members over the same time period.

RESULTS

Thirty-nine faculty members (equivalent to 36.4 full-time equivalents) reported a total of 1,223.5 hours of value-added work time (VAWT). A formula was used to calculate "value-added RVUs" (vRVUs) from VAWT. VAWT amounted to 5,793.6 vRVUs or 6.0% of total work performed (vRVUs plus work RVUs [wRVUs]). Were vRVUs considered equivalent to wRVUs for staffing purposes, this would require an additional 2.3 full-time equivalents, on the basis of average wRVU calculations. Mean data entry time was 56.1 seconds per day per faculty member.

CONCLUSIONS

As health care reimbursement evolves with an emphasis on value-based medicine, it is imperative that radiologists demonstrate the value they add to patient care beyond wRVUs. This free and easy-to-use cloud-based system allows the efficient quantification of value-added work activities.

Quantifying the Impact of Noninterpretive Tasks on Radiology Report Turn-Around Times

PURPOSE

Traditional radiology productivity metrics do not account for noninterpretive tasks (NITs). This study aimed to systematically quantify NITs and their impact on report turn-around time (RTAT) during solo academic neuroradiology overnight coverage in the emergency department.

METHODS

Retrospective analysis of 1 week of data, including phone call quantity and duration, clinician identification badge access to the reading room ("badge swipes"), suspected acute strokes, imaging examination volume, and emergency department patient volume, was performed. Univariate analyses were employed to quantify NITs. Multivariate linear regression was used to determine if NITs within an hour are predictive of RTAT of studies completed within that hour.

RESULTS

Sixty-three hours of overnight neuroradiology coverage were analyzed. The mean number of phone calls per hour was 8.7 (SD: 5.7), and mean duration of phone calls per hour was 12 min (SD: 9.6 min, range 1-46). The mean number of badge swipes per hour was 2.1 (SD 1.6). The mean number of examinations (CT and MRI) performed per hour was 2.2 (SD: 1.7). Regression analyses found total duration of phone calls in an hour as the strongest independent predictor of RTAT (unstandardized β = 4.25, P < .001). The overall multivariate model was also significant (P < .001, R² = 0.596; adjusted R² = 0.578).

CONCLUSIONS

For every 1-min increase in total duration of calls in an hour, mean RTAT increased by 4.25 min. Standardizing capture of NITs may aid development of strategies that address productivity, communication, and value in radiology.

What is the relation between number of sessions worked and productivity of radiologists-a pilot study?

Increasing workloads and the current austerity measures are putting UK radiology departments under considerable stress. We need to look at the most efficient ways to manage radiology departments in order to cope with increasing demand. Consequently, a system is needed that can compare productivity between radiologists with different jobs. We measured workload in a UK radiology department and compared the productivities of consultants working different numbers of sessions, which are called programmed activities (PAs), to identify the optimal job plan structure for reporting productivity. Reporting data was gathered from electronic records for 14 consultants working different numbers of PA during the period April 2010-March 2011. These were converted into relative value unit (RVU) scores using a modified RCSI RVU system. Crude and net workloads were calculated for each consultant by dividing their total RVU score by the number of PAs they were contracted for and how many they spent reporting. The consultants reported 118,001 imaging studies. There was statistically significant variation in productivity between consultants working different numbers of PAs on χ (2) analysis (p < 0.05). Consultants working 12 PAs were more productive than consultants working 11 PAs, with net workloads of 7636 RVU/PA/year versus net 6146 RVU/PA/year, p < 0.05. Although UK consultants working 12 PAs per week are more productive than their colleagues, the reasons why are unclear. We have identified a method that can be developed further to identify efficient working practices in UK radiology departments. However, a UK-specific RVU system would make this productivity analysis more accurate.

Academic Medical Centers and Community Hospitals Integration: Trends and Strategies

Academic medical centers are widely recognized as vital components of the American health care system, generally differentiated from their community hospital peers by their tripartite mission of clinical care, education, and research. Community hospitals fill a critical and complementary role, serving as the primary sites for health care in most communities. Health care reform initiatives and economic pressures have created incentives for hospitals and health systems to integrate, resulting in a nationwide trend toward consolidation with academic medical centers leveraging their substantial assets to merge, acquire, or establish partnerships with their community peers. As these alliances accelerate, they have and will continue to affect the radiology groups providing services at these institutions. A deeper understanding of these new marketplace dynamics, changing relationships and potential strategies will help both academic and private practice radiologists adapt to this ongoing change.

Of mice and men: escaping the relative value unit trap

Although radiologists have kept pace with the dramatic growth of medical imaging during the past two decades through the use of PACS, digital dictation, and 3-dimensional reconstruction, radiology is approaching the point of diminishing returns. As reimbursements further decline, attempts to increase radiologists' productivity risk commoditization of the specialty. The continued evolution of US health care policy, however, presents an opportunity for radiologists, using their core competencies, to shape the future direction of medicine.

Comprehensive review of the limitations of relative value unit measurements of productivity in academic departments

Measuring radiologist, especially sonologist, productivity has never been of greater interest than now, as radiology has moved from its historical status as a cost center to the largest revenue generator in most institutions. With more local institutional and federal oversight and regulation into the reimbursement and valuation of imaging procedures, including emphasis on outsourcing, overvalued procedures, and bundling, the ability to measure productivity and to draw meaningful conclusions from the data becomes increasingly imperative if we as radiologists and sonologists expect to contribute meaningfully to the process. This article presents a history of the "relative value unit" and discusses the valuation of radiologist/sonologist productivity in this era of ubiquitous high-technology implementation.

Workload of consultant radiologists in a large DGH and how it compares to international benchmarks

AIM

To measure radiologist workloads in a UK radiology department using relative value units (RVUs), and compared these data to current international standards from Australia and Ireland.

MATERIALS AND METHODS

Data on reporting throughput was gathered for 13.5 whole-time equivalent (WTE) consultants at the DGHs of East Lancashire Hospitals Trust (ELHT) between April 2010 to March 2011. RVUs were assigned to the reported imaging studies to create a crude RVU/WTE score. This was compared to benchmarks from Australia and results from a similar study in Ireland. Time spent on teaching, multidisciplinary teams, and administration was factored in to create a net RVU/WTE score, more accurately assessing workload.

RESULTS

Radiologists in ELHT reported 110,315 imaging studies, producing a total of 649,617 RVUs. Crude reporting workloads were 48,119.78 RVUs/WTE, and net workloads were 83,674.00 RVUs/WTE (with consultants spending 42.49% of their time on non-reporting commitments). These crude and net workloads are far above the Australian maximum reporting benchmark of 45,000 RVUs.

CONCLUSION

The workload is much higher than international benchmarks, indicating high-quality service and excellent value for money, but also highlights understaffing. Foreign RVU systems do not accurately reflect current UK practices and a UK-specific RVU system should be developed to assess staffing and analyse performance.