Current Status and Future Wish List of Peer Review: A National Questionnaire of U.S. Radiologists

Event-Based Learning and Improvement: Radiology's Move From Peer Review to Peer Learning

Over the past 15 years, the radiology community has made great progress moving from a system of score-based peer review to one of peer learning. Much has been learned along the way. In peer learning, cases in which learning opportunities are identified are reviewed solely for the purpose of fostering learning and improvement. This article defines peer learning and peer review and emphasizes the difference; looks back at the 20-year history of score-based peer review and transition to peer learning; outlines the problems with score-based peer review and the key elements of peer learning; discusses the current state of peer learning; and outlines future challenges and opportunities.

Peer learning is both preferable and less expensive than score-based peer review: Initial experience at a tertiary academic center

PURPOSE

To examine radiologist experiences and perceptions during a transition from score-based peer review to a peer learning program, and to assess differences in time-cost efficiency between the two models of quality improvement.

METHODS

Differences in Likert scale survey responses from radiologists (N = 27) in a multispecialty group at a single tertiary academic center before and following intervention were evaluated by Mann-Whitney U test. Multiple variable linear regression analysis assessed independent variables and program preference.

RESULTS

All positive impacts rated significantly higher for the peer learning program. Workflow disruption for the peer learning program rated significantly lower. 70.4 % (19 of 27) preferred the new program, and 25.9 % (7 of 27) preferred the old program. Only the "worth investment" questionnaire score demonstrated a significant correlation to program preference and with an effect that was greatest among all variables (Beta = 1.11, p = 0.02). There was a significantly decreased amount of time per month used to complete peer learning exercises (0.76 ± 0.45 h, N = 27) versus peer review exercises (1.71 ± 1.84 h, N = 34, p = 0.011). The result was a difference of 0.95 ± 1.89 h/month (11.4 ± 22.7 h/year), translating to an estimated direct salary time-cost saving of $1653.68/year/radiologists and a direct productivity time-cost saving of $3469.39/year/radiologist when utilizing the peer learning program.

CONCLUSIONS

There was a strongly positive perception of the new peer learning program. There was a substantial implied direct time-cost saving from the transition to the peer learning program.

PRECIS

The peer learning model emphasizes learning from errors via feedback in a non-punitive environment. This model was positively perceived and demonstrated substantial implied direct time-cost saving.

The Joint Commission's Ongoing Professional Practice Evaluation Process: Costly, Ineffective, and Potentially Harmful to Safety Culture

OBJECTIVE

To evaluate the estimated labor costs and effectiveness of Ongoing Professional Practice Evaluation (OPPE) processes at identifying outlier performers in a large sample of providers across multiple health care systems and to extrapolate costs and effectiveness nationally.

METHODS

Six hospital systems partnered to evaluate their labor expenses related to conducting OPPE. Estimates for mean labor hours and wages were created for the following: data analysts, medical staff office professionals, department physician leaders, and administrative assistants. The total number of outlier performers who were identified by OPPE metrics alone and that resulted in lack of renewal, limitation, or revoking of hospital privileges during the past annual OPPE cycle (2022) was recorded. National costs of OPPE were extrapolated. Literature review of the effect of OPPE on safety culture in radiology was performed.

RESULTS

The evaluated systems had 12,854 privileged providers evaluated by OPPE. The total estimated annual recurring labor cost per provider was $50.20. Zero of 12,854 providers evaluated were identified as outlier performers solely through the OPPE process. The total estimated annual recurring cost of administering OPPE nationally was $78.54 million. In radiology over the past 15 years, the use of error rates based on score-based peer review as an OPPE metric has been perceived as punitive and had an adverse effect on safety culture.

CONCLUSION

OPPE is expensive to administer, inefficient at identifying outlier performers, diverts human resources away from potentially more effective improvement work, and has been associated with an adverse impact on safety culture in radiology.

Transition from Peer Review to Peer Learning: Lessons Learned

Landmark publications, such as To Err is Human, confronted the healthcare community with the egregious toll medical errors played in both patient safety and overall healthcare costs. This heralded a paradigm shift and a call for action by professional organizations to enact methods to ensure physician competency and quality assurance. The American College of Radiology similarly convened a task force to discuss these concerns and how best to address quality assurance in radiology practice, leading to the development of RADPEER, a score-based peer review system. However, critics were quick to point out the deficiencies of this model, highlighting it as punitive and a poor evaluator of physician performance. The recognized deficiencies in score-based peer review prompted the pursuit of an alternate model that would instead emphasize learning and improvement. Peer learning was proposed and highlighted the necessity of an inclusive and collaborative environment where colleagues could discuss case errors as learning opportunities without fear of punitive consequence. This paper explores peer learning, its benefits and challenges, as well as how to identify specific learning opportunities by utilizing case examples.

Peer Learning Program Metrics: A Pediatric Neuroradiology Example

BACKGROUND AND PURPOSE

The American College of Radiology is now offering an accreditation pathway for programs that use peer learning. Here, we share feasibility and outcome data from a pilot peer learning program in a pediatric neuroradiology section that, in its design, follows the American College of Radiology peer learning accreditation pathway criteria.

MATERIALS AND METHODS

We retrospectively reviewed metrics from a peer learning program with 5 participating full-time pediatric neuroradiologists during 1 year: 1) number of cases submitted, 2) percentage of radiologists meeting targets, 3) monthly attendance, 4) number of cases reviewed, 5) learning points, and 6) improvement actions. In addition, a faculty survey was conducted and is reported here.

RESULTS

Three hundred twenty-four cases were submitted (mean, 7 cases/faculty/month). The faculty never met the monthly submission target. Peer learning meeting attendance was 100%. One hundred seventy-nine cases were reviewed during the peer learning meetings. There were 22 learning points throughout the year and 30 documented improvement actions. The faculty survey yielded the highest ratings (4.8 of 5) for ease of meeting the 100% attendance requirement and for the learning value of the peer learning sessions. The lowest rating (4.2 of 5) was given for the effectiveness of improvements as a result of peer learning discussions.

CONCLUSIONS

Implementing a peer learning program that follows the American College of Radiology peer learning accreditation pathway criteria is feasible. Program metric documentation can be time-consuming. Participant feedback led to meaningful program improvement, such as improving trust, expanding case submission categories, and delegating tasks to administrative staff. Effort to make peer learning operations more efficient and more effective is underway.

Implementation of a peer-learning program in an academic abdominal radiology practice and comparison with a traditional peer-review system

OBJECTIVE

The purpose of this study was to transition from a traditional score-based peer-review system to an education-oriented peer-learning program in our academic abdominal radiology practice.

MATERIAL AND METHODS

This retrospective study compared our experience with a score-based peer-review model used prior to September 2020 and a peer-learning model implemented and used exclusively beginning in October of 2020. In peer review, a web-based peer-review tool randomly generated a list of cases, which were blindly reviewed in consensus. Comparison of the consensus interpretation with the original report was used to categorize each reviewed case and to calculate the rates of significant and minor discrepancies. Only cases with a discrepancy were considered to represent a learning opportunity. In peer learning, faculty prospectively identified and submitted cases for review in several categories, including case interpretations with a discrepancy from subsequent opinion or result, interpretations considered to represent a great call, and interesting or challenging cases meriting further discussion. The peer-learning coordinator showed each case to the group in a manner which blinded the group to both submitting and interpreting radiologist and invited discussion during various stages of the case.

RESULTS

During peer review, a total of 172 cases were reviewed over 16 sessions occurring between April 2016 and September 2020. Only 3 cases (1.8%) yielded significant discrepancies whereas 13 (7.6%) yielded minor discrepancies, representing a total of 16 learning opportunities (3.6 per year). In peer learning, 64 cases were submitted and 52 reviewed over 7 sessions occurring between October 2020 and October 2021. 29 (56%) were submitted as an interesting or challenging case meriting further discussion, 18 (35%) were submitted for a discrepancy, and 5 (10%) were submitted for a great call. All 52 presented cases represented learning opportunities (48 per year).

CONCLUSION

An education-focused peer-learning program provided a platform for continuous quality improvement and yielded substantially more learning opportunities compared to score-based peer review.

The Current State of Gender Discrimination and Sexual Harassment in the Radiology Workplace: A Survey

RATIONALE AND OBJECTIVES

Our objectives were (1) to determine the extent to which gender discrimination and sexual harassment are experienced by female radiologists and trainees; (2) to examine whether experiencing harassment or discrimination influences perceptions of gender parity; and (3) to explore whether the existence of either formal institutional policies or the number of women in the workplace and/or in leadership positions influences perceptions of having achieved gender equity.

MATERIALS AND METHODS

An online anonymous questionnaire, developed through an Association of University Radiologists (AUR) - affiliated 2019-2020 Task Force, was used to assess participant demographics, perceptions of gender parity, and experiences of gender discrimination and sexual harassment.

RESULTS

A total of 375 complete responses were collected. All respondents were female with most practices consisting of fewer than 25% female radiologists. The majority of respondents reported having been a victim of sexual harassment (n = 226, 60.3%) and gender discrimination (n = 318, 84.8%) in the workplace. Approximately 87.5% of participants believed workplace gender parity would take longer than 10 years to achieve; 26.9% responded that it will never happen. Experiencing gender discrimination or harassment in the workplace was significantly associated with a negative outlook on achieving gender parity. Conversely, the presence and number of adequate formal institutional policies to address workplace gender equity and harassment were significantly associated with optimistic views on achieving gender parity. Higher percentages of women in one's practice as well as number of women in leadership positions were also significantly associated with more optimistic expectations.

CONCLUSION

Gender discrimination and sexual harassment are common in the field of radiology and influence victims' outlook on achieving gender parity in the workplace. Perceptions can be improved by implementing adequate institutional training policies on harassment and increasing the representation of female radiologists.

Radiology Trainees' Perceptions of Speaking up Culture Related to Safety and Unprofessional Behavior in Their Work Environments

OBJECTIVE. The purpose of this study was to compare radiology trainees' perceptions of the culture regarding speaking up about patient safety and unprofessional behavior in the clinical environment and to assess the likelihood that they will speak up in the presence of a medical hierarchy. MATERIALS AND METHODS. The study included radiology trainees from nine hospitals who attended a communication workshop. Trainees completed questionnaires assessing their perceptions of the support provided by their clinical environment regarding speaking up about patient safety and unprofessional behavior. We also queried their likelihood of speaking up within a team hierarchy about an error presented in a hypothetical clinical vignette. RESULTS. Of 61 participants, 58 (95%) completed questionnaires. Of these 58 participants, 84% felt encouraged by colleagues to speak up about safety concerns, and 57% felt encouraged to speak up about unprofessional behavior (p < .001). Moreover, 17% and 34% thought speaking up about safety concerns and unprofessional behavior, respectively, was difficult (p < .02). Trainees were less likely to agree that speaking up about unprofessional behavior (compared with speaking up about safety concerns) resulted in meaningful change (66% vs 95%; p < .001). In a vignette describing a sterile technique error, respondents were less likely to speak up to an attending radiologist (48%) versus a nurse, intern, or resident (79%, 84%, and 81%, respectively; p < .001). Significant predictors of the likelihood of trainees speaking up to an attending radiologist included perceived potential for patient harm as a result of the error (odds ratio [OR], 6.7; p < .001), perceptions of safety culture in the clinical environment (OR, 5.0; p = .03), and race or ethnicity (OR, 3.1; p = .03). CONCLUSION. Radiology trainees indicate gaps in workplace cultures regarding speaking up, particularly concerning unprofessional behavior and team hierarchy.