Are referral guidelines for CT examinations addressing all clinical scenarios? A comparison of EURO-2000 Guidelines and ESR iGuide

Impact and effect of imaging referral guidelines on patients and radiology services: a systematic review

OBJECTIVES

The objective of this systematic review was to offer a comprehensive overview and explore the associated outcomes from imaging referral guidelines on various key stakeholders, such as patients and radiologists.

MATERIALS AND METHODS

An electronic database search was conducted in Medline, Embase and Web of Science to retrieve citations published between 2013 and 2023. The search was constructed using medical subject headings and keywords. Only full-text articles and reviews written in English were included. The quality of the included papers was assessed using the mixed methods appraisal tool. A narrative synthesis was undertaken for the selected articles.

RESULTS

The search yielded 4384 records. Following the abstract, full-text screening, and removal of duplication, 31 studies of varying levels of quality were included in the final analysis. Imaging referral guidelines from the American College of Radiology were most commonly used. Clinical decision support systems were the most evaluated mode of intervention, either integrated or standalone. Interventions showed reduced patient radiation doses and waiting times for imaging. There was a general reduction in radiology workload and utilisation of diagnostic imaging. Low-value imaging utilisation decreased with an increase in the appropriateness of imaging referrals and ratings and cost savings. Clinical effectiveness was maintained during the intervention period without notable adverse consequences.

CONCLUSION

Using evidence-based imaging referral guidelines improves the quality of healthcare and outcomes while reducing healthcare costs. Imaging referral guidelines are one essential component of improving the value of radiology in the healthcare system.

CLINICAL RELEVANCE STATEMENT

There is a need for broader dissemination of imaging referral guidelines to healthcare providers globally in tandem with the harmonisation of the application of these guidelines to improve the overall value of radiology within the healthcare system.

KEY POINTS

The application of imaging referral guidelines has an impact and effect on patients, radiologists, and health policymakers. The adoption of imaging referral guidelines in clinical practice can impact healthcare costs and improve healthcare quality and outcomes. Implementing imaging referral guidelines contributes to the attainment of value-based radiology.

Evaluating the reliability of ChatGPT as a tool for imaging test referral: a comparative study with a clinical decision support system

OBJECTIVES

As the technology continues to evolve and advance, we can expect to see artificial intelligence (AI) being used in increasingly sophisticated ways to make a diagnosis and decisions such as suggesting the most appropriate imaging referrals. We aim to explore whether Chat Generative Pretrained Transformer (ChatGPT) can provide accurate imaging referrals for clinical use that are at least as good as the ESR iGuide.

METHODS

A comparative study was conducted in a tertiary hospital. Data was collected from 97 consecutive cases that were admitted to the emergency department with abdominal complaints. We compared the imaging test referral recommendations suggested by the ESR iGuide and the ChatGPT and analyzed cases of disagreement. In addition, we selected cases where ChatGPT recommended a chest abdominal pelvis (CAP) CT (n = 66), and asked four specialists to grade the appropriateness of the referral.

RESULTS

ChatGPT recommendations were consistent with the recommendations provided by the ESR iGuide. No statistical differences were found between the appropriateness of referrals by age or gender. Using a sub-analysis of CAP cases, a high agreement between ChatGPT and the specialists was found. Cases of disagreement (12.4%) were further analyzed and presented themes of vague recommendations such as "it would be advisable" and "this would help to rule out."

CONCLUSIONS

ChatGPT's ability to guide the selection of appropriate tests may be comparable to some degree with the ESR iGuide. Features such as the clinical, ethical, and regulatory implications are still warranted and need to be addressed prior to clinical implementation. Further studies are needed to confirm these findings.

CLINICAL RELEVANCE STATEMENT

The article explores the potential of using advanced language models, such as ChatGPT, in healthcare as a CDS for selecting appropriate imaging tests. Using ChatGPT can improve the efficiency of the decision-making process KEY POINTS: • ChatGPT recommendations were highly consistent with the recommendations provided by the ESR iGuide. • ChatGPT's ability in guiding the selection of appropriate tests may be comparable to some degree with ESR iGuide's.

Clinical decision support system recommendations: how often do radiologists and clinicians accept them?

OBJECTIVE

To assess the acceptance and reliability of clinical decision support system (CDSS) imaging referral scores (ESR iGuide).

METHODS

A pilot study was conducted in a tertiary hospital. Four different experts were invited to rate 40 simulated clinical cases on a 5-level scale, for the level of agreement with the ESR iGuide's recommended procedures. In cases of disagreement, physicians were asked to indicate the reason. Descriptive measures were calculated for the level of agreement. We also explored the degree of agreement between four different specialists, and examined the cases in which clinicians disagreed with ESR iGuide best practice recommendations.

RESULTS

The mean rating of the four experts for the 40 clinical simulated cases was 4.17 ± 0.65, median 4.25 (on a scale of 1-5). All four raters totally agreed with the system recommendation in 75% of cases. No significant relationship was found between the degree of agreement and the number of indications and the patient's age or gender. In an optimistic scenario, using a binary agree/disagree variable, the Overall Percentage Agreement for the rating of the 40 simulated cases between the four experts was 77.28%. There were a total of 20 disagreements out of 160 cases with the ESR iGuide, of which 7 were among the two radiologists.

CONCLUSIONS

CDSS can be an effective tool for guiding the selection of appropriate imaging examinations, thus cutting costs due to unnecessary imaging scans. Since this is a pilot study, further research on a larger scale, preferably at national level, is required.

KEY POINTS

• The average of the mean rating of the four experts was 4.17 ± 0.65, median 4.25, on a scale of 1-5 where 5 represents total agreement with the CDSS tool. • In an optimistic scenario, using a binary agree/disagree variable, the Overall Percentage Agreement between the four experts was 77.28%. • Radiologists had fewer disagreements with the recommendations of the CDSS tool than other physicians, indicating a better fit of the support system to radiology experts' perspective.