Subspecialist radiology

Effect of a dedicated PI-QUAL curriculum on the assessment of prostate MRI quality

PURPOSE

The Prostate Imaging Quality (PI-QUAL) score is a metric to evaluate the diagnostic quality of multiparametric magnetic resonance imaging (MRI) of the prostate. This study evaluated the impact of a prostate MRI quality training lecture on the participant's ability to assess prostate MRI image quality.

METHODS

Eighteen in-training-radiologists of varying experience in reviewing diagnostic prostate MRI assessed the image quality of ten examinations. Then, they attended a dedicated lecture on MRI quality assessment using the PI-QUAL score. After the lecture, the same participants evaluated the image quality of a new set of ten scans applying the PI-QUAL score. Results were assessed using receiver operating characteristic (ROC) analysis. The reference standard was the PI-QUAL score assessed by a fellowship trained abdominal radiologist with experience in reading prostate MRI.

RESULTS

There was a significant improvement in the average area under the curve (AUC) for assessment of prostate MRI image quality from baseline (0.82; [0.576 - 0.888]) to post teaching (1.0; [0.954-1]), with an improvement of 0.18 (p < 0.03). When ROC curves were computed for different cohorts stratified based on year of training, difference ranged from 0.48 for second year residents to 0.32 for fourth year residents (p < 0.001-0.01). For abdominal imaging fellows, the pre-teaching AUC was 0.9 [0.557-1] and post teaching AUC was 1 [0.957-1], a difference of 0.1 (p = 0.20).

CONCLUSIONS

A dedicated lecture on PI-QUAL improved the ability of radiologists-in-training to assess prostate MRI image quality, with variable impact depending on year of training.

Pancreas-protocol imaging at a high-volume center leads to improved preoperative staging of pancreatic ductal adenocarcinoma

BACKGROUND

High-quality preoperative cross-sectional imaging is vital to accurately stage patients with pancreatic ductal adenocarcinoma (PDAC). We hypothesized that imaging performed at a high-volume pancreatic cancer center with pancreatic imaging protocols more accurately stages patients compared with pre-referral imaging.

METHODS

We retrospectively reviewed data from all patients with PDAC who presented to the surgical oncology clinic at our institution between June 2005 and August 2009. Detailed preoperative imaging, staging, and operative data were collected for each patient.

RESULTS

A total of 230 patients with PDAC were identified, of which 169 had pre-referral imaging. Patients were selectively reimaged at our institution based on the quality and timing of imaging at the outside facility: 108 (47%) patients were deemed resectable, 54 (23.5%) were deemed borderline-resectable, and 68 (29.5%) were deemed unresectable. Of the resectable patients, 99 opted for resection. Eighty-two of those 99 patients underwent preoperative imaging at our institution, and of these 27% had unresectable disease at the time of surgery compared with 47% of patients who only had pre-referral imaging (p = 0.14). Reimaging altered staging and changed management in 56% of patients. Among that group were 55 patients, categorized as resectable on pre-referral imaging, who on repeat imaging were deemed to be borderline resectable (n = 27) or unresectable (n = 28).

CONCLUSIONS

Pancreas-protocol imaging at a high-volume center improves preoperative staging and alters management in a significant proportion of patients with PDAC who undergo pre-referral imaging. Thus, repeat imaging with pancreas protocols and dedicated radiologists is justified at high-volume centers.

Interactive dedicated training curriculum improves accuracy in the interpretation of MR imaging of prostate cancer

OBJECTIVE

To assess the effect of interactive dedicated training on radiology fellows' accuracy in assessing prostate cancer on MRI.

METHODS

Eleven radiology fellows, blinded to clinical and pathological data, independently interpreted preoperative prostate MRI studies, scoring the likelihood of tumour in the peripheral and transition zones and extracapsular extension. Each fellow interpreted 15 studies before dedicated training (to supply baseline interpretation accuracy) and 200 studies (10/week) after attending didactic lectures. Expert radiologists led weekly interactive tutorials comparing fellows' interpretations to pathological tumour maps. To assess interpretation accuracy, receiver operating characteristic (ROC) analysis was conducted, using pathological findings as the reference standard.

RESULTS

In identifying peripheral zone tumour, fellows' average area under the ROC curve (AUC) increased from 0.52 to 0.66 (after didactic lectures; p<0.0001) and remained at 0.66 (end of training; p<0.0001); in the transition zone, their average AUC increased from 0.49 to 0.64 (after didactic lectures; p=0.01) and to 0.68 (end of training; p=0.001). In detecting extracapsular extension, their average AUC increased from 0.50 to 0.67 (after didactic lectures; p=0.003) and to 0.81 (end of training; p<0.0001).

CONCLUSION

Interactive dedicated training significantly improved accuracy in tumour localization and especially in detecting extracapsular extension on prostate MRI.

Effect of directed training on reader performance for CT colonography: multicenter study

PURPOSE

To define the interpretative performance of radiologists experienced in computed tomographic (CT) colonography and to compare it with that of novice observers who had undergone directed training, with colonoscopy as the reference standard.

MATERIALS AND METHODS

Physicians at each participating center received ethical committee approval and followed the committees' requests regarding informed consent. Nine experienced radiologists, nine trained radiologists, and 10 trained technologists from nine centers read 40 CT colonographic studies selected from a data set of 51 studies and modeled to simulate a population with positive fecal occult blood test results: Studies were obtained in eight patients with cancer, 12 patients with large polyp, four patients with medium polyp, and 27 patients without colonic lesions. Findings were verified with colonoscopy. An experienced radiologist used 50 endoscopically validated studies to train novice observers before they were allowed to participate. Observers used one software platform to read studies over 2 days. Responses were collated and compared with the known diagnostic category for each subject. The number of correctly classified subjects was determined for each observer, and differences between groups were examined with bootstrap analysis.

RESULTS

Overall, 28 observers read 1084 studies and detected 121 cancers, 134 large polyps, and 33 medium polyps; 448 healthy subjects were categorized correctly. Experienced radiologists detected 116 lesions; trained radiologists and technologists detected 85 and 87 lesions, respectively. Overall accuracy of experienced observers (74.2%) was significantly better than that of trained radiologists (66.6%) and technologists (63.2%). There was no significant difference (P=.33) between overall accuracy of trained radiologists and that of technologists; however, some trainees reached the mean performance achieved by experienced observers.

CONCLUSION

Experienced observers interpreted CT colonographic images significantly better than did novices trained with 50 studies. On average, no difference between trained radiologists and trained technologists was found; however, individual performance was variable and some trainees outperformed some experienced observers.

Radiological error: analysis, standard setting, targeted instruction and teamworking

Diagnostic radiology does not have objective benchmarks for acceptable levels of missed diagnoses. Until now, data collection of radiological discrepancies has been very time consuming. The culture within the specialty did not encourage it. However, public concern about patient safety is increasing. There have been recent innovations in compiling radiological interpretive discrepancy rates which may facilitate radiological standard setting. However standard setting alone will not optimise radiologists' performance or patient safety. We must use these new techniques in radiological discrepancy detection to stimulate greater knowledge sharing, targeted instruction and teamworking among radiologists. Not all radiological discrepancies are errors. Radiological discrepancy programmes must not be abused as an instrument for discrediting individual radiologists. Discrepancy rates must not be distorted as a weapon in turf battles. Radiological errors may be due to many causes and are often multifactorial. A systems approach to radiological error is required. Meaningful analysis of radiological discrepancies and errors is challenging. Valid standard setting will take time. Meanwhile, we need to develop top-up training, mentoring and rehabilitation programmes.

Radiological error: analysis, standard setting, targeted instruction and teamworking

Diagnostic radiology does not have objective benchmarks for acceptable levels of missed diagnoses. Until now, data collection of radiological discrepancies has been very time consuming. The culture within the specialty did not encourage it. However, public concern about patient safety is increasing. There have been recent innovations in compiling radiological interpretive discrepancy rates which may facilitate radiological standard setting. However standard setting alone will not optimise radiologists' performance or patient safety. We must use these new techniques in radiological discrepancy detection to stimulate greater knowledge sharing, targeted instruction and teamworking among radiologists. Not all radiological discrepancies are errors. Radiological discrepancy programmes must not be abused as an instrument for discrediting individual radiologists. Discrepancy rates must not be distorted as a weapon in turf battles. Radiological errors may be due to many causes and are often multifactorial. A systems approach to radiological error is required. Meaningful analysis of radiological discrepancies and errors is challenging. Valid standard setting will take time. Meanwhile, we need to develop top-up training, mentoring and rehabilitation programmes.

MRI of Fistula In Ano: Inter- and Intraobserver Agreement and Effects of Directed Education

OBJECTIVE

Preoperative MRI of fistula in ano is becoming more common. This prospective study aimed to determine if a significant difference occurred in interpretation between one expert and one novice observer and to assess inter- and intraobserver agreement after both observers underwent a period of directed education. SUBJECTS AND METHODS. An outcome-derived reference standard was defined in 100 patients with suspected fistula in ano via a combination of preoperative MRI, surgical findings, and clinical outcome. The performances of a single expert and a single novice interpreter were compared with this reference standard both before and after a period of directed education, and inter- and intraobserver agreement was determined.

RESULTS

Initially the expert correctly classified significantly more fistulas than the novice (85% vs 63%, p = 0.024), but after directed education there was no significant difference, with good agreement for both the classification of the primary track (kappa = 0.71) and identification of extensions (k = 0.61). Intraobserver agreement was very good for the expert (kappa = 0.92) and novice (kappa = 0.88) for classification of the primary track and good (kappa = 0.64 and 0.74, respectively) for identification of extensions.

CONCLUSION

The diagnostic accuracy for fistula in ano classification using MRI was significantly higher for one expert than for one novice, though this was rectified by a short period of directed education.

CT colonography: effect of experience and training on reader performance

The purpose of this paper was to investigate the effect of radiologist experience and increasing exposure to CT colonography on reader performance. Three radiologists of differing general experience (consultant, research fellow, trainee) independently analysed 100 CT colonographic datasets. Readers had no prior experience of CT colonography and received feedback and training after the first 50 cases from an independent experienced radiologist. Diagnostic performance and reporting times were compared for the first and second 50 datasets and compared with the results of a radiologist experienced in CT colonography. Before training only the consultant reader achieved statistical equivalence with the reference standard for detection of larger polyps. After training, detection rates ranged between 25 and 58% for larger polyps. Only the trainee significantly improved after training ( P=0.007), with performance of other readers unchanged or even worse. Reporting times following training were reduced significantly for the consultant and fellow ( P<0.001 and P=0.03, respectively), but increased for the trainee ( P<0.001). In comparison to the consultant reader, the odds of detection of larger polyps was 0.36 (CI 0.16, 0.82) for the fellow and 0.36 (CI 0.14, 0.91) for the trainee. There is considerable variation in the ability to report CT colonography. Prior experience in gastrointestinal radiology is a distinct advantage. Competence cannot be assumed even after directed training via a database of 50 cases.

CT colonography practice in the UK: a national survey

AIM

To determine the provision of computed tomography (CT) colonography in UK radiology departments.

MATERIALS AND METHODS

A questionnaire relating to the availability of CT colonography, barriers to implementation, clinical indications, technique, and practitioners was posted to clinical directors of UK radiology departments.

RESULTS

One hundred and thirty-eight departments responded. Fifty (36%) offered CT colonography in day-to-day clinical practice. Of those that did not, 68 of 87 (64%) cited limited scanner capacity as the main barrier. Of the 50 departments offering a service, 39 (78%) offered CT after incomplete colonoscopy, 36 (72%), after failed barium enema, and 37 (74%) as an alternative to barium enema. Of those offering a service, the number of studies performed varied between one per month (38%) to more than one per day (8%). Total experience varied between 20 or fewer studies (28%) to more than 300 (12%). Full bowel preparation was common (92%), as was dual positioning (90%). Colonography was interpreted by radiologists with a subspecialty interest in gastrointestinal imaging in 64% of centres offering a service.

CONCLUSION

CT colonography is widely available in the UK, with approximately one-third of responders offering a service. Experience and throughput varies considerably. Limited CT scanner capacity is the major barrier to further dissemination.

Observer Variation in the Detection of Colorectal Neoplasia on Double-contrast Barium Enema: Implications for Colorectal Cancer Screening and Training

AIM

To assess inter-observer error for the diagnosis of neoplasia on double contrast barium enema (DCBE) in the light of claims that no additional interpretative training would be needed for implementation in a national screening programme.

MATERIALS AND METHODS

10 experts, 10 consultants, and 10 experienced trainees each reported 20 DCBE studies, of which two showed cancer, three showed large polyps, four showed small polyps, and 12 were normal. Inter-observer variation was compared using odds ratios with the trainee group as reference (baseline group).

RESULTS

Experts were significantly more likely to correctly identify neoplasia on DCBE than trainees. The odds of a correct diagnosis for experts were 2.79 (95% CI 2.04, 3.81) for cancer, 2.36 (1.88, 2.97) for large polyps, and 3.50 (1.98, 6.18) for small polyps. While consultants were more likely to correctly diagnose a large polyp than trainees, 1.45 (1.15, 1.84), there was no significant difference between these two groups for the correct diagnosis of either cancer, 1.24 (0.52, 2.96), or small polyps, 1.26 (0.83, 1.90). A cancer was missed by 6 (60%) experts, 9 (90%) consultants, and 8 (80%) trainees. Large polyps were missed by 4 (40%) experts, 5 (50%) consultants, and 6 (60%) trainees. There was no significant difference between any group when false positive diagnoses were considered.

CONCLUSIONS

There is considerable inter-observer perceptive error for the diagnosis of neoplasia on DCBE. Experts performed significantly better than other observers but the overall standard of performance was poor, even amongst experts.