Multiple diagnostic task performance in CT examination of the chest

Objectives In three experiments, we studied the detection of multiple abnormality types using the satisfaction of search (SOS) paradigm, the provision of a computer-aided detection (CAD) of pulmonary nodules and a focused nodule detection task. Methods 51 chest CT examinations (24 that demonstrated subtle pulmonary nodules and 27 that demonstrated no pulmonary nodules) were read by 15 radiology residents and fellows under two experimental conditions: (1) when there were no other abnormalities present except test abnormalities in the exams (non-SOS condition), and (2) when other abnormalities were present in the exams (SOS condition). Trials from the two conditions were intermixed. Readers were invited to return for two sessions: one in which the SOS condition was repeated with a simulated CAD; another in which only the non-SOS condition was presented. Detection accuracy was measured using receiver operating characteristic (ROC) analysis. Results An SOS effect (reduced detection accuracy for the test nodules in the presence of the diverse added abnormalities) was not found. Average accuracy was much higher when the CAD prompt was provided, without cost in the detection of the added abnormalities. Accuracy for detecting nodules appearing without intermixed SOS trials was also substantially improved. Conclusions CT interpretation was highly task dependent. Nodule detection was poor in the general search task. Therefore, CAD may offer a greater performance improvement than demonstrated in experiments assessing CAD using focused search. The absence of SOS may be due to limited nodule detection even without other abnormalities. Advances in knowledge CAD prompts of nodules increase the detection accuracy of nodules and decrease the time to detection-without impairing the detection accuracy-of non-nodule abnormalities.

Multiple diagnostic task performance in CT examination of the chest

OBJECTIVES

In three experiments, we studied the detection of multiple abnormality types using the satisfaction of search (SOS) paradigm, the provision of a computer-aided detection (CAD) of pulmonary nodules and a focused nodule detection task.

METHODS

51 chest CT examinations (24 that demonstrated subtle pulmonary nodules and 27 that demonstrated no pulmonary nodules) were read by 15 radiology residents and fellows under two experimental conditions: (1) when there were no other abnormalities present except test abnormalities in the exams (non-SOS condition), and (2) when other abnormalities were present in the exams (SOS condition). Trials from the two conditions were intermixed. Readers were invited to return for two sessions: one in which the SOS condition was repeated with a simulated CAD; another in which only the non-SOS condition was presented. Detection accuracy was measured using receiver operating characteristic (ROC) analysis.

RESULTS

An SOS effect (reduced detection accuracy for the test nodules in the presence of the diverse added abnormalities) was not found. Average accuracy was much higher when the CAD prompt was provided, without cost in the detection of the added abnormalities. Accuracy for detecting nodules appearing without intermixed SOS trials was also substantially improved.

CONCLUSIONS

CT interpretation was highly task dependent. Nodule detection was poor in the general search task. Therefore, CAD may offer a greater performance improvement than demonstrated in experiments assessing CAD using focused search. The absence of SOS may be due to limited nodule detection even without other abnormalities. Advances in knowledge CAD prompts of nodules increase the detection accuracy of nodules and decrease the time to detection-without impairing the detection accuracy-of non-nodule abnormalities.

Does computer-aided diagnosis for lung tumors change satisfaction of search in chest radiography?

RATIONALE AND OBJECTIVES

Computer-aided diagnosis (CAD) has been developed to ensure that the radiologist considers suspect focal opacities that may represent cancer in chest radiography. Although CAD was not developed to counteract the satisfaction of search (SOS) effect, it may be an effective intervention to do so. The objective of this study is to determine whether an idealized CAD can reduce SOS effects in chest radiography.

MATERIALS AND METHODS

Fifty-seven chest radiographs, half of which demonstrated diverse, native abnormalities were read twice by 16 observers, once with and once without the addition of a simulated pulmonary nodule. Simulated CAD prompts were provided during the interpretation, which unerringly pointed to the added simulated nodule. Area under the ROC curve for detecting the native abnormalities was estimated for each observer in each treatment condition. In addition to testing for the SOS effect in the presence of CAD prompts, results were compared to those of a previous SOS study.

RESULTS

Significantly more nodules were reported in the SOS with CAD experiment than in the original SOS experiment (49 versus 43, P < .01). An SOS effect was found even when CAD prompts were provided; ROC areas for detecting native abnormalities were reduced with added nodules [0.68 versus 0.65, P (one-tailed) < .05]. Comparison of the current experiment with CAD and the previous SOS experiments failed to show a significant difference of the magnitude of the SOS effect (P = .52). The threshold for reporting was more conservative with CAD prompts than in SOS studies (P = .052).

CONCLUSION

Our results indicate that the CAD prompts, even those that always point to their target lesion without false-positive error, fail to counteract SOS in chest radiography. The stricter decision thresholds with CAD prompts may indicate less visual search for native abnormalities.

Can order of report prevent satisfaction of search in abdominal contrast studies?

RATIONALE AND OBJECTIVE

A previous receiver operating characteristic (ROC) study showed a systematic shift in decision thresholds for detecting plain film abnormalities on contrast examinations rather than plain radiographs. A previous eye-position study showed that this shift was based on a relative visual neglect of plain film regions on the contrast studies. We now determine whether an intervention that changes visual search can reduce this search-based satisfaction of search effect in contrast studies of the abdomen.

MATERIALS AND METHODS

The authors measured detection of 23 plain film abnormalities in 44 patients who had plain film and contrast examinations. In 2 experiments, each plain-film and contrast study was examined independently in different sessions with observers providing a confidence rating of abnormality for each interpretation. There were 13 observers in the first experiment and 10 in the second experiment. The intervention required that for the contrast studies, observers first report abnormalities in the noncontrast region of the radiograph before reporting contrast findings. ROC curve areas for each observer in each treatment condition were estimated by using a proper ROC model. The analysis focused on changes in decision thresholds among the treatment conditions.

RESULTS

The SOS effect on decision thresholds in abdominal contrast studies was replicated. Although reduced, the shift in decision thresholds in detecting plain film abnormalities on contrast examinations remained when observers were required to report those abnormalities before contrast findings.

CONCLUSION

Reporting plain film abnormalities before reporting abnormalities demonstrated by contrast reduced somewhat the satisfaction of search effect on decision thresholds produced by a visual neglect of noncontrast regions on contrast examinations. This suggests that interventions that direct visual search do not offer protection against satisfaction of search effects that are based on faulty visual search.

An empirical comparison of discrete ratings and subjective probability ratings

RATIONALE AND OBJECTIVES

The authors compared receiver operating characteristic (ROC) data from a five-category discrete scale with that from a 101-category subjective probability scale to determine how well the latter categories define the ROC curve.

MATERIALS AND METHODS

The authors analyzed data from a pilot study performed for another purpose in which 10 radiologists provided both a five-point confidence rating and a subjective probability rating of abnormality for each interpretation. ROC operating points were plotted for a five-category scale and a 101-category scale to determine how well the observed points covered the range of false-positive probabilities. ROC curves were fitted to the subjective probability data according to the standard ROC model.

RESULTS

For these data, subjective probability ratings were somewhat more effective in populating the range of false-positive probability with ROC points. For three observers, the ROC curves inappropriately crossed the chance line. For another four, prevention of such crossing seemed to depend on one or two ROC points near the upper right corner of the ROC space, points based on discriminations within the discrete category "no abnormality to report."

CONCLUSION

Subjective probability rating should provide substantially better coverage of the ROC space with operating points, preventing inappropriate crossing of the chance line. Unfortunately, the protection offered by subjective probability ratings was unreliable and depended on ROC points derived from discriminations not directly related to apparent abnormality. The use of proper ROC models to fit data may offer a better solution.

Gaze dwell times on acute trauma injuries missed because of satisfaction of search

RATIONALE AND OBJECTIVES

The authors performed this study to determine whether satisfaction of search (SOS) errors in patients with multiple traumas are caused by faulty visual scanning, faulty recognition, or faulty decision making.

MATERIALS AND METHODS

A series of radiographs were obtained in patients with multiple traumas. Radiologists interpreted each series under two experimental conditions: when the first radiograph in the series included a fracture, and when it did not. In the first experiment, the initial radiographs showed nondisplaced fractures of the extremities (minor fractures); in the second experiment, the initial radiographs showed abnormalities of greater clinical importance (major fractures). Each series also included a radiograph with a subtle (test) fracture and a normal radiograph on which detection accuracy was measured. In each experiment, gaze dwell time was recorded as 10 radiologists reviewed images from 10 simulated cases of multiple trauma.

RESULTS

An SOS effect could be demonstrated only in the second experiment. Analysis of dwell times showed that search on subsequent radiographs was shortened when the initial radiograph contained a fracture; however, the errors were not based on faulty scanning.

CONCLUSION

The SOS effect in musculoskeletal trauma is not caused by faulty scanning. Demonstration of an SOS effect on test fractures with major but not minor additional fractures suggests that detection of other fractures is inversely related to the severity of the detected fracture.

Hepatic imaging in stage IV-S neuroblastoma

Stage IV-S neuroblastoma describes a group of infants with tumor spread limited to liver, skin, or bone marrow. Such patients, who constitute about 25% of affected infants with neuroblastoma, may expect spontaneous tumor remission. We report 18 infants with Stage IV-S neuroblastoma, 83% of whom had liver involvement. Imaging investigations included Technetium 99m sulfur colloid scan, ultrasound, and CT. Two patterns of liver metastasis were noted: ill-defined nodules or diffuse tumor throughout the liver. Distinction of normal and abnormal liver with diffuse type metastasis could be quite difficult, particularly with liver scans. We conclude that patients with Stage IV-S neuroblastoma have ultrasound or CT examination as an initial workup, with nuclear medicine scans reserved for followup studies.

Comparison of a PACS workstation with conventional film for interpretation of neonatal examinations: a paired comparison study

The diagnostic value of neonatal examinations using picture archiving and communication systems (PACS) was compared with that of conventional radiographs. A total of 202 consecutive chest or abdominal radiographs from the newborn intensive care unit were digitized for display on a commercially available PACS console. Experimental design was a paired comparison study. Plain films and PACS images were reviewed alternately in unbiased fashion. After the examination was evaluated using the second modality, any change in diagnosis or confidence in diagnosis was noted. Overall evaluation showed slight preference for the PACS modality. Change of diagnosis or in confidence of diagnosis was more than twice as likely to occur with evaluation of PACS (35%) after hardcopy than with evaluation of conventional radiographs (14%) after PACS. Of the variety of image processing features available on PACS, only window and leveling were judged to be of significant value. These results indicate that PACS and conventional radiographs of the neonatal chest and abdomen are of similar diagnostic value.

Nonoperative treatment of simple meconium ileus: a survey of the Society for Pediatric Radiology

To determine the practice pattern regarding nonoperative treatment of simple meconium ileus, a survey was sent to directors of 66 pediatric radiology departments in the USA and Canada. Thirty-nine responses were received reporting 1,236 patients. A wide variation of enema techniques and contrast media were used. While the success rate did not correlate with osmolality, mode of administration, catheter size, or perforation rate, there was a significantly higher overall success rate with the use of Gastrografin versus non-Gastrografin (p < 0.00076) and the use of additives such as Tween-80 and Mucomyst versus techniques without additives (p < 0.00001). Perforation did not correlate with success rate, osmolality, or type of contrast medium. The only two instances of rectal perforations were associated with the use of balloon-tip catheter.

Teleradiology for rural hospitals: analysis of a field study

We compared the accuracy of a low-cost teleradiology system with plain film at a small rural hospital. The comparison was a case-control, paired-comparison study. In total 377 consecutive cases were read prospectively by teleradiology and later by independent interpretation of the plain films. 'Truth' was determined in discrepant cases by further investigation of available records and images. Sensitivity and specificity were determined for each modality, and agreement using the kappa statistic. There was 90% agreement between teleradiology and plain film, with no significant differences. Sensitivities (0.88, 0.89) and specificities (0.98, 0.98) of the two methods were almost identical. McNemar's test indicated no significant differences in the accuracy of the two modalities. We conclude that inexpensive teleradiology for small rural hospitals is equivalent to plain film for radiologists' interpretation.

Teleradiology services for a rural hospital: a case study

We investigated the perceived quality of teleradiology services offered at a rural hospital in comparison with radiology provided by three other methods: 'circuit riding', an on-site radiologist, and a hybrid arrangement. The research design was a case study, with interviews of administrators, technologists and physicians at the rural hospital, followed by a structured survey of all staff physicians. Responses were analysed both qualitatively and quantitatively. Both interviews and the survey indicated that teleradiology was perceived to be as accurate as on-site film interpretation. All other aspects of the service--efficiency, reports, communications and the overall contribution to patient care--were judged to be poorer than on-site radiology. We conclude that the provision of acceptable teleradiology requires considerable attention to all aspects of the radiology service, with attention to differences in institutional culture and mission, and to communication.

Contrast enema diagnosis of necrotizing enterocolitis

Contrast enema was performed in a select group of 126 neonates with ambiguous diagnosis of necrotizing enterocolitis (NEC). Enema findings were compared with that on plain abdominal radiographs (AXR) and the clinical outcome. Thirty-one of the 34 patients with a discharge diagnosis of NEC were interpreted to have a positive contrast enema, and 91 of the 92 patients without a discharge diagnosis of NEC were interpreted to have a negative contrast enema. The AXR was interpreted as positive in 57 patients and as negative or indeterminate in 69. Not a single case of perforation was produced. Judicious use of contrast enema can improve specificity of diagnosis in cases with discordant clinical and plain radiographic findings.

Analysis of bowel perforation in necrotizing enterocolitis

The most severe complication of necrotizing enterocolitis (NEC) is bowel perforation. Identification of neonates at high risk for perforation and optimization of radiologic imaging to identify bowel perforation are necessary to reduce the high mortality rate associated with this catastrophic event. One hundred and fifty-five cases of NEC were seen at our institution during a 5.5-year period. Nineteen (12%) progressed to perforation. A review of surgical findings, autopsy results and radiographs from these patients shows only 63% had radiographic evidence of free air in the peritoneal cavity at the time of perforation. Twenty-one percent had radiographic evidence of ascites but no pneumoperitoneum, and 16% had neither free air nor ascites. Thus purely radiographic criteria for bowel perforation in NEC are imprecise, and paracentesis is mandatory in NEC patients with ascites or clinical findings indicative of peritonitis. Timing of radiographic studies and site of bowel involvement are also important. Seventy-nine percent of perforations occurred by 30 h from confirmation of diagnosis (by clinical or radiographic criteria). Surgery or autopsy revealed involvement of the ileo-cecal region in 89% of cases with the actual site of perforation occurring in this area in 58% of patients.

Subspecialty radiology consultation by interactive telemedicine

This investigation tested the hypothesis that interactive, low-resolution telemedicine is an effective method for radiologists' consultations. The case material consisted of radiographs and digital images from patients with 14 paediatric diseases readily diagnosed by imaging, and 10 matched controls. The original images were first evaluated by a general radiologist, who entered a diagnosis. He or she then discussed the case with a paediatric radiologist who had access to the same images via low-resolution interactive video. Following the consultation the general radiologist could change diagnosis. The experimental subjects were three teams of general and paediatric radiologists. Observer-performance and case-comparison methods were used for analysis. The results showed that the general radiologists' diagnostic accuracy improved after telemedicine consultation with the subspecialist, the area under the ROC curve improving from 0.648 to 0.709. The interactive consultation was judged valuable by all participants. We conclude that low-resolution interactive telemedicine is of value for consultation between generalists and subspecialists in radiology.

Role of faulty decision making in the satisfaction of search effect in chest radiography

RATIONALE AND OBJECTIVES

The authors performed this study to determine whether defective pattern recognition or defective decision making is more to blame for satisfaction of search (SOS) errors in chest radiography.

MATERIALS AND METHODS

Fifty-eight chest radiographs-half of which demonstrated diverse, native abnormalities-were read by 20 observers. The radiographs were read twice, once with and once without the addition of a simulated pulmonary nodule. Observers provided a verbal account of their focus of attention, indicating suspicious features and regions considered during their inspection of the radiograph. Observers also provided a separate account of the abnormalities they would include in a radiologic report.

RESULTS

When the authors considered only those reports that did not refer to the simulated nodules, they found no reduction in the area under the proper receiver operating characteristic (ROC) curves in cases that contained nodules. A smaller SOS effect, however, was demonstrated with analysis of events in which the native abnormality was missed in one condition but not the other. Verbal protocols suggested that the SOS errors were mainly caused by recognition failure rather than faulty decision making.

CONCLUSION

Describing their focus of attention may have prompted observers to inspect the radiographs in a more deliberate, systematic way, thus reducing the SOS effect. More residual SOS errors were caused by defective pattern recognition than by faulty decision making.

Proper ROC analysis and joint ROC analysis of the satisfaction of search effect in chest radiology

RATIONALE AND OBJECTIVES

The purpose of this study was to test whether the satisfaction of search (SOS) effect in chest radiology could be demonstrated with proper receiver operating characteristic (ROC) curves and with joint detection and localization ROC curves.

MATERIAL AND METHODS

Data from an earlier ROC study of SOS in chest radiology were analyzed with three proper ROC models and one ROC model for joint detection and localization. Fits of the models were compared on the basis of likelihood-ratio chi-squared statistics (G2). To examine further the validity of the SOS effect in chest radiology, the authors also replicated the earlier study with a new sample of readers, analyzing the new data with the same methods.

RESULTS

The proper contaminated binormal model fit the data better than the other two proper ROC models. Contaminated binormal analysis of the earlier and the replication experiment demonstrated an SOS effect: a reduction in area under the ROC curve for detection of the native abnormalities with the addition of nodules. Similarly, joint ROC analysis producing curves that appropriately cross the chance line gave similar results.

CONCLUSION

Preventing inappropriate chance line crossing reduces measurement error and provides more powerful statistical tests. Results of both experiments showed that the SOS effect in chest radiology can be demonstrated with ROC methods that avoid inappropriate crossing of the chance line.

Telemedicine for patient consultation: factors affecting use by rural primary-care physicians in Kansas

The purpose of our study was to investigate knowledge of, attitudes to, and use of interactive telemedicine for specialist consultation among rural practitioners Kansas. We interviewed 28 rural primary-care practitioners at seven remote health-care facilities in six locations. Content analyses of the interviews showed universal but superficial knowledge of telemedicine, appreciation of the value of the technology, but relatively low usage of the telemedicine service available (32% of subjects). Physicians did not appear to be afraid of change. Telemedicine usage was not related to the professional characteristics of the physicians. Our findings suggest that further growth in the use of telemedicine will depend on efforts directed towards physicians which are aimed at creating a more user-friendly environment and at accommodating the referral practices of potential users.

Role of faulty visual search in the satisfaction of search effect in chest radiography

RATIONALE AND OBJECTIVES

The authors tested the hypothesis that satisfaction of search effect, which is associated with the failure to detect native chest abnormalities in the presence of simulated nodules, is caused by reduced gaze on the native abnormalities.

MATERIALS AND METHODS

Gaze dwell time of 20 radiologists was recorded for the region around abnormalities on images. Ten radiographs were reviewed, nine of which contained native abnormalities. Each image was seen with and without a simulated nodule.

RESULTS

The decrease in the rate of true-positive findings in the detection of native abnormalities on images that contained simulated nodules confirmed the occurrence of a satisfaction of search effect. Gaze times on native abnormalities (up to the time of report of the abnormalities) were the same for images with nodules in which native abnormalities were missed (gaze time, 9.4 seconds) as they were for images without nodules in which native abnormalities were detected (gaze time, 9.5 seconds). Gaze time on missed native abnormalities was not affected by the presence (7.80 seconds) or absence (7.45 seconds) of nodules.

CONCLUSION

Reduction in gaze dwell time on the missed abnormalities is not the cause of satisfaction of search errors in chest radiographs.

Teleradiology consultation for a rural hospital: patterns of use

RATIONALE AND OBJECTIVES

The authors documented quantitatively how teleradiology was used for contemporaneous consultation with a radiologist by physicians at a small rural hospital.

MATERIALS AND METHODS

The authors collected 31/2 years worth of data at a small hospital on all patients (n = 327) for whom a radiologist's consultation by teleradiology was requested and compared the data with those from a control group of 309 patients whose studies were interpreted at the same hospital.

RESULTS

Teleradiology consultation was requested by the attending physician in 3.6% (519 of 14,586) of all examinations performed during the study period. Physicians requested teleradiology consultation most often for patients with multiple examinations (average 1.59 vs 1.35 for controls). Examinations of infants were 10 times more prevalent in the teleradiology group than in the control group. The indication for most (52%) teleradiology consultations was trauma. Requests for interpretation of spine and abdominal radiographs were relatively more frequent than were those of other studies.

CONCLUSION

Physicians in this rural practice used contemporaneous radiologic consultation for selected specific examinations, with emphasis on examinations for trauma, spine, abdomen, and the infant age group.

Pediatric radiology at a rural hospital: value of teleradiology and subspecialty consultation

OBJECTIVE

We performed a field study and subsequent laboratory investigation of pediatric radiology at a small rural hospital. Our investigation had three components: to describe the characteristics of pediatric radiology in a rural primary care facility, to test the diagnostic accuracy of interpretation of pediatric images transmitted by teleradiology, and to compare relative diagnostic accuracy of general and pediatric radiologists who interpreted pediatric images at a rural institution.

MATERIALS AND METHODS

All 196 pediatric radiographs obtained during a 4-month period comprised the database from which we determined practice characteristics. Reports of 153 unselected cases interpreted by general radiologists using teleradiology were compared with interpretation of the same cases by a pediatric radiologist who interpreted the original radiographs. Discrepant cases were further investigated by a receiver-operating-characteristic curve experiment in which general and pediatric radiologists interpreted each case twice: once viewing teleradiologic images and once viewing the original radiographs at another setting. We then compared interpretive accuracy of observers and techniques.

RESULTS

The pediatric radiographs were predominantly simple examinations for common acute disease, particularly pneumonia and fractures. Discrepancies of interpretation between teleradiology and original radiographs, which occurred in 13% of images, showed no significant difference in accuracy of interpretation for either teleradiologic images or original radiographs. Likewise, we found no significant advantage for accuracy of interpretation by general or pediatric radiologists. Receiver-operating-characteristic analysis of 18 discrepant cases showed slightly increased accuracy for interpretation of original radiographs by pediatric subspecialists.

CONCLUSION

Simple pediatric radiographs obtained at a rural primary care institution and transmitted by teleradiology can be adequately interpreted by general radiologists.

Imaging techniques in the diagnosis of gastrointestinal and hepatic disease

Abdominal imaging continues to grow in importance for diagnosis, anatomic staging, and functional analysis of gastrointestinal ailments. In this review, we attempt to delineate those publications, both original investigations and summary articles, that have advanced our understanding of abdominal imaging in infants and children. Special attention is given to the newer modalities that provide previously unavailable methods to analyze abnormalities. These include Doppler ultrasonography and magnetic resonance imaging as well as CT for abdominal trauma. Interventional radiology techniques useful for treatment of gastrointestinal diseases are also discussed.

Cause of satisfaction of search effects in contrast studies of the abdomen

RATIONALE AND OBJECTIVES

Extraintestinal abnormalities visible without contrast material on abdominal radiographs are reported less frequently when contrast examinations are performed. Gaze dwell time was used to determine whether this difference is due to failure by observers to scan plain-film regions of contrast studies or discounting of plain-film abnormalities that were actually scanned.

METHODS

Patients were included whose contrast studies had elicited the largest reductions in positive responses compared with their plain-film studies in a previous detection experiment. Gaze of 10 radiologists was studied.

RESULTS

Significantly less time was spent gazing at non-contrast regions of contrast examinations than at the corresponding regions of radiographs. Errors with radiographs were based primarily on failures of recognition and decision making, whereas errors with contrast studies were based primarily on faulty scanning.

CONCLUSION

Satisfaction of search errors on contrast examinations are caused by reduction in scanning of noncontrast regions.

Added value of radiologist consultation to family practitioners in the outpatient setting

PURPOSE

To measure the added value of a radiologist's consultation to the interpretation of radiographs previously read by a family practitioner.

MATERIALS AND METHODS

The authors reviewed 1,674 chest and extremity radiographs previously read by a family practitioner and consulting radiologist. The 196 radiographs in which there was a discrepancy between the family practitioner's and radiologist's report were evaluated by a radiologist and family physician not involved in and blinded to the original interpretations. The overall accuracy of the participants was determined and differences statistically quantified.

RESULTS

The overall sensitivity of the radiologists was greater than that of the family practitioners (92% vs 86%); specificity was not significantly different. For extremity examinations, there were no significant differences in accuracy of the radiologists and family practitioners; the sensitivity of radiologists for chest studies was considerably greater (89% vs 80%). Radiologic consultation was of particular value in the detection of pneumonia and masses.

CONCLUSION

At a family practice center, the radiologist's role for extremity radiographs might be limited to individual consultation, with review of all chest radiographs.

Radiologic interpretation by family physicians in an office practice setting

BACKGROUND

Radiology is an integral part of the office practice of many family physicians. Nevertheless, data are sparse on the performance of family physicians in this endeavor. This study investigated the performance of family physicians at interpreting radiographs ordered in a free-standing family practice office.

METHODS

A consecutive series of radiographic studies performed at a family practice office during a 3-year period was surveyed. All radiographic studies included in this analysis (N = 1674) were separately interpreted by the family physician ordering the study and an overreading radiologist. If the interpretations agreed, the studies were accepted as having been correctly interpreted. Cases in which the interpretations disagreed were reexamined.

RESULTS

Family physicians correctly interpreted 92.4% of the radiographic studies (95% confidence interval, 91.0 to 93.6). Their accuracy with extremity films (96.0%) was significantly higher than their accuracy with chest films (89.3%, P < .001). Family physicians were more likely to correctly interpret normal films (95.2%) than abnormal ones (85.9%, P < .001). Thirty-five percent of the cases in which there were differences between family physician and radiologist interpretations were correctly interpreted by family physicians.

CONCLUSIONS

Family physicians showed a high degree of accuracy in radiologic interpretation in an office setting. Chest films were inherently more difficult to interpret than extremity films. Because correct interpretation depends on body part examined and the prevalence of disease, the performance of family physicians will probably vary in different practice settings.

Iowa's National Laboratory for the study of Rural Telemedicine: a description of a work in progress

As the federal administration advances the idea of the "information superhighway," many disciplines are being challenged to find ways to use advanced telecommunications to improve access to information, enhance learning opportunities, and achieve higher levels of international competitiveness. Telemedicine, the use of communications technology in the practice of medicine, may change the way rural health care is provided by improving access to medical information, diagnostic tools, and consultations. The information and health care services required by health care professionals are rapidly changing, and dissemination of this information to isolated practitioners has proven to be difficult. By providing support electronically from a central site, the most current information is more readily available. Using test-bed hospitals in rural and urban settings, the National Library of Medicine-funded National Laboratory for the Study of Rural Telemedicine at the University of Iowa is currently developing the necessary infrastructure to support targeted projects studying how telemedicine applications can be made more effective and readily available.

Influence of clinical history on perception of abnormalities in pediatric radiographs

RATIONALE AND OBJECTIVES

We tested whether having clinical information would improve perception or simply decision making.

METHODS

Sixty-four pediatric chest and abdominal radiographs, half of which had abnormalities, were presented to nine radiologists under one of two conditions. In one condition, history consistent with abnormalities actually present for positive cases was provided for positive and matched negative cases before inspection. In a second condition, this information was provided only after inspection was completed and the radiograph was no longer available. Because detailed visual memory is short-lived, the image information was no longer available when the history was provided after inspection. A control condition measured detection without history.

RESULTS

Detection was significantly better with history provided before inspection. Detection did not differ for history provided after inspection and inspection without history.

CONCLUSION

The only difference between conditions with history was in whether history influenced perception; history affected decision making in both conditions. Clinical history affected perception in interpreting radiographs, not simply decision making.

Cost-effectiveness and safety of selective use of low-osmolality contrast media

RATIONALE AND OBJECTIVES

We estimated our potential institutional savings by switching from universal to selective use of low-osmolality contrast media.

METHODS

A total of 42,598 radiocontrast studies (26,595 with high-osmolality contrast agents and 16,003 with low-osmolality contrast agents [LOCA]) were performed over 42 months. Every radiocontrast reaction was classified and documented. All costs associated with the subsequent treatment of adverse reactions were recorded and added to the cost of contrast agents. The actual cost of contrast agent administration was determined and compared with the theoretical cost of total conversion to LOCA.

RESULTS

The overall rate of adverse reactions was 1.18%. The cost of radiocontrast materials and treatment of adverse reactions was $2,599,593. Over the same period, total conversion to LOCA would have cost $5,968,507.

CONCLUSIONS

Our institution saved an estimated $962,548 annually through the selective use of LOCA. The incremental cost of treating adverse reactions associated with selective use was minimal compared with the cost of total conversion to LOCA.

Satisfaction of search in the detection of plain-film abnormalities in abdominal contrast studies

RATIONALE AND OBJECTIVES

Satisfaction of search (SOS) occurs when a lesion is "missed" after detecting another lesion in the same radiograph. The authors investigated the SOS effect in abdominal contrast studies.

METHODS

The authors measured detection of 23 plain film abnormalities in 43 patients who had plain film and contrast examinations. Each plain-film and contrast study was examined independently by 10 radiologists in two sessions, with receiver operating characteristic (ROC) curve areas estimated with the computer program RSCORE-J (University of Iowa, Iowa City, IA) for each condition.

RESULTS

Observers more often missed plain film abnormalities present on contrast studies but also made fewer false-positive (FP) responses. There was no change in ROC area, but decision criteria grew more conservative.

CONCLUSIONS

The reduction of detecting plain-film abnormalities in contrast examinations differs from the SOS effect of other imaging studies. The reduction in true-positive (TP) and false-positive (FP) rates suggests that a different cause may underlie these misses.

Evaluation of resident applicants by letters of recommendation: a comparison of traditional and behavior-based formats

RATIONALE AND OBJECTIVES

Traditional, narrative letters of recommendation solicited by medical students applying for radiology residency are widely used as a selection tool. Letters of recommendation are considered a source of reliable information about the attitudes and behaviors (non-cognitive variables) of the resident applicant. However, in many instances, this information is not present or is highly encoded and cannot be extracted. This study attempted to document the deficiencies of traditional letters of recommendation and determine the effectiveness of a structured letter of recommendation in obtaining information regarding noncognitive variables.

METHODS

One hundred thirteen randomly selected letters of recommendation were analyzed by two radiologists with experience in residency selection. Deficiencies in inclusion of information or the ability to extract information about noncognitive variables were documented. A standard behavioral assessment was sent to the writers of these letters of recommendation and these results tabulated.

RESULTS

The traditional letters of recommendation were frequently deficient in data regarding the noncognitive variables. In letters that contained such data, two experienced reviewers could not reliably extract the information. The structured form produced clearly identifiable information about the letter writer's assessment of noncognitive variables.

CONCLUSIONS

Traditional letters of information are frequently deficient in data regarding noncognitive variables. A standardized statement is effective in eliciting information on noncognitive variables related to applicant performance.

The influence of clinical history on visual search with single and multiple abnormalities

RATIONALE AND OBJECTIVES

Facilitation of detection by clinical history generally has been found with a single abnormality per image but not with multiple abnormalities. Multiple abnormalities per image can occasion a "satisfaction-of-search" effect in which detection of one lesion is reduced in the presence of other distant lesions. Our experiment studied the combined effect of multiple abnormalities and clinical history on accuracy.

METHODS

Detection of native lesions was measured 1) with histories suggestive of the native abnormality; 2) with these histories and added simulated pulmonary nodules; and 3) with the same added nodules and histories suggestive of metastatic disease. These conditions also were compared with those of a previous experiment that were similar but included no history.

RESULTS

Detection was substantially improved for appropriately prompted abnormalities even in the presence of a pulmonary nodule. In fact, satisfaction of search was not found in the presence of an appropriate history. Detection of unprompted abnormalities was unchanged when prompts indicated other abnormalities actually present. Prompted abnormalities were detected earlier in search.

CONCLUSIONS

History appears to direct perceptual resources to the prompted abnormalities, thereby alleviating satisfaction of search. The presence of nodules yielded a small but consistent reduction in total search time for searches involving false responses, suggesting that satisfaction of search may depend more on reduction in search time than had been indicated by previous research.

Critical performance behaviors of radiology residents: evaluation of two category systems

OBJECTIVE

Previous research has suggested that certain behavioral aspects of job performance are critical for successful performance in a diagnostic radiology residency. We report two studies conducted to determine the long-term stability of critical performance behaviors for radiology residents and refine their definitions.

SUBJECTS AND METHODS

A trained psychology graduate student conducted critical incident interviews with 20 senior faculty in diagnostic radiology. From these interviews, the faculty generated 120 descriptions of exemplary or poor resident performance. These descriptors were then independently sorted by two radiologists into the previously defined categories of behaviors to evaluate consistency of the behaviors. As a second study, the 120 descriptors were sorted into an expanded behavioral definition system and the reproducibility, using the expanded system, was compared with the original results.

RESULTS

The interrater reliability for placing the current incidents in the originally described behavioral categories was good (Cohen's kappa 0.70). The overall distribution of incidents showed strong similarity to the original data. Sorting into the expanded categories improved the Cohen's kappa from 0.70 to 0.92, indicating that the expanded behavioral definitions improved reliability for categorizing a behavioral incident.

CONCLUSION

The critical behaviors necessary for successful performance in diagnostic radiology residency are stable over time. The expanded and refined system of definitions of these behaviors is more efficacious than the original system was.

Evaluation of a digital workstation for interpreting neonatal examinations. A receiver operating characteristic study

RATIONALE AND OBJECTIVES

The purpose of this study is to compare the diagnostic accuracy of interpreting clinical neonatal radiographs using a commercially available digital workstation versus conventional radiographic images.

METHODS

The case sample consists of 100 chest or abdominal radiographs from the neonatal intensive care unit in which diagnosis was confirmed. Four radiologists served as observers. During two initial reading sessions, half of the studies were viewed on digital radiography monitors and the other half by plain film. Observers indicated whether each patient had normal or abnormal findings and their degree of confidence in this judgment. Six weeks later, observers viewed cases on the alternate presentation system. Two statistical analyses were performed: the data from each observer were treated as a separate experiment in the first analysis; the data from all observers were combined using a new method in the second analysis.

RESULTS

No differences between areas under receiver operating characteristic (ROC) curves for viewing on the picture archiving and communication system (PACS) console and plain film were found for any observer (0.86 versus 0.86, 0.89 versus 0.86, 0.88 versus 0.85, 0.83 versus 0.82).

CONCLUSIONS

The study suggests that for pediatric plain film images, video images offer diagnostic information comparable with that of conventional radiographs for neonatal examinations.

The predictive utility of behavior-based interviewing compared with traditional interviewing in the selection of radiology residents

RATIONALE AND OBJECTIVES

This study compares the predictive use of measures based on traditional faculty and resident interviews of residency applicants with measures obtained through behavior-based interviewing. A special emphasis was placed on predicting residents' noncognitive abilities.

METHODS

One hundred fifty-one resident applicants, over a 3-year period, were interviewed using standard interviews by faculty and residents. These residents also were interviewed with an experimental behavior-based accomplishment interview. Four years later, during their diagnostic radiology residency, evaluations of performance were gathered on these applicants from their residency director.

RESULTS

Results indicated that scores based on responses given during the accomplishment interviews added considerable predictive utility to the low prediction demonstrated by traditional interviews.

CONCLUSIONS

These findings imply that improving unstructured faculty and resident interviews to obtain, in a more rigorous manner, desired information about noncognitive abilities may be a key to successful resident selection.

Error in radiology: classification and lessons in 182 cases presented at a problem case conference

The authors review and classify errors in 182 cases that were presented at problem case conferences between August 1986 and October 1990. Errors were classified by means of a system developed 20 years ago and by means of a system developed within the past several years. The authors found that sources of error have changed very little. Errors usually involved failure to consult old radiologic studies or reports, limitations in imaging technique, acquisition of inaccurate or incomplete clinical history, location of a lesion outside the area of interest on an image, lack of knowledge, failure to continue to search for abnormalities after the first abnormality was found, and failure to recognize a normal biologic variant. Errors included 126 perceptual errors (64 false-negative, 15 false-positive, and 47 misclassification errors) and 56 mishaps, including 38 complications and 18 communication errors. In seven cases nonperception errors occurred because established departmental routines were not followed, and in nine cases a new departmental routine was established after a complication occurred. Departmental policy exerts less effect on perception and interpretation errors.

Time course of satisfaction of search

"Satisfaction of search" (SOS) refers to the effect in which a second lesion remains undetected after detection of another lesion on the same radiograph. The objective of this study was to clarify our understanding of SOS by relating it to total time of inspection and time intervals before, between, and after discovery of lesions. Detection accuracy of native lesions in chest radiographs, before and after the addition of a simulated nodular lesion, was measured for ten observers. Analysis of data from this and a previous experiment showed that average perceptual accuracy of individual receiver operating characteristic curves was significantly reduced with the addition of the nodules. Plots and analyses of search time revealed that, on average, during a typical 46-second inspection of a case, simulated nodules were found at 18 seconds, native abnormalities at 25 seconds, and false positives occurred at 33 seconds. Time needed to find nodules did not depend on whether native lesions were present; time to find native lesions did not change with addition of nodules; and total search time was the same for images with one, two, or no lesions. The detection results show that the SOS effect was obtained, but that interrupting search in order to measure it also diminishes accuracy. Analysis of the time course data relates SOS to perceptual capture and strategic halting of search.