1
|
Haygood TM, Smith S, Sun J. Memory bias in observer-performance literature. J Med Imaging (Bellingham) 2018; 5:031412. [PMID: 30840725 PMCID: PMC6152535 DOI: 10.1117/1.jmi.5.3.031412] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2017] [Accepted: 08/23/2018] [Indexed: 11/16/2022] Open
Abstract
The objective of our study was to determine how authors of published observer–performance experiments dealt with memory bias in study design. We searched American Journal of Roentgenology online and Radiology using “observer study” and “observer performance.” We included articles from 1970 or later that reported an observer performance experiment using human observers. We recorded the methods used by the authors to order presentation of the conditions being tested and images within sets for viewing. We recorded use and length of any time gap between viewings. We included 110 experiments. Forty-five used methods not subject to memory bias. Of 68 remaining experiments, 30 (44.1%) ordered the viewing of tested conditions to decrease memory bias. Fifteen (22.1%) ordered the tested conditions in ways that may create memory bias. Eleven (16.2%) intermixed the tested conditions. Forty-three (63.2%) used random or pseudorandom ordering of images within sets. Forty-six (67.6%) used a time gap (median 14 days) between viewings. Six (8.8%) did not use a time gap. Thirty-six (52.9%) did not indicate what methods they used in at least one studied parameter. Therefore, we conclude that 22.1% of the experiments could improve their methods of ordering tested conditions. Completeness of reporting could be improved by including more details regarding methods of ameliorating memory bias.
Collapse
Affiliation(s)
- Tamara Miner Haygood
- University of Texas MD Anderson Cancer Center, Department of Diagnostic Radiology, Houston, Texas, United States
| | - Samantha Smith
- University of Texas MD Anderson Cancer Center, Department of Diagnostic Radiology, Houston, Texas, United States
| | - Jia Sun
- University of Texas MD Anderson Cancer Center, Department of Biostatistics, Houston, Texas, United States
| |
Collapse
|
2
|
Conradie A, Herbst CP. Evaluating the effect of reduced entrance surface dose on neonatal chest imaging using subjective image quality evaluation. Phys Med 2016; 32:1368-1374. [PMID: 27542577 DOI: 10.1016/j.ejmp.2016.07.005] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/29/2015] [Revised: 07/07/2016] [Accepted: 07/08/2016] [Indexed: 10/21/2022] Open
Abstract
INTRODUCTION As hospitalized neonates are radiosensitive and often require numerous X-rays, institutions should investigate the optimal beam parameter combinations to deliver diagnostically acceptable quality images at the lowest possible entrance surface dose (ESD). Using a subjective approach, this study evaluated the effect of different beam parameter combinations on image quality. METHODS AND MATERIALS Five rabbits simulated the neonatal chest. The ESD was reduced using a variation of voltage, tube current and filtration. Eight radiology registrars, blinded to the dose parameter information, ranked the digital X-ray images of three anatomical regions from best to worst using a variation of the multiple rank order method. T-tests compared the average values, obtained from the scores assigned by each observer to images acquired at different ESDs, for each region. The calculated intraclass correlation coefficient (ICC) assessed observer agreement. RESULTS Results showed that a 64% dose reduction was achievable by altering the beam parameters. Large variation among the observers was confirmed by an ICC value <0.5. A 95% confidence interval could not conclude that different ESD values, resulting in a 50-77% dose reduction compared to current practice, would result in different overall observed image quality. This was noted for all three regions indicating that no preference existed towards an image acquired with a specific beam parameter combination. CONCLUSIONS The large variation in observers' opinion of acceptable image quality emphasizes the importance of subjective image quality evaluation in the clinical environment. The rabbit phantom and multiple rank order method are considered appropriate for these evaluations.
Collapse
Affiliation(s)
- Anita Conradie
- Department of Medical Physics, University of the Free State, Bloemfontein 9300, South Africa.
| | - Charles Petrus Herbst
- Department of Medical Physics, University of the Free State, Bloemfontein 9300, South Africa.
| |
Collapse
|
3
|
Bowman A, Shunmugam M, Watts AR, Bramwell DC, Wilson C, Krishnan J. Inter-observer and intra-observer reliability of mechanical axis alignment before and after total knee arthroplasty using long leg radiographs. Knee 2016; 23:203-8. [PMID: 26796777 DOI: 10.1016/j.knee.2015.11.013] [Citation(s) in RCA: 27] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/17/2015] [Revised: 11/26/2015] [Accepted: 11/29/2015] [Indexed: 02/02/2023]
Abstract
BACKGROUND Long leg radiographs (LLRs) are commonly performed for assessment of mechanical alignment and operative planning in patients undergoing total knee arthroplasty (TKA). The aim of this study was to determine the inter- and intra-observer reliability of alignment measured by observers of different levels of experience. METHODS Forty patients on the waiting list for a TKA had pre- and post-operative standardised LLRs. We analysed the measurements of mechanical axis alignment between an orthopaedic surgeon, a senior orthopaedic registrar, a junior orthopaedic registrar, and a medical student. Reviewers performed blinded measurements on the same computer screen. These measurements were repeated three months later to assess intra-observer reliability. Furthermore high-resolution screens were compared with standard hospital computer screens to investigate whether monitoring quality influenced the accuracy of measurements of alignment. RESULTS Inter-observer reliability was high for pre-operative LLRs with an intra-class correlation (ICC) of >0.9 at all experience levels. Post-operative ICC was lowest between the surgeon and the medical student at 0.7. Intra-observer reliability was high at all experience levels. Larger deformities appeared to have exaggerated measurements for both pre- and post-operative images. There appeared to be no effect of the monitor size and quality on the accuracy of measurement. CONCLUSIONS Long leg radiographs can be used to measure mechanical axis alignment with strong reliability across different levels of experience. This information is important for the evaluation of knee alignment measurements in current clinical practice, to assess severity of deformity and to accompany pre-operative planning and post-operative evaluation. LEVEL OF EVIDENCE III.
Collapse
Affiliation(s)
- Anneka Bowman
- Department of Orthopaedics, School of Medicine, Flinders University, Adelaide, Australia; International Musculoskeletal Research Institute, Repatriation General Hospital, Daws Road, Daw Park, SA, Australia
| | - Meenalochani Shunmugam
- Department of Orthopaedics, School of Medicine, Flinders University, Adelaide, Australia
| | - Amy R Watts
- Department of Orthopaedics, School of Medicine, Flinders University, Adelaide, Australia; International Musculoskeletal Research Institute, Repatriation General Hospital, Daws Road, Daw Park, SA, Australia.
| | - Donald C Bramwell
- Department of Orthopaedics, School of Medicine, Flinders University, Adelaide, Australia; International Musculoskeletal Research Institute, Repatriation General Hospital, Daws Road, Daw Park, SA, Australia
| | - Christopher Wilson
- Department of Orthopaedics, School of Medicine, Flinders University, Adelaide, Australia; International Musculoskeletal Research Institute, Repatriation General Hospital, Daws Road, Daw Park, SA, Australia
| | - Jeganath Krishnan
- Department of Orthopaedics, School of Medicine, Flinders University, Adelaide, Australia; International Musculoskeletal Research Institute, Repatriation General Hospital, Daws Road, Daw Park, SA, Australia
| |
Collapse
|
4
|
Vasconcelos TV, Santaella GM, Nascimento HAR, Rovaris K, Ambrosano GMB, Freitas DQ. Digital radiographs displayed on different devices: effect on the detection of vertical root fractures. Int Endod J 2015; 49:386-92. [DOI: 10.1111/iej.12466] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2015] [Accepted: 05/13/2015] [Indexed: 11/29/2022]
Affiliation(s)
- T. V. Vasconcelos
- Division of Oral Radiology; Department of Oral Diagnosis; State University of Campinas; Piracicaba Brazil
| | - G. M. Santaella
- Division of Oral Radiology; Department of Oral Diagnosis; State University of Campinas; Piracicaba Brazil
| | - H. A. R. Nascimento
- Division of Oral Radiology; Department of Oral Diagnosis; State University of Campinas; Piracicaba Brazil
| | - K. Rovaris
- Division of Oral Radiology; Department of Oral Diagnosis; State University of Campinas; Piracicaba Brazil
| | - G. M. B. Ambrosano
- Division of Bioestatistcs; Department of Community Dentistry; Piracicaba Dental School; State University of Campinas; Piracicaba Brazil
| | - D. Q. Freitas
- Division of Oral Radiology; Department of Oral Diagnosis; State University of Campinas; Piracicaba Brazil
| |
Collapse
|
5
|
van Zyl S, Kekana R. Comparison of the perceived image quality between two digital imaging systems for neonatal bedside radiography – A case study. Radiography (Lond) 2015. [DOI: 10.1016/j.radi.2014.10.005] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/01/2022]
|
6
|
Han TS, Kwack KS, Park S, Min BH, Yoon SH, Lee HY, Lee KB. A superficial hyperechoic band in human articular cartilage on ultrasonography with histological correlation: preliminary observations. Ultrasonography 2014; 34:115-24. [PMID: 25656333 PMCID: PMC4372710 DOI: 10.14366/usg.14047] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2014] [Revised: 12/27/2014] [Accepted: 12/28/2014] [Indexed: 12/02/2022] Open
Abstract
Purpose: To demonstrate the superficial hyperechoic band (SHEB) in articular cartilage by using ultrasonography (US) and to assess its correlation with histological images. Methods: In total, 47 regions of interest (ROIs) were analyzed from six tibial osteochondral specimens (OCSs) that were obtained after total knee arthroplasty. Ultrasonograms were obtained for each OCS. Then, matching histological sections from all specimens were obtained for comparison with the ultrasonograms. Two types of histological staining were used: Safranin-O stain (SO) to identify glycosaminoglycans (GAG) and Masson’s trichrome stain (MT) to identify collagen. In step 1, two observers evaluated whether there was an SHEB in each ROI. In step 2, the two observers evaluated which histological staining method correlated better with the SHEB by using the ImageJ software. Results: In step 1 of the analysis, 20 out of 47 ROIs showed an SHEB (42.6%, kappa=0.579). Step 2 showed that the SHEB correlated significantly better with the topographical variation in stainability in SO staining, indicating the GAG distribution, than with MT staining, indicating the collagen distribution (P<0.05, kappa=0.722). Conclusion: The SHEB that is frequently seen in human articular cartilage on high-resolution US correlated better with variations in SO staining than with variations in MT staining. Thus, we suggest that a SHEB is predominantly related to changes in GAG. Identifying an SHEB by US is a promising method for assessing the thickness of articular cartilage or for monitoring early osteoarthritis.
Collapse
Affiliation(s)
- Tae Sun Han
- Department of Radiology, Ajou University School of Medicine, Suwon, Korea ; Musculoskeletal Imaging Laboratory, Ajou University Medical Center, Suwon, Korea
| | - Kyu-Sung Kwack
- Department of Radiology, Ajou University School of Medicine, Suwon, Korea ; Musculoskeletal Imaging Laboratory, Ajou University Medical Center, Suwon, Korea
| | - Sunghoon Park
- Department of Radiology, Ajou University School of Medicine, Suwon, Korea ; Musculoskeletal Imaging Laboratory, Ajou University Medical Center, Suwon, Korea
| | - Byoung-Hyun Min
- Department of Orthopedic Surgery, Ajou University School of Medicine, Suwon, Korea ; Cartilage Regeneration Center, Ajou University Medical Center, Suwon, Korea
| | - Seung-Hyun Yoon
- Cartilage Regeneration Center, Ajou University Medical Center, Suwon, Korea
| | - Hyun Young Lee
- Regional Clinical Trial Center, Ajou University Medical Center, Suwon, Korea ; Department of Biostatistics, Yonsei University College of Medicine, Seoul, Korea
| | - Kyi Beom Lee
- Department of Pathology, Ajou University School of Medicine, Suwon, Korea
| |
Collapse
|
7
|
The influence of ambient lighting on the detection of small contrast elements in digital dental radiographs. Clin Oral Investig 2012; 17:1727-31. [DOI: 10.1007/s00784-012-0858-2] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2012] [Accepted: 10/01/2012] [Indexed: 11/25/2022]
|
8
|
Yin J, Guo Q, Zhang W, Su H, Zhang J, Yue Y, Ding C, Lin A, Wang Y, Wang H. Effect of greyscale liquid crystal displays of different resolutions on observer performance during detection of small solitary pulmonary nodules. Br J Radiol 2012; 85:e549-55. [PMID: 22744323 DOI: 10.1259/bjr/19867962] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022] Open
Abstract
OBJECTIVE The aim of this study was to evaluate the effect of monochrome liquid crystal displays (LCDs) with different resolutions on observer performance during detection of small solitary pulmonary nodules. METHODS Chest images of digital radiography were selected online from the hospital's picture archiving and communication system. Of the 164 images selected, small solitary non-calcified pulmonary nodules were present in 63 images and absent in 101 images. Observer performance was assessed among 3 extremely experienced, 3 very experienced and 3 moderately experienced radiologists, who independently interpreted these images on 2, 3 and 5 megapixel greyscale LCDs. A five-point confidence level rating scale was used to represent the presence of nodules: definite absence, probable absence, indetermination, probable presence and definite presence. The observers were requested to rank each image on the given display according to the presence of the pulmonary nodule. Observer performance was analysed in terms of receiver operating characteristics (ROCs). RESULTS The areas under the ROC curves which represented the observer performance for the 2, 3 and 5 megapixel LCDs were found to be 0.705, 0.722 and 0.764, respectively, for the extremely experienced radiologists; 0.687, 0.712 and 0.721, respectively, for the very experienced radiologists; and 0.689, 0.696 and 0.711, respectively, for the moderately experienced radiologists. These differences were not statistically significant. CONCLUSION The observer performances for detection of small solitary non-calcified pulmonary nodules by radiologists with varying degrees of experience were comparable between the 2, 3 and 5 megapixel monochrome LCDs.
Collapse
Affiliation(s)
- J Yin
- Department of Radiology, Shengjing Hospital of China Medical University, Shenyang, Liaoning, China
| | | | | | | | | | | | | | | | | | | |
Collapse
|
9
|
A guide to stereoscopic 3D displays in medicine. Acad Radiol 2011; 18:1035-48. [PMID: 21652229 DOI: 10.1016/j.acra.2011.04.005] [Citation(s) in RCA: 61] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2011] [Revised: 04/08/2011] [Accepted: 04/22/2011] [Indexed: 01/21/2023]
Abstract
Stereoscopic displays can potentially improve many aspects of medicine. However, weighing the advantages and disadvantages of such displays remains difficult, and more insight is needed to evaluate whether stereoscopic displays are worth adopting. In this article, we begin with a review of monocular and binocular depth cues. We then apply this knowledge to examine how stereoscopic displays can potentially benefit diagnostic imaging, medical training, and surgery. It is apparent that the binocular depth information afforded by stereo displays 1) aid the detection of diagnostically relevant shapes, orientations, and positions of anatomical features, especially when monocular cues are absent or unreliable; 2) help novice surgeons orient themselves in the surgical landscape and perform complicated tasks; and 3) improve the three-dimensional anatomical understanding of students with low visual-spatial skills. The drawbacks of stereo displays are also discussed, including extra eyewear, potential three-dimensional misperceptions, and the hurdle of overcoming familiarity with existing techniques. Finally, we list suggested guidelines for the optimal use of stereo displays. We provide a concise guide for medical practitioners who want to assess the potential benefits of stereo displays before adopting them.
Collapse
|
10
|
Awan O, Safdar NM, Siddiqui KM, Moffitt R, Siegel EL. Detection of cervical spine fracture on computed radiography images a monitor resolution study. Acad Radiol 2011; 18:353-8. [PMID: 21215662 DOI: 10.1016/j.acra.2010.11.011] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2010] [Revised: 11/11/2010] [Accepted: 11/11/2010] [Indexed: 11/24/2022]
Abstract
RATIONALE AND OBJECTIVES The purpose of this study was to evaluate the diagnostic accuracy of radiologists using monochrome medical-grade 5 megapixel (MP), 3 MP, 2 MP, and 1 MP displays for the detection of cervical fractures on cervical radiographs, while controlling factors such as luminance and ambient conditions. MATERIALS AND METHODS Institutional review board approval was obtained. Two hundred lateral cervical computed radiography images, 97 with fractures, were randomly displayed on 5-MP, 3-MP, 2-MP, or 1-MP liquid crystal displays (LCDs) for a total of 450 interpretations per display. These radiographs were presented in eight sessions, each with 25 radiographs, to nine readers. The reference standard for all cases was computed tomography. Ambient lighting, monitor luminance, and gamma were controlled throughout the study. Measures included receiver operator characteristic areas under the curve (AUC), sensitivity, specificity, and accuracy, mean elapsed time by display, and mean confidence level by display. One way analysis of variance was performed. Results were considered to be significant at an alpha level of 0.05. RESULTS AUCs were 0.76 (95% CI, 0.72-0.80) for the 1 MP, 0.80 (95% CI, 0.76-0.84) for the 2 MP, 0.77 (95% CI, 0.73-0.81) for the 3 MP, and 0.76 (95% CI, 0.72-0.80) for the 5 MP medical grade LCDs. There was no significant difference in the AUCs (P values between .0651 and .8693), confidence (P = .158), or interpretation times (P = .751). CONCLUSION When controlling factors such as luminance and ambient light, a difference in accuracy in the detection of cervical fractures by resolution could not be detected when using medical-grade displays. Interpretation time and confidence were also not affected by resolution.
Collapse
|
11
|
DR and CR: Recent advances in technology. Eur J Radiol 2009; 72:194-201. [DOI: 10.1016/j.ejrad.2009.05.055] [Citation(s) in RCA: 24] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2009] [Accepted: 05/07/2009] [Indexed: 11/24/2022]
|
12
|
Sung YM, Chung MJ, Lee KS, Choe BK. The influence of liquid crystal display monitors on observer performance for the detection of interstitial lung markings on both storage phosphor and flat-panel-detector chest radiography. Eur J Radiol 2009; 74:275-9. [PMID: 19304429 DOI: 10.1016/j.ejrad.2009.02.009] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2008] [Revised: 10/14/2008] [Accepted: 02/13/2009] [Indexed: 11/25/2022]
Abstract
PURPOSE To compare observer performance with a flat-panel liquid crystal display (LCD) monitor and with a high-resolution gray-scale cathode ray tube (CRT) monitor in the detection of interstitial lung markings using a silicon flat-panel-detector direct radiography (DR) and storage phosphor computed radiography (CR) in a clinical setting. MATERIALS AND METHODS We displayed 39 sets of posteroanterior chest radiographs from the patients who were suspected of interstitial lung disease. Each sets consisted of DR, CR and thin-section CT as the reference standard. Image identities were masked, randomly sorted, and displayed on both five mega pixel (2048x2560x8 bits) LCD and CRT monitors. Ten radiologists independently rated their confidence in detection for the presence of linear opacities in the four fields of the lungs; right upper, left upper, right lower, and left lower quadrant. Performance of a total 6240 (39 setsx2 detector systemsx2 monitor systemx4 fieldsx10 observers) observations was analyzed by multi-reader multi-case receiver operating characteristic (ROC) analysis. Differences between monitor systems in combinations of detector systems were compared using ANOVA and paired-samples t-test. RESULTS Area under curves (AUC) for the presence of linear opacities measured by ROC analysis was higher on the LCDs than CRTs without statistical significance (p=0.082). AUC was significantly higher on the DR systems than CR systems (p=0.006). AUC was significantly higher on the LCDs than CRTs for DR systems (p=0.039) but not different for CR systems (p=0.301). CONCLUSION In clinical conditions, performance of the LCD monitor appears to be better for detecting interstitial lung markings when interfaced with DR systems.
Collapse
Affiliation(s)
- Yon Mi Sung
- Department of Radiology and Center for Imaging Science, Samsung Medical Center, Sungkyunkwan University School of Medicine, #50, Ilwon-dong, Kangnam-ku, Seoul 135-710, Republic of Korea
| | | | | | | |
Collapse
|
13
|
Sorantin E. Soft-copy display and reading: what the radiologist should know in the digital era. Pediatr Radiol 2008; 38:1276-84. [PMID: 18548242 DOI: 10.1007/s00247-008-0898-6] [Citation(s) in RCA: 12] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/29/2008] [Accepted: 02/03/2008] [Indexed: 11/28/2022]
Abstract
The basic scheme of modern medical imaging consists of the following steps: (1) data acquisition, (2) data (image) processing, (3) further data processing by the computer graphic system, (4) image display, and finally (5) visual perception and interpretation. Medical monitors represent a forgotten part of the whole chain. The purpose of this review is to present the key facts about the display technologies used today, as well as their characterization and quality assurance.
Collapse
Affiliation(s)
- Erich Sorantin
- Division of Paediatric Radiology, Department of Radiology, Medical University Graz, Auenbruggerplatz 34, A 8036 Graz, Austria.
| |
Collapse
|
14
|
Buls N, Shabana W, Verbeek P, Pevenage P, De Mey J. Influence of display quality on radiologists' performance in the detection of lung nodules on radiographs. Br J Radiol 2007; 80:738-43. [PMID: 17709363 DOI: 10.1259/bjr/48049509] [Citation(s) in RCA: 13] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022] Open
Abstract
The purpose of this study was to evaluate the influence of display quality on radiologists' performance in the detection of lung nodules. Display systems with various technical properties were considered based on their general availability in a radiology department. Their quality was assessed by physical tests. Multireader-multicase receiver operating characteristic (ROC) analysis was used to evaluate observer performance. The area under the curve (Az) was used as a metric for detectability of simulated lung nodules with diameters of 5 mm and 10 mm, and peak contrast values ranging from 0.1 (subtle) to 0.4 (evident) that were digitally superimposed on normal chest radiographs. Three experienced radiologists interpreted a batch of 60 radiographs on five different display systems; four monitors (two liquid crystal display (LCD) and two cathode ray tube (CRT) monitors) and one printed hardcopy. The physical tests showed superior performance of the two LCD monitors. ROC analysis resulted in the following Az scores: LCD-5MP Az = 0.78, hardcopy Az = 0.77, LCDc-2MP Az = 0.75, CRT-5MP Az = 0.72 and CRTc-1MP Az = 0.71. Difference in Az scores between the LCD-5MP monitor and both the CRT-5MP (p = 0.04) and CRTc-1MP (p = 0.01) monitors was significant. The primary class CRT-5MP monitor that showed reduced observer performance failed to comply with physical acceptance requirements. Luminance response was particularly observed to be insufficient. The results indicate that a quality assurance program has the potential to detect non-optimised display systems that could otherwise result in reduced observer performance.
Collapse
Affiliation(s)
- N Buls
- Universitair Ziekenhuis Brussel (UZ Brussel), Department of Radiology, Laarbeeklaan 101, B-1090, Belgium.
| | | | | | | | | |
Collapse
|
15
|
Zheng B, Mello-Thoms C, Wang XH, Abrams GS, Sumkin JH, Chough DM, Ganott MA, Lu A, Gur D. Interactive computer-aided diagnosis of breast masses: computerized selection of visually similar image sets from a reference library. Acad Radiol 2007; 14:917-27. [PMID: 17659237 PMCID: PMC2043128 DOI: 10.1016/j.acra.2007.04.012] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2007] [Revised: 04/15/2007] [Accepted: 04/18/2007] [Indexed: 10/23/2022]
Abstract
RATIONALE AND OBJECTIVES The clinical utility of interactive computer-aided diagnosis (ICAD) systems depends on clinical relevance and visual similarity between the queried breast lesions and the ICAD-selected reference regions. The objective of this study is to develop and test a new ICAD scheme that aims improve visual similarity of ICAD-selected reference regions. MATERIALS AND METHODS A large and diverse reference library involving 3,000 regions of interests was established. For each queried breast mass lesion by the observer, the ICAD scheme segments the lesion, classifies its boundary spiculation level, and computes 14 image features representing the segmented lesion and its surrounding tissue background. A conditioned k-nearest neighbor algorithm is applied to select a set of the 25 most "similar" lesions from the reference library. After computing the mutual information between the queried lesion and each of these initially selected 25 lesions, the scheme displays the six reference lesions with the highest mutual information scores. To evaluate the automated selection process of the six "visually similar" lesions to the queried lesion, we conducted a two-alternative forced-choice observer preference study using 85 queried mass lesions. Two sets of reference lesions selected by one new automated ICAD scheme and the other previously reported scheme using a subjective rating method were randomly displayed on the left and right side of the queried lesion. Nine observers were asked to decide for each of the 85 queried lesions which one of the two reference sets was "more visually similar" to the queried lesion. RESULTS In classification of mass boundary spiculation levels, the overall agreement rate between the automated scheme and an observer is 58.8% (Kappa = 0.31). In observer preference study, the nine observers preferred on average the reference lesion sets selected by the automated scheme as being more visually similar than the set selected by the subjective rating approach in 53.2% of the queried lesions. The results were not significantly different for the two methods (P = .128). CONCLUSIONS This study suggests that using the new automated ICAD scheme, the interobserver variability related issues can thus be avoided. Furthermore, the new scheme maintains the similar performance level as the previous scheme using the subjective rating method that can select reference sets that are significantly more visually similar (P < .05) than when using traditional ICAD schemes in which the mass boundary spiculation levels are not accurately detected and quantified.
Collapse
Affiliation(s)
- Bin Zheng
- Department of Radiology, Imaging Research Center, University of Pittsburgh, Pittsburgh, PA 15213, USA.
| | | | | | | | | | | | | | | | | |
Collapse
|
16
|
Fukushima H, Ikeda M, Ishigaki T, Usami H, Shimamoto K. Influence of liquid crystal display monitors on observer performance for detection of diffuse pulmonary disease on chest radiographs. ACTA ACUST UNITED AC 2007; 25:211-7. [PMID: 17581709 DOI: 10.1007/s11604-007-0126-6] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2006] [Accepted: 02/02/2007] [Indexed: 10/23/2022]
Abstract
PURPOSE The purpose of this study was to assess the influence of liquid crystal display (LCD) monitors on the detectability of diffuse pulmonary diseases depicted on chest radiographs by comparing them with a high-resolution cathode ray tube (CRT) monitor. MATERIALS AND METHODS A group of 17 radiologists interpreted 87 soft-copy images on LCD monitors with pixel arrays of 1024 x 1280, 1200 x 1600, 1536 x 2048, and 2048 x 2560 and on a CRT monitor with a pixel array of 2048 x 2560. They were asked to indicate their individual confidence levels regarding the presence of diffuse pulmonary diseases. The luminance distributions of all monitors were adjusted to the same distributions, and the ambient illumination was 200 lux. Observer performance was analyzed in terms of the receiver operating characteristics (ROC). RESULTS The average ROC curves for the five monitor types were similar, and there were no statistically reliable effects of the five monitor types on the readers' diagnostic performances (P = 0.7587). CONCLUSION The detectability of diffuse pulmonary disease on the LCD monitors with a spatial resolution equal to or higher than a matrix size of 1024 x 1280 was found to be equivalent to that on the high-resolution CRT monitor.
Collapse
Affiliation(s)
- Hiromichi Fukushima
- Department of Radiology, Nagoya University School of Medicine, 65 Tsurumai-cho, Showa-ku, Nagoya, 466-8560, Japan.
| | | | | | | | | |
Collapse
|
17
|
Sakai S, Soeda H, Takahashi N, Okafuji T, Yoshitake T, Yabuuchi H, Yoshino I, Yamamoto K, Honda H, Doi K. Computer-aided nodule detection on digital chest radiography: validation test on consecutive T1 cases of resectable lung cancer. J Digit Imaging 2007; 19:376-82. [PMID: 16763934 PMCID: PMC3045164 DOI: 10.1007/s10278-006-0626-4] [Citation(s) in RCA: 22] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022] Open
Abstract
PURPOSE To evaluate the usefulness of a commercially available computer-assisted diagnosis (CAD) system on operable T1 cases of lung cancer by use of digital chest radiography equipment. MATERIALS AND METHODS Fifty consecutive patients underwent surgery for primary lung cancer, and 50 normal cases were selected. All cancer cases were histopathologically confirmed T1 cases. All normal individuals were selected on the basis of chest computed tomography (CT) confirmation and were matched with cancer cases in terms of age and gender distributions. All chest radiographs were obtained with one computed radiography or two flat-panel detector systems. Eight radiologists (four chest radiologists and four residents) participated in observer tests and interpreted soft copy images by using an exclusive display system without and with CAD output. When radiologists diagnosed cases as positives, the locations of lesions were recorded on hard copies. The observers' performance was evaluated by receiver operating characteristic analysis. RESULTS The overall detectability of lung cancer cases with CAD system was 74% (37/50), and the false-positive rate was 2.28 (114/50) false positives per case for normal cases. The mean A(z) value increased significantly from 0.896 without CAD output to 0.923 with CAD output (P = 0.018). The main cause of the improvement in performance is attributable to changes from false negatives without CAD to true positives with CAD (19/31, 61%). Moreover, improvement in the location of the tumor was observed in 1.5 cases, on average, for radiology residents. CONCLUSION This CAD system for digital chest radiographs is useful in assisting radiologists in the detection of early resectable lung cancer.
Collapse
Affiliation(s)
- Shuji Sakai
- Department of Health Sciences, School of Medicine, Kyushu University, 3-1-1, Maidashi, Higashi-ku, Fukuoka, 812-8582, Japan.
| | | | | | | | | | | | | | | | | | | |
Collapse
|
18
|
Sakai S, Soeda H, Furuya A, Yabuuchi H, Okafuji T, Yamamoto K, Honda H, Doi K. Evaluation of the image quality of temporal subtraction images produced automatically in a PACS environment. J Digit Imaging 2007; 19:383-90. [PMID: 16741663 PMCID: PMC3045158 DOI: 10.1007/s10278-006-0625-5] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022] Open
Abstract
The aim of the study is to evaluate the reliable production of temporal subtraction images in a picture archiving and communication system environment and to establish objective criteria for the evaluation of image quality. A total of 117 temporal subtraction chest images (55 in the upright position, 62 in the supine position) were obtained in five consecutive days. In all of these, we confirmed that there were no interval changes on the original images, and cases with diffuse lung disease were excluded. The temporal subtraction images were classified by three chest radiologists into five levels: 5, excellent; 4, good; 3, acceptable; 2, poor; and 1, very poor. The following were examined: (1) the yield of adequate quality of the temporal subtraction images; (2) whether the temporal subtraction images were obtained in the warping or nonwarping mode; and (3) the correlation of the overall subjective image quality with the relative shift angles, relative shift distances, and the standard deviation of gray levels in the temporal subtraction images. The percentages of acceptable temporal subtraction images were 100% and 66% in the upright and supine positions, respectively. Sixteen (26%) of the 62 supine-position images were made in nonwarping mode, whereas all upright images were made in warping mode. Significant correlations were obtained in the relative shift angle (P < 0.05), relative horizontal shift distance (P < 0.05), and standard deviation of gray levels (P < 0.0001). Temporal subtraction images with acceptable image quality were obtained in the upright position. The objective criteria may be useful for the evaluation of image quality.
Collapse
Affiliation(s)
- Shuji Sakai
- Department of Health Sciences, School of Medicine, Kyushu University, 3-1-1, Maidashi, Higashi-ku, Fukuoka, 812-8582, Japan.
| | | | | | | | | | | | | | | |
Collapse
|
19
|
Abstract
There have been many remarkable advances in conventional thoracic imaging over the past decade. Perhaps the most remarkable is the rapid conversion from film-based to digital radiographic systems. Computed radiography is now the preferred imaging modality for bedside chest imaging. Direct radiography is rapidly replacing film-based chest units for in-department posteroanterior and lateral examinations. An exciting aspect of the conversion to digital radiography is the ability to enhance the diagnostic capabilities and influence of chest radiography. Opportunities for direct computer-aided detection of various lesions may enhance the radiologist's accuracy and improve efficiency. Newer techniques such as dual-energy and temporal subtraction radiography show promise for improved detection of subtle and often obscured or overlooked lung lesions. Digital tomosynthesis is a particularly promising technique that allows reconstruction of multisection images from a short acquisition at very low patient dose. Preliminary data suggest that, compared with conventional radiography, tomosynthesis may also improve detection of subtle lung lesions. The ultimate influence of these new technologies will, of course, depend on the outcome of rigorous scientific validation.
Collapse
Affiliation(s)
- H Page McAdams
- Department of Radiology, Duke Advanced Imaging Laboratories, Duke University Medical Center, Box 3808, Durham, NC 27710, USA.
| | | | | | | | | |
Collapse
|
20
|
Kamitani T, Yabuuchi H, Soeda H, Matsuo Y, Okafuji T, Sakai S, Furuya A, Hatakenaka M, Ishii N, Honda H. Detection of masses and microcalcifications of breast cancer on digital mammograms: comparison among hard-copy film, 3-megapixel liquid crystal display (LCD) monitors and 5-megapixel LCD monitors: an observer performance study. Eur Radiol 2006; 17:1365-71. [PMID: 17093968 DOI: 10.1007/s00330-006-0452-6] [Citation(s) in RCA: 37] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2006] [Revised: 07/03/2006] [Accepted: 08/22/2006] [Indexed: 11/24/2022]
Abstract
The purpose of the study was to compare observer performance in the detection of masses and microcalcifications of breast cancer among hard-copy reading and soft-copy readings using 3-megapixel (3M) and 5-megapixel (5M) liquid crystal display (LCD) monitors. For the microcalcification detection test, we prepared 100 mammograms: 40 surgically verified cancer cases and 60 normal cases. For the mass detection test, we prepared 100 mammograms: 50 cancer cases and 50 normal cases. After six readers assessed both microcalcifications and masses set for each modality, receiver operating characteristic (ROC) analysis was performed. The average A(z)s for mass detection using a hard copy and 3M and 5M LCD monitors were 0.923, 0.927 and 0.920, respectively; there were no significant differences. The average A(z) for microcalcification detection using hard copy, 3M and 5M LCD monitors was 0.977, 0.954 and 0.972, respectively. There were no significant differences, but the P-values between the hard copy and 3M LCD monitor and that between the 3M and 5M LCD monitor were 0.08 and 0.09, respectively. In conclusion, the observer performances for detecting masses of breast cancers were comparable among the hard copy and two LCD monitors; however, soft-copy reading with a 3M LCD monitor showed slightly lower observer performance for detecting microcalcifications of breast cancers than hard-copy or 5M LCD monitor reading.
Collapse
Affiliation(s)
- Takeshi Kamitani
- Department of Clinical Radiology, Graduate School of Medical Sciences, Kyushu University, 3-1-1 Maidashi, Higashi-ku, Fukuoka, 812-8582, Japan,
| | | | | | | | | | | | | | | | | | | |
Collapse
|
21
|
Brem MH, Böhner C, Brenning A, Gelse K, Radkow T, Blanke M, Schlechtweg PM, Neumann G, Wu IY, Bautz W, Hennig FF, Richter H. Evaluation of low-cost computer monitors for the detection of cervical spine injuries in the emergency room: an observer confidence-based study. Emerg Med J 2006; 23:850-3. [PMID: 17057136 PMCID: PMC2464403 DOI: 10.1136/emj.2006.036822] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 08/13/2006] [Indexed: 11/03/2022]
Abstract
BACKGROUND To compare the diagnostic value of low-cost computer monitors and a Picture Archiving and Communication System (PACS) workstation for the evaluation of cervical spine fractures in the emergency room. METHODS Two groups of readers blinded to the diagnoses (2 radiologists and 3 orthopaedic surgeons) independently assessed-digital radiographs of the cervical spine (anterior-posterior, oblique and trans-oral-dens views). The radiographs of 57 patients who arrived consecutively to the emergency room in 2004 with clinical suspicion of a cervical spine injury were evaluated. The diagnostic values of these radiographs were scored on a 3-point scale (1 = diagnosis not possible/bad image quality, 2 = diagnosis uncertain, 3 = clear diagnosis of fracture or no fracture) on a PACS workstation and on two different liquid crystal display (LCD) personal computer monitors. The images were randomised to avoid memory effects. We used logistic mixed-effects models to determine the possible effects of monitor type on the evaluation of x ray images. To determine the overall effects of monitor type, this variable was used as a fixed effect, and the image number and reader group (radiologist or orthopaedic surgeon) were used as random effects on display quality. Group-specific effects were examined, with the reader group and additional fixed effects as terms. A significance level of 0.05 was established for assessing the contribution of each fixed effect to the model. RESULTS Overall, the diagnostic score did not differ significantly between standard personal computer monitors and the PACS workstation (both p values were 0.78). CONCLUSION Low-cost LCD personal computer monitors may be useful in establishing a diagnosis of cervical spine fractures in the emergency room.
Collapse
Affiliation(s)
- M H Brem
- Department of Surgery, Division of Trauma Surgery and Orthopaedic Surgery, Friedrich-Alexander University Erlangen-Nuremberg, Krankenhausstr 12, 91054 Erlangen, Germany.
| | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
22
|
Thomas JA, Chakrabarti K, Kaczmarek RV, Maslennikov A, Mitchell CA, Romanyukha A. Optimization of reading conditions for flat panel displays. J Digit Imaging 2006; 19:181-7. [PMID: 16437286 PMCID: PMC3045187 DOI: 10.1007/s10278-006-9710-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022] Open
Abstract
Task Group 18 (TG 18) of the American Association of Physicists in Medicine has developed guidelines for Assessment of Display Performance for Medical Imaging Systems. In this document, a method for determination of the maximum room lighting for displays is suggested. It is based on luminance measurements of a black target displayed on each display device at different room illuminance levels. Linear extrapolation of the above luminance measurements vs. room illuminance allows one to determine diffuse and specular reflection coefficients. TG 18 guidelines have established recommended maximum room lighting. It is based on the characterization of the display by its minimum and maximum luminance and the description of room by diffuse and specular coefficients. We carried out these luminance measurements for three selected displays to determine their optimum viewing conditions: one cathode ray tube and two flat panels. We found some problems with the application of the TG 18 guidelines to optimize viewing conditions for IBM T221 flat panels. Introduction of the requirement for minimum room illuminance allows a more accurate determination of the optimal viewing conditions (maximum and minimum room illuminance) for IBM flat panels. It also addresses the possible loss of contrast in medical images on flat panel displays because of the effect of nonlinearity in the dependence of luminance on room illuminance at low room lighting.
Collapse
Affiliation(s)
- J. A. Thomas
- Uniformed Services University of the Health Sciences, Bethesda, MD 20814 USA
| | - K. Chakrabarti
- Center for Devices and Radiological Health, FDA, Rockville, MD 20850 USA
| | - R. V. Kaczmarek
- Center for Devices and Radiological Health, FDA, Rockville, MD 20850 USA
| | - A. Maslennikov
- Uniformed Services University of the Health Sciences, Bethesda, MD 20814 USA
| | - C. A. Mitchell
- Uniformed Services University of the Health Sciences, Bethesda, MD 20814 USA
| | - A. Romanyukha
- Uniformed Services University of the Health Sciences, Bethesda, MD 20814 USA
- 4301 Jones Bridge Road, Bethesda, MD 20814 USA
| |
Collapse
|
23
|
Bacher K, Smeets P, De Hauwere A, Voet T, Duyck P, Verstraete K, Thierens H. Image quality performance of liquid crystal display systems: influence of display resolution, magnification and window settings on contrast-detail detection. Eur J Radiol 2006; 58:471-9. [PMID: 16442770 DOI: 10.1016/j.ejrad.2005.12.016] [Citation(s) in RCA: 27] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/23/2005] [Accepted: 12/20/2005] [Indexed: 10/25/2022]
Abstract
The aim of this study was to investigate the combined effects of liquid crystal display (LCD) resolution, image magnification and window/level adjustment on the low-contrast performance in soft-copy image interpretation in digital radiography and digital mammography. In addition, the effect of a new LCD noise reduction mechanism on the low-contrast detectability was studied. Digital radiographs and mammograms of two dedicated contrast-detail phantoms (CDRAD 2.0 and CDMAM 3.4) were scored on five LCD devices with varying resolutions (1-3- and 5-megapixel) and one dedicated 5-megapixel cathode ray tube monitor. Two 5-megapixel LCDs were included. The first one was a standard 5-megapixel LCD and the second had a new (Per Pixel Uniformity) noise reduction mechanism. A multi-variate analysis of variance revealed a significant influence of LCD resolution, image magnification and window/level adjustment on the image quality performance assessed with both the CDRAD 2.0 and the CDMAM 3.4 phantoms. The interactive adjustment of brightness and contrast of digital images did not affect the reading time, whereas magnification to full resolution resulted in a significantly slower soft-copy interpretation. For digital radiography applications, a 3-megapixel LCD is comparable with a 5-megapixel CRT monitor in terms of low-contrast performance as well as in reading time. The use of a 2-megapixel LCD is only warranted when radiographs are analysed in full resolution and when using the interactive window/level adjustment. In digital mammography, a 5-megapixel monitor should be the first choice. In addition, the new PPU noise reduction system in the 5-megapixel LCD devices provides significantly better results for mammography reading as compared to a standard 5-magapixel LCD or CRT. If a 3-megapixel LCD is used in mammography setting, a very time-consuming magnification of the digital mammograms would be necessary.
Collapse
Affiliation(s)
- Klaus Bacher
- Department of Medical Physics and Radiation Protection, Ghent University, Proeftuinstraat 86, B-9000 Gent, Belgium.
| | | | | | | | | | | | | |
Collapse
|