1
|
Lee JH, Hwang EJ, Kim H, Park CM. A narrative review of deep learning applications in lung cancer research: from screening to prognostication. Transl Lung Cancer Res 2022; 11:1217-1229. [PMID: 35832457 PMCID: PMC9271435 DOI: 10.21037/tlcr-21-1012] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2021] [Accepted: 05/16/2022] [Indexed: 01/17/2023]
Abstract
Background and Objective Deep learning (DL) algorithms have been developed for various tasks, including lung nodule detection on chest radiographs or lung cancer computed tomography screening, potential candidate selection in lung cancer screening, malignancy prediction for indeterminate pulmonary nodules, lung cancer staging, treatment response prediction, prognostication, and prediction of genetic mutations in lung cancer. Furthermore, these DL algorithms have been applied in various clinical settings in order for them to be generalized in real-world clinical practice. Multiple DL algorithms have been corroborated to be on par with experts or current clinical prediction models for several specific tasks. However, no article has yet comprehensively reviewed DL algorithms dedicated to lung cancer research. This narrative review presents an overview of the literature dealing with DL techniques applied in lung cancer research and briefly summarizes the results according to the DL algorithms’ clinical use cases. Methods we performed a narrative review by searching the Embase and OVID-MEDLINE databases for articles published in English from October, 2016 until September, 2021 and reviewing the bibliographies of key references to identify important literature related to DL in lung cancer research. The background, development, results, and clinical implications of each DL algorithm are briefly discussed. Lastly, we end this review article by highlighting future directions in lung cancer research using DL techniques. Key Content and Findings DL algorithms have been introduced to show comparable or higher performance than human experts in various clinical settings. Specifically, they have been actively applied to detect lung nodules in chest radiographs or computed tomography (CT) examinations, optimize candidate selection for lung cancer screening (LCS), predict the malignancy of lung nodules, stage lung cancer, and predict treatment response, patients’ prognoses, and genetic mutations in lung cancers. Conclusions DL algorithms have corroborated their potential value for various tasks, ranging from lung cancer screening to prognostication of lung cancer patients. Future research is warranted for the clinical application of these algorithms in daily clinical practice and verification of their real-world clinical usefulness.
Collapse
Affiliation(s)
- Jong Hyuk Lee
- Department of Radiology, Seoul National University Hospital, Seoul, Korea.,Department of Radiology, Seoul National University College of Medicine, Seoul, Korea.,Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea
| | - Eui Jin Hwang
- Department of Radiology, Seoul National University Hospital, Seoul, Korea.,Department of Radiology, Seoul National University College of Medicine, Seoul, Korea.,Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea
| | - Hyungjin Kim
- Department of Radiology, Seoul National University Hospital, Seoul, Korea.,Department of Radiology, Seoul National University College of Medicine, Seoul, Korea.,Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea
| | - Chang Min Park
- Department of Radiology, Seoul National University Hospital, Seoul, Korea.,Department of Radiology, Seoul National University College of Medicine, Seoul, Korea.,Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea.,Institute of Medical and Biological Engineering, Medical Research Center, Seoul National University, Seoul, Korea
| |
Collapse
|
2
|
Yapp KE, Brennan P, Ekpo E. The Effect of Clinical History on Diagnostic Imaging Interpretation - A Systematic Review. Acad Radiol 2022; 29:255-266. [PMID: 33183952 DOI: 10.1016/j.acra.2020.10.021] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2020] [Revised: 10/21/2020] [Accepted: 10/21/2020] [Indexed: 12/25/2022]
Abstract
RATIONALE AND OBJECTIVES To provide updated information on the effect of clinical history on diagnostic image interpretation and to provide study methodology and design recommendations for future studies assessing the effect of clinical history on diagnostic image performance. MATERIALS AND METHODS A literature search of Medline, Embase, Scopus, Web of Science, and the Cochrane Central Register of Controlled Trials (CENTRAL) databases was conducted from database inception to July 21, 2020. Studies comparing diagnostic imaging performance with and without clinical history, using observers reading images under both conditions that used an independent reference standard were included. RESULTS Twenty-two studies met the inclusion criteria, with 15 showing clinical history improved diagnostic performance. One study reported a decrease in diagnostic performance with clinical history and the remaining six studies found no significant change in performance. Two studies used the free response paradigm with both reporting clinical history increased location sensitivity, decreased specificity and had no overall change in diagnostic performance. The disease spectrum of included cases was largely unreported and a balanced reading design was not used in 19 studies. CONCLUSION Most published studies found that clinical history improved diagnostic performance. More recent studies accounting for abnormality location and multiple abnormalities showed an increase in false positives and no significant change in overall diagnostic performance with clinical history.
Collapse
Affiliation(s)
- Kehn E Yapp
- Medical Image Optimisation and Perception Group (MIOPeG), Discipline of Medical Imaging Science, School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Australia.
| | - Patrick Brennan
- Medical Image Optimisation and Perception Group (MIOPeG), Discipline of Medical Imaging Science, School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Australia
| | - Ernest Ekpo
- Medical Image Optimisation and Perception Group (MIOPeG), Discipline of Medical Imaging Science, School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Australia
| |
Collapse
|
3
|
Lee JH, Sun HY, Park S, Kim H, Hwang EJ, Goo JM, Park CM. Performance of a Deep Learning Algorithm Compared with Radiologic Interpretation for Lung Cancer Detection on Chest Radiographs in a Health Screening Population. Radiology 2020; 297:687-696. [PMID: 32960729 DOI: 10.1148/radiol.2020201240] [Citation(s) in RCA: 24] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/17/2023]
Abstract
Background The performance of a deep learning algorithm for lung cancer detection on chest radiographs in a health screening population is unknown. Purpose To validate a commercially available deep learning algorithm for lung cancer detection on chest radiographs in a health screening population. Materials and Methods Out-of-sample testing of a deep learning algorithm was retrospectively performed using chest radiographs from individuals undergoing a comprehensive medical check-up between July 2008 and December 2008 (validation test). To evaluate the algorithm performance for visible lung cancer detection, the area under the receiver operating characteristic curve (AUC) and diagnostic measures, including sensitivity and false-positive rate (FPR), were calculated. The algorithm performance was compared with that of radiologists using the McNemar test and the Moskowitz method. Additionally, the deep learning algorithm was applied to a screening cohort undergoing chest radiography between January 2008 and December 2012, and its performances were calculated. Results In a validation test comprising 10 285 radiographs from 10 202 individuals (mean age, 54 years ± 11 [standard deviation]; 5857 men) with 10 radiographs of visible lung cancers, the algorithm's AUC was 0.99 (95% confidence interval: 0.97, 1), and it showed comparable sensitivity (90% [nine of 10 radiographs]) to that of the radiologists (60% [six of 10 radiographs]; P = .25) with a higher FPR (3.1% [319 of 10 275 radiographs] vs 0.3% [26 of 10 275 radiographs]; P < .001). In the screening cohort of 100 525 chest radiographs from 50 070 individuals (mean age, 53 years ± 11; 28 090 men) with 47 radiographs of visible lung cancers, the algorithm's AUC was 0.97 (95% confidence interval: 0.95, 0.99), and its sensitivity and FPR were 83% (39 of 47 radiographs) and 3% (2999 of 100 478 radiographs), respectively. Conclusion A deep learning algorithm detected lung cancers on chest radiographs with a performance comparable to that of radiologists, which will be helpful for radiologists in healthy populations with a low prevalence of lung cancer. © RSNA, 2020 Online supplemental material is available for this article. See also the editorial by Armato in this issue.
Collapse
Affiliation(s)
- Jong Hyuk Lee
- From the Department of Radiology and Institute of Radiation Medicine, Seoul National University College of Medicine, 101 Daehak-ro, Jongno-gu, Seoul 03080, Korea (J.H.L., H.K., E.J.H., J.M.G., C.M.P.); Department of Radiology, Healthcare Research Institute, Healthcare System Gangnam Center, Seoul National University Hospital, Seoul, Korea (H.Y.S.); and Lunit Inc, Seoul, Korea (S.P.)
| | - Hye Young Sun
- From the Department of Radiology and Institute of Radiation Medicine, Seoul National University College of Medicine, 101 Daehak-ro, Jongno-gu, Seoul 03080, Korea (J.H.L., H.K., E.J.H., J.M.G., C.M.P.); Department of Radiology, Healthcare Research Institute, Healthcare System Gangnam Center, Seoul National University Hospital, Seoul, Korea (H.Y.S.); and Lunit Inc, Seoul, Korea (S.P.)
| | - Sunggyun Park
- From the Department of Radiology and Institute of Radiation Medicine, Seoul National University College of Medicine, 101 Daehak-ro, Jongno-gu, Seoul 03080, Korea (J.H.L., H.K., E.J.H., J.M.G., C.M.P.); Department of Radiology, Healthcare Research Institute, Healthcare System Gangnam Center, Seoul National University Hospital, Seoul, Korea (H.Y.S.); and Lunit Inc, Seoul, Korea (S.P.)
| | - Hyungjin Kim
- From the Department of Radiology and Institute of Radiation Medicine, Seoul National University College of Medicine, 101 Daehak-ro, Jongno-gu, Seoul 03080, Korea (J.H.L., H.K., E.J.H., J.M.G., C.M.P.); Department of Radiology, Healthcare Research Institute, Healthcare System Gangnam Center, Seoul National University Hospital, Seoul, Korea (H.Y.S.); and Lunit Inc, Seoul, Korea (S.P.)
| | - Eui Jin Hwang
- From the Department of Radiology and Institute of Radiation Medicine, Seoul National University College of Medicine, 101 Daehak-ro, Jongno-gu, Seoul 03080, Korea (J.H.L., H.K., E.J.H., J.M.G., C.M.P.); Department of Radiology, Healthcare Research Institute, Healthcare System Gangnam Center, Seoul National University Hospital, Seoul, Korea (H.Y.S.); and Lunit Inc, Seoul, Korea (S.P.)
| | - Jin Mo Goo
- From the Department of Radiology and Institute of Radiation Medicine, Seoul National University College of Medicine, 101 Daehak-ro, Jongno-gu, Seoul 03080, Korea (J.H.L., H.K., E.J.H., J.M.G., C.M.P.); Department of Radiology, Healthcare Research Institute, Healthcare System Gangnam Center, Seoul National University Hospital, Seoul, Korea (H.Y.S.); and Lunit Inc, Seoul, Korea (S.P.)
| | - Chang Min Park
- From the Department of Radiology and Institute of Radiation Medicine, Seoul National University College of Medicine, 101 Daehak-ro, Jongno-gu, Seoul 03080, Korea (J.H.L., H.K., E.J.H., J.M.G., C.M.P.); Department of Radiology, Healthcare Research Institute, Healthcare System Gangnam Center, Seoul National University Hospital, Seoul, Korea (H.Y.S.); and Lunit Inc, Seoul, Korea (S.P.)
| |
Collapse
|
4
|
Lung Nodule: Imaging Features and Evaluation in the Age of Machine Learning. CURRENT PULMONOLOGY REPORTS 2019. [DOI: 10.1007/s13665-019-00229-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
|
5
|
van Beek EJR, Murchison JT. Artificial Intelligence and Computer-Assisted Evaluation of Chest Pathology. Artif Intell Med Imaging 2019. [DOI: 10.1007/978-3-319-94878-2_12] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/19/2022] Open
|
6
|
Itri JN, Tappouni RR, McEachern RO, Pesch AJ, Patel SH. Fundamentals of Diagnostic Error in Imaging. Radiographics 2018; 38:1845-1865. [DOI: 10.1148/rg.2018180021] [Citation(s) in RCA: 53] [Impact Index Per Article: 8.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
Affiliation(s)
- Jason N. Itri
- From the Department of Radiology, Wake Forest Baptist Medical Center, Medical Center Blvd, Winston-Salem, NC 27157-1088 (J.N.I., R.R.T.); and Department of Radiology and Medical Imaging, University of Virginia Health System, Charlottesville, Va (R.O.M., A.J.P., S.H.P.)
| | - Rafel R. Tappouni
- From the Department of Radiology, Wake Forest Baptist Medical Center, Medical Center Blvd, Winston-Salem, NC 27157-1088 (J.N.I., R.R.T.); and Department of Radiology and Medical Imaging, University of Virginia Health System, Charlottesville, Va (R.O.M., A.J.P., S.H.P.)
| | - Rachel O. McEachern
- From the Department of Radiology, Wake Forest Baptist Medical Center, Medical Center Blvd, Winston-Salem, NC 27157-1088 (J.N.I., R.R.T.); and Department of Radiology and Medical Imaging, University of Virginia Health System, Charlottesville, Va (R.O.M., A.J.P., S.H.P.)
| | - Arthur J. Pesch
- From the Department of Radiology, Wake Forest Baptist Medical Center, Medical Center Blvd, Winston-Salem, NC 27157-1088 (J.N.I., R.R.T.); and Department of Radiology and Medical Imaging, University of Virginia Health System, Charlottesville, Va (R.O.M., A.J.P., S.H.P.)
| | - Sohil H. Patel
- From the Department of Radiology, Wake Forest Baptist Medical Center, Medical Center Blvd, Winston-Salem, NC 27157-1088 (J.N.I., R.R.T.); and Department of Radiology and Medical Imaging, University of Virginia Health System, Charlottesville, Va (R.O.M., A.J.P., S.H.P.)
| |
Collapse
|
7
|
Affiliation(s)
- Ronald L. Arenson
- From the Department of Radiology, University of California, San Francisco, 505 Parnassus Ave, Room M-391, San Francisco, CA 94143-0628
| |
Collapse
|
8
|
Added value of double reading in diagnostic radiology,a systematic review. Insights Imaging 2018; 9:287-301. [PMID: 29594850 PMCID: PMC5990995 DOI: 10.1007/s13244-018-0599-0] [Citation(s) in RCA: 36] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2017] [Revised: 01/10/2018] [Accepted: 01/15/2018] [Indexed: 01/10/2023] Open
Abstract
Objectives Double reading in diagnostic radiology can find discrepancies in the original report, but a systematic program of double reading is resource consuming. There are conflicting opinions on the value of double reading. The purpose of the current study was to perform a systematic review on the value of double reading. Methods A systematic review was performed to find studies calculating the rate of misses and overcalls with the aim of establishing the added value of double reading by human observers. Results The literature search resulted in 1610 hits. After abstract and full-text reading, 46 articles were selected for analysis. The rate of discrepancy varied from 0.4 to 22% depending on study setting. Double reading by a sub-specialist, in general, led to high rates of changed reports. Conclusions The systematic review found rather low discrepancy rates. The benefit of double reading must be balanced by the considerable number of working hours a systematic double-reading scheme requires. A more profitable scheme might be to use systematic double reading for selected, high-risk examination types. A second conclusion is that there seems to be a value of sub-specialisation for increased report quality. A consequent implementation of this would have far-reaching organisational effects. Key Points • In double reading, two or more radiologists read the same images. • A systematic literature review was performed. • The discrepancy rates varied from 0.4 to 22% in various studies. • Double reading by sub-specialists found high discrepancy rates. Electronic supplementary material The online version of this article (10.1007/s13244-018-0599-0) contains supplementary material, which is available to authorised users.
Collapse
|
9
|
A review of lung cancer screening and the role of computer-aided detection. Clin Radiol 2017; 72:433-442. [DOI: 10.1016/j.crad.2017.01.002] [Citation(s) in RCA: 63] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2016] [Revised: 12/14/2016] [Accepted: 01/04/2017] [Indexed: 12/26/2022]
|
10
|
Yan PF, Yan L, Zhang Z, Salim A, Wang L, Hu TT, Zhao HY. Accuracy of conventional MRI for preoperative diagnosis of intracranial tumors: A retrospective cohort study of 762 cases. Int J Surg 2016; 36:109-117. [PMID: 27773598 DOI: 10.1016/j.ijsu.2016.10.023] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2016] [Revised: 09/30/2016] [Accepted: 10/15/2016] [Indexed: 12/20/2022]
Abstract
BACKGROUND Conventional magnetic resonance imaging (MRI) is considered a valuable tool for preoperative diagnosis of intracranial tumors. We assessed its accuracy in the diagnosis of intracranial tumors in usual clinical practice. MATERIALS AND METHODS MRI reports of 762 patients who had undergone conventional brain MRI prior to surgery were retrospectively reviewed. A 4-grade scoring system was devised to establish diagnostic agreement. Each tumor type was compared with the corresponding pathological diagnoses by dichotomization. The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy were calculated for the overall patient population as well as for each tumor type. RESULTS 664 cases (87.1%) were tumor-positive, and 98 cases (12.9%) were tumor-negative. The most common tumor types were meningiomas, gliomas, pituitary adenomas and schwannomas. These four types together comprised 74.5% of all cases reviewed. Sensitivity and PPV for the overall population were 72.0-90.7% and 91.9-95.4%, respectively. Diagnostic accuracy differed among tumor types. Meningiomas, pituitary adenomas, schwannomas and cholesteatomas were more likely to be diagnosed correctly (sensitivities were 82.6-96.9%, 86.1-96.7%, 88.9-98.2% and 91.3-100.0%, respectively); while some other types like solitary fibrous tumors (SFTs) seemed difficult to identify. Gliomas tended to be confused with metastases, meningiomas with SFTs, and pituitary adenomas with craniopharyngiomas. CONCLUSION The accuracy of conventional MRI for diagnosing intracranial tumors is generally satisfactory but should not be too heavily relied upon, especially for certain tumor types. In cases of discrepancy, neurosurgeons are encouraged to confer with the reporting neuroradiologists to achieve optimal preoperative diagnoses.
Collapse
Affiliation(s)
- Peng-Fei Yan
- Department of Neurosurgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei, 430022, China.
| | - Ling Yan
- Department of Medicine, University of British Columbia, Vancouver, BC, V6T 1Z3, Canada.
| | - Zhen Zhang
- Department of Radiology, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei, 430022, China.
| | - Adnan Salim
- Civil Hospital Karachi, Karachi, 74200, Pakistan.
| | - Lei Wang
- Department of Neurosurgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei, 430022, China.
| | - Ting-Ting Hu
- Department of Neurosurgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei, 430022, China.
| | - Hong-Yang Zhao
- Department of Neurosurgery, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei, 430022, China.
| |
Collapse
|
11
|
Pow RE, Mello-Thoms C, Brennan P. Evaluation of the effect of double reporting on test accuracy in screening and diagnostic imaging studies: A review of the evidence. J Med Imaging Radiat Oncol 2016; 60:306-14. [DOI: 10.1111/1754-9485.12450] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2015] [Accepted: 02/26/2016] [Indexed: 12/01/2022]
Affiliation(s)
- Richard E Pow
- Medical Radiation Sciences; Faculty of Health Sciences; The University of Sydney; Sydney New South Wales Australia
| | - Claudia Mello-Thoms
- Medical Radiation Sciences; Faculty of Health Sciences; The University of Sydney; Sydney New South Wales Australia
| | - Patrick Brennan
- Medical Radiation Sciences; Faculty of Health Sciences; The University of Sydney; Sydney New South Wales Australia
| |
Collapse
|
12
|
Lauritzen PM, Stavem K, Andersen JG, Stokke MV, Tennstrand AL, Bjerke G, Hurlen P, Sandbæk G, Dahl FA, Gulbrandsen P. Double reading of current chest CT examinations: Clinical importance of changes to radiology reports. Eur J Radiol 2016; 85:199-204. [DOI: 10.1016/j.ejrad.2015.11.012] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2015] [Revised: 10/22/2015] [Accepted: 11/04/2015] [Indexed: 11/17/2022]
|
13
|
Lauritzen PM, Hurlen P, Sandbæk G, Gulbrandsen P. Double reading rates and quality assurance practices in Norwegian hospital radiology departments: two parallel national surveys. Acta Radiol 2015; 56:78-86. [PMID: 24425793 DOI: 10.1177/0284185113519988] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
BACKGROUND Double reading as a quality assurance (QA) tool is employed extensively in Norwegian hospital radiology departments. The practice is resource consuming and regularly debated. PURPOSE To investigate the rates of double reading in Norwegian hospital radiology departments, to identify department characteristics associated with double reading rates, and to investigate associations between double reading and other quality improvement. MATERIAL AND METHODS We issued two parallel national surveys to management and to consultant radiologists, respectively. Management was defined as the chief medical officer and/or the head of the radiology department. The management survey covered staffing, perceived resource situation, double reading, guidelines, and quality improvement. The radiologist survey served to validate management responses concerning double reading. Management survey items concerning practices of quality improvement were organized into three indices reflecting different quality approaches, namely: appropriateness of investigations; personal performance feedback; and system performance feedback. RESULTS The response rates of the surveys were 100% (45/45) for management and 55% (266/483) for radiologists. Of all exams read by consultants, 33% were double read. The double reading rate was highest in university hospital departments (59%), intermediate in other teaching departments (30%), and lowest in non-teaching departments (11%) (P = 0.01). Among the quality indices, mean scores were highest on appropriateness index (68%), intermediate on the person index (56%), and lowest on system index (37%). There were no correlations between double reading rates and scores on any of the quality indices. CONCLUSION The rate of double reading in Norwegian hospital radiology is significantly correlated to department teaching status, but not to other practices of quality work.
Collapse
Affiliation(s)
- Peter M Lauritzen
- Department of Diagnostic Imaging, Akershus University Hospital, Norway
| | - Petter Hurlen
- Department of Diagnostic Imaging, Akershus University Hospital, Norway
- Institute of Clinical Medicine, University of Oslo, Norway
| | - Gunnar Sandbæk
- Institute of Clinical Medicine, University of Oslo, Norway
- Department of Radiology and Nuclear Imaging, Oslo University Hospital, Norway
| | - Pål Gulbrandsen
- Institute of Clinical Medicine, University of Oslo, Norway
- Health Services Research Unit, Akershus University Hospital, Norway
| |
Collapse
|
14
|
van der Gijp A, van der Schaaf MF, van der Schaaf IC, Huige JCBM, Ravesloot CJ, van Schaik JPJ, Ten Cate TJ. Interpretation of radiological images: towards a framework of knowledge and skills. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2014; 19:565-80. [PMID: 24449126 DOI: 10.1007/s10459-013-9488-y] [Citation(s) in RCA: 45] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/25/2013] [Accepted: 11/26/2013] [Indexed: 05/19/2023]
Abstract
The knowledge and skills that are required for radiological image interpretation are not well documented, even though medical imaging is gaining importance. This study aims to develop a comprehensive framework of knowledge and skills, required for two-dimensional and multiplanar image interpretation in radiology. A mixed-method study approach was applied. First, a literature search was performed to identify knowledge and skills that are important for image interpretation. Three databases, PubMed, PsycINFO and Embase, were searched for studies using synonyms of image interpretation skills or visual expertise combined with synonyms of radiology. Empirical or review studies concerning knowledge and skills for medical image interpretation were included and relevant knowledge and skill items were extracted. Second, a preliminary framework was built and discussed with nine selective experts in individual semi-structured interviews. The expert team consisted of four radiologists, one radiology resident, two education scientists, one cognitive psychologist and one neuropsychologist. The framework was optimised based on the experts comments. Finally, the framework was applied to empirical data, derived from verbal protocols of ten clerks interpreting two-dimensional and multiplanar radiological images. In consensus meetings adjustments were made to resolve discrepancies of the framework with the verbal protocol data. We designed a framework with three main components of image interpretation: perception, analysis and synthesis. The literature study provided four knowledge and twelve skill items. As a result of the expert interviews, one skill item was added and formulations of existing items were adjusted. The think-aloud experiment showed that all knowledge items and three of the skill items were applied within all three main components of the image interpretation process. The remaining framework items were apparent only within one of the main components. After combining two knowledge items, we finally identified three knowledge items and thirteen skills, essential for image interpretation by trainees. The framework can serve as a guideline for education and assessment of two- and three-dimensional image interpretation. Further validation of the framework in larger study groups with different levels of expertise is needed.
Collapse
Affiliation(s)
- A van der Gijp
- Radiology Department, E01.132, University Medical Center (UMC) Utrecht, Heidelberglaan 100, 3584 CX, Utrecht, The Netherlands,
| | | | | | | | | | | | | |
Collapse
|
15
|
Test M, Shah SS, Monuteaux M, Ambroggio L, Lee EY, Markowitz RI, Bixby S, Diperna S, Servaes S, Hellinger JC, Neuman MI. Impact of clinical history on chest radiograph interpretation. J Hosp Med 2013. [PMID: 23184766 DOI: 10.1002/jhm.1991] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
BACKGROUND The inclusion of clinical information may have unrecognized influence in the interpretation of diagnostic testing. OBJECTIVE The objective of the study was to determine the impact of clinical history on chest radiograph interpretation in the diagnosis of pneumonia. DESIGN Prospective case-based study. METHODS Radiologists interpreted 110 radiographs of children evaluated for suspicion of pneumonia. Clinical information was withheld during the first interpretation. After 6 months the radiographs were reviewed with clinical information. Radiologists reported on pneumonia indicators described by the World Health Organization (ie, any infiltrate, alveolar infiltrate, interstitial infiltrate, air bronchograms, hilar adenopathy, pleural effusion). SETTING Children's Hospital of Philadelphia and Boston Children's Hospital. PARTICIPANTS Six board-certified radiologists. OUTCOME MEASURES Inter- and inter-rater reliability were assessed using the kappa statistic. RESULTS The addition of clinical history did not have a substantial impact on the inter-rater reliability in the identification of any infiltrate, alveolar infiltrate, interstitial infiltrate, pleural effusion, or hilar adenopathy. Inter-rater reliability in the identification of air bronchograms improved from fair (k = 0.32) to moderate (k = 0.53). Intra-rater reliability for the identification of alveolar infiltrate remained substantial to almost perfect for all 6 raters with and without clinical information. One rater had a decrease in inter-rater reliability from almost perfect (k = 1.0) to fair (k = 0.21) in the identification of interstitial infiltrate with the addition of clinical history. CONCLUSIONS Alveolar infiltrate and pleural effusion are findings with high intra- and inter-rater reliability in the diagnosis of bacterial pneumonia. The addition of clinical information did not have a substantial impact on the reliability of these findings.
Collapse
Affiliation(s)
- Matthew Test
- Division of Hospital Medicine, Cincinnati Children's Hospital Medical Center, Cincinnati, OH 45229, USA
| | | | | | | | | | | | | | | | | | | | | |
Collapse
|
16
|
Husby JA, Espeland A, Kalyanpur A, Brocker C, Haldorsen IS. Double reading of radiological examinations in Norway. Acta Radiol 2011; 52:516-21. [PMID: 21498308 DOI: 10.1258/ar.2011.100347] [Citation(s) in RCA: 11] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
BACKGROUND Double reading of images is a part of the quality assurance activities at many radiological centers. PURPOSE To investigate the extent of and routines for double reading in Norway and the institutional heads' attitudes toward double reading. MATERIAL AND METHODS A questionnaire was addressed to the heads of all radiological institutions in Norway. The questionnaire concerned staffing, examinations performed, extent of double reading per imaging modality (except mammography screening), guidelines for double reading, checks of completed radiology reports, frequency of regular quality assurance meetings to discuss missed findings, and the heads' attitudes toward double reading. RESULTS The response rate was 73% (53/73). The percentage across imaging modalities of examinations being double read was 41% overall: 56% at university hospitals, 37% at local hospitals, and 18% at private centers. Double reading was most common for positron emission tomography (PET)/PET-computed tomography (CT) examinations (100%), and clinical mammography (91%). Almost all examinations read by residents were double read. Only 15% of institutions had written guidelines for double reading, 15% performed random double readings of completed examinations, and 55% organized regular meetings to discuss missed findings. Forty-six percent of the institutional heads wanted an increased use of double reading. CONCLUSION Double reading is common in Norway, especially in residency training, mammography, and PET/PET-CT. It is less common at private centers. Established routines for double reading are scarce. Many institutional heads want more double reading. The potential of double reading to assure quality in radiology should be better exploited.
Collapse
Affiliation(s)
- Jenny A Husby
- Department of Radiology, Haukeland University Hospital, Jonas Liesvei 65, Post Box 7800, 5021 Bergen, Norway
| | - Ansgar Espeland
- Department of Radiology, Haukeland University Hospital, Jonas Liesvei 65, Post Box 7800, 5021 Bergen, Norway
- Section for Radiology, Department of Surgical Sciences, University of Bergen, Bergen, Norway
| | | | - Carsten Brocker
- Department of Radiology, Ringerike Hospital, Ringerike, Norway
| | - Ingfrid S Haldorsen
- Department of Radiology, Haukeland University Hospital, Jonas Liesvei 65, Post Box 7800, 5021 Bergen, Norway
| |
Collapse
|
17
|
Niemeijer M, Loog M, Abramoff MD, Viergever MA, Prokop M, van Ginneken B. On combining computer-aided detection systems. IEEE TRANSACTIONS ON MEDICAL IMAGING 2011; 30:215-223. [PMID: 20813633 DOI: 10.1109/tmi.2010.2072789] [Citation(s) in RCA: 59] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/29/2023]
Abstract
Computer-aided detection (CAD) is increasingly used in clinical practice and for many applications a multitude of CAD systems have been developed. In practice, CAD systems have different strengths and weaknesses and it is therefore interesting to consider their combination. In this paper, we present generic methods to combine multiple CAD systems and investigate what kind of performance increase can be expected. Experimental results are presented using data from the ANODE09 and ROC09 online CAD challenges for the detection of pulmonary nodules in computed tomography scans and red lesions in retinal images, respectively. For both applications, combination results in a large and significant increase in performance when compared to the best individual CAD system.
Collapse
Affiliation(s)
- Meindert Niemeijer
- University Medical Center Utrecht, Image Sciences Institute, 3584 CX Utrecht, The Netherlands.
| | | | | | | | | | | |
Collapse
|
18
|
Hansson K, Häggström J, Kvart C, Lord P. Reader performance in radiographic diagnosis of signs of mitral regurgitation in cavalier King Charles spaniels. J Small Anim Pract 2009; 50 Suppl 1:44-53. [PMID: 19765219 DOI: 10.1111/j.1748-5827.2009.00669.x] [Citation(s) in RCA: 23] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Abstract
OBJECTIVES To measure accuracy and variability of diagnosis by radiography of heart enlargement (HE) and heart failure (HF) in mitral regurgitation (MR). METHODS Sixteen readers representing four levels of experience evaluated 50 sets of radiographs with varying severity of MR for presence or absence of HE, left atrial enlargement (LAE) and HF. The performance of the readers was compared with a reference standard, using area under the curve (AUC) of receiver operating characteristic (ROC) curves. The interreader agreement value kappa (K) was calculated. A subset of difficult cases of HF was analysed before and after removing an outlying reader from each group. RESULTS AUC for HE was 0.89, for LAE it was 0.93 and for HF it was 0.92. Experience increased certainty of diagnosis but not accuracy. K ranges were HE, 0.53 to 0.67; LAE, 0.61 to 0.69 and HF, 0.49 to 0.58. When only difficult cases of HF were read, accuracy decreased and experienced readers performed better than inexperienced. When outlying readers were excluded, the differences between experienced and inexperienced readers increased. CLINICAL SIGNIFICANCE LAE, not HE, should be used to evaluate the heart size and indirectly the severity of MR on radiographs. For HF, agreement among individual readers was only moderate. Studies of reader accuracy should consider the effects of interreader variability.
Collapse
Affiliation(s)
- K Hansson
- Department of Clinical Sciences and Department of Animal Physiology, Swedish University of Agricultural Sciences, Box 7054, SE-750 07 Uppsala, Sweden
| | | | | | | |
Collapse
|
19
|
Wu MH, Gotway M, Lee T, Chern MS, Cheng HC, Ko JC, Sheu MH, Chang CY. Features of non-small cell lung carcinomas overlooked at digital chest radiography. Clin Radiol 2008; 63:518-28. [DOI: 10.1016/j.crad.2007.09.011] [Citation(s) in RCA: 19] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2007] [Revised: 09/22/2007] [Accepted: 09/24/2007] [Indexed: 10/22/2022]
|
20
|
Elmore JG, Brenner RJ. The More Eyes, the Better to See? From Double to Quadruple Reading of Screening Mammograms. J Natl Cancer Inst 2007; 99:1141-3. [PMID: 17652275 DOI: 10.1093/jnci/djm079] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022] Open
|
21
|
Lakhani P, Menschik ED, Goldszal AF, Murray JP, Weiner MG, Langlotz CP. Development and validation of queries using structured query language (SQL) to determine the utilization of comparison imaging in radiology reports stored on PACS. J Digit Imaging 2006; 19:52-68. [PMID: 16132483 PMCID: PMC3043946 DOI: 10.1007/s10278-005-7667-y] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022] Open
Abstract
The purpose of this research was to develop queries that quantify the utilization of comparison imaging in free-text radiology reports. The queries searched for common phrases that indicate whether comparison imaging was utilized, not available, or not mentioned. The queries were iteratively refined and tested on random samples of 100 reports with human review as a reference standard until the precision and recall of the queries did not improve significantly between iterations. Then, query accuracy was assessed on a new random sample of 200 reports. Overall accuracy of the queries was 95.6%. The queries were then applied to a database of 1.8 million reports. Comparisons were made to prior images in 38.69% of the reports (693,955/1,793,754), were unavailable in 18.79% (337,028/1,793,754), and were not mentioned in 42.52% (762,771/1,793,754). The results show that queries of text reports can achieve greater than 95% accuracy in determining the utilization of prior images.
Collapse
Affiliation(s)
- Paras Lakhani
- Department of Radiology, University of Pennsylvania, Philadelphia, PA USA
| | | | | | | | - Mark G. Weiner
- Department of Medicine, University of Pennsylvania, Philadelphia, PA USA
| | - Curtis P. Langlotz
- Department of Radiology, University of Pennsylvania, Philadelphia, PA USA
- 719 Iron Post Road, Moorestown, NJ 08057 USA
| |
Collapse
|
22
|
Rubin GD, Lyo JK, Paik DS, Sherbondy AJ, Chow LC, Leung AN, Mindelzun R, Schraedley-Desmond PK, Zinck SE, Naidich DP, Napel S. Pulmonary nodules on multi-detector row CT scans: performance comparison of radiologists and computer-aided detection. Radiology 2004; 234:274-83. [PMID: 15537839 DOI: 10.1148/radiol.2341040589] [Citation(s) in RCA: 174] [Impact Index Per Article: 8.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
Abstract
PURPOSE To compare the performance of radiologists and of a computer-aided detection (CAD) algorithm for pulmonary nodule detection on thin-section thoracic computed tomographic (CT) scans. MATERIALS AND METHODS The study was approved by the institutional review board. The requirement of informed consent was waived. Twenty outpatients (age range, 15-91 years; mean, 64 years) were examined with chest CT (multi-detector row scanner, four detector rows, 1.25-mm section thickness, and 0.6-mm interval) for pulmonary nodules. Three radiologists independently analyzed CT scans, recorded the locus of each nodule candidate, and assigned each a confidence score. A CAD algorithm with parameters chosen by using cross validation was applied to the 20 scans. The reference standard was established by two experienced thoracic radiologists in consensus, with blind review of all nodule candidates and free search for additional nodules at a dedicated workstation for three-dimensional image analysis. True-positive (TP) and false-positive (FP) results and confidence levels were used to generate free-response receiver operating characteristic (ROC) plots. Double-reading performance was determined on the basis of TP detections by either reader. RESULTS The 20 scans showed 195 noncalcified nodules with a diameter of 3 mm or more (reference reading). Area under the alternative free-response ROC curve was 0.54, 0.48, 0.55, and 0.36 for CAD and readers 1-3, respectively. Differences between reader 3 and CAD and between readers 2 and 3 were significant (P < .05); those between CAD and readers 1 and 2 were not significant. Mean sensitivity for individual readings was 50% (range, 41%-60%); double reading resulted in increase to 63% (range, 56%-67%). With CAD used at a threshold allowing only three FP detections per CT scan, mean sensitivity was increased to 76% (range, 73%-78%). CAD complemented individual readers by detecting additional nodules more effectively than did a second reader; CAD-reader weighted kappa values were significantly lower than reader-reader weighted kappa values (Wilcoxon rank sum test, P < .05). CONCLUSION With CAD used at a level allowing only three FP detections per CT scan, sensitivity was substantially higher than with conventional double reading.
Collapse
Affiliation(s)
- Geoffrey D Rubin
- Department of Radiology, Stanford University School of Medicine, 300 Pasteur Drive, S-072, Stanford, CA 94305-5105, USA.
| | | | | | | | | | | | | | | | | | | | | |
Collapse
|