1
|
Li D, Pehrson LM, Tøttrup L, Fraccaro M, Bonnevie R, Thrane J, Sørensen PJ, Rykkje A, Andersen TT, Steglich-Arnholm H, Stærk DMR, Borgwardt L, Hansen KL, Darkner S, Carlsen JF, Nielsen MB. Inter- and Intra-Observer Agreement When Using a Diagnostic Labeling Scheme for Annotating Findings on Chest X-rays-An Early Step in the Development of a Deep Learning-Based Decision Support System. Diagnostics (Basel) 2022; 12:diagnostics12123112. [PMID: 36553118 PMCID: PMC9776917 DOI: 10.3390/diagnostics12123112] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2022] [Revised: 11/21/2022] [Accepted: 11/26/2022] [Indexed: 12/14/2022] Open
Abstract
Consistent annotation of data is a prerequisite for the successful training and testing of artificial intelligence-based decision support systems in radiology. This can be obtained by standardizing terminology when annotating diagnostic images. The purpose of this study was to evaluate the annotation consistency among radiologists when using a novel diagnostic labeling scheme for chest X-rays. Six radiologists with experience ranging from one to sixteen years, annotated a set of 100 fully anonymized chest X-rays. The blinded radiologists annotated on two separate occasions. Statistical analyses were done using Randolph's kappa and PABAK, and the proportions of specific agreements were calculated. Fair-to-excellent agreement was found for all labels among the annotators (Randolph's Kappa, 0.40-0.99). The PABAK ranged from 0.12 to 1 for the two-reader inter-rater agreement and 0.26 to 1 for the intra-rater agreement. Descriptive and broad labels achieved the highest proportion of positive agreement in both the inter- and intra-reader analyses. Annotating findings with specific, interpretive labels were found to be difficult for less experienced radiologists. Annotating images with descriptive labels may increase agreement between radiologists with different experience levels compared to annotation with interpretive labels.
Collapse
Affiliation(s)
- Dana Li
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2100 Copenhagen, Denmark
- Correspondence:
| | - Lea Marie Pehrson
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Computer Science, University of Copenhagen, 2100 Copenhagen, Denmark
| | | | | | | | | | - Peter Jagd Sørensen
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2100 Copenhagen, Denmark
| | - Alexander Rykkje
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2100 Copenhagen, Denmark
| | - Tobias Thostrup Andersen
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
| | - Henrik Steglich-Arnholm
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
| | - Dorte Marianne Rohde Stærk
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
| | - Lotte Borgwardt
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
| | - Kristoffer Lindskov Hansen
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2100 Copenhagen, Denmark
| | - Sune Darkner
- Department of Computer Science, University of Copenhagen, 2100 Copenhagen, Denmark
| | - Jonathan Frederik Carlsen
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2100 Copenhagen, Denmark
| | - Michael Bachmann Nielsen
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2100 Copenhagen, Denmark
| |
Collapse
|
2
|
Chen X, Yan J, Xiang L, Lin F. Misdiagnosis of a patient with congenital dysfibrinogenemia: A case report and literature review. J Clin Lab Anal 2022; 36:e24624. [PMID: 35949040 PMCID: PMC9459280 DOI: 10.1002/jcla.24624] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2022] [Revised: 06/26/2022] [Accepted: 07/13/2022] [Indexed: 12/17/2022] Open
Abstract
Background We reported a patient with congenital dysfibrinogenemia who was misdiagnosed and reviewed relevant literature, in order to discuss the methods to reduce misdiagnosis. Methods A 23‐year‐old pregnant woman was found to be with low fibrinogen in antenatal examination at another province teaching hospital, who was misdiagnosed to have hypofibrinogenemia. Fibrinogen infusion or cryoprecipitation was recommended if necessary. The patient came to our hospital for further diagnosis and treatment considering the safety of herself and the fetus. We examined the coagulation function and gene sequencing of the pregnant woman and her family members. Results Fibrinogen (Clauss method) was significantly reduced in the patient and her mother, while the level of fibrinogen (PT‐derived method) was normal. Thrombin time was prolonged. Heterozygous mutation site was found in exon 2 of the FGA gene, c.104G > A(p.Arg35His). Conclusion When the fibrinogen (Clauss method) is significantly reduced and the thrombin time is prolonged, PT‐derived method and the investigation of family coagulation function should be added, which can be used to diagnose and distinguish congenital dysfibrinogenemia from hypofibrinogenemia.
Collapse
Affiliation(s)
- Xinyan Chen
- Department of Clinical Laboratory, The First Affiliated Hospital of Guangxi Medical University, Nanning, China
| | - Jie Yan
- Department of Clinical Laboratory, The First Affiliated Hospital of Guangxi Medical University, Nanning, China
| | - Liqun Xiang
- Department of Clinical Laboratory, The First Affiliated Hospital of Guangxi Medical University, Nanning, China
| | - Faquan Lin
- Department of Clinical Laboratory, The First Affiliated Hospital of Guangxi Medical University, Nanning, China
| |
Collapse
|