1
|
Cortegoso Valdivia P, Deding U, Bjørsum-Meyer T, Baatrup G, Fernández-Urién I, Dray X, Boal-Carvalho P, Ellul P, Toth E, Rondonotti E, Kaalby L, Pennazio M, Koulaouzidis A. Inter/Intra-Observer Agreement in Video-Capsule Endoscopy: Are We Getting It All Wrong? A Systematic Review and Meta-Analysis. Diagnostics (Basel) 2022; 12:diagnostics12102400. [PMID: 36292089 PMCID: PMC9600122 DOI: 10.3390/diagnostics12102400] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2022] [Revised: 09/28/2022] [Accepted: 09/29/2022] [Indexed: 12/09/2022] Open
Abstract
Video-capsule endoscopy (VCE) reading is a time- and energy-consuming task. Agreement on findings between readers (either different or the same) is a crucial point for increasing performance and providing valid reports. The aim of this systematic review with meta-analysis is to provide an evaluation of inter/intra-observer agreement in VCE reading. A systematic literature search in PubMed, Embase and Web of Science was performed throughout September 2022. The degree of observer agreement, expressed with different test statistics, was extracted. As different statistics are not directly comparable, our analyses were stratified by type of test statistics, dividing them in groups of “None/Poor/Minimal”, “Moderate/Weak/Fair”, “Good/Excellent/Strong” and “Perfect/Almost perfect” to report the proportions of each. In total, 60 studies were included in the analysis, with a total of 579 comparisons. The quality of included studies, assessed with the MINORS score, was sufficient in 52/60 studies. The most common test statistics were the Kappa statistics for categorical outcomes (424 comparisons) and the intra-class correlation coefficient (ICC) for continuous outcomes (73 comparisons). In the overall comparison of inter-observer agreement, only 23% were evaluated as “good” or “perfect”; for intra-observer agreement, this was the case in 36%. Sources of heterogeneity (high, I2 81.8–98.1%) were investigated with meta-regressions, showing a possible role of country, capsule type and year of publication in Kappa inter-observer agreement. VCE reading suffers from substantial heterogeneity and sub-optimal agreement in both inter- and intra-observer evaluation. Artificial-intelligence-based tools and the adoption of a unified terminology may progressively enhance levels of agreement in VCE reading.
Collapse
Affiliation(s)
- Pablo Cortegoso Valdivia
- Gastroenterology and Endoscopy Unit, University Hospital of Parma, University of Parma, 43126 Parma, Italy
- Correspondence:
| | - Ulrik Deding
- Department of Clinical Research, University of Southern Denmark, 5230 Odense, Denmark
- Department of Surgery, Odense University Hospital, 5000 Odense, Denmark
| | - Thomas Bjørsum-Meyer
- Department of Clinical Research, University of Southern Denmark, 5230 Odense, Denmark
- Department of Surgery, Odense University Hospital, 5000 Odense, Denmark
| | - Gunnar Baatrup
- Department of Clinical Research, University of Southern Denmark, 5230 Odense, Denmark
- Department of Surgery, Odense University Hospital, 5000 Odense, Denmark
| | | | - Xavier Dray
- Center for Digestive Endoscopy, Sorbonne University, Saint Antoine Hospital, APHP, 75012 Paris, France
| | - Pedro Boal-Carvalho
- Gastroenterology Department, Hospital da Senhora da Oliveira, Creixomil, 4835 Guimarães, Portugal
| | - Pierre Ellul
- Division of Gastroenterology, Mater Dei Hospital, 2090 Msida, Malta
| | - Ervin Toth
- Department of Gastroenterology, Skåne University Hospital, Lund University, 20502 Malmö, Sweden
| | | | - Lasse Kaalby
- Department of Clinical Research, University of Southern Denmark, 5230 Odense, Denmark
- Department of Surgery, Odense University Hospital, 5000 Odense, Denmark
| | - Marco Pennazio
- University Division of Gastroenterology, City of Health and Science University Hospital, University of Turin, 10126 Turin, Italy
| | - Anastasios Koulaouzidis
- Department of Clinical Research, University of Southern Denmark, 5230 Odense, Denmark
- Department of Medicine, OUH Svendborg Sygehus, 5700 Svendborg, Denmark
- Surgical Research Unit, OUH, 5000 Odense, Denmark
- Department of Social Medicine and Public Health, Pomeranian Medical University, 70-204 Szczecin, Poland
| |
Collapse
|