1
|
Ernst BP, Dörsching C, Bozzato A, Gabrielpillai J, Becker S, Froelich MF, Kramer B, Sproll C, Schapher M, Goncalves M, Mansour N, Hofauer B, Sommer WH, von Scotti F, Weimer JM, Künzel J. Structured Reporting of Head and Neck Sonography Achieves Substantial Interrater Reliability. Ultrasound Int Open 2023; 9:E26-E32. [PMID: 37808417 PMCID: PMC10556873 DOI: 10.1055/a-2173-3966] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2023] [Accepted: 08/14/2023] [Indexed: 10/10/2023] Open
Abstract
Purpose Ultrasound examinations are often criticized for having higher examiner dependency compared to other imaging techniques. Compared to free-text reporting, structured reporting (SR) of head and neck sonography (HNS) achieves superior time efficiency as well as report quality. However, there are no findings concerning the influence of SR on the interrater reliability (IRR) of HNS. Materials and Methods Typical pathologies (n=4) in HNS were documented by video/images by two certified head and neck ultrasound instructors. Consequently, structured reports of these videos/images were created by n=9 senior physicians at departments of otolaryngology or maxillofacial surgery with DEGUM instructors on staff. Reports (n=36) were evaluated regarding overall completeness and IRR. Additionally, user satisfaction was assessed by a visual analog scale (VAS). Results SR yielded very high report completeness (91.8%) in all four cases with a substantial IRR (Fleiss' κ 0.73). Interrater agreement was high at 87.2% with very good user satisfaction (VAS 8.6). Conclusion SR has the potential to ensure high-quality examination reports with substantial comparability and very high user satisfaction. Furthermore, big data collection and analysis are facilitated by SR. Therefore, process quality, workflow, and scientific output are potentially enhanced by SR.
Collapse
Affiliation(s)
| | - Carla Dörsching
- Department of Otorhinolaryngology, University Medical Center Bonn,
Bonn, Germany
| | - Alessandro Bozzato
- Department of Otorhinolaryngology, Head & Neck Surgery,
Saarland University Hospital and Saarland University Faculty of Medicine,
Homburg, Germany
| | - Jennis Gabrielpillai
- Department of Otorhinolaryngology, University Medical Center Bonn,
Bonn, Germany
| | - Sven Becker
- Department of Otorhinolaryngology, Head and Neck Surgery, University of
Tübingen Medical Center, Tuebingen, Germany
| | - Matthias Frank Froelich
- Department of Radiology and Nuclear Medicine, University Medical Centre
Mannheim, Mannheim, Germany
| | - Benedikt Kramer
- Department of Otorhinolaryngology, Head and Neck Surgery, University
Medical Centre Mannheim, Mannheim, Germany
| | - Christoph Sproll
- Department of Oral and Maxillofacial Surgery, Medical Faculty and
University Hospital Düsseldorf, Duesseldorf, Germany
| | - Mirco Schapher
- Department of Otorhinolaryngology, Head and Neck Surgery, Paracelsus
Medical University, Nuremberg, Germany
| | - Miguel Goncalves
- Department of Otorhinolaryngology, Plastic Head and Neck Surgery, RWTH
Aachen University Hospital, Aachen, Germany
| | - Naglaa Mansour
- Department of Otorhinolaryngology, University Medical Center Freiburg,
Freiburg, Germany
| | - Benedikt Hofauer
- Department of Otorhinolaryngology, Head and Neck Surgery, Technical
University of Munich Hospital Rechts der Isar, Munich, Germany
| | - Wieland H Sommer
- Department of Radiology, LMU University Hospital, Munich,
Germany
| | - Felix von Scotti
- Ultrasound Division, Otorhinolaryngology Center Münsterland,
Münster, Germany
| | - Johannes Matthias Weimer
- Rudolf-Frey Teaching Department, University Medical Center of the
Johannes Gutenberg University Mainz, Mainz, Germany
| | - Julian Künzel
- Department of Otorhinolaryngology, Universitätsklinikum
Regensburg, Regensburg, Germany
| |
Collapse
|
2
|
Weimer JM, Rink M, Müller L, Arens C, Bozzato A, Künzel J. Sonografische Diagnostik im Kopf-Hals-Bereich – Teil 2: Transzervikale Sonografie. Laryngorhinootologie 2022; 101:156-175. [DOI: 10.1055/a-1667-8675] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Affiliation(s)
| | | | | | | | | | - Julian Künzel
- Klinik und Poliklinik für Hals-Nasen-Ohren-Heilkunde, Universitätsklinikum Regensburg, Regensburg
| |
Collapse
|
3
|
[Quality in the appraisal of head and neck sonography results in university hospitals-a random sample]. HNO 2021; 69:907-912. [PMID: 33439274 PMCID: PMC8545731 DOI: 10.1007/s00106-020-00989-9] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/21/2020] [Indexed: 02/07/2023]
Abstract
Hintergrund Die Ultraschalldiagnostik gilt für den Radiologen, Hals-Nasen-Ohren-Arzt (HNO) oder Mund-Kiefer-Gesichts-Chirurgen als Standard in der Abklärung zahlreicher Pathologien. Es besteht ein Konsens, dass die digitale Dokumentation heute dringend notwendig ist, um die Qualität der sonographischen Dokumentationen zu verbessern und zu standardisieren. Es häufen sich Publikationen zur Implementierung standardisierter Befunddokumentation einschließlich der Kopf- und Halssonographie. Ziel der Arbeit Die vorliegende Arbeit zielt darauf ab, die Qualität von routinemäßig angefertigten Kopf- und Halssonographiebefunden nach Kriterien der Kassenärztlichen Vereinigung (KV) Bayern an einer Auswahl deutscher HNO-Universitätskliniken stichprobenartig zu ermitteln. Material und Methoden Insgesamt wurden retrospektiv 70 zufällig ausgewählte, anonymisierte schriftliche Befunde einschließlich Bildmaterial von insgesamt 7 HNO-Universitätskliniken stichprobenartig nach KV-Kriterien durch einen erfahrenen Prüfer der KV Bayern ausgewertet und deskriptiv analysiert. Ergebnisse Von 70 Befunden konnten 69 ausgewertet werden. Die Dokumentationsvollständigkeit lag im Mittel bei 80,6 %. Neun Befunde waren vollständig korrekt dokumentiert (13 %). Die Dokumentationsvollständigkeit der einzelnen Kliniken lag zwischen 68,1 % und 93 %. Mit 88,5 % vs. 75 % erbrachte eine strukturierte Befundung eine höhere Befundvollständigkeit. In 75 % der Fälle verfügten die Kliniken mit strukturiertem Befund auch über digitale Dokumentationslösungen. Schlussfolgerung Die Vollständigkeit und Qualität von routinemäßig angefertigten Kopf- und Halssonographiebefunden an einer Auswahl von HNO-Universitätskliniken ist insgesamt optimierbar. Die Implementierung strukturierter Befundmasken und die Umstellung der analogen Dokumentation auf digitale Lösungen sowie Vernetzung mit dem Klinikinformationssystem (KIS) und Bildarchivierungs- und Kommunikationssystem (PACS) sollte weiter vorangetrieben werden. Darüber hinaus sind leitende Ärzte dazu angehalten, die Befundqualität unerfahrener Kollegen regelmäßig zu prüfen und im Rahmen der Facharztausbildung auf die Erfüllung entsprechender Standards wie der KV-Ultraschallvereinbarung hinzuarbeiten.
Collapse
|
4
|
The use of structured reporting of head and neck ultrasound ensures time-efficiency and report quality during residency. Eur Arch Otorhinolaryngol 2019; 277:269-276. [PMID: 31612337 DOI: 10.1007/s00405-019-05679-z] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2019] [Accepted: 10/01/2019] [Indexed: 02/07/2023]
Abstract
PURPOSE Free text reports (FTR) of head and neck ultrasound studies are currently deployed in most departments. Because of a lack of composition and language, these reports vary greatly in terms of quality and reliability. This may impair the learning process during residency. The purpose of the study was to analyze the longitudinal effects of using structured reports (SR) of head and neck ultrasound studies during residency. METHODS Attending residents (n = 24) of a tripartite course on head and neck ultrasound, accredited by the German Society for Ultrasound in Medicine (DEGUM), were randomly allocated to pictures of common diseases. Both SRs and FTRs were compiled. All reports were analyzed concerning completeness, acquired time and legibility. Overall user contentment was evaluated by a questionnaire. RESULTS SRs achieved significantly higher ratings regarding completeness (95.6% vs. 26.4%, p < 0.001), description of pathologies (72.2% vs. 58.9%, p < 0.001) and legibility (100% vs. 52.4%, p < 0.001) with a very high inter-rater reliability (Fleiss' kappa 0.9). Reports were finalized significantly faster (99.1 s vs. 115.0 s, p < 0.001) and user contentment was significantly better when using SRs (8.3 vs. 6.3, p < 0.001). In particular, only SRs showed a longitudinally increasing time efficiency (- 20.1 s, p = 0.036) while maintaining consistent completeness ratings. CONCLUSIONS The use of SRs of head and neck ultrasound studies results in an increased longitudinal time-efficiency while upholding the report quality at the same time. This may indicate an additive learning effect of structured reporting. Superior outcomes in terms of comprehensiveness, legibility and time-efficiency can be observed immediately after implementation.
Collapse
|
5
|
Ernst BP, Katzer F, Künzel J, Hodeib M, Strieth S, Eckrich J, Tattermusch A, Froelich MF, Matthias C, Sommer WH, Becker S. Impact of structured reporting on developing head and neck ultrasound skills. BMC MEDICAL EDUCATION 2019; 19:102. [PMID: 30971248 PMCID: PMC6458758 DOI: 10.1186/s12909-019-1538-6] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/07/2018] [Accepted: 03/31/2019] [Indexed: 05/06/2023]
Abstract
BACKGROUND Reports of head and neck ultrasound examinations are frequently written by hand as free texts. This is a serious obstacle to the learning process of the modality due to a missing report structure and terminology. Therefore, there is a great inter-observer variability in overall report quality. Aim of the present study was to evaluate the impact of structured reporting on the learning process as indicated by the overall report quality of head and neck ultrasound examinations within medical school education. METHODS Following an immersion course on head and neck ultrasound, previously documented images of three common pathologies were handed out to 58 medical students who asked to create both standard free text reports (FTR) and structured reports (SR). A template for structured reporting of head and neck ultrasound examinations was created using a web-based approach. FTRs and SRs were evaluated with regard to overall quality, completeness, required time to completion and readability by two independent raters (Paired Wilcoxon test, 95% CI). Ratings were assessed for inter-rater reliability (Fleiss' kappa). Additionally, a questionnaire was utilized to evaluate user satisfaction. RESULTS SRs received significantly better ratings in terms of report completeness (97.7% vs. 53.5%, p < 0.001) regarding all items. In addition, pathologies were described in more detail using SRs (70% vs. 51.1%, p < 0.001). Readability was significantly higher in all SRs when compared to FTRs (100% vs. 54.4%, p < 0.001). Mean time to complete was significantly lower (79.6 vs. 205.4 s, p < 0.001) and user satisfaction was significantly higher when using SRs (8.5 vs. 4.1, p < 0.001). Also, inter-rater reliability was very high (Fleiss' kappa 0.93). CONCLUSIONS SRs of head and neck ultrasound examinations provide more detailed information with a better readability in a time-saving manner within medical education. Also, medical students may benefit from SRs in their learning process due to the structured approach and standardized terminology.
Collapse
Affiliation(s)
- Benjamin P. Ernst
- Department of Otorhinolaryngology, University Medical Center Mainz, Langenbeckstraße 1, 55131 Mainz, Germany
| | - Fabian Katzer
- Department of Otorhinolaryngology, University Medical Center Mainz, Langenbeckstraße 1, 55131 Mainz, Germany
| | - Julian Künzel
- Department of Otorhinolaryngology, University Medical Center Mainz, Langenbeckstraße 1, 55131 Mainz, Germany
| | - Mohamed Hodeib
- Department of Otorhinolaryngology, University Medical Center Mainz, Langenbeckstraße 1, 55131 Mainz, Germany
| | - Sebastian Strieth
- Department of Otorhinolaryngology, University Medical Center Mainz, Langenbeckstraße 1, 55131 Mainz, Germany
| | - Jonas Eckrich
- Department of Otorhinolaryngology, University Medical Center Mainz, Langenbeckstraße 1, 55131 Mainz, Germany
| | | | - Matthias F. Froelich
- Institute of Clinical Radiology and Nuclear Medicine, Institute of Clinical Radiology and Nuclear Medicine, Faculty Mannheim-Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany
| | - Christoph Matthias
- Department of Otorhinolaryngology, University Medical Center Mainz, Langenbeckstraße 1, 55131 Mainz, Germany
| | - Wieland H. Sommer
- Department of Radiology, LMU University Hospital, Marchioninistraße 15, 81377 Munich, Germany
| | - Sven Becker
- Department of Otorhinolaryngology, University Medical Center Mainz, Langenbeckstraße 1, 55131 Mainz, Germany
- Department of Otolaryngology, Head and Neck Surgery, University of Tübingen, Elfriede-Aulhorn-Straße 5, 72076 Tübingen, Germany
| |
Collapse
|
6
|
Ernst BP, Hodeib M, Strieth S, Künzel J, Bischof F, Hackenberg B, Huppertz T, Weber V, Bahr K, Eckrich J, Hagemann J, Engelbarts M, Froelich MF, Solbach P, Linke R, Matthias C, Sommer WH, Becker S. Structured reporting of head and neck ultrasound examinations. BMC Med Imaging 2019; 19:25. [PMID: 30917796 PMCID: PMC6437950 DOI: 10.1186/s12880-019-0325-5] [Citation(s) in RCA: 16] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2018] [Accepted: 03/11/2019] [Indexed: 12/25/2022] Open
Abstract
Background Reports of head and neck ultrasound examinations are frequently written by hand as free texts. Naturally, quality and structure of free text reports is variable, depending on the examiner’s individual level of experience. Aim of the present study was to compare the quality of free text reports (FTR) and structured reports (SR) of head and neck ultrasound examinations. Methods Both standard FTRs and SRs of head and neck ultrasound examinations of 43 patients were acquired by nine independent examiners with comparable levels of experience. A template for structured reporting of head and neck ultrasound examinations was created using a web-based approach. FTRs and SRs were evaluated with regard to overall quality, completeness, required time to completion, and readability by four independent raters with different specializations (Paired Wilcoxon test, 95% CI) and inter-rater reliability was assessed (Fleiss’ kappa). A questionnaire was used to compare FTRs vs. SRs with respect to user satisfaction (Mann-Whitney U test, 95% CI). Results By comparison, completeness scores of SRs were significantly higher than FTRs’ completeness scores (94.4% vs. 45.6%, p < 0.001), and pathologies were described in more detail (91.1% vs. 54.5%, p < 0.001). Readability was significantly higher in all SRs when compared to FTRs (100% vs. 47.1%, p < 0.001). The mean time to complete a report, however, was significantly higher in SRs (176.5 vs. 107.3 s, p < 0.001). SRs achieved significantly higher user satisfaction ratings (VAS 8.87 vs. 1.41, p < 0.001) and a very high inter-rater reliability (Fleiss’ kappa 0.92). Conclusions As compared to FTRs, SRs of head and neck ultrasound examinations are more comprehensive and easier to understand. On the balance, the additional time needed for completing a SR is negligible. Also, SRs yield high inter-rater reliability and may be used for high-quality scientific data analyses.
Collapse
Affiliation(s)
- Benjamin P Ernst
- Department of Otorhinolaryngology, University Medical Center Mainz, Langenbeckstraße 1, 55131, Mainz, Germany.
| | - Mohamed Hodeib
- Department of Otorhinolaryngology, University Medical Center Mainz, Langenbeckstraße 1, 55131, Mainz, Germany
| | - Sebastian Strieth
- Department of Otorhinolaryngology, University Medical Center Mainz, Langenbeckstraße 1, 55131, Mainz, Germany
| | - Julian Künzel
- Department of Otorhinolaryngology, University Medical Center Mainz, Langenbeckstraße 1, 55131, Mainz, Germany
| | - Fabian Bischof
- Department of Otorhinolaryngology, University Medical Center Mainz, Langenbeckstraße 1, 55131, Mainz, Germany
| | - Berit Hackenberg
- Department of Otorhinolaryngology, University Medical Center Mainz, Langenbeckstraße 1, 55131, Mainz, Germany
| | - Tilmann Huppertz
- Department of Otorhinolaryngology, University Medical Center Mainz, Langenbeckstraße 1, 55131, Mainz, Germany
| | - Veronika Weber
- Department of Otorhinolaryngology, University Medical Center Mainz, Langenbeckstraße 1, 55131, Mainz, Germany
| | - Katharina Bahr
- Department of Otorhinolaryngology, University Medical Center Mainz, Langenbeckstraße 1, 55131, Mainz, Germany
| | - Jonas Eckrich
- Department of Otorhinolaryngology, University Medical Center Mainz, Langenbeckstraße 1, 55131, Mainz, Germany
| | - Jan Hagemann
- Department of Otorhinolaryngology, University Medical Center Mainz, Langenbeckstraße 1, 55131, Mainz, Germany
| | - Matthias Engelbarts
- Department of Otorhinolaryngology, University Medical Center Mainz, Langenbeckstraße 1, 55131, Mainz, Germany
| | - Matthias F Froelich
- Department of Radiology, LMU University Hospital, Marchioninistraße 15, 81377, Munich, Germany
| | - Philipp Solbach
- Department of Gastroenterology, Hepatology and Endocrinology, Hannover Medical School, Carl-Neuberg-Straße 1, 30625, Hannover, Germany
| | - Richard Linke
- Department of General and Visceral Surgery, University Hospital Frankfurt, Theodor-Stern-Kai 7, 60590, Frankfurt, Germany
| | - Christoph Matthias
- Department of Otorhinolaryngology, University Medical Center Mainz, Langenbeckstraße 1, 55131, Mainz, Germany
| | - Wieland H Sommer
- Department of Radiology, LMU University Hospital, Marchioninistraße 15, 81377, Munich, Germany
| | - Sven Becker
- Department of Otorhinolaryngology, University Medical Center Mainz, Langenbeckstraße 1, 55131, Mainz, Germany
| |
Collapse
|