Sonographie in der Nachsorge bei Kopf- und Halskarzinomen

Structured Reporting of Head and Neck Sonography Achieves Substantial Interrater Reliability

Purpose Ultrasound examinations are often criticized for having higher examiner dependency compared to other imaging techniques. Compared to free-text reporting, structured reporting (SR) of head and neck sonography (HNS) achieves superior time efficiency as well as report quality. However, there are no findings concerning the influence of SR on the interrater reliability (IRR) of HNS. Materials and Methods Typical pathologies (n=4) in HNS were documented by video/images by two certified head and neck ultrasound instructors. Consequently, structured reports of these videos/images were created by n=9 senior physicians at departments of otolaryngology or maxillofacial surgery with DEGUM instructors on staff. Reports (n=36) were evaluated regarding overall completeness and IRR. Additionally, user satisfaction was assessed by a visual analog scale (VAS). Results SR yielded very high report completeness (91.8%) in all four cases with a substantial IRR (Fleiss' κ 0.73). Interrater agreement was high at 87.2% with very good user satisfaction (VAS 8.6). Conclusion SR has the potential to ensure high-quality examination reports with substantial comparability and very high user satisfaction. Furthermore, big data collection and analysis are facilitated by SR. Therefore, process quality, workflow, and scientific output are potentially enhanced by SR.

[Quality in the appraisal of head and neck sonography results in university hospitals-a random sample]

Hintergrund

Die Ultraschalldiagnostik gilt für den Radiologen, Hals-Nasen-Ohren-Arzt (HNO) oder Mund-Kiefer-Gesichts-Chirurgen als Standard in der Abklärung zahlreicher Pathologien. Es besteht ein Konsens, dass die digitale Dokumentation heute dringend notwendig ist, um die Qualität der sonographischen Dokumentationen zu verbessern und zu standardisieren. Es häufen sich Publikationen zur Implementierung standardisierter Befunddokumentation einschließlich der Kopf- und Halssonographie.

Ziel der Arbeit

Die vorliegende Arbeit zielt darauf ab, die Qualität von routinemäßig angefertigten Kopf- und Halssonographiebefunden nach Kriterien der Kassenärztlichen Vereinigung (KV) Bayern an einer Auswahl deutscher HNO-Universitätskliniken stichprobenartig zu ermitteln.

Material und Methoden

Insgesamt wurden retrospektiv 70 zufällig ausgewählte, anonymisierte schriftliche Befunde einschließlich Bildmaterial von insgesamt 7 HNO-Universitätskliniken stichprobenartig nach KV-Kriterien durch einen erfahrenen Prüfer der KV Bayern ausgewertet und deskriptiv analysiert.

Ergebnisse

Von 70 Befunden konnten 69 ausgewertet werden. Die Dokumentationsvollständigkeit lag im Mittel bei 80,6 %. Neun Befunde waren vollständig korrekt dokumentiert (13 %). Die Dokumentationsvollständigkeit der einzelnen Kliniken lag zwischen 68,1 % und 93 %. Mit 88,5 % vs. 75 % erbrachte eine strukturierte Befundung eine höhere Befundvollständigkeit. In 75 % der Fälle verfügten die Kliniken mit strukturiertem Befund auch über digitale Dokumentationslösungen.

Schlussfolgerung

Die Vollständigkeit und Qualität von routinemäßig angefertigten Kopf- und Halssonographiebefunden an einer Auswahl von HNO-Universitätskliniken ist insgesamt optimierbar. Die Implementierung strukturierter Befundmasken und die Umstellung der analogen Dokumentation auf digitale Lösungen sowie Vernetzung mit dem Klinikinformationssystem (KIS) und Bildarchivierungs- und Kommunikationssystem (PACS) sollte weiter vorangetrieben werden. Darüber hinaus sind leitende Ärzte dazu angehalten, die Befundqualität unerfahrener Kollegen regelmäßig zu prüfen und im Rahmen der Facharztausbildung auf die Erfüllung entsprechender Standards wie der KV-Ultraschallvereinbarung hinzuarbeiten.

The use of structured reporting of head and neck ultrasound ensures time-efficiency and report quality during residency

PURPOSE

Free text reports (FTR) of head and neck ultrasound studies are currently deployed in most departments. Because of a lack of composition and language, these reports vary greatly in terms of quality and reliability. This may impair the learning process during residency. The purpose of the study was to analyze the longitudinal effects of using structured reports (SR) of head and neck ultrasound studies during residency.

METHODS

Attending residents (n = 24) of a tripartite course on head and neck ultrasound, accredited by the German Society for Ultrasound in Medicine (DEGUM), were randomly allocated to pictures of common diseases. Both SRs and FTRs were compiled. All reports were analyzed concerning completeness, acquired time and legibility. Overall user contentment was evaluated by a questionnaire.

RESULTS

SRs achieved significantly higher ratings regarding completeness (95.6% vs. 26.4%, p < 0.001), description of pathologies (72.2% vs. 58.9%, p < 0.001) and legibility (100% vs. 52.4%, p < 0.001) with a very high inter-rater reliability (Fleiss' kappa 0.9). Reports were finalized significantly faster (99.1 s vs. 115.0 s, p < 0.001) and user contentment was significantly better when using SRs (8.3 vs. 6.3, p < 0.001). In particular, only SRs showed a longitudinally increasing time efficiency (- 20.1 s, p = 0.036) while maintaining consistent completeness ratings.

CONCLUSIONS

The use of SRs of head and neck ultrasound studies results in an increased longitudinal time-efficiency while upholding the report quality at the same time. This may indicate an additive learning effect of structured reporting. Superior outcomes in terms of comprehensiveness, legibility and time-efficiency can be observed immediately after implementation.

Impact of structured reporting on developing head and neck ultrasound skills

BACKGROUND

Reports of head and neck ultrasound examinations are frequently written by hand as free texts. This is a serious obstacle to the learning process of the modality due to a missing report structure and terminology. Therefore, there is a great inter-observer variability in overall report quality. Aim of the present study was to evaluate the impact of structured reporting on the learning process as indicated by the overall report quality of head and neck ultrasound examinations within medical school education.

METHODS

Following an immersion course on head and neck ultrasound, previously documented images of three common pathologies were handed out to 58 medical students who asked to create both standard free text reports (FTR) and structured reports (SR). A template for structured reporting of head and neck ultrasound examinations was created using a web-based approach. FTRs and SRs were evaluated with regard to overall quality, completeness, required time to completion and readability by two independent raters (Paired Wilcoxon test, 95% CI). Ratings were assessed for inter-rater reliability (Fleiss' kappa). Additionally, a questionnaire was utilized to evaluate user satisfaction.

RESULTS

SRs received significantly better ratings in terms of report completeness (97.7% vs. 53.5%, p < 0.001) regarding all items. In addition, pathologies were described in more detail using SRs (70% vs. 51.1%, p < 0.001). Readability was significantly higher in all SRs when compared to FTRs (100% vs. 54.4%, p < 0.001). Mean time to complete was significantly lower (79.6 vs. 205.4 s, p < 0.001) and user satisfaction was significantly higher when using SRs (8.5 vs. 4.1, p < 0.001). Also, inter-rater reliability was very high (Fleiss' kappa 0.93).

CONCLUSIONS

SRs of head and neck ultrasound examinations provide more detailed information with a better readability in a time-saving manner within medical education. Also, medical students may benefit from SRs in their learning process due to the structured approach and standardized terminology.

Structured reporting of head and neck ultrasound examinations

Background

Reports of head and neck ultrasound examinations are frequently written by hand as free texts. Naturally, quality and structure of free text reports is variable, depending on the examiner’s individual level of experience. Aim of the present study was to compare the quality of free text reports (FTR) and structured reports (SR) of head and neck ultrasound examinations.

Methods

Both standard FTRs and SRs of head and neck ultrasound examinations of 43 patients were acquired by nine independent examiners with comparable levels of experience. A template for structured reporting of head and neck ultrasound examinations was created using a web-based approach. FTRs and SRs were evaluated with regard to overall quality, completeness, required time to completion, and readability by four independent raters with different specializations (Paired Wilcoxon test, 95% CI) and inter-rater reliability was assessed (Fleiss’ kappa). A questionnaire was used to compare FTRs vs. SRs with respect to user satisfaction (Mann-Whitney U test, 95% CI).

Results

By comparison, completeness scores of SRs were significantly higher than FTRs’ completeness scores (94.4% vs. 45.6%, p < 0.001), and pathologies were described in more detail (91.1% vs. 54.5%, p < 0.001). Readability was significantly higher in all SRs when compared to FTRs (100% vs. 47.1%, p < 0.001). The mean time to complete a report, however, was significantly higher in SRs (176.5 vs. 107.3 s, p < 0.001). SRs achieved significantly higher user satisfaction ratings (VAS 8.87 vs. 1.41, p < 0.001) and a very high inter-rater reliability (Fleiss’ kappa 0.92).

Conclusions

As compared to FTRs, SRs of head and neck ultrasound examinations are more comprehensive and easier to understand. On the balance, the additional time needed for completing a SR is negligible. Also, SRs yield high inter-rater reliability and may be used for high-quality scientific data analyses.