Identification of barriers and facilitators in nationwide implementation of standardized structured reporting in pathology: a mixed method study

Extraction and classification of structured data from unstructured hepatobiliary pathology reports using large language models: a feasibility study compared with rules-based natural language processing

AIMS

Structured reporting in pathology is not universally adopted and extracting elements essential to research often requires expensive and time-intensive manual curation. The accuracy and feasibility of using large language models (LLMs) to extract essential pathology elements, for cancer research is examined here.

METHODS

Retrospective study of patients who underwent pathology sampling for suspected hepatocellular carcinoma and underwent Ytrrium-90 embolisation. Five pathology report elements of interest were included for evaluation. LLMs (Generative Pre-trained Transformer (GPT) 3.5 turbo and GPT-4) were used to extract elements of interest. For comparison, a rules-based, regular expressions (REGEX) approach was devised for extraction. Accuracy for each approach was calculated.

RESULTS

88 pathology reports were identified. LLMs and REGEX were both able to extract research elements with high accuracy (average 84.1%-94.8%).

CONCLUSIONS

LLMs have significant potential to simplify the extraction of research elements from pathology reporting, and therefore, accelerate the pace of cancer research.

Recommendations for using artificial intelligence in clinical flow cytometry

Flow cytometry is a key clinical tool in the diagnosis of many hematologic malignancies and traditionally requires close inspection of digital data by hematopathologists with expert domain knowledge. Advances in artificial intelligence (AI) are transferable to flow cytometry and have the potential to improve efficiency and prioritization of cases, reduce errors, and highlight fundamental, previously unrecognized associations with underlying biological processes. As a multidisciplinary group of stakeholders, we review a range of critical considerations for appropriately applying AI to clinical flow cytometry, including use case identification, low and high risk use cases, validation, revalidation, computational considerations, and the present regulatory frameworks surrounding AI in clinical medicine. In particular, we provide practical guidance for the development, implementation, and suggestions for potential regulation of AI-based methods in the clinical flow cytometry laboratory. We expect these recommendations to be a helpful initial framework of reference, which will also require additional updates as the field matures.

Extracting structured information from unstructured histopathology reports using generative pre-trained transformer 4 (GPT-4)

Deep learning applied to whole-slide histopathology images (WSIs) has the potential to enhance precision oncology and alleviate the workload of experts. However, developing these models necessitates large amounts of data with ground truth labels, which can be both time-consuming and expensive to obtain. Pathology reports are typically unstructured or poorly structured texts, and efforts to implement structured reporting templates have been unsuccessful, as these efforts lead to perceived extra workload. In this study, we hypothesised that large language models (LLMs), such as the generative pre-trained transformer 4 (GPT-4), can extract structured data from unstructured plain language reports using a zero-shot approach without requiring any re-training. We tested this hypothesis by utilising GPT-4 to extract information from histopathological reports, focusing on two extensive sets of pathology reports for colorectal cancer and glioblastoma. We found a high concordance between LLM-generated structured data and human-generated structured data. Consequently, LLMs could potentially be employed routinely to extract ground truth data for machine learning from unstructured pathology reports in the future. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Impact of template-based synoptic reporting on completeness of surgical pathology reports

Synoptic reporting increases completeness and standardization of surgical pathology reports and thereby contributes to an increased quality of clinical cancer care. Nevertheless, its widespread practical implementation remains a challenge, which is in part related to the effort required for setup and maintenance of database structures. This prompted us to assess the effect of a simple template-based, database-free system for synoptic reporting on completeness of surgical pathology reports. For this purpose, we analyzed 200 synoptic reports (100 colon and 100 lung cancer resections each) for completeness as required by the pertinent College of American Pathologists (CAP) protocols and compared these to a control dataset of 200 narrative reports. Introduction of template-based synoptic reporting resulted in improved completeness (98% of mandatory data elements) as compared to narrative reports (77%). Narrative reports showed a high degree of completeness for data elements covered by previously existing dictation templates. In conclusion, template-based synoptic reporting without underlying database structure can be a useful transitory phase in the implementation of synoptic reporting. It can result in a similar degree of completeness as reported in the literature for database solutions and provides other benefits of synoptic reporting while facilitating its implementation.

Datasets for the reporting of primary tumour in bone: recommendations from the International Collaboration on Cancer Reporting (ICCR)

BACKGROUND AND OBJECTIVES

Bone tumours are relatively rare and, as a consequence, treatment in a centre with expertise is required. Current treatment guidelines also recommend review by a specialised pathologist. Here we report on international consensus-based datasets for the pathology reporting of biopsy and resection specimens of bone sarcomas. The datasets were produced under the auspices of the International Collaboration on Cancer Reporting (ICCR), a global alliance of major (inter-)national pathology and cancer organisations.

METHODS AND RESULTS

According to the ICCR's process for dataset development, an international expert panel consisting of pathologists, an oncologic orthopaedic surgeon, a medical oncologist, and a radiologist produced a set of core and noncore data items for biopsy and resection specimens based on a critical review and discussion of current evidence. All professionals involved were bone tumour experts affiliated with tertiary referral centres. Commentary was provided for each data item to explain the rationale for selecting it as a core or noncore element, its clinical relevance, and to highlight potential areas of disagreement or lack of evidence, in which case a consensus position was formulated. Following international public consultation, the documents were finalised and ratified, and the datasets, including a synoptic reporting guide, were published on the ICCR website.

CONCLUSION

These first international datasets for bone sarcomas are intended to promote high-quality, standardised pathology reporting. Their widespread adoption will improve the consistency of reporting, facilitate multidisciplinary communication, and enhance comparability of data, all of which will help to improve management of bone sarcoma patients.

Dataset for reporting of gastrointestinal stromal tumours: recommendations from the International Collaboration on Cancer Reporting (ICCR)

Gastrointestinal stromal tumours (GISTs) are the most common mesenchymal tumours of the gastrointestinal tract and are among the most frequent sarcomas. Accurate diagnosis, classification, and reporting are critical for prognostication and patient management, including selection of appropriate targeted therapy. Here we report on international consensus-based datasets for the pathology reporting of biopsy and resection specimens of GIST. The datasets were produced under the auspices of the International Collaboration on Cancer Reporting (ICCR), a global alliance of major international pathology and cancer organizations. An international expert panel consisting of pathologists, a surgical oncologist, and a medical oncologist produced a set of core and noncore data items for biopsy and resection specimens based on a critical review and discussion of current evidence. All professionals involved were subspecialized soft tissue tumour experts and affiliated with tertiary referral centres. Commentary was provided for each data item to explain its clinical relevance and the rationale for selection as a core or noncore element. Following international public consultation, the datasets, which include synoptic reporting guides, were finalized and ratified, and published on the ICCR website. These first international datasets for GIST are intended to promote high-quality, standardised pathology reporting. Their widespread adoption will improve consistency of reporting, facilitate multidisciplinary communication, and enhance comparability of data, all of which will ultimately help to improve the management of patients with GIST. All the ICCR datasets, including these on GIST, are freely available worldwide on the ICCR website (www.iccr-cancer.org/datasets).

Development and Validation of an Algorithm to Identify Patients with Advanced Cutaneous Squamous Cell Carcinoma from Pathology Reports

To facilitate nationwide epidemiological research on advanced cutaneous squamous cell carcinoma (cSCC), that is, locally advanced, recurrent, or metastatic cSCC, we sought to develop and validate a rule-based algorithm that identifies advanced cSCC from pathology reports. The algorithm was based on both hierarchical histopathological codes and free text from pathology reports recorded in the National Pathology Registry. Medical files from the Erasmus Medical Center of 186 patients with stage III/IV/recurrent cSCC and 184 patients with stage I/II cSCC were selected and served as the gold standard to assess the performance of the algorithm. The rule-based algorithm showed a sensitivity of 91.9% (95% confidence interval = 88.0‒95.9), a specificity of 96.7% (95% confidence interval = 94‒2-99.3), and a positive predictive value of 78.5% (95% confidence interval = 74.2‒82.8) for all advanced cSCC combined. The sensitivity was lower per subgroup: locally advanced (52.3‒86.2%), recurrent cSCC (23.3%), and metastatic cSCC (70.0%). The specificity per subgroup was above 97%, and the positive predictive value was above 78%, with the exception of metastatic cSCC, which had a positive predictive value of 62%. This algorithm can be used to identify advanced patients with cSCC from pathology reports and will facilitate large-scale epidemiological studies of advanced cSCC in the Netherlands and internationally after external validation.

Nationwide implementation of a multifaceted tailored strategy to improve uptake of standardized structured reporting in pathology: an effect and process evaluation

Background

Implementation strategies are aimed at improving guideline adherence. Both effect and process evaluations are conducted to provide insights into the success or failure of these strategies. In our study, we evaluate the nationwide implementation of standardized structured reporting (SSR) in pathology.

Methods

An interrupted time series analysis was conducted to evaluate the effect of a previously developed implementation strategy, which consisted of various digitally available elements, on SSR in pathology laboratories. A segmented regression analysis was performed to analyze the change in mean SSR percentages directly after the strategy introduction for pathology reporting and specific subcategories. In addition, we analyzed the change in trend in the weekly percentages after strategy introduction, also for subgroups of tumor groups, retrieval methods, and type of laboratory. The change in SSR use after the strategy introduction was determined for all pathology laboratories. We further conducted a process evaluation in which the exposure to the strategy elements was determined. Experiences of the users with all strategy elements and the remaining barriers and potential strategy elements were evaluated through an eSurvey. We also tested whether exposure to a specific element and a combination of elements resulted in a higher uptake of SSR after strategy introduction.

Results

There was a significant increase in an average use of SSR after the strategy introduction for reporting of gastrointestinal (p=.018) and urological (p=.003) oncological diagnoses. A significant increase was present for all oncological resections as a group (p=.007). Thirty-three out of 42 pathology laboratories increased SSR use after the strategy introduction. The “Feedback button”, an option within the templates for SSR to provide feedback to the provider and one of the elements of the implementation strategy, was most frequently used by the SSR users, and effectiveness results showed that it increased average SSR use after the strategy introduction. Barriers were still present for SSR implementation.

Conclusions

Nationwide SSR implementation improved for specific tumor groups and retrieval methods. The next step will be to further improve the use of SSR and, simultaneously, to further develop potential benefits of high SSR use, focusing on re-using discrete pathology data. In this way, we can facilitate proper treatment decisions in oncology.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-022-01224-5.

Systematic reporting of medical kidney biopsies

The medical kidney biopsy has an important added value in patient care in nephrology. In order to facilitate communication between the pathologist and the nephrologist and optimize patient care, both the content and form of the medical kidney biopsy report matter. With some exceptions, current guidelines in nephropathology focus on content rather than form and, not surprisingly, medical kidney biopsy reports mostly consist of unformatted and often lengthy free text. In contrast, in oncology, a more systematic reporting called synoptic reporting has become the dominant method. Synoptic formats enable complete, concise and clear reports that comply with agreed upon standards. In this review we discuss the possibilities of systematic reporting in nephropathology (including synoptic reporting). Furthermore, we explore applications of electronic formats with structured data and usage of international terminologies or coding systems. The benefits include the timely collection of high-quality data for benchmarking between centres as well as for epidemiologic and other research studies. Based on these developments, a scenario for future medical kidney biopsy reporting is drafted.

Improving Interdisciplinary Communication: Barriers and Facilitators for Implementation of Standardized Structured Reporting in Oncology

BACKGROUND

Standardized structured reporting (SSR) improves quality of diagnostic cancer reporting and interdisciplinary communication in multidisciplinary team (MDT) meetings, resulting in more adequate treatment decisions and better health outcomes. However, use of SSR varies widely among pathologists, but might be encouraged by MDT members (MDTMs). Our objectives were to identify barriers and facilitators (influencing factors) for SSR implementation in oncologic pathology from the perspective of MDTMs and their determinants.

METHODS

In a multimethod design, we identified influencing factors for SSR implementation related to MDT meetings, using 5 domains: (1) innovation factors, (2) individual professional factors, (3) social setting factors, (4) organizational factors, and (5) political and legal factors. Four focus groups with MDTMs in urologic, gynecologic, and gastroenterologic oncology were conducted. We used an eSurvey among MDTMs to quantify the qualitative findings and to analyze determinants affecting these influencing factors.

RESULTS

Twenty-three MDTMs practicing in 9 oncology-related disciplines participated in the focus groups and yielded 28 barriers and 28 facilitators in all domains. The eSurvey yielded 211 responses. Main barriers related to lack of readability of SSR: difficulties with capturing nuances (66%) and formulation of the conclusion (43%); lack of transparency in the development (50%) and feedback processes of SSR templates (38%); and lack of information exchange about SSR between pathologists and other MDTMs (45%). Main facilitators were encouragement of pathologists' SSR use by MDTMs (90%) and expanding the recommendation of SSR use in national guidelines (80%). Oncology-related medical discipline and MDT type were the most relevant determinants for SSR implementation barriers.

CONCLUSIONS

Although SSR makes diagnostic reports more complete, this study shows important barriers in implementing SSR in oncologic pathology. The next step is to use these factors for developing and testing implementation tools to improve SSR implementation.

Impact on Quality of Documentation and Workload of the Introduction of a National Information Standard for Tumor Board Reporting

PURPOSE

Tumor boards, clinical practice guidelines, and cancer registries are intertwined cancer care quality instruments. Standardized structured reporting has been proposed as a solution to improve clinical documentation, while facilitating data reuse for secondary purposes. This study describes the implementation and evaluation of a national standard for tumor board reporting for breast cancer on the basis of the clinical practice guideline and the potential for reusing clinical data for the Netherlands Cancer Registry (NCR).

METHODS

Previously, a national information standard for breast cancer was derived from the corresponding Dutch clinical practice guideline. Using data items from the information standard, we developed three different tumor board forms: preoperative, postoperative, and postneoadjuvant-postoperative. The forms were implemented in Amphia Hospital’s electronic health record. Quality of clinical documentation and workload before and after implementation were compared.

RESULTS

Both draft and final tumor board reports were collected from 27 and 31 patients in baseline and effect measurements, respectively. Completeness of final reports increased from 39.5% to 45.4% (P = .04). The workload for tumor board preparation and discussion did not change significantly. Standardized tumor board reports included 50% (61/122) of the data items carried in the NCR. An automated process was developed to upload information captured in tumor board reports to the NCR database.

CONCLUSION

This study shows implementation of a national standard for tumor board reports improves quality of clinical documentation, without increasing clinical workload. Simultaneously, our work enables data reuse for secondary purposes like cancer registration.