Transformation of the National Breast Cancer Guideline Into Data-Driven Clinical Decision Trees

Clinical decision support systems for multidisciplinary team decision-making in patients with solid cancer: Composition of an implementation model based on a scoping review

Generating guideline-based recommendations during multidisciplinary team (MDT) meetings in solid cancers is getting more complex due to increasing amount of information needed to follow the guidelines. Usage of clinical decision support systems (CDSSs) can simplify and optimize decision-making. However, CDSS implementation is lagging behind. Therefore, we aim to compose a CDSS implementation model. By performing a scoping review of the currently reported CDSSs for MDT decision-making we determined 102 barriers and 86 facilitators for CDSS implementation out of 44 papers describing 20 different CDSSs. The most frequently reported barriers and facilitators for CDSS implementation supporting MDT decision-making concerned CDSS maintenance (e.g. incorporating guideline updates), validity of recommendations and interoperability with electronic health records. Based on the identified barriers and facilitators, we composed a CDSS implementation model describing clinical utility, analytic validity and clinical validity to guide CDSS integration more successfully in the clinical workflow to support MDTs in the future.

The Application of Computer Technology to Clinical Practice Guideline Implementation: A Scoping Review

Implementation of clinical practice guidelines (CPG) is a complex and challenging task. Computer technology, including artificial intelligence (AI), has been explored to promote the CPG implementation. This study has reviewed the main domains where computer technology and AI has been applied to CPG implementation. PubMed, Embase, Web of science, the Cochrane Library, China National Knowledge Infrastructure database, WanFang DATA, VIP database, and China Biology Medicine disc database were searched from inception to December 2021. Studies involving the utilization of computer technology and AI to promote the implementation of CPGs were eligible for review. A total of 10429 published articles were identified, 117 met the inclusion criteria. 21 (17.9%) focused on the utilization of AI techniques to classify or extract the relative content of CPGs, such as recommendation sentence, condition-action sentences. 47 (40.2%) focused on the utilization of computer technology to represent guideline knowledge to make it understandable by computer. 15 (12.8%) focused on the utilization of AI techniques to verify the relative content of CPGs, such as conciliation of multiple single-disease guidelines for comorbid patients. 34 (29.1%) focused on the utilization of AI techniques to integrate guideline knowledge into different resources, such as clinical decision support systems. We conclude that the application of computer technology and AI to CPG implementation mainly concentrated on the guideline content classification and extraction, guideline knowledge representation, guideline knowledge verification, and guideline knowledge integration. The AI methods used for guideline content classification and extraction were pattern-based algorithm and machine learning. In guideline knowledge representation, guideline knowledge verification, and guideline knowledge integration, computer techniques of knowledge representation were the most used.

Agreement and consistency in the triaging of musculoskeletal primary care referrals by vetting clinicians using a knowledge-based triage tool

BACKGROUND

Primary care referrals received by secondary care services are vetted or triaged to pathways best suited for patients' needs. If knowledge-based triaging is used by vetting clinicians, accuracy is required to avoid incorrect decisions being made. With limited evidence to support best practice, we aimed to evaluate consistency across vetting clinicians' decisions and their agreement with a criterion decision.

METHODS

Twenty-nine trained vetting clinicians (18 female) representative of pay grades independently triaged five musculoskeletal physiotherapy referral cases into one of 10 decisions using an internally developed triage tool. Agreement across clinicians' decisions between and within cases was assessed using Fleiss's kappa overall and within pay grade. Proportions of triage decisions consistent with criterion decisions were assessed using Cochran's Q test.

RESULTS

Clinician agreement was fair for all cases (κ = 0.385) irrespective of pay grade but varied within clinical cases (κ = -0.014-0.786). Proportions of correct triage decisions were significantly different across cases [Q(4) = 33.80, P < 0.001] ranging from 17% to 83%.

CONCLUSIONS

Agreement and consistency in decisions were variable using the tool. Ensuring referrer information is accurate is vital, as is developing, automating and auditing standards for certain referrals with clear pathways. But we argue that variable vetting outcomes might represent healthy pathway abundance and should not simply be automated in response to perceived inefficiencies.

Interpretability of Clinical Decision Support Systems Based on Artificial Intelligence from Technological and Medical Perspective: A Systematic Review

Background

Artificial intelligence (AI) has developed rapidly, and its application extends to clinical decision support system (CDSS) for improving healthcare quality. However, the interpretability of AI-driven CDSS poses significant challenges to widespread application.

Objective

This study is a review of the knowledge-based and data-based CDSS literature regarding interpretability in health care. It highlights the relevance of interpretability for CDSS and the area for improvement from technological and medical perspectives.

Methods

A systematic search was conducted on the interpretability-related literature published from 2011 to 2020 and indexed in the five databases: Web of Science, PubMed, ScienceDirect, Cochrane, and Scopus. Journal articles that focus on the interpretability of CDSS were included for analysis. Experienced researchers also participated in manually reviewing the selected articles for inclusion/exclusion and categorization.

Results

Based on the inclusion and exclusion criteria, 20 articles from 16 journals were finally selected for this review. Interpretability, which means a transparent structure of the model, a clear relationship between input and output, and explainability of artificial intelligence algorithms, is essential for CDSS application in the healthcare setting. Methods for improving the interpretability of CDSS include ante-hoc methods such as fuzzy logic, decision rules, logistic regression, decision trees for knowledge-based AI, and white box models, post hoc methods such as feature importance, sensitivity analysis, visualization, and activation maximization for black box models. A number of factors, such as data type, biomarkers, human-AI interaction, needs of clinicians, and patients, can affect the interpretability of CDSS.

Conclusions

The review explores the meaning of the interpretability of CDSS and summarizes the current methods for improving interpretability from technological and medical perspectives. The results contribute to the understanding of the interpretability of CDSS based on AI in health care. Future studies should focus on establishing formalism for defining interpretability, identifying the properties of interpretability, and developing an appropriate and objective metric for interpretability; in addition, the user's demand for interpretability and how to express and provide explanations are also the directions for future research.

Comparison of Classification Success Rates of Different Machine Learning Algorithms in the Diagnosis of Breast Cancer

OBJECTIVE

To identify which Machine Learning (ML) algorithms are the most successful in predicting and diagnosing breast cancer according to accuracy rates.

METHODS

The "College of Wisconsin Breast Cancer Dataset", which consists of 569 data and 30 features, was classified using Support Vector Machine (SVM), Naive Bayes (NB), Random Forest (RF), Decision Tree (DT), K-Nearest Neighbor (KNN), Logistic Regression (LR), Multilayer Perceptron (MLP), Linear Discriminant Analysis (LDA), XgBoost (XGB), Ada-Boost (ABC) and Gradient Boosting (GBC) ML algorithms. Before the classification process, the dataset was preprocessed. Sensitivity, accuracy, and definiteness metrics were used to measure the success of the methods.

RESULT

Compared to other ML algorithms used in the study, the GBC ML algorithm was found to be the most successful method in the classification of tumors with an accuracy of 99.12%. The XGB ML algorithm was found to be the lowest method with an accuracy rate of 88.10%. In addition, it was determined that the general accuracy rates of the 11 ML algorithms used in the study varied between 88-95%.

CONCLUSION

When the results obtained from the ML classifiers used in the study are evaluated, the efficiency of the GBC algorithm in the classification of tumors is obvious. It can be said that the success rates obtained from 11 different ML algorithms used in the study are valuable in terms of being used to predict different cancer types.

Using Guideline-Based Clinical Decision Support in Oncological Multidisciplinary Team Meetings: A Prospective, Multicenter Concordance Study

Background

Multidisciplinary team meetings formulate guideline-based individual treatment plans based on patient and disease characteristics and motivate reasons for deviation. Clinical decision trees could support multidisciplinary teams to adhere more accurately to guidelines. Every clinical decision tree is tailored to a specific decision moment in a care pathway and is composed of patient and disease characteristics leading to a guideline recommendation.

Objective

This study investigated (1) the concordance between multidisciplinary team and clinical decision tree recommendations and (2) the completeness of patient and disease characteristics available during multidisciplinary team meetings to apply clinical decision trees such that it results in a guideline recommendation.

Methods

This prospective, multicenter, observational concordance study evaluated 17 selected clinical decision trees, based on the prevailing Dutch guidelines for breast, colorectal and prostate cancers. In cases with sufficient data, concordance between multidisciplinary team and clinical decision tree recommendations was classified as concordant, conditional concordant (multidisciplinary team specified a prerequisite for the recommendation) and non-concordant.

Results

Fifty-nine multidisciplinary team meetings were attended in 8 different hospitals, and 355 cases were included. For 296 cases (83.4%), all patient data were available for providing an unconditional clinical decision tree recommendation. In 59 cases (16.6%), insufficient data were available resulting in provisional clinical decision tree recommendations. From the 296 successfully generated clinical decision tree recommendations, the multidisciplinary team recommendations were concordant in 249 (84.1%) cases, conditional concordant in 24 (8.1%) cases and non-concordant in 23 (7.8%) cases of which in 7 (2.4%) cases the reason for deviation from the clinical decision tree generated guideline recommendation was not motivated.

Conclusion

The observed concordance of recommendations between multidisciplinary teams and clinical decision trees and data completeness during multidisciplinary team meetings in this study indicate a potential role for implementation of clinical decision trees to support multidisciplinary team decision-making.

Impact on Quality of Documentation and Workload of the Introduction of a National Information Standard for Tumor Board Reporting

PURPOSE

Tumor boards, clinical practice guidelines, and cancer registries are intertwined cancer care quality instruments. Standardized structured reporting has been proposed as a solution to improve clinical documentation, while facilitating data reuse for secondary purposes. This study describes the implementation and evaluation of a national standard for tumor board reporting for breast cancer on the basis of the clinical practice guideline and the potential for reusing clinical data for the Netherlands Cancer Registry (NCR).

METHODS

Previously, a national information standard for breast cancer was derived from the corresponding Dutch clinical practice guideline. Using data items from the information standard, we developed three different tumor board forms: preoperative, postoperative, and postneoadjuvant-postoperative. The forms were implemented in Amphia Hospital’s electronic health record. Quality of clinical documentation and workload before and after implementation were compared.

RESULTS

Both draft and final tumor board reports were collected from 27 and 31 patients in baseline and effect measurements, respectively. Completeness of final reports increased from 39.5% to 45.4% (P = .04). The workload for tumor board preparation and discussion did not change significantly. Standardized tumor board reports included 50% (61/122) of the data items carried in the NCR. An automated process was developed to upload information captured in tumor board reports to the NCR database.

CONCLUSION

This study shows implementation of a national standard for tumor board reports improves quality of clinical documentation, without increasing clinical workload. Simultaneously, our work enables data reuse for secondary purposes like cancer registration.

Conversion of a colorectal cancer guideline into clinical decision trees with assessment of validity

Objective

The interpretation and clinical application of guidelines can be challenging and time-consuming, which may result in noncompliance to guidelines. The aim of this study was to convert the Dutch guideline for colorectal cancer (CRC) into decision trees and subsequently implement decision trees in an online decision support environment to facilitate guideline application.

Methods

The recommendations of the Dutch CRC guidelines (published in 2014) were translated into decision trees consisting of decision nodes, branches and leaves that represent data items, data item values and recommendations, respectively. Decision trees were discussed with experts in the field and published as interactive open access decision support software (available at www.oncoguide.nl). Decision tree validation and a concordance analysis were performed using consecutive reports (January 2016–January 2017) from CRC multidisciplinary tumour boards (MTBs) at Amsterdam University Medical Centers, location AMC.

Results

In total, we developed 34 decision trees driven by 101 decision nodes based on the guideline recommendations. Decision trees represented recommendations for diagnostics (n = 1), staging (n = 10), primary treatment (colon: n = 1, rectum: n = 5, colorectal: n = 9), pathology (n = 4) and follow-up (n = 3) and included one overview decision tree for optimal navigation. We identified several guideline information gaps and areas of inconclusive evidence. A total of 158 patients’ MTB reports were eligible for decision tree validation and resulted in treatment recommendations in 80% of cases. The concordance rate between decision tree treatment recommendations and MTB advices was 81%. Decision trees reported in 22 out of 24 non-concordant cases (92%) that no guideline recommendation was available.

Conclusions

We successfully converted the Dutch CRC guideline into decision trees and identified several information gaps and areas of inconclusive evidence, the latter being the main cause of the observed disagreement between decision tree recommendations and MTB advices. Decision trees may contribute to future strategies to optimize quality of care for CRC patients.

Leveraging Pathology Informatics Concepts to Achieve Discrete Lab Data for Clinical Use and Translational Research

Clinical practice is most efficient when physicians have the right information, including pathology and laboratory results, at the point of contact with the patient. In downstream workflows, subsequent groups using lab data want to have it available in a format that is easy to manipulate. With the complexity of electronic medical records, hospital information systems, and the need to accommodate data from outside systems, this is not easy to accomplish. By utilizing a group of concepts from clinical and pathology informatics, system implementations may be improved to achieve relevant laboratory data in a format that is usable by healthcare entities to improve patient care and forward endeavors in precision medicine.

Introduction to Artificial Intelligence and Machine Learning for Pathology

CONTEXT.—

Recent developments in machine learning have stimulated intense interest in software that may augment or replace human experts. Machine learning may impact pathology practice by offering new capabilities in analysis, interpretation, and outcomes prediction using images and other data. The principles of operation and management of machine learning systems are unfamiliar to pathologists, who anticipate a need for additional education to be effective as expert users and managers of the new tools.

OBJECTIVE.—

To provide a background on machine learning for practicing pathologists, including an overview of algorithms, model development, and performance evaluation; to examine the current status of machine learning in pathology and consider possible roles and requirements for pathologists in local deployment and management of machine learning systems; and to highlight existing challenges and gaps in deployment methodology and regulation.

DATA SOURCES.—

Sources include the biomedical and engineering literature, white papers from professional organizations, government reports, electronic resources, and authors' experience in machine learning. References were chosen when possible for accessibility to practicing pathologists without specialized training in mathematics, statistics, or software development.

CONCLUSIONS.—

Machine learning offers an array of techniques that in recent published results show substantial promise. Data suggest that human experts working with machine learning tools outperform humans or machines separately, but the optimal form for this combination in pathology has not been established. Significant questions related to the generalizability of machine learning systems, local site verification, and performance monitoring remain to be resolved before a consensus on best practices and a regulatory environment can be established.

Pragmatic Considerations on Clinical Decision Support from the 2019 Literature

Objectives : To summarize significant research contributions published in 2019 in the field of computerized clinical decision support and select the best papers for the Decision Support section of the International Medical Informatics Association (IMIA) Yearbook.

Methods : Two bibliographic databases were searched for papers referring to clinical decision support systems (CDSSs) and computerized provider order entry (CPOE) systems. From search results, section editors established a list of candidate best papers, which were then peer-reviewed by external reviewers. The IMIA Yearbook editorial committee finally selected the best papers on the basis of all reviews including the section editors’ evaluation.

Results : A total of 1,378 articles were retrieved. Fifteen best paper candidates were selected, the reviews of which resulted in the selection of three best papers. One paper reports on a guideline modeling approach based on clinical decision trees, both clinically interpretable and suitable for implementation in CDSSs. In another paper, authors promote the use of extended Timed Transition Diagrams in CDSSs to formalize consistently recurrent medical processes for chronic diseases management. The third paper proposes a conceptual framework and a grid for assessing the performance of predictive tools based on the critical appraisal of published evidence.

Conclusions : As showed by the number and the variety of works related to decision support, research in the field is very active. This year’s selection highlighted pragmatic works that promote transparency and trust required by decision support tools.

Clinical decision trees support systematic evaluation of multidisciplinary team recommendations

Purpose

EUSOMA’s recommendation that “each patient has to be fully informed about each step in the diagnostic and therapeutic pathway” could be supported by guideline-based clinical decision trees (CDTs). The Dutch breast cancer guideline has been modeled into CDTs (www.oncoguide.nl). Prerequisites for adequate CDT usage are availability of necessary patient data at the time of decision-making and to consider all possible treatment alternatives provided in the CDT.

Methods

This retrospective single-center study evaluated 394 randomly selected female patients with non-metastatic breast cancer between 2012 and 2015. Four pivotal CDTs were selected. Two researchers analyzed patient records to determine to which degree patient data required per CDT were available at the time of multidisciplinary team (MDT) meeting and how often multiple alternatives were actually reported.

Results

The four selected CDTs were indication for magnetic resonance imaging (MRI) scan, preoperative and adjuvant systemic treatment, and immediate breast reconstruction. For 70%, 13%, 97% and 13% of patients, respectively, all necessary data were available. The two most frequent underreported data-items were “clinical M-stage” (87%) and “assessable mammography” (28%). Treatment alternatives were reported by MDTs in 32% of patients regarding primary treatment and in 28% regarding breast reconstruction.

Conclusion

Both the availability of data in patient records essential for guideline-based recommendations and the reporting of possible treatment alternatives of the investigated CDTs were low. To meet EUSOMA’s requirements, information that is supposed to be implicitly known must be explicated by MDTs. Moreover, MDTs have to adhere to clear definitions of data-items in their reporting.

Computational approaches in cancer multidrug resistance research: Identification of potential biomarkers, drug targets and drug-target interactions

Like physics in the 19th century, biology and molecular biology in particular, has been fertilized and enhanced like few other scientific fields, by the incorporation of mathematical methods. In the last decades, a whole new scientific field, bioinformatics, has developed with an output of over 30,000 papers a year (Pubmed search using the keyword "bioinformatics"). Huge databases of mass throughput data have been established, with ArrayExpress alone containing more than 2.7 million assays (October 2019). Computational methods have become indispensable tools in molecular biology, particularly in one of the most challenging areas of cancer research, multidrug resistance (MDR). However, confronted with a plethora of different algorithms, approaches, and methods, the average researcher faces key questions: Which methods do exist? Which methods can be used to tackle the aims of a given study? Or, more generally, how do I use computational biology/bioinformatics to bolster my research? The current review is aimed at providing guidance to existing methods with relevance to MDR research. In particular, we provide an overview on: a) the identification of potential biomarkers using expression data; b) the prediction of treatment response by machine learning methods; c) the employment of network approaches to identify gene/protein regulatory networks and potential key players; d) the identification of drug-target interactions; e) the use of bipartite networks to identify multidrug targets; f) the identification of cellular subpopulations with the MDR phenotype; and, finally, g) the use of molecular modeling methods to guide and enhance drug discovery. This review shall serve as a guide through some of the basic concepts useful in MDR research. It shall give the reader some ideas about the possibilities in MDR research by using computational tools, and, finally, it shall provide a short overview of relevant literature.