Effect of an Artificial Intelligence Clinical Decision Support System on Treatment Decisions for Complex Breast Cancer

An Advanced Machine Learning Model for a Web-Based Artificial Intelligence-Based Clinical Decision Support System Application: Model Development and Validation Study

BACKGROUND

Breast cancer is a leading global health concern, necessitating advancements in recurrence prediction and management. The development of an artificial intelligence (AI)-based clinical decision support system (AI-CDSS) using ChatGPT addresses this need with the aim of enhancing both prediction accuracy and user accessibility.

OBJECTIVE

This study aims to develop and validate an advanced machine learning model for a web-based AI-CDSS application, leveraging the question-and-answer guidance capabilities of ChatGPT to enhance data preprocessing and model development, thereby improving the prediction of breast cancer recurrence.

METHODS

This study focused on developing an advanced machine learning model by leveraging data from the Tri-Service General Hospital breast cancer registry of 3577 patients (2004-2016). As a tertiary medical center, it accepts referrals from four branches-3 branches in the northern region and 1 branch on an offshore island in our country-that manage chronic diseases but refer complex surgical cases, including breast cancer, to the main center, enriching our study population's diversity. Model training used patient data from 2004 to 2012, with subsequent validation using data from 2013 to 2016, ensuring comprehensive assessment and robustness of our predictive models. ChatGPT is integral to preprocessing and model development, aiding in hormone receptor categorization, age binning, and one-hot encoding. Techniques such as the synthetic minority oversampling technique address the imbalance of data sets. Various algorithms, including light gradient-boosting machine, gradient boosting, and extreme gradient boosting, were used, and their performance was evaluated using metrics such as the area under the curve, accuracy, sensitivity, and F1-score.

RESULTS

The light gradient-boosting machine model demonstrated superior performance, with an area under the curve of 0.80, followed closely by the gradient boosting and extreme gradient boosting models. The web interface of the AI-CDSS tool was effectively tested in clinical decision-making scenarios, proving its use in personalized treatment planning and patient involvement.

CONCLUSIONS

The AI-CDSS tool, enhanced by ChatGPT, marks a significant advancement in breast cancer recurrence prediction, offering a more individualized and accessible approach for clinicians and patients. Although promising, further validation in diverse clinical settings is recommended to confirm its efficacy and expand its use.

Evolving and Improving the Sustainability of Molecular Tumor Boards: The Value and Challenges

The increasing volume of information for cancer care, and the evolution of molecularly guided therapies, have increased the need for molecular tumor boards (MTBs), which can integrate such data into personalized treatment plans to improve patient outcomes. However, recommendations for improving the sustainability of MTBs are lacking. A diverse committee of MTB experts was assembled (February-March 2023), with extensive experience in sustainability in healthcare ecosystems. The aim was to identify MTB-related hurdles throughout the patient journey and develop a general framework for MTBs to operate on larger scales locally, nationally, and internationally. The committee identified ten key pillars for sustainable and scalable MTBs, including technical solutions for data integration and visualization, interoperability, learning loops, clinical trial access, legal considerations, criteria for patient testing, decision standardization, making MTBs official bodies for treatment decisions, local leaders, and international networks. The need for scalable frameworks at academic and community levels was recognized, along with integrating MTBs into national health systems to enhance sustainability and ensure optimal treatment decisions. Irrespective of the health ecosystem, the sustainability and scalability of MTBs are essential. Our framework provides guidelines to address this and to help MTBs evolve towards integrated, essential components of the oncology healthcare system.

Effectiveness of Artificial Intelligence (AI) in Clinical Decision Support Systems and Care Delivery

This review aims to assess the effectiveness of AI-driven CDSSs on patient outcomes and clinical practices. A comprehensive search was conducted across PubMed, MEDLINE, and Scopus. Studies published from January 2018 to November 2023 were eligible for inclusion. Following title and abstract screening, full-text articles were assessed for methodological quality and adherence to inclusion criteria. Data extraction focused on study design, AI technologies employed, reported outcomes, and evidence of AI-CDSS impact on patient and clinical outcomes. Thematic analysis was conducted to synthesise findings and identify key themes regarding the effectiveness of AI-CDSS. The screening of the articles resulted in the selection of 26 articles that satisfied the inclusion criteria. The content analysis revealed four themes: early detection and disease diagnosis, enhanced decision-making, medication errors, and clinicians' perspectives. AI-based CDSSs were found to improve clinical decision-making by providing patient-specific information and evidence-based recommendations. Using AI in CDSSs can potentially improve patient outcomes by enhancing diagnostic accuracy, optimising treatment selection, and reducing medical errors.

Investigating the effects of artificial intelligence on the personalization of breast cancer management: a systematic study

BACKGROUND

Providing appropriate specialized treatment to the right patient at the right time is considered necessary in cancer management. Targeted therapy tailored to the genetic changes of each breast cancer patient is a desirable feature of precision oncology, which can not only reduce disease progression but also potentially increase patient survival. The use of artificial intelligence alongside precision oncology can help physicians by identifying and selecting more effective treatment factors for patients.

METHOD

A systematic review was conducted using the PubMed, Embase, Scopus, and Web of Science databases in September 2023. We performed the search strategy with keywords, namely: Breast Cancer, Artificial intelligence, and precision Oncology along with their synonyms in the article titles. Descriptive, qualitative, review, and non-English studies were excluded. The quality assessment of the articles and evaluation of bias were determined based on the SJR journal and JBI indices, as well as the PRISMA2020 guideline.

RESULTS

Forty-six studies were selected that focused on personalized breast cancer management using artificial intelligence models. Seventeen studies using various deep learning methods achieved a satisfactory outcome in predicting treatment response and prognosis, contributing to personalized breast cancer management. Two studies utilizing neural networks and clustering provided acceptable indicators for predicting patient survival and categorizing breast tumors. One study employed transfer learning to predict treatment response. Twenty-six studies utilizing machine-learning methods demonstrated that these techniques can improve breast cancer classification, screening, diagnosis, and prognosis. The most frequent modeling techniques used were NB, SVM, RF, XGBoost, and Reinforcement Learning. The average area under the curve (AUC) for the models was 0.91. Moreover, the average values for accuracy, sensitivity, specificity, and precision were reported to be in the range of 90-96% for the models.

CONCLUSION

Artificial intelligence has proven to be effective in assisting physicians and researchers in managing breast cancer treatment by uncovering hidden patterns in complex omics and genetic data. Intelligent processing of omics data through protein and gene pattern classification and the utilization of deep neural patterns has the potential to significantly transform the field of complex disease management.

Use of artificial intelligence in breast surgery: a narrative review

Background and Objective

We have witnessed tremendous advances in artificial intelligence (AI) technologies. Breast surgery, a subspecialty of general surgery, has notably benefited from AI technologies. This review aims to evaluate how AI has been integrated into breast surgery practices, to assess its effectiveness in improving surgical outcomes and operational efficiency, and to identify potential areas for future research and application.

Methods

Two authors independently conducted a comprehensive search of PubMed, Google Scholar, EMBASE, and Cochrane CENTRAL databases from January 1, 1950, to September 4, 2023, employing keywords pertinent to AI in conjunction with breast surgery or cancer. The search focused on English language publications, where relevance was determined through meticulous screening of titles, abstracts, and full-texts, followed by an additional review of references within these articles. The review covered a range of studies illustrating the applications of AI in breast surgery encompassing lesion diagnosis to postoperative follow-up. Publications focusing specifically on breast reconstruction were excluded.

Key Content and Findings

AI models have preoperative, intraoperative, and postoperative applications in the field of breast surgery. Using breast imaging scans and patient data, AI models have been designed to predict the risk of breast cancer and determine the need for breast cancer surgery. In addition, using breast imaging scans and histopathological slides, models were used for detecting, classifying, segmenting, grading, and staging breast tumors. Preoperative applications included patient education and the display of expected aesthetic outcomes. Models were also designed to provide intraoperative assistance for precise tumor resection and margin status assessment. As well, AI was used to predict postoperative complications, survival, and cancer recurrence.

Conclusions

Extra research is required to move AI models from the experimental stage to actual implementation in healthcare. With the rapid evolution of AI, further applications are expected in the coming years including direct performance of breast surgery. Breast surgeons should be updated with the advances in AI applications in breast surgery to provide the best care for their patients.

AI in medical diagnosis: AI prediction & human judgment

AI has long been regarded as a panacea for decision-making and many other aspects of knowledge work; as something that will help humans get rid of their shortcomings. We believe that AI can be a useful asset to support decision-makers, but not that it should replace decision-makers. Decision-making uses algorithmic analysis, but it is not solely algorithmic analysis; it also involves other factors, many of which are very human, such as creativity, intuition, emotions, feelings, and value judgments. We have conducted semi-structured open-ended research interviews with 17 dermatologists to understand what they expect from an AI application to deliver to medical diagnosis. We have found four aggregate dimensions along which the thinking of dermatologists can be described: the ways in which our participants chose to interact with AI, responsibility, 'explainability', and the new way of thinking (mindset) needed for working with AI. We believe that our findings will help physicians who might consider using AI in their diagnosis to understand how to use AI beneficially. It will also be useful for AI vendors in improving their understanding of how medics want to use AI in diagnosis. Further research will be needed to examine if our findings have relevance in the wider medical field and beyond.

AI triage or manual triage? Exploring medical staffs' preference for AI triage in China

OBJECTIVES

The introduction of AI technology in healthcare presents both opportunities and challenges. The aim of this study was to investigate medical staffs' preference for AI triage and the influencing factors.

METHODS

A survey was conducted online among medical staffs in China from March 4th to April 28th, 2021. Participants were recruited through multiple channels, including medical professional platforms and social media. A total of 677 valid responses were obtained from medical staff members located in 28 provinces across China.

RESULTS

The results showed that AI triage had an overall acceptance rate of 77.1%, and 45.2% of the medical staffs surveyed preferred "AI triage exclusively." Direct experience was positively associated with medical staffs' preference for AI triage (β = 0.223, p < .001). Additionally, greater exposure to a variety of media was positively associated with the perceived value of AI technology, which, in turn, increased preference for AI triage (β = 0.040, SE = 0.013, p < .001, 95% CI = [0.017, 0.067]).

CONCLUSION

Medical staffs generally hold a favorable attitude towards AI triage, particularly in areas with a high medical burden and during pandemics. In a multimedia environment, media exposure variety impacts medical staffs' preferences through their perceived value of AI technology. This study has implications for the implementation of AI triage on a larger scale.

The Facilitators and Barriers of the Implementation of a Clinical Decision Support System for Breast Cancer Multidisciplinary Team Meetings-An Interview Study

BACKGROUND

AI-driven clinical decision support systems (CDSSs) hold promise for multidisciplinary team meetings (MDTMs). This study aimed to uncover the hurdles and aids in implementing CDSSs during breast cancer MDTMs.

METHODS

Twenty-four core team members from three hospitals engaged in semi-structured interviews, revealing a collective interest in experiencing CDSS workflows in clinical practice. All interviews were audio recorded, transcribed verbatim and analyzed anonymously. A standardized approach, 'the framework method', was used to create an analytical framework for data analysis, which was performed by two independent researchers.

RESULTS

Positive aspects included improved data visualization, time-saving features, automated trial matching, and enhanced documentation transparency. However, challenges emerged, primarily concerning data connectivity, guideline updates, the accuracy of AI-driven suggestions, and the risk of losing human involvement in decision making. Despite the complexities involved in CDSS development and integration, clinicians demonstrated enthusiasm to explore its potential benefits.

CONCLUSIONS

Acknowledging the multifaceted nature of this challenge, insights into the barriers and facilitators identified in this study offer a potential roadmap for smoother future implementations. Understanding these factors could pave the way for more effective utilization of CDSSs in breast cancer MDTMs, enhancing patient care through informed decision making.

Use and accuracy of decision support systems using artificial intelligence for tumor diseases: a systematic review and meta-analysis

Background

For therapy planning in cancer patients multidisciplinary team meetings (MDM) are mandatory. Due to the high number of cases being discussed and significant workload of clinicians, Clinical Decision Support System (CDSS) may improve the clinical workflow.

Methods

This review and meta-analysis aims to provide an overview of the systems utilized and evaluate the correlation between a CDSS and MDM.

Results

A total of 31 studies were identified for final analysis. Analysis of different cancers shows a concordance rate (CR) of 72.7% for stage I-II and 73.4% for III-IV. For breast carcinoma, CR for stage I-II was 72.8% and for III-IV 84.1%, P≤ 0.00001. CR for colorectal carcinoma is 63% for stage I-II and 67% for III-IV, for gastric carcinoma 55% and 45%, and for lung carcinoma 85% and 83% respectively, all P>0.05. Analysis of SCLC and NSCLC yields a CR of 94,3% and 82,7%, P=0.004 and for adenocarcinoma and squamous cell carcinoma in lung cancer a CR of 90% and 86%, P=0.02.

Conclusion

CDSS has already been implemented in clinical practice, and while the findings suggest that its use is feasible for some cancers, further research is needed to fully evaluate its effectiveness.

Assessing the decision quality of artificial intelligence and oncologists of different experience in different regions in breast cancer treatment

Background

AI-based clinical decision support system (CDSS) has important prospects in overcoming the current informational challenges that cancer diseases faced, promoting the homogeneous development of standardized treatment among different geographical regions, and reforming the medical model. However, there are still a lack of relevant indicators to comprehensively assess its decision-making quality and clinical impact, which greatly limits the development of its clinical research and clinical application. This study aims to develop and application an assessment system that can comprehensively assess the decision-making quality and clinical impacts of physicians and CDSS.

Methods

Enrolled adjuvant treatment decision stage early breast cancer cases were randomly assigned to different decision-making physician panels (each panel consisted of three different seniority physicians in different grades hospitals), each physician made an independent "Initial Decision" and then reviewed the CDSS report online and made a "Final Decision". In addition, the CDSS and guideline expert groups independently review all cases and generate "CDSS Recommendations" and "Guideline Recommendations" respectively. Based on the design framework, a multi-level multi-indicator system including "Decision Concordance", "Calibrated Concordance", " Decision Concordance with High-level Physician", "Consensus Rate", "Decision Stability", "Guideline Conformity", and "Calibrated Conformity" were constructed.

Results

531 cases containing 2124 decision points were enrolled; 27 different seniority physicians from 10 different grades hospitals have generated 6372 decision opinions before and after referring to the "CDSS Recommendations" report respectively. Overall, the calibrated decision concordance was significantly higher for CDSS and provincial-senior physicians (80.9%) than other physicians. At the same time, CDSS has a higher " decision concordance with high-level physician" (76.3%-91.5%) than all physicians. The CDSS had significantly higher guideline conformity than all decision-making physicians and less internal variation, with an overall guideline conformity variance of 17.5% (97.5% vs. 80.0%), a standard deviation variance of 6.6% (1.3% vs. 7.9%), and a mean difference variance of 7.8% (1.5% vs. 9.3%). In addition, provincial-middle seniority physicians had the highest decision stability (54.5%). The overall consensus rate among physicians was 64.2%.

Conclusions

There are significant internal variation in the standardization treatment level of different seniority physicians in different geographical regions in the adjuvant treatment of early breast cancer. CDSS has a higher standardization treatment level than all physicians and has the potential to provide immediate decision support to physicians and have a positive impact on standardizing physicians' treatment behaviors.

A comparison between clinical decision support system and clinicians in breast cancer

Objective

We are building a clinical decision support system (CSCO AI) for breast cancer patients to improve the efficiency of clinical decision-making. We aimed to assess cancer treatment regimens given by CSCO AI and different levels of clinicians.

Methods

400 breast cancer patients were screened from the CSCO database. Clinicians with similar levels were randomly assigned one of the volumes (200 cases). CSCO AI was asked to assess all cases. Three reviewers were independently asked to evaluate the regimens from clinicians and CSCO AI. Regimens were masked before evaluation. The primary outcome was the proportion of high-level conformity (HLC).

Results

The overall concordance between clinicians and CSCO AI was 73.9% (3621/4900). It was 78.8% (2757/3500) in the early-stage, higher than that in the metastatic stage (61.7% [864/1400], p < 0.001). The concordance was 90.7% (635/700) and 56.4% (395/700) in adjuvant radiotherapy and second-line therapy respectively. HLC in CSCO AI was 95.8% (95%CI:94.0%-97.6%), significantly higher than that in clinicians (90.8%, 95%CI:89.8%-91.8%). Considering professions, the HLC of surgeons was 85.9%, lower than that of CSCO AI (OR = 0.25,95%CI: 0.16-0.41). The most significant difference in HLC was in first-line therapy (OR = 0.06, 95%CI:0.01-0.41). When clinicians were divided according to their levels, there was no statistical significance between CSCO AI and higher level clinicians.

Conclusions

Decision from CSCO AI for breast cancer was superior than most clinicians did except in second-line therapy. The improvements in process outcomes suggest that CSCO AI can be widely used in clinical practice.

Artificial intelligence in clinical decision support systems for oncology

Artificial intelligence (AI) has been widely used in various medical fields, such as image diagnosis, pathological classification, selection of treatment schemes, and prognosis analysis. Especially in the image-aided diagnosis of tumors, the cooperation of human-computer interactions has become mature. However, the ethics of the application of AI as an emerging technology in clinical decision-making have not been fully supported, so the clinical decision support system (CDSS) based on AI technology has not fully realized human-computer interactions in clinical practice as the image-aided diagnosis system. The CDSS was currently used and promoted worldwide including Watson for Oncology, Chinese society of clinical oncology-artificial intelligence (CSCO AI) and so on. This paper summarized the applications and clarified the principle of AI in CDSS, analyzed the difficulties of AI in oncology decisions, and provided a reference scheme for the application of AI in oncology decisions in the future.

Towards effective clinical decision support systems: A systematic review

Background

Clinical Decision Support Systems (CDSS) are used to assist the decision-making process in the healthcare field. Developing an effective CDSS is an arduous task that can take advantage from prior assessment of the most promising theories, techniques and methods used at the present time.

Objective

To identify the features of Clinical Decision Support Systems and provide an analysis of their effectiveness. Thus, two research questions were formulated: RQ1—What are the most common trend characteristics in a CDSS? RQ2—What is the maturity level of the CDSS based on the decision-making theory proposed by Simon?

Methods

AIS e-library, Decision Support Systems journal, Nature, PlosOne and PubMed were selected as information sources to conduct this systematic literature review. Studies from 2000 to 2020 were chosen covering search terms in CDSS, selected according to defined eligibility criteria. The data were extracted and managed in a worksheet, based on the defined criteria. PRISMA statements were used to report the systematic review.

Results

The outcomes showed that rule-based module was the most used approach regarding knowledge management and representation. The most common technological feature adopted by the CDSS were the recommendations and suggestions. 19,23% of studies adopt the type of system as a web-based application, and 51,92% are standalone CDSS. Temporal evolution was also possible to visualize. This study contributed to the development of a Maturity Staging Model, where it was possible to verify that most CDSS do not exceed level 2 of maturity.

Conclusion

The trend characteristics addressed in the revised CDSS were identified, compared to the four predefined groups. A maturity stage model was developed based on Simon’s decision-making theory, allowing to assess the level of maturity of the most common features of the CDSS. With the application of the model, it was noticed that the phases of choice and implementation are underrepresented. This constitutes the main gap in the development of an effective CDSS.

Chinese Society of Clinical Oncology (CSCO) Breast Cancer Guidelines 2022

Developing guidelines for the diagnosis and treatment of common cancers in China based on the evidence-based practice, the availability of diagnosis and treatment products, and the up-to-date advances in precision medicine is one of the basic tasks of the Chinese Society of Clinical Oncology (CSCO). In recent years, the availability of medical resources has become a major concern in clinical guidelines, which is particularly important for developing countries or socioeconomically diverse countries and territories. China is the world's largest developing country, with a large territory and uneven economic and academic developments. The CSCO guidelines must take into account the differences in regional development, the availability of medicines and diagnostic methods, and the social value of cancer treatment. Therefore, for each clinical problem and intervention in the CSCO guidelines, the levels of evidence should be graded according to the currently available evidences and expert consensuses, and the grades of recommendations should be based on the availability and cost-effectiveness of the products. Protocols with high evidence level and good availability are used as the Level I recommendations; protocols with relatively high evidence level but slightly lower expert consensus or with poor availability are used as the Level II recommendations; and protocols that are clinically applicable but with low evidence level are regarded as the Level III recommendations. Based on the findings of clinical research at home and abroad and the opinions of CSCO experts, the CSCO guidelines determine the levels of recommendations for clinical application. The CSCO Guidance Working Group firmly believes that evidence-based, availability-concerned, and consensus-based guidelines will be more feasible for clinical practice. Again, any comments from our readers are greatly appreciated and will be considered in updates of these guidelines, so as to maintain the accuracy, fairness, and timeliness of the CSCO guidelines.

Clinical Decision Support Systems

BACKGROUND

By combining up-to-date medical knowledge and steadily increasing patient data, a new level of medical care can emerge.

SUMMARY AND KEY MESSAGES

Clinical decision support systems (CDSSs) are an arising solution to handling rich data and providing them to health care providers in order to improve diagnosis and treatment. However, despite promising examples in many areas, substantial evidence for a thorough benefit of these support solutions is lacking. This may be due to a lack of general frameworks and diverse health systems around the globe. We therefore summarize the current status of CDSSs in medicine but also discuss potential limitations that need to be overcome in order to further foster future development and acceptance.

Why a Clinical Decision Support System is needed for Tinnitus?

Tinnitus is the perception of a phantom sound and the individual's reaction to it. Although much progress has been made, tinnitus remains an unresolved scientific and clinical issue, affecting more than 10% of the general population and having a high prevalence and socioeconomic burden. Clinical decision support systems (CDSS) are used to assist clinicians in their complex decision-making processes, having been proved that they improve healthcare delivery. In this paper, we present a CDSS for tinnitus, attempting to address the question which treatment approach is optimal for a particular patient based on specific parameters. The CDSS will be developed in the context of the EU-funded "UNITI" project and, after the project completion, it will be able to determine the suitability and expected attachment of a particular patient to a list of available clinical interventions, utilizing predictive and classification machine learning models.Clinical Relevance - The proposed clinically utilizable CDSS will be able to suggest the optimal treatment strategy for the tinnitus patient based on a set of heterogeneous data.

Computer-Based Decision Tools for Shared Therapeutic Decision-making in Oncology: Systematic Review

BACKGROUND

Therapeutic decision-making in oncology is a complex process because physicians must consider many forms of medical data and protocols. Another challenge for physicians is to clearly communicate their decision-making process to patients to ensure informed consent. Computer-based decision tools have the potential to play a valuable role in supporting this process.

OBJECTIVE

This systematic review aims to investigate the extent to which computer-based decision tools have been successfully adopted in oncology consultations to improve patient-physician joint therapeutic decision-making.

METHODS

This review was conducted in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 checklist and guidelines. A literature search was conducted on February 4, 2021, across the Cochrane Database of Systematic Reviews (from 2005 to January 28, 2021), the Cochrane Central Register of Controlled Trials (December 2020), MEDLINE (from 1946 to February 4, 2021), Embase (from 1947 to February 4, 2021), Web of Science (from 1900 to 2021), Scopus (from 1969 to 2021), and PubMed (from 1991 to 2021). We used a snowball approach to identify additional studies by searching the reference lists of the studies included for full-text review. Additional supplementary searches of relevant journals and gray literature websites were conducted. The reviewers screened the articles eligible for review for quality and inclusion before data extraction.

RESULTS

There are relatively few studies looking at the use of computer-based decision tools in oncology consultations. Of the 4431 unique articles obtained from the searches, only 10 (0.22%) satisfied the selection criteria. From the 10 selected studies, 8 computer-based decision tools were identified. Of the 10 studies, 6 (60%) were conducted in the United States. Communication and information-sharing were improved between physicians and patients. However, physicians did not change their habits to take advantage of computer-assisted decision-making tools or the information they provide. On average, the use of these computer-based decision tools added approximately 5 minutes to the total length of consultations. In addition, some physicians felt that the technology increased patients' anxiety.

CONCLUSIONS

Of the 10 selected studies, 6 (60%) demonstrated positive outcomes, 1 (10%) showed negative results, and 3 (30%) were neutral. Adoption of computer-based decision tools during oncology consultations continues to be low. This review shows that information-sharing and communication between physicians and patients can be improved with the assistance of technology. However, the lack of integration with electronic health records is a barrier. This review provides key requirements for enhancing the chance of success of future computer-based decision tools. However, it does not show the effects of health care policies, regulations, or business administration on physicians' propensity to adopt the technology. Nevertheless, it is important that future research address the influence of these higher-level factors as well.

TRIAL REGISTRATION

PROSPERO International Prospective Register of Systematic Reviews CRD42021226087; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021226087.

Comparison of an oncology clinical decision-support system's recommendations with actual treatment decisions

OBJECTIVE

IBM(R) Watson for Oncology (WfO) is a clinical decision-support system (CDSS) that provides evidence-informed therapeutic options to cancer-treating clinicians. A panel of experienced oncologists compared CDSS treatment options to treatment decisions made by clinicians to characterize the quality of CDSS therapeutic options and decisions made in practice.

METHODS

This study included patients treated between 1/2017 and 7/2018 for breast, colon, lung, and rectal cancers at Bumrungrad International Hospital (BIH), Thailand. Treatments selected by clinicians were paired with therapeutic options presented by the CDSS and coded to mask the origin of options presented. The panel rated the acceptability of each treatment in the pair by consensus, with acceptability defined as compliant with BIH's institutional practices. Descriptive statistics characterized the study population and treatment-decision evaluations by cancer type and stage.

RESULTS

Nearly 60% (187) of 313 treatment pairs for breast, lung, colon, and rectal cancers were identical or equally acceptable, with 70% (219) of WfO therapeutic options identical to, or acceptable alternatives to, BIH therapy. In 30% of cases (94), 1 or both treatment options were rated as unacceptable. Of 32 cases where both WfO and BIH options were acceptable, WfO was preferred in 18 cases and BIH in 14 cases. Colorectal cancers exhibited the highest proportion of identical or equally acceptable treatments; stage IV cancers demonstrated the lowest.

CONCLUSION

This study demonstrates that a system designed in the US to support, rather than replace, cancer-treating clinicians provides therapeutic options which are generally consistent with recommendations from oncologists outside the US.

Future Medical Artificial Intelligence Application Requirements and Expectations of Physicians in German University Hospitals: Web-Based Survey

BACKGROUND

The increasing development of artificial intelligence (AI) systems in medicine driven by researchers and entrepreneurs goes along with enormous expectations for medical care advancement. AI might change the clinical practice of physicians from almost all medical disciplines and in most areas of health care. While expectations for AI in medicine are high, practical implementations of AI for clinical practice are still scarce in Germany. Moreover, physicians' requirements and expectations of AI in medicine and their opinion on the usage of anonymized patient data for clinical and biomedical research have not been investigated widely in German university hospitals.

OBJECTIVE

This study aimed to evaluate physicians' requirements and expectations of AI in medicine and their opinion on the secondary usage of patient data for (bio)medical research (eg, for the development of machine learning algorithms) in university hospitals in Germany.

METHODS

A web-based survey was conducted addressing physicians of all medical disciplines in 8 German university hospitals. Answers were given using Likert scales and general demographic responses. Physicians were asked to participate locally via email in the respective hospitals.

RESULTS

The online survey was completed by 303 physicians (female: 121/303, 39.9%; male: 173/303, 57.1%; no response: 9/303, 3.0%) from a wide range of medical disciplines and work experience levels. Most respondents either had a positive (130/303, 42.9%) or a very positive attitude (82/303, 27.1%) towards AI in medicine. There was a significant association between the personal rating of AI in medicine and the self-reported technical affinity level (H4=48.3, P<.001). A vast majority of physicians expected the future of medicine to be a mix of human and artificial intelligence (273/303, 90.1%) but also requested a scientific evaluation before the routine implementation of AI-based systems (276/303, 91.1%). Physicians were most optimistic that AI applications would identify drug interactions (280/303, 92.4%) to improve patient care substantially but were quite reserved regarding AI-supported diagnosis of psychiatric diseases (62/303, 20.5%). Of the respondents, 82.5% (250/303) agreed that there should be open access to anonymized patient databases for medical and biomedical research.

CONCLUSIONS

Physicians in stationary patient care in German university hospitals show a generally positive attitude towards using most AI applications in medicine. Along with this optimism comes several expectations and hopes that AI will assist physicians in clinical decision making. Especially in fields of medicine where huge amounts of data are processed (eg, imaging procedures in radiology and pathology) or data are collected continuously (eg, cardiology and intensive care medicine), physicians' expectations of AI to substantially improve future patient care are high. In the study, the greatest potential was seen in the application of AI for the identification of drug interactions, assumedly due to the rising complexity of drug administration to polymorbid, polypharmacy patients. However, for the practical usage of AI in health care, regulatory and organizational challenges still have to be mastered.