P1.01-069 Clinical Experience with IBM Watson for Oncology (WFO) Cognitive System for Lung Cancer Treatment in China

Physician Perception and Satisfaction with Artificial Intelligence in Cancer Treatment: The Watson for Oncology Experience and Implications for Low-Middle Income Countries (Preprint)

As technology continues to improve, health care systems have the opportunity to use a variety of innovative tools for decision-making, including artificial intelligence (AI) applications. However, there has been little research on the feasibility and efficacy of integrating AI systems into real-world clinical practice, especially from the perspectives of clinicians who use such tools. In this paper, we review physicians’ perceptions of and satisfaction with an AI tool, Watson for Oncology, which is used for the treatment of cancer. Watson for Oncology has been implemented in several different settings, including Brazil, China, India, South Korea, and Mexico. By focusing on the implementation of an AI-based clinical decision support system for oncology, we aim to demonstrate how AI can be both beneficial and challenging for cancer management globally and particularly for low-middle–income countries. By doing so, we hope to highlight the need for additional research on user experience and the unique social, cultural, and political barriers to the successful implementation of AI in low-middle–income countries for cancer care.

Concordance between treatment recommendations provided by IBM Watson for Oncology and a multidisciplinary tumor board for breast cancer in China

OBJECTIVE

Watson for Oncology (WFO), an artificial intelligence from IBM Corporation, can provide a treatment plan by analyzing patient's disease characteristics. The present study was performed to examine the concordance between treatment recommendations proposed by WFO and the multidisciplinary tumor board at our center. The aim was to explore the feasibility of using WFO for breast cancer cases in China and to ascertain the ways to make WFO more suitable for Chinese patients with breast cancer.

METHODS

Data from 302 breast cancer patients treated at the Second Affiliated Hospital of Xi'an Jiaotong University between October 2016 and February 2018 was retrieved and retrospectively analyzed by WFO. The recommendations were divided into 'recommended', 'considered' and 'not recommended' groups. Results were considered concordant when oncologists' recommendations were categorized as 'recommended' or 'for consideration' by WFO.

RESULTS

The concordance rate of 200 subjects with postoperative adjuvant therapy was 77%. However, the rate was 27.5% in the remaining 102 cases with metastatic disease receiving either first-line or no treatment. Further analysis demonstrated that inconsistencies were mainly due to different choices of chemotherapy regimens. Subgroup study indicates that tumor stage, receptor status and age also had influences at the concordance rate.

CONCLUSION

The results of this study suggest that WFO is a promising artificial intelligence system for the treatment of breast cancer. These findings can also serve as a reference framework for the inclusion of artificial intelligence in the ongoing medical reform in China.

Concordance Between Watson for Oncology and a Multidisciplinary Clinical Decision-Making Team for Gastric Cancer and the Prognostic Implications: Retrospective Study

Background

With the increasing number of cancer treatments, the emergence of multidisciplinary teams (MDTs) provides patients with personalized treatment options. In recent years, artificial intelligence (AI) has developed rapidly in the medical field. There has been a gradual tendency to replace traditional diagnosis and treatment with AI. IBM Watson for Oncology (WFO) has been proven to be useful for decision-making in breast cancer and lung cancer, but to date, research on gastric cancer is limited.

Objective

This study compared the concordance of WFO with MDT and investigated the impact on patient prognosis.

Methods

This study retrospectively analyzed eligible patients (N=235) with gastric cancer who were evaluated by an MDT, received corresponding recommended treatment, and underwent follow-up. Thereafter, physicians inputted the information of all patients into WFO manually, and the results were compared with the treatment programs recommended by the MDT. If the MDT treatment program was classified as “recommended” or “considered” by WFO, we considered the results concordant. All patients were divided into a concordant group and a nonconcordant group according to whether the WFO and MDT treatment programs were concordant. The prognoses of the two groups were analyzed.

Results

The overall concordance of WFO and the MDT was 54.5% (128/235) in this study. The subgroup analysis found that concordance was less likely in patients with human epidermal growth factor receptor 2 (HER2)-positive tumors than in patients with HER2-negative tumors (P=.02). Age, Eastern Cooperative Oncology Group performance status, differentiation type, and clinical stage were not found to affect concordance. Among all patients, the survival time was significantly better in concordant patients than in nonconcordant patients (P<.001). Multivariate analysis revealed that concordance was an independent prognostic factor of overall survival in patients with gastric cancer (hazard ratio 0.312 [95% CI 0.187-0.521]).

Conclusions

The treatment recommendations made by WFO and the MDT were mostly concordant in gastric cancer patients. If the WFO options are updated to include local treatment programs, the concordance will greatly improve. The HER2 status of patients with gastric cancer had a strong effect on the likelihood of concordance. Generally, survival was better in concordant patients than in nonconcordant patients.

Using Artificial Intelligence (Watson for Oncology) for Treatment Recommendations Amongst Chinese Patients with Lung Cancer: Feasibility Study

BACKGROUND

Artificial intelligence (AI) is developing quickly in the medical field and can benefit both medical staff and patients. The clinical decision support system Watson for Oncology (WFO) is an outstanding representative AI in the medical field, and it can provide to cancer patients prompt treatment recommendations comparable with ones made by expert oncologists. WFO is increasingly being used in China, but limited reports on whether WFO is suitable for Chinese patients, especially patients with lung cancer, exist. Here, we report a retrospective study based on the consistency between the lung cancer treatment recommendations made for the same patient by WFO and by the multidisciplinary team at our center.

OBJECTIVE

The aim of this study was to explore the feasibility of using WFO for lung cancer cases in China and to ascertain ways to make WFO more suitable for Chinese patients with lung cancer.

METHODS

We selected all lung cancer patients who were hospitalized and received antitumor treatment for the first time at the Second Xiangya Hospital Cancer Center from September to December 2017 (N=182). WFO made treatment recommendations for all supported cases (n=149). If the actual therapeutic regimen (administered by our multidisciplinary team) was recommended or for consideration according to WFO, we defined the recommendations as consistent; if the actual therapeutic regimen was not recommended by WFO or if WFO did not provide the same treatment option, we defined the recommendations as inconsistent. Blinded second round reviews were performed by our multidisciplinary team to reassess the incongruent cases.

RESULTS

WFO did not support 18.1% (33/182) of recommendations among all cases. Of the 149 supported cases, 65.8% (98/149) received recommendations that were consistent with the recommendations of our team. Logistic regression analysis showed that pathological type and staging had significant effects on consistency (P=.004, odds ratio [OR] 0.09, 95% CI 0.02-0.45 and P<.001, OR 9.5, 95% CI 3.4-26.1, respectively). Age, gender, and presence of epidermal growth factor receptor gene mutations had no effect on consistency. In 82% (42/51) of the inconsistent cases, our team administered two China-specific treatments, which were different from the recommendations made by WFO but led to excellent outcomes.

CONCLUSIONS

In China, most of the treatment recommendations of WFO are consistent with the recommendations of the expert group, although a relatively high proportion of cases are still not supported by WFO. Therefore, WFO cannot currently replace oncologists. WFO can improve the efficiency of clinical work by providing assistance to doctors, but it needs to learn the regional characteristics of patients to improve its assistive ability.