1
|
Fu YV, Ramachandran GK, Halwani A, McInnes BT, Xia F, Lybarger K, Yetisgen M, Uzuner Ö. CACER: Clinical concept Annotations for Cancer Events and Relations. J Am Med Inform Assoc 2024; 31:2583-2594. [PMID: 39225779 PMCID: PMC11491616 DOI: 10.1093/jamia/ocae231] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2024] [Revised: 08/08/2024] [Accepted: 08/12/2024] [Indexed: 09/04/2024] Open
Abstract
OBJECTIVE Clinical notes contain unstructured representations of patient histories, including the relationships between medical problems and prescription drugs. To investigate the relationship between cancer drugs and their associated symptom burden, we extract structured, semantic representations of medical problem and drug information from the clinical narratives of oncology notes. MATERIALS AND METHODS We present Clinical concept Annotations for Cancer Events and Relations (CACER), a novel corpus with fine-grained annotations for over 48 000 medical problems and drug events and 10 000 drug-problem and problem-problem relations. Leveraging CACER, we develop and evaluate transformer-based information extraction models such as Bidirectional Encoder Representations from Transformers (BERT), Fine-tuned Language Net Text-To-Text Transfer Transformer (Flan-T5), Large Language Model Meta AI (Llama3), and Generative Pre-trained Transformers-4 (GPT-4) using fine-tuning and in-context learning (ICL). RESULTS In event extraction, the fine-tuned BERT and Llama3 models achieved the highest performance at 88.2-88.0 F1, which is comparable to the inter-annotator agreement (IAA) of 88.4 F1. In relation extraction, the fine-tuned BERT, Flan-T5, and Llama3 achieved the highest performance at 61.8-65.3 F1. GPT-4 with ICL achieved the worst performance across both tasks. DISCUSSION The fine-tuned models significantly outperformed GPT-4 in ICL, highlighting the importance of annotated training data and model optimization. Furthermore, the BERT models performed similarly to Llama3. For our task, large language models offer no performance advantage over the smaller BERT models. CONCLUSIONS We introduce CACER, a novel corpus with fine-grained annotations for medical problems, drugs, and their relationships in clinical narratives of oncology notes. State-of-the-art transformer models achieved performance comparable to IAA for several extraction tasks.
Collapse
Affiliation(s)
- Yujuan Velvin Fu
- Department of Biomedical Informatics & Medical Education, University of Washington, Seattle, WA 98195, United States
| | | | - Ahmad Halwani
- Huntsman Cancer Institute, University of Utah, Salt Lake City, UT 84112, United States
| | - Bridget T McInnes
- Department of Computer Science, Virginia Commonwealth University, Richmond, VA 23284, United States
| | - Fei Xia
- Department of Linguistics, University of Washington, Seattle, WA 98195, United States
| | - Kevin Lybarger
- Department of Information Sciences and Technology, George Mason University, Fairfax, VA 22030, United States
| | - Meliha Yetisgen
- Department of Biomedical Informatics & Medical Education, University of Washington, Seattle, WA 98195, United States
| | - Özlem Uzuner
- Department of Information Sciences and Technology, George Mason University, Fairfax, VA 22030, United States
| |
Collapse
|
3
|
Bryant AK, Zamora‐Resendiz R, Dai X, Morrow D, Lin Y, Jungles KM, Rae JM, Tate A, Pearson AN, Jiang R, Fritsche L, Lawrence TS, Zou W, Schipper M, Ramnath N, Yoo S, Crivelli S, Green MD. Artificial intelligence to unlock real-world evidence in clinical oncology: A primer on recent advances. Cancer Med 2024; 13:e7253. [PMID: 38899720 PMCID: PMC11187737 DOI: 10.1002/cam4.7253] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2023] [Revised: 02/05/2024] [Accepted: 04/28/2024] [Indexed: 06/21/2024] Open
Abstract
PURPOSE Real world evidence is crucial to understanding the diffusion of new oncologic therapies, monitoring cancer outcomes, and detecting unexpected toxicities. In practice, real world evidence is challenging to collect rapidly and comprehensively, often requiring expensive and time-consuming manual case-finding and annotation of clinical text. In this Review, we summarise recent developments in the use of artificial intelligence to collect and analyze real world evidence in oncology. METHODS We performed a narrative review of the major current trends and recent literature in artificial intelligence applications in oncology. RESULTS Artificial intelligence (AI) approaches are increasingly used to efficiently phenotype patients and tumors at large scale. These tools also may provide novel biological insights and improve risk prediction through multimodal integration of radiographic, pathological, and genomic datasets. Custom language processing pipelines and large language models hold great promise for clinical prediction and phenotyping. CONCLUSIONS Despite rapid advances, continued progress in computation, generalizability, interpretability, and reliability as well as prospective validation are needed to integrate AI approaches into routine clinical care and real-time monitoring of novel therapies.
Collapse
Affiliation(s)
- Alex K. Bryant
- Department of Radiation OncologyUniversity of Michigan School of MedicineAnn ArborMichiganUSA
- Department of Radiation Oncology, Veterans Affairs Ann Arbor Healthcare SystemAnn ArborMichiganUSA
| | - Rafael Zamora‐Resendiz
- Applied Mathematics and Computational Research Division, Lawrence Berkeley National LaboratoryBerkeleyCaliforniaUSA
| | - Xin Dai
- Computational Science Initiative, Brookhaven National LaboratoryUptonNew YorkUSA
| | - Destinee Morrow
- Applied Mathematics and Computational Research Division, Lawrence Berkeley National LaboratoryBerkeleyCaliforniaUSA
| | - Yuewei Lin
- Computational Science Initiative, Brookhaven National LaboratoryUptonNew YorkUSA
| | - Kassidy M. Jungles
- Department of PharmacologyUniversity of Michigan School of MedicineAnn ArborMichiganUSA
| | - James M. Rae
- Department of PharmacologyUniversity of Michigan School of MedicineAnn ArborMichiganUSA
- Department of Internal MedicineUniversity of Michigan School of MedicineAnn ArborMichiganUSA
| | - Akshay Tate
- Department of Radiation OncologyUniversity of Michigan School of MedicineAnn ArborMichiganUSA
| | - Ashley N. Pearson
- Department of Radiation OncologyUniversity of Michigan School of MedicineAnn ArborMichiganUSA
| | - Ralph Jiang
- Department of Radiation OncologyUniversity of Michigan School of MedicineAnn ArborMichiganUSA
- Department of StatisticsUniversity of MichiganAnn ArborMichiganUSA
| | - Lars Fritsche
- Department of StatisticsUniversity of MichiganAnn ArborMichiganUSA
| | - Theodore S. Lawrence
- Department of Radiation OncologyUniversity of Michigan School of MedicineAnn ArborMichiganUSA
| | - Weiping Zou
- Department of StatisticsUniversity of MichiganAnn ArborMichiganUSA
- Center of Excellence for Cancer Immunology and ImmunotherapyUniversity of Michigan Rogel Cancer CenterAnn ArborMichiganUSA
- Department of PathologyUniversity of MichiganAnn ArborMichiganUSA
- Graduate Program in ImmunologyUniversity of MichiganAnn ArborMichiganUSA
| | - Matthew Schipper
- Department of Radiation OncologyUniversity of Michigan School of MedicineAnn ArborMichiganUSA
- Department of PharmacologyUniversity of Michigan School of MedicineAnn ArborMichiganUSA
| | - Nithya Ramnath
- Division of Hematology Oncology, Department of MedicineUniversity of Michigan School of MedicineAnn ArborMichiganUSA
- Division of Hematology Oncology, Department of MedicineVeterans Affairs Ann Arbor Healthcare SystemAnn ArborMichiganUSA
| | - Shinjae Yoo
- Computational Science Initiative, Brookhaven National LaboratoryUptonNew YorkUSA
| | - Silvia Crivelli
- Applied Mathematics and Computational Research Division, Lawrence Berkeley National LaboratoryBerkeleyCaliforniaUSA
| | - Michael D. Green
- Department of Radiation OncologyUniversity of Michigan School of MedicineAnn ArborMichiganUSA
- Department of Radiation Oncology, Veterans Affairs Ann Arbor Healthcare SystemAnn ArborMichiganUSA
- Graduate Program in ImmunologyUniversity of MichiganAnn ArborMichiganUSA
- Graduate Program in Cancer BiologyUniversity of MichiganAnn ArborMichiganUSA
- Department of Microbiology and ImmunologyUniversity of Michigan School of MedicineAnn ArborMichiganUSA
| |
Collapse
|
4
|
Li W, Gou F, Wu J. Artificial intelligence auxiliary diagnosis and treatment system for breast cancer in developing countries. JOURNAL OF X-RAY SCIENCE AND TECHNOLOGY 2024; 32:395-413. [PMID: 38189731 DOI: 10.3233/xst-230194] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/09/2024]
Abstract
BACKGROUND In many developing countries, a significant number of breast cancer patients are unable to receive timely treatment due to a large population base, high patient numbers, and limited medical resources. OBJECTIVE This paper proposes a breast cancer assisted diagnosis system based on electronic medical records. The goal of this system is to address the limitations of existing systems, which primarily rely on structured electronic records and may miss crucial information stored in unstructured records. METHODS The proposed approach is a breast cancer assisted diagnosis system based on electronic medical records. The system utilizes breast cancer enhanced convolutional neural networks with semantic initialization filters (BC-INIT-CNN). It extracts highly relevant tumor markers from unstructured medical records to aid in breast cancer staging diagnosis and effectively utilizes the important information present in unstructured records. RESULTS The model's performance is assessed using various evaluation metrics. Such as accuracy, ROC curves, and Precision-Recall curves. Comparative analysis demonstrates that the BC-INIT-CNN model outperforms several existing methods in terms of accuracy and computational efficiency. CONCLUSIONS The proposed breast cancer assisted diagnosis system based on BC-INIT-CNN showcases the potential to address the challenges faced by developing countries in providing timely treatment to breast cancer patients. By leveraging unstructured medical records and extracting relevant tumor markers, the system enables accurate staging diagnosis and enhances the utilization of valuable information.
Collapse
Affiliation(s)
- Wenxiu Li
- State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang, China
| | - Fangfang Gou
- State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang, China
| | - Jia Wu
- State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang, China
- Research Center for Artificial Intelligence, Monash University, Melbourne, Clayton VIC, Australia
| |
Collapse
|
5
|
Alqahtani T, Badreldin HA, Alrashed M, Alshaya AI, Alghamdi SS, Bin Saleh K, Alowais SA, Alshaya OA, Rahman I, Al Yami MS, Albekairy AM. The emergent role of artificial intelligence, natural learning processing, and large language models in higher education and research. Res Social Adm Pharm 2023:S1551-7411(23)00280-2. [PMID: 37321925 DOI: 10.1016/j.sapharm.2023.05.016] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2023] [Revised: 05/29/2023] [Accepted: 05/30/2023] [Indexed: 06/17/2023]
Abstract
Artificial Intelligence (AI) has revolutionized various domains, including education and research. Natural language processing (NLP) techniques and large language models (LLMs) such as GPT-4 and BARD have significantly advanced our comprehension and application of AI in these fields. This paper provides an in-depth introduction to AI, NLP, and LLMs, discussing their potential impact on education and research. By exploring the advantages, challenges, and innovative applications of these technologies, this review gives educators, researchers, students, and readers a comprehensive view of how AI could shape educational and research practices in the future, ultimately leading to improved outcomes. Key applications discussed in the field of research include text generation, data analysis and interpretation, literature review, formatting and editing, and peer review. AI applications in academics and education include educational support and constructive feedback, assessment, grading, tailored curricula, personalized career guidance, and mental health support. Addressing the challenges associated with these technologies, such as ethical concerns and algorithmic biases, is essential for maximizing their potential to improve education and research outcomes. Ultimately, the paper aims to contribute to the ongoing discussion about the role of AI in education and research and highlight its potential to lead to better outcomes for students, educators, and researchers.
Collapse
Affiliation(s)
- Tariq Alqahtani
- Department of Pharmaceutical Sciences, College of Pharmacy, King Saud bin Abdulaziz University for Health Sciences, Saudi Arabia; King Abdullah International Medical Research Center, Riyadh, Saudi Arabia.
| | - Hisham A Badreldin
- King Abdullah International Medical Research Center, Riyadh, Saudi Arabia; Department of Pharmacy Practice, College of Pharmacy, King Saud bin Abdulaziz University for Health Sciences, King Abdullah International Medical Research Center, Riyadh, Saudi Arabia; Pharmaceutical Care Department, King Abdulaziz Medical City, National Guard Health Affairs, Riyadh, Saudi Arabia
| | - Mohammed Alrashed
- King Abdullah International Medical Research Center, Riyadh, Saudi Arabia; Department of Pharmacy Practice, College of Pharmacy, King Saud bin Abdulaziz University for Health Sciences, King Abdullah International Medical Research Center, Riyadh, Saudi Arabia; Pharmaceutical Care Department, King Abdulaziz Medical City, National Guard Health Affairs, Riyadh, Saudi Arabia
| | - Abdulrahman I Alshaya
- King Abdullah International Medical Research Center, Riyadh, Saudi Arabia; Department of Pharmacy Practice, College of Pharmacy, King Saud bin Abdulaziz University for Health Sciences, King Abdullah International Medical Research Center, Riyadh, Saudi Arabia; Pharmaceutical Care Department, King Abdulaziz Medical City, National Guard Health Affairs, Riyadh, Saudi Arabia
| | - Sahar S Alghamdi
- Department of Pharmaceutical Sciences, College of Pharmacy, King Saud bin Abdulaziz University for Health Sciences, Saudi Arabia; King Abdullah International Medical Research Center, Riyadh, Saudi Arabia
| | - Khalid Bin Saleh
- King Abdullah International Medical Research Center, Riyadh, Saudi Arabia; Department of Pharmacy Practice, College of Pharmacy, King Saud bin Abdulaziz University for Health Sciences, King Abdullah International Medical Research Center, Riyadh, Saudi Arabia; Pharmaceutical Care Department, King Abdulaziz Medical City, National Guard Health Affairs, Riyadh, Saudi Arabia
| | - Shuroug A Alowais
- King Abdullah International Medical Research Center, Riyadh, Saudi Arabia; Department of Pharmacy Practice, College of Pharmacy, King Saud bin Abdulaziz University for Health Sciences, King Abdullah International Medical Research Center, Riyadh, Saudi Arabia; Pharmaceutical Care Department, King Abdulaziz Medical City, National Guard Health Affairs, Riyadh, Saudi Arabia
| | - Omar A Alshaya
- King Abdullah International Medical Research Center, Riyadh, Saudi Arabia; Department of Pharmacy Practice, College of Pharmacy, King Saud bin Abdulaziz University for Health Sciences, King Abdullah International Medical Research Center, Riyadh, Saudi Arabia; Pharmaceutical Care Department, King Abdulaziz Medical City, National Guard Health Affairs, Riyadh, Saudi Arabia
| | - Ishrat Rahman
- Department of Basic Dental Sciences, College of Dentistry, Princess Nourah bint Abdulrahman University, P.O. Box 84428, Riyadh, 11671, Saudi Arabia
| | - Majed S Al Yami
- King Abdullah International Medical Research Center, Riyadh, Saudi Arabia; Department of Pharmacy Practice, College of Pharmacy, King Saud bin Abdulaziz University for Health Sciences, King Abdullah International Medical Research Center, Riyadh, Saudi Arabia; Pharmaceutical Care Department, King Abdulaziz Medical City, National Guard Health Affairs, Riyadh, Saudi Arabia
| | - Abdulkareem M Albekairy
- King Abdullah International Medical Research Center, Riyadh, Saudi Arabia; Department of Pharmacy Practice, College of Pharmacy, King Saud bin Abdulaziz University for Health Sciences, King Abdullah International Medical Research Center, Riyadh, Saudi Arabia; Pharmaceutical Care Department, King Abdulaziz Medical City, National Guard Health Affairs, Riyadh, Saudi Arabia
| |
Collapse
|
6
|
Diab KM, Deng J, Wu Y, Yesha Y, Collado-Mesa F, Nguyen P. Natural Language Processing for Breast Imaging: A Systematic Review. Diagnostics (Basel) 2023; 13:diagnostics13081420. [PMID: 37189521 DOI: 10.3390/diagnostics13081420] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2023] [Revised: 04/05/2023] [Accepted: 04/11/2023] [Indexed: 05/17/2023] Open
Abstract
Natural Language Processing (NLP) has gained prominence in diagnostic radiology, offering a promising tool for improving breast imaging triage, diagnosis, lesion characterization, and treatment management in breast cancer and other breast diseases. This review provides a comprehensive overview of recent advances in NLP for breast imaging, covering the main techniques and applications in this field. Specifically, we discuss various NLP methods used to extract relevant information from clinical notes, radiology reports, and pathology reports and their potential impact on the accuracy and efficiency of breast imaging. In addition, we reviewed the state-of-the-art in NLP-based decision support systems for breast imaging, highlighting the challenges and opportunities of NLP applications for breast imaging in the future. Overall, this review underscores the potential of NLP in enhancing breast imaging care and offers insights for clinicians and researchers interested in this exciting and rapidly evolving field.
Collapse
Affiliation(s)
- Kareem Mahmoud Diab
- Institute for Data Science and Computing, University of Miami, Miami, FL 33146, USA
| | - Jamie Deng
- Department of Computer Science, University of Miami, Miami, FL 33146, USA
| | - Yusen Wu
- Institute for Data Science and Computing, University of Miami, Miami, FL 33146, USA
| | - Yelena Yesha
- Institute for Data Science and Computing, University of Miami, Miami, FL 33146, USA
- Department of Computer Science, University of Miami, Miami, FL 33146, USA
- Department of Radiology, Miller School of Medicine, University of Miami, Miami, FL 33146, USA
| | - Fernando Collado-Mesa
- Department of Radiology, Miller School of Medicine, University of Miami, Miami, FL 33146, USA
| | - Phuong Nguyen
- Institute for Data Science and Computing, University of Miami, Miami, FL 33146, USA
- Department of Computer Science, University of Miami, Miami, FL 33146, USA
- OpenKnect Inc., Halethorpe, MD 21227, USA
| |
Collapse
|