1
|
Harada Y, Sakamoto T, Sugimoto S, Shimizu T. Longitudinal Changes in Diagnostic Accuracy of a Differential Diagnosis List Developed by an AI-Based Symptom Checker: Retrospective Observational Study. JMIR Form Res 2024; 8:e53985. [PMID: 38758588 PMCID: PMC11143391 DOI: 10.2196/53985] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2023] [Revised: 03/23/2024] [Accepted: 04/24/2024] [Indexed: 05/18/2024] Open
Abstract
BACKGROUND Artificial intelligence (AI) symptom checker models should be trained using real-world patient data to improve their diagnostic accuracy. Given that AI-based symptom checkers are currently used in clinical practice, their performance should improve over time. However, longitudinal evaluations of the diagnostic accuracy of these symptom checkers are limited. OBJECTIVE This study aimed to assess the longitudinal changes in the accuracy of differential diagnosis lists created by an AI-based symptom checker used in the real world. METHODS This was a single-center, retrospective, observational study. Patients who visited an outpatient clinic without an appointment between May 1, 2019, and April 30, 2022, and who were admitted to a community hospital in Japan within 30 days of their index visit were considered eligible. We only included patients who underwent an AI-based symptom checkup at the index visit, and the diagnosis was finally confirmed during follow-up. Final diagnoses were categorized as common or uncommon, and all cases were categorized as typical or atypical. The primary outcome measure was the accuracy of the differential diagnosis list created by the AI-based symptom checker, defined as the final diagnosis in a list of 10 differential diagnoses created by the symptom checker. To assess the change in the symptom checker's diagnostic accuracy over 3 years, we used a chi-square test to compare the primary outcome over 3 periods: from May 1, 2019, to April 30, 2020 (first year); from May 1, 2020, to April 30, 2021 (second year); and from May 1, 2021, to April 30, 2022 (third year). RESULTS A total of 381 patients were included. Common diseases comprised 257 (67.5%) cases, and typical presentations were observed in 298 (78.2%) cases. Overall, the accuracy of the differential diagnosis list created by the AI-based symptom checker was 172 (45.1%), which did not differ across the 3 years (first year: 97/219, 44.3%; second year: 32/72, 44.4%; and third year: 43/90, 47.7%; P=.85). The accuracy of the differential diagnosis list created by the symptom checker was low in those with uncommon diseases (30/124, 24.2%) and atypical presentations (12/83, 14.5%). In the multivariate logistic regression model, common disease (P<.001; odds ratio 4.13, 95% CI 2.50-6.98) and typical presentation (P<.001; odds ratio 6.92, 95% CI 3.62-14.2) were significantly associated with the accuracy of the differential diagnosis list created by the symptom checker. CONCLUSIONS A 3-year longitudinal survey of the diagnostic accuracy of differential diagnosis lists developed by an AI-based symptom checker, which has been implemented in real-world clinical practice settings, showed no improvement over time. Uncommon diseases and atypical presentations were independently associated with a lower diagnostic accuracy. In the future, symptom checkers should be trained to recognize uncommon conditions.
Collapse
Affiliation(s)
- Yukinori Harada
- Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, Shimotsuga, Japan
- Department of General Medicine, Nagano Chuo Hospital, Nagano, Japan
| | - Tetsu Sakamoto
- Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, Shimotsuga, Japan
| | - Shu Sugimoto
- Department of Medicine (Neurology and Rheumatology), Shinshu University School of Medicine, Matsumoto, Japan
| | - Taro Shimizu
- Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, Shimotsuga, Japan
| |
Collapse
|
2
|
Martindale APL, Ng B, Ngai V, Kale AU, Ferrante di Ruffano L, Golub RM, Collins GS, Moher D, McCradden MD, Oakden-Rayner L, Rivera SC, Calvert M, Kelly CJ, Lee CS, Yau C, Chan AW, Keane PA, Beam AL, Denniston AK, Liu X. Concordance of randomised controlled trials for artificial intelligence interventions with the CONSORT-AI reporting guidelines. Nat Commun 2024; 15:1619. [PMID: 38388497 PMCID: PMC10883966 DOI: 10.1038/s41467-024-45355-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2023] [Accepted: 01/22/2024] [Indexed: 02/24/2024] Open
Abstract
The Consolidated Standards of Reporting Trials extension for Artificial Intelligence interventions (CONSORT-AI) was published in September 2020. Since its publication, several randomised controlled trials (RCTs) of AI interventions have been published but their completeness and transparency of reporting is unknown. This systematic review assesses the completeness of reporting of AI RCTs following publication of CONSORT-AI and provides a comprehensive summary of RCTs published in recent years. 65 RCTs were identified, mostly conducted in China (37%) and USA (18%). Median concordance with CONSORT-AI reporting was 90% (IQR 77-94%), although only 10 RCTs explicitly reported its use. Several items were consistently under-reported, including algorithm version, accessibility of the AI intervention or code, and references to a study protocol. Only 3 of 52 included journals explicitly endorsed or mandated CONSORT-AI. Despite a generally high concordance amongst recent AI RCTs, some AI-specific considerations remain systematically poorly reported. Further encouragement of CONSORT-AI adoption by journals and funders may enable more complete adoption of the full CONSORT-AI guidelines.
Collapse
Affiliation(s)
| | - Benjamin Ng
- Birmingham and Midland Eye Centre, Sandwell and West Birmingham NHS Trust, Birmingham, UK
- Christ Church, University of Oxford, Oxford, UK
| | - Victoria Ngai
- University College London Medical School, London, UK
| | - Aditya U Kale
- Institute of Inflammation and Ageing, University of Birmingham, Birmingham, UK
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, University of Birmingham, Birmingham, UK
| | | | - Robert M Golub
- Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Gary S Collins
- Centre for Statistics in Medicine//UK EQUATOR Centre, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - David Moher
- Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottowa, ON, Canada
| | - Melissa D McCradden
- Department of Bioethics, The Hospital for Sick Children, Toronto, ON, Canada
- Genetics & Genome Biology Research Program, Peter Gilgan Centre for Research & Learning, Toronto, ON, Canada
- Division of Clinical and Public Health, Dalla Lana School of Public Health, Toronto, ON, Canada
| | - Lauren Oakden-Rayner
- Australian Institute for Machine Learning, University of Adelaide, Adelaide, SA, Australia
| | - Samantha Cruz Rivera
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK
- Centre for Patient Reported Outcomes Research (CPROR), Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
| | - Melanie Calvert
- National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, University of Birmingham, Birmingham, UK
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK
- Centre for Patient Reported Outcomes Research (CPROR), Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
- NIHR Applied Research Collaboration (ARC) West Midlands, University of Birmingham, Birmingham, UK
- NIHR Blood and Transplant Research Unit (BTRU) in Precision Transplant and Cellular Therapeutics, University of Birmingham, Birmingham, UK
| | | | | | - Christopher Yau
- Nuffield Department of Women's and Reproductive Health, University of Oxford, Oxford, UK
- Health Data Research UK, London, UK
| | - An-Wen Chan
- Department of Medicine, Women's College Hospital. University of Toronto, Toronto, ON, Canada
| | - Pearse A Keane
- NIHR Biomedical Research Centre at Moorfields, Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, London, UK
| | - Andrew L Beam
- Department of Epidemiology, Harvard. T.H. Chan School of Public Health, Boston, MA, USA
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Alastair K Denniston
- Institute of Inflammation and Ageing, University of Birmingham, Birmingham, UK
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, University of Birmingham, Birmingham, UK
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK
- NIHR Biomedical Research Centre at Moorfields, Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, London, UK
| | - Xiaoxuan Liu
- Institute of Inflammation and Ageing, University of Birmingham, Birmingham, UK.
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK.
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK.
| |
Collapse
|
3
|
Pohanka M. Current trends in digital camera-based bioassays for point-of-care tests. Clin Chim Acta 2024; 552:117677. [PMID: 38000459 DOI: 10.1016/j.cca.2023.117677] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/07/2023] [Revised: 11/18/2023] [Accepted: 11/20/2023] [Indexed: 11/26/2023]
Abstract
Point-of-care and bedside tests are analytical devices suitable for a growing role in the current healthcare system and provide the opportunity to achieve an exact diagnosis by an untrained person and in various conditions and sites where it is necessary. Using a digital camera integrated into a well-accessible device like a smartphone brings a new way in which a colorimetric point-of-care diagnostic test can provide unbiased data. This review summarizes basic facts about the colorimetric point-of-care tests, principles of how to use a portable device with a camera in the assay, applications of digital cameras for the current tests, and new devices described in the recent papers. An overview of the recent literature and a discussion of recent developments and future trends are provided.
Collapse
Affiliation(s)
- Miroslav Pohanka
- Faculty of Military Health Sciences, University of Defense, Trebesska 1575, Hradec Kralove CZ-50001, Czech Republic.
| |
Collapse
|
4
|
Harada Y, Tomiyama S, Sakamoto T, Sugimoto S, Kawamura R, Yokose M, Hayashi A, Shimizu T. Effects of Combinational Use of Additional Differential Diagnostic Generators on the Diagnostic Accuracy of the Differential Diagnosis List Developed by an Artificial Intelligence-Driven Automated History-Taking System: Pilot Cross-Sectional Study. JMIR Form Res 2023; 7:e49034. [PMID: 37531164 PMCID: PMC10433017 DOI: 10.2196/49034] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2023] [Revised: 06/23/2023] [Accepted: 07/19/2023] [Indexed: 08/03/2023] Open
Abstract
BACKGROUND Low diagnostic accuracy is a major concern in automated medical history-taking systems with differential diagnosis (DDx) generators. Extending the concept of collective intelligence to the field of DDx generators such that the accuracy of judgment becomes higher when accepting an integrated diagnosis list from multiple people than when accepting a diagnosis list from a single person may be a possible solution. OBJECTIVE The purpose of this study is to assess whether the combined use of several DDx generators improves the diagnostic accuracy of DDx lists. METHODS We used medical history data and the top 10 DDx lists (index DDx lists) generated by an artificial intelligence (AI)-driven automated medical history-taking system from 103 patients with confirmed diagnoses. Two research physicians independently created the other top 10 DDx lists (second and third DDx lists) per case by imputing key information into the other 2 DDx generators based on the medical history generated by the automated medical history-taking system without reading the index lists generated by the automated medical history-taking system. We used the McNemar test to assess the improvement in diagnostic accuracy from the index DDx lists to the three types of combined DDx lists: (1) simply combining DDx lists from the index, second, and third lists; (2) creating a new top 10 DDx list using a 1/n weighting rule; and (3) creating new lists with only shared diagnoses among DDx lists from the index, second, and third lists. We treated the data generated by 2 research physicians from the same patient as independent cases. Therefore, the number of cases included in analyses in the case using 2 additional lists was 206 (103 cases × 2 physicians' input). RESULTS The diagnostic accuracy of the index lists was 46% (47/103). Diagnostic accuracy was improved by simply combining the other 2 DDx lists (133/206, 65%, P<.001), whereas the other 2 combined DDx lists did not improve the diagnostic accuracy of the DDx lists (106/206, 52%, P=.05 in the collective list with the 1/n weighting rule and 29/206, 14%, P<.001 in the only shared diagnoses among the 3 DDx lists). CONCLUSIONS Simply adding each of the top 10 DDx lists from additional DDx generators increased the diagnostic accuracy of the DDx list by approximately 20%, suggesting that the combinational use of DDx generators early in the diagnostic process is beneficial.
Collapse
Affiliation(s)
- Yukinori Harada
- Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, Mibu, Shimotsugagun, Japan
- Department of Internal Medicine, Nagano Chuo Hospital, Nagano, Japan
| | - Shusaku Tomiyama
- Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, Mibu, Shimotsugagun, Japan
| | - Tetsu Sakamoto
- Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, Mibu, Shimotsugagun, Japan
| | - Shu Sugimoto
- Department of Internal Medicine, Nagano Chuo Hospital, Nagano, Japan
| | - Ren Kawamura
- Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, Mibu, Shimotsugagun, Japan
| | - Masashi Yokose
- Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, Mibu, Shimotsugagun, Japan
| | - Arisa Hayashi
- Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, Mibu, Shimotsugagun, Japan
| | - Taro Shimizu
- Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, Mibu, Shimotsugagun, Japan
| |
Collapse
|
5
|
Mirkin S, Albensi BC. Should artificial intelligence be used in conjunction with Neuroimaging in the diagnosis of Alzheimer's disease? Front Aging Neurosci 2023; 15:1094233. [PMID: 37187577 PMCID: PMC10177660 DOI: 10.3389/fnagi.2023.1094233] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2022] [Accepted: 03/27/2023] [Indexed: 05/17/2023] Open
Abstract
Alzheimer's disease (AD) is a progressive, neurodegenerative disorder that affects memory, thinking, behavior, and other cognitive functions. Although there is no cure, detecting AD early is important for the development of a therapeutic plan and a care plan that may preserve cognitive function and prevent irreversible damage. Neuroimaging, such as magnetic resonance imaging (MRI), computed tomography (CT), and positron emission tomography (PET), has served as a critical tool in establishing diagnostic indicators of AD during the preclinical stage. However, as neuroimaging technology quickly advances, there is a challenge in analyzing and interpreting vast amounts of brain imaging data. Given these limitations, there is great interest in using artificial Intelligence (AI) to assist in this process. AI introduces limitless possibilities in the future diagnosis of AD, yet there is still resistance from the healthcare community to incorporate AI in the clinical setting. The goal of this review is to answer the question of whether AI should be used in conjunction with neuroimaging in the diagnosis of AD. To answer the question, the possible benefits and disadvantages of AI are discussed. The main advantages of AI are its potential to improve diagnostic accuracy, improve the efficiency in analyzing radiographic data, reduce physician burnout, and advance precision medicine. The disadvantages include generalization and data shortage, lack of in vivo gold standard, skepticism in the medical community, potential for physician bias, and concerns over patient information, privacy, and safety. Although the challenges present fundamental concerns and must be addressed when the time comes, it would be unethical not to use AI if it can improve patient health and outcome.
Collapse
Affiliation(s)
- Sophia Mirkin
- Dr. Kiran C. Patel College of Osteopathic Medicine, Nova Southeastern University, Fort Lauderdale, FL, United States
| | - Benedict C. Albensi
- Barry and Judy Silverman College of Pharmacy, Nova Southeastern University, Fort Lauderdale, FL, United States
- St. Boniface Hospital Research, Winnipeg, MB, Canada
- University of Manitoba, Winnipeg, MB, Canada
| |
Collapse
|
6
|
Miyachi Y, Ishii O, Torigoe K. Design, implementation, and evaluation of the computer-aided clinical decision support system based on learning-to-rank: collaboration between physicians and machine learning in the differential diagnosis process. BMC Med Inform Decis Mak 2023; 23:26. [PMID: 36732730 PMCID: PMC9896739 DOI: 10.1186/s12911-023-02123-5] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2022] [Accepted: 01/24/2023] [Indexed: 02/04/2023] Open
Abstract
BACKGROUND We are researching, developing, and publishing the clinical decision support system based on learning-to-rank. The main objectives are (1) To support for differential diagnoses performed by internists and general practitioners and (2) To prevent diagnostic errors made by physicians. The main features are that "A physician inputs a patient's symptoms, findings, and test results to the system, and the system outputs a ranking list of possible diseases". METHOD The software libraries for machine learning and artificial intelligence are TensorFlow and TensorFlow Ranking. The prediction algorithm is Learning-to-Rank with the listwise approach. The ranking metric is normalized discounted cumulative gain (NDCG). The loss functions are Approximate NDCG (A-NDCG). We evaluated the machine learning performance on k-fold cross-validation. We evaluated the differential diagnosis performance with validated cases. RESULTS The machine learning performance of our system was much higher than that of the conventional system. The differential diagnosis performance of our system was much higher than that of the conventional system. We have shown that the clinical decision support system prevents physicians' diagnostic errors due to confirmation bias. CONCLUSIONS We have demonstrated that the clinical decision support system is useful for supporting differential diagnoses and preventing diagnostic errors. We propose that differential diagnosis by physicians and learning-to-rank by machine has a high affinity. We found that information retrieval and clinical decision support systems have much in common (Target data, learning-to-rank, etc.). We propose that Clinical Decision Support Systems have the potential to support: (1) recall of rare diseases, (2) differential diagnoses for difficult-to-diagnoses cases, and (3) prevention of diagnostic errors. Our system can potentially evolve into an explainable clinical decision support system.
Collapse
Affiliation(s)
- Yasuhiko Miyachi
- The Society for Computer-Aided Clinical Decision Support System, Torigoe Clinic, Ibara, Okayama Japan
| | - Osamu Ishii
- The Society for Computer-Aided Clinical Decision Support System, Torigoe Clinic, Ibara, Okayama Japan
| | | |
Collapse
|
7
|
Kawamura R, Harada Y, Sugimoto S, Nagase Y, Katsukura S, Shimizu T. Incidence of diagnostic errors in unplanned hospitalized patients using an automated medical history-taking system with differential diagnosis generator: retrospective observational study (Preprint). JMIR Med Inform 2021; 10:e35225. [PMID: 35084347 PMCID: PMC8832260 DOI: 10.2196/35225] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2021] [Revised: 12/11/2021] [Accepted: 01/02/2022] [Indexed: 11/23/2022] Open
Abstract
Background Automated medical history–taking systems that generate differential diagnosis lists have been suggested to contribute to improved diagnostic accuracy. However, the effect of these systems on diagnostic errors in clinical practice remains unknown. Objective This study aimed to assess the incidence of diagnostic errors in an outpatient department, where an artificial intelligence (AI)–driven automated medical history–taking system that generates differential diagnosis lists was implemented in clinical practice. Methods We conducted a retrospective observational study using data from a community hospital in Japan. We included patients aged 20 years and older who used an AI-driven, automated medical history–taking system that generates differential diagnosis lists in the outpatient department of internal medicine for whom the index visit was between July 1, 2019, and June 30, 2020, followed by unplanned hospitalization within 14 days. The primary endpoint was the incidence of diagnostic errors, which were detected using the Revised Safer Dx Instrument by at least two independent reviewers. To evaluate the effect of differential diagnosis lists from the AI system on the incidence of diagnostic errors, we compared the incidence of these errors between a group where the AI system generated the final diagnosis in the differential diagnosis list and a group where the AI system did not generate the final diagnosis in the list; the Fisher exact test was used for comparison between these groups. For cases with confirmed diagnostic errors, further review was conducted to identify the contributing factors of these errors via discussion among three reviewers, using the Safer Dx Process Breakdown Supplement as a reference. Results A total of 146 patients were analyzed. A final diagnosis was confirmed for 138 patients and was observed in the differential diagnosis list from the AI system for 69 patients. Diagnostic errors occurred in 16 out of 146 patients (11.0%, 95% CI 6.4%-17.2%). Although statistically insignificant, the incidence of diagnostic errors was lower in cases where the final diagnosis was included in the differential diagnosis list from the AI system than in cases where the final diagnosis was not included in the list (7.2% vs 15.9%, P=.18). Conclusions The incidence of diagnostic errors among patients in the outpatient department of internal medicine who used an automated medical history–taking system that generates differential diagnosis lists seemed to be lower than the previously reported incidence of diagnostic errors. This result suggests that the implementation of an automated medical history–taking system that generates differential diagnosis lists could be beneficial for diagnostic safety in the outpatient department of internal medicine.
Collapse
Affiliation(s)
- Ren Kawamura
- Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, Mibu, Japan
| | - Yukinori Harada
- Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, Mibu, Japan
- Department of Internal Medicine, Nagano Chuo Hospital, Nagano, Japan
| | - Shu Sugimoto
- Department of Internal Medicine, Nagano Chuo Hospital, Nagano, Japan
| | - Yuichiro Nagase
- Department of Internal Medicine, Nagano Chuo Hospital, Nagano, Japan
| | - Shinichi Katsukura
- Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, Mibu, Japan
| | - Taro Shimizu
- Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, Mibu, Japan
| |
Collapse
|
8
|
Clinical Decision Support Systems for Diagnosis in Primary Care: A Scoping Review. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2021; 18:ijerph18168435. [PMID: 34444182 PMCID: PMC8391274 DOI: 10.3390/ijerph18168435] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/25/2021] [Revised: 07/29/2021] [Accepted: 08/04/2021] [Indexed: 01/18/2023]
Abstract
Diagnosis is one of the crucial tasks performed by primary care physicians; however, primary care is at high risk of diagnostic errors due to the characteristics and uncertainties associated with the field. Prevention of diagnostic errors in primary care requires urgent action, and one of the possible methods is the use of health information technology. Its modes such as clinical decision support systems (CDSS) have been demonstrated to improve the quality of care in a variety of medical settings, including hospitals and primary care centers, though its usefulness in the diagnostic domain is still unknown. We conducted a scoping review to confirm the usefulness of the CDSS in the diagnostic domain in primary care and to identify areas that need to be explored. Search terms were chosen to cover the three dimensions of interest: decision support systems, diagnosis, and primary care. A total of 26 studies were included in the review. As a result, we found that the CDSS and reminder tools have significant effects on screening for common chronic diseases; however, the CDSS has not yet been fully validated for the diagnosis of acute and uncommon chronic diseases. Moreover, there were few studies involving non-physicians.
Collapse
|
9
|
Harada Y, Katsukura S, Kawamura R, Shimizu T. Effects of a Differential Diagnosis List of Artificial Intelligence on Differential Diagnoses by Physicians: An Exploratory Analysis of Data from a Randomized Controlled Study. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2021; 18:ijerph18115562. [PMID: 34070958 PMCID: PMC8196999 DOI: 10.3390/ijerph18115562] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Received: 03/31/2021] [Revised: 05/07/2021] [Accepted: 05/21/2021] [Indexed: 11/16/2022]
Abstract
A diagnostic decision support system (DDSS) is expected to reduce diagnostic errors. However, its effect on physicians' diagnostic decisions remains unclear. Our study aimed to assess the prevalence of diagnoses from artificial intelligence (AI) in physicians' differential diagnoses when using AI-driven DDSS that generates a differential diagnosis from the information entered by the patient before the clinical encounter on physicians' differential diagnoses. In this randomized controlled study, an exploratory analysis was performed. Twenty-two physicians were required to generate up to three differential diagnoses per case by reading 16 clinical vignettes. The participants were divided into two groups, an intervention group, and a control group, with and without a differential diagnosis list of AI, respectively. The prevalence of physician diagnosis identical with the differential diagnosis of AI (primary outcome) was significantly higher in the intervention group than in the control group (70.2% vs. 55.1%, p < 0.001). The primary outcome was significantly >10% higher in the intervention group than in the control group, except for attending physicians, and physicians who did not trust AI. This study suggests that at least 15% of physicians' differential diagnoses were affected by the differential diagnosis list in the AI-driven DDSS.
Collapse
Affiliation(s)
- Yukinori Harada
- Department of General Internal Medicine, Nagano Chuo Hospital, Nagano 380-0814, Japan;
- Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, Tochigi 321-0293, Japan; (S.K.); (R.K.)
| | - Shinichi Katsukura
- Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, Tochigi 321-0293, Japan; (S.K.); (R.K.)
| | - Ren Kawamura
- Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, Tochigi 321-0293, Japan; (S.K.); (R.K.)
| | - Taro Shimizu
- Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, Tochigi 321-0293, Japan; (S.K.); (R.K.)
- Correspondence: ; Tel.: +81-282-86-1111
| |
Collapse
|