1
|
Scott IA. Using information technology to reduce diagnostic error: still a bridge too far? Intern Med J 2022; 52:908-911. [PMID: 35718736 DOI: 10.1111/imj.15804] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2022] [Accepted: 04/28/2022] [Indexed: 11/28/2022]
Affiliation(s)
- Ian A Scott
- Internal Medicine and Clinical Epidemiology, Princess Alexandra Hospital, Brisbane, Queensland, Australia.,School of Clinical Medicine, University of Queensland, Brisbane, Queensland, Australia
| |
Collapse
|
2
|
Sibbald M, Monteiro S, Sherbino J, LoGiudice A, Friedman C, Norman G. Should electronic differential diagnosis support be used early or late in the diagnostic process? A multicentre experimental study of Isabel. BMJ Qual Saf 2021; 31:426-433. [PMID: 34611040 PMCID: PMC9132870 DOI: 10.1136/bmjqs-2021-013493] [Citation(s) in RCA: 22] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2021] [Accepted: 09/09/2021] [Indexed: 12/17/2022]
Abstract
Background Diagnostic errors unfortunately remain common. Electronic differential diagnostic support (EDS) systems may help, but it is unclear when and how they ought to be integrated into the diagnostic process. Objective To explore how much EDS improves diagnostic accuracy, and whether EDS should be used early or late in the diagnostic process. Setting 6 Canadian medical schools. A volunteer sample of 67 medical students, 62 residents in internal medicine or emergency medicine, and 61 practising internists or emergency medicine physicians were recruited in May through June 2020. Intervention Participants were randomised to make use of EDS either early (after the chief complaint) or late (after the complete history and physical is available) in the diagnostic process while solving each of 16 written cases. For each case, we measured the number of diagnoses proposed in the differential diagnosis and how often the correct diagnosis was present within the differential. Results EDS increased the number of diagnostic hypotheses by 2.32 (95% CI 2.10 to 2.49) when used early in the process and 0.89 (95% CI 0.69 to 1.10) when used late in the process (both p<0.001). Both early and late use of EDS increased the likelihood of the correct diagnosis being present in the differential (7% and 8%, respectively, both p<0.001). Whereas early use increased the number of diagnostic hypotheses (most notably for students and residents), late use increased the likelihood of the correct diagnosis being present in the differential regardless of one’s experience level. Conclusions and relevance EDS increased the number of diagnostic hypotheses and the likelihood of the correct diagnosis appearing in the differential, and these effects persisted irrespective of whether EDS was used early or late in the diagnostic process.
Collapse
Affiliation(s)
- Matt Sibbald
- Department of Medicine, McMaster University, Hamilton, Ontario, Canada
| | - Sandra Monteiro
- Department of Health Evidence and Impact, McMaster University, Hamilton, Ontario, Canada
| | - Jonathan Sherbino
- Department of Medicine, McMaster University, Hamilton, Ontario, Canada
| | | | | | - Geoffrey Norman
- Department of Health Evidence and Impact, McMaster University, Hamilton, Ontario, Canada
| |
Collapse
|
3
|
What does it mean to provide decision support to a responsible and competent expert? EURO JOURNAL ON DECISION PROCESSES 2020. [DOI: 10.1007/s40070-020-00116-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|
4
|
Jia J, Wang R, An Z, Guo Y, Ni X, Shi T. RDAD: A Machine Learning System to Support Phenotype-Based Rare Disease Diagnosis. Front Genet 2018; 9:587. [PMID: 30564269 PMCID: PMC6288202 DOI: 10.3389/fgene.2018.00587] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2018] [Accepted: 11/15/2018] [Indexed: 01/21/2023] Open
Abstract
DNA sequencing has allowed for the discovery of the genetic cause for a considerable number of diseases, paving the way for new disease diagnostics. However, due to the lack of clinical samples and records, the molecular cause for rare diseases is always hard to identify, significantly limiting the number of rare Mendelian diseases diagnosed through sequencing technologies. Clinical phenotype information therefore becomes a major resource to diagnose rare diseases. In this article, we adopted both a phenotypic similarity method and a machine learning method to build four diagnostic models to support rare disease diagnosis. All the diagnostic models were validated using the real medical records from RAMEDIS. Each model provides a list of the top 10 candidate diseases as the prediction outcome and the results showed that all models had a high diagnostic precision (≥98%) with the highest recall reaching up to 95% while the models with machine learning methods showed the best performance. To promote effective diagnosis for rare disease in clinical application, we developed the phenotype-based Rare Disease Auxiliary Diagnosis system (RDAD) to assist clinicians in diagnosing rare diseases with the above four diagnostic models. The system is freely accessible through http://www.unimd.org/RDAD/.
Collapse
Affiliation(s)
- Jinmeng Jia
- The Center for Bioinformatics and Computational Biology, Shanghai Key Laboratory of Regulatory Biology, The Institute of Biomedical Sciences and School of Life Sciences, East China Normal University, Shanghai, China
| | - Ruiyuan Wang
- The Center for Bioinformatics and Computational Biology, Shanghai Key Laboratory of Regulatory Biology, The Institute of Biomedical Sciences and School of Life Sciences, East China Normal University, Shanghai, China
| | - Zhongxin An
- The Center for Bioinformatics and Computational Biology, Shanghai Key Laboratory of Regulatory Biology, The Institute of Biomedical Sciences and School of Life Sciences, East China Normal University, Shanghai, China
| | - Yongli Guo
- Beijing Key Laboratory for Pediatric Diseases of Otolaryngology, Head and Neck Surgery, The Ministry of Education Key Laboratory of Major Diseases in Children, Beijing Pediatric Research Institute, Beijing Children's Hospital, Capital Medical University, National Center for Children's Health, Beijing, China
| | - Xi Ni
- Beijing Key Laboratory for Pediatric Diseases of Otolaryngology, Head and Neck Surgery, The Ministry of Education Key Laboratory of Major Diseases in Children, Beijing Pediatric Research Institute, Beijing Children's Hospital, Capital Medical University, National Center for Children's Health, Beijing, China
| | - Tieliu Shi
- The Center for Bioinformatics and Computational Biology, Shanghai Key Laboratory of Regulatory Biology, The Institute of Biomedical Sciences and School of Life Sciences, East China Normal University, Shanghai, China
- National Center for International Research of Biological Targeting Diagnosis and Therapy/Guangxi Key Laboratory of Biological Targeting Diagnosis and Therapy Research/Collaborative Innovation Center for Targeting Tumor Diagnosis and Therapy, Guangxi Medical University, Nanning, Guangxi, China
| |
Collapse
|
5
|
Hossain MS, Rahaman S, Mustafa R, Andersson K. A belief rule-based expert system to assess suspicion of acute coronary syndrome (ACS) under uncertainty. Soft comput 2017. [DOI: 10.1007/s00500-017-2732-2] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
|
6
|
Baumgartner C, Caiani EG, Dickhaus H, Kulikowski CA, Schiecke K, van Bemmel JH, Witte H. Discussion of "Computational Electrocardiography: Revisiting Holter ECG Monitoring". Methods Inf Med 2016; 55:312-21. [PMID: 27406570 DOI: 10.3414/me15-15-0009] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
This article is part of a For-Discussion-Section of Methods of Information in Medicine about the paper "Computational Electrocardiography: Revisiting Holter ECG Monitoring" written by Thomas M. Deserno and Nikolaus Marx. It is introduced by an editorial. This article contains the combined commentaries invited to independently comment on the paper of Deserno and Marx. In subsequent issues the discussion can continue through letters to the editor.
Collapse
Affiliation(s)
| | | | | | | | | | | | - Herbert Witte
- Herbert Witte, Institute of Medical Statistics, Computer Sciences and Documentation, Jena University Hospital, Friedrich-Schiller University, Bachstraße 18, 07743 Jena, E-mail:
| |
Collapse
|
7
|
Farmer N. An update and further testing of a knowledge-based diagnostic clinical decision support system for musculoskeletal disorders of the shoulder for use in a primary care setting. J Eval Clin Pract 2014; 20:589-95. [PMID: 24828447 DOI: 10.1111/jep.12153] [Citation(s) in RCA: 18] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 04/03/2014] [Indexed: 12/27/2022]
Abstract
RATIONALE, AIMS AND OBJECTIVES A prototype diagnostic clinical decision support system (CDSS) was developed to assist primary care clinicians (general practitioners) in clinical decision making, aimed at reducing diagnostic errors. The prototype CDSS showed some promise with high levels of validity and reliability; however, issues regarding the underlying Bayesian belief network (BBN), small sample size and use of radiological imaging as a gold standard measure were highlighted that required further investigation before considering clinical testing. METHODS The prototype CDSS was reviewed and updated based on computer science literature and expert (orthopaedic consultant) opinion. The updated CDSS was tested by comparing its diagnostic outcome against the diagnosis of 93 case studies as determined by expert opinion combined with arthroscopy findings or radiological imaging. RESULTS The updated CDSS showed significant high levels of sensitivity (91%), specificity (98%), positive likelihood ratio (53.12) and negative likelihood ratio (0.08) with a kappa value of 0.88 to a confidence level of 99% compared with expert diagnosis combined with arthroscopy findings or radiological imaging. CONCLUSIONS The results suggest that the updated CDSS has addressed the issues highlighted from the initial research while maintaining high levels of validity and reliability. The updated CDSS is now ready for clinical testing.
Collapse
Affiliation(s)
- Nicholas Farmer
- Faculty of Medicine, University of Southampton, Highfield Campus, Southampton, Hampshire, UK
| |
Collapse
|
8
|
A bi-level belief rule based decision support system for diagnosis of lymph node metastasis in gastric cancer. Knowl Based Syst 2013. [DOI: 10.1016/j.knosys.2013.09.001] [Citation(s) in RCA: 49] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
|
9
|
Friedman C, Rindflesch TC, Corn M. Natural language processing: state of the art and prospects for significant progress, a workshop sponsored by the National Library of Medicine. J Biomed Inform 2013; 46:765-73. [PMID: 23810857 DOI: 10.1016/j.jbi.2013.06.004] [Citation(s) in RCA: 56] [Impact Index Per Article: 5.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2013] [Revised: 06/07/2013] [Accepted: 06/07/2013] [Indexed: 01/29/2023]
Abstract
Natural language processing (NLP) is crucial for advancing healthcare because it is needed to transform relevant information locked in text into structured data that can be used by computer processes aimed at improving patient care and advancing medicine. In light of the importance of NLP to health, the National Library of Medicine (NLM) recently sponsored a workshop to review the state of the art in NLP focusing on text in English, both in biomedicine and in the general language domain. Specific goals of the NLM-sponsored workshop were to identify the current state of the art, grand challenges and specific roadblocks, and to identify effective use and best practices. This paper reports on the main outcomes of the workshop, including an overview of the state of the art, strategies for advancing the field, and obstacles that need to be addressed, resulting in recommendations for a research agenda intended to advance the field.
Collapse
Affiliation(s)
- Carol Friedman
- Department of Biomedical Informatics, Columbia University, United States.
| | | | | |
Collapse
|
10
|
Papier A. Decision support in dermatology and medicine: history and recent developments. ACTA ACUST UNITED AC 2013; 31:153-9. [PMID: 22929351 DOI: 10.1016/j.sder.2012.06.005] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2012] [Revised: 06/06/2012] [Accepted: 06/19/2012] [Indexed: 11/29/2022]
Abstract
This article is focused on diagnostic decision support tools and will provide a brief history of clinical decision support (CDS), examine the components of CDS and its associated terminology, and discuss recent developments in the use and application of CDS systems, particularly in the field of dermatology. For this article, we use CDS to mean an interactive system allowing input of patient-specific information and providing customized medical knowledge-based results via automated reasoning, for example, a set of rules and/or an underlying logic, and associations.
Collapse
Affiliation(s)
- Art Papier
- Dermatology and Medical Informatics, University of Rochester College of Medicine, Rochester, NY 14642, USA.
| |
Collapse
|
11
|
Wachter RM. Why Diagnostic Errors Don’t Get Any Respect—And What Can Be Done About Them. Health Aff (Millwood) 2010; 29:1605-10. [DOI: 10.1377/hlthaff.2009.0513] [Citation(s) in RCA: 85] [Impact Index Per Article: 6.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Affiliation(s)
- Robert M. Wachter
- Robert M. Wachter ( ) is a professor and associate chair of the Department of Medicine at the University of California, San Francisco
| |
Collapse
|
12
|
Köhler S, Schulz MH, Krawitz P, Bauer S, Dölken S, Ott CE, Mundlos C, Horn D, Mundlos S, Robinson PN. Clinical diagnostics in human genetics with semantic similarity searches in ontologies. Am J Hum Genet 2009; 85:457-64. [PMID: 19800049 DOI: 10.1016/j.ajhg.2009.09.003] [Citation(s) in RCA: 323] [Impact Index Per Article: 21.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2009] [Revised: 08/04/2009] [Accepted: 09/01/2009] [Indexed: 10/20/2022] Open
Abstract
The differential diagnostic process attempts to identify candidate diseases that best explain a set of clinical features. This process can be complicated by the fact that the features can have varying degrees of specificity, as well as by the presence of features unrelated to the disease itself. Depending on the experience of the physician and the availability of laboratory tests, clinical abnormalities may be described in greater or lesser detail. We have adapted semantic similarity metrics to measure phenotypic similarity between queries and hereditary diseases annotated with the use of the Human Phenotype Ontology (HPO) and have developed a statistical model to assign p values to the resulting similarity scores, which can be used to rank the candidate diseases. We show that our approach outperforms simpler term-matching approaches that do not take the semantic interrelationships between terms into account. The advantage of our approach was greater for queries containing phenotypic noise or imprecise clinical descriptions. The semantic network defined by the HPO can be used to refine the differential diagnosis by suggesting clinical features that, if present, best differentiate among the candidate diagnoses. Thus, semantic similarity searches in ontologies represent a useful way of harnessing the semantic structure of human phenotypic abnormalities to help with the differential diagnosis. We have implemented our methods in a freely available web application for the field of human Mendelian disorders.
Collapse
|
13
|
Miller RA. Computer-assisted diagnostic decision support: history, challenges, and possible paths forward. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2009; 14 Suppl 1:89-106. [PMID: 19672686 DOI: 10.1007/s10459-009-9186-y] [Citation(s) in RCA: 26] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/14/2009] [Accepted: 07/14/2009] [Indexed: 05/28/2023]
Abstract
This paper presents a brief history of computer-assisted diagnosis, including challenges and future directions. Some ideas presented in this article on computer-assisted diagnostic decision support systems (CDDSS) derive from prior work by the author and his colleagues (see list in Acknowledgments) on the INTERNIST-1 and QMR projects. References indicate the original sources of many of these ideas.
Collapse
Affiliation(s)
- Randolph A Miller
- Department of Biomedical Informatics, Eskind Biomedical Library, Vanderbilt University Medical Center, Room B003C, Nashville, TN 37232-8340, USA.
| |
Collapse
|
14
|
Croskerry P. A universal model of diagnostic reasoning. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2009; 84:1022-8. [PMID: 19638766 DOI: 10.1097/acm.0b013e3181ace703] [Citation(s) in RCA: 495] [Impact Index Per Article: 33.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/13/2023]
Abstract
Clinical judgment is a critical aspect of physician performance in medicine. It is essential in the formulation of a diagnosis and key to the effective and safe management of patients. Yet, the overall diagnostic error rate remains unacceptably high. In more than four decades of research, a variety of approaches have been taken, but a consensus approach toward diagnostic decision making has not emerged. In the last 20 years, important gains have been made in psychological research on human judgment. Dual-process theory has emerged as the predominant approach, positing two systems of decision making, System 1 (heuristic, intuitive) and System 2 (systematic, analytical). The author proposes a schematic model that uses the theory to develop a universal approach toward clinical decision making. Properties of the model explain many of the observed characteristics of physicians' performance. Yet the author cautions that not all medical reasoning and decision making falls neatly into one or the other of the model's systems, even though they provide a basic framework incorporating the recognized diverse approaches. He also emphasizes the complexity of decision making in actual clinical situations and the urgent need for more research to help clinicians gain additional insight and understanding regarding their decision making.
Collapse
Affiliation(s)
- Pat Croskerry
- Department of Emergency Medicine, Faculty of Medicine and Division of Medical Education, Dalhousie University, Halifax, Nova Scotia, Canada.
| |
Collapse
|
15
|
|
16
|
Kong G, Xu DL, Yang JB. Clinical Decision Support Systems: A Review on Knowledge Representation and Inference Under Uncertainties. INT J COMPUT INT SYS 2008. [DOI: 10.1080/18756891.2008.9727613] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022] Open
|
17
|
Grand challenges in clinical decision support. J Biomed Inform 2007; 41:387-92. [PMID: 18029232 DOI: 10.1016/j.jbi.2007.09.003] [Citation(s) in RCA: 322] [Impact Index Per Article: 18.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2007] [Revised: 09/10/2007] [Accepted: 09/11/2007] [Indexed: 02/08/2023]
Abstract
There is a pressing need for high-quality, effective means of designing, developing, presenting, implementing, evaluating, and maintaining all types of clinical decision support capabilities for clinicians, patients and consumers. Using an iterative, consensus-building process we identified a rank-ordered list of the top 10 grand challenges in clinical decision support. This list was created to educate and inspire researchers, developers, funders, and policy-makers. The list of challenges in order of importance that they be solved if patients and organizations are to begin realizing the fullest benefits possible of these systems consists of: improve the human-computer interface; disseminate best practices in CDS design, development, and implementation; summarize patient-level information; prioritize and filter recommendations to the user; create an architecture for sharing executable CDS modules and services; combine recommendations for patients with co-morbidities; prioritize CDS content development and implementation; create internet-accessible clinical decision support repositories; use freetext information to drive clinical decision support; mine large clinical databases to create new CDS. Identification of solutions to these challenges is critical if clinical decision support is to achieve its potential and improve the quality, safety and efficiency of healthcare.
Collapse
|
18
|
Ramnarayan P, Winrow A, Coren M, Nanduri V, Buchdahl R, Jacobs B, Fisher H, Taylor PM, Wyatt JC, Britto J. Diagnostic omission errors in acute paediatric practice: impact of a reminder system on decision-making. BMC Med Inform Decis Mak 2006; 6:37. [PMID: 17087835 PMCID: PMC1654143 DOI: 10.1186/1472-6947-6-37] [Citation(s) in RCA: 43] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2006] [Accepted: 11/06/2006] [Indexed: 01/10/2023] Open
Abstract
Background Diagnostic error is a significant problem in specialities characterised by diagnostic uncertainty such as primary care, emergency medicine and paediatrics. Despite wide-spread availability, computerised aids have not been shown to significantly improve diagnostic decision-making in a real world environment, mainly due to the need for prolonged system consultation. In this study performed in the clinical environment, we used a Web-based diagnostic reminder system that provided rapid advice with free text data entry to examine its impact on clinicians' decisions in an acute paediatric setting during assessments characterised by diagnostic uncertainty. Methods Junior doctors working over a 5-month period at four paediatric ambulatory units consulted the Web-based diagnostic aid when they felt the need for diagnostic assistance. Subjects recorded their clinical decisions for patients (differential diagnosis, test-ordering and treatment) before and after system consultation. An expert panel of four paediatric consultants independently suggested clinically significant decisions indicating an appropriate and 'safe' assessment. The primary outcome measure was change in the proportion of 'unsafe' workups by subjects during patient assessment. A more sensitive evaluation of impact was performed using specific validated quality scores. Adverse effects of consultation on decision-making, as well as the additional time spent on system use were examined. Results Subjects attempted to access the diagnostic aid on 595 occasions during the study period (8.6% of all medical assessments); subjects examined diagnostic advice only in 177 episodes (30%). Senior House Officers at hospitals with greater number of available computer workstations in the clinical area were most likely to consult the system, especially out of working hours. Diagnostic workups construed as 'unsafe' occurred in 47/104 cases (45.2%); this reduced to 32.7% following system consultation (McNemar test, p < 0.001). Subjects' mean 'unsafe' workups per case decreased from 0.49 to 0.32 (p < 0.001). System advice prompted the clinician to consider the 'correct' diagnosis (established at discharge) during initial assessment in 3/104 patients. Median usage time was 1 min 38 sec (IQR 50 sec – 3 min 21 sec). Despite a modest increase in the number of diagnostic possibilities entertained by the clinician, no adverse effects were demonstrable on patient management following system use. Numerous technical barriers prevented subjects from accessing the diagnostic aid in the majority of eligible patients in whom they sought diagnostic assistance. Conclusion We have shown that junior doctors used a Web-based diagnostic reminder system during acute paediatric assessments to significantly improve the quality of their diagnostic workup and reduce diagnostic omission errors. These benefits were achieved without any adverse effects on patient management following a quick consultation.
Collapse
Affiliation(s)
| | - Andrew Winrow
- Department of Paediatrics, Kingston General Hospital, Galsworthy Road, Kingston-upon-Thames, KT2 7QB, UK
| | - Michael Coren
- Department of Paediatrics, St Mary's Hospital, Paddington, London, W2 1NY, UK
| | - Vasanta Nanduri
- Department of Paediatrics, Watford General Hospital, Vicarage Road, Watford, WD18 0HB, UK
| | - Roger Buchdahl
- Department of Paediatrics, Hillingdon Hospital, Pield Heath Road, Middlesex, UB8 3NN, UK
| | - Benjamin Jacobs
- Department of Paediatrics, Northwick Park Hospital, Watford Road, Harrow, Middlesex, HA1 3UJ, UK
| | - Helen Fisher
- Isabel Healthcare Ltd, Po Box 244, Haslemere, Surrey, GU27 1WU, UK
| | - Paul M Taylor
- Centre for Health Informatics and Multiprofessional Education (CHIME), Archway Campus, Highgate Hill, London, N19 5LW, UK
| | - Jeremy C Wyatt
- Health Informatics Centre, The Mackenzie Building, University of Dundee, Dundee, DD2 4BF, UK
| | - Joseph Britto
- Isabel Healthcare Ltd, Po Box 244, Haslemere, Surrey, GU27 1WU, UK
| |
Collapse
|
19
|
Ramnarayan P, Roberts GC, Coren M, Nanduri V, Tomlinson A, Taylor PM, Wyatt JC, Britto JF. Assessment of the potential impact of a reminder system on the reduction of diagnostic errors: a quasi-experimental study. BMC Med Inform Decis Mak 2006; 6:22. [PMID: 16646956 PMCID: PMC1513379 DOI: 10.1186/1472-6947-6-22] [Citation(s) in RCA: 46] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2005] [Accepted: 04/28/2006] [Indexed: 11/17/2022] Open
Abstract
Background Computerized decision support systems (DSS) have mainly focused on improving clinicians' diagnostic accuracy in unusual and challenging cases. However, since diagnostic omission errors may predominantly result from incomplete workup in routine clinical practice, the provision of appropriate patient- and context-specific reminders may result in greater impact on patient safety. In this experimental study, a mix of easy and difficult simulated cases were used to assess the impact of a novel diagnostic reminder system (ISABEL) on the quality of clinical decisions made by various grades of clinicians during acute assessment. Methods Subjects of different grades (consultants, registrars, senior house officers and medical students), assessed a balanced set of 24 simulated cases on a trial website. Subjects recorded their clinical decisions for the cases (differential diagnosis, test-ordering and treatment), before and after system consultation. A panel of two pediatric consultants independently provided gold standard responses for each case, against which subjects' quality of decisions was measured. The primary outcome measure was change in the count of diagnostic errors of omission (DEO). A more sensitive assessment of the system's impact was achieved using specific quality scores; additional consultation time resulting from DSS use was also calculated. Results 76 subjects (18 consultants, 24 registrars, 19 senior house officers and 15 students) completed a total of 751 case episodes. The mean count of DEO fell from 5.5 to 5.0 across all subjects (repeated measures ANOVA, p < 0.001); no significant interaction was seen with subject grade. Mean diagnostic quality score increased after system consultation (0.044; 95% confidence interval 0.032, 0.054). ISABEL reminded subjects to consider at least one clinically important diagnosis in 1 in 8 case episodes, and prompted them to order an important test in 1 in 10 case episodes. Median extra time taken for DSS consultation was 1 min (IQR: 30 sec to 2 min). Conclusion The provision of patient- and context-specific reminders has the potential to reduce diagnostic omissions across all subject grades for a range of cases. This study suggests a promising role for the use of future reminder-based DSS in the reduction of diagnostic error.
Collapse
Affiliation(s)
| | - Graham C Roberts
- Department of Paediatric Allergy and Respiratory Medicine, Southampton University Hospital Trust, Tremona Road, Southampton, SO16 6YD, UK
| | - Michael Coren
- Department of Paediatrics, St Mary's Hospital, Paddington, London, W2 1NY, UK
| | - Vasantha Nanduri
- Department of Paediatrics, Watford General Hospital, Vicarage Road, Watford, WD18 0HB, UK
| | - Amanda Tomlinson
- Isabel Healthcare Ltd, Po Box 244, Haslemere, Surrey, GU27 1WU, UK
| | - Paul M Taylor
- Centre for Health Informatics and Multiprofessional Education (CHIME), Archway Campus, Highgate Hill, London, N19 5LW, UK
| | - Jeremy C Wyatt
- Health Informatics Centre, The Mackenzie Building, University of Dundee, Dundee, DD2 4BF, UK
| | - Joseph F Britto
- Isabel Healthcare Ltd, Po Box 244, Haslemere, Surrey, GU27 1WU, UK
| |
Collapse
|
20
|
|
21
|
Cauvin JM, Le Guillou C, Solaiman B, Robaszkiewicz M, Le Beux P, Roux C. Computer-assisted diagnosis system in digestive endoscopy. ACTA ACUST UNITED AC 2004; 7:256-62. [PMID: 15000352 DOI: 10.1109/titb.2003.823293] [Citation(s) in RCA: 21] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
The purpose of this paper is to present an intelligent atlas of indexed endoscopic lesions that could be used in computer-assisted diagnosis as reference data. The development of such a system requires a mix of medical and engineering skills for analyzing and reproducing the cognitive processes that underlie the medical decision-making process. The analysis of both endoscopists experience and endoscopic terminologies developed by professional associations shows that diagnostic reasoning in digestive endoscopy uses a scene-object approach. The objects correspond to the endoscopic findings and the medical context of examination and the scene to the endoscopic diagnosis. According to expert assessment, the classes of endoscopic findings and diagnoses, their primitive characteristics (or indices), and their relationships have been listed. Each class describes an endoscopic finding or diagnosis in an intensive way. The retrieval method is based on a similarity metric that estimates the membership value of the case under investigation and the prototype of the class. A simulation test with randomized objects demonstrates a good classification of endoscopic findings. The correct class is the unique response in 68% of the tested objects, the first of multiple responses in 28%. Four descriptors are shown to be of major importance in the classification algorithm: anatomic location, shape, color, and relief. At the present time, the application database contains approximately 150 endoscopic images and is accessible via Internet. Experiments are in progress with endoscopists for the validation of the system and for the understanding of the similarity between images. The next step will integrate the system in a learning tool for junior endoscopists.
Collapse
Affiliation(s)
- Jean-Michel Cauvin
- Département d'Information Médicale, Centre Hospitalier Universitaire, Brest 29609, France.
| | | | | | | | | | | |
Collapse
|
22
|
Affiliation(s)
- Kevin B Johnson
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN 37232, USA.
| | | |
Collapse
|
23
|
Leitich H, Adlassnig KP, Kolarz G. Evaluation of two different models of semi-automatic knowledge acquisition for the medical consultant system CADIAG-II/RHEUMA. Artif Intell Med 2002; 25:215-25. [PMID: 12069760 DOI: 10.1016/s0933-3657(02)00025-8] [Citation(s) in RCA: 15] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
As part of a plan to promote semi-automatic knowledge acquisition for the medical consultant system CADIAG-II/RHEUMA, this study sought to explore and cope with the variability of results that may be anticipated when performing knowledge acquisition with patient data from different patient settings. Patient data were drawn both from a published study for the classification of rheumatoid arthritis (RA) and from a large database of rheumatological patient charts developed for the CADIAG-II/RHEUMA system. An analysis of the relationships between RA and selected CADIAG-II/RHEUMA symptoms was done using two models. In one of them, we controlled for the differences in baseline frequencies of symptoms and diseases in the two study populations as an important factor influencing the results of the calculations. Other factors that were identified included inconsistent definitions of symptoms and diseases, and the different composition of study groups in the two study populations. By eliminating differences in baseline frequencies as the most important bias, the results obtained from the two different knowledge sources became more consistent. All remaining inconsistencies and uncertainties about the contribution and relative importance of the factors were formalized using fuzzy intervals.
Collapse
Affiliation(s)
- Harald Leitich
- Department of Medical Computer Sciences, Section of Medical Expert and Knowledge-Based Systems, University of Vienna Medical School, Spitalgasse 23, A-1090 Vienna, Austria.
| | | | | |
Collapse
|
24
|
Elstein AS, Schwartz A, Nendaz MR. Medical Decision Making. INTERNATIONAL HANDBOOK OF RESEARCH IN MEDICAL EDUCATION 2002. [DOI: 10.1007/978-94-010-0462-6_9] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/03/2023]
|
25
|
|
26
|
|
27
|
Westberg EE, Miller RA. The basis for using the Internet to support the information needs of primary care. J Am Med Inform Assoc 1999; 6:6-25. [PMID: 9925225 PMCID: PMC61341 DOI: 10.1136/jamia.1999.0060006] [Citation(s) in RCA: 53] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/1998] [Accepted: 09/22/1998] [Indexed: 11/03/2022] Open
Abstract
Synthesizing the state of the art from the published literature, this review assesses the basis for employing the Internet to support the information needs of primary care. The authors survey what has been published about the information needs of clinical practice, including primary care, and discuss currently available information resources potentially relevant to primary care. Potential methods of linking information needs with appropriate information resources are described in the context of previous classifications of clinical information needs. Also described is the role that existing terminology mapping systems, such as the National Library of Medicine's Unified Medical Language System, may play in representing and linking information needs to answers.
Collapse
Affiliation(s)
- E E Westberg
- Vanderbilt University, Nashville, Tennessee 37232-8340, USA.
| | | |
Collapse
|
28
|
Miller RA. A heuristic approach to the multiple diagnoses problem. Artif Intell Med 1997. [DOI: 10.1007/bfb0029451] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
29
|
|
30
|
Berner ES, Webster GD, Shugerman AA, Jackson JR, Algina J, Baker AL, Ball EV, Cobbs CG, Dennis VW, Frenkel EP. Performance of four computer-based diagnostic systems. N Engl J Med 1994; 330:1792-6. [PMID: 8190157 DOI: 10.1056/nejm199406233302506] [Citation(s) in RCA: 156] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/29/2023]
Abstract
BACKGROUND Computer-based diagnostic systems are available commercially, but there has been limited evaluation of their performance. We assessed the diagnostic capabilities of four internal medicine diagnostic systems: Dxplain, Iliad, Meditel, and QMR. METHODS Ten expert clinicians created a set of 105 diagnostically challenging clinical case summaries involving actual patients. Clinical data were entered into each program with the vocabulary provided by the program's developer. Each of the systems produced a ranked list of possible diagnoses for each patient, as did the group of experts. We calculated scores on several performance measures for each computer program. RESULTS No single computer program scored better than the others on all performance measures. Among all cases and all programs, the proportion of correct diagnoses ranged from 0.52 to 0.71, and the mean proportion of relevant diagnoses ranged from 0.19 to 0.37. On average, less than half the diagnoses on the experts' original list of reasonable diagnoses were suggested by any of the programs. However, each program suggested an average of approximately two additional diagnoses per case that the experts found relevant but had not originally considered. CONCLUSIONS The results provide a profile of the strengths and limitations of these computer programs. The programs should be used by physicians who can identify and use the relevant information and ignore the irrelevant information that can be produced.
Collapse
|
31
|
Abstract
A review of the literature regarding computer-assisted diagnosis of rheumatic diseases is presented. After a general outline of the history and goals of computer programs intended to support physicians in the diagnostic process, 14 systems or projects are described. The scope of seven of these is general internal medicine, and the other seven are intended exclusively for rheumatic problems. The majority of these systems are prototypes. To date, none of them is widely used by physicians. Preliminary evaluation studies and/or independent reviews have been reported for all of the systems. The need for further evaluation studies is recognized, and strategies to carry these out are outlined. Furthermore, the potential usefulness for patient care and education is discussed. It is concluded that a new and interesting field is being developed that deserves more attention among rheumatologists.
Collapse
Affiliation(s)
- H J Moens
- Department of Rheumatology, Jan van Breemen Institute, Amsterdam, The Netherlands
| | | |
Collapse
|
32
|
Heckerling PS, Elstein AS, Terzian CG, Kushner MS. The effect of incomplete knowledge on the diagnoses of a computer consultant system. MEDICAL INFORMATICS = MEDECINE ET INFORMATIQUE 1991; 16:363-70. [PMID: 1762472 DOI: 10.3109/14639239109067658] [Citation(s) in RCA: 13] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/28/2022]
Abstract
The knowledge bases (KBs) of diagnostic decision support systems are often incomplete, and gaps in the KB could potentially lead systems to reach diagnoses that are implausible to physicians. To investigate this possibility we studied Iliad (Version 2.01), a computer consultant system that generates differential diagnosis across the domain of internal medicine. Data from the history, physical examination, and laboratory findings of 50 grand-rounds cases were entered into Iliad by a computer consultant aware of the diagnosis but blinded to its presence or absence in Iliad's KB. Two experienced internists were asked to diagnose these cases before and after seeing the results of the computer consultation, and to assess the plausibility of the computer's diagnoses. Twenty-eight of the 50 cases (56.0%) were diseases contained in Iliad's KB. After seeing Iliad's diagnoses for cases in the KB, physicians assigned to their correct diagnoses a higher mean ranked position (1.5 versus 2.0, p less than 0.008) and a higher mean probability (84.0% versus 77.6%, p less than 0.008) compared with their pre-Iliad values, whereas for cases not in the KB, mean position and probability for correct diagnoses did not change. Physician diagnostic accuracy did not change after consultation on cases included or not included in the KB. After adjusting for case difficulty, mean plausibility of Iliad's diagnoses was judged significantly higher (on a seven-point scale) for cases in the KB than for cases not in the KB (4.2 versus 3.2, p less than 0.02).(ABSTRACT TRUNCATED AT 250 WORDS)
Collapse
Affiliation(s)
- P S Heckerling
- Department of Medical Education, University of Illinois, Chicago 60680
| | | | | | | |
Collapse
|
33
|
Fugleberg S, Greulich A, Stenver DI. Computer-assisted diagnosis of acute azotaemia: diagnostic strategy and diagnostic criteria. Comput Biol Med 1991; 21:399-406. [PMID: 1790684 DOI: 10.1016/0010-4825(91)90041-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2022]
Abstract
This report describes the diagnostic strategy, cost-effectiveness and diagnostic accuracy of a rule-based, backward chaining diagnostic expert system designed to assist in diagnosing the cause(s) of acute azotaemia. The diagnostic strategy of the expert system seems to be cost-effective compared with that of a renal unit and the diagnostic criteria of the expert system accurately imitate the diagnoses of the renal unit. It is suggested that the expert system is diagnostically reliable and may reduce cost in the diagnostic work-up of patients with acute azotaemia.
Collapse
Affiliation(s)
- S Fugleberg
- Department of Nephrology, Hvidovre Hospital, Denmark
| | | | | |
Collapse
|