1
|
Hassoon A, Ng C, Lehmann H, Rupani H, Peterson S, Horberg MA, Liberman AL, Sharp AL, Johansen MC, McDonald K, Austin JM, Newman-Toker DE. Computable phenotype for diagnostic error: developing the data schema for application of symptom-disease pair analysis of diagnostic error (SPADE). Diagnosis (Berl) 2024; 11:295-302. [PMID: 38696319 DOI: 10.1515/dx-2023-0138] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2023] [Accepted: 04/01/2024] [Indexed: 05/04/2024]
Abstract
OBJECTIVES Diagnostic errors are the leading cause of preventable harm in clinical practice. Implementable tools to quantify and target this problem are needed. To address this gap, we aimed to generalize the Symptom-Disease Pair Analysis of Diagnostic Error (SPADE) framework by developing its computable phenotype and then demonstrated how that schema could be applied in multiple clinical contexts. METHODS We created an information model for the SPADE processes, then mapped data fields from electronic health records (EHR) and claims data in use to that model to create the SPADE information model (intention) and the SPADE computable phenotype (extension). Later we validated the computable phenotype and tested it in four case studies in three different health systems to demonstrate its utility. RESULTS We mapped and tested the SPADE computable phenotype in three different sites using four different case studies. We showed that data fields to compute an SPADE base measure are fully available in the EHR Data Warehouse for extraction and can operationalize the SPADE framework from provider and/or insurer perspective, and they could be implemented on numerous health systems for future work in monitor misdiagnosis-related harms. CONCLUSIONS Data for the SPADE base measure is readily available in EHR and administrative claims. The method of data extraction is potentially universally applicable, and the data extracted is conveniently available within a network system. Further study is needed to validate the computable phenotype across different settings with different data infrastructures.
Collapse
Affiliation(s)
- Ahmed Hassoon
- Department of Epidemiology, 25802 Johns Hopkins University Bloomberg School of Public Health , Baltimore, MD, USA
| | | | - Harold Lehmann
- 1500 The Johns Hopkins University School of Medicine , Baltimore, MD, USA
| | - Hetal Rupani
- 1500 Johns Hopkins School of Medicine , Baltimore, MD, USA
| | - Susan Peterson
- Emergency Medicine, 1500 Johns Hopkins University School of Medicine , Baltimore, MD, USA
| | - Michael A Horberg
- Mid-Atlantic Permanente Medical Group, 51637 Mid-Atlantic Permanente Research Institute , Rockville, MD, USA
| | - Ava L Liberman
- Neurology, 12295 Weill Cornell Medicine , New York, NY, USA
| | - Adam L Sharp
- Department of Research & Evaluation, 82579 Kaiser Permanente Southern California , Pasadena, CA, USA
| | - Michelle C Johansen
- Department of Neurology and the Armstrong Institute Center for Diagnostic Excellence, 1500 Johns Hopkins University School of Medicine , Baltimore, MD, USA
| | - Kathy McDonald
- Johns Hopkins University School of Nursing 15851 , Baltimore, MD, USA
| | - J Mathrew Austin
- Department of Anesthesia and Critical Care Medicine and the Armstrong Institute Center for Diagnostic Excellence, 1500 Johns Hopkins University School of Medicine , Baltimore, MD, USA
| | | |
Collapse
|
2
|
Aldridge SJ, Schmidt AE, Thißen M, Bernal-Delgado E, Estupiñán-Romero F, González-Galindo J, Dolanski-Aghamanoukjan L, Mathis-Edenhofer S, Buble T, Križ K, Vuković J, Palmieri L, Unim B, Meulman I, Owen RK, Lyons RA. Has the COVID-19 pandemic changed existing patterns of non-COVID-19 health care utilization? A retrospective analysis of six regions in Europe. Eur J Public Health 2024; 34:i67-i73. [PMID: 38946449 DOI: 10.1093/eurpub/ckad180] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/02/2024] Open
Abstract
BACKGROUND Resilience of national health systems in Europe remains a major concern in times of multiple crises and as more evidence is emerging relating to the indirect effects of the COVID-19 pandemic on health care utilization (HCU), resulting from de-prioritization of regular, non-pandemic healthcare services. Most extant studies focus on regional, disease specific or early pandemic HCU creating difficulties in comparing across multiple countries. We provide a comparatively broad definition of HCU across multiple countries, with potential to expand across regions and timeframes. METHODS Using a cross-country federated research infrastructure (FRI), we examined HCU for acute cardiovascular events, elective surgeries and serious trauma. Aggregated data were used in forecast modelling to identify changes from predicted European age-standardized counts via fitted regressions (2017-19), compared against post-pandemic data. RESULTS We found that elective surgeries were most affected, universally falling below predicted levels in 2020. For cardiovascular HCU, we found lower-than-expected cases in every region for heart attacks and displayed large sex differences. Serious trauma was the least impacted by the COVID-19 pandemic. CONCLUSION The strength of this study comes from the use of the European Population Health Information Research Infrastructure's (PHIRI) FRI, allowing for rapid analysis of regional differences to assess indirect impacts of events such as pandemics. There are marked differences in the capacity of services to return to normal in terms of elective surgery; additionally, we found considerable differences between men and women which requires further research on potential sex or gender patterns of HCU during crises.
Collapse
Affiliation(s)
- Sarah J Aldridge
- Population Data Science, Swansea University Medical School, Faculty of Medicine, Health, and Life Science, Swansea University, Swansea, UK
| | - Andrea E Schmidt
- Competence Centre Climate and Health, GÖG (Austrian National Public Health Institute), Vienna, Austria
| | - Martin Thißen
- Department for Health Monitoring and Epidemiology, Robert Koch Institute, Berlin, Germany
| | - Enrique Bernal-Delgado
- Data Science for Health Services and Policy Research Group, Institute for Health Sciences in Aragon (IACS), Zaragoza, Spain
| | - Francisco Estupiñán-Romero
- Data Science for Health Services and Policy Research Group, Institute for Health Sciences in Aragon (IACS), Zaragoza, Spain
| | - Javier González-Galindo
- Data Science for Health Services and Policy Research Group, Institute for Health Sciences in Aragon (IACS), Zaragoza, Spain
| | - Lorenz Dolanski-Aghamanoukjan
- International Affairs, Policy, Evaluation, Digitalisation, GÖG (Austrian National Public Health Institute), Vienna, Austria
| | - Stefan Mathis-Edenhofer
- Health Care Planning and System Development, GÖG (Austrian National Public Health Institute), Vienna, Austria
| | - Tamara Buble
- Croatian Institute of Public Health (HZJZ), Zagreb, Croatia
| | - Klea Križ
- Croatian Institute of Public Health (HZJZ), Zagreb, Croatia
| | - Jakov Vuković
- Croatian Institute of Public Health (HZJZ), Zagreb, Croatia
| | - Luigi Palmieri
- Department of Cardiovascular, Endocrine-Metabolic Diseases and Aging, Istituto Superiore di Sanità (ISS), Rome, Italy
| | - Brigid Unim
- Department of Cardiovascular, Endocrine-Metabolic Diseases and Aging, Istituto Superiore di Sanità (ISS), Rome, Italy
| | - Iris Meulman
- Center for Public Health, Health Services and Society, National Institute for Public Health and the Environment, Bilthoven, The Netherlands
- Tranzo, Tilburg School of Social and Behavioural Sciences, Tilburg University, Tilburg, The Netherlands
| | - Rhiannon K Owen
- Population Data Science, Swansea University Medical School, Faculty of Medicine, Health, and Life Science, Swansea University, Swansea, UK
| | - Ronan A Lyons
- Population Data Science, Swansea University Medical School, Faculty of Medicine, Health, and Life Science, Swansea University, Swansea, UK
| |
Collapse
|
3
|
Harada Y, Sakamoto T, Sugimoto S, Shimizu T. Longitudinal Changes in Diagnostic Accuracy of a Differential Diagnosis List Developed by an AI-Based Symptom Checker: Retrospective Observational Study. JMIR Form Res 2024; 8:e53985. [PMID: 38758588 PMCID: PMC11143391 DOI: 10.2196/53985] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2023] [Revised: 03/23/2024] [Accepted: 04/24/2024] [Indexed: 05/18/2024] Open
Abstract
BACKGROUND Artificial intelligence (AI) symptom checker models should be trained using real-world patient data to improve their diagnostic accuracy. Given that AI-based symptom checkers are currently used in clinical practice, their performance should improve over time. However, longitudinal evaluations of the diagnostic accuracy of these symptom checkers are limited. OBJECTIVE This study aimed to assess the longitudinal changes in the accuracy of differential diagnosis lists created by an AI-based symptom checker used in the real world. METHODS This was a single-center, retrospective, observational study. Patients who visited an outpatient clinic without an appointment between May 1, 2019, and April 30, 2022, and who were admitted to a community hospital in Japan within 30 days of their index visit were considered eligible. We only included patients who underwent an AI-based symptom checkup at the index visit, and the diagnosis was finally confirmed during follow-up. Final diagnoses were categorized as common or uncommon, and all cases were categorized as typical or atypical. The primary outcome measure was the accuracy of the differential diagnosis list created by the AI-based symptom checker, defined as the final diagnosis in a list of 10 differential diagnoses created by the symptom checker. To assess the change in the symptom checker's diagnostic accuracy over 3 years, we used a chi-square test to compare the primary outcome over 3 periods: from May 1, 2019, to April 30, 2020 (first year); from May 1, 2020, to April 30, 2021 (second year); and from May 1, 2021, to April 30, 2022 (third year). RESULTS A total of 381 patients were included. Common diseases comprised 257 (67.5%) cases, and typical presentations were observed in 298 (78.2%) cases. Overall, the accuracy of the differential diagnosis list created by the AI-based symptom checker was 172 (45.1%), which did not differ across the 3 years (first year: 97/219, 44.3%; second year: 32/72, 44.4%; and third year: 43/90, 47.7%; P=.85). The accuracy of the differential diagnosis list created by the symptom checker was low in those with uncommon diseases (30/124, 24.2%) and atypical presentations (12/83, 14.5%). In the multivariate logistic regression model, common disease (P<.001; odds ratio 4.13, 95% CI 2.50-6.98) and typical presentation (P<.001; odds ratio 6.92, 95% CI 3.62-14.2) were significantly associated with the accuracy of the differential diagnosis list created by the symptom checker. CONCLUSIONS A 3-year longitudinal survey of the diagnostic accuracy of differential diagnosis lists developed by an AI-based symptom checker, which has been implemented in real-world clinical practice settings, showed no improvement over time. Uncommon diseases and atypical presentations were independently associated with a lower diagnostic accuracy. In the future, symptom checkers should be trained to recognize uncommon conditions.
Collapse
Affiliation(s)
- Yukinori Harada
- Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, Shimotsuga, Japan
- Department of General Medicine, Nagano Chuo Hospital, Nagano, Japan
| | - Tetsu Sakamoto
- Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, Shimotsuga, Japan
| | - Shu Sugimoto
- Department of Medicine (Neurology and Rheumatology), Shinshu University School of Medicine, Matsumoto, Japan
| | - Taro Shimizu
- Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, Shimotsuga, Japan
| |
Collapse
|
4
|
Herasevich S, Soleimani J, Huang C, Pinevich Y, Dong Y, Pickering BW, Murad MH, Barwise AK. Diagnostic error among vulnerable populations presenting to the emergency department with cardiovascular and cerebrovascular or neurological symptoms: a systematic review. BMJ Qual Saf 2023; 32:676-688. [PMID: 36972982 DOI: 10.1136/bmjqs-2022-015038] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2022] [Accepted: 03/10/2023] [Indexed: 03/29/2023]
Abstract
BACKGROUND Diagnostic error (DE) is a common problem in clinical practice, particularly in the emergency department (ED) setting. Among ED patients presenting with cardiovascular or cerebrovascular/neurological symptoms, a delay in diagnosis or failure to hospitalise may be most impactful in terms of adverse outcomes. Minorities and other vulnerable populations may be at higher risk of DE. We aimed to systematically review studies reporting the frequency and causes of DE in under-resourced patients presenting to the ED with cardiovascular or cerebrovascular/neurological symptoms. METHODS We searched EBM Reviews, Embase, Medline, Scopus and Web of Science from 2000 through 14 August 2022. Data were abstracted by two independent reviewers using a standardised form. The risk of bias (ROB) was assessed using the Newcastle-Ottawa Scale, and the certainty of evidence was evaluated using the Grading of Recommendations Assessment, Development, and Evaluation approach. RESULTS Of the 7342 studies screened, we included 20 studies evaluating 7436,737 patients. Most studies were conducted in the USA, and one study was multicountry. 11 studies evaluated DE in patients with cerebrovascular/neurological symptoms, 8 studies with cardiovascular symptoms and 1 study examined both types of symptoms. 13 studies investigated missed diagnoses and 7 studies explored delayed diagnoses. There was significant clinical and methodological variability, including heterogeneity of DE definitions and predictor variable definitions as well as methods of DE assessment, study design and reporting.Among the studies evaluating cardiovascular symptoms, black race was significantly associated with higher odds of DE in 4/6 studies evaluating missed acute myocardial infarction (AMI)/acute coronary syndrome (ACS) diagnosis compared with white race (OR from 1.18 (1.12-1.24) to 4.5 (1.8-11.8)). The association between other analysed factors (ethnicity, insurance and limited English proficiency) and DE in this domain varied from study to study and was inconclusive.Among the studies evaluating DE in patients with cerebrovascular/neurological symptoms, no consistent association was found indicating higher or lower odds of DE. Although some studies showed significant differences, these were not consistently in the same direction.The overall ROB was low for most included studies; however, the certainty of evidence was very low, mostly due to serious inconsistency in definitions and measurement approaches across studies. CONCLUSIONS This systematic review demonstrated consistent increased odds of missed AMI/ACS diagnosis among black patients presenting to the ED compared with white patients in most studies. No consistent associations between demographic groups and DE related to cerebrovascular/neurological diagnoses were identified. More standardised approaches to study design, measurement of DE and outcomes assessment are needed to understand this problem among vulnerable populations. TRIAL REGISTRATION NUMBER The study protocol was registered in the International Prospective Register of Systematic Reviews PROSPERO 2020 CRD42020178885 and is available from: https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42020178885.
Collapse
Affiliation(s)
- Svetlana Herasevich
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, Rochester, Minnesota, USA
| | - Jalal Soleimani
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, Rochester, Minnesota, USA
| | - Chanyan Huang
- Department of Anaesthesiology, Sun Yat-sen University First Affiliated Hospital, Guangzhou, Guangdong, China
| | - Yuliya Pinevich
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, Rochester, Minnesota, USA
| | - Yue Dong
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, Rochester, Minnesota, USA
| | - Brian W Pickering
- Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, Rochester, Minnesota, USA
| | - Mohammad H Murad
- Center for Science of Healthcare Delivery, Division of Preventive Medicine, Mayo Clinic, Rochester, Minnesota, USA
| | - Amelia K Barwise
- Division of Pulmonary and Critical Care Medicine, Mayo Clinic, Rochester, Minnesota, USA
- Bioethics Research Program, Mayo Clinic, Rochester, MN, USA
| |
Collapse
|
5
|
Liberman AL, Wang Z, Zhu Y, Hassoon A, Choi J, Austin JM, Johansen MC, Newman-Toker DE. Optimizing measurement of misdiagnosis-related harms using symptom-disease pair analysis of diagnostic error (SPADE): comparison groups to maximize SPADE validity. Diagnosis (Berl) 2023; 10:225-234. [PMID: 37018487 PMCID: PMC10659025 DOI: 10.1515/dx-2022-0130] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2022] [Accepted: 03/06/2023] [Indexed: 04/07/2023]
Abstract
Diagnostic errors in medicine represent a significant public health problem but continue to be challenging to measure accurately, reliably, and efficiently. The recently developed Symptom-Disease Pair Analysis of Diagnostic Error (SPADE) approach measures misdiagnosis related harms using electronic health records or administrative claims data. The approach is clinically valid, methodologically sound, statistically robust, and operationally viable without the requirement for manual chart review. This paper clarifies aspects of the SPADE analysis to assure that researchers apply this method to yield valid results with a particular emphasis on defining appropriate comparator groups and analytical strategies for balancing differences between these groups. We discuss four distinct types of comparators (intra-group and inter-group for both look-back and look-forward analyses), detailing the rationale for choosing one over the other and inferences that can be drawn from these comparative analyses. Our aim is that these additional analytical practices will improve the validity of SPADE and related approaches to quantify diagnostic error in medicine.
Collapse
Affiliation(s)
- Ava L. Liberman
- Clinical and Translational Neuroscience Unit, Feil Family Brain and Mind Research Institute and Department of Neurology, Weill Cornell Medicine
| | - Zheyu Wang
- The Johns Hopkins University School of Medicine, Sidney Kimmel Comprehensive Cancer Center, Division of Biostatistics and Bioinformatics
- The Johns Hopkins Bloomberg School of Public Health, Department of Biostatistics
| | - Yuxin Zhu
- The Johns Hopkins University School of Medicine, Sidney Kimmel Comprehensive Cancer Center, Division of Biostatistics and Bioinformatics
- The Johns Hopkins University School of Medicine, Department of Neurology and the Armstrong Institute Center for Diagnostic Excellence
| | - Ahmed Hassoon
- The Johns Hopkins Bloomberg School of Public Health, Department of Biostatistics
| | - Justin Choi
- Department of Internal Medicine, Weill Cornell Medicine
| | - J. Matthew Austin
- The Johns Hopkins University School of Medicine, Department of Anesthesiology and Critical Care Medicine and the Armstrong Institute Center for Diagnostic Excellence
| | - Michelle C. Johansen
- The Johns Hopkins University School of Medicine, Department of Neurology and the Armstrong Institute Center for Diagnostic Excellence
| | - David E. Newman-Toker
- The Johns Hopkins University School of Medicine, Department of Neurology and the Armstrong Institute Center for Diagnostic Excellence
- The Johns Hopkins Bloomberg School of Public Health, Departments of Epidemiology and Health Policy & Management
| |
Collapse
|
6
|
Miller AC, Cavanaugh JE, Arakkal AT, Koeneman SH, Polgreen PM. A comprehensive framework to estimate the frequency, duration, and risk factors for diagnostic delays using bootstrapping-based simulation methods. BMC Med Inform Decis Mak 2023; 23:68. [PMID: 37060037 PMCID: PMC10103428 DOI: 10.1186/s12911-023-02148-w] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2022] [Accepted: 03/15/2023] [Indexed: 04/16/2023] Open
Abstract
BACKGROUND The incidence of diagnostic delays is unknown for many diseases and specific healthcare settings. Many existing methods to identify diagnostic delays are resource intensive or difficult to apply to different diseases or settings. Administrative and other real-world data sources may offer the ability to better identify and study diagnostic delays for a range of diseases. METHODS We propose a comprehensive framework to estimate the frequency of missed diagnostic opportunities for a given disease using real-world longitudinal data sources. We provide a conceptual model of the disease-diagnostic, data-generating process. We then propose a bootstrapping method to estimate measures of the frequency of missed diagnostic opportunities and duration of delays. This approach identifies diagnostic opportunities based on signs and symptoms occurring prior to an initial diagnosis, while accounting for expected patterns of healthcare that may appear as coincidental symptoms. Three different bootstrapping algorithms are described along with estimation procedures to implement the resampling. Finally, we apply our approach to the diseases of tuberculosis, acute myocardial infarction, and stroke to estimate the frequency and duration of diagnostic delays for these diseases. RESULTS Using the IBM MarketScan Research databases from 2001 to 2017, we identified 2,073 cases of tuberculosis, 359,625 cases of AMI, and 367,768 cases of stroke. Depending on the simulation approach that was used, we estimated that 6.9-8.3% of patients with stroke, 16.0-21.3% of patients with AMI and 63.9-82.3% of patients with tuberculosis experienced a missed diagnostic opportunity. Similarly, we estimated that, on average, diagnostic delays lasted 6.7-7.6 days for stroke, 6.7-8.2 days for AMI, and 34.3-44.5 days for tuberculosis. Estimates for each of these measures was consistent with prior literature; however, specific estimates varied across the different simulation algorithms considered. CONCLUSIONS Our approach can be easily applied to study diagnostic delays using longitudinal administrative data sources. Moreover, this general approach can be customized to fit a range of diseases to account for specific clinical characteristics of a given disease. We summarize how the choice of simulation algorithm may impact the resulting estimates and provide guidance on the statistical considerations for applying our approach to future studies.
Collapse
Affiliation(s)
- Aaron C Miller
- Department of Internal Medicine, Roy J. and Lucille A. Carver College of Medicine, University of Iowa, Iowa City, IA, 52242, USA.
| | - Joseph E Cavanaugh
- Department of Biostatistics, College of Public Health, University of Iowa, Iowa City, IA, 52242, USA
| | - Alan T Arakkal
- Department of Biostatistics, College of Public Health, University of Iowa, Iowa City, IA, 52242, USA
| | - Scott H Koeneman
- Department of Biostatistics, College of Public Health, University of Iowa, Iowa City, IA, 52242, USA
| | - Philip M Polgreen
- Department of Internal Medicine, Roy J. and Lucille A. Carver College of Medicine, University of Iowa, Iowa City, IA, 52242, USA
| |
Collapse
|
7
|
Loten C, Nutt H, Bull N. Death rates following emergency department chest pain assessment. Emerg Med Australas 2023; 35:525-527. [PMID: 36843305 DOI: 10.1111/1742-6723.14186] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2022] [Revised: 02/09/2023] [Accepted: 02/09/2023] [Indexed: 02/28/2023]
Abstract
OBJECTIVES We sought to define the rate of unexpected death from acute coronary syndrome or arrhythmia in chest pain patients directly discharged from the ED. METHODS Retrospective audit of all chest pain patients at a tertiary ED for 7 years. Medical and post-mortem records of the deceased were reviewed with independent cardiologist adjudication to determine outcomes. Primary outcome measure was 28-day unexpected death secondary to acute coronary syndrome or arrhythmia. RESULTS During the study period, 25 924 patients presented with chest pain, 292 (1.1%, 95% confidence interval [CI] 0.99-1.01%) died within 28 days. Of these, 16 680(64%, 95% CI 63.88-64.12%) were discharged by ED, two (0.01%, 95% CI 0-0.011%) of this group died from the primary outcome. CONCLUSION Unexpected death is very uncommon after ED discharge of chest pain patients.
Collapse
Affiliation(s)
- Conrad Loten
- Department of Emergency Medicine, John Hunter Hospital, Hunter New England Health, Newcastle, New South Wales, Australia
| | - Hannah Nutt
- Department of Emergency Medicine, John Hunter Hospital, Hunter New England Health, Newcastle, New South Wales, Australia
| | - Neva Bull
- School of Psychology, The University of Newcastle, Newcastle, New South Wales, Australia
| |
Collapse
|
8
|
Miller AC, Arakkal AT, Koeneman SH, Cavanaugh JE, Polgreen PM. A clinically-guided unsupervised clustering approach to recommend symptoms of disease associated with diagnostic opportunities. Diagnosis (Berl) 2023; 10:43-53. [PMID: 36127310 PMCID: PMC9934811 DOI: 10.1515/dx-2022-0044] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2022] [Accepted: 08/26/2022] [Indexed: 11/15/2022]
Abstract
OBJECTIVES A first step in studying diagnostic delays is to select the signs, symptoms and alternative diseases that represent missed diagnostic opportunities. Because this step is labor intensive requiring exhaustive literature reviews, we developed machine learning approaches to mine administrative data sources and recommend conditions for consideration. We propose a methodological approach to find diagnostic codes that exhibit known patterns of diagnostic delays and apply this to the diseases of tuberculosis and appendicitis. METHODS We used the IBM MarketScan Research Databases, and consider the initial symptoms of cough before tuberculosis and abdominal pain before appendicitis. We analyze diagnosis codes during healthcare visits before the index diagnosis, and use k-means clustering to recommend conditions that exhibit similar trends to the initial symptoms provided. We evaluate the clinical plausibility of the recommended conditions and the corresponding number of possible diagnostic delays based on these diseases. RESULTS For both diseases of interest, the clustering approach suggested a large number of clinically-plausible conditions to consider (e.g., fever, hemoptysis, and pneumonia before tuberculosis). The recommended conditions had a high degree of precision in terms of clinical plausibility: >70% for tuberculosis and >90% for appendicitis. Including these additional clinically-plausible conditions resulted in more than twice the number of possible diagnostic delays identified. CONCLUSIONS Our approach can mine administrative datasets to detect patterns of diagnostic delay and help investigators avoid under-identifying potential missed diagnostic opportunities. In addition, the methods we describe can be used to discover less-common presentations of diseases that are frequently misdiagnosed.
Collapse
Affiliation(s)
- Aaron C Miller
- Department of Internal Medicine, Carver College of Medicine, University of Iowa, Iowa City, IA, USA
| | - Alan T Arakkal
- Department of Biostatistics, College of Public Health, University of Iowa, Iowa City, IA, USA
| | - Scott H Koeneman
- Department of Biostatistics, College of Public Health, University of Iowa, Iowa City, IA, USA
| | - Joseph E Cavanaugh
- Department of Biostatistics, College of Public Health, University of Iowa, Iowa City, IA, USA
| | - Philip M Polgreen
- Department of Internal Medicine, Carver College of Medicine, University of Iowa, Iowa City, IA, USA
| |
Collapse
|
9
|
Development and validation of deep learning ECG-based prediction of myocardial infarction in emergency department patients. Sci Rep 2022; 12:19615. [PMID: 36380048 PMCID: PMC9666471 DOI: 10.1038/s41598-022-24254-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2022] [Accepted: 11/11/2022] [Indexed: 11/16/2022] Open
Abstract
Myocardial infarction diagnosis is a common challenge in the emergency department. In managed settings, deep learning-based models and especially convolutional deep models have shown promise in electrocardiogram (ECG) classification, but there is a lack of high-performing models for the diagnosis of myocardial infarction in real-world scenarios. We aimed to train and validate a deep learning model using ECGs to predict myocardial infarction in real-world emergency department patients. We studied emergency department patients in the Stockholm region between 2007 and 2016 that had an ECG obtained because of their presenting complaint. We developed a deep neural network based on convolutional layers similar to a residual network. Inputs to the model were ECG tracing, age, and sex; and outputs were the probabilities of three mutually exclusive classes: non-ST-elevation myocardial infarction (NSTEMI), ST-elevation myocardial infarction (STEMI), and control status, as registered in the SWEDEHEART and other registries. We used an ensemble of five models. Among 492,226 ECGs in 214,250 patients, 5,416 were recorded with an NSTEMI, 1,818 a STEMI, and 485,207 without a myocardial infarction. In a random test set, our model could discriminate STEMIs/NSTEMIs from controls with a C-statistic of 0.991/0.832 and had a Brier score of 0.001/0.008. The model obtained a similar performance in a temporally separated test set of the study sample, and achieved a C-statistic of 0.985 and a Brier score of 0.002 in discriminating STEMIs from controls in an external test set. We developed and validated a deep learning model with excellent performance in discriminating between control, STEMI, and NSTEMI on the presenting ECG of a real-world sample of the important population of all-comers to the emergency department. Hence, deep learning models for ECG decision support could be valuable in the emergency department.
Collapse
|
10
|
Sharp AL, Pallegadda R, Baecker A, Park S, Nassery N, Hassoon A, Peterson S, Pitts SI, Wang Z, Zhu Y, Newman-Toker DE. Are Mental Health and Substance Use Disorders Risk Factors for Missed Acute Myocardial Infarction Diagnoses Among Chest Pain or Dyspnea Encounters in the Emergency Department? Ann Emerg Med 2021; 79:93-101. [PMID: 34607739 DOI: 10.1016/j.annemergmed.2021.08.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2021] [Revised: 08/14/2021] [Accepted: 08/23/2021] [Indexed: 11/24/2022]
Abstract
STUDY OBJECTIVE To assess if having a mental health and/or substance use disorder is associated with a missed acute myocardial infarction diagnosis in the emergency department (ED). METHODS This was a retrospective cohort analysis (2009 to 2017) of adult ED encounters at Kaiser Permanente Southern California. We used the validated symptom-disease pair analysis of diagnostic error methodological approach to "look back" and "look forward" and identify missed acute myocardial infarctions within 30 days of a treat-and-release ED visit. We use adjusted logistic regression to report the odds of missed acute myocardial infarction among patients with a history of mental health and/or substance use disorders. RESULTS The look-back analysis identified 44,473 acute myocardial infarction hospital encounters; 574 (1.3%) diagnoses were missed. The odds of missed diagnoses were higher in patients with mental health disorders (odds ratio [OR] 1.48, 95% confidence interval [CI] 1.23 to 1.77) but not in those with substance abuse disorders (OR 1.22, 95% CI 0.91 to 1.62). The highest risk was observed in those with co-occurring disorders (OR 1.90, 95% CI 1.30 to 2.76). The look-forward analysis identified 325,088 chest pain/dyspnea ED encounters; 508 (0.2%) were missed acute myocardial infarctions. No significant associations of missed acute myocardial infarction were revealed in either group (mental health disorder: OR 0.92, 95% CI 0.71 to 1.18; substance use disorder: OR 1.22, 95% CI 0.80 to 1.85). CONCLUSION The look-back analysis identified patients with mental illness at increased risk of missed acute myocardial infarction diagnosis, with the highest risk observed in those with a history of comorbid substance abuse. Having substance use disorders alone did not increase this risk in either cohort. The look-forward analysis revealed challenges in prospectively identifying high-risk patients to target for improvement.
Collapse
Affiliation(s)
- Adam L Sharp
- Department of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, CA; Department of Health System Science, Kaiser Permanente Bernard J. Tyson School of Medicine, Pasadena, CA; Department of Emergency Medicine, Kaiser Permanente Southern California, Los Angeles Medical Center, Los Angeles, CA; Department of Clinical Science, Kaiser Permanente Bernard J. Tyson School of Medicine, Pasadena, CA.
| | - Rani Pallegadda
- Department of Emergency Medicine, Kaiser Permanente Southern California, Los Angeles Medical Center, Los Angeles, CA; Department of Clinical Science, Kaiser Permanente Bernard J. Tyson School of Medicine, Pasadena, CA
| | - Aileen Baecker
- Department of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, CA
| | - Stacy Park
- Department of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, CA
| | - Najlla Nassery
- Department of Internal Medicine, Division of General Internal Medicine, Johns Hopkins University School of Medicine, Baltimore, MD
| | - Ahmed Hassoon
- Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD
| | - Susan Peterson
- Department of Emergency Medicine, Johns Hopkins University School of Medicine, Baltimore, MD
| | - Samantha I Pitts
- Department of Internal Medicine, Division of General Internal Medicine, Johns Hopkins University School of Medicine, Baltimore, MD
| | - Zheyu Wang
- Division of Biostatistics and Bioinformatics, Sidney Kimmel Comprehensive Cancer Center, Johns Hopkins University School of Medicine, Baltimore, MD; Department of Biostatistics, Johns Hopkins University, Baltimore, MD
| | - Yuxin Zhu
- Division of Biostatistics and Bioinformatics, Sidney Kimmel Comprehensive Cancer Center, Johns Hopkins University School of Medicine, Baltimore, MD; Department of Biostatistics, Johns Hopkins University, Baltimore, MD
| | - David E Newman-Toker
- Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD; Department of Neurology, Johns Hopkins University School of Medicine, Baltimore, MD
| |
Collapse
|
11
|
Horberg MA, Nassery N, Rubenstein KB, Certa JM, Shamim EA, Rothman R, Wang Z, Hassoon A, Townsend JL, Galiatsatos P, Pitts SI, Newman-Toker DE. Rate of sepsis hospitalizations after misdiagnosis in adult emergency department patients: a look-forward analysis with administrative claims data using Symptom-Disease Pair Analysis of Diagnostic Error (SPADE) methodology in an integrated health system. ACTA ACUST UNITED AC 2021; 8:479-488. [PMID: 33894108 DOI: 10.1515/dx-2020-0145] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2020] [Accepted: 03/16/2021] [Indexed: 01/08/2023]
Abstract
OBJECTIVES Delays in sepsis diagnosis can increase morbidity and mortality. Previously, we performed a Symptom-Disease Pair Analysis of Diagnostic Error (SPADE) "look-back" analysis to identify symptoms at risk for delayed sepsis diagnosis. We found treat-and-release emergency department (ED) encounters for fluid and electrolyte disorders (FED) and altered mental status (AMS) were associated with downstream sepsis hospitalizations. In this "look-forward" analysis, we measure the potential misdiagnosis-related harm rate for sepsis among patients with these symptoms. METHODS Retrospective cohort study using electronic health record and claims data from Kaiser Permanente Mid-Atlantic States (2013-2018). Patients ≥18 years with ≥1 treat-and-release ED encounter for FED or AMS were included. Observed greater than expected sepsis hospitalizations within 30 days of ED treat-and-release encounters were considered potential misdiagnosis-related harms. Temporal analyses were employed to differentiate case and comparison (superficial injury/contusion ED encounters) cohorts. RESULTS There were 4,549 treat-and-release ED encounters for FED or AMS, 26 associated with a sepsis hospitalization in the next 30 days. The observed (0.57%) minus expected (0.13%) harm rate was 0.44% (absolute) and 4.5-fold increased over expected (relative). There was a spike in sepsis hospitalizations in the week following FED/AMS ED visits. There were fewer sepsis hospitalizations and no spike in admissions in the week following superficial injury/contusion ED visits. Potentially misdiagnosed patients were older and more medically complex. CONCLUSIONS Potential misdiagnosis-related harms from sepsis are infrequent but measurable using SPADE. This look-forward analysis validated our previous look-back study, demonstrating the SPADE approach can be used to study infectious disease syndromes.
Collapse
Affiliation(s)
- Michael A Horberg
- Mid-Atlantic Permanente Medical Group, Mid-Atlantic Permanente Research Institute, Kaiser Permanente Mid-Atlantic States, Rockville, MD, USA.,Mid-Atlantic Permanente Medical Group, Department of Infectious Diseases, Kaiser Permanente Mid-Atlantic States, Rockville, MD, USA
| | - Najlla Nassery
- Division of General Internal Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, USA.,Center for Diagnostic Excellence, Johns Hopkins Medicine, Armstrong Institute for Patient Safety and Quality, Baltimore, MD, USA
| | - Kevin B Rubenstein
- Mid-Atlantic Permanente Medical Group, Mid-Atlantic Permanente Research Institute, Kaiser Permanente Mid-Atlantic States, Rockville, MD, USA
| | - Julia M Certa
- Mid-Atlantic Permanente Medical Group, Mid-Atlantic Permanente Research Institute, Kaiser Permanente Mid-Atlantic States, Rockville, MD, USA
| | - Ejaz A Shamim
- Mid-Atlantic Permanente Medical Group, Mid-Atlantic Permanente Research Institute, Kaiser Permanente Mid-Atlantic States, Rockville, MD, USA.,Mid-Atlantic Permanente Medical Group, Department of Neurology, Kaiser Permanente Mid-Atlantic States, Rockville, MD, USA
| | - Richard Rothman
- Department of Emergency Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Zheyu Wang
- Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA
| | - Ahmed Hassoon
- Center for Diagnostic Excellence, Johns Hopkins Medicine, Armstrong Institute for Patient Safety and Quality, Baltimore, MD, USA.,Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.,Department of Neurology, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Jennifer L Townsend
- Division of Infectious Disease, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Panagis Galiatsatos
- Division of Pulmonary and Critical Care Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Samantha I Pitts
- Division of General Internal Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - David E Newman-Toker
- Center for Diagnostic Excellence, Johns Hopkins Medicine, Armstrong Institute for Patient Safety and Quality, Baltimore, MD, USA.,Department of Emergency Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, USA.,Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.,Department of Neurology, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| |
Collapse
|
12
|
Nassery N, Horberg MA, Rubenstein KB, Certa JM, Watson E, Somasundaram B, Shamim E, Townsend JL, Galiatsatos P, Pitts SI, Hassoon A, Newman-Toker DE. Antecedent treat-and-release diagnoses prior to sepsis hospitalization among adult emergency department patients: a look-back analysis employing insurance claims data using Symptom-Disease Pair Analysis of Diagnostic Error (SPADE) methodology. ACTA ACUST UNITED AC 2021; 8:469-478. [PMID: 33650389 DOI: 10.1515/dx-2020-0140] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/22/2020] [Accepted: 02/01/2021] [Indexed: 12/21/2022]
Abstract
OBJECTIVES The aim of this study was to identify delays in early pre-sepsis diagnosis in emergency departments (ED) using the Symptom-Disease Pair Analysis of Diagnostic Error (SPADE) approach. METHODS SPADE methodology was employed using electronic health record and claims data from Kaiser Permanente Mid-Atlantic States (KPMAS). Study cohort included KPMAS members ≥18 years with ≥1 sepsis hospitalization 1/1/2013-12/31/2018. A look-back analysis identified treat-and-release ED visits in the month prior to sepsis hospitalizations. Top 20 diagnoses associated with these ED visits were identified; two diagnosis categories were distinguished as being linked to downstream sepsis hospitalizations. Observed-to-expected (O:E) and temporal analyses were performed to validate the symptom selection; results were contrasted to a comparison group. Demographics of patients that did and did not experience sepsis misdiagnosis were compared. RESULTS There were 3,468 sepsis hospitalizations during the study period and 766 treat-and-release ED visits in the month prior to hospitalization. Patients discharged from the ED with fluid and electrolyte disorders (FED) and altered mental status (AMS) were most likely to have downstream sepsis hospitalizations (O:E ratios of 2.66 and 2.82, respectively). Temporal analyses revealed that these symptoms were overrepresented and temporally clustered close to the hospitalization date. Approximately 2% of sepsis hospitalizations were associated with prior FED or AMS ED visits. CONCLUSIONS Treat-and-release ED encounters for FED and AMS may represent harbingers for downstream sepsis hospitalizations. The SPADE approach can be used to develop performance measures that identify pre-sepsis.
Collapse
Affiliation(s)
- Najlla Nassery
- Division of General Internal Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, USA
- Center for Diagnostic Excellence, Armstrong Institute for Patient Safety and Quality, Johns Hopkins Medicine, Baltimore, MD, USA
| | - Michael A Horberg
- Mid-Atlantic Permanente Medical Group, Mid-Atlantic Permanente Research Institute, Kaiser Permanente Mid-Atlantic States, Rockville, MD, USA
- Mid-Atlantic Permanente Medical Group, Department of Infectious Diseases, Kaiser Permanente Mid-Atlantic States, Rockville, MD, USA
| | - Kevin B Rubenstein
- Mid-Atlantic Permanente Medical Group, Mid-Atlantic Permanente Research Institute, Kaiser Permanente Mid-Atlantic States, Rockville, MD, USA
| | - Julia M Certa
- Mid-Atlantic Permanente Medical Group, Mid-Atlantic Permanente Research Institute, Kaiser Permanente Mid-Atlantic States, Rockville, MD, USA
| | - Eric Watson
- Mid-Atlantic Permanente Medical Group, Mid-Atlantic Permanente Research Institute, Kaiser Permanente Mid-Atlantic States, Rockville, MD, USA
| | - Brinda Somasundaram
- Mid-Atlantic Permanente Medical Group, Mid-Atlantic Permanente Research Institute, Kaiser Permanente Mid-Atlantic States, Rockville, MD, USA
| | - Ejaz Shamim
- Mid-Atlantic Permanente Medical Group, Mid-Atlantic Permanente Research Institute, Kaiser Permanente Mid-Atlantic States, Rockville, MD, USA
- Mid-Atlantic Permanente Medical Group, Department of Neurology, Kaiser Permanente Mid-Atlantic States, Rockville, MD, USA
| | - Jennifer L Townsend
- Division of Infectious Disease, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Panagis Galiatsatos
- Division of Pulmonary and Critical Care Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Samantha I Pitts
- Division of General Internal Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Ahmed Hassoon
- Center for Diagnostic Excellence, Armstrong Institute for Patient Safety and Quality, Johns Hopkins Medicine, Baltimore, MD, USA
- Department of Neurology, Johns Hopkins University School of Medicine, Baltimore, MD, USA
- Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA
| | - David E Newman-Toker
- Center for Diagnostic Excellence, Armstrong Institute for Patient Safety and Quality, Johns Hopkins Medicine, Baltimore, MD, USA
- Department of Neurology, Johns Hopkins University School of Medicine, Baltimore, MD, USA
- Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA
- Department of Emergency Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| |
Collapse
|