1
|
Rajan SS, Sarvepalli S, Wei L, Meyer AND, Murphy DR, Choi DT, Singh H. Medical Home Implementation and Follow-Up of Cancer-Related Abnormal Test Results in the Veterans Health Administration. JAMA Netw Open 2024; 7:e240087. [PMID: 38483392 PMCID: PMC10940951 DOI: 10.1001/jamanetworkopen.2024.0087] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/26/2023] [Accepted: 12/18/2023] [Indexed: 03/17/2024] Open
Abstract
Importance Lack of timely follow-up of cancer-related abnormal test results can lead to delayed or missed diagnoses, adverse cancer outcomes, and substantial cost burden for patients. Care delivery models, such as the Veterans Affairs' (VA) Patient-Aligned Care Team (PACT), which aim to improve patient-centered care coordination, could potentially also improve timely follow-up of abnormal test results. PACT was implemented nationally in the VA between 2010 and 2012. Objective To evaluate the long-term association between PACT implementation and timely follow-up of abnormal test results related to the diagnosis of 5 different cancers. Design, Setting, and Participants This multiyear retrospective cohort study used 14 years of VA data (2006-2019), which were analyzed using panel data-based random-effects linear regressions. The setting included all VA clinics and facilities. The participants were adult patients who underwent diagnostic testing related to 5 different cancers and had abnormal test results. Data extraction and statistical analyses were performed from September 2021 to December 2023. Exposure Calendar years denoting preperiods and postperiods of PACT implementation, and the PACT Implementation Progress Index Score denoting the extent of implementation in each VA clinic and facility. Main Outcome and Measure Percentage of potentially missed timely follow-ups of abnormal test results. Results This study analyzed 6 data sets representing 5 different types of cancers. During the initial years of PACT implementation (2010 to 2013), percentage of potentially missed timely follow-ups decreased between 3 to 7 percentage points for urinalysis suggestive of bladder cancer, 12 to 14 percentage points for mammograms suggestive of breast cancer, 19 to 22 percentage points for fecal tests suggestive of colorectal cancer, and 6 to 13 percentage points for iron deficiency anemia laboratory tests suggestive of colorectal cancer, with no statistically significant changes for α-fetoprotien tests and lung cancer imaging. However, these beneficial reductions were not sustained over time. Better PACT implementation scores were associated with a decrease in potentially missed timely follow-up percentages for urinalysis (0.3-percentage point reduction [95% CI, -0.6 to -0.1] with 1-point increase in the score), and laboratory tests suggestive of iron deficiency anemia (0.5-percentage point reduction [95% CI,-0.8 to -0.2] with 1-point increase in the score). Conclusions and Relevance This cohort study found that implementation of PACT in the VA was associated with a potential short-term improvement in the quality of follow-up for certain test results. Additional multifaceted sustained interventions to reduce missed test results are required to prevent care delays.
Collapse
Affiliation(s)
- Suja S. Rajan
- Department of Management, Policy & Community Health, School of Public Health, The University of Texas Health Science Center at Houston
| | | | - Li Wei
- Center for Innovations in Quality, Effectiveness and Safety (IQuESt), Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas
| | - Ashley N. D. Meyer
- Department of Medicine, Baylor College of Medicine, Houston, Texas
- Center for Innovations in Quality, Effectiveness and Safety (IQuESt), Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas
| | - Daniel R. Murphy
- Department of Medicine, Baylor College of Medicine, Houston, Texas
- Center for Innovations in Quality, Effectiveness and Safety (IQuESt), Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas
| | - Debra T. Choi
- Center for Innovations in Quality, Effectiveness and Safety (IQuESt), Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas
| | - Hardeep Singh
- Department of Medicine, Baylor College of Medicine, Houston, Texas
- Center for Innovations in Quality, Effectiveness and Safety (IQuESt), Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas
| |
Collapse
|
2
|
Friedemann Smith C, Duncombe S, Fleming S, Hirst Y, Black GB, Bankhead C, Nicholson BD. Electronic safety-netting tool features considered important by UK general practice staff: an interview and Delphi consensus study. BJGP Open 2023; 7:BJGPO.2022.0163. [PMID: 37277171 PMCID: PMC10646209 DOI: 10.3399/bjgpo.2022.0163] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2022] [Revised: 03/16/2023] [Accepted: 04/03/2023] [Indexed: 06/07/2023] Open
Abstract
BACKGROUND The potential of the electronic health record to support safety netting has been recognised and a number of electronic safety-netting (E-SN) tools developed. AIM To establish the most important features of E-SN tools. DESIGN & SETTING User-experience interviews followed by a Delphi study in a primary care setting in the UK. METHOD The user-experience interviews were carried out remotely with primary care staff who had trialled the EMIS E-SN toolkit for suspected cancer. An electronic modified Delphi approach was used, with primary care staff involved in safety netting in any capacity, to measure consensus on tool features. RESULTS Thirteen user-experience interviews were carried out and features of E-SN tools seen as important formed the majority of the features included in the Delphi study. Three rounds of Delphi survey were administered. Sixteen responders (64%) completed all three rounds, and 28 out of 44 (64%) features reached consensus. Primary care staff preferred tools that were general in scope. CONCLUSION Primary care staff indicated that tools that were not specific to cancer or any other disease, and had features that promoted their flexible, efficient, and integrated use, were important. However, when the important features were discussed with the patient and public involvement (PPI) group, they expressed disappointment that features they believed would make E-SN tools robust and provide a safety net that is difficult to fall through did not reach consensus. The successful adoption of E-SN tools will rely on an evidence base of their effectiveness. Efforts should be made to assess the impact of these tools on patient outcomes.
Collapse
Affiliation(s)
| | | | - Susannah Fleming
- Nuffield Department of Primary Care Sciences, University of Oxford, Oxford, UK
| | - Yasemin Hirst
- Institute of Epidemiology & Health, University College London, Lancaster Medical School, Lancaster University, Lancaster, UK
| | - Georgia Bell Black
- Wolfson Institute of population Health, Queen Mary's University, London, UK
| | - Clare Bankhead
- Nuffield Department of Primary Care Sciences, University of Oxford, Oxford, UK
| | - Brian D Nicholson
- Nuffield Department of Primary Care Sciences, University of Oxford, Oxford, UK
| |
Collapse
|
3
|
Murphy DR, Zimolzak AJ, Upadhyay DK, Wei L, Jolly P, Offner A, Sittig DF, Korukonda S, Rekha RM, Singh H. Developing electronic clinical quality measures to assess the cancer diagnostic process. J Am Med Inform Assoc 2023; 30:1526-1531. [PMID: 37257883 PMCID: PMC10436145 DOI: 10.1093/jamia/ocad089] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2022] [Revised: 04/12/2023] [Accepted: 05/08/2023] [Indexed: 06/02/2023] Open
Abstract
OBJECTIVE Measures of diagnostic performance in cancer are underdeveloped. Electronic clinical quality measures (eCQMs) to assess quality of cancer diagnosis could help quantify and improve diagnostic performance. MATERIALS AND METHODS We developed 2 eCQMs to assess diagnostic evaluation of red-flag clinical findings for colorectal (CRC; based on abnormal stool-based cancer screening tests or labs suggestive of iron deficiency anemia) and lung (abnormal chest imaging) cancer. The 2 eCQMs quantified rates of red-flag follow-up in CRC and lung cancer using electronic health record data repositories at 2 large healthcare systems. Each measure used clinical data to identify abnormal results, evidence of appropriate follow-up, and exclusions that signified follow-up was unnecessary. Clinicians reviewed 100 positive and 20 negative randomly selected records for each eCQM at each site to validate accuracy and categorized missed opportunities related to system, provider, or patient factors. RESULTS We implemented the CRC eCQM at both sites, while the lung cancer eCQM was only implemented at the VA due to lack of structured data indicating level of cancer suspicion on most chest imaging results at Geisinger. For the CRC eCQM, the rate of appropriate follow-up was 36.0% (26 746/74 314 patients) in the VA after removing clinical exclusions and 41.1% at Geisinger (1009/2461 patients; P < .001). Similarly, the rate of appropriate evaluation for lung cancer in the VA was 61.5% (25 166/40 924 patients). Reviewers most frequently attributed missed opportunities at both sites to provider factors (84 of 157). CONCLUSIONS We implemented 2 eCQMs to evaluate the diagnostic process in cancer at 2 large health systems. Health care organizations can use these eCQMs to monitor diagnostic performance related to cancer.
Collapse
Affiliation(s)
- Daniel R Murphy
- Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas, USA
- Department of Medicine, Baylor College of Medicine, Houston, Texas, USA
| | - Andrew J Zimolzak
- Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas, USA
- Department of Medicine, Baylor College of Medicine, Houston, Texas, USA
| | - Divvy K Upadhyay
- Division of Quality, Safety and Patient Experience, Geisinger, Danville, Pennsylvania, USA
| | - Li Wei
- Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas, USA
- Department of Medicine, Baylor College of Medicine, Houston, Texas, USA
| | - Preeti Jolly
- Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas, USA
- Department of Medicine, Baylor College of Medicine, Houston, Texas, USA
| | - Alexis Offner
- Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas, USA
- Department of Medicine, Baylor College of Medicine, Houston, Texas, USA
| | - Dean F Sittig
- Department of Clinical and Health Informatics, The University of Texas Health Science Center at Houston’s School of Biomedical Informatics, Houston, Texas, USA
- The UT-Memorial Hermann Center for Healthcare Quality & Safety, Houston, Texas, USA
| | - Saritha Korukonda
- Investigator-Initiated Research Operations, Geisinger, Danville, Pennsylvania, USA
| | - Riyaa Murugaesh Rekha
- Division of Quality, Safety and Patient Experience, Geisinger, Danville, Pennsylvania, USA
| | - Hardeep Singh
- Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas, USA
- Department of Medicine, Baylor College of Medicine, Houston, Texas, USA
| |
Collapse
|
4
|
Giardina TD, Hunte H, Hill MA, Heimlich SL, Singh H, Smith KM. Defining Diagnostic Error: A Scoping Review to Assess the Impact of the National Academies' Report Improving Diagnosis in Health Care. J Patient Saf 2022; 18:770-778. [PMID: 35405723 PMCID: PMC9698189 DOI: 10.1097/pts.0000000000000999] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
Abstract
BACKGROUND Standards for accurate and timely diagnosis are ill-defined. In 2015, the National Academies of Science, Engineering, and Medicine (NASEM) committee published a landmark report, Improving Diagnosis in Health Care , and proposed a new definition of diagnostic error, "the failure to ( a ) establish an accurate and timely explanation of the patient's health problem(s) or ( b ) communicate that explanation to the patient." OBJECTIVE This study aimed to explore how researchers operationalize the NASEM's definition of diagnostic error with relevance to accuracy, timeliness, and/or communication in peer-reviewed published literature. METHODS Using the Arskey and O'Malley's framework framework, we identified published literature from October 2015 to February 2021 using Medline and Google Scholar. We also conducted subject matter expert interviews with researchers. RESULTS Of 34 studies identified, 16 were analyzed and abstracted to determine how diagnostic error was operationalized and measured. Studies were grouped by theme: epidemiology, patient focus, measurement/surveillance, and clinician focus. Nine studies indicated using the NASEM definition. Of those, 5 studies also operationalized with existing definitions proposed before the NASEM report. Four studies operationalized the components of the NASEM definition and did not cite existing definitions. Three studies operationalized error using existing definitions only. Subject matter experts indicated that the NASEM definition functions as foundation for researchers to conceptualize diagnostic error. CONCLUSIONS The NASEM report produced a common understanding of diagnostic error that includes accuracy, timeliness, and communication. In recent peer-reviewed literature, most researchers continue to use pre-NASEM report definitions to operationalize accuracy and timeliness. The report catalyzed the use of patient-centered concepts in the definition, resulting in emerging studies focused on examining errors related to communicating diagnosis to patients.
Collapse
Affiliation(s)
- Traber D. Giardina
- From the Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center
- Baylor College of Medicine, Houston, Texas
| | - Haslyn Hunte
- MedStar Institute for Quality and Safety (MIQS), Columbia
- Medstar Health, Baltimore, Maryland
| | - Mary A. Hill
- MedStar Institute for Quality and Safety (MIQS), Columbia
- Medstar Health, Baltimore, Maryland
| | | | - Hardeep Singh
- From the Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center
- Baylor College of Medicine, Houston, Texas
| | - Kelly M. Smith
- MedStar Institute for Quality and Safety (MIQS), Columbia
- Medstar Health, Baltimore, Maryland
- Michael Garron Hospital–Toronto East Health Network
- Institute of Health Policy, Management & Evaluation, University of Toronto, Toronto, Ontario, Canada
| |
Collapse
|
5
|
Zimolzak AJ, Singh H, Murphy DR, Wei L, Memon SA, Upadhyay DK, Korukonda S, Zubkoff L, Sittig DF. Translating electronic health record-based patient safety algorithms from research to clinical practice at multiple sites. BMJ Health Care Inform 2022; 29:bmjhci-2022-100565. [PMID: 35851287 PMCID: PMC9289019 DOI: 10.1136/bmjhci-2022-100565] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2022] [Accepted: 06/19/2022] [Indexed: 01/07/2023] Open
Abstract
Introduction Researchers are increasingly developing algorithms that impact patient care, but algorithms must also be implemented in practice to improve quality and safety. Objective We worked with clinical operations personnel at two US health systems to implement algorithms to proactively identify patients without timely follow-up of abnormal test results that warrant diagnostic evaluation for colorectal or lung cancer. We summarise the steps involved and lessons learned. Methods Twelve sites were involved across two health systems. Implementation involved extensive software documentation, frequent communication with sites and local validation of results. Additionally, we used automated edits of existing code to adapt it to sites’ local contexts. Results All sites successfully implemented the algorithms. Automated edits saved sites significant work in direct code modification. Documentation and communication of changes further aided sites in implementation. Conclusion Patient safety algorithms developed in research projects were implemented at multiple sites to monitor for missed diagnostic opportunities. Automated algorithm translation procedures can produce more consistent results across sites.
Collapse
Affiliation(s)
- Andrew J Zimolzak
- Center for Innovations in Quality, Effectiveness and Safety, Michael E DeBakey VA Medical Center, Houston, Texas, USA
- Department of Medicine, Baylor College of Medicine, Houston, Texas, USA
| | - Hardeep Singh
- Center for Innovations in Quality, Effectiveness and Safety, Michael E DeBakey VA Medical Center, Houston, Texas, USA
- Department of Medicine, Baylor College of Medicine, Houston, Texas, USA
| | - Daniel R Murphy
- Center for Innovations in Quality, Effectiveness and Safety, Michael E DeBakey VA Medical Center, Houston, Texas, USA
- Department of Medicine, Baylor College of Medicine, Houston, Texas, USA
| | - Li Wei
- Center for Innovations in Quality, Effectiveness and Safety, Michael E DeBakey VA Medical Center, Houston, Texas, USA
- Department of Medicine, Baylor College of Medicine, Houston, Texas, USA
| | - Sahar A Memon
- Center for Innovations in Quality, Effectiveness and Safety, Michael E DeBakey VA Medical Center, Houston, Texas, USA
- Department of Medicine, Baylor College of Medicine, Houston, Texas, USA
| | - Divvy K Upadhyay
- Division of Quality, Safety and Patient Experience, Geisinger, Danville, PA, USA
| | | | - Lisa Zubkoff
- Geriatric Research Education and Clinical Center, Birmingham VA Medical Center, Birmingham, Alabama, USA
- Division of Preventive Medicine, The University of Alabama at Birmingham, Birmingham, Alabama, USA
| | - Dean F Sittig
- School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA
| |
Collapse
|
6
|
Conderino S, Bendik S, Richards TB, Pulgarin C, Chan PY, Townsend J, Lim S, Roberts TR, Thorpe LE. The use of electronic health records to inform cancer surveillance efforts: a scoping review and test of indicators for public health surveillance of cancer prevention and control. BMC Med Inform Decis Mak 2022; 22:91. [PMID: 35387655 PMCID: PMC8985310 DOI: 10.1186/s12911-022-01831-8] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2021] [Accepted: 03/27/2022] [Indexed: 11/10/2022] Open
Abstract
INTRODUCTION State cancer prevention and control programs rely on public health surveillance data to set objectives to improve cancer prevention and control, plan interventions, and evaluate state-level progress towards achieving those objectives. The goal of this project was to evaluate the validity of using electronic health records (EHRs) based on common data model variables to generate indicators for surveillance of cancer prevention and control for these public health programs. METHODS Following the methodological guidance from the PRISMA Extension for Scoping Reviews, we conducted a literature scoping review to assess how EHRs are used to inform cancer surveillance. We then developed 26 indicators along the continuum of the cascade of care, including cancer risk factors, immunizations to prevent cancer, cancer screenings, quality of initial care after abnormal screening results, and cancer burden. Indicators were calculated within a sample of patients from the New York City (NYC) INSIGHT Clinical Research Network using common data model EHR data and were weighted to the NYC population using post-stratification. We used prevalence ratios to compare these estimates to estimates from the raw EHR of NYU Langone Health to assess quality of information within INSIGHT, and we compared estimates to results from existing surveillance sources to assess validity. RESULTS Of the 401 identified articles, 15% had a study purpose related to surveillance. Our indicator comparisons found that INSIGHT EHR-based measures for risk factor indicators were similar to estimates from external sources. In contrast, cancer screening and vaccination indicators were substantially underestimated as compared to estimates from external sources. Cancer screenings and vaccinations were often recorded in sections of the EHR that were not captured by the common data model. INSIGHT estimates for many quality-of-care indicators were higher than those calculated using a raw EHR. CONCLUSION Common data model EHR data can provide rich information for certain indicators related to the cascade of care but may have substantial biases for others that limit their use in informing surveillance efforts for cancer prevention and control programs.
Collapse
Affiliation(s)
- Sarah Conderino
- Department of Population Health, New York University Grossman School of Medicine, 180 Madison Ave, New York, NY, 10016, USA.
| | - Stefanie Bendik
- Department of Population Health, New York University Grossman School of Medicine, 180 Madison Ave, New York, NY, 10016, USA
| | - Thomas B Richards
- Division of Cancer Prevention and Control, Centers for Disease Control and Prevention, Atlanta, GA, 30333, USA
| | - Claudia Pulgarin
- Department of Population Health, New York University Grossman School of Medicine, 180 Madison Ave, New York, NY, 10016, USA
| | - Pui Ying Chan
- Division of Epidemiology, New York City Department of Health and Mental Hygiene, Long Island City, NY, 11101, USA
| | - Julie Townsend
- Division of Cancer Prevention and Control, Centers for Disease Control and Prevention, Atlanta, GA, 30333, USA
| | - Sungwoo Lim
- Division of Epidemiology, New York City Department of Health and Mental Hygiene, Long Island City, NY, 11101, USA
| | - Timothy R Roberts
- Health Sciences Library, New York University Grossman School of Medicine, New York, NY, 10016, USA
| | - Lorna E Thorpe
- Department of Population Health, New York University Grossman School of Medicine, 180 Madison Ave, New York, NY, 10016, USA
| |
Collapse
|
7
|
Knoll B, Horwitz LI, Garry K, McCloskey J, Nagler AR, Weerahandi H, Chung WY, Blecker S. Development of an Electronic Trigger to Identify Delayed Follow-up HbA1c Testing for Patients with Uncontrolled Diabetes. J Gen Intern Med 2022; 37:928-934. [PMID: 35037176 PMCID: PMC8904310 DOI: 10.1007/s11606-021-07224-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/09/2021] [Accepted: 10/19/2021] [Indexed: 10/19/2022]
Affiliation(s)
- Brianna Knoll
- Department of Medicine, NYU Langone Health, New York, NY, USA.
| | - Leora I Horwitz
- Department of Medicine, NYU Langone Health, New York, NY, USA.,Department of Population Health, NYU Langone Health, New York, NY, USA.,Center for Healthcare Innovation and Delivery Science, NYU Langone Health, New York, NY, USA
| | - Kira Garry
- Department of Population Health, NYU Langone Health, New York, NY, USA.,Center for Healthcare Innovation and Delivery Science, NYU Langone Health, New York, NY, USA
| | - Jeanne McCloskey
- Department of Population Health, NYU Langone Health, New York, NY, USA.,Center for Healthcare Innovation and Delivery Science, NYU Langone Health, New York, NY, USA
| | - Arielle R Nagler
- Center for Healthcare Innovation and Delivery Science, NYU Langone Health, New York, NY, USA.,The Ronald O. Perelman Department of Dermatology, NYU Grossman School of Medicine, New York, NY, USA
| | - Himali Weerahandi
- Department of Medicine, NYU Langone Health, New York, NY, USA.,Department of Population Health, NYU Langone Health, New York, NY, USA.,Center for Healthcare Innovation and Delivery Science, NYU Langone Health, New York, NY, USA
| | - Wei-Yi Chung
- Center for Healthcare Innovation and Delivery Science, NYU Langone Health, New York, NY, USA.,Clinical Research DataCore, NYU Langone Health, New York, NY, USA
| | - Saul Blecker
- Department of Medicine, NYU Langone Health, New York, NY, USA.,Department of Population Health, NYU Langone Health, New York, NY, USA.,Center for Healthcare Innovation and Delivery Science, NYU Langone Health, New York, NY, USA
| |
Collapse
|
8
|
Using Electronic Medical Records to Identify Potentially Eligible Study Subjects for Lung Cancer Screening with Biomarkers. Cancers (Basel) 2021; 13:cancers13215449. [PMID: 34771612 PMCID: PMC8582572 DOI: 10.3390/cancers13215449] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2021] [Revised: 10/18/2021] [Accepted: 10/19/2021] [Indexed: 11/18/2022] Open
Abstract
Simple Summary Recent cancer screening trials have found that using low-dose computed tomography (LDCT), compared to chest radiography, resulted in a significant reduction in lung cancer mortality. To effectively carry out this intervention, individuals at a high risk of developing lung cancer are targeted. However, accurately identifying and retaining these groups can be challenging. As electronic medical records (EMRs) contain important demographic and clinical information, they could be used to accurately identify subjects for screening. To determine whether EMRs can be used for this purpose, this paper examines the evidence around the use of EMRs in screening trials and the information contained in them that could be used to aid researchers in identifying eligible subjects. Abstract Lung cancer screening trials using low-dose computed tomography (LDCT) show reduced late-stage diagnosis and mortality rates. These trials have identified high-risk groups that would benefit from screening. However, these sub-populations can be difficult to access and retain in trials. Implementation of national screening programmes further suggests that there is poor uptake in eligible populations. A new approach to participant selection may be more effective. Electronic medical records (EMRs) are a viable alternative to population-based or health registries, as they contain detailed clinical and demographic information. Trials have identified that e-screening using EMRs has improved trial retention and eligible subject identification. As such, this paper argues for greater use of EMRs in trial recruitment and screening programmes. Moreover, this opinion paper explores the current issues in and approaches to lung cancer screening, whether records can be used to identify eligible subjects for screening and the challenges that researchers face when using EMR data.
Collapse
|
9
|
Vaghani V, Wei L, Mushtaq U, Sittig DF, Bradford A, Singh H. Validation of an electronic trigger to measure missed diagnosis of stroke in emergency departments. J Am Med Inform Assoc 2021; 28:2202-2211. [PMID: 34279630 DOI: 10.1093/jamia/ocab121] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2021] [Revised: 05/26/2021] [Accepted: 06/23/2021] [Indexed: 02/05/2023] Open
Abstract
OBJECTIVE Diagnostic errors are major contributors to preventable patient harm. We validated the use of an electronic health record (EHR)-based trigger (e-trigger) to measure missed opportunities in stroke diagnosis in emergency departments (EDs). METHODS Using two frameworks, the Safer Dx Trigger Tools Framework and the Symptom-disease Pair Analysis of Diagnostic Error Framework, we applied a symptom-disease pair-based e-trigger to identify patients hospitalized for stroke who, in the preceding 30 days, were discharged from the ED with benign headache or dizziness diagnoses. The algorithm was applied to Veteran Affairs National Corporate Data Warehouse on patients seen between 1/1/2016 and 12/31/2017. Trained reviewers evaluated medical records for presence/absence of missed opportunities in stroke diagnosis and stroke-related red-flags, risk factors, neurological examination, and clinical interventions. Reviewers also estimated quality of clinical documentation at the index ED visit. RESULTS We applied the e-trigger to 7,752,326 unique patients and identified 46,931 stroke-related admissions, of which 398 records were flagged as trigger-positive and reviewed. Of these, 124 had missed opportunities (positive predictive value for "missed" = 31.2%), 93 (23.4%) had no missed opportunity (non-missed), 162 (40.7%) were miscoded, and 19 (4.7%) were inconclusive. Reviewer agreement was high (87.3%, Cohen's kappa = 0.81). Compared to the non-missed group, the missed group had more stroke risk factors (mean 3.2 vs 2.6), red flags (mean 0.5 vs 0.2), and a higher rate of inadequate documentation (66.9% vs 28.0%). CONCLUSION In a large national EHR repository, a symptom-disease pair-based e-trigger identified missed diagnoses of stroke with a modest positive predictive value, underscoring the need for chart review validation procedures to identify diagnostic errors in large data sets.
Collapse
Affiliation(s)
- Viralkumar Vaghani
- Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center and Baylor College of Medicine, Houston, Texas, USA
| | - Li Wei
- Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center and Baylor College of Medicine, Houston, Texas, USA
| | - Umair Mushtaq
- Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center and Baylor College of Medicine, Houston, Texas, USA
| | - Dean F Sittig
- University of Texas-Memorial Hermann Center for Healthcare Quality & Safety, School of Biomedical Informatics, University of Texas Health Science Center, Houston, Texas, USA
| | - Andrea Bradford
- Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center and Baylor College of Medicine, Houston, Texas, USA
| | - Hardeep Singh
- Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center and Baylor College of Medicine, Houston, Texas, USA
| |
Collapse
|
10
|
Dolci E, Schärer B, Grossmann N, Musy SN, Zúñiga F, Bachnick S, Simon M. Automated Fall Detection Algorithm With Global Trigger Tool, Incident Reports, Manual Chart Review, and Patient-Reported Falls: Algorithm Development and Validation With a Retrospective Diagnostic Accuracy Study. J Med Internet Res 2020; 22:e19516. [PMID: 32955445 PMCID: PMC7536608 DOI: 10.2196/19516] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2020] [Revised: 06/26/2020] [Accepted: 07/26/2020] [Indexed: 11/13/2022] Open
Abstract
BACKGROUND Falls are common adverse events in hospitals, frequently leading to additional health costs due to prolonged stays and extra care. Therefore, reliable fall detection is vital to develop and test fall prevention strategies. However, conventional methods-voluntary incident reports and manual chart reviews-are error-prone and time consuming, respectively. Using a search algorithm to examine patients' electronic health record data and flag fall indicators offers an inexpensive, sensitive, cost-effective alternative. OBJECTIVE This study's purpose was to develop a fall detection algorithm for use with electronic health record data, then to evaluate it alongside the Global Trigger Tool, incident reports, a manual chart review, and patient-reported falls. METHODS Conducted on 2 campuses of a large hospital system in Switzerland, this retrospective diagnostic accuracy study consisted of 2 substudies: the first, targeting 240 patients, for algorithm development and the second, targeting 298 patients, for validation. In the development study, we compared the new algorithm's in-hospital fall rates with those indicated by the Global Trigger Tool and incident reports; in the validation study, we compared the algorithm's in-hospital fall rates with those from patient-reported falls and manual chart review. We compared the various methods by calculating sensitivity, specificity, and predictive values. RESULTS Twenty in-hospital falls were discovered in the development study sample. Of these, the algorithm detected 19 (sensitivity 95%), the Global Trigger Tool detected 18 (90%), and incident reports detected 14 (67%). Of the 15 falls found in the validation sample, the algorithm identified all 15 (100%), the manual chart review identified 14 (93%), and the patient-reported fall measure identified 5 (33%). Owing to relatively high numbers of false positives based on falls present on admission, the algorithm's positive predictive values were 50% (development sample) and 47% (validation sample). Instead of requiring 10 minutes per case for a full manual review or 20 minutes to apply the Global Trigger Tool, the algorithm requires only a few seconds, after which only the positive results (roughly 11% of the full case number) require review. CONCLUSIONS The newly developed electronic health record algorithm demonstrated very high sensitivity for fall detection. Applied in near real time, the algorithm can record in-hospital falls events effectively and help to develop and test fall prevention measures.
Collapse
Affiliation(s)
- Elisa Dolci
- MediZentrum Täuffelen, Täuffelen, Switzerland
| | - Barbara Schärer
- Nursing & Midwifery Research Unit, Inselspital Bern University Hospital, Bern, Switzerland
| | - Nicole Grossmann
- Department of General Internal Medicine, Inselspital Bern University Hospital, Bern, Switzerland.,Institute of Nursing Science, Department of Public Health, Faculty of Medicine, University of Basel, Basel, Switzerland
| | - Sarah Naima Musy
- Nursing & Midwifery Research Unit, Inselspital Bern University Hospital, Bern, Switzerland.,Institute of Nursing Science, Department of Public Health, Faculty of Medicine, University of Basel, Basel, Switzerland
| | - Franziska Zúñiga
- Institute of Nursing Science, Department of Public Health, Faculty of Medicine, University of Basel, Basel, Switzerland
| | - Stefanie Bachnick
- Institute of Nursing Science, Department of Public Health, Faculty of Medicine, University of Basel, Basel, Switzerland
| | - Michael Simon
- Nursing & Midwifery Research Unit, Inselspital Bern University Hospital, Bern, Switzerland.,Institute of Nursing Science, Department of Public Health, Faculty of Medicine, University of Basel, Basel, Switzerland
| |
Collapse
|
11
|
Fleming S, Nicholson BD, Bhuiya A, de Lusignan S, Hirst Y, Hobbs R, Perera R, Sherlock J, Yonova I, Bankhead C. CASNET2: evaluation of an electronic safety netting cancer toolkit for the primary care electronic health record: protocol for a pragmatic stepped-wedge RCT. BMJ Open 2020; 10:e038562. [PMID: 32843517 PMCID: PMC7449309 DOI: 10.1136/bmjopen-2020-038562] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/15/2020] [Revised: 06/16/2020] [Accepted: 07/01/2020] [Indexed: 11/03/2022] Open
Abstract
INTRODUCTION Safety-netting in primary care is the best practice in cancer diagnosis, ensuring that patients are followed up until symptoms are explained or have resolved. Currently, clinicians use haphazard manual solutions. The ubiquitous use of electronic health records provides an opportunity to standardise safety-netting practices.A new electronic safety-netting toolkit has been introduced to provide systematic ways to track and follow up patients. We will evaluate the effectiveness of this toolkit, which is embedded in a major primary care clinical system in England:Egerton Medical Information System(EMIS)-Web. METHODS AND ANALYSIS We will conduct a stepped-wedge cluster RCT in 60 general practices within the RCGP Research and Surveillance Centre (RSC) network. Groups of 10 practices will be randomised into the active phase at 2-monthly intervals over 12 months. All practices will be activated for at least 2 months. The primary outcome is the primary care interval measured as days between the first recorded symptom of cancer (within the year prior to diagnosis) and the subsequent referral to secondary care. Other outcomes include referrals rates and rates of direct access cancer investigation.Analysis of the clustered stepped-wedge design will model associations using a fixed effect for intervention condition of the cluster at each time step, a fixed effect for time and other covariates, and then include a random effect for practice and for patient to account for correlation between observations from the same centre and from the same participant. ETHICS AND DISSEMINATION Ethical approval has been obtained from the North West-Greater Manchester West National Health Service Research Ethics Committee (REC Reference 19/NW/0692). Results will be disseminated in peer-reviewed journals and conferences, and sent to participating practices. They will be published on the University of Oxford Nuffield Department of Primary Care and RCGP RSC websites. TRIAL REGISTRATION NUMBER ISRCTN15913081; Pre-results.
Collapse
Affiliation(s)
- Susannah Fleming
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
| | - Brian D Nicholson
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
| | - Afsana Bhuiya
- North Central and East London Cancer Alliance, London, UK
| | - Simon de Lusignan
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
- Department of Clinical and Experimental Medicine, University of Surrey, Guildford, Surrey, UK
- Research and Surveillance Centre, Royal College of General Practitioners, London, London, UK
| | - Yasemin Hirst
- Research Department of Behavioural Science and Health, University College, London, UK
| | - Richard Hobbs
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
| | - Rafael Perera
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
| | - Julian Sherlock
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
- Department of Clinical and Experimental Medicine, University of Surrey, Guildford, Surrey, UK
| | - Ivelina Yonova
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
- Department of Clinical and Experimental Medicine, University of Surrey, Guildford, Surrey, UK
- Research and Surveillance Centre, Royal College of General Practitioners, London, London, UK
| | - Clare Bankhead
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
| |
Collapse
|
12
|
Singh H, Bradford A, Goeschel C. Operational measurement of diagnostic safety: state of the science. ACTA ACUST UNITED AC 2020; 8:51-65. [PMID: 32706749 DOI: 10.1515/dx-2020-0045] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/13/2020] [Accepted: 04/18/2020] [Indexed: 12/15/2022]
Abstract
Reducing the incidence of diagnostic errors is increasingly a priority for government, professional, and philanthropic organizations. Several obstacles to measurement of diagnostic safety have hampered progress toward this goal. Although a coordinated national strategy to measure diagnostic safety remains an aspirational goal, recent research has yielded practical guidance for healthcare organizations to start using measurement to enhance diagnostic safety. This paper, concurrently published as an Issue Brief by the Agency for Healthcare Research and Quality, issues a "call to action" for healthcare organizations to begin measurement efforts using data sources currently available to them. Our aims are to outline the state of the science and provide practical recommendations for organizations to start identifying and learning from diagnostic errors. Whether by strategically leveraging current resources or building additional capacity for data gathering, nearly all organizations can begin their journeys to measure and reduce preventable diagnostic harm.
Collapse
Affiliation(s)
- Hardeep Singh
- Center for Innovations in Quality, Effectiveness, and Safety (IQuESt), Michael E. DeBakey Veterans Affairs Medical Center, Houston, TX, USA
- Baylor College of Medicine, 2002 Holcombe Blvd. #152, Houston, TX, USA
| | - Andrea Bradford
- Center for Innovations in Quality, Effectiveness, and Safety (IQuESt), Michael E. DeBakey Veterans Affairs Medical Center, Houston, TX, USA
- Section of Gastroenterology and Hepatology, Department of Medicine, Baylor College of Medicine, Houston, TX, USA
| | - Christine Goeschel
- MedStar Health Institute for Quality and Safety, MD, USA
- Department of Medicine, Georgetown University, Washington, DC, USA
| |
Collapse
|
13
|
Sheehan SE, Safdar N, Singh H, Sittig DF, Bruno MA, Keller K, Kinnard S, Brunner MC. Detection and Remediation of Misidentification Errors in Radiology Examination Ordering. Appl Clin Inform 2020; 11:79-87. [PMID: 31995835 PMCID: PMC6989264 DOI: 10.1055/s-0039-3402730] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2019] [Accepted: 12/06/2019] [Indexed: 02/08/2023] Open
Abstract
BACKGROUND Despite progress in patient safety, misidentification errors in radiology such as ordering imaging on the wrong anatomic side persist. If undetected, these errors can cause patient harm for multiple reasons, in addition to producing erroneous electronic health records (EHR) data. OBJECTIVES We describe the pilot testing of a quality improvement methodology using electronic trigger tools and preimaging checklists to detect "wrong-side" misidentification errors in radiology examination ordering, and to measure staff adherence to departmental policy in error remediation. METHODS We retrospectively applied and compared two methods for the detection of "wrong-side" misidentification errors among a cohort of all imaging studies ordered during a 1-year period (June 1, 2015-May 31, 2016) at our tertiary care hospital. Our methods included: (1) manual review of internal quality improvement spreadsheet records arising from the prospective performance of preimaging safety checklists, and (2) automated error detection via the development and validation of an electronic trigger tool which identified discrepant side indications within EHR imaging orders. RESULTS Our combined methods detected misidentification errors in 6.5/1,000 of study cohort imaging orders. Our trigger tool retrospectively identified substantially more misidentification errors than were detected prospectively during preimaging checklist performance, with a high positive predictive value (PPV: 88.4%, 95% confidence interval: 85.4-91.4). However, two third of errors detected during checklist performance were not detected by the trigger tool, and checklist-detected errors were more often appropriately resolved (p < 0.00001, 95% confidence interval: 2.0-6.9; odds ratio: 3.6). CONCLUSION Our trigger tool enabled the detection of substantially more imaging ordering misidentification errors than preimaging safety checklists alone, with a high PPV. Many errors were only detected by the preimaging checklist; however, suggesting that additional trigger tools may need to be developed and used in conjunction with checklist-based methods to ensure patient safety.
Collapse
Affiliation(s)
- Scott E. Sheehan
- Department of Radiology, William S. Middleton Veterans Hospital, Madison, Wisconsin, United States
| | - Nasia Safdar
- Department of Medicine, William S. Middleton Memorial Veterans Hospital and University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin, United States
| | - Hardeep Singh
- Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey VA Medical Center and Department of Medicine, Baylor College of Medicine, Houston, Texas, United States
| | - Dean F. Sittig
- School of Biomedical Informatics, University of Texas Health Science Center at Houston, Houston, Texas, United States
| | - Michael A. Bruno
- Department of Radiology, Penn State Hershey, Hershey, Pennsylvania, United States
| | - Kelli Keller
- Department of Radiology, William S. Middleton Veterans Hospital, Madison, Wisconsin, United States
| | - Samantha Kinnard
- Department of Radiology, William S. Middleton Veterans Hospital, Madison, Wisconsin, United States
| | - Michael C. Brunner
- Department of Radiology, William S. Middleton Memorial Veterans Hospital and University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin, United States
| |
Collapse
|
14
|
Murphy DR, Satterly T, Rogith D, Sittig DF, Singh H. Barriers and facilitators impacting reliability of the electronic health record-facilitated total testing process. Int J Med Inform 2019; 127:102-108. [PMID: 31128821 DOI: 10.1016/j.ijmedinf.2019.04.004] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2018] [Revised: 02/15/2019] [Accepted: 04/05/2019] [Indexed: 02/08/2023]
Abstract
OBJECTIVE Despite ongoing efforts to improve reliability of the total testing process (TTP), breakdowns continue to occur resulting in diagnostic delays and suboptimal patient outcomes. We performed an exploratory study to identify factors that impact TTP reliability in electronic health record (EHR)-enabled care. MATERIALS AND METHODS We interviewed experts at three large EHR-enabled health care organizations and identified all TTP steps performed from clinician test ordering to result communication to patients. Findings from all sites were combined to develop a detailed process map of known TTP activities. We additionally asked experts about factors that positively or negatively impacted TTP reliability at each step. We describe the specific TTP steps identified and associated barriers and facilitators to TTP reliability. RESULTS We interviewed 39 experts involved in or overseeing the TTP. Most TTP activities identified were similar across sites, but we found significant differences with test order transmission to diagnostic services and relay of results back to clinicians and patients. Twenty-five unique barriers were identified related to technology and EHR usability issues, time and resource constraints, suboptimal clinic workflows, patient-related factors, information access limitations, and insufficient clinician training. Twenty-four unique facilitators were identified related to personnel training, workflow optimization and standardization, helpful EHR features, and improved electronic communication between clinics and diagnostic services. DISCUSSION Barriers related to EHR usability and with communication between clinicians and diagnostic services increase TTP vulnerability and should be targeted by future efforts to improve process reliability. Several facilitators identified in the study could inform future strategies and solutions to improve TTP reliability.
Collapse
Affiliation(s)
- Daniel R Murphy
- Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, TX, United States; Department of Medicine, Baylor College of Medicine, Houston, TX, United States.
| | - Tyler Satterly
- Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, TX, United States; Department of Medicine, Baylor College of Medicine, Houston, TX, United States
| | - Deevakar Rogith
- The University of Texas Health Science Center at Houston's School of Biomedical Informatics, Houston, TX, United States
| | - Dean F Sittig
- The University of Texas Health Science Center at Houston's School of Biomedical Informatics, Houston, TX, United States; The UT-Memorial Hermann Center for Healthcare Quality & Safety, Houston, TX, United States
| | - Hardeep Singh
- Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, TX, United States; Department of Medicine, Baylor College of Medicine, Houston, TX, United States
| |
Collapse
|
15
|
Timely Interventions for Children with ADHD through Web-Based Monitoring Algorithms. Diseases 2019; 7:diseases7010020. [PMID: 30736492 PMCID: PMC6473761 DOI: 10.3390/diseases7010020] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2018] [Revised: 01/30/2019] [Accepted: 02/01/2019] [Indexed: 01/27/2023] Open
Abstract
The aim of this study was to evaluate an automated trigger algorithm designed to detect potentially adverse events in children with Attention-Deficit/Hyperactivity Disorder (ADHD), who were monitored remotely between visits. We embedded a trigger algorithm derived from parent-reported ADHD rating scales within an electronic patient monitoring system. We categorized clinicians’ alert resolution outcomes and compared Vanderbilt ADHD rating scale scores between patients who did or did not have triggered alerts. A total of 146 out of 1738 parent reports (8%) triggered alerts for 98 patients. One hundred and eleven alerts (76%) required immediate clinician review. Nurses successfully contacted parents for 68 (61%) of actionable alerts; 46% (31/68) led to a change in care plan prior to the next scheduled appointment. Compared to patients without alerts, patients with alerts demonstrated worsened ADHD severity (β = 5.8, 95% CI: 3.5–8.1 [p < 0.001] within 90 days prior to an alert. The trigger algorithm facilitated timely changes in the care plan in between face-to-face visits.
Collapse
|
16
|
Murphy DR, Meyer AN, Sittig DF, Meeks DW, Thomas EJ, Singh H. Application of electronic trigger tools to identify targets for improving diagnostic safety. BMJ Qual Saf 2019; 28:151-159. [PMID: 30291180 PMCID: PMC6365920 DOI: 10.1136/bmjqs-2018-008086] [Citation(s) in RCA: 75] [Impact Index Per Article: 15.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2018] [Revised: 06/20/2018] [Accepted: 08/14/2018] [Indexed: 02/05/2023]
Abstract
Progress in reducing diagnostic errors remains slow partly due to poorly defined methods to identify errors, high-risk situations, and adverse events. Electronic trigger (e-trigger) tools, which mine vast amounts of patient data to identify signals indicative of a likely error or adverse event, offer a promising method to efficiently identify errors. The increasing amounts of longitudinal electronic data and maturing data warehousing techniques and infrastructure offer an unprecedented opportunity to implement new types of e-trigger tools that use algorithms to identify risks and events related to the diagnostic process. We present a knowledge discovery framework, the Safer Dx Trigger Tools Framework, that enables health systems to develop and implement e-trigger tools to identify and measure diagnostic errors using comprehensive electronic health record (EHR) data. Safer Dx e-trigger tools detect potential diagnostic events, allowing health systems to monitor event rates, study contributory factors and identify targets for improving diagnostic safety. In addition to promoting organisational learning, some e-triggers can monitor data prospectively and help identify patients at high-risk for a future adverse event, enabling clinicians, patients or safety personnel to take preventive actions proactively. Successful application of electronic algorithms requires health systems to invest in clinical informaticists, information technology professionals, patient safety professionals and clinicians, all of who work closely together to overcome development and implementation challenges. We outline key future research, including advances in natural language processing and machine learning, needed to improve effectiveness of e-triggers. Integrating diagnostic safety e-triggers in institutional patient safety strategies can accelerate progress in reducing preventable harm from diagnostic errors.
Collapse
Affiliation(s)
- Daniel R Murphy
- Center for Innovations in Quality, Effectiveness and Safety, Michael E DeBakey Veterans Affairs Medical Center, Houston, Texas, USA
- Department of Medicine, Baylor College of Medicine, Houston, Texas, USA
| | - Ashley Nd Meyer
- Center for Innovations in Quality, Effectiveness and Safety, Michael E DeBakey Veterans Affairs Medical Center, Houston, Texas, USA
- Department of Medicine, Baylor College of Medicine, Houston, Texas, USA
| | - Dean F Sittig
- Center for Innovations in Quality, Effectiveness and Safety, Michael E DeBakey Veterans Affairs Medical Center, Houston, Texas, USA
- School of Biomedical Informatics, University of Texas Health Science Center, Houston, Texas, USA
- Department of Medicine, University of Texas-Memorial Hermann Center for Healthcare Quality and Safety, Houston, Texas, USA
| | - Derek W Meeks
- Center for Innovations in Quality, Effectiveness and Safety, Michael E DeBakey Veterans Affairs Medical Center, Houston, Texas, USA
- Department of Medicine, Baylor College of Medicine, Houston, Texas, USA
| | - Eric J Thomas
- Department of Medicine, University of Texas-Memorial Hermann Center for Healthcare Quality and Safety, Houston, Texas, USA
| | - Hardeep Singh
- Center for Innovations in Quality, Effectiveness and Safety, Michael E DeBakey Veterans Affairs Medical Center, Houston, Texas, USA
- Department of Medicine, Baylor College of Medicine, Houston, Texas, USA
| |
Collapse
|
17
|
Knight A, Jarrad GA, Schrader GD, Strobel J, Horton D, Bidargaddi N. Monte Carlo Simulations Demonstrate Algorithmic Interventions Over Time Reduce Hospitalisation in Patients With Schizophrenia and Bipolar Disorder. BIOMEDICAL INFORMATICS INSIGHTS 2018; 10:1178222618803076. [PMID: 30302053 PMCID: PMC6170953 DOI: 10.1177/1178222618803076] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/15/2018] [Accepted: 09/02/2018] [Indexed: 11/29/2022]
Abstract
Non-adherence with pharmacologic treatment is associated with increased rates of relapse and rehospitalisation among patients with schizophrenia and bipolar disorder. To improve treatment response, remission, and recovery, research efforts are still needed to elucidate how to effectively map patient’s response to medication treatment including both therapeutic and adverse effects, compliance, and satisfaction in the prodromal phase of illness (ie, the time period in between direct clinical consultation and relapse). The Actionable Intime Insights (AI2) application draws information from Australian Medicare administrative claims records in real time when compliance with treatment does not meet best practice guidelines for managing chronic severe mental illness. Subsequently, the AI2 application alerts clinicians and patients when patients do not adhere to guidelines for treatment. The aim of this study was to evaluate the impact of the AI2 application on the risk of hospitalisation among simulated patients with schizophrenia and bipolar disorder. Monte Carlo simulation methodology was used to estimate the impact of the AI2 intervention on the probability of hospitalisation over a 2-year period. Results indicated that when the AI2 algorithmic intervention had an efficacy level of (>0.6), over 80% of actioned alerts were contributing to reduced hospitalisation risk among the simulated patients. Such findings indicate the potential utility of the AI2 application should replication studies validate its methodologic and ecological rigour in real-world settings.
Collapse
Affiliation(s)
- Alissa Knight
- Personal Health Informatics, College of Medicine and Public Health, Flinders University, Adelaide, SA, Australia
| | - Geoff A Jarrad
- Data to Decisions Cooperative Research Centre, Adelaide, SA, Australia
| | - Geoff D Schrader
- Personal Health Informatics, College of Medicine and Public Health, Flinders University, Adelaide, SA, Australia.,Country Mental Health Services, SA Health, Adelaide, SA, Australia
| | - Jorg Strobel
- Personal Health Informatics, College of Medicine and Public Health, Flinders University, Adelaide, SA, Australia.,Country Mental Health Services, SA Health, Adelaide, SA, Australia
| | - Dennis Horton
- Data to Decisions Cooperative Research Centre, Adelaide, SA, Australia
| | - Niranjan Bidargaddi
- Personal Health Informatics, College of Medicine and Public Health, Flinders University, Adelaide, SA, Australia.,South Australian Health and Medical Research Institute, Adelaide, SA, Australia
| |
Collapse
|
18
|
Bhise V, Rajan SS, Sittig DF, Vaghani V, Morgan RO, Khanna A, Singh H. Electronic health record reviews to measure diagnostic uncertainty in primary care. J Eval Clin Pract 2018; 24:545-551. [PMID: 29675888 DOI: 10.1111/jep.12912] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/29/2017] [Revised: 02/19/2018] [Accepted: 02/20/2018] [Indexed: 02/05/2023]
Abstract
RATIONALE, AIMS AND OBJECTIVES Diagnostic uncertainty is common in primary care. Because it is challenging to measure, there is inadequate scientific understanding of diagnostic decision-making during uncertainty. Our objective was to understand how diagnostic uncertainty was documented in the electronic health record (EHR) and explore a strategy to retrospectively identify it using clinician documentation. METHODS We reviewed the literature to identify documentation language that could identify both direct expression and indirect inference of diagnostic uncertainty and designed an instrument to facilitate record review. Direct expression included clinician's use of question marks, differential diagnoses, symptoms as diagnosis, or vocabulary such as "probably, maybe, likely, unclear or unknown," while describing the diagnosis. Indirect inference included absence of documented diagnosis at the end of the visit, ordering of multiple consultations or diagnostic tests to resolve diagnostic uncertainty, and use of suspended judgement, test of treatment, and risk-averse disposition. Two physician-reviewers independently reviewed notes on a sample of outpatient visits to identify diagnostic uncertainty at the end of the visit. Documented Ninth Revision of the International Classification of Diseases (ICD-9) diagnosis codes and note quality were assessed. RESULTS Of 389 patient records reviewed, 218 had evidence of diagnostic activity and were included. In 156 visits (71.6%), reviewers identified clinicians who experienced diagnostic uncertainty with moderate inter-reviewer agreement (81.7%; Cohen's kappa: 0.609). Most cases (125, 80.1%) showed evidence of both direct expression and indirect inference. Uncertainty was directly expressed in 139 (89.1%) cases, most commonly by using symptoms as diagnosis (98, 62.8%), and inferred in 144 (92.3%). In more than 1/3 of visits (58, 37.2%), diagnostic uncertainty was recorded inappropriately using ICD-9 codes. CONCLUSIONS While current diagnosis coding mechanisms (ICD-9 and ICD-10) are unable to capture uncertainty, our study finds that review of EHR documentation can help identify diagnostic uncertainty with moderate reliability. Better measurement and understanding of diagnostic uncertainty could help inform strategies to improve the safety and efficiency of diagnosis.
Collapse
Affiliation(s)
- Viraj Bhise
- Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, TX, USA.,Department of Medicine, Baylor College of Medicine, Houston, TX, USA.,University of Texas School of Public Health, Houston, TX, USA.,John A Burns School of Medicine, University of Hawai'i at Manoa, Honolulu, HI, USA
| | - Suja S Rajan
- University of Texas School of Public Health, Houston, TX, USA
| | - Dean F Sittig
- School of Biomedical Informatics and UT-Memorial Hermann Center for Health Care Quality and Safety, University of Texas Health Science Center at Houston, Houston, TX, USA
| | - Viralkumar Vaghani
- Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, TX, USA.,Department of Medicine, Baylor College of Medicine, Houston, TX, USA
| | - Robert O Morgan
- University of Texas School of Public Health, Houston, TX, USA
| | - Arushi Khanna
- Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, TX, USA.,Department of Medicine, Baylor College of Medicine, Houston, TX, USA
| | - Hardeep Singh
- Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, TX, USA.,Department of Medicine, Baylor College of Medicine, Houston, TX, USA
| |
Collapse
|
19
|
Nicholson BD, Goyder CR, Bankhead CR, Toftegaard BS, Rose PW, Thulesius H, Vedsted P, Perera R. Responsibility for follow-up during the diagnostic process in primary care: a secondary analysis of International Cancer Benchmarking Partnership data. Br J Gen Pract 2018; 68:e323-e332. [PMID: 29686134 PMCID: PMC5916079 DOI: 10.3399/bjgp18x695813] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2017] [Accepted: 12/05/2017] [Indexed: 01/06/2023] Open
Abstract
BACKGROUND It is unclear to what extent primary care practitioners (PCPs) should retain responsibility for follow-up to ensure that patients are monitored until their symptoms or signs are explained. AIM To explore the extent to which PCPs retain responsibility for diagnostic follow-up actions across 11 international jurisdictions. DESIGN AND SETTING A secondary analysis of survey data from the International Cancer Benchmarking Partnership. METHOD The authors counted the proportion of 2879 PCPs who retained responsibility for each area of follow-up (appointments, test results, and non-attenders). Proportions were weighted by the sample size of each jurisdiction. Pooled estimates were obtained using a random-effects model, and UK estimates were compared with non-UK ones. Free-text responses were analysed to contextualise quantitative findings using a modified grounded theory approach. RESULTS PCPs varied in their retention of responsibility for follow-up from 19% to 97% across jurisdictions and area of follow-up. Test reconciliation was inadequate in most jurisdictions. Significantly fewer UK PCPs retained responsibility for test result communication (73% versus 85%, P = 0.04) and non-attender follow-up (78% versus 93%, P<0.01) compared with non-UK PCPs. PCPs have developed bespoke, inconsistent solutions to follow-up. In cases of greatest concern, 'double safety netting' is described, where both patient and PCP retain responsibility. CONCLUSION The degree to which PCPs retain responsibility for follow-up is dependent on their level of concern about the patient and their primary care system's properties. Integrated systems to support follow-up are at present underutilised, and research into their development, uptake, and effectiveness seems warranted.
Collapse
Affiliation(s)
- Brian D Nicholson
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
| | - Clare R Goyder
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
| | - Clare R Bankhead
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
| | | | - Peter W Rose
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
| | - Hans Thulesius
- Department of Clinical Sciences, Lund University, Växjö, Sweden
| | - Peter Vedsted
- Research Unit for General Practice, Aarhus University, Aarhus, Denmark
| | - Rafael Perera
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
| |
Collapse
|
20
|
Electronic Triggers to Identify Delays in Follow-Up of Mammography: Harnessing the Power of Big Data in Health Care. J Am Coll Radiol 2018; 15:287-295. [PMID: 29102539 DOI: 10.1016/j.jacr.2017.10.001] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2017] [Revised: 09/11/2017] [Accepted: 10/02/2017] [Indexed: 02/05/2023]
Abstract
PURPOSE We previously developed electronic triggers to automatically flag records for patients experiencing potential delays in diagnostic evaluation for certain cancers. Because of the unique clinical, logistic, and legal aspects of mammography, this study was conducted to evaluate the effectiveness of a trigger to flag delayed follow-up on mammography. METHODS An algorithm was developed to detect delays in follow-up of abnormal mammographic results (>60 days for BI-RADS® 0, 4, and 5 and >7 months for BI-RADS 3) using clinical data in the electronic health record. Flagged records were then manually reviewed to determine the trigger's performance characteristics (positive and negative predictive value, sensitivity, and specificity). The frequency of delays and patient communication related to abnormal results, reasons for lack of follow-up, and whether patients were subsequently diagnosed with breast cancer were also assessed. RESULTS Of 365,686 patients seen between January 1, 2010, and May 31, 2015, the trigger identified 2,129 patients with abnormal findings on mammography, of whom it flagged 552 as having delays in follow-up. From these, review of 400 randomly selected records revealed 283 true delays (positive predictive value, 71%; 95% confidence interval, 66%-75%), including 280 records without any documented plan and three patients with plans that were not adhered to. Transcription and reporting inconsistencies were identified in 27% of externally performed mammographic reports. Only 335 records (84%) contained specific documentation that the patient was informed of the abnormal result. CONCLUSIONS Care delays appear to continue despite federal laws requiring patient notification of mammographic results within 30 days. Clinical application of mammography-related triggers could help detect these delays.
Collapse
|
21
|
Murphy DR, Meyer AND, Vaghani V, Russo E, Sittig DF, Wei L, Wu L, Singh H. Development and Validation of Trigger Algorithms to Identify Delays in Diagnostic Evaluation of Gastroenterological Cancer. Clin Gastroenterol Hepatol 2018; 16:90-98. [PMID: 28804030 DOI: 10.1016/j.cgh.2017.08.007] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/03/2017] [Revised: 07/11/2017] [Accepted: 08/05/2017] [Indexed: 02/07/2023]
Abstract
BACKGROUND & AIMS Colorectal cancer (CRC) and hepatocellular cancer (HCC) are common causes of death and morbidity, and patients benefit from early detection. However, delays in follow-up of suspicious findings are common, and methods to efficiently detect such delays are needed. We developed, refined, and tested trigger algorithms that identify patients with delayed follow-up evaluation of findings suspicious of CRC or HCC. METHODS We developed and validated two trigger algorithms that detect delays in diagnostic evaluation of CRC and HCC using laboratory, diagnosis, procedure, and referral codes from the Department of Veteran Affairs National Corporate Data Warehouse. The algorithm initially identified patients with positive test results for iron deficiency anemia or fecal immunochemical test (for CRC) and elevated α-fetoprotein results (for HCC). Our algorithm then excluded patients for whom follow-up evaluation was unnecessary, such as patients with a terminal illness or those who had already completed a follow-up evaluation within 60 days. Clinicians reviewed samples of both delayed and nondelayed records, and review data were used to calculate trigger performance. RESULTS We applied the algorithm for CRC to 245,158 patients seen from January 1, 2013, through December 31, 2013 and identified 1073 patients with delayed follow up. In a review of 400 randomly selected records, we found that our algorithm identified patients with delayed follow-up with a positive predictive value of 56.0% (95% CI, 51.0%-61.0%). We applied the algorithm for HCC to 333,828 patients seen from January 1, 2011 through December 31, 2014, and identified 130 patients with delayed follow-up. During manual review of all 130 records, we found that our algorithm identified patients with delayed follow-up with a positive predictive value of 82.3% (95% CI, 74.4%-88.2%). When we extrapolated the findings to all patients with abnormal results, the algorithm identified patients with delayed follow-up evaluation for CRC with 68.6% sensitivity (95% CI, 65.4%-71.6%) and 81.1% specificity (95% CI, 79.5%-82.6%); it identified patients with delayed follow-up evaluation for HCC with 89.1% sensitivity (95% CI, 81.8%-93.8%) and 96.5% specificity (95% CI, 94.8%-97.7%). Compared to nonselective methods, use of the algorithm reduced the number of records required for review to identify a delay by more than 99%. CONCLUSIONS Using data from the Veterans Affairs electronic health record database, we developed an algorithm that greatly reduces the number of record reviews necessary to identify delays in follow-up evaluations for patients with suspected CRC or HCC. This approach offers a more efficient method to identify delayed diagnostic evaluation of gastroenterological cancers.
Collapse
Affiliation(s)
- Daniel R Murphy
- Houston Veterans Affairs Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas; Department of Medicine, Baylor College of Medicine, Houston, Texas.
| | - Ashley N D Meyer
- Houston Veterans Affairs Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas; Department of Medicine, Baylor College of Medicine, Houston, Texas
| | - Viralkumar Vaghani
- Houston Veterans Affairs Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas; Department of Medicine, Baylor College of Medicine, Houston, Texas
| | - Elise Russo
- Houston Veterans Affairs Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas; Department of Medicine, Baylor College of Medicine, Houston, Texas
| | - Dean F Sittig
- University of Texas Health Science Center, University of Texas-Memorial Hermann Center for Healthcare Quality and Safety, Houston, Texas
| | - Li Wei
- Houston Veterans Affairs Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas; Department of Medicine, Baylor College of Medicine, Houston, Texas
| | - Louis Wu
- Houston Veterans Affairs Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas
| | - Hardeep Singh
- Houston Veterans Affairs Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas; Department of Medicine, Baylor College of Medicine, Houston, Texas
| |
Collapse
|
22
|
Bhise V, Meyer AND, Singh H, Wei L, Russo E, Al-Mutairi A, Murphy DR. Errors in Diagnosis of Spinal Epidural Abscesses in the Era of Electronic Health Records. Am J Med 2017; 130:975-981. [PMID: 28366427 DOI: 10.1016/j.amjmed.2017.03.009] [Citation(s) in RCA: 32] [Impact Index Per Article: 4.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/22/2016] [Revised: 01/06/2017] [Accepted: 03/02/2017] [Indexed: 12/20/2022]
Abstract
PURPOSE With this study, we set out to identify missed opportunities in diagnosis of spinal epidural abscesses to outline areas for process improvement. METHODS Using a large national clinical data repository, we identified all patients with a new diagnosis of spinal epidural abscess in the Department of Veterans Affairs (VA) during 2013. Two physicians independently conducted retrospective chart reviews on 250 randomly selected patients and evaluated their records for red flags (eg, unexplained weight loss, neurological deficits, and fever) 90 days prior to diagnosis. Diagnostic errors were defined as missed opportunities to evaluate red flags in a timely or appropriate manner. Reviewers gathered information about process breakdowns related to patient factors, the patient-provider encounter, test performance and interpretation, test follow-up and tracking, and the referral process. Reviewers also determined harm and time lag between red flags and definitive diagnoses. RESULTS Of 250 patients, 119 had a new diagnosis of spinal epidural abscess, 66 (55.5%) of which experienced diagnostic error. Median time to diagnosis in error cases was 12 days, compared with 4 days in cases without error (P <.01). Red flags that were frequently not evaluated in error cases included unexplained fever (n = 57; 86.4%), focal neurological deficits with progressive or disabling symptoms (n = 54; 81.8%), and active infection (n = 54; 81.8%). Most errors involved breakdowns during the patient-provider encounter (n = 60; 90.1%), including failures in information gathering/integration, and were associated with temporary harm (n = 43; 65.2%). CONCLUSION Despite wide availability of clinical data, errors in diagnosis of spinal epidural abscesses are common and involve inadequate history, physical examination, and test ordering. Solutions should include renewed attention to basic clinical skills.
Collapse
Affiliation(s)
- Viraj Bhise
- Houston Veterans Affairs Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, Tex; Department of Medicine, Baylor College of Medicine, Houston, Tex
| | - Ashley N D Meyer
- Houston Veterans Affairs Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, Tex; Department of Medicine, Baylor College of Medicine, Houston, Tex
| | - Hardeep Singh
- Houston Veterans Affairs Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, Tex; Department of Medicine, Baylor College of Medicine, Houston, Tex
| | - Li Wei
- Houston Veterans Affairs Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, Tex; Department of Medicine, Baylor College of Medicine, Houston, Tex
| | - Elise Russo
- Houston Veterans Affairs Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, Tex; Department of Medicine, Baylor College of Medicine, Houston, Tex
| | - Aymer Al-Mutairi
- Department of Medicine, Baylor College of Medicine, Houston, Tex
| | - Daniel R Murphy
- Houston Veterans Affairs Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, Tex; Department of Medicine, Baylor College of Medicine, Houston, Tex.
| |
Collapse
|
23
|
Meyer AND, Murphy DR, Al-Mutairi A, Sittig DF, Wei L, Russo E, Singh H. Electronic Detection of Delayed Test Result Follow-Up in Patients with Hypothyroidism. J Gen Intern Med 2017; 32:753-759. [PMID: 28138875 PMCID: PMC5481223 DOI: 10.1007/s11606-017-3988-z] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/23/2016] [Revised: 12/20/2016] [Accepted: 01/05/2017] [Indexed: 02/05/2023]
Abstract
BACKGROUND Delays in following up abnormal test results are a common problem in outpatient settings. Surveillance systems that use trigger tools to identify delayed follow-up can help reduce missed opportunities in care. OBJECTIVE To develop and test an electronic health record (EHR)-based trigger algorithm to identify instances of delayed follow-up of abnormal thyroid-stimulating hormone (TSH) results in patients being treated for hypothyroidism. DESIGN We developed an algorithm using structured EHR data to identify patients with hypothyroidism who had delayed follow-up (>60 days) after an abnormal TSH. We then retrospectively applied the algorithm to a large EHR data warehouse within the Department of Veterans Affairs (VA), on patient records from two large VA networks for the period from January 1, 2011, to December 31, 2011. Identified records were reviewed to confirm the presence of delays in follow-up. KEY RESULTS During the study period, 645,555 patients were seen in the outpatient setting within the two networks. Of 293,554 patients with at least one TSH test result, the trigger identified 1250 patients on treatment for hypothyroidism with elevated TSH. Of these patients, 271 were flagged as potentially having delayed follow-up of their test result. Chart reviews confirmed delays in 163 of the 271 flagged patients (PPV = 60.1%). CONCLUSIONS An automated trigger algorithm applied to records in a large EHR data warehouse identified patients with hypothyroidism with potential delays in thyroid function test results follow-up. Future prospective application of the TSH trigger algorithm can be used by clinical teams as a surveillance and quality improvement technique to monitor and improve follow-up.
Collapse
Affiliation(s)
- Ashley N D Meyer
- Houston VA Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey VA Medical Center and Department of Medicine, Baylor College of Medicine, Houston, TX, USA.
| | - Daniel R Murphy
- Houston VA Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey VA Medical Center and Department of Medicine, Baylor College of Medicine, Houston, TX, USA
| | - Aymer Al-Mutairi
- Department of Family & Community Medicine, Baylor College of Medicine, Houston, TX, USA
| | - Dean F Sittig
- School of Biomedical Informatics and UT-Memorial Hermann Center for Healthcare Quality and Safety, The University of Texas Health Science Center at Houston, Houston, TX, USA
| | - Li Wei
- Houston VA Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey VA Medical Center and Department of Medicine, Baylor College of Medicine, Houston, TX, USA
| | - Elise Russo
- Houston VA Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey VA Medical Center and Department of Medicine, Baylor College of Medicine, Houston, TX, USA
| | - Hardeep Singh
- Houston VA Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey VA Medical Center and Department of Medicine, Baylor College of Medicine, Houston, TX, USA
| |
Collapse
|
24
|
Henriksen K, Dymek C, Harrison MI, Brady PJ, Arnold SB. Challenges and opportunities from the Agency for Healthcare Research and Quality (AHRQ) research summit on improving diagnosis: a proceedings review. Diagnosis (Berl) 2017. [DOI: 10.1515/dx-2017-0016] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
AbstractBackground:TheContent:The goals of the summit were to learn from the insights of participants; examine issues associated with definitions of diagnostic error and gaps in the evidence base; explore clinician and patient perspectives; gain a better understanding of data and measurement, health information technology, and organizational factors that impact the diagnostic process; and identify potential future directions for research.Summary and outlook:Plenary sessions focused on the state of the new diagnostic safety discipline followed by breakout sessions on the use of data and measurement, health information technology, and the role of organizational factors. The proceedings review captures many of the key challenges and areas deserving further research, revealing stimulating yet complex issues.
Collapse
|
25
|
McDonald KM, Su G, Lisker S, Patterson ES, Sarkar U. Implementation science for ambulatory care safety: a novel method to develop context-sensitive interventions to reduce quality gaps in monitoring high-risk patients. Implement Sci 2017; 12:79. [PMID: 28646886 PMCID: PMC5483297 DOI: 10.1186/s13012-017-0609-5] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2017] [Accepted: 06/13/2017] [Indexed: 11/15/2022] Open
Abstract
BACKGROUND Missed evidence-based monitoring in high-risk conditions (e.g., cancer) leads to delayed diagnosis. Current technological solutions fail to close this safety gap. In response, we aim to demonstrate a novel method to identify common vulnerabilities across clinics and generate attributes for context-flexible population-level monitoring solutions for widespread implementation to improve quality. METHODS Based on interviews with staff in otolaryngology, pulmonary, urology, breast, and gastroenterology clinics at a large urban publicly funded health system, we applied journey mapping to co-develop a visual representation of how patients are monitored for high-risk conditions. Using a National Academies framework and context-sensitivity theory, we identified common systems vulnerabilities and developed preliminary concepts for improving the robustness for monitoring patients with high-risk conditions ("design seeds" for potential solutions). Finally, we conducted a face validity and prioritization assessment of the design seeds with the original interviewees. RESULTS We identified five high-risk situations for potentially consequential diagnostic delays arising from suboptimal patient monitoring. All situations related to detection of cancer (head and neck, lung, prostate, breast, and colorectal). With clinic participants we created 5 journey maps, each representing specialty clinic workflow directed at evidence-based monitoring. System vulnerabilities common to the different clinics included challenges with: data systems, communications handoffs, population-level tracking, and patient activities. Clinic staff ranked 13 design seeds (e.g., keep patient list up to date, use triggered notifications) addressing these vulnerabilities. Each design seed has unique evaluation criteria for the usefulness of potential solutions developed from the seed. CONCLUSIONS We identified and ranked 13 design seeds that characterize situations that clinicians described 'wake them up at night', and thus could reduce their anxiety, save time, and improve monitoring of high-risk patients. We anticipate that the design seed approach promotes robust and context-sensitive solutions to safety and quality problems because it provides a human-centered link between the experienced problem and various solutions that can be tested for viability. The study also demonstrates a novel integration of industrial and human factors methods (journey mapping, process tracing and design seeds) linked to implementation theory for use in designing interventions that anticipate and reduce implementation challenges.
Collapse
Affiliation(s)
- Kathryn M. McDonald
- University of California Berkeley, School of Public Health, 50 University Hall, Berkeley, 94720 CA USA
- Stanford University, Center for Health Policy/Center for Primary Care and Outcomes Research, 117 Encina Commons, Stanford, 94305 CA USA
| | - George Su
- Department of Medicine, School of Medicine, University of California San Francisco, 1001 Potrero Avenue, San Francisco, 94110 CA USA
| | - Sarah Lisker
- Department of Medicine, School of Medicine, University of California San Francisco, 1001 Potrero Avenue, San Francisco, 94110 CA USA
| | - Emily S. Patterson
- Ohio State University, College of Medicine, School of Health and Rehabilitation Sciences, Division of Health Information Management and Systems, 453 W 10th Ave, Columbus, 43210 OH USA
| | - Urmimala Sarkar
- Department of Medicine, School of Medicine, University of California San Francisco, 1001 Potrero Avenue, San Francisco, 94110 CA USA
| |
Collapse
|
26
|
Singh H, Schiff GD, Graber ML, Onakpoya I, Thompson MJ. The global burden of diagnostic errors in primary care. BMJ Qual Saf 2017; 26:484-494. [PMID: 27530239 PMCID: PMC5502242 DOI: 10.1136/bmjqs-2016-005401] [Citation(s) in RCA: 182] [Impact Index Per Article: 26.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2016] [Revised: 06/15/2016] [Accepted: 07/13/2016] [Indexed: 12/20/2022]
Abstract
Diagnosis is one of the most important tasks performed by primary care physicians. The World Health Organization (WHO) recently prioritized patient safety areas in primary care, and included diagnostic errors as a high-priority problem. In addition, a recent report from the Institute of Medicine in the USA, 'Improving Diagnosis in Health Care', concluded that most people will likely experience a diagnostic error in their lifetime. In this narrative review, we discuss the global significance, burden and contributory factors related to diagnostic errors in primary care. We synthesize available literature to discuss the types of presenting symptoms and conditions most commonly affected. We then summarize interventions based on available data and suggest next steps to reduce the global burden of diagnostic errors. Research suggests that we are unlikely to find a 'magic bullet' and confirms the need for a multifaceted approach to understand and address the many systems and cognitive issues involved in diagnostic error. Because errors involve many common conditions and are prevalent across all countries, the WHO's leadership at a global level will be instrumental to address the problem. Based on our review, we recommend that the WHO consider bringing together primary care leaders, practicing frontline clinicians, safety experts, policymakers, the health IT community, medical education and accreditation organizations, researchers from multiple disciplines, patient advocates, and funding bodies among others, to address the many common challenges and opportunities to reduce diagnostic error. This could lead to prioritization of practice changes needed to improve primary care as well as setting research priorities for intervention development to reduce diagnostic error.
Collapse
Affiliation(s)
- Hardeep Singh
- Houston Veterans Affairs Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center and Baylor College of Medicine, Houston, Texas, USA
| | - Gordon D Schiff
- General Medicine, Brigham and Women's Hospital, Boston, Massachusetts, USA
| | - Mark L Graber
- RTI International, Research Triangle Park, North Carolina, USA
- SUNY Stony Brook School of Medicine, Stony Brook, New York, USA
| | - Igho Onakpoya
- Nuffield Department of Primary Care Health Sciences, University of Oxford, UK
| | | |
Collapse
|
27
|
Improving Diagnostic Safety in Primary Care by Unlocking Digital Data. Jt Comm J Qual Patient Saf 2017; 43:29-31. [PMID: 28334582 DOI: 10.1016/j.jcjq.2016.10.007] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/01/2023]
|
28
|
Murphy DR, Meyer AND, Vaghani V, Russo E, Sittig DF, Richards KA, Wei L, Wu L, Singh H. Application of Electronic Algorithms to Improve Diagnostic Evaluation for Bladder Cancer. Appl Clin Inform 2017; 8:279-290. [PMID: 28326433 PMCID: PMC5373770 DOI: 10.4338/aci-2016-10-ra-0176] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2016] [Accepted: 01/13/2017] [Indexed: 02/05/2023] Open
Abstract
BACKGROUND Strategies to ensure timely diagnostic evaluation of hematuria are needed to reduce delays in bladder cancer diagnosis. OBJECTIVE To evaluate the performance of electronic trigger algorithms to detect delays in hematuria follow-up. METHODS We developed a computerized trigger to detect delayed follow-up action on a urinalysis result with high-grade hematuria (>50 red blood cells/high powered field). The trigger scanned clinical data within a Department of Veterans Affairs (VA) national data repository to identify all patient records with hematuria, then excluded those where follow-up was unnecessary (e.g., terminal illness) or where typical follow-up action was detected (e.g., cystoscopy). We manually reviewed a randomly-selected sample of flagged records to confirm delays. We performed a similar analysis of records with hematuria that were marked as not delayed (non-triggered). We used review findings to calculate trigger performance. RESULTS Of 310,331 patients seen between 1/1/2012-12/31/2014, the trigger identified 5,857 patients who experienced high-grade hematuria, of which 495 experienced a delay. On manual review of 400 randomly-selected triggered records and 100 non-triggered records, the trigger achieved positive and negative predictive values of 58% and 97%, respectively. CONCLUSIONS Triggers offer a promising method to detect delays in care of patients with high-grade hematuria and warrant further evaluation in clinical practice as a means to reduce delays in bladder cancer diagnosis.
Collapse
Affiliation(s)
- Daniel R Murphy
- Daniel R. Murphy, MD MBA, Michael E. DeBakey Veterans Affairs Medical Center (MEDVAMC), Houston Center for Innovations in Quality, Effectiveness & Safety (IQuESt) (152), 2002 Holcombe Boulevard, Houston, TX 77030 USA, 713-440-4600 (o), 713-748-7359 (f),
| | | | | | | | | | | | | | | | | |
Collapse
|
29
|
Addition of the Fleischner Society Guidelines to Chest CT Examination Interpretive Reports Improves Adherence to Recommended Follow-up Care for Incidental Pulmonary Nodules. Acad Radiol 2017; 24:337-344. [PMID: 27793580 DOI: 10.1016/j.acra.2016.08.026] [Citation(s) in RCA: 32] [Impact Index Per Article: 4.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2016] [Revised: 08/25/2016] [Accepted: 08/29/2016] [Indexed: 12/21/2022]
Abstract
RATIONALE AND OBJECTIVES The study aimed to determine whether the addition of the Fleischner Society guidelines to chest computed tomography (CT) reports identifying incidental pulmonary nodules affects follow-up care. PATIENTS AND METHODS Beginning in 2008, a template containing the Fleischner Society guidelines was added at the interpreting radiologist's discretion to chest CT reports describing incidental solid pulmonary nodules at our institution. The records of all medical centers in Olmsted county were used to capture the complete medical history of local patients >35 years old diagnosed with a pulmonary nodule from April 1, 2008 to October 1, 2011. Patients with a history of cancer or previously diagnosed nodule, or who died before follow-up, were excluded. Patients were categorized according to whether they did ("template group") or did not ("control group") have the template added. Nodule size and smoking history were used to determine recommended follow-up care. Differences in follow-up were compared between groups using Pearson's chi-square test. RESULTS A total of 510 patients (276 in the template group, 234 in the control group) were included in the study. Only 198 patients (39%) received their recommended follow-up care. Template group patients were significantly more likely to receive recommended follow-up care compared to control group patients (45% vs 31%, P = .0014). Most patients whose management did not adhere to Fleischner Society guidelines did not receive a recommended follow-up chest CT (210 out of 312, 67%). CONCLUSIONS The addition of the Fleischner Society guidelines to chest CT reports significantly increases the likelihood of receiving recommended follow-up care for patients with incidental pulmonary nodules. Additional education is needed to improve appropriate guideline utilization by radiologists and adherence by ordering providers.
Collapse
|
30
|
Murphy DR, Meyer AN, Bhise V, Russo E, Sittig DF, Wei L, Wu L, Singh H. Computerized Triggers of Big Data to Detect Delays in Follow-up of Chest Imaging Results. Chest 2016; 150:613-20. [PMID: 27178786 DOI: 10.1016/j.chest.2016.05.001] [Citation(s) in RCA: 25] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2015] [Revised: 04/14/2016] [Accepted: 05/02/2016] [Indexed: 02/08/2023] Open
|
31
|
Murphy DR, Wu L, Thomas EJ, Forjuoh SN, Meyer AND, Singh H. Electronic Trigger-Based Intervention to Reduce Delays in Diagnostic Evaluation for Cancer: A Cluster Randomized Controlled Trial. J Clin Oncol 2015; 33:3560-7. [PMID: 26304875 PMCID: PMC4622097 DOI: 10.1200/jco.2015.61.1301] [Citation(s) in RCA: 64] [Impact Index Per Article: 7.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022] Open
Abstract
PURPOSE We tested whether prospective use of electronic health record-based trigger algorithms to identify patients at risk of diagnostic delays could prevent delays in diagnostic evaluation for cancer. METHODS We performed a cluster randomized controlled trial of primary care providers (PCPs) at two sites to test whether triggers that prospectively identify patients with potential delays in diagnostic evaluation for lung, colorectal, or prostate cancer can reduce time to follow-up diagnostic evaluation. Intervention steps included queries of the electronic health record repository for patients with abnormal findings and lack of associated follow-up actions, manual review of triggered records, and communication of this information to PCPs via secure e-mail and, if needed, phone calls to ensure message receipt. We compared times to diagnostic evaluation and proportions of patients followed up between intervention and control cohorts based on final review at 7 months. RESULTS We recruited 72 PCPs (36 in the intervention group and 36 in the control group) and applied the trigger to all patients under their care from April 20, 2011, to July 19, 2012. Of 10,673 patients with abnormal findings, the trigger flagged 1,256 patients (11.8%) as high risk for delayed diagnostic evaluation. Times to diagnostic evaluation were significantly lower in intervention patients compared with control patients flagged by the colorectal trigger (median, 104 v 200 days, respectively; n = 557; P < .001) and prostate trigger (40% received evaluation at 144 v 192 days, respectively; n = 157; P < .001) but not the lung trigger (median, 65 v 93 days, respectively; n = 19; P = .59). More intervention patients than control patients received diagnostic evaluation by final review (73.4% v 52.2%, respectively; relative risk, 1.41; 95% CI, 1.25 to 1.58). CONCLUSION Electronic trigger-based interventions seem to be effective in reducing time to diagnostic evaluation of colorectal and prostate cancer as well as improving the proportion of patients who receive follow-up. Similar interventions could improve timeliness of diagnosis of other serious conditions.
Collapse
Affiliation(s)
- Daniel R Murphy
- Daniel R. Murphy, Louis Wu, Ashley N.D. Meyer, and Hardeep Singh, Houston Veterans Affairs Health Services Research and Development, Michael E. DeBakey Veterans Affairs Medical Center, and Section of Health Services Research, Baylor College of Medicine; Eric J. Thomas, University of Texas Houston Medical School and University of Texas Houston-Memorial Hermann Center for Healthcare Quality and Safety, Houston; and Samuel N. Forjuoh, Scott and White Healthcare, Texas A&M Health Science Center, College of Medicine, Temple, TX
| | - Louis Wu
- Daniel R. Murphy, Louis Wu, Ashley N.D. Meyer, and Hardeep Singh, Houston Veterans Affairs Health Services Research and Development, Michael E. DeBakey Veterans Affairs Medical Center, and Section of Health Services Research, Baylor College of Medicine; Eric J. Thomas, University of Texas Houston Medical School and University of Texas Houston-Memorial Hermann Center for Healthcare Quality and Safety, Houston; and Samuel N. Forjuoh, Scott and White Healthcare, Texas A&M Health Science Center, College of Medicine, Temple, TX
| | - Eric J Thomas
- Daniel R. Murphy, Louis Wu, Ashley N.D. Meyer, and Hardeep Singh, Houston Veterans Affairs Health Services Research and Development, Michael E. DeBakey Veterans Affairs Medical Center, and Section of Health Services Research, Baylor College of Medicine; Eric J. Thomas, University of Texas Houston Medical School and University of Texas Houston-Memorial Hermann Center for Healthcare Quality and Safety, Houston; and Samuel N. Forjuoh, Scott and White Healthcare, Texas A&M Health Science Center, College of Medicine, Temple, TX
| | - Samuel N Forjuoh
- Daniel R. Murphy, Louis Wu, Ashley N.D. Meyer, and Hardeep Singh, Houston Veterans Affairs Health Services Research and Development, Michael E. DeBakey Veterans Affairs Medical Center, and Section of Health Services Research, Baylor College of Medicine; Eric J. Thomas, University of Texas Houston Medical School and University of Texas Houston-Memorial Hermann Center for Healthcare Quality and Safety, Houston; and Samuel N. Forjuoh, Scott and White Healthcare, Texas A&M Health Science Center, College of Medicine, Temple, TX
| | - Ashley N D Meyer
- Daniel R. Murphy, Louis Wu, Ashley N.D. Meyer, and Hardeep Singh, Houston Veterans Affairs Health Services Research and Development, Michael E. DeBakey Veterans Affairs Medical Center, and Section of Health Services Research, Baylor College of Medicine; Eric J. Thomas, University of Texas Houston Medical School and University of Texas Houston-Memorial Hermann Center for Healthcare Quality and Safety, Houston; and Samuel N. Forjuoh, Scott and White Healthcare, Texas A&M Health Science Center, College of Medicine, Temple, TX
| | - Hardeep Singh
- Daniel R. Murphy, Louis Wu, Ashley N.D. Meyer, and Hardeep Singh, Houston Veterans Affairs Health Services Research and Development, Michael E. DeBakey Veterans Affairs Medical Center, and Section of Health Services Research, Baylor College of Medicine; Eric J. Thomas, University of Texas Houston Medical School and University of Texas Houston-Memorial Hermann Center for Healthcare Quality and Safety, Houston; and Samuel N. Forjuoh, Scott and White Healthcare, Texas A&M Health Science Center, College of Medicine, Temple, TX.
| |
Collapse
|