Understanding the management of electronic test result notifications in the outpatient setting

Management of the Electronic Health Record Inbox: Results From a National Survey of Internal Medicine Program Directors

Background Internal medicine (IM) resident physicians spend a considerable amount of time managing their inbox as part of their longitudinal continuity clinic experience. There are no standardized guidelines for how programs should train, monitor, or supervise residents in this type of patient care. Objective To understand how IM residency programs educate, monitor, and supervise resident electronic health record (EHR) inbox management as part of their longitudinal continuity clinic and determine whether patient safety events have occurred due to EHR inbox-related patient care decisions made by unsupervised resident physicians. Methods In August 2021, 439 program directors at accredited US IM residency programs who were members of the Association of Program Directors in Internal Medicine (APDIM) were asked 12 questions developed by the study authors and APDIM survey committee members regarding resident EHR inbox management as part of the annual APDIM survey. Results Two hundred and sixty-seven (61%) PDs responded. The majority (224 of 267, 84%) of programs provided guidelines for expected message response times; less than half (115, 43%) monitored timeliness metrics. Only half (135; 51%) of programs required faculty supervision of inbox messages for all residents; 28% (76) did not require supervision for any residents. Twenty-one percent of PDs (56) reported awareness of a patient safety event occurring due to an unsupervised resident inbox-related patient care decision. Conclusions Substantial variability exists in how IM residency programs train, monitor, supervise, and provide coverage for resident inbox work. Program directors are aware of patient safety events resulting from unsupervised resident inbox management.

Why Test Results Are Still Getting "Lost" to Follow-up: a Qualitative Study of Implementation Gaps

BACKGROUND

Lack of timely follow-up of abnormal test results is common and has been implicated in missed or delayed diagnosis, resulting in potential for patient harm.

OBJECTIVE

As part of a larger project to implement change strategies to improve follow-up of diagnostic test results, this study sought to identify specifically where implementation gaps exist, as well as possible solutions identified by front-line staff.

DESIGN

We used a semi-structured interview guide to collect qualitative data from Veterans Affairs (VA) facility staff who had experience with test results management and patient safety.

SETTING

Twelve VA facilities across the USA.

PARTICIPANTS

Facility staff members (n = 27), including clinicians, lab and imaging professionals, nursing staff, patient safety professionals, and leadership.

APPROACH

We conducted a content analysis of interview transcripts to identify perceived barriers and high-risk areas for effective test result management, as well as recommendations for improvement.

RESULTS

We identified seven themes to guide further development of interventions to improve test result follow-up. Themes related to trainees, incidental findings, tracking systems for electronic health record notifications, outdated contact information, referrals, backup or covering providers, and responsibility for test results pending at discharge. Participants provided recommendations for improvement within each theme.

CONCLUSIONS

Perceived barriers and recommendations for improving test result follow-up often reflected previously known problems and their corresponding solutions, which have not been consistently implemented in practice. Better policy solutions and improvement methods, such as quality improvement collaboratives, may bridge the implementation gaps between knowledge and practice.

When no news is bad news: Improving diagnostic testing communication through patient engagement

Importance

Up to 17% of diagnostic test results are missed, lost or ignored despite conventional fixes such as electronic physician reminders – naïvely, patients assume: ‘No-News-is-Good-News’. These lapses can result in poor outcomes, complications, and even death. In response, Centers for Medicare and Medicaid Services (CMS)-led physician quality reporting system measure#265 emphasizes prevention. This study aims to improve the timely review of results through increasing patient engagement.

Design and Participants

Ninety patients undergoing diagnostic testing were included in this Internal Review Board (IRB)-approved study. Two groups, group-A (patients with medical chart access through our EHR, n = 40); and group-B (controls, n = 50) were included. Group-A was reminded via written AVS and EHR portal messages to ask about their test results at their next appointment. Controls were sent no reminders, mimicking the status- quo.

Main Outcomes

At subsequent visits whether patients ‘asked’ or ‘did not ask’ about their results was recorded and analyzed. Study group participants were also surveyed on their preferences for reminder communication.

Results

Patients that were sent reminders were up to twenty times more likely to ask their provider regarding their test results than the control group ( p < 0.0001). Eighty-one percent indicated that the reminders were helpful with 90% indicating they were ‘necessary’. Neither gender nor age seemed predictive factors of patient engagement.

Conclusions and Relevance

This pilot study demonstrates that engaging patients in their own care through already-existing tools (AVS, EHR portal messages) improves patient-physician communication, and could lead to lower rates of missed diagnostic tests.

Delayed or failure to follow-up abnormal breast cancer screening mammograms in primary care: a systematic review

Background

Successful breast cancer screening relies on timely follow-up of abnormal mammograms. Delayed or failure to follow-up abnormal mammograms undermines the potential benefits of screening and is associated with poorer outcomes. However, a comprehensive review of inadequate follow-up of abnormal mammograms in primary care has not previously been reported in the literature. This review could identify modifiable factors that influence follow-up, which if addressed, may lead to improved follow-up and patient outcomes.

Methods

A systematic literature review to determine the extent of inadequate follow-up of abnormal screening mammograms in primary care and identify factors impacting on follow-up was conducted. Relevant studies published between 1 January, 1990 and 29 October, 2020 were identified by searching MEDLINE®, Embase, CINAHL® and Cochrane Library, including reference and citation checking. Joanna Briggs Institute Critical Appraisal Checklists were used to assess the risk of bias of included studies according to study design.

Results

Eighteen publications reporting on 17 studies met inclusion criteria; 16 quantitative and two qualitative studies. All studies were conducted in the United States, except one study from the Netherlands. Failure to follow-up abnormal screening mammograms within 3 and at 6 months ranged from 7.2–33% and 27.3–71.6%, respectively. Women of ethnic minority and lower education attainment were more likely to have inadequate follow-up. Factors influencing follow-up included physician-patient miscommunication, information overload created by automated alerts, the absence of adequate retrieval systems to access patient’s results and a lack of coordination of patient records. Logistical barriers to follow-up included inconvenient clinic hours and inconsistent primary care providers. Patient navigation and case management with increased patient education and counselling by physicians was demonstrated to improve follow-up.

Conclusions

Follow-up of abnormal mammograms in primary care is suboptimal. However, interventions addressing amendable factors that negatively impact on follow-up have the potential to improve follow-up, especially for populations of women at risk of inadequate follow-up.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12885-021-08100-3.

The Development and Piloting of the Ambulatory Electronic Health Record Evaluation Tool: Lessons Learned

BACKGROUND

Substantial research has been performed about the impact of computerized physician order entry on medication safety in the inpatient setting; however, relatively little has been done in ambulatory care, where most medications are prescribed.

OBJECTIVE

To outline the development and piloting process of the Ambulatory Electronic Health Record (EHR) Evaluation Tool and to report the quantitative and qualitative results from the pilot.

METHODS

The Ambulatory EHR Evaluation Tool closely mirrors the inpatient version of the tool, which is administered by The Leapfrog Group. The tool was piloted with seven clinics in the United States, each using a different EHR. The tool consists of a medication safety test and a medication reconciliation module. For the medication test, clinics entered test patients and associated test orders into their EHR and recorded any decision support they received. An overall percentage score of unsafe orders detected, and order category scores were provided to clinics. For the medication reconciliation module, clinics demonstrated how their EHR electronically detected discrepancies between two medication lists.

RESULTS

For the medication safety test, the clinics correctly alerted on 54.6% of unsafe medication orders. Clinics scored highest in the drug allergy (100%) and drug-drug interaction (89.3%) categories. Lower scoring categories included drug age (39.3%) and therapeutic duplication (39.3%). None of the clinics alerted for the drug laboratory or drug monitoring orders. In the medication reconciliation module, three (42.8%) clinics had an EHR-based medication reconciliation function; however, only one of those clinics could demonstrate it during the pilot.

CONCLUSION

Clinics struggled in areas of advanced decision support such as drug age, drug laboratory, and drub monitoring. Most clinics did not have an EHR-based medication reconciliation function and this process was dependent on accessing patients' medication lists. Wider use of this tool could improve outpatient medication safety and can inform vendors about areas of improvement.

The impact of health information technology on the management and follow-up of test results - a systematic review

OBJECTIVE

To investigate the impact of health information technology (IT) systems on clinicians' work practices and patient engagement in the management and follow-up of test results.

MATERIALS AND METHODS

A search for studies reporting health IT systems and clinician test results management was conducted in the following databases: MEDLINE, EMBASE, CINAHL, Web of Science, ScienceDirect, ProQuest, and Scopus from January 1999 to June 2018. Test results follow-up was defined as provider follow-up of results for tests that were sent to the laboratory and radiology services for processing or analysis.

RESULTS

There are some findings from controlled studies showing that health IT can improve the proportion of tests followed-up (15 percentage point change) and increase physician awareness of test results that require action (24-28 percentage point change). Taken as whole, however, the evidence of the impact of health IT on test result management and follow-up is not strong.

DISCUSSION

The development of safe and effective test results management IT systems should pivot on several axes. These axes include 1) patient-centerd engagement (involving shared, timely, and meaningful information); 2) diagnostic processes (that involve the integration of multiple people and different clinical settings across the health care spectrum); and 3) organizational communications (the myriad of multi- transactional processes requiring feedback, iteration, and confirmation) that contribute to the patient care process.

CONCLUSION

Existing evidence indicates that health IT in and of itself does not (and most likely cannot) provide a complete solution to issues related to test results management and follow-up.

Application of Human Factors Methods to Understand Missed Follow-up of Abnormal Test Results

OBJECTIVE

This study demonstrates application of human factors methods for understanding causes for lack of timely follow-up of abnormal test results ("missed results") in outpatient settings.

METHODS

We identified 30 cases of missed test results by querying electronic health record data, developed a critical decision method (CDM)-based interview guide to understand decision-making processes, and interviewed physicians who ordered these tests. We analyzed transcribed responses using a contextual inquiry (CI)-based methodology to identify contextual factors contributing to missed results. We then developed a CI-based flow model and conducted a fault tree analysis (FTA) to identify hierarchical relationships between factors that delayed action.

RESULTS

The flow model highlighted barriers in information flow and decision making, and the hierarchical model identified relationships between contributing factors for delayed action. Key findings including underdeveloped methods to track follow-up, as well as mismatches, in communication channels, timeframes, and expectations between patients and physicians.

CONCLUSION

This case report illustrates how human factors-based approaches can enable analysis of contributing factors that lead to missed results, thus informing development of preventive strategies to address them.

Variation in electronic test results management and its implications for patient safety: A multisite investigation

Objective

The management and follow-up of diagnostic test results is a major patient safety concern. The aim of this qualitative study was to explore how clinicians manage test results on an everyday basis (work-as-done) in a health information technology–enabled emergency department setting. The objectives were to identify (1) variations in work-as-done in test results management and (2) the strategies clinicians use to ensure optimal management of diagnostic test results.

Materials and Methods

Qualitative interviews (n = 26) and field observations were conducted across 3 Australian emergency departments. Interview data coded for results management (ie, tracking, acknowledgment, and follow-up), and artifacts, were reviewed to identify variations in descriptions of work-as-done. Thematic analysis was performed to identify common themes.

Results

Despite using the same test result management application, there were variations in how the system was used. We identified 5 themes relating to electronic test results management: (1) tracking test results, (2) use and understanding of system functionality, (3) visibility of result actions and acknowledgment, (4) results inbox use, and (5) challenges associated with the absence of an inbox for results notifications for advanced practice nurses.

Discussion

Our findings highlight that variations in work-as-done can function to overcome perceived impediments to managing test results in a HIT-enabled environment and thus identify potential risks in the process. By illuminating work-as-done, we identified strategies clinicians use to enhance test result management including paper-based manual processes, cognitive reminders, and adaptive use of electronic medical record functionality.

Conclusions

Test results tracking and follow-up is a priority area in need of health information technology development and training to improve team-based collaboration/communication of results follow-up and diagnostic safety.

Does health information technology improve acknowledgement of radiology results for discharged Emergency Department patients? A before and after study

Background

The inadequate follow-up of test results is a key patient safety concern, carrying severe consequences for care outcomes. Patients discharged from the emergency department are at particular risk of having test results pending at discharge due to their short lengths of stay, with many hospitals acknowledging that they do not have reliable systems for managing such results. Health information technology hold the potential to reducing errors in the test result management process. This study aimed to measure changes in the proportion of acknowledged radiology reports pre and post introduction of an electronic result acknowledgement system and to determine the proportion of reports with abnormal results, including clinically significant abnormal results requiring follow-up action.

Methods

A before and after study was conducted in the emergency department of a 450-bed metropolitan teaching hospital in Australia. All radiology reports for discharged patients for a one-month period before and after implementation of the electronic result acknowledgement system were reviewed to determine; i) those that reported abnormal results; ii) evidence of test result acknowledgement. All unacknowledged radiology results with an abnormal finding were assessed by an independent panel of two senior emergency physicians for clinical significance.

Results

Of 1654 radiology reports in the pre-implementation period 70.6% (n = 1167) had documented evidence of acknowledgement by a clinician. For reports with abnormal results, 71.6% (n = 396) were acknowledged. Of 157 unacknowledged abnormal radiology reports reviewed by an independent emergency physician panel, 34.4% (n = 54) were identified as clinically significant and 50% of these (n = 27) were deemed to carry a moderate likelihood of patient morbidity if not followed up. Electronic acknowledgement occurred for all radiology reports in the post period (n = 1423), representing a 30.4% (95% CI: 28.1–32.6%) increase in acknowledgement rate, and an increase of 28.4% (95% CI: 24.6–32.2%) for abnormal radiology results.

Conclusions

The findings of this study demonstrate the potential of health information technology to improve the safety and effectiveness of the diagnostic process by increasing the rate of follow up of results pending at hospital discharge.

Evaluating the alert appropriateness of clinical decision support systems in supporting clinical workflow

The overwhelming number of medication alerts generated by clinical decision support systems (CDSS) has led to inappropriate alert overrides, which may lead to unintended patient harm. This review highlights the factors affecting the alert appropriateness of CDSS and barriers to the fit of CDSS alert with clinical workflow. A literature review was conducted to identify features and functions pertinent to CDSS alert appropriateness using the five rights of CDSS. Moreover, a process improvement method, namely, Lean, was used as a tool to optimise clinical workflows, and the appropriate design for CDSS alert using a human automation interaction (HAI) model was recommended. Evaluating the appropriateness of CDSS alert and its impact on workflow provided insights into how alerts can be designed and triggered effectively to support clinical workflow. The application of Lean methods and tools to analyse alert efficiencies in supporting workflow in this study provides an in-depth understanding of alert-workflow fit problems and their root cause, which is required for improving CDSS design. The application of the HAI model is recommended in the design of CDSS alerts to support various levels and stages of alert automations, namely, information acquisition and analysis, decision action and action implementation.

Follow-up of incidental pulmonary nodules and association with mortality in a safety-net cohort

Background Though incidental pulmonary nodules are common, rates of guideline-recommended surveillance and associations between surveillance and mortality are unclear. In this study, we describe adherence (categorized as complete, partial, late and none) to guideline-recommended surveillance among patients with incidental 5-8 mm pulmonary nodules and assess associations between adherence and mortality. Methods This was a retrospective cohort study of 551 patients (≥35 years) with incidental pulmonary nodules conducted from September 1, 2008 to December 31, 2016, in an integrated safety-net health network. Results Of the 551 patients, 156 (28%) had complete, 87 (16%) had partial, 93 (17%) had late and 215 (39%) had no documented surveillance. Patients were followed for a median of 5.2 years [interquartile range (IQR), 3.6-6.7 years] and 82 (15%) died during follow-up. Adjusted all-cause mortality rates ranged from 2.24 [95% confidence interval (CI), 1.24-3.25] deaths per 100 person-years for complete follow-up to 3.30 (95% CI, 2.36-4.23) for no follow-up. In multivariable models, there were no statistically significant associations between the levels of surveillance and mortality (p > 0.16 for each comparison with complete surveillance). Compared with complete surveillance, adjusted mortality rates were non-significantly increased by 0.45 deaths per 100 person-years (95% CI, -1.10 to 2.01) for partial, 0.55 (95% CI, -1.08 to 2.17) for late and 1.05 (95% CI, -0.35 to 2.45) for no surveillance. Conclusions Although guideline-recommended surveillance of small incidental pulmonary nodules was incomplete or absent in most patients, gaps in surveillance were not associated with statistically significant increases in mortality in a safety-net population.

A comparative study of the utilisation of an electronic test-result management system in emergency and intensive care settings

The purpose of this qualitative study was to identify differences in the utilisation of an electronic medical record test-result management system between two acute care departments. Field observations (130 min) and semi-structured interviews (n = 24) were conducted in the Intensive Care Unit and Emergency Department of an Australian hospital. Work processes identified from audio transcripts were modelled using business process modelling. Comparison of the Emergency Department and Intensive Care Unit identified the following: (1) test ordering variations according to clinical roles, (2) differences in the use of electronic medical record functionality according to specific demands of the clinical environment and (3) the non-linear components of the test-result management process. Variations were identified in the number of process decisions, external collaborations and temporal process workflows. Modelling the business processes, collaboration and communication needs of individual clinical environments can aid in enhancing the quality and appositeness of health information technology interventions and thus contribute to improving patient safety. Future health information technology interventions/evaluations aimed at improving the safety of test-result management processes need to address both the nuances of the clinical environment and accommodate the individual work practices of clinicians within that environment.

Factors Associated With Optimal Follow-up in Women With BI-RADS 3 Breast Findings

OBJECTIVE

Assess rate of and factors associated with optimal follow-up in patients with BI-RADS 3 breast findings.

METHODS

This Institutional Review Board-approved, retrospective cohort study, performed at an academic medical center, included all women undergoing breast imaging (ultrasound and mammography) in 2016. Index reports for unique patients with an assessment of BI-RADS 3 (retrieved via natural language processing) comprised the study population. Patient-specific and provider-related features were extracted from the Research Data Warehouse. The Institutional Cancer Registry identified patients diagnosed with breast cancer. Optimal follow-up rate was calculated as patients with follow-up imaging on the same breast 3 to 9 months from the index examination among patients with BI-RADS 3 assessments. Univariate analysis and multivariable logistic regression determined features associated with optimal follow-up. Malignancy rate and time to malignancy detection were recorded.

RESULTS

Among 93,685 breast imaging examinations, 64,771 were from unique patients of which 2,967 had BI-RADS 3 findings (4.6%). Excluding patients with off-site index examinations and those with another breast examination <3 months from the index, 1,125 of 1,511 patients (74%) had optimal follow-up. In univariate and multivariable analysis, prior breast cancer was associated with optimal follow-up; younger age, Hispanic ethnicity, divorced status, and lack of insurance were associated with not having optimal follow-up. Malignancy rate was 0.86%, and mean time to detection was 330 days.

DISCUSSION

Follow-up of BI-RADS 3 breast imaging findings is optimal in only 74% of women. Further interventions to promote follow-up should target younger, unmarried women, those with Hispanic ethnicity, and women without history of breast cancer and without insurance coverage.

An Exploration of Barriers, Facilitators, and Suggestions for Improving Electronic Health Record Inbox-Related Usability: A Qualitative Analysis

IMPORTANCE

Managing messages in the electronic health record (EHR) inbox consumes substantial amounts of physician time. Certain factors associated with inbox management, such as poor usability and excessive and unnecessary inbox messages, have been associated with physician burnout. Additionally, inbox design, usability, and workflows are associated with physicians' situational awareness (ie, perception, comprehension, and projection of clinical status) and efficiency of processing EHR inbox messages. Understanding factors associated with inbox usability could improve future EHR inbox designs and workflows, thus reducing risk of burnout while improving patient safety.

OBJECTIVE

To determine barriers, facilitators, and suggestions associated with EHR inbox-related usability.

DESIGN, SETTING, AND PARTICIPANTS

This qualitative study included cognitive walkthroughs of EHR inbox management with 25 physicians (17 primary care physicians and 8 specialists) at 6 large health care organizations using 4 different EHR systems between May 6, 2015, and September 19, 2016. While processing EHR inbox messages, participants identified facilitators and barriers associated with EHR inbox situational awareness and processing efficiency and potential interventions to address such barriers. A qualitative analysis was performed on transcribed recordings using an inductive thematic approach with an 8-dimension sociotechnical model as a theoretical lens from May 6, 2015, to August 15, 2019.

RESULTS

The cognitive walkthroughs identified 60 barriers, 32 facilitators, and 28 suggestions for improving the EHR inbox. Emergent data fit within 5 major themes: message processing complexity, inbox interface design, cognitive load, team communication, and inbox message content. Within these themes, similar barriers were identified across sites, such as poor usability due the high numbers of clicks needed to accomplish actions. In certain instances, an identified facilitator at one site provided the exact solution needed to address a barrier identified at another site.

CONCLUSIONS AND RELEVANCE

This qualitative study found that usability of the EHR inbox is often suboptimal and variable across sites, suggesting lack of shared best practices related to information management. Implementation of optimized design features and workflows will require EHR developers and health care organizations to collectively share this responsibility. Development of regional or national consortia to support collaborative sharing and implementation of EHR system best practices across EHR developers and health care organizations could also improve safety and efficiency and reduce physician burnout.

Practicing Clinicians' Recommendations to Reduce Burden from the Electronic Health Record Inbox: a Mixed-Methods Study

BACKGROUND

Workload from electronic health record (EHR) inbox notifications leads to information overload and contributes to job dissatisfaction and physician burnout. Better understanding of physicians' inbox requirements and workflows could optimize inbox designs, enhance efficiency, and reduce safety risks from information overload.

DESIGN

We conducted a mixed-methods study to identify strategies to enhance EHR inbox design and workflow. First, we performed a secondary analysis of national survey data of all Department of Veterans Affairs (VA) primary care practitioners (PCP) to identify major themes in responses to a free-text question soliciting suggestions to improve EHR inbox design and workflows. We then conducted expert interviews of clinicians at five health care systems (1 VA and 4 non-VA settings using 4 different EHRs) to understand existing optimal strategies to improve efficiency and situational awareness related to EHR inbox use. Themes from survey data were cross-validated with interview findings.

RESULTS

We analyzed responses from 2104 PCPs who completed the free-text inbox question (of 5001 PCPs who responded to survey) and used an inductive approach to identify five themes: (1) Inbox notification content should be actionable for patient care and relevant to recipient clinician, (2) Inboxes should reduce risk of losing messages, (3) Inbox functionality should be optimized to improve efficiency of processing notifications, (4) Team support should be leveraged to help with EHR inbox notification burden, (5) Sufficient time should be provided to all clinicians to process EHR inbox notifications. We subsequently interviewed 15 VA and non-VA clinicians and identified 11 unique strategies, each corresponding directly with one of these five themes.

CONCLUSION

Feedback from practicing end-user clinicians provides robust evidence to improve content and design of the EHR inbox and related clinical workflows and organizational policies. Several strategies we identified could improve clinicians' EHR efficiency and satisfaction as well as empower them to work with their local administrators, health IT personnel, and EHR developers to improve these systems.

Variation in Lipid-Lowering Therapy Use in Patients With Low-Density Lipoprotein Cholesterol ≥190 mg/dL: Insights From the National Cardiovascular Data Registry-Practice Innovation and Clinical Excellence Registry

BACKGROUND

Patients with low-density lipoprotein cholesterol (LDL-C) ≥190 mg/dL are at high risk of atherosclerotic cardiovascular disease events. Treatment guidelines recommend intensive treatment in these patients. Variation in the use of lipid-lowering therapies (LLTs) in these patients in a national sample of cardiology practices is not known.

METHODS AND RESULTS

Using data from the American College of Cardiology National Cardiovascular Data Registry-Practice Innovation and Clinical Excellence registry, we assessed the proportion of patients with LDL-C ≥190 mg/dL (n=49 447) receiving statin, high-intensity statin, LLT associated with ≥50% LDL-C lowering, ezetimibe, or a PCSK9 (proprotein convertase subtilisin/kexin type 9) inhibitor between January 2013 and December 2016. We assessed practice-level rates and variation in LLT use using median rate ratio (MRR) adjusted for patient and practice characteristics. MRRs represent the likelihood that 2 random practices would differ in treatment of identical patients with LDL-C ≥190 mg/dL. The proportion of patients receiving a statin, high-intensity statin, LLT associated with ≥50% LDL-C reduction, ezetimibe, or PCSK9 inhibitor were 58.5%, 31.9%, 34.6%, 8.5%, and 1.5%, respectively. Median practice-level rates and adjusted MRR for statin (56% [interquartile range, 47.3%-64.8%]; MRR, 1.20 [95% confidence interval [CI], 1.17-1.23]), high-intensity statin (30.2% [interquartile range, 12.1%-41.1%]; MRR, 2.31 [95% CI, 2.12-2.51]), LLT with ≥50% LDL-C lowering (31.8% [interquartile range, 15.3%-45.5%]; MRR, 2.12 [95% CI, 1.95-2.28]), ezetimibe (5.8% [interquartile range, 2.8%-9.8%]; MRR, 2.42 [95% CI, 2.21-2.63]), and PCSK9 inhibitors (0.16% [interquartile range, 0%-1.9%]; MRR, 2.38 [95% CI, 2.04-2.72]) indicated significant gaps and >200% variation in receipt of several of these medications for patients across practices. Among those without concomitant atherosclerotic cardiovascular disease, even larger treatment gaps were noted (proportion of patients on a statin, high-intensity statin, LLT with ≥50% LDL-C reduction, ezetimibe, or PCSK9 inhibitor were 50.8%, 25.25%, 26.8%, 4.9%, and 0.74%, respectively).

CONCLUSIONS

Evidence-based LLT use remains low among patients with elevated LDL-C with significant variation in care. System-level interventions are needed to address these gaps and reduce variation in care of these high-risk patients.

Association of the Usability of Electronic Health Records With Cognitive Workload and Performance Levels Among Physicians

IMPORTANCE

Current electronic health record (EHR) user interfaces are suboptimally designed and may be associated with excess cognitive workload and poor performance.

OBJECTIVE

To assess the association between the usability of an EHR system for the management of abnormal test results and physicians' cognitive workload and performance levels.

DESIGN, SETTING, AND PARTICIPANTS

This quality improvement study was conducted in a simulated EHR environment. From April 1, 2016, to December 23, 2016, residents and fellows from a large academic institution were enrolled and allocated to use either a baseline EHR (n = 20) or an enhanced EHR (n = 18). Data analyses were conducted from January 9, 2017, to March 30, 2018.

INTERVENTIONS

The EHR with enhanced usability segregated in a dedicated folder previously identified critical test results for patients who did not appear for a scheduled follow-up evaluation and provided policy-based decision support instructions for next steps. The baseline EHR displayed all patients with abnormal or critical test results in a general folder and provided no decision support instructions for next steps.

MAIN OUTCOMES AND MEASURES

Cognitive workload was quantified subjectively using NASA-Task Load Index and physiologically using blink rates. Performance was quantified according to the percentage of appropriately managed abnormal test results.

RESULTS

Of the 38 participants, 25 (66%) were female. The 20 participants allocated to the baseline EHR compared with the 18 allocated to the enhanced EHR demonstrated statistically significantly higher cognitive workload as quantified by blink rate (mean [SD] blinks per minute, 16 [9] vs 24 [7]; blink rate, -8 [95% CI, -13 to -2]; P = .01). The baseline group showed statistically significantly poorer performance compared with the enhanced group who appropriately managed 16% more abnormal test results (mean [SD] performance, 68% [19%] vs 98% [18%]; performance rate, -30% [95% CI, -40% to -20%]; P < .001).

CONCLUSIONS AND RELEVANCE

Relatively basic usability enhancements to the EHR system appear to be associated with better physician cognitive workload and performance; this finding suggests that next-generation systems should strip away non-value-added EHR interactions, which may help physicians eliminate the need to develop their own suboptimal workflows.

Patient groups, clinicians and healthcare professionals agree - all test results need to be seen, understood and followed up

Background Diagnostic testing provides integral information for the prevention, diagnosis, treatment and management of disease. Inadequate test result reporting and follow-up is a major risk to patient safety. Factors contributing to failure to follow-up test results include unclear delineation of responsibility about who is meant to act on a test result; poor coordination across different levels of care; and the absence of integrated health information systems for the efficient information communication. Methods A 2016 Australian Stakeholder Forum brought together over 30 representatives from 14 different consumer, clinical and management stakeholder organisations to discuss safe and effective test result communication, management and follow-up. Thematic analysis was conducted drawing on multimodal data collected in the form of observational fieldnotes and document artefacts produced by participants. Results The forum identified major challenges which pose immediate risks to patient safety. Participants recommended priorities for addressing issues relating to: (i) the governance of test result management processes; (ii) integration of health care processes through the utilisation of effective digital health solutions; and (iii) involving patients as key partners in the decision-making and care process. Conclusions Stakeholder groups diverged slightly in their priorities. Consumers highlighted the lack of patient involvement in the test result management process but were less concerned about standardisation of reports and critical result thresholds than pathologists. The forum foregrounded the need for a systems approach, capable of identifying and addressing interconnections and multiple factors that contribute to poor test result follow-up, with a strong emphasis on enhancing the contribution of patients.

Classifying Safety Events Related to Diagnostic Imaging From a Safety Reporting System Using a Human Factors Framework

Purpose:

To measure diagnostic imaging safety events reported to an electronic safety reporting system (ESRS) and assess steps where they occurred within the diagnostic imaging workflow and contributing socio-technical factors.

Methods:

We evaluated all ESRS safety reports related to diagnostic imaging during calendar 2015 at an academic medical center with 50,000 admissions, 950,000 ambulatory visits, and performing 680,000 diagnostic imaging studies annually. Each report was assigned a 0-4 harm score by the reporter; we classified scores of 2 (minor harm) to 4 (death) as “potential harm”. Two reviewers manually classified reports into steps involved in the diagnostic imaging chain and socio-technical factors per the Systems Engineering Initiative for Patient Safety (SEIPS) framework. Kappa measured inter-reviewer agreement on 10% of reports. The percentage of reports that could cause “potential harm” was compared for each step and socio-technical factor using chi-square analysis.

Results:

Of 11,570 safety reports submitted in 2015, 854 (7%) were related to diagnostic imaging. Although the most common step was Imaging Procedure (54% of reports), potential harm occurred more in Report Communication (Odds Ratio=2.36, p=0.05). Person factors most commonly contributed to safety reports (71%). Potential harm occurred more in safety reports that were related to Task compared to Person factors (OR=5.03, p<0.0001). Kappa was 0.79.

Conclusion:

Safety events were related to diagnostic imaging in 7% of reports and potential harm occurred primarily during Imaging Procedure and Report Communication. Safety events were attributed to multifactorial socio-technical factors. Further work is necessary to decrease safety events related to diagnostic imaging.

Monitoring InBaskets With Feedback to Providers Enhances the Timeliness of Patient Care

PURPOSE

Electronic health records have changed providers' workflow. Epic's InBasket supplants traditional communication and is a central hub for clinical information. Failure to promptly complete records impairs communication and revenue collection. By tracking providers' InBasket activities and offering feedback, we hoped to improve InBasket management and interdisciplinary communication.

METHODS

We created a report to track 273 providers' InBasket activities, including ambulatory transcriptions, chart cosignatures, order cosignatures, patient calls, results, and billable encounters. The report showed how often and for how long each activity was delinquent. We completed three Plan-Do-Study-Act cycles. During cycle 1 (November to December 2015), we sent all providers automated e-mails with their monthly results. During cycle 2 (January to April 2016), we focused solely on billable encounter closure and sent targeted e-mails to providers with > 50 delinquent encounters. The e-mails stated that providers had 30 days to complete encounters or their practices would be closed to new patients; at 30 days, noncompliant providers had 60 days before practice suspension. During cycle 3 (May to September 2016), we continued to monitor and send targeted e-mails to providers who accumulated > 50 encounters. We modeled the financial impact of the intervention using net closure data, the report's aging function, and billing logs.

RESULTS

InBasket monitoring with structured feedback decreased open encounters by 53.43%. We did not see improvements in the other metrics that the report tracked. We estimate that $231,724 was saved as a result of the intervention and $349,179 was lost to filing deadlines.

CONCLUSION

Automated e-mails did not reduce open encounters; targeted e-mails to providers improved InBasket management.

Critical Test Result Notification via Mobile Phone-Based Automated Text Message System in the Radiologic Field: Single Institutional Experience

PURPOSE

To investigate the feasibility of sharing critical test result (CTR) notifications (CTRNs) via automated text messaging.

MATERIALS AND METHODS

CTRNs via automated text messaging was used to notify physicians of CTRs in a tertiary hospital with 1,786 beds. From June 2016 to September 2016, notifications for 545 CTRs were given via a CTRN system. Among them, 490 CTRs (292 male and 198 female patients; mean age, 53.6 years old [range, 1-88]) were included in analysis. CTR levels (CTRLs) were assigned to four categories (CTRL1 to CTRL3 and unclassified) when reported, and reclassified into three CTRLs according to their clinical relevance and urgency. Response time was defined as time lapse between CTR reporting and documentation by physicians. Analysis of variance was performed to compare response times according to CTRLs and patients' location.

RESULTS

Corresponding actions were taken in 404 of 490 cases (82.4%) without any delayed CTRN-related morbidity. There were 15 CTRL1 (3.1%), 50 CTRL2 (10.2%), 112 CTRL3 (22.9%) cases, and the remaining 313 CTRL cases were unclassified. After reclassification, CTRL1, CTRL2, and CTRL3 were 81 (16.5%), 177 (36.1%), and 232 cases (47.3%), respectively. Response time of reclassified CTRL3 was significantly longer than that of reclassified CTRL1 (median 23.0, [interquartile range 2.0-133.5] hours versus 4.0 [0.0-22.0] hours; P < .001). Response time of outpatient cases (80.0 [6.0 to 157.0] hours) was significantly longer (P < .001) than those of inpatient (3.0 [0.0-16.0]) and emergency department cases (5.0 [1.0-21.0]).

CONCLUSION

Automated text messaging could be a feasible option for CTRNs in the radiologic field. Further large-scale investigations regarding efficiency of this system are warranted.

Using Natural Language Processing to Extract Abnormal Results From Cancer Screening Reports

OBJECTIVES

Numerous studies show that follow-up of abnormal cancer screening results, such as mammography and Papanicolaou (Pap) smears, is frequently not performed in a timely manner. A contributing factor is that abnormal results may go unrecognized because they are buried in free-text documents in electronic medical records (EMRs), and, as a result, patients are lost to follow-up. By identifying abnormal results from free-text reports in EMRs and generating alerts to clinicians, natural language processing (NLP) technology has the potential for improving patient care. The goal of the current study was to evaluate the performance of NLP software for extracting abnormal results from free-text mammography and Pap smear reports stored in an EMR.

METHODS

A sample of 421 and 500 free-text mammography and Pap reports, respectively, were manually reviewed by a physician, and the results were categorized for each report. We tested the performance of NLP to extract results from the reports. The 2 assessments (criterion standard versus NLP) were compared to determine the precision, recall, and accuracy of NLP.

RESULTS

When NLP was compared with manual review for mammography reports, the results were as follows: precision, 98% (96%-99%); recall, 100% (98%-100%); and accuracy, 98% (96%-99%). For Pap smear reports, the precision, recall, and accuracy of NLP were all 100%.

CONCLUSIONS

Our study developed NLP models that accurately extract abnormal results from mammography and Pap smear reports. Plans include using NLP technology to generate real-time alerts and reminders for providers to facilitate timely follow-up of abnormal results.

Development and Validation of Trigger Algorithms to Identify Delays in Diagnostic Evaluation of Gastroenterological Cancer

BACKGROUND & AIMS

Colorectal cancer (CRC) and hepatocellular cancer (HCC) are common causes of death and morbidity, and patients benefit from early detection. However, delays in follow-up of suspicious findings are common, and methods to efficiently detect such delays are needed. We developed, refined, and tested trigger algorithms that identify patients with delayed follow-up evaluation of findings suspicious of CRC or HCC.

METHODS

We developed and validated two trigger algorithms that detect delays in diagnostic evaluation of CRC and HCC using laboratory, diagnosis, procedure, and referral codes from the Department of Veteran Affairs National Corporate Data Warehouse. The algorithm initially identified patients with positive test results for iron deficiency anemia or fecal immunochemical test (for CRC) and elevated α-fetoprotein results (for HCC). Our algorithm then excluded patients for whom follow-up evaluation was unnecessary, such as patients with a terminal illness or those who had already completed a follow-up evaluation within 60 days. Clinicians reviewed samples of both delayed and nondelayed records, and review data were used to calculate trigger performance.

RESULTS

We applied the algorithm for CRC to 245,158 patients seen from January 1, 2013, through December 31, 2013 and identified 1073 patients with delayed follow up. In a review of 400 randomly selected records, we found that our algorithm identified patients with delayed follow-up with a positive predictive value of 56.0% (95% CI, 51.0%-61.0%). We applied the algorithm for HCC to 333,828 patients seen from January 1, 2011 through December 31, 2014, and identified 130 patients with delayed follow-up. During manual review of all 130 records, we found that our algorithm identified patients with delayed follow-up with a positive predictive value of 82.3% (95% CI, 74.4%-88.2%). When we extrapolated the findings to all patients with abnormal results, the algorithm identified patients with delayed follow-up evaluation for CRC with 68.6% sensitivity (95% CI, 65.4%-71.6%) and 81.1% specificity (95% CI, 79.5%-82.6%); it identified patients with delayed follow-up evaluation for HCC with 89.1% sensitivity (95% CI, 81.8%-93.8%) and 96.5% specificity (95% CI, 94.8%-97.7%). Compared to nonselective methods, use of the algorithm reduced the number of records required for review to identify a delay by more than 99%.

CONCLUSIONS

Using data from the Veterans Affairs electronic health record database, we developed an algorithm that greatly reduces the number of record reviews necessary to identify delays in follow-up evaluations for patients with suspected CRC or HCC. This approach offers a more efficient method to identify delayed diagnostic evaluation of gastroenterological cancers.

Primary Care Providers' Opening of Time-Sensitive Alerts Sent to Commercial Electronic Health Record InBaskets

BACKGROUND

Time-sensitive alerts are among the many types of clinical notifications delivered to physicians' secure InBaskets within commercial electronic health records (EHRs). A delayed alert review can impact patient safety and compromise care.

OBJECTIVE

To characterize factors associated with opening of non-interruptive time-sensitive alerts delivered into primary care provider (PCP) InBaskets.

DESIGN AND PARTICIPANTS

We analyzed data for 799 automated alerts. Alerts highlighted actionable medication concerns for older patients post-hospital discharge (2010-2011). These were study-generated alerts sent 3 days post-discharge to InBaskets for 75 PCPs across a multisite healthcare system, and represent a subset of all urgent InBasket notifications.

MAIN MEASURES

Using EHR access and audit logs to track alert opening, we performed bivariate and multivariate analyses calculating associations between patient characteristics, provider characteristics, contextual factors at the time of alert delivery (number of InBasket notifications, weekday), and alert opening within 24 h.

KEY RESULTS

At the time of alert delivery, the PCPs had a median of 69 InBasket notifications and had received a median of 379.8 notifications (IQR 295.0, 492.0) over the prior 7 days. Of the 799 alerts, 47.1% were opened within 24 h. Patients with longer hospital stays (>4 days) were marginally more likely to have alerts opened (OR 1.48 [95% CI 1.00-2.19]). Alerts delivered to PCPs whose InBaskets had a higher number of notifications at the time of alert delivery were significantly less likely to be opened within 24 h (top quartile >157 notifications: OR 0.34 [95% CI 0.18-0.61]; reference bottom quartile ≤42). Alerts delivered on Saturdays were also less likely to be opened within 24 h (OR 0.18 [CI 0.08-0.39]).

CONCLUSIONS

The number of total InBasket notifications and weekend delivery may impact the opening of time-sensitive EHR alerts. Further study is needed to support safe and effective approaches to care team management of InBasket notifications.

Clinician Perspectives on the Management of Abnormal Subcritical Tests in an Urban Academic Safety-Net Health Care System

BACKGROUND

Missed or delayed follow-up of abnormal subcritical tests (tests that do not require immediate medical attention) can lead to poor patient outcomes. Safety-net health systems with limited resources and socially complex patients are vulnerable to safety gaps resulting from delayed management. Clinician perspectives to identify system challenges, vulnerable situations, and potential solutions were sought in focus groups.

METHODS

Five semistructured focus groups were conducted in 2015 with purposefully sampled clinicians from radiology, hospital medicine, emergency medicine, risk management, and ambulatory care from an urban, academic, integrated, safety-net health system. Thematic analysis identified challenges of current management of abnormal subcritical tests, vulnerable situations, and solution characteristics. A total of 43 clinicians participated.

RESULTS

Clinicians cited challenges in assigning responsibility for follow-up and identified tests pending at discharge and tests requiring delayed follow-up as vulnerable situations. The lack of tracking systems and missing contact information for patients and providers exacerbated these challenges. Proposed solution characteristics involved protocols to aid in assigning responsibility, reliable paths of communication, and systems to track the status of tests. Clinicians noted a strong desire for integration of the work flow and technology solutions into existing structures.

CONCLUSION

In an urban safety-net setting, clinicians recommended outlining clear chains of responsibility and communication in the management of subcritical test results, and employing simple, integrated technological solutions that allow for tracking and management of tests. Existing test management solutions should be adapted to work within safety-net systems, which often have fewer resources and more complex patients and may function in the absence of integrated technology systems.

Electronic Health Record Alert-Related Workload as a Predictor of Burnout in Primary Care Providers

BACKGROUND

Electronic health records (EHRs) have been shown to increase physician workload. One EHR feature that contributes to increased workload is asynchronous alerts (also known as inbox notifications) related to test results, referral responses, medication refill requests, and messages from physicians and other health care professionals. This alert-related workload results in negative cognitive outcomes, but its effect on affective outcomes, such as burnout, has been understudied.

OBJECTIVES

To examine EHR alert-related workload (both objective and subjective) as a predictor of burnout in primary care providers (PCPs), in order to ultimately inform interventions aimed at reducing burnout due to alert workload.

METHODS

A cross-sectional questionnaire and focus group of 16 PCPs at a large medical center in the southern United States.

RESULTS

Subjective, but not objective, alert workload was related to two of the three dimensions of burnout, including physical fatigue (p = 0.02) and cognitive weariness (p = 0.04), when controlling for organizational tenure. To reduce alert workload and subsequent burnout, participants indicated a desire to have protected time for alert management, fewer unnecessary alerts, and improvements to the EHR system.

CONCLUSIONS

Burnout associated with alert workload may be in part due to subjective differences at an individual level, and not solely a function of the objective work environment. This suggests the need for both individual and organizational-level interventions to improve alert workload and subsequent burnout. Additional research should confirm these findings in larger, more representative samples.

Workarounds and Test Results Follow-up in Electronic Health Record-Based Primary Care

BACKGROUND

Electronic health records (EHRs) have potential to facilitate reliable communication and follow-up of test results. However, limitations in EHR functionality remain, leading practitioners to use workarounds while managing test results. Workarounds can lead to patient safety concerns and signify indications as to how to build better EHR systems that meet provider needs.

OBJECTIVE

To understand why primary care practitioners (PCPs) use workarounds to manage test results by analyzing data from a previously conducted national cross-sectional survey on test result management.

METHODS

We conducted a secondary data analysis of quantitative and qualitative data from a national survey of PCPs practicing in the Department of Veterans Affairs (VA) and explored the use of workarounds in test results management. We used multivariate logistic regression analysis to examine the association between key sociotechnical factors that could affect test results follow-up (e.g., both technology-related and those unrelated to technology, such as organizational support for patient notification) and workaround use. We conducted a qualitative content analysis of free text survey data to examine reasons for use of workarounds.

RESULTS

Of 2554 survey respondents, 1104 (43%) reported using workarounds related to test results management. Of these 1028 (93%) described the type of workaround they were using; 719 (70%) reported paper-based methods, while 230 (22%) used a combination of paper- and computer-based workarounds. Primary care practitioners who self-reported limited administrative support to help them notify patients of test results or described an instance where they personally (or a colleague) missed results, were more likely to use workarounds (p=0.02 and p=0.001, respectively). Qualitative analysis identified three main reasons for workaround use: 1) as a memory aid, 2) for improved efficiency and 3) for facilitating internal and external care coordination.

CONCLUSION

Workarounds to manage EHR-based test results are common, and their use results from unmet provider information management needs. Future EHRs and the respective work systems around them need to evolve to meet these needs.

Building a Quality Management System for a Thyroid Nodule Clinic

BACKGROUND

There are few published quality metrics relevant to fine-needle aspirations (FNA) of the thyroid and endocrinology. With the development of a Thyroid Nodule Clinic within an academic practice, the Division of Endocrinology, in collaboration with the Department of Medicine Quality Improvement Team, established a system to monitor the results and follow-up of thyroid biopsies to ensure patient safety and to prevent adverse clinical outcomes attributable to delayed or incomplete follow-up.

METHODS

All FNA performed are identified using billing data from the Thyroid Nodule Clinic. Results were followed using the Beth Israel Deaconess Online Medical Record (OMR) system, and information is collected on documented follow-up plan and actions taken. Missing data are flagged for review. Over the reporting period, the monitoring process and categories were modified to account for adoption of the Bethesda reporting criteria and implementation of gene expression classifier testing. Specific workflow plans were developed for each cytopathologic classification.

RESULTS

Between July 2007 and June 2014, 3895 FNAs were performed. The quality improvement tracking process found that 3856/3895 (99%) biopsy cases had documented follow-up since initiation of the project.

CONCLUSION

This monitoring process has ensured quality patient care, with confidence that patients having FNAs are receiving documented necessary follow-up for treatment of their nodular conditions. This system serves as a potential model for others to use in their endocrine practice in managing the results of thyroid nodule biopsies.

Patient Preferences for Test Result Notification

IMPORTANCE

Patients are increasingly being given access to their test results, but little is known about how preferences vary with the test under consideration or the results of the test (normal or abnormal).

OBJECTIVE

This study was conducted to examine preferences for test result communication.

DESIGN, SETTING, AND PARTICIPANTS

We surveyed adults to explore their preferences for test result notification for three common diagnostic tests of varying "emotional impact" (dual-energy x-ray absorptiometry [DXA], genital herpes, and cancer biopsy) when test results were 1) normal and 2) abnormal. We conducted our survey between June and August 2012 on the campus of an academic medical center. For each scenario, subjects were asked to rank seven methods that might be used to communicate test results (letter, unsecured email, secured email, text message, telephone call, secure Web portal, office visit) in order of acceptability.

MAIN OUTCOME MEASURES

The main measures were the percentage of respondents who ranked a particular test result notification method favorably and the percentage who ranked it as unacceptable.

RESULTS

When test results were normal, subjects' notification preferences were generally similar for DXA, herpes and cancer biopsy, with telephone and letter ranked most favorably for all three tests. Conversely, text message and unsecured email were viewed as unacceptable notification methods for normal results by 45.0-55.0 % of subjects across all three tests. When test results were abnormal, office visits became more popular. A higher proportion of subjects ranked office visits as their most preferred notification method for our test with high "emotional impact" (cancer biopsy) (38.4 %) as compared to DXA (28.2 %) and herpes (27.9 %) (P = 0.02). For most test scenarios, younger subjects appeared to rank electronic communication modalities (secure email or Web portal) higher than older subjects, though this difference did not reach statistical significance (P = 0.29).

CONCLUSIONS

Preferences for test result notification can differ substantially depending upon the test under consideration and results of the test. Providers and health care systems should consider these factors when deciding how to communicate results to patients.

Blame the Patient, Blame the Doctor or Blame the System? A Meta-Synthesis of Qualitative Studies of Patient Safety in Primary Care

Objective

Studies of patient safety in health care have traditionally focused on hospital medicine. However, recent years have seen more research located in primary care settings which have different features compared to secondary care. This study set out to synthesize published qualitative research concerning patient safety in primary care in order to build a conceptual model.

Method

Meta-ethnography, an interpretive synthesis method whereby third order interpretations are produced that best describe the groups of findings contained in the reports of primary studies.

Results

Forty-eight studies were included as 5 discrete subsets where the findings were translated into one another: patients’ perspectives of safety, staff perspectives of safety, medication safety, systems or organisational issues and the primary/secondary care interface. The studies were focused predominantly on issues seen to either improve or compromise patient safety. These issues related to the characteristics or behaviour of patients, staff or clinical systems and interactions between staff, patients and staff, or people and systems. Electronic health records, protocols and guidelines could be seen to both degrade and improve patient safety in different circumstances. A conceptual reading of the studies pointed to patient safety as a subjective feeling or judgement grounded in moral views and with potentially hidden psychological consequences affecting care processes and relationships. The main threats to safety appeared to derive from ‘grand’ systems issues, for example involving service accessibility, resources or working hours which may not be amenable to effective intervention by individual practices or health workers, especially in the context of a public health system.

Conclusion

Overall, the findings underline the human elements in patient safety primary health care. The key to patient safety lies in effective face-to-face communication between patients and health care staff or between the different staff involved in the care of an individual patient. Electronic systems can compromise safety when they override the opportunities for face-to-face communication. The circumstances under which guidelines or protocols are seen to either compromise or improve patient safety needs further investigation.

Lack of timely follow-up of abnormal imaging results and radiologists' recommendations

PURPOSE

Abnormal imaging results may not always lead to timely follow-up. We tested whether certain aspects of communication in radiology reports influence the response of the referring providers, and hence follow-up on abnormal findings.

METHODS

We focused on 2 communication-related items that we hypothesized could affect follow-up: expressions of doubt in the radiology report, and recommendations for further imaging. After institutional review board approval, we conducted a retrospective review of 250 outpatient radiology reports from a multispecialty ambulatory clinic of a tertiary-care Veterans Affairs facility. The selected studies included 92 cases confirmed to lack timely follow-up (ie, further tests or consultations, treatment, and/or communication to the patient within 4 weeks), as determined in a previous study. An additional 158 cases with documented timely follow-up served as controls. Doubt in the narrative was measured by the presence of key phrases (eg, "unable to exclude," "cannot exclude," "cannot rule out," "possibly," and "unlikely"), in the absence of which we used reviewer interpretation. A physician blinded to follow-up outcomes collected the data.

RESULTS

Patients whose reports contained recommendations for further imaging were more likely to have been lost to follow-up at 4 weeks compared with patients without such recommendations (P = .01). Language in the report suggestive of doubt did not affect the timeliness of follow-up (P = .59).

CONCLUSIONS

Abnormal imaging results with recommendations for additional imaging may be more vulnerable to lack of timely follow-up. Additional safeguards, such as tracking systems, should be developed to prevent failure to follow up on such results.

How context affects electronic health record-based test result follow-up: a mixed-methods evaluation

OBJECTIVES

Electronic health record (EHR)-based alerts can facilitate transmission of test results to healthcare providers, helping ensure timely and appropriate follow-up. However, failure to follow-up on abnormal test results (missed test results) persists in EHR-enabled healthcare settings. We aimed to identify contextual factors associated with facility-level variation in missed test results within the Veterans Affairs (VA) health system.

DESIGN, SETTING AND PARTICIPANTS

Based on a previous survey, we categorised VA facilities according to primary care providers' (PCPs') perceptions of low (n=20) versus high (n=20) risk of missed test results. We interviewed facility representatives to collect data on several contextual factors derived from a sociotechnical conceptual model of safe and effective EHR use. We compared these factors between facilities categorised as low and high perceived risk, adjusting for structural characteristics.

RESULTS

Facilities with low perceived risk were significantly more likely to use specific strategies to prevent alerts from being lost to follow-up (p=0.0114). Qualitative analysis identified three high-risk scenarios for missed test results: alerts on tests ordered by trainees, alerts 'handed off' to another covering clinician (surrogate clinician), and alerts on patients not assigned in the EHR to a PCP. Test result management policies and procedures to address these high-risk situations varied considerably across facilities.

CONCLUSIONS

Our study identified several scenarios that pose a higher risk for missed test results in EHR-based healthcare systems. In addition to implementing provider-level strategies to prevent missed test results, healthcare organisations should consider implementing monitoring systems to track missed test results.

Lessons learned from the introduction of an electronic safety net to enhance test result management in an Australian mothers' hospital

This study describes the implementation and impact of an electronic test result acknowledgement (RA) system in the Mater Mothers' Hospital in Brisbane, Australia. The Verdi application electronically records clinicians' acknowledgement of the review of results. Hospital data (August 2011-August 2012) were extracted to measure clinicians' acknowledgement practices. There were 27,354 inpatient test results for 6855 patients. All test results were acknowledged. 60% (95% CI 59% to 61%) of laboratory and 44% (95% CI 40% to 48%) of imaging results were acknowledged within 24 h. The median time between report availability and acknowledgement was 18.1 h for laboratory and 1 day 18 h for imaging results. The median time from when a result was first viewed to its acknowledgement was 7 min for laboratory and 1 min for imaging results. The longest recorded time to acknowledgement was 38 days. Electronic RA provides a safety net to enhance test result management.

Exploring the sociotechnical intersection of patient safety and electronic health record implementation

OBJECTIVE

The intersection of electronic health records (EHR) and patient safety is complex. To examine the applicability of two previously developed conceptual models comprehensively to understand safety implications of EHR implementation in the English National Health Service (NHS).

METHODS

We conducted a secondary analysis of interview data from a 30-month longitudinal, prospective, case study-based evaluation of EHR implementation in 12 NHS hospitals. We used a framework analysis approach to apply conceptual models developed by Sittig and Singh to understand better EHR implementation and use: an eight-dimension sociotechnical model and a three-phase patient safety model (safe technology, safe use of technology, and use of technology to improve safety).

RESULTS

The intersection of patient safety and EHR implementation and use was characterized by risks involving technology (hardware and software, clinical content, and human-computer interfaces), the interaction of technology with non-technological factors, and improper or unsafe use of technology. Our data support that patient safety improvement activities as well as patient safety hazards change as an organization evolves from concerns about safe EHR functionality, ensuring safe and appropriate EHR use, to using the EHR itself to provide ongoing surveillance and monitoring of patient safety.

DISCUSSION

We demonstrate the face validity of two models for understanding the sociotechnical aspects of safe EHR implementation and the complex interactions of technology within a healthcare system evolving from paper to integrated EHR.

CONCLUSIONS

Using sociotechnical models, including those presented in this paper, may be beneficial to help stakeholders understand, synthesize, and anticipate risks at the intersection of patient safety and health information technology.

Developing software to "track and catch" missed follow-up of abnormal test results in a complex sociotechnical environment

BACKGROUND

Abnormal test results do not always receive timely follow-up, even when providers are notified through electronic health record (EHR)-based alerts. High workload, alert fatigue, and other demands on attention disrupt a provider's prospective memory for tasks required to initiate follow-up. Thus, EHR-based tracking and reminding functionalities are needed to improve follow-up.

OBJECTIVES

The purpose of this study was to develop a decision-support software prototype enabling individual and system-wide tracking of abnormal test result alerts lacking follow-up, and to conduct formative evaluations, including usability testing.

METHODS

We developed a working prototype software system, the Alert Watch And Response Engine (AWARE), to detect abnormal test result alerts lacking documented follow-up, and to present context-specific reminders to providers. Development and testing took place within the VA's EHR and focused on four cancer-related abnormal test results. Design concepts emphasized mitigating the effects of high workload and alert fatigue while being minimally intrusive. We conducted a multifaceted formative evaluation of the software, addressing fit within the larger socio-technical system. Evaluations included usability testing with the prototype and interview questions about organizational and workflow factors. Participants included 23 physicians, 9 clinical information technology specialists, and 8 quality/safety managers.

RESULTS

Evaluation results indicated that our software prototype fit within the technical environment and clinical workflow, and physicians were able to use it successfully. Quality/safety managers reported that the tool would be useful in future quality assurance activities to detect patients who lack documented follow-up. Additionally, we successfully installed the software on the local facility's "test" EHR system, thus demonstrating technical compatibility.

CONCLUSION

To address the factors involved in missed test results, we developed a software prototype to account for technical, usability, organizational, and workflow needs. Our evaluation has shown the feasibility of the prototype as a means of facilitating better follow-up for cancer-related abnormal test results.

Safety Assurance Factors for Electronic Health Record Resilience (SAFER): study protocol

BACKGROUND

Implementation and use of electronic health records (EHRs) could lead to potential improvements in quality of care. However, the use of EHRs also introduces unique and often unexpected patient safety risks. Proactive assessment of risks and vulnerabilities can help address potential EHR-related safety hazards before harm occurs; however, current risk assessment methods are underdeveloped. The overall objective of this project is to develop and validate proactive assessment tools to ensure that EHR-enabled clinical work systems are safe and effective.

METHODS/DESIGN

This work is conceptually grounded in an 8-dimension model of safe and effective health information technology use. Our first aim is to develop self-assessment guides that can be used by health care institutions to evaluate certain high-risk components of their EHR-enabled clinical work systems. We will solicit input from subject matter experts and relevant stakeholders to develop guides focused on 9 specific risk areas and will subsequently pilot test the guides with individuals representative of likely users. The second aim will be to examine the utility of the self-assessment guides by beta testing the guides at selected facilities and conducting on-site evaluations. Our multidisciplinary team will use a variety of methods to assess the content validity and perceived usefulness of the guides, including interviews, naturalistic observations, and document analysis. The anticipated output of this work will be a series of self-administered EHR safety assessment guides with clear, actionable, checklist-type items.

DISCUSSION

Proactive assessment of patient safety risks increases the resiliency of health care organizations to unanticipated hazards of EHR use. The resulting products and lessons learned from the development of the assessment guides are expected to be helpful to organizations that are beginning the EHR selection and implementation process as well as those that have already implemented EHRs. Findings from our project, currently underway, will inform future efforts to validate and implement tools that can be used by health care organizations to improve the safety of EHR-enabled clinical work systems.

Primary care practitioners' views on test result management in EHR-enabled health systems: a national survey

Context

Failure to notify patients of test results is common even when electronic health records (EHRs) are used to report results to practitioners. We sought to understand the broad range of social and technical factors that affect test result management in an integrated EHR-based health system.

Methods

Between June and November 2010, we conducted a cross-sectional, web-based survey of all primary care practitioners (PCPs) within the Department of Veterans Affairs nationwide. Survey development was guided by a socio-technical model describing multiple inter-related dimensions of EHR use.

Findings

Of 5001 PCPs invited, 2590 (51.8%) responded. 55.5% believed that the EHRs did not have convenient features for notifying patients of test results. Over a third (37.9%) reported having staff support needed for notifying patients of test results. Many relied on the patient's next visit to notify them for normal (46.1%) and abnormal results (20.1%). Only 45.7% reported receiving adequate training on using the EHR notification system and 35.1% reported having an assigned contact for technical assistance with the EHR; most received help from colleagues (60.4%). A majority (85.6%) stayed after hours or came in on weekends to address notifications; less than a third reported receiving protected time (30.1%). PCPs strongly endorsed several new features to improve test result management, including better tracking and visualization of result notifications.

Conclusions

Despite an advanced EHR, both social and technical challenges exist in ensuring notification of test results to practitioners and patients. Current EHR technology requires significant improvement in order to avoid similar challenges elsewhere.

Effectiveness of an electronic health record-based intervention to improve follow-up of abnormal pathology results: a retrospective record analysis

BACKGROUND AND OBJECTIVE

On March 11, 2009, the Veterans Health Administration (VA) implemented an electronic health record (EHR)-based intervention that required all pathology results to be transmitted to ordering providers by mandatory automated notifications. We examined the impact of this intervention on improving follow-up of abnormal outpatient pathology results.

RESEARCH DESIGN AND SUBJECTS

We extracted pathology reports from the EHR of 2 VA sites. From 16,738 preintervention and 17,305 postintervention reports between 09/01/2008 and 09/30/2009, we randomly selected about 5% and evaluated follow-up outcomes using a standardized chart review instrument. Documented responses to the alerted report (eg, ordering follow-up tests or referrals, notifying patients, and prescribing/changing treatment) were recorded.

MEASURES

Primary outcome measures included proportion of timely follow-up responses (within 30 d) and median time to direct response for abnormal reports.

RESULTS

Of 816 preintervention and 798 postintervention reports reviewed, 666 (81.6%) and 688 (86.2%) were abnormal. Overall, there was no apparent intervention effect on timely follow-up (69% vs. 67.1%; P=0.4) or median time to direct response (8 vs. 8 d; P=0.7). However, logistic regression uncovered a significant intervention effect (preintervention odds ratio, 0.7; 95% confidence interval, 0.5-1.0) after accounting for site-specific differences in follow-up, with a lower likelihood of timely follow-up at one site (odds ratio, 0.4; 95% confidence interval, 0.2-0.7).

CONCLUSIONS

An electronic intervention to improve test result follow-up at 2 VA institutions using the same EHR was found effective only after accounting for certain local contextual factors. Aggregating the effect of EHR interventions across different institutions and EHRs without controlling for contextual factors might underestimate their potential benefits.

Technical support and delegation to practice staff - status quo and (possible) future perspectives for primary health care in Germany

BACKGROUND

Primary health care in industrialized countries faces major challenges due to demographic changes, an increasing prevalence of chronic diseases and a shortage of primary care physicians. One approach to counteract these developments might be to reduce primary care physicians' workload supported by the use of health information technology (HIT) and non-physician practice staff. In 2009, the U.S. Commonwealth Fund (CWF) conducted an international survey of primary care physicians which the present secondary descriptive analysis is based on. The aim of this analysis was twofold: First, to explore to what extend German primary care physicians already get support by HIT and non-physician practice staff, and second, to show possible future perspectives.

METHODS

The CWF questionnaire was sent to a representative random sample of 1,500 primary care physicians all over Germany. The data was descriptively analyzed. Group comparisons regarding differences in gender and age groups were made by means of Chi Square Tests for categorical variables. An alpha-level of p < 0.05 was used for statistical significance.

RESULTS

Altogether 715 primary care physicians answered the questionnaire (response rate 49%). Seventy percent of the physicians use electronic medical records. Technical features such as electronic ordering and access to laboratory parameters are mainly used. However, the majority does not routinely use technical functions for drug prescribing, reminder-systems for guideline-based interventions or recall of patients. Six percent of surveyed physicians are able to transfer prescriptions electronically to a pharmacy, 1% use email communication with patients regularly. Seventy-two percent of primary care physicians get support by non-physician practice staff in patient care, mostly in administrative tasks or routine preventive services. One fourth of physicians is supported in telephone calls to the patient or in patient education and counseling.

CONCLUSION

Within this sample the majority of primary care physicians get support by HIT and non-physician practice staff in their daily work. However, the potential has not yet been fully used. Supportive technical functions like electronic alarm functions for medication or electronic prescribing should be improved technically and more adapted to physicians' needs. To warrant pro-active health care, recall and reminder systems should get refined to encourage their use. Adequately qualified non-physician practice staff could play a more active role in patient care. Reimbursement should not only be linked to doctors', but also to non-physician practice staff services.

Notifications received by primary care practitioners in electronic health records: a taxonomy and time analysis

BACKGROUND

Asynchronous electronic health record (EHR)-based alerts used to notify practitioners via an inbox-like format rather than through synchronous computer "pop-up" messages are understudied. Our objective was to create an asynchronous alert taxonomy and measure the impact of different alert types on practitioner workload.

METHODS

We quantified and categorized asynchronous alerts according to the information they conveyed and conducted a time-motion analysis to assess practitioner workload. We reviewed alert information transmitted to all 47 primary care practitioners (PCPs) at a large, tertiary care Veterans Affairs facility over 4 evenly spaced 28-day periods. An interdisciplinary team used content analysis to categorize alerts according to their conveyed information. We then created an alert taxonomy and used it to calculate the mean number of alerts of each type PCPs received each day. We conducted a time-motion study of 26 PCPs while they processed their alerts. We used these data to estimate the uninterrupted time practitioners spend processing alerts each day.

RESULTS

We extracted 295,792 asynchronously generated alerts and created a taxonomy of 33 alert types categorized under 6 major categories: Test Results, Referrals, Note-Based Communication, Order Status, Patient Status Changes, and Incomplete Task Reminders. PCPs received a mean of 56.4 alerts/day containing new information. Based on 749 observed alert processing episodes, practitioners spent an estimated average of 49 minutes/day processing their alerts.

CONCLUSIONS

PCPs receive a large number of EHR-based asynchronous alerts daily and spend significant time processing them. The utility of transmitting large quantities and varieties of alerts to PCPs warrants further investigation.