Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA

Effect of digital tools to promote hospital quality and safety on adverse events after discharge

OBJECTIVES

Post-discharge adverse events (AEs) are common and heralded by new and worsening symptoms (NWS). We evaluated the effect of electronic health record (EHR)-integrated digital tools designed to promote quality and safety in hospitalized patients on NWS and AEs after discharge.

MATERIALS AND METHODS

Adult general medicine patients at a community hospital were enrolled. We implemented a dashboard which clinicians used to assess safety risks during interdisciplinary rounds. Post-implementation patients were randomized to complete a discharge checklist whose responses were incorporated into the dashboard. Outcomes were assessed using EHR review and 30-day call data adjudicated by 2 clinicians and analyzed using Poisson regression. We conducted comparisons of each exposure on post-discharge outcomes and used selected variables and NWS as independent predictors to model post-discharge AEs using multivariable logistic regression.

RESULTS

A total of 260 patients (122 pre, 71 post [dashboard], 67 post [dashboard plus discharge checklist]) enrolled. The adjusted incidence rate ratios (aIRR) for NWS and AEs were unchanged in the post- compared to pre-implementation period. For patient-reported NWS, aIRR was non-significantly higher for dashboard plus discharge checklist compared to dashboard participants (1.23 [0.97,1.56], P = .08). For post-implementation patients with an AE, aIRR for duration of injury (>1 week) was significantly lower for dashboard plus discharge checklist compared to dashboard participants (0 [0,0.53], P < .01). In multivariable models, certain patient-reported NWS were associated with AEs (3.76 [1.89,7.82], P < .01).

DISCUSSION

While significant reductions in post-discharge AEs were not observed, checklist participants experiencing a post-discharge AE were more likely to report NWS and had a shorter duration of injury.

CONCLUSION

Interventions designed to prompt patients to report NWS may facilitate earlier detection of AEs after discharge.

CLINICALTRIALS.GOV

NCT05232656.

Factors Influencing Data Quality in Electronic Health Record Systems in 50 Health Facilities in Rwanda and the Role of Clinical Alerts: Cross-Sectional Observational Study

BACKGROUND

Electronic health records (EHRs) play an increasingly important role in delivering HIV care in low- and middle-income countries. The data collected are used for direct clinical care, quality improvement, program monitoring, public health interventions, and research. Despite widespread EHR use for HIV care in African countries, challenges remain, especially in collecting high-quality data.

OBJECTIVE

We aimed to assess data completeness, accuracy, and timeliness compared to paper-based records, and factors influencing data quality in a large-scale EHR deployment in Rwanda.

METHODS

We randomly selected 50 health facilities (HFs) using OpenMRS, an EHR system that supports HIV care in Rwanda, and performed a data quality evaluation. All HFs were part of a larger randomized controlled trial, with 25 HFs receiving an enhanced EHR with clinical decision support systems. Trained data collectors visited the 50 HFs to collect 28 variables from the paper charts and the EHR system using the Open Data Kit app. We measured data completeness, timeliness, and the degree of matching of the data in paper and EHR records, and calculated concordance scores. Factors potentially affecting data quality were drawn from a previous survey of users in the 50 HFs.

RESULTS

We randomly selected 3467 patient records, reviewing both paper and EHR copies (194,152 total data items). Data completeness was >85% threshold for all data elements except viral load (VL) results, second-line, and third-line drug regimens. Matching scores for data values were close to or >85% threshold, except for dates, particularly for drug pickups and VL. The mean data concordance was 10.2 (SD 1.28) for 15 (68%) variables. HF and user factors (eg, years of EHR use, technology experience, EHR availability and uptime, and intervention status) were tested for correlation with data quality measures. EHR system availability and uptime was positively correlated with concordance, whereas users' experience with technology was negatively correlated with concordance. The alerts for missing VL results implemented at 11 intervention HFs showed clear evidence of improving timeliness and completeness of initially low matching of VL results in the EHRs and paper records (11.9%-26.7%; P<.001). Similar effects were seen on the completeness of the recording of medication pickups (18.7%-32.6%; P<.001).

CONCLUSIONS

The EHR records in the 50 HFs generally had high levels of completeness except for VL results. Matching results were close to or >85% threshold for nondate variables. Higher EHR stability and uptime, and alerts for entering VL both strongly improved data quality. Most data were considered fit for purpose, but more regular data quality assessments, training, and technical improvements in EHR forms, data reports, and alerts are recommended. The application of quality improvement techniques described in this study should benefit a wide range of HFs and data uses for clinical care, public health, and disease surveillance.

Integrating clinical research into electronic health record workflows to support a learning health system

Objective

Integrating clinical research into routine clinical care workflows within electronic health record systems (EHRs) can be challenging, expensive, and labor-intensive. This case study presents a large-scale clinical research project conducted entirely within a commercial EHR during the COVID-19 pandemic.

Case Report

The UCSD and UCSDH COVID-19 NeutraliZing Antibody Project (ZAP) aimed to evaluate antibody levels to SARS-CoV-2 virus in a large population at an academic medical center and examine the association between antibody levels and subsequent infection diagnosis.

Results

The project rapidly and successfully enrolled and consented over 2000 participants, integrating the research trial with standing COVID-19 testing operations, staff, lab, and mobile applications. EHR-integration increased enrollment, ease of scheduling, survey distribution, and return of research results at a low cost by utilizing existing resources.

Conclusion

The case study highlights the potential benefits of EHR-integrated clinical research, expanding their reach across multiple health systems and facilitating rapid learning during a global health crisis.

Electronic health records and e-prescribing in Australia: An exploration of technological utilisation in Australian community pharmacies

OBJECTIVE

This study aimed to assess the utilisation, benefits, and challenges associated with Electronic Health Records (EHR) and e-prescribing systems in Australian Community Pharmacies, focusing on their integration into daily practice and the impacts on operational efficiency, while also gathering qualitative insights from community pharmacists.

METHODS

A mixed-methods online survey was carried out among community pharmacists throughout Australia to assess the utilisation of EHR and e-prescribing systems, including the benefits and challenges associated with their use. Data was analysed based on pharmacists' age, gender, and practice location (metropolitan vs. regional). The chi-square test was applied to examine the relationship between these demographic factors and the utilisation and operational challenges of EHR and e-prescribing systems.

RESULTS

The survey engaged 120 Australian community pharmacists. Of the participants, 67 % reported usability and efficiency issues with EHR systems. Regarding e-prescribing, 58 % of pharmacists faced delays due to slow software performance, while 42 % encountered errors in data transmission. Despite these challenges, the benefits of e-prescribing were evident, with 79 % of respondents noting the elimination of illegible prescriptions and 40 % observing a reduction in their workload. Issues with prescription quantity discrepancies and the reprinting process were highlighted, indicating areas for improvement in workflow and system usability. The analysis revealed no significant statistical relationship between the utilisation and challenges of EHR and e-prescribing systems with the demographic variables of age, gender and location (p > 0.05), emphasising the necessity for healthcare solutions that address the needs of all pharmacists regardless of specific demographic segments.

CONCLUSION

In Australian community pharmacies, EHR and e-prescribing may enhance patient care but come with challenges such as data completeness, technical issues, and usability concerns. Implementing successful integration relies on user-centric design, standardised practices, and robust infrastructure. While demanding for pharmacists, the digital transition improves efficiency and quality of care. Ensuring user-friendly tools is crucial for the smooth utilisation of digital health.

The Safety of Outpatient Health Care : Review of Electronic Health Records

BACKGROUND

Despite considerable emphasis on delivering safe care, substantial patient harm occurs. Although most care occurs in the outpatient setting, knowledge of outpatient adverse events (AEs) remains limited.

OBJECTIVE

To measure AEs in the outpatient setting.

DESIGN

Retrospective review of the electronic health record (EHR).

SETTING

11 outpatient sites in Massachusetts in 2018.

PATIENTS

3103 patients who received outpatient care.

MEASUREMENTS

Using a trigger method, nurse reviewers identified possible AEs and physicians adjudicated them, ranked severity, and assessed preventability. Generalized estimating equations were used to assess the association of having at least 1 AE with age, sex, race, and primary insurance. Variation in AE rates was analyzed across sites.

RESULTS

The 3103 patients (mean age, 52 years) were more often female (59.8%), White (75.1%), English speakers (90.8%), and privately insured (70.4%) and had a mean of 4 outpatient encounters in 2018. Overall, 7.0% (95% CI, 4.6% to 9.3%) of patients had at least 1 AE (8.6 events per 100 patients annually). Adverse drug events were the most common AE (63.8%), followed by health care-associated infections (14.8%) and surgical or procedural events (14.2%). Severity was serious in 17.4% of AEs, life-threatening in 2.1%, and never fatal. Overall, 23.2% of AEs were preventable. Having at least 1 AE was less often associated with ages 18 to 44 years than with ages 65 to 84 years (standardized risk difference, -0.05 [CI, -0.09 to -0.02]) and more often associated with Black race than with Asian race (standardized risk difference, 0.09 [CI, 0.01 to 0.17]). Across study sites, 1.8% to 23.6% of patients had at least 1 AE and clinical category of AEs varied substantially.

LIMITATION

Retrospective EHR review may miss AEs.

CONCLUSION

Outpatient harm was relatively common and often serious. Adverse drug events were most frequent. Rates were higher among older adults. Interventions to curtail outpatient harm are urgently needed.

PRIMARY FUNDING SOURCE

Controlled Risk Insurance Company and the Risk Management Foundation of the Harvard Medical Institutions.

Determinants of appropriate antibiotic and NSAID prescribing in unscheduled outpatient settings in the veterans health administration

BACKGROUND

Despite efforts to enhance the quality of medication prescribing in outpatient settings, potentially inappropriate prescribing remains common, particularly in unscheduled settings where patients can present with infectious and pain-related complaints. Two of the most commonly prescribed medication classes in outpatient settings with frequent rates of potentially inappropriate prescribing include antibiotics and nonsteroidal anti-inflammatory drugs (NSAIDs). In the setting of persistent inappropriate prescribing, we sought to understand a diverse set of perspectives on the determinants of inappropriate prescribing of antibiotics and NSAIDs in the Veterans Health Administration.

METHODS

We conducted a qualitative study guided by the Consolidated Framework for Implementation Research and Theory of Planned Behavior. Semi-structured interviews were conducted with clinicians, stakeholders, and Veterans from March 1, 2021 through December 31, 2021 within the Veteran Affairs Health System in unscheduled outpatient settings at the Tennessee Valley Healthcare System. Stakeholders included clinical operations leadership and methodological experts. Audio-recorded interviews were transcribed and de-identified. Data coding and analysis were conducted by experienced qualitative methodologists adhering to the Consolidated Criteria for Reporting Qualitative Studies guidelines. Analysis was conducted using an iterative inductive/deductive process.

RESULTS

We conducted semi-structured interviews with 66 participants: clinicians (N = 25), stakeholders (N = 24), and Veterans (N = 17). We identified six themes contributing to potentially inappropriate prescribing of antibiotics and NSAIDs: 1) Perceived versus actual Veterans expectations about prescribing; 2) the influence of a time-pressured clinical environment on prescribing stewardship; 3) Limited clinician knowledge, awareness, and willingness to use evidence-based care; 4) Prescriber uncertainties about the Veteran condition at the time of the clinical encounter; 5) Limited communication; and 6) Technology barriers of the electronic health record and patient portal.

CONCLUSIONS

The diverse perspectives on prescribing underscore the need for interventions that recognize the detrimental impact of high workload on prescribing stewardship and the need to design interventions with the end-user in mind. This study revealed actionable themes that could be addressed to improve guideline concordant prescribing to enhance the quality of prescribing and to reduce patient harm.

Assessing the Usability and Feasibility of Digital Assistant Tools for Direct Support Professionals: Participatory Design and Pilot-Testing

BACKGROUND

The United States is experiencing a direct support professional (DSP) crisis, with demand far exceeding supply. Although generating documentation is a critical responsibility, it is one of the most wearisome aspects of DSPs' jobs. Technology that enables DSPs to log informal time-stamped notes throughout their shift could help reduce the burden of end-of-shift documentation and increase job satisfaction, which in turn could improve the quality of life of the individuals with intellectual and developmental disabilities (IDDs) whom DSPs support. However, DSPs, with varied ages, levels of education, and comfort using technology, are not likely to adopt tools that detract from caregiving responsibilities or increase workload; therefore, technological tools for them must be relatively simple, extremely intuitive, and provide highly valued capabilities.

OBJECTIVE

This paper describes the development and pilot-testing of a digital assistant tool (DAT) that enables DSPs to create informal notes throughout their shifts and use these notes to facilitate end-of-shift documentation. The purpose of the pilot study was to assess the usability and feasibility of the DAT.

METHODS

The research team applied an established user-centered participatory design process to design, develop, and test the DAT prototypes between May 2020 and April 2023. Pilot-testing entailed having 14 DSPs who support adults with IDDs use the first full implementation of the DAT prototypes during 2 or 3 successive work shifts and fill out demographic and usability questionnaires.

RESULTS

Participants used the DAT prototypes to create notes and help generate end-of-shift reports. The System Usability Scale score of 81.79 indicates that they found the prototypes easy to use. Survey responses imply that using the DAT made it easier for participants to produce required documentation and suggest that they would adopt the DAT if this tool were available for daily use.

CONCLUSIONS

Simple technologies such as the DAT prototypes, which enable DSPs to use mobile devices to log time-stamped notes throughout their shift with minimal effort and use the notes to help write reports, have the potential to both reduce the burden associated with producing documentation and enhance the quality (level of detail and accuracy) of this documentation. This could help to increase job satisfaction and reduce turnover in DSPs, both of which would help improve the quality of life of the individuals with IDDs whom they support. The pilot test results indicate that DSPs found the DAT easy to use. Next steps include (1) producing more robust versions of the DAT with additional capabilities, such as storing data locally on mobile devices when Wi-Fi is not available; and (2) eliciting input from agency directors, families, and others who use data about adults with IDDs to help care for them to ensure that data produced by DSPs are relevant and useful.

Clinical Informatics Team Members' Perspectives on Health Information Technology Safety After Experiential Learning and Safety Process Development: Qualitative Descriptive Study

BACKGROUND

Although intended to support improvement, the rapid adoption and evolution of technologies in health care can also bring about unintended consequences related to safety. In this project, an embedded researcher with expertise in patient safety and clinical education worked with a clinical informatics team to examine safety and harm related to health information technologies (HITs) in primary and community care settings. The clinical informatics team participated in learning activities around relevant topics (eg, human factors, high reliability organizations, and sociotechnical systems) and cocreated a process to address safety events related to technology (ie, safety huddles and sociotechnical analysis of safety events).

OBJECTIVE

This study aimed to explore clinical informaticians' experiences of incorporating safety practices into their work.

METHODS

We used a qualitative descriptive design and conducted web-based focus groups with clinical informaticians. Thematic analysis was used to analyze the data.

RESULTS

A total of 10 informants participated. Barriers to addressing safety and harm in their context included limited prior knowledge of HIT safety, previous assumptions and perspectives, competing priorities and organizational barriers, difficulty with the reporting system and processes, and a limited number of reports for learning. Enablers to promoting safety and mitigating harm included participating in learning sessions, gaining experience analyzing reported events, participating in safety huddles, and role modeling and leadership from the embedded researcher. Individual outcomes included increased ownership and interest in HIT safety, the development of a sociotechnical systems perspective, thinking differently about safety, and increased consideration for user perspectives. Team outcomes included enhanced communication within the team, using safety events to inform future work and strategic planning, and an overall promotion of a culture of safety.

CONCLUSIONS

As HITs are integrated into care delivery, it is important for clinical informaticians to recognize the risks related to safety. Experiential learning activities, including reviewing safety event reports and participating in safety huddles, were identified as particularly impactful. An HIT safety learning initiative is a feasible approach for clinical informaticians to become more knowledgeable and engaged in HIT safety issues in their work.

EHR-BERT: A BERT-based model for effective anomaly detection in electronic health records

OBJECTIVE

Physicians and clinicians rely on data contained in electronic health records (EHRs), as recorded by health information technology (HIT), to make informed decisions about their patients. The reliability of HIT systems in this regard is critical to patient safety. Consequently, better tools are needed to monitor the performance of HIT systems for potential hazards that could compromise the collected EHRs, which in turn could affect patient safety. In this paper, we propose a new framework for detecting anomalies in EHRs using sequence of clinical events. This new framework, EHR-Bidirectional Encoder Representations from Transformers (BERT), is motivated by the gaps in the existing deep-learning related methods, including high false negatives, sub-optimal accuracy, higher computational cost, and the risk of information loss. EHR-BERT is an innovative framework rooted in the BERT architecture, meticulously tailored to navigate the hurdles in the contemporary BERT method; thus, enhancing anomaly detection in EHRs for healthcare applications.

METHODS

The EHR-BERT framework was designed using the Sequential Masked Token Prediction (SMTP) method. This approach treats EHRs as natural language sentences and iteratively masks input tokens during both training and prediction stages. This method facilitates the learning of EHR sequence patterns in both directions for each event and identifies anomalies based on deviations from the normal execution models trained on EHR sequences.

RESULTS

Extensive experiments on large EHR datasets across various medical domains demonstrate that EHR-BERT markedly improves upon existing models. It significantly reduces the number of false positives and enhances the detection rate, thus bolstering the reliability of anomaly detection in electronic health records. This improvement is attributed to the model's ability to minimize information loss and maximize data utilization effectively.

CONCLUSION

EHR-BERT showcases immense potential in decreasing medical errors related to anomalous clinical events, positioning itself as an indispensable asset for enhancing patient safety and the overall standard of healthcare services. The framework effectively overcomes the drawbacks of earlier models, making it a promising solution for healthcare professionals to ensure the reliability and quality of health data.

Development and Usability Testing of a Mobile App-Based Clinical Decision Support System for Delirium: Randomized Crossover Trial

Background

The 3-Minute Diagnostic Interview for Confusion Assessment Method-Defined Delirium (3D-CAM) is an instrument specially developed for the assessment of delirium in general wards, with high reported sensitivity and specificity. However, the use of the 3D-CAM by bedside nurses in routine practice showed relatively poor usability, with multiple human errors during assessment.

Objective

This study aimed to develop a mobile app-based delirium assessment tool based on the 3D-CAM and evaluate its usability among older patients by bedside nurses.

Methods

The Delirium Assessment Tool With Decision Support Based on the 3D-CAM (3D-DST) was developed to address existing issues of the 3D-CAM and optimize the assessment process. Following a randomized crossover design, questionnaires were used to evaluate the usability of the 3D-DST among older adults by bedside nurses. Meanwhile, the performances of both the 3D-DST and the 3D-CAM paper version, including the assessment completion rate, time required for completing the assessment, and the number of human errors made by nurses during assessment, were recorded, and their differences were compared.

Results

The 3D-DST included 3 assessment modules, 9 evaluation interfaces, and 16 results interfaces, with built-in reminders to guide nurses in completing the delirium assessment. In the usability testing, a total of 432 delirium assessments (216 pairs) on 148 older adults were performed by 72 bedside nurses with the 3D-CAM paper version and the 3D-DST. Compared to the 3D-CAM paper version, the mean usability score was significantly higher when using the 3D-DST (4.35 vs 3.40; P<.001). The median scores of the 6 domains of the satisfactory evaluation questionnaire for nurses using the 3D-CAM paper version and the 3D-DST were above 2.83 and 4.33 points, respectively (P<.001). The average time for completing the assessment reduced by 2.1 minutes (4.4 vs 2.3 min; P<.001) when the 3D-DST was used.

Conclusions

This study demonstrated that the 3D-DST significantly improved the efficiency of delirium assessment and was considered highly acceptable by bedside nurses.

Use of Electronic Medical Records (EMR) in Gerontology: Benefits, Considerations and a Promising Future

Electronic medical records (EMRs) have many benefits in clinical research in gerontology, enabling data analysis, development of prognostic tools and disease risk prediction. EMRs also offer a range of advantages in clinical practice, such as comprehensive medical records, streamlined communication with healthcare providers, remote data access, and rapid retrieval of test results, ultimately leading to increased efficiency, enhanced patient safety, and improved quality of care in gerontology, which includes benefits like reduced medication use and better patient history taking and physical examination assessments. The use of artificial intelligence (AI) and machine learning (ML) approaches on EMRs can further improve disease diagnosis, symptom classification, and support clinical decision-making. However, there are also challenges related to data quality, data entry errors, as well as the ethics and safety of using AI in healthcare. This article discusses the future of EMRs in gerontology and the application of AI and ML in clinical research. Ethical and legal issues surrounding data sharing and the need for healthcare professionals to critically evaluate and integrate these technologies are also emphasized. The article concludes by discussing the challenges related to the use of EMRs in research as well as in their primary intended use, the daily clinical practice.

Harnessing the power of clinical decision support systems: challenges and opportunities

Clinical decision support systems (CDSSs) are increasingly integrated into healthcare settings to improve patient outcomes, reduce medical errors and enhance clinical efficiency by providing clinicians with evidence-based recommendations at the point of care. However, the adoption and optimisation of these systems remain a challenge. This review aims to provide an overview of the current state of CDSS, discussing their development, implementation, benefits, limitations and future directions. We also explore the potential for enhancing their effectiveness and provide an outlook for future developments in this field. There are several challenges in CDSS implementation, including data privacy concerns, system integration and clinician acceptance. While CDSS have demonstrated significant potential, their adoption and optimisation remain a challenge.

An open-source system for efficient clinical trial support: The COMET study experience

Exercise clinical trials are complex, logistically burdensome, and require a well-coordinated multi-disciplinary approach. Challenges include managing, curating, and reporting on many disparate information sources, while remaining responsive to a variety of stakeholders. The Combined Exercise Trial (COMET, NCT04848038) is a one-year comparison of three exercise modalities delivered in the community. Target enrollment is 280 individuals over 4 years. To support rigorous execution of COMET, the study team has developed a suite of scripts and dashboards to assist study stakeholders in each of their various functions. The result is a highly automated study system that preserves rigor, increases communication, and reduces staff burden. This manuscript describes system considerations and the COMET approach to data management and use, with a goal of encouraging further development and adaptation by other study teams in various fields.

Identifying the data elements and functionalities of clinical decision support systems to administer medication for neonates and pediatrics: a systematic literature review

BACKGROUND

Patient safety is a central healthcare policy worldwide. Adverse drug events (ADE) are among the main threats to patient safety. Children are at a higher risk of ADE in each stage of medication management process. ADE rate is high in the administration stage, as the final stage of preventing medication errors in pediatrics and neonates. The most effective way to reduce ADE rate is using medication administration clinical decision support systems (MACDSSs). The present study reviewed the literature on MACDSS for neonates and pediatrics. It identified and classified the data elements that mapped onto the Fast Healthcare Interoperability Resources (FHIR) standard and the functionalities of these systems to guide future research.

METHODS

PubMed/ MEDLINE, Embase, CINAHL, and ProQuest databases were searched from 1995 to June 31, 2021. Studies that addressed developing or applying medication administration software for neonates and pediatrics were included. Two authors reviewed the titles, abstracts, and full texts. The quality of eligible studies was assessed based on the level of evidence. The extracted data elements were mapped onto the FHIR standard.

RESULTS

In the initial search, 4,856 papers were identified. After removing duplicates, 3,761 titles, and abstracts were screened. Finally, 56 full-text papers remained for evaluation. The full-text review of papers led to the retention of 10 papers which met the eligibility criteria. In addition, two papers from the reference lists were included. A total number of 12 papers were included for analysis. Six papers were categorized as high-level evidence. Only three papers evaluated their systems in a real environment. A variety of data elements and functionalities could be observed. Overall, 84 unique data elements were extracted from the included papers. The analysis of reported functionalities showed that 18 functionalities were implemented in these systems.

CONCLUSION

Identifying the data elements and functionalities as a roadmap by developers can significantly improve MACDSS performance. Though many CDSSs have been developed for different medication processes in neonates and pediatrics, few have actually evaluated MACDSSs in reality. Therefore, further research is needed on the application and evaluation of MACDSSs in the real environment.

PROTOCOL REGISTRATION

(dx.doi.org/10.17504/protocols.io.bwbwpape).

Electronic health record "gag clauses" and the prevalence of screenshots in peer-reviewed literature

OBJECTIVE

To determine whether the Office of the National Coordinator's policy change restricting the use of "gag clauses" in contracts between electronic health record (EHR) vendors and healthcare facilities increased the prevalence of screenshots in peer-reviewed literature.

MATERIALS AND METHODS

We reviewed EHR usability and safety-related peer-reviewed journal articles from 2015 to 2023 and quantified the number of articles containing screenshots. For those that did not contain screenshots, we identified whether they would have benefited from screenshots.

RESULTS

When gag clauses were permitted 6 of 79 (7.6%) of articles contained screenshots and 8 (10.1%) would have benefited from screenshots. When gag clauses were restricted 3 of 40 (7.5%) contained screenshots and 8 (20%) would have benefited from screenshots.

DISCUSSION

The policy change does not appear to have an impact on the prevalence of screenshots in peer-reviewed literature.

CONCLUSIONS

Additional steps are necessary to promote the use of screenshots in peer-reviewed literature.

Trauma Matters: Integrating Genetic and Environmental Components of PTSD

Trauma is ubiquitous, but only a subset of those who experience trauma will develop posttraumatic stress disorder (PTSD). In this review, it is argued that to determine who is at risk of developing PTSD, it is critical to examine the genetic etiology of the disorder and individual trauma profiles of those who are susceptible. First, the state of current PTSD genetic research is described, with a particular focus on studies that present evidence for trauma type specificity, or for differential genetic etiology according to gender or race. Next, approaches that leverage non-traditional phenotyping approaches are reviewed to identify PTSD-associated variants and biology, and the relative advantages and limitations inherent in these studies are reflected on. Finally, it is discussed how trauma might influence the heritability of PTSD, through type, risk factors, genetics, and associations with PTSD symptomology.

Using multiple qualitative methods to inform intervention development: Improving functional status measurement for older veterans in primary care settings

Functional status, or the ability to perform activities of daily living, is central to older adults' health and quality of life. However, health systems have been slow to incorporate routine measurement of function into patient care. We used multiple qualitative methods to develop a patient-centered, interprofessional intervention to improve measurement of functional status for older veterans in primary care settings. We conducted semi-structured interviews with patients, clinicians, and operations staff (n = 123) from 7 Veterans Health Administration (VHA) Medical Centers. Interviews focused on barriers and facilitators to measuring function. We used concepts from the Consolidated Framework for Implementation Science and sociotechnical analysis to inform rapid qualitative analyses and a hybrid deductive/inductive approach to thematic analysis. We mapped qualitative findings to intervention components. Barriers to measurement included time pressures, cumbersome electronic tools, and the perception that measurement would not be used to improve patient care. Facilitators included a strong interprofessional environment and flexible workflows. Findings informed the development of five intervention components, including (1) an interprofessional educational session; (2) routine, standardized functional status measurement among older patients; (3) annual screening by nurses using a standardized instrument and follow-up assessment by primary care providers; (4) electronic tools and templates to facilitate increased identification and improved management of functional impairment; and (5) tailored reports on functional status for clinicians and operations leaders. These findings show how qualitative methods can be used to develop interventions that are more responsive to real-world contexts, increasing the chances of successful implementation. Using a conceptually-grounded approach to intervention development has the potential to improve patient and clinician experience with measuring function in primary care.

User-Centered Design and Evaluation of Clinical Decision Support to Improve Early Peanut Introduction: Formative Study

BACKGROUND

Peanut allergy has recently become more prevalent. Peanut introduction recommendations have evolved from suggesting peanut avoidance until the age of 3 years to more recent guidelines encouraging early peanut introduction after the Learning Early about Peanut Allergy (LEAP) study in 2015. Guideline adherence is poor, leading to missed care opportunities.

OBJECTIVE

In this study, we aimed to develop a user-centered clinical decision support (CDS) tool to improve implementation of the most recent early peanut introduction guidelines in the primary care clinic setting.

METHODS

We edited the note template of the well-child check (WCC) visits at ages 4 and 6 months with CDS prompts and point-of-care education. Formative and summative usability testing were completed with pediatric residents in a simulated electronic health record (EHR). We estimated task completion rates and perceived usefulness of the CDS in summative testing, comparing a test EHR with and without the CDS.

RESULTS

Formative usability testing with the residents provided qualitative data that led to improvements in the build for both the 4-month and 6-month WCC note templates. During summative usability testing, the CDS tool significantly improved discussion of early peanut introduction at the 4-month WCC visit compared to scenarios without the CDS tool (9/15, 60% with CDS and 0/15, 0% without CDS). All providers except one at the 4-month WCC scenario gave at least an adequate score for the ease of use of the CDS tool for the history of present illness and assessment and plan sections. During the summative usability testing with the 6-month WCC new build note template, providers more commonly provided comprehensive care once obtaining a patient history concerning for an immunoglobulin E-mediated peanut reaction by placing a referral to allergy/immunology (P=.48), prescribing an epinephrine auto-injector (P=.07), instructing on how to avoid peanut products (P<.001), and providing an emergency treatment plan (P=.003) with CDS guidance. All providers gave at least an adequate score for ease of use of the CDS tool in the after-visit summary.

CONCLUSIONS

User-centered CDS improved application of early peanut introduction recommendations and comprehensive care for patients who have symptoms concerning for peanut allergy in a simulation.

Development and Validation of SafeHIT: An Instrument to Assess the Self-Reported Safe Use of Health Information Technology

BACKGROUND

Implementing health information technology (HIT) may cause unintended consequences and safety risks when incorrectly designed and used. Yet, the tools to assess self-reported safe use of HIT are not well established.

OBJECTIVE

This study aims to develop and validate SafeHIT, an instrument to assess self-reported safe use of HIT among health care practitioners.

METHODS

Systematic literature review and a semistructured interview with 31 experts were adopted to generate SafeHIT instrument items. In total, 450 physicians from various departments at three Malaysian public hospitals participated in the questionnaire survey to validate SafeHIT. Exploratory factor analysis and confirmatory factor analysis (CFA) were undertaken to explore the items that best represent a specific construct and to confirm the reliability and validity of the SafeHIT, respectively.

RESULTS

The final SafeHIT consisted of 14 constructs and 58 items in total. The result of the CFA confirmed that all constructs demonstrated adequate convergent and discriminant validity.

CONCLUSION

A reliable and valid theoretically underpinned measure of determinants of safe HIT use behavior has been developed. Understanding external factors that influence safe HIT use is useful for developing targeted interventions that favor the quality and safety of health care.

How Digital Systems Are Used in Swedish Home Care Nursing Practice: A Qualitative Interview Study to Identify Challenges and Opportunities

This study investigates how digital technology is used (or not used) in home care nursing, to identify challenges and opportunities that can move the digitalization of home care nursing forward. The use of digital technology in daily practices of home care nursing was analyzed based on semistructured interviews in four Swedish home care organizations. The results correspond to the two initial stages of a design thinking process: (1) knowledge about users and the use context , presented as an up-to-date description of how digital technology has affected home care nursing practice, and (2) definition of areas for improvement , presented as challenges and opportunities. We identified the following areas to proceed with in the subsequent stages of designing home care nursing digitalization: (1) nurses' ability to access information and conduct documentation in the field, particularly how new digital systems are integrated with existing routines and systems; (2) the multitude of communication processes that nurses must manage, including communication channels that are used and the level of synchronicity; (3) an increasingly complex digital working environment, where evaluating existing systems and routines is one way to learn where improvements can be most efficient.

Best Practices for Technology in Clinical Social Work and Mental Health Professions to Promote Well-being and Prevent Fatigue

The shift to communication technologies during the pandemic has had positive and negative effects on clinical social worker practice. Best practices are identified for clinical social workers to maintain emotional well-being, prevent fatigue, and avoid burnout when using technology. A scoping review from 2000 to 21 of 15 databases focused on communication technologies for mental health care within four areas: (1) behavioral, cognitive, emotional, and physical impact; (2) individual, clinic, hospital, and system/organizational levels; (3) well-being, burnout, and stress; and (4) clinician technology perceptions. Out of 4795 potential literature references, full text review of 201 papers revealed 37 were related to technology impact on engagement, therapeutic alliance, fatigue and well-being. Studies assessed behavioral (67.5%), emotional (43.2%), cognitive (57.8%), and physical (10.8%) impact at the individual (78.4%), clinic (54.1%), hospital (37.8%) and system/organizational (45.9%) levels. Participants were clinicians, social workers, psychologists, and other providers. Clinicians can build a therapeutic alliance via video, but this requires additional skill, effort, and monitoring. Use of video and electronic health records were associated with clinician physical and emotional problems due to barriers, effort, cognitive demands, and additional workflow steps. Studies also found high user ratings on data quality, accuracy, and processing, but low satisfaction with clerical tasks, effort required and interruptions. Studies have overlooked the impact of justice, equity, diversity and inclusion related to technology, fatigue and well-being, for the populations served and the clinicians providing care. Clinical social workers and health care systems must evaluate the impact of technology in order to support well-being and prevent workload burden, fatigue, and burnout. Multi-level evaluation and clinical, human factor, training/professional development and administrative best practices are suggested.

A usability evaluation of the perceived user friendliness, accessibility, and inclusiveness of a personalized digital care pathway tool

OBJECTIVE

This study aimed to acquire insight into the perceived user friendliness, accessibility and inclusiveness of a personalized digital care pathway.

MATERIALS & METHODS

Usability of the tool was tested in an experimental setting. Mixed methods data collection consisted of scenario-based eye tracking tests in a web- or mobile-based prototype of the tool, followed by a questionnaire assessing user friendliness (System Usability Scale; SUS) and a structured interview. Inclusiveness was assessed by subgroup comparisons based on language proficiency, age and education level. Via purposive sampling a heterogeneous population of users (N = 24) was recruited. Eye tracking was used to measure gaze behavior.

RESULTS

Overall, participants were satisfied with the tool (scale 0-10, 7.5; SD = 1.29). User friendliness of the mobile version (68.3; SD = 21.6) was higher than the web version (50.9; SD = 17.3) measured by SUS score (0-100). With regard to accessibility, eye tracking scenarios showed that the menu bar was hard to find (17% mobile, 55% web). In all scenario's, information was found faster in the mobile version than the web version. Attention was easily drawn to images. Regarding inclusiveness of the tool, we found significantly longer completing time of the scenario tasks for low language proficiency (p-value = 0.029) and higher age subgroups (p-value = 0.049). Lower language proficiency scored a significant lower SUS score (p-value = 0.012).

CONCLUSIONS

Overall, user friendliness and accessibility were positively evaluated. Assessment of inclusiveness emphasized the need for tailoring digital tools to those with low language proficiency and/or an older age. Co-creation of digital care tools with users is therefore important to match users' needs, make tools easily understandable and accessible to all users, and ultimately result in better uptake and impact.

Automation of Hematopoietic Cell Transplantation Outcomes Reporting Leads to Dramatic Reduction in Errors Reported to Real-World Data Registry

Institutions that perform hematopoietic cell transplantation (HCT) are required by law to report standardized, structured data on transplantation outcomes. A key post-transplantation outcome is engraftment, the time between HCT infusion and reemergence of circulating neutrophils and platelets. At our center, we found that manual chart abstraction for engraftment data was highly error-prone. We developed a custom R/Shiny application that automatically calculates engraftment dates and displays them in an intuitive format to augment the manual chart review. Our hypothesis was that use of the application to assist with calculating and reporting engraftment dates would be associated with a decreased error rate. The study was conducted at a single tertiary care institution. The application was developed in a collaborative, multidisciplinary fashion by members of an embedded cellular therapy informatics team. Retrospective validation of the application's accuracy was conducted on all malignant HCTs from February 2016 to December 2020 (n = 198). Real-world use of the application was evaluated prospectively from April 2021 through April 2022 (n = 53). The Welch 2-sample t test was used to compare error rates preimplementation and postimplementation. Data were visualized using p charts, and standard special cause variation rules were applied. The accuracy of reported data postdeployment increased dramatically; the engraftment error rate decreased from 15% to 3.8% for neutrophils (P = .003) and from 28% to 1.9% for platelets (P < .001). This study demonstrates the effective deployment of a custom R/Shiny application that was associated with significantly reduced error rates in HCT engraftment reporting for operational, research, and regulatory purposes. Users reported subjective satisfaction with the application and that it addressed difficulties with the legacy manual process. Identifying and correcting erroneous data in engraftment reporting could lead to a more efficient and accurate nationwide assessment of transplantation success. Furthermore, we show that it is possible and practical for academic medical centers to create and support embedded informatics teams that can quickly build applications for clinical operations in a manner compliant with regulatory requirements.

Usability of a mobile application for health professionals in home care services: a user-centered approach

The use of mobile devices by healthcare professionals has led to rapid growth in the development of mobile healthcare applications designed to improve healthcare services. This study was conducted to assess the acceptability and usability of a mobile application for health professionals in relation to their work in hospitalization at home. A mixed methods approach was used. Acceptance, included the satisfaction of the professionals, attitudes toward using the application, and intention or willingness to continue using the application. Usability tests were performed in laboratory analyzing five controlled clinical tasks, and the interaction of the participants with the mobile application was based on the six basic facial expressions published by the American Psychological Association. Perceived satisfaction was assessed using the computer system usability questionnaire. Thirty-two participants completed the task scenarios and questionnaire. More than 90 per cent of participants were able to complete the tasks with only some difficult with vital signs. Satisfaction had a score of 6.18/7 (SD: 0.76), and recommendation of the mobile application had a score of 6.21/7 (SD: 0.81). This study showed a significant usability and acceptability of this mobile application, in terms of effectiveness, efficiency, and satisfaction.

Problem-oriented documentation: design and widespread adoption of a novel toolkit in a commercial electronic health record

Background

Problem-oriented documentation is an accepted method of note construction which facilitates clinical thought processes. However, problem-oriented documentation is challenging to put into practice using commercially available electronic health record (EHR) systems.

Objective

Our goal was to create, iterate, and distribute a problem-oriented documentation toolkit within a commercial EHR that maximally supported clinicians' thinking, was intuitive to use, and produced clear documentation.

Materials and Methods

We used an iterative design process that stressed visual simplicity, data integration, a predictable interface, data reuse, and clinician efficiency. Creation of the problem-oriented documentation toolkit required the use of EHR-provided tools and custom programming.

Results

We developed a problem-oriented documentation interface with a 3-column view showing (1) a list of visit diagnoses, (2) the current overview and assessment and plan for a selected diagnosis, and (3) a list of medications, labs, data, and orders relevant to that diagnosis. We also created a series of macros to bring information collected through the interface into clinicians' notes. This toolkit was put into a live environment in February 2019. Over the first 9 months, the custom problem-oriented documentation toolkit was used in a total of 8385 discrete visits by 28 clinicians in 13 ambulatory departments. After 9 months, the go-live education and EHR optimization teams in our health system began promoting the toolkit to new and existing users of our EHR resulting in a significantly increased uptake by outpatient clinicians. In April 2022 alone, the toolkit was used in more than 92 000 ambulatory visits by 894 users in 271 departments across our health system.

Conclusions

As a health-system client of a commercial EHR, we developed and deployed a revised problem-oriented documentation toolkit that is used by clinicians more than 92 000 times a month. Key success elements include an emphasis on usability and an effective training effort.

POPDx: an automated framework for patient phenotyping across 392 246 individuals in the UK Biobank study

OBJECTIVE

For the UK Biobank, standardized phenotype codes are associated with patients who have been hospitalized but are missing for many patients who have been treated exclusively in an outpatient setting. We describe a method for phenotype recognition that imputes phenotype codes for all UK Biobank participants.

MATERIALS AND METHODS

POPDx (Population-based Objective Phenotyping by Deep Extrapolation) is a bilinear machine learning framework for simultaneously estimating the probabilities of 1538 phenotype codes. We extracted phenotypic and health-related information of 392 246 individuals from the UK Biobank for POPDx development and evaluation. A total of 12 803 ICD-10 diagnosis codes of the patients were converted to 1538 phecodes as gold standard labels. The POPDx framework was evaluated and compared to other available methods on automated multiphenotype recognition.

RESULTS

POPDx can predict phenotypes that are rare or even unobserved in training. We demonstrate substantial improvement of automated multiphenotype recognition across 22 disease categories, and its application in identifying key epidemiological features associated with each phenotype.

CONCLUSIONS

POPDx helps provide well-defined cohorts for downstream studies. It is a general-purpose method that can be applied to other biobanks with diverse but incomplete data.

Design propositions for nudging in healthcare: Adoption of national electronic health record systems

Objectives

Electronic health records (EHRs) are considered important for improving efficiency and reducing costs of a healthcare system. However, the adoption of EHR systems differs among countries and so does the way the decision to participate in EHRs is presented. Nudging is a concept that deals with influencing human behaviour within the research stream of behavioural economics. In this paper, we focus on the effects of the choice architecture on the decision for the adoption of national EHRs. Our study aims to link influences on human behaviour through nudging with the adoption of EHRs to investigate how choice architects can facilitate the adoption of national information systems.

Methods

We employ a qualitative explorative research design, namely the case study method. Using theoretical sampling, we selected four cases (i.e., countries) for our study: Estonia, Austria, the Netherlands, and Germany. We collected and analyzed data from various primary and secondary sources: ethnographic observation, interviews, scientific papers, homepages, press releases, newspaper articles, technical specifications, publications from governmental bodies, and formal studies.

Results

The findings from our European case studies show that designing for EHR adoption should encompass choice architecture elements (i.e., defaults), technical elements (i.e., choice granularity and access transparency), and institutional elements (i.e., regulations for data protection, information campaigns, and financial incentives) in combination.

Conclusions

Our findings provide insights on the design of the adoption environments of large-scale, national EHR systems. Future research could estimate the magnitude of effects of the determinants.

Exploring facilitators of the implementation of electronic health records in Saudi Arabia

INTRODUCTION

The introduction of information technology was one of the key priorities for policy-makers in health care organisations over the last two decades due to the potential benefits of this technology to improve health care services and quality. However, approximately 50% of those projects failed to achieve their intended aims. This was a result of several factors, including the cost of these projects. The Saudi Ministry of Health (MoH) planned to implement an electronic health record system (EHRS) in approximately 2100 primary health care centres nationwide. It was acknowledged that this project may face hurdles, which might result in the failure of the project if implementation facilitators were not first determined. According to the Saudi MoH, previous electronic health record system implementation in primary health care centres failed as a consequence of several barriers, such as poor infrastructure, lack of connectivity and lack of interoperability. However, the facilitators of successful electronic health record system implementation in Saudi primary health care centres are not understood.

AIM

To determine the facilitators that enhance the success of the implementation of an EHRS in public primary health care centres in SA.

METHOD

A mixed methods approach was used with both qualitative and quantitative methods (qualitative using semistructured interviews and quantitative with a closed survey). The purpose of the utilisation of exploratory mixed methods was to identify a wide range of facilitators that may influence EHRS implementation. The data were obtained from two different perspectives, primary health care centre practitioners and project team members. A total of 351 practitioners from 21 primary health care centres participated in the online survey, and 14 key informants at the Saudi Ministry of Health who were directly involved in the electronic health record system implementation in the primary health care centres agreed to be interviewed face to face.

RESULTS

The findings from both studies revealed several facilitators. Among these facilitators, financial resources were found to be the most influential factor that assisted in overcoming some barriers, such as software selection. The size of the primary health care centres was the second facilitator of successful implementation, despite the scale of the project. Perceived usefulness was another facilitator identified in both the interviews and the survey. More than 90% of the participants thought that the electronic health record system was useful and could contribute to improving the quality of health care services. While a high level of satisfaction was expressed towards the electronic health record system's usability and efficiency, low levels of satisfaction were recorded for organisational factors such as user involvement, training and support. Hence, system usability and efficiency were documented to be other facilitators of successful electronic health record system implementation in Saudi primary health care centres.

CONCLUSION

The findings of the present study suggest that sufficient financial support is essential to enhance the success of electronic health record system implementation despite the scale of the project. Additionally, effective leadership and project management are core factors to overcome many obstacles and ensure the success of large-scale projects.

Electronic surveillance of patient safety events using natural language processing

Objective

We describe our approach to surveillance of reportable safety events captured in hospital data including free-text clinical notes. We hypothesize that a) some patient safety events are documented only in the clinical notes and not in any other accessible source; and b) large-scale abstraction of event data from clinical notes is feasible.

Materials and Methods

We use regular expressions to generate a training data set for a machine learning model and apply this model to the full set of clinical notes and conduct further review to identify safety events of interest. We demonstrate this approach on peripheral intravenous (PIV) infiltrations and extravasations (PIVIEs).

Results

During Phase 1, we collected 21,362 clinical notes, of which 2342 were reviewed. We identified 125 PIV events, of which 44 cases (35%) were not captured by other patient safety systems. During Phase 2, we collected 60,735 clinical notes and identified 440 infiltrate events. Our classifier demonstrated accuracy above 90%.

Conclusion

Our method to identify safety events from the free text of clinical documentation offers a feasible and scalable approach to enhance existing patient safety systems. Expert reviewers, using a machine learning model, can conduct routine surveillance of patient safety events.

Usability of human-computer interaction in neonatal care

While a goal for Electronic Health Record (EHR) technologies was to improve quality, efficiency, and safety, the usability of EHRs has remained poor. The relation to patient harm and user satisfaction cannot be ignored. Optimization of EHR usability is imperative to improving the outcomes for critically ill patients, especially neonates who are at the extremes of physiologic variability. Further development and integration of metadata with predictive modeling and clinical protocols can support provider decision making, increase efficiency and safety, and reduce clinician burnout. This paper reviews EHR usability and identifies opportunities to improve the EHR specific to neonatal care.

Documentation Burden in Nursing and Its Role in Clinician Burnout Syndrome

OBJECTIVES

The purpose of this study is to understand the relationship between documentation burden and clinician burnout syndrome in nurses working in direct patient care. The Office of the National Coordinator considers documentation burden a high priority problem. However, the presence of documentation burden in nurses working in direct patient care is not well known. Furthermore, the presence of documentation burden has not been linked to the development of clinician burnout syndrome.

METHODS

This paper reports that the results of a cross-sectional survey study comprised of three tools: (1) The burden of documentation for nurses and mid-wives survey, (2) the system usability scale, and (3) Maslach's burnout inventory for medical professionals.

RESULTS

Documentation burden has a weak to moderate correlation to clinician burnout syndrome. Furthermore, poor usability of the electronic health record (EHR) is also associated with documentation burden and clinician burnout syndrome.

CONCLUSION

This study suggests that there is a relationship between documentation burden and clinician burnout syndrome. The correlation of poor usability and domains of clinician burnout syndrome implies the need for more work on improving the usability of EHR for nursing documentation. Further study regarding the presence of documentation burden and its correlation to clinician burnout syndrome should focus on specific areas of nursing to understand the drivers of documentation burden variation within and across specialty domains.

Assessing the use of a clinical decision support tool for pain management in primary care

Objective

Given time constraints, poorly organized information, and complex patients, primary care providers (PCPs) can benefit from clinical decision support (CDS) tools that aggregate and synthesize problem-specific patient information. First, this article describes the design and functionality of a CDS tool for chronic noncancer pain in primary care. Second, we report on the retrospective analysis of real-world usage of the tool in the context of a pragmatic trial.

Materials and methods

The tool known as OneSheet was developed using user-centered principles and built in the Epic electronic health record (EHR) of 2 health systems. For each relevant patient, OneSheet presents pertinent information in a single EHR view to assist PCPs in completing guideline-recommended opioid risk mitigation tasks, review previous and current patient treatments, view patient-reported pain, physical function, and pain-related goals.

Results

Overall, 69 PCPs accessed OneSheet 2411 times (since November 2020). PCP use of OneSheet varied significantly by provider and was highly skewed (site 1: median accesses per provider: 17 [interquartile range (IQR) 9-32]; site 2: median: 8 [IQR 5-16]). Seven "power users" accounted for 70% of the overall access instances across both sites. OneSheet has been accessed an average of 20 times weekly between the 2 sites.

Discussion

Modest OneSheet use was observed relative to the number of eligible patients seen with chronic pain.

Conclusions

Organizations implementing CDS tools are likely to see considerable provider-level variation in usage, suggesting that CDS tools may vary in their utility across PCPs, even for the same condition, because of differences in provider and care team workflows.

Patient Safety Issues From Information Overload in Electronic Medical Records

BACKGROUND AND OBJECTIVE

Electronic health records (EHRs) have become ubiquitous in medicine and continue to grow in informational content. Little has been documented regarding patient safety from the resultant information overload. The objective of this literature review is to better understand how information overload in EHR affects patient safety.

METHODS

A literature search was performed using the Transparent Reporting of Systematic Reviews and Meta-Analyses standards for literature review. PubMed and Web of Science were searched and articles selected that were relevant to EHR information overload based on keywords.

RESULTS

The literature search yielded 28 articles meeting the criteria for the study. Information overload was found to increase physician cognitive load and error rates in clinical simulations. Overabundance of clinically irrelevant information, poor data display, and excessive alerting were consistently identified as issues that may lead to information overload.

CONCLUSIONS

Information overload in EHRs may result in higher error rates and negatively impact patient safety. Further studies are necessary to define the role of EHR in adverse patient safety events and to determine methods to mitigate these errors. Changes focused on the usability of EHR should be considered with the end user (physician) in mind. Federal agencies have a role to play in encouraging faster adoption of improved EHR interfaces.

Scalable relevance ranking algorithm via semantic similarity assessment improves efficiency of medical chart review

OBJECTIVE

Accurately assigning phenotype information to individual patients via computational phenotyping using Electronic Health Records (EHRs) has been seen as the first step towards enabling EHRs for precision medicine research. Chart review labels annotated by clinical experts, also known as "gold standard" labels, are essential for the development and validation of computational phenotyping algorithms. However, given the complexity of EHR systems, the process of chart review is both labor intensive and time consuming. We propose a fully automated algorithm, referred to as pGUESS, to rank EHR notes according to their relevance to a given phenotype. By identifying the most relevant notes, pGUESS can greatly improve the efficiency and accuracy of chart reviews.

METHOD

pGUESS uses prior guided semantic similarity to measure the informativeness of a clinical note to a given phenotype. We first select candidate clinical concepts from a pool of comprehensive medical concepts using public knowledge sources and then derive the semantic embedding vector (SEV) for a reference article (SEV_ref) and each note (SEV_note). The algorithm scores the relevance of a note as the cosine similarity between SEV_note and SEV_ref.

RESULTS

The algorithm was validated against four sets of 200 notes that were manually annotated by clinical experts to assess their informativeness to one of three disease phenotypes. pGUESS algorithm substantially outperforms existing unsupervised approaches for classifying the relevance status with respect to both accuracy and scalability across phenotypes. Averaging over the three phenotypes, the rank correlation between the algorithm ranking and gold standard label was 0.64 for pGUESS, but only 0.47 and 0.35 for the next two best performing algorithms. pGUESS is also much more computationally scalable compared to existing algorithms.

CONCLUSION

pGUESS algorithm can substantially reduce the burden of chart review and holds potential in improving the efficiency and accuracy of human annotation.

Development and pilot evaluation of an electronic health record usability and safety self-assessment tool

Objective

Poor electronic health record (EHR) usability contributes to clinician burnout and poses patent safety risks. Site-specific customization and configuration of EHRs require individual EHR system usability and safety testing which is resource intensive. We developed and pilot-tested a self-administered EHR usability and safety assessment tool, focused on computerized provider order entry (CPOE), which can be used by any facility to identify specific issues. In addition, the tool provides recommendations for improvement.

Materials and Methods

An assessment tool consisting of 104 questions was developed and pilot-tested at 2 hospitals, one using a Cerner EHR and the other using Epic. Five physicians at each site participated in and completed the assessment. Participant response accuracy compared to actual EHR interactions, consistency across participants, and usability issues identified through the tool were measured at each site.

Results

Across sites, participants answered an average of 46 questions in 23 min with 89.9% of responses either correct or partially correct. The tool identified 8 usability and safety issues at one site and 7 at the other site across medication, laboratory, and radiology CPOE functions.

Discussion

The tool shows promise as a method to rapidly evaluate EHR usability and safety and provide guidance on specific areas for improvement. Important improvements to the evaluation tool were identified including the need to clarify certain questions and provide definitions for usability terminology.

Conclusion

A self-administered usability and safety assessment tool can serve to identify specific usability and safety issues in the EHR and provide guidance for improvements.

Real-world implementation evaluation of an electronic health record-integrated consumer informatics tool that collects patient-generated contextual data

OBJECTIVE

Use the RE-AIM framework to examine the implementation of a patient contextual data (PCD) Tool designed to share patients' needs, values, and preferences with care teams ahead of clinical encounters.

MATERIALS & METHODS

Observational study that follows initial PCD Tool scaling across primary care at a Midwestern academic health network. Program invitations, enrollment, patient submissions, and clinician views were tracked over a 1-year study period. Logistic regression modeled the likelihood of using the PCD Tool, accounting for patient covariates.

RESULTS

Of 58,874 patients who could be contacted by email, 9,183 (15.6%) became PCD Tool users. Overall, 76% of primary care providers had patients who used the PCD Tool. Older age, female gender, non-minority race, patient portal activation, and Medicare coverage were significantly associated with increased likelihood of use. Number of office visits, medical issues, and behavioral health conditions also associated with use. Primary care staff viewed 18.7% of available PCD Tool summaries, 1.1% to 57.6% per clinic.

DISCUSSION

The intervention mainly reached non-minority patients and patients who used more health services. Given the requirement for an email address on file, some patients may have been underrepresented. Overall, patient reach and adoption and clinician adoption, implementation, and maintenance of this Tool were modest but stable, consistent with a non-directive approach to fostering adoption by introducing the Tool in the absence of clear expectations for use.

CONCLUSION

Healthcare organizations must implement effective methods to increase the reach, adoption, implementation, and maintenance of PCD tools across all patient populations. Assisting people, particularly racial minorities, with PCD Tool registration and actively supporting clinician use are critical steps in implementing technology that facilitates care.

Utilizing eye tracking to assess electronic health record use by pharmacists in the intensive care unit

DISCLAIMER

In an effort to expedite the publication of articles related to the COVID-19 pandemic, AJHP is posting these manuscripts online as soon as possible after acceptance. Accepted manuscripts have been peer-reviewed and copyedited, but are posted online before technical formatting and author proofing. These manuscripts are not the final version of record and will be replaced with the final article (formatted per AJHP style and proofed by the authors) at a later time.

PURPOSE

A study was conducted using high-fidelity electronic health record (EHR)-based simulations with incorporated eye tracking to understand the workflow of critical care pharmacists within the EHR, with specific attention to the data elements most frequently viewed.

METHODS

Eight critical care pharmacists were given 25 minutes to review 3 simulated intensive care unit (ICU) charts deployed in the simulation instance of the EHR. Using monitor-based eye trackers, time spent reviewing screens, clinical information accessed, and screens used to access specific information were reviewed and quantified to look for trends.

RESULTS

Overall, pharmacists viewed 25.5 total and 15.1 unique EHR screens per case. The majority of time was spent looking at screens focused on medications, followed by screens displaying notes, laboratory values, and vital signs. With regard to medication data, the vast majority of screen visitations were to view information on opioids/sedatives and antibiotics. With regard to laboratory values, the majority of views were focused on basic chemistry and hematology data. While there was significant variance between pharmacists, individual navigation patterns remained constant across cases.

CONCLUSION

The study results suggest that in addition to medication information, laboratory data and clinical notes are key focuses of ICU pharmacist review of patient records and that navigation to multiple screens is required in order to view these data with the EHR. New pharmacy-specific EHR interfaces should consolidate these elements within a primary interface.

Electronic health records using a resource advantage theory perspective: an interdisciplinary literature review

Purpose

Warren Buffett asserted that the greatest issue confronting American business and the economy is rising health-care costs, which have risen to 17% of gross domestic product. Public policymakers, health-care providers and other stakeholders grapple with cost-containment and increased health-care delivery efficiencies. There exists a paucity of theory-driven research addressing how information technology vis-à-vis electronic health records (EHR) may supply a managerial mechanism for increasing bottom-line hospital performance, thereby attaining competitive advantage.

Design/methodology/approach

A systematic interdisciplinary literature review motivated by resource advantage theory (RAT) offers a conceptual foundation for analyzing the financial, informational and physical workflows that are core elements of supply chain management in a hospital.

Findings

RAT links how EHR impacts profitability, competitive advantage and macromarketing factors in hospital supply chains. The literature review provides a research synthesis of the implementation and adoption of EHR to reveal its impact on a hospital’s competitive advantage. Although legislative initiatives like the 2009 Health Information Technology for Economic and Clinical Health Act and the Affordable Care Act encourage EHR adoption, there remains a reluctance for hospitals to do so.

Originality/value

The extant literature precedes the relevant legislation, has incomplete data or focuses solely on patient outcomes.

Clinician collaboration to improve clinical decision support: the Clickbusters initiative

OBJECTIVE

We describe the Clickbusters initiative implemented at Vanderbilt University Medical Center (VUMC), which was designed to improve safety and quality and reduce burnout through the optimization of clinical decision support (CDS) alerts.

MATERIALS AND METHODS

We developed a 10-step Clickbusting process and implemented a program that included a curriculum, CDS alert inventory, oversight process, and gamification. We carried out two 3-month rounds of the Clickbusters program at VUMC. We completed descriptive analyses of the changes made to alerts during the process, and of alert firing rates before and after the program.

RESULTS

Prior to Clickbusters, VUMC had 419 CDS alerts in production, with 488 425 firings (42 982 interruptive) each week. After 2 rounds, the Clickbusters program resulted in detailed, comprehensive reviews of 84 CDS alerts and reduced the number of weekly alert firings by more than 70 000 (15.43%). In addition to the direct improvements in CDS, the initiative also increased user engagement and involvement in CDS.

CONCLUSIONS

At VUMC, the Clickbusters program was successful in optimizing CDS alerts by reducing alert firings and resulting clicks. The program also involved more users in the process of evaluating and improving CDS and helped build a culture of continuous evaluation and improvement of clinical content in the electronic health record.

Impact of Electronic Health Records on Information Practices in Mental Health Contexts: Scoping Review

Background

The adoption of electronic health records (EHRs) and electronic medical records (EMRs) has been slow in the mental health context, partly because of concerns regarding the collection of sensitive information, the standardization of mental health data, and the risk of negatively affecting therapeutic relationships. However, EHRs and EMRs are increasingly viewed as critical to improving information practices such as the documentation, use, and sharing of information and, more broadly, the quality of care provided.

Objective

This paper aims to undertake a scoping review to explore the impact of EHRs on information practices in mental health contexts and also explore how sensitive information, data standardization, and therapeutic relationships are managed when using EHRs in mental health contexts.

Methods

We considered a scoping review to be the most appropriate method for this review because of the relatively recent uptake of EHRs in mental health contexts. A comprehensive search of electronic databases was conducted with no date restrictions for articles that described the use of EHRs, EMRs, or associated systems in the mental health context. One of the authors reviewed all full texts, with 2 other authors each screening half of the full-text articles. The fourth author mediated the disagreements. Data regarding study characteristics were charted. A narrative and thematic synthesis approach was taken to analyze the included studies’ results and address the research questions.

Results

The final review included 40 articles. The included studies were highly heterogeneous with a variety of study designs, objectives, and settings. Several themes and subthemes were identified that explored the impact of EHRs on information practices in the mental health context. EHRs improved the amount of information documented compared with paper. However, mental health–related information was regularly missing from EHRs, especially sensitive information. EHRs introduced more standardized and formalized documentation practices that raised issues because of the focus on narrative information in the mental health context. EHRs were found to disrupt information workflows in the mental health context, especially when they did not include appropriate templates or care plans. Usability issues also contributed to workflow concerns. Managing the documentation of sensitive information in EHRs was problematic; clinicians sometimes watered down sensitive information or chose to keep it in separate records. Concerningly, the included studies rarely involved service user perspectives. Furthermore, many studies provided limited information on the functionality or technical specifications of the EHR being used.

Conclusions

We identified several areas in which work is needed to ensure that EHRs benefit clinicians and service users in the mental health context. As EHRs are increasingly considered critical for modern health systems, health care decision-makers should consider how EHRs can better reflect the complexity and sensitivity of information practices and workflows in the mental health context.

AI and Clinical Decision Making: The Limitations and Risks of Computational Reductionism in Bowel Cancer Screening

Advances in artificial intelligence in healthcare are frequently promoted as ‘solutions’ to improve the accuracy, safety, and quality of clinical decisions, treatments, and care. Despite some diagnostic success, however, AI systems rely on forms of reductive reasoning and computational determinism that embed problematic assumptions about clinical decision-making and clinical practice. Clinician autonomy, experience, and judgement are reduced to inputs and outputs framed as binary or multi-class classification problems benchmarked against a clinician’s capacity to identify or predict disease states. This paper examines this reductive reasoning in AI systems for colorectal cancer (CRC) to highlight their limitations and risks: (1) in AI systems themselves due to inherent biases in (a) retrospective training datasets and (b) embedded assumptions in underlying AI architectures and algorithms; (2) in the problematic and limited evaluations being conducted on AI systems prior to system integration in clinical practice; and (3) in marginalising socio-technical factors in the context-dependent interactions between clinicians, their patients, and the broader health system. The paper argues that to optimise benefits from AI systems and to avoid negative unintended consequences for clinical decision-making and patient care, there is a need for more nuanced and balanced approaches to AI system deployment and evaluation in CRC.

Evaluation of an Electronic Medical Record Module for Nursing Documentation in Paediatric Palliative Care: Involvement of Nurses with a Think-Aloud Approach

Background: Paediatric palliative care (PPC) is a noncurative approach to the care of children and adolescents with life-limiting and life-threatening illnesses. Electronic medical records (EMRs) play an important role in documenting such complex processes. Despite their benefits, they can introduce unintended consequences if future users are not involved in their development. Aim: The aim of this study was to evaluate the acceptance of a novel module for nursing documentation by nurses working in the context of PPC. Methods: An observational study employing concurrent think-aloud and semi-structured qualitative interviews were conducted with 11 nurses working in PPC. Based on the main determinants of the unified theory of acceptance and use of technology (UTAUT), data were analysed using qualitative content analysis. Results: The main determinants of UTAUT were found to potentially influence acceptance of the novel module. Participants perceived the module to be self-explanatory and intuitive. Some adaptations, such as the reduction of fragmentation in the display, the optimization of confusing mouseover fields, and the use of familiar nursing terminology, are reasonable ways of increasing software adoption. Conclusions: After adaptation of the modules based on the results, further evaluation with the participation of future users is required.

A Remote Patient-Monitoring System for Intensive Care Medicine: Mixed Methods Human-Centered Design and Usability Evaluation

Background

Continuous monitoring of vital signs is critical for ensuring patient safety in intensive care units (ICUs) and is becoming increasingly relevant in general wards. The effectiveness of health information technologies such as patient-monitoring systems is highly determined by usability, the lack of which can ultimately compromise patient safety. Usability problems can be identified and prevented by involving users (ie, clinicians).

Objective

In this study, we aim to apply a human-centered design approach to evaluate the usability of a remote patient-monitoring system user interface (UI) in the ICU context and conceptualize and evaluate design changes.

Methods

Following institutional review board approval (EA1/031/18), a formative evaluation of the monitoring UI was performed. Simulated use tests with think-aloud protocols were conducted with ICU staff (n=5), and the resulting qualitative data were analyzed using a deductive analytic approach. On the basis of the identified usability problems, we conceptualized informed design changes and applied them to develop an improved prototype of the monitoring UI. Comparing the UIs, we evaluated perceived usability using the System Usability Scale, performance efficiency with the normative path deviation, and effectiveness by measuring the task completion rate (n=5). Measures were tested for statistical significance using a 2-sample t test, Poisson regression with a generalized linear mixed-effects model, and the N-1 chi-square test. P<.05 were considered significant.

Results

We found 37 individual usability problems specific to monitoring UI, which could be assigned to six subcodes: usefulness of the system, response time, responsiveness, meaning of labels, function of UI elements, and navigation. Among user ideas and requirements for the UI were high usability, customizability, and the provision of audible alarm notifications. Changes in graphics and design were proposed to allow for better navigation, information retrieval, and spatial orientation. The UI was revised by creating a prototype with a more responsive design and changes regarding labeling and UI elements. Statistical analysis showed that perceived usability improved significantly (System Usability Scale design A: mean 68.5, SD 11.26, n=5; design B: mean 89, SD 4.87, n=5; P=.003), as did performance efficiency (normative path deviation design A: mean 8.8, SD 5.26, n=5; design B: mean 3.2, SD 3.03, n=5; P=.001), and effectiveness (design A: 18 trials, failed 7, 39% times, passed 11, 61% times; design B: 20 trials, failed 0 times, passed 20 times; P=.002).

Conclusions

Usability testing with think-aloud protocols led to a patient-monitoring UI with significantly improved usability, performance, and effectiveness. In the ICU work environment, difficult-to-use technology may result in detrimental outcomes for staff and patients. Technical devices should be designed to support efficient and effective work processes. Our results suggest that this can be achieved by applying basic human-centered design methods and principles.

Trial Registration

ClinicalTrials.gov NCT03514173; https://clinicaltrials.gov/ct2/show/NCT03514173

Defining recommendations to guide user interface design: a multimethod approach (Preprint)

Background

For the development of digital solutions, different aspects of user interface design must be taken into consideration. Different technologies, interaction paradigms, user characteristics and needs, and interface design components are some of the aspects that designers and developers should pay attention to when designing a solution. Many user interface design recommendations for different digital solutions and user profiles are found in the literature, but these recommendations have numerous similarities, contradictions, and different levels of detail. A detailed critical analysis is needed that compares, evaluates, and validates existing recommendations and allows the definition of a practical set of recommendations.

Objective

This study aimed to analyze and synthesize existing user interface design recommendations and propose a practical set of recommendations that guide the development of different technologies.

Methods

Based on previous studies, a set of recommendations on user interface design was generated following 4 steps: (1) interview with user interface design experts; (2) analysis of the experts’ feedback and drafting of a set of recommendations; (3) reanalysis of the shorter list of recommendations by a group of experts; and (4) refining and finalizing the list.

Results

The findings allowed us to define a set of 174 recommendations divided into 12 categories, according to usability principles, and organized into 2 levels of hierarchy: generic (69 recommendations) and specific (105 recommendations).

Conclusions

This study shows that user interface design recommendations can be divided according to usability principles and organized into levels of detail. Moreover, this study reveals that some recommendations, as they address different technologies and interaction paradigms, need further work.

Increased Intraoperative Faculty Entrustment and Resident Entrustability Does Not Compromise Patient Outcomes After General Surgery Procedures

BACKGROUND

Intraoperative resident autonomy has been compromised secondary to expectations for increased supervision without defined parameters for safe progressive independence, diffusion of training experience, and more to learn with less time. Surgical residents who are insufficiently entrusted during training attain less autonomy, confidence, and even clinical competency, potentially affecting future patient outcomes.

OBJECTIVE

To determine if OpTrust, an educational intervention for increasing intraoperative faculty entrustment and resident entrustability, negatively impacts patient outcomes after general surgery procedures.

METHODS

Surgical faculty and residents received OpTrust training and instruction to promote intraoperative faculty entrustment and resident entrustability. A post-intervention OpTrust cohort was compared to historical and pre-intervention OpTrust cohorts. Multivariable logistic and negative binomial regression was used to evaluate the impact of the OpTrust intervention and time on patient outcomes.

SETTING

Single tertiary academic center.

PARTICIPANTS

General surgery faculty and residents.

MAIN OUTCOMES AND MEASURES

Thirty-day postoperative outcomes, including mortality, any complication, reoperation, readmission, and length of stay.

RESULTS

A total of 8890 surgical procedures were included. After risk adjustment, overall patient outcomes were similar. Multivariable regression estimating the effect of the OpTrust intervention and time revealed similar patient outcomes with no increased risk (P > 0.05) of mortality {odds ratio (OR), 2.23 [95% confidence interval (CI), 0.87-5.6]}, any complication [OR, 0.98 (95% CI, 0.76-1.3)], reoperation [OR, 0.65 (95% CI, 0.42-1.0)], readmission [OR, 0.82 (95% CI, 0.57-1.2)], and length of stay [OR, 0.99 (95% CI, 0.86-1.1)] compared to the historic and pre-intervention OpTrust cohorts.

CONCLUSIONS

OpTrust, an educational intervention to increase faculty entrustment and resident entrustability, does not compromise postoperative patient outcomes. Integrating faculty and resident development to further enhance entrustment and entrustability through OpTrust may help facilitate increased resident autonomy within the safety net of surgical training without negatively impacting clinical outcomes.

Development and Usability Evaluation of a Telemedicine System for Management and Monitoring of Patients with Diabetic Foot

Objectives: The aim of the present study was to develop and evaluate the usability of a telemedicine system for management and monitoring of patients with diabetic foot.Methods: This study was conducted in four phases. In the first phase, the information needs and characteristics required to design the telemedicine system were identified based on a literature review. Then, in a two-stage Delphi survey, 15 experts approved the identified information needs and characteristics. The prototype telemedicine system was then designed. In the third phase, system usability was evaluated through a semi-structured interview. In the fourth phase, users’ satisfaction with the designed system was analyzed.Results: Out of 115 information needs and required characteristics, 95 were considered in the system design. Eight main pages for enabling patient-physician interactions and physician-physician interactions, monitoring the patient and controlling the disease process, providing medical consultation, and prescribing medications were considered. In the third phase, 26 distinct problems were identified. However, 75% of the participants were very satisfied with the system.Conclusions: This study presents an attempt to design and evaluate a telemedicine system for the management and monitoring of patients with diabetic foot. In this system, patients receiving medical services or physicians who encounter rare cases can send the complete medical history, clinical test results, and videos and images related to the foot to specialist physicians. After examining the medical history or images and videos, the physician can provide the necessary medication prescriptions and laboratory tests or other recommendations.

Saving Time for Patient Care by Optimizing Physician Note Templates: A Pilot Study

Background: At times, electronic medical records (EMRs) have proven to be less than optimal, causing longer hours behind computers, shorter time with patients, suboptimal patient safety, provider dissatisfaction, and physician burnout. These concerning healthcare issues can be positively affected by optimizing EMR usability, which in turn would lead to substantial benefits to healthcare professionals such as increased healthcare professional productivity, efficiency, quality, and accuracy. Documentation issues, such as non-standardization of physician note templates and tedious, time-consuming notes in our mother-baby unit (MBU), were discussed during meetings with stakeholders in the MBU and our hospital's EMR analysts.

Objective: The objective of this study was to assess physician note optimization on saving time for patient care and improving provider satisfaction.

Methods: This quality improvement pilot investigation was conducted in our MBU where four note templates were optimized: History and Physical (H and P), Progress Note (PN), Discharge Summary (DCS), and Hand-Off List (HOL). Free text elements documented elsewhere in the EMR (e.g., delivery information, maternal data, lab result, etc.) were identified and replaced with dynamic links that automatically populate the note with these data. Discrete data pick lists replaced necessary elements that were previously free texts. The new note templates were given new names for ease of accessibility. Ten randomly chosen pediatric residents completed both the old and new note templates for the same control newborn encounter during a period of one year. Time spent and number of actions taken (clicks, keystrokes, transitions, and mouse-keyboard switches) to complete these notes were recorded. Surveys were sent to MBU providers regarding overall satisfaction with the new note templates.

Results: The ten residents' average time saved was 23 min per infant. Reflecting this saved time on the number of infants admitted to our MBU between January 2016 and September, 2019 which was 9373 infants; resulted in 2.6 hours saved per day, knowing that every infant averages two days length of stay. The new note templates required 69 fewer actions taken than the old ones (H and P: 11, PN: 8, DCS: 18, HOL: 32). The provider surveys were consistent with improved provider satisfaction.

Conclusion: Optimizing physician notes saved time for patient care and improved physician satisfaction.

Comparing International Experiences With Electronic Health Records Among Emergency Medicine Physicians in the United States and Norway: Semistructured Interview Study

Background

The variability in physicians’ attitudes regarding electronic health records (EHRs) is widely recognized. Both human and technological factors contribute to user satisfaction. This exploratory study considers these variables by comparing emergency medicine physician experiences with EHRs in the United States and Norway.

Objective

This study is unique as it aims to compare individual experiences with EHRs. It creates an opportunity to expand perspective, challenge the unknown, and explore how this technology affects clinicians globally. Research often highlights the challenge that health information technology has created for users: Are the negative consequences of this technology shared among countries? Does it affect medical practice? What determines user satisfaction? Can this be measured internationally? Do specific factors account for similarities or differences? This study begins by investigating these questions by comparing cohort experiences. Fundamental differences between nations will also be addressed.

Methods

We used semistructured, participant-driven, in-depth interviews (N=12) for data collection in conjunction with ethnographic observations. The conversations were recorded and transcribed. Texts were then analyzed using NVivo software (QSR International) to develop codes for direct comparison among countries. Comprehensive understanding of the data required triangulation, specifically using thematic and interpretive phenomenological analysis. Narrative analysis ensured appropriate context of the NVivo (QSR International) query results.

Results

Each interview resulted in mixed discussions regarding the benefits and disadvantages of EHRs. All the physicians recognized health care’s dependence on this technology. In Norway, physicians perceived more benefits compared with those based in the United States. Americans reported fewer benefits and disproportionally high disadvantages. Both cohorts believed that EHRs have increased user workload. However, this was mentioned 2.6 times more frequently by Americans (United States [n=40] vs Norway [n=15]). Financial influences regarding health information technology use were of great concern for American physicians but rarely mentioned among Norwegian physicians (United States [n=37] vs Norway [n=6]). Technology dysfunctions were the most common complaint from Norwegian physicians. Participants from each country noted increased frustration among older colleagues.

Conclusions

Despite differences spanning geographical, organizational, and cultural boundaries, much is to be learned by comparing individual experiences. Both cohorts experienced EHR-related frustrations, although etiology differed. The overall number of complaints was significantly higher among American physicians. This study augments the idea that policy, regulation, and administration have compelling influence on user experience. Global EHR optimization requires additional investigation, and these results help to establish a foundation for future research.