A comparative observational study of inpatient clinical note-entry and reading/retrieval styles adopted by physicians

Impact of clinical note format on diagnostic accuracy and efficiency

BACKGROUND

Clinician notes are structured in a variety of ways. This research pilot tested an innovative study design and explored the impact of note formats on diagnostic accuracy and documentation review time.

OBJECTIVE

To compare two formats for clinical documentation (narrative format vs. list of findings) on clinician diagnostic accuracy and documentation review time.

METHOD

Participants diagnosed written clinical cases, half in narrative format, and half in list format. Diagnostic accuracy (defined as including correct case diagnosis among top three diagnoses) and time spent processing the case scenario were measured for each format. Generalised linear mixed regression models and bias-corrected bootstrap percentile confidence intervals for mean paired differences were used to analyse the primary research questions.

RESULTS

Odds of correctly diagnosing list format notes were 26% greater than with narrative notes. However, there is insufficient evidence that this difference is significant (75% CI 0.8-1.99). On average the list format notes required 85.6 more seconds to process and arrive at a diagnosis compared to narrative notes (95% CI -162.3, -2.77). Of cases where participants included the correct diagnosis, on average the list format notes required 94.17 more seconds compared to narrative notes (75% CI -195.9, -8.83).

CONCLUSION

This study offers note format considerations for those interested in improving clinical documentation and suggests directions for future research. Balancing the priority of clinician preference with value of structured data may be necessary.

IMPLICATIONS

This study provides a method and suggestive results for further investigation in usability of electronic documentation formats.

A human centered design approach to define and measure documentation quality using an EHR virtual simulation

Electronic health record (EHR) documentation serves multiple functions, including recording patient health status, enabling interprofessional communication, supporting billing, and providing data to support the quality infrastructure of a Learning Healthcare System. There is no definition and standardized method to assess documentation quality in EHRs. Using a human-centered design (HCD) approach, we define and describe a method to measure documentation quality. Documentation quality was defined as timely, accurate, user-centered, and efficient. Measurement of quality used a virtual simulated standardized patient visit via an EHR vendor platform. By observing and recording documentation efforts, nurse practitioners (NPs) (N = 12) documented the delivery of an Age-Friendly Health System (AFHS) 4Ms (what Matters, Medication, Mentation, and Mobility) clinic visit using a standardized case. Results for timely documentation indicated considerable variability in completion times of documenting the 4Ms. Accuracy varied, as there were many types of episodes of erroneous documentation and extra time in seconds in documenting the 4Ms. The type and frequency of erroneous documentation efforts were related to navigation burden when navigating to different documentation tabs. The evaluated system demonstrated poor usability, with most participants scoring between 60 and 70 on the System Usability Scale (SUS). Efficiency, measured as click burden (the number of clicks used to navigate through a software system), revealed significant variability in the number of clicks required, with the NPs averaging approximately 13 clicks above the minimum requirement. The HCD methodology used in this study to assess the documentation quality proved feasible and provided valuable information on the quality of documentation. By assessing the quality of documentation, the gathered data can be leveraged to enhance documentation, optimize user experience, and elevate the quality of data within a Learning Healthcare System.

Examining the Generalizability of Pretrained De-identification Transformer Models on Narrative Nursing Notes

BACKGROUND

 Narrative nursing notes are a valuable resource in informatics research with unique predictive signals about patient care. The open sharing of these data, however, is appropriately constrained by rigorous regulations set by the Health Insurance Portability and Accountability Act (HIPAA) for the protection of privacy. Several models have been developed and evaluated on the open-source i2b2 dataset. A focus on the generalizability of these models with respect to nursing notes remains understudied.

OBJECTIVES

 The study aims to understand the generalizability of pretrained transformer models and investigate the variability of personal protected health information (PHI) distribution patterns between discharge summaries and nursing notes with a goal to inform the future design for model evaluation schema.

METHODS

 Two pretrained transformer models (RoBERTa, ClinicalBERT) fine-tuned on i2b2 2014 discharge summaries were evaluated on our data inpatient nursing notes and compared with the baseline performance. Statistical testing was deployed to assess differences in PHI distribution across discharge summaries and nursing notes.

RESULTS

 RoBERTa achieved the optimal performance when tested on an external source of data, with an F1 score of 0.887 across PHI categories and 0.932 in the PHI binary task. Overall, discharge summaries contained a higher number of PHI instances and categories of PHI compared with inpatient nursing notes.

CONCLUSION

 The study investigated the applicability of two pretrained transformers on inpatient nursing notes and examined the distinctions between nursing notes and discharge summaries concerning the utilization of personal PHI. Discharge summaries presented a greater quantity of PHI instances and types when compared with narrative nursing notes, but narrative nursing notes exhibited more diversity in the types of PHI present, with some pertaining to patient's personal life. The insights obtained from the research help improve the design and selection of algorithms, as well as contribute to the development of suitable performance thresholds for PHI.

Management Opportunities and Challenges After Achieving Widespread Health System Digitization

The adoption of electronic health records (EHRs) and digitization of health data over the past decade is ushering in the next generation of digital health tools that leverage artificial intelligence (AI) to improve varied aspects of health system performance. The decade ahead is therefore shaping up to be one in which digital health becomes even more at the forefront of health care delivery - demanding the time, attention, and resources of health care leaders and frontline staff, and becoming inextricably linked with all dimensions of health care delivery. In this chapter, we look back and look ahead. There are substantive lessons learned from the first era of large-scale adoption of enterprise EHRs and ongoing challenges that organizations are wrestling with - particularly related to the tension between standardization and flexibility/customization of EHR systems and the processes they support. Managing this tension during efforts to implement and optimize enterprise systems is perhaps the core challenge of the past decade, and one that has impeded consistent realization of value from initial EHR investments. We describe these challenges, how they manifest, and organizational strategies to address them, with a specific focus on alignment with broader value-based care transformation. We then look ahead to the AI wave - the massive number of applications of AI to health care delivery, the expected benefits, the risks and challenges, and approaches that health systems can consider to realize the benefits while avoiding the risks.

Through the Narrative Looking Glass: Commentary on “Impact of Electronic Health Records on Information Practices in Mental Health Contexts: A Scoping Review” (Preprint)

The authors of “Impact of Electronic Health Records on Information Practices in Mental Health Contexts: Scoping Review” have effectively brought to our attention the failure of the electronic health record (EHR) to represent the human context. Because mental health or behavioral disorders (and functional status in general) emerge from an interaction between the individual’s characteristics and the social context, it is essentially a failure to represent the human context. The assessment and treatment of these disorders must reflect how the person lives, their degree of social connectedness, their personal motivation, and their cultural background. This type of information is best communicated both through narrative and in collaboration with other providers and the patient—largely because human social memory is organized around situation models and natural episodes. Neither functionality is currently available in most EHRs. Narrative communication is effective for several reasons: (1) it supports the communication of goals between providers; (2) it allows the author to express their belief in others’ perspectives (theory of mind), for example, those who will be reading these notes; and (3) it supports the incorporation of the patient’s personal perspective. The failure of the EHR to support mental health information data and information practices is, therefore, essentially a failure to support the basic communication functions necessary for the narrative. The authors have rightly noted the problems of the EHR in this domain, but perhaps they did not completely link the problems to the lack of functionality to support narrative communication. Suggestions for adding design elements are discussed.

Diagnosis Documentation of Critically Ill Children at Admission to a PICU

OBJECTIVES

Multidisciplinary PICU teams must effectively share information while caring for critically ill children. Clinical documentation helps clinicians develop a shared understanding of the patient's diagnosis, which informs decision-making. However, diagnosis-related documentation in the PICU is understudied, thus limiting insights into how pediatric intensivists convey their diagnostic reasoning. Our objective was to describe how pediatric critical care clinicians document patients' diagnoses at PICU admission.

DESIGN

Retrospective mixed methods study describing diagnosis documentation in electronic health records.

SETTING

Academic tertiary referral PICU.

PATIENTS

Children 0-17 years old admitted nonelectively to a single PICU over 1 year.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

One hundred PICU admission notes for 96 unique patients were reviewed. In 87% of notes, both attending physicians and residents or advanced practice providers documented a primary diagnosis; in 13%, primary diagnoses were documented by residents or advanced practice providers alone. Most diagnoses (72%) were written as narrative free text, 11% were documented as problem lists/billing codes, and 17% used both formats. At least one rationale was documented to justify the primary diagnosis in 91% of notes. Diagnostic uncertainty was present in 52% of notes, most commonly suggested by clinicians' use of words indicating uncertainty (65%) and documentation of differential diagnoses (60%). Clinicians' integration and interpretation of information varied in terms of: 1) organization of diagnosis narratives, 2) use of contextual details to clarify the diagnosis, and 3) expression of diagnostic uncertainty.

CONCLUSIONS

In this descriptive study, most PICU admission notes documented a rationale for the primary diagnosis and expressed diagnostic uncertainty. Clinicians varied widely in how they organized diagnostic information, used contextual details to clarify the diagnosis, and expressed uncertainty. Future work is needed to determine how diagnosis narratives affect clinical decision-making, patient care, and outcomes.

Characterizing styles of clinical note production and relationship to clinical work hours among first-year residents

OBJECTIVE

To characterize variation in clinical documentation production patterns, how this variation relates to individual resident behavior preferences, and how these choices relate to work hours.

MATERIALS AND METHODS

We used unsupervised machine learning with clinical note metadata for 1265 progress notes written for 279 patient encounters by 50 first-year residents on the Hospital Medicine service in 2018 to uncover distinct note-level and user-level production patterns. We examined average and 95% confidence intervals of median user daily work hours measured from audit log data for each user-level production pattern.

RESULTS

Our analysis revealed 10 distinct note-level and 5 distinct user-level production patterns (user styles). Note production patterns varied in when writing occurred and in how dispersed writing was through the day. User styles varied in which note production pattern(s) dominated. We observed suggestive trends in work hours for different user styles: residents who preferred producing notes in dispersed sessions had higher median daily hours worked while residents who preferred producing notes in the morning or in a single uninterrupted session had lower median daily hours worked.

DISCUSSION

These relationships suggest that note writing behaviors should be further investigated to understand what practices could be targeted to reduce documentation burden and derivative outcomes such as resident work hour violations.

CONCLUSION

Clinical note documentation is a time-consuming activity for physicians; we identify substantial variation in how first-year residents choose to do this work and suggestive trends between user preferences and work hours.

Feeling and thinking: can theories of human motivation explain how EHR design impacts clinician burnout?

The psychology of motivation can help us understand the impact of electronic health records (EHRs) on clinician burnout both directly and indirectly. Informatics approaches to EHR usability tend to focus on the extrinsic motivation associated with successful completion of clearly defined tasks in clinical workflows. Intrinsic motivation, which includes the need for autonomy, sense-making, creativity, connectedness, and mastery is not well supported by current designs and workflows. This piece examines existing research on the importance of 3 psychological drives in relation to healthcare technology: goal-based decision-making, sense-making, and agency/autonomy. Because these motives are ubiquitous, foundational to human functioning, automatic, and unconscious, they may be overlooked in technological interventions. The results are increased cognitive load, emotional distress, and unfulfilling workplace environments. Ultimately, we hope to stimulate new research on EHR design focused on expanding functionality to support intrinsic motivation, which, in turn, would decrease burnout and improve care.

Building the evidence-base to reduce electronic health record-related clinician burden

Clinicians face competing pressures of being clinically productive while using imperfect electronic health record (EHR) systems and maximizing face-to-face time with patients. EHR use is increasingly associated with clinician burnout and underscores the need for interventions to improve clinicians' experiences. With an aim of addressing this need, we share evidence-based informatics approaches, pragmatic next steps, and future research directions to improve 3 of the highest contributors to EHR burden: (1) documentation, (2) chart review, and (3) inbox tasks. These approaches leverage speech recognition technologies, natural language processing, artificial intelligence, and redesign of EHR workflow and user interfaces. We also offer a perspective on how EHR vendors, healthcare system leaders, and policymakers all play an integral role while sharing responsibility in helping make evidence-based sociotechnical solutions available and easy to use.

Electronic health record note review in an outpatient specialty clinic: who is looking?

Note entry and review in electronic health records (EHRs) are time-consuming. While some clinics have adopted team-based models of note entry, how these models have impacted note review is unknown in outpatient specialty clinics such as ophthalmology. We hypothesized that ophthalmologists and ancillary staff review very few notes. Using audit log data from 9775 follow-up office visits in an academic ophthalmology clinic, we found ophthalmologists reviewed a median of 1 note per visit (2.6 ± 5.3% of available notes), while ancillary staff reviewed a median of 2 notes per visit (4.1 ± 6.2% of available notes). While prior ophthalmic office visit notes were the most frequently reviewed note type, ophthalmologists and staff reviewed no such notes in 51% and 31% of visits, respectively. These results highlight the collaborative nature of note review and raise concerns about how cumbersome EHR designs affect efficient note review and the utility of prior notes in ophthalmic clinical care.

Physicians' perceptions about narrative note sections format and content: A multi-specialty survey

OBJECTIVE

To assess physicians' perceptions about narrative note sections format and content commonly reported in visit notes to inform future research and EHR development.

METHODS

We conducted two online surveys with a multi-specialty panel of outpatient physicians from a large health system to collect their perceptions of the usefulness of three narrative formats and the relevance of content reported in the note sections history of present illness (HPI) and assessment and plan (AP). Survey questions were responded with a 7-point Likert scale and include two open-ended questions for comments on challenges and suggestions related to electronic clinical documentation.

RESULTS

Eighty-eight physicians completed the surveys. The most preferred format for HPI was story (i.e., coherent paragraph), followed by list without categories (i.e., non-categorized sentences) and list with categories (i.e., categorized sentences). The most preferred format for AP was list with categories, followed by story and list without categories. The most relevant type of content in HPI was temporal information and finding/condition. The most relevant type of content reported in AP was intervention and reasons and justifications. Challenges frequently mentioned include suboptimal note creation interfaces and bloated notes, and the most common suggestions for improvements are related to note entry facilitators and organizational improvements.

CONCLUSION

Physicians' input is extremely valuable to inform improvements to EHRs. More effective clinical documentation systems should include less intrusive, more intuitive and automated user interfaces for note creation, smarter autopoluation functionality and linkage between note content and data from other parts of the record.

Effect of Outpatient Note Templates on Note Quality: NOTE (Notation Optimization through Template Engineering) Randomized Clinical Trial

BACKGROUND

This is the first randomized controlled trial evaluating the impact of note template design on note quality using a simulated patient encounter and a validated assessment tool.

OBJECTIVE

To compare note quality between two different templates using a novel randomized clinical simulation process.

DESIGN

A randomized non-blinded controlled trial of a standard note template versus redesigned template.

PARTICIPANTS

PGY 1-3 IM residents.

INTERVENTIONS

Residents documented the simulated patient encounter using one of two templates. The standard template was modeled after the usual outpatient progress note. The new template placed the assessment and plan section in the beginning, grouped subjective data into the assessment, and deemphasized less useful elements.

MAIN MEASURES

Note length; time to note completion; note template evaluation by resident authors; note evaluation by faculty reviewers.

KEY RESULTS

36 residents participated, 19 randomized to standard template, 17 to new. New template generated shorter notes (103 vs 285 lines, p < 0.001) that took the same time to complete (19.8 vs 21.6 min, p = 0.654). Using a 5-point Likert scale, residents considered new notes to have increased visual appeal (4 vs 3, p = 0.05) and less redundancy and clutter (4 vs 3, p = 0.006). Overall template satisfaction was not statistically different. Faculty reviewers rated the standard note more up-to-date (4.3 vs 2.7, p = 0.001), accurate (3.9 vs 2.6, p = 0.003), and useful (4 vs 2.8, p = 0.002), but less organized (3.3 vs 4.5, p < 0.001). Total quality was not statistically different.

CONCLUSIONS

Residents rated the new note template more visually appealing, shorter, and less cluttered. Faculty reviewers rated both note types equivalent in the overall quality but rated new notes inferior in terms of accuracy and usefulness though better organized. This study demonstrates a novel method of a simulated clinical encounter to evaluate note templates before the introduction into practice.

TRIAL REGISTRATION

ClinicalTrials.gov ID: NCT04333238.

The national health information technology human factors and ergonomics agenda

Health information technology (IT) implementation has encompassed much of the United States health care system over the past decade, and user frustration with health IT has steadily increased. Human factors and ergonomics (HFE) methods and approaches can improve the design, implementation, and use of health IT for clinicians and consumers. To better understand the effect of federal HFE in health IT research funding, the authors conducted a review of several key, specific initiatives. The review focused on the goals and accomplishments of these initiatives. Findings to date show that HFE is improving the usefulness of health IT, but additional research and new methods are needed. Corresponding research funding and policy priorities are identified. New HFE work and innovative approaches are needed to capitalize on HFE knowledge, principles, and methods to improve the design, implementation, and use of health IT at a broader scale.

[Legitimate resistance without technophobia: Analysis of electronic medical records impacts on the medical profession]

The objective of this short narrative literature review is to highlight the different difficulties encountered by medical doctor in the daily use of EMR. We show that these are not simple transitional phenomena related to a "resistance to change", but rather the fact of a deeper and unfinished transformation. Beyond the "perception of misfit with work processes" or the threat of a loss of autonomy, we propose to analyze this so-called "resistance" in relation to the formalization of medical work induced by EMR. Our question concerns the compatibility of the multiple objectives of EMR, the potential influence of computerization on the steps of entering and consulting medical information, the impact on the clinical reasoning, the reality of assistance to medical "performance". The question is not so much what EMRs do less well than the paper record, but to provide insights into how tomorrow's EMRs will do better than today's.

Clinicians' reasoning as reflected in electronic clinical note-entry and reading/retrieval: a systematic review and qualitative synthesis

Objective

To describe the literature exploring the use of electronic health record (EHR) systems to support creation and use of clinical documentation to guide future research.

Materials and Methods

We searched databases including MEDLINE, Scopus, and CINAHL from inception to April 20, 2018, for studies applying qualitative or mixed-methods examining EHR use to support creation and use of clinical documentation. A qualitative synthesis of included studies was undertaken.

Results

Twenty-three studies met the inclusion criteria and were reviewed in detail. We briefly reviewed 9 studies that did not meet the inclusion criteria but provided recommendations for EHR design. We identified 4 key themes: purposes of electronic clinical notes, clinicians' reasoning for note-entry and reading/retrieval, clinicians' strategies for note-entry, and clinicians' strategies for note-retrieval/reading. Five studies investigated note purposes and found that although patient care is the primary note purpose, non-clinical purposes have become more common. Clinicians' reasoning studies (n = 3) explored clinicians' judgement about what to document and represented clinicians' thought process in cognitive pathways. Note-entry studies (n = 6) revealed that what clinicians document is affected by EHR interfaces. Lastly, note-retrieval studies (n = 12) found that "assessment and plan" is the most read note section and what clinicians read is affected by external stimuli, care/information goals, and what they know about the patient.

Conclusion

Despite the widespread adoption of EHRs, their use to support note-entry and reading/retrieval is still understudied. Further research is needed to investigate approaches to capture and represent clinicians' reasoning and improve note-entry and retrieval/reading.

Copy-and-Paste in Medical Student Notes: Extent, Temporal Trends, and Relationship to Scholastic Performance

BACKGROUND

Medical students may observe and subsequently perpetuate redundancy in clinical documentation, but the degree of redundancy in student notes and whether there is an association with scholastic performance are unknown.

OBJECTIVES

This study sought to quantify redundancy, defined generally as the proportion of similar text between two strings, in medical student notes and evaluate the relationship between note redundancy and objective indicators of student performance.

METHODS

Notes generated by medical students rotating through their medicine clerkship during a single academic year at our institution were analyzed. A student-patient interaction (SPI) was defined as a history and physical and at least two contiguous progress notes authored by the same student during a single patient's hospitalization. For some students, SPI pairs were available from early and late in the clerkship. Redundancy between analogous sections of consecutive notes was calculated on a 0 to 100% scale and was derived from edit distance, the number of changes needed to transform one text string into another. Indicators of student performance included United States Medical Licensing Exam (USMLE) scores.

RESULTS

Ninety-four single SPIs and 58 SPI pairs were analyzed. Redundancy in the assessment/plan section was high (40%) and increased within individual SPIs (to 60%; p < 0.001) and between SPI pairs over the course of the clerkship (by 30-40%; p < 0.001). Students in the lowest tertile of USMLE step II clinical knowledge scores had higher redundancy in the assessment/plan section than their classmates (67 ± 24% vs. 38 ± 22%; p = 0.002).

CONCLUSION

During the medicine clerkship, the assessment/plan section of medical student notes became more redundant over a patient's hospital course and as students gained clinical experience. These trends may be indicative of deficiencies in clinical knowledge or reasoning, as evidenced by performance on some standardized evaluations.

A Daily Hospital Progress Note that Increases Physician Usability of the Electronic Health Record by Facilitating a Problem-Oriented Approach to the Patient and Reducing Physician Clerical Burden

We suggest changes in the electronic health record (EHR) in hospitalized patients to increase EHR usability by optimizing the physician's ability to approach the patient in a problem-oriented fashion and by reducing physician data entry and chart navigation. The framework for these changes is a Physician's Daily Hospital Progress Note organized into 3 sections: Subjective, Objective, and a combined Assessment and Plan section, subdivided by problem titles. The EHR would consolidate information for each problem by: 1) juxtaposing to each problem title relevant medications, key durable results, and limitations; 2) entering in the running lists under Assessment and Plan the most relevant information for that day, including abbreviated versions of relevant reports; and 3) generating a flow sheet in a problem's progress note for any key results tracked daily. To reduce physician EHR navigation, the EHR would place in the Objective section abbreviated versions of notes of other physicians, nurses, and allied health professionals as well as recent orders. The physician would enter only the analysis and plan and new information not included in the EHR. The consolidation of information for each problem would facilitate physician communication at points of transition of care including generation of a problem-oriented discharge summary.

Unintended Consequences of Nationwide Electronic Health Record Adoption: Challenges and Opportunities in the Post-Meaningful Use Era

The US health system has recently achieved widespread adoption of electronic health record (EHR) systems, primarily driven by financial incentives provided by the Meaningful Use (MU) program. Although successful in promoting EHR adoption and use, the program, and other contributing factors, also produced important unintended consequences (UCs) with far-reaching implications for the US health system. Based on our own experiences from large health information technology (HIT) adoption projects and a collection of key studies in HIT evaluation, we discuss the most prominent UCs of MU: failed expectations, EHR market saturation, innovation vacuum, physician burnout, and data obfuscation. We identify challenges resulting from these UCs and provide recommendations for future research to empower the broader medical and informatics communities to realize the full potential of a now digitized health system. We believe that fixing these unanticipated effects will demand efforts from diverse players such as health care providers, administrators, HIT vendors, policy makers, informatics researchers, funding agencies, and outside developers; promotion of new business models; collaboration between academic medical centers and informatics research departments; and improved methods for evaluations of HIT.

Electronic decision support for treatment of hospitalized smokers: A qualitative analysis of physicians' knowledge, attitudes, and practices

BACKGROUND

We recently demonstrated the ability of a suite of tools embedded in an electronic medical record (EMR) to improve tobacco cessation treatment for adult smokers admitted to the hospital. A randomized controlled trial conducted by our group demonstrated the ability of an EMR-embedded tobacco use disorder treatment tool, the Electronic Support Tool and Orders for the Prevention of Smoking (E-STOPS), to increase the identification and treatment of smokers, but its uptake varied among 126 physicians randomized to the intervention arm. The purpose of this study was to identify facilitators and barriers to using E-STOPS.

METHODS

Semi-structured individual interviews from a purposive sample of 12 hospitalist attending physicians and nine internal medicine residents who were randomized to the E-STOPS intervention were analyzed thematically.

RESULTS

Three themes shaped E-STOPS use: the inpatient environment, prescriber attitudes and beliefs, and information needs. Overall, participants were pleased with E-STOPS, but had specific suggestions for improvements regarding the timing of the intervention, suppression logic, and additional decision support and training. A few had concerns about the clinical appropriateness of beginning treatment for tobacco dependence during a hospitalization and the proper role of the inpatient team in that treatment.

CONCLUSIONS

Tobacco dependence treatment for hospitalized smokers and facilitated by the EMR is generally acceptable to hospitalists and resident physicians. Improvements in provider training and feedback as well as the timing and content of the electronic tools may increase their utilization by inpatient physicians.

Clinical Documentation in Electronic Health Record Systems: Analysis of Patient Record Review During Outpatient Ophthalmology Visits

Busy clinicians struggle with productivity and usability in electronic health record systems (EHRs). While previous studies have investigated documentation practices and strategies in the inpatient setting, outpatient documentation and review practices by clinicians using EHRs are relatively unknown. In this study, we look at clinicians' patterns of note review in the EHR during outpatient follow-up office visits in ophthalmology. Key findings from this study are that the number and percentage of notes reviewed is very low, there is variation between providers, specialties, and users, and staff access more notes than physicians. These findings suggest that the vast majority of content in the EHR is not being used by clinicians; improved EHR designs would better present this data and support the information needs of outpatient clinicians.

Usability Evaluation of Electronic Health Record System around Clinical Notes Usage-An Ethnographic Study

Background A substantial gap exists between current Electronic Health Record (EHR) usability and potential optimal usability. One of the fundamental reasons for this discrepancy is poor incorporation of a User-Centered Design (UCD) approach during the Graphical User Interface (GUI) development process. Objective To evaluate usability strengths and weaknesses of two widely implemented EHR GUIs for critical clinical notes usage tasks. Methods Twelve Internal Medicine resident physicians interacting with one of the two EHR systems (System-1 at Location-A and System-2 at Location-B) were observed by two usability evaluators employing an ethnographic approach. User comments and observer findings were analyzed for two critical tasks: (1) clinical notes entry and (2) related information-seeking tasks. Data were analyzed from two standpoints: (1) usability references categorized by usability evaluators as positive, negative, or equivocal and (2) usability impact of each feature measured through a 7-point severity rating scale. Findings were also validated by user responses to a post observation questionnaire. Results For clinical notes entry, System-1 surpassed System-2 with more positive (26% vs. 12%) than negative (12% vs. 34%) usability references. Greatest impact features on EHR usability (severity score pertaining to each feature) for clinical notes entry were: autopopulation (6), screen options (5.5), communication (5), copy pasting (4.5), error prevention (4.5), edit ability (4), and dictation and transcription (3.5). Both systems performed equally well on information-seeking tasks and features with greatest impacts on EHR usability were navigation for notes (7) and others (e.g., looking for ancillary data; 5.5). Ethnographic observations were supported by follow-up questionnaire responses. Conclusion This study provides usability-specific insights to inform future, improved, EHR interface that is better aligned with UCD approach.

Use of Simulation Based on an Electronic Health Records Environment to Evaluate the Structure and Accuracy of Notes Generated by Medical Scribes: Proof-of-Concept Study

Background

The increasing adoption of electronic health records (EHRs) has been associated with a number of unintended negative consequences with provider efficiency and job satisfaction. To address this, there has been a dramatic increase in the use of medical scribes to perform many of the required EHR functions. Despite this rapid growth, little has been published on the training or assessment tools to appraise the safety and efficacy of scribe-related EHR activities. Given the number of reports documenting that other professional groups suffer from a number of performance errors in EHR interface and data gathering, scribes likely face similar challenges. This highlights the need for new assessment tools for medical scribes.

Objective

The objective of this study was to develop a virtual video-based simulation to demonstrate and quantify the variability and accuracy of scribes’ transcribed notes in the EHR.

Methods

From a pool of 8 scribes in one department, a total of 5 female scribes, intent on pursuing careers in health care, with at least 6 months of experience were recruited for our simulation study. We created three simulated patient-provider scenarios. Each scenario contained a corresponding medical record in our simulation instance of our EHR. For each scenario, we video-recorded a standardized patient-provider encounter. Five scribes with at least 6 months of experience both with our EHR and in the specialty of the simulated cases were recruited. Each scribe watched the simulated encounter and transcribed notes into a simulated EHR environment. Transcribed notes were evaluated for interscribe variability and compared with a gold standard for accuracy.

Results

All scribes completed all simulated cases. There was significant interscribe variability in note structure and content. Overall, only 26% of all data elements were unique to the scribe writing them. The term data element was used to define the individual pieces of data that scribes perceived from the simulation. Note length was determined by counting the number of words varied by 31%, 37%, and 57% between longest and shortest note between the three cases, and word economy ranged between 23% and 71%. Overall, there was a wide inter- and intrascribe variation in accuracy for each section of the notes with ranges from 50% to 76%, resulting in an overall positive predictive value for each note between 38% and 81%.

Conclusions

We created a high-fidelity, video-based EHR simulation, capable of assessing multiple performance indicators in medical scribes. In this cohort, we demonstrate significant variability both in terms of structure and accuracy in clinical documentation. This form of simulation can provide a valuable tool for future development of scribe curriculum and assessment of competency.

The Effect of Medical Recording Training on Quantity and Quality of Recording in Gynecology Residents of Tabriz University of Medical Sciences

Introduction: Medical records contain valuable information about a patient's medical history and treatment. Patient safety is one of the most important dimensions of health care quality assurance and performance improvement. Completing the process of documentation is necessary to continue patient care and continuous quality improvement of basic services. The aim of the present study was to evaluate the effect of medical recording education on the quantity and quality of recording in gynecology residents of Tabriz University of Medical Sciences.

Methods: This study is a quasi-experimental study and was conducted at Al-Zahra Teaching Hospital, Tabriz, Iran, in 2016. Thirty-two second through fourth year gynecologic residents of Tabriz University of Medical Sciences who were willing to participate in the study were included by census sampling and participated in training workshop. Three evaluators reviewed the residents’ records before and after training course by a checklist. Statistical analyses were performed using SPSS 13 software. P-values less than 0.05 were considered statistically significant.

Results: The results showed that before the intervention, there were significant differences in the quantity of information status among the evaluators and no significant difference was observed in the recording of qualitative status. After the workshop, among the 3 evaluators, there were also significant differences in the quantity of data recording status; however, no significant change was observed in recording of qualitative status.

Conclusion: The study findings revealed that a sectional training course of correct and standardized medical records has no effect on reforming the process of recording.