A new instrument for measuring clinician satisfaction with electronic health records

Measuring cognitive effort using tabular transformer-based language models of electronic health record-based audit log action sequences

OBJECTIVES

To develop and validate a novel measure, action entropy, for assessing the cognitive effort associated with electronic health record (EHR)-based work activities.

MATERIALS AND METHODS

EHR-based audit logs of attending physicians and advanced practice providers (APPs) from four surgical intensive care units in 2019 were included. Neural language models (LMs) were trained and validated separately for attendings' and APPs' action sequences. Action entropy was calculated as the cross-entropy associated with the predicted probability of the next action, based on prior actions. To validate the measure, a matched pairs study was conducted to assess the difference in action entropy during known high cognitive effort scenarios, namely, attention switching between patients and to or from the EHR inbox.

RESULTS

Sixty-five clinicians performing 5 904 429 EHR-based audit log actions on 8956 unique patients were included. All attention switching scenarios were associated with a higher action entropy compared to non-switching scenarios (P < .001), except for the from-inbox switching scenario among APPs. The highest difference among attendings was for the from-inbox attention switching: Action entropy was 1.288 (95% CI, 1.256-1.320) standard deviations (SDs) higher for switching compared to non-switching scenarios. For APPs, the highest difference was for the to-inbox switching, where action entropy was 2.354 (95% CI, 2.311-2.397) SDs higher for switching compared to non-switching scenarios.

DISCUSSION

We developed a LM-based metric, action entropy, for assessing cognitive burden associated with EHR-based actions. The metric showed discriminant validity and statistical significance when evaluated against known situations of high cognitive effort (ie, attention switching). With additional validation, this metric can potentially be used as a screening tool for assessing behavioral action phenotypes that are associated with higher cognitive burden.

CONCLUSION

An LM-based action entropy metric-relying on sequences of EHR actions-offers opportunities for assessing cognitive effort in EHR-based workflows.

Utilizing nursing standards in electronic health records: A descriptive qualitative study

BACKGROUND

The electronic health record (EHR), including standardized structures and languages, represents an important data source for nurses, to continually update their individual and shared perceptual understanding of clinical situations. Registered nurses' utilization of nursing standards, such as standardized nursing care plans and language in EHRs, has received little attention in the literature. Further research is needed to understand nurses' care planning and documentation practice.

AIMS

This study aimed to describe the experiences and perceptions of nurses' EHR documentation practices utilizing standardized nursing care plans including standardized nursing language, in the daily documentation of nursing care for patients living in special dementia-care units in nursing homes in Norway.

METHODS

A descriptive qualitative study was conducted between April and November 2021 among registered nurses working in special dementia care units in Norwegian nursing homes. In-depth interviews were conducted, and data was analyzed utilizing reflexive thematic analysis with a deductive orientation. Findings Four themes were generated from the analysis. First, the knowledge, skills, and attitude of system users were perceived to influence daily documentation practice. Second, management and organization of documentation work, internally and externally, influenced motivation and engagement in daily documentation processes. Third, usability issues of the EHR were perceived to limit the daily workflow and the nurses' information-needs. Last, nursing standards in the EHR were perceived to contribute to the development of documentation practices, supporting and stimulating ethical awareness, cognitive processes, and knowledge development.

CONCLUSION

Nurses and nursing leaders need to be continuously involved and engaged in EHR documentation to safeguard development and implementation of relevant nursing standards.

Improving usability of Electronic Health Records in a UK Mental Health setting: a feasibility study

Background

Electronic Health Records (EHRs) can help clinicians to plan, document and deliver care for patients in healthcare services. When used consistently, EHRs can advance patient safety and quality, and reduce clinician’s workload. However, usability problems can make it difficult for clinicians to use EHRs effectively, which can negatively impact both healthcare professionals and patients.

Objective

To improve usability of EHRs within a mental health service in the UK.

Methods

This was a feasibility study conducted with two mental health teams. A mixed-methods approach was employed. Focus group discussions with clinicians identified existing usability problems in EHRs and changes were made to address these problems. Updated EHR assessment forms were evaluated by comparing the following measures pre and post changes: (1) usability testing to monitor time spent completing and duplicating patient information in EHRs, (2) clinician’s experience of using EHRs, and (3) proportion of completed EHR assessment forms.

Results

Usability testing with clinicians (n = 3) showed that the time taken to complete EHR assessment forms and time spent duplicating patient information decreased. Clinician’s experience of completing EHR assessment forms also significantly improved post changes compared to baseline (n = 71; p < 0.005). There was a significant increase in completion of most EHR forms by both teams after EHR usability improvements (all at p < 0.01).

Conclusions

Usability improvements to EHRs can reduce the time taken to complete forms, advance clinician’s experience and increase usage of EHRs. It is important to engage healthcare professionals in the usability improvement process of EHRs in mental health services.

Supplementary information

The online version contains supplementary material available at 10.1007/s10916-022-01832-0.

Frontline Nurse Feedback During the Development of a System to Track Cleaning of Portable Medical Equipment

As part of the development and testing of an innovative technology for tracking disinfection of portable medical equipment, end-user feedback was obtained during an initial trial on two acute care hospital units. The disinfection tracking device was installed on the computers-on-wheels and vital signs machines. Each device had the capability of detecting a cleaning event, reporting the event to an online database, and displaying the time since last cleaning event on a visual display. End-user feedback regarding functionality, usefulness of information provided, and impact on workflow was obtained by survey and facilitated group discussions. Seventeen frontline nurses completed the anonymous survey, and 22 participated in the facilitated group discussions. End users found the system functionally easy to use and the information about time since last cleaning useful and reported minimum disruption of workflow. Functionality of the system was confirmed by consistency between recorded and self-reported cleaning patterns. Managers found the data on cleaning of portable medical equipment helpful in validating compliance with hospital equipment cleaning policy. Frontline staff expressed appreciation for technology that helps them and improves outcomes but also discussed concerns about the potential for technology that creates extra work and disruption in the busy frontline nursing care delivery environment. Nurses were appreciative of opportunities to provide feedback and input into efforts to develop and introduce technology. Recorded cleaning events coincided with self-reported equipment cleaning patterns and illustrated that the device efficiently collects information deemed useful by the end user.

Electronic Health Record Usability: Associations With Nurse and Patient Outcomes in Hospitals

BACKGROUND

Electronic health record (EHR) usability issues represent an emerging threat to the wellbeing of nurses and patients; however, few large studies have examined these relationships.

OBJECTIVE

To examine associations between EHR usability and nurse job (burnout, job dissatisfaction, and intention to leave) and surgical patient (inpatient mortality and 30-day readmission) outcomes.

METHODS

A cross-sectional analysis of linked American Hospital Association, state patient discharge, and nurse survey data was conducted. The sample included 343 hospitals, 1,281,848 surgical patients, and 12,004 nurses. Logistic regression models were used to assess relationships between EHR usability and outcomes, before and after accounting for EHR adoption level (comprehensive vs. basic or less) and other confounders.

RESULTS

In fully adjusted models, nurses who worked in hospitals with poorer EHR usability had significantly higher odds of burnout [odds ratio (OR), 1.41; 95% confidence interval (CI), 1.21-1.64], job dissatisfaction (OR, 1.61; 95% CI, 1.37-1.90) and intention to leave (OR, 1.31; 95% CI, 1.09-1.58) compared with nurses working in hospitals with better usability. Surgical patients treated in hospitals with poorer EHR usability had significantly higher odds of inpatient mortality (OR, 1.21; 95% CI, 1.09-1.35) and 30-day readmission (OR, 1.06; 95% CI, 1.01-1.12) compared with patients in hospitals with better usability. Comprehensive EHR adoption was associated with higher odds of nurse burnout (OR, 1.14; 95% CI, 1.01-1.28).

CONCLUSION

Employing EHR systems with suboptimal usability was associated with higher odds of adverse nurse job outcomes and surgical patient mortality and readmission. EHR usability may be more important to nurse job and patient outcomes than comprehensive EHR adoption.

Conceptual considerations for using EHR-based activity logs to measure clinician burnout and its effects

Electronic health records (EHR) use is often considered a significant contributor to clinician burnout. Informatics researchers often measure clinical workload using EHR-derived audit logs and use it for quantifying the contribution of EHR use to clinician burnout. However, translating clinician workload measured using EHR-based audit logs into a meaningful burnout metric requires an alignment with the conceptual and theoretical principles of burnout. In this perspective, we describe a systems-oriented conceptual framework to achieve such an alignment and describe the pragmatic realization of this conceptual framework using 3 key dimensions: standardizing the measurement of EHR-based clinical work activities, implementing complementary measurements, and using appropriate instruments to assess burnout and its downstream outcomes. We discuss how careful considerations of such dimensions can help in augmenting EHR-based audit logs to measure factors that contribute to burnout and for meaningfully assessing downstream patient safety outcomes.

Investigating Evaluation Frameworks for Electronic Health Record: A Literature Review

BACKGROUND: There are various electronic health records (EHRs) evaluation frameworks with multiple dimensions and numerous sets of evaluation measures, while the coverage rate of evaluation measures in a common framework varies in different studies. AIM: This study provides a literature review of the current EHR evaluation frameworks and a model for measuring the coverage rate of evaluation measures in EHR frameworks. METHODS: The current study was a comprehensive literature review and a critical appraisal study. The study was conducted in three phases. In Phase 1, a literature review of EHR evaluation frameworks was conducted. In Phase 2, a three-level hierarchical structure was developed, which includes three aspects, 12 dimensions, and 110 evaluation measures. Subsequently, evaluation measures in the identified studies were categorized based on the hierarchical structure. In Phase 3, relative frequency (RF) of evaluation measures in different dimensions and aspects for each of the identified studies were determined and categorized as follows: Appropriate, moderate, and low coverage. RESULTS: Out of a total of 8276 retrieved articles, 62 studies were considered relevant. The RF range in the second and third level of the hierarchical structure was between 8.6%–91.94% and 0.2%–61%, respectively. “Ease of use” and “system quality” were the most frequent evaluation measure and dimension. Our results indicate that identified studies cover at least one and at most nine evaluation dimensions and current evaluation frameworks focus more on the technology aspect. Almost in all identified studies, evaluation measures related to the technology aspect were covered. However, evaluation measures related to human and organization aspects were covered in 68% and 84% of the identified studies, respectively. CONCLUSION: In this study, we systematically reviewed all literature presenting any type of EHR evaluation framework and analyzed and discussed their aspects and features. We believe that the findings of this study can help researchers to review and adopt the EHR evaluation frameworks for their own particular field of usage.

Implementation outcome instruments for use in physical healthcare settings: a systematic review

BACKGROUND

Implementation research aims to facilitate the timely and routine implementation and sustainment of evidence-based interventions and services. A glaring gap in this endeavour is the capability of researchers, healthcare practitioners and managers to quantitatively evaluate implementation efforts using psychometrically sound instruments. To encourage and support the use of precise and accurate implementation outcome measures, this systematic review aimed to identify and appraise studies that assess the measurement properties of quantitative implementation outcome instruments used in physical healthcare settings.

METHOD

The following data sources were searched from inception to March 2019, with no language restrictions: MEDLINE, EMBASE, PsycINFO, HMIC, CINAHL and the Cochrane library. Studies that evaluated the measurement properties of implementation outcome instruments in physical healthcare settings were eligible for inclusion. Proctor et al.'s taxonomy of implementation outcomes was used to guide the inclusion of implementation outcomes: acceptability, appropriateness, feasibility, adoption, penetration, implementation cost and sustainability. Methodological quality of the included studies was assessed using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. Psychometric quality of the included instruments was assessed using the Contemporary Psychometrics checklist (ConPsy). Usability was determined by number of items per instrument.

RESULTS

Fifty-eight publications reporting on the measurement properties of 55 implementation outcome instruments (65 scales) were identified. The majority of instruments assessed acceptability (n = 33), followed by appropriateness (n = 7), adoption (n = 4), feasibility (n = 4), penetration (n = 4) and sustainability (n = 3) of evidence-based practice. The methodological quality of individual scales was low, with few studies rated as 'excellent' for reliability (6/62) and validity (7/63), and both studies that assessed responsiveness rated as 'poor' (2/2). The psychometric quality of the scales was also low, with 12/65 scales scoring 7 or more out of 22, indicating greater psychometric strength. Six scales (6/65) rated as 'excellent' for usability.

CONCLUSION

Investigators assessing implementation outcomes quantitatively should select instruments based on their methodological and psychometric quality to promote consistent and comparable implementation evaluations. Rather than developing ad hoc instruments, we encourage further psychometric testing of instruments with promising methodological and psychometric evidence.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO 2017 CRD42017065348.

A model to measure self-assessed proficiency in electronic medical records: Validation using maturity survey data from Canadian community-based physicians

OBJECTIVE

Adoption of electronic medical records (EMRs) does not necessarily translate to proficiency -referred to here as EMR maturity. To realize the full benefit of wide scale EMR adoption, the focus must shift from adoption to advancing mature use. This calls for validated assessment models so that researchers, health system planners and digital health developers can better understand what contributes to maturity among physicians. This research aims to validate a measurement model for self-assessed EMR maturity among community-based physicians.

METHODS

As part of an Ontario government-funded EMR adoption program, the EMR Maturity Model for community-based practices was adapted from a hospital-based EMR maturity model. A survey instrument was developed on the foundation of the new model and revised by experts and stakeholders. Content validity, face validity and user acceptance were established before survey administration. Internal consistency and construct validity of the model were tested after survey data were collected. Finally, physicians' comments collected via the survey were qualitatively analyzed to provide additional insights that can be applied to refinement of the model and survey.

RESULTS

As of August 1, 2019, 1588 physicians completed the survey. Ordinal alpha tests for reliability and content validity yielded an alpha value of 0.86 across all key measures specifically associated with maturity. Among most of these, there was a pattern of weak to moderate significant (p < .0001) positive Spearman inter-correlations. One factor was extracted for items measuring dimensions of maturity and all factor loadings of the key measures were greater than 0.40. The fit of the one-factor model was moderately adequate. This indicates the model is valid and reliable, with consistency across key measures for measuring one factor: maturity.

CONCLUSIONS

This is the first known validated model published in English that measures EMR maturity among community-based physicians. While the model is shown to be valid and reliable statistically and qualitative analysis supports this, there is room for improvement. Both the statistical analysis and portions of the qualitative analysis suggest areas of exploration to strengthen the model and survey. Future efforts will include refining the survey to improve user interface and accrue further data, as the sample to date is insufficient for generalizability.

Mayo Clinic Registry of Operational Tasks (ROOT): A Paradigm Shift in Electronic Health Reco Implementation Evaluation

OBJECTIVE

To systematically examine clinical workflows before and after a major electronic health record (EHR) implementation, we performed this study. EHR implementation and/or conversion are associated with many challenges, which are barriers to optimal care. Clinical workflows may be significantly affected by EHR implementations and conversions, resulting in provider frustration and reduced efficiency.

PATIENTS AND METHODS

Our institution completed a large EHR conversion and workflow standardization converting from 3 EHRs (GE Centricity and 2 versions of Cerner) to a system-wide Epic platform. To study this quantitatively and qualitatively, we collected and curated clinical workflows through rapid ethnography, workflow observation, video ethnography, and log-file analyses of hundreds of providers, patients, and more than 100,000 log files. The study included 5 geographic sites in 4 states (Arizona, Minnesota, Florida, and Wisconsin). This project began in April 2016, and will be completed by December 2019. Our study began on May 1, 2016, and is ongoing.

RESULTS

Salient themes include the importance of prioritizing clinical areas with the most intensive EHR use, the value of tools to identify bottlenecks in workflow that cause delays, and desire for additional training to optimize navigation. Video microanalyses identified marked differences in patterns of workflow and EHR navigation patterns across sites. Log-file analyses and social network analyses identified differences in personnel roles, which led to differences in patient-clinician interaction, time spent using the EHR, and paper-based artifacts.

CONCLUSION

Assessing and curating workflow data before and after EHR conversion may provide opportunities for unexpected efficiencies in workflow optimization and information-system redesign. This project may be a model for capturing significant new knowledge in using EHRs to improve patient care, workflow efficiency, and outcomes.

An evaluation of the user-friendliness of Bayesian forecasting programs in a clinical setting

AIMS

To evaluate 3 Bayesian forecasting (BF) programs-TDMx, InsightRx and DoseMe-on their user-friendliness and common liked and disliked features through a survey of hospital pharmacists.

METHODS

Clinical pharmacists across 3 Australian hospitals that did not use a BF program were invited to a BF workshop and complete a survey on programs they trialled. Participants were given 4 case scenarios to work through and asked to complete a 5-point Likert scale survey evaluating the program's user-friendliness. Liked and disliked features of each program were ascertained through written responses to open-ended questions. Survey results were compared using a χ² test of equal or given proportions to identify significant differences in response.

RESULTS

Twenty-seven pharmacists, from hospitals, participated. BF programs were rated overall as user-friendly with 70%, 41% and 37% (P = .02) of participants recording a Likert score of 4 or 5 for DoseMe, TDMx and InsightRx, respectively. Participants found it easy to access all required information to use the programs, understood dosing recommendations and visualisations given by each program, and thought programs supported decision-making with >50% of participants scoring a 4 or 5 across the programs in these categories. Common liked features across all programs were the graphical displays and ease of data entry, while common disliked features were related to the units, layout and information display.

CONCLUSION

Although differences exist between programs, all 3 programs were most commonly rated as user-friendly across all themes evaluated, which provides useful information for healthcare facilities wanting to implement a BF program.

Electronic Health Record Adoption and Nurse Reports of Usability and Quality of Care: The Role of Work Environment

BACKGROUND

Despite evidence suggesting higher quality and safer care in hospitals with comprehensive electronic health record (EHR) systems, factors related to advanced system usability remain largely unknown, particularly among nurses. Little empirical research has examined sociotechnical factors, such as the work environment, that may shape the relationship between advanced EHR adoption and quality of care.

OBJECTIVE

The objective of this study was to examine the independent and joint effects of comprehensive EHR adoption and the hospital work environment on nurse reports of EHR usability and nurse-reported quality of care and safety.

METHODS

This study was a secondary analysis of nurse and hospital survey data. Unadjusted and adjusted logistic regression models were used to assess the relationship between EHR adoption level, work environment, and a set of EHR usability and quality/safety outcomes. The sample included 12,377 nurses working in 353 hospitals.

RESULTS

In fully adjusted models, comprehensive EHR adoption was associated with lower odds of nurses reporting poor usability outcomes, such as dissatisfaction with the system (odds ratio [OR]: 0.75; 95% confidence interval [CI]: 0.61-0.92). The work environment was associated with all usability outcomes with nurses in better environments being less likely to report negatively. Comprehensive EHRs (OR: 0.83; 95% CI: 0.71-0.96) and better work environments (OR: 0.47; 95% CI: 0.42-0.52) were associated with lower odds of nurses reporting fair/poor quality of care, while poor patient safety grade was associated with the work environment (OR: 0.50; 95% CI: 0.46-0.54), but not EHR adoption level.

CONCLUSION

Our findings suggest that adoption of a comprehensive EHR is associated with more positive usability ratings and higher quality of care. We also found that-independent of EHR adoption level-the hospital work environment plays a significant role in how nurses evaluate EHR usability and whether EHRs have their intended effects on improving quality and safety of care.

Satisfaction of health-care providers with electronic health records and perceived barriers to its implementation in the United Arab Emirates

The integration of electronic health records (EHRs) has shown promise in improving health-care quality. In the United Arab Emirates, EHRs have been recently adopted to improve the quality and safety of patient care. A cross-sectional survey of 680 health-care providers (HCPs) was conducted to assess the satisfaction of HCPs in the United Arab Emirates with EHRs' impact on access/viewing, documentation and medication administration and to explore the barriers encountered in their use. Data were collected over 6 months from April to September 2014. High overall satisfaction with EHRs was reported by HCPs, suggesting their acceptance. Physicians reported the greatest overall satisfaction with EHRs, although nurses showed significantly higher satisfaction with the impact on medication administration compared with other HCPs. The most significant barriers reported by nurses were lack of belief in the value of EHRs for patients and lack of adequate computer skills. Given the large investment in technology, additional research is necessary to promote the full utilization of EHRs. Nurses need to be aware of the value of EHRs for patient care and be involved in all stages of EHR implementations to maximize its meaningful use for better clinical outcomes.

Going Live

This study evaluated the immediate effects of implementing an electronic health record (EHR) system on physician workflow in the emergency department. Two sets of observations were conducted in one emergency department. The first set of observations, the baseline period, was completed in the 22 days prior to the implementation of a new EHR. The second set of observations, the go-live period, was completed during the seven-day period immediately after the EHR go-live date. A comparison across four task-based categories revealed that during the go-live period there was a decrease in the proportion of time physicians spent in patient rooms and engaged with paper-based tasks, and an increase in the proportion of time physicians spent at computer stations and working with other people. In addition, physicians engaged in more information seeking behaviors during the go-live period than during the baseline period. The impact of these shifts in tasks and behaviors is discussed with a focus on the importance of fully understanding the EHR go-live process.

Impact of homecare electronic health record on timeliness of clinical documentation, reimbursement, and patient outcomes

BACKGROUND

Homecare is an important and effective way of managing chronic illnesses using skilled nursing care in the home. Unlike hospitals and ambulatory settings, clinicians visit patients at home at different times, independent of each other. Twenty-nine percent of 10,000 homecare agencies in the United States have adopted point-of-care EHRs. Yet, relatively little is known about the growing use of homecare EHRs.

OBJECTIVE

Researchers compared workflow, financial billing, and patient outcomes before and after implementation to evaluate the impact of a homecare point-of-care EHR.

METHODS

The design was a pre/post observational study embedded in a mixed methods study. The setting was a Philadelphia-based homecare agency with 137 clinicians. Data sources included: (1) clinician EHR documentation completion; (2) EHR usage data; (3) Medicare billing data; (4) an EHR Nurse Satisfaction survey; (5) clinician observations; (6) clinician interviews; and (7) patient outcomes.

RESULTS

Clinicians were satisfied with documentation timeliness and team communication. Following EHR implementation, 90% of notes were completed within the 1-day compliance interval (n = 56,702) compared with 30% of notes completed within the 7-day compliance interval in the pre-implementation period (n = 14,563; OR 19, p <. 001). Productivity in the number of clinical notes documented post-implementation increased almost 10-fold compared to pre-implementation. Days to Medicare claims fell from 100 days pre-implementation to 30 days post-implementation, while the census rose. EHR implementation impact on patient outcomes was limited to some behavioral outcomes.

DISCUSSION

Findings from this homecare EHR study indicated clinician EHR use enabled a sustained increase in productivity of note completion, as well as timeliness of documentation and billing for reimbursement with limited impact on improving patient outcomes. As EHR adoption increases to better meet the needs of the growing population of older people with chronic health conditions, these results can inform homecare EHR development and implementation.

An observational study of the accuracy and completeness of an anesthesia information management system: recommendations for documentation system changes

Anesthesia information management systems must often be tailored to fit the environment in which they are implemented. Extensive customization necessitates that systems be analyzed for both accuracy and completeness of documentation design to ensure that the final record is a true representation of practice. The purpose of this study was to determine the accuracy of a recently installed system in the capture of key perianesthesia data. This study used an observational design and was conducted using a convenience sample of nurse anesthetists. Observational data of the nurse anesthetists'delivery of anesthesia care were collected using a touch-screen tablet computer utilizing an Access database customized observational data collection tool. A questionnaire was also administered to these nurse anesthetists to assess perceived accuracy, completeness, and satisfaction with the electronic documentation system. The major sources of data not documented in the system were anesthesiologist presence (20%) and placement of intravenous lines (20%). The major sources of inaccuracies in documentation were gas flow rates (45%), medication administration times (30%), and documentation of neuromuscular function testing (20%)-all of the sources of inaccuracies were related to the use of charting templates that were not altered to reflect the actual interventions performed.

Challenges and facilitators to adoption of a point-of-care electronic health record in home care

Electronic health records (EHRs), intended to improve the clinical process, are understudied in home care. The researchers assessed clinician satisfaction, informed by workflow and patient outcomes, to identify EHR adoption challenges. The mixed methods study setting was a Philadelphia agency with 137 clinicians. Adoption challenges included: (a) hardware problems coupled with lack of field support; (b) inadequate training; and (c) mismatch of EHR usability/functionality and workflow resulting in decreased efficiency. Adoption facilitators were support for team communication and improved clinical data timeliness. Opportunities for improved adoption included sharing with front-line clinicians EHR data related to patient care and health outcomes.

Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures

BACKGROUND

Two of the current methodological barriers to implementation science efforts are the lack of agreement regarding constructs hypothesized to affect implementation success and identifiable measures of these constructs. In order to address these gaps, the main goals of this paper were to identify a multi-level framework that captures the predominant factors that impact implementation outcomes, conduct a systematic review of available measures assessing constructs subsumed within these primary factors, and determine the criterion validity of these measures in the search articles.

METHOD

We conducted a systematic literature review to identify articles reporting the use or development of measures designed to assess constructs that predict the implementation of evidence-based health innovations. Articles published through 12 August 2012 were identified through MEDLINE, CINAHL, PsycINFO and the journal Implementation Science. We then utilized a modified five-factor framework in order to code whether each measure contained items that assess constructs representing structural, organizational, provider, patient, and innovation level factors. Further, we coded the criterion validity of each measure within the search articles obtained.

RESULTS

Our review identified 62 measures. Results indicate that organization, provider, and innovation-level constructs have the greatest number of measures available for use, whereas structural and patient-level constructs have the least. Additionally, relatively few measures demonstrated criterion validity, or reliable association with an implementation outcome (e.g., fidelity).

DISCUSSION

In light of these findings, our discussion centers on strategies that researchers can utilize in order to identify, adapt, and improve extant measures for use in their own implementation research. In total, our literature review and resulting measures compendium increases the capacity of researchers to conceptualize and measure implementation-related constructs in their ongoing and future research.

Community-based, interdisciplinary geriatric care team satisfaction with an electronic health record: a multimethod study

This multimethod study measured the impact of an electronic health record (EHR) on clinician satisfaction with clinical process. Subjects were 39 clinicians at a Program of All-inclusive Care for Elders (PACE) site in Philadelphia utilizing an EHR. Methods included the evidence-based evaluation framework, Health Information Technology Research-Based Evaluation Framework, which guided assessment of clinician satisfaction with surveys, observations, follow-up interviews, and actual EHR use at two points in time. Mixed-methods analysis of findings provided context for interpretation and improved validity. The study found that clinicians were satisfied with the EHR; however, satisfaction declined between time periods. Use of EHR was universal and wide and was differentiated by clinical role. Between time periods, EHR use increased in volume, with increased timeliness and decreased efficiency. As the first EHR evaluation at a PACE site from the perspective of clinicians who use the system, this study provides insights into EHR use in the care of older people in community-based healthcare settings.

Advice for Decision Makers Based on an Electronic Health Record Evaluation at a Program for All-inclusive Care for Elders Site

OBJECTIVE

Provide evidence-based advise to "Program of All-inclusive Care for the Elderly" (PACE) decision makers considering implementing an electronic health record (EHR) system, drawing on the results of a mixed methods study to examine: (1) the diffusion of an EHR among clinicians documenting direct patient care in a PACE day care site, (2) the impact of the use of the EHR on the satisfaction levels of clinicians, and (3) the impact of the use of the EHR on patient functional outcomes.

METHODS

Embedded mixed methods design with a post-test design quantitative experiment and concurrent qualitative component. Quantitative methods included: (1) the EHR audit log used to determine the frequency and timing during the week of clinicians' usage of the system; (2) a 22-item clinician satisfaction survey; and (3) a 16-item patient functional outcome questionnaire related to locomotion, mobility, personal hygiene, dressing, feeding as well the use of adaptive devices. Qualitative methods included observations and open-ended, semi-structured follow-up interviews. Qualitative data was merged with the quantitative data by comparing the findings along themes. The setting was a PACE utilizing an EHR in Philadelphia: PACE manages the care of nursing-home eligible members to enable them to avoid nursing home admission and reside in their homes. Participants were 39 clinicians on the multi-disciplinary teams caring for the elders and 338 PACE members.

RESULTS

Clinicians did not use the system as intended, which may help to explain why the benefits related to clinical processes and patient outcomes as expected for an EHR were not reflected in the results. Clinicians were satisfied with the EHR, although there was a non-significant decline between 11 and 17 months post implementation of the EHR. There was no significant difference in patient functional outcome the two time periods. However, the sample size of 48 was too small to allow any conclusive statements to be made. Interpretation of findings underscores the importance of the interaction of workflow and EHR functionality and usability to impact clinician satisfaction, efficiency, and clinician use of the EHR.

CONCLUSION

This research provides insights into EHR use in the care of the older people in community-based health care settings. This study assessed the adoption of an EHR outside the acute hospital setting and in the community setting to provide evidence-based recommendations to PACE decision makers considering implementing an EHR.