Clinical information technology capabilities in four U.S. hospitals: testing a new structural performance measure

Investigation of the Usability of Computerized Critical Care Information Systems in Germany

INTRODUCTION

The term "usability" describes how effectively, efficiently, and with what level of user satisfaction an information system can be used to accomplish specific goals. Computerized critical care information systems (CCISs) with high usability increase quality of care and staff satisfaction, while reducing medication errors. Conversely, systems lacking usability can interrupt clinical workflow, facilitate errors, and increase charting time. The aim of this study was to investigate and compare usability across CCIS currently used in Germany.

METHODS

In this study, German intensive care unit (ICU) nurses and physicians completed a specialized, previously validated, web-based questionnaire. The questionnaire assessed CCIS usability based on three rating models: an overall rating of the systems, a model rating technical usability, and a model rating task-specific usability.

RESULTS

We analyzed results from 535 survey participants and compared eight different CCIS commonly used in Germany. Our results showed that usability strongly differs across the compared systems. The system ICUData had the best overall rating and technical usability, followed by the platforms ICM and MetaVision. The same three systems performed best in the rating of task-specific usability without significant differences between each other. Across all systems, overall ratings were more dependent on ease-of-use aspects than on aspects of utility/functionality, and the general scope of the functions offered was rated better than how well the functions are realized.

DISCUSSION

Our results suggest that manufacturers should shift some of their effort away from the development of new features and focus more on improving the ease-of-use and quality of existing features.

Care coordination gaps due to lack of interoperability in the United States: a qualitative study and literature review

Background

Health information technology (HIT) could improve care coordination by providing clinicians remote access to information, improving legibility, and allowing asynchronous communication, among other mechanisms. We sought to determine, from a clinician perspective, how care is coordinated and to what extent HIT is involved when transitioning patients between emergency departments, acute care hospitals, skilled nursing facilities, and home health agencies in settings across the United States.

Methods

We performed a qualitative study with clinicians and information technology professionals from six regions of the U.S. which were chosen as national leaders in HIT. We analyzed data through a two person consensus approach, assigning responses to each of nine care coordination activities. We also conducted a literature review of MEDLINE®, CINAHL®, and Embase, analyzing results of studies that examined interventions to improve information transfer during transitions of care.

Results

We enrolled 29 respondents from 17 organizations and conducted six focus groups. Respondents reported how HIT is currently used for care coordination activities. HIT is currently used to monitor patients and to align systems-level resources with population needs. However, we identified multiple areas where the lack of interoperability leads to inefficient processes and missing data. Additionally, the literature review identified ten intervention studies that address information transfer, seven of which employed HIT and three of which utilized other communication methods such as telephone calls, faxed records, and nurse case management.

Conclusions

Significant care coordination gaps exist due to the lack of interoperability across the United States. We must design, evaluate, and incentivize the use of HIT for care coordination. We should focus on the domains where we found the largest gaps: information transfer, systems to monitor patients, tools to support patients’ self-management goals, and tools to link patients and their caregivers with community resources.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-016-1373-y) contains supplementary material, which is available to authorized users.

How is the electronic health record being used? Use of EHR data to assess physician-level variability in technology use

BACKGROUND

Studies of the effects of electronic health records (EHRs) have had mixed findings, which may be attributable to unmeasured confounders such as individual variability in use of EHR features.

OBJECTIVE

To capture physician-level variations in use of EHR features, associations with other predictors, and usage intensity over time.

METHODS

Retrospective cohort study of primary care providers eligible for meaningful use at a network of federally qualified health centers, using commercial EHR data from January 2010 through June 2013, a period during which the organization was preparing for and in the early stages of meaningful use.

RESULTS

Data were analyzed for 112 physicians and nurse practitioners, consisting of 430,803 encounters with 99,649 patients. EHR usage metrics were developed to capture how providers accessed and added to patient data (eg, problem list updates), used clinical decision support (eg, responses to alerts), communicated (eg, printing after-visit summaries), and used panel management options (eg, viewed panel reports). Provider-level variability was high: for example, the annual average proportion of encounters with problem lists updated ranged from 5% to 60% per provider. Some metrics were associated with provider, patient, or encounter characteristics. For example, problem list updates were more likely for new patients than established ones, and alert acceptance was negatively correlated with alert frequency.

CONCLUSIONS

Providers using the same EHR developed personalized patterns of use of EHR features. We conclude that physician-level usage of EHR features may be a valuable additional predictor in research on the effects of EHRs on healthcare quality and costs.

Information technology capacities assessment tool in hospitals: Instrument development and validation

Objectives:This research integrates existing literature on information technology (IT) in hospitals, and proposes and validates a comprehensive IT capacities assessment tool in these settings.

Methods:A comprehensive literature review was conducted on Medline until September 2006 to identify studies that used specific IT measures in hospitals. The results were mapped and used as a basis for the development of the proposed instrument, which was tested through a survey of Canadian healthcare organizations (N= 221).

Results:A total of seventeen studies provided indicators of clinical and administrative IT capacities in hospitals. Based on the mapping of these indicators, a comprehensive IT capacities assessment instrument was developed including thirty-four items exploring computerized processes, thirteen items assessing contemporary technologies, and eleven items investigating internal and external information sharing. A time frame was inserted in the tool to reflect “plans for” versus “current” implementation of IT; in the latter, the extent of current use of computerized processes and technologies was measured on a (1–7) scale. Overall, the survey yielded a total of 106 responses (52.2 percent response rate), and the results demonstrated a good level of reliability and validity of the instrument.

Conclusions:This study unifies existing work in this area, and presents the psychometric properties of an IT capacities assessment tool in hospitals. By developing scores for capturing IT capacities in hospitals, it is possible to further address important research questions related to the determinants and impacts of IT sophistication in these settings.

Hospital characteristics associated with highly automated and usable clinical information systems in Texas, United States

Background

A hospital's clinical information system may require a specific environment in which to flourish. This environment is not yet well defined. We examined whether specific hospital characteristics are associated with highly automated and usable clinical information systems.

Methods

This was a cross-sectional survey of 125 urban hospitals in Texas, United States using the Clinical Information Technology Assessment Tool (CITAT), which measures a hospital's level of automation based on physician interactions with the information system. Physician responses were used to calculate a series of CITAT scores: automation and usability scores, four automation sub-domain scores, and an overall clinical information technology (CIT) score. A multivariable regression analysis was used to examine the relation between hospital characteristics and CITAT scores.

Results

We received a sufficient number of physician responses at 69 hospitals (55% response rate). Teaching hospitals, hospitals with higher IT operating expenses (>$1 million annually), IT capital expenses (>$75,000 annually) and hospitals with larger IT staff (≥ 10 full-time staff) had higher automation scores than hospitals that did not meet these criteria (p < 0.05 in all cases). These findings held after adjustment for bed size, total margin, and ownership (p < 0.05 in all cases). There were few significant associations between the hospital characteristics tested in this study and usability scores.

Conclusion

Academic affiliation and larger IT operating, capital, and staff budgets are associated with more highly automated clinical information systems.

Measuring clinical information technology in the ICU setting: application in a quality improvement collaborative

OBJECTIVE

Few instruments are available to measure the performance of intensive care unit (ICU) clinical information systems. Our objectives were: 1) to develop a survey-based metric that assesses the automation and usability of an ICU's clinical information system; 2) to determine whether higher scores on this instrument correlate with improved outcomes in a multi-institution quality improvement collaborative.

DESIGN

This is a cross-sectional study of the medical directors of 19 Michigan ICUs participating in a state-wide quality improvement collaborative designed to reduce the rate of catheter-related blood stream infections (CRBSI). Respondents completed a survey assessing their ICU's information systems.

MEASUREMENTS

The mean of 54 summed items on this instrument yields the clinical information technology (CIT) index, a global measure of the ICU's information system performance on a 100 point scale. The dependent variable in this study was the rate of CRBSI after the implementation of several evidence-based recommendations. A multivariable linear regression analysis was used to examine the relationship between the CIT score and the post-intervention CRBSI rates after adjustment for the pre-intervention rate.

RESULTS

In this cross-sectional analysis, we found that a 10 point increase in the CIT score is associated with 4.6 fewer catheter related infections per 1,000 central line days for ICUs who participate in the quality improvement intervention for 1 year (95% CI: 1.0 to 8.0).

CONCLUSIONS

This study presents a new instrument to examine ICU information system effectiveness. The results suggest that the presence of more sophisticated information systems was associated with greater reductions in the bloodstream infection rate.