Human-centered computing in health information systems part 2: evaluation

Human Factors and Ergonomics in the Design of Health Information Technology: Trends and Progress in 2014

OBJECTIVE

To summarize significant contributions to the research on human factors and organizational issues in medical informatics.

METHODS

An extensive search using PubMed/Medline and Web of Science® was conducted to identify the scientific contributions, published in 2014, to human factors and organizational issues in medical informatics, with a focus on health information technology (HIT) usability. The selection process comprised three steps: (i) 15 candidate best papers were selected by the two section editors, (ii) external reviewers from a pool of international experts reviewed each candidate best paper, and (iii) the final selection of three best papers was made by the editorial board of the IMIA Yearbook.

RESULTS

Noteworthy papers published in 2014 describe an efficient, easy to implement, and useful process for detecting and mitigating human factors and ergonomics (HFE) issues of HIT. They contribute to promote the HFE approach with interventions based on rigorous and well-conducted methods when designing and implementing HIT.

CONCLUSION

The application of HFE in the design and implementation of HIT remains limited, and the impact of incorporating HFE principles on patient safety is understudied. Future works should be conducted to advance this field of research, so that the safety and quality of patient care are not compromised by the increasing adoption of HIT.

Are three methods better than one? A comparative assessment of usability evaluation methods in an EHR

OBJECTIVE

To comparatively evaluate the effectiveness of three different methods involving end-users for detecting usability problems in an EHR: user testing, semi-structured interviews and surveys.

MATERIALS AND METHODS

Data were collected at two major urban dental schools from faculty, residents and dental students to assess the usability of a dental EHR for developing a treatment plan. These included user testing (N=32), semi-structured interviews (N=36), and surveys (N=35).

RESULTS

The three methods together identified a total of 187 usability violations: 54% via user testing, 28% via the semi-structured interview and 18% from the survey method, with modest overlap. These usability problems were classified into 24 problem themes in 3 broad categories. User testing covered the broadest range of themes (83%), followed by the interview (63%) and survey (29%) methods.

DISCUSSION

Multiple evaluation methods provide a comprehensive approach to identifying EHR usability challenges and specific problems. The three methods were found to be complementary, and thus each can provide unique insights for software enhancement. Interview and survey methods were found not to be sufficient by themselves, but when used in conjunction with the user testing method, they provided a comprehensive evaluation of the EHR.

CONCLUSION

We recommend using a multi-method approach when testing the usability of health information technology because it provides a more comprehensive picture of usability challenges.

The science of human factors: separating fact from fiction

Background

Interest in human factors has increased across healthcare communities and institutions as the value of human centred design in healthcare becomes increasingly clear. However, as human factors is becoming more prominent, there is growing evidence of confusion about human factors science, both anecdotally and in scientific literature. Some of the misconceptions about human factors may inadvertently create missed opportunities for healthcare improvement.

Methods

The objective of this article is to describe the scientific discipline of human factors and provide common ground for partnerships between healthcare and human factors communities.

Results

The primary goal of human factors science is to promote efficiency, safety and effectiveness by improving the design of technologies, processes and work systems. As described in this article, human factors also provides insight on when training is likely (or unlikely) to be effective for improving patient safety. Finally, we outline human factors specialty areas that may be particularly relevant for improving healthcare delivery and provide examples to demonstrate their value.

Conclusions

The human factors concepts presented in this article may foster interdisciplinary collaborations to yield new, sustainable solutions for healthcare quality and patient safety.

User-centered requirements engineering in health information systems: a study in the hemophilia field

The use of sophisticated information and communication technologies (ICTs) in the health care domain is a way to improve the quality of services. However, there are also hazards associated with the introduction of ICTs in this domain and a great number of projects have failed due to the lack of systematic consideration of human and other non-technology issues throughout the design or implementation process, particularly in the requirements engineering process. This paper presents the methodological approach followed in the design process of a web-based information system (WbIS) for managing the clinical information in hemophilia care, which integrates the values and practices of user-centered design (UCD) activities into the principles of software engineering, particularly in the phase of requirements engineering (RE). This process followed a paradigm that combines a grounded theory for data collection with an evolutionary design based on constant development and refinement of the generic domain model using three well-known methodological approaches: (a) object-oriented system analysis; (b) task analysis; and, (c) prototyping, in a triangulation work. This approach seems to be a good solution for the requirements engineering process in this particular case of the health care domain, since the inherent weaknesses of individual methods are reduced, and emergent requirements are easier to elicit. Moreover, the requirements triangulation matrix gives the opportunity to look across the results of all used methods and decide what requirements are critical for the system success.

A holistic framework to improve the uptake and impact of eHealth technologies

BACKGROUND

Many eHealth technologies are not successful in realizing sustainable innovations in health care practices. One of the reasons for this is that the current development of eHealth technology often disregards the interdependencies between technology, human characteristics, and the socioeconomic environment, resulting in technology that has a low impact in health care practices. To overcome the hurdles with eHealth design and implementation, a new, holistic approach to the development of eHealth technologies is needed, one that takes into account the complexity of health care and the rituals and habits of patients and other stakeholders.

OBJECTIVE

The aim of this viewpoint paper is to improve the uptake and impact of eHealth technologies by advocating a holistic approach toward their development and eventual integration in the health sector.

METHODS

To identify the potential and limitations of current eHealth frameworks (1999-2009), we carried out a literature search in the following electronic databases: PubMed, ScienceDirect, Web of Knowledge, PiCarta, and Google Scholar. Of the 60 papers that were identified, 44 were selected for full review. We excluded those papers that did not describe hands-on guidelines or quality criteria for the design, implementation, and evaluation of eHealth technologies (28 papers). From the results retrieved, we identified 16 eHealth frameworks that matched the inclusion criteria. The outcomes were used to posit strategies and principles for a holistic approach toward the development of eHealth technologies; these principles underpin our holistic eHealth framework.

RESULTS

A total of 16 frameworks qualified for a final analysis, based on their theoretical backgrounds and visions on eHealth, and the strategies and conditions for the research and development of eHealth technologies. Despite their potential, the relationship between the visions on eHealth, proposed strategies, and research methods is obscure, perhaps due to a rather conceptual approach that focuses on the rationale behind the frameworks rather than on practical guidelines. In addition, the Web 2.0 technologies that call for a more stakeholder-driven approach are beyond the scope of current frameworks. To overcome these limitations, we composed a holistic framework based on a participatory development approach, persuasive design techniques, and business modeling.

CONCLUSIONS

To demonstrate the impact of eHealth technologies more effectively, a fresh way of thinking is required about how technology can be used to innovate health care. It also requires new concepts and instruments to develop and implement technologies in practice. The proposed framework serves as an evidence-based roadmap.

Value of human factors to medication and patient safety in the intensive care unit

Conventional wisdom suggests that the "human factor" in critical care environments is reason for inadequate medication and patient safety. "Human factors" (or human factors engineering) is also a scientific discipline and practice of improving human performance. Using decades of human factors research, this paper evaluates a range of common beliefs about patient safety through a human factors lens. This evaluation demonstrates that human factors provides a framework for understanding safety failures in critical care settings, offers insights into how to improve medication and patient safety, and reminds us that the "human factor" in critical care units is what allows these time-pressured, information-intense, mentally challenging, interruption-laden, and life-or-death environments to function so safely so much of the time.

Novel Representation of Clinical Information in the ICU: Developing User Interfaces which Reduce Information Overload

The introduction of electronic medical records (EMR) and computerized physician order entry (CPOE) into the intensive care unit (ICU) is transforming the way health care providers currently work. The challenge facing developers of EMR's is to create products which add value to systems of health care delivery. As EMR's become more prevalent, the potential impact they have on the quality and safety, both negative and positive, will be amplified. In this paper we outline the key barriers to effective use of EMR and describe the methodology, using a worked example of the output. AWARE (Ambient Warning and Response Evaluation), is a physician led, electronic-environment enhancement program in an academic, tertiary care institution's ICU. The development process is focused on reducing information overload, improving efficiency and eliminating medical error in the ICU.

Using task analysis to improve the requirements elicitation in health information system

This paper describes the application of task analysis within the design process of a Web-based information system for managing clinical information in hemophilia care, in order to improve the requirements elicitation and, consequently, to validate the domain model obtained in a previous phase of the design process (system analysis). The use of task analysis in this case proved to be a practical and efficient way to improve the requirements engineering process by involving users in the design process.