Decision-Support Systems as Safety-Critical Components: Towards a Safety Culture for Medical Informatics

Cognitive and usability engineering methods for the evaluation of clinical information systems

Increasingly healthcare policy and decision makers are demanding evidence to justify investments in health information systems. This demand requires an adequate evaluation of these systems. A wide variety of approaches and methodologies have been applied in assessing the impact of information systems in health care, ranging from controlled clinical trials to use of questionnaires and interviews with users. In this paper we describe methodological approaches which we have applied and refined for the past 10 years for the evaluation of health information systems. The approaches are strongly rooted in theories and methods from cognitive science and the emerging field of usability engineering. The focus is on assessing human computer interaction and in particular, the usability of computer systems in both laboratory and naturalistic settings. The methods described can be a part of the formative evaluation of systems during their iterative development, and can also complement more traditional assessment methods used in summative system evaluation of completed systems. The paper provides a review of the general area of systems evaluation with the motivation and rationale for methodological approaches underlying usability engineering and cognitive task analysis as applied to health information systems. This is followed by a detailed description of the methods we have applied in a variety of settings in conducting usability testing and usability inspection of systems such as computer-based patient records. Emerging trends in the evaluation of complex information systems are discussed.

Towards productive Knowledge-Based Systems in clinical organizations: a methods perspective

This paper discusses an approach towards Knowledge-Based Systems (KBS) development which emphazises fit in the clinical organization, utility and safety. KBS design is identified as a subset of Decision Support System (DSS) design, and experiences from use of development methods is made available from the more general field. Generic and specific issues related to making KBS development result in systems used in clinical practice are discussed, and an integration of the Logic Engineering KBS technique into the Action Design DSS requirements specification method is outlined. Group-based knowledge modeling is identified as the bridge between the methods. It is concluded that while Action Design provides organizational validation (Are we building the right system?), Logic Engineering adds on KBS design verification (Are we going to build the system right?).