User Testing of an Admission, Discharge, Transfer System: Usability Evaluation

Introduction: To improve the first step of the hospitalization procedure, appropriate interaction must be established between users and the admission, discharge, and transfer system. The aim of this study was to evaluate the usability of the ADT system in some of selected Iranian non-teaching hospitals.Material and Methods: This study was cross-sectional research that has evaluated the usability of a selected ADT system using the think-aloud method by 11 medical record administrators. Users were asked to follow the provided scenario, then share and elaborate on what they saw, thought about, did, felt, and decided during their interaction with the system. Users' feedbacks were collected and organized into four main categories for further processing.Results: To evaluate the usability of an ADT system, four routine scenario tasks were followed by users and only 45.45% of them could implement all tasks. Overall, 36 independent problems were identified. All problems were related to the data entry categories that accounted for the largest share. The most important problems were related to the issues regarding "date of birth" field in this category which deals with the outpatient admission process.Conclusion: The study of the usability testing method indicated that the ADT subsystem of non-teaching hospital has many problems in interact with real users with the system. It showed that more than half of the users could not completely and successfully perform the entire real-world scenario tasks. Furthermore, the most usability problems were found in data entry categories.

Usability testing of bed information management system: A think-aloud method

Not considering the usability in designing clinical information systems causes problems in human–computer interaction and patient dissatisfaction. Therefore, in this study, the usability of the bed information management system (BIMS) was examined by think-aloud method. This cross-sectional study was conducted on the BIMS in 50 noneducational hospitals. Participants consisted of three groups including users, facilitators, and technical support. To carry out the study, a scenario consisting of four tasks was designed. Three researchers analyzed the recorded files to identify the usability problems and their severity. The mean time of the evaluation process was 20:33 ± 4:47 s. The total number of the problems identifies by users was 80 cases. Data entry and layout problems with 38 (48%) and 33 (41%) cases were the most frequently found problems, respectively. About 61% and 55% of the data entry and layout problems had a minor severity (Severity 2), respectively. Furthermore, 43 (54%) cases of the problems were resolved by the users and 32 (40%) cases by the facilitator assistance. This study showed that a large number of the problems were due to the system poor design. Furthermore, by increasing the users’ level of knowledge about the system, it is possible to enhance user-system interaction. It is recommended that before designing and implementing a system, the system should be evaluated for usability, and the users should be educated in clinical information systems.

Usability evaluation of obstetrics and gynecology information system using cognitive walkthrough method

Background and aim

Obstetrics and gynecology information system is a critical component of the HIS in social security organization health centers. The objective of this study was to evaluate the usability of this system using the cognitive walkthrough method. Also, the present study provided a detailed formal description of how the cognitive usability evaluation can be applied and reported for a health care information system.

Methods

This study was conducted at the Mashhad University of Medical Sciences' usability lab from March 2016 to June 2017. A two-phase approach was used to conduct the cognitive walkthrough evaluation: preparatory and evaluation. The preparation was done in three stages: first, we investigated users' capabilities and background knowledge through a semi-structured interview. Second, the evaluation scenario was developed based on the most common tasks in routine workflow of users. Finally, each task was broken down into sequences of actions. In the evaluation phase, three usability experts independently assessed each action using a four-item checklist. Problems were categorized thematically and were reported from three different perspectives: Question-based, Task-based, and Evaluator-based. The data were then analyzed to understand the contribution of each task, along with its mean severity score.

Results

Evaluators' responses were compared and any conflict was resolved in an expert panel. A total of 116 usability problems were identified based on the consensus of the evaluators. Inadequate system feedback was found to be the main source of 43% of the problems, and resulted in users confusion.

Conclusion

Since the system was evaluated in its pilot implementation phase, there was an opportunity to prevent future potential usability problems. The use of a mixed quantitative and qualitative approach in this usability study provided a more comprehensive perspective of the system problems. This study provided a detailed description of conducting CW usability evaluation which can be used as a practical guide for future studies.