The relationship between user interface problems of an admission, discharge and transfer module and usability features: a usability testing method

Comparison of usability evaluation methods for a health information system: heuristic evaluation versus cognitive walkthrough method

Background

There are differences of opinion regarding the selection of the most practical usability evaluation method among different methods. The present study aimed to compare two expert-based evaluation methods in order to assess a nursing module as the most widely used module of a Hospital Information System (HIS).

Methods

Five independent evaluators used the Heuristic Evaluation (HE) and Cognitive Walkthrough (CW) methods to evaluate the nursing module of Shafa HIS. In this regard, the number and severity of the recognized problems according to the usability attributes were compared using two evaluation methods.

Results

The HE and CW evaluation methods resulted in the identification of 104 and 24 unique problems, respectively, of which 33.3% of recognized problems in the CW evaluation method overlapped with the HE method. The average severity of the recognized problems was considered to be minor (2.34) in the HE method and major (2.77) in the CW evaluation method. There was a significant difference in terms of the total number and average severity of the recognized problems by these methods (P < 0.001). Based on the usability attribute, the HE method identified a larger number of problems concerning all usability attributes, and a significant difference was observed in terms of the number of recognized problems in both methods for all attributes except ‘memorability’. Also, there was a significant difference between the two methods based on the average severity of recognized problems only in terms of ‘learnability’.

Conclusion

The HE method identified more problems with lower average severity while the CW was able to recognize fewer problems with higher average severity. Regarding the evaluation goal, the HE method was able to be used to improve the effectiveness and satisfaction of the HIS. Furthermore, the CW evaluation method is recommended to identify usability problems with the highest average severity, especially in terms of ‘learnability’.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-022-01905-7.

Design, development, and evaluation of an interactive training simulator for teaching hospital information systems

BACKGROUND

In their apprenticeship program, health information technology (HIT) students are deprived of the ample opportunity to work with the hospital information system (HIS). This study aimed to design an interactive simulator for the HIS training and evaluate its effects on the informatics skills of HIT students.

MATERIALS AND METHODS

This study was conducted on 16 Bachelor of Science students of HIT at Kashan University of Medical Sciences in 2019. After the functionalities and features of the simulator were determined based on similar existing simulators, expert opinions were received to simulate eight important processes of admission, discharge, and transfer module in HIS. The scores of students' skills and time taken to perform the processes were recorded and analyzed before and after the educational intervention. After they were trained by the simulator, the students filled out a usability evaluation questionnaire. The data were then analyzed in SPSS version 21.

RESULTS

The simulators of health information systems were characterized by interactivity, multimedia applications, practice exercises, tests, and feedback. After the students were trained by the developed simulator, their skills scores improved significantly in 75% (6/8) of the processes, and the timespans of all processes decreased significantly (P < 0.05). The usability evaluation indicated the usability of the simulator was at a "good" level.

CONCLUSIONS

According to the study results, using the simulator improves the informatics skills of HIT students in working with HIS. It is recommended that this method also be used in other apprenticeship programs to teach health information systems.

Usability evaluation of a nursing information system by applying cognitive walkthrough method

BACKGROUND AND OBJECTIVE

The user interface usability of the nursing information system (NIS) should be such that the nurses can learn and interact with it easily and quickly. Therefore, it is necessary to identify and solve the usability problems of these systems. The present study aimed to evaluate the usability of a NIS using the cognitive walkthrough (CW) evaluation method.

METHODS

Based on five selected scenarios, five evaluators evaluated the NIS in Shafa Hospital Information System. After identifying the problems, the evaluators assigned each problem to one of the usability attributes. The severity of each identified problem was determined by the evaluators and five real users of the system.

RESULTS

In total, 24 unique problems were identified. The average severity of the problems was determined by the evaluators (2.77) and the actual users (2.82) in the "major problem" category. The highest number of problems were assigned to the scenarios 3 and 2 with 15 and 14 problems, respectively. The highest average severity in terms of evaluations and actual users was related to the scenario 5 (3.06 and 2.94, respectively), which was in the "major problem" category. The highest number of problems were associated with learnability (8 problems) and efficiency (6 problems).

CONCLUSION

Since most of the nurses do not have enough time for learning the system, and given that the selected scenarios for evaluation were based on the most frequent daily tasks that nurses performed using the NIS; The correction of usability problems in the scenarios 3 and 2 that had the highest number of problems and the scenario 5 that had the highest average severity and execution time; it can reduce the nurses' cognitive load and the learning time of the system and increase the efficiency of nurses.

Evaluating hospital information system according to ISO 9241 part 12

Objectives

Compliance with standards in designing information systems leads to better utilization and ease of use for users. In this study, the compliance of a widely used hospital information system (HIS) with ISO 9241 part 12 was assessed.

Methods

This applied research is a descriptive, cross-sectional study in which the HIS of 8 hospitals affiliated with Kerman University of Medical Sciences was evaluated based on ISO 9241 part 12. Data were collected by using ISO 9241/12 checklist. The data was analyzed in SPSS 16 using descriptive statistics.

Results

The analysis of data showed that the total compliance of the software with the ISO 9241/12 was 72%. The compliance of the software based on different groups of recommendations was 79% with Organization of information, 91% with Graphic objects, and 58% with Coding techniques. Compliance with different subgroups of ISO recommendations ranged from 28% related to “color coding” in coding techniques to 97% related to “General recommendation for graphical objects” in Graphic objects.

Conclusion

According to this study, the design of a widely used HIS has fairly good compliance with the standard but still suffers from some problems. Considering the role of accurate, valid and timely information in management of the hospitals, and the difficulty of system optimization after implementation, it is necessary that software developers follow existing standards when designing health information systems.

The efficiency and effectiveness of surgery information systems in Iran

Background

Despite the prevalent use and advantages of information systems in hospitals, some have failed to meet their predefined objectives. Surgery information system (SIS) is a sub-system of a hospital information system. Its effective and efficient operation could enhance patient care in the busy environment of operating rooms with multiple tasks. The objective of this study was to evaluate the effectiveness and efficiency of SIS in three educational hospitals.

Methods

Data were collected using a questionnaire completed by 82 users of SIS. This questionnaire contains three parts: 1) participants’ demographic information, 2) questions regarding the efficiency of SIS, and 3) questions about its effectiveness. An independent sample t-test was used to compare the efficiency and effectiveness among systems. Chi-squared and Fisher tests were used to determine the relationship between the participants’ demographics and efficiency and effectiveness as well as the relationship between efficiency and effectiveness.

Results

About 23% of the participants rated the system’s efficiency as low, 29% as medium, and 48% as high. Besides, 24% of the participants considered the effectiveness of the system as low, 31% as medium, and 45% as high. There was a significant correlation between the efficiency and effectiveness of SIS (p ≤ 0.0001).

Conclusion

Based on the perspective of most participants (44%)the efficiency and effectiveness of both surgery information systems were acceptable. The results suggest that these systems should be designed in a way that facilitate user’s interaction and reduce the time takes to complete tasks. The results could be useful for developing and designing an efficient and effective system.