1
|
Farzandipour M, Nabovati E, Sadeqi Jabali M. Comparison of usability evaluation methods for a health information system: heuristic evaluation versus cognitive walkthrough method. BMC Med Inform Decis Mak 2022; 22:157. [PMID: 35717183 PMCID: PMC9206256 DOI: 10.1186/s12911-022-01905-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/03/2021] [Accepted: 06/15/2022] [Indexed: 11/16/2022] Open
Abstract
Background There are differences of opinion regarding the selection of the most practical usability evaluation method among different methods. The present study aimed to compare two expert-based evaluation methods in order to assess a nursing module as the most widely used module of a Hospital Information System (HIS). Methods Five independent evaluators used the Heuristic Evaluation (HE) and Cognitive Walkthrough (CW) methods to evaluate the nursing module of Shafa HIS. In this regard, the number and severity of the recognized problems according to the usability attributes were compared using two evaluation methods. Results The HE and CW evaluation methods resulted in the identification of 104 and 24 unique problems, respectively, of which 33.3% of recognized problems in the CW evaluation method overlapped with the HE method. The average severity of the recognized problems was considered to be minor (2.34) in the HE method and major (2.77) in the CW evaluation method. There was a significant difference in terms of the total number and average severity of the recognized problems by these methods (P < 0.001). Based on the usability attribute, the HE method identified a larger number of problems concerning all usability attributes, and a significant difference was observed in terms of the number of recognized problems in both methods for all attributes except ‘memorability’. Also, there was a significant difference between the two methods based on the average severity of recognized problems only in terms of ‘learnability’. Conclusion The HE method identified more problems with lower average severity while the CW was able to recognize fewer problems with higher average severity. Regarding the evaluation goal, the HE method was able to be used to improve the effectiveness and satisfaction of the HIS. Furthermore, the CW evaluation method is recommended to identify usability problems with the highest average severity, especially in terms of ‘learnability’. Supplementary Information The online version contains supplementary material available at 10.1186/s12911-022-01905-7.
Collapse
Affiliation(s)
- Mehrdad Farzandipour
- Health Information Management Research Center, Kashan University of Medical Sciences, Kashan, Islamic Republic of Iran.,Department of Health Information Management and Technology, Kashan University of Medical Sciences, Kashan, Islamic Republic of Iran
| | - Ehsan Nabovati
- Health Information Management Research Center, Kashan University of Medical Sciences, Kashan, Islamic Republic of Iran.,Department of Health Information Management and Technology, Kashan University of Medical Sciences, Kashan, Islamic Republic of Iran
| | - Monireh Sadeqi Jabali
- Health Information Management Research Center, Kashan University of Medical Sciences, Kashan, Islamic Republic of Iran. .,Department of Health Information Management and Technology, Kashan University of Medical Sciences, Kashan, Islamic Republic of Iran.
| |
Collapse
|
2
|
Farzandipour M, Nabovati E, Tadayon H, Sadeqi Jabali M. Identification and Classification of Usability Problems in a Nursing Information System: A Heuristic Evaluation. Comput Inform Nurs 2021; 40:121-130. [PMID: 35115439 DOI: 10.1097/cin.0000000000000803] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
Nursing information system is a system that nurses are required to use in order to perform their daily activities. Poor user interface design of nursing information system can cause problems in nurses' interaction with the system. This research aimed to evaluate the usability of nursing information system. During this study, five evaluators examined the nursing information system with the heuristic evaluation method and by using the checklist of Nielsen usability principles. Then, the identified problems were categorized into 10 Nielsen usability principles, and the severity of the problems was determined. They also attributed each problem to one of usability attributes. A total of 104 unique problems were identified. Most of the problems were related to the principle of "consistency and standards." More than a third of the identified problems were classed as major and catastrophe, with the highest severity in the average problems of "help and documentation" and "error prevention." Most of the identified problems were attributed to effectiveness and satisfaction. Because a significant number of identified usability problems in nursing information system were major and catastrophe and were attributed to effectiveness and satisfaction, being able to remedy the problems could improve nurses' interaction with the system user interface and increase satisfaction and effectiveness of nurses.
Collapse
Affiliation(s)
- Mehrdad Farzandipour
- Author affiliations: Health Information Management Research Center, and Department of Health Information Management & Technology (Dr Farzandipour, Dr Nabovati, and Ms Sadeqi Jabali), Kashan University of Medical Sciences, Kashan, I.R.Iran; Health Information Management Research Center (Mr Tadayon), Kashan University of Medical Sciences, Kashan I.R.Iran, and Department of Health Information Technology (Mr Tadayon), Neyshabur University of Medical Sciences, Neyshabur, I.R.Iran
| | | | | | | |
Collapse
|
3
|
Lloyd S, Long K, Oshni Alvandi A, Di Donato J, Probst Y, Roach J, Bain C. A National Survey of EMR Usability: Comparisons between medical and nursing professions in the hospital and primary care sectors in Australia and Finland. Int J Med Inform 2021; 154:104535. [PMID: 34425552 DOI: 10.1016/j.ijmedinf.2021.104535] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2020] [Revised: 06/09/2021] [Accepted: 06/16/2021] [Indexed: 11/28/2022]
Abstract
BACKGROUND Electronic Medical Record Systems (EMRs) are now part of nursing and medical professionals daily work in the acute and primary care sectors in Australia. Usability is an important factor in their successful adoption and impacts upon clinical workflow, safety and quality, communication, and collaboration. This study replicates a significant body of work conducted by Finnish researchers applying a usability focused survey to understand medical and nursing professionals' experiences in the Australian context. As we implement EMRs across health systems, their usability and design to support clinicians to effectively deliver and document care, is essential. METHODS We conducted an observational study using a cross sectional survey, the National Usability-Focused HIS Scale (NuHISS) developed and validated by Finnish researchers. For this study 13 usability statements collected clinician impressions of EMRs related to technical quality, ease of use, benefits, and collaboration. We report the responses from medical and nursing professionals working in clinical practice settings in Australia, including primary care and hospital sectors in 2020. RESULTS Nursing and medical professionals have different experiences with EMR usability. This depends on the sector they work in and the usability feature measured. In our sample, technical quality features were more positively experienced by doctors in the primary care sector than nurses as well as ease of obtaining patient information and prevention of errors. In the hospital sector nurses experiences with EMRs were more positive with respect to support for routine task completion, learnability, ease of obtaining patient information and entry of patient data. CONCLUSIONS The NuHISS is a suitable tool for measuring the usability experiences of Australian clinicians and the EMRs utilised. Differences in usability experiences were noted between professional groups and sectors. A focus on the usability perspectives of clinicians when enhancing or developing EMR solutions is advocated.
Collapse
Affiliation(s)
| | - Karrie Long
- Nursing Research Hub, The Royal Melbourne Hospital, Melbourne, Australia
| | | | - Josie Di Donato
- QUT Online, Queensland University of Technology, Brisbane, Australia
| | - Yasmine Probst
- School of Medicine, University of Wollongong, Wollongong, NSW, Australia; Illawarra Health and Medical Research Institute, Wollongong, NSW, Australia
| | - Jeremy Roach
- Clinical Informatics Community of Practice, Australasian Institute of Digital Health, Melbourne, Australia
| | - Christopher Bain
- Department of Human Centred Computing, Monash Data Futures Institute, Monash University, Melbourne, Australia
| |
Collapse
|
4
|
Farzandipour M, Nabovati E, Tadayon H, Sadeqi Jabali M. Usability evaluation of a nursing information system by applying cognitive walkthrough method. Int J Med Inform 2021; 152:104459. [PMID: 34091145 DOI: 10.1016/j.ijmedinf.2021.104459] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2020] [Revised: 03/05/2021] [Accepted: 04/06/2021] [Indexed: 11/25/2022]
Abstract
BACKGROUND AND OBJECTIVE The user interface usability of the nursing information system (NIS) should be such that the nurses can learn and interact with it easily and quickly. Therefore, it is necessary to identify and solve the usability problems of these systems. The present study aimed to evaluate the usability of a NIS using the cognitive walkthrough (CW) evaluation method. METHODS Based on five selected scenarios, five evaluators evaluated the NIS in Shafa Hospital Information System. After identifying the problems, the evaluators assigned each problem to one of the usability attributes. The severity of each identified problem was determined by the evaluators and five real users of the system. RESULTS In total, 24 unique problems were identified. The average severity of the problems was determined by the evaluators (2.77) and the actual users (2.82) in the "major problem" category. The highest number of problems were assigned to the scenarios 3 and 2 with 15 and 14 problems, respectively. The highest average severity in terms of evaluations and actual users was related to the scenario 5 (3.06 and 2.94, respectively), which was in the "major problem" category. The highest number of problems were associated with learnability (8 problems) and efficiency (6 problems). CONCLUSION Since most of the nurses do not have enough time for learning the system, and given that the selected scenarios for evaluation were based on the most frequent daily tasks that nurses performed using the NIS; The correction of usability problems in the scenarios 3 and 2 that had the highest number of problems and the scenario 5 that had the highest average severity and execution time; it can reduce the nurses' cognitive load and the learning time of the system and increase the efficiency of nurses.
Collapse
Affiliation(s)
- Mehrdad Farzandipour
- Research Centre for Health Information Management, Kashan University of Medical Sciences, Kashan, Islamic Republic of Iran; Department of Health Information Management & Technology, Kashan University of Medical Sciences, Kashan, Islamic Republic of Iran
| | - Ehsan Nabovati
- Research Centre for Health Information Management, Kashan University of Medical Sciences, Kashan, Islamic Republic of Iran; Department of Health Information Management & Technology, Kashan University of Medical Sciences, Kashan, Islamic Republic of Iran
| | - Hamidreza Tadayon
- Department of Health Information Management & Technology, Kashan University of Medical Sciences, Kashan, Islamic Republic of Iran; Department of Health Information Technology, Neyshabur University of Medical Sciences, Neyshabur, Islamic Republic of Iran
| | - Monireh Sadeqi Jabali
- Department of Health Information Management & Technology, Kashan University of Medical Sciences, Kashan, Islamic Republic of Iran.
| |
Collapse
|
5
|
Heshmati A, Moftian N, Rezaei-Hachesu P, Samad-Soltani T. Simple and Efficient Measurement of the User Experience in Health Information Systems: A Persian Version. IRANIAN JOURNAL OF PUBLIC HEALTH 2021; 50:217-218. [PMID: 34178788 PMCID: PMC8213632 DOI: 10.18502/ijph.v50i1.5097] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
Abstract
The article's abstract is not available.
Collapse
Affiliation(s)
- Arefe Heshmati
- Department of Health Information Technology, School of Management and Medical Informatics, Tabriz University of Medical Sciences, Tabriz, Iran
| | - Nazila Moftian
- Department of Health Information Technology, School of Management and Medical Informatics, Tabriz University of Medical Sciences, Tabriz, Iran
| | - Peyman Rezaei-Hachesu
- Department of Health Information Technology, School of Management and Medical Informatics, Tabriz University of Medical Sciences, Tabriz, Iran
| | - Taha Samad-Soltani
- Department of Health Information Technology, School of Management and Medical Informatics, Tabriz University of Medical Sciences, Tabriz, Iran
| |
Collapse
|
6
|
Simon A. Usability of electronic patient record systems: Instrument validation study conducted for hospitals in Germany. Health Informatics J 2020; 26:1969-1982. [DOI: 10.1177/1460458219895910] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
In Germany and elsewhere, few hospital electronic patient record usability surveys are available. Moreover, there seems to be a complete lack of validated instruments in the respective literature. Hence, this study’s purpose is to validate a scale for measuring the usability of hospital electronic patient record systems. The data used for the study’s analyses (n = 949) were originally obtained during the German national initiative ‘Hospital IT User Questionnaire’. In the course of the study, reliability and exploratory factor analyses were conducted and psychometric tests showed a reliable, valid and suitable instrument. Descriptive data analysis suggests a generally low user perception and variances hint at a high potential for future improvements. To our knowledge, this study shows that a general inventory (subscales from the IsoMetrics inventory) can be applied to measuring hospital electronic patient record usability as well. The validated instrument can be used to inform healthcare providers, decision makers and politicians of a given state of usability, discrepancies between different hospitals or systems providers, and serve as a basis for improvements.
Collapse
Affiliation(s)
- Anke Simon
- Baden-Wuerttemberg Cooperative State University Stuttgart, Germany
| |
Collapse
|
7
|
Serafim RC, Temer MJ, Parada CMGDL, Peres HHC, Serafim CTR, Jensen R. System for reception and risk classification in obstetrics: a technical quality assessment. Rev Lat Am Enfermagem 2020; 28:e3330. [PMID: 32696923 PMCID: PMC7365607 DOI: 10.1590/1518-8345.3327.3330] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2019] [Accepted: 04/13/2020] [Indexed: 08/26/2023] Open
Abstract
Objective: to assess the technical quality of a decision support system for reception
and risk classification in obstetrics. Method: a methodological study of assessment of the system. 12 nurses and 11
information technology (IT) professionals were invited to evaluate the
Obstetrics Reception and Risk Classification System (Sistema de
Acolhimento e Classificação de Risco em Obstetrícia, SACR-O).
Based on the standards of the International Organization for
Standardization, the minimum number of evaluators and the characteristics to
be evaluated were established: functional suitability, reliability,
usability, performance efficiency, compatibility, safety, maintainability,
and portability. The characteristics assessed should be given a ≥70%
positive assessment to be considered suitable. Results: the characteristics assessed by the nurses and the IT professionals,
respectively, were considered adequate: Functional suitability (97% and
98%), Reliability (91% and 94%), Usability (89% and 93%), Performance
efficiency (97% and 98%), Compatibility (93% and 100%), and System security
(95% and 97%). Maintainability (87%) and Portability (97%) were also
evaluated by IT professionals. Conclusion: the technical quality of the SACR-O system was considered excellent by nurses
and IT professionals.
Collapse
Affiliation(s)
| | | | | | | | | | - Rodrigo Jensen
- Universidade Estadual Paulista “Júlio de Mesquita Filho”, Brazil
| |
Collapse
|