Clinical information systems end user satisfaction: The expectations and needs congruencies effects

Data management system for diabetes clinical trials: a pre-post evaluation study

BACKGROUND

Data management system for diabetes clinical trials is used to support clinical data management processes. The purpose of this study was to evaluate the quality and usability of this system from the users' perspectives.

METHODS

This study was conducted in 2020, and the pre-post evaluation method was used to examine the quality and usability of the designed system. Initially, a questionnaire was designed and distributed among the researchers who were involved in the diabetes clinical trials (n = 30) to investigate their expectations. Then, the researchers were asked to use the system and explain their perspectives about it by completing two questionnaires.

RESULTS

There was no statistically significant differences between the users' perspectives about the information quality, service quality, achievements, and communication before and after using the system. However, in terms of the system quality (P = 0.042) and users' autonomy (P = 0.026), the users' expectations were greater than the system performance. The system usability was at a good level based on the users' opinions.

CONCLUSION

It seems that the designed system largely met the users' expectations in most areas. However, the system quality and users' autonomy need further attentions. In addition, the system should be used in multicenter trials and re-evaluated by a larger group of users.

A Clinical Data Management System for Diabetes Clinical Trials

BACKGROUND

The use of novel medications and methods to prevent, diagnose, treat, and manage diabetes requires confirmation of safety and efficacy in a well-designed study prior to widespread adoption. Diabetes clinical trials are the studies that examine these issues. The aim of the present study was to develop a web-based system for data management in diabetes clinical trials.

METHODS

The present research was a mixed-methods study conducted in 2019. To identify the required data elements and functions to develop the system, 60 researchers completed a questionnaire. The designed system was evaluated using two methods. The usability of the system was initially evaluated by a group of researchers (n = 6) using the think-aloud method, and after system improvement, the system functions were evaluated by other researchers (n = 30) using a questionnaire.

RESULTS

The main data elements which were required to develop a case report form included "study data," "participant's personal data," and "clinical data." The functional requirements of the system were "managing the study," "creating case report forms," "data management," "data quality control," and "data security and confidentiality." After using the system, researchers rated the system functions at a "good" level (6.3 ± 0.73) on a seven-point Likert scale.

CONCLUSION

Given the complexity of the data management processes in diabetes clinical trials and the widespread use of information technologies in research, the use of clinical data management systems in diabetes clinical trials seems inevitable. The system developed in the current study can facilitate and improve the process of creating and managing case report forms as well as collecting data in diabetes clinical trials.

Evaluation of the Integrated Health Information System (IHIS) in Public Hospitals in Cyprus Utilizing the DIPSA Framework

INTRODUCTION

The Ministry of Health of the Republic of Cyprus has implemented an Integrated Health Information System (IHIS) in two hospitals. However, no evaluation of IHIS has been conducted to assess its safety, efficiency and effectiveness. The proper utilization of IHIS is essential for the provision of quality healthcare services.

AIM

The purpose of this study was to evaluate the current IHIS in public hospitals in Cyprus utilizing the DIPSA evaluation framework.

METHODS

A total of 309 subjects, including doctors, nurses and other healthcare professionals, participated in the study. The DIPSA evaluation framework assessed the users' perception in five categories namely, satisfaction, collaboration, system quality, safety and procedures, using Likert scale and 3 open questions. Correlation between the categories was assessed using the Pearson correlation coefficient, and multiple regression analysis was used to examine the relationship between the demographic characteristics and categories. Data analysis was done using SPSS v24.

RESULTS

All five categories were rated moderately, between 2.5 and 3, by the participants. All categories were correlated (P < 0.01). Multiple regression analysis indicated the need for improvement between the professionals (mainly doctors and nurses) and the categories. The open questions pointed out the need for improvement in all 3 factors examined (Technology, Human Factor, Organization).

DISCUSSION

The moderately rated categories, in the Cyprus IHIS, suggest that there is a lot of room for improvement. Some interventions are suggested that could positively and simultaneously affect one or more categories.

Investigation of Resistance, Perception and Attitudes of Employees against Change in Information Systems Using Change Management Approach: A Study in a University Hospital

The need for change in the rapidly developing health sector is essential for the provision of quality health care services. In this study, the technological change experienced in the process of transition from the old hospital information management system used in a university hospital to the new hospital information management system was discussed and employees’ resistance, perceptions and attitudes to change have been examined through a change management approach. In this study, we aimed to examine the level of usage of the change management tools and change management phases in a university hospital through a change management approach in the transition phase to the new HIS. We also aim to discuss faced problems encountered in the change process, help to create consciousness for management of change, examine behavioural reasons of resistance of healthcare professionals and present our experiences and propose solutions in the view of adoption and acceptance management tools against change in Hospital Information Systems (HIS) for healthcare managers. In the research, it has been observed that in the transition to the new information system, the change management tools are applied partially whereas change management phases have been implemented to a large extent. University hospital employees are resistant to Hospital Information Management System replacement. The most resistant groups are doctors and nurses and the least resistant employees are administrative staff. The rise in hospital revenues following the change is noteworthy.

Most Influential Qualities in Creating Satisfaction Among the Users of Health Information Systems: Study in Seven European Union Countries

Background

Several models suggest how the qualities of a product or service influence user satisfaction. Models such as the Customer Satisfaction Index (CSI), Technology Acceptance Model (TAM), and Delone and McLean Information Systems Success demonstrate those relations and have been used in the context of health information systems.

Objective

This study aimed to investigate which qualities foster greater satisfaction among patient and professional users. In addition, we are interested in knowing to what extent improvement in those qualities can explain user satisfaction and whether this makes user satisfaction a proxy indicator of those qualities.

Methods

The Unified eValuation using ONtology (UVON) method was used to construct an ontology of the required qualities for 7 electronic health (eHealth) apps being developed in the Future Internet Social and Technological Alignment Research (FI-STAR) project, a European Union (EU) project in electronic health (eHealth). The eHealth apps were deployed across 7 EU countries. The ontology included and unified the required qualities of those systems together with the aspects suggested by the Model for ASsessment of Telemedicine apps (MAST) evaluation framework. Moreover, 2 similar questionnaires for 87 patient users and 31 health professional users were elicited from the ontology. In the questionnaires, the user was asked if the system has improved the specified qualities and if the user was satisfied with the system. The results were analyzed using Kendall correlation coefficients matrices, incorporating the quality and satisfaction aspects. For the next step, 2 partial least squares structural equation modeling (PLS-SEM) path models were developed using the quality and satisfaction measure variables and the latent construct variables that were suggested by the UVON method.

Results

Most of the quality aspects grouped by the UVON method are highly correlated. Strong correlations in each group suggest that the grouped qualities can be measures that reflect a latent quality construct. The PLS-SEM path analysis for the patients reveals that the effectiveness, safety, and efficiency of treatment provided by the system are the most influential qualities in achieving and predicting user satisfaction. For the professional users, effectiveness and affordability are the most influential. The parameters of the PLS-SEM that are calculated allow for the measurement of a user satisfaction index similar to CSI for similar health information systems.

Conclusions

For both patients and professionals, the effectiveness of systems highly contributes to their satisfaction. Patients care about improvements in safety and efficiency, whereas professionals care about improvements in the affordability of treatments with health information systems. User satisfaction is reflected more in the users’ evaluation of system output and fulfillment of expectations but slightly less in how far the system is from ideal. Investigating satisfaction scores can be a simple and fast way to infer if the system has improved the abovementioned qualities in treatment and care.

Development and Evaluation of a User-Centered Mobile Telestroke Platform

Background: Reperfusion is the most effective acute treatment for ischemic stroke within a narrow therapeutic time window. Ambulance-based telestroke is a novel way to improve stroke diagnosis and timeliness of treatment. This study aims to (1) assess the usability of our ambulance-based telestroke platform and (2) identify strengths and limitations of the system from the user's perspective. Materials and Methods: An ambulance was equipped with a mobile telemedicine system to perform remote stroke assessments. Scripted scenarios were performed by actors during transport and evaluated by physicians using the National Institutes of Health Stroke Scale (NIHSS). Scores obtained during transport were compared with original scripted NIHSS scores. Participants completed the System Usability Scale (SUS), NASA Task Load Index (NASA TLX), audio/video quality scale, and a modified Acceptability of Technology survey to assess perceptions and usability. In addition, interviews were conducted to evaluate user's experience. Descriptive analysis was used for all surveys. Weighted kappa statistics was used to compare the agreement in NIHSS scores. Results: Ninety-one percent (59/65) of mobile scenarios were completed. Median completion time was 9 min (range 4-17 min). There was moderate inter-rater agreement (weighted kappa = 0.46 [95% confidence interval 0.33-0.60, p = 0.0018]) among mobile and original scripted scenarios. The mean SUS score was 68.8 (standard deviation = 15.9). There was variability between usability score and formative feedback among all end-users in the areas of usability issues (i.e., audibility and equipment stability) and safety. Conclusion: Before implementation of a mobile prehospital telestroke program, the use of combined clinical simulation and Plan-Do-Study-Act methodology can improve the quality and optimization of the telemedicine system.

The use of information technology to enhance patient safety and nursing efficiency

BACKGROUND

Issues in patient safety and nursing efficiency have long been of concern. Advancing the role of nursing informatics is seen as the best way to address this.

OBJECTIVES

The aim of this study was to determine if the use, outcomes and satisfaction with a nursing information system (NIS) improved patient safety and the quality of nursing care in a hospital in Taiwan.

METHOD

This study adopts a quasi-experimental design. Nurses and patients were surveyed by questionnaire and data retrieval before and after the implementation of NIS in terms of blood drawing, nursing process, drug administration, bar code scanning, shift handover, and information and communication integration.

RESULTS

Physiologic values were easier to read and interpret; it took less time to complete electronic records (3.7 vs. 9.1 min); the number of errors in drug administration was reduced (0.08% vs. 0.39%); bar codes reduced the number of errors in blood drawing (0 vs. 10) and transportation of specimens (0 vs. 0.42%); satisfaction with electronic shift handover increased significantly; there was a reduction in nursing turnover (14.9% vs. 16%); patient satisfaction increased significantly (3.46 vs. 3.34).

CONCLUSIONS

Introduction of NIS improved patient safety and nursing efficiency and increased nurse and patient satisfaction. Medical organizations must continually improve the nursing information system if they are to provide patients with high quality service in a competitive environment.

Evaluation of Nursing Information Systems: Application of Usability Aspects in the Development of Systems

Objectives

As the largest group providing healthcare services, nurses require well-designed information systems in their practice. This study aims to evaluate the usability of nursing information systems (NIS).

Methods

This cross-sectional survey was conducted in 2015. The settings of the study consisted of four hospitals affiliated with three medical universities in Tehran (Iran). The subjects of the study included nurses who had access to and used a NIS developed by four major software companies. The data were collected using a modified version of a usability questionnaire known as IsoMetrics, based on the International Standard ISO 9241, Part 11. The questionnaire is composed of 35 questions divided into seven general criteria. The validity of the questionnaire was determined by experts in the field, and the reliability was checked using Cronbach's alpha (α = 0.91). The questionnaire was then distributed to 184 nurses.

Results

The response rate was 64.6%. Among the seven ISO usability criteria, suitability for the task (3.10 ± 1.24) and suitability for learning (3.10 ± 1.27) had the highest mean value. The lowest mean value (2.37 ± 1.29) was related to the suitability for individualization.

Conclusions

Addressing issues related to individualization and self-descriptiveness could improve the usability of nursing systems. Considering usability requirements in the design of a NIS will lead to the efficient and effective use of these systems.