Development and initial evaluation of the Clinical Information Systems Success Model (CISSM)

A scalable approach for critical care data extraction and analysis in an academic medical center

BACKGROUND

Electronic health records are a valuable asset for research, but their use is challenging due to inconsistencies of records, heterogeneous formats and the distribution over multiple, non-integrated information systems. Hence, specialized health data engineering and data science expertise are required to enable research. To facilitate secondary use of clinical routine data collected in our intensive care wards, we developed a scalable approach, consisting of cohort generation, variable filtering and data extraction steps.

OBJECTIVE

With this report we share our workflow of data request, cohort identification and data extraction. We present an algorithm for automatic data extraction from our critical care information system (CCIS) that can be adapted to other object-oriented data bases.

METHODS

We introduced a data request process with functionalities for automated identification of patient cohorts and a specialized hierarchical data structure that supports filtering relevant variables from the CCIS and further systems for the specified cohorts. The data extraction algorithm takes patient pseudonyms and variable lists as inputs. Algorithms are implemented in Python, leveraging the PySpark framework running on our data lake infrastructure.

RESULTS

Our data request process is in operational use since June 2022. Since then we have served 121 projects with 148 service requests in total. We discuss the hierarchical structure and the frequently used data items of our CCIS in detail and present an application example, including cohort selection, data extraction and data transformation into an analyses-ready format.

CONCLUSIONS

Using clinical routine data for secondary research is challenging and requires an interdisciplinary team. We developed a scalable approach that automates steps for cohort identification, data extraction and common data pre-processing steps. Additionally, we facilitate data harmonization, integration and consult on typical data analysis scenarios, machine learning algorithms and visualizations in dashboards.

Development and validation of an instrument to evaluate the perspective of using the electronic health record in a hospital setting

BACKGROUND

Evaluating healthcare information systems, such as the Electronic Health Records (EHR), is both challenging and essential, especially in resource-limited countries. This study aims to psychometrically develop and validate an instrument (questionnaire) to assess the factors influencing the successful adoption of the EHR system by healthcare professionals in Moroccan university hospitals.

METHODS

The questionnaire validation process occurred in two main stages. Initially, data collected from a pilot sample of 164 participants underwent analysis using exploratory factor analysis (EFA) to evaluate the validity and reliability of the retained factor structure. Subsequently, the validity of the overall measurement model was confirmed using confirmatory factor analysis (CFA) in a sample of 368 healthcare professionals.

RESULTS

The structure of the modified HOT-fit model, comprising seven constructs (System Quality, Information Quality, Information technology Service Quality, User Satisfaction, Organization, Environment, and Clinical Performance), was confirmed through confirmatory factor analysis. Absolute, incremental, and parsimonious fit indices all indicated an appropriate level of acceptability, affirming the robustness of the measurement model. Additionally, the instrument demonstrated adequate reliability and convergent validity, with composite reliability values ranging from 0.75 to 0.89 and average variance extracted (AVE) values ranging from 0.51 to 0.63. Furthermore, the square roots of AVE values exceeded the correlations between different pairs of constructs, and the heterotrait-monotrait ratio of correlations (HTMT) was below 0.85, confirming suitable discriminant validity.

CONCLUSIONS

The resulting instrument, due to its rigorous development and validation process, can serve as a reliable and valid tool for assessing the success of information technologies in similar contexts.

Multidisciplinary user experience of a newly implemented electronic patient record in Ireland: An exploratory qualitative study

BACKGROUND

Implementation of an Electronic Patient Record (EPR) in a key milestone in the digital strategy of modern healthcare organisations. The implementation of EPR systems can be viewed as challenging and complex.

OBJECTIVE

The aim of the study was to investigate user perspectives and experiences of the implementation of an Electronic Medical Record in a major academic teaching hospital, with simultaneous 'go-live' across the whole hospital taking place.

METHODS

Focus groups and individual in-depth interviews were conducted with stakeholders and users (n = 105), approximately nine months post-EPR implementation. The study explored EPR users' perceptions using an extended theoretical framework of the DeLone and McLean Information Systems Success Model (2003), which measured information systems, system quality, information quality, service quality, use/perceived usefulness & user satisfaction and net benefits.

RESULTS

Staff engagement and satisfaction was high and the EPR is accepted as the new standard way of completing care. There was agreement that the EPR affords transparency, and greater accountability. There was some concern expressed regarding impact of the EPR on interprofessional and patient/provider interactions and communication. Physicians reported the inputting of social history through free text as an issue of concern and time consuming. The Big Bang approach with mandatory conversion was key to the successful adoption of EPR. There was consensus across professional and administrative respondents that there was no appetite to return to paper-based records.

CONCLUSION

The successful roll out of the EPR reflects the digital readiness of healthcare providers and organisations. The potential for unintended consequences on work process requires continual monitoring. A key future benefit of the EPR will be the capacity to reach a broader understanding and analysis of variation in processes and outcomes within healthcare organisations. It is clear that skills in data analytics will be needed to mine data successfully.

Healthcare professionals' satisfaction toward the use of district health information system and its associated factors in southwest Ethiopia: using the information system success model

Background

Ethiopia has the potential to use the district health information system, which is a building block of the health system. Thus, it needs to assess the performance level of the system by identifying the satisfaction of end users. There is little evidence about users' satisfaction with using this system. As a result, this study was conducted to fill this gap by evaluating user satisfaction and associated factors of district health information system among healthcare providers in Ethiopia, using the information system success model.

Methods

An institutional-based cross-sectional study was conducted from November to December 2022 in the Oromia region of southwest Ethiopia. A total of 391 health professionals participated in the study. The study participants were selected using a census. Using a self-administered questionnaire, data were collected. Measurement and structural equation modeling analyses were used to evaluate reliability, the validity of model fit, and to test the relationship between the constructs, respectively, using analysis of moment structure (AMOS) V 26.

Results

System quality had a positive direct effect on the respondent's system use (β = 0.18, P-value < 0.001), and satisfaction (β = 0.44, P-value < 0.001). Service quality had also a direct effect on the respondent's system use (β = 0.37, P-value < 0.01), and satisfaction with using the district health information system (β = 0.36, P-value < 0.01). Similarly, system use had also a direct effect on the respondent's satisfaction (β = 0.53, P-value < 0.05). Moreover, computer literacy had a direct effect on the respondent's system use (β = 0.63, P-value < 0.05), and satisfaction (β = 0.51, P-value < 0.01).

Concussions

The overall user satisfaction with using the district health information system in Ethiopia was low. System quality, service quality, and computer literacy had a direct positive effect on system use and user satisfaction. In addition, system use and information quality had a direct positive effect on healthcare professionals' satisfaction with using the district health information system. The most important factor for enhancing system use and user satisfaction was computer literacy. Accordingly, for the specific user training required for the success of the district health information system in Ethiopia, the manager should offer additional basic computer courses for better use of the system.

Acceptance of e-learning in higher education: The role of task-technology fit with the information systems success model

The COVID-19 global epidemic has compelled higher education institutions to reconsider their teaching methods. Because of this public health emergency, universities in higher education have adopted e-learning techniques as a solution to face-to-face education. Thus, e-learning has emerged as a critical technology in education at higher education institutions. Nonetheless, the effectiveness of e-learning systems is largely dependent on students' adoption of such systems. The study aims to evaluate the usefulness of task-technology fit (TTF) with the information system success model (ISSM) in perceiving students' adoption of e-learning with the goal of encouraging them to adopt e-learning in the context of higher education. The study employed a quantitative approach, and a theoretical model was evaluated with proposed hypotheses to find the relationships between the constructs. A questionnaire based on TTF and ISSM was distributed among the students, and 260 valid responses were received using a sample random sampling technique. Data was analyzed with the help of SPSS and Partial Least Squares-Structural Equation Modeling (PLS-SEM). After analyzing the data, it was found that perceived ease of use, perceived usefulness, system use, and task technology fit of e-learning are positively and significantly influenced by system quality, information quality, perceived enjoyment, technology characteristics, and task characteristics. The results of TTF and ISSM on system use show a positive effect on e-learning benefits in educational institutions, with all male and female students completely satisfied with the use of e-learning systems. As a result, we advise students to use e-learning systems for educational purposes and should have motivated them to do so through lecturers at higher-level educational institutions.

An extended DeLone and McLean's model to determine the success factors of e-learning platform

Due to the COVID-19 pandemic, all Saudi universities have adopted e-learning systems to ensure that educational activities continue. Shaqra University adopted a platform called the Shaqra University e-learning platform. This study aimed to identify the factors contributing to the success of that platform in Shaqra University, based on students' responses. This research has proposed an extension of well-known DeLone and McLean's Information Systems Success (D&M ISS) model to check and validate the success factors of the Shaqra University platform. The questionnaire was adopted in this study to collect data from students currently enrolled at Shaqra University. One thousand online links to the questionnaire were randomly distributed among current students enrolled in Shaqra University. The results revealed that the instrument adopted in this study was valid and reliable. Also, the results showed that the model was a good fit for the Saudi context. The proposed factors of instructor's quality, learner quality, and perceived usefulness positively impacted the e-learning platform. On the other hand, the factors information quality, system quality and service quality had no positive impact on the use of the e-learning platform.

Investigating Students' Perceptions of Online Learning Use as a Digital Tool for Educational Sustainability During the COVID-19 Pandemic

Previous research on e-learning in underdeveloped countries has seldom taken a comprehensive approach. A literature review of recent published research in the field of e-learning use during the COVID-19 epidemic is also included in this study. Therefore, the aim of this study is to look at the technology acceptance model (TAM) and information systems (IS) performance models to see how system quality (SYQ), service quality (SEQ), and quality of life (QoL) are related, as well as the mediating impact of perceived ease of use (PEU) and perceived usefulness (PU), affect students' behavioral intention to use (BIU), and actual use of an e-learning system (AUE) as sustainability for education during the COVID-19 pandemic. Path analysis and structural equation modeling (SEM) were used to evaluate the research model, using the data from e-learning users obtained through a survey. Participants were e-learning users from two Saudi Arabian public universities. The findings revealed that PU and ease of use were positively correlated and influenced by SYQ, SEQ, and QoL in education, and that PEU and PU were positively influenced by students' BIU and AUE system. In the sense of e-learning in developing countries, previous studies rarely looked at an integrated model. This paper also attempts to provide a recently published study in the area of the use of an e-learning system as sustainability for education during the COVID-19 pandemic. There is a lot of ongoing research.

Evaluation frameworks for digital nursing technologies: analysis, assessment, and guidance. An overview of the literature

Background

The evaluation of digital nursing technologies (DNT) plays a major role in gaining knowledge about certain aspects of a technology such as acceptance, effectiveness, or efficiency. Evaluation frameworks can help to classify the success or failure of a DNT or to further develop the technology. In general, there are many different evaluation frameworks in the literature that provide overviews of a wide variety of aspects, which makes this a highly diverse field and raises the question how to select a suitable framework. The aim of this article is to provide orientation in the field of comprehensive evaluation frameworks that can be applied to the field of DNT and to conduct a detailed analysis and assessment of these frameworks to guide field researchers.

Methods

This overview was conducted using a three-component search process to identify relevant frameworks. These components were (1) a systematized literature search in PubMed; (2) a narrative review and (3) expert consultations. Data relating to the frameworks’ evaluation areas, purpose, perspectives, and success definitions were extracted. Quality criteria were developed in an expert workshop and a strength and weakness assessment was carried out.

Results

Eighteen relevant comprehensive evaluation frameworks for DNT were identified. Nine overarching evaluation areas, seven categories of purposes, five evaluation perspectives and three categories of success definitions could be identified. Eleven quality criteria for the strengths and weaknesses of DNT-related evaluation frameworks were developed and the included frameworks were assessed against them.

Conclusion

Evaluators can use the concise information and quality criteria of this article as a starting point to select and apply appropriate DNT evaluation frameworks for their research projects or to assess the quality of an evaluation framework for DNT, as well as a basis for exploring the questions raised in this article. Future research could address gaps and weaknesses in existing evaluation frameworks, which could improve the quality of future DNT evaluations.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12912-021-00654-8.

An Essential Clinical Dataset Intervention for Nursing Documentation of a Pediatric Admission History Database

PURPOSE

The purpose of this study was to improve nursing documentation efficiencies and satisfaction of a pediatric admission history workflow. Secondary aims determined if defining essential data elements was associated with decreased pediatric admission history documentation time, increased dataset completion rate, and increased satisfaction.

DESIGN AND METHODS

A quasi-experimental between-group difference comparison was conducted for a nurse-led quality improvement study that included implementation of a pediatric essential clinical dataset (ECD) tool for pre/post-intervention analysis of nursing admission history documentation time, dataset completion rate, and satisfaction. A survey was administered to nurses pre- and post-intervention to compare documentation satisfaction.

RESULTS

Nursing admission history documentation time decreased by 1 min 31 s and the number of clicks decreased 38%. Dataset utilization increased 8% indicating improved nursing documentation of essential questions within a pediatric admission history form. Nursing documentation satisfaction with the pediatric admission history form was minimally impacted by the pediatric ECD study intervention.

CONCLUSIONS

Defining what is essential for nurses to document positively influenced nursing documentation time, dataset completion rate, and satisfaction.

PRACTICE IMPLICATIONS

The study contributed to EHR content standardization, optimization, and documentation efficiencies for nurses within a pediatric organization with implications for clinical and informatics collaboration to create real-world evidence, leveraging an intervention that decreased documentation burden and increased time for children and families.

Investigation the success rate of hospital information system (HIS): Development of a questionnaire and case study

INTRODUCTION AND OBJECTIVES

The implantation of hospital information systems (HISs) has grown dramatically in recent years. Understanding the success rate of HIS is key in health organizations. In this study, a validated questionnaire for HISs evaluation based on the Information System Success Model (ISSM) has been provided. In addition, the HIS success rate was determined.

MATERIALS AND METHODS

The current study was conducted at one of the largest teaching hospitals in eastern Iran. The 44-items questionnaire was developed for data gathering. The questionnaire covered ISSM dimensions, which include analysis of quality system, quality of information, quality service, system use, usefulness, satisfaction, and net benefits. Content validity, constructs validity, and reliability of the ISSM questionnaire was measured. HIS success rate has been determined and categorized based on users' perspective as follows: appropriate (75%≤HIS success rate), moderate (50%≤HIS success rate<75%), low coverage (25%≤HIS success rate<50%), and poor (coverage rate<25%).

RESULTS

In total, 253 users participated in the study. The ISSM questionnaire was validated by an expert panel with CVI: 85.12% and CVR: 88.22%. The overall Cronbach's alpha value of the instrument was determined as 92.2%. Nine factors with eigenvalues greater than 1.00 were identified, jointly accounting for 66.91% of the total variance. The value of KMO was.866 showed that the sample size was adequate for factor analysis. The highly significant Bartlett's test (p<0.000) indicated that variables were correlated and the factor analysis was appropriate. Our results demonstrated that the total mean of HIS success was "moderate" base on the users' point of view.

CONCLUSIONS

The findings of the current study, provide valuable scientific evidence for key affecting factors on hospital EHR in Iran as a developing country.

Investigating Evaluation Frameworks for Electronic Health Record: A Literature Review

BACKGROUND: There are various electronic health records (EHRs) evaluation frameworks with multiple dimensions and numerous sets of evaluation measures, while the coverage rate of evaluation measures in a common framework varies in different studies. AIM: This study provides a literature review of the current EHR evaluation frameworks and a model for measuring the coverage rate of evaluation measures in EHR frameworks. METHODS: The current study was a comprehensive literature review and a critical appraisal study. The study was conducted in three phases. In Phase 1, a literature review of EHR evaluation frameworks was conducted. In Phase 2, a three-level hierarchical structure was developed, which includes three aspects, 12 dimensions, and 110 evaluation measures. Subsequently, evaluation measures in the identified studies were categorized based on the hierarchical structure. In Phase 3, relative frequency (RF) of evaluation measures in different dimensions and aspects for each of the identified studies were determined and categorized as follows: Appropriate, moderate, and low coverage. RESULTS: Out of a total of 8276 retrieved articles, 62 studies were considered relevant. The RF range in the second and third level of the hierarchical structure was between 8.6%–91.94% and 0.2%–61%, respectively. “Ease of use” and “system quality” were the most frequent evaluation measure and dimension. Our results indicate that identified studies cover at least one and at most nine evaluation dimensions and current evaluation frameworks focus more on the technology aspect. Almost in all identified studies, evaluation measures related to the technology aspect were covered. However, evaluation measures related to human and organization aspects were covered in 68% and 84% of the identified studies, respectively. CONCLUSION: In this study, we systematically reviewed all literature presenting any type of EHR evaluation framework and analyzed and discussed their aspects and features. We believe that the findings of this study can help researchers to review and adopt the EHR evaluation frameworks for their own particular field of usage.

Willingness to Use Electronic Medical Record (EMR) System in Healthcare Facilities of Bahir Dar City, Northwest Ethiopia

Background

Globally, electronic information and communication technology has been applied and much expanded in the healthcare industry. However, in developing counties including Ethiopia, EMR system adoption and utilization for medical practice are still inconsistent, and healthcare institutions which started utilization currently have also failed to sustain. A desirable readiness of healthcare experts is mandatory to expand digital health service delivery. Thus, this study is aimed at estimating the proportion of the willingness of professionals in Bahir Dar city to use EMR and at identifying factors associated with this proportion.

Methods

An institution-based cross-sectional study was conducted from September 1 to October 30, 2019, among 634 health professionals. Respondents were selected using a simple random sampling method. Data were entered into EpiData version 3.1 and exported to SPSS version 23 for further analysis. Descriptive statistics were computed to describe study variables and presented using tables. Willingness to use the EMR system was computed. Bivariable and multivariable binary logistic regression models were fitted to identify the associated factors. The odds ratio with 95% confidence interval was used to measure the strength of association.

Results

A total of 616 health professionals participated in the study with a response rate of 97%. The proportion of willingness to use the EMR system was 85.9%. Among health professionals who were not willing to use EMR, lack of access to EMR training (73.4%) was a major barrier to the willingness to use EMR. A multivariable logistic regression analysis showed that those health professionals who had good computer skill (AOR = 2.5; 95% CI: 1.3-4.6), good knowledge on EMR (AOR = 2.1; 95% CI: 1-4.4), gotten EMR training (AOR = 3.8; 95% CI: 1.7-8.1), EMR guideline access (AOR = 2.8; 95% CI: 1.4-5.6), and management support (AOR = 2.6; 95% CI: 1.4-4.8) were more likely willing to use the EMR system.

Conclusions

Majority of the professionals were willing to use the EMR system. EMR program should involve computer illiterate, less knowledgeable, those unable to access EMR guidelines, and managerially unsupported professionals. Enhancing health professionals' attitude and contextualizing EMR training in the healthcare curricula are highly recommended to scale up EMR use.

Implementation outcome instruments for use in physical healthcare settings: a systematic review

BACKGROUND

Implementation research aims to facilitate the timely and routine implementation and sustainment of evidence-based interventions and services. A glaring gap in this endeavour is the capability of researchers, healthcare practitioners and managers to quantitatively evaluate implementation efforts using psychometrically sound instruments. To encourage and support the use of precise and accurate implementation outcome measures, this systematic review aimed to identify and appraise studies that assess the measurement properties of quantitative implementation outcome instruments used in physical healthcare settings.

METHOD

The following data sources were searched from inception to March 2019, with no language restrictions: MEDLINE, EMBASE, PsycINFO, HMIC, CINAHL and the Cochrane library. Studies that evaluated the measurement properties of implementation outcome instruments in physical healthcare settings were eligible for inclusion. Proctor et al.'s taxonomy of implementation outcomes was used to guide the inclusion of implementation outcomes: acceptability, appropriateness, feasibility, adoption, penetration, implementation cost and sustainability. Methodological quality of the included studies was assessed using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. Psychometric quality of the included instruments was assessed using the Contemporary Psychometrics checklist (ConPsy). Usability was determined by number of items per instrument.

RESULTS

Fifty-eight publications reporting on the measurement properties of 55 implementation outcome instruments (65 scales) were identified. The majority of instruments assessed acceptability (n = 33), followed by appropriateness (n = 7), adoption (n = 4), feasibility (n = 4), penetration (n = 4) and sustainability (n = 3) of evidence-based practice. The methodological quality of individual scales was low, with few studies rated as 'excellent' for reliability (6/62) and validity (7/63), and both studies that assessed responsiveness rated as 'poor' (2/2). The psychometric quality of the scales was also low, with 12/65 scales scoring 7 or more out of 22, indicating greater psychometric strength. Six scales (6/65) rated as 'excellent' for usability.

CONCLUSION

Investigators assessing implementation outcomes quantitatively should select instruments based on their methodological and psychometric quality to promote consistent and comparable implementation evaluations. Rather than developing ad hoc instruments, we encourage further psychometric testing of instruments with promising methodological and psychometric evidence.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO 2017 CRD42017065348.

Factors Influencing User Satisfaction with Information Systems: A Systematic Review

User satisfaction has been considered as the measure of information system effectiveness success. User satisfaction is difficult to define but is considered an evaluation construct. Globally health organizations, particularly hospitals, invest a huge amount of money on information system projects. If hospital information systems (HISs) are to be successful, factors influencing or related to user satisfaction should be taken into account at the time of designing, developing or adopting such systems. The current study aimed to provide a comprehensive review of factors related to user satisfaction with information systems. The researchers systematically searched PubMed, Science Direct, and IEEE electronic databases for articles published from January 1990 to June 2016. A search strategy was developed using a combination of the following keywords: "model," "user satisfaction," "information system," "measurement," "instrument," and " tool." Reported dimensions, factors, and their possible influence on user satisfaction with information systems were extracted from the studies wherever was possible. Overall factors influencing user satisfaction with information systems can be categorized in seven dimensions: Information quality, system quality, vendor support quality, system use, perceived usefulness, user characteristics, and organizational structure & management style. If all these factors are considered properly in the process of developing, designing, implementing, or purchasing information systems, the higher user satisfaction with the system will be likely. Otherwise, it would end up with unsatisfied users that will finally contribute to the system failure.

Ensuring continued use of a digital library: a qualitative approach

Purpose

This paper aims to identify the factors affecting continued use of digital libraries in a military context.

Design/methodology/approach

Semi-structured interviews with four focus groups consisting of 21 respondents, who are military education instructors and librarians, were carried out. This qualitative approach study adopted content analysis methods that were designed to contrast and make comparisons based on the participants’ responses. The valid responses were analyzed using NVivo 12 Plus.

Findings

Using semi-structured interviews, this study presents five dimensions of critical success factors generated from the analysis of the content of the qualitative data. The dimensions highlighted are as follows: perceived quality factors; perceived instrumental support; perceived ease of use; users’ expectation and users’ satisfaction; and net benefits and perceived usefulness.

Research limitations/implications

This study is only focussing on military education instructors and librarians in the vicinity of four regions in Peninsular Malaysia. For practical implications, it provides an understanding of how the organization could sustain the continued use of a military-context digital library (DL).

Practical implications

This study makes a new practical contribution to DL information systems’ successful implementation practices in a military context. This study also serves as a guideline for the organizational stakeholders to have a better understanding of their knowledge and the digitalization environment. The findings of this study provide an understanding of how the organization could sustain the continued use of digital libraries in a military context.

Originality/value

This study fills the void in the literature by investigating the DL use in the context of a military setting.

Performance Evaluation of an Information Technology Intervention Regarding Charging for Inpatient Medical Materials at a Regional Teaching Hospital in Taiwan: Empirical Study

Background

The process of manually recording the consumption of medical materials can be time consuming and prone to omission owing to its detailed and complicated nature. Implementing an information system will better improve work performance.

Objective

The Information System Success Model was adopted as the theoretical foundation. The opinions of nursing staff were collected to verify the impact of the system intervention on their work performance.

Methods

This cross-sectional study was conducted at a regional teaching hospital. Nursing staff were invited to participate in the field survey. A total of 296 questionnaires were collected, and of these, 284 (95.9%) were valid and returned.

Results

The key findings showed that two critical factors (“subjective norm” and “system quality”) had significant positive effects (both P<.001) on user satisfaction (R²=0.709). The path of “service quality” to “user satisfaction” showed marginal significance (P=.08) under the 92% CI. Finally, the explanatory power of the model reached 68.9%.

Conclusions

Support from the top management, appointment of a nurse supervisor as the change agent, recruitment of seed members to establish a pioneer team, and promotion of the system through the influence of opinion leaders in small groups were critical success factors needed for implementing the system in the case hospital. The target system was proven to be able to improve work performance, and the time saved could be further used for patient care, thereby increasing the value of nursing work. The positive experiences gained from this study could lay the foundation for the further promotion of the new system, and this is for future studies to replicate. The example of the successful experience of the case hospital could also serve as a reference for other hospitals in developing countries like Taiwan with regard to the promotion of nursing informatization.

Critical success factors of the continued usage of digital library successful implementation in military-context

Purpose

This paper aims to identify the critical success factors of the continued usage of digital library (DL) successful implementation from the perspective of users with organisation support factors as key antecedents.

Design/methodology/approach

Based on the extensive literature review, this study has constructed a conceptual framework based on organisational support perspectives and updated Delone and McLean’s information system success model. A pilot study was carried out on a sample of 105 respondents from military-context. A survey instrument including 22 measurement items was designed to identify the level of DL information system (IS) successful implementation practices in military context. Descriptive statistical analysis and reliability analysis were used to analyse the data with SPSS software.

Findings

The findings indicate that training and education, DL environment and communication support, continuation of usage intention, net benefits and user satisfaction are the critical success factors that play key role in ensuring the continued usage DL successful implementation in military context. The military-context DL needs to obtain critical success factors of DL practices implementation to sustain its continued usage, particularly for the active users of its members.

Research limitations/implications

This study only considered the military personnel who have visited the five main military libraries. For practical implications, the development of instruments in this study may be valuable tools to evaluate DL in other DL settings in Malaysian context other than academic settings.

Originality/value

This study makes a new contribution to DL IS successful implementation practices in military context. This study also provides important information for decision-makers involved in DL IS successful implementation practices and provides useful reference for future researchers in this research area.

External validation of the unified model of information systems continuance (UMISC): An international comparison

CONTEXT

The Unified Model of Information Systems Continuance (UMISC) is a metamodel for the evaluation of clinical information systems (CISs) that integrates constructs from five models that have previously been published in the literature. UMISC was developed at the Georges Pompidou University Hospital (HEGP) in Paris and was partially validated at the Saint Joseph Hospital Group (HPSJ), another acute care institution using the same CIS as HEGP.

OBJECTIVE

The aim of this replication study was twofold: (1) to perform an external validation of UMISC in two different hospitals and country contexts: the Italian Hospital of Buenos Aires (HIBA) in Argentina and the Hospital Sirio Libanes in Sao Paulo, Brazil (HSL); (2) to compare, using the same evaluation model, the determinants of satisfaction, use, and continuance intention observed at HIBA and HSL with those previously observed at HEGP and HPSJ.

METHODS

The UMISC evaluation questionnaires were translated from their original languages (English and French) to Brazilian Portuguese and Spanish following the translation/back-translation method. These questionnaires were then applied at each target site. The 21 UMISC-associated hypotheses were tested using structural equation modeling (SEM).

RESULTS

A total of 3020 users, 1079 at HIBA and 1941 at the HSL, were included in the analysis. The respondents included 1406 medical staff and 1001 nursing staff. The average profession-adjusted use, overall satisfaction and continuance intention were significantly higher at HIBA than at HSL in the medical and nursing groups. In SEM analysis, UMISC explained 23% and 11% of the CIS use dimension, 72% and 85% of health professionals' satisfaction, and 41% and 60% of continuance intention at HIBA and HSL, respectively. Twenty of the 21 UMISC-related hypotheses were validated in at least one of the four evaluation sites, and 16 were validated in two or more sites.

CONCLUSION

The UMISC evaluation metamodel appears to be a robust comparison and explanatory model of satisfaction, use and continuance intention for CISs in late post adoption situations.

Determining the Hospital Information System (HIS) Success Rate: Development of a New Instrument and Case Study

BACKGROUND:

Hospital Information System (HIS) is a type of health information system which is widely used in clinical settings. Determining the success rate of HISs is an ongoing area of research since its implications are of interest for researchers, physicians and managers.

AIM:

In the present study, we develop a novel instrument to measure HIS success rate based on users’ viewpoints in a teaching hospital.

METHODS:

The study was conducted in Ebnesina and Dr Hejazi Psychiatric Hospital and education centre in Mashhad, Iran. The instrument for data collection was a self-administered structured questionnaire based on ISSM, covering seven dimensions, which includes system quality, information quality, service quality, system use, usefulness, satisfaction, and net benefits. The verification of content validity was carried out by an expert panel. The internal consistency of dimensions was measured by Cronbach’s alpha. Pearson’s correlation coefficient was calculated to evaluate the significance of associations between dimensions. The HIS success rate on users’ viewpoints was determined.

RESULTS:

A total of 125 users participated in the study. The instrument was validated by an expert panel with the Content Validity Index (CVI): 0.85 and Content Validity Ratio (CVR): 0.86. The overall Cronbach’s alpha value of the instrument was 0.93. The Pearson correlation coefficient showed significant positive relationships among the investigated dimensions. On average, HIS success rate in the hospital under study was 65% (CI: 64%, 67%). The dimensions of “usefulness”, “system quality”, and “net benefits” showed the highest rates of success, respectively.

CONCLUSION:

The instrument used in this study can be adopted for HIS evaluation in future studies. In the current study, a method was developed to determine the HIS success rate based on users’ viewpoints. This method allows for the comparison of HIS success rates in various hospitals. As well, our findings underscore the viewpoints of HIS users in a developing country.’

Assessing electronic immunization registries: the Pan American Health Organization experience

Objective.

To develop a methodology to assess electronic immunization registries (EIRs) in low- and middle-income countries (LMICs) in Latin America and the Caribbean.

Methods.

A team from the Immunization Unit at the Pan American Health Organization (PAHO) reviewed existing methodologies to evaluate health information systems, particularly the Performance of Routine Information System Management (PRISM) framework and methodologies used to assess information systems. In 2014, the PAHO team convened a small working group to develop an evaluation approach to be added to the existing World Health Organization immunization data quality self-assessment (DQS) tool. The resulting DQS with an added EIR component was named “DQS Plus.” The DQS Plus methodology was used in Panama in May 2014 and in Honduras in November 2015.

Results.

The DQS Plus tool proved feasible and easy to implement in Panama and Honduras, including by not adding much time or resources to those needed for a usual DQS. The information obtained from the DQS Plus assessment was practical and helped provide health authorities with recommendations to update and improve their EIR, strengthen the use of the registry, and enhance the data the assessment produced, at all levels of the health system. These recommendations are currently being implemented in the two countries.

Conclusions.

The DQS Plus proved to be a practical and useful approach for assessing an EIR in an LMIC and generating actionable recommendations. Further work on defining operational and related EIR functional standards in LMICs will help develop an improved EIR assessment tool for Latin America and the Caribbean, and potentially elsewhere.

Strategic Improvement for Quality and Satisfaction of Hospital Information Systems

The purpose of our study aimed to identify attributes capable of improving physicians' satisfaction levels with the use of a hospital information system (HIS). A model inclusive of system quality, information quality, and service quality related to an HIS is used to form antecedents of user satisfaction. Survey methodology was used to collect an attributive set representing the system quality, information quality, and service quality made available from 150 physicians at a large health-care system in southern Taiwan. Responses were segmented into low and high satisfaction and analyzed with partial least squares and importance-performance analysis. The results reveal that system quality, information quality, and service quality may be used to significantly predict physicians' satisfaction. Two system quality attributes (reliability and response time) were identified as the highest priorities for intervention by low- and high-satisfaction users. Low-satisfaction users further expect improvement of the HIS service quality to take place. The subject health-care system should produce coping interventions for those high priorities to enhance the satisfaction of physicians.

Development and validation of a model for the adoption of structured and standardised data recording among healthcare professionals

Background

Healthcare professionals provide care to patients and during that process, record large quantities of data in patient records. Data in an Electronic Health Record should ideally be recorded once and be reusable within the care process as well as for secondary purposes. A common approach to realise this is to let healthcare providers record data in a standardised and structured way at the point of care. Currently, it is not clear to what extent this structured and standardised recording has been adopted by healthcare professionals and what barriers to their adoption exist. Therefore, we developed and validated a multivariable model to capture the concepts underlying the adoption of structured and standardised recording among healthcare professionals.

Methods

Based on separate models from the literature we developed a new theoretical model describing the underlying concepts of the adoption of structured and standardised recording. Using a questionnaire built upon this model we gathered data to perform a summative validation of our model. Validation was done through partial least squares structural equation modelling (PLS-SEM). The quality of both levels defined in PLS-SEM analysis, i.e., the measurement model and the structural model, were assessed on performance measures defined in literature.

Results

The theoretical model we developed consists of 29 concepts related to information systems as well as organisational factors and personal beliefs. Based on these concepts, 59 statements with a 5 point Likert-scale (fully disagree to fully agree) were specified in the questionnaire. We received 3584 responses. The validation shows our model is supported to a large extent by the questionnaire data. Intention to record in a structured and standardised way emerged as a significant factor of reported behaviour (β = 0.305, p < 0.001). This intention is influenced most by attitude (β = 0.512, p < 0.001).

Conclusions

This model can be used to measure the perceived level of adoption of structured and standardised recording among healthcare professionals and further improve knowledge on the barriers and facilitators of this adoption.

Electronic supplementary material

The online version of this article (10.1186/s12911-018-0640-8) contains supplementary material, which is available to authorized users.

Evaluation of the unified model of information systems continuance (UMISC) in two hospital environments

CONTEXT

The deployment and long-term acceptance of clinical information systems (CISs) are faced with multiple difficulties. They include insufficient quality of the systems in place and resistance to the multiple changes they induce in care processes. Permanent evaluation of deployed solutions is a prerequisite to their continuous improvement.

OBJECTIVE

The purpose of this study was twofold: (1) To validate the post-adoption unified model of information systems continuance (UMISC) progressively developed at the Georges Pompidou University Hospital (HEGP) in Paris (internal validation); and (2) To compare, using the same evaluation model, the results observed at HEGP with those of the Saint-Joseph Hospital Group (HPSJ), another Paris acute care institution (external validation).

METHODS

The UMISC post-adoption model is built around nine dimensions: end-user characteristics, social norm (SN), IS quality (ISQ), facilitating conditions (FC), perceived usefulness (PU), confirmation of expectations (CE), profession-adjusted use (PAU), satisfaction (SAT), and continuance intention (CI). Two semi-quantitative evaluation surveys were performed at HEGP in 2014 and 2015, and one at HPSJ in 2015. Statistical analysis included multiple regression analysis and structural equation modeling (SEM).

RESULTS

The analysis concerned 459 responders, 264 at HEGP and 195 at HPSJ. UMISC indicators, with the exception of SN, are superior at HEGP than at HPSJ, which had a shorter CIS anteriority than HEGP. In SEM analysis, the UMISC model explained 25% and 40% of the CIS use, 92% and 93% of health professionals' satisfaction, and 72% and 71% of continuance intention at HEGP and HPSJ, respectively. Seventeen of the 21 tested UMISC hypotheses were supported in at least one of the two sites.

CONCLUSION

The UMISC evaluation model can be used as a comparison and explanatory model of CIS use, satisfaction and continuance intention in post-CIS adoption situations that become prevalent in current electronic hospitals.

Developing a theoretical model and questionnaire survey instrument to measure the success of electronic health records in residential aged care

Electronic health records (EHR) are introduced into healthcare organizations worldwide to improve patient safety, healthcare quality and efficiency. A rigorous evaluation of this technology is important to reduce potential negative effects on patient and staff, to provide decision makers with accurate information for system improvement and to ensure return on investment. Therefore, this study develops a theoretical model and questionnaire survey instrument to assess the success of organizational EHR in routine use from the viewpoint of nursing staff in residential aged care homes. The proposed research model incorporates six variables in the reformulated DeLone and McLean information systems success model: system quality, information quality, service quality, use, user satisfaction and net benefits. Two variables training and self-efficacy were also incorporated into the model. A questionnaire survey instrument was designed to measure the eight variables in the model. After a pilot test, the measurement scale was used to collect data from 243 nursing staff members in 10 residential aged care homes belonging to three management groups in Australia. Partial least squares path modeling was conducted to validate the model. The validated EHR systems success model predicts the impact of the four antecedent variables—training, self-efficacy, system quality and information quality—on the net benefits, the indicator of EHR systems success, through the intermittent variables use and user satisfaction. A 24-item measurement scale was developed to quantitatively evaluate the performance of an EHR system. The parsimonious EHR systems success model and the measurement scale can be used to benchmark EHR systems success across organizations and units and over time.

Development of an Evaluation Framework for Health Information Systems (DIPSA)

Introduction:

Use of Integrated Health Information Systems (IHIS) for the provision of healthcare services benefits both healthcare professionals and patients, while requiring continuous evaluation and upgrading to fully support its role.

Aim:

The main purpose of the study was to develop an evaluation framework for hospitals utilizing IHIS, within the three main areas identified as Human factor, Technology and Organization.

Material and methods:

The questionnaire consisted of 43 questions, with 17 questions (related to categories procedures, system quality and satisfaction), 25 questions (related to categories, safety and collaboration) and 1 question related to accessibility to the system (within the category system quality). Three open questions were added to evaluate users’ perception on what was needed for the improvement of health services in their respective hospitals for all 3 variables being evaluated. The open questions were included to allow participants to express their opinion in a more detailed setting. A database was developed, and the data were processed and analyzed.

Results:

Factor analysis formed 5 categories for the evaluation framework. Cronbach’s alpha coefficient was found in all categories to be above > 0.85.

Conclusion:

Evaluation frameworks can be designed, developed and implemented by using different methodologies. For an evaluation framework to be effective it should be designed and implemented based on the aims and purpose of the research and the specific needs of the particular healthcare setting or hospital. Considering the categories satisfaction, collaboration, safety, system quality, procedures, and by using Likert scale and open questions in the current study, DIPSA can provide a holistic image of IHIS by evaluating any hospital system.

Continued use of an interactive computer game-based visual perception learning system in children with developmental delay

This study developed an interactive computer game-based visual perception learning system for special education children with developmental delay. To investigate whether perceived interactivity affects continued use of the system, this study developed a theoretical model of the process in which learners decide whether to continue using an interactive computer game-based visual perception learning system. The technology acceptance model, which considers perceived ease of use, perceived usefulness, and perceived playfulness, was extended by integrating perceived interaction (i.e., learner-instructor interaction and learner-system interaction) and then analyzing the effects of these perceptions on satisfaction and continued use. Data were collected from 150 participants (rehabilitation therapists, medical paraprofessionals, and parents of children with developmental delay) recruited from a single medical center in Taiwan. Structural equation modeling and partial-least-squares techniques were used to evaluate relationships within the model. The modeling results indicated that both perceived ease of use and perceived usefulness were positively associated with both learner-instructor interaction and learner-system interaction. However, perceived playfulness only had a positive association with learner-system interaction and not with learner-instructor interaction. Moreover, satisfaction was positively affected by perceived ease of use, perceived usefulness, and perceived playfulness. Thus, satisfaction positively affects continued use of the system. The data obtained by this study can be applied by researchers, designers of computer game-based learning systems, special education workers, and medical professionals.

Health information systems evaluation frameworks: A systematic review

BACKGROUND

Evaluation of health information systems (HISs) is complicated because of the complex nature of the health care domain. Various studies have proposed different frameworks to reduce the complexity in the assessment of these systems. The aim of these frameworks is to provide a set of guidelines for the evaluation of the adequacy of health care information systems.

OBJECTIVE

This paper aims to analyse studies on the evaluation of HISs by applying a content, context and process (CCP) framework to address the 'who', 'what', 'how', 'when', and 'why' of the evaluation processes used. This will allow for a better understanding of the relative strengths and weaknesses of various HISs evaluation frameworks, and will pave the way for developing a more complete framework for HISs.

METHOD

A systematic literature review on HIS evaluation studies was undertaken to identify the currently available HIS evaluation frameworks. Five academic databases were selected to conduct this systematic literature review.

RESULTS

Most of the studies only address some, but not all, of the five main questions, i.e. the who, what, how, when, why, and that there was a lack of consensus in the way these questions were addressed. The critical role of context was also largely neglected in these studies.

CONCLUSIONS

Evaluation of HISs is complex. The health care domain is highly context sensitive and in order to have a complete assessment of HISs, consideration of contextual factors is necessary. Specifically, to have the right set of criteria to measure the 'what', the answer to the 'who' of the evaluation is necessary.

Organizational Capabilities for Integrating Care: A Review of Measurement Tools

The success of integrated care interventions is highly dependent on the internal and collective capabilities of the organizations in which they are implemented. Yet, organizational capabilities are rarely described, understood, or measured with sufficient depth and breadth in empirical studies or in practice. Assessing these capabilities can contribute to understanding why some integrated care interventions are more effective than others. We identified, organized, and assessed survey instruments that measure the internal and collective organizational capabilities required for integrated care delivery. We conducted an expert consultation and searched Medline and Google Scholar databases for survey instruments measuring factors outlined in the Context and Capabilities for Integrating Care Framework. A total of 58 instruments were included in the review and assessed based on their psychometric properties, practical considerations, and applicability to integrated care efforts. This study provides a bank of psychometrically sound instruments for describing and comparing organizational capabilities. Greater use of these instruments across integrated care interventions and studies can enhance standardized comparative analyses and inform change management. Further research is needed to build an evidence base for these instruments and to explore the associations between organizational capabilities and integrated care processes and outcomes.

How to overcome the digital divide? An empirical study of Taiwan’s DOCs

Purpose

The digital divide is a concern, as the inequality of information access might have significant influences on social development and quality of life. The purpose of this paper is to examine the perceived benefit of Digital Opportunity Centers (DOCs) programs on remote area participants from the perspective of computer anxiety and personal information ability.

Design/methodology/approach

The Taiwanese Government has built DOCs in remote areas to provide information technology (IT) training and learning programs to citizens residing in these areas. DOC program participants in Taiwan voluntarily completed a self-report questionnaire; the authors received 2,105 completed questionnaires, with a response rate of 84.2 percent. This research used partial least-squares (PLS) to empirical the research model.

Findings

Using PLS, the results show that information and communication technology ability influences the perceived benefit of DOC programs; computer anxiety has significantly negative effects on package software use, internet use, and IT usefulness; and internet use and IT usefulness have positive effects on perceived benefits.

Originality/value

IT is continuously advancing, but digital resources are still lacking within remote areas. DOCs provide citizens different types of learning experiences related to economic, social, and educational development. DOC programs provide participants with opportunities to obtain and improve basic IT knowledge and abilities and decreasing the digital divide.

Information system end-user satisfaction and continuance intention: A unified modeling approach

CONTEXT

Permanent evaluation of end-user satisfaction and continuance intention is a critical issue at each phase of a clinical information system (CIS) project, but most validation studies are concerned with the pre- or early post-adoption phases.

OBJECTIVE

The purpose of this study was twofold: to validate at the Pompidou University Hospital (HEGP) an information technology late post-adoption model built from four validated models and to propose a unified metamodel of evaluation that could be adapted to each context or deployment phase of a CIS project.

METHODS

Five dimensions, i.e., CIS quality (CISQ), perceived usefulness (PU), confirmation of expectations (CE), user satisfaction (SAT), and continuance intention (CI) were selected to constitute the CI evaluation model. The validity of the model was tested using the combined answers to four surveys performed between 2011 and 2015, i.e., more than ten years after the opening of HEGP in July 2000. Structural equation modeling was used to test the eight model-associated hypotheses.

RESULTS

The multi-professional study group of 571 responders consisted of 158 doctors, 282 nurses, and 131 secretaries. The evaluation model accounted for 84% of variance of satisfaction and 53% of CI variance for the period 2011-2015 and for 92% and 69% for the period 2014-2015. In very late post adoption, CISQ appears to be the major determinant of satisfaction and CI. Combining the results obtained at various phases of CIS deployment, a Unified Model of Information System Continuance (UMISC) is proposed.

CONCLUSION

In a meaningful CIS use situation at HEGP, this study confirms the importance of CISQ in explaining satisfaction and CI. The proposed UMISC model that can be adapted to each phase of CIS deployment could facilitate the necessary efforts of permanent CIS acceptance and continuance evaluation.

Conceptual Models in Health Informatics Research: A Literature Review and Suggestions for Development

BACKGROUND

Contributing to health informatics research means using conceptual models that are integrative and explain the research in terms of the two broad domains of health science and information science. However, it can be hard for novice health informatics researchers to find exemplars and guidelines in working with integrative conceptual models.

OBJECTIVES

The aim of this paper is to support the use of integrative conceptual models in research on information and communication technologies in the health sector, and to encourage discussion of these conceptual models in scholarly forums.

METHODS

A two-part method was used to summarize and structure ideas about how to work effectively with conceptual models in health informatics research that included (1) a selective review and summary of the literature of conceptual models; and (2) the construction of a step-by-step approach to developing a conceptual model.

RESULTS

The seven-step methodology for developing conceptual models in health informatics research explained in this paper involves (1) acknowledging the limitations of health science and information science conceptual models; (2) giving a rationale for one's choice of integrative conceptual model; (3) explicating a conceptual model verbally and graphically; (4) seeking feedback about the conceptual model from stakeholders in both the health science and information science domains; (5) aligning a conceptual model with an appropriate research plan; (6) adapting a conceptual model in response to new knowledge over time; and (7) disseminating conceptual models in scholarly and scientific forums.

CONCLUSIONS

Making explicit the conceptual model that underpins a health informatics research project can contribute to increasing the number of well-formed and strongly grounded health informatics research projects. This explication has distinct benefits for researchers in training, research teams, and researchers and practitioners in information, health, and other disciplines.

14 Years longitudinal evaluation of clinical information systems acceptance: The HEGP case

CONTEXT

Meaningful use and end-user satisfaction are two major components of the success of a clinical information system (CIS). The purpose of this study was to longitudinally measure and analyze the CIS use and satisfaction determinants in a multi-professional group at the Georges Pompidou university hospital (HEGP) in Paris.

METHODS

From the different evaluation surveys performed at HEGP, three periods were considered corresponding to 4, 8 and over 10 years after the first CIS deployment in 2000, respectively. Six acceptance dimensions were considered: CIS quality (CISQ), facilitating conditions (FC), perceived usefulness (PU), confirmation of expectations (CE), use, and global satisfaction (GS). Relationships between these constructs were tested through multiple regressions analysis and structural equation modeling (SEM).

RESULTS

Responses were obtained from 298, 332, and 448 users for the three periods considered. CIS acceptance dimensions progressively and significantly increased over time. Significant differences between professions were observed with an initial low PU among medical staff. In the early deployment phase, GS appeared to be determined by CIS use, CISQ and PU (R(2)=.53 in SEM). In the very late post-adoption phase, GS was strongly determined by CISQ, CE, and PU (R(2)=.86 in SEM) and was no longer associated with CIS use.

CONCLUSION

Acceptance models should be adapted to the phase of deployment of a CIS and integrate end-users' individual characteristics. Progressive reduction over time of the positive relationships between CIS use and satisfaction could possibly be considered as a maturity indicator of CIS deployment. These observations validate the introduction in post-adoption models of a continuance intention dimension.

Development and evaluation of the mobile library service system success model

Purpose

– The purpose of this study is to develop and analyse the usage status, level of satisfaction and success model of a mobile library service system in Taiwan. The research results could serve as a reference for the development, evaluation and improvement of libraries’ mobile services.

Design/methodology/approach

– This study designed a mobile library service system and used the informational system success model to evaluate its use status and level of satisfaction and, thus, to develop a success model for a mobile library service system.

Findings

– Results showed that students’ level of satisfaction with the mobile library service system had a positive correlation with the usage status indicated that the system could help them to improve their work efficiency, such as by reducing time spent searching for books and cost of obtaining electronic resources. The students also expressed a willingness to continue using the mobile library service system.

Practical implications

– In general, the students thought that the mobile library service system could reduce the time needed to search for book-related information, but that the service quality should be improved. Therefore, this study suggests that the education and training of system service personnel should be enhanced and that easy-to-use functions be provided to promote the users’ level of satisfaction and willingness to use the system.

Originality/value

– This study developed a mobile library service system and invited students studying in the National University of Tainan, in southern Taiwan, to be the subjects in an experiment, with a goal to evaluate the success of a model mobile library. The findings showed that the overall explanatory power (R2) of this model was 55 per cent and, thus, that the findings of this work have practical significance.

Modeling antecedents of electronic medical record system implementation success in low-resource setting hospitals

BACKGROUND

With the increasing implementation of Electronic Medical Record Systems (EMR) in developing countries, there is a growing need to identify antecedents of EMR success to measure and predict the level of adoption before costly implementation. However, less evidence is available about EMR success in the context of low-resource setting implementations. Therefore, this study aims to fill this gap by examining the constructs and relationships of the widely used DeLone and MacLean (D&M) information system success model to determine whether it can be applied to measure EMR success in those settings.

METHODS

A quantitative cross sectional study design using self-administered questionnaires was used to collect data from 384 health professionals working in five governmental hospitals in Ethiopia. The hospitals use a comprehensive EMR system since three years. Descriptive and structural equation modeling methods were applied to describe and validate the extent of relationship of constructs and mediating effects.

RESULTS

The findings of the structural equation modeling shows that system quality has significant influence on EMR use (β = 0.32, P < 0.05) and user satisfaction (β = 0.53, P < 0.01); information quality has significant influence on EMR use (β = 0.44, P < 0.05) and user satisfaction (β = 0.48, P < 0.01) and service quality has strong significant influence on EMR use (β = 0.36, P < 0.05) and user satisfaction (β = 0.56, P < 0.01). User satisfaction has significant influence on EMR use (β = 0.41, P < 0.05) but the effect of EMR use on user satisfaction was not significant. Both EMR use and user satisfaction have significant influence on perceived net-benefit (β = 0.31, P < 0.01; β = 0.60, P < 0.01), respectively. Additionally, computer literacy was found to be a mediating factor in the relationship between service quality and EMR use (P < 0.05) as well as user satisfaction (P < 0.01). Among all the constructs, user satisfaction showed the strongest effect on perceived net-benefit of health professionals.

CONCLUSION

EMR implementers and managers in developing countries are in urgent need of implementation models to design proper implementation strategies. In this study, the constructs and relationships depicted in the updated D&M model were found to be applicable to assess the success of EMR in low resource settings. Additionally, computer literacy was found to be a mediating factor in EMR use and user satisfaction of health professionals. Hence, EMR implementers and managers in those settings should give priority in improving service quality of the hospitals like technical support and infrastructure; providing continuous basic computer trainings to health professionals; and give attention to the system and information quality of the systems they want to implement.

Comprehensive evaluation of electronic medical record system use and user satisfaction at five low-resource setting hospitals in ethiopia

Background

Electronic medical record (EMR) systems are increasingly being implemented in hospitals of developing countries to improve patient care and clinical service. However, only limited evaluation studies are available concerning the level of adoption and determinant factors of success in those settings.

Objective

The objective of this study was to assess the usage pattern, user satisfaction level, and determinants of health professional’s satisfaction towards a comprehensive EMR system implemented in Ethiopia where parallel documentation using the EMR and the paper-based medical records is in practice.

Methods

A quantitative, cross-sectional study design was used to assess the usage pattern, user satisfaction level, and determinant factors of an EMR system implemented in Ethiopia based on the DeLone and McLean model of information system success. Descriptive statistical methods were applied to analyze the data and a binary logistic regression model was used to identify determinant factors.

Results

Health professionals (N=422) from five hospitals were approached and 406 responded to the survey (96.2% response rate). Out of the respondents, 76.1% (309/406) started to use the system immediately after implementation and user training, but only 31.7% (98/309) of the professionals reported using the EMR during the study (after 3 years of implementation). Of the 12 core EMR functions, 3 were never used by most respondents, and they were also unaware of 4 of the core EMR functions. It was found that 61.4% (190/309) of the health professionals reported over all dissatisfaction with the EMR (median=4, interquartile range (IQR)=1) on a 5-level Likert scale. Physicians were more dissatisfied (median=5, IQR=1) when compared to nurses (median=4, IQR=1) and the health management information system (HMIS) staff (median=2, IQR=1). Of all the participants, 64.4% (199/309) believed that the EMR had no positive impact on the quality of care. The participants indicated an agreement with the system and information quality (median=2, IQR=0.5) but strongly disagreed with the service quality (median=5, IQR=1). The logistic regression showed a strong correlation between system use and dissatisfaction (OR 7.99, 95% CI 5.62-9.10) and service quality and satisfaction (OR 8.23, 95% CI 3.23-17.01).

Conclusions

Health professionals’ use of the EMR is low and they are generally dissatisfied with the service of the implemented system. The results of this study show that this dissatisfaction is caused mainly and strongly by the poor service quality, the current practice of double documentation (EMR and paper-based), and partial departmental use of the system in the hospitals. Thus, future interventions to improve the current use or future deployment projects should focus on improving the service quality such as power infrastructure, user support, trainings, and more computers in the wards. After service quality improvement, other departments (especially inter-dependent departments) should be motivated and supported to use the EMR to avoid the dependency deadlock.

Privacy preserving index for encrypted electronic medical records

With the development of electronic systems, privacy has become an important security issue in real-life. In medical systems, privacy of patients' electronic medical records (EMRs) must be fully protected. However, to combine the efficiency and privacy, privacy preserving index is introduced to preserve the privacy, where the EMR can be efficiently accessed by this patient or specific doctor. In the literature, Goh first proposed a secure index scheme with keyword search over encrypted data based on a well-known primitive, Bloom filter. In this paper, we propose a new privacy preserving index scheme, called position index (P-index), with keyword search over the encrypted data. The proposed index scheme is semantically secure against the adaptive chosen keyword attack, and it also provides flexible space, lower false positive rate, and search privacy. Moreover, it does not rely on pairing, a complicate computation, and thus can search over encrypted electronic medical records from the cloud server efficiently.