Evaluating the success of an emergency response medical information system

Predictive Factors of Physicians' Satisfaction and Quality of Work Under Teleconsultation Conditions: Structural Equation Analysis

BACKGROUND

The COVID-19 pandemic contributed to an increase in teleconsultation adoption in the Polish primary health care system. It is expected that in the long run, teleconsultations will successfully replace a significant part of face-to-face visits. Therefore, a significant challenge facing primary health care facilities (PHCs) is the acceptance of teleconsultations by their users, especially physicians.

OBJECTIVE

This study aimed to explore physicians' acceptance of teleconsultations during the COVID-19 pandemic in Poland.

METHODS

A representative survey was conducted among 361 physicians of PHCs across Poland in 2021. For the purposes of the study, we developed a modified Technology Acceptance Model (TAM) model. Based on the modified TAM, we analyzed the impact of perceived usefulness (PU), perceived ease of use (PEU), and intention to use teleconsultation (INT) on physicians' satisfaction (SAT) and quality of work (Q). The psychometric properties of the research instrument were examined using exploratory factor analysis. Finally, structural equation modeling was used for data analysis.

RESULTS

The results indicated a generally high level of PU (mean 3.85-4.36, SD 0.87-1.18), PEU (mean 3.81-4.60, SD 0.60-1.42), INT (mean 3.87-4.22, SD 0.89-1.12), and SAT (mean 3.55-4.13, SD 0.88-1.16); the lowest rated dimension in TAM was Q (mean 3.28-3.73, SD 1.06-1.26). The most important independent variable was PU. The influence of PU on INT (estimate=0.63, critical ratio [CR]=15.84, P<.001) and of PU on SAT (estimate=0.44, CR= 9.53, P<.001) was strong. INT was also a key factor influencing SAT (estimate=0.4, CR=8.57, P<.001). A weaker relationship was noted in the effect of PEU on INT (estimate=0.17, CR=4.31, P<.001). In turn, Q was positively influenced by INT (estimate=0.179, CR=3.64, P<.001), PU (estimate=0.246, CR=4.79, P<.001), PEU (estimate=0.18, CR=4.93, P<.001), and SAT (estimate=0.357, CR=6.97, P<.001). All paths between the constructs (PU, PEU, INT, SAT, and Q) were statistically significant, which highlights the multifaceted nature of the adoption of teleconsultations among physicians.

CONCLUSIONS

Our findings provide strong empirical support for the hypothesized relationships in TAM. The findings suggest that the PU and PEU of teleconsultation have a significant impact on the intention of physicians to adopt teleconsultation. This results in an improvement in the satisfaction of Polish physicians with the use of teleconsultation and an increase in Q. The study contributes to both theory and practice by identifying important prognostic factors affecting physicians' acceptance of teleconsultation systems.

Improve the satisfaction of medical staff on the use of home nursing mobile APP by using a hybrid multi-standard decision model

OBJECTIVE

To identify critical satisfaction gaps in a home nursing mobile application (APP) using a systematic decision-making model.

METHODS

Initially, the decision-making trial and evaluation laboratory method was used to analyze the relationship structure and corresponding weights among the indicators. The Importance-Performance Analysis (IPA) method was used to identify the categories of all indicators and their corresponding strategic directions. Twenty-six home nursing specialists currently providing home nursing services were recruited for this study.

RESULTS

The IPA results revealed that "Assurance," "Reliability," and "Personal security protection" are critical satisfaction gaps. From the influence network and weight results, "information quality" and "system quality" were the critical quality factors in the home nursing mobile APP. The influence of the network relationship structure and weight demonstrated a 98.12% significance level, indicating good stability.

CONCLUSION

Continuous improvement in information and system quality is recommended to optimize the overall quality of the home nursing mobile APP. Additionally, user demands should be considered, and personal safety guarantee functions should be developed and integrated into the system to ensure the safety of home nursing workers.

Designing a doctor evaluation index system for an online medical platform based on the information system success model in China

Objective

In the context of "internet + medical health" and emphasis on evaluation mechanism for medical and health talents in China, we design an evaluation index system for doctors on online medical platforms by synthesizing two patterns of existing online medical platforms, which is the first step to enhance the capabilities of doctors on online medical platforms.

Methods

Based on the doctor evaluation model integrating information systems success model (ISS-DE model) and grounded theory, the evaluation indicators were obtained through expert interviews, offline medical institutions investigation, online platforms investigation, and literature research, and were assigned weights using the analytic hierarchy process (AHP) method. A working group composed of 23 experts was set up to review and determine the competency standards of doctors on the online medical platforms.

Results

A new indicator framework covering 3 dimensions of system quality, service quality and information quality was constructed in this study. The index system included 3 first-level indicators, 8 s-level indicators and 60 third-level indicators, and each indicator was given different weightage.

Conclusion

The complete index system constructed by the Delphi method in this study is suitable for China's online medical platforms, which will help to improve the quality of platforms and the ability of doctors, thus promoting the process of internet medical integration.

Factors Influencing Clinicians' Use of Hospital Information Systems for Infection Prevention and Control: Cross-Sectional Study Based on the Extended DeLone and McLean Model

BACKGROUND

Healthcare-associated infections have become a serious public health problem. Various types of information systems have begun to be applied in hospital infection prevention and control (IPC) practice. Clinicians are the key users of these systems, but few studies have assessed the use of infection prevention and control information systems (IPCISs) from their perspective.

OBJECTIVE

This study aimed to (1) apply the extended DeLone and McLean Information Systems Success model (D&M model) that incorporates IPC culture to examine how technical factors like information quality, system quality, and service quality, as well as organizational culture factors affect clinicians' use intention, satisfaction, and perceived net benefits, and (2) identify which factors are the most important for clinicians' use intention.

METHODS

A total of 12,317 clinicians from secondary and tertiary hospitals were surveyed online. Data were analyzed using partial least squares-structural equation modeling and the importance-performance matrix analysis.

RESULTS

Among the technical factors, system quality (β=.089-.252; P<.001), information quality (β=.294-.102; P<.001), and service quality (β=.126-.411; P<.001) were significantly related to user satisfaction (R2=0.833), use intention (R2=0.821), and perceived net benefits (communication benefits [R2=0.676], decision-making benefits [R2=0.624], and organizational benefits [R2=0.656]). IPC culture had an effect on use intention (β=.059; P<.001), and it also indirectly affected perceived net benefits (β=.461-.474; P<.001). In the importance-performance matrix analysis, the attributes of service quality (providing user training) and information quality (readability) were present in the fourth quadrant, indicating their high importance and low performance.

CONCLUSIONS

This study provides valuable insights into IPCIS usage among clinicians from the perspectives of technology and organization culture factors. It found that technical factors (system quality, information quality, and service quality) and hospital IPC culture have an impact on the successful use of IPCISs after evaluating the application of IPCISs based on the extended D&M model. Furthermore, service quality and information quality showed higher importance and lower performance for use intention. These findings provide empirical evidence and specific practical directions for further improving the construction of IPCISs.

A scale to measure the perceived quality of mHealth by elderly patients with hypertension in China

BACKGROUND

The elderly population in China is growing, with hypertension being the most prevalent chronic disease in older adults. Despite the rapid penetration and efficient management effect of mobile health on hypertension healthcare, elderly patients are often less adopted and continue to use mobile health services. Quality perception significantly affects an individual's satisfaction and continued intention to use mobile health services. The evaluation of the significant factors affecting mobile health quality perception by elderly individuals remains largely unexplored. The aim of this study was to develop and validate an evaluation scale to measure the perceived quality of mobile health applications for hypertension and determine the underlying influencing factors.

METHODS

A cross-sectional survey was conducted between November 2018 and October 2019. A quality evaluation scale with three factors and seven indicators was developed based on the Information Systems Success model. Data was analyzed using structural equations modelling (SEM) and one-way analysis of variance (ANOVA). All tests were two-sided and statistically significant at P < 0.05.

RESULTS

The proposed mobile health application quality evaluation scale from the perspective of the elderly was shown to be a hierarchical, multidimensional construct with valid reliability, convergent validity and discriminant validity, which consists of three factors and seven indicators. The SEM results suggested that information quality and service quality had a significant impact on the satisfaction of elderly individual's with mobile health applications for hypertension management. The results also suggest that the elderly individuals had a low evaluation of mobile medical service quality(4.06 ± 0.70), while the score of information quality was the highest, with an average score of 4.36(SD 0.83) out of 5. Male patients were shown to more readily accept mobile health applications, with their perception of system quality being 0.27 higher than female ones (95% CI 0.00 ~ 0.52; P < .05). Patients with 1-5 years hypertension histories assessed the system (95% CI 0.03 ~ 0.63; P < .05), information (95% CI 0.11 ~ 0.65; P < .05), and service quality (95% CI 0.00 ~ 0.47; P < .05) higher than those with hypertension histories > 10 years. Elderly patients who regularly visited primary hospitals assessed the information quality 0.13 higher (95% CI -0.08 ~ 0.34; P < .05) than those visited tertiary hospitals.

CONCLUSIONS

These findings have significant implications for theoretical and practical research on mobile health application quality evaluation, which will be helpful for policymakers and mobile health providers in improving the context and utilisation of mobile health to include elderly users. More mobile health applications attributes, such as timely information and interactive services that meet the characteristics of elderly patients with different mental and health demands need to be considered. Deeply embedding mobile health into primary health services is recommended to help increase the perceived quality of mobile health, and ensure the continuous use.

An empirical assessment of enterprise information systems success in a developing country: the Jordanian experience

Purpose

The paper discusses the assessment of enterprise information systems (EIS) success in an organizational context is one area that is not sufficiently researched. This lack of theoretical attention prompted the authors to measure EIS success in the organizational setting of Jordanian listed firms.

Design/methodology/approach

Based on the DeLone and McLean success model and by adopting a quantitative approach, survey data were collected through questionnaires. The established questionnaires were distributed to 250 senior IT executives who use EIS and a total of 134 valid questionnaires were empirically tested via the Smart-PLS technique.

Findings

The investigation findings indicate that information and system quality had a positive impact on user satisfaction and individual impact. User satisfaction had a positive influence on individual impact, both of which eventually lead to organizational impact. The results also showed that the model used has adequate convergent and discriminant validities, as well as sufficient reliability.

Practical implications

The outcomes can help managers and practitioners more effectively understand the factors influencing EIS success among Jordanian listed firms.

Originality/value

This research paper is the very first in assessing EIS success from an organizational perspective in developing country such as Jordan.

Assessing the effectiveness of accounting information systems in the era of COVID-19 pandemic

Purpose

This study aims to evaluate the Accounting Information Systems (AIS) effectiveness on the organizational level by extending DeLone and McLean success model in developing countries like Jordan. Unlike other studies, this study investigates the effects of system quality, information quality, process quality, collaboration quality, service quality, individual performance and workgroup performance toward organizational performance during the period of the COVID-19 pandemic outbreak.

Design/methodology/approach

A self-administrated survey was used to collect data from 104 accounting managers operating in listed Jordanian firms in the Amman Stock Exchange (ASE). The partial least squares-structural equation modeling method was used for data analysis.

Findings

Eight relationships were tested, whereby six were supported as was expected. Particularly, the empirical results reveal that individual performance is positively and significantly influenced by information quality, process quality and service quality. Furthermore, workgroup performance is positively and significantly influenced by individual performance. The results also show that organizational performance is impacted by individual performance and workgroup performance.

Originality/value

The current study is among the first empirical studies that has extended the information systems success model to evaluate AIS success during COVID-19 pandemic. This study also offers insights into the importance of AIS success among listed Jordanian firms in ASE.

Factors Influencing User Satisfaction with Information Systems: A Systematic Review

User satisfaction has been considered as the measure of information system effectiveness success. User satisfaction is difficult to define but is considered an evaluation construct. Globally health organizations, particularly hospitals, invest a huge amount of money on information system projects. If hospital information systems (HISs) are to be successful, factors influencing or related to user satisfaction should be taken into account at the time of designing, developing or adopting such systems. The current study aimed to provide a comprehensive review of factors related to user satisfaction with information systems. The researchers systematically searched PubMed, Science Direct, and IEEE electronic databases for articles published from January 1990 to June 2016. A search strategy was developed using a combination of the following keywords: "model," "user satisfaction," "information system," "measurement," "instrument," and " tool." Reported dimensions, factors, and their possible influence on user satisfaction with information systems were extracted from the studies wherever was possible. Overall factors influencing user satisfaction with information systems can be categorized in seven dimensions: Information quality, system quality, vendor support quality, system use, perceived usefulness, user characteristics, and organizational structure & management style. If all these factors are considered properly in the process of developing, designing, implementing, or purchasing information systems, the higher user satisfaction with the system will be likely. Otherwise, it would end up with unsatisfied users that will finally contribute to the system failure.

Performance Evaluation of an Information Technology Intervention Regarding Charging for Inpatient Medical Materials at a Regional Teaching Hospital in Taiwan: Empirical Study

Background

The process of manually recording the consumption of medical materials can be time consuming and prone to omission owing to its detailed and complicated nature. Implementing an information system will better improve work performance.

Objective

The Information System Success Model was adopted as the theoretical foundation. The opinions of nursing staff were collected to verify the impact of the system intervention on their work performance.

Methods

This cross-sectional study was conducted at a regional teaching hospital. Nursing staff were invited to participate in the field survey. A total of 296 questionnaires were collected, and of these, 284 (95.9%) were valid and returned.

Results

The key findings showed that two critical factors (“subjective norm” and “system quality”) had significant positive effects (both P<.001) on user satisfaction (R²=0.709). The path of “service quality” to “user satisfaction” showed marginal significance (P=.08) under the 92% CI. Finally, the explanatory power of the model reached 68.9%.

Conclusions

Support from the top management, appointment of a nurse supervisor as the change agent, recruitment of seed members to establish a pioneer team, and promotion of the system through the influence of opinion leaders in small groups were critical success factors needed for implementing the system in the case hospital. The target system was proven to be able to improve work performance, and the time saved could be further used for patient care, thereby increasing the value of nursing work. The positive experiences gained from this study could lay the foundation for the further promotion of the new system, and this is for future studies to replicate. The example of the successful experience of the case hospital could also serve as a reference for other hospitals in developing countries like Taiwan with regard to the promotion of nursing informatization.

Factors Influencing the Adoption of Online Health Consultation Services: The Role of Subjective Norm, Trust, Perceived Benefit, and Offline Habit

The cyberspace plays an important role in improving the quality, equity, and efficiency of health services. Studying people's adoption of online health services, such as online health consultation services (OHCS) can benefit both industry and policy in the health service sector. This paper investigates influencing factors and paths of people's intention of adopting OHCS by employing the extended valence framework, with our new contribution of integrating subjective norm and offline habit into the model. Five hundred forty-three university students participated in the survey. Structural equation models and Sobel-Goodman tests were applied to test the models. The results show that subjective norm (β = 0.077, p = 0.041), trust in providers (β = 0.194, p = 0.002) and perceived benefit (β = 0.463, p < 0.001) positively affect the intention to adopt OHCS, while offline habit (β = -0.111, p = 0.026) has a negative effect. However, the association of perceived risk (β = -0.062, p = 0.315) and adoption is not supported. Moreover, trust in providers plays a mediating role between subjective norm and the intention of adopting, while perceived benefit mediates the relationship between trust in providers and the intention of adopting. This study highlights the importance of trust, subjective norm, perceived benefit, and persisting habits in promoting the adoption of OHCS.

Developing a theoretical model and questionnaire survey instrument to measure the success of electronic health records in residential aged care

Electronic health records (EHR) are introduced into healthcare organizations worldwide to improve patient safety, healthcare quality and efficiency. A rigorous evaluation of this technology is important to reduce potential negative effects on patient and staff, to provide decision makers with accurate information for system improvement and to ensure return on investment. Therefore, this study develops a theoretical model and questionnaire survey instrument to assess the success of organizational EHR in routine use from the viewpoint of nursing staff in residential aged care homes. The proposed research model incorporates six variables in the reformulated DeLone and McLean information systems success model: system quality, information quality, service quality, use, user satisfaction and net benefits. Two variables training and self-efficacy were also incorporated into the model. A questionnaire survey instrument was designed to measure the eight variables in the model. After a pilot test, the measurement scale was used to collect data from 243 nursing staff members in 10 residential aged care homes belonging to three management groups in Australia. Partial least squares path modeling was conducted to validate the model. The validated EHR systems success model predicts the impact of the four antecedent variables—training, self-efficacy, system quality and information quality—on the net benefits, the indicator of EHR systems success, through the intermittent variables use and user satisfaction. A 24-item measurement scale was developed to quantitatively evaluate the performance of an EHR system. The parsimonious EHR systems success model and the measurement scale can be used to benchmark EHR systems success across organizations and units and over time.

Changes in User Perceptions of a Telemedicine System Over Time: From Initial Implementation to Everyday Use

BACKGROUND

Benefits associated with telemedicine are contingent upon positive user perceptions. Despite this, research on user perceptions of telemedicine remains limited.

INTRODUCTION

Usability approaches offer a robust way to assess user perceptions, but have rarely been applied in telemedicine. In this study, a usability approach was employed to examine how user perceptions toward a telemedicine system changed over the course of everyday use.

MATERIALS AND METHODS

A telemedicine system was introduced to a hospital in the home service. Ten mobile nurses completed the System Usability Scale (SUS) after initial use, then again after 18 months of everyday use. Results were compared. Analysis included Bangor et al.'s (2009) adjective rating scale.

RESULTS

The initial SUS mean was 83 (standard deviation [SD] = 7.98), indicating "excellent" usability. After 18 months, the SUS mean was 64.38, indicating "OK" usability (SD = 14.25, p < 0.05, 95% confidence interval [CI]). Over time, users had lower desire to use the system frequently (p < 0.05, 95% CI), found it more complex (p < 0.05, 95% CI), and perceived greater inconsistency in its design (p < 0.05, 95% CI).

DISCUSSION

Considered with existing evidence, our usability findings indicate that a temporary period of positive user perceptions occurs when new telemedicine systems are used for the first few months. This fades with everyday use, with design inconsistency and perceived complexity becoming more noticeable. Although other factors such as user satisfaction and efficiency may also contribute, further studies are needed for confirmation.

CONCLUSIONS

User perceptions of telemedicine vary with time. To help maximize the benefits and longevity of telemedicine systems, responding to intermittent user appraisal is desirable.

Nurses' Satisfaction With Using Nursing Information Systems From Technology Acceptance Model and Information Systems Success Model Perspectives: A Reductionist Approach

Nursing information systems can enhance nursing practice and the efficiency and quality of administrative affairs within the nursing department and thus have been widely considered for implementation. Close alignment of human-computer interaction can advance optimal clinical performance with the use of information systems. However, a lack of introduction of the concept of alignment between users' perceptions and technological functionality has caused dissatisfaction, as shown in the existing literature. This study provides insight into the alignment between nurses' perceptions and how technological functionality affects their satisfaction with Nursing Information System use through a reductionist perspective of alignment. This cross-sectional study collected data from 531 registered nurses in Taiwan. The results indicated that "perceived usefulness in system quality alignment," "perceived usefulness in information quality alignment," "perceived ease of use in system quality alignment," "perceived ease of use in information quality alignment," and "perceived ease of use in service quality alignment" have significantly affected nurses' satisfaction with Nursing Information System use. However, "perceived usefulness in service quality alignment" had no significant effect on nurses' satisfaction. This study also provides some meaningful implications for theoretical and practical aspects of design.

Validation of the DeLone and McLean Information Systems Success Model

Objectives

This study is an adaptation of the widely used DeLone and McLean information system success model in the context of hospital information systems in a developing country.

Methods

A survey research design was adopted in the study. A structured questionnaire was used to collect data from 442 health information management personnel in five Nigerian teaching hospitals. A structural equation modeling technique was used to validate the model's constructs.

Results

It was revealed that system quality significantly influenced use (β = 0.53, p < 0.001) and user satisfaction (β = 0.17, p < 0.001). Information quality significantly influenced use (β = 0.24, p < 0.001) and user satisfaction (β = 0.17, p < 0.001). Also, service quality significantly influenced use (β = 0.22, p < 0.001) and user satisfaction (β = 0.51, p < 0.001). However, use did not significantly influence user satisfaction (β = 0.00, p > 0.05), but it significantly influenced perceived net benefits (β = 0.21, p < 0.001). Furthermore, user satisfaction did not significantly influence perceived net benefits (β = 0.00, p > 0.05).

Conclusions

The study validates the DeLone and McLean information system success model in the context of a hospital information system in a developing country. Importantly, system quality and use were found to be important measures of hospital information system success. It is, therefore, imperative that hospital information systems are designed in such ways that are easy to use, flexible, and functional to serve their purpose.

Acceptance model of a Hospital Information System

PURPOSE

The purpose of this study is to develop a model of Hospital Information System (HIS) user acceptance focusing on human, technological, and organizational characteristics for supporting government eHealth programs. This model was then tested to see which hospital type in Indonesia would benefit from the model to resolve problems related to HIS user acceptance.

METHOD

This study used qualitative and quantitative approaches with case studies at four privately owned hospitals and three government-owned hospitals, which are general hospitals in Indonesia. The respondents involved in this study are low-level and mid-level hospital management officers, doctors, nurses, and administrative staff who work at medical record, inpatient, outpatient, emergency, pharmacy, and information technology units. Data was processed using Structural Equation Modeling (SEM) and AMOS 21.0.

RESULTS

The study concludes that non-technological factors, such as human characteristics (i.e. compatibility, information security expectancy, and self-efficacy), and organizational characteristics (i.e. management support, facilitating conditions, and user involvement) which have level of significance of p<0.05, significantly influenced users' opinions of both the ease of use and the benefits of the HIS. This study found that different factors may affect the acceptance of each user in each type of hospital regarding the use of HIS. Finally, this model is best suited for government-owned hospitals.

CONCLUSIONS

Based on the results of this study, hospital management and IT developers should have more understanding on the non-technological factors to better plan for HIS implementation. Support from management is critical to the sustainability of HIS implementation to ensure HIS is easy to use and provides benefits to the users as well as hospitals. Finally, this study could assist hospital management and IT developers, as well as researchers, to understand the obstacles faced by hospitals in implementing HIS.

How to overcome the digital divide? An empirical study of Taiwan’s DOCs

Purpose

The digital divide is a concern, as the inequality of information access might have significant influences on social development and quality of life. The purpose of this paper is to examine the perceived benefit of Digital Opportunity Centers (DOCs) programs on remote area participants from the perspective of computer anxiety and personal information ability.

Design/methodology/approach

The Taiwanese Government has built DOCs in remote areas to provide information technology (IT) training and learning programs to citizens residing in these areas. DOC program participants in Taiwan voluntarily completed a self-report questionnaire; the authors received 2,105 completed questionnaires, with a response rate of 84.2 percent. This research used partial least-squares (PLS) to empirical the research model.

Findings

Using PLS, the results show that information and communication technology ability influences the perceived benefit of DOC programs; computer anxiety has significantly negative effects on package software use, internet use, and IT usefulness; and internet use and IT usefulness have positive effects on perceived benefits.

Originality/value

IT is continuously advancing, but digital resources are still lacking within remote areas. DOCs provide citizens different types of learning experiences related to economic, social, and educational development. DOC programs provide participants with opportunities to obtain and improve basic IT knowledge and abilities and decreasing the digital divide.

Impact of doctors' resistance on success of drug utilization review system

OBJECTIVES

The drug utilization review (DUR) system, which checks any conflict event of medications, contributes to improve patient safety. One of the important barriers in its adoption is doctors' resistance. This study aimed to analyze the impacts of doctors' resistance on the success of the DUR system.

METHODS

This study adopted an augmented the DeLone and McLean Information System (D&M IS) Success Model (2003), which used doctors' resistance as a socio-technological measure. This study framework is the same as that of the D&M IS Success Model in that it is based on qualities, such as system, information, and services. The major difference is that this study excluded the variable 'use' because it was not statistically significant for mandatory systems. A survey of doctors who used computers to enter prescriptions was conducted at a Korean tertiary hospital in February 2012.

RESULTS

This study is very meaningful in that it is the first study to explore the success factors of the DUR system associated with doctors' resistance. Doctors' resistance to the DUR system was not statistically associated with user usefulness, whereas it affected user satisfaction.

CONCLUSIONS

The results indicate that doctors still complain of discomfort in using the DUR system in the outpatient clinical setting, even though they admit that it contributes to patient safety. To mitigate doctors' resistance and raise user satisfaction, more opinions from doctors regarding the DUR system have to be considered and have to be reflected in the system.

A review of data quality assessment methods for public health information systems

High quality data and effective data quality assessment are required for accurately evaluating the impact of public health interventions and measuring public health outcomes. Data, data use, and data collection process, as the three dimensions of data quality, all need to be assessed for overall data quality assessment. We reviewed current data quality assessment methods. The relevant study was identified in major databases and well-known institutional websites. We found the dimension of data was most frequently assessed. Completeness, accuracy, and timeliness were the three most-used attributes among a total of 49 attributes of data quality. The major quantitative assessment methods were descriptive surveys and data audits, whereas the common qualitative assessment methods were interview and documentation review. The limitations of the reviewed studies included inattentiveness to data use and data collection process, inconsistency in the definition of attributes of data quality, failure to address data users’ concerns and a lack of systematic procedures in data quality assessment. This review study is limited by the coverage of the databases and the breadth of public health information systems. Further research could develop consistent data quality definitions and attributes. More research efforts should be given to assess the quality of data use and the quality of data collection process.

Evaluation of a comprehensive EHR based on the DeLone and McLean model for IS success: approach, results, and success factors

OBJECTIVE

The article describes the methodological approach to, and results of an evaluation of a comprehensive electronic health record (EHR) in the shake down phase, shortly after its implementation at a regional hospital in Denmark.

DESIGN

A formative evaluation based on a mixed-methods case study, designed to be interactive and concurrent was conducted at two hospital departments based on the updated DeLone and McLean framework for evaluating information systems success.

METHODS

To ascertain user assessments of the EHR, we distributed a questionnaire two months after implementation to four groups of staff (physicians, nurses, medical secretaries, and physiotherapists; n=244), and at the same time we conducted thirteen individual, semi-structured interviews with representatives from these four groups. Subsequently, seven follow-up focus group interviews were conducted with the four above-mentioned groups, in order to go deeper into specific user assessments. Simultaneously, focus group interviews with two IT departments and the implementation team were conducted, to gain insight into system provider assessments of the implementation process and the EHR. Before, during, and after implementation, 88 h of ethnographic observation were carried out, to give the researchers an understanding of the daily routine of staff, and their use of health records. Finally, daily system performance data were obtained, to gather factual information on system response and downtime.

RESULTS

Overall, staff had positive experiences with the EHR and its operational reliability, response time, login and support. Performance was acceptable. Medical secretaries found the use of the patient administration module cumbersome, and physicians found the establishment of the overview of professionally relevant data challenging. There were demands for improvements to these and other functionalities, and for the EHR to be integrated with other systems and databases.

LIMITATIONS

Evaluations immediately following implementation are inherently difficult, but was required because a key role was to inform decision-making upon enrollment at other hospitals and systematically identify barriers in this respect. The strength of the evaluation is the mixed-methods approach. Further, the evaluation was based on assessments from staff in two departments that comprise around 50% of hospital staff. A weakness may be that staff assessment plays a major role in interviews and survey. These though are supplemented by performance data and observation. Also, the evaluation primarily reports upon the dimension 'user satisfaction', since use of the EHR is mandatory. Finally, generalizability may be low, since the evaluation was not based on a validated survey. All in all, however, the evaluation proposes an evaluation design in constrained circumstances.

CONCLUSIONS

Despite inherent limitations, evaluation of a comprehensive EHR shortly after implementation may be necessary, can be conducted, and may inform political decision making. The updated DeLone and McLean framework was constructive in the overall design of the evaluation of the EHR implementation, and allowed the model to be adapted to the health care domain by being methodological flexible. The mixed-methods case study produced valid and reliable results, and was accepted by staff, system providers, and political decision makers. The successful implementation may be attributed to the configurability of the EHR and to factors such as an experienced, competent implementation organization at the hospital, upgraded soft- and hardware, and a high degree of user involvement.

Physicians' Participation in Practice of Knowledge Management Systems

Healthcare organisations are knowledge-based establishments. At a healthcare organisation, computer-based information systems (ISs) are powerful tools for improving care using decision support such as patient health data recording, storing, delivery and retrieval. This study investigates the physicians' perspective for the success of ISs, and the relationships between the users' satisfactions and intellectual capital. We adopted a questionnaire survey for the purpose of outcome assessment and the correlations among each dimension, from a sample of 181 physicians in Taiwan's international medical service centres. The results indicate that IS success is significantly related to intellectual capital. With regard to IS quality, system quality has no significant effects on information quality and patients' privacy protection. This research provides empirical evidence to evaluate IS success in healthcare organisations and suggestions for how to promote physicians using knowledge management systems.

An online expert network for high quality information on occupational safety and health: cross-sectional study of user satisfaction and impact

Background

Many people have difficulties finding information on health questions, including occupational safety and health (OSH) issues. One solution to alleviate these difficulties could be to offer questioners free-of-charge, online access to a network of OSH experts who provide tailored, high-quality information. The aim of this study was to assess whether network quality, respectively information quality, as perceived by the questioners, is associated with questioners' overall satisfaction and to explore the impact of the information received on questioners' knowledge, work and work functioning.

Methods

We evaluated the experiences of OSH questioners with the online network ArboAntwoord.com over a two-year period. In this network, approximately 80 qualified experts are available to answer OSH questions. By means of a questionnaire, we assessed questioners' overall satisfaction with the network, whether the network was user-friendly, easily accessible and easy to handle and whether the information provided was complete, applicable and received in a timely manner. The impact of the information on questioners' knowledge, work or work functioning was explored with seven questions. In the study period, 460 unique OSH questioners asked 851 OSH questions. In total, 205 of the 460 questioners completed the questionnaire (response rate 45%).

Results

Of the responders, 71% were satisfied with the ArboAntwoord network. Multiple logistic regression analysis showed that the applicability of the information had a positive influence on the questioners' overall satisfaction (OR = 16.0, 95% CI: 7.0-36.4). Also, user friendliness of the network (OR = 3.3, 95% CI: 1.3-8.6) and completeness of the information provided (OR = 3.0, 95% CI: 1.3-6.8) were positively related to the questioners' satisfaction. For 74% of the questioners, the information helped to increase their knowledge and understanding. Overall, 25% of the questioners indicated that the received information improved their work, work functioning or health.

Conclusions

A free-of-charge, online expert network in the field of OSH can be a useful strategy to provide OSH questioners with applicable, complete and timely information that may help improve safety and health at work. This study provides more insight in how to satisfy network questioners and about the potential impact of provided information on OSH.