An evaluation framework for Health Information Systems: human, organization and technology-fit factors (HOT-fit)

Electronic health records: critical success factors in implementation

Introduction:

EHR implementation results in the improved quality of care, customer-orientation and timely access to complete information. Despite the potential benefits of EHR, its implementation is a difficult and complex task whose success depends on many factors. The purpose of this research is indeed to identify the key success factors of EHR.

Method and materials:

This is a cross-sectional survey conducted with participation of 340 work forces from different types of job from Hospitals of TUMS in 2014. Data were collected using a self-structured questionnaire which was estimated as both reliable and valid. The data were analyzed by SPSS software descriptive statistics and analytical statistics.

Results:

58.2% of respondents were female and their mean age and work experience were 37.7 and 11.2 years, respectively and most respondents (52.5%) was bachelor. In terms of job, the maximum rate was related to nursing (33 %) and physician (21 %). the main category of critical success factors in Implementation EHRs, the highest rate related to Project Management (4.62) and lowest related to Organizational factors (3.98).

Conclusion:

success in implementation EHRs requirement more centralization to project management and human factors. Therefore must be Creating to EHR roadmap implementation, establishment teamwork to participation of end-users and select prepare leadership, users obtains sufficient training to use of system and also prepare support from maintain and promotion system.

A case study evaluation of a Critical Care Information System adoption using the socio-technical and fit approach

BACKGROUND

Clinical information systems have long been used in intensive care units but reports on their adoption and benefits are limited. This study evaluated a Critical Care Information System implementation.

METHODS

A case study summative evaluation was conducted, employing observation, interview, and document analysis in operating theatres and 16-bed adult intensive care units in a 400-bed Malaysian tertiary referral centre from the perspectives of users (nurses and physicians), management, and information technology staff. System implementation, factors influencing adoption, fit between these factors, and the impact of the Critical Care Information System were evaluated after eight months of operation.

RESULTS

Positive influences on system adoption were associated with technical factors, including system ease of use, usefulness, and information relevancy; human factors, particularly user attitude; and organisational factors, namely clinical process-technology alignment and champions. Organisational factors such as planning, project management, training, technology support, turnover rate, clinical workload, and communication were barriers to system implementation and use. Recommendations to improve the current system problems were discussed. Most nursing staff positively perceived the system's reduction of documentation and data access time, giving them more time with patients. System acceptance varied among doctors. System use also had positive impacts on timesaving, data quality, and clinical workflow.

CONCLUSIONS

Critical Care Information Systems is crucial and has great potentials in enhancing and delivering critical care. However, the case study findings showed that the system faced complex challenges and was underutilised despite its potential. The role of socio-technical factors and their fit in realizing the potential of Critical Care Information Systems requires continuous, in-depth evaluation and stakeholder understanding and acknowledgement. The comprehensive and specific evaluation measures of the Human-Organisation-Technology Fit framework can flexibly evaluate Critical Care Information Systems.

Health information technology in hospitals: current issues and future trends

Healthcare is at an important crossroads in that current models of care are increasingly seen by politicians and policymakers as unsustainable. Furthermore, there is a need to move away from the reactive, doctor-centred model of care to one that is more patient-centred and that consistently delivers accessible, high-quality and safe care to all. Greater use of health information technology (HIT) is seen by many key decision makers as crucial to this transformation process and, hence, substantial investments are made in this area. However, healthcare, particularly in UK hospitals, remains a laggard in HIT adoption. To uncover the underlying reasons, we discuss current implementation and adoption challenges and explore potential ways to address these. We outline strategic, organisational, technical and social factors that can 'make or break' technological implementations. Most importantly, we suggest that efforts should be characterised by an underlying awareness of the complexity of the hospital environment and the need to develop tools that support provision of integrated multidisciplinary care. We conclude with a discussion of promising future developments, including increased patient involvement; access and contribution to shared records; the penetration of smart devices; greater health information exchange and interoperability; and innovative real-time secondary uses of data. We argue that there is considerable merit in evaluating the introduction of these interventions to help ensure that they are optimised for effectiveness, support efficient working and minimise the possibility of inadvertently introducing new risks into already complex health systems. Our over-riding message is that HIT should be seen as, where appropriate, a potential enabler and an important source of data to support healthcare redesign and that critical, ongoing evaluation is key to maximising benefits. However, it should not be seen as a silver bullet.

Development and Evaluation of an Educational E-Tool to Help Patients With Non-Hodgkin's Lymphoma Manage Their Personal Care Pathway

BACKGROUND

An overload of health-related information is available for patients on numerous websites, guidelines, and information leaflets. However, the increasing need for personalized health-related information is currently unmet.

OBJECTIVE

This study evaluates an educational e-tool for patients with non-Hodgkin's lymphoma (NHL) designed to meet patient needs with respect to personalized and complete health-related information provision. The e-tool aims to help NHL patients manage and understand their personal care pathway, by providing them with insight into their own care pathway, the possibility to keep a diary, and structured health-related information.

METHODS

Together with a multidisciplinary NHL expert panel, we developed an e-tool consisting of two sections: (1) a personal section for patients' own care pathway and their experiences, and (2) an informative section including information on NHL. We developed an ideal NHL care pathway based on the available (inter)national guidelines. The ideal care pathway, including date of first consultation, diagnosis, and therapy start, was used to set up the personal care pathway. The informative section was developed in collaboration with the patient association, Hematon. Regarding participants, 14 patients and 6 laymen were asked to evaluate the e-tool. The 24-item questionnaire used discussed issues concerning layout (6 questions), user convenience (3 questions), menu clarity (3 questions), information clarity (5 questions), and general impression (7 questions). In addition, the panel members were asked to give their feedback by email.

RESULTS

A comprehensive overview of diagnostics, treatments, and aftercare can be established by patients completing the questions from the personal section. The informative section consisted of NHL information regarding NHL in general, diagnostics, therapy, aftercare, and waiting times. Regarding participants, 6 patients and 6 laymen completed the questionnaire. Overall, the feedback was positive, with at least 75% satisfaction on each feedback item. Important strengths mentioned were the use of a low health-literacy level, the opportunity to document the personal care pathway and experiences, and the clear overview of the information provided. The added value of the e-tool in general was pointed out as very useful for preparing the consultation with one's doctor and for providing all information on one website, including the opportunity for a personalized care pathway and diary. The majority of the revisions concerned wording and clarity. In addition, more explicit information on immunotherapy, experimental therapy, and psychosocial support was added.

CONCLUSIONS

We have developed a personal care management e-tool for NHL patients. This tool contains a unique way to help patients manage their personal care pathway and give them insight into their NHL by providing health-related information and a personal diary. This evaluation showed that our e-tool meets patients' needs concerning personalized health-related information, which might serve as a good example for other oncologic diseases. Future research should focus on the possible impact of the e-tool on doctor-patient communication during consultations.

Organizational decision to adopt hospital information system: an empirical investigation in the case of Malaysian public hospitals

OBJECTIVES

This study mainly integrates the mature Technology-Organization-Environment (TOE) framework and recently developed Human-Organization-Technology (HOT) fit model to identify factors that affect the hospital decision in adopting Hospital Information System (HIS).

METHODS

Accordingly, a hybrid Multi-Criteria-Decision-Making (MCDM) model is used to address the dependence relationships of factors with the aid of Analytic Network Processes (ANP) and Decision Making Trial and Evaluation Laboratory (DEMATEL) approaches. The initial model of the study is designed by considering four main dimensions with 13 variables as organizational innovation adoption factors with respect to HIS. By using DEMATEL, the interdependencies strength among the dimensions and variables are tested. The ANP method is then adopted in order to determine the relative importance of the adoption factors, and is used to identify how these factors are weighted and prioritized by the public hospital professionals, who are wholly familiar with the HIS and have years of experience in decision making in hospitals' Information System (IS) department.

RESULTS

The results of this study indicate that from the experts' viewpoint "Perceived Technical Competence" is the most important factor in the Human dimension. In the Technology dimension, the experts agree that the "Relative Advantage" is more important in relation to the other factors. In the Organization dimension, "Hospital Size" is considered more important rather than others. And, in the Environment dimension, according to the experts judgment, "Government Policy" is the most important factor. The results of ANP survey from experts also reveal that the experts in the HIS field believed that these factors should not be overlooked by managers of hospitals and the adoption of HIS is more related to more consideration of these factors. In addition, from the results, it is found that the experts are more concerned about Environment and Technology for the adoption HIS.

CONCLUSIONS

The findings of this study make a novel contribution in the context of healthcare industry that is to improve the decision process of innovation in adoption stage and to help enhance more the diffusion of IS in the hospital setting, which by doing so, can provide plenty of profits to the patient community and the hospitals.

Assessing organizational capacity for achieving meaningful use of electronic health records

BACKGROUND

Health care institutions are scrambling to manage the complex organizational change required for achieving meaningful use (MU) of electronic health records (EHR). Assessing baseline organizational capacity for the change can be a useful step toward effective planning and resource allocation.

PURPOSE

The aim of this article is to describe an adaptable method and tool for assessing organizational capacity for achieving MU of EHR. Data on organizational capacity (people, processes, and technology resources) and barriers are presented from outpatient clinics within one integrated health care delivery system; thus, the focus is on MU requirements for eligible professionals, not eligible hospitals.

METHODS

We conducted 109 interviews with representatives from 46 outpatient clinics.

FINDINGS

Most clinics had core elements of the people domain of capacity in place. However, the process domain was problematic for many clinics, specifically, capturing problem lists as structured data and having standard processes for maintaining the problem list in the EHR. Also, nearly half of all clinics did not have methods for tracking compliance with their existing processes. Finally, most clinics maintained clinical information in multiple systems, not just the EHR. The most common perceived barriers to MU for eligible professionals included EHR functionality, changes to workflows, increased workload, and resistance to change.

PRACTICE IMPLICATIONS

Organizational capacity assessments provide a broad institutional perspective and an in-depth clinic-level perspective useful for making resource decisions and tailoring strategies to support the MU change effort for eligible professionals.

Impact of clinical decision support on receipt of antibiotic prescriptions for acute bronchitis and upper respiratory tract infection

OBJECTIVE

Antibiotics are commonly recognized as non-indicated for acute bronchitis and upper respiratory tract infection (URI), yet their widespread use persists. Clinical decision support in the form of electronic warnings is hypothesized to prevent non-indicated prescriptions. The purpose of this study was to identify the effect of clinical decision support on a common type of non-indicated prescription.

MATERIALS AND METHODS

Using National Ambulatory Medical Care Survey data from 2006 to 2010, ambulatory visits with a primary diagnosis of acute bronchitis or URI and orders for antibiotic prescriptions were identified. Visits were classified on the basis of clinician report of decision-support use. Generalized estimating equations were used to assess the effect of decision support on likelihood of antibiotic prescription receipt, controlling for patient, provider, and practice characteristics.

RESULTS

Clinician use of decision support increased sharply between 2006 (16% of visits) and 2010 (55%). Antibiotic prescribing for acute bronchitis and URI increased from ∼35% of visits in 2006 to ∼45% by 2010. Use of decision support was associated with a 19% lower likelihood of receiving an antibiotic prescription, controlling for patient, provider, and practice characteristics.

DISCUSSION

In spite of the increased use of decision-support systems and the relatively fewer non-indicated antibiotic prescriptions resulting from the use of decision support, a secular upward trend in non-indicated antibiotic prescribing offset these improvements.

CONCLUSIONS

The overall effect of decision support suggests an important role for technology in reducing non-indicated prescriptions. Decision support alone may not be sufficient to eliminate non-indicated prescriptions given secular trends.

Evaluating a federated medical search engine: tailoring the methodology and reporting the evaluation outcomes

BACKGROUND

Federated medical search engines are health information systems that provide a single access point to different types of information. Their efficiency as clinical decision support tools has been demonstrated through numerous evaluations. Despite their rigor, very few of these studies report holistic evaluations of medical search engines and even fewer base their evaluations on existing evaluation frameworks.

OBJECTIVES

To evaluate a federated medical search engine, MedSocket, for its potential net benefits in an established clinical setting.

METHODS

This study applied the Human, Organization, and Technology (HOT-fit) evaluation framework in order to evaluate MedSocket. The hierarchical structure of the HOT-factors allowed for identification of a combination of efficiency metrics. Human fit was evaluated through user satisfaction and patterns of system use; technology fit was evaluated through the measurements of time-on-task and the accuracy of the found answers; and organization fit was evaluated from the perspective of system fit to the existing organizational structure.

RESULTS

Evaluations produced mixed results and suggested several opportunities for system improvement. On average, participants were satisfied with MedSocket searches and confident in the accuracy of retrieved answers. However, MedSocket did not meet participants' expectations in terms of download speed, access to information, and relevance of the search results. These mixed results made it necessary to conclude that in the case of MedSocket, technology fit had a significant influence on the human and organization fit. Hence, improving technological capabilities of the system is critical before its net benefits can become noticeable.

CONCLUSIONS

The HOT-fit evaluation framework was instrumental in tailoring the methodology for conducting a comprehensive evaluation of the search engine. Such multidimensional evaluation of the search engine resulted in recommendations for system improvement.

Organizational learning in the implementation and adoption of national electronic health records: Case studies of two hospitals participating in the National Programme for Information Technology in England

Objective:

To explore the role of organizational learning in enabling implementation and supporting adoption of electronic health record systems into two English hospitals.

Methods and setting:

In the course of conducting our prospective and sociotechnical evaluation of the implementation and adoption of electronic health record into 12 “early adopter” hospitals across England, we identified two hospitals implementing virtually identical versions of the same “off-the-shelf” software (Millennium) within a comparable timeframe. We undertook a longitudinal qualitative case study–based analysis of these two hospitals (referred to hereafter as Alpha and Omega) and their implementation experiences. Data included the following: 63 in-depth interviews with various groups of internal and external stakeholders; 41-h on-site observation; and content analysis of 218 documents of various types. Analysis was both inductive and deductive, the latter being informed by the “sociotechnical changing” theoretical perspective.

Results:

Although Alpha and Omega shared a number of contextual similarities, our evaluation revealed fundamental differences in visions of electronic health record and the implementation strategy between the hospitals, which resulted in distinct local consequences of electronic health record implementation and impacted adoption. Both hospitals did not, during our evaluation, see the hoped-for benefits to the organization as a result of the introduction of electronic health record, such as speeding-up tasks. Nonetheless, the Millennium software worked out to be easier to use at Omega. Interorganizational learning was at the heart of this difference.

Conclusion:

Despite the turbulent overall national “roll out” of electronic health record systems into the English hospitals, considerable opportunities for organizational learning were offered by sequential delivery of the electronic health record software into “early adopter” hospitals. We argue that understanding the process of organizational learning and its enabling factors has the potential to support efforts at implementing national electronic health record implementation endeavors.

A comparative study of the proposed models for the components of the national health information system

Introduction:

National Health Information System plays an important role in ensuring timely and reliable access to Health information, which is essential for strategic and operational decisions that improve health, quality and effectiveness of health care. In other words, using the National Health information system you can improve the quality of health data, information and knowledge used to support decision making at all levels and areas of the health sector. Since full identification of the components of this system – for better planning and management influential factors of performanceseems necessary, therefore, in this study different attitudes towards components of this system are explored comparatively.

Methods:

This is a descriptive and comparative kind of study. The society includes printed and electronic documents containing components of the national health information system in three parts: input, process and output. In this context, search for information using library resources and internet search were conducted, and data analysis was expressed using comparative tables and qualitative data.

Results:

The findings showed that there are three different perspectives presenting the components of national health information system Lippeveld and Sauerborn and Bodart model in 2000, Health Metrics Network (HMN) model from World Health Organization in 2008, and Gattini’s 2009 model. All three models outlined above in the input (resources and structure) require components of management and leadership, planning and design programs, supply of staff, software and hardware facilities and equipment. Plus, in the “process” section from three models, we pointed up the actions ensuring the quality of health information system, and in output section, except for Lippeveld Model, two other models consider information products and use and distribution of information as components of the national health information system.

Conclusion:

the results showed that all the three models have had a brief discussion about the components of health information in input section. But Lippeveld model has overlooked the components of national health information in process and output sections. Therefore, it seems that the health measurement model of network has a comprehensive presentation for the components of health system in all three sections-input, process and output.

National evaluation of the benefits and risks of greater structuring and coding of the electronic health record: exploratory qualitative investigation

OBJECTIVE

We aimed to explore stakeholder views, attitudes, needs, and expectations regarding likely benefits and risks resulting from increased structuring and coding of clinical information within electronic health records (EHRs).

MATERIALS AND METHODS

Qualitative investigation in primary and secondary care and research settings throughout the UK. Data were derived from interviews, expert discussion groups, observations, and relevant documents. Participants (n=70) included patients, healthcare professionals, health service commissioners, policy makers, managers, administrators, systems developers, researchers, and academics.

RESULTS

Four main themes arose from our data: variations in documentation practice; patient care benefits; secondary uses of information; and informing and involving patients. We observed a lack of guidelines, co-ordination, and dissemination of best practice relating to the design and use of information structures. While we identified immediate benefits for direct care and secondary analysis, many healthcare professionals did not see the relevance of structured and/or coded data to clinical practice. The potential for structured information to increase patient understanding of their diagnosis and treatment contrasted with concerns regarding the appropriateness of coded information for patients.

CONCLUSIONS

The design and development of EHRs requires the capture of narrative information to reflect patient/clinician communication and computable data for administration and research purposes. Increased structuring and/or coding of EHRs therefore offers both benefits and risks. Documentation standards within clinical guidelines are likely to encourage comprehensive, accurate processing of data. As data structures may impact upon clinician/patient interactions, new models of documentation may be necessary if EHRs are to be read and authored by patients.

Team climate and attitudes toward information and communication technology among nurses on acute psychiatric wards

Objectives: To describe the association of team climate with attitudes toward information and communication technology among nursing staff working on acute psychiatric wards. Background: Implementation of ICT applications in nursing practice brings new operating models to work environments, which may affect experienced team climate on hospital wards. Method: Descriptive survey was used as a study design. Team climate was measured by the Finnish modification of the Team Climate Inventory, and attitudes toward ICT by Burkes' questionnaire. The nursing staff (N = 181, n = 146) on nine acute psychiatric wards participated in the study. Results: It is not self-evident that experienced team climate associates with attitudes toward ICT, but there are some positive relationships between perceived team climate and ICT attitudes. The study showed that nurses' motivation to use ICT had statistically significant connections with experienced team climate, participative safety (p = 0.021), support for innovation (p = 0.042) and task orientation (p = 0.042). Conclusion: The results suggest that asserting team climate and supporting innovative operations may lead to more positive attitudes toward ICT. It is, in particular, possible to influence nurses' motivation to use ICT. More attention should be paid to psychosocial factors such as group education and co-operation at work when ICT applications are implemented in nursing.

Application Portfolio Management in Hospitals

Application portfolio management concerns the management of all technology and applications, and is a complex task under pressure of increasing collaboration among hospitals. Various approaches to application portfolio management are described in existing literature, but are directed at commercial enterprises rather than health care organizations. Addressing this deficiency, this article presents the results of three case studies conducted at Dutch hospitals surveying current application portfolio management processes. Results show differences between the three hospitals. One hospital implemented a continuous application portfolio management process. The other hospitals perform ad-hoc IT management due to, among others, lacking support from management, decentralized IT governance structures and an increasing need for technical integration. This article can assist IT executives in making better informed decisions, while it provides a step towards a better understanding of the complex application portfolio management process in hospitals for academia.

Determining of factors influencing the success and failure of hospital information system and their evaluation methods: a systematic review

BACKGROUND

Nowadays, using new information technology (IT) has provided remarkable opportunities to decrease medical errors, support health care specialist, increase the efficiency and even the quality of patient's care and safety.

OBJECTIVES

The purpose of this study was the identification of Hospital Information System (HIS) success and failure factors and the evaluation methods of these factors. This research emphasizes the need to a comprehensive evaluation of HISs which considers a wide range of success and failure factors in these systems.

MATERIALS AND METHODS

We searched for relevant English language studies based on keywords in title and abstract, using PubMed, Ovid Medline (by applying MeSH terms), Scopus, ScienceDirect and Embase (earliest entry to march 17, 2012). Studies which considered success models and success or failure factors, or studied the evaluation models of HISs and the related ones were chosen. Since the studies used in this systematic review were heterogeneous, the combination of extracted data was carried out by using narrative synthesis method.

RESULTS

We found 16 articles which required detailed analysis. Finally, the suggested framework includes 12 main factors (functional, organizational, behavioral, cultural, management, technical, strategy, economy, education, legal, ethical and political factors), 67 sub factors, and 33 suggested methods for the evaluation of these sub factors.

CONCLUSIONS

The results of the present research indicates that the emphasis of the HIS evaluation moves from technical subjects to human and organizational subjects, and from objective to subjective issues. Therefore, this issue entails more familiarity with more qualitative evaluation methods. In most of the reviewed studies, the main focus has been laid on the necessity of using multi-method approaches and combining methods to obtain more comprehensive and useful results.

The computerized medical record as a tool for clinical governance in Australian primary care

Background

Computerized medical records (CMR) are used in most Australian general practices. Although CMRs have the capacity to amalgamate and provide data to the clinician about their standard of care, there is little research on the way in which they may be used to support clinical governance: the process of ensuring quality and accountability that incorporates the obligation that patients are treated according to best evidence.

Objective

The objective of this study was to explore the capability, capacity, and acceptability of CMRs to support clinical governance.

Methods

We conducted a realist review of the role of seven CMR systems in implementing clinical governance, developing a four-level maturity model for the CMR. We took Australian primary care as the context, CMR to be the mechanism, and looked at outcomes for individual patients, localities, and for the population in terms of known evidence-based surrogates or true outcome measures.

Results

The lack of standardization of CMRs makes national and international benchmarking challenging. The use of the CMR was largely at level two of our maturity model, indicating a relatively simple system in which most of the process takes place outside of the CMR, and which has little capacity to support benchmarking, practice comparisons, and population-level activities. Although national standards for coding and projects for record access are proposed, they are not operationalized.

Conclusions

The current CMR systems can support clinical governance activities; however, unless the standardization and data quality issues are addressed, it will not be possible for current systems to work at higher levels.

The USE IT-adoption-model to predict and evaluate adoption of information and communication technology in healthcare

BACKGROUND AND OBJECTIVE

The USE IT-model integrates theories about adoption and diffusion of innovations and is suitable to predict and evaluate the success of an information system from a user's perspective. The USE IT-model consists of four determinants: relevance, requirements, resources and resistance, which are measured at the macro-level (organizational), and at the micro-level (individual). After applying the USE IT approach in several researches we evaluated and updated the USE IT-model.

METHODS

We used the USE IT-model in ten case studies in healthcare and compared the results of the studies with the determinants and dimensions of the USE IT-model.

RESULTS

The quality of the implementation process is part of the innovation process-dimension and therefore relocated as a dimension of macro-resistance. The improvements and value in the relevance determinant are made more concrete by quality, efficiency, effectiveness, and task support. The dimensions of micro-resistance are reduced, and the dimension negative consequences is added. Also the dimensions of macro- and micro-requirements are made more specific to express the importance of information quality, availability and accessibility.

DISCUSSION AND CONCLUSION

The research resulted in the updated USE IT-adoption-model to predict and evaluate the adoption of information systems in healthcare. The structure and determinants of the original USE IT-model with a distinction between the macro- and micro-level remained unchanged.

Evaluation of a comprehensive EHR based on the DeLone and McLean model for IS success: approach, results, and success factors

OBJECTIVE

The article describes the methodological approach to, and results of an evaluation of a comprehensive electronic health record (EHR) in the shake down phase, shortly after its implementation at a regional hospital in Denmark.

DESIGN

A formative evaluation based on a mixed-methods case study, designed to be interactive and concurrent was conducted at two hospital departments based on the updated DeLone and McLean framework for evaluating information systems success.

METHODS

To ascertain user assessments of the EHR, we distributed a questionnaire two months after implementation to four groups of staff (physicians, nurses, medical secretaries, and physiotherapists; n=244), and at the same time we conducted thirteen individual, semi-structured interviews with representatives from these four groups. Subsequently, seven follow-up focus group interviews were conducted with the four above-mentioned groups, in order to go deeper into specific user assessments. Simultaneously, focus group interviews with two IT departments and the implementation team were conducted, to gain insight into system provider assessments of the implementation process and the EHR. Before, during, and after implementation, 88 h of ethnographic observation were carried out, to give the researchers an understanding of the daily routine of staff, and their use of health records. Finally, daily system performance data were obtained, to gather factual information on system response and downtime.

RESULTS

Overall, staff had positive experiences with the EHR and its operational reliability, response time, login and support. Performance was acceptable. Medical secretaries found the use of the patient administration module cumbersome, and physicians found the establishment of the overview of professionally relevant data challenging. There were demands for improvements to these and other functionalities, and for the EHR to be integrated with other systems and databases.

LIMITATIONS

Evaluations immediately following implementation are inherently difficult, but was required because a key role was to inform decision-making upon enrollment at other hospitals and systematically identify barriers in this respect. The strength of the evaluation is the mixed-methods approach. Further, the evaluation was based on assessments from staff in two departments that comprise around 50% of hospital staff. A weakness may be that staff assessment plays a major role in interviews and survey. These though are supplemented by performance data and observation. Also, the evaluation primarily reports upon the dimension 'user satisfaction', since use of the EHR is mandatory. Finally, generalizability may be low, since the evaluation was not based on a validated survey. All in all, however, the evaluation proposes an evaluation design in constrained circumstances.

CONCLUSIONS

Despite inherent limitations, evaluation of a comprehensive EHR shortly after implementation may be necessary, can be conducted, and may inform political decision making. The updated DeLone and McLean framework was constructive in the overall design of the evaluation of the EHR implementation, and allowed the model to be adapted to the health care domain by being methodological flexible. The mixed-methods case study produced valid and reliable results, and was accepted by staff, system providers, and political decision makers. The successful implementation may be attributed to the configurability of the EHR and to factors such as an experienced, competent implementation organization at the hospital, upgraded soft- and hardware, and a high degree of user involvement.

Ten key considerations for the successful implementation and adoption of large-scale health information technology

The implementation of health information technology interventions is at the forefront of most policy agendas internationally. However, such undertakings are often far from straightforward as they require complex strategic planning accompanying the systemic organizational changes associated with such programs. Building on our experiences of designing and evaluating the implementation of large-scale health information technology interventions in the USA and the UK, we highlight key lessons learned in the hope of informing the on-going international efforts of policymakers, health directorates, healthcare management, and senior clinicians.

An information systems model of the determinants of electronic health record use

OBJECTIVES

The prominence given to universal implementation of electronic health record (EHR) systems in U.S. health care reform, underscores the importance of devising reliable measures of factors that predict medical care providers' use of EHRs. This paper presents an easily administered provider survey instrument that includes measures corresponding to core dimensions of DeLone and McClean's (D & M) model of information system success.

METHODS

Study data came from self-administered surveys completed by 460 primary care providers, who had recently begun using an EHR.

RESULTS

Based upon assessment of psychometric properties of survey items, a revised D&M causal model was formulated that included four measures of the determinants of EHR use (system quality, IT support, ease of use, user satisfaction) and five indicators of provider beliefs about the impact on an individual's clinical practice. A structural equation model was estimated that demonstrated a high level of inter-correlation between the four scales measuring determinants of EHR use. All four variables had positive association with each of the five individual impact measures. Consistent with our revised D&M model, the association of system quality and IT support with the individual impact measures was entirely mediated by ease of use and user satisfaction.

CONCLUSIONS

Survey research provides important insights into provider experiences with EHR. Additional studies are in progress to investigate how the variables constructed for this study are related to direct measures of EHR use.

Towards an evaluation framework for information quality management (IQM) practices for health information systems--evaluation criteria for effective IQM practices

RATIONALE, AIMS AND OBJECTIVES

Poor information quality (IQ) must be understood as a business problem rather than systems problem. In health care organization, what is required is an effective quality management that continuously manages and reviews the factors influencing IQ in health information systems (HIS) so as to achieve the desired outcomes. Hence, in order to understand the issues of information quality management (IQM) practices in health care organizations, a more holistic evaluation study should be undertaken to investigate the IQM practices in health care organizations. It is the aim of this paper to identify the significant evaluation criteria that influence the production of good IQ in HIS.

METHODS

Six selected frameworks and best practices both from health informatics and information systems literature have been reviewed to identify the evaluation criteria from the perspective of human, organizational and technological factors.

RESULTS

From the review, it was found that human and organization factors are of greater significance in influencing HIS IQ. Our review depicts that there is still shortage in finding a comprehensive IQM evaluation framework. Thus, the criteria from the frameworks reviewed can be used in combination for more comprehensive evaluation criteria. Integrated IQM evaluation criteria for HIS are then proposed in this study.

CONCLUSIONS

Poor IQ is the result of complex interdependency within sociotechnical factors in health care organization and lack of formal and structured IQM practices. Thus, a feedback mechanism such as evaluation is needed to understand the issues in depth in the future.

The Need for a Socio-Technical Analysis in E-Health

Healthcare systems around the globe are facing a number of challenges. Thus Increasing focus is being placed on constructing appropriate healthcare reforms which are attempting to address how to tackle these challenges. A critical enabler in these reforms is the adoption of an e-health solution. Such e-health solutions are not only expensive and complex endeavours, but also have far reaching implications. Given that the implementation and adoption of these e-health solutions is so important, not to mention also requiring a substantial investment in various resources such as time and money, it is therefore essential to ensure their success. The following proffers a socio-technical analysis as an appropriate strategy to ensure more successful outcomes. An exemplar case study of the Personally Controlled Electronic Health Record (PCEHR), the chosen e-health solution by the Australian government is provided to illustrate the benefits such an analysis might provide

Factors of accepting pain management decision support systems by nurse anesthetists

BACKGROUND

Pain management is a critical but complex issue for the relief of acute pain, particularly for postoperative pain and severe pain in cancer patients. It also plays important roles in promoting quality of care. The introduction of pain management decision support systems (PM-DSS) is considered a potential solution for addressing the complex problems encountered in pain management. This study aims to investigate factors affecting acceptance of PM-DSS from a nurse anesthetist perspective.

METHODS

A questionnaire survey was conducted to collect data from nurse anesthetists in a case hospital. A total of 113 questionnaires were distributed, and 101 complete copies were returned, indicating a valid response rate of 89.3%. Collected data were analyzed by structure equation modeling using the partial least square tool.

RESULTS

The results show that perceived information quality (γ=.451, p<.001), computer self-efficacy (γ=.315, p<.01), and organizational structure (γ=.210, p<.05), both significantly impact nurse anesthetists' perceived usefulness of PM-DSS. Information quality (γ=.267, p<.05) significantly impacts nurse anesthetists' perceptions of PM-DSS ease of use. Furthermore, both perceived ease of use (β=.436, p<.001, R(2)=.487) and perceived usefulness (β=.443, p<.001, R(2)=.646) significantly affected nurse anesthetists' PM-DSS acceptance (R2=.640). Thus, the critical role of information quality in the development of clinical decision support system is demonstrated.

CONCLUSIONS

The findings of this study enable hospital managers to understand the important considerations for nurse anesthetists in accepting PM-DSS, particularly for the issues related to the improvement of information quality, perceived usefulness and perceived ease of use of the system. In addition, the results also provide useful suggestions for designers and implementers of PM-DSS in improving system development.

Comparative case study investigating sociotechnical processes of change in the context of a national electronic health record implementation

The introduction of electronic health records (EHRs) lies at the heart of many international efforts to improve the safety and quality of healthcare. England has attempted to introduce nationally procured EHR software--the first country in the world to do so. In this qualitative comparative case study tracing local developments over time we sought to generate a detailed picture of the implementation landscape characterising this first attempt at implementing nationally procured software through studying three purposefully selected hospitals. Despite differences in relation to demographic considerations and local implementation strategies, implementing hospitals faced similar technical and political challenges. These were coped with differently by the various organisations and individual stakeholders, their responses being shaped by contextual contingencies. We conclude that national implementation efforts need to allow effective technology adoption to occur locally before considering larger-scale interoperability. This should involve the allocation of sufficient time for individual users and organisations to adjust to the complex changes that often accompany such service re-design initiatives.

Evaluation of the clinical process in a critical care information system using the Lean method: a case study

Background

There are numerous applications for Health Information Systems (HIS) that support specific tasks in the clinical workflow. The Lean method has been used increasingly to optimize clinical workflows, by removing waste and shortening the delivery cycle time. There are a limited number of studies on Lean applications related to HIS. Therefore, we applied the Lean method to evaluate the clinical processes related to HIS, in order to evaluate its efficiency in removing waste and optimizing the process flow. This paper presents the evaluation findings of these clinical processes, with regards to a critical care information system (CCIS), known as IntelliVue Clinical Information Portfolio (ICIP), and recommends solutions to the problems that were identified during the study.

Methods

We conducted a case study under actual clinical settings, to investigate how the Lean method can be used to improve the clinical process. We used observations, interviews, and document analysis, to achieve our stated goal. We also applied two tools from the Lean methodology, namely the Value Stream Mapping and the A3 problem-solving tools. We used eVSM software to plot the Value Stream Map and A3 reports.

Results

We identified a number of problems related to inefficiency and waste in the clinical process, and proposed an improved process model.

Conclusions

The case study findings show that the Value Stream Mapping and the A3 reports can be used as tools to identify waste and integrate the process steps more efficiently. We also proposed a standardized and improved clinical process model and suggested an integrated information system that combines database and software applications to reduce waste and data redundancy.

Organizational issues in the implementation and adoption of health information technology innovations: an interpretative review

PURPOSE

Implementations of health information technologies are notoriously difficult, which is due to a range of inter-related technical, social and organizational factors that need to be considered. In the light of an apparent lack of empirically based integrated accounts surrounding these issues, this interpretative review aims to provide an overview and extract potentially generalizable findings across settings.

METHODS

We conducted a systematic search and critique of the empirical literature published between 1997 and 2010. In doing so, we searched a range of medical databases to identify review papers that related to the implementation and adoption of eHealth applications in organizational settings. We qualitatively synthesized this literature extracting data relating to technologies, contexts, stakeholders, and their inter-relationships.

RESULTS

From a total body of 121 systematic reviews, we identified 13 systematic reviews encompassing organizational issues surrounding health information technology implementations. By and large, the evidence indicates that there are a range of technical, social and organizational considerations that need to be deliberated when attempting to ensure that technological innovations are useful for both individuals and organizational processes. However, these dimensions are inter-related, requiring a careful balancing act of strategic implementation decisions in order to ensure that unintended consequences resulting from technology introduction do not pose a threat to patients.

CONCLUSIONS

Organizational issues surrounding technology implementations in healthcare settings are crucially important, but have as yet not received adequate research attention. This may in part be due to the subjective nature of factors, but also due to a lack of coordinated efforts toward more theoretically-informed work. Our findings may be used as the basis for the development of best practice guidelines in this area.

Collaborative research: opinions and information technology utilization potential

Purpose

The purpose of this paper is twofold. First, to study collaborative research in general and identify its characteristics, advantages and disadvantages by conglomerating various views and ideas from the literature and from expert opinion research. Second, to determine the importance of and attitudes toward the usage of information technology (IT) in collaborative research projects.

Design/methodology/approach

Initially, an extensive literature review and ten in‐depth interviews have been conducted to determine the identifying characteristics of collaborative research. Consequently, a questionnaire is used as the major data collection tool to assess academicians' opinions about collaborative research and attitudes toward IT utilization in such studies.

Findings

Collaborative research has been most distinctively defined with the concepts of teamwork, knowledge and experience sharing and direct and continuous communication. Increased visibility and recognition of such projects and enhanced access to various resources have also been pinpointed as important advantages. Academicians have shown a highly positive tendency to utilize IT and adopt IT tools that will enhance convenience and communication and contribute to the conduction of various basic and support processes of collaborative research projects.

Research limitations/implications

The major implication of this research is the general need for more collaborative research projects in academia and the necessity to develop and employ various IT tools that can be used in such studies. Similar studies can be done with larger sample sizes or across various contexts, for comparative purposes, to overcome the limitations of the study.

Originality/value

In this paper, the dispersed literature about collaborative research has been assembled and a unified scope has been drawn around the concept. Attitudes toward collaborative research and the potential value of utilizing IT in such projects, which has been studied at an inadequate level in the literature, have been evaluated comprehensively from the perspective of academicians.

Feasibility of a wiki as a participatory tool for patients in clinical guideline development

Background

Patient participation is essential in developing high-quality guidelines but faces practical challenges. Evidence on timing, methods, evaluations, and outcomes of methodologies for patient participation in guideline development is lacking.

Objective

To assess the feasibility of a wiki as a participatory tool for patients in the development of a guideline on infertility determined by (1) use of the wiki (number of page views and visitors), (2) benefits of the wiki (ie, number, content, and eligibility of the recommendations to be integrated into the guideline), and (3) patients’ facilitators of and barriers to adoption, and the potential challenges to be overcome in improving this wiki.

Methods

To obtain initial content for the wiki, we conducted in-depth interviews (n = 12) with infertile patients. Transcripts from the interviews were translated into 90 draft recommendations. These were presented on a wiki. Over 7 months, infertile patients were invited through advertisements or mailings to formulate new or modify existing recommendations. After modifying the recommendations, we asked patients to select their top 5 or top 3 recommendations for each of 5 sections on fertility care. Finally, the guideline development group assessed the eligibility of the final set of recommendations within the scope of the guideline. We used a multimethod evaluation strategy to assess the feasibility of the wiki as a participatory tool for patients in guideline development.

Results

The wiki attracted 298 unique visitors, yielding 289 recommendations. We assessed the 21 recommendations ranked as the top 5 or top 3 for their eligibility for being integrated into the clinical practice guideline. The evaluation identified some challenges needed to be met to improve the wiki tool, concerning its ease of use, website content and layout, and characteristics of the wiki tool.

Conclusions

The wiki is a promising and feasible participatory tool for patients in guideline development. A modified version of this tool including new modalities (eg, automatically limiting the number and length of recommendations, using a fixed format for recommendations, including a motivation page, and adding a continuous prioritization system) should be developed and evaluated in a patient-centered design.

Factors determining nurse acceptance of hospital information systems

This study proposed an evaluation model, derived from the Technology Acceptance Model and Information System Success Model, to explore factors influencing the acceptance of hospital information systems by nurses. Although many healthcare institutions have applied hospital information systems, relatively few studies have investigated the perceptions of nurses regarding the usefulness, ease of use, or acceptance of these systems. This study recruited the nursing staff of a regional hospital in Taiwan. A total of 297 questionnaires were sent, and 277 were returned, for a response rate of 93.3%. The results indicated that system quality, information quality, and service quality were positively correlated with the perceived ease of use (R=0.69) and perceived usefulness (R=0.72). Information quality has the greatest influence on perceived usefulness (γ3=0.57, P<.001) and ease of use (γ4=0.61, P<.001). Perceived usefulness and ease of use have a significant influence on system acceptance (R=0.75). Perceived usefulness (β1=0.61, P<.001) has a significant influence on system acceptance. These findings indicated that nursing care requires high-quality healthcare information to support the daily activities of nursing professionals. The results of this study also provide a valuable reference for hospital administrators in developing hospital information systems.

Health care professionals' beliefs about using wiki-based reminders to promote best practices in trauma care

Background

Wikis are knowledge translation tools that could help health professionals implement best practices in acute care. Little is known about the factors influencing professionals’ use of wikis.

Objectives

To identify and compare the beliefs of emergency physicians (EPs) and allied health professionals (AHPs) about using a wiki-based reminder that promotes evidence-based care for traumatic brain injuries.

Methods

Drawing on the theory of planned behavior, we conducted semistructured interviews to elicit EPs’ and AHPs’ beliefs about using a wiki-based reminder. Previous studies suggested a sample of 25 EPs and 25 AHPs. We purposefully selected participants from three trauma centers in Quebec, Canada, to obtain a representative sample. Using univariate analyses, we assessed whether our participants’ gender, age, and level of experience were similar to those of all eligible individuals. Participants viewed a video showing a clinician using a wiki-based reminder, and we interviewed participants about their behavioral, control, and normative beliefs—that is, what they saw as advantages, disadvantages, barriers, and facilitators to their use of a reminder, and how they felt important referents would perceive their use of a reminder. Two reviewers independently analyzed the content of the interview transcripts. We considered the 75% most frequently mentioned beliefs as salient. We retained some less frequently mentioned beliefs as well.

Results

Of 66 eligible EPs and 444 eligible AHPs, we invited 55 EPs and 39 AHPs to participate, and 25 EPs and 25 AHPs (15 nurses, 7 respiratory therapists, and 3 pharmacists) accepted. Participating AHPs had more experience than eligible AHPs (mean 14 vs 11 years; P = .04). We noted no other significant differences. Among EPs, the most frequently reported advantage of using a wiki-based reminder was that it refreshes the memory (n = 14); among AHPs, it was that it provides rapid access to protocols (n = 16). Only 2 EPs mentioned a disadvantage (the wiki added stress). The most frequently reported favorable referent was nurses for EPs (n = 16) and EPs for AHPs (n = 19). The most frequently reported unfavorable referents were people resistant to standardized care for EPs (n = 8) and people less comfortable with computers for AHPs (n = 11). The most frequent facilitator for EPs was ease of use (n = 19); for AHPs, it was having a bedside computer (n = 20). EPs’ most frequently reported barrier was irregularly updated wiki-based reminders (n = 18); AHPs’ was undetermined legal responsibility (n = 10).

Conclusions

We identified EPs’ and AHPs’ salient beliefs about using a wiki-based reminder. We will draw on these beliefs to construct a questionnaire to measure the importance of these determinants to EPs’ and AHPs’ intention to use a wiki-based reminder promoting evidence-based care for traumatic brain injuries.

An empirical study of physicians' acceptance of hospital information systems in Taiwan

OBJECTIVE

This study investigates the key factors affecting the acceptance of hospital information systems (HIS) by physicians based on an extensive technology acceptance model. This model consists of six constructs: System quality, information quality, service quality, perceived usefulness, perceived ease of use, and HIS acceptance.

MATERIALS AND METHODS

Survey research targeted physicians in the selected case hospital as subjects. Of 131 questionnaires sent out, 81 completed forms were returned (a valid response rate of 64.8%). The partial least squares approach was used to analyze the collected data.

RESULTS

This study shows that system quality and information quality are significant factors influencing perceived ease of use of HIS, while information quality and service quality are key factors affecting perceived usefulness of HIS. Results indicate that perceived usefulness and perceived ease of use significantly affect HIS acceptance (R(2)=0.545) from the physician's perspective.

CONCLUSIONS

The findings of this study can help managers understand the key considerations affecting HIS development and use and provide important reference material for system improvement.

Promoting Success in the Introduction of Health Information Systems

The significant number of publications describing unsuccessful cases in the introduction of health information systems makes it advisable to analyze the factors that may be contributing to such failures. However, the very notion of success is not equally assumed in all publications. Based in a literature review, the authors argue that the introduction of systems must be based in an eclectic combination of knowledge fields, adopting methodologies that strengthen the role of organizational culture and human resources in this project, as a whole. On the other hand, the authors argue that the introduction of systems should be oriented by a previously defined matrix of factors, against which the success can be measured.

How can routine health information systems improve health systems functioning in low- and middle-income countries? Assessing the evidence base

PURPOSE

Health system performance depends on production and use of quality health data and information. Routine health information systems (RHIS) are defined as systems that provide information at regular intervals of a year or less to meet predictable information needs. These include paper-based or electronic health records and facility- and district-level management information systems. RHIS are receiving increasing attention as an essential component of efficient, country-owned, integrated national systems. To guide investment decisions on RHIS, evidence is needed on which types of RHIS interventions work and which do not.

DESIGN/METHODOLOGY/APPROACH

This chapter is a systematic review of the literature on the evaluation of RHIS interventions in low- and middle-income countries, starting from the premise that investments in RHIS could be better understood and so produce greater benefits than they currently do.

FINDINGS

We describe the conceptual literature on the determinants of RHIS performance and its role in improving health systems functioning, discuss the evidence base on the effectiveness of strategies to improve RHIS performance, provide an overview of RHIS evaluation challenges, and make suggestions to improve the evidence base.

ORIGINALITY/VALUE

The goal is to help ensure that (a) RHIS interventions are appropriately designed and implemented to improve health systems functioning and (b) resulting RHIS information is used more effectively.

A holistic framework to improve the uptake and impact of eHealth technologies

BACKGROUND

Many eHealth technologies are not successful in realizing sustainable innovations in health care practices. One of the reasons for this is that the current development of eHealth technology often disregards the interdependencies between technology, human characteristics, and the socioeconomic environment, resulting in technology that has a low impact in health care practices. To overcome the hurdles with eHealth design and implementation, a new, holistic approach to the development of eHealth technologies is needed, one that takes into account the complexity of health care and the rituals and habits of patients and other stakeholders.

OBJECTIVE

The aim of this viewpoint paper is to improve the uptake and impact of eHealth technologies by advocating a holistic approach toward their development and eventual integration in the health sector.

METHODS

To identify the potential and limitations of current eHealth frameworks (1999-2009), we carried out a literature search in the following electronic databases: PubMed, ScienceDirect, Web of Knowledge, PiCarta, and Google Scholar. Of the 60 papers that were identified, 44 were selected for full review. We excluded those papers that did not describe hands-on guidelines or quality criteria for the design, implementation, and evaluation of eHealth technologies (28 papers). From the results retrieved, we identified 16 eHealth frameworks that matched the inclusion criteria. The outcomes were used to posit strategies and principles for a holistic approach toward the development of eHealth technologies; these principles underpin our holistic eHealth framework.

RESULTS

A total of 16 frameworks qualified for a final analysis, based on their theoretical backgrounds and visions on eHealth, and the strategies and conditions for the research and development of eHealth technologies. Despite their potential, the relationship between the visions on eHealth, proposed strategies, and research methods is obscure, perhaps due to a rather conceptual approach that focuses on the rationale behind the frameworks rather than on practical guidelines. In addition, the Web 2.0 technologies that call for a more stakeholder-driven approach are beyond the scope of current frameworks. To overcome these limitations, we composed a holistic framework based on a participatory development approach, persuasive design techniques, and business modeling.

CONCLUSIONS

To demonstrate the impact of eHealth technologies more effectively, a fresh way of thinking is required about how technology can be used to innovate health care. It also requires new concepts and instruments to develop and implement technologies in practice. The proposed framework serves as an evidence-based roadmap.

Quality of human-computer interaction--results of a national usability survey of hospital-IT in Germany

Background

Due to the increasing functionality of medical information systems, it is hard to imagine day to day work in hospitals without IT support. Therefore, the design of dialogues between humans and information systems is one of the most important issues to be addressed in health care. This survey presents an analysis of the current quality level of human-computer interaction of healthcare-IT in German hospitals, focused on the users' point of view.

Methods

To evaluate the usability of clinical-IT according to the design principles of EN ISO 9241-10 the IsoMetrics Inventory, an assessment tool, was used. The focus of this paper has been put on suitability for task, training effort and conformity with user expectations, differentiated by information systems. Effectiveness has been evaluated with the focus on interoperability and functionality of different IT systems.

Results

4521 persons from 371 hospitals visited the start page of the study, while 1003 persons from 158 hospitals completed the questionnaire. The results show relevant variations between different information systems.

Conclusions

Specialised information systems with defined functionality received better assessments than clinical information systems in general. This could be attributed to the improved customisation of these specialised systems for specific working environments. The results can be used as reference data for evaluation and benchmarking of human computer engineering in clinical health IT context for future studies.

A study of factors affecting acceptance of hospital information systems: a nursing perspective

BACKGROUND

Hospital information systems (HISs) are widely used in Taiwan, and HIS performance must be carefully evaluated. Nursing personnel are the largest group of staff in a hospital and are the center of care delivery; thus, they play an important role in the adoption and evaluation of HISs.

PURPOSE

The primary objective of this study was to explore the critical factors affecting the acceptance of HISs in Taiwan from a nursing perspective. On the basis of the technology acceptance model, we used six exogenous variables (system quality, information quality, user self-efficacy, compatibility, top management support, and project team competency) as investigation factors.

METHODS

Survey research targeted nursing personnel in the selected case hospital as participants. A total of 545 questionnaires were sent out, and 501 were returned, indicating a valid response rate of 91.9%. Collected data were analyzed using multiple regression analysis.

RESULTS

: Results indicate that user self-efficacy, top management support, compatibility, and information quality have significant impacts on perceived ease of use. In addition, top management support, compatibility, and information quality were identified as having significant impacts on perceived usefulness. Furthermore, nurses' perceived ease of use and perceived usefulness on HISs was found to impact significantly on system acceptance, with 45.1% of the total explained variance.

CONCLUSIONS/IMPLICATIONS FOR PRACTICE

Results can help managers understand key considerations affecting HIS development and use and may be applied as a reference for system development and improvement.

Guideline for good evaluation practice in health informatics (GEP-HI)

OBJECTIVE

Development of a good practice guideline to plan and perform scientifically robust evaluation studies in health informatics.

METHODS

Issues to be addressed in evaluation studies were identified and guidance drafted based on the evaluation literature and on experiences by key players. Successive drafts of the guideline were discussed in several rounds by an increasing number of experts during conferences and by e-mail. At a fairly early point the guideline was put up for comments on the web.

RESULTS

Sixty issues were identified that are of potential relevance for planning, implementation and execution of an evaluation study in the health informatics domain. These issues cover all phases of an evaluation study: Preliminary outline, study design, operationalization of methods, project planning, execution and completion of the evaluation study. Issues of risk management and project control as well as reporting and publication of the evaluation results are also addressed.

CONCLUSION

A comprehensive list of issues is presented as a guideline for good evaluation practice in health informatics (GEP-HI). The strengths and weaknesses of the guideline are discussed. Application of this guideline will support better handling of an evaluation study, potentially leading to a higher quality of evaluation studies. This guideline is an important step towards building stronger evidence and thus to progress towards evidence-based health informatics.

Effects of the implementation of the web-based patient support system on staff's attitudes towards computers and IT use: a randomised controlled trial

Utilisation of information technology (IT) in the treatment of people with severe mental health problems is an unknown area in Europe. Use of IT and guiding patients to relevant sources of health information requires that nursing staff have positive attitudes toward computers and accept IT use as a part of daily practises. The aim of the study was to assess the effects of the implementation of a web-based patient support system on staff's attitudes towards computers and IT use on psychiatric wards. Hundred and forty-nine nurses in two psychiatric hospitals in Finland were randomised to two groups to deliver patient education for patients with schizophrenia and psychosis with a web-based system (n = 76) or leaflets (n = 73). After baseline nurses were followed-up for 18 months after the introduction of the system. The primary outcome was nurses' motivation to utilise computers, and the secondary outcomes were nurses' beliefs in and satisfaction with computers, and use of computer and internet. There were no statistically significant differences between study groups in attitudes towards computers (motivation p = 0.936, beliefs p = 0.270, satisfaction p = 0.462) and internet use (p = 0.276). However, nurses' general computer use (p = 0.029) increased more in the leaflet group than in the IT intervention group. We can conclude that IT has promise as an alternative method in patient education, as the implementation of the web-based patient support system in daily basis did not have a negative effect on nurses' attitudes towards IT.

Determining patient preferences for remote monitoring

This paper presents the patient preferences for an application in remote health monitoring. The data was collected through a mobile service prototype. Analytical Hierarchy Process and Conjoint Analysis were used to extract the patient preferences. The study was limited to diabetes and obesity patients in Istanbul, Turkey. Results indicated that sending users' data automatically, availability of technical support, and price are key factors impacting patient's decisions. This implies that e-health service providers and designers should focus on the services that enable users to send measurement results automatically instead of manually.

Systematic review of factors influencing the adoption of information and communication technologies by healthcare professionals

This systematic review of mixed methods studies focuses on factors that can facilitate or limit the implementation of information and communication technologies (ICTs) in clinical settings. Systematic searches of relevant bibliographic databases identified studies about interventions promoting ICT adoption by healthcare professionals. Content analysis was performed by two reviewers using a specific grid. One hundred and one (101) studies were included in the review. Perception of the benefits of the innovation (system usefulness) was the most common facilitating factor, followed by ease of use. Issues regarding design, technical concerns, familiarity with ICT, and time were the most frequent limiting factors identified. Our results suggest strategies that could effectively promote the successful adoption of ICT in healthcare professional practices.