Integrating Quantitative and Qualitative Methods in Patient Care Information System Evaluation

Radiation Oncologists' Perceptions of Adopting an Artificial Intelligence-Assisted Contouring Technology: Model Development and Questionnaire Study

Background

An artificial intelligence (AI)–assisted contouring system benefits radiation oncologists by saving time and improving treatment accuracy. Yet, there is much hope and fear surrounding such technologies, and this fear can manifest as resistance from health care professionals, which can lead to the failure of AI projects.

Objective

The objective of this study was to develop and test a model for investigating the factors that drive radiation oncologists’ acceptance of AI contouring technology in a Chinese context.

Methods

A model of AI-assisted contouring technology acceptance was developed based on the Unified Theory of Acceptance and Use of Technology (UTAUT) model by adding the variables of perceived risk and resistance that were proposed in this study. The model included 8 constructs with 29 questionnaire items. A total of 307 respondents completed the questionnaires. Structural equation modeling was conducted to evaluate the model’s path effects, significance, and fitness.

Results

The overall fitness indices for the model were evaluated and showed that the model was a good fit to the data. Behavioral intention was significantly affected by performance expectancy (β=.155; P=.01), social influence (β=.365; P<.001), and facilitating conditions (β=.459; P<.001). Effort expectancy (β=.055; P=.45), perceived risk (β=−.048; P=.35), and resistance bias (β=−.020; P=.63) did not significantly affect behavioral intention.

Conclusions

The physicians’ overall perceptions of an AI-assisted technology for radiation contouring were high. Technology resistance among Chinese radiation oncologists was low and not related to behavioral intention. Not all of the factors in the Venkatesh UTAUT model applied to AI technology adoption among physicians in a Chinese context.

System dynamics modelling of health workforce planning to address future challenges of Thailand's Universal Health Coverage

BACKGROUND

System dynamics (SD) modelling can inform policy decisions under Thailand's Universal Health Coverage. We report on this thinking approach to Thailand's strategic health workforce planning for the next 20 years (2018-2037).

METHODS

A series of group model building (GMB) sessions involving 110 participants from multi-sectors of Thailand's health systems was conducted in 2017 and 2018. We facilitated policymakers, administrators, practitioners and other stakeholders to co-create a causal loop diagram (CLD) representing a shared understanding of why the health workforce's demands and supplies in Thailand were mismatched. A stock and flow diagram (SFD) was also co-created for testing the consequences of policy options by simulation modelling.

RESULTS

The simulation modelling found hospital utilisation created a vicious cycle of constantly increasing demands for hospital care and a constant shortage of healthcare providers. Moreover, hospital care was not designed for effectively dealing with the future demands of ageing populations and prevalent chronic illness. Hence, shifting emphasis to professions that can provide primary care, intermediate care, long-term care, palliative care, and end-of-life care can be more effective.

CONCLUSIONS

Our SD modelling confirmed that shifting the care models to address the changing health demands can be a high-leverage policy of health workforce planning, although very difficult to implement in the short term. of health workforce planning, although very difficult to implement in the short term.

Dengue Surveillance System in Brazil: A Qualitative Study in the Federal District

Dengue's increasing trends raise concerns over global health and pose a challenge to the Brazilian health system, highlighting the necessity of a strong surveillance system to reduce morbidity, mortality, and the economic burden of this disease. Although the Brazilian surveillance system reports more dengue cases than any other country, recent studies suggest that non-reported cases are the majority. The aim of the study is to explore the strengths and weaknesses of the Brazilian surveillance system, particularly looking at the functioning of data collection and reporting. This was done through qualitative semi-structured interviews with 17 experts in dengue surveillance, supported by quantitative data from the official notification system. To select the interviewees, purposive and theoretical sampling were used. Data were analyzed through thematic analysis. The research highlighted that a lack of human and technological resources in healthcare units and surveillance departments slows down the notification process and data analysis. Due to a lack of integration in the private sector, the surveillance system fails to detect the socioeconomic profile of the patients. Investments in public healthcare, human and technological resources for surveillance and better integration in the private healthcare system, and vector surveillance may improve dengue surveillance.

The Last Mile: Where Artificial Intelligence Meets Reality

Although much effort is focused on improving the technical performance of artificial intelligence, there are compelling reasons to focus more on the implementation of this technology class to solve real-world applications. In this “last mile” of implementation lie many complex challenges that may make technically high-performing systems perform poorly. Instead of viewing artificial intelligence development as a linear one of algorithm development through to eventual deployment, there are strong reasons to take a more agile approach, iteratively developing and testing artificial intelligence within the context in which it finally will be used.

Key Issues in the Development of an Evidence-Based Stratified Surgical Patient Safety Improvement Information System: Experience From a Multicenter Surgical Safety Program

Surgery is still far from being completely safe and reliable. Surgical safety has, therefore, been the focus of considerable attention over the last few decades, and there are a growing number of national drives to improve it. There are also a number of large surgical complication reporting systems and system-based interventions, both of which have made remarkable progress in the past two decades. These systems, however, have either mainly focused on reporting complications and played a limited role in guiding practice or have provided nonselective interventions to all patients, perhaps imposing unnecessary burdens on frontline medical staff. We have, therefore, developed an evidence-based stratified surgical safety information system based on a multicenter surgical safety improvement program. This study discusses some critical issues in the process of developing this information system, including (1) decisions about data gathering, (2) establishing and sharing knowledge, (3) developing functions for the system, (4) system implementation, and (5) evaluation and continuous improvement. Using examples drawn from the surgical safety improvement program, we have shown how this type of system can be fitted into day-to-day clinical practice and how it can guide medical practice by incorporating inherent patient-related risk and providing tailored interventions for patients with different levels of risk. We concluded that multidisciplinary collaboration, involving experts in health care (including senior staff in surgery, nursing, and anesthesia), data science, health care management, and health information technology, can help build an evidence-based stratified surgical patient safety improvement system. This can provide an information-intensified surgical safety learning platform and, therefore, benefit surgical patients by delivering tailored interventions and an integrated workflow.

Present Situation and Prospect of Medical Knowledge Based Systems in German-speaking Countries

Background: After a decrease of interest in classical medical expert systems, the publication activity concerning the medical application of Artificial Intelligence and the interest in medical decision support have markedly increased. Nonetheless, no systematic exploratory study has yet been carried out, which directly considers the actual fields of applications, exemplary approaches, obstacles, challenges, and future prospect as seen by pioneering users and developers in a given region.

Objectives: This paper reports the results of an online survey designed to fill this gap with the “Knowledge Based Systems” working group of the German Society for Medical Informatics, Biometry and Epidemiology (GMDS) in 2010.

Methods: The survey was based on an online questionnaire (5 single and multiple choice questions, 8 Likert-scaled items, 7 free text questions) consented to by the working group. The answers were analyzed by descriptive statistics and a qualitative analysis (bottom-up coding). All academic institutions of Medical Informatics in the German-speaking countries and contributors reporting KBS-related projects at the relevant scientific conferences and in a journal specialized in the field were invited to participate.

Results: The survey reached a response rate of 33.4%. The results show a gap between the reported obstacles of medical KBS (mainly low acceptance and rare use in clinical practice) and their future prospect as stated by the participants. Problems previously discussed in the literature like low acceptance, integration, and sustainability of KBS projects were confirmed. The current situation was characterized by naming exemplary existing systems and specifying promising fields of application.

Conclusions: The field of KBS in medicine is more diversified and has evolved beyond expectations in the German-speaking countries.

Health Services Research Evaluation Principles

Background: Our forthcoming national experiment in increased health information technology (HIT) adoption funded by the American Recovery and Reinvestment Act of 2009 will require a comprehensive approach to evaluating HIT. The quality of evaluation studies of HIT to date reveals a need for broader evaluation frameworks that limits the generalizability of findings and the depth of lessons learned.

Objective: Develop an informatics evaluation framework for health information technology (HIT) integrating components of health services research (HSR) evaluation and informatics evaluation to address identified shortcomings in available HIT evaluation frameworks.

Method: A systematic literature review updated and expanded the exhaustive review by Ammenwerth and deKeizer (AdK). From retained studies, criteria were elicited and organized into classes within a framework. The resulting Health Information Technology Research-based Evaluation Framework (HITREF) was used to guide clinician satisfaction survey construction, multi-dimensional analysis of data, and interpretation of findings in an evaluation of a vanguard community health care EHR.

Results: The updated review identified 128 electronic health record (EHR) evaluation studies and seven evaluation criteria not in AdK: EHR Selection/Development/Training; Patient Privacy Concerns; Unintended Consequences/ Benefits; Functionality; Patient Satisfaction with EHR; Barriers/Facilitators to Adoption; and Patient Satisfaction with Care. HITREF was used productively and was a complete evaluation framework which included all themes that emerged.

Conclusions: We can recommend to future EHR evaluators that they consider adding a complete, research-based HIT evaluation framework, such as HITREF, to their evaluation tools suite to monitor HIT challenges as the federal government strives to increase HIT adoption.

Health information systems evaluation frameworks: A systematic review

BACKGROUND

Evaluation of health information systems (HISs) is complicated because of the complex nature of the health care domain. Various studies have proposed different frameworks to reduce the complexity in the assessment of these systems. The aim of these frameworks is to provide a set of guidelines for the evaluation of the adequacy of health care information systems.

OBJECTIVE

This paper aims to analyse studies on the evaluation of HISs by applying a content, context and process (CCP) framework to address the 'who', 'what', 'how', 'when', and 'why' of the evaluation processes used. This will allow for a better understanding of the relative strengths and weaknesses of various HISs evaluation frameworks, and will pave the way for developing a more complete framework for HISs.

METHOD

A systematic literature review on HIS evaluation studies was undertaken to identify the currently available HIS evaluation frameworks. Five academic databases were selected to conduct this systematic literature review.

RESULTS

Most of the studies only address some, but not all, of the five main questions, i.e. the who, what, how, when, why, and that there was a lack of consensus in the way these questions were addressed. The critical role of context was also largely neglected in these studies.

CONCLUSIONS

Evaluation of HISs is complex. The health care domain is highly context sensitive and in order to have a complete assessment of HISs, consideration of contextual factors is necessary. Specifically, to have the right set of criteria to measure the 'what', the answer to the 'who' of the evaluation is necessary.

Crossing the Quality Chasm: The Role of Information Technology Departments

Integrating information technology (IT) into medical settings is considered essential to transforming hospitals into 21st-century health care institutions. Yet the role of IT departments in maximizing the effectiveness of information systems is not well understood. This article reports a 3-round Delphi panel of Veterans Administration personnel experienced with provider order entry electronic systems. In round 1, 35 administrative, clinical, and IT personnel answered 10 open-ended questions about IT strategies and structures that best support successful transformation. In round 2, panelists rated item importance and ranked proposed strategies. In round 3, panelists received aggregate feedback and rerated the items. Four domains emerged from round 1: IT organization, IT performance monitoring, user-support activities, and core IT responsibilities (eg, computer security, training). In rounds 2 and 3, IT performance monitoring was rated the most important, closely followed by clinical support. Strategies associated with each domain are identified and discussed.

Determining the Most Important Evaluation Indicators of Healthcare Information Systems (HCIS) in Iran

Accurate evaluation of healthcare information systems (HCIS) relies upon the choice of appropriate indicators. Iranian healthcare and health industry professionals were surveyed, by means of a descriptive cross sectional study, in order to identify the indicators they considered most relevant to the evaluation of healthcare information systems currently in use in Iran. It was concluded that effective evaluation of HCIS should encompass a variety of perspectives and methodologies (including qualitative methodologies), focus upon technical, economic and organisational concerns, and involve diversely constituted research teams.

Review of Social and Organizational Issues in Health Information Technology

Objectives

This paper reviews organizational and social issues (OSIs) in health information technology (HIT).

Methods

A review and synthesis of the literature on OSIs in HIT was conducted.

Results

Five overarching themes with respect to OSIs in HIT were identified and discussed: scope and frameworks for defining OSIs in HIT, context matters, process immaturity and complexity, trade-offs will happen and need to be discussed openly, and means of studying OSIs in HIT.

Conclusions

There is a wide body of literature that provides insight into OSIs in HIT, even if many of the studies are not explicitly labelled as such. The two biggest research needs are more explicit and theoretical studies of OSI in HITs and more research on integrating micro and macro perspectives of HIT use in organizations.

From expert-derived user needs to user-perceived ease of use and usefulness: a two-phase mixed-methods evaluation framework

Underspecified user needs and frequent lack of a gold standard reference are typical barriers to technology evaluation. To address this problem, this paper presents a two-phase evaluation framework involving usability experts (phase 1) and end-users (phase 2). In phase 1, a cross-system functionality alignment between expert-derived user needs and system functions was performed to inform the choice of "the best available" comparison system to enable a cognitive walkthrough in phase 1 and a comparative effectiveness evaluation in phase 2. During phase 2, five quantitative and qualitative evaluation methods are mixed to assess usability: time-motion analysis, software log, questionnaires - System Usability Scale and the Unified Theory of Acceptance of Use of Technology, think-aloud protocols, and unstructured interviews. Each method contributes data for a unique measure (e.g., time motion analysis contributes task-completion-time; software log contributes action transition frequency). The measures are triangulated to yield complementary insights regarding user-perceived ease-of-use, functionality integration, anxiety during use, and workflow impact. To illustrate its use, we applied this framework in a formative evaluation of a software called Integrated Model for Patient Care and Clinical Trials (IMPACT). We conclude that this mixed-methods evaluation framework enables an integrated assessment of user needs satisfaction and user-perceived usefulness and usability of a novel design. This evaluation framework effectively bridges the gap between co-evolving user needs and technology designs during iterative prototyping and is particularly useful when it is difficult for users to articulate their needs for technology support due to the lack of a baseline.

Understanding critical barriers to implementing a clinical information system in a nursing home through the lens of a socio-technical perspective

This paper addresses key barriers to implementing a clinical information system (CIS) in a Hong Kong nursing home setting, from a healthcare specific socio-technical perspective. Data was collected through field observations (n = 12) and semi-structured individual interviews (n = 18) of CIS stakeholders in a Hong Kong nursing home, and analyzed using the immersion/crystallization approach. Complex interactions relevant to our case were contextualized and interpreted within the perspective of the Sittig-Singh Healthcare Socio-Technical Framework (HSTF). Three broad clusters of implementation barriers from the eight HSTF dimensions were identified: (a) Infrastructure-based barriers, which relate to conflict between government regulations and system functional needs of users; lack of financial support; inconsistency between workflow, work policy, and procedures; and inadequacy of hardware-software infrastructural and technical support; (b) Process-based barriers, which relate to mismatch between the technology, existing work practice and workflow, and communication; low system speed, accessibility, and stability; deficient computer literacy; more experience in health care profession; clinical content inadequacy and unavailability; as well as poor system usefulness and user interface design; and (c) Outcome-based barriers, which relate to the lack of measurement and monitoring of system effectiveness. Two additional dimensions underlining the importance of the ability of a CIS to change are proposed to extend the Sittig-Singh HSTF. First, advocacy would promote the articulation and influence of changes in the system and subsequent outcomes by CIS stakeholders, and second, adaptability would ensure the ability of the system to adjust to emerging needs. The broad set of discovered implementation shortcomings expands prior research on why CIS can fail in nursing home settings. Moreover, our investigation offers a knowledge base and recommendations that can serve as a guide for future implementation strategies and policies in CIS initiatives.

The SAMS: Smartphone Addiction Management System and verification

While the popularity of smartphones has given enormous convenience to our lives, their pathological use has created a new mental health concern among the community. Hence, intensive research is being conducted on the etiology and treatment of the condition. However, the traditional clinical approach based surveys and interviews has serious limitations: health professionals cannot perform continual assessment and intervention for the affected group and the subjectivity of assessment is questionable. To cope with these limitations, a comprehensive ICT (Information and Communications Technology) system called SAMS (Smartphone Addiction Management System) is developed for objective assessment and intervention. The SAMS system consists of an Android smartphone application and a web application server. The SAMS client monitors the user's application usage together with GPS location and Internet access location, and transmits the data to the SAMS server. The SAMS server stores the usage data and performs key statistical data analysis and usage intervention according to the clinicians' decision. To verify the reliability and efficacy of the developed system, a comparison study with survey-based screening with the K-SAS (Korean Smartphone Addiction Scale) as well as self-field trials is performed. The comparison study is done using usage data from 14 users who are 19 to 50 year old adults that left at least 1 week usage logs and completed the survey questionnaires. The field trial fully verified the accuracy of the time, location, and Internet access information in the usage measurement and the reliability of the system operation over more than 2 weeks. The comparison study showed that daily use count has a strong correlation with K-SAS scores, whereas daily use times do not strongly correlate for potentially addicted users. The correlation coefficients of count and times with total K-SAS score are CC = 0.62 and CC =0.07, respectively, and the t-test analysis for the contrast group of potential addicts and the values for the non-addicts were p = 0.047 and p = 0.507, respectively.

We are bitter, but we are better off: case study of the implementation of an electronic health record system into a mental health hospital in England

BACKGROUND

In contrast to the acute hospital sector, there have been relatively few implementations of integrated electronic health record (EHR) systems into specialist mental health settings. The National Programme for Information Technology (NPfIT) in England was the most expensive IT-based transformation of public services ever undertaken, which aimed amongst other things, to implement integrated EHR systems into mental health hospitals. This paper describes the arrival, the process of implementation, stakeholders' experiences and the local consequences of the implementation of an EHR system into a mental health hospital.

METHODS

Longitudinal, real-time, case study-based evaluation of the implementation and adoption of an EHR software (RiO) into an English mental health hospital known here as Beta. We conducted 48 in-depth interviews with a wide range of internal and external stakeholders, undertook 26 hours of on-site observations, and obtained 65 sets of relevant documents from various types relating to Beta. Analysis was both inductive and deductive, the latter being informed by the 'sociotechnical changing' theoretical framework.

RESULTS

Many interviewees perceived the implementation of the EHR system as challenging and cumbersome. During the early stages of the implementation, some clinicians felt that using the software was time-consuming leading to the conclusion that the EHR was not fit for purpose. Most interviewees considered the chain of deployment of the EHR-which was imposed by NPfIT-as bureaucratic and obstructive, which restricted customization and as a result limited adoption and use. The low IT literacy among users at Beta was a further barrier to the implementation of the EHR. This along with inadequate training in using the EHR software led to resistance to the significant cultural and work environment changes initiated by EHR. Despite the many challenges, Beta achieved some early positive results. These included: the ability to check progress notes and monitor staff activities; improving quality of care as a result of real-time, more accurate and shared patient records across the hospital; and potentially improving the safety of care through increasing the legibility of the clinical record.

CONCLUSIONS

Notwithstanding what was seen as a turbulent, painful and troublesome implementation of the EHR system, Beta achieved some early clinical and managerial benefits from implementing EHRs. The 'sociotechnical changing' framework helped us go beyond the dichotomy of success versus failure, when conducting the evaluation and interpreting findings. Given the scope for continued development, there are good reasons, we argue, to scale up the intake of EHR systems by mental health care settings. Software customization and appropriate support are essential to work EHR out in such organizations.

Improving the effectiveness of electronic health record-based referral processes

Electronic health records are increasingly being used to facilitate referral communication in the outpatient setting. However, despite support by technology, referral communication between primary care providers and specialists is often unsatisfactory and is unable to eliminate care delays. This may be in part due to lack of attention to how information and communication technology fits within the social environment of health care. Making electronic referral communication effective requires a multifaceted “socio-technical” approach. Using an 8-dimensional socio-technical model for health information technology as a framework, we describe ten recommendations that represent good clinical practices to design, develop, implement, improve, and monitor electronic referral communication in the outpatient setting. These recommendations were developed on the basis of our previous work, current literature, sound clinical practice, and a systems-based approach to understanding and implementing health information technology solutions. Recommendations are relevant to system designers, practicing clinicians, and other stakeholders considering use of electronic health records to support referral communication.

Boundaries and e-health implementation in health and social care

Background

The major problem facing health and social care systems globally today is the growing challenge of an elderly population with complex health and social care needs. A longstanding challenge to the provision of high quality, effectively coordinated care for those with complex needs has been the historical separation of health and social care. Access to timely and accurate data about patients and their treatments has the potential to deliver better care at less cost.

Methods

To explore the way in which structural, professional and geographical boundaries have affected e-health implementation in health and social care, through an empirical study of the implementation of an electronic version of Single Shared Assessment (SSA) in Scotland, using three retrospective, qualitative case studies in three different health board locations.

Results

Progress in effectively sharing electronic data had been slow and uneven. One cause was the presence of established structural boundaries, which lead to competing priorities, incompatible IT systems and infrastructure, and poor cooperation. A second cause was the presence of established professional boundaries, which affect staffs’ understanding and acceptance of data sharing and their information requirements. Geographical boundaries featured but less prominently and contrasting perspectives were found with regard to issues such as co-location of health and social care professionals.

Conclusions

To provide holistic care to those with complex health and social care needs, it is essential that we develop integrated approaches to care delivery. Successful integration needs practices such as good project management and governance, ensuring system interoperability, leadership, good training and support, together with clear efforts to improve working relations across professional boundaries and communication of a clear project vision. This study shows that while technological developments make integration possible, long-standing boundaries constitute substantial risks to IT implementations across the health and social care interface which those initiating major changes would do well to consider before committing to the investment.

Video observation in HIT development: lessons learned on benefits and challenges

BACKGROUND

Experience shows that the precondition for the development of successful health information technologies is a thorough insight into clinical work practice. In contemporary clinical work practice, clinical work and health information technology are integrated, and part of the practice is tacit. When work practice becomes routine, it slips to the background of the conscious awareness and becomes difficult to recognize without the context to support recall. This means that it is difficult to capture with traditional ethnographic research methods or in usability laboratories or clinical set ups. Observation by the use of the video technique within healthcare settings has proven to be capable of providing a thorough insight into the complex clinical work practice and its context - including parts of the tacit practice. The objective of this paper is 1) to argue for the video observation technique to inform and improve health-information-technology development and 2) to share insights and lessons learned on benefits and challenges when using the video observation technique within healthcare settings.

METHODS

A multiple case study including nine case studies conducted by DaCHI researchers 2004-2011 using audio-visual, non-participant video observation for data collection within different healthcare settings.

RESULTS

In HIT development, video observation is beneficial for 1) informing and improving system design 2) studying changes in work practice 3) identifying new potentials and 4) documenting current work practices.

CONCLUSIONS

The video observation technique used within healthcare settings is superior to other ethnographic research methods when it comes to disclosing the complexity in clinical work practice. The insights gained are far more realistic compared to traditional ethnographic studies or usability studies and studies in clinical set ups. Besides, the data generated through video recordings provide a solid basis for dialog between the health care professionals involved. The most important lessons learned are that a well considered methodology and clear formulated objectives are imperative, in order to stay focused during the data rich analysis phase. Additionally, the video observation technique is primarily recommended for studies of specific clinical work practices within delimited clinical settings. Overall, the video observation technique has proven to be capable of improving our understanding of the interwoven relation between clinical work practice and HIT and to inform us about user requirements and needs for HIT, which is a precondition for the development of more successful HIT systems in the future.

An empirically-derived approach for investigating Health Information Technology: the Elementally Entangled Organisational Communication (EEOC) framework

BACKGROUND

The purpose of this paper is to illustrate the Elementally Entangled Organisational Communication (EEOC) framework by drawing on a set of three case studies which assessed the impact of new Health Information Technology (HIT) on a pathology service. The EEOC framework was empirically developed as a tool to tackle organisational communication challenges in the implementation and evaluation of health information systems.

METHODS

The framework was synthesised from multiple research studies undertaken across a major metropolitan hospital pathology service during the period 2005 to 2008. These studies evaluated the impact of new HIT systems in pathology departments (Laboratory Information System) and an Emergency Department (Computerised Provider Order Entry) located in Sydney, Australia.

RESULTS

Key dimensions of EEOC are illustrated by the following case studies: 1) the communication infrastructure between the Blood Bank and the ward for the coordination and distribution of blood products; 2) the organisational environment in the Clinical Chemistry and Haematology departments and their attempts to organise, plan and control the processing of laboratory specimens; and 3) the temporal make up of the organisation as revealed in changes to the way the Central Specimen Reception allocated, sequenced and synchronised work tasks.

CONCLUSIONS

The case studies not only highlight the pre-existing communication architecture within the organisation but also the constitutive role communication plays in the way organisations go about addressing their requirements. HIT implementation involves a mutual transformation of the organisation and the technology. This is a vital consideration because of the dangers associated with poor organisational planning and implementation of HIT, and the potential for unintended adverse consequences, workarounds and risks to the quality and safety of patient care. The EEOC framework aims to account for the complex range of contextual factors and triggers that play a role in the success or otherwise of new HITs, and in the realisation of their innovation potential.

Cancer patients' experiences of using an Interactive Health Communication Application (IHCA)

Interactive Health Communication Applications (IHCAs) are increasingly used in health care. Studies document that IHCAs provide patients with knowledge and social support, enhance self- efficacy and can improve behavioural and clinical outcomes. However, research exploring patients' experiences of using IHCAs has been scarce. The aim of this study was to explore cancer patients' perspectives and experiences related to the use of an IHCA called WebChoice in their homes. Qualitative interviews were conducted with infrequent, medium and frequent IHCA users-six women and four men with breast and prostate cancer. The interviews were transcribed and analyzed inspired by interactionistic perspectives. We found that some patients' perceived WebChoice as a "friend," others as a "stranger." Access to WebChoice stimulated particularly high frequency users to position themselves as "information seeking agents," assuming an active patient role. However, to position oneself as an "active patient" was ambiguous and emotional. Feelings of "calmness", "normalization of symptoms", feelings of "being part of a community", feeling "upset" and "vulnerable", as well as "feeling supported" were identified. Interaction with WebChoice implied for some users an increased focus on illness. Our findings indicate that the interaction between patients and an IHCA such as WebChoice occurs in a variety of ways, some of which are ambivalent or conflicting. Particularly for frequent and medium frequency users, it offers support, but may at the same time reinforce an element of uncertainty in their life. Such insights should be taken into consideration in the future development of IHCAs in healthcare in general and in particular for implementation into patients' private sphere.

Building a house on shifting sand: methodological considerations when evaluating the implementation and adoption of national electronic health record systems

BACKGROUND

A commitment to Electronic Health Record (EHR) systems now constitutes a core part of many governments' healthcare reform strategies. The resulting politically-initiated large-scale or national EHR endeavors are challenging because of their ambitious agendas of change, the scale of resources needed to make them work, the (relatively) short timescales set, and the large number of stakeholders involved, all of whom pursue somewhat different interests. These initiatives need to be evaluated to establish if they improve care and represent value for money.

METHODS

Critical reflections on these complexities in the light of experience of undertaking the first national, longitudinal, and sociotechnical evaluation of the implementation and adoption of England's National Health Service's Care Records Service (NHS CRS).

RESULTS/DISCUSSION

We advance two key arguments. First, national programs for EHR implementations are likely to take place in the shifting sands of evolving sociopolitical and sociotechnical and contexts, which are likely to shape them in significant ways. This poses challenges to conventional evaluation approaches which draw on a model of baseline operations → intervention → changed operations (outcome). Second, evaluation of such programs must account for this changing context by adapting to it. This requires careful and creative choice of ontological, epistemological and methodological assumptions.

SUMMARY

New and significant challenges are faced in evaluating national EHR implementation endeavors. Based on experiences from this national evaluation of the implementation and adoption of the NHS CRS in England, we argue for an approach to these evaluations which moves away from seeing EHR systems as Information and Communication Technologies (ICT) projects requiring an essentially outcome-centred assessment towards a more interpretive approach that reflects the situated and evolving nature of EHR seen within multiple specific settings and reflecting a constantly changing milieu of policies, strategies and software, with constant interactions across such boundaries.

Radiation oncology information systems and clinical practice compatibility: Workflow evaluation and comprehensive assessment

PURPOSE

To map the level of clinical practice compatibility with a radiation oncology information system (ROIS) through a workflow- and clinical process-based method aimed at optimizing the safety, efficacy, and efficiency of patient care; to improve the understanding of the critical relationship between the clinical practice and ROIS.

METHODS AND MATERIALS

Clinic-specific workflow and infrastructure were classified into clinical processes, information management, and technological innovation integration. Clinical information systems-information technology infrastructure and process maps were generated by a team of experts, representing clinical constituents. These maps served as the basis for evaluating connectivity and process flow and to guide the development of a quantitative survey where all clinical tasks and subprocesses were ranked according to importance in patient care and scored by the team of experts for performance. Process maps and survey output were used to measure ROIS compatibility with the practice and to guide practice improvement.

RESULTS

Practice-specific process and infrastructure maps were generated. The developed survey was applied and results indicate a range of ROIS compatibility with clinical workflow and infrastructure. Survey results combined with experiential feedback provided specific prioritized guidance to improve both ROIS performance and clinic-specific processes and infrastructure.

CONCLUSIONS

This work provides a systematic and customizable tool to understand and evaluate clinical information and workflow and its compatibility with a given ROIS. The analysis provides insight into workflow improvements and information systems and information technology infrastructure limitations. Participating in such a process provides the entire team with a deeper understanding of the critical relationship between the clinical practice and the ROIS.

The e-Health Implementation Toolkit: qualitative evaluation across four European countries

BACKGROUND

Implementation researchers have attempted to overcome the research-practice gap in e-health by developing tools that summarize and synthesize research evidence of factors that impede or facilitate implementation of innovation in healthcare settings. The e-Health Implementation Toolkit (e-HIT) is an example of such a tool that was designed within the context of the United Kingdom National Health Service to promote implementation of e-health services. Its utility in international settings is unknown.

METHODS

We conducted a qualitative evaluation of the e-HIT in use across four countries--Finland, Norway, Scotland, and Sweden. Data were generated using a combination of interview approaches (n = 22) to document e-HIT users' experiences of the tool to guide decision making about the selection of e-health pilot services and to monitor their progress over time.

RESULTS

e-HIT users evaluated the tool positively in terms of its scope to organize and enhance their critical thinking about their implementation work and, importantly, to facilitate discussion between those involved in that work. It was easy to use in either its paper- or web-based format, and its visual elements were positively received. There were some minor criticisms of the e-HIT with some suggestions for content changes and comments about its design as a generic tool (rather than specific to sites and e-health services). However, overall, e-HIT users considered it to be a highly workable tool that they found useful, which they would use again, and which they would recommend to other e-health implementers.

CONCLUSION

The use of the e-HIT is feasible and acceptable in a range of international contexts by a range of professionals for a range of different e-health systems.

Biomedical informatics: changing what physicians need to know and how they learn

The explosive growth of biomedical complexity calls for a shift in the paradigm of medical decision making-from a focus on the power of an individual brain to the collective power of systems of brains. This shift alters professional roles and requires biomedical informatics and information technology (IT) infrastructure. The authors illustrate this future role of medical informatics with a vignette and summarize the evolving understanding of both beneficial and deleterious effects of informatics-rich environments on learning, clinical care, and research. The authors also provide a framework of core informatics competencies for health professionals of the future and conclude with broad steps for faculty development. They recommend that medical schools advance on four fronts to prepare their faculty to teach in a biomedical informatics-rich world: (1) create academic units in biomedical informatics; (2) adapt the IT infrastructure of academic health centers (AHCs) into testing laboratories; (3) introduce medical educators to biomedical informatics sufficiently for them to model its use; and (4) retrain AHC faculty to lead the transformation to health care based on a new systems approach enabled by biomedical informatics. The authors propose that embracing this collective and informatics-enhanced future of medicine will provide opportunities to advance education, patient care, and biomedical science.

A Sociotechnical Approach to Evaluating the Impact of ICT on Clinical Care Environments

INTRODUCTION

Process-supporting information technology holds the potential to increase efficiency, reduce errors, and alter professional roles and responsibilities in a manner which allows improvement in the delivery of patient care. However, clashes between the model of health care work inscribed in these tools with the actual nature of work has resulted in staff resistance and decreased organisational uptake of ICT, as well as the facilitation of unexpected and negative effects in efficiency and patient safety. Sociotechnical theory provides a paradigm against which workflow and transfusion of ICT in healthcare could be better explored and understood.

DESIGN

This paper will conceptualise a formative, multi-method longitudinal evaluation process to explore the impact of ICT with an appreciation of the relationship between the social and technical systems within a clinical department.

METHOD

Departmental culture, including clinical work processes and communication patterns will be thoroughly explored before system implementation using both quantitative and qualitative research methods. Findings will be compared with post implementation data, which will incorporate measurement of safety and workflow efficiency indicators.

DISCUSSION

Sociotechnical theory provides a paradigm against which workflow and transfusion of ICT in healthcare could be better explored and understood. However, sociotechnical and multimethod approaches to evaluation do not exist without criticism. Inherent in the protocol are limitations of sociotechnical theory and criticism of the multimethod approach; testing of the methodology in real clinical settings will serve to verify efficacy and refine the process.

Organizational challenges in developing one of the Nationwide Health Information Network trial implementation awardees

Health care in the United States is rarely delivered in a coordinated manner. Current methods to share patient information are inefficient and may lead to medical errors, higher readmission rates, and delays in the delivery of needed health services. This qualitative study describes lessons learned concerning the early implementation of one Nationwide Health Information Network (NHIN) site in Long Beach, CA during its first year of operation. The Long Beach Network for Health (LBNH) focused on an incremental effort to exchange health information. Despite a limited concentration on emergency department care, virtually all respondents noted concerns regarding the sustainability, or business case, for the exchange of health information. Nevertheless, respondents were encouraged by progress on technological challenges and user requirements during this first year. The early gains in this process may, in turn, have laid the groundwork for future efforts to expand beyond the emergency department.

An informatics model for guiding assembly of telemicrobiology workstations for malaria collaborative diagnostics using commodity products and open-source software

BACKGROUND

Deficits in clinical microbiology infrastructure exacerbate global infectious disease burdens. This paper examines how commodity computation, communication, and measurement products combined with open-source analysis and communication applications can be incorporated into laboratory medicine microbiology protocols. Those commodity components are all now sourceable globally. An informatics model is presented for guiding the use of low-cost commodity components and free software in the assembly of clinically useful and usable telemicrobiology workstations.

METHODS

The model incorporates two general principles: 1) collaborative diagnostics, where free and open communication and networking applications are used to link distributed collaborators for reciprocal assistance in organizing and interpreting digital diagnostic data; and 2) commodity engineering, which leverages globally available consumer electronics and open-source informatics applications, to build generic open systems that measure needed information in ways substantially equivalent to more complex proprietary systems. Routine microscopic examination of Giemsa and fluorescently stained blood smears for diagnosing malaria is used as an example to validate the model.

RESULTS

The model is used as a constraint-based guide for the design, assembly, and testing of a functioning, open, and commoditized telemicroscopy system that supports distributed acquisition, exploration, analysis, interpretation, and reporting of digital microscopy images of stained malarial blood smears while also supporting remote diagnostic tracking, quality assessment and diagnostic process development.

CONCLUSION

The open telemicroscopy workstation design and use-process described here can address clinical microbiology infrastructure deficits in an economically sound and sustainable manner. It can boost capacity to deal with comprehensive measurement of disease and care outcomes in individuals and groups in a distributed and collaborative fashion. The workstation enables local control over the creation and use of diagnostic data, while allowing for remote collaborative support of diagnostic data interpretation and tracking. It can enable global pooling of malaria disease information and the development of open, participatory, and adaptable laboratory medicine practices. The informatic model highlights how the larger issue of access to generic commoditized measurement, information processing, and communication technology in both high- and low-income countries can enable diagnostic services that are much less expensive, but substantially equivalent to those currently in use in high-income countries.

Telemedicine in interdisciplinary work practices: on an IT system that met the criteria for success set out by its sponsors, yet failed to become part of every-day clinical routines

Background

Information systems can play a key role in care innovations including task redesign and shared care. Many demonstration projects have presented evidence of clinical and cost effectiveness and high levels of patient satisfaction. Yet these same projects often fail to become part of everyday clinical routines. The aim of the paper is to gain insight into a common paradox that a technology can meet the criteria for success set out at the start of the project yet fail to become part of everyday clinical routines.

Methods

We evaluated a telecare service set up to reduce the workload of ophthalmologists. In this project, optometrists in 10 optical shops made digital images to detect patients with glaucoma which were further assessed by trained technicians in the hospital. Over a period of three years, we conducted interviews with the project team and the users about the workability of the system and its integration in practice. Beside the interviews, we analyzed record data to measure the quality of the images. We compared the qualitative accounts with these measurements.

Results

According to our measurements, the quality of the images was at least satisfactory in 90% of the cases, i.e. the images could be used to screen the patients – reducing the workload of the ophthalmologist considerably. However, both the ophthalmologist and the optometrists became increasingly dissatisfied respectively with the perceived quality of the pictures and the perceived workload.

Through a detailed analysis of how the professionals discussed the quality of the pictures, we re-constructed how the notion of quality of the images and being a good professional were constructed and linked. The IT system transformed into a quality system and, at the same time, transformed the notions of being a good professional. While a continuous dialogue about the quality of the pictures became an emblem for the quality of care, this dialogue was hindered by the system and the way the care process was structured.

Conclusion

To conceptualize what telemedicine does in interdisciplinary work practices, a fine-tuned analysis is needed to assess how IT systems re-shape the social relations between professional groups. Such transformations should not be exclusively attributed to the technology itself or to the professionals working with it. Instead we need to assess these technologies through an empirically grounded study of the sociotechnical functioning of telemedicine.

Hospital characteristics associated with highly automated and usable clinical information systems in Texas, United States

Background

A hospital's clinical information system may require a specific environment in which to flourish. This environment is not yet well defined. We examined whether specific hospital characteristics are associated with highly automated and usable clinical information systems.

Methods

This was a cross-sectional survey of 125 urban hospitals in Texas, United States using the Clinical Information Technology Assessment Tool (CITAT), which measures a hospital's level of automation based on physician interactions with the information system. Physician responses were used to calculate a series of CITAT scores: automation and usability scores, four automation sub-domain scores, and an overall clinical information technology (CIT) score. A multivariable regression analysis was used to examine the relation between hospital characteristics and CITAT scores.

Results

We received a sufficient number of physician responses at 69 hospitals (55% response rate). Teaching hospitals, hospitals with higher IT operating expenses (>$1 million annually), IT capital expenses (>$75,000 annually) and hospitals with larger IT staff (≥ 10 full-time staff) had higher automation scores than hospitals that did not meet these criteria (p < 0.05 in all cases). These findings held after adjustment for bed size, total margin, and ownership (p < 0.05 in all cases). There were few significant associations between the hospital characteristics tested in this study and usability scores.

Conclusion

Academic affiliation and larger IT operating, capital, and staff budgets are associated with more highly automated clinical information systems.

Expanding multi-disciplinary approaches to healthcare information technologies: what does information systems offer medical informatics?

The effective use of information technology (IT) is a crucial component for the delivery of effective services in health care. Current approaches to medical informatics (MI) research have significantly contributed to the success of IT use in health care but important challenges remain to be addressed. We believe that expanding the multi-disciplinary basis for MI research is important to meeting these research challenges. In this paper, we outline theories and methods used in information systems (IS) research that we believe can inform our understanding of health care IT applications and outcomes. To do so, we discuss some general differences in the focus and methods of MI and IS research to identify broad opportunities. We then review conceptual and methodological approaches in IS that have been applied in health care IT research. These include: technology-use mediation, collaborative work, genre theory, interpretive research, action research, and modeling. Examples of these theories and methods in healthcare IS research are illustrated.

Investigating evaluation frameworks for health information systems

BACKGROUND AND PURPOSE

Evaluation of health information systems (HIS) enables the assessment of the extent to which HIS are fulfilling their objectives in supporting the services of healthcare delivery. This paper presents an overview of evaluation in health informatics and information systems.

METHODS

Literature review on discourses, dimensions and methods of HIS and IS evaluation. A critical appraisal of selected HIS and IS evaluation frameworks is undertaken in order to identify HIS evaluation dimensions and measures. The frameworks are compared based on their inclusion of human, organizational and technological factors.

RESULTS

We found that an increasing number of evaluation studies deal with two distinct trends of HIS: one considers human and organizational issues and the other is concerned with the employment of a subjectivist approach. Our review indicates that current evaluation methods complement each other in that they evaluate different aspects of HIS and they can be improved upon.

CONCLUSIONS

Evaluation is complex; it is easy to measure many things but not necessarily the right ones. Nevertheless, it is possible to consider, a HIS evaluation framework with more comprehensive and specific measures that would incorporate technological, human and organizational issues to facilitate HIS evaluation.

Contextual implementation model: a framework for assisting clinical information system implementations

OBJECTIVE

This paper presents a multiple perspectives model of clinical information system implementation, the CONTEXTual Implementation Model (CIM). Although other implementation models have been developed, few are grounded in data and others fail to take adequate account of the clinical environment and users' requirements.

DESIGN

The CIM arose from qualitative data collected from four clinical units in two large Australian teaching hospitals. The aim of the study was to explore physicians' test management work practices associated with the compulsory use of a hospital-wide, mandatory computerized provider order entry (CPOE) system.(1) The dataset consisted of non-participatory observations of physicians using CPOE (n=55 sessions) and interviews with health professionals (n=28) about test management work practices. Data were analyzed by two researchers independently using an iterative grounded approach.

RESULTS

A core underlying theme of 'contextual differences' emerged which explained physicians' use of the CPOE system in the sites. The CIM focuses attention on diversity at three contextual levels: the organizational level; the clinical or departmental level, and the individual level. Within each of these levels there are dimensions for consideration (for example, organizational culture, leadership and diverse ways of working) which affect physicians' attitudes to, and use of, CPOE.

CONCLUSION

The CIM provides a contextual differences perspective which can be used to facilitate the implementation of clinical information systems. Developing a clinical information system implementation model serves as a framework to guide future implementations to ensure their safe and efficient use and also improve the likelihood of uptake by physicians.

Multiple perspectives on the impact of electronic ordering on hospital organisational and communication processes

Electronic ordering systems provide many potential benefits for improving the efficiency and effectiveness of healthcare delivery. They also have major implications for organisational and communication processes within hospitals. We undertook a qualitative study using focus groups and interviews with doctors, nurses, IT managers, and pathology laboratory managers to investigate the impact of the system on their work processes and relations within a major teaching hospital. This study revealed that the new electronic ordering system involved major alterations to the information management processes within the hospital, which in turn affected communication processes and work relations.

Completeness of medical records in emergency trauma care and an IT-based strategy for improvement

The medical trauma record, produced in the Accident & Emergency Departments (AEDs) receives much attention from both health-care professionals and parties interested in quality of care. While it is an important data source for health-care professionals in their everyday work, and for quality assessment by third parties, the (paper) medical record is usually negatively evaluated because of incompleteness. In this article, we show that completeness is relative to the purpose for which the record is used. We distinguish two contexts in which the trauma record is used: the primary-care process at the AED, and assessment and monitoring of trauma care. Incompleteness of the medical record is valued differently in these contexts. Especially with regard to the information demands of quality assessment, and more specifically the national trauma registry, the work processes in the AED have not evolved sufficiently as yet. Information technology has great power to improve completeness and to facilitate quality assessment, but it cannot solve the problem of incompleteness in itself. One solution we propose is to restructure the recording process by introducing a clerk. This clerk could also be a nurse or physician who is temporarily released from direct patient care.

A cognitive task analysis of information management strategies in a computerized provider order entry environment

OBJECTIVE

Computerized Provider Order Entry (CPOE) with electronic documentation, and computerized decision support dramatically changes the information environment of the practicing clinician. Prior work patterns based on paper, verbal exchange, and manual methods are replaced with automated, computerized, and potentially less flexible systems. The objective of this study is to explore the information management strategies that clinicians use in the process of adapting to a CPOE system using cognitive task analysis techniques.

DESIGN

Observation and semi-structured interviews were conducted with 88 primary-care clinicians at 10 Veterans Administration Medical Centers.

MEASUREMENTS

Interviews were taped, transcribed, and extensively analyzed to identify key information management goals, strategies, and tasks. Tasks were aggregated into groups, common components across tasks were clarified, and underlying goals and strategies identified.

RESULTS

Nearly half of the identified tasks were not fully supported by the available technology. Six core components of tasks were identified. Four meta-cognitive information management goals emerged: 1) Relevance Screening; 2) Ensuring Accuracy; 3) Minimizing memory load; and 4) Negotiating Responsibility. Strategies used to support these goals are presented.

CONCLUSION

Users develop a wide array of information management strategies that allow them to successfully adapt to new technology. Supporting the ability of users to develop adaptive strategies to support meta-cognitive goals is a key component of a successful system.