Contextual dynamics during health information systems implementation: an event-based actor-network approach

Comparison of usability evaluation methods for a health information system: heuristic evaluation versus cognitive walkthrough method

Background

There are differences of opinion regarding the selection of the most practical usability evaluation method among different methods. The present study aimed to compare two expert-based evaluation methods in order to assess a nursing module as the most widely used module of a Hospital Information System (HIS).

Methods

Five independent evaluators used the Heuristic Evaluation (HE) and Cognitive Walkthrough (CW) methods to evaluate the nursing module of Shafa HIS. In this regard, the number and severity of the recognized problems according to the usability attributes were compared using two evaluation methods.

Results

The HE and CW evaluation methods resulted in the identification of 104 and 24 unique problems, respectively, of which 33.3% of recognized problems in the CW evaluation method overlapped with the HE method. The average severity of the recognized problems was considered to be minor (2.34) in the HE method and major (2.77) in the CW evaluation method. There was a significant difference in terms of the total number and average severity of the recognized problems by these methods (P < 0.001). Based on the usability attribute, the HE method identified a larger number of problems concerning all usability attributes, and a significant difference was observed in terms of the number of recognized problems in both methods for all attributes except ‘memorability’. Also, there was a significant difference between the two methods based on the average severity of recognized problems only in terms of ‘learnability’.

Conclusion

The HE method identified more problems with lower average severity while the CW was able to recognize fewer problems with higher average severity. Regarding the evaluation goal, the HE method was able to be used to improve the effectiveness and satisfaction of the HIS. Furthermore, the CW evaluation method is recommended to identify usability problems with the highest average severity, especially in terms of ‘learnability’.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-022-01905-7.

Determinants of dynamic process of emerging ICT adoption in SMEs – actor network theory perspective

Purpose

While traditional theories of information and communication technology (ICT) adoption have been deployed to study ICT adoption in the past, these theories considers ICT adoption as static and rely on conceptualising factors as variables and predicting their levels of outcome at a single decision stage. Although much has been credited to these theories, they have continuously ignored the fact that as decisions to adopt are made and challenged along the adoption process, such decision can be influenced by the same, different or combination of factors at different stages of the adoption process. This paper aims to examine ICT adoption from a dynamic process perspective and to explore the key determinants and how these differ from one stage of the adoption process to another.

Design/methodology/approach

Qualitative method was adopted in this study. Both unstructured and semi-structured interviews were conducted in two separate stages using purposeful random sampling. Hybrid approach of thematic analysis was adopted in analysing the data.

Findings

This study develops a framework informed by actor network theory (ANT) concepts and finds that using ANT to examine the process of adoption helps to unveil the recursive nature of the process. The study reveals that the 14 determinants identified in this study influenced adoption at different stages. Factors such as ease of use, managerial time, customer focus and adoption influenced adoption at all the stages. This reveals that factors influencing ICT adoption are not static at one particular stage rather it may influence ICT adoption at different stages.

Research limitations/implications

Qualitative research is often subjective and interpretive in nature, and one of the limitations is the sample used in this research. Considering the small number of interviews carried out in this study, the generalisation of finding and the framework remains to be established across a wider population. Therefore, the factors presented could be limited considering that a number of practitioners that are involved in establishing emerging ICT are numerous. Therefore, other factors are prevailing to other industries or sectors that may provide scholars another way of examining these factors. Also, the framework demonstrates that it is a valuable analytical tool for researchers to examine how and why different actors including small and medium enterprise (SME) managers act around emerging ICT.

Originality/value

This study develops a framework that revealed the interactive and recursive nature of ICT adoption and the determinants influencing the process of ICT adoption at different stages thereby advancing ICT adoption research. The study challenges researchers to always consider ICT adoption as dynamic and unpredictable instead of one-off action as factors influencing its adoption are not static rather; they vary from one stage to another.

Exploring nurses’ reactions to electronic nursing documentation at the point of care

Purpose

The purpose of this paper is to investigate nurses’ attitudes, perceptions, and reactions to a new point-of-care information system for documenting nursing care.

Design/methodology/approach

A design science research methodology (DSRM) was used to examine the feasibility and usability of a novel nursing informatics solution in the context of acute hospital care. Data were collected using focus groups and non-participant observations. Analyses were guided by the theoretical lens of actor-network theory (ANT).

Findings

The findings unpack an understanding of the potential value of a new technology, rather than a binary understanding of positive or negative value. Using the ANT lens, the study reveals the dynamics of the nurse-technology relationships and consequent disruptions throughout the translation process. The findings highlight the central role of negotiation in the socio-technical construction of the hybrid actor-network during the implementation of new technology in acute hospital contexts.

Research limitations/implications

Further studies are needed to investigate the dynamics and complexity of the translation process that occurs during technology adoption, reactions of the involved actors to the emerging network and impacts on their role and work process.

Practical implications

Engaging nurses early during development and testing; aligning the new system’s functionality and interface with nurses’ interests and work practices; and supporting changes to clinical work process to enable an effective heterogeneous actor-network to emerge and become stable.

Originality/value

This study presents a novel use of ANT in a DSRM to understand an enterprise-wide system involving nurses and real clinical settings. The emerged actor-network provides insights into the translation process when nurses adapt to using new technology in their work.

Evaluation of health information systems research in information systems research: A meta-analysis

Given the importance of the health-care industry and the promise of health information systems, researchers are encouraged to build on the shoulders of giants as the saying goes. The health information systems field has a unique opportunity to learn from and extend the work that has already been done by the highly correlated information systems field. As a result, this research article presents a past, present and future meta-analysis of health information systems research in information systems journals over the 2000–2015 time period. Our analysis reviewed 126 articles on a variety of topics related to health information systems research published in the “Senior Scholars” list of the top eight ranked information systems academic journals. Across the selected information systems academic journals, our findings compare research methodologies applied, health information systems topic areas investigated and research trends. Interesting results emerge in the range and evolution of health information systems research and opportunities for health information systems researchers and practitioners to consider moving forward.

Nurses and electronic health records in a Canadian hospital: examining the social organisation and programmed use of digitised nursing knowledge

Institutional ethnography (IE) is used to examine transformations in a professional nurse's work associated with her engagement with a hospital's electronic health record (EHR) which is being updated to integrate professional caregiving and produce more efficient and effective health care. We review in the technical and scholarly literature the practices and promises of information technology and, especially of its applications in health care, finding useful the more critical and analytic perspectives. Among the latter, scholarship on the activities of economising is important to our inquiry into the actual activities that transform 'things' (in our case, nursing knowledge and action) into calculable information for objective and financially relevant decision-making. Beginning with an excerpt of observational data, we explicate observed nurse-patient interactions, discovering in them traces of institutional ruling relations that the nurse's activation of the EHR carries into the nursing setting. The EHR, we argue, materialises and generalises the ruling relations across institutionally located caregivers; its authorised information stabilises their knowing and acting, shaping health care towards a calculated effective and efficient form. Participating in the EHR's ruling practices, nurses adopt its ruling standpoint; a transformation that we conclude needs more careful analysis and debate.

A literature review for large-scale health information system project planning, implementation and evaluation

Information technology is perceived as a potential panacea for healthcare organisations to manage pressure to improve services in the face of increased demand. However, the implementation and evaluation of health information systems (HIS) is plagued with problems and implementation shortcomings and failures are rife. HIS implementation is complex and relies on organisational, structural, technological, and human factors to be successful. It also requires reflective, nuanced, multidimensional evaluation to provide ongoing feedback to ensure success. This article provides a comprehensive review of the literature about evaluating and implementing HIS, detailing the challenges and recommendations for both evaluators and healthcare organisations. The factors that inhibit or promote successful HIS implementation are identified and effective evaluation strategies are described with the goal of informing teams evaluating complex HIS.

Examining the pre-adoption stages of healthcare IT: A case study of vital signs monitoring systems

Pervasive healthcare systems can reduce the costs and improve the quality of healthcare. However, insufficient care in managing the process before the organizational decision to adopt information technology (IT) can result in poor outcomes. With most previous research focusing on IT adoption, this paper develops a multi-stage theoretical framework for the pre-adoption phase of healthcare IT to address this practical challenge and gap in the literature. With a priori concepts identified from previous multi-stage models, our framework was developed by analyzing two cases of the introduction of vital signs monitoring systems in hospitals to identify the important stages and influencing factors for healthcare IT pre-adoption.

Radio frequency identification-enabled capabilities in a healthcare context: An exploratory study

Increasingly, the adoption and use of radio frequency identification systems in hospital settings is gaining prominence. However, despite the transformative impact that radio frequency identification has in healthcare settings, few studies have examined how and why this change may occur. The purpose of this study is to systematically understand how radio frequency identification can transform work practices in an operational process that directly impacts cost and operational efficiency and indirectly contributes to impacting patient safety and quality of care. We leverage an interdisciplinary framework to explore the contextual characteristics that shape the assimilation of radio frequency identification in healthcare settings. By linking the use of radio frequency identification with specific contextual dimensions in healthcare settings, we provide a data-driven account of how and why radio frequency identification can be useful in inventory management in this setting. In doing so, we also contribute to recent work by information systems scholars who argue for a reconfiguration of conventional assumptions regarding the role of technology in contemporary organizations.

Organizational learning in the implementation and adoption of national electronic health records: Case studies of two hospitals participating in the National Programme for Information Technology in England

Objective:

To explore the role of organizational learning in enabling implementation and supporting adoption of electronic health record systems into two English hospitals.

Methods and setting:

In the course of conducting our prospective and sociotechnical evaluation of the implementation and adoption of electronic health record into 12 “early adopter” hospitals across England, we identified two hospitals implementing virtually identical versions of the same “off-the-shelf” software (Millennium) within a comparable timeframe. We undertook a longitudinal qualitative case study–based analysis of these two hospitals (referred to hereafter as Alpha and Omega) and their implementation experiences. Data included the following: 63 in-depth interviews with various groups of internal and external stakeholders; 41-h on-site observation; and content analysis of 218 documents of various types. Analysis was both inductive and deductive, the latter being informed by the “sociotechnical changing” theoretical perspective.

Results:

Although Alpha and Omega shared a number of contextual similarities, our evaluation revealed fundamental differences in visions of electronic health record and the implementation strategy between the hospitals, which resulted in distinct local consequences of electronic health record implementation and impacted adoption. Both hospitals did not, during our evaluation, see the hoped-for benefits to the organization as a result of the introduction of electronic health record, such as speeding-up tasks. Nonetheless, the Millennium software worked out to be easier to use at Omega. Interorganizational learning was at the heart of this difference.

Conclusion:

Despite the turbulent overall national “roll out” of electronic health record systems into the English hospitals, considerable opportunities for organizational learning were offered by sequential delivery of the electronic health record software into “early adopter” hospitals. We argue that understanding the process of organizational learning and its enabling factors has the potential to support efforts at implementing national electronic health record implementation endeavors.

Changes in healthcare professional work afforded by technology: The introduction of a national electronic patient record in an English hospital

This article considers changes in healthcare professional work afforded by technology. It uses the sociology of professionals’ literature together with a theory of affordances to examine how and when technology allows change in healthcare professional work. The study draws from research into the introduction of a national electronic patient record in an English hospital. We argue that electronic patient record affords changes through its materiality as it interacts with healthcare professional practice. Its affordances entail some level of standardisation of healthcare professional conduct and practice, curtailment of professional autonomy, enlargement of nurses’ roles and redistribution of clinical work within and across professional boundaries. The article makes a contribution to the growing literature advocating a cultural approach to the study of technological affordances in organisations and to studies that explore healthcare professional practice in conjunction with the materiality of technology. Two main lines of argument are developed here. First, that technological affordances do not solely lie with the materiality of technology nor with individual perceptions, but are cultivated and nurtured within a broader cultural–institutional context, in our case a professional context of use. Second, that technological affordance of change is realised when healthcare professionals’ (individual and collective) perceptions of technology (and of its materiality) fit with their sense of (professional) self. In this respect, the article shows the extent to which the materiality of technology plays out with professional identity and frames the level and extent to which technology can and cannot afford restructuring of work and redistribution of power across professional groups.

Alexander P. A study using a graphical syntax for actor-network theory

Purpose

– Actor-network theory (ANT) is considered to be both a theoretical position and a methodology. ANT has been the centre of vigorous debates regarding its ontological viewpoint but has also been found to have some drawbacks as a methodology. The purpose of this paper is to use a graphical syntax for ANT to describe the development and implementation of a health information system (IS) in order to assess whether the graphical syntax improves the use of ANT as a methodology.

Design/methodology/approach

– An extensive example derived from a case study, describing the development and implementation of an electronic patients’ records system, is illustrated using this graphical syntax. This serves to makes the actors, translations and black boxes in the case visible.

Findings

– The syntax is found to help the researcher to conceptualise the research, to highlight assumptions as black boxes and to follow the actor but above all it encourages the researcher to understand the translations being made between actors and to make them explicit. Hence the syntax is found to assist at the analytical phases of a research project. The graphic syntax found to address the criticisms identified for ANT as an IS research methodology.

Research limitations/implications

– The proposed graphical syntax assists researchers who use ANT as a methodology. Use of the syntax in education is also foreseen to be practical. The evaluation of ANT-gs in the paper provides an argument for its use but a more reliable argument would be made by collating and analysing feedback from independent modellers who actually use the graphical syntax. This is an important part of future research.

Practical implications

– The visualisation of the actor-network provided by ANT-gs provides a relatively simple representation while at the same time it makes key ANT concepts explicit. This is expected to address the issue of superficial understanding of ANT and selective use of its constructs; it makes the actor-network boundaries immediately visible. Thus the paper proposes that ANT-gs will be useful both by IS researchers and as an educational tool but future research is required to verify both these practical implications. The development of a computer-based modelling tool based on this syntax is proposed to facilitate effective and efficient modelling.

Originality/value

– This paper proposes a unique tool to support ANT as a methodology. Its use encourages the researcher to focus on constructs that are essential elements of ANT and, by making each of the translation instances explicit, it strengthens the analysis in a way that is true to the view of ANT as a sociology of translation.

Interpretive flexibility in mobile health: lessons from a government-sponsored home care program

Background

Mobile technologies have emerged as important tools that health care personnel can use to gain easy access to client data anywhere. This is particularly useful for nurses and care workers in home health care as they provide services to clients in many different settings. Although a growing body of evidence supports the use of mobile technologies, the diverse implications of mobile health have yet to be fully documented.

Objective

Our objective was to examine a large-scale government-sponsored mobile health implementation program in the Danish home care sector and to understand how the technology was used differently across home care agencies.

Methods

We chose to perform a longitudinal case study with embedded units of analysis. We included multiple data sources, such as written materials, a survey to managers across all 98 Danish municipalities, and semistructured interviews with managers, care workers, and nurses in three selected home care agencies. We used process models of change to help analyze the overall implementation process from a longitudinal perspective and to identify antecedent conditions, key events, and practical outcomes.

Results

Strong collaboration between major stakeholders in the Danish home care sector (government bodies, vendors, consultants, interest organizations, and managers) helped initiate and energize the change process, and government funding supported quick and widespread technology adoption. However, although supported by the same government-sponsored program, mobile technology proved to have considerable interpretive flexibility with variation in perceived nature of technology, technology strategy, and technology use between agencies. What was first seen as a very promising innovation across the Danish home care sector subsequently became the topic of debate as technology use arrangements ran counter to existing norms and values in individual agencies.

Conclusions

Government-sponsored programs can have both positive and negative results, and managers need to be aware of this and the interpretive flexibility of mobile technology. Mobile technology implementation is a complex process that is best studied by combining organization-level analysis with features of the wider sociopolitical and interorganizational environment.

We are bitter, but we are better off: case study of the implementation of an electronic health record system into a mental health hospital in England

BACKGROUND

In contrast to the acute hospital sector, there have been relatively few implementations of integrated electronic health record (EHR) systems into specialist mental health settings. The National Programme for Information Technology (NPfIT) in England was the most expensive IT-based transformation of public services ever undertaken, which aimed amongst other things, to implement integrated EHR systems into mental health hospitals. This paper describes the arrival, the process of implementation, stakeholders' experiences and the local consequences of the implementation of an EHR system into a mental health hospital.

METHODS

Longitudinal, real-time, case study-based evaluation of the implementation and adoption of an EHR software (RiO) into an English mental health hospital known here as Beta. We conducted 48 in-depth interviews with a wide range of internal and external stakeholders, undertook 26 hours of on-site observations, and obtained 65 sets of relevant documents from various types relating to Beta. Analysis was both inductive and deductive, the latter being informed by the 'sociotechnical changing' theoretical framework.

RESULTS

Many interviewees perceived the implementation of the EHR system as challenging and cumbersome. During the early stages of the implementation, some clinicians felt that using the software was time-consuming leading to the conclusion that the EHR was not fit for purpose. Most interviewees considered the chain of deployment of the EHR-which was imposed by NPfIT-as bureaucratic and obstructive, which restricted customization and as a result limited adoption and use. The low IT literacy among users at Beta was a further barrier to the implementation of the EHR. This along with inadequate training in using the EHR software led to resistance to the significant cultural and work environment changes initiated by EHR. Despite the many challenges, Beta achieved some early positive results. These included: the ability to check progress notes and monitor staff activities; improving quality of care as a result of real-time, more accurate and shared patient records across the hospital; and potentially improving the safety of care through increasing the legibility of the clinical record.

CONCLUSIONS

Notwithstanding what was seen as a turbulent, painful and troublesome implementation of the EHR system, Beta achieved some early clinical and managerial benefits from implementing EHRs. The 'sociotechnical changing' framework helped us go beyond the dichotomy of success versus failure, when conducting the evaluation and interpreting findings. Given the scope for continued development, there are good reasons, we argue, to scale up the intake of EHR systems by mental health care settings. Software customization and appropriate support are essential to work EHR out in such organizations.

Building a house on shifting sand: methodological considerations when evaluating the implementation and adoption of national electronic health record systems

BACKGROUND

A commitment to Electronic Health Record (EHR) systems now constitutes a core part of many governments' healthcare reform strategies. The resulting politically-initiated large-scale or national EHR endeavors are challenging because of their ambitious agendas of change, the scale of resources needed to make them work, the (relatively) short timescales set, and the large number of stakeholders involved, all of whom pursue somewhat different interests. These initiatives need to be evaluated to establish if they improve care and represent value for money.

METHODS

Critical reflections on these complexities in the light of experience of undertaking the first national, longitudinal, and sociotechnical evaluation of the implementation and adoption of England's National Health Service's Care Records Service (NHS CRS).

RESULTS/DISCUSSION

We advance two key arguments. First, national programs for EHR implementations are likely to take place in the shifting sands of evolving sociopolitical and sociotechnical and contexts, which are likely to shape them in significant ways. This poses challenges to conventional evaluation approaches which draw on a model of baseline operations → intervention → changed operations (outcome). Second, evaluation of such programs must account for this changing context by adapting to it. This requires careful and creative choice of ontological, epistemological and methodological assumptions.

SUMMARY

New and significant challenges are faced in evaluating national EHR implementation endeavors. Based on experiences from this national evaluation of the implementation and adoption of the NHS CRS in England, we argue for an approach to these evaluations which moves away from seeing EHR systems as Information and Communication Technologies (ICT) projects requiring an essentially outcome-centred assessment towards a more interpretive approach that reflects the situated and evolving nature of EHR seen within multiple specific settings and reflecting a constantly changing milieu of policies, strategies and software, with constant interactions across such boundaries.

Improvisation during Process-Technology Adoption: A Longitudinal Study of a Software Firm

Most software firms struggle to take advantage of the potential benefits of software process improvement (SPI) as they adopt this technology into the complex and dynamic realities of their day-to-day operation. Such efforts are therefore typically fluctuating between management's attempt to control SPI technology adoption and events that causes the process to drift in unpredictable directions. To further understand how management's attempt to control the process is complemented by drifting, this article investigates the role of improvisation in adoption of SPI technology in a Danish software firm, SmallSoft, over a 10-year period (1996–2005). We found that micro-level and macro-level improvisations interacted, often in uncoordinated ways, to shape SPI technology adoption at SmallSoft. The improvisations enhanced employee creativity, motivation and empowerment, created momentum in the adoption process despite constrained resources, and, most importantly, helped adapt SPI technology to the everyday practices at SmallSoft. However, we also identified un-called for improvisations and outcomes that were uncoordinated with SmallSoft's goals. Based on these findings we discuss how management in small software firms can exploit improvisations to facilitate adoption of complex technologies like SPI.