E-health: implementation and evaluation research in Scotland -- a scoping exercise

Health economic evaluation of digital nursing technologies: a review of methodological recommendations

BACKGROUND

Health economic evaluation of digital nursing technologies (DNT) is important to provide information that helps avoid undesirable developments and implementations as well as increase the chances of success of developed applications. At the same time, studies and evidence on cost-effectiveness are still very rare in this field. Review studies in related technology areas such as telemedicine frequently criticise the quality and comparability of health economic evaluations conducted in this field. Based on a content analysis of methodological literature on the economic evaluation of innovative (digital) technologies in health and nursing, this article aims to identify specific challenges in this research area and offers recommendations on how to address these challenges to promote more sound health economic evaluations in the future.

METHODS

A rapid review was conducted, consisting of a systematic search in the Pubmed database as well as Google Scholar. In addition, the literature lists of the analysed texts were scoured for additional texts to be included. Methodological literature, single studies, and reviews were included. A total of 536 studies were screened, of which 29 were included in the full text analysis.

RESULTS

Based on the systematic content analysis of the studies under consideration, 10 specific methodological challenges are identified, and the methodological recommendations were examined for consideration. A particular focus was given to whether specific methodological approaches might be needed in the context of evaluating the efficiency of DNT.

CONCLUSION

Many of the challenges identified for the health economic evaluations of digital nursing technologies are comparable to those of other complex health care interventions. The recommendations discussed can help to alleviate those challenges. Future research should focus on alternative approaches to assessing causality in different phases of technology development while maintaining high evidence standards. High-evidence economic assessment of technologies in nursing care should be carried out in routine use, especially if they are intended to be reimbursed by the social insurance.

The case for randomized controlled trials to assess the impact of clinical information systems

There is a persistent view of a significant minority in the medical informatics community that the randomized controlled trial (RCT) has a limited role to play in evaluating clinical information systems. A common reason voiced by skeptics is that these systems are fundamentally different from drug interventions, so the RCT is irrelevant. There is an urgent need to promote the use of RCTs, given the shift to evidence-based policy and the need to demonstrate cost-effectiveness of these systems. The authors suggest returning to first principles and argue that what is required is clarity about how to match methods to evaluation questions. The authors address common concerns about RCTs, and the extent to which they are fallacious, and also discuss the challenges of conducting RCTs in informatics and alternative study designs when randomized trials are infeasible. While neither a perfect nor universal evaluation method, RCTs form an important part of an evaluator's toolkit.

Exploring public perspectives on e-health: findings from two citizen juries

BACKGROUND

Interest and investment in e-health continue to grow world-wide, but there remains relatively little engagement with the public on this subject, despite calls for more public involvement in health-care planning.

DESIGN

This study used two modified citizen juries to explore barriers and facilitators to e-health implementation and the priorities for future e-health research from the perspective of health service users and lay representatives. Citizen juries bring together a group of people to deliberate over a specific issue. They are given information and invited to 'cross-examine' witnesses during the process.

RESULTS

Jurors were very keen for lay views to be included in e-health development and embraced the citizen jury approach. They agreed unanimously that e-health should be developed and thought it was in many ways inevitable. Although there was much enthusiasm for a health-care system which offered e-health as an option, there was as much concern about what it might mean for patients if implemented inappropriately. E-health was preferred as an enhancement rather than substitute for, existing services. Lack of universal access was seen as a potential barrier to implementation but problems such as lack of computer literacy were seen as a temporary issue. Participants emphasized that e-health research needed to demonstrate both clinical and economic benefits.

CONCLUSION

There was broad support from the citizen juries for the development of e-health, although participants stressed that e-health should enhance, rather than substitute, face-to-face services. One-day citizen juries proved a practical method of public engagement on this subject.

The influence of context and process when implementing e-health

Background

Investing in computer-based information systems is notoriously risky, since many systems fail to become routinely used as part of everyday working practices, yet there is clear evidence about the management practices which improve the acceptance and integration of such systems. Our aim in this study was to identify to what extent these generic management practices are evident in e-health projects, and to use that knowledge to develop a theoretical model of e-health implementation. This will support the implementation of appropriate e-health systems.

Methods

This study consisted of qualitative semi-structured interviews with managers and health professionals in Scotland, UK. We contacted the Scottish Ethics Committee, who advised that formal application to that body was not necessary for this study. The interview guide aimed to identify the issues which respondents believed had affected the successful implementation of e-health projects. We drew on our research into information systems in other sectors to identify likely themes and questions, which we piloted and revised. Eighteen respondents with experience of e-health projects agreed to be interviewed. These were recorded, transcribed, coded, and then analysed with 'Nvivo' data analysis software.

Results

Respondents identified factors in the context of e-health projects which had affected implementation, including clarity of the strategy; supportive structures and cultures; effects on working processes; and how staff perceived the change. The results also identified useful implementation practices such as balancing planning with adaptability; managing participation; and using power effectively.

Conclusion

The interviews confirmed that the contextual factors that affect implementation of information systems in general also affect implementation of e-health projects. As expected, these take place in an evolving context of strategies, structures, cultures, working processes and people. Respondents also confirmed that those managing such projects seek to change these contexts through observable implementation processes of planning, adaptation, participation and using power. This study confirms that research to support the delivery of appropriate e-health projects can usefully draw on the experience of information systems in other sectors.