A novel research design can aid disinvestment from existing health technologies with uncertain effectiveness, cost-effectiveness, and/or safety

Disinvestment in the presence of uncertainty: Description of a novel, multi-group, disinvestment trial design and protocol for an application to reduce or cease use of mobilisation alarms for preventing falls in hospitals

Disinvestment is the removal or reduction of previously provided practices or services, and has typically been undertaken where a practice or service has been clearly shown to be ineffective, inefficient and/or harmful. However, practices and services that have uncertain evidence of effectiveness, efficiency and safety can also be considered as candidates for disinvestment. Disinvestment from these practices and services is risky as they may yet prove to be beneficial if further evidence becomes available. A novel research approach has previously been described for this situation, allowing disinvestment to take place while simultaneously generating evidence previously missing from consideration. In this paper, we describe how this approach can be expanded to situations where three or more conditions are of relevance, and describe the protocol for a trial examining the reduction and elimination of use of mobilisation alarms on hospital wards to prevent patient falls. Our approach utilises a 3-group, concurrent, non-inferiority, stepped wedge, randomised design with an embedded parallel, cluster randomised design. Eighteen hospital wards with high rates of alarm use (≥3%) will be paired within their health service and randomly allocated to a calendar month when they will transition to a “Reduced” (<3%) or “Eliminated” (0%) mobilisation alarm condition. Dynamic randomisation will be used to determine which ward in each pair will be allocated to either the reduced or eliminated condition to promote equivalence between wards for the embedded parallel, cluster randomised component of the design. A project governance committee will set non-inferiority margins. The primary outcome will be rates of falls. Secondary clinical, process, safety, and economic outcomes will be collected and a concurrent economic evaluation undertaken.

Divesting from a Scored Hospital Fall Risk Assessment Tool (FRAT): A Cluster Randomized Non-Inferiority Trial

Background/Objectives

We investigated the impact of ceasing routine falls risk assessment tool (FRAT) completion and instead used clinical reasoning to select fall mitigation strategies.

Design

Two‐group, multi‐site cluster‐randomized active‐control non‐inferiority trial.

Setting

Hospital wards.

Participants

Adult inpatients admitted to participating hospitals (n = 10 hospitals, 123,176 bed days).

Intervention

Hospitals were randomly assigned (1:1) to a usual care control group that continued to use a historical FRAT to assign falls risk scores and accompanying mitigation strategies, or an experimental group whereby clinicians did not assign risk scores and instead used clinical reasoning to select fall mitigation strategies using a decision support list.

Measurements

The primary measure was between‐group difference in mean fall rates (falls/1000 bed days). Falls were identified from incident reports supplemented by hand searches of medical records over three consecutive months at each hospital. The incidence rate ratio (IRR) of monthly falls rates in control versus experimental hospitals was also estimated.

Results

The experimental clinical reasoning approach was non‐inferior to the usual care FRAT that assigned fall risk ratings when compared to a‐priori stakeholder derived and sensitivity non‐inferiority margins. The mean fall rates were 3.84 falls/1000 bed days for the FRAT continuing sites and 3.11 falls/1000 bed days for experimental sites. After adjusting for historical fall rates at each hospital, the IRR (95%CI) was 0.78 (0.64, 0.95), where IRR < 1.00 indicated fewer falls among the experimental group. There were 4 and 3 serious events in the control and experimental groups, respectively.

Conclusion

Replacing a FRAT scoring system with clinical reasoning did not lead to inferior fall outcomes in the short term and may even reduce fall incidence.

Conceptualising characteristics of resources withdrawal from medical services: a systematic qualitative synthesis

BACKGROUND

Terms used to describe government-led resource withdrawal from ineffective and unsafe medical services, including 'rationing' and 'disinvestment', have tended to be used interchangeably, despite having distinct characteristics. This lack of descriptive precision for arguably distinct terms contributes to the obscurity that hinders effective communication and the achievement of evidence-based decision-making. The objectives of this study are to (1) identify the various terms used to describe resource withdrawal and (2) propose definitions for the key or foundational terms, which includes a clear description of the unique characteristics of each.

METHODS

This is a systematic qualitative synthesis of characteristics and terms found through a search of the academic and grey literature. This approach involved identifying commonly used resource withdrawal terms, extracting data about resource withdrawal characteristics associated with each term and conducting a comparative analysis by categorising elements as antecedents, attributes or outcomes.

RESULTS

Findings from an analysis of 106 documents demonstrated that terms used to describe resource withdrawal are inconsistently defined and applied. The characteristics associated with these terms, mainly antecedents and attributes, are used interchangeably by many authors but are differentiated by others. Our analysis resulted in the development of a framework that organises these characteristics to demonstrate the unique attributes associated with each term. To enhance precision, these terms were classified as either policy options or patient health outcomes and refined definitions for rationing and disinvestment were developed. Rationing was defined as resource withdrawal that denies, on average, patient health benefits. Disinvestment was defined as resource withdrawal that results in, on average, improved or no change in health benefits.

CONCLUSION

Agreement on the definition of various resource withdrawal terms and their key characteristics is required for transparent government decision-making regarding medical service withdrawal. This systematic qualitative synthesis presents the proposed definitions of resource withdrawal terms that will promote consistency, benefit public policy dialogue and enhance the policy-making process for health systems.

Falls risk score removal does not impact inpatient falls: A stepped-wedge, cluster-randomised trial

AIMS AND OBJECTIVES

To investigate the impact of removing a falls risk screening tool from an overall falls risk assessment programme on the rate of falls, injurious falls and completion of falls prevention activities by staff.

BACKGROUND

Falls in older patients are common adverse events in hospital settings. Screening and assessing individual patients for risk of falls are a common, but controversial element of falls prevention strategies in hospitals.

DESIGN

A stepped-wedge, cluster-randomised controlled trial using a disinvestment approach.

METHODS

This trial was carried out according to the Consolidated Standards of Reporting Trials (CONSORT). All patients were admitted to 20 health service wards (9 units) over the 10-month study period. The control condition contained a falls risk screening tool element, a full falls risk factor assessment and intervention provision section. In the intervention condition, only the full falls risk factor assessment and intervention provision section was applied, and the falls risk screening tool element was removed. Fall rates were extracted from hospital level data, files were audited for tool completion, and nurses surveyed about tool use.

RESULTS

There did not appear to be an impact on the falls rate per month when the risk screening tool component was removed (incidence rate ratio (IRR) = 0.84-favours intervention, 95%CI = 0.67 to 1.05, p = .14) nor on the falls rate with serious injury (IRR = 0.90, 95%CI = 0.26 to 3.09, p = .87). There was a thirty-six second reduction of time per patient reported by staff to complete paperwork (p < .001). There was no difference in the proportion of patients for whom the tool was completed, nor the number of falls prevention interventions identified for implementation.

CONCLUSION

Removing the falls risk screening tool section did not negatively impact falls and reduced time spent completing falls prevention paperwork.

RELEVANCE TO CLINICAL PRACTICE

Falls prevention is an important issue in health services. Removal of a screening risk tool is unlikely to impact falls. This has the potential to reduce nursing administration time that may be otherwise redirected to individual approaches to falls prevention.

[National Pharmacovigilance Systems in Brazil and Portugal: similarities, differences, and challenges]

National Pharmacovigilance Systems (PVS) manage health risks and identify, assess, and act to minimize them, contributing to adequate use of medicines, patient safety, and improved quality of care. Fast-track drug registration, which has become increasingly frequent, hinders assessment of the efficacy and safety of new drugs, adding difficulties to current regulation and health protection. The article applies indicators proposed by the World Health Organization to analyze the National PVS of Portugal and Brazil. Brazil's PVS was established later than that of Portugal, generates fewer safety signals, has a lower notification rate for suspected adverse drug events (ADEs), and displays difficulty in producing and disseminating information to health professionals and the population. Portugal has the advantage of being a member state of the European Medicines Agency. The article also suggests that the differences are related to the political and social context that hinders the implementation of public policies and compromises the effectiveness of the Brazilian PVS. Challenges for PVS include awareness-raising of health professionals, the adoption of methods to complement voluntary notification, pharmacovigilance of biological and genetic drugs, and assessment of the system's impact. An additional challenge for the Brazilian PVS is to improve the notifications' uptake and quality, including from industry, generate safety signals in the national context, and communicate risk in timely fashion to health professionals and the population.

Collective constructions of 'waste': epistemic practices for disinvestment in the context of Dutch social health insurance

Background

Faced with growing budget pressure, policymakers worldwide recognize the necessity of strategic disinvestment from ineffective, inefficient or harmful medical practices. However, disinvestment programs face substantial social, political and cultural challenges: mistrust, struggles for clinical autonomy or stakeholders’ reluctance to engage in what can be perceived as ‘rationing’. Academic literature says little about effective strategies to address these challenges. This paper provides insights on this matter. We analyzed the epistemic work of a group of policymakers at the National Health Care Institute on what was initially a disinvestment initiative within the context of the Dutch basic benefits package: the ‘Appropriate Care’ program. The Institute developed a strategy using national administrative data to identify and tackle low-value care covered from public funds as well as potential underuse, and achieve savings through improved organization of efficiency and quality in health care delivery. How did the Institute deal with the socio-political sensitivities associated with disinvestment by means of their epistemic work?

Method

We conducted ethnographic research into the National Health Care Institute’s epistemic practices. Research entailed document analysis, non-participant observation, in-depth conversations, and interviews with key-informants.

Results

The Institute dealt with the socio-political sensitivities associated with disinvestment by democratizing the epistemic practices to identify low-value care, by warranting data analysis by clinical experts, by creating an epistemic safe space for health care professionals who were the object of research into low-value care, and by de-emphasizing the economization measure. Ultimately, this epistemic work facilitated a collaborative construction of problems relating to low-value care practices and their solutions.

Conclusions

This case shows that – apart from the right data and adequate expertise – disinvestment requires clinical leadership and political will on the part of stakeholders. Our analysis of the Institute’s Appropriate Care program shows how the epistemic effort to identify low-value care became a co-construction between policymakers, care providers, patients and insurers of problems of ‘waste’ in Dutch social health insurance. This collective epistemic work gave cognitive, moral and political standing to the idea of ‘waste’ in public health expenditure.

Electronic supplementary material

The online version of this article (10.1186/s12913-019-4434-1) contains supplementary material, which is available to authorized users.

Understanding Health Professional Responses to Service Disinvestment: A Qualitative Study

Background: Disinvestment from inefficient health services may be a potential solution to rising healthcare costs, but there has been poor uptake of disinvestment recommendations. This Australian study aims to understand how health professionals react when confronted with a plan to disinvest from a health service they previously provided to their patients.

Methods: This qualitative study took place prior to the disinvestment phase of a trial which removed weekend allied health services from acute hospital wards, to evaluate the effectiveness and cost effectiveness of the service. Observations and focus groups were used to collect data from 156 participants which was analysed thematically.

Results: Initial reactions to the disinvestment were almost universally negative, with staff extremely concerned about the impact on the safety and quality of patient care and planning ways to circumvent the trial. Removal of existing services was perceived as a loss and created a direct threat to some clinicians’ professional identity. With time, discussion, and understanding of the project’s context, some staff moved towards acceptance and perceived the trial as an opportunity, particularly given the service was to be reinstated after the disinvestment.

Conclusion: Clinicians and health service managers are protective of the services they deliver and can create barriers to disinvestment. Even when services are removed to ascertain their value, health professionals may continue to provide services to their patients. Measuring the impact of the disinvestment may assist staff to accept the removal of a service.

Trans-disciplinary advanced allied health practitioners for acute hospital inpatients: a feasibility study

OBJECTIVE

To explore cost-efficiency, safety and acceptability of trans-disciplinary advanced allied health (AH) practitioners for acute adult general medicine inpatients.

DESIGN

Quasi-experimental feasibility study.

SETTING

Three acute general medical units in an Australian urban hospital.

PARTICIPANTS

Two hundred and fifty-six acute hospital inpatients.

MAIN OUTCOME MEASURES

Cost-efficiency measures included AH service utilization and length of stay (LOS). Patient outcomes were functional independence, discharge destination, adverse events, unplanned admissions within 28 days, patient satisfaction and quality of life data on admission, and 30 days post-discharge. Ward staff were surveyed regarding satisfaction with the service model, and advanced health practitioners (AHPs) rated their confidence in their own ability to meet the performance standards of the role.

RESULTS

Patients allocated to AHPs (n = 172) received 0.91 less hours of AH intervention (adjusted for LOS) (95% confidence intervals (CI): -1.68 to -0.14; P = 0.02) and had 1.76 days shorter LOS relative to expected (95%CI: 0.18-3.34; P = 0.03) compared with patients receiving standard AH (n = 84). There were no differences in patient outcomes or satisfaction. AHPs demonstrated growth in job satisfaction and skill confidence.

CONCLUSIONS

Trans-disciplinary advanced AH roles may be feasible and cost-efficient compared with traditional roles for acute general medical inpatients. Further development of competency frameworks is recommended.

A novel counterbalanced implementation study design: methodological description and application to implementation research

Background

Implementation research is increasingly being recognised for optimising the outcomes of clinical practice. Frequently, the benefits of new evidence are not implemented due to the difficulties applying traditional research methodologies to implementation settings. Randomised controlled trials are not always practical for the implementation phase of knowledge transfer, as differences between individual and organisational readiness for change combined with small sample sizes can lead to imbalances in factors that impede or facilitate change between intervention and control groups. Within-cluster repeated measure designs could control for variance between intervention and control groups by allowing the same clusters to receive a sequence of conditions. Although in implementation settings, they can contaminate the intervention and control groups after the initial exposure to interventions. We propose the novel application of counterbalanced design to implementation research where repeated measures are employed through crossover, but contamination is averted by counterbalancing different health contexts in which to test the implementation strategy.

Methods

In a counterbalanced implementation study, the implementation strategy (independent variable) has two or more levels evaluated across an equivalent number of health contexts (e.g. community-acquired pneumonia and nutrition for critically ill patients) using the same outcome (dependent variable). This design limits each cluster to one distinct strategy related to one specific context, and therefore does not overburden any cluster to more than one focussed implementation strategy for a particular outcome, and provides a ready-made control comparison, holding fixed. The different levels of the independent variable can be delivered concurrently because each level uses a different health context within each cluster to avoid the effect of treatment contamination from exposure to the intervention or control condition.

Results

An example application of the counterbalanced implementation design is presented in a hypothetical study to demonstrate the comparison of ‘video-based’ and ‘written-based’ evidence summary research implementation strategies for changing clinical practice in community-acquired pneumonia and nutrition in critically ill patient health contexts.

Conclusion

A counterbalanced implementation study design provides a promising model for concurrently investigating the success of research implementation strategies across multiple health context areas such as community-acquired pneumonia and nutrition for critically ill patients.

Electronic supplementary material

The online version of this article (10.1186/s13012-019-0896-0) contains supplementary material, which is available to authorized users.

Costs will rather increase

Purpose

The purpose of this paper was to study the unfolding of an urgent and extensive decommissioning program in Sweden, focusing on the public’s reactions and their arguments when opposing the decommissioning activities.

Design/methodology/approach

The public’s responses were studied through local media. Its content was surveyed and divided into actions and arguments. The arguments were further analyzed and categorized into inductively developed themes.

Findings

Protest activities, such as demonstrations, meetings and petitions, were not coordinated, but mostly carried out for withdrawals of unique services and services in remote areas. The public questioned the decision makers’ information, calculations and competence, the adequacy of the consequence analyses and whether the decommissioning activities would lead to any real savings. Patient and public safety, the vulnerable in society, and effects on the local areas were important topics. Thus, it seems the decision makers did not fully succeed in communicating the demonstrable benefits or create clarity of the rationales for decommissioning the particular services. Furthermore, it seems the public has a more inclusive approach to health services and their value compared to decision makers that need to keep the budget.

Originality/value

Decommissioning is an emerging field of research, and this study of the unfolding of an urgent and extensive decommissioning program contributes with evidence that may improve decommissioning policy and practice. The study illustrates that it may be possible to implement a decommissioning program despite public protest, but that the longer-term effects on the health system’s legitimacy need to be studied.

Knowledge translation and health technology reassessment: identifying synergy

Background

Health Technology Reassessment (HTR) is an emerging field that shifts the focus from traditional methods of technology adoption to managing technology throughout its lifecycle. HTR is a mechanism to improve patient care and system efficiency through a reallocation of resources away from low-value care towards interventions and technologies that are high value. To achieve this, the outputs of HTR and its recommendations must be translated into practice. The evolving field of knowledge translation (KT) can provide guidance to improve the uptake of evidence-informed policies and recommendations resulting from the process of HTR. This paper argues how the theories, models and frameworks from KT could advance the HTR process.

Discussion

First, common KT theories, models and frameworks are presented. Second, facilitators and barriers to KT within the context of HTR are summarized from the literature. Facilitators and barriers to KT include ensuring a solid research evidence-base for the technology under reassessment, assessing the climate and context, understanding the social an political context, initiating linkage and exchange, having a structured HTR Process, adequate resources, and understanding the roles of researchers, knowledge users, and stakeholders can enhance knowledge translation of HTR outputs. Third, three case examples at the individual (micro), organizational (meso), and policy (macro) levels are used to illustrate to describe how a KT theory, model or framework could be applied to a HTR project. These case studies show how selecting and applying KT theories, models and frameworks can facilitate the implementation of HTR recommendations.

Conclusion

HTR and KT are synergistic processes that can be used to optimize technology use throughout its lifecycle. We argue that the application of KT theories, models and frameworks, and the assessment of barriers and facilitators to KT can facilitate translation of HTR recommendations into practice.

Models and applications for measuring the impact of health research: update of a systematic review for the Health Technology Assessment programme

BACKGROUND

This report reviews approaches and tools for measuring the impact of research programmes, building on, and extending, a 2007 review.

OBJECTIVES

(1) To identify the range of theoretical models and empirical approaches for measuring the impact of health research programmes; (2) to develop a taxonomy of models and approaches; (3) to summarise the evidence on the application and use of these models; and (4) to evaluate the different options for the Health Technology Assessment (HTA) programme.

DATA SOURCES

We searched databases including Ovid MEDLINE, EMBASE, Cumulative Index to Nursing and Allied Health Literature and The Cochrane Library from January 2005 to August 2014.

REVIEW METHODS

This narrative systematic literature review comprised an update, extension and analysis/discussion. We systematically searched eight databases, supplemented by personal knowledge, in August 2014 through to March 2015.

RESULTS

The literature on impact assessment has much expanded. The Payback Framework, with adaptations, remains the most widely used approach. It draws on different philosophical traditions, enhancing an underlying logic model with an interpretative case study element and attention to context. Besides the logic model, other ideal type approaches included constructionist, realist, critical and performative. Most models in practice drew pragmatically on elements of several ideal types. Monetisation of impact, an increasingly popular approach, shows a high return from research but relies heavily on assumptions about the extent to which health gains depend on research. Despite usually requiring systematic reviews before funding trials, the HTA programme does not routinely examine the impact of those trials on subsequent systematic reviews. The York/Patient-Centered Outcomes Research Institute and the Grading of Recommendations Assessment, Development and Evaluation toolkits provide ways of assessing such impact, but need to be evaluated. The literature, as reviewed here, provides very few instances of a randomised trial playing a major role in stopping the use of a new technology. The few trials funded by the HTA programme that may have played such a role were outliers.

DISCUSSION

The findings of this review support the continued use of the Payback Framework by the HTA programme. Changes in the structure of the NHS, the development of NHS England and changes in the National Institute for Health and Care Excellence's remit pose new challenges for identifying and meeting current and future research needs. Future assessments of the impact of the HTA programme will have to take account of wider changes, especially as the Research Excellence Framework (REF), which assesses the quality of universities' research, seems likely to continue to rely on case studies to measure impact. The HTA programme should consider how the format and selection of case studies might be improved to aid more systematic assessment. The selection of case studies, such as in the REF, but also more generally, tends to be biased towards high-impact rather than low-impact stories. Experience for other industries indicate that much can be learnt from the latter. The adoption of researchfish^® (researchfish Ltd, Cambridge, UK) by most major UK research funders has implications for future assessments of impact. Although the routine capture of indexed research publications has merit, the degree to which researchfish will succeed in collecting other, non-indexed outputs and activities remains to be established.

LIMITATIONS

There were limitations in how far we could address challenges that faced us as we extended the focus beyond that of the 2007 review, and well beyond a narrow focus just on the HTA programme.

CONCLUSIONS

Research funders can benefit from continuing to monitor and evaluate the impacts of the studies they fund. They should also review the contribution of case studies and expand work on linking trials to meta-analyses and to guidelines.

FUNDING

The National Institute for Health Research HTA programme.

Decommissioning care: The need for rigorous multifaceted evaluations of decisions to withdraw health services

In this Perspective on the two clinical trials of Terry Haines and colleagues that incrementally removed and reinstated allied healthcare services, Aziz Sheikh discusses the evidence base for the routine provision of such services.

Impact of disinvestment from weekend allied health services across acute medical and surgical wards: 2 stepped-wedge cluster randomised controlled trials

BACKGROUND

Disinvestment (removal, reduction, or reallocation) of routinely provided health services can be difficult when there is little published evidence examining whether the services are effective or not. Evidence is required to understand if removing these services produces outcomes that are inferior to keeping such services in place. However, organisational imperatives, such as budget cuts, may force healthcare providers to disinvest from these services before the required evidence becomes available. There are presently no experimental studies examining the effectiveness of allied health services (e.g., physical therapy, occupational therapy, and social work) provided on weekends across acute medical and surgical hospital wards, despite these services being routinely provided internationally. The aim of this study was to understand the impact of removing weekend allied health services from acute medical and surgical wards using a disinvestment-specific non-inferiority research design.

METHODS AND FINDINGS

We conducted 2 stepped-wedge cluster randomised controlled trials between 1 February 2014 and 30 April 2015 among patients on 12 acute medical or surgical hospital wards spread across 2 hospitals. The hospitals involved were 2 metropolitan teaching hospitals in Melbourne, Australia. Data from n = 14,834 patients were collected for inclusion in Trial 1, and n = 12,674 in Trial 2. Trial 1 was a disinvestment-specific non-inferiority stepped-wedge trial where the 'current' weekend allied health service was incrementally removed from participating wards each calendar month, in a random order, while Trial 2 used a conventional non-inferiority stepped-wedge design, where a 'newly developed' service was incrementally reinstated on the same wards as in Trial 1. Primary outcome measures were patient length of stay (proportion staying longer than expected and mean length of stay), the proportion of patients experiencing any adverse event, and the proportion with an unplanned readmission within 28 days of discharge. The 'no weekend allied health service' condition was considered to be not inferior if the 95% CIs of the differences between this condition and the condition with weekend allied health service delivery were below a 2% increase in the proportion of patients who stayed in hospital longer than expected, a 2% increase in the proportion who had an unplanned readmission within 28 days, a 2% increase in the proportion who had any adverse event, and a 1-day increase in the mean length of stay. The current weekend allied health service included physical therapy, occupational therapy, speech therapy, dietetics, social work, and allied health assistant services in line with usual care at the participating sites. The newly developed weekend allied health service allowed managers at each site to reprioritise tasks being performed and the balance of hours provided by each professional group and on which days they were provided. Analyses conducted on an intention-to-treat basis demonstrated that there was no estimated effect size difference between groups in the proportion of patients staying longer than expected (weekend versus no weekend; estimated effect size difference [95% CI], p-value) in Trial 1 (0.40 versus 0.38; estimated effect size difference 0.01 [-0.01 to 0.04], p = 0.31, CI was both above and below non-inferiority margin), but the proportion staying longer than expected was greater with the newly developed service compared to its no weekend service control condition (0.39 versus 0.40; estimated effect size difference 0.02 [0.01 to 0.04], p = 0.04, CI was completely below non-inferiority margin) in Trial 2. Trial 1 and 2 findings were discordant for the mean length of stay outcome (Trial 1: 5.5 versus 6.3 days; estimated effect size difference 1.3 days [0.9 to 1.8], p < 0.001, CI was both above and below non-inferiority margin; Trial 2: 5.9 versus 5.0 days; estimated effect size difference -1.6 days [-2.0 to -1.1], p < 0.001, CI was completely below non-inferiority margin). There was no difference between conditions for the proportion who had an unplanned readmission within 28 days in either trial (Trial 1: 0.01 [-0.01 to 0.03], p = 0.18, CI was both above and below non-inferiority margin; Trial 2: -0.01 [-0.02 to 0.01], p = 0.62, CI completely below non-inferiority margin). There was no difference between conditions in the proportion of patients who experienced any adverse event in Trial 1 (0.01 [-0.01 to 0.03], p = 0.33, CI was both above and below non-inferiority margin), but a lower proportion of patients had an adverse event in Trial 2 when exposed to the no weekend allied health condition (-0.03 [-0.05 to -0.004], p = 0.02, CI completely below non-inferiority margin). Limitations of this research were that 1 of the trial wards was closed by the healthcare provider after Trial 1 and could not be included in Trial 2, and that both withdrawing the current weekend allied health service model and installing a new one may have led to an accommodation period for staff to adapt to the new service settings. Stepped-wedge trials are potentially susceptible to bias from naturally occurring change over time at the service level; however, this was adjusted for in our analyses.

CONCLUSIONS

In Trial 1, criteria to say that the no weekend allied health condition was non-inferior to current weekend allied health condition were not met, while neither the no weekend nor current weekend allied health condition demonstrated superiority. In Trial 2, the no weekend allied health condition was non-inferior to the newly developed weekend allied health condition across all primary outcomes, and superior for the outcomes proportion of patients staying longer than expected, proportion experiencing any adverse event, and mean length of stay.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry ACTRN12613001231730 and ACTRN12613001361796.

Sustainability in Health care by Allocating Resources Effectively (SHARE) 9: conceptualising disinvestment in the local healthcare setting

BACKGROUND

This is the ninth in a series of papers reporting a program of Sustainability in Health care by Allocating Resources Effectively (SHARE) in a local healthcare setting. The disinvestment literature has broadened considerably over the past decade; however there is a significant gap regarding systematic, integrated, organisation-wide approaches. This debate paper presents a discussion of the conceptual aspects of disinvestment from the local perspective.

DISCUSSION

Four themes are discussed: Terminology and concepts, Motivation and purpose, Relationships with other healthcare improvement paradigms, and Challenges to disinvestment. There are multiple definitions for disinvestment, multiple concepts underpin the definitions and multiple alternative terms convey these concepts; some definitions overlap and some are mutually exclusive; and there are systematic discrepancies in use between the research and practice settings. Many authors suggest that the term 'disinvestment' should be avoided due to perceived negative connotations and propose that the concept be considered alongside investment in the context of all resource allocation decisions and approached from the perspective of optimising health care. This may provide motivation for change, reduce disincentives and avoid some of the ethical dilemmas inherent in other disinvestment approaches. The impetus and rationale for disinvestment activities are likely to affect all aspects of the process from identification and prioritisation through to implementation and evaluation but have not been widely discussed. A need for mechanisms, frameworks, methods and tools for disinvestment is reported. However there are several health improvement paradigms with mature frameworks and validated methods and tools that are widely-used and well-accepted in local health services that already undertake disinvestment-type activities and could be expanded and built upon. The nature of disinvestment brings some particular challenges for policy-makers, managers, health professionals and researchers. There is little evidence of successful implementation of 'disinvestment' projects in the local setting, however initiatives to remove or replace technologies and practices have been successfully achieved through evidence-based practice, quality and safety activities, and health service improvement programs.

CONCLUSIONS

These findings suggest that the construct of 'disinvestment' may be problematic at the local level. A new definition and two potential approaches to disinvestment are proposed to stimulate further research and discussion.

DRUG DISINVESTMENT FRAMEWORKS: COMPONENTS, CHALLENGES, AND SOLUTIONS

OBJECTIVES

Value assessments of marketed drug technologies have been developed through disinvestment frameworks. Components of these frameworks are varied and implementation challenges are prevalent. The objective of this systematic literature review was to describe disinvestment framework process components for drugs and to report on framework components, challenges, and solutions.

METHODS

A systematic literature search was conducted using the terms: reassessment, reallocation, reinvestment, disinvestment, delist, decommission or obsolescence in MEDLINE, EMBASE, NLM PubMed, the Cochrane Library, and CINAHL from January 1, 2000, until November 14, 2015. Additional citations were identified through a gray literature search of Health Technology Assessment international (HTAi) and the International Network of Agencies for Health Technology Assessment (INAHTA) member Web sites and from bibliographies of full-text reviewed manuscripts.

RESULTS

Sixty-three articles underwent full text review and forty were included in the qualitative analysis. Framework components including disinvestment terms and definitions, identification and prioritization criteria and methods, assessment processes, stakeholders and dissemination strategies, challenges, and solutions were compiled. This review finds that stakeholders lack the political, administrative, and clinical will to support disinvestment and that there is not one disinvestment framework that is considered best practice.

CONCLUSIONS

Drug technology disinvestment components and processes vary and challenges are numerous. Future research should focus on lessening value assessment challenges. This could include adopting more neutral framework terminology, setting fixed reassessment timelines, conducting therapeutic reviews, and modifying current qualitative decision-making assessment frameworks.

Minimum number of clusters and comparison of analysis methods for cross sectional stepped wedge cluster randomised trials with binary outcomes: A simulation study

BACKGROUND

Stepped wedge cluster randomised trials frequently involve a relatively small number of clusters. The most common frameworks used to analyse data from these types of trials are generalised estimating equations and generalised linear mixed models. A topic of much research into these methods has been their application to cluster randomised trial data and, in particular, the number of clusters required to make reasonable inferences about the intervention effect. However, for stepped wedge trials, which have been claimed by many researchers to have a statistical power advantage over the parallel cluster randomised trial, the minimum number of clusters required has not been investigated.

METHODS

We conducted a simulation study where we considered the most commonly used methods suggested in the literature to analyse cross-sectional stepped wedge cluster randomised trial data. We compared the per cent bias, the type I error rate and power of these methods in a stepped wedge trial setting with a binary outcome, where there are few clusters available and when the appropriate adjustment for a time trend is made, which by design may be confounding the intervention effect.

RESULTS

We found that the generalised linear mixed modelling approach is the most consistent when few clusters are available. We also found that none of the common analysis methods for stepped wedge trials were both unbiased and maintained a 5% type I error rate when there were only three clusters.

CONCLUSIONS

Of the commonly used analysis approaches, we recommend the generalised linear mixed model for small stepped wedge trials with binary outcomes. We also suggest that in a stepped wedge design with three steps, at least two clusters be randomised at each step, to ensure that the intervention effect estimator maintains the nominal 5% significance level and is also reasonably unbiased.

Stepped wedge cluster randomised trials: a review of the statistical methodology used and available

BACKGROUND

Previous reviews have focussed on the rationale for employing the stepped wedge design (SWD), the areas of research to which the design has been applied and the general characteristics of the design. However these did not focus on the statistical methods nor addressed the appropriateness of sample size methods used.This was a review of the literature of the statistical methodology used in stepped wedge cluster randomised trials.

METHODS

Literature Review. The Medline, Embase, PsycINFO, CINAHL and Cochrane databases were searched for methodological guides and RCTs which employed the stepped wedge design.

RESULTS

This review identified 102 trials which employed the stepped wedge design compared to 37 from the most recent review by Beard et al. 2015. Forty six trials were cohort designs and 45 % (n = 46) had fewer than 10 clusters. Of the 42 articles discussing the design methodology 10 covered analysis and seven covered sample size. For cohort stepped wedge designs there was only one paper considering analysis and one considering sample size methods. Most trials employed either a GEE or mixed model approach to analysis (n = 77) but only 22 trials (22 %) estimated sample size in a way which accounted for the stepped wedge design that was subsequently used.

CONCLUSIONS

Many studies which employ the stepped wedge design have few clusters but use methods of analysis which may require more clusters for unbiased and efficient intervention effect estimates. There is the need for research on the minimum number of clusters required for both types of stepped wedge design. Researchers should distinguish in the sample size calculation between cohort and cross sectional stepped wedge designs. Further research is needed on the effect of adjusting for the potential confounding of time on the study power.

Issues of cost-benefit and cost-effectiveness for simulation in health professions education

Background

Simulation education can be costly—however, costs need to be considered against what you get in return to determine whether these costs are justified. Unfortunately in simulation education, evaluations that yield information about the return on investment are scarce. An economic evaluation provides a comparison of value. In short—what is it that is being obtained, what do you need to give up to get it, and how does that compare to what you get with the next best alternative? When educators are equipped with this knowledge, they will be better informed to know the place that simulation-based learning approaches should take in optimal course structures.

Main body

This article provides an overview of the costs and consequences associated with simulation in healthcare education. It provides an outline of the benefits of using economic evaluations to inform decision-making by educators and clinicians concerning the most appropriate educational approaches. It also provides guidance for educational researchers interested in investigating the cost and value of their innovations.

Conclusion

Measures of cost and value in simulation are required to provide information about the viability and sustainability of simulation education, enabling simulation education in health care to demonstrate its worth.

Embedding research culture and productivity in hospital physiotherapy departments: challenges and opportunities

Few studies have investigated research culture in the Australian hospital system. Although physiotherapists working in tertiary hospital departments conduct and publish research, a conflict between service delivery and research productivity remains. Few departments record research achievements, which limits the accuracy of investigating factors associated with research productivity within allied health. The conduct and translation of research within acute physiotherapy and allied health departments is imperative to improve patient health outcomes, optimise health service efficiency and cost-effectiveness and to improve staff and patient satisfaction and staff retention. Allied health departments should institute a research register and consider implementing other strategies to improve research culture and productivity, such as dedicating equivalent full-time staff to research, supporting staff with joint clinical and academic appointments, ensuring a research register is available and used and having events available for the dissemination of research. Future research should focus on improving research productivity within acute allied health departments to provide Level 1 and 2 evidence of service effectiveness and cost-effectiveness to optimise health care delivery and to maximise the benefit of allied health staff to Australia's healthcare system.

DISINVESTING FROM INEFFECTIVE TECHNOLOGIES: LESSONS LEARNED FROM CURRENT PROGRAMS

OBJECTIVES

Many of the currently used health technologies have never been systematically assessed or are misused, overused or superseded. Therefore, they may be ineffective. Active identification of ineffectiveness in health care is gaining importance to facilitate best care for patients and optimal use of limited resources. The present research analyzed processes and experiences of programs for identifying ineffective health technologies. The goal of this study was to elucidate factors that facilitate implementation.

METHODS

Based on an overview article, a systematic literature search and unsystematic hand-search were conducted. Further information was gained from international experts.

RESULTS

Seven programs were identified that include identification, prioritization and assessment of ineffective health technologies and dissemination of recommendations. The programs are quite similar regarding their goals, target groups and criteria for identification and prioritization. Outputs, mainly HTA reports or lists, are mostly disseminated by means of the internet. Top-down and bottom-up programs both have benefits in terms of implementation of recommendations, either as binding guidelines and decisions or as nonbinding information for physicians and other stakeholders. Crucial facilitators of implementation are political will, transparent processes and broad stakeholder involvement focusing on physicians.

CONCLUSIONS

All programs can improve the quality of health care and enable cost reduction in supportive surrounding conditions. Physicians and patients must be continuously involved in the process of evaluating health technologies. Additionally, decision makers must support programs and translate recommendations into concrete actions.

Towards understanding the de-adoption of low-value clinical practices: a scoping review

BACKGROUND

Low-value clinical practices are common in healthcare, yet the optimal approach to de-adopting these practices is unknown. The objective of this study was to systematically review the literature on de-adoption, document current terminology and frameworks, map the literature to a proposed framework, identify gaps in our understanding of de-adoption, and identify opportunities for additional research.

METHODS

MEDLINE, EMBASE, the Cochrane Central Register of Controlled Trials, the Cochrane Database of Systematic Reviews, the Cochrane Database of Abstracts and Reviews of Effects, and CINAHL Plus were searched from 1 January 1990 to 5 March 2014. Additional citations were identified from bibliographies of included citations, relevant websites, the PubMed 'related articles' function, and contacting experts in implementation science. English-language citations that referred to de-adoption of clinical practices in adults with medical, surgical, or psychiatric illnesses were included. Citation selection and data extraction were performed independently and in duplicate.

RESULTS

From 26,608 citations, 109 were included in the final review. Most citations (65%) were original research with the majority (59%) published since 2010. There were 43 unique terms referring to the process of de-adoption-the most frequently cited was "disinvest" (39% of citations). The focus of most citations was evaluating the outcomes of de-adoption (50%), followed by identifying low-value practices (47%), and/or facilitating de-adoption (40%). The prevalence of low-value practices ranged from 16% to 46%, with two studies each identifying more than 100 low-value practices. Most articles cited randomized clinical trials (41%) that demonstrate harm (73%) and/or lack of efficacy (63%) as the reason to de-adopt an existing clinical practice. Eleven citations described 13 frameworks to guide the de-adoption process, from which we developed a model for facilitating de-adoption. Active change interventions were associated with the greatest likelihood of de-adoption.

CONCLUSIONS

This review identified a large body of literature that describes current approaches and challenges to de-adoption of low-value clinical practices. Additional research is needed to determine an ideal strategy for identifying low-value practices, and facilitating and sustaining de-adoption. In the meantime, this study proposes a model that providers and decision-makers can use to guide efforts to de-adopt ineffective and harmful practices.

Study protocol for two randomized controlled trials examining the effectiveness and safety of current weekend allied health services and a new stakeholder-driven model for acute medical/surgical patients versus no weekend allied health services

Background

Disinvestment from inefficient or ineffective health services is a growing priority for health care systems. Provision of allied health services over the weekend is now commonplace despite a relative paucity of evidence supporting their provision. The relatively high cost of providing this service combined with the paucity of evidence supporting its provision makes this a potential candidate for disinvestment so that resources consumed can be used in other areas.

This study aims to determine the effectiveness, cost-effectiveness and safety of the current model of weekend allied health service and a new stakeholder-driven model of weekend allied health service delivery on acute medical and surgical wards compared to having no weekend allied health service.

Methods/Design

Two stepped wedge, cluster randomised trials of weekend allied health services will be conducted in six acute medical/surgical wards across two public metropolitan hospitals in Melbourne (Australia). Wards have been chosen to participate by management teams at each hospital. The allied health services to be investigated will include physiotherapy, occupational therapy, speech therapy, dietetics, social work and allied health assistants. At baseline, all wards will be receiving weekend allied health services. Study 1 intervention will be the sequential disinvestment (roll-in) of the current weekend allied health service model from each participating ward in monthly intervals and study 2 will be the roll-out of a new stakeholder-driven model of weekend allied health service delivery. The order in which weekend allied health services will be rolled in and out amongst participating wards will be determined randomly. This trial will be conducted in each of the two participating hospitals at a different time interval. Primary outcomes will be length of stay, rate of unplanned hospital readmission within 28 days and rate of adverse events. Secondary outcomes will be number of complaints and compliments, staff absenteeism, and patient discharge destination, satisfaction, and functional independence at discharge.

Discussion

This is the world’s first application of the recently described non-inferiority (roll-in) stepped wedge trial design, and the largest investigation of the effectiveness of weekend allied health services on acute medical surgical wards to date.

Trial registration

Australian New Zealand Clinical Trials Registry.

Registration number: ACTRN12613001231730 (first study) and ACTRN12613001361796 (second study).

Was this trial prospectively registered?: Yes.

Date registered: 8 November 2013 (first study), 12 December 2013 (second study).

Anticipated completion: June 2015.

Protocol version: 1.

Role of trial sponsor: KP and DL are directly employed by one of the trial sponsors, their roles were: KP assisted with overall development of research design and assisted with overall project management; DL contributed to project management, administration and communications strategy.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-015-0619-z) contains supplementary material, which is available to authorized users.

Future of specialised roles in allied health practice: who is responsible?

Allied health professions have developed specialised advanced and extended scope roles over the past decade, for the benefit of patient outcomes, allied health professionals’ satisfaction and to meet labour and workforce demands. There is an essential need for formalised, widely recognised training to support these roles, and significant challenges to the delivery of such training exist. Many of these roles function in the absence of specifically defined standards of clinical practice and it is unclear where the responsibility for training provision lies. In a case example of physiotherapy practice in the intensive care unit, clinical placements and independence of practice are not core components of undergraduate physiotherapy degrees. Universities face barriers to the delivery of postgraduate specialised training and, although hospital physiotherapy departments are ideally placed, resources for training are lacking and education is not traditionally considered part of healthcare service providers’ core business. Substantial variability in training, and its evaluation, leads to variability in practice and may affect patient outcomes. Allied health professionals working in specialised roles should develop specific clinical standards of practice, restructure models of health care delivery to facilitate training, continue to develop the evidence base for their roles and target and evaluate training efficacy to achieve independent practice in a cost-effective manner. Healthcare providers must work with universities, the vocational training sector and government to optimise the ability of allied health to influence decision making and care outcomes for patients.

Stepped-wedge cluster randomised controlled trials: a generic framework including parallel and multiple-level designs

Stepped-wedge cluster randomised trials (SW-CRTs) are being used with increasing frequency in health service evaluation. Conventionally, these studies are cross-sectional in design with equally spaced steps, with an equal number of clusters randomised at each step and data collected at each and every step. Here we introduce several variations on this design and consider implications for power.

One modification we consider is the incomplete cross-sectional SW-CRT, where the number of clusters varies at each step or where at some steps, for example, implementation or transition periods, data are not collected. We show that the parallel CRT with staggered but balanced randomisation can be considered a special case of the incomplete SW-CRT. As too can the parallel CRT with baseline measures. And we extend these designs to allow for multiple layers of clustering, for example, wards within a hospital. Building on results for complete designs, power and detectable difference are derived using a Wald test and obtaining the variance–covariance matrix of the treatment effect assuming a generalised linear mixed model. These variations are illustrated by several real examples.

We recommend that whilst the impact of transition periods on power is likely to be small, where they are a feature of the design they should be incorporated. We also show examples in which the power of a SW-CRT increases as the intra-cluster correlation (ICC) increases and demonstrate that the impact of the ICC is likely to be smaller in a SW-CRT compared with a parallel CRT, especially where there are multiple levels of clustering. Finally, through this unified framework, the efficiency of the SW-CRT and the parallel CRT can be compared.