Developmental evaluation as a strategy to enhance the uptake and use of deprescribing guidelines: protocol for a multiple case study

Connecting the science and practice of implementation - applying the lens of context to inform study design in implementation research

The saying "horses for courses" refers to the idea that different people and things possess different skills or qualities that are appropriate in different situations. In this paper, we apply the analogy of "horses for courses" to stimulate a debate about how and why we need to get better at selecting appropriate implementation research methods that take account of the context in which implementation occurs. To ensure that implementation research achieves its intended purpose of enhancing the uptake of research-informed evidence in policy and practice, we start from a position that implementation research should be explicitly connected to implementation practice. Building on our collective experience as implementation researchers, implementation practitioners (users of implementation research), implementation facilitators and implementation educators and subsequent deliberations with an international, inter-disciplinary group involved in practising and studying implementation, we present a discussion paper with practical suggestions that aim to inform more practice-relevant implementation research.

Building a practice-based research network for healthcare integration: a protocol paper for a mixed-method project

INTRODUCTION

Practice-based research networks (PBRNs) are sustained collaborations between healthcare professionals, researchers and members of the community that develop, conduct and report on research relevant to local needs. While PBRNs have traditionally been focused towards primary care practices and their patients, there has been increasing interest in how they may help facilitate healthcare integration. Yet, little is known on the ways in which PBRNs can best integrate with the broader healthcare system, in particular Advanced Health Research and Translation Centres. The overall project aim is to build a sustainable collaboration between a PBRN and an Advanced Health Research and Translation Centre to generate a research platform suitable for planning, undertaking and translating research to improve care across the healthcare continuum.

METHODS AND ANALYSIS

We will use a developmental evaluation design. Our iterative approach will be informed by a programme logic model and consists of: preparation work (pre-implementation assessment, literature review, community and stakeholder engagement), adaptation and building for a sustainable collaboration (strategy for recruitment and sustainment of members) and planning for network action (designing and implementing priority initiatives, monitoring and follow-up).

ETHICS AND DISSEMINATION

This project was approved by the Monash Health ethics committee (ERM Reference Number: 76281; Monash Health Ref: RES-21-0000-392L) and the Monash University Human Research ethics committee (Reference Number: 29786). Dissemination will take place via various channels, including relevant national and international committees and conferences, peer-reviewed journals and social media. Continuous dissemination to and communication with all participants in this project as well as other relevant stakeholders will help strengthen and sustain the network.

Identifying Best Implementation Practices for Smoking Cessation in Complex Cancer Settings

Background: In response to evidence about the health benefits of smoking cessation at time of cancer diagnosis, Ontario Health (Cancer Care Ontario) (OH-CCO) instructed Regional Cancer Centres (RCC) to implement smoking cessation interventions (SCI). RCCs were given flexibility to implement SCIs according to their context but were required to screen new patients for tobacco status, advise patients about the importance of quitting, and refer patients to cessation supports. The purpose of this evaluation was to identify practices that influenced successful implementation across RCCs. Methods: A realist evaluation approach was employed. Realist evaluations examine how underlying processes of an intervention (mechanisms) in specific settings (contexts) interact to produce results (outcomes). A realist evaluation may thus help to generate an understanding of what may or may not work across contexts. Results: The RCCs with the highest Tobacco Screening Rates used a centralized system. Regarding the process for advising and referring, three RCCs offered robust smoking cessation training, resulting in advice and referral rates between 80% and 100%. Five RCCs surpassed the target for Accepted Referral Rates; acceptance rates for internal referral were highest overall. Conclusion: Findings highlight factors that may influence successful SCI implementation.

Practical Considerations for the Implementation of a Stereotactic Body Radiation Therapy Program for Oligo-Metastases

Purpose

With multiple phase 2 trials supporting the use of stereotactic body radiation therapy (SBRT) in oligo-metastatic disease, we evaluated practices that could inform effective implementation of an oligo-metastasis SBRT program.

Methods and Materials

Using a context-focused realist methodology, an advisory committee of interprofessional clinicians met over a series of semistructured teleconference meetings to identify challenges in implementing an oligo-metastasis SBRT program. Consideration was given to 2 models of care: a subspecialist anatomic expertise model versus a single-practitioner “quarterback” model.

Results

The advisory committee structured recommendations within a context-mechanism-outcome framework. In summary, the committee recommends that during patient workup, a single practitioner arranges the minimum number of necessary tests, with case presentation at an appropriate multidisciplinary tumor board, including careful review of all previous treatments, and enrollment on clinical trials when possible. At simulation, common patient positions and immobilization on a single simulation scan for multiple sites is recommended. During radiation planning, dose-fractionation regimens should safely facilitate cumulative dose calculations, a single isocenter should be considered for multiple close targets to reduce treatment time, and adherence to strict quality assurance protocols is strongly recommended. Treatment duration should be minimized by treating multiple sites on the same day or choosing shorter dose fractionations. Team communication, thorough documentation, and standardized nomenclature can reduce system errors. Follow-up should aim to minimize redundant clinical appointments and imaging scans. Expert radiology review may be required to interpret post-SBRT imaging.

Conclusions

These guidelines inform best clinical practices for implementing an oligo-metastasis SBRT program. Iterations using a realist approach may further expand on local contexts.

The Service of Research Analytics to Optimize Digital Health Evidence Generation: Multilevel Case Study

Background

The widespread adoption of digital health interventions for chronic disease self-management has catalyzed a paradigm shift in the selection of methodologies used to evidence them. Recently, the application of digital health research analytics has emerged as an efficient approach to evaluate these data-rich interventions. However, there is a growing mismatch between the promising evidence base emerging from analytics mediated trials and the complexity of introducing these novel research methods into evaluative practice.

Objective

This study aimed to generate transferable insights into the process of implementing research analytics to evaluate digital health interventions. We sought to answer the following two research questions: (1) how should the service of research analytics be designed to optimize digital health evidence generation? and (2) what are the challenges and opportunities to scale, spread, and sustain this service in evaluative practice?

Methods

We conducted a qualitative multilevel embedded single case study of implementing research analytics in evaluative practice that comprised a review of the policy and regulatory climate in Ontario (macro level), a field study of introducing a digital health analytics platform into evaluative practice (meso level), and interviews with digital health innovators on their perceptions of analytics and evaluation (microlevel).

Results

The practice of research analytics is an efficient and effective means of supporting digital health evidence generation. The introduction of a research analytics platform to evaluate effective engagement with digital health interventions into a busy research lab was ultimately accepted by research staff, became routinized in their evaluative practice, and optimized their existing mechanisms of log data analysis and interpretation. The capacity for research analytics to optimize digital health evaluations is highest when there is (1) a collaborative working relationship between research client and analytics service provider, (2) a data-driven research agenda, (3) a robust data infrastructure with clear documentation of analytic tags, (4) in-house software development expertise, and (5) a collective tolerance for methodological change.

Conclusions

Scientific methods and practices that can facilitate the agile trials needed to iterate and improve digital health interventions warrant continued implementation. The service of research analytics may help to accelerate the pace of digital health evidence generation and build a data-rich research infrastructure that enables continuous learning and evaluation.

Optimizing Practices, Use, Care and Services-Antipsychotics (OPUS-AP) in Long-term Care Centers in Québec, Canada: A Strategy for Best Practices

OBJECTIVES

Antipsychotic medications are often used for the first-line management of behavioral and psychological symptoms of dementia (BPSD) contrary to guideline recommendations. The Optimizing Practices, Use, Care and Services-Antipsychotics (OPUS-AP) strategy aims to improve the well-being of long-term care (LTC) residents with major neurocognitive disorder (MNCD) by implementing a resident-centered approach, nonpharmacologic interventions, and antipsychotic deprescribing in inappropriate indications.

DESIGN

Prospective, closed cohort supplemented by a developmental evaluation.

SETTING AND PARTICIPANTS

Residents of designated wards in 24 LTC centers in Québec, Canada.

METHODS

Provincial guidelines were disseminated, followed by the implementation of an integrated knowledge translation and mobilization strategy, including training, coaching, clinical tools, evaluation of clinical practices, and a change management strategy. Antipsychotic, benzodiazepine, and antidepressant prescriptions; BPSD; and falls were evaluated every 3 months, for 9 months, from January to October 2018. Semistructured interviews (n = 20) were conducted with LTC teams to evaluate the implementation of OPUS-AP.

RESULTS

Of 1054 residents, 78.3% had an MNCD diagnosis and 51.7% an antipsychotic prescription. The cohort included 464 residents with both MNCD and antipsychotic prescription. Antipsychotic deprescribing (cessation or dose decrease) was attempted in 220 of the 344 residents still admitted at 9 months. Complete cessation was observed in 116 of these residents (52.7%) and dose reduction in 72 (32.7%), for a total of 188 residents (85.5%; 95% confidence interval: 80.1%, 89.8%). A decrease in benzodiazepine prescriptions and improvements in Cohen-Mansfield Agitation Inventory scores were observed among residents who had their antipsychotics deprescribed. Caregivers and clinicians expressed satisfaction as a result of observing an improved quality of life among residents.

CONCLUSIONS AND IMPLICATIONS

Antipsychotic deprescribing was successful in a vast majority of LTC residents with MNCD without worsening of BPSD. Based on this success, phase 2 of OPUS-AP is now under way in 129 LTC centers in Québec.

Using developmental evaluation to support knowledge translation: reflections from a large-scale quality improvement project in Indigenous primary healthcare

BACKGROUND

Developmental evaluation is a growing area of evaluation practice, advocated for informing the adaptive development of change initiatives in complex social environments. The utilisation focus, complexity perspective and systems thinking of developmental evaluation suggest suitability for evaluating knowledge translation initiatives in primary healthcare. However, there are few examples in the literature to guide its use in these contexts and in Indigenous settings. In this paper, we reflect on our experience of using developmental evaluation to implement a large-scale knowledge translation research project in Australian Aboriginal and Torres Strait Islander primary healthcare. Drawing on principles of knowledge translation and key features of developmental evaluation, we debate the key benefits and challenges of applying this approach to engage diverse stakeholders in using aggregated quality improvement data to identify and address persistent gaps in care delivery.

DISCUSSION

The developmental evaluation enabled the team to respond to stakeholder feedback and apply learning in real-time to successfully refine theory-informed research and engagement processes, tailor the presentation of findings to stakeholders and context, and support the project's dissemination and knowledge co-production aim. It thereby contributed to the production of robust, useable research findings for informing policy and system change. The use of developmental evaluation appeared to positively influence stakeholders' use of the project reports and their responses to the findings. Challenges included managing a high volume of evaluation data and multiple evaluation purposes, balancing facilitative sense-making processes and change with task-focused project management, and lack of experience in using this evaluation approach. Use of an embedded evaluator with facilitation skills and background knowledge of the project helped to overcome these challenges, as did similarities observed between features of developmental evaluation and continuous quality improvement.

CONCLUSION

Our experience of developmental evaluation confirmed our expectations of the potential value of this approach for strengthening improvement interventions and implementation research, and particularly for adapting healthcare innovations in Indigenous settings. In our project, developmental evaluation successfully encompassed evaluation, project adaptation, capacity development and knowledge translation. Further work is warranted to apply this approach more widely to improve primary healthcare initiatives and outcomes, and to evaluate implementation research.

A developmental evaluation to enhance stakeholder engagement in a wide-scale interactive project disseminating quality improvement data: study protocol for a mixed-methods study

INTRODUCTION

Bringing together continuous quality improvement (CQI) data from multiple health services offers opportunities to identify common improvement priorities and to develop interventions at various system levels to achieve large-scale improvement in care. An important principle of CQI is practitioner participation in interpreting data and planning evidence-based change. This study will contribute knowledge about engaging diverse stakeholders in collaborative and theoretically informed processes to identify and address priority evidence-practice gaps in care delivery. This paper describes a developmental evaluation to support and refine a novel interactive dissemination project using aggregated CQI data from Aboriginal and Torres Strait Islander primary healthcare centres in Australia. The project aims to effect multilevel system improvement in Aboriginal and Torres Strait Islander primary healthcare.

METHODS AND ANALYSIS

Data will be gathered using document analysis, online surveys, interviews with participants and iterative analytical processes with the research team. These methods will enable real-time feedback to guide refinements to the design, reports, tools and processes as the interactive dissemination project is implemented. Qualitative data from interviews and surveys will be analysed and interpreted to provide in-depth understanding of factors that influence engagement and stakeholder perspectives about use of the aggregated data and generated improvement strategies. Sources of data will be triangulated to build up a comprehensive, contextualised perspective and integrated understanding of the project's development, implementation and findings.

ETHICS AND DISSEMINATION

The Human Research Ethics Committee (HREC) of the Northern Territory Department of Health and Menzies School of Health Research (Project 2015-2329), the Central Australian HREC (Project 15-288) and the Charles Darwin University HREC (Project H15030) approved the study. Dissemination will include articles in peer-reviewed journals, policy and research briefs. Results will be presented at conferences and quality improvement network meetings. Researchers, clinicians, policymakers and managers developing evidence-based system and policy interventions should benefit from this research.

Deprescribing: A narrative review of the evidence and practical recommendations for recognizing opportunities and taking action

Deprescribing can be defined as the process of withdrawal or dose reduction of medications which are considered inappropriate in an individual. The aim of this narrative review is to provide an overview of "deprescribing"; firstly discussing the potential benefits and harms followed by the barriers to and enablers of deprescribing. We also provide practical recommendations to recognise opportunities and strategies for deprescribing in practice. Studies focused on minimizing polypharmacy indicate that deprescribing may be associated with potential benefits including resolution of adverse drug reactions, improved quality of life and medication adherence and a reduction in drug costs. While the data on the benefits is inconsistent, deprescribing appears to be safe. There are, however, potential harms including return of medical conditions or symptoms and adverse drug withdrawal reactions which emphasise the need for the process to be supervised and monitored by a health care professional. Taking action on deprescribing can be facilitated by knowledge of potential barriers, implementing a deprescribing process (utilising developed tools and resources) and identifying opportunities for deprescribing through engaging with patients and caregivers and other health care professionals and considering deprescribing in a variety of populations. Important areas for future research include the suitability of deprescribing of certain medications in specific populations, how to implement deprescribing processes into clinical care in a feasible and cost effective manner and how to engage consumers throughout the process to achieve positive health and quality of life outcomes.

Self-efficacy for deprescribing: A survey for health care professionals using evidence-based deprescribing guidelines

BACKGROUND

Although polypharmacy is associated with significant morbidity, deprescribing can be challenging. In particular, clinicians express difficulty with their ability to deprescribe (i.e. reduce or stop medications that are potentially inappropriate). Evidence-based deprescribing guidelines are designed to help clinicians take action on reducing or stopping medications that may be causing more harm than benefit.

OBJECTIVES

Determine if implementation of evidence-based guidelines increases self-efficacy for deprescribing proton pump inhibitor (PPI), benzodiazepine receptor agonist (BZRA) and antipsychotic (AP) drug classes.

METHODS

A deprescribing self-efficacy survey was developed and administered to physicians, nurse practitioners and pharmacists at 3 long-term care (LTC) and 3 Family Health Teams in Ottawa, Canada at baseline and approximately 6 months after sequential implementation of each guideline. For each drug class, overall and domain-specific self-efficacy mean scores were calculated. The effects of implementation of each guideline on self-efficacy were tested by estimating the difference in scores using paired t-test. A linear mixed-effects model was used to investigate change over time and over practice sites.

RESULTS

Of eligible clinicians, 25, 21, 18 and 13 completed the first, second, third and fourth survey respectively. Paired t-tests compared 14 participants for PPI and BZRA, and 9 for AP. Overall self-efficacy score increased for AP only (95% confidence intervals (CI) 0.32 to 19.79). Scores for domain 2 (develop a plan to deprescribe) increased for PPI (95% CI 0.52 to 24.12) and AP guidelines (95% CI 2.46 to 18.11); scores for domain 3 (implement the plan for deprescribing) increased for the PPI guideline (95% CI 0.55 to 14.24). Longitudinal analysis showed an increase in non-class specific scores, with a more profound effect for clinicians in LTC where guidelines were routinely used.

CONCLUSION

Implementation of evidence-based deprescribing guidelines appears to increase clinicians' self-efficacy in developing and implementing a deprescribing plan for specific drug classes.

An Ecological Approach to Reducing Potentially Inappropriate Medication Use: Canadian Deprescribing Network

Polypharmacy is growing in Canada, along with adverse drug events and drug-related costs. Part of the solution may be deprescribing, the planned and supervised process of dose reduction or stopping of medications that may be causing harm or are no longer providing benefit. Deprescribing can be a complex process, involving the intersection of patients, health care providers, and organizational and policy factors serving as enablers or barriers. This article describes the justification, theoretical foundation, and process for developing a Canadian Deprescribing Network (CaDeN), a network of individuals, organizations, and decision-makers committed to promoting the appropriate use of medications and non-pharmacological approaches to care, especially among older people in Canada. CaDeN will deploy multiple levels of action across multiple stakeholder groups simultaneously in an ecological approach to health system change. CaDeN proposes a unique model that might be applied both in national settings and for different transformational challenges in health care.

Deprescribing: a primary care perspective

Polypharmacy is an increasing and global issue affecting primary care. Although sometimes appropriate, polypharmacy can also be problematic, leading to a range of adverse consequences. Deprescribing is the process of supervised withdrawal of an inappropriate medication and has the potential to reduce some of the problems associated with polypharmacy. It is a complex and sensitive process. We examine the issue of deprescribing from the perspective of primary care. Key steps in the deprescribing process are a review of medications and corresponding indications, consideration of harms, assessment of eligibility for discontinuation, prioritisation of medications and implementation of a stopping plan with appropriate monitoring. Patient involvement is a key feature of this process. Deprescribing should be considered in the context of end-of-life care and medication safety, but approaches are also required to identify other situations where deprescribing is appropriate. General practitioners are well positioned to facilitate deprescribing, usually through formal medication review, with decisions informed by a range of other healthcare professionals. Guidelines are available that help guide these processes. A range of studies have explored attitudes towards deprescribing; patients are generally supportive of the concept, although clinician views are varied. The successful implementation of deprescribing strategies still requires important patient and clinician barriers to be overcome, and clinical trial evidence of effectiveness and safety is essential.

Parental acceptance of human papillomavirus vaccinations and community pharmacies as vaccination settings: A qualitative study in Alabama

Purpose

To determine parents’ knowledge and attitudes regarding human papillomavirus (HPV) vaccinations in their adolescent children and to describe parents’ perceptions of adolescent vaccinations in community pharmacies.

Methods

In-depth interviews were completed with parents or guardians of children ages 11–17 years from Alabama's Lee and Macon counties. One-hour long, open-ended telephonic or in-person interviews were conducted until the saturation point was reached. Using ATLAS.ti software and thematic analysis, interview transcripts were coded to identify themes.

Results

Twenty-six parents were interviewed, most of whom were female (80.8%) and white (50%). A total of 12 themes were identified. First, two themes emerged regarding elements facilitating children's HPV vaccination, the most common being positive perception of the HPV vaccine. Second, elements hindering children's vaccination contained seven themes, the top one being lack of correct or complete information about the HPV vaccine. The last topic involved acceptance/rejection of community pharmacies as vaccination settings, and the most frequently cited theme was concern about pharmacists’ clinical training.

Conclusions

Physician-to-parent vaccine education is important, and assurances of adequate pharmacy immunization training will ease parents’ fears and allow pharmacists to better serve adolescents, especially those who do not see physicians regularly.

•

Physicians play a crucial role in parents’ HPV vaccination decisions.

•

Parents are reluctant to use pharmacists as HPV vaccine providers.

•

Parents are concerned about pharmacists’ training and pharmacy infrastructure.

•

Community pharmacists must work in conjunction with physicians.

Review of structured guides for deprescribing

Avoiding inappropriate polypharmacy has become increasingly recognised as a safety imperative for older patient care. Deprescribing is an active process of reviewing all medications being used by individual patients that prompts clinicians to consider which medications have unfavourable risk-benefit trade-offs in the context of illness severity, advanced age, multi-morbidity, physical and emotional capacity, life expectancy, care goals and personal preferences. Structured guides to deprescribing include algorithms, flow charts or tables which are patient-directed and aim to guide the clinician through sequential steps in deciding which medications should be targeted for discontinuation. In this narrative review, we describe seven structured deprescribing guides whose stated purpose included the reduction of polypharmacy, their use was not restricted to a single drug or drug class and they had undergone some form of efficacy testing. There was considerable heterogeneity in guide design and content, with some guides constituting little more than a set of principles while others entail detailed processes and sub-steps which addressed multiple determinants of drug appropriateness. Evidence of effectiveness for each guide was limited in that none have been evaluated in randomised controlled trials, that pilot or feasibility studies have involved relatively small patient samples, that intra-rater and inter-rater reliabilities have not been determined and that most have been studied in hospital settings. Which is most useful to clinicians is unknown in the absence of head-to-head comparisons. While most guides have face validity, more research is needed for determining effectiveness and ease of use in routine clinical practice, especially in primary care settings.

Sex and gender-based analysis in pharmacy practice research: A scoping review

BACKGROUND

Recognizing the potential effect of sex and gender on health outcomes, there is a shift toward conducting sex and gender-based analysis (SGBA) within health research. However, little is known about the extent to which SGBA has been incorporated into pharmacy practice research.

OBJECTIVES

To understand the extent to which SGBA is included in pharmacy practice research.

METHOD

Scoping review of English-language studies identified through MEDLINE, Embase, International Pharmacy Abstracts (IPA), and CINAHL (inception to Jan 2014). Two raters independently screened citations to identify titles and abstracts that included key words related to sex or gender and studies that could be categorized as pharmacy practice research. One author extracted data from included studies related to study design, population, intervention/exposure and outcomes, with results reviewed by another. All authors reviewed eligible articles to categorize them based on a previously-developed typology, and to assess four criteria: 1) the inclusion of sex or gender in research objectives, 2) the depth of sex/gender analysis incorporated into study designs and reporting, 3) the inclusion of sex or gender considerations in interpretation of study results, 4) the intentional and accurate use of sex/gender language.

RESULTS

Of 458 unique search results, only six articles met the inclusion criteria. Two of these six publications included sex/gender considerations in a model consistent with sex/gender based analysis as described by Hammarström. Three of the six studies inaccurately applied sex and gender terminology, whereas the two studies that featured sex or gender in their primary research question did use these terms appropriately.

CONCLUSION

Despite increasing attention on the need for considering sex and gender, there was a paucity of pharmacy practice research publications that conducted SGBA. This presents an opportunity to explore sex, gender and intersectionality when pursuing studies that explore the impact of pharmacists interventions on patient outcomes.

Methodology for Developing Deprescribing Guidelines: Using Evidence and GRADE to Guide Recommendations for Deprescribing

BACKGROUND

Class specific deprescribing guidelines could help clinicians taper and stop medications no longer needed or which may be causing more harm than benefit. We set out to develop methodology to create such guidelines using evidence-based methods for guideline development, evidence synthesis and recommendation rating.

METHODS AND FINDINGS

Using a comprehensive checklist for a successful guideline enterprise, we conducted a national modified Delphi consensus process to identify priorities for deprescribing guidelines, then conducted scoping exercises to identify feasible topics, and sequentially developed three deprescribing guidelines. We selected guideline development team members for clinical expertise; a GRADE member worked with staff to ensure guideline development processes were followed. We conducted or used systematic searches and reviews of deprescribing trials of selected drug classes, reviews or systematic reviews of drug class effectiveness, reviews of reviews of drug class harm and narrative syntheses of contextual questions to inform recommendations and guideline development. Our 8 step process for guideline development included defining scope and purpose, developing a logic model to guide the process and generate key clinical questions, setting criteria for admissible evidence and conducting systematic reviews, synthesizing evidence considering additional contextual information and performing quality estimates, formulating recommendations and providing strength estimations, adding clinical considerations, conducting clinical and stakeholder review and finally updating content pre-publication. Innovative aspects of the guideline development process included synthesizing evidence for outcomes of tapering or stopping medication, and incorporating evidence for medication harm into the recommendation strength rating. Through the development of three deprescribing guidelines (for proton pump inhibitors, benzodiazepine receptor agonists and antipsychotics) and associated decision-support algorithms, we were able to gradually hone the methodology; each guideline will be published separately.

CONCLUSION

Our methodology demonstrates the importance of searching for short and long-term outcomes, showing the benefits of deprescribing and studying patient preferences. This publication will support development of future deprescribing guidelines.