The role of scientific evidence in decisions to adopt complex innovations in cancer care settings: a multiple case study in Nova Scotia, Canada

Factors Influencing the Adoption of Magnetic Resonance-Guided High-Intensity Focused Ultrasound for Painful Bone Metastases in Europe, A Group Concept Mapping Study

Magnetic resonance imaging-guided high-intensity focused ultrasound (MR-HIFU) is an innovative treatment for patients with painful bone metastases. The adoption of MR-HIFU will be influenced by several factors beyond its effectiveness. To identify contextual factors affecting the adoption of MR-HIFU, we conducted a group concept mapping (GCM) study in four European countries. The GCM was conducted in two phases. First, the participants brainstormed statements guided by the focus prompt "One factor that may influence the uptake of MR-HIFU in clinical practice is...". Second, the participants sorted statements into categories and rated the statements according to their importance and changeability. To generate a concept map, multidimensional scaling and cluster analysis were conducted, and average ratings for each (cluster of) factors were calculated. Forty-five participants contributed to phase I and/or II (56% overall participation rate). The resulting concept map comprises 49 factors, organized in 12 clusters: "competitive treatments", "physicians' attitudes", "alignment of resources", "logistics and workflow", "technical disadvantages", "radiotherapy as first-line therapy", "aggregating knowledge and improving awareness", "clinical effectiveness", "patients' preferences", "reimbursement", "cost-effectiveness" and "hospital costs". The factors identified echo those from the literature, but their relevance and interrelationship are case-specific. Besides evidence on clinical effectiveness, contextual factors from 10 other clusters should be addressed to support adoption of MR-HIFU.

Monitor to innovate with feedback loops: process evaluation protocol for an anemia prevention intervention

Background: Digital process monitoring and evaluation tools designed to capture near-to-real-time intervention data paired with feedback loops have the potential to innovate intervention delivery.

Objective: To describe how a multilevel social norms field trial (RANI) is using feedback loops to enhance intervention delivery.

Methods: We use a mixed-methods process evaluation design to monitor the Reduction of Anemia through Normative Innovations (RANI) project; a three-year randomized control trial which aims to lower rates of anemia among women in Odisha, India. Surveys and structured observation monitor fidelity to implementation and acceptability of implementation activities among study participants. Quantitative data evaluates implementation dose, coverage, exposure, and reach of intervention activities, and qualitative data will delve more deeply into reasons for high or low functioning. Iron folic acid supplement supply and demand are also monitored for stock-outs. Data collected from 130 intervention villages is processed, visualized, and triangulated in near to real-time via Real-time Monitoring for Knowledge Generation (RPM4K), a locally developed software application. Data visualization products facilitate the examination of monitoring data to mitigate bottlenecks and identify and implement tweaks to our intervention delivery strategy on an ongoing basis.

Discussion: Feedback loops facilitate timely course corrections. Feedback loops can also engender a shared understanding of ground realities for a geographically dispersed and culturally diverse team. Leveraging feedback loops, we identify opportunities to provide on-going supportive supervision for our community facilitators promoting joint problem-solving, and communication. Monthly media and hemoglobin level demonstration strategies are informed by participant engagement and acceptability. Stock-outs of iron folic acid tablets activate contingency plans to mobilize local stakeholders and advocate for timely resolutions. Unintended effects are monitored based on ongoing feedback from community facilitators.

Conclusions: Documenting our processes can inform the future implementation or scale up of similar projects embracing feedback loops to iterate and innovate their intervention delivery.

Developing an implementation research logic model: using a multiple case study design to establish a worked exemplar

Background

Implementation science frameworks explore, interpret, and evaluate different components of the implementation process. By using a program logic approach, implementation frameworks with different purposes can be combined to detail complex interactions. The Implementation Research Logic Model (IRLM) facilitates the development of causal pathways and mechanisms that enable implementation. Critical elements of the IRLM vary across different study designs, and its applicability to synthesizing findings across settings is also under-explored. The dual purpose of this study is to develop an IRLM from an implementation research study that used case study methodology and to demonstrate the utility of the IRLM to synthesize findings across case sites.

Method

The method used in the exemplar project and the alignment of the IRLM to case study methodology are described. Cases were purposely selected using replication logic and represent organizations that have embedded exercise in routine care for people with cancer or mental illness. Four data sources were selected: semi-structured interviews with purposely selected staff, organizational document review, observations, and a survey using the Program Sustainability Assessment Tool (PSAT). Framework analysis was used, and an IRLM was produced at each case site. Similar elements within the individual IRLM were identified, extracted, and re-produced to synthesize findings across sites and represent the generalized, cross-case findings.

Results

The IRLM was embedded within multiple stages of the study, including data collection, analysis, and reporting transparency. Between 33-44 determinants and 36-44 implementation strategies were identified at sites that informed individual IRLMs. An example of generalized findings describing “intervention adaptability” demonstrated similarities in determinant detail and mechanisms of implementation strategies across sites. However, different strategies were applied to address similar determinants. Dependent and bi-directional relationships operated along the causal pathway that influenced implementation outcomes.

Conclusions

Case study methods help address implementation research priorities, including developing causal pathways and mechanisms. Embedding the IRLM within the case study approach provided structure and added to the transparency and replicability of the study. Identifying the similar elements across sites helped synthesize findings and give a general explanation of the implementation process. Detailing the methods provides an example for replication that can build generalizable knowledge in implementation research.

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-022-00337-8.

Examining the sustainability and effectiveness of co-created physical activity interventions in vocational education and training: a multimethod evaluation

Background

Co-creation approaches are increasingly used in physical activity promotion to develop interventions tailored to the target group and setting. The resulting complexity of such interventions raises challenges in evaluation. Accordingly, little is known about the effectiveness of co-created interventions and the underlying processes that impact their sustainable implementation. In this study, we attempt to fill this gap by evaluating co-created multi-component physical activity interventions in vocational education and training in nursing care and automotive mechatronics regarding (1) their sustainable implementation at the institutional level and (2) the effectiveness of single intervention components at the individual level.

Methods

Following a multimethod design, we conducted a questionnaire survey (n = 7) and semi-structured interviews (n = 4) to evaluate the sustainability of the interventions. Quantitative data were analyzed descriptively, and qualitative data were analyzed using qualitative content analysis. To examine the interventions’ effectiveness, we conducted two non-randomized controlled trials (n = 111). Analysis of variance was used to examine differences between groups.

Results

At the institutional level, long-term implementation of single intervention components in nursing care was observed; in contrast, long-term implementation in automotive mechatronics was not observed. In this context, various factors at the outer contextual (e.g., COVID-19 pandemic), inner contextual (e.g., health-promoting leadership), intervention (e.g., acceptance), and personal levels (e.g., champion) influenced sustainability. At the individual level, no significant intervention effects were found for changes in physical activity behavior and physical activity-related health competence.

Conclusion

The role of co-creation on the effectiveness and sustainability of physical activity promotion in vocational education and training cannot be answered conclusively. Only in the nursing care sector, a co-creation approach appeared promising for long-term intervention implementation. Sustainable implementation depends on various influencing factors that should be considered from the outset. Demonstrating effectiveness at the individual level was challenging. To conclusively clarify both the role and impact of co-creation, methodologically complex and elaborate evaluation designs will be required in future research projects.

Trial registration

This study was retrospectively registered at clinicaltrials.gov on 24/08/2021 (NCT05018559).

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-022-13133-9.

Monitor to innovate with feedback loops: process evaluation protocol for an anemia prevention intervention

Background: With the proliferation of the digital age, information and communication technologies paired with feedback loops have the potential to innovate process evaluations. Objective: To describe how a multilevel social norms field trial (RANI) is using feedback loops to enhance intervention delivery. Methods: We use a mixed-methods process evaluation design to monitor the Reduction of Anemia through Normative Innovations (RANI) project; a three-year randomized control trial which aims to lower rates of anemia among women in Odisha, India. Surveys and structured observation monitor fidelity to implementation and receptivity to implementation activities among study participants. Quantitative data evaluates implementation dose, coverage, exposure, and reach of intervention activities, and qualitative data will delve more deeply into reasons for high or low functioning. Iron folic acid supplement supply and demand are also monitored for stock-outs. Data collected from 130 intervention villages is processed, visualized, and triangulated in near to real-time via Real-time Monitoring for Knowledge Generation (RPM4K), a locally developed software application. Data visualization products facilitate the examination of monitoring data to mitigate bottlenecks and identify and implement tweaks to our intervention delivery strategy on an ongoing basis. Discussion: Feedback loops facilitate timely course corrections. Feedback loops can also engender a shared understanding of ground realities for a geographically dispersed and culturally diverse team. Leveraging feedback loops, we identify opportunities to provide on-going supportive supervision for our community facilitators promoting joint problem-solving, and communication. Monthly media and hemoglobin level demonstration strategies are informed by participant engagement and receptivity. Stock-outs of iron folic acid tablets activate contingency plans to mobilize local stakeholders and advocate for timely resolutions. Unintended effects are monitored based on ongoing feedback from community facilitators. Conclusions: Documenting our processes can inform the future implementation or scale up of similar projects embracing feedback loops to iterate and innovate their intervention delivery.

Stakeholder-Identified Interventions to Address Cancer Survivors' Psychosocial Needs after Completing Treatment

The interventions used in cancer-survivorship care do not always address outcomes important to survivors. This study sought to understand stakeholders’ views on the key concerns of cancer survivors after treatment and the interventions needed to meet survivors’ and families’ psychosocial needs after completing cancer treatment. We conducted a descriptive qualitative study using semi-structured interviews with stakeholders (survivors, family/friend caregivers, oncology providers, primary care providers, and cancer system decision-/policy-makers) from across Canada. For the data analysis, we used techniques commonly employed in descriptive qualitative research, such as coding, grouping, detailing, and comparing the data. There were 44 study participants: 11 survivors, seven family/friend caregivers, 18 health care providers, and eight decision-/policy-makers. Stakeholder-relevant interventions to address survivors’ psychosocial needs were categorized into five groups, as follows: information provision, peer support, navigation, knowledge translation interventions, and caregiver-specific supports. These findings, particularly interventions that deliver timely and relevant information about the post-treatment period and knowledge translation interventions that strive to integrate effective tools and programs into survivorship care, have implications for future research and practice.

Implementation of innovative medical technologies in German inpatient care: patterns of utilization and evidence development

Background

Innovative medical technologies are commonly associated with positive expectations. At the time of their introduction into care, there is often little evidence available regarding their benefits and harms. Accordingly, some innovative medical technologies with a lack of evidence are used widely until or even though findings of adverse events emerge, while others with study results supporting their safety and effectiveness remain underused. This study aims at examining the diffusion patterns of innovative medical technologies in German inpatient care between 2005 and 2017 while simultaneously considering evidence development.

Methods

Based on a qualitatively derived typology and a quantitative clustering of the adoption curves, a representative sample of 21 technologies was selected for further evaluation. Published scientific evidence on efficacy/effectiveness and safety of the technologies was identified and extracted in a systematic approach. Derived from a two-dimensional classification according to the degree of utilization and availability of supportive evidence, the diffusion patterns were then assigned to the categories “Success” (widespread/positive), “Hazard” (widespread/negative), “Overadoption” (widespread/limited or none), “Underadoption” (cautious/positive), “Vigilance” (cautious/negative), and “Prudence” (cautious/limited or none).

Results

Overall, we found limited evidence on the examined technologies regarding both the quantity and quality of published randomized controlled trials. Thus, the categories “Prudence” and “Overadoption” together account for nearly three-quarters of the years evaluated, followed by “Success” with 17%. Even when evidence is available, the transfer of knowledge into practice appears to be inhibited.

Conclusions

The successful implementation of safe and effective innovative medical technologies into practice requires substantial further efforts by policymakers to strengthen systematic knowledge generation and translation. Creating an environment that encourages the conduct of rigorous studies, promotes knowledge translation, and rewards innovative medical technologies according to their added value is a prerequisite for the diffusion of valuable health care.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-021-01159-3.

Implementation of targeted screening for poverty in a large primary care team in Toronto, Canada: a feasibility study

Background

Poverty has a significant influence on health. Efforts to optimize income and reduce poverty could make a difference to the lives of patients and their families. Routine screening for poverty in primary care is an important first step but rarely occurs in Canada. We aimed to implement a targeted screening and referral process in a large, distributed primary care team in Toronto, Ontario, Canada. The main outcome was the proportion of targeted patients screened.

Methods

This implementation evaluation was conducted with a large community-based primary care team in north Toronto. The primary care team serves relatively wealthy neighborhoods with pockets of poverty. Physicians were invited to participate. We implemented targeted screening by combining census information on neighborhood-level deprivation with postal codes in patient records. For physicians agreeing to participate, we added prompts to screen for poverty to the charts of adult patients living in the most deprived areas. Standardized electronic medical record templates recommended a referral to a team case worker for income optimization, for those patients screening positive. We recorded the number and percentages of participants at each stage, from screening to receiving advice on income optimization.

Results

128 targeted patients with at least one visit (25%) were screened. The primary care team included 86 physicians distributed across 19 clinical locations. Thirty-four physicians (39%) participated. Their practices provided care for 27,290 patients aged 18 or older; 852 patients (3%) were found to be living in the most deprived neighborhoods. 509 (60%) had at least one office visit over the 6 months of follow up. 25 patients (20%) screened positive for poverty, and 13 (52%) were referred. Eight patients (62% of those referred) were ultimately seen by a caseworker for income optimization.

Conclusions

We implemented a targeted poverty screening program combined with resources to optimize income for patients in a large, distributed community-based primary care team. Screening was feasible; however, only a small number of patients were linked to the intervention Further efforts to scale and spread screening and mitigation of poverty are warranted; these should include broadening the targeted population beyond those living in the most deprived areas.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12875-021-01514-9.

The use of molecular biomarker tests: an interview study with healthcare providers about a molecular biomarker test for prostate cancer

Aim: Investigate why healthcare providers are not always willing to use molecular biomarker tests, even though they promise to personalize disease diagnosis and treatment. Materials & methods: We interviewed 20 Dutch urological healthcare providers to ascertain why they used or did not use SelectMDx, a biomarker test for prostate cancer. Results: Whether and how it was used differed from the developers' expectations, because users and nonusers disagreed about its perceived advantages; the scientific and clinical evidence; the advantages of MRI; and the value of PCA3 testing. Financial issues and the absence of SelectMDx in professional guidelines and hospital care pathways also hampered its use. Conclusion: Eliciting users' and nonusers' views is important to better understand how biomarker tests can be embedded in clinical practice.

Lessons learned about the effective operationalization of champions as an implementation strategy: results from a qualitative process evaluation of a pragmatic trial

BACKGROUND

Though the knowledge base on implementation strategies is growing, much remains unknown about how to most effectively operationalize these strategies in diverse contexts. For example, while evidence shows that champions can effectively support implementation efforts in some circumstances, little has been reported on how to operationalize this role optimally in different settings, or on the specific pathways through which champions enact change.

METHODS

This is a secondary analysis of data from a pragmatic trial comparing implementation strategies supporting the adoption of guideline-concordant cardioprotective prescribing in community health centers in the USA. Quantitative data came from the community health centers' shared electronic health record; qualitative data sources included community health center staff interviews over 3 years. Using a convergent mixed-methods design, data were collected concurrently and merged for interpretation to identify factors associated with improved outcomes. Qualitative analysis was guided by the constant comparative method. As results from the quantitative and initial qualitative analyses indicated the essential role that champions played in promoting guideline-concordant prescribing, we conducted multiple immersion-crystallization cycles to better understand this finding.

RESULTS

Five community health centers demonstrated statistically significant increases in guideline-concordant cardioprotective prescribing. A combination of factors appeared key to their successful practice change: (1) A clinician champion who demonstrated a sustained commitment to implementation activities and exhibited engagement, influence, credibility, and capacity; and (2) organizational support for the intervention. In contrast, the seven community health centers that did not show improved outcomes lacked a champion with the necessary characteristics, and/or organizational support. Case studies illustrate the diverse, context-specific pathways that enabled or prevented study implementers from advancing practice change.

CONCLUSION

This analysis confirms the important role of champions in implementation efforts and offers insight into the context-specific mechanisms through which champions enact practice change. The results also highlight the potential impact of misaligned implementation support and key modifiable barriers and facilitators on implementation outcomes. Here, unexamined assumptions and a lack of evidence-based guidance on how best to identify and prepare effective champions led to implementation support that failed to address important barriers to intervention success.

TRIAL REGISTRATION

ClinicalTrials.gov , NCT02325531 . Registered 15 December 2014.

Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors

Objective

The objective was to identify barriers and facilitators to the implementation of artificial intelligence (AI) applications in clinical radiology in The Netherlands.

Materials and methods

Using an embedded multiple case study, an exploratory, qualitative research design was followed. Data collection consisted of 24 semi-structured interviews from seven Dutch hospitals. The analysis of barriers and facilitators was guided by the recently published Non-adoption, Abandonment, Scale-up, Spread, and Sustainability (NASSS) framework for new medical technologies in healthcare organizations.

Results

Among the most important facilitating factors for implementation were the following: (i) pressure for cost containment in the Dutch healthcare system, (ii) high expectations of AI’s potential added value, (iii) presence of hospital-wide innovation strategies, and (iv) presence of a “local champion.” Among the most prominent hindering factors were the following: (i) inconsistent technical performance of AI applications, (ii) unstructured implementation processes, (iii) uncertain added value for clinical practice of AI applications, and (iv) large variance in acceptance and trust of direct (the radiologists) and indirect (the referring clinicians) adopters.

Conclusion

In order for AI applications to contribute to the improvement of the quality and efficiency of clinical radiology, implementation processes need to be carried out in a structured manner, thereby providing evidence on the clinical added value of AI applications.

Key Points

• Successful implementation of AI in radiology requires collaboration between radiologists and referring clinicians.

• Implementation of AI in radiology is facilitated by the presence of a local champion.

• Evidence on the clinical added value of AI in radiology is needed for successful implementation.

Electronic supplementary material

The online version of this article (10.1007/s00330-020-06946-y) contains supplementary material, which is available to authorized users.