Implementing a protocol for a research impact assessment of the Centre for Research Excellence in Stroke Rehabilitation and Brain Recovery

What impact has the Centre of Research Excellence in Digestive Health made in the field of gastrointestinal health in Australia and internationally? Study protocol for impact evaluation using the FAIT framework

INTRODUCTION

The need for public research funding to be more accountable and demonstrate impact beyond typical academic outputs is increasing. This is particularly challenging and the science behind this form of research is in its infancy when applied to collaborative research funding such as that provided by the Australian National Health and Medical Research Council to the Centre for Research Excellence in Digestive Health (CRE-DH).

METHODS AND ANALYSIS

In this paper, we describe the protocol for applying the Framework to Assess the Impact from Translational health research to the CRE-DH. The study design involves a five-stage sequential mixed-method approach. In phase I, we developed an impact programme logic model to map the pathway to impact and establish key domains of benefit such as knowledge advancement, capacity building, clinical implementation, policy and legislation, community and economic impacts. In phase 2, we have identified and selected appropriate, measurable and timely impact indicators for each of these domains and established a data plan to capture the necessary data. Phase 3 will develop a model for cost-consequence analysis and identification of relevant data for microcosting and valuation of consequences. In phase 4, we will determine selected case studies to include in the narrative whereas phase 5 involves collation, data analysis and completion of the reporting of impact.We expect this impact evaluation to comprehensively describe the contribution of the CRE-DH for intentional activity over the CRE-DH lifespan and beyond to improve outcomes for people suffering with chronic and debilitating digestive disorders.

ETHICS AND DISSEMINATION

This impact evaluation study has been registered with the Hunter New England Human Research Ethics Committee as project 2024/PID00336 and ethics application 2024/ETH00290. Results of this study will be disseminated via medical conferences, peer-reviewed publications, policy submissions, direct communication with relevant stakeholders, media and social media channels such as X (formely Twitter).

Impact assessment of the Centre for Research Excellence in Stroke Rehabilitation and Brain Recovery

BACKGROUND

Research impact is an emerging measure of research achievement alongside traditional academic outputs such as publications. We present the results of applying the Framework to Assess the Impact from Translational health research (FAIT) to the Centre for Research Excellence (CRE) in Stroke Rehabilitation and Brain Recovery (CRE-Stroke, 2014-2019) and report on the feasibility and lessons from the application of FAIT to a CRE rather than a discrete research project.

METHODS

Data were gathered via online surveys, in-depth interviews, document analysis and review of relevant websites/databases to report on the three major FAIT methods: the modified Payback Framework, an assessment of costs against monetized consequences, and a narrative account of the impact generated from CRE-Stroke activities. FAIT was applied during the last 4 years of CRE-Stroke operation.

RESULTS

With an economic investment of AU$ 3.9 million over 5 years, CRE-Stroke delivered a return on investment that included AU$ 18.8 million in leveraged grants, fellowships and consultancies. Collectively, CRE-Stroke members produced 354 publications that were accessed 470,000 times and cited over 7220 times. CRE-Stroke supported 26 PhDs, 39 postdocs and seven novice clinician researchers. There were 59 capacity-building events benefiting 744 individuals including policy-makers and consumers. CRE-Stroke created research infrastructure (including a research register of stroke survivors and a brain biobank), and its global leadership produced international consensus recommendations to influence the stroke research landscape worldwide. Members contributed to the Australian Living Stroke Guidelines: four researchers' outputs were directly referenced. Based only on the consequences that could be monetized, CRE-Stroke returned AU$ 4.82 for every dollar invested in the CRE.

CONCLUSION

This case example in the developing field of impact assessment illustrates how researchers can use evidence to demonstrate and report the impact of and returns on research investment. The prospective application of FAIT by a dedicated research impact team demonstrated impact in broad categories of knowledge-gain, capacity-building, new infrastructure, input to policy and economic benefits. The methods can be used by other research teams to provide comprehensive evidence to governments and other research funders about what has been generated from their research investment but requires dedicated resources to complete.

Research protocol for impact assessment of a project to scale up food policies in the Pacific

BACKGROUND

One of the challenges for countries implementing food policy measures has been the difficulty in demonstrating impact and retaining stakeholder support. Consequently, research funded to help countries overcome these challenges should assess impact and translation into practice, particularly in low-resource settings. However, there are still few attempts to prospectively, and comprehensively, assess research impact. This protocol describes a study co-created with project implementers, collaborative investigators and key stakeholders to optimize and monitor the impact of a research project on scaling up food policies in Fiji.

METHODS

To develop this protocol, our team of researchers prospectively applied the Framework to Assess the Impact from Translational health research (FAIT). Activities included (i) developing a logic model to map the pathway to impact and establish domains of benefit; (ii) identifying process and impact indicators for each of these domains; (iii) identifying relevant data for impact indicators and a cost-consequence analysis; and (iv) establishing a process for collecting quantitative and qualitative data to measure progress. Impact assessment data will be collected between September 2022 and December 2024, through reports, routine monitoring activities, group discussions and semi-structured interviews with key implementers and stakeholders. The prospective application of the protocol, and interim and final research impact assessments of each project stream and the project as a whole, will optimize and enable robust measurement of research impact.

DISCUSSION

By applying this protocol, we aim to increase understanding of pathways to impact and processes that need to be put in place to achieve this. This impact evaluation will inform future projects with a similar scope and will identify transferable and/or translatable lessons for other Pacific Island states and low- and middle-income countries.

Effect of opposing needling on motor cortex excitability in healthy participants and in patients with post-stroke hemiplegia: study protocol for a single-blind, randomised controlled trial

Background

Opposing needling has an obvious curative effect in the treatment of post-stroke hemiplegia; however, the mechanism of the opposing needling in the treatment of post-stroke hemiplegia is still not clear. The purpose of this study is to investigate the effect of opposing needling on the excitability of primary motor cortex (M1) of healthy participants and patients with post-stroke hemiplegia, which may provide insight into the mechanisms of opposing needling in treating post-stroke hemiplegia.

Methods

This will be a single-blind, randomised, sham-controlled trial in which 80 healthy participants and 40 patients with post-stroke hemiplegia will be recruited. Healthy participants will be randomised 1:1:1:1 to the 2-Hz, 50-Hz, 100-Hz, and sham electroacupuncture groups. Patients with post-stroke hemiplegia will be randomised 1:1 to the opposing needling or conventional treatment groups. The M1 will be located in all groups by using neuroimaging-based navigation. The stimulator coil of transcranial magnetic stimulation (TMS) will be moved over the left and right M1 in order to identify the TMS hotspot, followed by a recording of resting motor thresholds (RMTs) and motor-evoked potentials (MEPs) of the thenar muscles induced by TMS before and after the intervention. The primary outcome measure will be the percent change in the RMTs of the thenar muscles at baseline and after the intervention. The secondary outcome measures will be the amplitude (μV) and latency (ms) of the MEPs of the thenar muscles at baseline and after the intervention.

Discussion

The aim of this trial is to explore the effect of opposing needling on the excitability of M1 of healthy participants and patients with post-stroke hemiplegia.

Trial registration

Chinese Clinical Trial Registry ChiCTR1900028138. Registered on 13 December 2019.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-021-05443-x.

A mixed-methods study to explore opinions of research translation held by researchers working in a Centre of Research Excellence in Australia

OBJECTIVE

There is a growing need for researchers to demonstrate impact, which is reliant on successful research translation. The Australian National Health and Medical Research Council funded a Centre of Research Excellence in Stroke Rehabilitation and Brain Recovery (CRE-Stroke) to enhance collaborations between researchers conducting different types of stroke rehabilitation research. The purpose of this study was to explore opinions about research translation held by CRE-Stroke researchers conducting preclinical and clinical research, in terms of scope, importance, responsibility and perceived skills and knowledge.

DESIGN

Mixed-methods study, comprising a paper-based survey and semistructured interviews. Interview data were inductively coded and thematically analysed. Survey and interview data were compared and synthesised.

PARTICIPANTS

55 (7 preclinical, 48 clinical) researchers attending a CRE-Stroke research forum completed a paper-based survey. Semistructured interviews with 22 CRE-Stroke (5 preclinical, 17 clinical) researchers were conducted.

RESULTS

Research translation was described as translating to other research and translating to clinical practice and policy. Most researchers (n=54, 98%) reported that research translation was important, particularly in terms of generating research impact, but the most common sign of project completion reported by researchers (n=7, 100% preclinical; n=37, 77% clinical) was publication. Most researchers (preclinical n=4, 57%; clinical n=37, 77%) reported having responsibility for translating research, but less than half reported having the necessary skills (n=1, 14% preclinical; n=17, 35% clinical) and knowledge (n=3, 43% preclinical; n=19, 40% clinical). Differing opinions about who should be responsible for translating findings to clinical practice were expressed.

CONCLUSIONS

Stroke rehabilitation researchers appear confident to translate their research via the traditional mechanism of publications. To optimise impact, clarity is needed regarding who is best placed to translate research findings to clinical practice and policy. Education and skills development to apply broader translation processes are needed to maximise the use of research at all stages.