1
|
Bailie J, Matthews V, Laycock AF, Conte K, Feeney L, Bainbridge R. Reflecting on the quality of a methodologically pluralist evaluation of a large-scale Indigenous health research collaboration in Australia. BMJ Glob Health 2024; 9:e014433. [PMID: 39097294 PMCID: PMC11298732 DOI: 10.1136/bmjgh-2023-014433] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2023] [Accepted: 06/21/2024] [Indexed: 08/05/2024] Open
Abstract
BACKGROUND Indigenous communities worldwide lead calls for all evaluations of research, programmes and policies affecting their communities to reflect the values, priorities and perspectives of the Indigenous peoples and communities involved. Tools, such as the Quality Appraisal Tool (QAT), are available to assess research quality through an Indigenous cultural lens. Good evaluation requires that evaluation efforts be evaluated. We found that critical reflection on the quality of evaluations from an Indigenous perspective is largely absent from the published literature. To ensure that we strive for quality in evaluation as determined by Indigenous people with whom we work, we examined the quality of our own evaluation of an Indigenous health research collaboration by conducting a reflexive dialogue. METHODS The QAT was used to assess our evaluation according to Indigenous health research principles. Our qualitative study used analytical coautoethnography to generate data through a series of reflexive dialogue sessions with Indigenous and non-Indigenous members of the research collaboration, using the QAT criteria as discussion prompts. Our ideas and reflections were compared and contrasted through a collaborative and iterative writing process, multiple review cycles and discussions. RESULTS We documented our findings against the QAT framework. We found examples that each QAT principle had, to some extent, been adhered to, but constantly needed to assess whether the principles were fully achieved to our satisfaction. Strengths of the evaluation included being adaptable and responsive to emerging issues for the research collaboration, while areas for improvement included more Indigenous leadership of, and involvement in, evaluation. CONCLUSIONS Although reflexive evaluation practice is not always comfortable, it does provide an opportunity to generate insights for improvement. Reflecting as we did-in a partnership between Indigenous and non-Indigenous colleagues-enabled deeper insights and meaning. We anticipate that our process models how other research in Indigenous contexts might better advance ethical, quality Indigenous research through working in collaboration with Indigenous researchers and communities.
Collapse
Affiliation(s)
- Jodie Bailie
- University Centre for Rural Health, The University of Sydney, Lismore, New South Wales, Australia
- Centre for Disability Research and Policy, The University of Sydney, Sydney, New South Wales, Australia
| | - Veronica Matthews
- University Centre for Rural Health, The University of Sydney, Lismore, New South Wales, Australia
| | - Alison Frances Laycock
- University Centre for Rural Health, The University of Sydney, Lismore, New South Wales, Australia
| | - Kathleen Conte
- University Centre for Rural Health, The University of Sydney, Lismore, New South Wales, Australia
- Homelessness Research and Action Collaborative, Portland State University, Portland, Oregon, USA
| | - Lynette Feeney
- University Centre for Rural Health, The University of Sydney, Lismore, New South Wales, Australia
| | - Roxanne Bainbridge
- Poche Centre for Indigenous Health, The University of Queensland, Brisbane, Queensland, Australia
| |
Collapse
|
2
|
Koloski N, Duncanson K, Ramanathan SA, Rao M, Holtmann G, Talley NJ. What impact has the Centre of Research Excellence in Digestive Health made in the field of gastrointestinal health in Australia and internationally? Study protocol for impact evaluation using the FAIT framework. BMJ Open 2024; 14:e076839. [PMID: 38514142 PMCID: PMC10961498 DOI: 10.1136/bmjopen-2023-076839] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/18/2023] [Accepted: 02/20/2024] [Indexed: 03/23/2024] Open
Abstract
INTRODUCTION The need for public research funding to be more accountable and demonstrate impact beyond typical academic outputs is increasing. This is particularly challenging and the science behind this form of research is in its infancy when applied to collaborative research funding such as that provided by the Australian National Health and Medical Research Council to the Centre for Research Excellence in Digestive Health (CRE-DH). METHODS AND ANALYSIS In this paper, we describe the protocol for applying the Framework to Assess the Impact from Translational health research to the CRE-DH. The study design involves a five-stage sequential mixed-method approach. In phase I, we developed an impact programme logic model to map the pathway to impact and establish key domains of benefit such as knowledge advancement, capacity building, clinical implementation, policy and legislation, community and economic impacts. In phase 2, we have identified and selected appropriate, measurable and timely impact indicators for each of these domains and established a data plan to capture the necessary data. Phase 3 will develop a model for cost-consequence analysis and identification of relevant data for microcosting and valuation of consequences. In phase 4, we will determine selected case studies to include in the narrative whereas phase 5 involves collation, data analysis and completion of the reporting of impact.We expect this impact evaluation to comprehensively describe the contribution of the CRE-DH for intentional activity over the CRE-DH lifespan and beyond to improve outcomes for people suffering with chronic and debilitating digestive disorders. ETHICS AND DISSEMINATION This impact evaluation study has been registered with the Hunter New England Human Research Ethics Committee as project 2024/PID00336 and ethics application 2024/ETH00290. Results of this study will be disseminated via medical conferences, peer-reviewed publications, policy submissions, direct communication with relevant stakeholders, media and social media channels such as X (formely Twitter).
Collapse
Affiliation(s)
- Natasha Koloski
- School of Medicine and Public Health, University of Newcastle, Callaghan, New South Wales, Australia
- School of Health & Behavioural Sciences, University of Queensland, St Lucia, Queensland, Australia
- Department of Gastroenterology & Hepatology, Princess Alexandra Hospital, Woolloongabba, Queensland, Australia
| | - Kerith Duncanson
- School of Medicine and Public Health, University of Newcastle, Callaghan, New South Wales, Australia
- Hunter Medical Research Institute, Newcastle, New South Wales, Australia
| | - Shanthi Ann Ramanathan
- School of Medicine and Public Health, University of Newcastle, Callaghan, New South Wales, Australia
- Hunter Medical Research Institute, Newcastle, New South Wales, Australia
| | - Melanie Rao
- Hunter Medical Research Institute, Newcastle, New South Wales, Australia
| | - Gerald Holtmann
- Department of Gastroenterology & Hepatology, Princess Alexandra Hospital, Woolloongabba, Queensland, Australia
- School of Medicine, University of Queensland, St Lucia, Queensland, Australia
| | - Nicholas J Talley
- School of Medicine and Public Health, University of Newcastle, Callaghan, New South Wales, Australia
- Hunter Medical Research Institute, Newcastle, New South Wales, Australia
| |
Collapse
|
3
|
Cunningham FC, Potts BA, Ramanathan SA, Bailie J, Bainbridge RG, Searles A, Laycock AF, Bailie RS. Network evaluation of an innovation platform in continuous quality improvement in Australian Indigenous primary healthcare. Health Res Policy Syst 2022; 20:119. [PMID: 36316678 PMCID: PMC9620635 DOI: 10.1186/s12961-022-00909-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Accepted: 09/13/2022] [Indexed: 11/04/2022] Open
Abstract
Background From 2014 to 2019, the Centre for Research Excellence in Integrated Quality Improvement (CRE-IQI) was evaluated as an innovation platform focusing on continuous quality improvement in Indigenous Australian primary healthcare. Although social network analysis (SNA) is a recognized method for evaluating the functioning, collaboration and effectiveness of innovation platforms, applied research is limited. This study applies SNA to evaluate the CRE-IQI’s functioning as an innovation platform. Methods Two surveys (2017, 2019) were conducted using social survey and network methods. Survey items covered respondent characteristics, their perceptions of the CRE-IQI’s performance, and its impact and sociometric relationships. Members’ relationship information was captured for the CRE-IQI at three time points, namely start (retrospectively), midpoint and final year, on three network types (knew, shared information, collaborated). SNA software was used to compute standard network metrics including diameter, density and centrality, and to develop visualizations. Survey and network results were addressed in a workshop held by members to develop improvement strategies. Results The response rate was 80% in 2017 and 65% in 2019 (n = 49 and 47, respectively). Between 2017 and 2019, respondents’ mean ratings of the CRE-IQI’s functioning and achievements in meeting its goals were sustained. They perceived the CRE-IQI as multidisciplinary, having effective management and governance, and incorporating Indigenous research leadership, representation and ways of working. Respondents recognized high levels of trust amongst members, rated “good communication and coordination with participants” highly, and “facilitating collaboration” as the CRE’s most strongly recognized achievement. In collaboration and information-sharing networks, average path length remained low in 2017 and 2019, indicating good small-world network properties for relaying information. On average, respondents shared information and collaborated with more CRE members in 2017 than 2019. However, in both 2017 and 2019 there were new collaborations and information-sharing outside of direct collaborations. CRE-IQI outcomes included: evidence generation; knowledge transfer and skills development in quality improvement; research capacity-building, career development; mentoring; grant support; development of new projects; health service support; and policy impact. Conclusions This study shows the utility of network analysis in evaluating the functioning, and collaboration, at the individual, organizational and health system levels, of an innovation platform, and adds to our understanding of factors enabling successful innovation platforms. Supplementary Information The online version contains supplementary material available at 10.1186/s12961-022-00909-z.
Collapse
Affiliation(s)
- Frances Clare Cunningham
- grid.271089.50000 0000 8523 7955Wellbeing and Preventable Chronic Diseases Division, Menzies School of Health Research, Charles Darwin University, Brisbane, QLD Australia
| | - Boyd Alexander Potts
- grid.271089.50000 0000 8523 7955Wellbeing and Preventable Chronic Diseases Division, Menzies School of Health Research, Charles Darwin University, Brisbane, QLD Australia
| | - Shanthi Ann Ramanathan
- grid.413648.cHealth Research Economics, Hunter Medical Research Institute, Newcastle, NSW Australia ,grid.266842.c0000 0000 8831 109XCollege of Health, Medicine and Wellbeing, The University of Newcastle, Newcastle, NSW Australia
| | - Jodie Bailie
- grid.1013.30000 0004 1936 834XUniversity Centre for Rural Health, The University of Sydney, Lismore, NSW Australia ,grid.1013.30000 0004 1936 834XSchool of Public Health, The University of Sydney, Sydney, NSW Australia
| | - Roxanne Gwendalyn Bainbridge
- grid.1003.20000 0000 9320 7537Poche Centre for Indigenous Health, The University of Queensland, Toowong, QLD Australia
| | - Andrew Searles
- grid.413648.cHealth Research Economics, Hunter Medical Research Institute, Newcastle, NSW Australia ,grid.266842.c0000 0000 8831 109XCollege of Health, Medicine and Wellbeing, The University of Newcastle, Newcastle, NSW Australia
| | - Alison Frances Laycock
- grid.1013.30000 0004 1936 834XUniversity Centre for Rural Health, The University of Sydney, Lismore, NSW Australia
| | - Ross Stewart Bailie
- grid.1013.30000 0004 1936 834XFaculty of Medicine and Health, The University of Sydney, Sydney, NSW Australia
| |
Collapse
|
4
|
Patay D, Trieu K, McKenzie B, Ramanathan S, Hure A, Bell C, Thow AM, Allender S, Reeve E, Palu A, Woodward M, Waqa G, Webster J. Research protocol for impact assessment of a project to scale up food policies in the Pacific. Health Res Policy Syst 2022; 20:117. [PMID: 36309749 PMCID: PMC9617745 DOI: 10.1186/s12961-022-00927-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2022] [Accepted: 10/12/2022] [Indexed: 11/29/2022] Open
Abstract
Background One of the challenges for countries implementing food policy measures has been the difficulty in demonstrating impact and retaining stakeholder support. Consequently, research funded to help countries overcome these challenges should assess impact and translation into practice, particularly in low-resource settings. However, there are still few attempts to prospectively, and comprehensively, assess research impact. This protocol describes a study co-created with project implementers, collaborative investigators and key stakeholders to optimize and monitor the impact of a research project on scaling up food policies in Fiji. Methods To develop this protocol, our team of researchers prospectively applied the Framework to Assess the Impact from Translational health research (FAIT). Activities included (i) developing a logic model to map the pathway to impact and establish domains of benefit; (ii) identifying process and impact indicators for each of these domains; (iii) identifying relevant data for impact indicators and a cost–consequence analysis; and (iv) establishing a process for collecting quantitative and qualitative data to measure progress. Impact assessment data will be collected between September 2022 and December 2024, through reports, routine monitoring activities, group discussions and semi-structured interviews with key implementers and stakeholders. The prospective application of the protocol, and interim and final research impact assessments of each project stream and the project as a whole, will optimize and enable robust measurement of research impact. Discussion By applying this protocol, we aim to increase understanding of pathways to impact and processes that need to be put in place to achieve this. This impact evaluation will inform future projects with a similar scope and will identify transferable and/or translatable lessons for other Pacific Island states and low- and middle-income countries. Supplementary Information The online version contains supplementary material available at 10.1186/s12961-022-00927-x.
Collapse
|
5
|
Bailie J, Cunningham F, Abimbola S, Laycock A, Bainbridge R, Bailie R, Conte K, Passey M, Peiris D. Methodological pluralism for better evaluations of complex interventions: lessons from evaluating an innovation platform in Australia. Health Res Policy Syst 2022; 20:14. [PMID: 35090472 PMCID: PMC8796351 DOI: 10.1186/s12961-022-00814-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2021] [Accepted: 01/10/2022] [Indexed: 11/10/2022] Open
Abstract
Complex interventions, such as innovation platforms, pose challenges for evaluators. A variety of methodological approaches are often required to build a more complete and comprehensive understanding of how complex interventions work. In this paper, we outline and critically appraise a methodologically pluralist evaluation of an innovation platform to strengthen primary care for Aboriginal and Torres Strait Islander Australians. In doing so, we aim to identify lessons learned from the approach taken and add to existing literature on implementing evaluations in complex settings, such as innovation platforms. The pluralist design used four evaluation approaches-developmental evaluation, principles-focused evaluation, network analysis, and framework analysis-with differing strengths and challenges. Taken together, the multiple evaluation approaches yielded a detailed description and nuanced understanding of the formation, functioning and outcomes of the innovation platform that would be difficult to achieve with any single evaluation method. While a methodologically pluralist design may place additional pressure on logistical and analytic resources available, it enables a deeper understanding of the mechanisms that underlie complex interventions.
Collapse
Affiliation(s)
- J Bailie
- The University Centre for Rural Health, The University of Sydney, 61 Uralba Street, Lismore, NSW, 2480, Australia.
- The School of Public Health, The University of Sydney, Sydney, Australia.
| | - F Cunningham
- Menzies School of Health Research, Charles Darwin University, Brisbane, Australia
| | - S Abimbola
- The School of Public Health, The University of Sydney, Sydney, Australia
- The George Institute for Global Health, The University of New South Wales, Sydney, Australia
| | - A Laycock
- The University Centre for Rural Health, The University of Sydney, 61 Uralba Street, Lismore, NSW, 2480, Australia
| | - R Bainbridge
- School of Health, Medical and Applied Sciences, Central Queensland University, Cairns, Australia
| | - R Bailie
- The University Centre for Rural Health, The University of Sydney, 61 Uralba Street, Lismore, NSW, 2480, Australia
| | - K Conte
- The University Centre for Rural Health, The University of Sydney, 61 Uralba Street, Lismore, NSW, 2480, Australia
- The School of Public Health, De Paul University, Chicago, USA
| | - M Passey
- The University Centre for Rural Health, The University of Sydney, 61 Uralba Street, Lismore, NSW, 2480, Australia
| | - D Peiris
- The School of Public Health, The University of Sydney, Sydney, Australia
- The George Institute for Global Health, The University of New South Wales, Sydney, Australia
| |
Collapse
|
6
|
Sarkies MN, Robinson S, Briffa T, Duffy SJ, Nelson M, Beltrame J, Cullen L, Chew D, Smith J, Brieger D, Macdonald P, Liew D, Reid C. Applying a framework to assess the impact of cardiovascular outcomes improvement research. Health Res Policy Syst 2021; 19:67. [PMID: 33882947 PMCID: PMC8059028 DOI: 10.1186/s12961-021-00710-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2020] [Accepted: 03/21/2021] [Indexed: 01/06/2023] Open
Abstract
Background Health and medical research funding agencies are increasingly interested in measuring the impact of funded research. We present a research impact case study for the first four years of an Australian National Health and Medical Research Council funded Centre of Research Excellence in Cardiovascular Outcomes Improvement (2016–2020). The primary aim of this paper was to explore the application of a research impact matrix to assess the impact of cardiovascular outcomes improvement research. Methods We applied a research impact matrix developed from a systematic review of existing methodological frameworks used to measure research impact. This impact matrix was used as a bespoke tool to identify and understand various research impacts over different time frames. Data sources included a review of existing internal documentation from the research centre and publicly available information sources, informal iterative discussions with 10 centre investigators, and confirmation of information from centre grant and scholarship recipients. Results By July 2019, the impact on the short-term research domain category included over 41 direct publications, which were cited over 87 times (median journal impact factor of 2.84). There were over 61 conference presentations, seven PhD candidacies, five new academic collaborations, and six new database linkages conducted. The impact on the mid-term research domain category involved contributions towards the development of a national cardiac registry, cardiovascular guidelines, application for a Medicare Benefits Schedule reimbursement item number, introduction of patient-reported outcome measures into several databases, and the establishment of nine new industry collaborations. Evidence of long-term impacts were described as the development and use of contemporary management for aortic stenosis, a cardiovascular risk prediction model and prevention targets in several data registries, and the establishment of cost-effectiveness for stenting compared to surgery. Conclusions We considered the research impact matrix a feasible tool to identify evidence of academic and policy impact in the short- to midterm; however, we experienced challenges in capturing long-term impacts. Cost containment and broader economic impacts represented another difficult area of impact to measure. Supplementary Information The online version contains supplementary material available at 10.1186/s12961-021-00710-4.
Collapse
Affiliation(s)
- Mitchell N Sarkies
- Centre for Healthcare Resilience and Implementation Science, Australian Institute of Health Innovation, Faculty of Medicine, Health and Human Sciences, Macquarie University, 75 Talavera Road, Sydney, NSW, 2109, Australia. .,Health Systems and Health Economics Group, Health Research and Data Analytics Hub, School of Public Health, Faculty of Health Sciences, Curtin University, Perth, Australia.
| | - Suzanne Robinson
- Health Systems and Health Economics Group, Health Research and Data Analytics Hub, School of Public Health, Faculty of Health Sciences, Curtin University, Perth, Australia
| | - Tom Briffa
- Faculty of Health and Medical Sciences, Population and Public Health, The University of Western Australia, Perth, Australia
| | - Stephen J Duffy
- Department of General Cardiology, Alfred Health, Melbourne, Australia.,Department of Epidemiology and Preventative Medicine, Monash University, Melbourne, Australia
| | - Mark Nelson
- Menzies Institute for Medical Research, University of Tasmania, Hobart, Australia
| | - John Beltrame
- Discipline of Medicine, University of Adelaide, Adelaide, Australia.,Cardiology Department, Central Adelaide Local Health Network, Adelaide, Australia.,Cardiology Department, Lyell McEwin Hospital, Adelaide, Australia
| | - Louise Cullen
- Institute of Health and Biomedical Innovation and School of Public Health and Social Work, Faculty of Health, Queensland University of Technology, Brisbane, Australia.,Emergency and Trauma Centre, Royal Brisbane and Women's Hospital, Brisbane, Australia.,School of Medicine, Faculty of Health and Behavioural Sciences, The University of Queensland, Brisbane, Australia
| | - Derek Chew
- Department of Cardiovascular Medicine, Flinders University, Adelaide, Adelaide, Australia
| | - Julian Smith
- Department of Surgery (School of Clinical Sciences At Monash Health), Monash University, Melbourne, Australia.,Department of Cardiothoracic Surgery, Monash Health, Melbourne, Australia
| | - David Brieger
- Division of Cardiology, Concord Hospital and University of Sydney, Sydney, Australia
| | - Peter Macdonald
- St Vincent's Hospital, Victor Chang Cardiac Research Institute, University of New South Wales, Sydney, Australia
| | - Danny Liew
- Department of Epidemiology and Preventative Medicine, Monash University, Melbourne, Australia
| | - Chris Reid
- Department of Epidemiology and Preventative Medicine, Monash University, Melbourne, Australia.,NHMRC Centre for Research Excellence in Cardiovascular Outcomes Improvement, Health Research and Data Analytics Hub, School of Public Health, Faculty of Health Sciences, Curtin University, Perth, Australia
| |
Collapse
|
7
|
Ramanathan SA, Larkins S, Carlisle K, Turner N, Bailie RS, Thompson S, Bainbridge R, Deeming S, Searles A. What was the impact of a participatory research project in Australian Indigenous primary healthcare services? Applying a comprehensive framework for assessing translational health research to Lessons for the Best. BMJ Open 2021; 11:e040749. [PMID: 33622941 PMCID: PMC7907854 DOI: 10.1136/bmjopen-2020-040749] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/06/2023] Open
Abstract
OBJECTIVES To (1) apply the Framework to Assess the Impact from Translational health research (FAIT) to Lessons from the Best to Better the Rest (LFTB), (2) report on impacts from LFTB and (3) assess the feasibility and outcomes from a retrospective application of FAIT. SETTING Three Indigenous primary healthcare (PHC) centres in the Northern Territory, Australia; project coordinating centre distributed between Townsville, Darwin and Cairns and the broader LFTB learning community across Australia. PARTICIPANTS LFTB research team and one representative from each PHC centre. PRIMARY AND SECONDARY OUTCOME MEASURES Impact reported as (1) quantitative metrics within domains of benefit using a modified Payback Framework, (2) a cost-consequence analysis given a return on investment was not appropriate and (3) a narrative incorporating qualitative evidence of impact. Data were gathered through in-depth stakeholder interviews and a review of project documentation, outputs and relevant websites. RESULTS LFTB contributed to knowledge advancement in Indigenous PHC service delivery; enhanced existing capacity of health centre staff, researchers and health service users; enhanced supportive networks for quality improvement; and used a strengths-based approach highly valued by health centres. LFTB also leveraged between $A1.4 and $A1.6 million for the subsequent Leveraging Effective Ambulatory Practice (LEAP) Project to apply LFTB learnings to resource development and creation of a learning community to empower striving PHC centres. CONCLUSION Retrospective application of FAIT to LFTB, although not ideal, was feasible. Prospective application would have allowed Indigenous community perspectives to be included. Greater appreciation of the full benefit of LFTB including a measure of return on investment will be possible when LEAP is complete. Future assessments of impact need to account for the limitations of fully capturing impact when intermediate/final impacts have not yet been realised and captured.
Collapse
Affiliation(s)
- Shanthi Ann Ramanathan
- Health Research Economics, Hunter Medical Research Institute, Newcastle, New South Wales, Australia
- School of Medicine and Public Health, University of Newcastle, Newcastle, New South Wales, Australia
| | - Sarah Larkins
- College of Medicine and Dentistry, James Cook University, Townsville, Queensland, Australia
| | - Karen Carlisle
- College of Medicine and Dentistry, James Cook University, Townsville, Queensland, Australia
| | - Nalita Turner
- College of Medicine and Dentistry, James Cook University, Townsville, Queensland, Australia
| | - Ross Stewart Bailie
- The University of Sydney, University Centre for Rural Health, Lismore, New South Wales, Australia
| | - Sandra Thompson
- University of Western Australia, Western Australian Centre for Rural Health, Geraldton, Western Australia, Australia
| | - Roxanne Bainbridge
- Centre for Indigenous Health Equity Research, School of Health, Medical and Applied Sciences, Central Queensland University, Cairns, Queensland, Australia
| | - Simon Deeming
- School of Medicine and Public Health, University of Newcastle, Newcastle, New South Wales, Australia
| | - Andrew Searles
- School of Medicine and Public Health, University of Newcastle, Newcastle, New South Wales, Australia
| |
Collapse
|
8
|
Bailie J, Laycock AF, Peiris D, Bainbridge RG, Matthews V, Cunningham FC, Conte KP, Abimbola S, Passey ME, Bailie RS. Using developmental evaluation to enhance continuous reflection, learning and adaptation of an innovation platform in Australian Indigenous primary healthcare. Health Res Policy Syst 2020; 18:45. [PMID: 32398136 PMCID: PMC7218558 DOI: 10.1186/s12961-020-00562-4] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2019] [Accepted: 04/08/2020] [Indexed: 12/21/2022] Open
Abstract
Effective efforts to strengthen health systems need diverse, multi-stakeholder networks working together on complex or 'wicked' problems such as prevention and control of chronic diseases, solutions to which go beyond the role and capability of one organisation. The contextual complexities inherent in 'wicked' problems mean that solutions warrant a systems approach that encompasses innovation and new ways of thinking about, facilitating and implementing collective decision-making processes and change practices.Innovation platforms are a mechanism for facilitating communication and collaboration among diverse stakeholders, promoting joint action and stimulating innovation. Developmental evaluation is an approach that is increasingly being used to evaluate innovative and emergent programmes and projects, as it enables evaluators to provide real-time feedback so that evaluation findings can be used to guide development and adaptations. Developmental evaluation emphasises learning and adaptation, and aligns well with the implementation of innovation platforms that have continuous reflection, learning and adaptation as a specific design principle.Here, we outline our rationale for applying a developmental evaluation to enhance the formation, functioning and outcomes of an innovation platform aimed at accelerating and strengthening large-scale quality improvement efforts in Australian Aboriginal and Torres Strait Islander primary healthcare. We provide examples to explain how the developmental evaluation findings were used for adaptation of the innovation platform and assess to what extent our application of developmental evaluation was consistent with, and reflective of, its essential principles.Our evaluation aligned strongly with the principles of developmental evaluation, and the approach we took was well suited to situations with a developmental purpose, innovation niche and complexity such as innovation platforms. As a result, along with the increasing interest in multi-stakeholder platforms (e.g. innovation platforms) and the inherent challenges with evaluating these complex networks, we anticipate our use of this approach being of interest globally.
Collapse
Affiliation(s)
- Jodie Bailie
- The University Centre for Rural Health, The University of Sydney, 61 Uralba Street, Lismore, NSW, 2480, Australia. .,The School of Public Health, The University of Sydney, Sydney, NSW, 2006, Australia.
| | - Alison Frances Laycock
- The University Centre for Rural Health, The University of Sydney, 61 Uralba Street, Lismore, NSW, 2480, Australia.,Menzies School of Health Research, Charles Darwin University, Casuarina, Australia
| | - David Peiris
- The School of Public Health, The University of Sydney, Sydney, NSW, 2006, Australia.,The George Institute for Global Health, The University of New South Wales, Sydney, Australia
| | | | - Veronica Matthews
- The University Centre for Rural Health, The University of Sydney, 61 Uralba Street, Lismore, NSW, 2480, Australia
| | | | - Kathleen Parker Conte
- The University Centre for Rural Health, The University of Sydney, 61 Uralba Street, Lismore, NSW, 2480, Australia
| | - Seye Abimbola
- The School of Public Health, The University of Sydney, Sydney, NSW, 2006, Australia.,The George Institute for Global Health, The University of New South Wales, Sydney, Australia
| | - Megan Elizabeth Passey
- The University Centre for Rural Health, The University of Sydney, 61 Uralba Street, Lismore, NSW, 2480, Australia
| | - Ross Stewart Bailie
- The University Centre for Rural Health, The University of Sydney, 61 Uralba Street, Lismore, NSW, 2480, Australia
| |
Collapse
|
9
|
Dodd R, Ramanathan S, Angell B, Peiris D, Joshi R, Searles A, Webster J. Strengthening and measuring research impact in global health: lessons from applying the FAIT framework. Health Res Policy Syst 2019; 17:48. [PMID: 31060617 PMCID: PMC6501392 DOI: 10.1186/s12961-019-0451-0] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2018] [Accepted: 04/04/2019] [Indexed: 12/27/2022] Open
Abstract
Background To date, efforts to measure impact have largely focused on health research in high-income countries, reflecting where the majority of health research funding is spent. Nevertheless, there is a growing body of health and medical research being undertaken in low- and middle-income countries (LMICs), supported by both development aid and established research funders. The Framework to Assess the Impact of Translational health research (FAIT) combines three approaches to measuring research impact (Payback, economic assessment and case study narrative). Its aim is to strengthen the focus on translation and impact measurement in health research. FAIT has been used by several Australian research initiatives; however, it has not been used in LMICs. Our aim was to apply FAIT in an LMIC context and evaluate its utility. Methods We retrospectively applied all three FAIT methods to two LMIC studies using available data, supplemented with group discussion and further economic analyses. Results were presented in a scorecard format. Results FAIT helped clarify pathways of impact for the projects and provided new knowledge on areas of impact in several domains, including capacity-building for research, policy development and economic impact. However, there were constraints, particularly associated with calculating the return on investment in the LMIC context. The case study narrative provided a layperson’s summary of the research that helped to explain outcomes and succinctly communicate lessons learnt. Conclusion Use of FAIT to assess the impact of LMIC research was both feasible and useful. We make recommendations related to prospective use, identification of metrics to support use of the Payback framework, and simplification of the economic assessment, which may facilitate further application in LMIC environments. Electronic supplementary material The online version of this article (10.1186/s12961-019-0451-0) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Rebecca Dodd
- The George Institute for Global Health, University of New South Wales, 1 King Street, Newtown, Sydney, 2042, Australia.
| | - Shanthi Ramanathan
- Hunter Medical Research Institute, Locked Bag 1000, New Lambton, NSW, 2305, Australia.,School of Medicine and Public Health, University of Newcastle, University Drive, Callaghan, NSW, 2308, Australia
| | - Blake Angell
- The George Institute for Global Health, University of New South Wales, 1 King Street, Newtown, Sydney, 2042, Australia
| | - David Peiris
- The George Institute for Global Health, University of New South Wales, 1 King Street, Newtown, Sydney, 2042, Australia
| | - Rohina Joshi
- The George Institute for Global Health, University of New South Wales, 1 King Street, Newtown, Sydney, 2042, Australia
| | - Andrew Searles
- Hunter Medical Research Institute, Locked Bag 1000, New Lambton, NSW, 2305, Australia.,School of Medicine and Public Health, University of Newcastle, University Drive, Callaghan, NSW, 2308, Australia
| | - Jacqui Webster
- The George Institute for Global Health, University of New South Wales, 1 King Street, Newtown, Sydney, 2042, Australia
| |
Collapse
|
10
|
Bailie J, Cunningham FC, Bainbridge RG, Passey ME, Laycock AF, Bailie RS, Larkins SL, Brands JSM, Ramanathan S, Abimbola S, Peiris D. Comparing and contrasting 'innovation platforms' with other forms of professional networks for strengthening primary healthcare systems for Indigenous Australians. BMJ Glob Health 2018; 3:e000683. [PMID: 29862056 PMCID: PMC5969724 DOI: 10.1136/bmjgh-2017-000683] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2017] [Revised: 04/23/2018] [Accepted: 04/27/2018] [Indexed: 11/16/2022] Open
Abstract
Efforts to strengthen health systems require the engagement of diverse, multidisciplinary stakeholder networks. Networks provide a forum for experimentation and knowledge creation, information exchange and the spread of good ideas and practice. They might be useful in addressing complex issues or ‘wicked’ problems, the solutions to which go beyond the control and scope of any one agency. Innovation platforms are proposed as a novel type of network because of their diverse stakeholder composition and focus on problem solving within complex systems. Thus, they have potential applicability to health systems strengthening initiatives, even though they have been predominantly applied in the international agricultural development sector. In this paper, we compare and contrast the concept of innovation platforms with other types of networks that can be used in efforts to strengthen primary healthcare systems, such as communities of practice, practice-based research networks and quality improvement collaboratives. We reflect on our ongoing research programme that applies innovation platform concepts to drive large-scale quality improvement in primary healthcare for Aboriginal and Torres Strait Islander Australians and outline our plans for evaluation. Lessons from our experience will find resonance with others working on similar initiatives in global health.
Collapse
Affiliation(s)
- Jodie Bailie
- University Centre for Rural Health, University of Sydney, Sydney, New South Wales, Australia
| | | | | | - Megan E Passey
- University Centre for Rural Health, University of Sydney, Sydney, New South Wales, Australia
| | - Alison Frances Laycock
- Centre for Indigenous Health Equity Research, Central Queensland University, Brisbane, Queensland, Australia
| | - Ross Stewart Bailie
- University Centre for Rural Health, University of Sydney, Sydney, New South Wales, Australia
| | - Sarah L Larkins
- College of Medicine and Dentistry, James Cook University, Townsville, Queensland, Australia
| | - Jenny S M Brands
- Centre for Indigenous Health Equity Research, Central Queensland University, Brisbane, Queensland, Australia
| | - Shanthi Ramanathan
- Hunter Research Medical Institute, University of Newcastle, Newcastle, New South Wales, Australia
| | - Seye Abimbola
- School of Public Health, The University of Sydney, Sydney, New South Wales, Australia
| | - David Peiris
- The George Institute for Global Health, University of New South Wales, Sydney, New South Wales, Australia
| |
Collapse
|
11
|
McPhail-Bell K, Matthews V, Bainbridge R, Redman-MacLaren ML, Askew D, Ramanathan S, Bailie J, Bailie R. An "All Teach, All Learn" Approach to Research Capacity Strengthening in Indigenous Primary Health Care Continuous Quality Improvement. Front Public Health 2018; 6:107. [PMID: 29761095 PMCID: PMC5936797 DOI: 10.3389/fpubh.2018.00107] [Citation(s) in RCA: 20] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2017] [Accepted: 04/03/2018] [Indexed: 11/13/2022] Open
Abstract
In Australia, Indigenous people experience poor access to health care and the highest rates of morbidity and mortality of any population group. Despite modest improvements in recent years, concerns remains that Indigenous people have been over-researched without corresponding health improvements. Embedding Indigenous leadership, participation, and priorities in health research is an essential strategy for meaningful change for Indigenous people. To centralize Indigenous perspectives in research processes, a transformative shift away from traditional approaches that have benefited researchers and non-Indigenous agendas is required. This shift must involve concomitant strengthening of the research capacity of Indigenous and non-Indigenous researchers and research translators-all must teach and all must learn. However, there is limited evidence about how to strengthen systems and stakeholder capacity to participate in and lead continuous quality improvement (CQI) research in Indigenous primary health care, to the benefit of Indigenous people. This paper describes the collaborative development of, and principles underpinning, a research capacity strengthening (RCS) model in a national Indigenous primary health care CQI research network. The development process identified the need to address power imbalances, cultural contexts, relationships, systems requirements and existing knowledge, skills, and experience of all parties. Taking a strengths-based perspective, we harnessed existing knowledge, skills and experiences; hence our emphasis on capacity "strengthening". New insights are provided into the complex processes of RCS within the context of CQI in Indigenous primary health care.
Collapse
Affiliation(s)
- Karen McPhail-Bell
- Poche Centre for Indigenous Health, Sydney Medical School, The University of Sydney, Camperdown, NSW, Australia.,University Centre for Rural Health, The University of Sydney, Lismore, NSW, Australia
| | - Veronica Matthews
- University Centre for Rural Health, The University of Sydney, Lismore, NSW, Australia
| | - Roxanne Bainbridge
- Centre for Indigenous Health Equity Research, CQUniversity, Cairns, QLD, Australia
| | - Michelle Louise Redman-MacLaren
- Centre for Indigenous Health Equity Research, CQUniversity, Cairns, QLD, Australia.,College of Medicine and Dentistry, James Cook University, Cairns, QLD, Australia
| | - Deborah Askew
- Primary Care Clinical Unit, The University of Queensland, Herston, QLD, Australia.,Southern Queensland Centre of Excellence in Aboriginal and Torres Strait Islander Primary Health Care (Inala Indigenous Health Service), Queensland Health, Inala, QLD, Australia
| | - Shanthi Ramanathan
- Hunter Medical Research Institute, University of Newcastle, Newcastle, NSW, Australia
| | - Jodie Bailie
- University Centre for Rural Health, The University of Sydney, Lismore, NSW, Australia
| | - Ross Bailie
- University Centre for Rural Health, The University of Sydney, Lismore, NSW, Australia
| | | |
Collapse
|