Setting health research priorities using the CHNRI method: III. Involving stakeholders

A framework for remotely enabled co-design with young people: its development and application with neurodiverse children and their caregivers

Introduction

This paper describes an innovative Framework for Remotely Enabled Co-Design with Young people (FREDY), which details an adaptable four-stage process for generating design concepts with children and other key stakeholders in a naturalistic and inclusive way.

Methods

Recommendations from existing patient engagement and design methodologies were combined to provide research teams with procedures to capture and analyse end-user requirements rapidly. Resulting insights were applied through iterative design cycles to achieve accelerated and user-driven innovation.

Results

Applying this framework with neurodiverse children within the context of healthcare, shows how creative design methods can give rise to new opportunities for co-creating across diverse geographies, abilities, and backgrounds as well as strengthen co-designer approval of the co-design process and resulting product.

Discussion

We summarise key learnings and principles for fostering trust and sustaining participation with remote activities, and facilitating stakeholder design input through continuous collaboration, as well as highlight the potential benefits and challenges of utilising FREDY with neurotypical populations.

Setting research priorities for global pandemic preparedness: An international consensus and comparison with ChatGPT's output

Background

In this priority-setting exercise, we sought to identify leading research priorities needed for strengthening future pandemic preparedness and response across countries.

Methods

The International Society of Global Health (ISoGH) used the Child Health and Nutrition Research Initiative (CHNRI) method to identify research priorities for future pandemic preparedness. Eighty experts in global health, translational and clinical research identified 163 research ideas, of which 42 experts then scored based on five pre-defined criteria. We calculated intermediate criterion-specific scores and overall research priority scores from the mean of individual scores for each research idea. We used a bootstrap (n = 1000) to compute the 95% confidence intervals.

Results

Key priorities included strengthening health systems, rapid vaccine and treatment production, improving international cooperation, and enhancing surveillance efficiency. Other priorities included learning from the coronavirus disease 2019 (COVID-19) pandemic, managing supply chains, identifying planning gaps, and promoting equitable interventions. We compared this CHNRI-based outcome with the 14 research priorities generated and ranked by ChatGPT, encountering both striking similarities and clear differences.

Conclusions

Priority setting processes based on human crowdsourcing - such as the CHNRI method - and the output provided by ChatGPT are both valuable, as they complement and strengthen each other. The priorities identified by ChatGPT were more grounded in theory, while those identified by CHNRI were guided by recent practical experiences. Addressing these priorities, along with improvements in health planning, equitable community-based interventions, and the capacity of primary health care, is vital for better pandemic preparedness and response in many settings.

Russian Biodiversity Collections: A Professional Opinion Survey

Biodiversity collections are important vehicles for protecting endangered wildlife in situations of adverse anthropogenic influence. In Russia, there are currently a number of institution- and museum-based biological collections, but there are no nation-wide centres of biodiversity collections. In this paper, we report on the results of our survey of 324 bioconservation, big-data, and ecology specialists from different regions of Russia in regard to the necessity to create several large national biodiversity centres of wildlife protection. The survey revealed specific goals that have to be fulfilled during the development of these centres for the protection and restoration of endangered wildlife species. The top three problems/tasks (topics) are the following: (1) the necessity to create large national centres for different types of specimens; (2) the full sequencing and creation of different "omic" (genomic, proteomic, transcriptomic, etc.) databases; (3) full digitisation of a biodiversity collection/centre. These goals may constitute a guideline for the future of biodiversity collections in Russia that would be targeted at protecting and restoring endangered species. With the due network service level, the translation of the website into English, and permission from the regulator (Ministry of Science and Higher Education of Russian Federation), it can also become an international project.

Bridging the gap: Using CHNRI to align migration health research priorities in India with local expertise and global perspectives

Background

Migration and health are increasingly recognised as a global public health priority, but concerns have been raised on the skewed nature of current research and the potential disconnect between health needs and policy and governance responses. The Migration Health South Asia (MiHSA) network led the first systematic research priority-setting exercise for India, aligned with the global call to develop a clearly defined migration health research agenda that will inform research investments and guide migrant-responsive policies by the year 2030.

Methods

We adapted the Child Health and Nutrition Research Initiative (CHNRI) method for this priority setting exercise for migration health. Guided by advisory groups established at international and country levels, we sought research topics from 51 experts from diverse disciplines and sectors across India. We consolidated 223 responses into 59 research topics across five themes and scored them against five predefined criteria: answerability, effectiveness, feasibility, impact, and effect on equity. We then calculated research priority scores (RPS) and average expert agreement (AEA) each research topic and theme.

Results

A third of the 59 research topics were on migrants' health and health care access, 12 on social determinants of migrants' health, 10 on policies, law and migration health governance, eight on health systems' responsiveness, and five on migration health discourse. Three of the top five priority topics pertained to migrants' health care access. The policies, law, and governance theme had the highest overall RPS score.

Conclusions

There is a noticeable gap between research priorities identified by experts at the country-level and the current research focus and priorities set globally. This disconnect between the global and local perspectives in migration health scholarship hinders the development of context-specific and suitable policy agendas for improving migrants' health. Our co-developed agenda emphasises the need to prioritise research on the capacity of existing systems and policies so as to make them more migration-aware and responsive to migrants' health needs.

Research prioritisation on prevention and management of preterm birth in low and middle-income countries (LMICs) with a special focus on Bangladesh using the Child Health and Nutrition Research Initiative (CHNRI) method

Background

Fifteen million babies are born preterm globally each year, with 81% occurring in low- and middle-income countries (LMICs). Preterm birth complications are the leading cause of newborn deaths and significantly impact health, quality of life, and costs of health services. Improving outcomes for newborns and their families requires prioritising research for developing practical, scalable solutions, especially in low-resource settings such as Bangladesh. We aimed to identify research priorities related to preventing and managing preterm birth in LMICs for 2021-2030, with a special focus on Bangladesh.

Methods

We adopted the Child Health and Nutrition Research Initiative (CHNRI) method to set research priorities for preventing and managing preterm birth. Seventy-six experts submitted 490 research questions online, which we collated into 95 unique questions and sent for scoring to all experts. A hundred and nine experts scored the questions using five pre-selected criteria: answerability, effectiveness, deliverability, maximum potential for burden reduction, and effect on equity. We calculated weighted and unweighted research priority scores and average expert agreement to generate a list of top-ranked research questions for LMICs and Bangladesh.

Results

Health systems and policy research dominated the top 20 identified priorities for LMICs, such as understanding and improving uptake of the facility and community-based Kangaroo Mother Care (KMC), promoting breastfeeding, improving referral and transport networks, evaluating the impact of the use of skilled attendants, quality improvement activities, and exploring barriers to antenatal steroid use. Several of the top 20 questions also focused on screening high-risk women or the general population of women, understanding the causes of preterm birth, or managing preterm babies with illnesses (jaundice, sepsis and retinopathy of prematurity). There was a high overlap between research priorities in LMICs and Bangladesh.

Conclusions

This exercise, aimed at identifying priorities for preterm birth prevention and management research in LMICs, especially in Bangladesh, found research on improving the care of preterm babies to be more important in reducing the burden of preterm birth and accelerating the attainment of Sustainable Development Goal 3 target of newborn deaths, by 2030.

What interventions should we implement in England's mental health services? The mental health implementation network (MHIN) mixed-methods approach to rapid prioritisation

Introduction

Setting mental health priorities helps researchers, policy makers, and service funders improve mental health services. In the context of a national mental health implementation programme in England, this study aims to identify implementable evidence-based interventions in key priority areas to improve mental health service delivery.

Methods

A mixed-methods research design was used for a three step prioritisation approach involving systematic scoping reviews (additional manuscript under development), expert consultations and data triangulation. Groups with diverse expertise, including experts by experience, worked together to improve decision-making quality by promoting more inclusive and comprehensive discussions. A multi-criteria decision analysis (MCDA) model was used to combine participants' varied opinions, data and judgments about the data's relevance to the issues at hand during a decision conferencing workshop where the priorities were finalised.

Results

The study identified mental health interventions in three mental health priority areas: mental health inequalities, child and adolescent mental health, comorbidities with a focus on integration of mental and physical health services and mental health and substance misuse problems. Key interventions in all the priority areas are outlined. The programme is putting some of these evidence-based interventions into action nationwide in each of these three priority mental health priority areas.

Conclusion

We report an inclusive attempt to ensure that the list of mental health service priorities agrees with perceived needs on the ground and focuses on evidence-based interventions. Other fields of healthcare may also benefit from this methodological approach if they need to make rapid health-prioritisation decisions.

Setting national research priorities for difficult-to-treat depression in the UK between 2021-2026

Background

Difficult-to-treat depression (DTD) presents a substantial health care challenge, with around one-third of people diagnosed with a depressive episode in the UK finding that their symptoms persist following treatment. This study aimed to identify priority research questions (RQs) that could inform the development of new and improved treatments, interventions, and support for people with DTD.

Methods

Using an adapted Child Health and Nutrition Research Initiative (CHNRI) method, this national prioritisation exercise engaged 60 leading researchers and health care professionals in the UK, as well as 25 wider stakeholders with relevant lived experience to produce a ranked list of priority RQs in DTD. The final list of 99 distinct RQs was independently scored by 42 individuals against a list of five criteria: answerability, effectiveness, impact on health, deliverability, and equity.

Results

Highly ranked RQs covered a range of novel and existing treatments. The three highest scoring RQs included evaluation of psychological and pharmacological therapies (eg, behavioural activation, and augmentation therapies), as well as social interventions to reduce loneliness or increase support for people with DTD.

Conclusions

This exercise identified and prioritised 99 RQs that could inform future research and funding decisions over the next five years. The results of this research could improve treatment and support for people affected by DTD. It also serves as an example of ways in which the CHNRI method can be adapted in a collaborative manner to provide a more active role for patients, carers, and health care professionals.

Research priority setting related to older adults: a scoping review to inform the Cochrane-Campbell Global Ageing Partnership work programme

OBJECTIVE

To explore and map the findings of prior research priority-setting initiatives related to improving the health and well-being of older adults.

DESIGN

Scoping review.

DATA SOURCES

Searched MEDLINE, EMBASE, AgeLine, CINAHL and PsycINFO databases from January 2014 to 26 April 2021, and the James Lind Alliance top 10 priorities.

ELIGIBILITY CRITERIA

We included primary studies reporting research priorities gathered from stakeholders that focused on ageing or the health of older adults (≥60 years). There were no restrictions by setting, but language was limited to English and French.

DATA EXTRACTION AND SYNTHESIS

We used a modified Reporting Guideline for Priority Setting of Health Research (REPRISE) guideline to assess the transparency of the reported methods. Population-intervention-control-outcome (PICO) priorities were categorised according to their associated International Classification of Health Interventions (ICHI) and International Classification of Functioning (ICF) outcomes. Broad research topics were categorised thematically.

RESULTS

Sixty-four studies met our inclusion criteria. The studies gathered opinions from various stakeholder groups, including clinicians (n=56 studies) and older adults (n=35), and caregivers (n=24), with 75% of the initiatives involving multiple groups. None of the included priority-setting initiatives reported gathering opinions from stakeholders located in low-income or middle-income countries. Of the priorities extracted, 272 were identified as broad research topics, while 217 were identified as PICO priorities. PICO priorities that involved clinical outcomes (n=165 priorities) and interventions concerning health-related behaviours (n=59) were identified most often. Broad research topics on health services and systems were identified most often (n=60). Across all these included studies, the reporting of six REPRISE elements was deemed to be critically low.

CONCLUSION

Future priority setting initiatives should focus on documenting a more detailed methodology with all initiatives eliciting opinions from caregivers and older adults to ensure priorities reflect the opinions of all key stakeholder groups.

Defining research priorities for youth public mental health: reflections on a coproduction approach to transdisciplinary working

BACKGROUND

With most mental health problems established during childhood/adolescence, young people must be a key focus of public mental health approaches. Despite the range of factors known to influence mental health, evidence for effective interventions is lacking for this age group. This study aimed to define priorities for future public health intervention-focused research to support youth mental health by engaging with transdisciplinary stakeholder groups.

METHODS

Our coproduction approach involved priority-setting workshops with young people, researchers, practitioners and policy-makers. Each workshop focused on three thematic areas: social connections and relationships; schools and other education settings; and key groups at greater risk of mental ill-health, specifically LGBTQ+ and care-experienced young people. Workshop outputs were synthesized to define research priorities.

RESULTS

This paper presents the research priorities that were defined through the priority-setting workshops, and our reflections on the coproduction approach to guide future similar activities undertaken by others. Ten priorities for youth public mental health research were defined, covering the following areas: building supportive relationships; whole system approaches; social media; support at times of transition; improving links between different services; development and training for those who support young people; staff mental health; engaging with families; awareness of and access to services; and out-of-school and community settings.

CONCLUSIONS

These research priorities can inform future intervention development to support youth public mental health. Our transdisciplinary approach means the identified research priorities are likely to be relevant to young people's experiences and needs, and to fit with the needs of those working in practice and policy to support young people.

Research priorities to reduce the impact of COVID-19 in low- and middle-income countries

Background

The COVID-19 pandemic has caused disruptions to the functioning of societies and their health systems. Prior to the pandemic, health systems in low- and middle-income countries (LMIC) were particularly stretched and vulnerable. The International Society of Global Health (ISoGH) sought to systematically identify priorities for health research that would have the potential to reduce the impact of the COVID-19 pandemic in LMICs.

Methods

The Child Health and Nutrition Research Initiative (CHNRI) method was used to identify COVID-19-related research priorities. All ISoGH members were invited to participate. Seventy-nine experts in clinical, translational, and population research contributed 192 research questions for consideration. Fifty-two experts then scored those questions based on five pre-defined criteria that were selected for this exercise: 1) feasibility and answerability; 2) potential for burden reduction; 3) potential for a paradigm shift; 4) potential for translation and implementation; and 5) impact on equity.

Results

Among the top 10 research priorities, research questions related to vaccination were prominent: health care system access barriers to equitable uptake of COVID-19 vaccination (ranked 1st), determinants of vaccine hesitancy (4th), development and evaluation of effective interventions to decrease vaccine hesitancy (5th), and vaccination impacts on vulnerable population/s (6th). Health care delivery questions also ranked highly, including: effective strategies to manage COVID-19 globally and in LMICs (2nd) and integrating health care for COVID-19 with other essential health services in LMICs (3rd). Additionally, the assessment of COVID-19 patients’ needs in rural areas of LMICs was ranked 7th, and studying the leading socioeconomic determinants and consequences of the COVID-19 pandemic in LMICs using multi-faceted approaches was ranked 8th. The remaining questions in the top 10 were: clarifying paediatric case-fatality rates (CFR) in LMICs and identifying effective strategies for community engagement against COVID-19 in different LMIC contexts.

Interpretation

Health policy and systems research to inform COVID-19 vaccine uptake and equitable access to care are urgently needed, especially for rural, vulnerable, and/or marginalised populations. This research should occur in parallel with studies that will identify approaches to minimise vaccine hesitancy and effectively integrate care for COVID-19 with other essential health services in LMICs. ISoGH calls on the funders of health research in LMICs to consider the urgency and priority of this research during the COVID-19 pandemic and support studies that could make a positive difference for the populations of LMICs.

Setting research priorities to achieve long-term national road safety goals in Iran

Background

Road traffic crashes (RTCs) and its associated injuries are one of the most important public health problems in the world. In Iran, RTCs rank second in terms of mortality. To address this issue, there is a need for research-based interventions. Prioritizing researches using a variety of approaches and frameworks to determine the most effective interventions is a key nodal point in the RTCs' research policy planning cycle. Thus, this study aims to generate and prioritize research questions in the field of RTCs in Iran.

Methods

By adapting the Child Health and Nutrition Research Initiative (CHNRI) method, this study engaged 25 prominent Iranian academic leaders having role in setting Iran’s long-term road safety goals, a group of research funders, and policymakers. The experts' proposed research questions were independently scored on a set of criteria: feasibility, impact on health, impact on the economy, capacity building, and equity. Following the prioritization of Research Questions (RQs), they were all classified using the 5 Pillar frameworks.

Results

In total, 145 Research Questions were systematically scored by experts against five criteria. Iran's top 20 road traffic safety priorities were established. The RQs related to “road safety management” and “road and infrastructure” achieved a high frequency.

Conclusions

The top 20 research questions in the area of RTCs in Iran were determined by experts. The majority of these RQs were related to “road safety management”. The results of this study may contribute to the optimal use of resources in achieving long-term goals in the prevention and control of road traffic crashes and its related injuries. Considering these RQs as research investment options will improve the current status of Road Traffic Injuries (RTIs) at a national level and further advance toward compliance with international goals. If these research priorities are addressed, and their findings are implemented, we can anticipate a significant reduction in the number of crashes, injuries, and deaths.

Methods for Identifying Health Research Gaps, Needs, and Priorities: a Scoping Review

BACKGROUND

Well-defined, systematic, and transparent processes to identify health research gaps, needs, and priorities are vital to ensuring that available funds target areas with the greatest potential for impact.

OBJECTIVE

The purpose of this review is to characterize methods conducted or supported by research funding organizations to identify health research gaps, needs, or priorities.

METHOD

We searched MEDLINE, PsycINFO, and the Web of Science up to September 2019. Eligible studies reported on methods to identify health research gaps, needs, and priorities that had been conducted or supported by research funding organizations. Using a published protocol, we extracted data on the method, criteria, involvement of stakeholders, evaluations, and whether the method had been replicated (i.e., used in other studies).

RESULTS

Among 10,832 citations, 167 studies were eligible for full data extraction. More than half of the studies employed methods to identify both needs and priorities, whereas about a quarter of studies focused singularly on identifying gaps (7%), needs (6%), or priorities (14%) only. The most frequently used methods were the convening of workshops or meetings (37%), quantitative methods (32%), and the James Lind Alliance approach, a multi-stakeholder research needs and priority setting process (28%). The most widely applied criteria were importance to stakeholders (72%), potential value (29%), and feasibility (18%). Stakeholder involvement was most prominent among clinicians (69%), researchers (66%), and patients and the public (59%). Stakeholders were identified through stakeholder organizations (51%) and purposive (26%) and convenience sampling (11%). Only 4% of studies evaluated the effectiveness of the methods and 37% employed methods that were reproducible and used in other studies.

DISCUSSION

To ensure optimal targeting of funds to meet the greatest areas of need and maximize outcomes, a much more robust evidence base is needed to ascertain the effectiveness of methods used to identify research gaps, needs, and priorities.

Research priorities to address the global burden of chronic obstructive pulmonary disease (COPD) in the next decade

Background

The global prevalence of chronic obstructive pulmonary disease (COPD) has increased markedly in recent decades. Given the scarcity of resources available to address global health challenges and respiratory medicine being relatively under-invested in, it is important to define research priorities for COPD globally. In this paper, we aim to identify a ranked set of COPD research priorities that need to be addressed in the next 10 years to substantially reduce the global impact of COPD.

Methods

We adapted the Child Health and Nutrition Research Initiative (CHNRI) methodology to identify global COPD research priorities.

Results

62 experts contributed 230 research ideas, which were scored by 34 researchers according to six pre-defined criteria: answerability, effectiveness, feasibility, deliverability, burden reduction, and equity. The top-ranked research priority was the need for new effective strategies to support smoking cessation. Of the top 20 overall research priorities, six were focused on feasible and cost-effective pulmonary rehabilitation delivery and access, particularly in primary/community care and low-resource settings. Three of the top 10 overall priorities called for research on improved screening and accurate diagnostic methods for COPD in low-resource primary care settings. Further ideas that drew support involved a better understanding of risk factors for COPD, development of effective training programmes for health workers and physicians in low resource settings, and evaluation of novel interventions to encourage physical activity.

Conclusions

The experts agreed that the most pressing feasible research questions to address in the next decade for COPD reduction were on prevention, diagnosis and rehabilitation of COPD, especially in low resource settings. The largest gains should be expected in low- and middle-income countries (LMIC) settings, as the large majority of COPD deaths occur in those settings. Research priorities identified by this systematic international process should inform and motivate policymakers, funders, and researchers to support and conduct research to reduce the global burden of COPD.

Advancing Diagnostic Safety Research: Results of a Systematic Research Priority Setting Exercise

BACKGROUND

Diagnostic errors are a major source of preventable harm but the science of reducing them remains underdeveloped.

OBJECTIVE

To identify and prioritize research questions to advance the field of diagnostic safety in the next 5 years.

PARTICIPANTS

Ninety-seven researchers and 42 stakeholders were involved in the identification of the research priorities.

DESIGN

We used systematic prioritization methods based on the Child Health and Nutrition Research Initiative (CHNRI) methodology. We first invited a large international group of expert researchers in various disciplines to submit research questions while considering five prioritization criteria: (1) usefulness, (2) answerability, (3) effectiveness, (4) potential for translation, and (5) maximal potential for effect on diagnostic safety. After consolidation, these questions were prioritized at an in-person expert meeting in April 2019. Top-ranked questions were subsequently reprioritized through scoring on the five prioritization criteria using an online questionnaire. We also invited non-research stakeholders to assign weights to the five criteria and then used these weights to adjust the final prioritization score for each question.

KEY RESULTS

Of the 207 invited researchers, 97 researchers responded and 78 submitted 333 research questions which were then consolidated. Expert meeting participants (n = 21) discussed questions in different breakout sessions and prioritized 50, which were subsequently reduced to the top 20 using the online questionnaire. The top 20 questions addressed mostly system factors (e.g., implementation and evaluation of information technologies), teamwork factors (e.g., role of nurses and other health professionals in the diagnostic process), and strategies to engage patients in the diagnostic process.

CONCLUSIONS

Top research priorities for advancing diagnostic safety in the short-term include strengthening systems and teams and engaging patients to support diagnosis. High-priority areas identified using these systematic methods can inform an actionable research agenda for reducing preventable diagnostic harm.

A common framework of steps and criteria for prioritizing topics for evidence syntheses: a systematic review

OBJECTIVE

The objective of the study was to systematically review the literature for proposed approaches and exercises conducted to prioritize topics or questions for systematic reviews and other types of evidence syntheses in any health-related area.

STUDY DESIGN AND SETTING

A systematic review. We searched Medline and CINAHL databases in addition to Cochrane website and Google Scholar. Teams of two reviewers independently screened the studies and extracted data.

RESULTS

We included 31 articles reporting on 29 studies: seven proposed approaches for prioritization and 25 conducted prioritization exercises (three studies did both). The included studies addressed the following fields: clinical (n = 19; 66%), public health (n = 10; 34%), and health policy and systems (n = 8; 28%), with six studies (21%) addressing more than one field. We categorized prioritization into 11 steps clustered in 3 phases (preprioritization, prioritization, and postprioritization). Twenty-eight studies (97%) involved or proposed involving stakeholders in the priority-setting process. These 28 studies referred to twelve stakeholder categories, most frequently to health care providers (n = 24; 86%) and researchers (n = 21; 75%). A common framework of 25 prioritization criteria was derived, clustered in 10 domains.

CONCLUSION

We identified literature that addresses different aspects of prioritizing topics or questions for evidence syntheses, including prioritization steps and criteria. The identified steps and criteria can serve as a menu of options to select from, as judged appropriate to the context.

Setting priorities in child health research in India for 2016-2025: a CHNRI exercise undertaken by the Indian Council for Medical Research and INCLEN Trust

Background

Millennium Development Goal 4 (MDGs) mobilised countries to reduce child mortality by two thirds the 1990 rate in 2015. While India did not reach MDG 4, it considerably reduced child mortality in the MDG-era. Efficient and targeted interventions and adequate monitoring are necessary to further progress in improvements to child health. Looking forward to the Sustainable Development Goal (SDG)-era, the Indian Council of Medical Research and The INCLEN Trust International conducted a national research priority setting exercise for maternal, child, newborn health, and maternal and child nutrition. Here, results are reported for child health.

Methods

The Child Health and Nutrition Research Initiative (CHNRI) method for research priority setting was employed. Research ideas were crowd-sourced from a network of child health experts from across India; these were refined and consolidated into research options (ROs) which were scored against five weighted criteria to arrive weighted Research Priority Scores (wRPS). National and regional priority lists were prepared.

Results

90 experts contributed 596 ideas that were consolidated into 101 research options (ROs). These were scored by 233 experts nationwide. National wRPS for ROs ranged between 0.92 and 0.51. The majority of the top research priorities related to development of cost-effective interventions and their implementation, and impact evaluations, improving data quality; and monitoring of existing programs, or improving the management of morbidities. The research priorities varied between regions, the Economic Action Group and North-Eastern states prioritised questions relating to delivering interventions at community- or household-level, whereas the North-Eastern states and Union Territories prioritised research questions involving managing and measuring malaria, and the Southern and Western states prioritised research questions involving pharmacovigilance of vaccines, impact of newly introduced vaccines, and delivery of vaccines to hard-to-reach populations.

Conclusions

Research priorities varied geographically, according the stage of development of the area and mostly pertained to implementation sciences, which was expected given diversity in epidemiological profiles. Priority setting should help guide investment decisions by national and international agencies, therefore encouraging researchers to focus on priority areas. The ICMR has launched a grants programme for implementation research on maternal and child health to pursue research priorities identified by this exercise.

Research co-design in health: a rapid overview of reviews

Background

Billions of dollars are lost annually in health research that fails to create meaningful benefits for patients. Engaging in research co-design – the meaningful involvement of end-users in research – may help address this research waste. This rapid overview of reviews addressed three related questions, namely (1) what approaches to research co-design exist in health settings? (2) What activities do these research co-design approaches involve? (3) What do we know about the effectiveness of existing research co-design approaches? The review focused on the study planning phase of research, defined as the point up to which the research question and study design are finalised.

Methods

Reviews of research co-design were systematically identified using a rapid overview of reviews approach (PROSPERO: CRD42019123034). The search strategy encompassed three academic databases, three grey literature databases, and a hand-search of the journal Research Involvement and Engagement. Two reviewers independently conducted the screening and data extraction and resolved disagreements through discussion. Disputes were resolved through discussion with a senior author (PB). One reviewer performed quality assessment. The results were narratively synthesised.

Results

A total of 26 records (reporting on 23 reviews) met the inclusion criteria. Reviews varied widely in their application of ‘research co-design’ and their application contexts, scope and theoretical foci. The research co-design approaches identified involved interactions with end-users outside of study planning, such as recruitment and dissemination. Activities involved in research co-design included focus groups, interviews and surveys. The effectiveness of research co-design has rarely been evaluated empirically or experimentally; however, qualitative exploration has described the positive and negative outcomes associated with co-design. The research provided many recommendations for conducting research co-design, including training participating end-users in research skills, having regular communication between researchers and end-users, setting clear end-user expectations, and assigning set roles to all parties involved in co-design.

Conclusions

Research co-design appears to be widely used but seldom described or evaluated in detail. Though it has rarely been tested empirically or experimentally, existing research suggests that it can benefit researchers, practitioners, research processes and research outcomes. Realising the potential of research co-design may require the development of clearer and more consistent terminology, better reporting of the activities involved and better evaluation.

Exploring individual and demographic characteristics and their relation to CHNRI Criteria from an international public stakeholder group: an analysis using random intercept and logistic regression modelling

INTRODUCTION

The Child Health and Nutrition Research Initiative (CHNRI) method for health research prioritisation relies on stakeholders weighting criteria used to assess research options. These weights in turn impact on the final scores and ranks assigned to research options. Three quarters of CHNRI studies published to date have not involved stakeholders in criteria weighting. Of those that have, few incorporated members of the public into stakeholder groups. Those that have compared different stakeholder groups, such as donors, researchers, or policy makers, showed that different groups place different values upon CHNRI criteria. When choosing the composition of a stakeholder group, it may be important to understand factors that may influence weighting. Drawing upon a group of international public stakeholders, this study explores some of the effects of individual and demographic characteristics has on the weights assigned to the most commonly used CHNRI criteria, with the aim of informing future researchers on avoiding future biases.

METHODS

Individual and demographic information and 5-point Likert scale responses to questions about the importance of 15 CHNRI criteria were collected from 1031 "Turkers" (Amazon Mechanical Turk workers) via Amazon Mechanical Turk (AMT), which is an online crowdsourcing platform. Thirteen of the fifteen criteria were analysed using random-intercept models and the remaining two were analysed through logistic regression.

RESULTS

Self-reported health status explained most of the variability in participants' responses across criteria (11/15 criteria), followed by being female (10/15), ethnicity (9/15), employment (8/15), and religion (7/15). Differences across criteria indicate that when choosing stakeholder groups, researchers need to consider these factors to minimise bias.

CONCLUSION

Researchers should collect and report more detailed information from stakeholders, including individual and demographic characteristics, and ensure participation from both genders, multiple ethnicities, religious beliefs, and people with differing health statuses to be transparent regarding possible biases in health research prioritisation. Our analyses indicate that these factors do influence the relative importance of these values, even when the data appears fairly homogeneous.

Setting weights for fifteen CHNRI criteria at the global and regional level using public stakeholders: an Amazon Mechanical Turk study

Introduction

Stakeholder involvement has been described as an indispensable part of health research priority setting. Yet, more than 75% of the exercises using the Child Health and Nutrition Research Initiative (CHNRI) methodology have omitted the step involving stakeholders in priority setting. Those that have used stakeholders have rarely used the public, possibly due to the difficulty of assembling and/or accessing a public stakeholder group. In order to strengthen future exercises using the CHNRI methodology, we have used a public stakeholder group to weight 15 CHNRI criteria, and have explored regional differences or being a health stakeholder is influential, and whether the criteria are collapsible.

Methods

Using Amazon Mechanical Turk (AMT), an online crowdsourcing platform, we collected demographic information and conducted a Likert-scale format survey about the importance of the CHNRI criteria from 1051 stakeholders. The Kruskal-Wallis test, with Dunn's test for posthoc comparisons, was used to examine regional differences and Wilcoxon rank-sum test was used to analyse differences between stakeholders with health training/background and stakeholders without a health background and by region. A Factor Analysis (FA) was conducted on the criteria to identify the main domains connecting them. Criteria means were converted to weights.

Results

There were regional differences in thirteen of fifteen criteria according to the Kruskal-Wallis test and differences in responses from health stakeholders vs those who were not in eleven of fifteen criteria using the Wilcoxon rank-sum test. Three components were identified: improve and impact results; implementation and affordability; and, study design and dissemination. A formula is provided to convert means to weights for future studies.

Conclusion

In future CHNRI studies, researchers will need to ensure adequate representation from stakeholders to undue bias of CHNRI results. These results should be used in combination with other stakeholder groups, including government, donors, policy makers, and bilateral agencies. Global and regional stakeholder groups scored CHNRI criteria differently; due to this, researchers should consider which group to use in their CHNRI exercises.

Agreeing on global research priorities for medication safety: an international prioritisation exercise

Objectives

Medication errors continue to contribute substantially to global morbidity and mortality. In the context of the recent launch of the World Health Organization’s (WHO) Third Global Patient Safety Challenge: Medication Without Harm, we sought to establish agreement on research priorities for medication safety.

Methods

We undertook a consensus prioritisation exercise using an approach developed by the Child Health and Nutrition Research Initiative. Based on a combination of productivity and citations, we identified leading researchers in patient and medication safety and invited them to participate. We also extended the invitation to a further pool of experts from the WHO Global Patient Safety Network. All experts independently generated research ideas, which they then independently scored based on the criteria of: answerability, effectiveness, innovativeness, implementation, burden reduction and equity. An overall Research Priority Score and Average Expert Agreement were calculated for each research question.

Findings

131 experts submitted 333 research ideas, and 42 experts then scored the proposed research questions. The top prioritised research areas were: (1) deploying and scaling technology to enhance medication safety; (2) developing guidelines and standard operating procedures for high-risk patients, medications and contexts; (3) score-based approaches to predicting high-risk patients and situations; (4) interventions to increase patient medication literacy; (5) focused training courses for health professionals; and (6) universally applicable pictograms to avoid medication-related harm. Whilst there was a focus on promoting patient education and involvement across resource settings, priorities identified in high-resource settings centred on the optimisation of existing systems through technology. In low- and middle-resource settings, priorities focused on identifying systemic issues contributing to high-risk situations.

Conclusions

WHO now plans to work with global, regional and national research funding agencies to catalyse the investment needed to enable teams to pursue these research priorities in medication safety across high-, middle- and low-resource country settings.

Stakeholder involvement in health research priority setting in low income countries: the case of Zambia

SUMMARY

While there is increasing recognition of the importance of stakeholder involvement in health research priority setting there is a paucity of literature reporting on stakeholder involvement in health research priority setting in low income countries. This paper fills this gap by identifying and discussing the roles and legitimacy of different stakeholders (including the public and patients) involved in the health research priority setting process in Zambia; identifying the barriers to public participation and proposing improvement strategies.We interviewed 28 policy makers and practitioners who had participated in the national level health research priority setting in Zambia. Reported participants in health research priority setting included research users, researchers, research funders and the community/ public. Research funders were thought to have undue influence while the public and patients were not effectively involved. This could be due to the public's lack of education, lack of resources to facilitate public involvement and limited skills to meaningfully engage the public. Participation of people from rural areas, women and young professionals was also limited.While there is a commitment to broad stakeholder involvement in health research priority setting, there's limited public/patient involvement. Public education, availing more resources, and skills to meaningfully engage the public need to be explored. The undue influence of research funders should be mitigated and incentives availed to ensure that they align their research funding with the national priorities. These efforts would strengthen meaningful stakeholder engagement in health research prioritization within Zambia and other similar contexts.

ABSTRACT

Background Stakeholder involvement in health research priority setting contributes to the legitimacy and acceptability of the priorities. Hence legitimate priority setting should involve a broad representation of stakeholders including the public. While there is a growing body of literature on health research prioritization in low income countries, there is a paucity of literature reporting on stakeholder involvement in the process. The objectives of this paper are to; 1) identify the stakeholders who were involved in the health research priority setting process in Zambia; 2) discuss the roles and perceived legitimacy of the stakeholders and analyze the degree to which patients/ public was involved; 3) To discuss some of the barriers to stakeholder participation in Zambia and similar contexts and to propose improvement strategies.Methods This was a qualitative study involving 28 in-depth interviews with stakeholders who had participated in the national level health research priority setting exercises in Zambia. An interview guide was used. Audio recorded interviews were transcribed and analyzed using INVIVO 10. Analysis of the Stakeholders' theme involved identifying the different dimensions of stakeholder involvement as discussed in the interviews.Results Identified stakeholders included; research users, researchers, research funders and the community/ public. We found that health research priority setting involved research users, researchers, research funders and the community/ public. However, research funders were thought to have undue influence while the public and patients were not effectively involved. While the respondents recognized the advantages of involving the public and patients, they were not effectively involved. This could be due to the public's limited understanding of the technicalities of priority setting, lack of resources to facilitate public involvement and limited skills to meaningfully engage the public. Participation from rural areas, women, and young professionals was also limited.Conclusions While there is a commitment to broad stakeholder involvement in health research priority setting, the public is left out. Efforts such as public education, availing more resources, and skills to meaningfully engage the public need to be explored. The undue influence of research funders should be mitigated through their direct involvement in the prioritization process and incentives to ensure that they align their research funding with the national priorities. These efforts would strengthen meaningful stakeholder engagement in health research prioritization within Zambia and other similar contexts.

Setting research priorities to achieve long-term health targets in Iran

Background

In 2015, it was estimated that the burden of disease in Iran comprised of 19 million disability-adjusted life years (DALYs), 74% of which were due to non-communicable diseases (NCDs). The observed leading causes of death were cardiovascular diseases (41.9%), neoplasms (14.9%), and road traffic injuries (7.4%). Even so, the health research investment in Iran continues to remain limited. This study aims to identify national health research priorities in Iran for the next five years to assist the efficient use of resources towards achieving the long-term health targets.

Methods

Adapting the Child Health and Nutrition Research Initiative (CHNRI) method, this study engaged 48 prominent Iranian academic leaders in the areas related to Iran’s long-term health targets, a group of research funders and policy makers, and 68 stakeholders from the wider society. 128 proposed research questions were scored independently using a set of five criteria: feasibility, impact on health, impact on economy, capacity building, and equity.

Findings

The top-10 priorities were focused on the research questions relating to: health insurance system reforms to improve equity; integration of NCDs prevention strategy into primary health care; cost-effective population-level interventions for NCDs and road traffic injury prevention; tailoring medical qualifications; epidemiological assessment of NCDs by geographic areas; equality in the distribution of health resources and services; current and future common health problems in Iran’s elderly and strategies to reduce their economic burden; the status of antibiotic resistance in Iran and strategies to promote rational use of antibiotics; the health impacts of water crisis; and research to replace the physician-centered health system with a team-based one.

Conclusions

These findings highlight consensus amongst various prominent Iranian researchers and stakeholders over the research priorities that require investment to generate information and knowledge relevant to the national health targets and policies. The exercise should assist in addressing the knowledge gaps to support both the National General Health Policies by 2025 and the health targets of the United Nations’ Sustainable Development Goals by 2030.

Research priorities in Maternal, Newborn, & Child Health & Nutrition for India: An Indian Council of Medical Research-INCLEN Initiative

In India, research prioritization in Maternal, Newborn, and Child Health and Nutrition (MNCHN) themes has traditionally involved only a handful of experts mostly from major cities. The Indian Council of Medical Research (ICMR)-INCLEN collaboration undertook a nationwide exercise engaging faculty from 256 institutions to identify top research priorities in the MNCHN themes for 2016-2025. The Child Health and Nutrition Research Initiative method of priority setting was adapted. The context of the exercise was defined by a National Steering Group (NSG) and guided by four Thematic Research Subcommittees. Research ideas were pooled from 498 experts located in different parts of India, iteratively consolidated into research options, scored by 893 experts against five pre-defined criteria (answerability, relevance, equity, investment and innovation) and weighed by a larger reference group. Ranked lists of priorities were generated for each of the four themes at national and three subnational (regional) levels [Empowered Action Group & North-Eastern States, Southern and Western States, & Northern States (including West Bengal)]. Research priorities differed between regions and from overall national priorities. Delivery domain of research which included implementation research constituted about 70 per cent of the top ten research options under all four themes. The results were endorsed in the NSG meeting. There was unanimity that the research priorities should be considered by different governmental and non-governmental agencies for investment with prioritization on implementation research and issues cutting across themes.

Research priorities for adolescent health in low- and middle-income countries: A mixed-methods synthesis of two separate exercises

Background

In order to clarify priorities and stimulate research in adolescent health in low- and middle-income countries (LMICs), the World Health Organization (WHO) conducted two priority-setting exercises based on the Child Health and Nutrition Research Initiative (CHNRI) methodology related to 1) adolescent sexual and reproductive health and 2) eight areas of adolescent health including communicable diseases prevention and management, injuries and violence, mental health, non-communicable diseases management, nutrition, physical activity, substance use, and health policy. Although the CHNRI methodology has been utilized in over 50 separate research priority setting exercises, none have qualitatively synthesized the ultimate findings across studies. The purpose of this study was to conduct a mixed-method synthesis of two research priority-setting exercises for adolescent health in LMICs based on the CHNRI methodology and to situate the priority questions within the current global health agenda.

Methods

All of the 116 top-ranked questions presented in each exercise were analyzed by two independent reviewers. Word clouds were generated based on keywords from the top-ranked questions. Questions were coded and content analysis was conducted based on type of delivery platform, vulnerable populations, and the Survive, Thrive, and Transform framework from the United Nations Global Strategy for Women’s, Children’s, and Adolescents’ Health, 2016-2030.

Findings

Within the 53 top-ranked intervention-related questions that specified a delivery platform, the platforms specified were schools (n = 17), primary care (n = 12), community (n = 11), parenting (n = 6), virtual media (n = 5), and peers (n = 2). Twenty questions specifically focused on vulnerable adolescents, including those living with HIV, tuberculosis, mental illness, or neurodevelopmental disorders; victims of gender-based violence; refugees; young persons who inject drugs; sex workers; slum dwellers; out-of-school youth; and youth in armed conflict. A majority of the top-ranked questions (108/116) aligned with one or a combination of the Survive (n = 39), Thrive (n = 67), and Transform (n = 28) agendas.

Conclusions

This study advances the CHNRI methodology by conducting the first mixed-methods synthesis of multiple research priority-setting exercises by analyzing keywords (using word clouds) and themes (using content analysis).

Setting health research priorities using the CHNRI method: VII. A review of the first 50 applications of the CHNRI method

Background

Several recent reviews of the methods used to set research priorities have identified the CHNRI method (acronym derived from the “Child Health and Nutrition Research Initiative”) as an approach that clearly became popular and widely used over the past decade. In this paper we review the first 50 examples of application of the CHNRI method, published between 2007 and 2016, and summarize the most important messages that emerged from those experiences.

Methods

We conducted a literature review to identify the first 50 examples of application of the CHNRI method in chronological order. We searched Google Scholar, PubMed and so–called grey literature.

Results

Initially, between 2007 and 2011, the CHNRI method was mainly used for setting research priorities to address global child health issues, although the first cases of application outside this field (eg, mental health, disabilities and zoonoses) were also recorded. Since 2012 the CHNRI method was used more widely, expanding into the topics such as adolescent health, dementia, national health policy and education. The majority of the exercises were focused on issues that were only relevant to low– and middle–income countries, and national–level applications are on the rise. The first CHNRI–based articles adhered to the five recommended priority–setting criteria, but by 2016 more than two–thirds of all conducted exercises departed from recommendations, modifying the CHNRI method to suit each particular exercise. This was done not only by changing the number of criteria used, but also by introducing some entirely new criteria (eg, “low cost”, “sustainability”, “acceptability”, “feasibility”, “relevance” and others).

Conclusions

The popularity of the CHNRI method in setting health research priorities can be attributed to several key conceptual advances that have addressed common concerns. The method is systematic in nature, offering an acceptable framework for handling many research questions. It is also transparent and replicable, because it clearly defines the context and priority–setting criteria. It is democratic, as it relies on “crowd–sourcing”. It is inclusive, fostering “ownership” of the results by ensuring that various groups invest in the process. It is very flexible and adjustable to many different contexts and needs. Finally, it is simple and relatively inexpensive to conduct, which we believe is one of the main reasons for its uptake by many groups globally, particularly those in low– and middle–income countries.