Systematic review of international Delphi surveys for core outcome set development: representation of international patients

Core outcome sets for trials of interventions to prevent and to treat multimorbidity in adults in low and middle-income countries: the COSMOS study

INTRODUCTION

The burden of multimorbidity is recognised increasingly in low- and middle-income countries (LMICs), creating a strong emphasis on the need for effective evidence-based interventions. Core outcome sets (COS) appropriate for the study of multimorbidity in LMICs do not presently exist. These are required to standardise reporting and contribute to a consistent and cohesive evidence-base to inform policy and practice. We describe the development of two COS for intervention trials aimed at preventing and treating multimorbidity in adults in LMICs.

METHODS

To generate a comprehensive list of relevant prevention and treatment outcomes, we conducted a systematic review and qualitative interviews with people with multimorbidity and their caregivers living in LMICs. We then used a modified two-round Delphi process to identify outcomes most important to four stakeholder groups (people with multimorbidity/caregivers, multimorbidity researchers, healthcare professionals and policymakers) with representation from 33 countries. Consensus meetings were used to reach agreement on the two final COS.

REGISTRATION

https://www.comet-initiative.org/Studies/Details/1580.

RESULTS

The systematic review and qualitative interviews identified 24 outcomes for prevention and 49 for treatment of multimorbidity. An additional 12 prevention and 6 treatment outcomes were added from Delphi round 1. Delphi round 2 surveys were completed by 95 of 132 round 1 participants (72.0%) for prevention and 95 of 133 (71.4%) participants for treatment outcomes. Consensus meetings agreed four outcomes for the prevention COS: (1) adverse events, (2) development of new comorbidity, (3) health risk behaviour and (4) quality of life; and four for the treatment COS: (1) adherence to treatment, (2) adverse events, (3) out-of-pocket expenditure and (4) quality of life.

CONCLUSION

Following established guidelines, we developed two COS for trials of interventions for multimorbidity prevention and treatment, specific to adults in LMIC contexts. We recommend their inclusion in future trials to meaningfully advance the field of multimorbidity research in LMICs.

PROSPERO REGISTRATION NUMBER

CRD42020197293.

Development of a core outcome set for physiotherapy trials in adults with bronchiectasis (the COS-PHyBE study): A Delphi study and consensus meeting

Background

Physiotherapy is recommended for bronchiectasis management, but there is disparity in evidence supporting its use. This is partly because of inconsistency and poor reporting of outcomes in available studies. A Core Outcome Set (COS) may improve trial consistency and decrease reporting bias. This study aimed to develop a COS for physiotherapy clinical trials in adults with bronchiectasis.

Methods

A comprehensive list of outcomes was developed using a systematic review and qualitative semi-structured interviews with patients with bronchiectasis and physiotherapists.An international two-round online Delphi survey was conducted. Outcomes scored 7-9 (crucial) by ≥ 70 % of participants and 1-3 (not that important) by ≤ 15 % of participants from each stakeholder in the Likert scale were nominated for inclusion in the COS. Nominated outcomes and those considered crucial by only one of the stakeholders' groups were further discussed and voted in an international consensus meeting.

Results

A list of 137 outcomes was generated; 104 from literature and 33 from interviews. A Delphi survey containing 59 outcomes was completed by 171 participants from 20 countries. After the consensus meeting, representatives agreed on seven outcomes: health-related quality of life, respiratory symptoms, physical functioning, emotional and psychological functioning, fatigue, adherence to treatment, and functional exercise capacity.

Conclusion

A minimum set of seven outcomes are recommended to be included as measurements in future trials evaluating physiotherapy interventions for bronchiectasis.

Experiences of core outcome set developers on including stakeholders from low- and middle-income countries: An online survey

Core outcome set (COS) development and use enhances comparability of research findings. It may also enhance the translation of research into practice and reduce research waste. However, there is limited involvement of stakeholders from low- and middle-income countries (LMICs) in COS development and use. In this study, we explored the experiences of researchers in COS development projects who included stakeholders from LMICs. Online survey conducted in English of 70 COS developers from HICs who had included LMIC stakeholders in the process of developing a COS, published before the end of 2019. Respondents were identified from the COMET database and sent a link to the survey via a personalised email. Quantitative data were analysed using simple descriptive statistics. Qualitative data analysis was based on qualitative content analysis. There were 37 respondents yielding a 53% overall response rate. Analysis was limited to the responses related to 29 COS developed in the years 2015 to 2019, to reduce the potential for recall bias for earlier COS. Most respondents 20/29 (69%) were researchers. Determining 'what to measure' was reported as the most common stage of inclusion of LMIC stakeholders. Respondents cited (24/29, 83%) their ongoing collaborations with LMIC stakeholders such as clinicians or researchers as their main rationale for including LMICs stakeholders and reported that translation of the Delphi into languages other than English may be useful to enhance wider stakeholder participation. Involvement of LMIC stakeholders only in the later stages of COS development, lack of adequate resources to support their involvement, and lack of networks and contacts were thought to limit fuller participation of stakeholders from LMICs. To improve the involvement of LMIC stakeholders in COS development and use, COS developers need to raise awareness on the utility of COS. The need for and feasibility of translation into multiple languages warrants further discussion.

Core outcome sets for trials of interventions to prevent and to treat multimorbidity in low- and middle-income countries: the COSMOS study

Introduction

The burden of multimorbidity is recognised increasingly in low- and middle-income countries (LMICs), creating a strong emphasis on the need for effective evidence-based interventions. A core outcome set (COS) appropriate for the study of multimorbidity in LMIC contexts does not presently exist. This is required to standardise reporting and contribute to a consistent and cohesive evidence-base to inform policy and practice. We describe the development of two COS for intervention trials aimed at the prevention and treatment of multimorbidity in LMICs.

Methods

To generate a comprehensive list of relevant prevention and treatment outcomes, we conducted a systematic review and qualitative interviews with people with multimorbidity and their caregivers living in LMICs. We then used a modified two-round Delphi process to identify outcomes most important to four stakeholder groups with representation from 33 countries (people with multimorbidity/caregivers, multimorbidity researchers, healthcare professionals, and policy makers). Consensus meetings were used to reach agreement on the two final COS. Registration: https://www.comet-initiative.org/Studies/Details/1580.

Results

The systematic review and qualitative interviews identified 24 outcomes for prevention and 49 for treatment of multimorbidity. An additional 12 prevention, and six treatment outcomes were added from Delphi round one. Delphi round two surveys were completed by 95 of 132 round one participants (72.0%) for prevention and 95 of 133 (71.4%) participants for treatment outcomes. Consensus meetings agreed four outcomes for the prevention COS: (1) Adverse events, (2) Development of new comorbidity, (3) Health risk behaviour, and (4) Quality of life; and four for the treatment COS: (1) Adherence to treatment, (2) Adverse events, (3) Out-of-pocket expenditure, and (4) Quality of life.

Conclusion

Following established guidelines, we developed two COS for trials of interventions for multimorbidity prevention and treatment, specific to LMIC contexts. We recommend their inclusion in future trials to meaningfully advance the field of multimorbidity research in LMICs.

Measuring the success of programmes of care for people living with dementia: a protocol for consensus building with consumers to develop a set of Core Outcome Measures for Improving Care (COM-IC)

INTRODUCTION

The Core Outcome Measures for Improving Care (COM-IC) project aims to deliver practical recommendations on the selection and implementation of a suite of core outcomes to measure the effectiveness of interventions for dementia care.

METHODS AND ANALYSIS

COM-IC embeds a participatory action approach to using the Alignment-Harmonisation-Results framework for measuring dementia care in Australia. Using this framework, suitable core outcome measures will be identified, analysed, implemented and audited. The methods for analysing each stage will be codesigned with stakeholders, through the conduit of a Stakeholder Reference Group including people living with dementia, formal and informal carers, aged care industry representatives, researchers, clinicians and policy actors. The codesigned evaluation methods consider two key factors: feasibility and acceptability. These considerations will be tested during a 6-month feasibility study embedded in aged care industry partner organisations.

ETHICS AND DISSEMINATION

COM-IC has received ethical approval from The University of Queensland (HREC 2021/HE001932). Results will be disseminated through networks established over the project, and in accordance with both the publication schedule and requests from the Stakeholder Reference Group. Full access to publications and reports will be made available through UQ eSpace (https://espace.library.uq.edu.au/), an open access repository hosted by The University of Queensland.

Awareness and experiences on core outcome set development and use amongst stakeholders from low- and middle- income countries: An online survey

Harmonization of outcomes to be measured in clinical trials can reduce research waste and enhance research translation. One of the ways to standardize measurement is through development and use of core outcome sets (COS). There is limited involvement of low- and middle-income country (LMIC) stakeholders in COS development and use. This study explores the level of awareness and experiences of LMIC stakeholders in the development and use of COS. We conducted an online survey of LMIC stakeholders. Three existing COS (pre-eclampsia, COVID-19, palliative care) were presented as case scenarios, and respondents asked to state (with reason(s)) if they would or would not use the COS if they were working in that area. Quantitative data were analyzed descriptively while qualitative data were analyzed thematically. Of 81 respondents, 26 had COS experience, 9 of whom had been involved in COS development. Personal research interests and prevalence of disease are key drivers for initiation/participation in a given COS project. Most respondents would use the COS for pre-eclampsia (18/26) and COVID-19 (19/26) since the development process included key stakeholders. More than half of the respondents were not sure or would not use the palliative care COS as they felt stakeholder engagement was limited and it was developed for a different resource setting. Respondents reported that use of COS can be limited by (i) feasibility of measuring the outcomes in the COS, (ii) knowledge on the usefulness and availability of COS and (iii) lack of wide stakeholder engagement in the COS development process including having patients and carers in the development process. To ensure the development and use of COS in LMICs, collaborations are essential in awareness raising on COS utility, training, and COS development. The COS also needs to be made accessible in locally understandable languages and feasible to measure in LMICs.

Research priority setting in plastic and reconstructive surgery: A systematic review

BACKGROUND

The health research agenda has historically been led by researchers; however, their priorities may not necessarily align with those of patients, caregivers and clinicians. Research priority setting initiatives identify and prioritise topics which lack evidence. This is particularly important in plastic surgery, a speciality lacking high-quality evidence to definitively answer many common clinical questions. Research priorities direct research activity and funding, so their selection process must be representative and transparent. This review appraised all priority setting initiatives in plastic surgery using the reporting guideline for priority setting of health research (REPRISE).

METHODS

OVID Medline, EMBASE, CINAHL and the James Lind Alliance (JLA) repository were searched (inception - 11/06/21) using search terms for 'research priority setting' and 'plastic and reconstructive surgery'. Dual-author screening and data extraction were conducted, according to PRISMA.

RESULTS

Of 3899 de-duplicated citations, 17 were included. Most studies were conducted in national (14/17), high-income (16/17) settings. More priority setting initiatives focussed on burns (6/17) and hand surgery (4/17) than other subspecialties. The JLA (5/17) and qualitative (5/17) approaches were most used for prioritisation, followed by Delphi techniques (3/17), other surveys (3/17) and mixed methods (1/17). A minority included patient (8/17) or multi-disciplinary (8/17) stakeholders. Few reported strategies for implementing research priorities (6/17) or measuring their impact (2/17).

CONCLUSIONS

Stakeholders from lower-income countries are underrepresented in priority setting initiatives for plastic surgery, despite the global burden of disease. Future studies should recruit more patient and multidisciplinary stakeholders, to achieve meaningful consensus. Clear implementation strategies are needed to maximise impact.

The Core Rehabilitation Outcome Set for Single-Sided Deafness (CROSSSD) study: International consensus on outcome measures for trials of interventions for adults with single-sided deafness

Background

Single-sided deafness (SSD) has functional, psychological, and social consequences. Interventions for adults with SSD include hearing aids and auditory implants. Benefits and harms (outcome domains) of these interventions are until now reported inconsistently in clinical trials. Inconsistency in reporting outcome measures prevents meaningful comparisons or syntheses of trial results. The Core Rehabilitation Outcome Set for Single-Sided Deafness (CROSSSD) international initiative used structured communication techniques to achieve consensus among healthcare users and professionals working in the field of SSD. The novel contribution is a set of core outcome domains that experts agree are critically important to assess in all clinical trials of SSD interventions.

Methods

A long list of candidate outcome domains compiled from a systematic review and published qualitative data, informed the content of a two-round online Delphi survey. Overall, 308 participants from 29 countries were enrolled. Of those, 233 participants completed both rounds of the survey and scored each outcome domain on a 9-point scale. The set of core outcome domains was finalised via a web-based consensus meeting with 12 participants. Votes involved all stakeholder groups, with an approximate 2:1 ratio of professionals to healthcare users participating in the Delphi survey, and a 1:1 ratio participating in the consensus meeting.

Results

The first round of the survey listed 44 potential outcome domains, organised thematically. A further five outcome domains were included in Round 2 based on participant feedback. The structured voting at round 2 identified 17 candidate outcome domains which were voted on at the consensus meeting. Consensus was reached for a core outcome domain set including three outcome domains: spatial orientation, group conversations in noisy social situations, and impact on social situations. Seventy-seven percent of the remaining Delphi participants agreed with this core outcome domain set.

Conclusions

Adoption of the internationally agreed core outcome domain set would promote consistent assessment and reporting of outcomes that are meaningful and important to all relevant stakeholders. This consistency will in turn enable comparison of outcomes reported across clinical trials comparing SSD interventions in adults and reduce research waste. Further research will determine how those outcome domains should best be measured.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-022-06702-1.

The Management of Asymptomatic Congenital Pulmonary Airway Malformation: Results of a European Delphi Survey

Consensus on the optimal management of asymptomatic congenital pulmonary airway malformation (CPAM) is lacking, and comparison between studies remains difficult due to a large variety in outcome measures. We aimed to define a core outcome set (COS) for pediatric patients with an asymptomatic CPAM. An online, three-round Delphi survey was conducted in two stakeholder groups of specialized caregivers (surgeons and non-surgeons) in various European centers. Proposed outcome parameters were scored according to level of importance, and the final COS was established through consensus. A total of 55 participants (33 surgeons, 22 non-surgeons) from 28 centers in 13 European countries completed the three rounds and rated 43 outcome parameters. The final COS comprises seven outcome parameters: respiratory insufficiency, surgical complications, mass effect/mediastinal shift (at three time-points) and multifocal disease (at two time-points). The seven outcome parameters included in the final COS reflect the diversity in priorities among this large group of European participants. However, we recommend the incorporation of these outcome parameters in the design of future studies, as they describe measurable and validated outcomes as well as the accepted age at measurement.

Development of a core outcome set for multimorbidity trials in low/middle-income countries (COSMOS): study protocol

INTRODUCTION

'Multimorbidity' describes the presence of two or more long-term conditions, which can include communicable, non-communicable diseases, and mental disorders. The rising global burden from multimorbidity is well documented, but trial evidence for effective interventions in low-/middle-income countries (LMICs) is limited. Selection of appropriate outcomes is fundamental to trial design to ensure cross-study comparability, but there is currently no agreement on a core outcome set (COS) to include in trials investigating multimorbidity specifically in LMICs. Our aim is to develop international consensus on two COSs for trials of interventions to prevent and treat multimorbidity in LMIC settings.

METHODS AND ANALYSIS

Following methods recommended by the Core Outcome Measures in Effectiveness Trials initiative, the development of these two COSs will occur in parallel in three stages: (1) generation of a long list of potential outcomes for inclusion; (2) two-round online Delphi surveys and (3) consensus meetings. First, to generate an initial list of outcomes, we will conduct a systematic review of multimorbidity intervention and prevention trials and interviews with people living with multimorbidity and their caregivers in LMICs. Outcomes will be classified using an outcome taxonomy. Two-round Delphi surveys will be used to elicit importance scores for these outcomes from people living with multimorbidity, caregivers, healthcare professionals, policy makers and researchers in LMICs. Finally, consensus meetings including all of these stakeholders will be held to agree outcomes for inclusion in the two COSs.

ETHICS AND DISSEMINATION

The study has been approved by the Research Governance Committee of the Department of Health Sciences, University of York, UK (HSRGC/2020/409/D:COSMOS). Each participating country/research group will obtain local ethics board approval. Informed consent will be obtained from all participants. We will disseminate findings through peer-reviewed open access publications, and presentations at global conferences selected to reach a wide range of LMIC stakeholders.

PROSPERO REGISTATION NUMBER

CRD42020197293.

Synthesizing Core Outcome Sets for outcomes research in cohort studies: a systematic review

BACKGROUND

Life course studies are designed to "collect once, use multiple times" for observational and, increasingly, interventional research. Core Outcome Sets (COS) are minimum sets developed for clinical trials by multi-stakeholder consensus methodologies. We aimed to synthesize published COS that might guide outcomes selection for early life cohorts with an interventional focus.

METHODS

We searched PubMed, Medline, COMET, and CROWN for COS published before January 2021 relevant to four life stages (pregnancy, newborns, children <8 years, and parents (adults aged 18-50 years)). We synthesized core outcomes into overarching constructs.

RESULTS

From 46 COS we synthesized 414 core outcomes into 118 constructs. "Quality of life", "adverse events", "medication use", "hospitalization", and "mortality" were consistent across all stages. For pregnancy, common constructs included "preterm birth", "delivery mode", "pre-eclampsia", "gestational weight gain", "gestational diabetes", and "hemorrhage"; for newborns, "birthweight", "small for gestational age", "neurological damage", and "morbidity" and "infection/sepsis"; for pediatrics, "pain", "gastrointestinal morbidity", "growth/weight", "breastfeeding", "feeding problems", "hearing", "neurodevelopmental morbidity", and "social development"; and for adults, "disease burden", "mental health", "neurological function/stroke", and "cardiovascular health/morbidity".

CONCLUSION

This COS synthesis generated outcome constructs that are of high value to stakeholders (participants, health providers, services), relevant to life course research, and could position cohorts for trial capabilities.

IMPACT

We synthesized existing Core Outcome Sets as a transparent methodology that could prioritize outcomes for lifecourse cohorts with an interventional focus. "Quality of life", "adverse events", "medication use", "hospitalization", and "mortality" are important outcomes across pregnancy, newborns, childhood, and early-to-mid-adulthood (the age range relevant to parents). Other common outcomes (such as "birthweight", "cognitive function/ability", "psychological health") are also highly relevant to lifecourse research. This synthesis could assist new early life cohorts to pre-select outcomes that are of high value to stakeholders (participants, health providers, services), are relevant to lifecourse research, and could position them for future trials and interventional capability.

Secondary analysis of data from a core outcome set for burns demonstrated the need for involvement of lower income countries

Objective

To compare the views of participants from different income-status countries on outcome selection for a burn care Core Outcome Set (COS).

Methods

A retrospective analysis of data collected during a two round Delphi survey to prioritise the most important outcomes in burn care research.

Results

There was considerable agreement between participants from low- and middle-income countries (LMICs) and high-income countries (HICs) across outcomes. The groups agreed on 91% of 88 outcomes in round 1 and 92% of 100 in round 2. In cases of discordance, the consensus of participants from LMICs was to include the outcome and for participants from HICs to exclude. There was also considerable agreement between the groups for the top-ten ranking outcomes. Discordance in outcome prioritisation gives an insight into the different values clinicians from LMICs place on outcomes compared to those from HICs. Limitations of the study were that outcome rankings from international patients were not available. Healthcare professionals from LMICs were not involved in the final consensus meeting.

Conclusion

COS developers should consider the need for a COS to be global at protocol stage. Global COS should include equal representation from both LMICs and HICs at all stages of development.