Choosing important health outcomes for comparative effectiveness research: An updated systematic review and involvement of low and middle income countries

Experiences of core outcome set developers on including stakeholders from low- and middle-income countries: An online survey

Core outcome set (COS) development and use enhances comparability of research findings. It may also enhance the translation of research into practice and reduce research waste. However, there is limited involvement of stakeholders from low- and middle-income countries (LMICs) in COS development and use. In this study, we explored the experiences of researchers in COS development projects who included stakeholders from LMICs. Online survey conducted in English of 70 COS developers from HICs who had included LMIC stakeholders in the process of developing a COS, published before the end of 2019. Respondents were identified from the COMET database and sent a link to the survey via a personalised email. Quantitative data were analysed using simple descriptive statistics. Qualitative data analysis was based on qualitative content analysis. There were 37 respondents yielding a 53% overall response rate. Analysis was limited to the responses related to 29 COS developed in the years 2015 to 2019, to reduce the potential for recall bias for earlier COS. Most respondents 20/29 (69%) were researchers. Determining 'what to measure' was reported as the most common stage of inclusion of LMIC stakeholders. Respondents cited (24/29, 83%) their ongoing collaborations with LMIC stakeholders such as clinicians or researchers as their main rationale for including LMICs stakeholders and reported that translation of the Delphi into languages other than English may be useful to enhance wider stakeholder participation. Involvement of LMIC stakeholders only in the later stages of COS development, lack of adequate resources to support their involvement, and lack of networks and contacts were thought to limit fuller participation of stakeholders from LMICs. To improve the involvement of LMIC stakeholders in COS development and use, COS developers need to raise awareness on the utility of COS. The need for and feasibility of translation into multiple languages warrants further discussion.

Outcomes of interventions in neonatal sepsis: A systematic review of qualitative research

BACKGROUND

While a systematic review exists detailing neonatal sepsis outcomes from clinical trials, there remains an absence of a qualitative systematic review capturing the perspectives of key stakeholders.

OBJECTIVES

Our aim is to identify outcomes from qualitative research on any intervention to prevent or improve the outcomes of neonatal sepsis that are important to parents, other family members, healthcare providers, policymakers, and researchers as a part of the development of a core outcome set (COS) for neonatal sepsis.

SEARCH STRATEGY

A literature search was carried out using MEDLINE, EMBASE, CINAHL, and PsycInfo databases.

SELECTION CRITERIA

Publications describing qualitative data relating to neonatal sepsis outcomes were included.

DATA COLLECTION AND ANALYSIS

Drawing on the concepts of thematic synthesis, texts related to outcomes were coded and grouped. These outcomes were then mapped to the domain headings of an existing model.

MAIN RESULTS

Out of 6777 records screened, six studies were included. Overall, 19 outcomes were extracted from the included studies. The most frequently reported outcomes were those in the domains related to parents, healthcare workers and individual organ systemas such as gastrointestinal system. The remaining outcomes were classified under the headings of general outcomes, miscellaneous outcomes, survival, and infection.

CONCLUSIONS

The outcomes identified in this review are different from those reported in neonatal sepsis clinical trials, thus highlighting the importance of incorporating qualitative studies into COS development to encapsulate all relevant stakeholders' perspectives.

Development of a core outcome set for use in research evaluations of interventions for venous leg ulceration: International eDelphi consensus

INTRODUCTION

Venous leg ulceration (VLU) is a chronic, recurring condition with associated pain, malodour, impaired mobility and susceptibility to infection which in turn significantly impacts an individual's health-related quality of life. Randomised controlled trials (RCTs) aim to determine the efficacy of interventions to improve outcomes. To be useful, these outcomes should be consistently and fully reported across RCTs. A core outcome set (COS) is an agreed-upon standardised set of outcomes which should be, at a minimum, reported in all RCTs for a given indication including that of VLU.

AIM

To gain consensus on which outcome domains and outcomes should be considered as core and therefore included in all RCTs of interventions in VLU treatment.

METHOD

Two sequential, two round e-Delphi surveys were completed. The first gained consensus on core outcome domains and the second on core outcomes within those domains. Participants included: people with direct experience of having VLUs and their carers, healthcare professionals whose practice included VLU care and researchers within wound care (clinical, academic, industry).

RESULTS

Five outcome domains; healing, pain, quality of life, resource use and adverse events, and 11 outcomes were rated as core by participants. The patient and not the limb or ulcer was the preferred unit of analysis for reporting.

RECOMMENDATIONS

We recommend investigators report on all five outcome domains, regardless of the type of intervention being evaluated. Future research is needed to identify measurement methods for the 11 identified outcomes. We also recommend investigators follow the CONSORT guidelines (http://www.consort-statement.org/).

Core outcome sets for trials of interventions to prevent and to treat multimorbidity in low- and middle-income countries: the COSMOS study

Introduction

The burden of multimorbidity is recognised increasingly in low- and middle-income countries (LMICs), creating a strong emphasis on the need for effective evidence-based interventions. A core outcome set (COS) appropriate for the study of multimorbidity in LMIC contexts does not presently exist. This is required to standardise reporting and contribute to a consistent and cohesive evidence-base to inform policy and practice. We describe the development of two COS for intervention trials aimed at the prevention and treatment of multimorbidity in LMICs.

Methods

To generate a comprehensive list of relevant prevention and treatment outcomes, we conducted a systematic review and qualitative interviews with people with multimorbidity and their caregivers living in LMICs. We then used a modified two-round Delphi process to identify outcomes most important to four stakeholder groups with representation from 33 countries (people with multimorbidity/caregivers, multimorbidity researchers, healthcare professionals, and policy makers). Consensus meetings were used to reach agreement on the two final COS. Registration: https://www.comet-initiative.org/Studies/Details/1580.

Results

The systematic review and qualitative interviews identified 24 outcomes for prevention and 49 for treatment of multimorbidity. An additional 12 prevention, and six treatment outcomes were added from Delphi round one. Delphi round two surveys were completed by 95 of 132 round one participants (72.0%) for prevention and 95 of 133 (71.4%) participants for treatment outcomes. Consensus meetings agreed four outcomes for the prevention COS: (1) Adverse events, (2) Development of new comorbidity, (3) Health risk behaviour, and (4) Quality of life; and four for the treatment COS: (1) Adherence to treatment, (2) Adverse events, (3) Out-of-pocket expenditure, and (4) Quality of life.

Conclusion

Following established guidelines, we developed two COS for trials of interventions for multimorbidity prevention and treatment, specific to LMIC contexts. We recommend their inclusion in future trials to meaningfully advance the field of multimorbidity research in LMICs.

Current status and implementation strategies of patient education in core outcome set development

OBJECTIVE

Patient participation is essential for Core Outcome Set (COS) development studies. Patient education during participation may help patients better express their views in COS studies. This study aimed to investigate the current status of patient participation and the specified educational information in COS studies.

METHODS

We conducted a systematic review of COS development studies. Information on patient participation in COS research, and especially details of patient education, was analyzed.

RESULTS

In total, 146 COS development studies were included in this review. Of these, 125 studies (85.6%) mentioned patient participation. Most studies did not provide explicit information on patient participation. Some studies mentioned recruiting patients, but ultimately, none of them responded. Six studies reported conducting patient education through workshops, creating patient forums, or providing videos and slides. However, these studies did not provide details on education. Twenty-three studies used the plain language to explain patient outcomes.

CONCLUSION

COS developers are increasingly focusing on patient participation. However, only a few COS studies have explicitly reported conducting measures related to patient education. Further patient education is necessary when they participate in the development of a new Core Outcome Set.

PRACTICE IMPLICATION

This article provides implementation strategies related to patient education for future COS development studies.

A systematic review of studies reporting the development of core outcome sets for use in routine care

OBJECTIVES

Core outcome sets (COS) represent the minimum health outcomes to be measured for a given health condition. Interest is growing in using COS within routine care to support delivery of patient-focused care. This review aims to systematically map COS developed for routine care to understand their scope, stakeholder involvement, and development methods.

METHODS

Medline (Ovid), Scopus, and Web of Science Core collection were searched for studies reporting development of COS for routine care. Data on scope, methods, and stakeholder groups were analyzed in subgroups defined by setting.

RESULTS

Screening 25,301 records identified 262 COS: 164 for routine care only and 98 for routine care and research. Nearly half of the COS (112/254, 44%) were developed with patients, alongside input from experts in registries, insurance, legal, outcomes measurement, and performance management. Research publications were often searched to generate an initial list of outcomes (115/198, 58%) with few searching routine health records (47/198, 24%).

CONCLUSION

An increasing number of COS is being developed for routine care. Although involvement of patient stakeholders has increased in recent years, further improvements are needed. Methodology and scope are broadly similar to COS for research but implementation of the final set is a greater consideration during development.

Patient participation impacts outcome domain selection in core outcome sets for research: an updated systematic review

OBJECTIVES

The importance of including patients, carers, and the public in health research is well recognized, including the need to consider outcomes in health care research that reflect the priorities of patients. Core outcome sets (COS) define the minimum set of outcomes that should be measured and reported in research of a given condition, determined through consensus among key stakeholders. The Core Outcome Measures in Effectiveness Trials Initiative undertakes an annual systematic review (SR) to identify newly published COS to update its online database of COS for research. The objective of this study was to assess the impact of patient participation on COS.

STUDY DESIGN AND SETTING

SR methods used in previous updates were applied to identify research studies published or indexed in 2020 and 2021 (conducted as separate reviews) that report development of a COS, regardless of any specifications relating to condition, population, intervention, or setting. Studies were assessed according to published standards for COS development, and core outcomes extracted from study publications were categorized according to an outcome taxonomy and added to an existing database of core outcome classifications of all previously published COS. The effect of patient participation on core domains was examined.

RESULTS

Searches identified 56 new studies published in 2020 and 54 in 2021. All studies met all four minimum standards for scope, and 42 (75%) of the 2020 studies and 45 (83%) of the 2021 studies met all three standards for stakeholders involved. However, only 19 (34%) of the 2020 studies and 18 (33%) of the 2021 studies met all four standards for the consensus process. COS that involved patients or their representatives are more likely to include life impact outcomes (239, 86%) than COS without patient participation (193, 62%). Physiological/clinical outcomes are almost always specified at a granular level, whereas life impact outcomes are often described at a higher level.

CONCLUSION

This study adds to the body of evidence demonstrating the importance and impact of including patients, carers, and the public in COS development, in particular by demonstrating that the impact of interventions on patients' lives is more likely to be represented in COS that involve patients or their representatives. COS developers are encouraged to pay increased attention to methods and reporting relating to the consensus process. Further work is required to understand the appropriateness and rationale for the discrepancy in granularity levels between outcome domains.

The importance of establishing a core outcome set for endodontic clinical trials and outcomes studies

Endodontic therapy aims to preserve teeth by preventing and treating apical disease, therefore, evaluation of treatment outcome in clinical trials and outcomes studies should effectively assess if it achieves these aims. Traditionally, treatment outcomes have been reported by clinicians after history and clinical examination in what is known as clinician-reported outcomes (CROs). Much less commonly employed however, are patient-reported outcomes (PROs) in which patients directly report on their condition. Endodontic treatment outcome reporting is evolving from a focus on CROs to increasing consideration for patient and disease-focused outcomes, with different criteria being proposed for assessment of treatment outcomes. Unfortunately, this has led to considerable variability and a lack of consensus on the definition, appropriate measurement and reporting of these outcomes. Heterogeneity in outcome reporting in clinical research provides a significant major barrier to conduct meta-analysis, guidelines development, clinical decision making, and ultimately affecting patient care. These effects could, however, be reduced by the establishment of a core outcome set (COS) in endodontics, which is defined as an agreed, standardized set of outcomes that should be included, measured and reported as a minimum in all trials and outcome studies. COS development is a regulated and validated process requiring involvement of appropriate stakeholders as well as a rigorous methodology. To date, COS has been developed for the management of traumatic dental injuries, orthodontic and periodontal treatment and is currently being developed for endodontic treatment. The aim of this review is to discuss the importance of COS in endodontics with focus on the evidence for and impact of heterogeneity in reporting endodontic treatment outcomes. An overview of an ongoing process for development of COS for different endodontic treatment modalities will also be provided.

Developing a core outcome set for hand fractures and joint injuries in adults

Aims

The aim of this study was to develop a core outcome set of what to measure in all future clinical research on hand fractures and joint injuries in adults.

Methods

Phase 1 consisted of steps to identify potential outcome domains through systematic review of published studies, and exploration of the patient perspective through qualitative research, consisting of 25 semi-structured interviews and five focus groups. Phase 2 involved key stakeholder groups (patients, hand surgeons, and hand therapists) prioritizing the outcome domains via a three-round international Delphi survey, with a final consensus meeting to agree the final core outcome set.

Results

The systematic review of 160 studies identified 74 outcome domains based on the World Health Organization International Classification of Functioning, Disability, and Health. Overall, 35 domains were generated through thematic analysis of the patient interviews and focus groups. The domains from these elements were synthesised to develop 37 outcome domains as the basis of the Delphi survey, with a further four generated from participant suggestions in Round 1. The Delphi survey identified 20 outcome domains as ‘very important’ for the core outcome set. At the consensus meeting, 27 participants from key stakeholder groups selected seven outcomes for the core outcome set: pain/discomfort with activity, pain/discomfort with rest, fine hand use/dexterity, self-hygiene/personal care, return to usual work/job, range of motion, and patient satisfaction with outcome/result.

Conclusion

This set of core outcome domains is recommended as a minimum to be reported in all clinical research on hand fractures and joint injuries in adults. While this establishes what to measure, future work will focus on determining how best to measure these outcomes. By adopting this patient-centred core outcome set, consistency and comparability of studies will be improved, aiding meta-analysis and strengthening the evidence base for management of these common and impactful injuries.

Cite this article: Bone Jt Open 2023;4(2):87–95.

The research status and progress of core outcome set in oral health

The core outcome set (COS) refers to the minimum set of outcomes that should be reported by all clinical trials in a particular health field. The use of COS in clinical studies can reduce the heterogeneity caused by using different outcomes across different clinical studies, facilitate the systematic review of different clinical studies on the same topic, reduce selective reporting bias, and increase the utility of clinical studies. The importance of COS in oral health has recently been recognized. This review summarizes the history, necessity, and key methodological points of COS development, with emphasis on the research status and existing problems in COS development, in the field of oral health.

Completeness and Mismatch of Patient-Important Outcomes After Trauma

To assess the completeness of the collection of patient-important outcomes and the mismatch between outcomes measured in research and patients' important issues after trauma.

Summary Background Data

To date, severe trauma has mainly been assessed using in-hospital mortality. Yet, with 80 to 90% survivors discharged from hospital, it is critical to assess the collection of patient important long-term outcomes of trauma.

Methods

Mixed methods study combining a systematic review of outcomes and their comparison with domains elicited by patients during a qualitative study. We searched Medline, EMBASE and clinicaltrials.gov from January 1, 2014 to September 30, 2019 and extracted all outcomes from reports including severe trauma. We compared these outcomes with 97 domains that matter to trauma survivors identified in a previous qualitative study. We defined as patient-important outcome as the 10 most frequently elicited domains in the qualitative study. We assessed the number of domains captured in each report to illustrate the completeness of the collection of patient-important outcomes. We also assessed the mismatch between outcomes collected and what matters to patients.

Findings

Among the 116 reports included in the systematic review, we identified 403 outcomes collected with 154 unique measurements tools. Beside mortality, measurement tools most frequently used were the Glasgow Outcome Scale (31.0%, n=36), questions on patients' return to work (20,7%, n=24) and the EQ-5D (19.0%, n=22). The comparison between the outcomes identified in the systematic review and the domains from the qualitative study found that 10.3% (n=12) reports did not collect any patient-important domains and one collected all 10 patient-important domains. By examining each of the 10 patient-important domains, none was collected in more than 72% of reports and only five were among the ten most frequently measured domains in studies.

Conclusion

The completeness of the collection of the long-term patient-important outcomes after trauma can be improved. There was a mismatch between the domains used in the literature and those considered important by patients during a qualitative study.

Comparison of published core outcome sets with outcomes recommended in regulatory guidance from the US Food and Drug Administration and European Medicines Agency: cross sectional analysis

Objective

To compare the outcomes in published core outcome sets with the outcomes recommended in corresponding guidance documents from the European Medicines Agency (EMA) and US Food and Drug Administration (FDA), matched by health condition.

Design

Cross sectional analysis.

Setting

US and Europe.

Population

Sample of core outcome sets related to drugs, devices, and gene therapy that involved patients in the consensus process, published between 1 January 2015 and 31 December 2019; and corresponding EMA and FDA guidance documents.

Main outcome measures

The extent of matches between outcomes included within core outcome sets and those recommended in corresponding EMA and FDA guidance documents were assessed. Matches were considered to be general (ie, non-specific) or specific (ie, exact). General matches were assessed to determine whether the core outcome set or guidance document outcome was narrower.

Results

Relevant guidance documents were found for for 38 (39%) of 98 eligible published core outcome sets. Among outcomes in core outcome sets, medians of 70% (interquartile range 48-86%) and 52% (33-77%) were matches with outcomes recommended in EMA and FDA documents, respectively. Medians of 46% (27-68%) and 26% (18-46%) were specific matches with outcomes in EMA and FDA documents, respectively. When outcomes were generally matched, the outcomes from core outcome sets were more frequently narrower than the regulatory outcomes (83% and 75% for EMA and FDA, respectively).

Conclusion

Greater adoption of, and reference to, core outcome sets in regulatory guidance documents can encourage clinical trialists, especially those in industry, to measure and report consistent and agreed outcomes and improve the quality of guidance. Given the overlap between outcomes in core outcome sets and regulatory guidance, and given that most core outcome sets now involve patients in the consensus process, these sets could serve as a useful resource for regulators when recommending outcomes for studies evaluating regulated products. Developers are encouraged to appraise recommended outcomes in salient regulatory documents when planning a core outcome set.

Core Outcome Sets in Child Health: A Systematic Review

IMPORTANCE

Developing core outcome sets is essential to ensure that results of clinical trials are comparable and useful. A number of core outcome sets in pediatrics have been published, but a comprehensive in-depth understanding of core outcome sets in this field is lacking.

OBJECTIVE

To systematically identify core outcome sets in child health, collate the diseases to which core outcome sets have been applied, describe the methods used for development and stakeholder participation, and evaluate the methodological quality of existing core outcome sets.

EVIDENCE REVIEW

MEDLINE, SCOPUS, Cochrane Library, and CINAHL were searched using relevant search terms, such as clinical trials, core outcome, and children, along with relevant websites, such as Core Outcome Measures in Effectiveness Trials (COMET). Four researchers worked in teams of 2, performed literature screening and data extraction, and evaluated the methodological quality of core outcome sets using the Core Outcome Set-Standards for Development (COS-STAD).

FINDINGS

A total of 77 pediatric core outcome sets were identified, mainly developed by organizations or researchers in Europe, North America, and Australia and mostly from the UK (22 [29%]) and the US (22 [29%]). A total of 77 conditions were addressed; the most frequent International Classification of Diseases, 11th Revision category was diseases of the digestive system (14 [18%]). Most of the outcomes in pediatric core outcome sets were unordered (34 [44%]) or presented in custom classifications (29 [38%]). Core outcome sets used 1 or more of 8 development methods; the most frequent combination of methods was systematic review/literature review/scoping review, together with the Delphi approach and consensus for decision-making (10 [14%]). Among the 6 main types of stakeholders, clinical experts were the most frequently involved (74 [100%]), while industry representatives were rarely involved (4 [5%]). Only 6 core outcome sets (8%) met the 12 criteria of COS-STAD.

CONCLUSIONS AND RELEVANCE

Future quality of pediatric core outcome sets should be improved based on the standards proposed by the COMET initiative, while core outcome sets methodology and reporting standards should be extended to pediatric populations to help improve the quality of core outcome sets in child health. In addition, the COMET outcome taxonomy should also add items applicable to children.

Usefulness of Cochrane Reviews in Clinical Guideline Development-A Survey of 585 Recommendations

The Danish Health Authority develops clinical practice guidelines to support clinical decision-making based on the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) system and prioritizes using Cochrane reviews. The objective of this study was to explore the usefulness of Cochrane reviews as a source of evidence in the development of clinical recommendations. Evidence-based recommendations in guidelines published by the Danish Health Authority between 2014 and 2021 were reviewed. For each recommendation, it was noted if and how Cochrane reviews were utilized. In total, 374 evidence-based recommendations and 211 expert consensus recommendations were published between 2014 and 2021. Of the 374 evidence-based recommendations, 106 included evidence from Cochrane reviews. In 28 recommendations, all critical and important outcomes included evidence from Cochrane reviews. In 36 recommendations, a minimum of all critical outcomes included evidence from Cochrane reviews, but not all important outcomes. In 33 recommendations, some but not all critical outcomes included evidence from Cochrane reviews. Finally, in nine recommendations, some of the important outcomes included evidence from Cochrane reviews. In almost one-third of the evidence-based recommendations, Cochrane reviews were used to inform clinical recommendations. This evaluation should inform future evaluations of Cochrane review uptake in clinical practice guidelines concerning outcomes important for clinical decision-making.

Secondary analysis of data from a core outcome set for burns demonstrated the need for involvement of lower income countries

Objective

To compare the views of participants from different income-status countries on outcome selection for a burn care Core Outcome Set (COS).

Methods

A retrospective analysis of data collected during a two round Delphi survey to prioritise the most important outcomes in burn care research.

Results

There was considerable agreement between participants from low- and middle-income countries (LMICs) and high-income countries (HICs) across outcomes. The groups agreed on 91% of 88 outcomes in round 1 and 92% of 100 in round 2. In cases of discordance, the consensus of participants from LMICs was to include the outcome and for participants from HICs to exclude. There was also considerable agreement between the groups for the top-ten ranking outcomes. Discordance in outcome prioritisation gives an insight into the different values clinicians from LMICs place on outcomes compared to those from HICs. Limitations of the study were that outcome rankings from international patients were not available. Healthcare professionals from LMICs were not involved in the final consensus meeting.

Conclusion

COS developers should consider the need for a COS to be global at protocol stage. Global COS should include equal representation from both LMICs and HICs at all stages of development.

A protocol for the Development of Core Outcome Sets for Endodontic Treatment modalities (COSET): an international consensus process

Background

The outcome of endodontic treatment is generally assessed using a range of patient and clinician-centred, non-standardised clinical and radiographic outcome measures. This makes it difficult to synthesise evidence for systematic analysis of the literature and the development of clinical guidelines. Core outcome sets (COS) represent a standardised list of outcomes that should be measured and reported in all clinical studies in a particular field. Recently, clinical researchers and guideline developers have focussed on the need for the integration of a patient-reported COS with clinician-centred measures. This study aims to develop a COS that includes both patient-reported outcomes and clinician-centred measures for various endodontic treatment modalities to be used in clinical research and practice.

Methods

To identify reported outcomes (including when and how they are measured), systematic reviews and their included clinical studies, which focus on the outcome of endodontic treatment and were published between 1990 and 2020 will be screened. The COSs will be defined by a consensus process involving key stakeholders using semi-structured interviews and an online Delphi methodology followed by an interactive virtual consensus meeting. A heterogeneous group of key ‘stakeholders’ including patients, general dental practitioners, endodontists, endodontic teachers, clinical researchers, students and policy-makers will be invited to participate. Patients will establish, via interactive interviews, which outcomes they value and feel should be included in a COS. In the Delphi process, other stakeholders will be asked to prioritise outcomes identified from the literature and patient interviews and will have the opportunity at the end of the first round to add outcomes that are not included, but which they consider relevant. Feedback will be provided in the second round, when participants will be asked to prioritise the list again. If consensus is reached, the remaining outcomes will be discussed at an online meeting and agreement established via defined consensus rules of outcome inclusion. If consensus is not reached after the second round, a third round will be conducted with feedback, followed by the online meeting. Following the identification of a COS, we will proceed to identify how and when these outcomes are measured.

Discussion

Using a rigorous methodology, the proposed consensus process aims to develop a COS for endodontic treatment that will be relevant to stakeholders. The results of the study will be shared with participants and COS users. To increase COS uptake, it will also be actively shared with clinical guideline developers, research funders and the editors of general dental and endodontology journals.

Trial registration

COMET 1879. 21 May 2021.

The standards of obstetrics and gynecology core outcome sets: A scoping review

Background

Core outcome sets (COSs) are the minimum outcomes which should be measured and reported by researchers investigating a specific condition. The definition of standards of COSs vary across different health-related areas. This investigated the characteristics of COSs regarding obstetrics and gynecology (OG) and examined the reports and designs of standards of OG COSs.

Methods

A comprehensive search was conduced on the COMET database on December 20, 2019 to identify systematic reviews on COSs. Two reviewers independently evaluated whether the reported OG COS met the reporting requirements as stipulated in the Core Outcome Set-STAndards for Reporting (COS-STAR) statement checklist and the minimum design recommendations as outlined in the Core Outcome Set-STAndards for Development (COS-STAD) checklist.

Results

Forty-four OG COSs related to 26 topics were identified. None of them met all the 25 standards of COS-STAR statement which representing 18 items considered essential for transparent and complete reporting list for all COS studies (range: 6.0-24.0, median: 14.0). The compliance rates to 16 standards of methods and result sections ranged from 27.3%–68.2%. Total COS-STAR compliance items for OG COSs with the prior protocol was significantly higher than without prior protocol (MD = 3.846, 95% CI: 0.835–6.858, P = 0.012). None of the OG COSs met all the 12 criteria in the COS-STAD minimum standards (range: 3.0-11.0, median: 5.0). The compliance rates for all three standards of stakeholders involved and all four standards of the consensus process were lower than 60%.

Conclusions

Methodological and reporting standards of OG COSs should be improved.

Inclusion of participants from low-income and middle-income countries in core outcome sets development: a systematic review

OBJECTIVE

Our study aims to describe differences or similarities in the scope, participant characteristics and methods used in core outcome sets (COS) development when only participants from high-income countries (HICs) were involved compared with when participants from low-income and middle-income countries (LMICs) were also involved.

DESIGN

Systematic review.

DATA SOURCES

Annual Core Outcome Measures in Effectiveness Trials systematic reviews of COS which are updated based on SCOPUS and MEDLINE, searches. The latest systematic review included studies published up to the end of 2019.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

We included studies reporting development of a COS for use in research regardless of age, health condition or setting. Studies reporting the development of a COS for patient-reported outcomes or adverse events or complications were also included.

DATA EXTRACTION AND SYNTHESIS

Data were extracted in relation to scope of the COS study, participant categories and the methods used in outcome selection.

RESULTS

Studies describing 370 COS were identified in the database. Of these, 75 (20%) included participants from LMICs. Only four COS were initiated from an LMIC setting. More than half of COS with LMIC participants were developed in the last 5 years. Cancer and rheumatology were the dominant disease domains. Overall, over 259 (70%) of COS explicitly reported including clinical experts; this was higher where LMIC participants were also included 340 (92%). Most LMIC participants were from China, Brazil and South Africa. Mixed methods for consensus building were used across the two settings.

CONCLUSION

Progress has been made in including LMIC participants in the development of COS, however, there is a need to explore how to enable initiation of COS development from a range of LMIC settings, how to ensure prioritisation of COS that better reflects the burden of disease in these contexts and how to improve public participation from LMICs.

Patient participation in Delphi surveys to develop core outcome sets: systematic review

OBJECTIVES

To describe the design and conduct of core outcome set (COS) studies that have included patients as participants, exploring how study characteristics might impact their response rates.

DESIGN

Systematic review of COS studies published between 2015 and 2019 that included more than one patient, carer or representative as participants (hereafter referred to as patients for brevity) in scoring outcomes in a Delphi.

RESULTS

There were variations in the design and conduct of COS studies that included patients in the Delphi process, including differing: scoring and feedback systems, approaches to recruiting patients, length of time between rounds, use of reminders, incentives, patient and public involvement, and piloting. Minimal reporting of participant characteristics and a lack of translation of Delphi surveys into local languages were found. Additionally, there were indications that studies that recruited patients through treatment centres had higher round two response rates than studies recruiting through patient organisations.

CONCLUSIONS

Variability was striking in how COS Delphi surveys were designed and conducted to include patient participants and other stakeholders. Future research is needed to explore what motivates patients to take part in COS studies and what factors influence COS developer recruitment strategies. Improved reporting would increase knowledge of how methods affect patient participation in COS Delphi studies.

Protocol for developing core outcome sets for evaluation of psychosocial interventions for children and families with experience or at risk of child maltreatment or domestic abuse

INTRODUCTION

Recognition that child maltreatment (CM) and domestic violence and abuse (DVA) are common and have serious and long-term adverse health consequences has resulted in policies and programmes to ensure that services respond to and safeguard children and their families. However, high-quality evidence about how services can effectively intervene is scant. The value of the current evidence base is limited partly because of the variety of outcomes and measures used in evaluative studies. One way of addressing this limitation is to develop a core outcome set (COS) which is measured and reported as a minimum standard in the context of trials and other types of evaluative research. The study described in this protocol aims to develop two discrete COSs for use in future evaluation of psychosocial interventions aimed at improving outcomes for children and families at risk or with experience of (1) CM or (2) DVA.

METHODS AND ANALYSIS

A two-phase mixed methods design: (1) rapid reviews of evidence, stakeholder workshops and semistructured interviews with adult survivors of CM/DVA and parents of children who have experienced CM/DVA and (2) a three panel adapted E-Delphi Study and consensus meeting. This study protocol adheres to reporting guidance for COS protocols and has been registered on the Core Outcome Measures for Effectiveness Trials (COMET) database.

ETHICS AND DISSEMINATION

We will disseminate our findings through peer-reviewed and open access publications, the COMET website and presentations at international conferences. We will engage with research networks, journal editors and funding agencies to promote awareness of the CM-COS and DVA-COS. We will work with advisory and survivor and public involvement groups to coproduce a range of survivor, policy and practice facing outputs.Approval for this study has been granted by the Research Ethics Committee at University College London.

Development of the CORE-Kids core set of outcome domains for studies of childhood limb fractures

Aims

The aim of this study is to develop a core set of outcome domains that should be considered and reported in all future trials of childhood limb fractures.

Methods

A four-phase study was conducted to agree a set of core outcome domains. Identification of candidate outcome domains were identified through systematic review of trials, and outcome domains relevant to families were identified through semi-structured interviews with 20 families (parent-child pairing or group). Outcome domains were prioritized using an international three-round Delphi survey with 205 panellists and then condensed into a core outcome set through a consensus workshop with 30 stakeholders.

Results

The systematic review and interviews identified 85 outcome domains as relevant to professionals or families. The Delphi survey prioritized 30 upper and 29 lower limb outcome domains at first round, an additional 17 upper and 18 lower limb outcomes at second round, and four additional outcomes for upper and lower limb at the third round as important domains. At the consensus workshop, the core outcome domains were agreed as: 1) pain and discomfort; 2) return to physical and recreational activities; 3) emotional and psychosocial wellbeing; 4) complications from the injury and treatment; 5) rturn to baseline activities daily living; 6) participation in learning; 7) appearance and deformity; and 8) time to union. In addition, 9a) recovery of mobility and 9b) recovery of manual dexterity was recommended as a core outcome for lower and upper limb fractures, respectively.

Conclusion

This set of core outcome domains is recommended as a minimum set of outcomes to be reported in all trials. It is not an exhaustive set and further work is required to identify what outcome tools should be used to measure each of these outcomes. Adoption of this outcome set will improve the consistency of research for these children that can be combined for more meaningful meta-analyses and policy development.

Cite this article: Bone Joint J 2021;103-B(12):1821–1830.

More than half of systematic reviews have relevant core outcome sets

OBJECTIVES

Using recent systematic reviews (SRs), our objectives were to: (1) develop a framework to assess whether a given COS is relevant to the scope of a SR; (2) examine the proportion of SRs for which relevant COS exist; and (3) for SRs for which COS exist, examine the extent to which outcomes in the COS and outcomes in the SR match.

STUDY DESIGN AND SETTING

We included a sample of SRs published by the Agency for Healthcare Research and Quality Evidence-based Practice Center Program between January 1, 2018 and October 12, 2020. We searched for potentially relevant COS from the Core Outcome Measures for Effectiveness Trials (COMET) database. We assessed the matching between outcomes recommended by COS and those included in corresponding SRs. When outcomes were matched, we considered matches to be specific (i.e., exact) or general (i.e., non-specific).

RESULTS

Sixty-seven SRs met criteria. We found relevant COS for 36 of 67 SRs (54%). Our framework for comparing the scope of a SR and a COS describes 16 scenarios arising when the breadth of the populations and the interventions are considered. The framework guides systematic reviewers to determine whether a COS is very likely to be relevant, may be relevant, or unlikely to be relevant. Sixty-two percent of outcomes in COS (interquartile range, 40% - 80%) were either specific or general matches to outcomes in SRs.

CONCLUSION

We found a COS with relevant scope for more than half of the SRs in our sample, with almost two-thirds of the recommended core outcomes matched to outcomes chosen for the SRs. Consideration of COS appears relevant for SR planning and our framework for assessing relevance of a given COS may help with this process.

Methods for development of a core outcome set for clinical trials integrating traditional Chinese medicine and Western medicine

Clinical trial outcome reporting differs between studies integrating traditional Chinese medicine (TCM) and Western medicine, so that some clinical trials are not eligible for inclusion in a systematic review. The excluded studies are therefore less widely disseminated, and even valid studies are less likely to yield impact. This problem may be addressed by developing core outcome sets (COSs) for integrative medicine in specific healthcare areas. The first stage of development is to define the scope of the COS for integrative medicine, the second stage is to establish the need for such a COS, and the third stage is to develop a protocol and register the COS. The final stage involves three steps: (i) development of a comprehensive list of outcomes (including efficacy outcomes and safety outcomes and TCM syndromes) using systematic review, qualitative or cross-sectional research, and reviews of package inserts and medical records; (ii) merging and grouping of outcomes within domains; (iii) conducting two rounds of Delphi survey and consensus meetings with a range of stakeholders. The final COS will include a general COS and core TCM syndrome- set. Development of COSs for clinical trials of integrative medicine may help to standardize outcome reporting and reduce publication bias in the future.

A core outcome set for the treatment of pregnant women with pregestational diabetes: an international consensus study

Objective

To develop a core outcome set (COS) for randomised controlled trials (RCTs) evaluating the effectiveness of interventions for the treatment of pregnant women with pregestational diabetes mellitus (PGDM).

Design

A consensus developmental study.

Setting

International.

Population

Two hundred and five stakeholders completed the first round.

Methods

The study consisted of three components. (1) A systematic review of the literature to produce a list of outcomes reported in RCTs assessing the effectiveness of interventions for the treatment of pregnant women with PGDM. (2) A three-round, online eDelphi survey to prioritise these outcomes by international stakeholders (including healthcare professionals, researchers and women with PGDM). (3) A consensus meeting where stakeholders from each group decided on the final COS.

Main outcome measures

All outcomes were extracted from the literature.

Results

We extracted 131 unique outcomes from 67 records meeting the full inclusion criteria. Of the 205 stakeholders who completed the first round, 174/205 (85%) and 165/174 (95%) completed rounds 2 and 3, respectively. Participants at the subsequent consensus meeting chose 19 outcomes for inclusion into the COS: trimester-specific haemoglobin A1c, maternal weight gain during pregnancy, severe maternal hypoglycaemia, diabetic ketoacidosis, miscarriage, pregnancy-induced hypertension, pre-eclampsia, maternal death, birthweight, large for gestational age, small for gestational age, gestational age at birth, preterm birth, mode of birth, shoulder dystocia, neonatal hypoglycaemia, congenital malformations, stillbirth and neonatal death.

Conclusions

This COS will enable better comparison between RCTs to produce robust evidence synthesis, improve trial reporting and optimise research efficiency in studies assessing treatment of pregnant women with PGDM.

165 key stakeholders have developed #Treatment #CoreOutcomes in pregnant women with #diabetes existing before pregnancy.

Core outcome set for surgical trials in gastric cancer (GASTROS study): international patient and healthcare professional consensus

BACKGROUND

Surgery is the primary treatment that can offer potential cure for gastric cancer, but is associated with significant risks. Identifying optimal surgical approaches should be based on comparing outcomes from well designed trials. Currently, trials report different outcomes, making synthesis of evidence difficult. To address this, the aim of this study was to develop a core outcome set (COS)-a standardized group of outcomes important to key international stakeholders-that should be reported by future trials in this field.

METHODS

Stage 1 of the study involved identifying potentially important outcomes from previous trials and a series of patient interviews. Stage 2 involved patients and healthcare professionals prioritizing outcomes using a multilanguage international Delphi survey that informed an international consensus meeting at which the COS was finalized.

RESULTS

Some 498 outcomes were identified from previously reported trials and patient interviews, and rationalized into 56 items presented in the Delphi survey. A total of 952 patients, surgeons, and nurses enrolled in round 1 of the survey, and 662 (70 per cent) completed round 2. Following the consensus meeting, eight outcomes were included in the COS: disease-free survival, disease-specific survival, surgery-related death, recurrence, completeness of tumour removal, overall quality of life, nutritional effects, and 'serious' adverse events.

CONCLUSION

A COS for surgical trials in gastric cancer has been developed with international patients and healthcare professionals. This is a minimum set of outcomes that is recommended to be used in all future trials in this field to improve trial design and synthesis of evidence.

Identification and categorisation of relevant outcomes for symptomatic uncomplicated gallstone disease: in-depth analysis to inform the development of a core outcome set

BACKGROUND

Many completed trials of interventions for uncomplicated gallstone disease are not as helpful as they could be due to lack of standardisation across studies, outcome definition, collection and reporting. This heterogeneity of outcomes across studies hampers useful synthesis of primary studies and ultimately negatively impacts on decision making by all stakeholders. Core outcome sets offer a potential solution to this problem of heterogeneity and concerns over whether the 'right' outcomes are being measured. One of the first steps in core outcome set generation is to identify the range of outcomes reported (in the literature or by patients directly) that are considered important.

OBJECTIVES

To develop a systematic map that examines the variation in outcome reporting of interventions for uncomplicated symptomatic gallstone disease, and to identify other outcomes of importance to patients with gallstones not previously measured or reported in interventional studies.

RESULTS

The literature search identified 794 potentially relevant titles and abstracts of which 137 were deemed eligible for inclusion. A total of 129 randomised controlled trials, 4 gallstone disease specific patient-reported outcome measures (PROMs) and 8 qualitative studies were included. This was supplemented with data from 6 individual interviews, 1 focus group (n=5 participants) and analysis of 20 consultations. A total of 386 individual recorded outcomes were identified across the combined evidence: 330 outcomes (which were reported 1147 times) from trials evaluating interventions, 22 outcomes from PROMs, 17 outcomes from existing qualitative studies and 17 outcomes from primary qualitative research. Areas of overlap between the evidence sources existed but also the primary research contributed new, unreported in this context, outcomes.

CONCLUSIONS

This study took a rigorous approach to catalogue and map the outcomes of importance in gallstone disease to enhance the development of the COS 'long' list. A COS for uncomplicated gallstone disease that considers the views of all relevant stakeholders is needed.

Core Outcome Sets Relevant to Maternity Service Users: A Scoping Review

INTRODUCTION

Variation in outcomes reported in maternity-related clinical trials and practice stifles data synthesis and contributes to ineffective or harmful treatments and interventions. Variation can be addressed using core outcome sets (COSs), minimum agreed sets of outcomes that should be measured and reported in all clinical trials in a specific area of health or health care. This scoping review identified studies that developed maternity-related COSs; evaluated the extent, scope, quality, and consistency of outcomes across similar COSs; and identified current gaps in evidence.

METHODS

A multifaceted search of 2 COS registers (Core Outcome Measures in Effectiveness Trials, Core Outcomes in Women's and Newborn Health), the International Consortium for Health Outcomes Measurement website, electronic databases (MEDLINE, Embase, CINAHL), and hand search was conducted. Published, English-language studies describing maternal and neonatal COSs for any health condition and published from inception to January 2020 were included. COS development process was evaluated against recently published COS Standards for Development: scope, stakeholder involvement, and consensus process.

RESULTS

Thirty-two articles relating to 26 COSs were included (maternal: 18 articles that addressed 17 COSs; neonatal: 14 articles that addressed 9 COSs) and covered a range of obstetric and neonatal conditions. COSs were published between 2006 and 2020, 58% since 2017. Maternal COSs included more outcomes (median, 17; range, 50) than neonatal COSs (median, 8; range, 20). Overlap in COSs was seen for maternity care and gestational diabetes. Overlap in outcomes was seen across similar COSs, which were mostly inconsistent or poorly defined. No included COS met all minimum standards for development. Two COSs extended recommendations for how and when to measure outcomes.

DISCUSSION

Growth in COS development in the last 3 years signifies increasing commitment to address variation and improve data synthesis. Although the quality of the development process has improved in the last 3 years, there is a need for improvement. This article presents an urgent need to minimize overlap in outcomes and standardize outcome measurement, case definitions, and timing of measurement between COSs.

Choosing important health outcomes for comparative effectiveness research: 6th annual update to a systematic review of core outcome sets for research

Background

An annual update to a systematic review of core outcome sets (COS) for research ensures that the COMET database is up-to-date. The aims of this study were to: (i) identify COS that were published or indexed in 2019 and to describe the methodological approaches used in these studies; (ii) investigate whether children have been included as participants in published COS development studies, and which methods have been used to facilitate their participation; iii) update a previous exercise to identify COS relevant to the most burdensome global diseases and injuries.

Methods

MEDLINE and SCOPUS were searched to identify studies published or indexed between (and inclusive of) January 2019 and December 2019. Automated screening methods were used to rank the citations in order of relevance; the top 25% in ranked priority order were screened for eligibility. COS were assessed against each of the Core Outcome Set-STAndards for Development (COS-STAD). A search of the COMET database was undertaken to identify COS relevant to the 25 leading causes of disease burden.

Results

Thirty-three studies, describing the development of 37 COS, were included in this update. These studies have been added to the COMET database, which now contains 370 published (1981–2019) COS studies for clinical research. Six (18%) of the 33 studies in this update were deemed to have met all of the minimum standards for COS development (range = 4 to 12 criteria, median = 9 criteria). Of the 370 COS studies published to date, 82 COS have been developed for paediatric health conditions and children would have been eligible to participate in 68/82 of these studies. Eleven of these 68 (16%) COS studies have included children as participants within the development process, most commonly through participation in Delphi surveys. Relevant COS were identified for 22/25 leading causes of global disease burden.

Conclusion

There has been a demonstrated increase in COS developed for both research and routine practice, and consistently high inclusion of patient participants. COS developed for paediatric conditions need to further incorporate the perspectives of children, alongside parents and other adults, and adopt research methods fit for this purpose. COS developers should consider the gaps identified in this update as priorities for COS development.

International consensus on a standard set of outcome measures for child and youth anxiety, depression, obsessive-compulsive disorder, and post-traumatic stress disorder

A major barrier to improving care effectiveness for mental health is a lack of consensus on outcomes measurement. The International Consortium for Health Outcomes Measurement (ICHOM) has already developed a consensus-based standard set of outcomes for anxiety and depression in adults (including the Patient Health Questionnaire-9, the Generalised Anxiety Disorder 7-item Scale, and the WHO Disability Schedule). This Position Paper reports on recommendations specifically for anxiety, depression, obsessive-compulsive disorder, and post-traumatic stress disorder in children and young people aged between 6 and 24 years. An international ICHOM working group of 27 clinical, research, and lived experience experts formed a consensus through teleconferences, an exercise using an adapted Delphi technique (a method for reaching group consensus), and iterative anonymous voting, supported by sequential research inputs. A systematic scoping review identified 70 possible outcomes and 107 relevant measurement instruments. Measures were appraised for their feasibility in routine practice (ie, brevity, free availability, validation in children and young people, and language translation) and psychometric performance (ie, validity, reliability, and sensitivity to change). The final standard set recommends tracking symptoms, suicidal thoughts and behaviour, and functioning as a minimum through seven primarily patient-reported outcome measures: the Revised Children's Anxiety and Depression Scale, the Obsessive Compulsive Inventory for Children, the Children's Revised Impact of Events Scale, the Columbia Suicide Severity Rating Scale, the KIDSCREEN-10, the Children's Global Assessment Scale, and the Child Anxiety Life Interference Scale. The set's recommendations were validated through a feedback survey involving 487 participants across 45 countries. The set should be used alongside the anxiety and depression standard set for adults with clinicians selecting age-appropriate measures.

A systematic review finds Core Outcome Set uptake varies widely across different areas of health

OBJECTIVE

The aim of our review was to bring together studies that had assessed the uptake of core outcome sets (COS) to explore the level of uptake across different COS and areas of health.

STUDY DESIGN AND SETTING

We examined the citations of 337 COS reports to identify studies that had assessed the uptake of a particular COS in randomized controlled trials (RCTs) or systematic reviews (SRs).

RESULTS

We identified 24 studies that had assessed uptake in RCTs and two studies that had assessed uptake in SRs. The studies covered a total of 17/337 (5%) COS. Uptake rates reported for RCTs varied from 0% of RCTs (gout) to 82% RCTs (rheumatoid arthritis) measuring the full COS. Studies that assessed uptake of individual core outcomes showed a wide variation in uptake between the outcomes. Suggested barriers to uptake included lack of validated measures, lack of patient and other key stakeholder involvement in COS development, and lack of awareness of the COS.

CONCLUSIONS

Few studies have been undertaken to assess the uptake of COS in RCTs and SRs. Further studies are needed to assess whether COS have been implemented across a wider range of disease categories and to explore the barriers and facilitators to COS uptake.

Pain assessment in clinical trials: a narrative review

Pain is a symptom measured in many clinical trials. For pain as an outcome domain, trialists need to choose adequate outcome measure(s), as there are myriad outcome measures for pain to choose from. To ensure consistency and uniformity in clinical trials and systematic reviews, core outcome sets (COS) have been defined; COS includes a predefined minimal list of core outcomes that should be measured within a trial, to ensure their consistency and comparability. COS is defined via consensus procedure, which includes relevant stakeholders such as experts from a specific field and patients. Along with outcomes, outcome measures for each outcome need to be defined to make sure that the outcomes will be measured consistently and uniformly. Hereby we reviewed studies that have examined use of recommended core outcome domains and outcome measures in clinical trials that would be expected to measure pain. Despite the existence of COS and defined core outcome measures (COMs), multiple studies have shown that these are not necessarily used in clinical trials, or in the relevant systematic reviews, which further increases heterogeneity of existing evidence, hinders evidence synthesis and trial comparability, and assessment of comparative effectiveness of interventions. Trialists are encouraged to use COS and COMs when designing clinical trials. Research community is encouraged to design interventions that will help with identifying barriers for using COS and COMs and interventions to foster their uptake. Use of consistent pain outcomes and pain outcome measures is in the interest of patients, research community, healthcare workers and decision-makers. For clinical conditions for which there are no COS and COMs, efforts to design them would be beneficial.

Systematic review of international Delphi surveys for core outcome set development: representation of international patients

OBJECTIVES

A core outcome set (COS) describes a minimum set of outcomes to be reported by all clinical trials of one healthcare condition. Delphi surveys are frequently used to achieve consensus on core outcomes. International input is important to achieve global COS uptake. We aimed to investigate participant representation in international Delphi surveys, with reference to the inclusion of patients and participants from low and middle income countries as stakeholders (LMICs).

DESIGN

Systematic review.

DATA SOURCES

EMBASE, Medline, Web of Science, COMET database and hand-searching.

ELIGIBILITY CRITERIA

Protocols and studies describing Delphi surveys used to develop an international COS for trial reporting, published between 1 January 2017 and 6 June 2019.

DATA EXTRACTION AND SYNTHESIS

Delphi participants were grouped as patients or healthcare professionals (HCPs). Participants were considered international if their country of origin was different to that of the first or senior author. Data extraction included participant numbers, country of origin, country income group and whether Delphi surveys were translated. We analysed the impact these factors had on outcome prioritisation.

RESULTS

Of 90 included studies, 69% (n=62) were completed and 31% (n=28) were protocols. Studies recruited more HCPs than patients (median 60 (IQR 30-113) vs 30 (IQR 14-66) participants, respectively). A higher percentage of HCPs was international compared with patients (57% (IQR 37-78) vs 20% (IQR 0-68)). Only 31% (n=28) studies recruited participants from LMICs. Regarding recruitment from LMICs, patients were under-represented (16% studies; n=8) compared with HCPs (22%; n=28). Few (7%; n=6) studies translated Delphi surveys. Only 3% studies (n=3) analysed Delphi responses by geographical location; all found differences in outcome prioritisation.

CONCLUSIONS

There is a disproportionately lower inclusion of international patients, compared with HCPs, in COS-development Delphi surveys, particularly within LMICs. Future international Delphi surveys should consider exploring for geographical and income-based differences in outcome prioritisation.

PROSPERO REGISTRATION NUMBER

CRD42019138519.

Assessing the relevance and uptake of core outcome sets (an agreed minimum collection of outcomes to measure in research studies) in Cochrane systematic reviews: a review

OBJECTIVES

A core outcome set (COS) is an agreed standardised minimum collection of outcomes that should be measured and reported in research in a specific area of health. Cochrane systematic reviews ('reviews') are rigorous reviews on health-related topics conducted under the auspices of Cochrane. This study examines the use of existing COS to inform the choice of outcomes in Cochrane systematic reviews ('reviews') and investigates the views of the coordinating editors of Cochrane Review Groups (CRGs) on this topic.

METHODS

A cohort of 100 recently published or updated Cochrane reviews were assessed for reference to a COS being used to inform the choice of outcomes for the review. Existing COS, published 2 or more years before the review publication, were then identified to assess how often a reviewer could have used a relevant COS if it was available. We asked 52 CRG coordinating editors about their involvement in COS development, how outcomes are selected for reviews in their CRG and their views of the advantages and challenges surrounding the standardisation of outcomes within their CRG.

RESULTS

In the cohort of reviews from 2019, 40% (40/100) of reviewers noted problems due to outcome inconsistency across the included studies. In 7% (7/100) of reviews, a COS was referenced in relation to the choice of outcomes for the review. Relevant existing COS could be considered for a review update in 35% of the others (33/93). Most editors who responded (31/36, 86%) thought that COS should definitely or possibly be used to inform the choice of outcomes in a review.

CONCLUSIONS

Systematic reviewers are continuing to note outcome heterogeneity but are starting to use COS to inform their reviews. There is potential for greater uptake of COS in Cochrane reviews.

Effects of Workplace-Based Physical Activity Interventions on Cardiorespiratory Fitness: A Systematic Review and Meta-Analysis of Controlled Trials

BACKGROUND

Cardiorespiratory fitness is a strong predictor of all-cause mortality. Physical activity of at least moderate intensity can improve cardiorespiratory fitness. Workplaces may provide a relatively controlled setting in which to improve cardiorespiratory fitness through physical activity. Limited work has been conducted to quantify the impact of delivering physical activity in the workplace on cardiorespiratory fitness.

OBJECTIVE

The objective of this systematic review was to quantify the effects of workplace physical activity interventions on peak oxygen consumption (VO_2peak) and explore study and participant characteristics as putative moderators.

METHODS

Seven databases were searched up to September 2018. Search terms included "workplace", "physical activity" and "intervention". Inclusion criteria were controlled trials where physical activity of at least moderate intensity was delivered in the workplace and compared to controls or non-active comparators; and cardiorespiratory fitness measured by actual or predicted VO_2peak. Risk of bias was assessed using the PEDro scale. A random-effects meta-analysis was conducted with between-study variation quantified and then explored for putative predictors with a meta-regression. Pooled estimate uncertainty was expressed as 90% confidence intervals (CIs) and assessed against our threshold value for clinical relevance of 1 mL·kg^-1·min^-1.

RESULTS

The final dataset consisted of 25 estimates of VO_2peak from 12 trials. The pooled mean differences between intervention and control arms was a beneficial improvement of 2.7 mL·kg^-1·min^-1 (90% CI 1.6-3.8). The 95% prediction interval ranged from a reduction in VO_2peak of - 1.1 to an improvement of 6.5 mL·kg^-1·min^-1. Between-study heterogeneity (τau) was ± 1.6 mL·kg^-1·min^-1. The meta-regression showed longer interventions (3.2 mL·kg^-1·min^-1; 90% CI 1.6-3.8) to have an additive effect and studies with a low risk of bias (- 2.5 mL·kg^-1·min^-1; 90% CI - 4.0 to - 1.0), and participants of greater baseline VO_2peak (- 1.6 mL·kg^-1·min^-1; 90% CI - 3.6 to 0.4), and age (- 1.4 mL·kg^-1·min^-1; 90% CI - 3.2 to 0.3) having a lesser effect. Participant sex (percentage female) had an additive effect on VO_2peak (0.4 mL·kg^-1·min^-1; 90% CI - 1.6 to 2.4).

CONCLUSIONS

Workplace-based physical activity interventions consisting of at least moderate-intensity activity improve cardiorespiratory fitness. At the present time, we surmise that no single group of employees (e.g. older employees or less fit individuals) can be definitively identified as standing to benefit more from workplace physical activity interventions than others. This demonstrates the potential utility of workplace physical activity interventions for improving cardiorespiratory fitness in a broad range of healthy employees. Protocol registration: PROSPERO (registration number: 42017057498).

Pediatric perioperative outcomes: Protocol for a systematic literature review and identification of a core outcome set for infants, children, and young people requiring anesthesia and surgery

Clinical outcomes are measurable changes in health, function, or quality of life that are important for evaluating the quality of care and comparing the efficacy of interventions. However, clinical outcomes and related measurement tools need to be well-defined, relevant, and valid. In adults, Core Outcome Measures in Effectiveness Trials (COMET) methodology has been used to develop core outcome sets for perioperative care. Systematic literature reviews identified standardized endpoints (StEP) and valid measurement tools, and consensus across a broader range of relevant stakeholders was achieved via a Delphi process to establish Core Outcome Measures in Perioperative and Anaesthetic Care (COMPAC). Core outcome sets for pediatric perioperative care cannot be directly extrapolated from adult data. The type and weighting of endpoints within particular domains can be influenced by age-dependent differences in the indications for and/or nature of surgery and medical comorbidities, and the validity and utility of many measurement tools vary significantly with developmental stage and age. The involvement of parents/carers is essential as they frequently act as surrogate responders for preverbal and developmentally delayed children, parental response may influence child outcome, and parental and/or child ranking of outcomes may differ from those of health professionals. Here, we describe the formation of the international Pediatric Perioperative Outcomes Group, which aims to identify and create validated, broadly applicable, patient-centered outcome measures for infants, children, and young people. Methodologies parallel that of the StEP and COMPAC projects, and systematic literature searches have been performed within agreed age-dependent subpopulations to identify reported outcomes and measurement tools. This represents the first steps for developing core outcome sets for pediatric perioperative care.

Evaluation of an automatic article selection method for timelier updates of the Comet Core Outcome Set database

Curated databases of scientific literature play an important role in helping researchers find relevant literature, but populating such databases is a labour intensive and time-consuming process. One such database is the freely accessible Comet Core Outcome Set database, which was originally populated using manual screening in an annually updated systematic review. In order to reduce the workload and facilitate more timely updates we are evaluating machine learning methods to reduce the number of references needed to screen. In this study we have evaluated a machine learning approach based on logistic regression to automatically rank the candidate articles. Data from the original systematic review and its four first review updates were used to train the model and evaluate performance. We estimated that using automatic screening would yield a workload reduction of at least 75% while keeping the number of missed references around 2%. We judged this to be an acceptable trade-off for this systematic review, and the method is now being used for the next round of the Comet database update.

Choosing important health outcomes for comparative effectiveness research: 5th annual update to a systematic review of core outcome sets for research

BACKGROUND

A systematic review of core outcome sets (COS) for research is updated annually to populate an online database. It is a resource intensive review to do annually but automation techniques have potential to aid the process. The production of guidance and standards in COS development means that there is now an expectation that COS are being developed and reported to a higher standard. This is the fifth update to the systematic review and will explore these issues.

METHODS

Searches were carried out to identify studies published or indexed in 2018. Automated screening methods were used to rank the citations in order of relevance. The cut-off for screening was set to the top 25% in ranked priority order, following development and validation of the algorithm. Studies were eligible for inclusion if they reported the development of a COS, regardless of any restrictions by age, health condition or setting. COS were assessed against each of the Core Outcome Set-STAndards for Development (COS-STAD).

RESULTS

Thirty studies describing the development of 44 COS were included in this update. Six COS (20%) were deemed to have met all 12 criteria representing the 11 minimum standards for COS development (range = 4 to 12 criteria, median = 10 criteria). All 30 COS studies met all four minimum standards for scope. Twenty-one (70%) COS met all three minimum standards for stakeholders. Twenty-three studies (77%) included patients with the condition or their representatives. The number of countries involved in the development of COS ranged from 1 to 39 (median = 10). Six studies (20%) met all four minimum standards [five criteria] for the consensus process.

CONCLUSION

Automated ranking was successfully used to assist the screening process and reduce the workload of this systematic review update. With the provision of guidelines, COS are better reported and being developed to a higher standard.

Assessing the impact of a research funder's recommendation to consider core outcome sets

Background

Core outcome sets (COS) have the potential to reduce waste in research by improving the consistency of outcomes measured in trials of the same health condition. However, this reduction in waste will only be realised through the uptake of COS by clinical trialists. Without uptake, the continued development of COS that are not implemented may add to waste in research. Funders of clinical trials have the potential to have an impact on COS uptake by recommending their use to those applying for funding. The aim of our study was to assess the extent to which applicants followed the National Institute for Health Research Health Technology Assessment (NIHR HTA) programme’s recommendation to search for a COS to include in their clinical trial.

Methods and findings

We examined the outcomes section and detailed project descriptions of all 95 researcher-led primary research applications submitted to the NIHR HTA between January 2012, when the recommendation to search for a COS was included in the guidance for applicants, and December 2015 for evidence that a search for a COS had taken place and rationale for outcome choice in the absence of COS. A survey of applicants was conducted to further explore their use of COS and choice of outcomes with a response rate of 49%. Nine out of 95 applicants (10%) stated in their application that they had searched the COMET (Core Outcome Measures for Effectiveness Trials) Initiative database for a COS and another nine referred to searching for a COS using another method, e.g. a review of the literature. Of the 77 (81%) applicants that did not mention COMET or COS in their application, eight stated in the survey that they had searched the COMET database and ten carried out a search using another method. Some applicants who did not search for a COS gave reasons for their choice of outcomes including taking advice from patients and the public and choosing outcomes used in previous trials.

Conclusion

A funding body can have an impact on COS uptake by encouraging trialists to search for a COS. Funders could take further steps by putting processes in place to prompt applicants to be explicit about searching for COS in their application and notifying the funding board if a search has not taken place. The sources of information used by trialists to make decisions about outcomes in the absence of COS may suggest methods of dissemination for COS.

Critical care outcomes in resource-limited settings

PURPOSE OF REVIEW

The burden of critical illness in low-income and middle-income countries (LMICs) is substantial. A better understanding of critical care outcomes is essential for improving critical care delivery in resource-limited settings. In this review, we provide an overview of recent literature reporting on critical care outcomes in LMICs. We discuss several barriers and potential solutions for a better understanding of critical care outcomes in LMICs.

RECENT FINDINGS

Epidemiologic studies show higher in-hospital mortality rates for critically ill patients in LMICs as compared with patients in high-income countries (HICs). Recent findings suggest that critical care interventions that are effective in HICs may not be effective and may even be harmful in LMICs. Little data on long-term and morbidity outcomes exist. Better outcomes measurement is beginning to emerge in LMICs through decision support tools that report process outcome measures, studies employing mobile health technologies with community health workers and the development of context-specific severity of illness scores.

SUMMARY

Outcomes from HICs cannot be reliably extrapolated to LMICs, so it is important to study outcomes for critically ill patients in LMICs. Specific challenges to achieving meaningful outcomes studies in LMICs include defining the critically ill population when few ICU beds exist, the resource-intensiveness of long-term follow-up, and the need for reliable severity of illness scores to interpret outcomes. Although much work remains to be done, examples of studies overcoming these challenges are beginning to emerge.

Outcomes for Pressure Ulcer Trials (OUTPUTs): protocol for the development of a core domain set for trials evaluating the clinical efficacy or effectiveness of pressure ulcer prevention interventions

Background

Core outcome sets (COS) are being developed in many clinical areas to increase the quality and comparability of clinical trial results as well as to ensure their relevance for patients. A COS represents an agreed standardized set of outcomes that describes the minimum that should be consistently reported in all clinical trials of a defined area. It comprises a core domain set (defining what core outcomes should be measured) and a core measurement set (defining measurement/assessment instruments for each core domain). For pressure ulcer prevention trials a COS is lacking. The great heterogeneity of reported outcomes in this field indicates the need for a COS.

Methods/design

The first part of this project aims to develop a core domain set by following established methods, which incorporates four steps: (1) definition of the scope, (2) conducting a scoping review, (3) organizing facilitated workshops with service users, (4) performing Delphi surveys and establishing consensus in a face-to-face meeting with different stakeholders.

Discussion

After achieving consensus on the core domain set, further work will be undertaken to determine a corresponding core measurement set. This will lead to better pressure ulcer prevention research in the future. There are a number of methodological challenges in the field of COS development. To meet these challenges and to ensure a high-quality COS, the OUTPUTS project affiliates to current standards and works in close collaboration with international experts and with existing international service user groups.

Trial registration

The OUTPUTs project is registered in the COMET database: (http://www.comet-initiative.org/studies/details/283). Registered on 2015.

Improvement was needed in the standards of development for cancer core outcome sets

OBJECTIVE

The Core Outcome Set-STAndards for Development (COS-STAD) contains 11 standards (12 criteria) that are deemed to be the minimum design recommendations for all core outcome set (COS) development projects. Cancer is currently the disease area with the highest number of published COSs and is a major cause of worldwide morbidity and mortality. The aim of this study was to provide a baseline of cancer COS standards.

STUDY DESIGN AND SETTING

Systematic reviews of COSs have identified 307 published COS studies. Cancer COSs were eligible for inclusion. Two reviewers independently assessed each of the COSs against the 12 criteria.

RESULTS

Forty-nine cancer COSs were included; none met all 12 criteria representing the 11 minimum standards assessed in this study (range = 4-11 criteria, median = 6 criteria). All studies met the four scope standards, eight (16%) met all three standards for stakeholders involved, and two (4%) met all four standards for consensus process standards.

CONCLUSION

With the exception of "scope" specification, there is much need for improvement. Poor reporting often made it challenging to assess whether minimum standards were met. The consensus process criteria were most difficult to assess, particularly those that required an assessment of being a priori. This is the first application of COS-STAD criteria to studies that have developed COSs and provides a baseline of cancer COS standards of development.

Navigating the landscape of core outcome set development in dermatology

The development of core outcome sets (COSs; ie, a minimum set of core outcomes that should be measured and reported in all trials or in clinical practice for a specific condition) in dermatology is increasing in pace. A total of 44 dermatology-related COS projects have been registered in the online Core Outcome Measures in Effectiveness Trials database (http://www.comet-initiative.org/studies/search) and include studies on 26 different skin diseases. With the increasing number of COSs in dermatology, care is needed to ensure the delivery of high-quality COSs that meet quality standards when using state-of-the-art methods. In 2015, the Cochrane Skin-Core Outcome Set Initiative (CS-COUSIN) was established. CS-COUSIN is an international, multidisciplinary working group aiming to improve the development and implementation of COSs in dermatology. CS-COUSIN has developed guidance on how to develop high-quality COSs for skin diseases and supports dermatology-specific COS initiatives. Currently, 17 COS development groups are affiliated with CS-COUSIN and following standardized COS development processes. To ensure successful uptake of COSs in dermatology, researchers, clinicians, systematic reviewers, guideline developers, and other stakeholders should use existing COSs in their work.

Core Outcome Set-STAndardised Protocol Items: the COS-STAP Statement

BACKGROUND

Several hundred core outcome set (COS) projects have been systematically identified to date which, if adopted, ensure that researchers measure and report those outcomes that are most likely to be relevant to users of their research. The uptake of a COS by COS users will depend in part on the transparency and robustness of the methods used in the COS development study, which would be increased by the use of a standardised protocol. This article describes the development of the COS-STAP (Core Outcome Set-STAndardised Protocol Items) Statement for the content of a COS development study protocol.

METHODS

The COS-STAP Statement was developed following the Enhancing the Quality and Transparency Of Health Research (EQUATOR) Network's methodological framework for guideline development. This included an initial item generation stage, a two-round Delphi survey involving more than 150 participants representing three stakeholder groups (COS developers, journal editors and patient and public involvement researchers interested in COS development), followed by a consensus meeting with eight voting participants.

RESULTS

The COS-STAP Statement consists of a checklist of 13 items considered essential documentation in a protocol, outlining the scope of the COS, stakeholder involvement, COS development plans and consensus processes.

CONCLUSIONS

Journal editors and peer reviewers can use the guidance to assess the completeness of a COS development study protocol submitted for publication. By providing guidance for key content, the COS-STAP Statement will enhance the drafting of high-quality protocols and determine how the COS development study will be carried out.

Incorporating patients' perspectives into the initial stages of core outcome set development: a rapid review of qualitative studies of type 2 diabetes

Conducting systematic reviews of qualitative studies to incorporate patient perspectives within the early stages of core outcome set (COS) development can be resource intensive. We aimed to identify an expedited approach to be used as part of the wider COS development process. Specifically, we undertook a rapid review of qualitative studies of patients' views and experiences of type 2 diabetes. We searched MEDLINE from inception to June 2017 to identify studies reporting qualitative empirical findings of perspectives of people with type 2 diabetes. Qualitative methodological filters were used to minimize irrelevant references. Drawing on content analysis, data synthesis involved identifying text in eligible studies relevant to outcomes of type 2 diabetes and interpreting and categorizing this according to the 38 core domains of the Core Outcome Measures in Effectiveness Trials taxonomy. Of 146 studies screened, 26 were included. Four hundred and fifty-eight outcomes were derived from the included studies. In comparison to the outcomes extracted from clinical trials, more life impact outcomes were derived from the qualitative studies, but fewer physiological/clinical outcomes. Outcomes relating to 'mortality/survival' and 'role functioning' were more prevalent in studies conducted in low/middle-income countries. This rapid review and synthesis of qualitative studies identified outcomes that had not previously been identified by a systematic review of clinical trials. It also identified differences in the types of outcomes given prominence to in the clinical trials and qualitative literatures. Incorporating qualitative evidence on patient perspectives from the outset of the COS development process can help to ensure outcomes that matter to patients are not overlooked. Our method provides a pragmatic and resource-efficient way to do this. For those developing international COS, our method has potential for incorporating the perspectives of patients from diverse countries in the early stages of COS development.

Choosing important health outcomes for comparative effectiveness research: 4th annual update to a systematic review of core outcome sets for research

Background

The Core Outcome Measures in Effectiveness Trials (COMET) database is a publically available, searchable repository of published and ongoing core outcome set (COS) studies. An annual systematic review update is carried out to maintain the currency of database content.

Methods

The methods used in the fourth update of the systematic review followed the same approach used in the original review and previous updates. Studies were eligible for inclusion if they reported the development of a COS, regardless of any restrictions by age, health condition or setting. Searches were carried out in March 2018 to identify studies that had been published or indexed between January 2017 and the end of December 2017.

Results

Forty-eight new studies, describing the development of 56 COS, were included. There has been an increase in the number of studies clearly specifying the scope of the COS in terms of the population (n = 43, 90%) and intervention (n = 48, 100%) characteristics. Public participation has continued to rise with over half (n = 27, 56%) of studies in the current review including input from members of the public. The rate of inclusion of all stakeholder groups has increased, in particular participation from non-clinical research experts has risen from 32% (mean average in previous reviews) to 62% (n = 29). Input from participants located in Australasia (n = 17; 41%), Asia (n = 18; 44%), South America (n = 13; 32%) and Africa (n = 7; 17%) have all increased since the previous reviews.

Conclusion

This update included a pronounced increase in the number of new COS identified compared to the previous three updates. There was an improvement in the reporting of the scope, stakeholder participants and methods used. Furthermore, there has been an increase in participation from Australasia, Asia, South America and Africa. These advancements are reflective of the efforts made in recent years to raise awareness about the need for COS development and uptake, as well as developments in COS methodology.