Choosing important health outcomes for comparative effectiveness research: 4th annual update to a systematic review of core outcome sets for research

A review of core outcome sets (COS) developed for different settings finds there is a subset of outcomes relevant for both research and routine care

OBJECTIVES

To compare the outcomes selected for the same condition in core outcome sets (COSs) for research with those in COS for the routine care setting.

METHODS

A sample of COS was created from the most frequent five health areas within previous systematic reviews of COS for research and COS for routine care. Outcomes were extracted and categorized using an outcome taxonomy. Frequency of outcome domains included within COS were analyzed in subgroups according to research or care setting, patient involvement in COS development and health area. Matched sets of COS were created, where at least one research COS and one routine care COS exist for the same health condition, to identify the outcomes that were recommended for both settings. A similar process was used for a subset of paired COS matched in scope for both intervention and population as well as health condition.

RESULTS

The sample of COS comprised: 246 COS for research only, 76 COS for routine care only and 55 COS for both research and routine care. Across the 18 sets matched by health condition the median number (range) of outcomes included in both research COS and routine care COS was 6 (3-15), with differences noted across health areas. For the 11 paired COS matched by scope and health condition, the corresponding figures were 2 (2-8). Across all settings, COS that did not include patients as participants were less likely to include life impact outcomes.

CONCLUSION

Within a given health condition, a small number of core outcomes were found to be relevant for both research and care, offering a meaningful starting point for linking research and real-world evaluation.

Core Outcome Set for IgE-mediated food allergy clinical trials and observational studies of interventions: International Delphi consensus study 'COMFA'

BACKGROUND

IgE-mediated food allergy (FA) is a global health concern with substantial individual and societal implications. While diverse intervention strategies have been researched, inconsistencies in reported outcomes limit evaluations of FA treatments. To streamline evaluations and promote consistent reporting, the Core Outcome Measures for Food Allergy (COMFA) initiative aimed to establish a Core Outcome Set (COS) for FA clinical trials and observational studies of interventions.

METHODS

The project involved a review of published clinical trials, trial protocols and qualitative literature. Outcomes found as a result of review were categorized and classified, informing a two-round online-modified Delphi process followed by hybrid consensus meeting to finalize the COS.

RESULTS

The literature review, taxonomy mapping and iterative discussions with diverse COMFA group yielded an initial list of 39 outcomes. The iterative online and in-person meetings reduced the list to 13 outcomes for voting in the formal Delphi process. One more outcome was added based on participant suggestions after the first Delphi round. A total of 778 participants from 52 countries participated, with 442 participating in both Delphi rounds. No outcome met a priori criteria for inclusion, and one was excluded as a result of the Delphi. Thirteen outcomes were brought to the hybrid consensus meeting as a result of Delphi and two outcomes, 'allergic symptoms' and 'quality of life' achieved consensus for inclusion as 'core' outcomes.

CONCLUSION

In addition to the mandatory reporting of adverse events for FA clinical trials or observational studies of interventions, allergic symptoms and quality of life should be measured as core outcomes. Future work by COMFA will define how best to measure these core outcomes.

Healthcare professionals' views on the most important outcomes for non-infectious uveitis of the posterior segment: A qualitative study

BACKGROUND

Uveitis comprises a range of conditions that result in intraocular inflammation. Most sight-threatening uveitis falls into the broad category known as Non-infectious Posterior Segment-Involving Uveitis (PSIU). To evaluate treatments, trialists and clinicians must select outcome measures. The aim of this study was to understand healthcare professionals' perspectives on what outcomes are important to adult patients with PSIU and their carers.

METHODS

Twelve semi-structured telephone interviews were undertaken to understand the perspectives of healthcare professionals. Interviews were audio recorded, transcribed and thematically analysed. Findings were compared with the views of patients and carers and outcomes abstracted from a previously published systematic review.

RESULTS

Eleven core domains were identified as important to healthcare professionals: (1) visual function, (2) symptoms, (3) functional ability, (4) impact on relationships, (5) financial impact, (6) psychological morbidity and emotional well-being (7) psychosocial adjustment to uveitis, (8) doctor / patient / interprofessional relationships and access to health care, (9) treatment burden, (10) treatment side effects, (11) disease control. Healthcare professionals recognised a similar range of domains to patients and carers but placed more emphasis on certain outcomes, particularly in the disease control domain. In contrast the range of outcomes identified via the systematic review was limited.

CONCLUSION

Healthcare professionals recognise all of the published outcome domains as patients/carers in the previous publication but with subtly differing emphasis within some domains and with a priority for certain types of measures. Healthcare professionals discussed the disease control and side effects/complications to a greater degree than patients and carers in the focus groups.

Improving continence management for people with dementia in the community in Aotearoa, New Zealand: Protocol for a mixed methods study

BACKGROUND

The number of people living with dementia (PLWD) in Aotearoa New Zealand (NZ) was estimated at 96,713 in 2020 and it is anticipated that this number will increase to 167,483 by 2050, including an estimated 12,039 Māori (indigenous people of NZ) with dementia. Experiencing urinary incontinence (UI) or faecal incontinence (FI) is common for PLWD, particularly at the later stages of the disease. However, there is no robust estimate for either prevalence or incidence of UI or FI for PLWD in NZ. Although caregivers rate independent toilet use as the most important activity of daily living to be preserved, continence care for PLWD in the community is currently not systematised and there is no structured care pathway. The evidence to guide continence practice is limited, and more needs to be known about caregiving and promoting continence and managing incontinence for PLWD in the community. This project will seek to understand the extent of the challenge and current practices of health professionals, PLWD, caregivers and family; identify promising strategies; co-develop culturally appropriate guidelines and support materials to improve outcomes; and identify appropriate quality indicators so that good continence care can be measured in future interventions.

METHODS AND ANALYSIS

A four-phase mixed methods study will be delivered over three years: three phases will run concurrently, followed by a fourth transformative sequential phase. Phase 1 will identify the prevalence and incidence of incontinence for PLWD in the community using a cohort study from standardised home care interRAI assessments. Phase 2 will explore continence management for PLWD in the community through a review of clinical policies and guidance from publicly funded continence services, and qualitative focus group interviews with health professionals. Phase 3 will explore experiences, strategies, impact and consequences of promoting continence and managing incontinence for PLWD in the community through secondary data analysis of an existing carers' study, and collecting new cross-sectional and longitudinal qualitative data from Māori and non-Māori PLWD and their caregivers. In Phase 4, two adapted 3-stage Delphi processes will be used to co-produce clinical guidelines and a core outcome set, while a series of workshops will be used to co-produce caregiver resources.

A systematic review of studies reporting the development of core outcome sets for use in routine care

OBJECTIVES

Core outcome sets (COS) represent the minimum health outcomes to be measured for a given health condition. Interest is growing in using COS within routine care to support delivery of patient-focused care. This review aims to systematically map COS developed for routine care to understand their scope, stakeholder involvement, and development methods.

METHODS

Medline (Ovid), Scopus, and Web of Science Core collection were searched for studies reporting development of COS for routine care. Data on scope, methods, and stakeholder groups were analyzed in subgroups defined by setting.

RESULTS

Screening 25,301 records identified 262 COS: 164 for routine care only and 98 for routine care and research. Nearly half of the COS (112/254, 44%) were developed with patients, alongside input from experts in registries, insurance, legal, outcomes measurement, and performance management. Research publications were often searched to generate an initial list of outcomes (115/198, 58%) with few searching routine health records (47/198, 24%).

CONCLUSION

An increasing number of COS is being developed for routine care. Although involvement of patient stakeholders has increased in recent years, further improvements are needed. Methodology and scope are broadly similar to COS for research but implementation of the final set is a greater consideration during development.

Patient participation impacts outcome domain selection in core outcome sets for research: an updated systematic review

OBJECTIVES

The importance of including patients, carers, and the public in health research is well recognized, including the need to consider outcomes in health care research that reflect the priorities of patients. Core outcome sets (COS) define the minimum set of outcomes that should be measured and reported in research of a given condition, determined through consensus among key stakeholders. The Core Outcome Measures in Effectiveness Trials Initiative undertakes an annual systematic review (SR) to identify newly published COS to update its online database of COS for research. The objective of this study was to assess the impact of patient participation on COS.

STUDY DESIGN AND SETTING

SR methods used in previous updates were applied to identify research studies published or indexed in 2020 and 2021 (conducted as separate reviews) that report development of a COS, regardless of any specifications relating to condition, population, intervention, or setting. Studies were assessed according to published standards for COS development, and core outcomes extracted from study publications were categorized according to an outcome taxonomy and added to an existing database of core outcome classifications of all previously published COS. The effect of patient participation on core domains was examined.

RESULTS

Searches identified 56 new studies published in 2020 and 54 in 2021. All studies met all four minimum standards for scope, and 42 (75%) of the 2020 studies and 45 (83%) of the 2021 studies met all three standards for stakeholders involved. However, only 19 (34%) of the 2020 studies and 18 (33%) of the 2021 studies met all four standards for the consensus process. COS that involved patients or their representatives are more likely to include life impact outcomes (239, 86%) than COS without patient participation (193, 62%). Physiological/clinical outcomes are almost always specified at a granular level, whereas life impact outcomes are often described at a higher level.

CONCLUSION

This study adds to the body of evidence demonstrating the importance and impact of including patients, carers, and the public in COS development, in particular by demonstrating that the impact of interventions on patients' lives is more likely to be represented in COS that involve patients or their representatives. COS developers are encouraged to pay increased attention to methods and reporting relating to the consensus process. Further work is required to understand the appropriateness and rationale for the discrepancy in granularity levels between outcome domains.

How to reach agreement: the impact of different analytical approaches to Delphi process results in core outcomes set development

BACKGROUND

Core outcomes sets are increasingly used to define research outcomes that are most important for a condition. Different consensus methods are used in the development of core outcomes sets; the most common is the Delphi process. Delphi methodology is increasingly standardised for core outcomes set development, but uncertainties remain. We aimed to empirically test how the use of different summary statistics and consensus criteria impact Delphi process results.

METHODS

Results from two unrelated child health Delphi processes were analysed. Outcomes were ranked by mean, median, or rate of exceedance, and then pairwise comparisons were undertaken to analyse whether the rankings were similar. The correlation coefficient for each comparison was calculated, and Bland-Altman plots produced. Youden's index was used to assess how well the outcomes ranked highest by each summary statistic matched the final core outcomes sets. Consensus criteria identified in a review of published Delphi processes were applied to the results of the two child-health Delphi processes. The size of the consensus sets produced by different criteria was compared, and Youden's index was used to assess how well the outcomes that met different criteria matched the final core outcomes sets.

RESULTS

Pairwise comparisons of different summary statistics produced similar correlation coefficients. Bland-Altman plots showed that comparisons involving ranked medians had wider variation in the ranking. No difference in Youden's index for the summary statistics was found. Different consensus criteria produced widely different sets of consensus outcomes (range: 5-44 included outcomes). They also showed differing abilities to identify core outcomes (Youden's index range: 0.32-0.92). The choice of consensus criteria had a large impact on Delphi results.

DISCUSSION

The use of different summary statistics is unlikely to affect how outcomes are ranked during a Delphi process: mean, median, and rates of exceedance produce similar results. Different consensus criteria have a large impact on resultant consensus outcomes and potentially on subsequent core outcomes sets: our results confirm the importance of adhering to pre-specified consensus criteria.

Endpoints and Outcomes After Immunotherapy for Food Allergy: What Is Meaningful for Patients?

Multiple novel interventions for food allergy are currently at various stages of development with the goal of reducing or eliminating allergic reactions. However, the relative success of these therapeutics in achieving meaningful, long-term improvements to patients' lives is difficult to determine as there is currently very limited understanding of the degree of alignment between clinical trial efficacy endpoints and patient-centered outcomes. Furthermore, outcome measures used in clinical trials of food allergy immunotherapies vary widely, are often misinterpreted, and not necessarily consistent with what patients expect to achieve through treatment. This review aims to assist clinicians in critically interpreting outcomes reported in clinical trials and accurately communicating risks and outcomes to patients when practicing shared decision-making.

The research status and progress of core outcome set in oral health

The core outcome set (COS) refers to the minimum set of outcomes that should be reported by all clinical trials in a particular health field. The use of COS in clinical studies can reduce the heterogeneity caused by using different outcomes across different clinical studies, facilitate the systematic review of different clinical studies on the same topic, reduce selective reporting bias, and increase the utility of clinical studies. The importance of COS in oral health has recently been recognized. This review summarizes the history, necessity, and key methodological points of COS development, with emphasis on the research status and existing problems in COS development, in the field of oral health.

Comparison of published core outcome sets with outcomes recommended in regulatory guidance from the US Food and Drug Administration and European Medicines Agency: cross sectional analysis

Objective

To compare the outcomes in published core outcome sets with the outcomes recommended in corresponding guidance documents from the European Medicines Agency (EMA) and US Food and Drug Administration (FDA), matched by health condition.

Design

Cross sectional analysis.

Setting

US and Europe.

Population

Sample of core outcome sets related to drugs, devices, and gene therapy that involved patients in the consensus process, published between 1 January 2015 and 31 December 2019; and corresponding EMA and FDA guidance documents.

Main outcome measures

The extent of matches between outcomes included within core outcome sets and those recommended in corresponding EMA and FDA guidance documents were assessed. Matches were considered to be general (ie, non-specific) or specific (ie, exact). General matches were assessed to determine whether the core outcome set or guidance document outcome was narrower.

Results

Relevant guidance documents were found for for 38 (39%) of 98 eligible published core outcome sets. Among outcomes in core outcome sets, medians of 70% (interquartile range 48-86%) and 52% (33-77%) were matches with outcomes recommended in EMA and FDA documents, respectively. Medians of 46% (27-68%) and 26% (18-46%) were specific matches with outcomes in EMA and FDA documents, respectively. When outcomes were generally matched, the outcomes from core outcome sets were more frequently narrower than the regulatory outcomes (83% and 75% for EMA and FDA, respectively).

Conclusion

Greater adoption of, and reference to, core outcome sets in regulatory guidance documents can encourage clinical trialists, especially those in industry, to measure and report consistent and agreed outcomes and improve the quality of guidance. Given the overlap between outcomes in core outcome sets and regulatory guidance, and given that most core outcome sets now involve patients in the consensus process, these sets could serve as a useful resource for regulators when recommending outcomes for studies evaluating regulated products. Developers are encouraged to appraise recommended outcomes in salient regulatory documents when planning a core outcome set.

Establishment of a core outcome set for burn care research: development and international consensus

Objective

To develop a core outcome set for international burn research.

Design

Development and international consensus, from April 2017 to November 2019.

Methods

Candidate outcomes were identified from systematic reviews and stakeholder interviews. Through a Delphi survey, international clinicians, researchers, and UK patients prioritised outcomes. Anonymised feedback aimed to achieve consensus. Pre-defined criteria for retaining outcomes were agreed. A consensus meeting with voting was held to finalise the core outcome set.

Results

Data source examination identified 1021 unique outcomes grouped into 88 candidate outcomes. Stakeholders in round 1 of the survey, included 668 health professionals from 77 countries (18% from low or low middle income countries) and 126 UK patients or carers. After round 1, one outcome was discarded, and 13 new outcomes added. After round 2, 69 items were discarded, leaving 31 outcomes for the consensus meeting. Outcome merging and voting, in two rounds, with prespecified thresholds agreed seven core outcomes: death, specified complications, ability to do daily tasks, wound healing, neuropathic pain and itch, psychological wellbeing, and return to school or work.

Conclusions

This core outcome set caters for global burn research, and future trials are recommended to include measures of these outcomes.

Evaluations of training and education interventions for improved infectious disease management in low-income and middle-income countries: a systematic literature review

OBJECTIVES

To identify most vital input and outcome parameters required for evaluations of training and education interventions aimed at addressing infectious diseases in low-income and middle-income countries.

DESIGN

Systematic review.

DATA SOURCES

PubMed/Medline, Web of Science and Scopus were searched for eligible studies between January 2000 and November 2021.

STUDY SELECTION

Health economic and health-outcome studies on infectious diseases covering an education or training intervention in low-income and middle-income countries were included.

RESULTS

A total of 59 eligible studies covering training or education interventions for infectious diseases were found; infectious diseases were categorised as acute febrile infections (AFI), non-AFI and other non-acute infections. With regard to input parameters, the costs (direct and indirect) were most often reported. As outcome parameters, five categories were most often reported including final health outcomes, intermediate health outcomes, cost outcomes, prescription outcomes and health economic outcomes. Studies showed a wide range of per category variables included and a general lack of uniformity across studies.

CONCLUSIONS

Further standardisation is needed on the relevant input and outcome parameters in this field. A more standardised approach would improve generalisability and comparability of results and allow policy-makers to make better informed decisions on the most effective and cost-effective interventions.

Inclusion of participants from low-income and middle-income countries in core outcome sets development: a systematic review

OBJECTIVE

Our study aims to describe differences or similarities in the scope, participant characteristics and methods used in core outcome sets (COS) development when only participants from high-income countries (HICs) were involved compared with when participants from low-income and middle-income countries (LMICs) were also involved.

DESIGN

Systematic review.

DATA SOURCES

Annual Core Outcome Measures in Effectiveness Trials systematic reviews of COS which are updated based on SCOPUS and MEDLINE, searches. The latest systematic review included studies published up to the end of 2019.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

We included studies reporting development of a COS for use in research regardless of age, health condition or setting. Studies reporting the development of a COS for patient-reported outcomes or adverse events or complications were also included.

DATA EXTRACTION AND SYNTHESIS

Data were extracted in relation to scope of the COS study, participant categories and the methods used in outcome selection.

RESULTS

Studies describing 370 COS were identified in the database. Of these, 75 (20%) included participants from LMICs. Only four COS were initiated from an LMIC setting. More than half of COS with LMIC participants were developed in the last 5 years. Cancer and rheumatology were the dominant disease domains. Overall, over 259 (70%) of COS explicitly reported including clinical experts; this was higher where LMIC participants were also included 340 (92%). Most LMIC participants were from China, Brazil and South Africa. Mixed methods for consensus building were used across the two settings.

CONCLUSION

Progress has been made in including LMIC participants in the development of COS, however, there is a need to explore how to enable initiation of COS development from a range of LMIC settings, how to ensure prioritisation of COS that better reflects the burden of disease in these contexts and how to improve public participation from LMICs.

Patient participation in Delphi surveys to develop core outcome sets: systematic review

OBJECTIVES

To describe the design and conduct of core outcome set (COS) studies that have included patients as participants, exploring how study characteristics might impact their response rates.

DESIGN

Systematic review of COS studies published between 2015 and 2019 that included more than one patient, carer or representative as participants (hereafter referred to as patients for brevity) in scoring outcomes in a Delphi.

RESULTS

There were variations in the design and conduct of COS studies that included patients in the Delphi process, including differing: scoring and feedback systems, approaches to recruiting patients, length of time between rounds, use of reminders, incentives, patient and public involvement, and piloting. Minimal reporting of participant characteristics and a lack of translation of Delphi surveys into local languages were found. Additionally, there were indications that studies that recruited patients through treatment centres had higher round two response rates than studies recruiting through patient organisations.

CONCLUSIONS

Variability was striking in how COS Delphi surveys were designed and conducted to include patient participants and other stakeholders. Future research is needed to explore what motivates patients to take part in COS studies and what factors influence COS developer recruitment strategies. Improved reporting would increase knowledge of how methods affect patient participation in COS Delphi studies.

More than half of systematic reviews have relevant core outcome sets

OBJECTIVES

Using recent systematic reviews (SRs), our objectives were to: (1) develop a framework to assess whether a given COS is relevant to the scope of a SR; (2) examine the proportion of SRs for which relevant COS exist; and (3) for SRs for which COS exist, examine the extent to which outcomes in the COS and outcomes in the SR match.

STUDY DESIGN AND SETTING

We included a sample of SRs published by the Agency for Healthcare Research and Quality Evidence-based Practice Center Program between January 1, 2018 and October 12, 2020. We searched for potentially relevant COS from the Core Outcome Measures for Effectiveness Trials (COMET) database. We assessed the matching between outcomes recommended by COS and those included in corresponding SRs. When outcomes were matched, we considered matches to be specific (i.e., exact) or general (i.e., non-specific).

RESULTS

Sixty-seven SRs met criteria. We found relevant COS for 36 of 67 SRs (54%). Our framework for comparing the scope of a SR and a COS describes 16 scenarios arising when the breadth of the populations and the interventions are considered. The framework guides systematic reviewers to determine whether a COS is very likely to be relevant, may be relevant, or unlikely to be relevant. Sixty-two percent of outcomes in COS (interquartile range, 40% - 80%) were either specific or general matches to outcomes in SRs.

CONCLUSION

We found a COS with relevant scope for more than half of the SRs in our sample, with almost two-thirds of the recommended core outcomes matched to outcomes chosen for the SRs. Consideration of COS appears relevant for SR planning and our framework for assessing relevance of a given COS may help with this process.

Methods for conducting international Delphi surveys to optimise global participation in core outcome set development: a case study in gastric cancer informed by a comprehensive literature review

Background

Core outcome sets (COS) should be relevant to key stakeholders and widely applicable and usable. Ideally, they are developed for international use to allow optimal data synthesis from trials. Electronic Delphi surveys are commonly used to facilitate global participation; however, this has limitations. It is common for these surveys to be conducted in a single language potentially excluding those not fluent in that tongue. The aim of this study is to summarise current approaches for optimising international participation in Delphi studies and make recommendations for future practice.

Methods

A comprehensive literature review of current approaches to translating Delphi surveys for COS development was undertaken. A standardised methodology adapted from international guidance derived from 12 major sets of translation guidelines in the field of outcome reporting was developed. As a case study, this was applied to a COS project for surgical trials in gastric cancer to translate a Delphi survey into 7 target languages from regions active in gastric cancer research.

Results

Three hundred thirty-two abstracts were screened and four studies addressing COS development in rheumatoid and osteoarthritis, vascular malformations and polypharmacy were eligible for inclusion. There was wide variation in methodological approaches to translation, including the number of forward translations, the inclusion of back translation, the employment of cognitive debriefing and how discrepancies and disagreements were handled. Important considerations were identified during the development of the gastric cancer survey including establishing translation groups, timelines, understanding financial implications, strategies to maximise recruitment and regulatory approvals. The methodological approach to translating the Delphi surveys was easily reproducible by local collaborators and resulted in an additional 637 participants to the 315 recruited to complete the source language survey. Ninety-nine per cent of patients and 97% of healthcare professionals from non-English-speaking regions used translated surveys.

Conclusion

Consideration of the issues described will improve planning by other COS developers and can be used to widen international participation from both patients and healthcare professionals.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-021-05338-x.

Core Outcome Sets Relevant to Maternity Service Users: A Scoping Review

INTRODUCTION

Variation in outcomes reported in maternity-related clinical trials and practice stifles data synthesis and contributes to ineffective or harmful treatments and interventions. Variation can be addressed using core outcome sets (COSs), minimum agreed sets of outcomes that should be measured and reported in all clinical trials in a specific area of health or health care. This scoping review identified studies that developed maternity-related COSs; evaluated the extent, scope, quality, and consistency of outcomes across similar COSs; and identified current gaps in evidence.

METHODS

A multifaceted search of 2 COS registers (Core Outcome Measures in Effectiveness Trials, Core Outcomes in Women's and Newborn Health), the International Consortium for Health Outcomes Measurement website, electronic databases (MEDLINE, Embase, CINAHL), and hand search was conducted. Published, English-language studies describing maternal and neonatal COSs for any health condition and published from inception to January 2020 were included. COS development process was evaluated against recently published COS Standards for Development: scope, stakeholder involvement, and consensus process.

RESULTS

Thirty-two articles relating to 26 COSs were included (maternal: 18 articles that addressed 17 COSs; neonatal: 14 articles that addressed 9 COSs) and covered a range of obstetric and neonatal conditions. COSs were published between 2006 and 2020, 58% since 2017. Maternal COSs included more outcomes (median, 17; range, 50) than neonatal COSs (median, 8; range, 20). Overlap in COSs was seen for maternity care and gestational diabetes. Overlap in outcomes was seen across similar COSs, which were mostly inconsistent or poorly defined. No included COS met all minimum standards for development. Two COSs extended recommendations for how and when to measure outcomes.

DISCUSSION

Growth in COS development in the last 3 years signifies increasing commitment to address variation and improve data synthesis. Although the quality of the development process has improved in the last 3 years, there is a need for improvement. This article presents an urgent need to minimize overlap in outcomes and standardize outcome measurement, case definitions, and timing of measurement between COSs.

Choosing important health outcomes for comparative effectiveness research: 6th annual update to a systematic review of core outcome sets for research

Background

An annual update to a systematic review of core outcome sets (COS) for research ensures that the COMET database is up-to-date. The aims of this study were to: (i) identify COS that were published or indexed in 2019 and to describe the methodological approaches used in these studies; (ii) investigate whether children have been included as participants in published COS development studies, and which methods have been used to facilitate their participation; iii) update a previous exercise to identify COS relevant to the most burdensome global diseases and injuries.

Methods

MEDLINE and SCOPUS were searched to identify studies published or indexed between (and inclusive of) January 2019 and December 2019. Automated screening methods were used to rank the citations in order of relevance; the top 25% in ranked priority order were screened for eligibility. COS were assessed against each of the Core Outcome Set-STAndards for Development (COS-STAD). A search of the COMET database was undertaken to identify COS relevant to the 25 leading causes of disease burden.

Results

Thirty-three studies, describing the development of 37 COS, were included in this update. These studies have been added to the COMET database, which now contains 370 published (1981–2019) COS studies for clinical research. Six (18%) of the 33 studies in this update were deemed to have met all of the minimum standards for COS development (range = 4 to 12 criteria, median = 9 criteria). Of the 370 COS studies published to date, 82 COS have been developed for paediatric health conditions and children would have been eligible to participate in 68/82 of these studies. Eleven of these 68 (16%) COS studies have included children as participants within the development process, most commonly through participation in Delphi surveys. Relevant COS were identified for 22/25 leading causes of global disease burden.

Conclusion

There has been a demonstrated increase in COS developed for both research and routine practice, and consistently high inclusion of patient participants. COS developed for paediatric conditions need to further incorporate the perspectives of children, alongside parents and other adults, and adopt research methods fit for this purpose. COS developers should consider the gaps identified in this update as priorities for COS development.

A systematic review finds Core Outcome Set uptake varies widely across different areas of health

OBJECTIVE

The aim of our review was to bring together studies that had assessed the uptake of core outcome sets (COS) to explore the level of uptake across different COS and areas of health.

STUDY DESIGN AND SETTING

We examined the citations of 337 COS reports to identify studies that had assessed the uptake of a particular COS in randomized controlled trials (RCTs) or systematic reviews (SRs).

RESULTS

We identified 24 studies that had assessed uptake in RCTs and two studies that had assessed uptake in SRs. The studies covered a total of 17/337 (5%) COS. Uptake rates reported for RCTs varied from 0% of RCTs (gout) to 82% RCTs (rheumatoid arthritis) measuring the full COS. Studies that assessed uptake of individual core outcomes showed a wide variation in uptake between the outcomes. Suggested barriers to uptake included lack of validated measures, lack of patient and other key stakeholder involvement in COS development, and lack of awareness of the COS.

CONCLUSIONS

Few studies have been undertaken to assess the uptake of COS in RCTs and SRs. Further studies are needed to assess whether COS have been implemented across a wider range of disease categories and to explore the barriers and facilitators to COS uptake.

Systematic review of international Delphi surveys for core outcome set development: representation of international patients

OBJECTIVES

A core outcome set (COS) describes a minimum set of outcomes to be reported by all clinical trials of one healthcare condition. Delphi surveys are frequently used to achieve consensus on core outcomes. International input is important to achieve global COS uptake. We aimed to investigate participant representation in international Delphi surveys, with reference to the inclusion of patients and participants from low and middle income countries as stakeholders (LMICs).

DESIGN

Systematic review.

DATA SOURCES

EMBASE, Medline, Web of Science, COMET database and hand-searching.

ELIGIBILITY CRITERIA

Protocols and studies describing Delphi surveys used to develop an international COS for trial reporting, published between 1 January 2017 and 6 June 2019.

DATA EXTRACTION AND SYNTHESIS

Delphi participants were grouped as patients or healthcare professionals (HCPs). Participants were considered international if their country of origin was different to that of the first or senior author. Data extraction included participant numbers, country of origin, country income group and whether Delphi surveys were translated. We analysed the impact these factors had on outcome prioritisation.

RESULTS

Of 90 included studies, 69% (n=62) were completed and 31% (n=28) were protocols. Studies recruited more HCPs than patients (median 60 (IQR 30-113) vs 30 (IQR 14-66) participants, respectively). A higher percentage of HCPs was international compared with patients (57% (IQR 37-78) vs 20% (IQR 0-68)). Only 31% (n=28) studies recruited participants from LMICs. Regarding recruitment from LMICs, patients were under-represented (16% studies; n=8) compared with HCPs (22%; n=28). Few (7%; n=6) studies translated Delphi surveys. Only 3% studies (n=3) analysed Delphi responses by geographical location; all found differences in outcome prioritisation.

CONCLUSIONS

There is a disproportionately lower inclusion of international patients, compared with HCPs, in COS-development Delphi surveys, particularly within LMICs. Future international Delphi surveys should consider exploring for geographical and income-based differences in outcome prioritisation.

PROSPERO REGISTRATION NUMBER

CRD42019138519.

Use of wearable biometric monitoring devices to measure outcomes in randomized clinical trials: a methodological systematic review

BACKGROUND

Wearable biometric monitoring devices (BMDs) have the potential to transform the conduct of randomized controlled trials (RCTs) by shifting the collection of outcome data from single measurements at predefined time points to dense continuous measurements.

METHODS

Methodological systematic review to understand how recent RCTs used BMDs to measure outcomes and to describe the reporting of these RCTs. Electronic search was performed in the Cochrane Central Register of Controlled Trials, PubMed, and EMBASE and completed a page-by-page hand search in five leading medical journals between January 1, 2018, and December 31, 2018. Three reviewers independently extracted all primary and secondary outcomes collected using BMDs, and assessed (1) the definitions used to summarize BMD outcome data; (2) whether the validity, reliability, and responsiveness of sensors was reported; (3) the discrepancy with outcomes prespecified in public clinical trial registries; and (4) the methods used to manage missing and incomplete BMD outcome data.

RESULTS

Of the 4562 records screened, 75 RCTs were eligible. Among them, 24% tested a pharmacological intervention and 57% used an inertial measurement sensor to measure physical activity. Included trials involved 464 outcomes (average of 6 [SD = 8] outcomes per trial). In total, 35 trials used a BMD to measure a primary outcome. Several issues affected the value and transparency of trials using BMDs to measure outcomes. First, the definition of outcomes used in the trials was highly heterogeneous (e.g., 21 diabetes trials had 266 outcomes and 153 had different unique definitions to measure diabetes control), which limited the combination and comparison of results. Second, information on the validity, reliability, and responsiveness of sensors used was lacking in 74% of trials. Third, half (53%) of the outcomes measured with BMDs had not been prespecified, with a high risk of outcome reporting bias. Finally, reporting on the management of incomplete outcome data (e.g., due to suboptimal compliance with the BMD) was absent in 68% of RCTs.

CONCLUSIONS

Use of BMDs to measure outcomes is becoming the norm rather than the exception in many fields. Yet, trialists need to account for several methodological issues when specifying and conducting RCTs using these novel tools.

Dental Research Waste in Design, Analysis, and Reporting: A Scoping Review

Research waste is highly prevalent across biomedical investigations. We aimed to assess the evidence on the extent of research waste in dental research. We performed a scoping review of empirical evaluations of dental studies assessing the prevalence and impact of limitations in design, conduct, analysis, and reporting of research. PubMed was searched using specific terms to retrieve studies dealing with design, conduct, analysis, and reporting of studies in dentistry, with no year or language restrictions. Of the 1,807 publications identified from the search and from manual searches, 71 were included in this review. The topic and article selection was based on the expert opinion of the authors. The existing evidence suggests that, although there are improvements over time, substantial deficiencies in all areas (design, conduct, analysis, reporting) were prevalent in dental research publications. Waste in research is a multifaceted problem without a simple solution. However, an appreciation of optimal research design and execution is a prerequisite and should be underpinned by policies that include appropriate training in research methods and properly aligned incentives.

Developing consensus on core outcome domains and measurement instruments for assessing effectiveness in perioperative pain management after sternotomy, breast cancer surgery, total knee arthroplasty, and surgery related to endometriosis : The IMI-PainCare PROMPT protocol for achieving a consensus on core outcome domains

INTRODUCTION

Evidence synthesis of clinical trials requires consistent outcome assessment. For pain management after surgery, inconsistency of effectiveness assessment is still observed. A subproject of IMI-PainCare (Innovative Medicine Initiatives, www.imi-paincare.eu ) aims for identifying core outcome domains and measurement instruments for postoperative pain in four surgical fields (sternotomy, breast cancer surgery, total knee arthroplasty, and surgery related to endometriosis) in order to harmonize outcome assessment for perioperative pain management.

METHODS

A multifaceted process will be performed according to existing guidelines (Core Outcome Measures in Effectiveness Trials (COMET), COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN)). In a first step, outcome domains will be identified via systematic literature review and consented on during a 1-day consensus meeting by 10 stakeholder groups, including patient representatives, forming an IMI PROMPT consensus panel. In a second step, outcome measurement instruments regarding the beforehand consented core outcome domains and their psychometric properties will be searched for via systematic literature review and approved by COSMIN checklist for study quality and scale quality separately. In a three-step online survey, the IMI PROMPT consensus panel will vote for most suitable measurement instruments. The process is planned to be conducted between 11/2017 (systematic literature review on common outcome domains) and 3/2022 (final voting on core outcome measurement).

Assessing the relevance and uptake of core outcome sets (an agreed minimum collection of outcomes to measure in research studies) in Cochrane systematic reviews: a review

OBJECTIVES

A core outcome set (COS) is an agreed standardised minimum collection of outcomes that should be measured and reported in research in a specific area of health. Cochrane systematic reviews ('reviews') are rigorous reviews on health-related topics conducted under the auspices of Cochrane. This study examines the use of existing COS to inform the choice of outcomes in Cochrane systematic reviews ('reviews') and investigates the views of the coordinating editors of Cochrane Review Groups (CRGs) on this topic.

METHODS

A cohort of 100 recently published or updated Cochrane reviews were assessed for reference to a COS being used to inform the choice of outcomes for the review. Existing COS, published 2 or more years before the review publication, were then identified to assess how often a reviewer could have used a relevant COS if it was available. We asked 52 CRG coordinating editors about their involvement in COS development, how outcomes are selected for reviews in their CRG and their views of the advantages and challenges surrounding the standardisation of outcomes within their CRG.

RESULTS

In the cohort of reviews from 2019, 40% (40/100) of reviewers noted problems due to outcome inconsistency across the included studies. In 7% (7/100) of reviews, a COS was referenced in relation to the choice of outcomes for the review. Relevant existing COS could be considered for a review update in 35% of the others (33/93). Most editors who responded (31/36, 86%) thought that COS should definitely or possibly be used to inform the choice of outcomes in a review.

CONCLUSIONS

Systematic reviewers are continuing to note outcome heterogeneity but are starting to use COS to inform their reviews. There is potential for greater uptake of COS in Cochrane reviews.

Outcomes reported in trials of childhood fractures: a systematic review

Aims

To analyze outcomes reported in trials of childhood fractures.

Methods

OVID MEDLINE, Embase, and Cochrane CENTRAL databases were searched on the eighth August 2019. A manual search of trial registries, bibliographic review and internet search was used to identify additional studies. 11,476 studies were screened following PRISMA guidelines. 100 trials were included in the analysis. Data extraction was completed by two researchers for each trial. Study quality was not evaluated. Outcomes reported by trials were mapped onto domains in the World Health Organization (WHO) International Classification of Function framework.

Results

In all, 525 outcomes were identified representing 52 WHO domains. Four domains were reported in more than 50% of trials: structure of upper/lower limb, sensation of pain, mobility of joint function, and health services, systems and policies. The Activities Scale for Kids performance (ASK-p) score was the most common outcome score reported in 6/72 upper limb and 4/28 lower limb trials.

Conclusion

There is a diverse range of outcomes reported in trials of childhood fractures covering all areas in the International Classification of Functioning, Disability and Health (ICF) framework. There were three common upper limb and three common lower limb outcomes. In the absence of a core outcome set, we recommend that upper limb trials report pain, range of movement and radiograph appearance of the arm and lower limb trials report pain, radiograph appearance of the leg and healthcare costs to improve consistency of reporting in future trials.

Cite this article: Bone Joint Open 2020;1-5:167–174.

Outcomes important to patients with non-infectious posterior segment-involving uveitis: a qualitative study

OBJECTIVE

Uveitis, a group of disorders characterised by intraocular inflammation, causes 10%-15% of total blindness in the developed world. The most sight-threatening forms of non-infectious uveitis are those affecting the posterior segment of the eye, collectively known as posterior segment-involving uveitis (PSIU). Numerous different clinical outcomes have been used in trials evaluating treatments for PSIU, but these may not represent patients' and carers' concerns. Therefore, the aims of this study were to understand the impact of PSIU on adult patients' and carers' lives and to explore what outcomes of treatment are important to them.

METHODS AND ANALYSIS

Four focus group discussions were undertaken to understand the perspectives of adult patients (=18) and carers (10) with PSIU. Participants were grouped according to whether or not their uveitis was complicated by the sight-threatening condition uveitic macular oedema. Discussions were audio-recorded, transcribed and analysed using the framework analytical approach. Outcomes were identified and grouped into outcome domains.

RESULTS

Eleven core domains were identified as important to patients and carers undergoing treatment for PSIU, comprising (1) visual function, (2) symptoms, (3) functional ability, (4) impact on relationships, (5) financial impact, (6) psychological morbidity and emotional well-being, (7) psychosocial adjustment to uveitis, (8) doctor/patient/interprofessional relationships and access to healthcare, (9) treatment burden, (10) treatment side effects, and (11) disease control.

CONCLUSION

The domains identified represent patients' and carers' experience and perspectives and can be used to reflect on outcomes assessed in PSIU. They will directly inform the development of a core outcome set for PSIU clinical trials.

Methods used to assess outcome consistency in clinical studies: A literature-based evaluation

Evaluation studies of outcomes used in clinical research and their consistency are appearing more frequently in the literature, as a key part of the core outcome set (COS) development. Current guidance suggests such evaluation studies should use systematic review methodology as their default. We aimed to examine the methods used. We searched the Core Outcome Measures in Effectiveness Trials (COMET) database (up to May 2019) supplementing it with additional resources. We included evaluation studies of outcome consistency in clinical studies across health subjects and used a subset of A MeaSurement Tool to Assess systematic Reviews (AMSTAR) 2 (items 1-9) to assess their methods. Of 93 included evaluation studies of outcome consistency (90 full reports, three summaries), 91% (85/93) reported performing literature searches in at least one bibliographic database, and 79% (73/93) was labelled as a "systematic review". The evaluations varied in terms of satisfying AMSTAR 2 criteria, such that 81/93 (87%) had implemented PICO in the research question, whereas only 5/93 (6%) had included the exclusions list. None of the evaluation studies explained how inconsistency of outcomes was detected, however, 80/90 (88%) concluded inconsistency in individual outcomes (66%, 55/90) or outcome domains (20%, 18/90). Methods used in evaluation studies of outcome consistency in clinical studies differed considerably. Despite frequent being labelled as a "systematic review", adoption of systematic review methodology is selective. While the impact on COS development is unknown, authors of these studies should refrain from labelling them as "systematic review" and focus on ensuring that the methods used to generate the different outcomes and outcome domains are reported transparently.

Core outcomes in neonatology: development of a core outcome set for neonatal research

BACKGROUND

Neonatal research evaluates many different outcomes using multiple measures. This can prevent synthesis of trial results in meta-analyses, and selected outcomes may not be relevant to former patients, parents and health professionals.

OBJECTIVE

To define a core outcome set (COS) for research involving infants receiving neonatal care in a high-income setting.

DESIGN

Outcomes reported in neonatal trials and qualitative studies were systematically reviewed. Stakeholders were recruited for a three-round international Delphi survey. A consensus meeting was held to confirm the final COS, based on the survey results.

PARTICIPANTS

Four hundred and fourteen former patients, parents, healthcare professionals and researchers took part in the eDelphi survey; 173 completed all three rounds. Sixteen stakeholders participated in the consensus meeting.

RESULTS

The literature reviews identified 104 outcomes; these were included in round 1. Participants proposed 10 additional outcomes; 114 outcomes were scored in rounds 2 and 3. Round 1 scores showed different stakeholder groups prioritised contrasting outcomes. Twelve outcomes were included in the final COS: survival, sepsis, necrotising enterocolitis, brain injury on imaging, general gross motor ability, general cognitive ability, quality of life, adverse events, visual impairment/blindness, hearing impairment/deafness, retinopathy of prematurity and chronic lung disease/bronchopulmonary dysplasia.

CONCLUSIONS AND RELEVANCE

A COS for clinical trials and other research studies involving infants receiving neonatal care in a high-income setting has been identified. This COS for neonatology will help standardise outcome selection in clinical trials and ensure these are relevant to those most affected by neonatal care.

Core outcome sets through the healthcare ecosystem: the case of type 2 diabetes mellitus

BACKGROUND

It is increasingly accepted that insufficient attention has been given to the patient health outcomes that are important to measure in comparative effectiveness research that will inform decision-making. The relationship between outcomes chosen for comparative effectiveness research, outcomes used in decision-making in routine care, and outcome data recorded in electronic health records (EHR) is also poorly understood. The COMET Initiative (http://www.comet-initiative.org/. Accessed 3 Apr 2020) supports and encourages the development and use of 'core outcome sets' (COS), which represent the minimum set of patient health outcomes that should be measured and reported for a specific condition. There is growing interest in identifying how COS might fit into the different stages of the healthcare research and delivery ecosystem, and whether inclusion in the EHR might facilitate this.

METHODS

We sought to determine the degree of overlap between outcomes within COS for research and routine care, EMA, FDA and NICE guidelines, NICE quality statements/indicators, EHR and a point-of-care randomised clinical trial, using type 2 diabetes (T2D) as a case study.

RESULTS

There is substantial agreement about important patient outcomes for T2D for research and healthcare, with associated coverage within the UK general practice EHR.

CONCLUSIONS

This case study has demonstrated the potential for efficient research and value-based healthcare when the EHR can include COS for both research and care, where the COS comprises outcomes of importance to all relevant stakeholders. However, this concordance may not hold more generally, as the focus on patient-centred outcomes may well be greater in T2D than in other conditions. Work is ongoing to examine other clinical areas, in order to highlight any current inefficiencies when health outcomes in research and healthcare do not agree with core outcomes identified by patients, clinicians and other key stakeholders.

Analysis of responder-based endpoints: improving power through utilising continuous components

BACKGROUND

Clinical trials and other studies commonly assess the effectiveness of an intervention through the use of responder-based endpoints. These classify patients based on whether they meet a number of criteria which often involve continuous variables categorised as being above or below a threshold. The proportion of patients who are responders is estimated and, where relevant, compared between groups. An alternative method called the augmented binary method keeps the definition of the endpoint the same but utilises information contained within the continuous component to increase the power considerably (equivalent to increasing the sample size by > 30%). In this article we summarise the method and investigate the variety of clinical conditions that use endpoints to which it could be applied.

METHODS

We reviewed a database of core outcome sets (COSs) that covered physiological and mortality trial endpoints recommended for collection in clinical trials of different disorders. We identified responder-based endpoints where the augmented binary method would be useful for increasing power.

RESULTS

Out of the 287 COSs reviewed, we identified 67 new clinical areas where endpoints were used that would be more efficiently analysed using the augmented binary method. Clinical areas that had particularly high numbers were rheumatology (11 clinical disorders identified), non-solid tumour oncology (10 identified), neurology (9 identified) and cardiovascular (8 identified).

CONCLUSIONS

The augmented binary method can potentially provide large benefits in a vast array of clinical areas. Further methodological development is needed to account for some types of endpoints.

Ensuring young voices are heard in core outcome set development: international workshops with 70 children and young people

PLAIN ENGLISH SUMMARY

Researchers test treatments to ensure these work and are safe. They do this by studying the effects that treatments have on patients by measuring outcomes, such as pain and quality of life. Often research teams measure different outcomes even though each team is studying the same condition. This makes it hard to compare the findings from different studies and it can reduce the accuracy of the treatment advice available to patients. Increasingly, researchers are tackling this problem by developing 'core outcome sets'. These are lists of outcomes that all researchers working on a given condition should measure in their studies. It is important that patients have a voice in the development of core outcome sets and children and young people are no exception. But their voices have rarely been heard when core outcome sets are developed. Researchers are trying to address this problem and make sure that core outcome sets are developed in ways that are suitable for children and young people. As a first step, we held two international workshops with children and young people to listen to their views. They emphasised the importance of motivating young people to participate in developing core outcome sets, making them feel valued, and making the development process more interactive, enjoyable and convenient. We hope this commentary will encourage researchers to include children and young people when developing core outcome sets and to adapt their methods so these are suitable for young participants. Future research is important to examine whether these adaptations are effective.

ABSTRACT

Background Different research teams looking at treatments for the same condition often select and measure inconsistent treatment outcomes. This makes it difficult to synthesise the results of different studies, leads to selective outcome reporting and impairs the quality of evidence about treatments. 'Core outcome sets' (COS) can help to address these problems. A COS is an agreed, minimum list of outcomes that researchers are encouraged to consistently measure and report in their studies. Including children and young people (CYP) as participants in the development of COS for paediatric conditions ensures that clinically meaningful outcomes are measured and reported. However, few published COS have included CYP as participants. COS developers have described difficulties in recruiting and retaining CYP and there is a lack of guidance on optimising COS methods for them. We aimed to explore CYP's views on the methods used to develop COS and identify ways to optimise these methods.Main body This commentary summarises discussions during two workshops with approximately 70 CYP (aged 10-18 years old) at the International Children's Advisory Network Research and Advocacy Summit, 2018. Delegates described what might motivate them to participate in a COS study, including feeling valued, understanding the need for COS and the importance of input from CYP in their development, and financial and other incentives (e.g. certificates of participation). For Delphi surveys, delegates suggested that lists of outcomes should be as brief as possible, and that scoring and feedback methods should be simplified. For consensus meetings, delegates advised preparing CYP in advance, supporting them during meetings (e.g. via mentors) and favoured arrangements whereby CYP could meet separately from parents and other stakeholders. Overall, they wanted COS methods that were convenient, enjoyable and engaging.Conclusion This commentary points to the limitations of the methods currently used to develop COS with CYP. It also points to ways to motivate CYP to participate in COS studies and to enhancements of methods to make participation more engaging for CYP. Pending much needed research on COS methods for CYP, the perspectives offered in the workshops should help teams developing COS in paediatrics and child health.

Core outcome set in paediatric sepsis in low- and middle-income countries: a study protocol

INTRODUCTION

Sepsis is the leading cause of death in children worldwide and has recently been declared a major global health issue. New interventions and a concerted effort to enhance our understanding of sepsis are required to address the huge burden of disease, especially in low- and middle-income countries (LMIC) where it is highest. An opportunity therefore exists to ensure that ongoing research in this area is relevant to all stakeholders and is of consistently high quality. One method to address these issues is through the development of a core outcome set (COS).

METHODS AND ANALYSIS

This study protocol outlines the phases in the development of a core outcome set for paediatric sepsis in LMIC. The first step involves performing a systematic review of all outcomes reported in the research of paediatric sepsis in low middle-income countries. A three-stage international Delphi process will then invite a broad range of participants to score each generated outcome for inclusion into the COS. This will include an initial two-step online survey and finally, a face-to-face consensus meeting where each outcome will be reviewed, voted on and ratified for inclusion into the COS.

ETHICS AND DISSEMINATION

No core outcome sets exist for clinical trials in paediatric sepsis. This COS will serve to not only highlight the heavy burden of paediatric sepsis in this setting and aid collaboration and participation between all stakeholders, but to promote ongoing essential high quality and relevant research into the topic. A COS in paediatric sepsis in LMIC will advocate for a common language and facilitate interpretation of findings from a variety of settings. A waiver for ethics approval has been granted by University of British Columbia Children's and Women's Research Ethics Board.

Pediatric perioperative outcomes: Protocol for a systematic literature review and identification of a core outcome set for infants, children, and young people requiring anesthesia and surgery

Clinical outcomes are measurable changes in health, function, or quality of life that are important for evaluating the quality of care and comparing the efficacy of interventions. However, clinical outcomes and related measurement tools need to be well-defined, relevant, and valid. In adults, Core Outcome Measures in Effectiveness Trials (COMET) methodology has been used to develop core outcome sets for perioperative care. Systematic literature reviews identified standardized endpoints (StEP) and valid measurement tools, and consensus across a broader range of relevant stakeholders was achieved via a Delphi process to establish Core Outcome Measures in Perioperative and Anaesthetic Care (COMPAC). Core outcome sets for pediatric perioperative care cannot be directly extrapolated from adult data. The type and weighting of endpoints within particular domains can be influenced by age-dependent differences in the indications for and/or nature of surgery and medical comorbidities, and the validity and utility of many measurement tools vary significantly with developmental stage and age. The involvement of parents/carers is essential as they frequently act as surrogate responders for preverbal and developmentally delayed children, parental response may influence child outcome, and parental and/or child ranking of outcomes may differ from those of health professionals. Here, we describe the formation of the international Pediatric Perioperative Outcomes Group, which aims to identify and create validated, broadly applicable, patient-centered outcome measures for infants, children, and young people. Methodologies parallel that of the StEP and COMPAC projects, and systematic literature searches have been performed within agreed age-dependent subpopulations to identify reported outcomes and measurement tools. This represents the first steps for developing core outcome sets for pediatric perioperative care.

A scoping review of core outcome sets and their 'mapping' onto real-world data using prostate cancer as a case study

Background

A Core Outcomes Set (COS) is an agreed minimum set of outcomes that should be reported in all clinical studies related to a specific condition. Using prostate cancer as a case study, we identified, summarized, and critically appraised published COS development studies and assessed the degree of overlap between them and selected real-world data (RWD) sources.

Methods

We conducted a scoping review of the Core Outcome Measures in Effectiveness Trials (COMET) Initiative database to identify all COS studies developed for prostate cancer. Several characteristics (i.e., study type, methods for consensus, type of participants, outcomes included in COS and corresponding measurement instruments, timing, and sources) were extracted from the studies; outcomes were classified according to a predefined 38-item taxonomy. The study methodology was assessed based on the recent COS-STAndards for Development (COS-STAD) recommendations. A ‘mapping’ exercise was conducted between the COS identified and RWD routinely collected in selected European countries.

Results

Eleven COS development studies published between 1995 and 2017 were retrieved, of which 8 were classified as ‘COS for clinical trials and clinical research’, 2 as ‘COS for practice’ and 1 as ‘COS patient reported outcomes’. Recommended outcomes were mainly categorized into ‘mortality and survival’ (17%), ‘outcomes related to neoplasm’ (18%), and ‘renal and urinary outcomes’ (13%) with no relevant differences among COS study types. The studies generally fulfilled the criteria for the COS-STAD ‘scope specification’ domain but not the ‘stakeholders involved’ and ‘consensus process’ domains. About 72% overlap existed between COS and linked administrative data sources, with important gaps. Linking with patient registries improved coverage (85%), but was sometimes limited to smaller follow-up patient groups.

Conclusions

This scoping review identified few COS development studies in prostate cancer, some quite dated and with a growing level of methodological quality over time. This study revealed promising overlap between COS and RWD sources, though with important limitations; linking established, national patient registries to administrative data provide the best means to additionally capture patient-reported and some clinical outcomes over time. Thus, increasing the combination of different data sources and the interoperability of systems to follow larger patient groups in RWD is required.

Outcome choice and definition in systematic reviews leads to few eligible studies included in meta-analyses: a case study

BACKGROUND

There is broad recognition of the importance of evidence in informing clinical decisions. When information from all studies included in a systematic review ("review") does not contribute to a meta-analysis, decision-makers can be frustrated. Our objectives were to use the field of eyes and vision as a case study and examine the extent to which authors of Cochrane reviews conducted meta-analyses for their review's pre-specified main outcome domain and the reasons that some otherwise eligible studies were not incorporated into meta-analyses.

METHODS

We examined all completed systematic reviews published by Cochrane Eyes and Vision, as of August 11, 2017. We extracted information about each review's outcomes and, using an algorithm, categorized one outcome as its "main" outcome. We calculated the percentage of included studies incorporated into meta-analyses for any outcome and for the main outcome. We examined reasons for non-inclusion of studies into the meta-analysis for the main outcome.

RESULTS

We identified 175 completed reviews, of which 125 reviews included two or more studies. Across these 125 reviews, the median proportions of studies incorporated into at least one meta-analysis for any outcome and for the main outcome were 74% (interquartile range [IQR] 0-100%) and 28% (IQR 0-71%), respectively. Fifty-one reviews (41%) could not conduct a meta-analysis for the main outcome, mostly because fewer than two included studies measured the outcome (21/51 reviews) or the specific measurements for the outcome were inconsistent (16/51 reviews).

CONCLUSIONS

Outcome choice during systematic reviews can lead to few eligible studies included in meta-analyses. Core outcome sets and improved reporting of outcomes can help solve some of these problems.

Adding value to core outcome set development using multimethod systematic reviews

Trials evaluating the same interventions rarely measure or report identical outcomes. This limits the possibility of aggregating effect sizes across studies to generate high-quality evidence through systematic reviews and meta-analyses. To address this problem, core outcome sets (COS) establish agreed sets of outcomes to be used in all future trials. When developing COS, potential outcome domains are identified by systematically reviewing the outcomes of trials, and increasingly, through primary qualitative research exploring the experiences of key stakeholders, with relevant outcome domains subsequently determined through transdisciplinary consensus development. However, the primary qualitative component can be time consuming with unclear impact. We aimed to examine the potential added value of a qualitative systematic review alongside a quantitative systematic review of trial outcomes to inform COS development in neonatal care using case analysis methods. We compared the methods and findings of a scoping review of neonatal trial outcomes and a scoping review of qualitative research on parents', patients', and professional caregivers' perspectives of neonatal care. Together, these identified a wider range and greater depth of health and social outcome domains, some unique to each review, which were incorporated into the subsequent Delphi process and informed the final set of core outcome domains. Qualitative scoping reviews of participant perspectives research, used in conjunction with quantitative scoping reviews of trials, could identify more outcome domains for consideration and could provide greater depth of understanding to inform stakeholder group discussion in COS development. This is an innovation in the application of research synthesis methods.

Choosing important health outcomes for comparative effectiveness research: 5th annual update to a systematic review of core outcome sets for research

BACKGROUND

A systematic review of core outcome sets (COS) for research is updated annually to populate an online database. It is a resource intensive review to do annually but automation techniques have potential to aid the process. The production of guidance and standards in COS development means that there is now an expectation that COS are being developed and reported to a higher standard. This is the fifth update to the systematic review and will explore these issues.

METHODS

Searches were carried out to identify studies published or indexed in 2018. Automated screening methods were used to rank the citations in order of relevance. The cut-off for screening was set to the top 25% in ranked priority order, following development and validation of the algorithm. Studies were eligible for inclusion if they reported the development of a COS, regardless of any restrictions by age, health condition or setting. COS were assessed against each of the Core Outcome Set-STAndards for Development (COS-STAD).

RESULTS

Thirty studies describing the development of 44 COS were included in this update. Six COS (20%) were deemed to have met all 12 criteria representing the 11 minimum standards for COS development (range = 4 to 12 criteria, median = 10 criteria). All 30 COS studies met all four minimum standards for scope. Twenty-one (70%) COS met all three minimum standards for stakeholders. Twenty-three studies (77%) included patients with the condition or their representatives. The number of countries involved in the development of COS ranged from 1 to 39 (median = 10). Six studies (20%) met all four minimum standards [five criteria] for the consensus process.

CONCLUSION

Automated ranking was successfully used to assist the screening process and reduce the workload of this systematic review update. With the provision of guidelines, COS are better reported and being developed to a higher standard.

PREgnancy Nutrition: A protocol for the development of a Core Outcome Set (PRENCOS)

OBJECTIVE

To develop a Core Outcome Set (COS) for pregnancy nutrition research that is relevant to varied stakeholders and resource settings.

METHODS

This study has three distinct phases. The first phase involves generating a list of outcomes for consideration for the COS. This includes a systematic review of studies evaluating nutrition during pregnancy where all outcomes reported in relevant literature will be extracted. Qualitative interviews with currently or previously pregnant women will also be conducted. This step will supplement the findings of the systematic review by identifying additional outcomes of importance to this stakeholder group. In the second phase of the study, healthcare professionals, researchers, and mothers from various international resource settings will be invited to participate in a two-round modified Delphi survey. The aim of the survey is to gain consensus on which outcomes are most important to include in the COS. Finally, a face-face consensus meeting will be held with a select group of participants to finalize the COS.

CONCLUSION

This COS will support standardization of outcome reporting in pregnancy nutrition research and ensure that selected outcomes are considered important by a variety of stakeholders. This will enhance the evidence behind nutrition interventions in pregnancy to improve outcomes for pregnant women.

Assessing the impact of a research funder's recommendation to consider core outcome sets

Background

Core outcome sets (COS) have the potential to reduce waste in research by improving the consistency of outcomes measured in trials of the same health condition. However, this reduction in waste will only be realised through the uptake of COS by clinical trialists. Without uptake, the continued development of COS that are not implemented may add to waste in research. Funders of clinical trials have the potential to have an impact on COS uptake by recommending their use to those applying for funding. The aim of our study was to assess the extent to which applicants followed the National Institute for Health Research Health Technology Assessment (NIHR HTA) programme’s recommendation to search for a COS to include in their clinical trial.

Methods and findings

We examined the outcomes section and detailed project descriptions of all 95 researcher-led primary research applications submitted to the NIHR HTA between January 2012, when the recommendation to search for a COS was included in the guidance for applicants, and December 2015 for evidence that a search for a COS had taken place and rationale for outcome choice in the absence of COS. A survey of applicants was conducted to further explore their use of COS and choice of outcomes with a response rate of 49%. Nine out of 95 applicants (10%) stated in their application that they had searched the COMET (Core Outcome Measures for Effectiveness Trials) Initiative database for a COS and another nine referred to searching for a COS using another method, e.g. a review of the literature. Of the 77 (81%) applicants that did not mention COMET or COS in their application, eight stated in the survey that they had searched the COMET database and ten carried out a search using another method. Some applicants who did not search for a COS gave reasons for their choice of outcomes including taking advice from patients and the public and choosing outcomes used in previous trials.

Conclusion

A funding body can have an impact on COS uptake by encouraging trialists to search for a COS. Funders could take further steps by putting processes in place to prompt applicants to be explicit about searching for COS in their application and notifying the funding board if a search has not taken place. The sources of information used by trialists to make decisions about outcomes in the absence of COS may suggest methods of dissemination for COS.

A Core Outcome Set to Evaluate Physical Activity Interventions for People Living With Dementia

Background and Objectives

A core outcome set (COS) to evaluate physical activity (PA) interventions for people living with dementia is needed, as the development of guidelines is currently limited by important heterogeneity in this field of research. Development of COS often includes Delphi surveys, but people living with dementia are often excluded. This study aimed to reach consensus on this COS using a modified Delphi survey to enable the participation of people living with dementia.

Research Design and Methods

Two stakeholders groups took part in a Delphi survey (Group 1: people living with dementia and family caregivers; Group 2: professionals from different backgrounds, including physiotherapists, occupational therapists, and researchers). Caregivers and professionals completed the survey remotely. Participants living with dementia took part face-to-face, using a card sorting strategy. The consensus process was finalized with a consensus meeting.

Results

Ninety-five participants of both groups completed the modified Delphi. Of those, 11 attended the consensus meeting. The card sorting strategy was successful at including people living with dementia. Seven outcomes reached consensus: preventing falls; doing what you can do; staying healthy and fit; walking better, being able to stand up and climb stairs; feeling brighter; enjoying the moment; and, feeling useful and having a purpose.

Discussion and Implications

Robust and innovative methodological strategies were used to reach a consensus on a COS (what to measure) to evaluate PA for people living with dementia. Future work will focus on the selection of the most appropriate tools to measure these outcomes (how to measure).

Industry funding was associated with increased use of core outcome sets

OBJECTIVES

The objective of this study was to assess the uptake of the rheumatoid arthritis core outcome set (RA-COS) using data from multiple data providers, and to investigate factors that may influence this uptake.

STUDY DESIGN AND SETTING

An observational review was carried out on all clinical trials of rheumatoid arthritis that were indexed on the World Health Organization-International Clinical Trials Registry Platform (WHO-ICTRP). Various measures of COS uptake were calculated from information presented in the trial registries and trial publications. Logistic regression was conducted to investigate factors thought to be associated with planned COS uptake.

RESULTS

A total of 341 trials were eligible for the evaluation of RA-COS uptake. In the decade leading up to 2019, the assessment of uptake based on published results was maintained at just over 80%. Trials that were not commercially funded were much less likely to plan to measure the RA-COS than those with industry funding (60% vs. 80%; adjusted OR 0.18, 95% CI 0.11 to 0.32; P < 0.001).

CONCLUSION

This study has demonstrated that the use of the WHO-ICTRP can identify a large and geographically diverse range of trials to include in the evaluation of COS uptake.

Improvement was needed in the standards of development for cancer core outcome sets

OBJECTIVE

The Core Outcome Set-STAndards for Development (COS-STAD) contains 11 standards (12 criteria) that are deemed to be the minimum design recommendations for all core outcome set (COS) development projects. Cancer is currently the disease area with the highest number of published COSs and is a major cause of worldwide morbidity and mortality. The aim of this study was to provide a baseline of cancer COS standards.

STUDY DESIGN AND SETTING

Systematic reviews of COSs have identified 307 published COS studies. Cancer COSs were eligible for inclusion. Two reviewers independently assessed each of the COSs against the 12 criteria.

RESULTS

Forty-nine cancer COSs were included; none met all 12 criteria representing the 11 minimum standards assessed in this study (range = 4-11 criteria, median = 6 criteria). All studies met the four scope standards, eight (16%) met all three standards for stakeholders involved, and two (4%) met all four standards for consensus process standards.

CONCLUSION

With the exception of "scope" specification, there is much need for improvement. Poor reporting often made it challenging to assess whether minimum standards were met. The consensus process criteria were most difficult to assess, particularly those that required an assessment of being a priori. This is the first application of COS-STAD criteria to studies that have developed COSs and provides a baseline of cancer COS standards of development.

Evaluation of an automatic article selection method for timelier updates of the Comet Core Outcome Set database

Curated databases of scientific literature play an important role in helping researchers find relevant literature, but populating such databases is a labour intensive and time-consuming process. One such database is the freely accessible Comet Core Outcome Set database, which was originally populated using manual screening in an annually updated systematic review. In order to reduce the workload and facilitate more timely updates we are evaluating machine learning methods to reduce the number of references needed to screen. In this study we have evaluated a machine learning approach based on logistic regression to automatically rank the candidate articles. Data from the original systematic review and its four first review updates were used to train the model and evaluate performance. We estimated that using automatic screening would yield a workload reduction of at least 75% while keeping the number of missed references around 2%. We judged this to be an acceptable trade-off for this systematic review, and the method is now being used for the next round of the Comet database update.

Selecting Core Outcomes for Randomised Effectiveness trials In Type 2 diabetes (SCORE-IT): a patient and healthcare professional consensus on a core outcome set for type 2 diabetes

OBJECTIVES

Heterogeneity in outcomes measured across trials of glucose-lowering interventions for people with type 2 diabetes impacts on the ability to compare findings and may mean that the results have little importance to healthcare professionals and the patients that they care for. The SCORE-IT study (Selecting Core Outcomes for Randomised Effectiveness trials In Type 2 diabetes) has addressed this issue by establishing consensus on the most important outcomes for non-surgical interventions for hyperglycemia in type 2 diabetes.

RESEARCH DESIGN AND METHODS

A comprehensive list of outcomes was developed from registered clinical trials, online patient resources, qualitative literature and long-term studies in the field. This list was then scored in a two-round online Delphi survey completed by healthcare professionals, people with type 2 diabetes, researchers in the field and healthcare policymakers. The results of this online Delphi were discussed and ratified at a face-to-face consensus meeting.

RESULTS

173 people completed both rounds of the online survey (116 people with type 2 diabetes, 37 healthcare professionals, 14 researchers and 6 policymakers), 20 of these attended the consensus meeting (13 people with type 2 diabetes and 7 healthcare professionals). Consensus was reached on 18 core outcomes across five domains, which include outcomes related to diabetes care, quality of life and long-term diabetes-related complications.

CONCLUSIONS

Implementation of the core outcome set in future trials will ensure that outcomes of importance to all stakeholders are measured and reported, enhancing the relevance of trial findings and facilitating the comparison of results across trials.