Participating in core outcome set development via Delphi surveys: qualitative interviews provide pointers to inform guidance

A standard set of outcome measures for the comprehensive assessment of oral health and occlusion in individuals with osteogenesis imperfecta

BACKGROUND

Osteogenesis imperfecta (OI) is a group of inherited connective tissue disorders of varying severity characterized by bone fragility. The primary objective of this international multidisciplinary collaboration initiative was to reach a consensus for a standardized set of clinician and patient-reported outcome measures, as well as associated measuring instruments for dental care of individuals with OI, based on the aspects considered important by both experts and patients. This project is a subsequent to the Key4OI project initiated by the Care4BrittleBones foundation which aims to develop a standard set of outcome measures covering a large domain of factors affecting quality of life for people with OI. An international team of experts comprising orthodontists, pediatric dentists, oral and maxillofacial surgeons, and prosthetic dentists used a modified Delphi consensus process to select clinician-reported outcome measures (CROMs) and patient-reported outcome measures (PROMs) to evaluate oral health in individuals with OI. Important domains were identified through a literature review and by professional expertise (both CROMs and PROMs). In three focus groups of individuals with OI, important and relevant issues regarding dental health were identified. The input from the focus groups was used as the basis for the final set of outcome measures: the selected issues were attributed to relevant CROMs and, when appropriate, matched with validated questionnaires to establish the final PROMs which represented best the specific oral health-related concerns of individuals with OI.

RESULTS

Consensus was reached on selected CROMs and PROMs for a standard set of outcome measures and measuring instruments of oral health in individuals with OI.

CONCLUSIONS

Our project resulted in consensus statements for standardization oral health PROMs and CROMs in individuals with OI. This outcome set can improve the standard of care by incorporating recommendations of professionals involved in dental care of individuals with OI. Further, it can facilitate research and international research co-operation. In addition, the significant contribution of the focus groups highlights the relevance of dental and oral health-related problems of individuals with OI.

The NeoPACE study: study protocol for the development of a core outcome set for neonatal palliative care

BACKGROUND

Neonatal death is the leading category of death in children under the age of 5 in the UK. Many babies die following decisions between parents and the neonatal team; when a baby is critically unwell, with the support of healthcare professionals, parents may make the decision to stop active treatment and focus on ensuring their baby has a 'good' death. There is very little evidence to support the clinical application of neonatal palliative care and/or end-of-life care, resulting in variation in clinical provision between neonatal units. Developing core outcomes for neonatal palliative care would enable the development of measures of good practice and enhance our care of families. The aim of this study is to develop a core outcome set with associated tools for measuring neonatal palliative care.

METHOD

This study has four phases: (1) identification of potential outcomes through systematic review and qualitative interviews with key stakeholders, including parents and healthcare professionals (2) an online Delphi process with key stakeholders to determine core outcomes (3) identification of outcome measures to support clinical application of outcome use (4) dissemination of the core outcome set for use across neonatal units in the UK. Key stakeholders include parents, healthcare professionals, and researchers with a background in neonatal palliative care.

DISCUSSION

Developing a core outcome set will standardise minimum reported outcomes for future research and quality improvement projects designed to determine the effectiveness of interventions and clinical care during neonatal palliative and/or end-of-life care. The core outcome set will provide healthcare professionals working in neonatal palliative and/or end-of-life support with an increased and consistent evidence base to enhance practice in this area.

TRIAL REGISTRATION

The study has been registered with the COMET initiative ( https://www.comet-initiative.org/Studies/Details/1470 ) and the systematic review is registered with the International Prospective Register of Systematic Reviews (PROSPERO) (CRD42023451068).

Core outcome set for pulmonary rehabilitation of patients with COPD: results of a modified Delphi survey

INTRODUCTION

There is high heterogeneity of outcomes and measures reported in pulmonary rehabilitation (PR) trials of people with chronic obstructive pulmonary disease (COPD). This hinders study comparability and benchmarking of PR. We have developed a core outcome set (COS) to overcome these challenges.

METHODS

This study was informed by a systematic review and two qualitative studies and had patient involvement since its inception. A two-round Delphi survey was available in seven languages. Outcomes (n=63) scored 7-9 (crucial) by ≥70% of the participants and 1-3 (not that important) by ≤15% of participants from both groups in the Likert scale were automatically included in the COS, while outcomes that were considered crucial by only one of the groups were further discussed by the authors in a meeting.

RESULTS

A total of 299 people (n=229 healthcare professionals/researchers/policy-makers; n=70 people with COPD and informal caregivers) participated in the survey (83% retention), which covered 29 countries/five continents. After the second round, six outcomes were included and three were added in the meeting. The final COS contains dyspnoea, fatigue, functional exercise capacity, health-related quality of life, health behaviours/lifestyle, knowledge about the disease, lower limb muscle function, personal goals and problematic activities of daily living.

CONCLUSION

A COS for PR of people with COPD is now available and can be used by different stakeholders to improve consistency and comparability of studies, benchmark PR and improve the quality of care provided. Future research should establish the core measures and investigate the uptake of this COS.

Developing a Core Outcome Set for Assessing Clinical Safety Outcomes of Prostate Cancer in Clinical Trials of Traditional Chinese Medicine: Protocol for a Mixed Methods Study

BACKGROUND

Among the common malignant tumors in men worldwide, the incidence of prostate cancer ranks second to lung cancer. This disease will bring an economic burden to patients and their families and can reduce the quality of life of patients. Researchers have conducted numerous clinical trials on the efficacy and safety of different interventions in the treatment of prostate cancer with traditional Chinese medicine (TCM) combined with standard treatment regimens. However, the currently published clinical trials exhibit inconsistent and irregular reporting of outcome measures.

OBJECTIVE

The objective of this paper is to emphasize the need for a core outcome set (COS) to facilitate future prostate cancer research, aiming to improve the quality of trials and generate high-quality evidence.

METHODS

This mixed methods project has three phases, as follows: (1) a scoping review of the literature to identify outcomes that have been reported in clinical trials and systematic reviews of interventions involving TCM for the treatment of prostate cancer as well as a qualitative component using interviews to obtain the views of patients with prostate cancer, their families, and their caregivers who have a history of TCM treatment; (2) a Delphi survey among stakeholders to prioritize the core outcomes-Participants will include traditional Chinese and Western medicine clinicians in prostate cancer-related directions, nurses, and methodology experts who will participate in 2 rounds of the Delphi method expert consultation to score each outcome in the list of outcome indicators; and (3) a face-to-face consensus meeting to discuss and agree on the final COS for the application of TCM in the treatment of prostate cancer.

RESULTS

The protocol has been registered in PROSPERO (CRD42022356184) before the start of the review process, and we will initiate the review on August 1, 2023; results should be expected by September 1, 2023. The Delphi survey among stakeholders is expected to start in October 2023.

CONCLUSIONS

The development of a core outcome set for assessing clinical safety outcomes of prostate cancer in clinical trials of TCM will provide a significant first step to assist Chinese doctors, researchers, and policy makers.

TRIAL REGISTRATION

PROSPERO CRD42022356184; https://tinyurl.com/ysakz74r.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/46794.

Multi-Round versus Real-Time Delphi survey approach for achieving consensus in the COHESION core outcome set: a randomised trial

BACKGROUND

Delphi surveys are commonly used to prioritise critical outcomes in core outcome set (COS) development. This trial aims to compare a three-round (Multi-Round) Delphi (MRD) with a Real-Time Delphi (RTD) in the prioritisation of outcomes for inclusion in a COS for neonatal encephalopathy treatments and explore whether 'feedback', 'iteration', and 'initial condition' effects may occur in the two survey methods.

METHODS

We recruited 269 participants (parents/caregivers, healthcare providers and researchers/academics) of which 222 were randomised to either the MRD or the RTD. We investigated the outcomes prioritised in each survey and the 'feedback', 'iteration', and 'initial condition' effects to identify differences between the two survey methods.

RESULTS

In the RTD, n = 92 participants (83%) fully completed the survey. In the MRD, n = 60 participants (54%) completed all three rounds. Of the 92 outcomes presented, 26 (28%) were prioritised differently between the RTD and MRD. Significantly fewer participants amended their scores when shown stakeholder responses in the RTD compared to the MRD ('feedback effect'). The 'iteration effect' analysis found most experts appeared satisfied with their initial ratings in the RTD and did not amend their scores following stakeholder response feedback. Where they did amend their scores, ratings were amended substantially, suggesting greater convergence. Variance in scores reduced with subsequent rounds of the MRD ('iteration effect'). Whilst most participants did not change their initial scores in the RTD, of those that did, later recruits tended to align their final score more closely to the group mean final score than earlier recruits (an 'initial condition' effect).

CONCLUSION

The feedback effect differed between the two Delphi methods but the magnitude of this difference was small and likely due to the large number of observations rather than because of a meaningfully large difference. It did not appear to be advantageous to require participants to engage in three rounds of a survey due to the low change in scores. Larger drop-out through successive rounds in the MRD, together with a lesser convergence of scores and longer time to completion, indicate considerable benefits of the RTD approach.

TRIAL REGISTRATION

NCT04471103. Registered on 14 July 2020.

A systematic review of studies reporting the development of core outcome sets for use in routine care

OBJECTIVES

Core outcome sets (COS) represent the minimum health outcomes to be measured for a given health condition. Interest is growing in using COS within routine care to support delivery of patient-focused care. This review aims to systematically map COS developed for routine care to understand their scope, stakeholder involvement, and development methods.

METHODS

Medline (Ovid), Scopus, and Web of Science Core collection were searched for studies reporting development of COS for routine care. Data on scope, methods, and stakeholder groups were analyzed in subgroups defined by setting.

RESULTS

Screening 25,301 records identified 262 COS: 164 for routine care only and 98 for routine care and research. Nearly half of the COS (112/254, 44%) were developed with patients, alongside input from experts in registries, insurance, legal, outcomes measurement, and performance management. Research publications were often searched to generate an initial list of outcomes (115/198, 58%) with few searching routine health records (47/198, 24%).

CONCLUSION

An increasing number of COS is being developed for routine care. Although involvement of patient stakeholders has increased in recent years, further improvements are needed. Methodology and scope are broadly similar to COS for research but implementation of the final set is a greater consideration during development.

The importance of establishing a core outcome set for endodontic clinical trials and outcomes studies

Endodontic therapy aims to preserve teeth by preventing and treating apical disease, therefore, evaluation of treatment outcome in clinical trials and outcomes studies should effectively assess if it achieves these aims. Traditionally, treatment outcomes have been reported by clinicians after history and clinical examination in what is known as clinician-reported outcomes (CROs). Much less commonly employed however, are patient-reported outcomes (PROs) in which patients directly report on their condition. Endodontic treatment outcome reporting is evolving from a focus on CROs to increasing consideration for patient and disease-focused outcomes, with different criteria being proposed for assessment of treatment outcomes. Unfortunately, this has led to considerable variability and a lack of consensus on the definition, appropriate measurement and reporting of these outcomes. Heterogeneity in outcome reporting in clinical research provides a significant major barrier to conduct meta-analysis, guidelines development, clinical decision making, and ultimately affecting patient care. These effects could, however, be reduced by the establishment of a core outcome set (COS) in endodontics, which is defined as an agreed, standardized set of outcomes that should be included, measured and reported as a minimum in all trials and outcome studies. COS development is a regulated and validated process requiring involvement of appropriate stakeholders as well as a rigorous methodology. To date, COS has been developed for the management of traumatic dental injuries, orthodontic and periodontal treatment and is currently being developed for endodontic treatment. The aim of this review is to discuss the importance of COS in endodontics with focus on the evidence for and impact of heterogeneity in reporting endodontic treatment outcomes. An overview of an ongoing process for development of COS for different endodontic treatment modalities will also be provided.

The research status and progress of core outcome set in oral health

The core outcome set (COS) refers to the minimum set of outcomes that should be reported by all clinical trials in a particular health field. The use of COS in clinical studies can reduce the heterogeneity caused by using different outcomes across different clinical studies, facilitate the systematic review of different clinical studies on the same topic, reduce selective reporting bias, and increase the utility of clinical studies. The importance of COS in oral health has recently been recognized. This review summarizes the history, necessity, and key methodological points of COS development, with emphasis on the research status and existing problems in COS development, in the field of oral health.

Patient participation in Delphi surveys to develop core outcome sets: systematic review

OBJECTIVES

To describe the design and conduct of core outcome set (COS) studies that have included patients as participants, exploring how study characteristics might impact their response rates.

DESIGN

Systematic review of COS studies published between 2015 and 2019 that included more than one patient, carer or representative as participants (hereafter referred to as patients for brevity) in scoring outcomes in a Delphi.

RESULTS

There were variations in the design and conduct of COS studies that included patients in the Delphi process, including differing: scoring and feedback systems, approaches to recruiting patients, length of time between rounds, use of reminders, incentives, patient and public involvement, and piloting. Minimal reporting of participant characteristics and a lack of translation of Delphi surveys into local languages were found. Additionally, there were indications that studies that recruited patients through treatment centres had higher round two response rates than studies recruiting through patient organisations.

CONCLUSIONS

Variability was striking in how COS Delphi surveys were designed and conducted to include patient participants and other stakeholders. Future research is needed to explore what motivates patients to take part in COS studies and what factors influence COS developer recruitment strategies. Improved reporting would increase knowledge of how methods affect patient participation in COS Delphi studies.

A standard set of outcome measures for the comprehensive assessment of osteogenesis imperfecta

BACKGROUND

Osteogenesis Imperfecta (OI) is a genetic disorder also known as 'brittle bone disease'. The clinical manifestation of OI shows a wide variation. Therefore, care for patients with OI requires an interdisciplinary approach. The effectiveness of particular interventions and treatment protocols of interdisciplinary teams is not clear due to a non-standardized and wide variation of patient outcomes thus making the comparison of outcome measures available in the literature difficult. It is only by agreeing on a common, standard set of outcome measures for the comprehensive appraisal of OI that comparisons across interdisciplinary treatment centers for OI will be possible in the future.

METHODS

The Key4OI international interdisciplinary working group of 27 members used a consensus-driven modified Delphi approach to develop a set of global outcome measures for patients with OI. The International Classification of Functioning, Disability and Health (ICF), was used to define domains and organize the outcomes from the literature search. After reviewing the outcomes extracted from the literature, trials and registries, the working group agreed on a final selection of domains and their definition (ICF definition as well as a lay description). These domains were then presented to the focus groups who prioritized the outcome domains by taking into account the items important to the OI community. All content was collected and analyzed and final domains were determined. A consensus of appropriate measuring instruments for each domain was reached with Delphi rounds. The entire approach was in line with the International Consortium for Health Outcomes Measurement ICHOM methodology.

RESULTS

More than 400 different outcome measures were identified in our literature search. After three Delphi rounds, 24 domains were selected. After the focus group sessions, the number of domains were reduced to 15. A consensus was reached on the measuring instruments to cover these domains for both children and adults.

CONCLUSION

The Key4OI project resulted in standard set of outcome measures focused on the needs and wishes of individuals with OI and their families. This outcome set will enable healthcare teams and systems to compare and to improve their care pathways and quality of care worldwide. Further studies are needed to evaluate the implementation of this standardized outcome set.

Methodology in core outcome set (COS) development: the impact of patient interviews and using a 5-point versus a 9-point Delphi rating scale on core outcome selection in a COS development study

BACKGROUND

As the development of core outcome sets (COS) increases, guidance for developing and reporting high-quality COS continues to evolve; however, a number of methodological uncertainties still remain. The objectives of this study were: (1) to explore the impact of including patient interviews in developing a COS, (2) to examine the impact of using a 5-point versus a 9-point rating scale during Delphi consensus methods on outcome selection and (3) to inform and contribute to COS development methodology by advancing the evidence base on COS development techniques.

METHODS

Semi-structured patient interviews and a nested randomised controlled parallel group trial as part of the Pelvic Girdle Pain Core Outcome Set project (PGP-COS). Patient interviews, as an adjunct to a systematic review of outcomes reported in previous studies, were undertaken to identify preliminary outcomes for including in a Delphi consensus survey. In the Delphi survey, participants were randomised (1:1) to a 5-point or 9-point rating scale for rating the importance of the list of preliminary outcomes.

RESULTS

Four of the eight patient interview derived outcomes were included in the preliminary COS, however, none of these outcomes were included in the final PGP-COS. The 5-point rating scale resulted in twice as many outcomes reaching consensus after the 3-round Delphi survey compared to the 9-point scale. Consensus on all five outcomes included in the final PGP-COS was achieved by participants allocated the 5-point rating scale, whereas consensus on four of these was achieved by those using the 9-point scale.

CONCLUSIONS

Using patient interviews to identify preliminary outcomes as an adjunct to conducting a systematic review of outcomes measured in the literature did not appear to influence outcome selection in developing the COS in this study. The use of different rating scales in a Delphi survey, however, did appear to impact on outcome selection. The 5-point scale demonstrated greater congruency than the 9-point scale with the outcomes included in the final PGP-COS. Future research to substantiate our findings and to explore the impact of other rating scales on outcome selection during COS development, however, is warranted.

Core Outcome Sets for Studies of Diabetes in Pregnancy: A Review

Core Outcome Sets (COS) contain an agreed minimum set of outcomes to be measured and reported in all studies in a specific area, with the objective of standardizing outcome reporting. COS may minimize research waste by identifying outcomes important to key stakeholders, allowing for improved evidence synthesis, and facilitating translation of research findings to clinical practice. Over the past 5 years, there has been significant progress in developing COS relevant to studies of diabetes in pregnancy. This review summarizes work in this area, reviews the role of patient and public involvement in COS development, and suggests areas for future research.