Increasing uptake through collaboration in the development of core outcome sets: Lessons learned at OMERACT 2023

OBJECTIVE

This manuscript highlights the importance of enhancing the uptake of Core Outcome Sets (COS) by building partnerships with Collaborators and addressing their needs in COS development.

METHODS AND SETTING

This session was structured as a simulation, resembling a format akin to a classic television game show. The moderator posed a series of questions to eight different Collaborator groups who briefly described the importance of COS within their areas of interest. Previous studies examining the uptake of individual core outcomes revealed disparities in uptake rates. The Identified barriers to the uptake of COS include the lack of recommendations for validated instruments for each domain, insufficient involvement of patients and key Collaborator groups in COS development, and a lack of awareness regarding the existence of COS.

CONCLUSIONS

This analysis underscores the need for COS development approaches that prioritize the inclusion of patients and diverse Collaborator groups at every stage. While current studies on COS uptake are limited, future research should explore the broader implementation of COS across diverse disease categories and delve into the factors that hinder or facilitate their uptake such as, the importance of COS developers extending their work to recommending domains with well validated instruments. Embracing patient leadership and multifaceted engagement is essential for advancing the relevance and impact of COS in clinical research.

Defining domains: developing consensus-based definitions for foundational domains in OMERACT core outcome sets

OBJECTIVE

To develop a set of detailed definitions for foundational domains commonly used in OMERACT (Outcome Measures in Rheumatology) core domain sets.

METHODS

We identified candidate domain definitions from prior OMERACT publications and websites and publications of major organizations involved in outcomes research for six domains commonly used in OMERACT Core Domain Sets: pain intensity, pain interference, physical function, fatigue, patient global assessment, and health-related quality of life. We conducted a two-round survey of OMERACT working groups, patient research partners, and then the OMERACT Technical Advisory Group to establish their preferred domain definitions. Results were presented at the OMERACT 2023 Methodology Workshop, where participants discussed their relevant lived experience and identified potential sources of variability giving the needed detail in our domain definitions.

RESULTS

One-hundred four people responded to both rounds of the survey, and a preferred definition was established for each of the domains except for patient global assessment for which no agreement was reached. Seventy-five participants at the OMERACT 2023 Methodology Workshop provided lived experience examples, which were used to contextualise domain definition reports for each of the five domains.

CONCLUSION

Using a consensus-based approach, we have created a detailed definition for five of the foundational domains in OMERACT core domain sets; patient global assessment requires further research. These definitions, although not mandatory for working groups to use, may facilitate the initial domain-match assessment step of instrument selection, and reduce the time and resources required by future OMERACT groups when developing core outcome sets.

Embracing unity at OMERACT: Valuing equity, promoting diversity, fostering inclusivity

OBJECTIVE

To increase awareness and understanding of the principles of Equity, Diversity, and Inclusivity (EDI) within Outcome Measures in Rheumatology's (OMERACT) members. For this, we aimed to obtain ideas on how to promote and foster these principles within the organization and determine the diversity of the current membership in order to focus future efforts.

METHODS

We held a plenary workshop session at OMERACT 2023 with roundtable discussions on barriers and solutions to increased diversity within OMERACT. We conducted an anonymous, web-based survey of members to record characteristics including population group, gender identity, education level, age, and ability.

RESULTS

The workshop generated ideas to increase diversity of participants across the themes of building relationships [12 topics], materials and methods [5 topics], and conference-specific [6 topics]. Four hundred and seven people responded to the survey (25 % response rate). The majority of respondents were White (75 %), female (61 %), university-educated (94 %), Christian (42 %), spoke English at home (60 %), aged 35 to 55 years (50 %), and did not report a disability (64 %).

CONCLUSION

OMERACT is committed to improving its diversity. Next steps include strategic recruitment of members to the EDI working group, drafting an EDI mission statement centering equity and inclusivity in the organization, and developing guidance for the OMERACT Handbook to help all working groups create actionable plans for promoting EDI principles.

Navigating the path of progress: The OMERACT 2023 emerging leaders program

OBJECTIVES

The Outcome Measures in Rheumatology Clinical Trials (OMERACT) Emerging Leaders Program (ELP) aims to cultivate a cohort of skilled leaders within the OMERACT community empowering them with expertise and knowledge to help shape and steer the organization into the future. This publication highlights the significance of the ELP in driving leadership excellence, its impact on OMERACT's evolution, and the outcomes and learnings from the OMERACT 2023 ELP.

METHODS

Insights from the 2018 ELP report informed 2023 program improvements. Engagement was measured by attendance and WhatsApp interactions. Positive program aspects, areas for improvement and ideas for enhancing future ELPs were captured via anonymous survey and participant focus groups.

RESULTS

Engagement with the ELP was high with 9 participants, 96 % attendance at all workshops, 154 WhatsApp interactions. All program components were highly rated, with the highest being the 'Psychological Safety' and 'Methodology/Process/Politics' workshops. Future enhancements included creating further networking, connection and support activities, practical leadership and methodological skill development opportunities, and a new stream focussing on organisational advancement.

CONCLUSIONS

The 2023 OMERACT ELP was well received and successfully addressed areas previously identified as requiring improvement. New educational enhancements were valued, and the importance of fostering psychological safety at all levels was highlighted. The ELP fortifies OMERACT by nurturing a diverse array of skilled leaders who embody OMERACTs core values. Continuing to refine and evolve the ELP over time will help OMERACT sustain its global influence in patient-centered outcome research.

Evolving and evaluating the OMERACT fellows program: insights and implications from OMERACT 2023 fellows

OBJECTIVE

To describe the evolution of the OMERACT Fellows Program (OM FP) and to evaluate the innovative changes implemented in the 2023 program.

METHODS

The OM FP, the first of its kind in global rheumatology, was developed in 2000 to mentor early career researchers in methods and processes for reaching evidence-driven consensus for outcome measures in clinical studies. The OM FP has evolved through continuing iterations of face to face and online feedback. Key new features delivered in 2023 included e-learning modules, virtual introductory pre-meetings, increased networking with Patient Research Partners (PRPs), learning opportunities to give and receive personal feedback, ongoing performance feedback during the meeting from Fellow peers, PRPs, senior OMERACTers (members of the OMERACT community) and Emerging Leader mentors, involvement in pitching promotions, two-minute Lightning Talks in a plenary session and an embedded poster tour. An online survey was distributed after the meeting to evaluate the program.

RESULTS

OM FP has included 208 fellows from 16 countries across 4 continents covering 47 different aspects of rheumatology outcomes since its inception. Over 50 % have remained engaged with OMERACT work. In 2023, 18 Fellows attended and 15 (83 %) completed the post-meeting survey. A dedicated OM FP was deemed important by all respondents, and 93 % would attend the meeting in future. The PRP/Fellow Connection Carousel and Lightning Talks were rated exceptional by 93 %. Key components to improve included clarification of expectations, overall workload, the Emerging Leaders Mentoring Program, and the content and duration of daily summary sessions.

CONCLUSION

The innovations in the 2023 OM FP were well received by the majority of participants and supports early career rheumatology researchers to develop collaborations, skills and expertise in outcome measurement. Implementation of feedback from Fellows will enhance the program for future meetings, continuing to facilitate learning and succession planning within OMERACT.

Patient outcomes in longitudinal observational studies (POLOS) of rheumatoid arthritis: Determining the OMERACT core domain set

OBJECTIVE

To define and select rheumatoid arthritis (RA)-specific core domain set for Longitudinal Observational Studies (LOS) within the Outcome Measures in Rheumatology (OMERACT) framework.

METHODS

A three-round online Delphi exercise, including patient research partners (PRPs) and other community partners in healthcare, was conducted. Domains scored 7-9 (i.e., critically important to include) by ≥ 70 % of participants in both groups were included. Items were consolidated in a subsequent dedicated meeting.

RESULTS

Nineteen domains scored ≥ 70 % consensus in both groups. The focus group refined these into a list of twelve domains.

CONCLUSION

The achieved consensus will inform the next steps of developing the core domain set for LOS in RA.

Composite outcomes at OMERACT: Multi-outcome domains and composite outcome domains

The OMERACT Technical Advisory Group recognises that working groups during the process of creating a core outcome set may identify an outcome domain that would be best represented as a composite that encapsulates these component outcome domains by bringing them together into a single outcome. A multi-outcome domain (MOD) is a within-patient combination of component outcomes, and an individual patient's evaluation depends on the observation of all of the components in that patient with a single overall rating determined according to a specified rule; which is often applicable when we consider a disease activity score. A composite outcome domain (COD) consists of a number of component outcomes and is defined as the occurrence in a patient of one, some or all of these specified components; which is often applicable when we consider the risk of adverse events or remission criteria. We review the general benefits, challenges, reporting and interpretation of using MODs and CODs. The development of the MOD or COD instrument for an OMERACT core outcome measurement set is considered through four distinct steps: choosing relevant outcome domains; finding high quality instruments for each of these outcome domains; weighting the outcome domain instruments in the MOD/COD instrument; and putting MOD/COD instrument through the OMERACT Filter. Guidance and training are in preparation for working groups who will be completing the OMERACT Instrument Selection Algorithm (OFISA). As for other initiatives in OMERACT, we will seek feedback from OMERACT working groups who complete the development of their MOD/COD, which will then be incorporated into the refinement of the guidance and training.

The evolution of instrument selection for inclusion in core outcome sets at OMERACT: Filter 2.2

INTRODUCTION

OMERACT uses an evidence-based framework known as the 'OMERACT Filter Instrument Selection Algorithm' (OFISA) to guide decisions in the assessment of outcome measurement instruments for inclusion in a core outcome set for interventional and observational clinical trials.

METHODS

A group of OMERACT imaging and patient-centered outcome methodologists worked with imaging outcome groups to facilitate the selection of imaging outcome measurement instruments using the OFISA approach. The lessons learned from this work influenced the evolution to Filter 2.2 and necessitated changes to OMERACT's documentation and processes.

RESULTS

OMERACT has revised documentation and processes to incorporate the evolution of instrument selection to Filter 2.2. These revisions include creation of a template for detailed definitions of the target domain which is a necessary first step for instrument selection, modifications to the Summary of Measurement Properties (SOMP) table to account for sources of variability, and development of standardized reporting tables for each measurement property.

CONCLUSIONS

OMERACT Filter 2.2 represents additional modifications of the OMERACT guide for working groups in their rigorous assessment of measurement properties of instruments of various types, including imaging outcome measurement instruments. Enhanced reporting aims to increase the transparency of the evidence base leading to judgements for the endorsement of instruments in core outcome sets.

Core Domain Set Selection According to OMERACT Filter 2.1: The OMERACT Methodology

Objective.

To describe the Outcome Measures in Rheumatology (OMERACT) Filter 2.1 methodology for core domain set selection.

Methods.

The “OMERACT Way for Core Domain Set selection” framework consists of 3 stages: first, generating candidate domains through literature reviews and qualitative work, then a process of consensus to obtain agreement from those involved, and finally formal voting on the OMERACT Onion. The OMERACT Onion describes the placement of domains in layers/circles: mandatory in all trials/mandatory in specific circumstances (inner circle); important but optional (middle circle); or research agenda (outer circle). Five OMERACT working groups presented their core domain sets for endorsement by the OMERACT community. Tools including a workbook and whiteboard video were created to assist the process. The methods workshop at OMERACT 2018 introduced participants to this framework.

Results.

The 5 OMERACT working groups achieved consensus on their proposed core domain sets. After the Methodology Workshop training exercise at OMERACT 2018, over 90% of participants voted that they were confident that they understood the process of core domain set selection.

Conclusion.

The methods described in this paper were successfully used by the 5 working groups voting on domains at the OMERACT 2018 meeting, demonstrating the feasibility of the process. In addition, participants at OMERACT 2018 expressed increased confidence and understanding of the core domain set selection process after the training exercise. This methodology will continue to evolve, and we will use innovative technology such as whiteboard videos as a key part of our dissemination and implementation strategy for new methods.

Consensus Building in OMERACT: Recommendations for Use of the Delphi for Core Outcome Set Development

Objective.

Developing international consensus on outcome measures for clinical trials is challenging. The following paper will review consensus building in Outcome Measures in Rheumatology (OMERACT), with a focus on the Delphi.

Methods.

Based on the literature and feedback from delegates at OMERACT 2018, a set of recommendations is provided in the form of the OMERACT Delphi Consensus Checklist.

Results.

The OMERACT delegates generally supported the use of the checklist as a guide. The checklist provides guidance for clearly outlining the multiple aspects of the Delphi process.

Conclusion.

OMERACT is deeply committed to consensus building and these recommendations should be considered a work in progress.

Instrument Selection Using the OMERACT Filter 2.1: The OMERACT Methodology

Objective.

Outcome Measures in Rheumatology (OMERACT) Filter 2.1 revised the process used for core outcome measurement set selection to add rigor and transparency in decision making. This paper describes OMERACT’s methodology for instrument selection.

Methods.

We presented instrument selection processes, tools, and reporting templates at OMERACT 2018, introducing the concept of “3 pillars, 4 questions, 7 measurement properties, 1 answer.” Truth, discrimination, and feasibility are the 3 original OMERACT pillars. Based on these, we developed 4 signaling questions. We introduced the Summary of Measurement Properties table that summarizes the 7 measurement properties: truth (domain match, construct validity), discrimination [test-retest reliability, longitudinal construct validity (responsiveness), clinical trial discrimination, thresholds of meaning], and feasibility. These properties address a set of standards which, when met, answer the one question: Is there enough evidence to support the use of this instrument in clinical research of the benefits and harms of treatments in the population and study setting described? The OMERACT Filter 2.1 was piloted on 2 instruments by the Psoriatic Arthritis Working Group.

Results.

The methodology was reviewed in a full plenary session and facilitated breakout groups. Tools to facilitate retention of the process (i.e., “The OMERACT Way”) were provided. The 2 instruments were presented, and the recommendation of the working group was endorsed in the first OMERACT Filter 2.1 Instrument Selection votes.

Conclusion.

Instrument selection using OMERACT Filter 2.1 is feasible and is now being implemented.

The Third Biennial 2018 OMERACT First-time Participant Program: A Qualitative and Quantitative Study

OBJECTIVE

To assess the expanded/refined first-time participant training program.

METHODS

We conducted a refined new participant program at OMERACT 2018 on days 1-4, in which first-time participants provided feedback with online surveys and a nominal group on Day 4.

RESULTS

Twenty first-time participants attended the introductory session and 8-12 attended followup sessions. A high proportion valued the newbie session (100%), rating it overall (91%), content-wise (62%), for presentation quality (82%), and value for the money (82%) as outstanding or good. The nominal group technique identified opportunities for further improvement of breakouts/voting.

CONCLUSION

The expanded new participant training program is valued by attendees.

Improving Benefit-harm Assessment of Therapies from the Patient Perspective: OMERACT Premeeting Toward Consensus on Core Sets for Randomized Controlled Trials

Objective.

Outcome Measures in Rheumatology (OMERACT) convened a premeeting in 2018 to bring together patients, regulators, researchers, clinicians, and consumers to build upon previous OMERACT drug safety work, with patients fully engaged throughout all phases.

Methods.

Day 1 included a brief introduction to the history of OMERACT and methodology, and an overview of current efforts within and outside OMERACT to identify patient-reported medication safety concerns. On Day 2, two working groups presented results; after each, breakout groups were assembled to discuss findings.

Results.

Five themes pertaining to drug safety measurement emerged.

Conclusion.

Current approaches have failed to include data from the patient’s perspective. A better understanding of how individuals with rheumatic diseases view potential benefits and harms of therapies is essential.

The OMERACT First-time Participant Program: Fresh Eye from the New Guys

OBJECTIVE

To describe the experience of the first-time participant (newbie) training program at the Outcome Measures in Rheumatology (OMERACT) 2016 meeting.

METHODS

We conducted new participant sessions at OMERACT 2016, including a 2-h introductory session on Day 1 followed by 1-h evening followup sessions on days 1-4. Pre- and post-meeting surveys assessed participants' levels of comfort with the principles of the OMERACT Filter 2.0 (the essential tools for OMERACT methodology) and the different types of OMERACT sessions, and whether participants felt welcome. In addition, on the final day, a nominal group technique was used to elicit problematic components of the meeting and to develop solutions to those problems.

RESULTS

Of the 43 new attendees, 38 participated in the introductory session and 14-18 attended the followup sessions. Comparing Day 1 (preintroductory session) to days 1-3 (post), a similar proportion understood different types of sessions extremely well [45% (pre) versus 47%, 44%, and 36% (post), respectively], and a higher proportion understood principles of the OMERACT filter extremely well [22% (pre) versus 55%, 44%, and 40% (post), respectively]. Most reported feeling welcome (86.7%) and felt they contributed to breakout sessions (93.3%) on the evening of Day 1; results were sustained on days 2-3. The most commonly reported "best" experience included the OMERACT culture and the most common reported experience needing improvement included facilitation issues during breakouts.

CONCLUSION

The first-time participants came to OMERACT 2016 with a high baseline level of understanding. They rapidly attained a high comfort level with participation and provided concrete and innovative solutions to the most commonly reported experiences needing improvement.

Current State of Reporting Pain Outcomes in Cochrane Reviews of Chronic Musculoskeletal Pain Conditions and Considerations for an OMERACT Research Agenda

OBJECTIVE

To assess the current state of reporting of pain outcomes in Cochrane reviews on chronic musculoskeletal painful conditions and to elicit opinions of patients, healthcare practitioners, and methodologists on presenting pain outcomes to patients, clinicians, and policymakers.

METHODS

We identified all reviews in the Cochrane Library of chronic musculoskeletal pain conditions from Cochrane review groups (Back, Musculoskeletal, and Pain, Palliative, and Supportive Care) that contained a summary of findings (SoF) table. We extracted data on reported pain domains and instruments and conducted a survey and interviews on considerations for SoF tables (e.g., pain domains, presentation of results).

RESULTS

Fifty-seven SoF tables in 133 Cochrane reviews were eligible. SoF tables reported pain in 56/57, with all presenting results for pain intensity (20 different outcome instruments), pain interference in 8 SoF tables (5 different outcome instruments), and pain frequency in 1 multiple domain instrument. Other domains like pain quality or pain affect were not reported. From the survey and interviews [response rate 80% (36/45)], we derived 4 themes for a future research agenda: pain domains, considerations for assessing truth, discrimination, and feasibility; clinically important thresholds for responder analyses and presenting results; and establishing hierarchies of outcome instruments.

CONCLUSION

There is a lack of standardization in the domains of pain selected and the manner that pain outcomes are reported in SoF tables, hampering efforts to synthesize evidence. Future research should focus on the themes identified, building partnerships to achieve consensus and develop guidance on best practices for reporting pain outcomes.

The Spirit of OMERACT: Q Methodology Analysis of Conference Characteristics Valued by Delegates

OBJECTIVE

To identify the major features of OMERACT meetings as valued by frequent participants and to explore whether there are groups of participants with different opinions.

METHODS

Using Q methodology (a qualitative and quantitative approach to grouping people according to subjective opinion), participants (who attended more than 1 OMERACT conference) sorted 66 statements relating to the "spirit of OMERACT" according to level of agreement across a normal distribution grid. Data were examined using Q factor analysis.

RESULTS

Of 226 potential participants, 105 responded (46%). All participants highly ranked the focus on global standardization of methods, outcome measures, data-driven research, methodological discussion, and international collaboration. Four factors describing the "spirit of OMERACT" were identified: "Evidence not eminence" (n = 31) valued the data- and evidence-driven research above personality and status; "Collaboration and collegiality" (n = 19) valued the international and cross-stakeholder collaboration, interaction, and collegiality; "Equal voices, equal votes, common goals" (n = 12) valued equality in discussion and voting, with everyone striving toward the same goal; "principles and product, not process" (n = 8) valued the principles of focusing on outcome measures and the product of guiding clinical trials, but were unsure whether the process is necessary to reach this. The factors did not segregate different stakeholder groups.

CONCLUSION

Delegates value different elements of OMERACT, and thus the "spirit of OMERACT" encompasses evidence-based research, collaboration, and equality, although a small group are unsure whether the process is necessary to achieve the end result. Q methodology may prove useful for conference organizers to identify their delegates' different needs to tailor conference content.

The OMERACT First-time Participant ("Newbie") Program: Initial Assessment and Lessons Learned

OBJECTIVE

To describe the experience of a first-time participant ("newbie") training program at the Outcome Measures in Rheumatology 12 meeting in 2014.

METHODS

We conducted newbie sessions at OMERACT 12, including a 2-hour introductory session on Day 1, followed by 1-h evening followup sessions on days 1-4 of OMERACT 12. Pre- and postmeeting surveys assessed participants' level of comfort with the principles of the OMERACT Filters 1.0 (truth, discrimination, feasibility), and Filter 2.0 (the essential tools for OMERACT methodology), the different types of OMERACT sessions, and whether participants felt welcome.

RESULTS

In all, 25 new attendees participated in the introductory session and 10-16 attended followup sessions. Fewer participants reported being somewhat or extremely uncomfortable with the meeting, comparing Day 1 (preintroductory session) to days 1-4 (post): (1) with different OMERACT sessions: 56% (pre) versus 6%, 0%, 8%, and 6% (post days 1-4, respectively); and (2) with principles of the OMERACT filter, 64% (pre) versus 7%, 0%, 8%, and 0% (post), respectively. Most reported feeling welcome (100%) and that they were able to contribute substantively to breakout sessions (87%) on Day 1 evening; results were sustained on days 2-4.

CONCLUSION

First-time participant training sessions increased the comfort level of the participants with the OMERACT meeting structure and filter, and increased the ability of the new attendees to feel they could contribute to the OMERACT process.