Study protocol for developing, piloting and disseminating the PRISMA-COSMIN guideline: a new reporting guideline for systematic reviews of outcome measurement instruments

BACKGROUND

Systematic reviews of outcome measurement instruments are important tools in the evidence-based selection of these instruments. COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments) has developed a comprehensive and widespread guideline to conduct systematic reviews of outcome measurement instruments, but key information is often missing in published reviews. This hinders the appraisal of the quality of outcome measurement instruments, impacts the decisions of knowledge users regarding their appropriateness, and compromises reproducibility and interpretability of the reviews' findings. To facilitate sufficient, transparent, and consistent reporting of systematic reviews of outcome measurement instruments, an extension of the PRISMA (Preferred Reporting of Items for Systematic reviews and Meta-Analyses) 2020 guideline will be developed: the PRISMA-COSMIN guideline.

METHODS

The PRISMA-COSMIN guideline will be developed in accordance with recommendations for reporting guideline development from the EQUATOR (Enhancing the QUAlity and Transparency Of health Research) Network. First, a candidate reporting item list will be created through an environmental literature scan and expert consultations. Second, an international Delphi study will be conducted with systematic review authors, biostatisticians, epidemiologists, psychometricians/clinimetricians, reporting guideline developers, journal editors as well as patients, caregivers, and members of the public. Delphi panelists will rate candidate items for inclusion on a 5-point scale, suggest additional candidate items, and give feedback on item wording and comprehensibility. Third, the draft PRISMA-COSMIN guideline and user manual will be iteratively piloted by applying it to systematic reviews in several disease areas to assess its relevance, comprehensiveness, and comprehensibility, along with usability and user satisfaction. Fourth, a consensus meeting will be held to finalize the PRISMA-COSMIN guideline through roundtable discussions and voting. Last, a user manual will be developed and the final PRISMA-COSMIN guideline will be disseminated through publications, conferences, newsletters, and relevant websites. Additionally, relevant journals and organizations will be invited to endorse and implement PRISMA-COSMIN. Throughout the project, evaluations will take place to identify barriers and facilitators of involving patient/public partners and employing a virtual process.

DISCUSSION

The PRISMA-COSMIN guideline will ensure that the reports of systematic reviews of outcome measurement instruments are complete and informative, enhancing their reproducibility, ease of use, and uptake.

Establishing a core outcome set for neurogenic bladder trials: study protocol for a scoping review and Delphi surveys

Background

Neurogenic bladder (NGB) is a chronic and disabling condition with a high prevalence rate, which can cause economic burden on patients and their families and reduce the quality of life of patients. Researchers have carried out a large number of clinical trials on the effectiveness and safety of different interventions for the treatment of NGB. The published clinical trials of NGB generally suffered from inconsistent and irregular reporting of outcome indicators. To facilitate future research studies of NGB, a core outcome set (COS) is required, which helps translate the results into high-quality evidence.

Methods and analysis

This mixed-method project has four phases instrument: in phase 1, a scoping review of the literature to identify outcomes that have been reported in clinical trials and systematic reviews of clinical trials of interventions for NGB; in phase 2, a qualitative component using interviews to obtain the views of NGB patients, families, and their caregivers; in phase 3, Delphi survey among stakeholders to prioritize the core outcomes; and in phase 4, a face-to-face consensus meeting to discuss and agree on the final NBG COS.

Conclusions

We will develop a COS that should be reported in future clinical trials of NGB.

Trial registration

Core Outcome Measures in Effectiveness Trials (COMET) Initiative database registration: http://www.comet-initiative.org/studies/details/1985. Registered on 02 January 2022. INPLASY INPLASY202210007

New treatments and new assessment instruments for hidradenitis suppurativa

Research interest in Hidradenitis Suppurativa (HS) has grown exponentially over the past decades. Several groups have worked to develop novel scores that address the drawbacks of existing investigator assessed and patient reported outcome measures currently used in HS trials, clinical practice and research. In clinical trial settings, the drawbacks of the HiSCR have become apparent; mainly it's lack of a dynamic measurement of draining tunnels. The newly developed (dichotomous) IHS4 and HASI-R are backed up by adequate validation data and are good contenders to become the new primary outcome measure in HS clinical trials. Patient reported outcomes, as well as physician reported measures, are being developed by the HIdradenitis SuppuraTiva cORe outcomes set International Collaboration (HISTORIC). For example, the Hidradenitis Suppurativa Quality of Life (HiSQOL) score is a validated measure of HS-specific quality of life and is already being used in many HS trials. Magnitude of pain measurement via a 0-10 numerical rating scale is well-established, however consensus is still required to ensure consistent administration and interpretation of the instrument. A longitudinal measurement over multiple days rather than at one time point, such as for example the Pain Index could provide increased reliability and reduced recall bias. Ultimately, these newly developed scores and tools can be included in a standardized registry to be used in routine clinical practice.

Improving domain definition and outcome instrument selection: Lessons learned for OMERACT from imaging

OBJECTIVES

Imaging is one of the most rapidly evolving fields in medicine. Unfortunately, many imaging technologies have been applied as measurement instruments without rigorous evaluation of the evidence supporting their truth, discriminatory capability and feasibility for that context of use. The Outcome Measures in Rheumatology (OMERACT) Filter 2.1 Instrument Selection Algorithm (OFISA) is used to evaluate such evidence for use of an instrument in a research setting. The objectives of this work are to: [1] define and describe the key conceptual aspects that are essential for the evaluation of imaging as an outcome measurement instrument and [2] describe how these aspects can be assessed through OFISA.

METHODS

Experts in imaging and/or methodology met to formalize concepts and define key steps. These concepts were discussed with a team of patient research partners with interest in imaging to refine technical and methodological aspects into comprehensible information. A workshop was held at OMERACT2020 and feedback was incorporated into existing OMERACT process for domain and instrument selection.

RESULTS

Three key lessons were identified: (1) a clear definition of the domain we want to measure is a necessary prerequisite to the selection of a good instrument, (2) the sources of variability that can directly influence the instrument should be clearly identified, (3) incorporating these first two lessons into OFISA improves the quality of every instrument selection process.

CONCLUSIONS

The incorporation of these lessons in the updated OMERACT Filter (now 2.2) will improve the quality of the selection process for all types of outcome measurement instruments.

The evolution of instrument selection for inclusion in core outcome sets at OMERACT: Filter 2.2

INTRODUCTION

OMERACT uses an evidence-based framework known as the 'OMERACT Filter Instrument Selection Algorithm' (OFISA) to guide decisions in the assessment of outcome measurement instruments for inclusion in a core outcome set for interventional and observational clinical trials.

METHODS

A group of OMERACT imaging and patient-centered outcome methodologists worked with imaging outcome groups to facilitate the selection of imaging outcome measurement instruments using the OFISA approach. The lessons learned from this work influenced the evolution to Filter 2.2 and necessitated changes to OMERACT's documentation and processes.

RESULTS

OMERACT has revised documentation and processes to incorporate the evolution of instrument selection to Filter 2.2. These revisions include creation of a template for detailed definitions of the target domain which is a necessary first step for instrument selection, modifications to the Summary of Measurement Properties (SOMP) table to account for sources of variability, and development of standardized reporting tables for each measurement property.

CONCLUSIONS

OMERACT Filter 2.2 represents additional modifications of the OMERACT guide for working groups in their rigorous assessment of measurement properties of instruments of various types, including imaging outcome measurement instruments. Enhanced reporting aims to increase the transparency of the evidence base leading to judgements for the endorsement of instruments in core outcome sets.

Relevant domains and outcome measurement instruments in Neuropsychiatric Systemic Lupus Erythematosus: a systematic literature review

OBJECTIVES

Although neuropsychiatric involvement in Systemic Lupus Erythematosus (NPSLE) is one of the most complex and troubling manifestations of the disease, validated outcome instruments to be used as sensitive endpoints in controlled clinical trials are lacking. We set out a systematic literature review (SLR) to identify outcome measurement instruments and domains used to assess NPSLE.

METHODS

The Preferred Reporting Items for systematic reviews and Meta-analysis (PRISMA) guidelines were used. Articles available in English (1967-2020), listed in PubMed, EMBASE, PsycINFO, Cochrane Library and EULAR outcome measures library were screened. All domains and outcome measurement instruments were characterized according to the OMERACT Filter 2.1, considering core areas (manifestations/abnormalities, life impact, death/lifespan, societal/resource use) and contextual factors.

RESULTS

Of 3,392 abstracts evaluated, 83 studies were included in the SLR (15,974 patients, females 89.9%). Eligible studies included domains and instruments pertinent to all core areas defined by OMERACT, except for "societal/resource use". The most common core areas were "manifestations/abnormalities", covering 10 domains pertinent to laboratory and instrumental markers, indexes and neuropsychiatric dimension (cognitive, neurologic and psychiatric field), and "life impact", covering 7 domains related to physical function (from both the perspective of the patient and the physician), pain and quality of life.

CONCLUSION

Our study revealed great heterogeneity in the instruments derived from populations with NPSLE and none of these had high-quality evidence. This supports the need to develop and further validate a core domain set and outcome measurement instruments to promote clinical research in this field, enhancing comparability across studies.

COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments: a Delphi study

BACKGROUND

Scores on an outcome measurement instrument depend on the type and settings of the instrument used, how instructions are given to patients, how professionals administer and score the instrument, etc. The impact of all these sources of variation on scores can be assessed in studies on reliability and measurement error, if properly designed and analyzed. The aim of this study was to develop standards to assess the quality of studies on reliability and measurement error of clinician-reported outcome measurement instruments, performance-based outcome measurement instrument, and laboratory values.

METHODS

We conducted a 3-round Delphi study involving 52 panelists.

RESULTS

Consensus was reached on how a comprehensive research question can be deduced from the design of a reliability study to determine how the results of a study inform us about the quality of the outcome measurement instrument at issue. Consensus was reached on components of outcome measurement instruments, i.e. the potential sources of variation. Next, we reached consensus on standards on design requirements (n = 5), standards on preferred statistical methods for reliability (n = 3) and measurement error (n = 2), and their ratings on a four-point scale. There was one term for a component and one rating of one standard on which no consensus was reached, and therefore required a decision by the steering committee.

CONCLUSION

We developed a tool that enables researchers with and without thorough knowledge on measurement properties to assess the quality of a study on reliability and measurement error of outcome measurement instruments.

Outcome measurement instruments for peripheral vascular malformations and an assessment of the measurement properties: a systematic review

Purpose

The Outcome measures for vascular malformation (OVAMA) group reached consensus on the core outcome domains for the core outcome set (COS) for peripheral vascular malformations (venous, lymphatic and arteriovenous malformations). However, it is unclear which instruments should be used to measure these domains. Therefore, our aims were to identify all outcome measurement instruments available for vascular malformations, and to evaluate their measurement properties.

Methods

With the first literature search, we identified outcomes and instruments previously used in prospective studies on vascular malformations. A second search yielded studies on measurement properties of patient- and physician-reported instruments that were either developed for vascular malformations, or used in prospective studies. If the latter instruments were not specifically validated for vascular malformations, we performed a third search for studies on measurement properties in clinically similar diseases (vascular or lymphatic diseases and benign tumors). We assessed the methodological quality of these studies following the Consensus-based Standards for the selection of health Measurement Instruments methodology, and evaluated the quality of the measurement properties.

Results

The first search yielded 27 studies, none using disease-specific instruments. The second and third search included 22 development and/or validation studies, concerning six instruments. Only the Lymphatic Malformation Function Instrument was developed specifically for vascular malformations. Other instruments were generic QoL instruments developed and/or partly validated for clinically similar diseases.

Conclusions

Additional research on measurement properties is needed to assess which instruments may be included in the COS. This review informs the instrument selection and/or the development of new instruments.

Systematic review registration

PROSPERO, 42017056242.

Electronic supplementary material

The online version of this article (10.1007/s11136-019-02301-x) contains supplementary material, which is available to authorized users.

[Core outcome measurements in hand eczema trials : The HECOS initiative]

BACKGROUND

To enhance the informative value and comparability of hand eczema trials, the consensus of a set of core outcomes is necessary.

OBJECTIVES

How is success of therapeutic and preventive interventions currently determined in hand eczema trials? Which standard should apply in the future?

MATERIALS AND METHODS

As a first step, a systematic literature review was conducted to describe the status in controlled and randomized-controlled hand eczema trials from 2000-2017 and to identify the applied outcomes. The Hand Eczema Core Outcome Set (HECOS) initiative follows the guidelines of the Cochrane Skin Group Core Outcomes Set Initiative (CS-COUSIN) so that its further process is going to ensure the validity of the proposed measurement instruments and the appropriate inclusion of patients', physicians', and researchers' perspectives.

RESULTS

Previous trials predominantly assessed erythema, scaling, fissures, vesicles, or pruritus as clinical signs and symptoms of hand eczema. These variables were often not reported separately but as part of a variety of scores. Only few hand eczema scores were applied in more than one trial-most commonly the Hand Eczema Severity Index (HECSI, in 10% of the trials). Overall skin condition, without mentioning specific signs or symptoms, was assessed in 51% of the trials.

CONCLUSIONS

Although certain clinical signs and symptoms were assessed by a majority of hand eczema trials, comparability of results was not ensured because various measurement instruments were applied. HECOS is going to agree on a set of relevant and valid core outcome measurement instruments.

COSMIN Risk of Bias checklist for systematic reviews of Patient-Reported Outcome Measures

PURPOSE

The original COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist was developed to assess the methodological quality of single studies on measurement properties of Patient-Reported Outcome Measures (PROMs). Now it is our aim to adapt the COSMIN checklist and its four-point rating system into a version exclusively for use in systematic reviews of PROMs, aiming to assess risk of bias of studies on measurement properties.

METHODS

For each standard (i.e., a design requirement or preferred statistical method), it was discussed within the COSMIN steering committee if and how it should be adapted. The adapted checklist was pilot-tested to strengthen content validity in a systematic review on the quality of PROMs for patients with hand osteoarthritis.

RESULTS

Most important changes were the reordering of the measurement properties to be assessed in a systematic review of PROMs; the deletion of standards that concerned reporting issues and standards that not necessarily lead to biased results; the integration of standards on general requirements for studies on item response theory with standards for specific measurement properties; the recommendation to the review team to specify hypotheses for construct validity and responsiveness in advance, and subsequently the removal of the standards about formulating hypotheses; and the change in the labels of the four-point rating system.

CONCLUSIONS

The COSMIN Risk of Bias checklist was developed exclusively for use in systematic reviews of PROMs to distinguish this application from other purposes of assessing the methodological quality of studies on measurement properties, such as guidance for designing or reporting a study on the measurement properties.

COSMIN Risk of Bias checklist for systematic reviews of Patient-Reported Outcome Measures

PURPOSE

The original COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist was developed to assess the methodological quality of single studies on measurement properties of Patient-Reported Outcome Measures (PROMs). Now it is our aim to adapt the COSMIN checklist and its four-point rating system into a version exclusively for use in systematic reviews of PROMs, aiming to assess risk of bias of studies on measurement properties.

METHODS

For each standard (i.e., a design requirement or preferred statistical method), it was discussed within the COSMIN steering committee if and how it should be adapted. The adapted checklist was pilot-tested to strengthen content validity in a systematic review on the quality of PROMs for patients with hand osteoarthritis.

RESULTS

Most important changes were the reordering of the measurement properties to be assessed in a systematic review of PROMs; the deletion of standards that concerned reporting issues and standards that not necessarily lead to biased results; the integration of standards on general requirements for studies on item response theory with standards for specific measurement properties; the recommendation to the review team to specify hypotheses for construct validity and responsiveness in advance, and subsequently the removal of the standards about formulating hypotheses; and the change in the labels of the four-point rating system.

CONCLUSIONS

The COSMIN Risk of Bias checklist was developed exclusively for use in systematic reviews of PROMs to distinguish this application from other purposes of assessing the methodological quality of studies on measurement properties, such as guidance for designing or reporting a study on the measurement properties.

Choosing the right outcome measurement instruments for patients with low back pain

Choosing the most fit-for-purpose outcome measurement instruments is fundamental because using inappropriate instruments can lead to detection bias and measurement inconsistency. Recent recommendations, consensus procedures and systematic reviews on existing patient-reported outcome measures (PROMs) informed this manuscript, which provides suggestions on which outcome domains and measurement instruments to use in patients with low back pain (LBP). Six domains are identified as highly relevant: (1) physical functioning, (2) pain intensity, (3) health-related quality of life, (4) work, (5) psychological functioning and (6) pain interference. For each domain, one or more PROMs are suggested for clinical research and practice, selecting among those that are most frequently used and recommended, and that have satisfactory measurement properties in patients with LBP. Further research on the measurement properties of these suggested PROMs is needed while also considering other emerging instruments, such as the PROMIS computerised adaptive testing and short forms.

The quality of systematic reviews of health-related outcome measurement instruments

BACKGROUND

Systematic reviews of outcome measurement instruments are important tools for the selection of instruments for research and clinical practice. Our aim was to assess the quality of systematic reviews of health-related outcome measurement instruments and to determine whether the quality has improved since our previous study in 2007.

METHODS

A systematic literature search was performed in MEDLINE and EMBASE between July 1, 2013, and June 19, 2014. The quality of the reviews was rated using a study-specific checklist.

RESULTS

A total of 102 reviews were included. In many reviews the search strategy was considered not comprehensive; in only 59 % of the reviews a search was performed in EMBASE and in about half of the reviews there was doubt about the comprehensiveness of the search terms used for type of measurement instruments and measurement properties. In 41 % of the reviews, compared to 30 % in our previous study, the methodological quality of the included studies was assessed. In 58 %, compared to 55 %, the quality of the included instruments was assessed. In 42 %, compared to 7 %, a data synthesis was performed in which the results from multiple studies on the same instrument were somehow combined.

CONCLUSION

Despite a clear improvement in the quality of systematic reviews of outcome measurement instruments in comparison with our previous study in 2007, there is still room for improvement with regard to the search strategy, and especially the quality assessment of the included studies and the included instruments, and the data synthesis.