Tools to assess the measurement properties of quality of life instruments: a meta-review

Development and validation of meta-measurement instruments: A methodological approach

A valid and reliable nursing assessment is essential for identifying required care and ensuring patient safety. The convenience of conducting a comprehensive assessment of the patient has led to a significant increase in assessment tools that may slow down the process. Nevertheless, the possibility of consolidating various instruments that measure common or similar constructs into a meta-instrument is considered an alternative that could enhance assessment efficiency. A meta-instrument can be defined as a measurement tool that consolidates other instruments based on measuring related constructs and sharing dimensions or items, aiming to achieve a more parsimonious measurement. Literature on such assessment tools is scarce, and there are numerous options for their construction and initial validation. Additionally, it is advisable to confirm their psychometric properties and ensure that they maintain, at the very least, the same diagnostic capacity as the original instruments. This article presents a proposal for the phases to follow in constructing meta-instruments, along with various methodological alternatives that can be employed based on the characteristics of the original instruments and the purpose of creating the meta-instrument. Furthermore, special attention is given to the checklists that should be used to study the psychometric properties and diagnostic capacity of the meta-instruments. Finally, future lines of research and challenges in the development of nursing assessment meta-instruments are discussed.

Guideline for reporting systematic reviews of outcome measurement instruments (OMIs): PRISMA-COSMIN for OMIs 2024

BACKGROUND AND OBJECTIVE

Although comprehensive and widespread guidelines on how to conduct systematic reviews of outcome measurement instruments (OMIs) exist, for example from the COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments) initiative, key information is often missing in published reports. This article describes the development of an extension of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 guideline: PRISMA-COSMIN for OMIs 2024.

METHODS

The development process followed the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) guidelines and included a literature search, expert consultations, a Delphi study, a hybrid workgroup meeting, pilot testing, and an end-of-project meeting, with integrated patient/public involvement.

RESULTS

From the literature and expert consultation, 49 potentially relevant reporting items were identified. Round 1 of the Delphi study was completed by 103 panelists, whereas round 2 and 3 were completed by 78 panelists. After 3 rounds, agreement (≥67%) on inclusion and wording was reached for 44 items. Eleven items without consensus for inclusion and/or wording were discussed at a workgroup meeting attended by 24 participants. Agreement was reached for the inclusion and wording of 10 items, and the deletion of 1 item. Pilot testing with 65 authors of OMI systematic reviews further improved the guideline through minor changes in wording and structure, finalized during the end-of-project meeting. The final checklist to facilitate the reporting of full systematic review reports contains 54 (sub)items addressing the review's title, abstract, plain language summary, open science, introduction, methods, results, and discussion. Thirteen items pertaining to the title and abstract are also included in a separate abstract checklist, guiding authors in reporting for example conference abstracts.

CONCLUSION

PRISMA-COSMIN for OMIs 2024 consists of two checklists (full reports; abstracts), their corresponding explanation and elaboration documents detailing the rationale and examples for each item, and a data flow diagram. PRISMA-COSMIN for OMIs 2024 can improve the reporting of systematic reviews of OMIs, fostering their reproducibility and allowing end-users to appraise the quality of OMIs and select the most appropriate OMI for a specific application. NOTE: This paper was jointly developed by Journal of Clinical Epidemiology, Quality of Life Research, Journal of Patient Reported Outcomes, Health and Quality of Life Outcomes and jointly published by Elsevier Inc, Springer Nature Switzerland AG, and BioMed Central Ltd., part of Springer Nature. The articles are identical except for minor stylistic and spelling differences in keeping with each journal's style. Either citation can be used when citing this article.

Estimation of Societal Values of Health States Preferences at the National Level for Low- and Middle-Income Countries

OBJECTIVES

Social preference values of health states are a fundamental input for the preparation of studies in health economics. Several countries have undertaken studies to obtain these values. Our objective was to conduct a structured and systematic literature review of articles that calculates this set of representative values at the national level in low- and middle-income countries (LMICs).

METHODS

In this systematic review, we searched the Embase, MEDLINE, Ovid, SciELO, and LILACS databases, among others, for studies published up to June 2022 that estimated nationally representative health states preferences values for LMICs. We summarized the information qualitatively and assessed the risk of bias in each article using the consensus-based standards for selecting health measurement instruments checklist tool.

RESULTS

Of the 23 663 articles identified, 35 studies were eligible for inclusion. The studies were from 19 countries in Latin-American, Europe, Africa, and Asia. No studies were found for low-income countries. The most commonly applied generic instrument for measuring health-related quality of life was the 5-level version of EQ-5D and 3-level version of EQ-5D. Preference was given to face-to-face administration of these instruments. The sociodemographic variables with the most significant negative correlation versus utility were older adults, marital status (widowed or divorced), and low educational level and income.

CONCLUSIONS

Worldwide, there have been few studies that have estimated, in a nationally representative manner, the social values of health states preferences in LMICs. We consider the local estimate of this set of societal values relevant for any society to improve decision making in allocating resources in health budgets.

Assessing independence in mobility activities in trauma care: Validity and reliability of the Activity Independence Measure-Trauma (AIM-T) in humanitarian settings

The importance of measuring outcomes after injury beyond mortality and morbidity is increasingly recognized, though underreported in humanitarian settings. To address shortcomings of existing outcome measures in humanitarian settings, the Activity Independence Measure-Trauma (AIM-T) was developed, and is structured in three subscales (i.e., core, lower limb, and upper limb). This study aimed to assess the AIM-T construct validity (structural validity and hypothesis testing) and reliability (internal consistency, inter-rater reliability and measurement error) in four humanitarian settings (Burundi, Iraq, Cameroon and Central African Republic). Patients with acute injury (n = 195) were assessed using the AIM-T, the Barthel Index (BI), and two pain scores. Structural validity was assessed through confirmatory factor analysis. Hypotheses were tested regarding correlations with BI and pain scores using Pearson correlation coefficient (PCC) and differences in AIM-T scores between patients' subgroups, using standardized effect size Cohen's d (d). Internal consistency was assessed with Cronbach's alpha (α). AIM-T was reassessed by a second rater in 77 participants to test inter-rater reliability using intraclass correlation coefficient (ICC). The results showed that the AIM-T structure in three subscales had an acceptable fit. The AIM-T showed an inverse weak to moderate correlation with both pain scores (PCC<0.7, p≤0.05), positive strong correlation with BI (PCC≥0.7, p≤0.05), and differed between all subgroups (d≥0.5, p≤0.05). The inter-rater reliability in the (sub)scales was good to excellent (ICC 0.86-0.91) and the three subscales' internal consistency was adequate (α≥0.7). In conclusion, this study supports the AIM-T validity in measuring independence in mobility activities and its reliability in humanitarian settings, as well as it informs on its interpretability. Thus, the AIM-T could be a valuable measure to assess outcomes after injury in humanitarian settings.

Study protocol for developing, piloting and disseminating the PRISMA-COSMIN guideline: a new reporting guideline for systematic reviews of outcome measurement instruments

BACKGROUND

Systematic reviews of outcome measurement instruments are important tools in the evidence-based selection of these instruments. COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments) has developed a comprehensive and widespread guideline to conduct systematic reviews of outcome measurement instruments, but key information is often missing in published reviews. This hinders the appraisal of the quality of outcome measurement instruments, impacts the decisions of knowledge users regarding their appropriateness, and compromises reproducibility and interpretability of the reviews' findings. To facilitate sufficient, transparent, and consistent reporting of systematic reviews of outcome measurement instruments, an extension of the PRISMA (Preferred Reporting of Items for Systematic reviews and Meta-Analyses) 2020 guideline will be developed: the PRISMA-COSMIN guideline.

METHODS

The PRISMA-COSMIN guideline will be developed in accordance with recommendations for reporting guideline development from the EQUATOR (Enhancing the QUAlity and Transparency Of health Research) Network. First, a candidate reporting item list will be created through an environmental literature scan and expert consultations. Second, an international Delphi study will be conducted with systematic review authors, biostatisticians, epidemiologists, psychometricians/clinimetricians, reporting guideline developers, journal editors as well as patients, caregivers, and members of the public. Delphi panelists will rate candidate items for inclusion on a 5-point scale, suggest additional candidate items, and give feedback on item wording and comprehensibility. Third, the draft PRISMA-COSMIN guideline and user manual will be iteratively piloted by applying it to systematic reviews in several disease areas to assess its relevance, comprehensiveness, and comprehensibility, along with usability and user satisfaction. Fourth, a consensus meeting will be held to finalize the PRISMA-COSMIN guideline through roundtable discussions and voting. Last, a user manual will be developed and the final PRISMA-COSMIN guideline will be disseminated through publications, conferences, newsletters, and relevant websites. Additionally, relevant journals and organizations will be invited to endorse and implement PRISMA-COSMIN. Throughout the project, evaluations will take place to identify barriers and facilitators of involving patient/public partners and employing a virtual process.

DISCUSSION

The PRISMA-COSMIN guideline will ensure that the reports of systematic reviews of outcome measurement instruments are complete and informative, enhancing their reproducibility, ease of use, and uptake.

Instruments used to measure knowledge and attitudes of healthcare professionals towards antibiotic use for the treatment of urinary tract infections: A systematic review

Background

Urinary tract infections (UTIs) are the second most common condition (after upper respiratory tract infections) for which adults receive antibiotics, and this prevalence may contribute to antibiotic resistance. Knowledge and attitudes have been identified as potential determinants of antibiotic prescribing behaviour among healthcare professionals in the treatment and management of UTIs. An instrument that captures prescribers’ baseline knowledge of and attitudes towards antibiotic prescribing for UTIs could inform interventions to enhance prescribing. The current systematic review evaluates the psychometric properties of instruments already available and describes the theoretical constructs they measure.

Methods

Five electronic databases were searched for published studies and instruments. The Consensus-based Standards for the selection of health status Measurement Instruments checklist was used to assess the psychometric quality reporting of the instruments. The items included in each instrument were mapped onto the theoretical constructs underlying knowledge and attitudes using a mixed-theoretical model developed for this study.

Results

Fourteen studies met the review inclusion criteria. All instruments were available for review. None of the instruments had all the psychometric properties evaluated. Most of the instruments sought to identify knowledge and/or attitude factors influencing antibiotic prescribing for UTIs rather than to measure/assess knowledge and attitudes.

Conclusions

Few instruments for the assessment of knowledge and attitudes of healthcare professionals towards antibiotic use and UTI treatment are available. None of the instruments underwent the full development process to ensure that all psychometric properties were met. Furthermore, none of the instruments assessed all domains of knowledge and attitudes. Therefore, the ability of the instruments to provide a robust measurement of knowledge and attitudes is doubtful. There is a need for an instrument that fully and accurately measures the constructs of knowledge and attitude of healthcare professionals in the treatment of UTIs.

Conducting Systematic Reviews of the Quality and Psychometric Properties of Health-Related Measurement Instruments: Finding the Right Tool for the Job

Many health-related measurement instruments have been developed to measure psychological constructs and whilst several instruments are usually available for a particular study, finding the right tool for the job is important. Systematic reviews of measurement properties of instruments have long been identified as a valuable strategy to ensure that we select the right tool to assess mental health. There are many important steps and procedures to guide these types of systematic reviews to find the "best fit" and this paper summarizes some of these key processes and steps. The selection of instrument(s) to use should be made considering the most recent comprehensive review of the quality of the outcome measurement instrument based on unbiased assessment of its psychometric properties, responsiveness, and generalizability of results. Researchers planning to conduct a systematic review of health-related measurement instruments should design the review beforehand using standardized frameworks. Conducting systematic reviews of the quality and psychometric properties of health-related measurement instruments is important to ensure we choose the best tool for the research question and target population.

Quality, methods, and recommendations of systematic reviews on measures of evidence-based practice: an umbrella review

OBJECTIVES

The objective of the review was to estimate the quality of systematic reviews on evidence-based practice measures across health care professions and identify differences between systematic reviews regarding approaches used to assess the adequacy of evidence-based practice measures and recommended measures.

INTRODUCTION

Systematic reviews on the psychometric properties of evidence-based practice measures guide researchers, clinical managers, and educators in selecting an appropriate measure for use. The lack of psychometric standards specific to evidence-based practice measures, in addition to recent findings suggesting the low methodological quality of psychometric systematic reviews, calls into question the quality and methods of systematic reviews examining evidence-based practice measures.

INCLUSION CRITERIA

We included systematic reviews that identified measures that assessed evidence-based practice as a whole or of constituent parts (eg, knowledge, attitudes, skills, behaviors), and described the psychometric evidence for any health care professional group irrespective of assessment context (education or clinical practice).

METHODS

We searched five databases (MEDLINE, Embase, CINAHL, PsycINFO, and ERIC) on January 18, 2021. Two independent reviewers conducted screening, data extraction, and quality appraisal following the JBI approach. A narrative synthesis was performed.

RESULTS

Ten systematic reviews, published between 2006 and 2020, were included and focused on the following groups: all health care professionals (n = 3), nurses (n = 2), occupational therapists (n = 2), physical therapists (n = 1), medical students (n = 1), and family medical residents (n = 1). The overall quality of the systematic reviews was low: none of the reviews assessed the quality of primary studies or adhered to methodological guidelines, and only one registered a protocol. Reporting of psychometric evidence and measurement characteristics differed. While all the systematic reviews discussed internal consistency, feasibility was only addressed by three. Many approaches were used to assess the adequacy of measures, and five systematic reviews referenced tools. Criteria for the adequacy of individual properties and measures varied, but mainly followed standards for patient-reported outcome measures or The Standards of Educational and Psychological Testing. Two hundred and four unique measures were identified across 10 reviews. One review explicitly recommended measures for occupational therapists, and four reviews identified adequate measures for all health care professionals (n = 3) and medical students (n = 1). The 27 measures deemed adequate by these five systematic reviews are described.

CONCLUSIONS

Our results suggest a need to improve the overall methodological quality and reporting of systematic reviews on evidence-based practice measures to increase the trustworthiness of recommendations and allow comprehensive interpretation by end-users. Risk of bias is common to all the included systematic reviews as the quality of primary studies was not assessed. The diversity of tools and approaches used to evaluate the adequacy of evidence-based practice measures reflects tensions regarding the conceptualization of validity, suggesting a need to reflect on the most appropriate application of validity theory to evidence-based practice measures.

SYSTEMATIC REVIEW REGISTRATION NUMBER

PROSPERO CRD42020160874.

Evidence-based evaluation of owner-reported outcome measures for canine orthopedic care - a COSMIN evaluation of 6 instruments

OBJECTIVE

To systematically review the currently available validated owner-reported outcome measures (OROMs) for assessing pain and function in dogs with osteoarthritis and other joint diseases.

STUDY DESIGN

Systematic review.

METHODS

A systematic literature search of PubMed and Web of Science was performed in December, 2019, using search terms relevant to OROMs for pain and function in dogs with osteoarthritis and other joint diseases. The measurement properties of the resulting outcome instruments and validation studies were evaluated with the Consensus-Based Standards for the Selection of Health Measurement Instruments (COSMIN).

RESULTS

Seventeen publications describing the validation of 6 OROMs were selected and evaluated with the COSMIN. The Canine Brief Pain Inventory (CBPI), Canine Orthopedic Index (COI), and Liverpool Osteoarthritis in Dogs (LOAD) provided evidence of sufficient content validity. Internal consistency, reliability, construct validity, and responsiveness were the most frequently validated other properties. Both CBPI and COI had sufficient internal consistency; LOAD, being formative in construct, need not be assessed for internal consistency. No instruments provided information on measurement errors.

CONCLUSION

The CBPI, COI, and LOAD can be recommended for use in dogs with osteoarthritis. Further evidence is necessary to increase the interpretability of these instruments or expand their application into other canine orthopedic conditions. Further evidence is also needed before the other 3 instruments can be recommended.

CLINICAL SIGNIFICANCE

This systematic evaluation of 6 OROMs provides clinicians and researchers with a tool that facilitates and improves evidence-based selection of outcome measures for evaluating canine osteoarthritis and other joint disease treatments.

Psychological resilience during COVID-19: a meta-review protocol

INTRODUCTION

The global COVID-19 pandemic continues to have wide-ranging implications for health, including psychological well-being. A growing corpus of research reviews has emerged on the topic of psychological resilience in the context of the pandemic. However, this body of work has not been systematically reviewed for its quality, nor with respect to findings on the effectiveness of tools and strategies for psychological resilience. To this end, a meta-review protocol is proposed with the following objectives: (1) identify review work on the topic of psychological resilience during COVID-19; (2) assess the quality of this review work using A MeaSurement Tool to Assess systematic Reviews; (3) assess the risk of bias in this work; (4) generate a narrative summary of the key points, strengths and weaknesses; (5) identify the psychological resilience strategies that have been reviewed; (6) identify how these strategies have been evaluated for their effectiveness; (7) identify what outcomes were measured and (8) summarise the findings on strategies for psychological resilience so far, providing recommendations, if possible.

METHODS AND ANALYSIS

A systematic meta-review will be conducted in accordance with the Preferred Reporting Items for Systematic Reviews for Protocols and Joanna Briggs Institute umbrella review guidelines. Electronic searches of general databases, especially Web of Science, Scopus and PubMed, will be conducted. Only results from January 2020 onwards will be considered, coinciding with the COVID-19 pandemic. Only results in English will be included. Descriptive statistics, thematic analysis and narrative summaries describing the nature of the reviewed work and evaluation of psychological resilience strategies will be carried out.

ETHICS AND DISSEMINATION

Ethical approval is not needed for systematic review protocols. The results of the meta-review will be published in an international peer-reviewed journal. The raw and summarised data will be shared in the journal or other open venues.

PROSPERO REGISTRATION NUMBER

CRD42021235288.

Dental patient reported outcome and oral health-related quality of life measures: protocol for a systematic evidence map of reviews

AIMS

This research synthesis protocol addresses the question: what is the evidence concerning measurement properties of dental patient reported outcome measures (dPROMs), and regarding the real-world value of dPROMs, and where are the gaps in this evidence? Evidence mapping will systematically examine reviews of quantitative dPROMs used to assess the impact of oral health on the quality of life of dental patients and research participants. Evidence gaps where future research or systematic reviews are required will be identified.

MATERIALS AND METHODS

This protocol accords with the PRISMA-P guideline. Open Science Framework Registration https://doi.org/10.17605/OSF.IO/RZD3N . Biomedical and grey literature databases will be searched, adapting the same search strategy. Published or unpublished reviews evaluating any dPROM will be considered for inclusion. There will be no restriction by date, setting, or language. AMSTAR2 and ROBIS will evaluate risk of bias. Psychometric criteria will be adapted from COSMIN. Data will be summarised separately for specific populations and conditions.

DISCUSSION

The findings will enable clinicians and researchers to identify methodologically robust dPROMs, appropriate for use with relevant populations and conditions. Implications for real-world practice and research will be discussed.