Evaluation of Studies on the Measurement Properties of Self-Reported Instruments

A critical evaluation of the content validity of patient-reported outcome measures assessing health-related quality of life in children with cancer: a systematic review

BACKGROUND

With increasing survival rates in pediatric oncology, the need to monitor health-related quality of life (HRQOL) is becoming even more important. However, available patient-reported outcome measures (PROMs) have been criticized. This review aims to systematically evaluate the content validity of PROMs for HRQOL in children with cancer.

METHODS

In December 2021, a systematic literature search was conducted in PubMed. PROMs were included if they were used to assess HRQOL in children with cancer and had a lower age-limit between 8 and 12 years and an upper age-limit below 21 years. The COSMIN methodology for assessing the content validity of PROMs was applied to grade evidence for relevance, comprehensiveness, and comprehensibility based on quality ratings of development studies (i.e., studies related to concept elicitation and cognitive interviews for newly developed questionnaires) and content validity studies (i.e., qualitative studies in new samples to evaluate the content validity of existing questionnaires).

RESULTS

Twelve PROMs were included. Due to insufficient patient involvement and/or poor reporting, the quality of most development studies was rated 'doubtful' or 'inadequate'. Few content validity studies were available, and these were mostly 'inadequate'. Following the COSMIN methodology, evidence for content validity was 'low' or 'very low' for almost all PROMs. Only the PROMIS Pediatric Profile had 'moderate' evidence. In general, the results indicated that the PROMs covered relevant issues, while results for comprehensiveness and comprehensibility were partly inconsistent or insufficient.

DISCUSSION

Following the COSMIN methodology, there is scarce evidence for the content validity of available PROMs for HRQOL in children with cancer. Most instruments were developed before the publication of milestone guidelines and therefore were not able to fulfill all requirements. Efforts are needed to catch up with methodological progress made during the last decade. Further research should adhere to recent guidelines to develop new instruments and to strengthen the evidence for existing PROMs.

Quality of the adaptation procedures and psychometric properties of the scale of oral health outcomes for 5-year-old children (SOHO-5): a systematic review

PURPOSE

The aim of this study was to assess the quality of the cross-cultural adaptations and psychometric properties of the Scale of Oral Health Outcomes for 5-year-old children (SOHO-5).

METHODS

A systematic search was performed in three databases, PubMed, SCOPUS, and Dentistry and Oral Science Source (EBSCO). Studies relating to the cross-cultural adaptation and psychometric properties of SOHO-5 were included. The quality of cross-cultural adaptation was assessed in five aspects including translation, synthesis, back-translation, expert committee review, and pretesting. The Consensus-based Standards for the selection of health Measurement Instruments (COSMIN) checklist was used to assess the psychometric properties in seven domains including responsiveness, internal consistency, reliability, measurement error, and structural, construct, and cross-cultural validity.

RESULTS

The search identified 162 papers for screening; from these, 107 articles and 9 systematic reviews were excluded. 21 full papers were retrieved and 6 studies that met the inclusion criteria were included. Out of six, two reported all five aspects of cross-cultural adaptation procedures. Regarding the psychometric properties, none of the studies evaluated all aspects of psychometric properties. Almost all of the studies reported internal consistency (Cronbach's alpha ranging from 0.71 to 0.90), test-retest reliability (Intraclass Correlation Coefficient 0.46-0.98), and construct validity. None reported the cross-cultural validity and responsiveness of the measures.

CONCLUSION

While these studies provided satisfactory results regarding the validity and reliability of SOHO-5, the majority of the included studies did not report all domains of the cross-cultural adaptations and psychometric properties. Overall, the quality of the cross-cultural adaptations and psychometric properties of the included studies is doubtful. Future research should follow the guidelines to assess all aspects of psychometric properties and adaptation procedures in cross-cultural settings.

Psychometric properties of the revised Diabetes Knowledge Test using Rasch analysis

OBJECTIVE

The revised Diabetes Knowledge Test (DKT2) is the instrument that is the most widely used for measuring diabetes knowledge; nevertheless, its structural validity has never been evaluated. This study aimed to determine the psychometric properties of the DKT2.

METHODS

This study performed a secondary data analysis of people with diabetes recruited at university hospitals. Rasch analysis was used.

RESULTS

One item in the 14-item DKT2 measuring general diabetes knowledge exhibited a poor-fit, and so it was eliminated. The person-item map showed that items of greater difficulty need to be added to the instrument. The principal-components analysis of residuals revealed a unidimensional structure. The person reliability was 0.50, with a person separation index of 1.01. Measurement invariance was not satisfied for items 11 and 2 according to gender and age.

CONCLUSIONS

The unidimensional structure of the 13-item DKT2 demonstrated poor person reliability and a low person separation index. Females and elderly persons found it more difficult to respond to items 11 and 2, respectively.

PRACTICE IMPLICATIONS

Careful consideration is necessary when using the 13-item DKT2 in practice and research. It is recommended for future studies to add items of greater difficulty to the instrument.