Statistical considerations in the psychometric validation of outcome measures

Assessing the Psychometric Validity of the Epistaxis Severity Score: Internal Consistency and Test-Retest Reliability

BACKGROUND

The Epistaxis Severity Score (ESS) is the gold-standard patient-reported outcome measure for evaluating nosebleed severity in patients with hereditary hemorrhagic telangiectasia (HHT). To date, the ESS has been assessed only for content validity and concurrent validity.

OBJECTIVE

We evaluate the internal consistency and test-retest reliability of the ESS.

MATERIALS AND METHODS

After receiving institutional review board approval, we sent an online survey battery, including the ESS survey, to 305 (39% male) English-speaking HHT patients ≥18 years old at a single center. Of those, 140 (46%) patients completed the battery, and 110/140 (79%) reported epistaxis. Cronbach's alpha and correlation analyses were used to evaluate internal consistency. For the test-retest reliability evaluation, we recruited 69 HHT patients during HHT clinic to complete 2 self-administered ESS surveys 2 weeks apart. Participants also completed a modified Clinical Global Impression-Improvement scale with readministration of the ESS survey. We calculated the intraclass correlation coefficient in a 2-way mixed model with absolute agreement.

RESULTS

The ESS survey demonstrated low internal consistency (Cronbach's alpha = 0.495), suggesting that it measured multiple unrelated concepts. Factor analysis revealed 3 latent factors with moderate intercorrelation, suggesting the presence of 3 related but distinct constructs underlying the ESS. However, the ESS demonstrated excellent test-retest reliability (intraclass correlation coefficient = 0.955; 95% CI, 0.91-0.98).

CONCLUSION

Although the ESS demonstrates high test-retest reliability, it may not adequately assess different dimensions of nosebleed severity. Additional correlated survey questions and sub-scores may be needed to increase internal consistency to accurately measure each component of epistaxis severity. It is necessary to acknowledge epistaxis severity from different dimensions and to consider evaluating individual ESS items separately for a comprehensive understanding.

Brazilian Portuguese translation, cross-cultural adaptation, and content validation of the pediatric lower urinary tract symptom score

BACKGROUND

Early screening instruments for professionals are important to reduce the cost and time of diagnosing pediatric lower urinary tract dysfunction. This study aimed to translate, cross-culturally adapt, and validate pediatric lower urinary tract symptom scores.

METHODS

This methodological study was conducted in 2019 in two phases: translation and cross-cultural adaptation, and content validation. In the first phase, we used translators, specialists, and 30 parents of children with voiding dysfunction. In the second phase, validation was performed by 11 pediatric urology specialists.

FINDINGS

The instrument was translated and adapted to Brazilian Portuguese and presented an excellent level of verbal comprehension. Further, the instrument presented a good content validity index >0.8 for all items, except for the nomenclature and clarity of two items, as indicated by the Cronbach's alpha value of 0.75 for internal consistency.

DISCUSSION

The final version showed semantic, idiomatic, experiential, and conceptual equivalence with the original instrument, indicating that it is adequate for other assessments of psychometric qualities.

PRACTICE IMPLICATIONS

The translation and adaptation of an important instrument for diagnosing lower urinary tract dysfunction in children to other countries helps nurses identify the disease early and monitor the treatment.

Business process nature assessment matrix – a novel approach to the assessment of business process nature

Purpose

The purpose of this study is to give an in-depth understanding of the nature of business processes (BPs) from the perspective of their dynamism and knowledge intensity that will allow for their correct classification and provide practical and useful implications for their more relevant and effective management. A simple and low labor-intensive BP nature assessment approach is proposed that will allow for objective assessment and internal benchmarking of all BPs in a specific context of execution based on their nature.

Design/methodology/approach

The research is divided into two parts. The first comprises a systematic literature review (SLR) based on the resources of the ProQuest, Springer Nature and ScienceDirect full-text databases and the second includes illustrative case studies.

Findings

Building on the SLR, the authors identified and reviewed 3,385 articles and defined a set of criteria by which the nature of BPs can be assessed. Further, the authors proposed a BP nature assessment matrix together with complementary questionnaires for the evaluation of process dynamism and knowledge intensity. To demonstrate the logic of this approach, two illustrative case studies were presented.

Originality/value

The article contributes to the theoretical reflection on the nature of BPs in the knowledge economy. From the practical point of view, a novel approach to the assessment of the nature of BPs is offered. The approach is open and as experience is accumulated it will develop according to the data and recommendations collected.

Psychometric properties of measures of upper limb activity performance in adults with and without spasticity undergoing neurorehabilitation-A systematic review

INTRODUCTION

This systematic review appraises the measurement quality of tools which assess activity and/or participation in adults with upper limb spasticity arising from neurological impairment, including methodological quality of the psychometric studies. Differences in the measurement quality of the tools for adults with a neurological impairment, but without upper limb spasticity, is also presented.

METHODS

29 measurement tools identified in a published review were appraised in this systematic review. For each identified tool, we searched 3 databases (Medline, Embase, CINAHL) to identify psychometric studies completed with neurorehabilitation samples. Methodological quality of instrument evaluations was assessed with use of the Consensus-based Standards for the Selection of Health Status Measurement Instruments (COSMIN) checklist. Synthesis of ratings allowed an overall rating of the psychometric evidence for each measurement tool to be calculated.

RESULTS

149 articles describing the development or evaluation of psychometric properties of 22 activity and/or participation measurement tools were included. Evidence specific to tool use for adults with spasticity was identified within only 15 of the 149 articles and provided evidence for 9 measurement tools only. Overall, COSMIN appraisal highlighted a lack of evidence of measurement quality. Synthesis of ratings demonstrated all measures had psychometric weaknesses or gaps in evidence (particularly for use of tools with adults with spasticity).

CONCLUSIONS

The systematic search, appraisal and synthesis revealed that currently there is insufficient measurement quality evidence to recommend one tool over another. Notwithstanding this conclusion, newer tools specifically designed for use with people with neurological conditions who have upper limb spasticity, have emergent measurement properties that warrant further research.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42014013190.

Validation of a modified Berger HIV stigma scale for use among patients with hepatitis C virus (HCV) infection

BACKGROUND

Stigma around hepatitis C virus (HCV) infection is an important and understudied barrier to HCV prevention, treatment, and elimination. To date, no validated instrument exists to measure patients' experiences of HCV stigma. This study aimed to revise the Berger (2001) HIV stigma scale and evaluate its psychometric properties among patients with HCV infection.

METHODS

The Berger HIV stigma scale was revised to ask about HCV and administered to patients with HCV (n = 270) in Philadelphia, Pennsylvania. Scale reliability was evaluated as internal consistency by calculating Cronbach's alpha. Exploratory factor analysis was performed to evaluate construct validity by comparing item clustering to the Berger HIV stigma scale subscales. Item response theory was employed to further evaluate individual items and to calibrate items for simulated computer adaptive testing sessions in order to identify potential shortened instruments.

RESULTS

The revised HCV Stigma Scale was found to have good reliability (α = 0.957). After excluding items for low loadings, the exploratory factor analysis indicated good construct validity with 85% of items loading on pre-defined factors. Analyses strongly suggested the predominance of an underlying unidimensional factor solution, which yielded a 33-item scale after items were removed for low loading and differential item functioning. Adaptive simulations indicated that the scale could be substantially shortened without detectable information loss.

CONCLUSIONS

The 33-item HCV Stigma Scale showed sufficient reliability and construct validity. We also conducted computer adaptive testing simulations and identified shortened six- and three-item scale alternatives that performed comparably to the original 40-item scale.

Translation, validation and reliability of the functional capacity questionnaire Haemophilia Activities List for haemophilia patients in Brazil

INTRODUCTION

To date, none of the available assessment instruments to evaluate functional abilities for individuals with haemophilia has been translated and validated in Brazil.

AIM

To translate, and test the construct validity, internal consistency and the reliability of the Haemophilia Activities List (HAL) for the Brazilian population with severe and moderate haemophilia (type A or B) and to investigate differences in the self-perception of functional abilities in patients adults with haemophilia classified according to the presence of joint bleeding and the performance of orthopaedic surgeries.

METHODS

The translation and transcultural adaptation following the steps: (a) translation; (b) synthesis (consensual version); (c) back-translation; (d) review by the committee of experts; (e) pretest in the target population; and (f) final version of the instrument. Correlations between HAL scores and the scores of the Health Assessment Questionnaire (HAQ) and Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) were determined for construct validation. Reliability was tested using the test and retest method, and internal consistency was calculated by Cronbach's α.

RESULTS

The HAL-BR was applied to 52 individuals with haemophilia (34.51 ± 12.87 years). The process of translation and cross-cultural adaptation produced similar versions between the translations A moderate correlation was observed between HAL and the HAQ (r = -0.55) and WOMAC scores (r = -0.58). The reliability was ICC = 0.972, CI (0.917-0.997) ICC = 0.876, CI (0.631-0.978) for inter- and intra-examiners. No difference was found in the total HAL score between the groups with and without bleeding. The group subjected to surgery compared to the group of not subjected to surgery presented in the HAL domains worse function in 'lying down/sitting/kneeling/standing', 'self-care' and 'complex lower extremities activities' as well as in total HAL score.

CONCLUSION

The Brazilian version of the HAL was proven to be a valid and reliable evaluation tool for adults with haemophilia in Brazil.

Odor evaluation scales for odor in neoplastic wounds: an integrative review

OBJECTIVE

Checking for the existence of assessment instruments for odor in neoplastic wounds.

METHOD

Integrative review performed in the databases Lilacs, SCiELO, Ibecs and MedLine, including national and international publications.

RESULTS

15 articles were analyzed, in its majority performed by nurses and being of the revision type. Nine scales were found, of which the majority assesses only the odor intensity. The scale most used to evaluate products and/or bandage covers for neoplastic injuries was the Odor Evaluation Guideline, of qualitative-quantitative nature, that establishes the symptom within four levels; only one of which being validated (Teler scale).

CONCLUSION

The results of this study showed that, currently, there is only one scale that is validated for assessing odor in neoplastic wounds, pointing towards the need to develop new instruments that incorporate validated and reliable instruments in clinical practice.

The Development and Validity of the Adult Burn Outcome Questionnaire Short Form

Patient Reported Outcome Measures (PROMs) are useful for understanding the health needs and outcomes of the general public. We aim to develop a burn-specific metric-Adult Burn Outcome Questionnaire (ABOQ)-that is brief and can be administered electronically to all burn survivors over the age of 18. The 14-item ABOQ was developed from the already validated Young Adult Burn Outcome Questionnaire (YABOQ) long form. The ABOQ questionnaire, along with Patient-Reported Outcomes Measurement System-10 (PROMIS-10), was administered to 120 outpatient burn survivors at three hospitals. Clinical validity of the ABOQ was measured by testing associations between ABOQ items and burn size, the PROMIS-10 generic items and composite scales using correlational analysis including multivariate canonical analysis. Nine out of 14 ABOQ items were significantly correlated with burn size (correlations ranging from -0.25 to -0.46, P < .01). The canonical correlation between ABOQ and burn size was 0.68 (P = .0002). The overall canonical correlation between two instruments was also significant (P < .0001). At the item level, at least 25% of the variation in each of the five ABOQ items could be explained by PROMIS-10 items and composite scores, while six other items could only be accounted for by less than 15% of the variation. ABOQ short form assessment can be used to efficiently measure burn outcomes across a range of relevant clinical domains with credible validity. A large proportion of the variation in ABOQ scores was not accounted for by PROMIS-10, suggesting that ABOQ provided additional health-related information specifically for the burn population beyond the generic instrument.