A critical analysis of test-retest reliability in instrument validation studies of cancer patients under palliative care: a systematic review

Cultural translation of the ethical dimension: a study on the reliability and validity of the Chinese nurses' professional ethical dilemma scale

BACKGROUND

Quantifying the professional ethical challenges that nurses encounter is crucial for both theoretical insights and practical outcomes. The objective of this research is to assess the psychometric properties of the Chinese adaptation of the Moral Distress Scale for Healthcare Professionals (MD-APPS).

METHODS

In 2024, a survey approach was utilized to engage with several tertiary-level healthcare institutions throughout China. A cohort of 448 nursing professionals who satisfied the specified selection benchmarks was consequently incorporated into the study. To evaluate the scale's reliability and validity, methods including the Content Validity Index (CVI), Factor Analysis-both Exploratory (EFA) and Confirmatory (CFA)-alongside assessments of internal consistency and test-retest reliability were employed.

RESULTS

Expert evaluations yielded an I-CVI of 0.90, suggesting good content validity for the MD-APPS's Chinese adaptation. Exploratory Factor Analysis (EFA) revealed a bi-dimensional framework with 7 components, explaining 56.34% of the cumulative variance. Confirmatory Factor Analysis (CFA) outcomes displayed a χ-square/df ratio of 1.542. The estimate for Robust RMSEA was 0.054, and the SRMR was ascertained to be 0.041. Indices for both Robust TLI and Robust CFI surpassed the 0.9 threshold, indicating an acceptable fit; this aspect was supported by a P-value (Chi-square) of 0.094. The internal consistency, measured by Cronbach's α, was found to be 0.74, while the test-retest reliability over a two-week period reached 0.964. These findings provide initial evidence for the psychometric properties of the Chinese MD-APPS.

CONCLUSION

The Chinese adaptation of the MD-APPS demonstrates promising initial psychometric properties, suggesting its potential suitability for exploring nurses' professional ethical challenges within the Chinese cultural context. This scale may facilitate the identification of diverse elements influencing nurses' professional ethics and the assessment of the ethical climate in nursing practices. However, further validation studies are needed to fully establish its psychometric robustness across various healthcare settings in China.

Cross-cultural adaptation, reliability, and validity of the Turkish Pediatric Quality of Life Inventory-Cognitive Functioning Scale (PedsQLTM-CFS) in children with cancer

The Pediatric Quality of Life Inventory-Cognitive Functioning Scale (PedsQLTM-CFS) was developed as a brief, general, symptom-specific tool to measure cognitive function. The 6-item PedsQL™ Cognitive Functioning Scale and PedsQL 3.0 Cancer Module answered 369 parents and 330 children with 5-18 years. Parents also completed Behavior Rating Inventory of Executive Function (BRIEF). The PedsQL™ Cognitive Functioning Scale evidenced excellent reliability (parent proxy-report α = 0.980/Fleiss Kappa: 0.794; children self-report α = 0.963/Fleiss Kappa: 0.790). Both child self-report and parent proxy-report PedsQL™ Cognitive Functioning Scale scores exhibited significant correlations with all parent-report BRIEF summary and subscale scores (p < .05). Both child self-report and parent proxy-report PedsQL™ Cognitive Functioning Scale scores exhibited significant correlations with PedsQL 3.0 Cancer Module total score and subscale scores (p < .05). The PedsQLTM-CFS can be used in high-risk populations with substantial to perfect reliability, both in regards to total/subcategory scores as well as in children with cancer.

Psychometric Evaluation of a Tablet-Based Tool to Detect Mild Cognitive Impairment in Older Adults: Mixed Methods Study

BACKGROUND

With the rapid aging of the global population, the prevalence of mild cognitive impairment (MCI) and dementia is anticipated to surge worldwide. MCI serves as an intermediary stage between normal aging and dementia, necessitating more sensitive and effective screening tools for early identification and intervention. The BrainFx SCREEN is a novel digital tool designed to assess cognitive impairment. This study evaluated its efficacy as a screening tool for MCI in primary care settings, particularly in the context of an aging population and the growing integration of digital health solutions.

OBJECTIVE

The primary objective was to assess the validity, reliability, and applicability of the BrainFx SCREEN (hereafter, the SCREEN) for MCI screening in a primary care context. We conducted an exploratory study comparing the SCREEN with an established screening tool, the Quick Mild Cognitive Impairment (Qmci) screen.

METHODS

A concurrent mixed methods, prospective study using a quasi-experimental design was conducted with 147 participants from 5 primary care Family Health Teams (FHTs; characterized by multidisciplinary practice and capitated funding) across southwestern Ontario, Canada. Participants included health care practitioners, patients, and FHT administrative executives. Individuals aged ≥55 years with no history of MCI or diagnosis of dementia rostered in a participating FHT were eligible to participate. Participants were screened using both the SCREEN and Qmci. The study also incorporated the Geriatric Anxiety Scale-10 to assess general anxiety levels at each cognitive screening. The SCREEN's scoring was compared against that of the Qmci and the clinical judgment of health care professionals. Statistical analyses included sensitivity, specificity, internal consistency, and test-retest reliability assessments.

RESULTS

The study found that the SCREEN's longer administration time and complex scoring algorithm, which is proprietary and unavailable for independent analysis, presented challenges. Its internal consistency, indicated by a Cronbach α of 0.63, was below the acceptable threshold. The test-retest reliability also showed limitations, with moderate intraclass correlation coefficient (0.54) and inadequate κ (0.15) values. Sensitivity and specificity were consistent (63.25% and 74.07%, respectively) between cross-tabulation and discrepant analysis. In addition, the study faced limitations due to its demographic skew (96/147, 65.3% female, well-educated participants), the absence of a comprehensive gold standard for MCI diagnosis, and financial constraints limiting the inclusion of confirmatory neuropsychological testing.

CONCLUSIONS

The SCREEN, in its current form, does not meet the necessary criteria for an optimal MCI screening tool in primary care settings, primarily due to its longer administration time and lower reliability. As the number of digital health technologies increases and evolves, further testing and refinement of tools such as the SCREEN are essential to ensure their efficacy and reliability in real-world clinical settings. This study advocates for continued research in this rapidly advancing field to better serve the aging population.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/25520.

Development and psychometric testing of a pediatric chronic graft-versus-host disease symptom scale: protocol for a two-phase, mixed methods study

Background

Chronic graft-versus-host disease (cGVHD) is a debilitating late complication of hematopoietic stem cell transplantation. It is often accompanied by extensive symptom burden. No validated cGVHD patient-reported outcome (PRO) measure exists to evaluate cGVHD symptom bother in children and adolescents younger than 18 years. This paper presents the study protocol for a multi-center, two-phase protocol to develop a psychometrically valid pediatric cGVHD Symptom Scale (PCSS) and a companion caregiver-proxy measure to capture the symptom burden experienced by children with cGVHD. In the first phase of the study, our aim is to evaluate the comprehension, clarity and ease of response of the PCSS through cognitive interviewing and to iteratively refine the measure to optimize content validity. In the second phase of the study, we will quantitatively examine the measurement properties of the PCSS in children and their caregiver-proxies.

Methods and analysis

Eligible participants are children/adolescents ages 5-17 with cGVHD who are receiving systemic immunosuppressive treatment or have recently tapered to discontinuation. In the first phase, we are enrolling 60 child and caregiver-proxy dyads in three child age strata (5-7, 8-12, and 13-17 years old). Semi-scripted cognitive debriefing interviews are conducted to assess comprehension, clarity, and ease of response of each PCSS item with the child alone, and then jointly with the caregiver-proxy to explore discordant ratings. In phase two, an age-stratified cohort of 120 child-caregiver dyads will be enrolled to evaluate test-retest reliability, construct validity, and responsiveness. Anchors for known-groups validity include the PedsQL module and clinical variables, including cGVHD clinician-rated severity scores. In participants ages 13-17, we will also compare responses on the PCSS with those from the Lee cGVHD Symptom Scale, to gauge the youngest age at which adolescent respondents can comprehend this adult measure.

Discussion

This study will yield a well-validated, counterpart measure to the Lee cGVHD Symptom Scale for use in children with cGVHD and their caregiver-proxies. This new patient-reported outcome measure can be integrated into clinical trials and care delivery for pediatric transplant survivors to improve the precision and accuracy with which their cGVHD symptom experience is captured.

Clinical trial registration

www.ClinicalTrials.gov, NCT04044365.

Screening Children in India: Translation and Psychometric Evaluation of the Parents' Evaluation of Developmental Status and the Strength and Difficulties Questionnaire

Timely screening and surveillance of children for developmental delay and social-emotional learning difficulties are essential in Low- and Middle-Income Countries like India. Screening measures like the Parents' Evaluation of Developmental Status (PEDS) and Strength and Difficulties Questionnaire (SDQ) are considered suitable for India due to their low cost, easy accessibility, and no training requirement for administration. However, India lacks validated screening measures, and the PEDS and SDQ have yet to be validated for children in India. The study aimed to translate the PEDS and SDQ from English to Hindi and psychometrically evaluate the same measures on children aged 4-8 years in India. The original PEDS and SDQ forms and their translations were pilot tested on 55 participants and evaluated using data from 407 children with typical development (TD) and 59 children with developmental disability (DD). Parents and teachers reported no meaningful discrepancy between the original and translated (Hindi) questionnaires. Internal consistency for the PEDS was acceptable, but unacceptable for most subscales on the SDQ, for both TD and DD samples. Test-retest reliability was poor for the PEDS but adequate for the SDQ. Results from known-group validity testing showed that the PEDS scores could be used to distinguish between the TD and DD samples. The results from this study provide further support for the use of the PEDS and SDQ in developing countries like India.

The Oral Symptom Assessment Scale (OSAS): criterion validation with the EORTC QLQ-OH15 and reliability testing

PURPOSE

The aim of this study was to investigate the criterion validity and reliability of the Oral Symptom Assessment Scale (OSAS) in patients with advanced cancer receiving specialist palliative care.

METHODS

To examine criterion validity, participants completed the OSAS, EORTC QLQ-C30 and EORTC QLQ-OH15. Twenty-four hours later participants repeated the OSAS to investigate test-retest reliability.

RESULTS

54 participants were recruited (median age 70; range: 35- 93 years). 51 complete datasets were obtained. Cohen's kappa test was used to evaluate the agreement for the presence / absence of symptoms on the OSAS on the first and second days (test-retest reliability). This analysis showed values of moderate and higher for agreement for all symptoms. All kappa values were statistically significant. The test-retest reliability for symptom severity, frequency and distress was assessed using Intraclass Correlation Coefficient. Spearman's rank correlation coefficients were used to evaluate agreement between similar questions on the OSAS from day one and the EORTC QLQ-OH15 also on day one to examine criterion validity.

CONCLUSION

This study supports the validity of the OSAS, and provides evidence for the reliability of this novel oral symptom assessment tool, in patients with advanced cancer. Further research is needed to corroborate the findings of this study.

TRIAL REGISTRATION

CancerTrials.gov registry registration no.: PM202166.

[Spanish translation and cross-cultural adaptation of the Pressure Ulcer Quality of Life Questionnaire (PU-QOL).]

OBJECTIVE

Pressure ulcers (PU) are the most prevalent of the dependency-related injuries, affecting the quality of life of the patients who suffer them. However, there are no instruments adapted to the Spanish context to evaluate this quality of life. The use of specific tools in Spanish to evaluate the quality of life perceived by patients with PUs is considered an indispensable element for healthcare decisions. The aim of this paper was to translate and culturally adapt the Pressure Ulcer Quality of Life Questionnaire (PU-QOL) into Spanish for the measurement of health-related quality of life in patients with pressure ulcers.

METHODS

A translation, back-translation, and pre-test method was used on the target population to obtain an adapted version of the original PU-QOL instrument. The area was Primary Care. The participants were fifteen Primary Care patients. Main steps: 1) Direct translation; 2) Synthesis and concordance of versions by an expert committee; 3) Back translation; 4) Consistency of the back translation with the author from the original questionnaire; and 5) Analysis of comprehensibility through cognitive interviews with a sample of patients.

RESULTS

An instrument to measure perceived quality of life in patients with PU was obtained, composed of ten scales and eighty-three items. The scales and items of the original questionnaire were maintained. Conceptual and semantic analysis produced adjustments in wording, clarification and reformulations adapted to the Spanish context.

CONCLUSIONS

We present this first phase of translation and cross-cultural adaptation of the PU-QOL questionnaire in Spanish, which could be a useful tool for decision-making on health care in patients with PUs.

Knowledge, attitude, opinion, perspective, and agreement of Palestinian medical students on strategies/recommendations to curb plagiarism: A multicenter cross-sectional study

Plagiarism is a common issue in written academic assignments and graduation theses. This multicenter study was conducted to assess the knowledge, attitude, opinion, perspective, and agreement of Palestinian medical students on strategies/recommendations to curb plagiarism. The study was conducted in a cross-sectional design using a questionnaire in all universities with medical education programs. The questionnaire contained 12 knowledge items, 8 attitude items, 6 opinion/perspective items, and 8 strategies/recommendations to curb plagiarism. Of the 550 invited medical students, 474 completed the study tool. Knowledge, attitude, opinion, perspective, and agreement on strategies/recommendations scores correlated positively. Higher knowledge, attitude, opinion, perspective, and agreement on strategies/recommendations scores were significantly associated with higher academic/training year, grade point average, satisfaction with academic achievement, academic writing skills, informational skills, using citation managers, receiving courses/workshops/lecturers on plagiarism, using plagiarism checking tools, and participation in a scientific paper/graduation thesis writing. Gaps in knowledge about plagiarism were identified among Palestinian medical students. Educators/trainers and other decision-makers in medical schools and higher academic institutions might use the strategies on which the students agreed to curb plagiarism.

Test-retest reliability of single-item assessments of immune fitness, mood, and quality of life

The use of single-item assessments is increasingly important and popular, as these enable quick real-time assessments in clinical practice or research. In this study we investigated the test-retest reliability of single-item assessments of mood ("stress", "anxiety", "depression", "fatigue", "loneliness", "being active", "optimism", and "happiness"), quality of life, and immune fitness in N = 108 participants. The analysis revealed high test-retest correlations between the single-item assessments (r = 0.67 to 0.90), moderate to excellent intraclass correlations (r = 0.672 to 0.889), and the Bland-Altman analysis revealed agreement between all test-retest assessments, except for depression. Taken together, it can be concluded that the single-item assessments of mood, quality and immune fitness have a good test-retest reliability. This strengthens the rationale for using these single item assessments.

Validating the Remote First Aid Self-Efficacy Scale for Use in Evaluation and Training of First Responders in Remote Contexts

INTRODUCTION

The purpose of this study was to investigate the psychometric properties of the remote first aid self-efficacy scale (RFA SES). The RFA SES is a 30-item self-report scale developed in response to emerging evidence showing that self-efficacy is predictive of performance.

METHODS

Trained alumni from Wilderness Medical Associates (WMA) training courses and less trained students from Laurentian University (LU) were recruited via email to complete an online questionnaire at 2 different periods (T1 and T2). The questionnaire included demographic questions, the 30-item RFA SES, the 10-item Connor-Davidson resilience scale (CD-RISC), and the 10-item generalized self-efficacy scale (GSES). Data analysis included assessment of the dimensionality, reliability, and validity of the scale.

RESULTS

There were 448 alumni from WMA and 1106 students from LU who participated in the study. The RFA SES demonstrated a clear unidimensional structure. The mean interitem correlation was 0.75 at T1. Test-retest reliability (T1 to T2) was high for both the LU group (intraclass correlation [ICC]=0.90) and the WMA group (ICC=0.92). Moderate correlations were found between RFA SES and CD-RISC (r=0.42, P<0.001), a general measure of resilience, and the GSES (r=0.48, P<0.001), a general measure of self-efficacy. Wilderness Medical Associates participants showed higher mean scores than LU students at T1 (t [569]=16.2, P<0.001).

CONCLUSIONS

The RFA SES is a unidimensional, reliable, and potentially valid scale. Further research should focus on item reduction followed by additional tests of reliability and validity.

Developing a Questionnaire on Knowledge, Perceptions and Application of Vascular-Aging Measurements

BACKGROUND

Vascular age (VA) is independent and chronological age for assessing cardiovascular disease (CVD) risk. However, tools for the implementation of VA are currently lacking. We aimed to develop a questionnaire to assess the current knowledge gaps related to VA and barriers to its implementation in routine practice.

METHODS

Using a stepwise mixed-method approach, a quantitative questionnaire was constructed in four phases: (1) basic item generation and the development of a semi-qualitative questionnaire (SQQ); (2) dissemination to the VascAgeNet extended network and an analysis of the semi-qualitative questionnaire responses; (3) the development of a quantitative questionnaire (QQ); and (4) an assessment of the content and face validity and internal reliability in an additional sample.

RESULTS

Based on six main topics initially identified through an expert panel, a SQQ was developed and disseminated. Finally, a 22-item QQ was developed, with questions grouped around three main themes: knowledge of VA and its risk factors; perceptions and beliefs regarding the importance and contribution of VA to risk classification; and the application of VA measurements in clinical and research practice and its potential limitations (Cronbach's alpha between 0.920 and 0.982 for all three categories).

CONCLUSION

We report the development of a QQ on VA addressed to both clinicians and non-clinicians aiming to assess their knowledge, perceptions and application of VA measurements.

Translation and Validation of Low Back Pain Knowledge Questionnaire Among Hindi-Speaking Indian Women

Low back pain (LBP) is a common problem encountered among women worldwide. This research aimed at the cross-cultural translation, adaptation, and equivalence assessment of the Low Back Pain Knowledge Questionnaire (LBPKQ) in the Hindi language. The LBPKQ, originally in English (E-LBPKQ) was translated and validated in the Hindi language (H-LBPKQ). The forward-backward procedure was adapted from the recommended guidelines for cross-cultural adaptation of measures. Psychometric properties of the H-LBPKQ were evaluated among 250 Hindi-speaking Indian women with LBP. Test-retest reliability was evaluated in the first 161 participants of the study sample. The overall LBPKQ scores obtained a mean of 3.1 ± 1.1. The internal consistency was excellent for both E-LBPKQ and H-LBPKQ, with Cronbach's alpha of 0.983 (95% confidence interval [CI], 0.980-0.986) and 0.975 (95% CI, 0.970-0.979), respectively. Construct validity was good, with Kaiser-Meyer-Olkin value of 0.892 for E-LBPKQ and 0.852 for H-LBPKQ. Bartlett's test of sphericity (P < .0001) was significant for E-LBPKQ and H-LBPKQ. Two factors were extracted through principal component analysis. The H-LBPKQ is valid and reliable to assess LBP knowledge among Hindi speaking population. Low LBPKQ scores indicate poor LBP knowledge; hence, LBP sensitization is needed among Indian women.

Psychometric properties of the Chinese version of the Cognitive Reserve Assessment Scale in Health in patients with cancer

BACKGROUND

Cognitive reserve is a modifiable factor that could prevent cognitive decline in patients with cancer. The Cognitive Reserve Assessment Scale in Health (CRASH) is an instrument used to assess cognitive reserve. This study aims to develop and examine the psychometric properties of the Chinese version of the CRASH for patients with cancer.

METHODS

A cross-sectional survey was conducted with 167 cancer patients from four wards of two hospitals in China. Thirty-one patients were re-assessed to examine the test-retest reliability. Four translators and three reviewers developed the Chinese version of the scale. We assessed its structural validity, concurrent validity, internal consistency, test-retest reliability, measurement error, and floor/ceiling effects.

RESULTS

Confirmatory factor analysis showed a good model fit with the four-factor structure of the original CRASH. The CRASH scores were statistically significantly associated with neuropsychological test scores, indicating sufficient concurrent validity. The internal consistency was acceptable, except for leisure activities, with standardized Cronbach's alphas (0.64-0.94) and standardized Omega (0.66-0.95). There was excellent test-retest reliability, with a high intraclass correlation coefficient (0.914-0.993) of total scores and scores for each domain. The measurement error was acceptable, and no floor or ceiling effects were observed.

CONCLUSIONS

The Chinese version of the CRASH is a valid and reliable instrument to assess cognitive reserve in patients with cancer. Moreover, cognitive reserve measured by the CRASH was associated with low cognitive performance in cancer patients.

Cross-sectional study examining psychometric properties of the Slovenian version of the 14-item Resilience Scale (RS-14-SL)

PURPOSE

Resilience has become an important concept in health research, addressing mental health outcomes. The purpose of this study was to translate, adapt, and evaluate psychometric properties of the Slovenian version of the 14-item Resilience Scale (RS-14), using a sample of general population and cancer survivors.

METHODS

The original version of the RS-14 was first back-translated and pilot tested. The factor structure was assessed with an exploratory analysis and confirmed with a confirmatory analysis. To assess reliability, internal consistency (Cronbach's alpha) and test-retest (ICC, t student test) were determined. Measurement validity was assessed with demographic (age and gender) and psychological characteristics (symptoms of anxiety and depression and health-related quality of life).

RESULTS

An exploratory factor analysis revealed one-factor solution in both samples and the fit indexes showed an acceptable model fit. Internal consistency showed excellent values (0.91-0.96), and test-retest reliability was found to be acceptable (ICC = 0.89). Significant correlations were found between RS-14 and anxiety and depression HADS subscales in Sample 1 (r = - 0.62, - 0.72 for anxiety and depression) and Sample 2 (r = - 0.43, r = - 0.51 for anxiety and depression) and the majority of EQ-5D subscales (usual activities, pain, anxiety/depression).

CONCLUSION

The study showed that the Slovenian version of the RS-14 test scores is valid and stable.

TRIAL REGISTRATION NUMBER

0120-25/2019/6.

Psychometric properties of the Slovenian version of the Cancer Survivors’ Unmet Needs (CaSUN-SL) measure in post-treatment cancer survivors

Background

As the number of cancer survivors is growing, valid instruments for assessing cancer survivors' needs are required. Thus, the aim of this study was to translate and validate the Cancer Survivors Unmet Needs (CaSUN) scale.

Methods

Cancer survivors were recruited from 30 family medicine practices and separated into two samples (sample 1, n = 147; sample 2, n = 148). Factor structure was explored with an exploratory analysis in sample 1 and determined with a confirmatory analysis in sample 2. Psychometric properties were assessed with internal consistency, test–retest reliability and construct validity.

Results

A translation and cultural adaptation of the CaSUN scale resulted in 34 items being included in the final version. The factor structure confirmed the five-factors solution of the CaSUN-SL. Cronbach’s alpha was 0.94 for the CaSUN-SL and ranged from 0.71–0.88 for specific domains. Test–retest reliability showed moderate-high stability over time. The CaSUN-SL significantly and positively correlated with anxiety (r = 0.49), depression (r = 0.44), health-related quality of life (r = 0.36), and negatively with self-perceived health (r =  − 0.36) and resilience (r =  − 0.47), which confirms the construct validity. In addition, we found a significant correlation between unmet needs and age (r =  − 0.29), gender (r = 0.14), cancer stage (r = 0.20), cancer type (r = 0.19), and time since treatment (r =  − 0.20).

Conclusions

Results indicate that CaSUN-SL is a valid and reliable measure to assess the Slovenian cancer survivors’ unmet, met and total needs and can be used for further prospective studies.

Trial Registration: No. 0120-25/2019/6.

Self-assessed tactical skills in tennis players: Psychometric evaluation of the Tactical Skills Questionnaire in Tennis

To our knowledge, no feasible, valid and reliable instrument exists to examine tactical skills over the course of multiple training and game situations in tennis yet. Therefore, the aim of this study was to develop and evaluate the psychometric properties of the Tactical Skills Questionnaire in Tennis (TSQT). The TSQT is a new instrument with closed-ended questions designed to examine tactical skills in tennis players. Participants were 233 competitive tennis players (age: 17.06 ± 4.74 years) competing on national or regional levels. With a principal component analysis (PCA) we identified four theoretically meaningful subscales for the 31-item TSQT: “Anticipation and positioning,” “Game intelligence and adaptability,” “Decision-making,” and “Recognizing game situations” and confirmed them with a confirmatory factor analysis (CFA) (χ² = 527.02, df = 426, p < 0.001, CFI = 0.93, RMSEA = 0.045, SRMR = 0.079). Internal consistency was good, with Cronbach's alpha of 0.89 for the entire scale and McDonald's omega ranging from 0.69 to 0.78 for the separate subscales. A subsample of 57 players completed the TSQT twice to assess test-retest reliability. Absolute test-retest reliability of the subscales was good with no significant differences in mean scores between test and retest (p > 0.05). Relative test-retest reliability was moderate with ICC values ranging from 0.65 to 0.71. National players outperformed regional players on the subscales “Game intelligence and adaptability,” “Decision-making,” and “Recognizing game situations” (p < 0.05), and there was a trend toward significance for “Anticipation and positioning” (p = 0.07). This study supported the psychometric properties of the TSQT. Evaluating tactical skills with the TSQT provides players, coaches and other professionals with insight in players' self-assessed tactical skills over the course of multiple training and game situations. It creates the opportunity for players to reflect on their skills and detect personal development areas with their coach. We advise to use this information as input for tailor-made training programs.

Hindi translation and reliability testing of the international spinal cord injury quality of life basic data set version 1.0

STUDY DESIGN

Methodological study design: translation and test-retest reliability testing.

OBJECTIVES

To translate International Spinal Cord Injury Quality of Life Basic Data Set (ISCIQoLBDS) Version 1.0 into Hindi language and conduct reliability testing of Hindi ISCIQoLBDS (hISCIQoLBDS).

SETTING

The study was conducted at Indian Spinal Injuries Centre, New Delhi, India.

METHODS

The study was done in two phases-translation of ISCIQoLBDS Version 1.0 into Hindi language and reliability testing of hISCIQoLBDS. In the translation phase, the data set was forward translated by two independent professional translators. Then backward translation was done by another pair of professional translators to ensure the essence of the data set. For their feedback, field testing of hISCIQoLBDS was done on ten individuals with Spinal Cord Injury (SCI). After this, an expert panel reviewed their recommendations, and the final hISCIQoLBDS was administered on 50 participants with SCI for test-retest reliability of the data set.

RESULTS

hISCIQoLBDS was found to be equivalent to ISCIQoLBDS Version 1.0 on back-translation. It showed high test-retest reliability for each item in the data set. Intraclass Correlation Coefficient (95% CI) for items of hISCIQoLBDS was within the range of (0.91-0.98).

CONCLUSION

The hISCIQoLBDS is a stepwise systematic Hindi translation of the ISCIQoLBDS Version 1.0 for the Hindi literate population with high test-retest reliability. It can facilitate international data exchange on the quality of life (QoL) of Hindi literate individuals with SCI.

Cultural adaptation of the Italian version of the Patient-Reported Outcomes Common Terminology Criteria for Adverse Event (PRO-CTCAE®)

INTRODUCTION

US National Cancer Institute's (NCI) Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE^®) is a library of 78 symptom terms and 124 items enabling patient reporting of symptomatic adverse events in cancer trials. This multicenter study used mixed methods to develop an Italian language version of this widely accepted measure, and describe the content validity and reliability in a diverse sample of Italian-speaking patients.

METHODS

All PRO-CTCAE items were translated in accordance with international guidelines. Subsequently, the content validity of the PRO-CTCAE-Italian was explored and iteratively refined through cognitive debriefing interviews. Participants (n=96; 52% male; median age 64 years; 26% older adults; 18% lower educational attainment) completed a PRO-CTCAE survey and participated in a semi-structured interview to determine if the translation captured the concepts of the original English language PRO-CTCAE, and to evaluate comprehension, clarity and ease of judgement. Test-retest reliability of the finalized measure was explored in a second sample (n=135).

RESULTS

Four rounds of cognitive debriefing interviews were conducted. The majority of PRO-CTCAE symptom terms, attributes and associated response choices were well-understood, and respondents found the items easy to judge. To improve comprehension and clarity, the symptom terms for nausea and pain were rephrased and retested in subsequent interview rounds. Test-retest reliability was excellent for 41/49 items (84%); the median intraclass correlation coefficient was 0.83 (range 0.64-0.94).

DISCUSSION

Results support the semantic, conceptual and pragmatic equivalence of PRO-CTCAE-Italian to the original English version, and provide preliminary descriptive evidence of content validity and reliability.

Spanish cross-cultural adaptation and validation of the Australian Pelvic Floor Questionnaire in running women

Pelvic floor dysfunctions are a wide range of disorders in the gynaecological, lower urinary and gastrointestinal tracts that affect the structure and/or function of the pelvic organs. The objective of this study was to carry out a cross-cultural adaptation and a psychometric analysis of the Spanish version of the Australian Pelvic Floor Questionnaire. Observational study divided into two main phases: (1) translation and cross-cultural adaptation and (2) psychometric tests. Women runners from all over the Spanish territory, from different federations, clubs and levels were recruited. Participants: 424 female runners, native Spanish, over 18 years of age and who had been practicing running for more than 6 months. The instruments used in this study were the Australian Pelvic Floor Questionnaire, Female Sexual Function Index, King Health Questionnaire, Quality of Life SF-12 and EuroQoL 5-D. The Spanish version of Australian Pelvic Floor Questionnaire has proven to be an understandable and easy-to-use tool. The general internal consistency of the questionnaire was 0.972 and the intraclass correlation coefficient ranged between ICC 0.596-0.960. The Spanish version of Australian Pelvic Floor Questionnaire is a valid and reliable measure that can be used clinically to assess pelvic floor dysfunctions among the female Spanish population.

Intra-rater reliability and feasibility of the HexCom advanced disease complexity assessment model

Aim

To study the intra-rater reliability and feasibility of the HexCom complexity assessment model by analyzing internal consistency, intra-rater reliability and response time.

Design

Test–retest study with a selection of 11 clinical situations that cover the full scope of situations assessed by the HexCom model and which are responded to individually.

Location

Home care, primary care, hospital and sociosanitary care. Two specialized palliative home care teams (PADES).

Participants

A total of 20 professionals comprising 10 experts in palliative home care (PADES) and 10 professionals from general palliative care (primary care, hospital and sociosanitary care). These professionals came from the fields of family medicine (5), internal medicine (2), geriatrics (2), nursing (9), psychology (1) and social work (1).

Main measurements

Cronbach's alpha, weighted kappa, response time.

Results

Cronbach's alpha of 0.91 for HexCom-Red and 0.87 for HexCom-Clin. Intra-rater reliability ranging from good to very good for HexCom-Red (kappa: 0.78–1) and from moderate to very good for HexCom-Clin (kappa: 0.58–0.91). Average response time of 0:57 for HexCom-Red and 3:80 min for HexComClin.

Conclusions

HexCom-Red and HexCom-Clin are reliable tools and feasible for use by all professionals involved in both general and specialized palliative care at different levels.

Psychometric Assessment of SpiDiscI: Spiritual Distress Scale for Palliative Care Patients in India

Objectives:

Spirituality has an impact on the quality of life of palliative care patients and it influences the way in which they experience their disease. Spiritual distress is a common issue among palliative care patients in India that is best assessed through a tool specifically designed for them. This study presents the findings of a psychometric assessment of SpiDiscI: a 16-item spiritual distress scale for Hindi speaking palliative care patients in India.

Materials and Methods:

This study used a non-experimental cross-sectional sample survey design. 400 cancer patients undergoing pain treatment at the pain and palliative care unit of a tertiary cancer hospital in New Delhi enrolled in the study and completed the study’s questionnaire. The questionnaire contained a section on demographic information, a numerical rating scale (NRS) for pain, SpiDiscI, FACIT-Sp-12, and WHOQOL-BREF. A subset of 50 patients with stable pain completed SpiDiscI a 2^nd time 2 weeks after the first administration of the scale to assess test-retest reliability.

Results:

The scale’s internal consistency is very good (Cronbach’s Alpha 0.85). Spiritual distress scores ranged from 0 to 93.8 (mean 37.1, SD 23.6). Significant negative correlations between SpiDiScI and both FACIT-Sp-12 (R = –0.16, P = 0.001) and WHOQOL-BREF (R = –0.27, P < 0.001) confirmed convergent validity. There was no significant association with NRS pain scores (R = 0.06, P = 0.224). The highly significant association (R = 0.75, P < 0.001) of the scores on assessments 2 weeks apart confirmed test-retest reliability.

Conclusion:

SpiDiScI is a reliable and valid measure to assess spiritual distress in research among Hindi speaking palliative care patients in India.

Exploring moderators of mirror exposure on pre- to post changes in body image outcomes: Systematic review and meta-analysis

PURPOSE

Mirror exposure (ME) is embedded within cognitive behavioral treatment of body image concern (BIC), and existing literature demonstrates its potential value as a BIC intervention. The purpose of the study is to provide a meta-analytic review of ME effects on attitudinal and behavioral BIC.

METHODS

Twelve studies met inclusion criteria, yielding six between-group and 18 within-group comparisons.

RESULTS

Random effects models demonstrated a medium effect of ME versus control conditions, on attitudinal BIC, from pre- to immediate posttreatment, small effect at follow-up (1 to 4 weeks posttreatment), and large effect on behavioral BIC pre- to immediate posttreatment. However, small sample sizes yielded nonsignificant findings and significant heterogeneity. Exploratory moderator analyses-ME type, ME dosage (session frequency, duration), and diagnosis-demonstrated that, among single group pre-post designs, higher ME dosage may be instrumental to pre-posttreatment improvements. Consistent with this finding, individual studies with high dosages of ME demonstrated the largest effect sizes. Conclusion: Results may suggest a strong ME effect on BIC at high dosages (>120 minutes). The current review highlights gaps in ME literature and the need for future clinical trials assessing ME, using a priori, high-quality research designs that further the understanding of ME's mechanisms of change.

Validity and reliability of the Chinese version of the partners at care transitions measure

BACKGROUND

The Partners at Care Transitions Measure (PACT-M) is a measure that assesses the quality and safety of care during the transition from hospital to home from the patient's perspective. The aim of this study was to examine the psychometric properties of the Chinese version of the PACT-M in Mainland China.

METHODS

This was a cross-sectional study. A convenience sample of patients was recruited from three tertiary hospitals affiliated with Zhengzhou University, China. A total of 402 participants were interviewed before discharge, and 306 participants were interviewed one month after discharge from hospital to home using the Chinese version of the PACT-M. The statistical methods used in this study include the critical ratio value, item total correlation, test-retest, Cronbach's alpha, confirmatory factor analysis and exploratory factor analysis.

RESULTS

The Chinese version of the PACT-M consists of PACT-M₁ and PACT-M₂, both of which have two dimensions, the number of items in both parts are consistent with the original English language version. The Cronbach's alpha values of the PACT-M₁ and PACT-M₂ were 0.802 and 0.741, and the test-retest reliability values were 0.885 and 0.837. The item content validity index and scale content validity index values of the PACT-M₁ and PACT-M₂ were all 1.0.

CONCLUSION

The Chinese version of the PACT-M shows acceptable validity and reliability and can be used to assess the quality and safety of transitional care from hospital to home from the patient's perspective in mainland China.

Reliability, validity and acceptability of the traditional Chinese version of the carer support needs assessment tool in Hong Kong palliative care settings

Background

Among the few existing needs assessment tools for family carers, the 14-item Carer Support Needs Assessment Tool (CSNAT) is the only brief and holistic needs screening tool designed for everyday use in palliative care practices. The aim of this study was to evaluate the reliability, validity, and acceptability of the traditional Chinese version of the CSNAT in palliative care settings in Hong Kong.

Methods

This adopted a cross-sectional and correlation design with repeated measures. The participants were 125 family carers of palliative cancer patients and 10 healthcare providers (HCPs) that were recruited from two local hospitals. The evaluation of psychometric properties included the following: (1) content validity through HCPs including frontline physicians, nurses, social workers, and clinical psychologists; (2) construct validity between the CSNAT items and those of the validated tools that measured caregiver burden, social support, and caregiving self-efficacy; and (3) one-week test-retest reliability in a sub-sample of 81 caregivers. The acceptability of the tool was assessed by the carers using several closed-ended questions.

Results

The content validity index of the CSNAT at the scale level was 0.98. Each item of the CSNAT was significantly and moderately correlated with caregiver burden (Spearman’s r = 0.24 to 0.50) and caregiving self-efficacy (r = − 0.21 to − 0.52), but not for social support. All CSNAT items had fair to moderate test-retest reliability (weighted kappa = 0.21 to 0.48), with the exception of two items “managing your relatives’ symptoms, including giving medicines” and “having time for yourself in the day”.

Regarding the acceptability of the CSNAT, almost all HCPs were willing to use the CSNAT for carer assessment and support. 89.6% of the carers demonstrated a comprehensibility of the CSNAT tool and 92.9% felt comfortable answering the questions. Around 90% of the carers agreed to use the tool for screening, discussing needs, and making referrals.

Conclusion

The traditional Chinese version of the CSNAT is a tool with high validity and acceptability and adequate reliability that measures family carers’ support needs, which should be considered for wide application in local palliative care practices.

Both subscales of the Keratoconus End-Points Assessment Questionnaire have excellent test-retest reliability

Purpose:

The keratoconus end-points assessment questionnaire (KEPAQ) is a disease-specific scale designed to evaluate the quality of life in keratoconus patients and provides the measurement of both functional and emotional compromise in keratoconus. It was previously developed, tested, and validated and now we want to evaluate the test-retest reliability of the KEPAQ, in an effort to contribute evidence on its internal consistency and capability of measuring clinical state with minimal inference of random chance.

Methods:

This is a prospective analytical study, designed to evaluate the test-retest reliability of the KEPAQ through the repeated application of the questionnaire to a group of clinically stable individuals. A number of patients with a confirmed diagnosis of keratoconus underwent double application of the KEPAQ, seven days apart. Mean KEPAQ score was obtained through Rasch analysis, while test-retest reliability was evaluated through Spearman rank-order correlation and intraclass correlation coefficient. Rasch analysis was performed in JMetrik version 4.1.1 (Psychomeasurement Systems LLC; Charlottesville, VA, USA) in a MacBook Air computer running macOS Catalina version 10.15.2 (Apple Inc.; Cupertino, CA, USA).

Results:

A total of 100 patients were included. For KEPAQ-E, Spearman correlation was R = 0.963 while ICC was 0.981 (95% confidence interval 0.972–0.987). For KEPAQ-F, Spearman correlation was R = 0.921 while ICC was 0.952 (95% confidence interval 0.929–0.968).

Conclusion:

The KEPAQ is a robust, well-developed, extremely reliable scale which can be confidently used for clinical and research endeavors.

COVID-19 related mental health issues: a narrative review of psychometric properties of scales and methodological concerns in scale development

OBJECTIVES

The global crisis of COVID-19 and its consequential strict public health measures placed around the world have impacted mental health. New scales and tools have been developed to measure these mental health effects. This narrative review assesses the psychometric properties of these scales and tools and methodological aspects of their development.

METHODS

PubMed, PubMed Central, and Google Scholar were searched for articles published from 15 May 2020 to 15 August 2020. This search used three groups of terms ("tool" OR "scale" AND "mental" OR "psychological"; AND "COVID-19" OR "coronavirus"). The identified scales were further evaluated for their psychometric properties and methodological aspects of their development.

RESULTS

Though the studies developing these scales (n = 12) have demonstrated their robust psychometric properties, some methodological concerns are noteworthy. Most of the scales were validated using internet-based surveys, and detailed descriptions of the mode of administration, sampling process, response rates, and augmentation strategies were missing.

CONCLUSIONS

The heterogeneous and inadequate reporting of methods adopted to evaluate the psychometric properties of the identified scales can limit their utility in clinical and research settings. We suggest developing guidelines and checklists to improve the design and testing, and result in reporting of online-administered scales to assess the mental health effects of the COVID-19 pandemic.

The Central Aspects of Pain in the Knee (CAP-Knee) questionnaire; a mixed-methods study of a self-report instrument for assessing central mechanisms in people with knee pain

OBJECTIVES

Pain is the prevailing symptom of knee osteoarthritis. Central sensitisation creates discordance between pain and joint pathology. We previously reported a Central Pain Mechanisms trait derived from eight discrete characteristics: Neuropathic-like pain, Fatigue, Cognitive-impact, Catastrophising, Anxiety, Sleep disturbance, Depression, and Pain distribution. We here validate and show that an 8-item questionnaire, Central Aspects of Pain in the Knee (CAP-Knee) is associated both with sensory- and affective- components of knee pain severity.

METHODS

Participants with knee pain were recruited from the Investigating Musculoskeletal Health and Wellbeing study in the East Midlands, UK. CAP-Knee items were refined following cognitive interviews. Psychometric properties were assessed in 250 participants using Rasch-, and factor-analysis, and Cronbach's alpha. Intra-class correlation coefficients tested repeatability. Associations between CAP-Knee and McGill Pain questionnaire pain severity scores were assessed using linear regression.

RESULTS

CAP-Knee targeted the knee pain sample well. Cognitive interviews indicated that participants interpreted CAP-Knee items in diverse ways, which aligned to their intended meanings. Fit to the Rasch model was optimised by rescoring each item, producing a summated score from 0 to 16. Internal consistency was acceptable (Cronbach's alpha = 0.74) and test-retest reliability was excellent (ICC2,1 = 0.91). Each CAP-Knee item contributed uniquely to one discrete 'Central Mechanisms trait' factor. High CAP-Knee scores associated with worse overall knee pain intensity, and with each of sensory- and affective- McGill Pain Questionnaire scores.

CONCLUSION

CAP-Knee is a simple and valid self-report questionnaire, which measures a single 'Central Mechanisms' trait, and may help identify and target centrally-acting treatments aiming to reduce the burden of knee pain.

Psychometric Evaluation and Workflow Integration Study of a Tablet-Based Tool to Detect Mild Cognitive Impairment in Older Adults: Protocol for a Mixed Methods Study

BACKGROUND

With the rapid aging of the global population, experts anticipate a surge in the prevalence of mild cognitive impairment (MCI) and dementia worldwide. It is argued that developing more sensitive, easy to administer, and valid MCI screening tools for use in primary care settings may initiate timely clinical and personal care planning and treatment, enabling early access to programs and services. Including functional competence measures in screening tests makes them more ecologically valid and may help to identify cognitive deficits at an earlier stage.

OBJECTIVE

We aim to conduct a preliminary evaluative study comparing the sensitivity, specificity, and reliability of the BrainFx Screen (referred to as SCREEN hereafter), a novel digital tool designed to assess functional competence and detect early signs of cognitive impairment, with the Quick Mild Cognitive Impairment, a validated and highly sensitive tool that detects MCI in the older adult population. We will also investigate the perceived usefulness and integration of the SCREEN into primary care practice to identify demonstrable impacts on clinical workflow and health care providers' (HCP) perceptions of its success as a screening tool. Patients' perceptions of completing the SCREEN and its impact on their quality of life will also be explored.

METHODS

This study has a concurrent, mixed methods, prospective, and quasi-experimental design. Participants will be recruited from 5 primary care family health teams (FHTs; defined by multidisciplinary practice and capitated funding) across southwestern Ontario, Canada. Participants will include HCPs, patients, care partners, and FHT administrative executives. Patients 55 years and older with no history of diagnoses for MCI, dementia, or Alzheimer disease rostered in one of the FHTs participating in the study will be eligible to participate. Their care partners will help triangulate the qualitative data collected from patients. Participating FHTs will identify an occupational therapist from their site to participate in the study; this HCP will both administer the research protocol and participate in semistructured in-depth interviews and questionnaires. Principal component analysis will be conducted on the SCREEN data to understand the test components better. Tests comparing sensitivity, specificity, and test-retest reliability will assess the validity of SCREEN as a screening tool for MCI.

RESULTS

This paper describes the study protocol and its activities to date. Data collection was halted early because of COVID-19 restrictions on research activity, and data analysis is currently in progress.

CONCLUSIONS

At the end of the project, we anticipate having an initial comparative evaluation of the SCREEN as a tool for early detection of MCI in primary care older adult patient populations. Resource constraints on this research study limit our ability to conduct a randomized controlled trial; however, the results will assist developers of the SCREEN in determining whether rigorous controlled testing is warranted.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/25520.

Cross-cultural adaptation and validation of the Norwegian version of the Leicester Cough Questionnaire in chronic obstructive pulmonary disease

Background: Cough is a common symptom among patients with chronic obstructive pulmonary disease (COPD), but there are currently no patient-reported outcome measures (PROM) for subjective cough symptoms in Norwegian.Objective: The objective was to translate and validated the most widely used PROM for chronic cough, the Leicester Cough Questionnaire (LCQ) from English to Norwegian (NLCQ) using established guidelines.Methods: Forward and backward translations were performed using external translators. All phases were discussed in an expert workgroup until consensus was achieved. Ten patients were interviewed about the pre-final version to assess understandability and ease to complete. The final NLCQ was externally proofread and tested for content and construct validity, internal consistency, test-retest reliability and measurement error.Results: Mean (SD) age was 67 (7.9) years, and 62% of the sample were female (n = 56/89). According to GOLD stages, 10% had mild, 27% moderate, 35% severe and 22% very severe COPD, respectively. GOLD stages were missing for five patients (6%). The NLCQ showed acceptable accordance with the original, acceptable understandability and ease to complete, satisfactory content reliability, total score construct validity, internal consistency and test-retest reliability. Test-retest bias was low, but measurement error was larger than previously reported. The standard error of measurement and smallest detectable change were 1.56 and 4.32, respectively. Construct validity and factor analysis raised concerns regarding the three subdomains.Conclusion: The present cross-cultural adaptation of the NLCQ showed satisfactory reliability and overall validity. Due to concerns for the validity of the three domains, we suggest use should be restricted to the NLCQ total score until these are reassessed.

Comprehensive School Physical Activity Program Policies and Practices Questionnaire (CSPAP-Q)

Purpose: The Comprehensive School Physical Activity Program (CSPAP) model has recently been adopted as a national framework for physical activity (PA) promotion in schools. As a result, there is a need to develop an instrument to help leaders of PA measure policies and practices across the five component areas of a CSPAP. To address this gap, our team systematically developed and assessed the psychometric properties of the CSPAP-Q, an instrument designed to assess school policies and practices related to the five components of CSPAP. Methods: The instrument development process began by reviewing existing school health and PA surveys. The Delphi method was then used to evaluate face and content validity through expert feedback. Thirty-eight physical education teachers completed the CSPAP-Q twice over a 14-day period to further establish validity and to assess test-retest reliability. Results: In total, 117 items were identified for the initial review. Upon completion of expert feedback, 22 items were eliminated and 66 items were revised. After reliability testing, three additional items were eliminated due to low test-retest agreement levels (Kappa <.40; non-significant χ²). Ninety-two items were retained in the final version of the CSPAP-Q and kappa values indicated moderate to substantial agreement among items. Conclusions: After validity and reliability testing, results suggest that the CSPAP-Q is a valid and reliable tool for assessing policies and practices related to CSPAP.

The Reliability of Remote Patient-Reported Outcome Measures via Mobile Apps to Replace Outpatient Visits After Rotator Cuff Repair Surgery: Repetitive Test-Retest Comparison Study for 1-Year Follow-up

BACKGROUND

With the development of health care-related mobile apps, attempts have been made to implement remote patient-reported outcome measures (PROMs). In order for remote PROMs to be widely used by mobile apps, the results should not be different depending on the location; that is, remote PROM results performed in locations other than hospitals should be able to obtain reliable results equivalent to those performed in hospitals, and this is very important. However, to our knowledge, there are no studies that have assessed the reliability of PROMs using mobile apps according to the location by comparing the results performed remotely from the hospital and performed at the outpatient visits.

OBJECTIVE

The purpose of this study was to evaluate the reliability of remote PROMs using mobile apps compared to PROMs performed during outpatient follow-up visits after arthroscopic shoulder surgery.

METHODS

A total of 174 patients who underwent arthroscopic rotator cuff repair completed questionnaires 2 days before visiting the clinic for the 1-, 2-, 3-, 6-, and 12-month follow-ups (test A). The patients completed the questionnaires at the clinic (test B) using the same mobile app and device for the 1-, 2-, 3-, 6-, and 12-month follow-ups. Test-retest comparisons were performed to analyze the differences and reliability of the PROMs according to the period.

RESULTS

Comparisons of tests A and B showed statistically significant differences at 1, 2, and 3 months (all Ps<.05 except for the ASES function scale at 3-months) but not 6 or 12 months after surgery (all Ps>.05). The intraclass correlation values between the two groups were relatively low at the 1-, 2-, and 3-month follow-ups but were within the reliable range at 6 and 12 months after surgery. The rate of completion of tests A and B using the mobile app was significantly lower in the group older than 70 years than in the other groups for all postoperative periods (P<.001).

CONCLUSIONS

PROMs using mobile apps with different locations differed soon after surgery but were reliably similar after 6 months. The remote PROMs using mobile apps could be used reliably for the patient more than 6 months after surgery. However, it is to be expected that the use of mobile app-based questionnaires is not as useful in the group older than 70 years as in other age groups.

Selection of trustworthy crowd workers for telemedical diagnosis of pediatric autism spectrum disorder

Crowd-powered telemedicine has the potential to revolutionize healthcare, especially during times that require remote access to care. However, sharing private health data with strangers from around the world is not compatible with data privacy standards, requiring a stringent filtration process to recruit reliable and trustworthy workers who can go through the proper training and security steps. The key challenge, then, is to identify capable, trustworthy, and reliable workers through high-fidelity evaluation tasks without exposing any sensitive patient data during the evaluation process. We contribute a set of experimentally validated metrics for assessing the trustworthiness and reliability of crowd workers tasked with providing behavioral feature tags to unstructured videos of children with autism and matched neurotypical controls. The workers are blinded to diagnosis and blinded to the goal of using the features to diagnose autism. These behavioral labels are fed as input to a previously validated binary logistic regression classifier for detecting autism cases using categorical feature vectors. While the metrics do not incorporate any ground truth labels of child diagnosis, linear regression using the 3 correlative metrics as input can predict the mean probability of the correct class of each worker with a mean average error of 7.51% for performance on the same set of videos and 10.93% for performance on a distinct balanced video set with different children. These results indicate that crowd workers can be recruited for performance based largely on behavioral metrics on a crowdsourced task, enabling an affordable way to filter crowd workforces into a trustworthy and reliable diagnostic workforce.

Development and Validation of a Clinical Grading Scale to Assess the Vulvar Region: The Vulvar Architecture Severity Scale

BACKGROUND

The vulva is composed of aesthetic units that can be affected differently by vulvar conditions. A reliable, comprehensive, and quick-to-use clinical scoring system is required to assess the disease extent in the vulvar area.

OBJECTIVES

The aim of this study was to develop and validate a grading scale based on the aesthetic unit principle to evaluate the extent of vulvar lichen sclerosus (VLS).

METHODS

After reviewing photographs of 100 patients affected by VLS, the authors targeted the aesthetic units most frequently affected. The disease signs were recorded and graded in 4 levels of severity (none, mild, moderate, severe) taking into account the vulvar architecture and skin involvement. To validate the scale, 14 observers were asked to apply it to photographs of 25 VLS patients on 2 different occasions. Intra- and inter-observer reliabilities were determined employing Pearson's and intraclass correlation coefficients.

RESULTS

A 6-region, 4-point grading system was designed and identified as the Vulvar Architecture Severity Scale (VASS). In all 6 areas, the Pearson's r was greater than 0.9 (mean, 0.994; 95% confidence interval [CI] = 0.992), indicating that the intra-observer reliability of the VASS was consistent over time (P < 0.001). Intraclass correlation at time 1 was 0.928 (95% CI = 0.910, 0.943) and at time 2 was 0.944 (95% CI = 0.931, 0.996), indicating a high reliability level among different observers.

CONCLUSIONS

The VASS is a reliable scale to assess the severity of VLS, and it might be considered as an outcome measure in future VLS trials.

LEVEL OF EVIDENCE: 4

Post-stroke Quality of Life Index: A quality of life tool for stroke survivors from Sri Lanka

Background

Burden of stroke is rising due to the demographic and epidemiological transitions in Sri Lanka. Assessment of success of stroke-management requires tools to assess the quality of life (QOL) of stroke survivors. Most of currently used QOL tools are developed in high-income countries and may not reflect characteristics relevant to resource-constrained countries. The aim was to develop and validate a new QOL tool for stroke survivors in Sri Lanka.

Methods

The COnsensus-based Standards for the selection of health Measurement Instruments (COSMIN) checklist was referred. A conceptual framework was prepared. Item generation was done reviewing the existing QOL tools, inputs from experts and from stroke survivors. Non-statistical item reduction was done for the 36 generated items with modified-Delphi technique. Retained 21 items were included in the draft tool. A cross sectional study was done with 180 stroke survivors. Exploratory Factor Analysis was done and identified factors were subjected to varimax rotation. Further construct validity was tested with 6 a-priori hypothesis using already validated tools (SF-36, EQ-5D-3 L) and a formed construct. Internal consistency reliability was assessed with Cronbach alpha.

Results

Four factors identified with principal-component-analysis explained 72.02% of the total variance. All 21 items loaded with a level > 0.4. The developed tool was named as the Post-stroke QOL Index (PQOLI). Four domains were named as “physical and social function”, “environment”, “financial-independence” and “pain and emotional-wellbeing”. Four domain scores of PQOLI correlated as expected with the SF-36, EQ-5D Index and EQ-5D-VAS scores. Higher domain scores were obtained for ambulatory-group than the hospitalized-group. Higher scores for financial-independence domain were obtained for the group without financial-instability. Five a-priori hypothesis were completely proven to be true. Cronbach-alpha level ranged from 0.682 to 0.906 for the four domains.

Conclusions

There is first evidence for sufficient construct validity of the PQOLI as a valid QOL tool for measuring the QOL of stroke survivors with satisfactory internal consistency reliability.

Psychometric properties of a single-item visual analog scale measuring goals of care in patients with advanced cancer

PURPOSE

The purpose of this study was to examine the psychometric properties of a single-item visual analog scale (VAS) to measure goals of care in patients with advanced cancer.

METHODS

Data were obtained from 378 patients with diagnoses of advanced lung, gastrointestinal, or pancreatic cancer. Goal of care was measured at baseline and every 3 months until patient death or completion of the 15-month study period. A single-item VAS ranging from 0 (quality of life is all that matters) to 100 (length of life is all that matters) was used to measure patients' goals of care for all study subjects; a subsample of subjects also completed the Quality of Life-Length of Life scale which asked patients to select categories of preferences. Test-retest reliability (intra-class correlation) and construct validity (known-groups, convergent, divergent) were evaluated.

RESULTS

At 9 and 12 months, the test-retest reliability for patients with stable symptoms (n = 107) was established with the ICC_(1,3) = 0.81, p < .001. Known-groups (r = 0.99, p < .001), convergent (r = 0.78, p < .001), and divergent (r = .06, p = 0.24) validity all demonstrated evidence of good construct validity.

CONCLUSIONS

Preliminary psychometric testing for a single-item VAS that measures goals of care in a sample of patients with advanced cancer met standard requirements for reliability and validity. While further testing with a larger sample size is recommended, the tool's use in the clinical area to assess cancer patients' goals of care is appropriate. Such a tool could facilitate goals of care discussions in the clinical area.

Bristol rheumatoid arthritis fatigue scale is valid in patients with psoriatic arthritis and is associated with overall severe disease and higher comorbidities

AIMS

To (1) determine the reliability and validity of the Bristol Rheumatoid Arthritis Fatigue scale (BRAF-NRS) in patients with psoriatic arthritis (PsA) and (2) examine possible clinical associations of worse fatigue in PsA.

METHODS

Study phase 1: BRAF-NRS scale validation cohort. A consecutive cohort of 70 PsA patients was recruited to complete the 3-item BRAF-NRS and the 13-item Functional Assessment of Chronic Illness Therapy-Fatigue (FACIT-F) questionnaires, alongside disease activity assessment. All patients also completed the BRAF-NRS questionnaire, 1 day later. Study phase 2: Identifying the potential clinical associations of fatigue by using BRAF-NRS (n = 283). A second cohort of 283 PsA patients underwent detailed skin and rheumatologic assessments including disease activity measures. Comorbidities were measured using the Charlson comorbidity index (CCI). Factors predicting worse fatigue as measured by BRAF-NRS were determined using regression analysis.

RESULTS

In phase 1, 67 out of 70 patients from the first cohort had complete assessments. The internal consistency of BRAF-NRS as measured by Cronbach's alpha was 0.92. Test-retest reliability as measured by the intra-class correlation coefficient was 0.97. There was excellent correlation between the BRAF-NRS and FACIT-F score(r = - 0.83) (p = <0.001, 95% CI - 0.74 to - 0.91). In phase 2, using data from the second cohort of 283 PsA patients, possible clinical associations of worse fatigue were examined. On multiple linear regression analyses, the model predicted significant association of worse fatigue scores with low education status (p = 0.03), number of deformed joints (p = 0.01), not achieving minimal disease activity state (p < 0.001), higher CCI scores, and worse health assessment questionnaire score (p < 0.001).

CONCLUSIONS

BRAF-NRS is a reliable, reproducible, and valid instrument for measuring fatigue in PsA. Fatigue in PsA is associated with low education status and overall more severe disease.Key Points• Fatigue is increasingly recognized as an important measure to examine among patients with PsA, but the available valid fatigue scores in PsA are relatively long and time-consuming especially when other core domains also need to be measured• BRAF-NRS is a short, easily readable, only 3-item tool to measure fatigue, and this is the first study which has examined its performance among the patients with PsA. Our results show that it is a reliable, reproducible, and valid (construct validity) instrument for measuring fatigue in PsA• This study also clearly showed a significant positive relationship between fatigue and comorbidities, and it was also found that comorbidities play the largest role in the multivariate model.

Reliability and Validity of the Korean Language Version of the U.S. National Cancer Institute's Patient-Reported Outcomes Common Terminology Criteria for Adverse Events

CONTEXT

To improve precision and accuracy in the capture of symptomatic adverse events (AEs) by self-report, the U.S. National Cancer Institute has developed a library of 124 patient-reported outcome (PRO) items reflecting 78 symptomatic AEs drawn from the Common Terminology Criteria for Adverse Events (CTCAE). The PRO-CTCAE™ item library has been translated and linguistically validated in the Korean language.

OBJECTIVES

The aim of this study was to examine the psychometric properties of PRO-CTCAE-Korean.

METHODS

PRO-CTCAE-Korean and the European Organization for Research and Treatment of Cancer Core Quality of Life Questionnaire Core 30 (QLQ-C30) were administered to 1358 Korean-speaking individuals receiving treatment for cancer at two medical centers in Korea (mean age 55.1 years; SD ±11.9; 60% females; and 61% high school education or less). A subset of 82 study participants completed the same two measures on a second occasion approximately three days later.

RESULTS

Correlations between PRO-CTCAE-Korean and conceptually relevant QLQ-C30 items were all greater than r = 0.30 except for headache severity. Most PRO-CTCAE-Korean items correlated at least moderately with QLQ-C30 summary scores. Monotonically decreasing total QLQ-C30 scores were observed across worsening levels of symptom frequency, severity, and interference (all P < 0.01), indicating that PRO-CTCAE-Korean response choices are well comprehended, and that PRO-CTCAE-Korean discriminates respondents with different levels of symptom burden. PRO-CTCAE-Korean also demonstrated generally acceptable to good reliability (88% of items intraclass correlation coefficient >0.50).

CONCLUSION

PRO-CTCAE-Korean is a reliable and valid instrument to capture symptomatic AEs by self-report in patients on cancer clinical trials.

Understanding the Fertility Desires and Intentions among HIV-Positive Men Living in Ontario: Survey Instrument Development and Validation

Men with HIV have highlighted the importance of understanding their fertility desires. However, most research has focused on women. We aimed (1) to develop a survey instrument to assess fertility desires and intentions among HIV-positive men and (2) to assess its face, content, and construct validity, as well as test–retest reliability and internal consistency. Principal component analysis was used for construct validity analysis in a sample of 60 men with HIV. The test–retest reliability and internal consistency were assessed using Spearman correlation and Cronbach α, respectively. The initial and the final version of the questionnaire consisted of 10 domains and 14 constructs. We found a one-component model for the 3 constructs analyzed and Cronbach α values were ≥.70. Test–retest statistic was stable with Spearman correlation >0.70. In conclusion, a reliable and valid questionnaire was developed for determining the fertility desires and intentions of men with HIV.

Development and Initial Psychometric Properties of Two Itch-Related Measures: Scratch Intensity and Impact, Sleep-Related Itch and Scratch

Self-report measures are needed to better understand the relationships among sleep, itching, scratching, and chronic itch conditions and their associations with disease severity, QOL, health, and functioning. Two scales related to sleep and/or scratch were recently developed and assessed in 137 patients with chronic itch and atopic dermatitis or psoriasis. The Scratch Intensity and Impact Scale consisted of two factors (scratching intensity and impact of scratching on QOL) that accounted for 64.59% of the variance with a total of 13 items, overall Cronbach's α of 0.93, and test-retest reliability of 0.66. The Sleep-Related Itch and Scratch Scale consisted of one factor that accounted for 63.01% of the variance with a total of 16 items, overall Cronbach's α of 0.98, and test-retest reliability of 0.66. Both measures demonstrated significant correlations with each other as well as other itch-related measures and nonsignificant correlations with scales hypothesized to be unrelated. The final measures demonstrated adequate preliminary psychometric characteristics. It is hoped that these scales will be used for future research and clinical purposes to help fill recognized gaps in understanding about sleep, itch, scratching, atopic dermatitis, and psoriasis.

Is my patient ready for a safe transfer to a lower-intensity care setting? Nursing complexity as an independent predictor of adverse events risk after ICU discharge

BACKGROUND

Patients discharged from intensive care units (ICUs) are at risk for adverse events (AEs). Establishing safe discharge criteria is challenging. No available criteria consider nursing complexity among risk factors.

OBJECTIVES

To investigate whether nursing complexity upon ICU discharge is an independent predictor for AEs.

METHODS

Prospective observational study. The Patient Acuity and Complexity Score (PACS) was developed to measure nursing complexity. Its predictive power for AEs was tested using multivariate regression analysis.

RESULTS

The final regression model showed a very-good discrimination power (AUC 0.881; p<0.001) for identifying patients who experienced AEs. Age, ICU admission reason, PACS, cough strength, PaCO₂, serum creatinine and sodium, and transfer to Internal Medicine showed to be predictive of AEs. Exceeding the identified PACS threshold increased by 3.3 times the AEs risk.

CONCLUSIONS

The level of nursing complexity independently predicts AEs risk and should be considered in establishing patient's eligibility for a safe ICU discharge.

Observational Gait Assessment Scales in Patients with Walking Disorders: Systematic Review

Objective

To compile and analyze the characteristics and methodological quality of observational gait assessment scales validated to date.

Methods

PubMed, Scopus, the Cochrane Library, Physiotherapy Evidence Database, Web of Science, Cumulative Index to Nursing and Allied Health Literature, Dialnet, Spanish Medical Index, and Nursing, Physiotherapy, and Podiatry databases were searched up to August 2019. The main inclusion criteria were validated tools based on a conceptual framework developed to evaluate gait, validation design studies of observational scales in their entirety, and articles written in English or Spanish. Evaluators extracted descriptive information of the scales and the metric properties of the studies, which were further analyzed with Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) and COnsensus-based Standards for the selection of health Measurement Instruments (COSMIN checklist).

Results

Eighteen articles based on 14 scales were included. The populations were neurological patients (72.22%), musculoskeletal disorders (11.11%), and other areas such as vestibular disorders (11.11%). The most addressed items were orthopedic aids (64.29%); phases of the gait cycle and kinematics of the leg and trunk (57.14% each one); and spatial and temporal parameters (50%). All studies analyzed criterion validity, and five included content or structural validity (27.78%). Fifteen articles considered reliability (83.33%). Regarding the seven-item scale QUADAS-2, five studies obtained six results on “low” risk of bias or “low” concerns regarding applicability. Nine articles obtained at least a “fair” result on COSMIN checklist.

Conclusions

A necessary compilation of the observational gait assessment scales validated to date was conducted. Besides, their characteristics and methodological quality were analyzed. Most scales were applied in neurological signs. The most approached topics were orthopedic aids, phases of the gait cycle, and kinematics of the leg and trunk. The scale that demonstrated a higher methodological quality was Visual Gait Assessment Scale, followed by CHAGS, Salford Gait Tool, and Edinburgh Visual Gait Score.

Psychometric Properties of the Glaucoma Treatment Compliance Assessment Tool (GTCAT) in a Brazilian Population

PURPOSE

To determine the psychometric properties of a reduced version of the Glaucoma Treatment Compliance Assessment Tool (GTCAT), which was translated to Brazilian Portuguese.

PATIENTS AND METHODS

We translated into Portuguese the GTCAT using validated techniques and administered it to a group of glaucoma patients using at least 1 ocular hypotensive eye drop medication. We used principal components analysis to determine construct validity, Rasch analysis, and Cronbach α for internal consistency reliability, frequency analysis for floor and ceiling effects, and Spearman ρ for test-retest reliability.

RESULTS

We included 76 glaucoma patients. Principal component analysis loaded 14 questions into 7 components that were consistent with the Health Belief Model. The components were related to "barriers due to lack of drops," "self-efficacy," "experience of negative effects of the disease," "well-being," "general glaucoma knowledge," "glaucoma symptoms," and "cues-to-action." No statements had floor or ceiling effects, and all statements had acceptable test-retest reliability. Components had internal consistency Cronbach α reliability between 0.125 and 0.794. and average Spearman ρ reliability was 0.73, ranging from 0.44 to 1.00. According to Rasch analysis, the mean (±SD) of the person measures was 0.24±0.15 logits, person separation index was 0.58, and person reliability 0.25.

CONCLUSIONS

The Portuguese-translated version of the GTCAT showed acceptable psychometric properties. With further refinement, clinicians and researchers could use it to better investigate glaucoma adherence issues in the Brazilian population.

Translation, validation and cross-cultural adaptation of the Nepali version of WOMAC® LK 3.1

BACKGROUND

Among many self-reported outcome measures, Western Ontario and McMaster Universities Arthritis Index (WOMAC) is one of the commonly used indexes for osteoarthritis patients. Despite high prevalence of musculoskeletal problems in Nepal, there is no validated tool available in the local language. Thus, this study aimed to translate the English WOMAC® into Nepali, and validate it for use in the future.

METHODS

Guidelines by Beaton et al were followed for translation and cross-cultural adaptation of the original WOMAC into Nepali language. Diagnosed cases of knee osteoarthritis (OA) attending the rheumatology outpatient department of National Center for Rheumatic Diseases, Kathmandu were enrolled in the study. These patients were interviewed with the Nepali version. Internal consistency was measured by Cronbach's alpha and test-retest reliability by intra-class correlation coefficient (ICC). Correlation between domains of WOMAC was tested with visual analog scale (VAS) and numerical rating scale (NRS) of pain and stiffness.

RESULTS

The test-retest reliability was good with ICC of >0.75 for all domains and items. Internal consistency was acceptable with Cronbach's alpha scores of 0.852, 0.704 and 0.955 for pain, stiffness and physical function domains, respectively. Strong correlation was observed between WOMAC stiffness domain and VAS for stiffness and NRS for stiffness with rho (ρ) values of 0.750 and 0.759, respectively. Moderate correlation was seen between WOMAC pain and VAS for pain and NRS for pain with ρ of 0.658 and 0.584, respectively.

CONCLUSIONS

Nepali WOMAC is a reliable and valid instrument for assessment of disease severity and its impact in Nepali-speaking patients with OA.

Symptom screening in paediatrics tool for screening multiple symptoms in Brazilian patients with cancer: a cross-sectional validation study

OBJECTIVE

The objective of this study was to translate, culturally adapt and validate the Symptom Screening in Paediatrics Tool (SSPedi) into the Brazilian Portuguese language to be used by paediatric oncology patients in Brazil.

DESIGN

A descriptive, cross-sectional study that follows an established methodology for translation and cultural adaptation, developed in two phases: phase I, linguistic translation and cultural adaptation of the SSPedi scale and phase II, psychometric properties evaluation.

SETTING

Children's Hospital for Cancer Treatment in Latin America.

PARTICIPANTS

Paediatric patients between 7 and 18 years of age and proxies of patients between 2 and 6 years of age, diagnosed with cancer and undergoing chemotherapy treatment. Patients and proxies with significant neuropsychiatric disorders and/or visual impairment that prevented the ability to read were excluded.

PRIMARY OUTCOME MEASURES

Construct validation of SSPedi using convergent validity and contrasted groups. Reliability was evaluated using Cronbach's alpha test and assessing the retest using the intraclass correlation coefficient (ICC).

RESULTS

The psychometric properties of the symptom screening tool were evaluated using 157 participants, of which 116 were patients and 41 were proxies. Convergent validity and hypothesised correlations (Spearman's r>0.4) were confirmed for both self- and proxy-reported versions of the assessment tool. No significant differences found between the two contrasting groups. Assessment of SSPedi resulted in an internal consistency of reliability of α=0.77 (95% CI 0.70 to 0.82) for the self and α=0.81 (95% CI 0.71 to 0.88) for the proxy and overall reproducibility ICC values of (95% CI), 0.54 (0.15 to 0.77) and 0.77 (0.64 to 0.86).

CONCLUSION

SSPedi was found to be culturally and linguistically adaptable and considered valid and reliable for use by paediatric oncology patients in Brazil. The new translated and adapted version was named SSPedi-BR.

Validity and internal consistency of EQ-5D-3L quality of life tool among pre-dialysis patients with chronic kidney disease in Sri Lanka, a lower middle-income country

Objective

EQ-5D-3L is a generic QOL tool used mainly in economic evaluations. Burden of Chronic Kidney Disease (CKD) is rising in Sri Lanka. Assessing the validity of generic QOL tools creates new opportunities of their utilization among patients with CKD.

Methods

A cross-sectional study was conducted among 1036 CKD patients, selected using the simple random sampling technique. The validity was tested with six a-priori hypotheses. These included construct validity assessments, evaluating convergent validity and performing known group comparisons. EQ-5D-3L, Short Form-36 (SF-36) were used to assess QOL. Center for Epidemiological Studies Depression Scale (CES-D-20) and General Health Questionnaire-12 (GHQ-12) were used to assess the presence of depression and psychological distress respectively. Internal consistency of the whole tool and when each item is removed was assessed by Cronbach alpha.

Results

The response rate was 99.2%. Majority of participants were males (n = 646,62.4%) in the age category of 41–60 (n = 530; 51.2%). Most were in either stage 4 or 5 of CKD (n = 646,75.1%). The summary measures of SF-36, positively and significantly correlated with the EQ-5D-3L index and VAS scores (p<0.001). EQ-5D-3L QOL scores were significantly different between the group with depression and without as measured by CES-D-20 (p<0.001). Assessed using GHQ-12, similar significance was detected between the group with psychological distress and without (p<0.001). The Cronbach alpha was 0.834 and when each item was removed, ranged from 0.782 to 0.832.

Conclusion

EQ-5D-3L is a valid generic QOL tool with satisfactory internal consistency to be used among CKD patients in the pre-dialysis stage.

The Relationship Between Patients' Safety-Related Behaviors and Safety Events and Patient Experiences in Korean Hospitals

BACKGROUND AND PURPOSE

There is increasing interest in patient participation in healthcare aimed at enhancing safety. The purposes of this study were to examine levels of patients' safety-related behaviors and to determine their relationships with safety events and care experiences.

METHODS

A cross-sectional survey was conducted with 360 patients in two tertiary general hospitals in Korea. The final dataset consisted of 341 patient responses. A questionnaire was developed to collect information concerning patients' safety-related behaviors, care experiences, and the occurrence of safety events. Univariate and multivariate analyses were performed.

RESULTS

On average, patients participated in 10.0 (SD 5.4) safety-related behaviors. Sixty-five (19.1%) patients reported experiencing at least one adverse event related to medical management. The mean problem score for care experience was 39.4 (SD 27.8) out of 100.0. Multiple logistic analysis showed that patients with adequate health literacy and higher activation levels were more likely to have high safety-related behavior participation levels. High participation in safety-related behaviors was negatively associated with the occurrence of safety events. However, no significant relationship was found between patient participation and the total problem scores for care experience.

IMPLICATIONS FOR PRACTICE

These findings suggest that efforts to support patients with limited health literacy and low activation levels are needed to facilitate patient participation. Active participation in safety-related behaviors can contribute to improving patient safety.

Thai adaptation and reliability of three versions of the Barratt Impulsiveness Scale (BIS 11, BIS-15, and BIS-Brief)

Long, short, and brief versions of the Barratt Impulsiveness scale (BIS-11, BIS-15, and BIS-Brief) were tested in an adult Thai population. The BIS-11T and BIS-15T were translated, back-translated, and administered to a non-clinical population (n = 305) of native Thai speakers who returned 2 weeks later for re-test. BIS-Brief-T psychometrics were calculated post-hoc. Impulsivity scores were normally distributed for the BIS-11T and BIS-15T, but not BIS-Brief-T. Excellent internal consistency was observed, with Cronbach's alpha coefficients above 0.80 for all translated instruments: BIS-11T (α = 0.86), BIS-15T (α = 0.81), BIS-Brief-T (α = 0.81). A total of 260 participants completed both instruments (85%), with test-retest reliability exceeding r = 0.81. All three instruments were highly correlated (r = 0.83-0.89). Confirmatory factor analysis supports a three factor structure (attention, motor, non-planning) for BIS-15T and two factor structure for BIS-11T. BIS scales can support measurement of a range of impulsivity scores in an adult Thai population, though predictive validity of these scales remains unexplored.

The Reliability and Relevance of a Quality of Decision Making Instrument, Quality of Decision-Making Orientation Scheme (QoDoS), for Use During the Lifecycle of Medicines

Introduction: The Quality of Decision-Making Orientation Scheme (QoDoS) was developed to provide organisations involved in submission, approval and reimbursement of new medicines with a tool to improve the quality of their decision-making processes and is considered the most promising tool for such purpose. This study aimed to further establish the measurement properties of the QoDoS by evaluating its reliability (internal consistency and test-retest reliability) and relevance in the target population.

Methods: The study participants consisted of 55 individuals recruited from pharmaceutical companies, regulatory and HTA agencies. It was designed as a longitudinal study with participants assessed on two different occasions, at baseline (test 1) and then 7 days later (test 2). Internal consistency reliability was assessed with Cronbach’s alpha and the test-retest reliability was evaluated using the intraclass correlation coefficients (ICC) based on absolute agreement, 2 way mixed-effects model for the four QoDoS domains. The relevance of the QoDoS was evaluated by applying cognitive debriefing using five short feedback questions following test 1.

Results: Test 1 was completed by 44 study participants (80% response rate) and test 2 was completed by 32 of the 44 individuals, resulting in a 73% response rate. Cronbach’s alpha coefficient was greater than 0.7 across all the domains for test 1 and test 2, ranging from 0.71 to 0.79, indicating good consistency of responses. For the overall score across all 47 items, the Cronbach’s alpha coefficient was 0.81 for test 1 and 0.86 for test 2, which is rated as very good. The four QoDoS domains showed moderate to strong reproducibility (ICC range: 0.63–0.86). The outcome of the cognitive debriefing from the 43 respondents (98% response rate) confirmed the relevance (95% agreement), language clarity (95%) and completeness of items (86%); the clarity of the scaling (91%) as well as spontaneity of responses (95%).

Conclusion: These results provide strong support for the relevance and reliability of the QoDoS, which are key properties for future longitudinal and cross-sectional applications of the instrument when evaluating quality of decision making by those involved in the lifecycle of medicines.