Can rehabilitation adherence among stroke patients be measured using a single item?

AIMS AND OBJECTIVES

To assess the concurrent validity between logbooks and a single-item rehabilitation adherence measurement for patients with stroke. Agreement between caregivers and patients and between caregivers and physical therapists regarding a single-item measurement was investigated, and its predictive validity was explored.

BACKGROUND

Adherence to therapy is a primary determinant of treatment success. There are no standard instruments for measuring rehabilitation adherence available for stroke patients.

DESIGN

Prospective longitudinal study.

METHODS

Seventy-five patients with stroke were recruited, measured four times and followed for 6 months. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) checklist was used to ensure comprehensive reporting. Adherence was documented in logbooks, and single-item measurements were compared. Predictive validity was explored by assessing associations between adherence levels, self-care ability and health-related quality of life. The Spearman's correlation coefficients, weighted kappa, and generalised estimating equations statistics were used to explore the concurrent validity, measurement agreement, and predictive validity, respectively.

RESULTS

Logbook records had a fair correlation (r_s  = .23, p = .04) with the single-item rehabilitation adherence measurements. There was moderate agreement (kappa = 0.42, p < .001) between caregiver and patient assessments and fair agreement (kappa = 0.29, p = .017) between caregiver and physical therapist assessments of patients' rehabilitation adherence levels. Perfect rehabilitation adherence, based on the logbook and single-item measurements, predicted better scores for self-care ability and quality of life than imperfect rehabilitation adherence during 6 months after inclusion.

CONCLUSIONS

There was fair concurrent validity between logbooks and single-item rehabilitation adherence measurements and moderate and fair adherence measure agreement between caregivers and patients and caregivers and physical therapists, respectively. Logbooks and single-item rehabilitation adherence measurements had adequate predictive validity.

RELEVANCE TO CLINICAL PRACTICE

Single-item rehabilitation adherence measurement is a workable and straightforward method to assess stroke patients' rehabilitation adherence in busy clinical care settings. Caregivers can represent stroke patients regarding their reported rehabilitation adherence.

PATIENT OR PUBLIC CONTRIBUTION

Patients were diagnosed with stroke in the study hospital. Rehabilitation physicians transferred patients to a research nurse who then screened them for the inclusion criteria and invited them and their family caregivers to participate in this study if they met the requirements. We also recruited seven physical therapists responsible for the physical therapy of the study participants. After participants signed informed consent, the research nurse encouraged participants to respond to research questions face to face, including rehabilitation adherence data, daily physical function, and quality of life. Each participant was measured four times at baseline and at 1, 3, and 6 months after inclusion in this study. Physical therapists had to score their patients' rehabilitation adherence levels before discharge.

TRIAL REGISTRATION DETAILS

Not applicable.

Predictors of diagnostically defined insomnia in child and adolescent community samples: a literature review

Diagnostically defined insomnia is prevalent, persistent, and associated with a range of negative outcomes in childhood and adolescence. To inform prevention and treatment, we need to identify relevant predictors that can be addressed in such intervention efforts. Therefore, a systematic search for longitudinal studies involving child and adolescent samples (ages 4 to 19) examining predictors of diagnostically defined insomnia adjusted for previous insomnia was conducted. The search identified 6419 studies, resulting in six included papers involving five samples (n = 9949) conducted in five different countries (the US, New Zealand, Norway, China, and Japan). Few longitudinal studies investigated the predictors of diagnostically defined insomnia in children and adolescents, and insomnia is rarely defined according to diagnostic manuals. The results suggested that poor mental health (most notably depression) and female sex may be involved in the etiology of diagnostically defined insomnia. Stress might be the most modifiable factor identified. However, the diversity of the predictors studied in previous reports combined with the lack of replication prevent any firm conclusions from being drawn. This review serves as a summary of the best available evidence.

Functional Correlations in the Pursuit of Performance Assessment of Classifiers

In statistical classification and machine learning, as well as in social and other sciences, a number of measures of association have been proposed for assessing and comparing individual classifiers, raters, as well as their groups. In this paper, we introduce, justify, and explore several new measures of association, which we call CO-, ANTI-, and COANTI-correlation coefficients, that we demonstrate to be powerful tools for classifying confusion matrices. We illustrate the performance of these new coefficients using a number of examples, from which we also conclude that the coefficients are new objects in the sense that they differ from those already in the literature.

Self-assessed puberty is reliable in a low-income setting in rural Pakistan

Objectives Staging sexual maturation is an integral component of adolescent research. The Pubertal Development Scale (PDS) is commonly used as a puberty self-assessment tool because it avoids the use of images. Among the youth living in rural Pakistan, we determined the accuracy of self-reported pubertal assessments using a modified PDS compared to the 'gold standard' of physically assessed Tanner stages by a physician. Methods The strength of agreement between self-assessed puberty using a modified PDS and the 'gold' standard of physician-assessed Tanner stages was reported using weighted kappa (κ w) for girls (n = 723) of 9.0-14.9 years of age or boys (n = 662) of 10.0-15.9 years of age living in the rural District of Matiari. Results Agreement between the gold standard and self-assessment for puberty was substantial, with a κ w of 0.73 (95% confidence interval [CI]: 0.67; 0.79) for girls and a κ w of 0.61 (95% CI: 0.55; 0.66) for boys. Substantial agreement was observed for both boys and girls classified as thinness but only for girls with a normal body mass index. Those who were classified as severely thin had moderate agreement. The prevalence of overestimation was 18.5% (95% CI: 15.9-21.5) for girls and 2.7% (95% CI: 1.7-4.3) for boys, while the prevalence of underestimation estimation was 8.0% (95% CI: 6.2-10.2) for girls and 29.0% (95% CI: 25.8-32.6) for boys. Conclusions Most girls and boys assessed their pubertal development with substantial agreement with physician assessment. Girls were better able to assess their puberty, but they were more likely to overestimate. Agreement for boys was also substantial, but they were more likely to underestimate their pubertal development. In this rural Pakistan population, the PDS seems to be a promising tool for self-assessed puberty.

Test-Retest Reliability, Agreement and Responsiveness of Productivity Loss (iPCQ-VR) and Healthcare Utilization (TiCP-VR) Questionnaires for Sick Workers with Chronic Musculoskeletal Pain

The purpose of this study was to assess test-retest reliability, agreement, and responsiveness of questionnaires on productivity loss (iPCQ-VR) and healthcare utilization (TiCP-VR) for sick-listed workers with chronic musculoskeletal pain who were referred to vocational rehabilitation. Methods Test-retest reliability and agreement was assessed with a 2-week interval. Responsiveness was assessed at discharge after a 15-week vocational rehabilitation (VR) program. Data was obtained from six Dutch VR centers. Test-retest reliability was determined with intraclass correlation coefficient (ICC) and Cohen's kappa. Agreement was determined by Standard Error of Measurement (SEM), smallest detectable changes (on group and individual level), and percentage observed, positive and negative agreement. Responsiveness was determined with area under the curve (AUC) obtained from receiver operation characteristic (ROC). Results A sample of 52 participants on test-retest reliability and agreement, and a sample of 223 on responsiveness were included in the analysis. Productivity loss (iPCQ-VR): ICCs ranged from 0.52 to 0.90, kappa ranged from 0.42 to 0.96, and AUC ranged from 0.55 to 0.86. Healthcare utilization (TiCP-VR): ICC was 0.81, and kappa values of the single healthcare utilization items ranged from 0.11 to 1.00. Conclusions The iPCQ-VR showed good measurement properties on working status, number of hours working per week and long-term sick leave, and low measurement properties on short-term sick leave and presenteeism. The TiCP-VR showed adequate reliability on all healthcare utilization items together and medication use, but showed low measurement properties on the single healthcare utilization items.

A New Interpretation of the Weighted Kappa Coefficients

Reliability and agreement studies are of paramount importance. They do contribute to the quality of studies by providing information about the amount of error inherent to any diagnosis, score or measurement. Guidelines for reporting reliability and agreement studies were recently provided. While the use of the kappa-like family is advised for categorical and ordinal scales, no further guideline in the choice of a weighting scheme is given. In the present paper, a new simple and practical interpretation of the linear- and quadratic-weighted kappa coefficients is given. This will help researchers in motivating their choice of a weighting scheme.