Interobserver Reliability and Sensitivity to Change of a Composite Ocular Inflammatory Activity Index: UVEDAI©

INTRODUCTION

This was a multicenter, prospective, longitudinal, observational study involving eight Spanish tertiary hospitals to determine the interobserver reliability of an uveitis disease activity index, (UVEDAI) and assess its sensitivity to change in patients with receiving pharmacologic treatment.

METHODS

Patients aged ≥ 18 years diagnosed with active noninfectious uveitis were included. A complete baseline assessment was performed by two ophthalmologists who determined ocular inflammatory activity using the UVEDAI index independently of each other. The principal ophthalmologist made a new visit at 4 weeks to determine the change in inflammatory activity. The interobserver reliability analysis was performed by calculating the intraclass correlation coefficient (ICC), with the values of the variables and the UVEDAI obtained by both ophthalmologists in the more active eye at the baseline visit. Sensitivity to change in the UVEDAI index was assessed at 4 weeks from the start of pharmacologic treatment by determining the clinically relevant change, defined as a change in UVEDAI of ≥ 0.8 points over baseline. The mean change between both measures was compared using the repeated-measures t-test.

RESULTS

A total of 111 patients were included. In the interobserver reliability analysis, the ICC for the UVEDAI value was 0.9, and, when compared with the mean UVEDAI values obtained by the ophthalmologists, no statistically significant differences were found (p value > 0.05). As for the sensitivity to change in UVEDAI, statistically significant differences (p value = 0.00) were found for the mean values of the index compared with baseline. In all cases, the index value decreased by > 1 point at the 4-week visit.

CONCLUSIONS

The interobserver reliability of the UVEDAI was high in the total sample. Furthermore, the index was sensitive in determining the change in inflammatory activity after treatment. We believe that UVEDAI is a disease activity index that enables objective comparison of results in clinical practice and trials.

Responsiveness and minimal important change of the Family Reported Outcome Measure (FROM-16)

BACKGROUND

The FROM-16 is a generic family quality of life (QoL) instrument that measures the QoL impact of patients' disease on their family members/partners. The study aimed to assess the responsiveness of FROM-16 to change and determine Minimal Important Change (MIC).

METHODS

Responsiveness and MIC for FROM-16 were assessed prospectively with patients and their family members recruited from outpatient departments of the University Hospital Wales and University Hospital Llandough, Cardiff, United Kingdom. Patients completed the EQ-5D-3L and a global severity question (GSQ) online at baseline and at 3-month follow-up. Family members completed FROM-16 at baseline and a Global Rating of Change (GRC) in addition to FROM-16 at follow-up. Responsiveness was assessed using the distribution-based (effect size-ES, standardized response mean -SRM) and anchor-based (area under the receiver operating characteristics curve ROC-AUC) approaches and by testing hypotheses on expected correlation strength between FROM-16 change score and patient assessment tools (GSQ and EQ-5D). Cohen's criteria were used for assessing ES. The AUC ≥ 0.7 was considered a good measure of responsiveness. MIC was calculated using anchor-based (ROC analysis and adjusted predictive modelling) and distribution methods based on standard deviation (SD) and standard error of the measurement (SEM).

RESULTS

Eighty-three patients with 15 different health conditions and their relatives completed baseline and follow-up questionnaires and were included in the responsiveness analysis. The mean FROM-16 change over 3 months = 1.43 (SD = 4.98). The mean patient EQ-5D change over 3 months = -0.059 (SD = 0.14). The responsiveness analysis showed that the FROM-16 was responsive to change (ES = 0.2, SRM = 0.3; p < 0.01). The ES and SRM of FROM-16 change score ranged from small (ES = 0.2; SRM = 0.3) for the distribution-based method to large (ES = 0.8, SRM = 0.85) for anchor-based methods. The AUC value was above 0.7, indicating good responsiveness. There was a significant positive correlation between the FROM-16 change scores and the patient's disease severity change scores (p < 0.001). The MIC analysis was based on data from 100 family members of 100 patients. The MIC value of 4 was suggested for FROM-16.

CONCLUSIONS

The results of this study confirm the longitudinal validity of FROM-16 which refers to the degree to which an instrument is able to measure change in the construct to be measured. The results yield a MIC value of 4 for FROM-16. These psychometric attributes of the FROM-16 instrument are useful in both clinical research as well as clinical practice.

Are suicide risk scales sensitive to change? A scoping review

INTRODUCTION

Clinician- or self-administered scales are frequently used to assist in detecting risk of death by suicide and to determine the effectiveness of interventions. No recent review studies have examined whether these scales are sensitive to change. We conducted a scoping review to identify suicide risk scales that are sensitive to change.

MATERIAL AND METHODS

We searched Medline and Excerpta Medica Database from inception through March 17, 2022, to identify randomized trials, pooled analysis, quasi-experimental studies, and cohort studies reporting on sensitivity to change of suicide risk scales. We assessed sensitivity to change by examining internal and external responsiveness. Internal responsiveness evaluates whether the scale measures changes in suicide-related symptoms in response to an intervention while external responsiveness assess whether changes in the scale correspond to changes in risk of suicide. We summarized findings across studies using descriptive analysis.

RESULTS

Among 38 eligible scales, we identified 27 scales that included items that were modifiable to change. However, only 7 scales had been studied to determine their sensitivity to change based on internal or external responsiveness. While the results of studies suggested that 6 scales have internal responsiveness, none of the included studies confirmed that scales have external responsiveness.

DISCUSSION

A few suicide risk scales are internally responsive and may be useful in a clinical or research setting. It is unclear, however, whether changes in scores correspond to an actual change in suicide risk. Future research should confirm the external responsiveness of scales using robust metrics including suicide mortality.

Sensitivity to change of generic preference-based instruments (EQ-5D-3L, EQ-5D-5L, and HUI3) in the context of treatment for people with prescription-type opioid use disorder in Canada

PURPOSE

Using data from a randomized controlled trial for treatment of prescription-type opioid use disorder in Canada, this study examines sensitivity to change in three preference-based instruments [EQ-5D-3L, EQ-5D-5L, and the Health Utilities Index Mark 3 (HUI3)] and explores an oft-overlooked consideration when working with contemporaneous responses for similar questions-data quality.

METHODS

Analyses focused on the relative abilities of three instruments to capture change in health status. Distributional methods were used to categorize individuals as 'improved' or 'not improved' for eight anchors (seven clinical, one generic). Sensitivity to change was assessed using area under the ROC (receiver operating characteristics) curve (AUC) analysis and comparisons of mean change scores for three time periods. A 'strict' data quality criteria, defined a priori, was applied. Analyses were replicated using 'soft' and 'no' criteria.

RESULTS

Data from 160 individuals were used in the analysis; 30% had at least one data quality violation at baseline. Despite mean index scores being significantly lower for the HUI3 compared with EQ-5D instruments at each time point, the magnitudes of change scores were similar. No instrument demonstrated superior sensitivity to change. While six of the 10 highest AUC estimates were for the HUI3, 'moderate' classifications of discriminative ability were identified in 12 (of 22) analyses for each EQ-5D instrument, compared with eight for the HUI3.

CONCLUSION

Negligible differences were observed between the EQ-5D-3L, EQ-5D-5L, and HUI3 regarding the ability to measure change. The prevalence of data quality violations-which differed by ethnicity-requires further investigation.

Validation of the Radiographic Global Impression of Change (RGI-C) score to assess healing of rickets in pediatric X-linked hypophosphatemia (XLH)

BACKGROUND

Rickets is a primary manifestation of pediatric X-linked hypophosphatemia (XLH) - a rare progressive hereditary phosphate-wasting disease. Severity is quantified from radiographs using the Rickets Severity Scale (RSS). The Radiographic Global Impression of Change (RGI-C) is a complementary assessment in which a change score is assigned based on differences in the appearance of rickets on pairs of radiographs compared side by side.

OBJECTIVE

The current study evaluated the reliability, validity, and sensitivity to change of the RGI-C specifically in pediatric XLH.

METHODS

The reliability, validity, and sensitivity to change of the RGI-C were evaluated using data from two studies in pediatric XLH (113 children aged 1-12 years) in which burosumab treatment significantly improved rickets severity. Intra-rater and inter-rater reliability were assessed by three pediatric radiologists.

RESULTS

Intra-rater reliability for RGI-C global score was >90% for agreement within 1 point, with weighted kappa values >0.5, indicating moderate to almost perfect agreement. Inter-rater reliability was also >90% (0.47-0.52 for all reader pairs; moderate agreement). The RGI-C global score showed significant relationships with changes from baseline to week 64 in serum phosphorus (r = -0.397), alkaline phosphatase (-0.611), total RSS (-0.672), standing height (0.268), and patient-reported global functioning (0.306) and comfort/pain functioning (0.409). Based on standardized response means, RGI-C global scores were sensitive to change in RSS, differentiating between those considered improved and greatly improved. Results for validity and sensitivity to change were similar for the RGI-C wrist, knee, and standing long leg scores.

CONCLUSION

The RGI-C is a reliable, valid, and sensitive measure in pediatric XLH, and complementary to the RSS.

Responsiveness of PROMIS and Patient Health Questionnaire (PHQ) Depression Scales in three clinical trials

BACKGROUND

The PROMIS depression scales are reliable and valid measures that have extensive normative data in general population samples. However, less is known about how responsive they are to detect change in clinical settings and how their responsiveness compares to legacy measures. The purpose of this study was to assess and compare the responsiveness of the PROMIS and Patient Health Questionnaire (PHQ) depression scales in three separate samples.

METHODS

We used data from three clinical trials (two in patients with chronic pain and one in stroke survivors) totaling 651 participants. At both baseline and follow-up, participants completed four PROMIS depression fixed-length scales as well as legacy measures: Patient Health Questionnaire 9-item and 2-item scales (PHQ-9 and PHQ-2) and the SF-36 Mental Health scale. We measured global ratings of depression change, both prospectively and retrospectively, as anchors to classify patients as improved, unchanged, or worsened. Responsiveness was assessed with standardized response means, statistical tests comparing change groups, and area-under-curve analysis.

RESULTS

The PROMIS depression and legacy scales had generally comparable responsiveness. Moreover, the four PROMIS depression scales of varying lengths were similarly responsive. In general, measures performed better in detecting depression improvement than depression worsening. For all measures, responsiveness varied based on the study sample and on whether depression improved or worsened.

CONCLUSIONS

Both PROMIS and PHQ depression scales are brief public domain measures that are responsive (i.e., sensitive to change) and thus appropriate as outcome measures in research as well as for monitoring treatment in clinical practice. Trial registration ClinicalTrials.gov ID: NCT01236521, NCT01583985, NCT01507688.

Change in pain and its relation to change in activity in osteoarthritis

Objective

Trials testing promising interventions in knee osteoarthritis (OA) often fail to show pain reductions. This may be due to change in activity whereby a person's pain decreases, leading them to increase their activity levels, in turn increasing pain back to baseline levels. Using data from a trial of a beneficial treatment for knee pain, we explored whether activity changes might mask a treatment's effect on pain, by looking at whether activity levels increased with effective treatment and whether change in activity level related to change in pain.

Design

During the InRespond trial (ISRCTN55059760) participants wore an accelerometer for 7 days before and during treatments. We assessed change in pain on treatment using scores for overall knee pain and pain in a nominated pain-aggravating activity both in the last week and evaluated change in different types of activity using accelerometer data. Principal components analysis tested whether change in activity and pain outcomes were correlated and created composites combining them. We then tested whether activity, pain or the composites showed a treatment effect, and examined their responsiveness.

Results

In the 61 participants (mean age 64.5 years, 38% women, mean overall knee pain score 5.08 (0-10)), activity levels mostly decreased during the trial. Principal component analyses suggested that pain and activity did not correlate highly, loading on different components. Treatment that showed significant effects on pain did not show similar effects on either activity (e.g. the active treatment had a slightly greater reduction in total steps taken than the control treatment (difference 1942.6 steps/week, p = 0.42) nor on composites combining activity and pain. Pain outcomes were the most responsive; static loading (standing) outcomes were the most responsive activity outcome.

Conclusion

We found no evidence to support the hypothesis that activity levels increase during effective OA treatment and might account for the negligible pain effects of OA treatments.

Five direct behavior rating multi-item scales: Sensitivity to the effects of classroom interventions

Direct Behavior Rating (DBR) is a tool designed for the assessment of behavioral changes over time. Unlike methods for summative evaluations, the development of progress monitoring tools requires evaluation of sensitivity to change. The present study aimed to evaluate this psychometric feature of five newly developed DBR Multi-Item Scales (DBR-MIS). Teachers identified students with behaviors interfering with their learning or the learning of others and implemented a Daily Report Card (DRC) intervention in the classroom settings for two months. The analyses were performed on 31 AB single case studies. Change metrics were calculated at an individual level by using Tau-U_{A vs. B + trend B} and Hedges' g and at a scale-level by using Mixed Effect Meta-Analysis, Hierarchical Linear Models (HLMs), and Between-Case Standardized Mean Difference (BC-SMD). HLMs were estimated considering both fixed and random effects of intervention and linear trend within the intervention phase. The results supported sensitivity to change for three DBR-MIS (i.e., Academic Engagement, Organizational Skills, and Disruptive Behavior), and the relative magnitudes were consistent across the metrics. Sensitivity to change of DBR-MIS Interpersonal Skills received moderate support. Conversely, empirical evidence was not provided for sensitivity to change of DBR-MIS Oppositional Behavior. Particular emphasis was placed on the intervention trend in that responses to behavioral interventions might occur gradually or require consistency over time in order to be observed by raters. Implications for the use of the new DBR-MIS in the context of progress monitoring of social-emotional behaviors are discussed.

Sensitivity to change and minimal clinically important difference of the 7-item Generalized Anxiety Disorder Questionnaire (GAD-7)

BACKGROUND

Effective treatment requires regular follow-up and monitoring of symptoms. We investigated sensitivity to change and minimal clinically important difference of the Generalized Anxiety Disorder Scale (GAD-7).

METHODS

This study included all participants from a multisite trial of chronic depression. Baseline and follow-up (12 and 48 weeks) data were used to assess treatment response. Effect sizes (ES) and standardized response means (SRM) of pre- and post-GAD-7 mean changes were calculated for subgroups of patients, who did or did not improve according to ratings in the Hamilton Rating Scale for Depression (HRSD-24).

RESULTS

N = 261 patients were included in the analyses. In the subgroup of patients who improved according to HRSD-24, GAD-7 scores were significantly lower after 12 weeks (t = -6.31, df = 120, p < .001; ES = -0.51, SRM = -0.57), and 48 weeks of treatment (t = -12.68, df = 141, p < .001; ES = -1.0, SRM = -1.7), when compared to admission. In the group who worsened, GAD-7 scores were significantly higher after 12 weeks (t = 2.96, df = 41, p = .005; ES = 0.30, SRM = 0.46), and increased after 48 weeks (t = 1.99, df = 21, p = .059; ES = 0.37, SRM = 0.43), when compared to baseline. The unchanged group showed no significant difference between baseline and follow-up. MCID was estimated 4 points on the GAD-7 total score.

LIMITATIONS

Confirmation of these findings and further investigation of the GAD-7 in populations and trials focusing on anxiety-specific treatment is highly recommended.

CONCLUSIONS

Results show that the GAD-7 is sensitive to detect change in psychopathology over the course of treatment.

Validation of a Motor-Cognitive Assessment for a Stepping Exergame in Older Adults: Use of Game-Specific, Internal Data Stream

Objective: To comprehensively validate an assessment tailored to an exergaming training program with motor-cognitive challenges. Materials and Methods: Fifty-eight cognitively intact, multimorbid, independently living older adults (mean age [standard deviation]: 78.3 [6.5] years) with moderate functional limitations participated in the study. For construct validity, Spearman's rank correlations (r_s) between game parameters and established cognitive (Number-Connection-Test [Zahlen-Verbindungs-Test], Simple Response Time Task, Simon Task) and motor (Short Physical Performance Battery, Physiomat^® balance tests) measures were calculated. Test-retest reliability was documented by intraclass correlation coefficients (ICCs), sensitivity to change by effect sizes using partial eta squared (η_p²), and feasibility by mean completion time and completion rates. Results: Good construct validity of the assessment was observed, with on average moderate-to-high correlations between game parameters and cognitive tests, measures of lower extremity function and dynamic balance (range of r_s including extreme outliers = 0.00-0.70, P < 0.001-0.998). Test-retest reliability was good, with ICCs mostly ranging from moderate to high (ICCs = 0.37-0.93, P < 0.001-0.130), and sensitivity to change was excellent (η_p² = 0.16-0.81, P < 0.001-0.044). Completion rates for the initial challenge levels were 100%, mean completion time 36.3 minutes, and no clinical events or safety problems were observed. Conclusion: Study results documented on average good validity, test-retest reliability and feasibility, with an extraordinary high responsiveness of the presented game-based assessment in older adults with moderate functional limitations. The innovative, data-based assessment validated in this study may serve as a blueprint for future, tailored assessments for exergaming.

The Italian version of the Quebec Back Pain Disability Scale: cross-cultural adaptation, reliability and validity in patients with chronic low back pain

BACKGROUND AND AIM

Growing attention is being given to physical functioning measures to assess interventions for low back pain (LBP). The Quebec Back Pain Disability Questionnaire (QBPDS) has never been validated in Italian patients, and the aim of the study was culturally adapting and validating the Italian version of the QBPDS (QBPDS-I), to allow its use with Italian-speaking patients with chronic LBP.

METHODS

The QBPDS-I was developed by means of forward-backward translation, a final review by an expert committee and a test of the prefinal version to evaluate its comprehensibility. The psychometric testing included structural validity by exploratory factor analysis (EFA), reliability by internal consistency (Cronbach's alpha) and test-retest reliability (intraclass correlation coefficient, ICC 2.1), measurement error by calculating the minimum detectable change (MDC), construct validity by assessing hypotheses of QBPDS correlations with the Roland Morris Disability Scale (RMDQ), the Oswestry Disability Questionnaire (ODI) and a pain numerical rating scale (NRS) (Spearman's correlations).

RESULTS

It took one month to develop a consensus-based version of the QBPDS-I. The questionnaire was administered to 201 subjects with chronic LBP and was well accepted. EFA suggested a one-factor 20-item solution (first factor variance explained = 54.7%). Internal consistency (α = 0.95) and test-retest reliability (ICC = 0.90) were excellent. The MDC was 12 scale points. Construct validity was good as all of the hypotheses were met; correlations: RMDQ (r = 0.40), ODI (r = 0.48) and NRS (r = 0.44).

CONCLUSIONS

The QBPDS-I is unidimensional, reliable and valid in patients with chronic LBP. Its use is recommended for clinical and research purposes.

Inter-rater reliability, sensitivity to change and responsiveness of the orthopaedic Wolf-Motor-Function-Test as functional capacity measure before and after rehabilitation in patients with proximal humeral fractures

BACKGROUND

The incidence of proximal humeral fractures (PHF) increased by more than 30% over the last decade, which is accompanied by an increased number of operations. However, the evidence on operative vs. non-operative treatment and post-operative treatments is limited and mostly based on expert opinion. It is mandatory to objectively assess functional capacity to compare different treatments. Clinical tools should be valid, reliable and sensitive to change assessing functional capacity after PHFs. This study aimed to analyse inter-rater reliability of the videotaped Wolf-Motor-Function-Test-Orthopaedic (WMFT-O) and the association between the clinical WMFT-O and the Disability of the Arm, Shoulder and Hand (DASH) and to determine the sensitivity to change of the WMFT-O and the DASH to measure functional capacity before and after rehabilitation in PHF patients.

METHODS

Fifty-six patients (61.7 ± 14.7 years) after surgical treatment of PHF were assessed using the WMFT-O at two different time points. To determine inter-rater reliability, the videotaped WMFT-O was evaluated through three blinded raters. Inter-rater agreement was determined by Fleiss' Kappa statistics. Pearson correlation coefficients were calculated to assess the association between the clinical WMFT-O and the video rating as well as the DASH. Sensitivity to change and responsiveness were analysed for the WMFT-O and the DASH in a subsample of forty patients (53.8 ± 1.4 years) who were assessed before and after a three week robotic-assisted training intervention.

RESULTS

Inter-rater agreement was indicated by Fleiss' Kappa values ranging from 0.33-0.66 for functional capacity and from 0.27-0.54 for quality of movement. The correlation between the clinical WMFT-O and the video rating was higher than 0.77. The correlation between the clinical WMFT-O and the DASH was weak. Sensitivity to change was high for the WMFT-O and the DASH and responsiveness was given. In comparison to the DASH, the sensitivity to change of the WMFT-O was higher.

CONCLUSION

The overall results indicate that the WMFT-O is a reliable, sensitive and responsive instrument to measure more objectively functional change over time in rehabilitation after PHF. Furthermore, it has been shown that video assessment is eligible for studies to ensure a full blinding of raters.

TRIAL REGISTRATION

Clinicaltrials.gov, NCT03100201 . Registered on 28 March 2017. The trial was retrospectively registered.

One-minute stair climbing, 50-foot walk, and timed up-and-go were responsive measures for patients with chronic low back pain undergoing lumbar fusion surgery

Background

Physical capacity tasks are useful tools to assess functioning in patients with low back pain (LBP), but evidence is scarce regarding the responsiveness (ability to detect change over time) and minimal important change (MIC). The aim was to investigate the responsiveness and MIC of 5-min walk, 1-min stair climbing, 50-ft walk, and timed up-and-go in patients with chronic LBP undergoing lumbar fusion surgery.

Methods

In this clinimetric study, 118 patients scheduled for lumbar fusion surgery for motion-elicited chronic LBP with degenerative changes were included. All patients performed the physical capacity tasks 5-min walk, 1-min stair climbing, 50-ft walk, and timed up-and-go 8–12 weeks before and six months after surgery. Responsiveness was evaluated by testing five a priori responsiveness hypotheses. The hypotheses concerned the area under the receiver operating characteristics (ROC) curve and correlations (Spearman’s rho) between the change scores of the physical capacity tasks, the Oswestry Disability Index 2.0 (ODI), and back pain intensity measured with visual analog scale (VAS). At least 80% of the hypotheses would have to be confirmed for adequate responsiveness. Absolute and relative MICs for improvement were determined by the optimal cut-off point of the ROC curve based on the classification of improved and unchanged patients according to construct-specific global perceived effect (GPE) scales.

Results

One-minute stair climbing, 50-ft walk and timed up-and-go displayed adequate responsiveness (≥ 80% of hypotheses confirmed), while 5-min walk did not (40% of hypotheses confirmed). The absolute MICs for improvement were 45.5 m for 5-min walk, 20.0 steps for 1-min stair climbing, − 0.6 s for 50-ft walk, and − 1.3 s for timed up-and-go.

Conclusions

The results of responsiveness for 1-min stair climbing, 50-ft walk, and timed up-and-go implies that these have the ability to detect changes in physical capacity over time in patients with chronic LBP who have undergone lumbar fusion surgery.

Validity and sensitivity to change of the Somatic Symptom Disorder-B Criteria Scale (SSD-12) in a clinical population

OBJECTIVE

The SSD-12 is a brief self-report questionnaire to measure the psychological criteria of DSM-5 Somatic Symptom Disorder. This study examines its psychometric properties in a German inpatient sample from a psychosomatic rehabilitation setting, and provides evidence to its sensitivity to change.

METHOD

Patients completed the SSD-12 and the Health49-subscale on somatoform complaints before and after receiving inpatient treatment. Therapists evaluated the psychological improvement of their patients at the end of treatment. Effect sizes (ES) and standardized response means (SRM) of pre- and post-SSD-12 mean changes were calculated for subgroups of patients who did or did not improve.

RESULTS

SSD-12 scores at discharge were significantly lower compared to scores at admission for subgroups of patients who improved according to clinicians (t=2976, df=103, p=.004), and for patients who improved according to self-report (t=5.059, df=159, p<.001). Effect sizes of change in SSD-12 scores in the improved subgroups were ES=-0.19 and ES=-0.30, and standardized response means were SRM=-0.29 and SRM=-0.40, respectively.

CONCLUSION

The SSD-12 shows sound psychometric properties and is useful and time-efficient for monitoring psychological burden associated with bothersome somatic symptoms. Its sensitivity to change over time could be documented.

Test-retest Reliability and Sensitivity to Change of a New Fall Risk Assessment System: A Pilot Study

Background

The new fall risk assessment (FRA) system is a composite and comprehensive assessment tool developed to predict the risk of falls. The aim of this pilot study was to examine the new FRA system’s test-retest reliability and sensitivity to change in community-dwelling older adults.

Methods

This was an observational study with a test-retest design and an 8-week fall prevention exercise program. A sample of 28 community-dwelling older adults with a mean age of 73.0 years (range, 65–80 years) participated in the study. The new FRA system was administered twice within a 7-day period for test-retest reliability expressed as intraclass correlation coefficient (ICC) and standard error of measurement (SEM) assessment. Eighteen subjects of them completed the 8-week fall prevention exercise intervention to evaluate the new FRA system’s sensitivity to change.

Results

In the evaluation of interrater reliability for the new FRA system, the ICC (95% confidence interval) of the total score was 0.77 (0.47–0.98), with good reliability. The SEM was 11.61 for the total FRA score. A good to excellent reliability was observed, with ICC levels of 0.73 to 0.91 for the 4 composite scores of the new FRA system. Following the 8-week exercise intervention, the mean total FRA score (effect size, 0.58) significantly increased (p=0.028).

Conclusion

The new FRA system has generally moderate to excellent interrater reliability and reliable sensitivity to change in community-dwelling older adults. Our findings provide support for the reliability of the new FRA system in healthy older adults without a fall history.

Sensitivity to change of mobility measures in musculoskeletal conditions on lower extremities in outpatient rehabilitation settings

STUDY DESIGN

Prospective longitudinal study.

OBJECTIVE

To examine the sensitivity of the Mobility Activities Measure for lower extremities and to compare it to the sensitivity of the Physical Functioning Scale (PF-10) and the Patient-Specific Functional Scale (PSFS) at week 4 and week 8 post-hospitalization in outpatient rehabilitation settings.

BACKGROUND

Mobility Activities Measure is a set of short mobility measures to track outpatient rehabilitation progress: its scales have shown good properties but its sensitivity to change has not been reported.

METHODS

Patients with musculoskeletal conditions were recruited at admission in three outpatient rehabilitation settings in Spain. Data were collected at admission, week 4 and week 8 from an initial sample of 236 patients (mean age ± SD = 36.7 ± 11.1).

MAIN OUTCOME MEASURES

Mobility Activities Measure scales for lower extremity; PF-10; and PSFS.

RESULTS

All the Mobility Activities Measure scales were sensitive to both positive and negative changes (the Standardized Response Means (SRMs) ranged between 1.05 and 1.53 at week 4, and between 0.63 and 1.47 at week 8). The summary measure encompassing the three Mobility Activities Measure scales detected a higher proportion of participants who had improved beyond the minimal detectable change (MDC) than detected by the PSFS and the PF-10 both at week 4 (86.64% vs. 69.81% and 42.23%, respectively) and week 8 (71.14% vs. 55.65% and 60.81%, respectively).

CONCLUSIONS

The three Mobility Activities Measure scales assessing the lower extremity can be used across outpatient rehabilitation settings to provide consistent and sensitive measures of changes in patients' mobility. Implications for rehabilitation All the scales of the Mobility Activities Measure for the lower extremity were sensitive to both positive and negative change across the follow-up periods. Overall, the summary measure encompassing the three Mobility Activities Measure scales for the lower extremity appeared more sensitive to positive changes than the Physical Functioning Scale, especially during the first four weeks of treatment. The summary measure also detected a higher percentage of participants with positive change that exceeded the minimal detectable change than the Patient-Specific Functional Scale and the Physical Functioning Scale at the first follow-up period. By demonstrating their consistency and sensitivity to change, the three Mobility Activities Measures scales can now be considered in order to track patients' functional progress. Mobility Activities Measure can be therefore used in patients with musculoskeletal conditions across outpatient rehabilitation settings to provide estimates of change in mobility activities focusing on the lower extremity.

Construct validity and sensitivity to change of Belgrade Ultrasound Enthesitis Score in patients with spondyloarthritis: a pilot study

The objective of this study was to determine the construct validity and sensitivity to change of Belgrade Ultrasound Enthesitis Score (BUSES) in spondyloarthritis patients. Seventy-six spondyloarthritis patients with enthesitis were included in this pilot, prospective, double-blinded ultrasound study. Thirty-four patients received biological and forty-two patients received non-biological therapy. BUSES was determined at the beginning, after 1, 3, and 6 months. Spearman's correlation coefficient was calculated between BUSES and baseline characteristics. Brunner-Langer mixed non-parametric ANOVA was used to examine sensitivity to change of BUSES and effect of biological therapy on BUSES. Effect of time on the presence of each of the ultrasound enthesitis signs (increased thickness, hypoehogenicity, Power Doppler, enthesophytes, and erosions) was assessed using Cochran Q test. There was a weak, positive correlation between BUSES and disease duration, clinical enthesitis score, BASFI, BASDAI, and ASDAS-ESR/CRP. BUSES was higher at the beginning than after 1 month (p = 0.004), after 3 months (p < 0.001) and after 6 months (p < 0.001), as well as BUSES was higher after 1 month than after 3 months (p < 0.001) and after 6 months (p = 0.002). There is no difference in efficiency between non-biological and biological therapies on BUSES. Increased thickness, hypoechogenicity, and Power Doppler have decreased on Achilles tendon's and plantar fascia's enthesis over time. BUSES has a certain degree of construct validity because of the weak, positive correlation with parameters referring to severity of spondyloarthritis. BUSES demonstrated sensitivity to change over time due to decreasing of ultrasound acute enthesitis signs in treated spondyloarthritis patients. BUSES could be useful for monitoring the progression of enthesitis and effectiveness of the treatment.

Monitoring somatic symptoms in patients with mental disorders: Sensitivity to change and minimal clinically important difference of the Somatic Symptom Scale - 8 (SSS-8)

OBJECTIVE

The SSS-8 is a brief questionnaire for the assessment of somatic symptom burden. This study examines its sensitivity to change and the minimal clinically important difference (MCID) in patients with mental disorders.

METHOD

55 outpatients with mental disorders completed the SSS-8 and measures of anxiety, depression, and disability before and after receiving treatment. Effect sizes and correlations between the change scores were calculated. The MCID was estimated using a one standard error of measurement threshold and the change in disability as an external criterion.

RESULTS

There was a medium decline in somatic symptom burden for the complete sample (n=55, d_z=0.53) and a large decline in a subgroup with very high somatic symptom burden at baseline (n=11, d_z=0.94). Decreases in somatic symptom burden were associated with decreases in anxiety (r=0.68, p<0.001), depression (r=0.62, p<0.001) and disability (r=0.51, p<0.001). The MCID was estimated as a 3-point decrease.

CONCLUSION

The SSS-8 is sensitive to change. A 3-point decrease reflects a clinically important improvement. Due to its brevity and sound psychometric properties, the SSS-8 is useful for monitoring somatic symptom burden.

Validation of the World Health Organization Disability Assessment Schedule in people with severe mental disorders in rural Ethiopia

Background

The World Health Organization Disability Assessment Schedule (WHODAS-2.0) has been adapted and validated in several cultures, but data on performance in the African context are lacking. The aim of the study was to evaluate the validity and psychometric properties of the WHODAS-2.0 among people with severe mental disorders (SMD) and their caregivers in a rural African setting.

Methods

The content validity of the 36 item WHODAS was assessed using free listing and pile sorting in 36 community members. Cognitive interviewing was conducted with 20 people with SMD and 20 caregivers to assess comprehensibility. Convergent validity and sensitivity to change were evaluated in a facility-based cohort study of new or acutely relapsed cases of people with SMD (n = 150) and their caregivers (n = 150) consecutively recruited from a psychiatric clinic. A repeat assessment was conducted in a sub-sample (n = 84) after 6 weeks. Confirmatory factor analysis was used to evaluate construct validity in people with SMD (n = 250) and their caregivers (n = 250).

Results

Internal consistency of the items of the overall scale and each domain ranged from very good (alpha = 0.82) to excellent (alpha = 0.98). Scores on the WHODAS-2.0 correlated highly with a locally developed measure of functioning (r = 0.88) and moderately with clinical symptom severity (r = 0.52). The WHODAS- 2.0 was sensitive to treatment changes (effect size = 0.50). As hypothesized, the six sub-scales loaded highly onto the general disability factor and each item loaded significantly onto their respective domains. The factor loadings of each item in the one factor model of the brief version of WHODAS (12 item) were also high. For both 12- and 36-item scales the goodness of fit indices, were close to, but outside of, recommended ranges. The caregiver data of both the 36 and 12 item versions had similar psychometric properties, but higher mean values and better responsiveness to change.

Conclusions

Our study showed that both the 12 and 36 item versions of the WHODAS 2.0 have acceptable validity and psychometric properties and can be used as a cross-cultural measure; however, careful and rigorous adaptation is required for rural African settings.

Electronic supplementary material

The online version of this article (doi:10.1186/s12955-017-0647-3) contains supplementary material, which is available to authorized users.

Development of the LaComm 1.0, A French medical communication analysis software: A study assessing its sensitivity to change

OBJECTIVE

To test and compare the sensitivity to change of a communication analysis software, the LaComm 1.0, to the CRCWEM's using data from a randomized study assessing the efficacy of a communication skills training program designed for nurses.

METHODS

The program assessment included the recording of two-person simulated interviews at baseline and after training or 3 months later. Interview transcripts were analyzed using the CRCWEM and the LaComm 1.0 tools.

RESULTS

One hundred and nine oncology nurses (mainly graduated or certified) were included in the study. The CRCWEM detected 5 changes out of 13 expected changes (38%) (e.g., more open directive questions after training) and the LaComm 1.0, 4 changes out of 7 expected changes (57%) (e.g., more empathic statements after training). For open directive question, the effect sizes of the group-by-time changes were slightly different between tools (CRCWEM: Cohen's d=0.97; LaComm 1.0: Cohen's d=0.67).

CONCLUSIONS

This study shows that the LaComm 1.0 is sensitive to change.

PRACTICE IMPLICATIONS

The LaComm 1.0 is a valid method to assess training effectiveness in French. The use of the Lacomm 1.0 in future French communication skills training programs will allow comparisons of studies.

Development and validation of a contextual measure of functioning for people living with severe mental disorders in rural Africa

BACKGROUND

Most measures of functioning in people with severe mental disorders (SMD) have been developed in Western societies. Many of the questions in these scales are culture-bound, fail to capture differentiation of tasks by gender and are difficult to adapt to other contexts. The aim of this study was to develop a measure of functioning for people with SMD which is contextually appropriate for a rural African setting.

METHODS

A review of existing scales, a qualitative study, free listing and pile sorting exercises, and expert consensus were used to establish a pool of items. Cognitive interviewing guided initial item reduction and refinement. The resulting scale was pilot-tested in people with SMD (n = 200) and their caregivers (n = 200) to inform further item reduction based on psychometric properties. The final Butajira Functioning Scale (BFS) comprised 33 items that were common to both men and women, and an additional eight items for women only, covering the following domains: self-care, work, and family and community participation. Psychometric properties of the finalized BFS were examined in a facility-based sample of 150 people with SMD and their caregivers (n = 150), with longitudinal follow-up of n = 84.

RESULTS

The BFS in people with SMD had excellent internal consistency (Cronbach's α = 0.99), acceptable convergent validity (r = 0.88 with the World Health Organization Disability Assessment Schedule [WHODAS-2.0] and r = 0.32 with the Brief Psychiatric Rating Scale [BPRS-E]) and was sensitive to change following treatment (effect size =0.50). Addition of the items specific to women did not improve the psychometric properties. The caregiver version had similar psychometric properties but higher mean values for each item and better responsiveness to change. Exploratory factor analysis of the BFS provided evidence of construct validity, with four underlying dimensions.

CONCLUSIONS

We have developed a measure of functioning for people with SMD in a rural, low income country setting with acceptable psychometric properties. The BFS is easy to administer, sensitive to changes following treatment and has content, construct and convergent validity. The BFS includes domains from existing measures, but has more emphasis on social and occupational domains, which reflects priorities in the setting.

Increasing the Sensitivity of Measures to Change

Little attention is paid in prevention research to the ability of measures to accurately assess change, termed "responsiveness" or "sensitivity to change." This paper reviews definitions and measures of responsiveness, and suggests five strategies for increasing sensitivity to change, with central focus on prevention research with small samples: (a) improving understandability and cultural validity, (b) assuring that the measure covers the full range of the latent construct being measured, (c) eliminating redundant items, (d) maximizing sensitivity of the device used to collect responses; and (e) asking directly about change. Examples of the application of each strategy are provided. The discussion focuses on using the issues as a checklist for improving measures and the implications of sensitivity to change for prevention research with small samples.

Development of a metric for tracking and comparing population health based on the minimal generic set of domains of functioning and health

Background

The following minimal set of valid health domains for tracking the health of both clinical and general populations has recently been proposed: 1) energy and drive functions, 2) emotional functions, 3) sensation of pain, 4) carrying out daily routine, 5) walking and moving around, and 6) remunerative employment. This study investigates whether these domains can be integrated into a sound psychometric measure to adequately assess, compare, and monitor the health of populations.

Methods

Data from waves 3 and 4 of the English Longitudinal Study of Ageing (ELSA) were analysed (N = 9779 and 11,050). From ELSA, 12 items operationalizing the six domains of the minimal generic set were identified. The Partial Credit Model (PCM) was applied to create a health metric based on these items. The Item Response Theory (IRT) model assumptions of unidimensionality, local independence, and monotonicity were evaluated, and Differential Item Functioning (DIF) was examined for sex and age groups. The psychometric properties of: 1) internal consistency reliability, 2) construct validity, and 3) sensitivity to change were evaluated to establish the final health metric.

Results

IRT model assumptions were found to be fulfilled. None of the items showed DIF by sex or age group. The final health metric demonstrated sound psychometric properties.

Conclusions

The health metric developed in this study – based on the domains of the minimal generic set – proved useful for a wide range of health comparisons, especially for different groups of persons, and both cross-sectionally and over time. Monitoring health over time provides especially useful information for health care providers and health policymakers and both in clinical settings and the general population. The developed health metric offers a wide range of applications, including comparisons of levels of health among different groups in the general population, clinical populations, and even populations within and across different countries.

Electronic supplementary material

The online version of this article (doi:10.1186/s12963-016-0088-y) contains supplementary material, which is available to authorized users.

Differences in subchondral bone size after one year in osteoarthritic and healthy knees

OBJECTIVE

Increase of subchondral bone area (tAB) in OA has been reported, but it remains unclear if this is specific to OA. We investigated differences in knee tAB after one year in healthy subjects and in those with radiographic OA (rOA).

METHOD

MR images of 899 right knees from the OA Initiative were acquired at baseline and one year follow-up (year-1). Medial and lateral tibial cartilage (MT and LT) and weight-bearing femoral cartilage (cMF and cLF) were segmented and tAB computed. Subjects were stratified into: healthy controls, pre-rOA (K&L grades 0 and 1, with OA risk factors), established rOA (K&L grades 2-4), and independently with regards to joint space narrowing (without, with medial, lateral and bilateral JSN). Primary analysis tested if tAB was different between baseline and year-1 in rOA. Exploratory analyses investigated whether: (1) tAB changes differed between healthy controls and those with rOA; (2) tAB differences were greater in higher K&L grades; and (3) tAB was different between baseline and year-1 in JSN. Significance was set at P < 0.0125.

RESULTS

Differences in tAB were found in rOA in MT, cMF and cLF (ranging from +0.2% to +0.4%; P < 0.001), but not in healthy controls or pre-rOA. Rates of change did not differ between groups. Within the JSN groups differences of 0.2-0.4% were found in the femur (P < 0.05).

CONCLUSION

We find that knee tABs differ in rOA between baseline and year-1, but the change was not greater than in healthy knees, and is restricted to the femur in JSN.

Evaluation of Computerized Adaptive Tests (CATs) for longitudinal monitoring of depression, anxiety, and stress reactions

BACKGROUND

Computerized adaptive testing (CAT) based on Item Response Theory, (IRT) offers an efficient way for accurate measurement of patient reported outcomes. The efficiency lies within a minimal response burden and a high measurement precision over a broad measurement range. The objective of the study was to evaluate and compare the responsiveness of CATs measuring anxiety, depression, and stress reaction to standard static self-assessment tools.

METHODS

Longitudinal data of n=595 psychosomatic inpatients were analyzed for evaluating retest-reliability and sensitivity to change of the CATs compared to static measures (GAD-7, PHQ-9, and PSQ) using correlational and ANOVA statistics. The study hypothesized that CATs are at least as retest-reliable and as sensitive to change as static tools.

RESULTS

The three CATs show a low burden for patients, administering on average 5-7 (±2-6SD) items with similar retest-reliability compared to the static tools applied (A-CAT: r=.78 vs. GAD-7: r=.75, D-CAT: r=.71 vs. PHQ-9: r=.75, S-CAT: r=.80 vs. PSQworries scale: r=.80). The CATs were overall as sensitive to change as the static tools (Cohen׳s d ranged between .19 and .69).

LIMITATIONS

This is a monocenter, observational, longitudinal study without external clinical criteria; thus generalization to other settings may be limited.

CONCLUSIONS

The tested CATs belong to the first generation of CATs being used in daily routine for more than a decade. They are as retest reliable and sensitive to change as static tools. Newer CATs may provide further practical advantages.

Validity and sensitivity to change of three scales for the radiographic assessment of knee osteoarthritis using images from the Multicenter Osteoarthritis Study (MOST)

OBJECTIVES

The purpose of this study was to assess the concurrent validity and sensitivity to change of three knee osteoarthritis (OA) grading scales. The Kellgren-Lawrence (KL) and the Osteoarthritis Research Society International (OARSI) joint space narrowing (JSN) grading scales are well-established. The third scale, the compartmental grading scale for OA (CG) is a novel scale which grades JSN, femoral osteophytes, tibial erosion and subluxation to create a total score.

METHODS

One sample of 72 posteroanterior (PA) fixed-flexion radiographs displaying mild to moderate knee OA was selected from the Multicenter Osteoarthritis Study (MOST) to study validity. A second sample of 75 radiograph pairs, which showed an increase in OA severity over 30 months, was selected to study sensitivity to change. The three radiographic grading scales were applied to each radiograph in both samples. Spearman's rank correlation coefficients were used to correlate the radiographic grades and the change in grades over 30 months with a Whole-organ Magnetic Resonance Imaging Score (WORMS)-based composite score which included five articular features of knee OA.

RESULTS

Correlations between the KL, OARSI JSN and CG grading scales and the magnetic resonance image (MRI)-based score were 0.836, 0.840 and 0.773 (P < 0.0001) respectively while correlations between change in the radiographic grading scales and change in the MRI-based score were 0.501, 0.525 and 0.492 (P < 0.0001).

CONCLUSIONS

All three radiographic grading scales showed high validity and are suitable to assess knee OA severity. They showed moderate sensitivity to change; therefore caution should be taken when using ordinal radiographic grading scales to monitor knee OA over time.

The Satisfaction with Daily Occupations (SDO-13) Scale: Psychometric Properties among Clients in Primary Care in Sweden

This article describes the results to expand and develop the use of the Satisfaction with Daily Occupations (SDO-13) Scale. Data were collected in primary care before (I) and after intervention (II) among clients with stress-related disorders and musculoskeletal pain. The Cronbach's alpha values of the SDO-13 Scale were 0.80 and 0.88. Convergent validity was assessed against global occupational satisfaction and general health, resulting in rs = -0.65 (p < 0.001) and rs = -0.46 (p < 0.001). The SDO-13 Scale could not discriminate between the primary care sample and a psychiatric sample (p = 0.15), whereas number of current occupations could (p < 0.001). The SDO-13 was not sensitive to change (p = 0.92). Future studies need to explore criterion and construct validity based on more dissimilar samples and more standardized interventions. Applications of these results to practice are discussed.

Preliminary evaluation of the sensitivity to change of DE-STA motor unit number estimation in the upper trapezius muscle in amyotrophic lateral sclerosis

OBJECTIVE

To compare the sensitivity to change of decomposition-enhanced spike-triggered averaging (DE-STA) motor unit number estimation (MUNE) in the upper trapezius (UT) to that of various clinical outcome measures in subjects with amyotrophic lateral sclerosis (ALS).

METHODS

Ten patients with clinically probable or definite ALS were assessed at baseline, 2, 4 and 6months with the following outcome measures: manual muscle testing in five upper extremity muscle groups, scapular elevation and elbow flexion peak force measured with hand-held dynamometry, MUNE, forced vital capacity and the Revised ALS Functional Rating Scale (ALSFRS-R).

RESULTS

ALSFRS-R was the only outcome measure for which there was a significant difference between baseline and 6months (p=0.034). ALSFRS-R had the largest standardized response mean (SRM), and was thus the most sensitive to change. MUNE demonstrated a decline over 6months and a moderate SRM (-0.63).

CONCLUSIONS

This study has demonstrated a moderate degree of sensitivity to change for DE-STA MUNE as applied to the UT in subjects with ALS.

SIGNIFICANCE

In this preliminary study, DE-STA MUNE detected motor unit loss over 6months, with a moderate degree of sensitivity, in the upper trapezius of subjects with ALS.

Rates and sensitivity of knee cartilage thickness loss in specific central reading radiographic strata from the osteoarthritis initiative

OBJECTIVE

To compare the rate and sensitivity to change of quantitative cartilage thickness change with magnetic resonance imaging (MRI) across specific radiographic strata of knee osteoarthritis (KOA) from central expert readings of the Osteoarthritis Initiative (OAI). Specifically, we explored whether Kellgren Lawrence grade (KLG) 2 knees with radiographic joint space narrowing (JSN) displayed greater cartilage loss than those without JSN, and whether knees with medial JSN grade2 had greater loss than those with grade1.

METHODS

One-year femorotibial cartilage thickness change was obtained for 836 knees, 112 without, and 724 with definite radiographic KOA based on baseline site readings. The maximum subregional cartilage loss, and cartilage thickness change in the total femorotibial joint (FTJ) and medial femorotibial compartment (MFTC) were analyzed across different radiographic strata (central vs site readings).

RESULTS

The maximum subregional rate of change was significantly greater in central_KLG2 knees with than in those without JSN (172 ± 152 vs 134 ± 100 μm; P = 0.03). In contrast, the rate did not differ significantly between central_KLG1 knees with and without JSN. MFTC cartilage loss in central_medial_grade2 JSN knees was substantially and significantly greater than in grade1 knees (-70 ± 159 vs -31 ± 126 μm; P = 0.02). For comparison, the loss in grade3 knees was -72 ± 122 μm.

CONCLUSIONS

In KLG2 knees, presence of radiographic JSN was associated with significantly and substantially greater rates of subregional cartilage loss. Differentiating knees with mild vs moderate medial JSN, and definite radiographic OA knees with vs without JSN is important in predicting structural progression of KOA, and for planning clinical trials testing the efficacy of disease modifying drugs (DMOADs).