Challenging the understanding of significant improvement and outcome in schizophrenia - the concept of reliable and clinically significant change methods

Health changes from trans-theoretical model-based education in older adults with mild cognitive impairment: A randomized controlled trial

BACKGROUND

Low awareness and misconceptions surrounding mild cognitive impairment highlight the urgent need for effective health education. Reluctance to seek intervention and poor adherence to management strategies make behavior-oriented health education essential.

OBJECTIVE

To assess the effectiveness and clinical significance of a trans-theoretical model-based health education program on cognitive-behavioral outcomes in older adults with mild cognitive impairment.

DESIGN

A two-arm and assessor-blinded randomized controlled trial.

SETTINGS AND PARTICIPANTS

100 community-dwelling older adults with mild cognitive impairment in Huzhou, China.

METHODS

Participants were randomly assigned to a trans-theoretical model-based health education program (weekly 45-60 min sessions for 8 weeks, followed by 12 weeks of unsupervised practice) or a wait-list control group receiving standard health education. Disease knowledge, behavioral stage, and adherence to health management behaviors were assessed at baseline, 8-week, and 20-week. Effects were evaluated at the group level via generalized estimating equation and at the individual level using reliable and clinically significant change.

RESULTS

The trans-theoretical model-based health education program demonstrated significant effects over the wait-listed control. Generalized estimating equation analyses showed statistically significant effects on behavioral stage (β8-week = 1.04, 95%CI = 0.34-1.75; β20-week = 1.72, 95%CI = 0.95-2.49), disease knowledge (β8-week = 1.14, 95%CI = 0.26-2.02; β20-week = 1.78, 95%CI = 0.87-2.69), and adherence to health management behaviors (β8-week = 6.20, 95%CI = 2.03-10.37; β20-week = 10.74, 95%CI = 6.47-15.01) at both measured intervals. Additionally, global cognitive function (β8-week = 0.60, 95%CI = - 0.18-1.38; β20-week = 2.42, 95%CI = 1.64-3.20), Purdue Pegboard Test Assembly and Bimanual Tasks (β8-week = 0.16/0.38, 95%CI = - 0.21-0.53/-0.18-0.94; β20-week = 0.96/1.80, 95%CI = 0.57-1.35/1.17-2.43) improved significantly over time. Reliable and clinically significant change analyses at 8 weeks indicated significant improvements in the intervention group: 57 % of participants improved in disease knowledge (22 % clinically significant), 90 % in adherence to health management behaviors (17 % clinically significant), and 61 % in global cognitive function (10 % clinically significant). By 20 weeks, these rates increased to 63 % (29 %), 100 % (25 %), and 78 % (27 %). However, non-significant improvements in depression symptoms and sleep quality were found at individual-level assessment.

CONCLUSIONS

This study shows that the trans-theoretical model-based health education program effectively enhances cognitive-behavioral health outcomes in older adults with mild cognitive impairment, with benefits persisting for 12 weeks. Future research should further explore the potential mechanisms underlying the cognition and behavior-enhancing effects of this program.

REGISTRATION NUMBER

ChiCTR1900028351.

Utilizing a learning health system to capture real-world patient data: Application of the reliable change index to evaluate and improve the outcome of a pain rehabilitation program

BACKGROUND AND OBJECTIVES

The learning healthcare system (LHS) has been developed to integrate patients' clinical data into clinical decisions and improve treatment outcomes. Having little guidance on this integration process, we aim to explain (a) an applicable analytic tool for clinicians to evaluate the clinical outcomes at a group and an individual level and (b) our quality improvement (QI) project, analyzing the outcomes of a new outpatient pain rehabilitation program ("Back-in-Action": BIA) and applying the analysis results to modify our clinical practice.

METHODS

Through our LHS (CHOIR; https://choir.stanford.edu), we administered the Pain Catastrophizing Scale (PCS), Chronic Pain Acceptance Questionnaire (CPAQ), and Patient-Reported Outcomes Measures (PROMIS)® before and after BIA. After searching for appropriate analytic tools, we decided to use the Reliable Change Index (RCI) to determine if an observed change in the direction of better (improvement) or worse (deterioration) would be beyond or within the measurement error (no change).

RESULTS

Our RCI calculations revealed that at least a 9-point decrease in the PCS scores and 10-point increase in the CPAQ scores would indicate reliable improvement. RCIs for the PROMIS measures ranged from 5 to 8 T-score points (i.e., 0.5-0.8 SD). When evaluating change scores of the PCS, CPAQ, and PROMIS measures, we found that 94% of patients showed improvement in at least one domain after BIA and 6% showed no reliable improvement.

CONCLUSIONS

Our QI project revealed RCI as a useful tool to evaluate treatment outcomes at a group and an individual level, and RCI could be incorporated into the LHS to generate a progress report automatically for clinicians. We further explained how clinicians could use RCI results to modify a clinical practice, to improve the outcomes of a pain program, and to develop individualized care plans. Lastly, we suggested future research areas to improve the LHS application in pain practice.

Adapting the multilevel model for estimation of the reliable change index (RCI) with multiple timepoints and multiple sources of error

OBJECTIVE

One of the primary tools in the assessment of individual-level patient outcomes is Jacobson and Truax, (1991's) Reliable Change Index (RCI). Recent efforts to optimize the RCI have revolved around three issues: (a) extending the RCI beyond two timepoints, (b) estimating the RCI using scale scores from item response theory or factor analysis and (c) estimation of person- and time-specific standard errors of measurement.

METHOD

We present an adaptation of a two-stage procedure, a measurement error-corrected multilevel model, as a tool for RCI estimation (with accompanying Statistical Analysis System syntax). Using DASS-21 data from a community-based mental health center (N = 379), we illustrate the potential for the model as unifying framework for simultaneously addressing all three limitations in modeling individual-level RCI estimates.

RESULTS

Compared to the optimal-fitting RCI model (moderated nonlinear factor analysis scoring with measurement error correction), an RCI model that uses DASS-21 total scores produced errors in RCI inferences in 50.8% of patients; this was largely driven by overestimation of the proportion of patients with statistically significant improvement.

CONCLUSION

Estimation of the RCI can now be enhanced by the use of latent variables, person- and time-specific measurement errors, and multiple timepoints.

Longitudinal analysis of statistical and clinically significant psychosocial change following mental health rehabilitation

Purpose: With appropriate mental health rehabilitation, schizophrenia is increasingly associated with reports of recovery and stability. However, there is little empirical evidence evaluating the efficacy of services delivering this care. This study evaluated the effectiveness of rehabilitation for improving psychosocial function in consumers with schizophrenia.Methods: An electronic database of standardized assessment instruments mandated and maintained by the health service was retrospectively reviewed to extract ratings of psychosocial function, daily living skills, and mood state from consecutive admissions to an inpatient rehabilitation service. Outcomes were compared at admission, discharge, and one-year follow-up to identify statistically significant change. Individual reliable and clinically significant change was also assessed by comparison with a normative group of clients functioning independently in the community.Results: From admission to discharge the rehabilitation group made statistically significant gains in psychosocial function and daily living skills. Improvements were reliable and clinically significant in one-quarter to one-third of individual consumers. Approximately half sustained their improvements at follow-up, although this represented only a small fraction of the overall cohort. Consumers not demonstrating gains exhibited psychometric floor effects at admission.Conclusions: Rehabilitation can produce statistically and clinically significant immediate improvement in psychosocial function for a sub-set of consumers with elevated scores at admission. The durability of any gains is less clear, and strategies promoting longer-term maintenance are encouraged. Furthermore, currently mandated outcome measures are confounded by issues of sensitivity and reporting compliance, and exploration of alternative instruments for assessing recovery is recommended.Implications for RehabilitationRoutinely collected standardized outcome measures can be used to investigate the effectiveness of mental health rehabilitationIn addition to statistical significance, the clinical significance of outcomes should be evaluated to identify change that is individually meaningfulCurrently mandated outcomes instruments do not adequately evaluate many individuals' recovery journeyMental health service evaluation and quality improvement processes would likely benefit from adoption of recovery-oriented measures.

Challenging the understanding of significant improvement and outcome in schizophrenia - the concept of reliable and clinically significant change methods

Significant changes of schizophrenia patients during inpatient treatment were evalutaed and compared to established outcome criteria. The concept of reliable and clinically significant change methods was applied to three hundred and ninety-six patients suffering from a schizophrenia spectrum disorder. First, information on whether or not the change of the patient's condition is sufficient in order to declare that it is beyond a measurement error or random effect (= reliable change) was evaluated and in a second step it was observed if the reliable change was clinically meaningful (= clinically significant change). Different Positive and Negative Syndrome Scale for Schizophrenia (PANSS) thresholds were applied to define the clinically significant change (40, 45 and 50 points). These changes were then compared to established outcome criteria such as response and remission. Seventy-nine of the 396 patients (20%) showed a reliable improvement of symptoms, whereas 70% improved without achieving a reliable change of their condition. Of the 79 patients achieving a reliable change during treatment 8-15% concurrently showed a clinically significant change depending on the respective PANSS threshold. In contrast, 56% of the patients achieved response and 60% were in remission at discharge when applying established outcome criteria. Our results showed that a rather small number of schizophrenia patients were found to reliably change during inpatient treatment, with even less patients achieving a clinically significant change. The concept of reliable and clinically significant changes revealed to be a lot more stringent than today's established outcome criteria and should be critically evaluated regarding its use in schizophrenia patients.