Adapting the multilevel model for estimation of the reliable change index (RCI) with multiple timepoints and multiple sources of error

Minimal important difference metrics and test-retest reliability of the PTSD Checklist for DSM-5 with a primary care sample

The PTSD Checklist for DSM-5 (PCL-5) is a measure of posttraumatic stress disorder (PTSD) symptom severity that is widely used for clinical and research purposes. Although previous work has examined metrics of minimal important difference (MID) of the PCL-5 in veteran samples, no work has identified PCL-5 MID metrics among adults in primary care in the United States. In this secondary analysis, data were evaluated from primary care patients (N = 971) who screened positive for PTSD and participated in a large clinical trial in federally qualified health centers in three U.S. states. Participants primarily self-identified as women (70.2%) and White (70.3%). We calculated test-retest reliability using clinic registry data and multiple distribution- and anchor-based metrics of MID using baseline and follow-up survey data. Test-retest reliability (Pearson's r, Spearman's ρ, intraclass correlation coefficient) ranged from adequate to excellent (.79-.94), with the shortest time lag demonstrating the highest reliability estimate. The MID for the PCL-5 was estimated using multiple approaches. Distribution-based approaches indicated an MID range of 8.5-12.5, and anchor-based approaches indicated an MID range of 9.8-11.7. Taken together, the MID metrics indicate that PCL-5 change scores of 9-12 likely reflect real change in PTSD symptoms and indicate at least an MID for patients, whereas PCL-5 change scores of 5 or less likely are not reliable. These findings can help inform clinicians using the PCL-5 in similar populations to track patient responses to treatment and help researchers interpret PCL-5 score changes in clinical trials.

Preventing Mental Health Problems in Children After High Conflict Parental Separation/Divorce Study: An Optimization Randomized Controlled Trial Protocol

Parental divorce is a childhood stressor that affects approximately 1.1 million children in the U.S. annually. The children at greatest risk for deleterious mental health consequences are those exposed to high interparental conflict (IPC) following the separation/divorce. Research shows that children's emotional security and coping efficacy mediate the impact of IPC on their mental health. Interventions targeting their adaptive coping in response to IPC events may bolster their emotional security and coping efficacy. However, existing coping interventions have not been tested with children exposed to high post-separation/divorce IPC, nor has any study assessed the effects of individual intervention components on children's coping with IPC and their mental health. This intensive longitudinal intervention study examines the mechanisms through which coping intervention components impact children's responses to interactions in interparental relationships. A 23 factorial experiment will assess whether, and to what extent, three candidate intervention components demonstrate main and interactive effects on children's coping and mental health. Children aged 9-12 (target N = 144) will be randomly assigned to one of eight combinations of three components with two levels each: (1) reappraisal (present vs. absent), (2) distraction (present vs. absent), (3) relaxation (present vs. absent). The primary outcomes are child-report emotional security and coping efficacy at one-month post-intervention. Secondary outcomes include internalizing and externalizing problems at the three-month follow-up. Based on data from this optimization phase RCT, intervention components will be selected to comprise a multi-component intervention and assessed for effectiveness in a subsequent evaluation phase RCT.

Introduction to the Special Issue on Innovations and Applications of Integrative Data Analysis (IDA) and Related Data Harmonization Procedures in Prevention Science

This paper serves as an introduction to the special issue of Prevention Science entitled, "Innovations and Applications of Integrative Data Analysis (IDA) and Related Data Harmonization Procedures in Prevention Science." This special issue includes a collection of original papers from multiple disciplines that apply individual-level data synthesis methodologies, including IDA, individual participant meta-analysis, and other related methods to harmonize and integrate multiple datasets from intervention trials of the same or similar interventions. This work builds on a series of papers appearing in a prior Prevention Science special issue, entitled "Who Benefits from Programs to Prevent Adolescent Depression?" (Howe, Pantin, & Perrino, 2018). Since the publication of this prior work, the use of individual-level data synthesis has increased considerably in and outside of prevention. As such, there is a need for an update on current and future directions in IDA, with careful consideration of innovations and applications of these methods to fill important research gaps in prevention science. The papers in this issue are organized into two broad categories of (1) evidence synthesis papers that apply best practices in data harmonization and individual-level data synthesis and (2) new and emerging design, psychometric, and methodological issues and solutions. This collection of original papers is followed by two invited commentaries which provide insight and important reflections on the field and future directions for prevention science.

Differential symptom weighting in estimating empirical thresholds for underlying PTSD severity: Toward a "platinum" standard for diagnosis?

OBJECTIVE

Symptom counts as the basis for Post-Traumatic Stress Disorder (PTSD) diagnoses in the DSM presume each symptom is equally reflective of underlying disorder severity. However, the "equal weight" assumption fails to fit PTSD symptom data when tested. The present study developed an enhanced PTSD diagnosis based on (a) a conventional PTSD diagnosis from a clinical interview and (b) an empirical classification of full PTSD that reflected the relative clinical weights of each symptom.

METHOD

Baseline structured interview data from Project Harmony (N = 2658) was used. An enhanced diagnosis for full PTSD was estimated using an empirical threshold from moderated nonlinear factor analysis (MNLFA) latent PTSD scale scores, in combination with a full conventional PTSD diagnosis based on interview data.

RESULTS

One in 4 patients in the sample had a PTSD diagnosis that was inconsistent with their empirical PTSD grouping, such that the enhanced diagnostic standard reduced the diagnostic discrepancy rate by 20%. Veterans, and in particular female Veterans, were at greatest odds for discrepancy between their underlying PTSD severity and DSM diagnosis.

CONCLUSION

Psychometric methodologies that differentially weight symptoms can complement DSM criteria and may serve as a platform for symptom prioritization for diagnoses in future editions of DSM.

Determining the reliable feature change in longitudinal radiomics studies: A methodological approach using the reliable change index

PURPOSE

Determination of reliable change of radiomics feature over time is essential and vital in delta-radiomics, but has not yet been rigorously examined. This study attempts to propose a methodological approach using reliable change index (RCI), a statistical metric to determine the reliability of quantitative biomarker changes by accounting for the baseline measurement standard error, in delta-radiomics. The use of RCI was demonstrated with the MRI data acquired from a group of prostate cancer (PCa) patients treated by 1.5 T MRI-guided radiotherapy (MRgRT).

METHODS

Fifty consecutive PCa patients who underwent five-fractionated MRgRT were retrospectively included, and 1023 radiomics features were extracted from the clinical target volume (CTV) and planning target volume (PTV). The two MRI datasets acquired at the first fraction (MRI11 and MRI21) were used to calculate the baseline feature reliability against image acquisition using intraclass correlation coefficient (ICC). The RCI was constructed based on the baseline feature measurement standard deviation, ICC, and feature value differences at two time points between the fifth (MRI51) and the first fraction MRI (MRI11). The reliable change of features was determined in each patient only if the calculated RCI was over 1.96 or smaller than -1.96. The feature changes between MRI51 and MRI11 were correlated to two patient-reported quality-of-life clinical endpoints of urinary domain summary score (UDSS) and bowel domain summary score (BDSS) in 35 patients using the Spearman correlation test. Only the significant correlations between a feature that was reliably changed in ≥7 patients (20%) by RCI and an endpoint were considered as true significant correlations.

RESULTS

The 352 (34.4%) and 386 (37.7%) features among all 1023 features were determined by RCI to be reliably changed in more than five (10%) patients in the CTV and PTV, respectively. Nineteen features were found reliably changed in the CTV and 31 features in the PTV, respectively, in 10 (20%) or more patients. These features were not necessarily associated with significantly different longitudinal feature values (group p-value < 0.05). Most reliably changed features in more than 10 patients had excellent or good baseline test-retest reliability ICC, while none showed poor reliability. The RCI method ruled out the features to be reliably changed when substantial feature measurement bias was presented. After applying the RCI criterion, only four and five true significant correlations were confirmed with UDSS and BDSS in the CTV, respectively, with low true significance correlation rates of 10.8% (4/37) and 17.9% (5/28). No true significant correlations were found in the PTV.

CONCLUSIONS

The RCI method was proposed for delta-radiomics and demonstrated using PCa MRgRT data. The RCI has advantages over some other statistical metrics commonly used in the previous delta-radiomics studies, and is useful to reliably identify the longitudinal radiomics feature change on an individual basis. This proposed RCI method should be helpful for the development of essential feature selection methodology in delta-radiomics.

Reliable change and the reliable change index: still useful after all these years?

In 1984 Jacobson and colleagues introduced the concept of reliable change, viz the amount of change on a measure that an individual needed to show to determine that it exceeded the extent of change likely due to measurement error alone. Establishing reliable change was a pre-requisite for determining clinical significance. This paper summarizes the rationale for determining reliable change as providing an individual-focused, idiographic alternative to the dominant nomothetic approach to clinical outcome research based on group mean data and statistical significance. The conventional computational steps for calculating an individual’s standardized difference (reliable change) score and the minimum raw change score on the measure (a reliable change index) required to classify individuals as reliably positively changed, indeterminate, or reliably deteriorated are described. Two methods for graphically representing reliable change are presented, and a range of possible uses in both research and practice settings are summarized. A number of issues and debates concerning the calculation of reliable change are reviewed. It is concluded that the concept of reliable change remains useful for both cognitive behavioural researchers and practitioners, but that there are options regarding methods of computation. In any use of reliable change, the rationale for selecting among method options and the exact computations used need clear and careful description so that we can continuously judge the utility and appropriateness of the use of reliable change and enhance its value to the field.

Key learning aims

(1)