Using Cluster Bootstrapping to Analyze Nested Data With a Few Clusters

Common odds ratio test and interval estimation for stratified bilateral and unilateral data

In clinical research, data are commonly collected bilaterally from paired organs or bodily parts within individual subjects. However, unilateral data arise when constraints or limiting factors impede the collection of complete bilateral data. In this article, we propose three large-sample tests and five confidence interval methods for making inferences on the common treatment effect, measured by the odds ratio, in a stratified design under integrated bilateral and unilateral data. Our simulation results show that the likelihood ratio-based and score-based tests, along with their associated confidence interval methods, demonstrate robust control of type I error and close-to-nominal coverage probabilities. We apply the proposed methods to real-world datasets of acute otitis media and myopic eyes to showcase their validity and applicability in clinical practice.

Circulating Tumor DNA Assay Detects Merkel Cell Carcinoma Recurrence, Disease Progression, and Minimal Residual Disease: Surveillance and Prognostic Implications

PURPOSE

Merkel cell carcinoma (MCC) is an aggressive skin cancer with a 40% recurrence rate, lacking effective prognostic biomarkers and surveillance methods. This prospective, multicenter, observational study aimed to evaluate circulating tumor DNA (ctDNA) as a biomarker for detecting MCC recurrence.

METHODS

Plasma samples, clinical data, and imaging results were collected from 319 patients. A tumor-informed ctDNA assay was used for analysis. Patients were divided into discovery (167 patients) and validation (152 patients) cohorts. Diagnostic performance, including sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), was assessed.

RESULTS

ctDNA showed high sensitivity, 95% (discovery; 95% CI, 87 to 99) and 94% (validation; 95% CI, 85 to 98), for detecting disease at enrollment, with corresponding specificities of 90% (95% CI, 82 to 95) and 86% (95% CI, 77 to 93). A positive ctDNA during surveillance indicated increased recurrence risk, with hazard ratios (HRs) of 6.8 (discovery; 95% CI, 2.9 to 16) and 20 (validation; 95% CI, 8.3 to 50). The PPV for clinical recurrence at 1 year after a positive ctDNA test was 69% (discovery; 95% CI, 32 to 91) and 94% (validation; 95% CI, 71 to 100), respectively. The NPV at 135 days after a negative ctDNA test was 94% (discovery; 95% CI, 90 to 97) and 93% (validation; 95% CI, 89 to 97), respectively. Patients positive for ctDNA within 4 months after treatment had higher rates of recurrence, with 1-year rates of 74% versus 21% (adjusted HR, 7.4 [95% CI, 2.7 to 20]).

CONCLUSION

ctDNA testing exhibited high prognostic accuracy in detecting MCC recurrence, suggesting its potential to reduce frequent surveillance imaging. ctDNA also identifies high-risk patients who need more frequent imaging and may be best suited for adjuvant therapy trials.

Enhancing the Capabilities of Continuous Glucose Monitoring With a Predictive App

BACKGROUND

Despite abundant evidence demonstrating the benefits of continuous glucose monitoring (CGM) in diabetes management, a significant proportion of people using this technology still struggle to achieve glycemic targets. To address this challenge, we propose the Accu-Chek® SmartGuide Predict app, an innovative CGM digital companion that incorporates a suite of advanced glucose predictive functionalities aiming to inform users earlier about acute glycemic situations.

METHODS

The app's functionalities, powered by three machine learning models, include a two-hour glucose forecast, a 30-minute low glucose detection, and a nighttime low glucose prediction for bedtime interventions. Evaluation of the models' performance included three data sets, comprising subjects with T1D on MDI (n = 21), subjects with type 2 diabetes (T2D) on MDI (n = 59), and subjects with T1D on insulin pump therapy (n = 226).

RESULTS

On an aggregated data set, the two-hour glucose prediction model, at a forecasting horizon of 30, 45, 60, and 120 minutes, achieved a percentage of data points in zones A and B of Consensus Error Grid of: 99.8%, 99.3%, 98.7%, and 96.3%, respectively. The 30-minute low glucose prediction model achieved an accuracy, sensitivity, specificity, mean lead time, and area under the receiver operating characteristic curve (ROC AUC) of: 98.9%, 95.2%, 98.9%, 16.2 minutes, and 0.958, respectively. The nighttime low glucose prediction model achieved an accuracy, sensitivity, specificity, and ROC AUC of: 86.5%, 55.3%, 91.6%, and 0.859, respectively.

CONCLUSIONS

The consistency of the performance of the three predictive models when evaluated on different cohorts of subjects with T1D and T2D on different insulin therapies, including real-world data, offers reassurance for real-world efficacy.

Deep learning-based image quality assessment for optical coherence tomography macular scans: a multicentre study

AIMS

To develop and externally test deep learning (DL) models for assessing the image quality of three-dimensional (3D) macular scans from Cirrus and Spectralis optical coherence tomography devices.

METHODS

We retrospectively collected two data sets including 2277 Cirrus 3D scans and 1557 Spectralis 3D scans, respectively, for training (70%), fine-tuning (10%) and internal validation (20%) from electronic medical and research records at The Chinese University of Hong Kong Eye Centre and the Hong Kong Eye Hospital. Scans with various eye diseases (eg, diabetic macular oedema, age-related macular degeneration, polypoidal choroidal vasculopathy and pathological myopia), and scans of normal eyes from adults and children were included. Two graders labelled each 3D scan as gradable or ungradable, according to standardised criteria. We used a 3D version of the residual network (ResNet)-18 for Cirrus 3D scans and a multiple-instance learning pipline with ResNet-18 for Spectralis 3D scans. Two deep learning (DL) models were further tested via three unseen Cirrus data sets from Singapore and five unseen Spectralis data sets from India, Australia and Hong Kong, respectively.

RESULTS

In the internal validation, the models achieved the area under curves (AUCs) of 0.930 (0.885-0.976) and 0.906 (0.863-0.948) for assessing the Cirrus 3D scans and Spectralis 3D scans, respectively. In the external testing, the models showed robust performance with AUCs ranging from 0.832 (0.730-0.934) to 0.930 (0.906-0.953) and 0.891 (0.836-0.945) to 0.962 (0.918-1.000), respectively.

CONCLUSIONS

Our models could be used for filtering out ungradable 3D scans and further incorporated with a disease-detection DL model, allowing a fully automated eye disease detection workflow.

Target trial framework for determining the effect of changes in training load on injury risk using observational data: a methodological commentary

In recent years, a large focus has been placed on managing training load for injury prevention. To minimise injuries, training recommendations should be based on research that examines causal relationships between load and injury risk. While observational studies can be used to estimate causal effects, conventional methods to study the relationship between load and injury are prone to bias. The target trial framework is a valuable tool that requires researchers to emulate a hypothetical randomised trial using observational data. This framework helps to explicitly define research questions and design studies in a way that estimates causal effects. This article provides an overview of the components of the target trial framework as applied to studies on load and injury and describes various considerations that should be made in study design and analyses to minimise bias.

Evaluating Effects of Multilevel Interventions on Disparity in Health and Healthcare Decisions

In this paper, we introduce an analytic approach for assessing effects of multilevel interventions on disparity in health outcomes and health-related decision outcomes (i.e., a treatment decision made by a healthcare provider). We outline common challenges that are encountered in interventional health disparity research, including issues of effect scale and interpretation, choice of covariates for adjustment and its impact on effect magnitude, and the methodological challenges involved with studying decision-based outcomes. To address these challenges, we introduce total effects of interventions on disparity for the entire sample and the treated sample, and corresponding direct effects that are relevant for decision-based outcomes. We provide weighting and g-computation estimators in the presence of study attrition and sketch a simulation-based procedure for sample size determinations based on precision (e.g., confidence interval width). We validate our proposed methods through a brief simulation study and apply our approach to evaluate the RICH LIFE intervention, a multilevel healthcare intervention designed to reduce racial and ethnic disparities in hypertension control.

Predicting Glaucoma Surgical Outcomes Using Neural Networks and Machine Learning on Electronic Health Records

Purpose

To develop machine learning (ML) and deep learning (DL) models to predict glaucoma surgical outcomes, including postoperative intraocular pressure, use of ocular antihypertensive medications, and need for repeat surgery.

Methods

We identified glaucoma surgeries performed at Stanford from 2013-2024, with two or more postoperative visits with intraocular pressure (IOP) measurement. Patient features were identified from the electronic health record (EHR), including demographics, prior diagnosis and procedure codes, medications and eye exam findings. Classical ML and DL models were developed to predict which glaucoma surgeries would result in surgical failure, defined as (1) IOP not reduced by more than 20% of preoperative baseline on two consecutive postoperative visits, (2) increased classes of glaucoma medications, and (3) need for additional glaucoma surgery or revision of original surgery.

Results

A total of 2398 glaucoma surgeries of 1571 patients were included, of which 1677 surgeries met failure criteria. Random forest performed best for prediction of overall surgical failure, with accuracy of 75.5% and area under the receiver operator curve (AUROC) of 76.7%, similar to the deep learning model (accuracy 75.5%, AUROC 76.6%). Across all models, prediction performance was better for IOP outcomes (AUROC 86%) than need for an additional surgery (AUROC 76%) or need for additional glaucoma medication (AUC 70%).

Conclusions

ML and DL algorithms can predict glaucoma surgery outcomes using structured data inputs from EHRs.

Translational Relevance

Models that predict outcomes of glaucoma surgery may one day provide the basis for clinical decision support tools supporting surgeons in personalizing glaucoma treatment plans.

Estimating infants' language exposure: A comparison of random and volume sampling from daylong recordings collected in a bilingual community

In North America, the characteristics of a child's language environment predict language outcomes. For example, differences in bilingual language exposure, exposure to electronic media, and exposure to child-directed speech (CDS) relate to children's language growth. Recently, these predictors have been studied through the use of daylong recordings, followed by manual annotation of audio samples selected from these recordings. Using a dataset of daylong recordings collected from bilingually raised infants in the United States as an example, we ask whether two of the most commonly used sampling methods, random sampling and sampling based on high adult speech, differ from each other with regard to estimating the frequencies of specific language behaviors. Daylong recordings from 37 Spanish-English speaking families with infants between 4 and 22 months of age were analyzed. From each child's recording, samples were extracted in two ways (at random/based on high adult speech) and then annotated for Language (Spanish/English/Mixed), CDS, Electronic Media, Social Context, Turn-Taking, and Infant Babbling. Correlation and agreement analyses were performed, in addition to paired sample t-tests, to assess how the choice of one or the other sampling method may affect the estimates. For most behaviors studied, correlation and agreement between the two sampling methods was high (Pearson r values between 0.79 and 0.99 for 16 of 17 measures; Intraclass Correlation Coefficient values between 0.78 and 0.99 for 13 of 17 measures). However, interesting between-sample differences also emerged: the degree of language mixing, the amount of CDS, and the number of conversational turns were all significantly higher when sampling was performed based on high adult speech compared to random sampling. By contrast, the presence of electronic media and one-on-one social contexts was higher when sampling was performed at random. We discuss advantages of choosing one sampling technique over the other, depending on the research question and variables at hand.

Baseline characteristics of eyes with early residual fluid post loading phase of aflibercept therapy in neovascular AMD: PRECISE study report 3

PURPOSE

To compare the baseline characteristics in patients with and without early residual fluid (ERF) after aflibercept loading phase (LP) in patients with treatment naïve neovascular age related macular degeneration (nAMD).

METHODS

Patients with nAMD initiated on LP of three intravitreal aflibercept doses were recruited from December 2019 to August 2021. Baseline demographic and OCT features associated with any ERF were analysed using Generalised Estimating Equations to account for inter-eye correlation. Receiver operating characteristic (ROC) curve was performed for selection of CST threshold.

RESULTS

Of 2128 patients enrolled, 1999 eyes of 1862 patients with complete data were included. After LP, ERF was present in 1000 (50.0%), eSRF in 746(37.3%) and eIRF in 428 (21.4%) eyes. In multivariable analysis of baseline features, eyes with increased central subfield thickness (CST) (OR 1.31 per 100 microns increase [95% CI 1.22 to 1.41]; P < 0.001), eyes with IRF and SRF at baseline (1.62 [95% CI 1.17 to 2.22]; P = 0.003), and those with SRF only (OR 2.26 [95% CI 1.59 to 3.20]; P < 0.001) relative to IRF only were determinants of ERF. CST ≥ 418 microns had 57% sensitivity and 58% specificity to distinguish ERF from no ERF at visit 4.

CONCLUSION

On average, 50% of eyes have ERF after aflibercept LP. Clinically relevant baseline determinants of ERF include CST ≥ 418 µ and presence of only SRF. These eyes may require further monthly treatment before extending treatment intervals.

Multiply robust generalized estimating equations for cluster randomized trials with missing outcomes

Generalized estimating equations (GEEs) provide a useful framework for estimating marginal regression parameters based on data from cluster randomized trials (CRTs), but they can result in inaccurate parameter estimates when some outcomes are informatively missing. Existing techniques to handle missing outcomes in CRTs rely on correct specification of a propensity score model, a covariate-conditional mean outcome model, or require at least one of these two models to be correct, which can be challenging in practice. In this article, we develop new weighted GEEs to simultaneously estimate the marginal mean, scale, and correlation parameters in CRTs with missing outcomes, allowing for multiple propensity score models and multiple covariate-conditional mean models to be specified. The resulting estimators are consistent provided that any one of these models is correct. An iterative algorithm is provided for implementing this more robust estimator and practical considerations for specifying multiple models are discussed. We evaluate the performance of the proposed method through Monte Carlo simulations and apply the proposed multiply robust estimator to analyze the Botswana Combination Prevention Project, a large HIV prevention CRT designed to evaluate whether a combination of HIV-prevention measures can reduce HIV incidence.

A comparison of automatic and manual measures of turn-taking in monolingual and bilingual contexts

The Language ENvironment Analysis system (LENA) records children's language environment and provides an automatic estimate of adult-child conversational turn count (CTC) by automatically identifying adult and child speech in close temporal proximity. To assess the reliability of this measure, we examine correlation and agreement between LENA's CTC estimates and manual measurement of adult-child turn-taking in two corpora collected in the USA: a bilingual corpus of Spanish-English-speaking families with infants between 4 and 22 months (n = 37), and a corpus of monolingual families with English-speaking 5-year-olds (n = 56). In each corpus for each child, 100 30-second segments were extracted from daylong recordings in two ways, yielding a total of 9300 minutes of manually annotated audio. LENA's CTC estimate for the same segments was obtained through the LENA software. The two measures of CTC had low correlations for the segments from the monolingual 5-year-olds sampled in both ways, and somewhat higher correlations for the bilingual samples. LENA substantially overestimated CTC on average, relative to manual measurement, for three out of four analysis conditions, and limits of agreement were wide in all cases. Segment-level analyses demonstrated that accidental contiguity had the largest individual impact on LENA's average CTC error, affecting 12-17% of analyzed segments. Other factors significantly contributing to CTC error were speech from other children, presence of multiple adults, and presence of electronic media. These results indicate wide discrepancies between LENA's CTC estimates and manual CTCs, and call into question the comparability of LENA's CTC measure across participants, conditions, and developmental time points.

Diagnostic Performance of Point-of-Care Apparent Diffusion Coefficient Measures to Reduce Biopsy in Breast Lesions at MRI: Clinical Validation

Background The Eastern Cooperative Oncology Group-American College of Radiology Imaging Network Cancer Research Group multicenter A6702 trial identified an optimal apparent diffusion coefficient (ADC) cutoff to potentially reduce biopsies by 21% without affecting sensitivity. Whether this performance can be achieved in clinical settings has not yet been established. Purpose To validate the performance of point-of-care ADC measurements with the A6702 trial ADC cutoff for reducing unnecessary biopsies in lesions detected at breast MRI. Materials and Methods Consecutive breast MRI examinations performed from May 2015 to January 2019 at a single medical center and showing biopsy-confirmed Breast Imaging Reporting and Data System category 4 or 5 lesions, without ipsilateral cancer, were identified. Point-of-care lesion ADC measurements collected at clinical interpretation were retrospectively evaluated. MRI examinations included axial T2-weighted, diffusion-weighted, and dynamic contrast-enhanced sequences. Sensitivity and biopsy reduction rates were calculated by applying the A6702 optimal (ADC, 1.53 × 10-3 mm2/sec) and alternate conservative (1.68 × 10-3 mm2/sec) cutoffs. Lesion pathologic outcomes were the reference standard. To assess reproducibility, one radiologist repeated ADC measurements, and agreement was summarized using the intraclass correlation coefficient. Results A total of 240 lesions in 201 women (mean age, 49 years ± 13 [SD]) with pathologic outcomes (63 malignant and 177 benign) were included. Applying the optimal ADC cutoff produced an overall biopsy reduction rate of 15.8% (38 of 240 lesions [95% CI: 11.2, 20.9]), with a sensitivity of 92.1% (58 of 63 lesions [95% CI: 82.4, 97.4]; sensitivity was 97.2% [35 of 36 lesions] [95% CI: 82.7, 99.6] for invasive cancers). Results were similar for screening versus diagnostic examinations (P = .92 and .40, respectively). Sensitivity was higher for masses than for nonmass enhancements (NMEs) (100% vs 85.3%; P = .009). Applying the conservative ADC cutoff achieved a sensitivity of 95.2% (60 of 63 lesions [95% CI: 86.7, 99.0]), with a biopsy reduction rate of 10.4% (25 of 240 lesions [95% CI: 6.7, 14.5]). Repeated single-reader measurements showed good agreement with clinical ADCs (intraclass correlation coefficient, 0.72 [95% CI: 0.58, 0.81]). Conclusion This study validated the clinical use of ADC cutoffs to reduce MRI-prompted biopsies by up to 16%, with a suggested tradeoff of lowered sensitivity for in situ and microinvasive disease manifesting as NME. Clinical trial registration no. NCT02022579 © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Honda and Iima in this issue.

A mixed model approach to estimate the survivor average causal effect in cluster-randomized trials

In many medical studies, the outcome measure (such as quality of life, QOL) for some study participants becomes informatively truncated (censored, missing, or unobserved) due to death or other forms of dropout, creating a nonignorable missing data problem. In such cases, the use of a composite outcome or imputation methods that fill in unmeasurable QOL values for those who died rely on strong and untestable assumptions and may be conceptually unappealing to certain stakeholders when estimating a treatment effect. The survivor average causal effect (SACE) is an alternative causal estimand that surmounts some of these issues. While principal stratification has been applied to estimate the SACE in individually randomized trials, methods for estimating the SACE in cluster-randomized trials are currently limited. To address this gap, we develop a mixed model approach along with an expectation-maximization algorithm to estimate the SACE in cluster-randomized trials. We model the continuous outcome measure with a random intercept to account for intracluster correlations due to cluster-level randomization, and model the principal strata membership both with and without a random intercept. In simulations, we compare the performance of our approaches with an existing fixed-effects approach to illustrate the importance of accounting for clustering in cluster-randomized trials. The methodology is then illustrated using a cluster-randomized trial of telecare and assistive technology on health-related QOL in the elderly.

Examining the Relationship Between Multilevel Resilience Resources and Cardiovascular Disease Incidence, Overall and by Psychosocial Risks, Among Participants in the Jackson Heart Study, the Multi-Ethnic Study of Atherosclerosis, and the Mediators of Atherosclerosis in South Asians Living in America (MASALA) Study

We examined relationships between resilience resources (optimism, social support, and neighborhood social cohesion) and cardiovascular disease (CVD) incidence and assessed potential effect-measure modification by psychosocial risk factors (e.g., stress, depression) among adults without CVD in 3 cohort studies (2000-2018): the Jackson Heart Study, the Multi-Ethnic Study of Atherosclerosis, and the Mediators of Atherosclerosis in South Asians Living in America (MASALA) Study. We fitted adjusted Cox models accounting for within-neighborhood clustering while censoring at dropout or non-CVD death. We assessed for effect-measure modification by psychosocial risks. In secondary analyses, we estimated standardized risk ratios using inverse-probability-weighted Aalen-Johansen estimators to account for confounding, dropout, and competing risks (non-CVD deaths) and obtained 95% confidence intervals (CIs) using cluster bootstrapping. For high and medium (versus low) optimism (n = 6,243), adjusted hazard ratios (HRs) for incident CVD were 0.94 (95% CI: 0.78, 1.13) and 0.90 (95% CI: 0.75, 1.07), respectively. Corresponding HRs were 0.88 (95% CI: 0.74, 1.04) and 0.92 (95% CI: 0.79, 1.06) for social support (n = 7,729) and 1.10 (95% CI: 0.94, 1.29) and 0.99 (95% CI: 0.85, 1.16) for social cohesion (n = 7,557), respectively. Some psychosocial risks modified CVD HRs. Secondary analyses yielded similar findings. For optimism and social support, an inverse relationship was frequently most compatible with the data, but a positive relationship was also compatible. For neighborhood social cohesion, positive and null relationships were most compatible. Thus, specific resilience resources may be potential intervention targets, especially among certain subgroups.

Relationship between systolic blood pressure and mortality in older vs younger trauma patients - a retrospective multicentre observational study

BACKGROUND

The population of older trauma patients is increasing. Those patients have heterogeneous presentations and need senior-friendly triaging tools. Systolic blood pressure (SBP) is commonly used to assess injury severity, and some authors advocated adjusting SBP threshold for older patients. We aimed to describe and compare the relationship between mortality and SBP in older trauma patients and their younger counterparts.

METHODS

We included patients admitted to three level-I trauma centres and performed logistic regressions with age and SBP to obtain mortality curves. Multivariable Logistic regressions were performed to measure the association between age and mortality at different SBP ranges. Subgroup analyses were conducted for major trauma and severe traumatic brain injury admissions.

RESULTS

A total of 47,661 patients were included, among which 12.9% were aged 65-74 years and 27.3% were ≥ 75 years. Overall mortality rates were 3.9%, 8.1%, and 11.7% in the groups aged 16-64, 65-74, and ≥ 75 years, respectively. The relationship between prehospital SBP and mortality was nonlinear (U-shape), mortality increased with each 10 mmHg SBP decrement from 130 to 50 mmHg and each 10-mmHg increment from 150 to 220 mmHg across all age groups. Older patients were at higher odd for mortality in all ranges of SBP. The highest OR in patients aged 65-74 years was 3.67 [95% CI: 2.08-6.45] in the 90-99 mmHg SBP range and 7.92 [95% CI: 5.13-12.23] for those aged ≥ 75 years in the 100-109 mmHg SBP range.

CONCLUSION

The relationship between SBP and mortality is nonlinear, regardless of trauma severity and age. Older age was associated with a higher odd of mortality at all SBP points. Future triage tools should therefore consider SBP as a continuous rather than a dichotomized predictor.

Effect of an Integrated School Garden and Home Garden Intervention on Anemia Among School-Aged Children in Nepal: Evidence From a Cluster Randomised Controlled Trial

BACKGROUND

Integrated school and home garden interventions can improve health outcomes in low-income countries, but rigorous evidence remains scarce, particularly for school-aged children and to reduce anemia.

OBJECTIVE

We test if an integrated school and home garden intervention, implemented at pilot stage, improves hemoglobin levels among school children (aged 9-13 years) in a rural district in the mid-hills of Nepal.

METHODS

We use a cluster randomized controlled trial with 15 schools each in the control and treatment groups (n = 680 school children). To test if nutritional improvements translate into a reduction of anemia prevalence, hemoglobin data were collected 6 months after intervention support had ended. Using structural equation modeling, we estimate the direct and indirect effects of the treatment through several pathways, including nutritional knowledge, good food and hygiene practices, and dietary diversity.

RESULTS

The integrated school and home garden intervention did not lead to a direct significant reduction in anemia. Causal positive changes of the treatment on nutritional outcomes, although significant, are not strong enough to impact hemoglobin levels. The program improved hemoglobin levels indirectly for children below 12 by increasing the use of good food and hygiene practices at home. These practices are associated with higher hemoglobin levels, particularly for girls, young children, and in households where caregivers are literate.

CONCLUSIONS

Even integrated school and home garden interventions are not sufficient to reduce anemia among school children. Incorporating behavioral change components around food and hygiene practices into integrated garden interventions is important to unlocking their health impacts.

Comparative Performance of Contrast-enhanced Mammography, Abbreviated Breast MRI, and Standard Breast MRI for Breast Cancer Screening

Background Contrast-enhanced mammography (CEM) and abbreviated breast MRI (ABMRI) are emerging alternatives to standard MRI for supplemental breast cancer screening. Purpose To compare the diagnostic performance of CEM, ABMRI, and standard MRI. Materials and Methods This single-institution, prospective, blinded reader study included female participants referred for breast MRI from January 2018 to June 2021. CEM was performed within 14 days of standard MRI; ABMRI was produced from standard MRI images. Two readers independently interpreted each CEM and ABMRI after a washout period. Examination-level performance metrics calculated were recall rate, cancer detection, and false-positive biopsy recommendation rates per 1000 examinations and sensitivity, specificity, and positive predictive value of biopsy recommendation. Bootstrap and permutation tests were used to calculate 95% CIs and compare modalities. Results Evaluated were 492 paired CEM and ABMRI interpretations from 246 participants (median age, 51 years; IQR, 43-61 years). On 49 MRI scans with lesions recommended for biopsy, nine lesions showed malignant pathology. No differences in ABMRI and standard MRI performance were identified. Compared with standard MRI, CEM demonstrated significantly lower recall rate (14.0% vs 22.8%; difference, -8.7%; 95% CI: -14.0, -3.5), lower false-positive biopsy recommendation rate per 1000 examinations (65.0 vs 162.6; difference, -97.6; 95% CI: -146.3, -50.8), and higher specificity (87.8% vs 80.2%; difference, 7.6%; 95% CI: 2.3, 13.1). Compared with standard MRI, CEM had significantly lower cancer detection rate (22.4 vs 36.6; difference, -14.2; 95% CI: -28.5, -2.0) and sensitivity (61.1% vs 100%; difference, -38.9%; 95% CI: -66.7, -12.5). The performance differences between CEM and ABMRI were similar to those observed between CEM and standard MRI. Conclusion ABMRI had comparable performance to standard MRI and may support more efficient MRI screening. CEM had lower recall and higher specificity compared with standard MRI or ABMRI, offset by lower cancer detection rate and sensitivity compared with standard MRI. These trade-offs warrant further consideration of patient population characteristics before widespread screening with CEM. Clinical trial registration no. NCT03517813 © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Chang in this issue.

Hypertension and one-year risk of all-cause mortality among women with treated HIV in the United States

OBJECTIVE

Hypertension is a critical cause of cardiovascular disease, and women with HIV have a higher prevalence of hypertension than women without HIV. The relationship between hypertension and mortality has not been well characterized in women with treated HIV. Here, we estimate the effect of hypertension on 1-year risk of all-cause mortality among women with HIV on antiretroviral therapy (ART) in the United States.

DESIGN

An analysis of multicenter, observational cohort data from the Women's Interagency HIV Study (WIHS) collected between 1995 and 2019.

METHODS

We included women with HIV who reported ever using ART. We used parametric g-computation to estimate the effect of hypertension (SBP ≥140  mmHg, DBP ≥90 mmHg, or use of hypertensive medication) on all-cause mortality within 1 year of a WIHS visit.

RESULTS

Among 2929 unique women, we included 57 034 visits with a median age of 45 (interquartile range: 39, 52) years. Women had hypertension at 34.5% of visits, and 641 deaths occurred within 1 year of a study visit. Comparing women at visits with hypertension to women at visits without hypertension, the standardized 1-year risk ratio for mortality was 1.16 [95% confidence interval (95% CI): 1.01-1.33]. The risk ratios were higher in Hispanic (risk ratio: 1.23, 95% CI: 0.86-1.77) and non-Hispanic black women (risk ratio: 1.19, 95% CI: 1.04-1.37) and lower in non-Hispanic white women (risk ratio: 0.93, 95% CI: 0.58-1.48).

CONCLUSION

Among women with treated HIV, those with hypertension, compared with those without, had an increased 1-year risk of all-cause mortality.

A scoping review of statistical methods for trial-based economic evaluations: The current state of play

The statistical quality of trial-based economic evaluations is often suboptimal, while a comprehensive overview of available statistical methods is lacking. Therefore, this review summarized and critically appraised available statistical methods for trial-based economic evaluations. A literature search was performed to identify studies on statistical methods for dealing with baseline imbalances, skewed costs and/or effects, correlated costs and effects, clustered data, longitudinal data, missing data and censoring in trial-based economic evaluations. Data was extracted on the statistical methods described, their advantages, disadvantages, relative performance and recommendations of the study. Sixty-eight studies were included. Of them, 27 (40%) assessed methods for baseline imbalances, 39 (57%) assessed methods for skewed costs and/or effects, 27 (40%) assessed methods for correlated costs and effects, 18 (26%) assessed methods for clustered data, 7 (10%) assessed methods for longitudinal data, 26 (38%) assessed methods for missing data and 10 (15%) assessed methods for censoring. All identified methods were narratively described. This review provides a comprehensive overview of available statistical methods for dealing with the most common statistical complexities in trial-based economic evaluations. Herewith, it can provide valuable input for researchers when deciding which statistical methods to use in a trial-based economic evaluation.

Management and Prognosis of Cardiac Metastatic Merkel Cell Carcinoma: A Case-Control Study and Literature Review

Merkel cell carcinoma (MCC), an aggressive neuroendocrine skin cancer, has a high rate (20%) of distant metastasis. Within a prospective registry of 582 patients with metastatic MCC (mMCC) diagnosed between 2003-2021, we identified 9 (1.5%) patients who developed cardiac metastatic MCC (mMCC). We compared overall survival (OS) between patients with cardiac and non-cardiac metastases in a matched case-control study. Cardiac metastasis was a late event (median 925 days from initial MCC diagnosis). The right heart was predominantly involved (8 of 9; 89%). Among 7 patients treated with immunotherapy, 6 achieved a complete or partial response of the cardiac lesion. Among these 6 responders, 5 received concurrent cardiac radiotherapy (median 20 Gray) with immunotherapy; 4 of 5 did not have local disease progression or recurrence in the treated cardiac lesion. One-year OS was 44%, which was not significantly different from non-cardiac mMCC patients (45%, p = 0.96). Though it occurs relatively late in the disease course, cardiac mMCC responded to immunotherapy and/or radiotherapy and was not associated with worse prognosis compared to mMCC at other anatomic sites. These results are timely as cardiac mMCC may be increasingly encountered in the era of immunotherapy as patients with metastatic MCC live longer.

Can the Academic and Experiential Study of Flourishing Improve Flourishing in College Students? A Multi-University Study

Objectives

Significant concerns have been raised about the "mental health crisis" on college campuses, with attention turning to what colleges can do beyond counseling services to address students' mental health and well-being. We examined whether primarily first-year (89.1%) undergraduate students (n=651) who enrolled in the Art and Science of Human Flourishing (ASHF), a novel academic and experiential for-credit elective course on human flourishing, would demonstrate improved mental health and strengthen skills, perspectives, and behaviors associated with flourishing relative to students who did not enroll in this course.

Methods

In a two-wave, multi-site, propensity-score matched controlled trial (ASHF n=217, Control n=434; N=651), we used hierarchal linear models and false discovery rate corrected doubly robust estimates to evaluate the impact of the ASHF on attention and social-emotional skill development, flourishing perspectives, mental health, health, and risk behavior outcomes.

Results

ASHF participants reported significantly improved mental health (i.e., reduced depression) and flourishing, improvements on multiple attention and social-emotional skills (e.g., attention function, self-compassion), and increases in prosocial attitudes (empathic concern, shared humanity; Cohen's ds= 0.18-0.46) compared to controls. There was no evidence for ASHF course impacts on health or risk behaviors, raising the possibility that these outcomes take more time to change.

Conclusions

This research provides initial evidence that the ASHF course may be a promising curricular approach to reduce and potentially prevent poor mental health while promoting flourishing in college students. Continued research is needed to confirm these conclusions.

Analyzing nested experimental designs—A user-friendly resampling method to determine experimental significance

While hierarchical experimental designs are near-ubiquitous in neuroscience and biomedical research, researchers often do not take the structure of their datasets into account while performing statistical hypothesis tests. Resampling-based methods are a flexible strategy for performing these analyses but are difficult due to the lack of open-source software to automate test construction and execution. To address this, we present Hierarch, a Python package to perform hypothesis tests and compute confidence intervals on hierarchical experimental designs. Using a combination of permutation resampling and bootstrap aggregation, Hierarch can be used to perform hypothesis tests that maintain nominal Type I error rates and generate confidence intervals that maintain the nominal coverage probability without making distributional assumptions about the dataset of interest. Hierarch makes use of the Numba JIT compiler to reduce p-value computation times to under one second for typical datasets in biomedical research. Hierarch also enables researchers to construct user-defined resampling plans that take advantage of Hierarch’s Numba-accelerated functions.

An important step in analyzing experimental data is quantifying uncertainty in the experimenter’s conclusions. One mechanism for doing so is by using a statistical hypothesis test, which allows the experimenter to control what percentage of the time they make erroneous conclusions over the course of their career. Biological experimental designs often have hierarchical data-gathering schemes that traditional hypothesis tests are not well-suited for (for example, an experimenter may make measurements of several tissue samples that were collected from subjects who were given a treatment). While traditional tests can be adapted to hierarchical experimental designs, we propose a simple resampling-based hypothesis test that applies to a variety of experimental designs while maintaining control over error rate. In this manuscript, we describe Hierarch, the Python package that enables users to carry out this test and validate it under several conditions.

Correlation of Optical Coherence Tomography Angiography Characteristics with Visual Function to Define Vision-Threatening Diabetic Macular Ischemia

The thresholds of macular microvasculature parameters associated with mild visual impairment in diabetic macular ischemia (DMI) patients are unclear. Therefore, this prospective observational study is aimed at demonstrating the optical coherence tomography angiography parameters that best correlate with mild visual impairment (<70 Early Treatment Diabetic Retinopathy Study (ETDRS) letters, Snellen equivalent 20/40) in DMI. The study was completed at the Moorfields Eye Hospital from December 2019 to August 2021. A total of 123 eyes of 87 patients with stable-treated proliferative diabetic retinopathy following panretinal photocoagulation were recruited. DMI was defined as an irregular foveal avascular zone (FAZ) area ≥ 0.5 mm2 or a smaller FAZ area with parafoveal capillary dropout in at least one quadrant. The analysis showed that the whole image deep vascular complex vessel density (DVC VD) in the 3 × 3 mm area had the best discriminatory ability to identify participants with mild visual impairment at 41.9% (area under the curve = 0.77, sensitivity 94%, specificity 54%, likelihood ratio [LR] = 2.04), and the FAZ area had the greatest post-test LR = 4.21 at 0.64 mm2. The 3 × 3 mm whole image DVC VD and FAZ area cutoffs are useful for screening vision-threatening DMI, but DVC VD has low specificity.

Evaluation of the type I error rate when using parametric bootstrap analysis of a cluster randomized controlled trial with binary outcomes and a small number of clusters

BACKGROUND

Cluster randomized controlled trials (cRCTs) are increasingly used but must be analyzed carefully. We conducted a simulation study to evaluate the validity of a parametric bootstrap (PB) approach with respect to the empirical type I error rate for a cRCT with binary outcomes and a small number of clusters.

METHODS

We simulated a case study with a binary (0/1) outcome, four clusters, and 100 subjects per cluster. To compare the validity of the test with respect to error rate, we simulated the same experiment with K=10, 20, and 30 clusters, each with 2,000 simulated datasets. To test the null hypothesis, we used a generalized linear mixed model including a random intercept for clusters and obtained p-values based on likelihood ratio tests (LRTs) using the parametric bootstrap method as implemented in the R package "pbkrtest".

RESULTS

The PB test produced error rates of 9.1%, 5.5%, 4.9%, and 5.0% on average across all ICC values for K=4, K=10, K=20, and K=30, respectively. The error rates were higher, ranging from 9.1% to 36.5% for K=4, in the models with singular fits (i.e., ignoring clustering) because the ICC was estimated to be zero.

CONCLUSION

Using the parametric bootstrap for cRCTs with a small number of clusters results in inflated error rates and is not valid.

Selective exposure in different political information environments - How media fragmentation and polarization shape congruent news use

Previous research posits that individual predispositions play an essential role in explaining patterns of selective exposure to political information. Yet the contextual factors in the political information environment have received far less attention. Using a cross-national and quasi-experimental design, this article is one of the first to investigate how political information environments shape selective exposure. We rely on a unique two-wave online survey quasi-experiment in five countries (Switzerland, Denmark, Italy, Poland and the United States) with 4349 participants to test the propositions that (a) the level of polarization and fragmentation in information environments and (b) the type of media source used affect selective exposure. Our results reveal that selective exposure is slightly more frequent among regular social media users but is less common among users of TV, radio and newspapers; crucially, it is more common in information environments that are highly fragmented and polarized. Nevertheless, news users from less fragmented-polarized media landscapes show one surprising yet intriguing behaviour: in a quasi-experimentally manipulated setting with more opportunities to self-select than they may be accustomed to, their coping strategy is to pick larger amounts of congruent news stories. All our findings imply that contextual factors play a crucial role in moderating individuals' tendency to select information that aligns with their political views.

Antimicrobial Stewardship with and without Infectious Diseases Specialist Services to Improve Quality-of-Care in Secondary and Tertiary Care Hospitals in Germany: Study Protocol of the ID ROLL OUT Study

BACKGROUND

Antimicrobial stewardship (AMS) programs aim to secure the rational prescription of antibiotics through implementing department- or hospital-level activities. Infectious disease (ID) specialists improve the quality of care and outcomes in infection patients predominantly by individual consultations and patient-level interventions. While hospital AMS programs are established to various extents in Germany, ID specialist services are rarely available in this country. In the ID ROLL OUT study, we will implement and evaluate hospital-level AMS tools with and without ID specialist services in secondary and tertiary care hospitals. We aim to identify means to comprehensively and sustainably improve the quality of care of patients with infectious diseases.

METHODS

This project is a clustered, two-armed intervention study, which will be conducted in ten secondary and tertiary (non-university) care hospitals in Germany. The intervention groups are stratified by key characteristics of the hospitals. We will compare two interventional strategies: implementation of AMS teams and implementation of AMS teams combined with the activities of ID specialists (AMS + IDS).

PLANNED OUTCOMES

The primary outcome is the quality of care as measured in changes in a Staphylococcus aureus bacteremia (SAB) score (as an indicator of difficult-to-treat infections) and a community-acquired pneumonia (CAP) score (as an indicator of common infections) compared to a baseline pre-interventional period. Our secondary outcomes comprise patient- and hospital-level outcomes, such as the quality and frequency of antibiotic treatment, in-hospital mortality, duration of hospitalization, and C. difficile incidence (associated diarrhea episodes). The study may provide urgently needed key information for the aspired advancement of ID care in Germany.

TRIAL REGISTRATION

DRKS00023710 (registered on 9th April 2021).

Using cluster-robust standard errors when analyzing group-randomized trials with few clusters

Accounting for dependent observations in cluster-randomized trials (CRTs) using nested data is necessary in order to avoid misestimated standard errors resulting in questionable inferential statistics. Cluster-robust standard errors (CRSEs) are often used to address this issue. However, CRSEs are still well-known to underestimate standard errors for group-level variables when the number of clusters is low (e.g., < 50) and with CRTs, a small number of clusters, due to logistical or financial considerations, is the norm rather than the exception. Using a simulation with various conditions, we investigate the use of a small sample correction (i.e., CR2 estimator) proposed by Bell and McCaffrey (2002) together with empirically derived degrees of freedom estimates (dof_BM). Findings indicate that even with as few as 10 clusters, the CR2 estimator used with dof_BM yields generally unbiased results with acceptable type I error and coverage rates. Results show that coverage and type I error rates can be largely influenced by the choice of dof, not just the standard error adjustments. An applied example is provided together with R syntax to conduct the analysis. To facilitate the use of different CRSEs, a free graphical, menu-driven SPSS add-on to compute the various cluster-robust variance estimates can be downloaded from https://github.com/flh3/CR2/tree/master/SPSS .

Bootstrap Confidence Intervals for Multilevel Standardized Effect Size

Although many methodologists and professional organizations have urged applied researchers to compute and report effect size measures accompanying tests of statistical significance, discussions on obtaining confidence intervals (CIs) for effect size with clustered/multilevel data have been scarce. In this paper, I explore the bootstrap as a viable and accessible alternative for obtaining CIs for multilevel standardized mean difference effect size for cluster-randomized trials. A simulation was carried out to compare 17 analytic and bootstrap procedures for constructing CIs for multilevel effect size, in terms of empirical coverage rate and width, for both normal and nonnormal data. Results showed that, overall, the residual bootstrap with studentized CI had the best coverage rates (94.75% on average), whereas the residual bootstrap with basic CI had better coverage in small samples. These two procedures for constructing CIs showed better coverage than using analytic methods for both normal and nonnormal data. In addition, I provide an illustrative example showing how bootstrap CIs for multilevel effect size can be easily obtained using the statistical software R and the R package bootmlm. I strongly encourage applied researchers to report CIs to adequately convey the uncertainty of their effect size estimates.

Tutorial on Biostatistics: Receiver-Operating Characteristic (ROC) Analysis for Correlated Eye Data

Purpose: To demonstrate methods for receiver-operating characteristic (ROC) analysis of correlated eye data.Methods: We applied the Obuchowski's nonparametric approach and cluster bootstrap for estimating and comparing the area under ROC curve (AUC) between different sets of predictors to three datasets with varying inter-eye correlation.Results: In an optic neuritis (ON) study of 152 eyes (80 patients), the AUC of optical coherence tomography retinal nerve fiber layer thickness for diagnosing ON (inter-eye kappa = 0.13) was 0.71 [95% confidence interval (95% CI): 0.622, 0.792] from the naïve approach without accounting for inter-eye correlation was narrower than from nonparametric (95% CI: 0.613, 0.801) or cluster bootstrap (95% CI: 0.614, 0.797) approaches. In an analysis of 198 eyes (135 patients), the baseline Age-related Eye disease Study scale predicted 5-year incidence of advanced age-related macular degeneration (inter-eye kappa = 0.23) with AUC of 0.72. The 95% CI from the naïve approach was slightly narrower (0.645, 0.794) than from the nonparametric (0.641, 0.797) or cluster bootstrap (0.641, 0.793) approaches. In an analysis of 1542 eyes (771 infants), birthweight and gestational age predicted treatment-requiring retinopathy of prematurity (inter-eye kappa = 0.98) with AUC of 0.80. Furthermore, the 95% CI from the naïve approach was narrower (0.769, 0.835) than from the nonparametric (0.755, 0.848) or cluster bootstrap (0.755, 0.845) approaches. 95% CIs for AUC differences between different models were narrower in the naïve approach than the nonparametric or cluster bootstrap approaches.Conclusion: In ROC analysis of correlated eye data, ignoring inter-eye correlation leads to narrower 95% CI with underestimation dependent on magnitude of inter-eye correlation. Nonparametric and cluster bootstrap approaches properly account for inter-eye correlation.

Adult-like neural representation of species-specific songs in the auditory forebrain of zebra finch nestlings

Encoding of conspecific signals during development can reinforce species barriers as well as set the stage for learning and production of species-typical vocalizations. In altricial songbirds, the development of the auditory system is not complete at hatching, so it is unknown the degree to which recently hatched young can process auditory signals like birdsong. We measured in vivo extracellular responses to song stimuli in a zebra finch (Taeniopygia guttata) secondary auditory forebrain region, the caudomedial nidopallium (NCM). We recorded from three age groups between 13 days post-hatch and adult to identify possible shifts in stimulus encoding that occur before the opening of the sensitive period of song motor learning. We did not find differences in putative cell type composition, firing rate, response strength, and selectivity across ages. Across ages narrow-spiking units had higher firing rates, response strength, accuracy, and trial-by-trial reliability along with lower selectivity than broad-spiking units. In addition, we showed that stimulus-specific adaptation, a characteristic of adult NCM, was also present in nestlings and fledglings. These results indicate that most features of secondary auditory processing are already adult-like shortly after hatching. Furthermore, we showed that selectivity for species-specific stimuli is similar across all ages, with the greatest fidelity in temporal coding in response to conspecific song and domesticated Bengalese finch song, and reduced fidelity in response to owl finch song, a more ecologically relevant heterospecific, and white noise. Our study provides the first evidence that the electrophysiological properties of higher-order auditory neurons are already mature in nestling songbirds.

Quantitative T2 -mapping magnetic resonance imaging for assessment of muscle motor unit recruitment patterns

INTRODUCTION

In this study, we aimed to determine whether muscle transverse relaxation time (T₂ ) magnetic resonance (MR) mapping results correlate with motor unit loss, as defined by motor unit recruitment patterns on electromyography (EMG).

METHODS

EMG and 3-Tesla MRI exams were acquired no more than 31 days apart in subjects referred for peripheral nerve MRI. Two musculoskeletal radiologists qualitatively graded T₂ -weighted, fat-suppressed sequences for severity of muscle edema-like patterns and manually placed regions of interest within muscles to obtain T₂ values from T₂ -mapping sequences. Concordance was calculated between qualitative and quantitative MR grades and EMG recruitment categories (none, discrete, decreased) as well as interobserver agreement for both MR grades.

RESULTS

Thirty-four muscles (21 abnormal, 13 control) were assessed in 13 subjects (5 females and 8 males; mean age, 46 years) with 14 EMG-MRI pairs. T₂ -relaxation times were significantly (P < .001) increased in all EMG recruitment categories compared with control muscles. T₂ differences were not significant between EMG grades of motor unit recruitment (P = .151-.702). T₂ and EMG score concordance was acceptable (Harrell's concordance index [c index]: rater A, 0.71; 95% confidence interval [CI], 0.51-0.87; rater B, 0.77; 95% CI, 0.57-0.91). Qualitative MRI and EMG score concordance was poor to acceptable (c index: rater A, 0.60; 95% CI, 0.50-0.79; rater B, 0.72; 95% CI, 0.55-0.89). T₂ values had moderate-to-substantial ability to distinguish between absent vs incomplete (ie, decreased or discrete) motor unit recruitment (c index: rater A, 0.78; 95% CI, 0.50-1.00; rater B, 0.86; 95% CI, 0.57-1.00).

DISCUSSION

Quantitative T₂ MR muscle mapping is a promising tool for noninvasive evaluation of the degree of motor unit recruitment loss.

Care partner-assisted intervention to improve oral health for older adults with cognitive impairment: A feasibility study

BACKGROUND AND OBJECTIVES

Older adults with cognitive impairment often experience poor oral health outcomes due to inadequate oral hygiene practices. This pilot study aimed to evaluate the feasibility of a care partner-assisted intervention to improve the oral hygiene of community-dwelling older adults with cognitive impairment.

MATERIAL AND METHODS

The 6-month intervention included 25 older adults with mild dementia or mild cognitive impairment, who were randomly assigned to Treatment Group 1 or Treatment Group 2. Treatment Group 1 (n = 7) received an educational booklet. Treatment Group 2 (n = 18) received a booklet, a tailored care plan for the participants with cognitive impairment and the care partner received four coaching sessions to learn to facilitate good oral hygiene. Both groups received electric toothbrushes. The study consisted of a 3-month active intervention and 3-month maintenance phase. The outcomes of gingival index, plaque index and overall oral health status based on the Oral Health Assessment Tool were measured at baseline, 3 months (end of active intervention) and 6 months of the study.

RESULTS

This study had very low dropout rate. Participants' oral hygiene improved in this study. In comparison to Treatment Group 1, participants in Treatment Group 2 had a greater reduction in plaque level and gingival inflammation, and greater improvement in overall oral health status.

CONCLUSION

This study demonstrates the feasibility of this intervention designed to improve the oral health of persons with cognitive impairment and it lays the foundation for using this protocol in a future large randomised clinical trial.

Application of the hierarchical bootstrap to multi-level data in neuroscience

A common feature in many neuroscience datasets is the presence of hierarchical data structures, most commonly recording the activity of multiple neurons in multiple animals across multiple trials. Accordingly, the measurements constituting the dataset are not independent, even though the traditional statistical analyses often applied in such cases (e.g., Student's t-test) treat them as such. The hierarchical bootstrap has been shown to be an effective tool to accurately analyze such data and while it has been used extensively in the statistical literature, its use is not widespread in neuroscience - despite the ubiquity of hierarchical datasets. In this paper, we illustrate the intuitiveness and utility of this approach to analyze hierarchically nested datasets. We use simulated neural data to show that traditional statistical tests can result in a false positive rate of over 45%, even if the Type-I error rate is set at 5%. While summarizing data across non-independent points (or lower levels) can potentially fix this problem, this approach greatly reduces the statistical power of the analysis. The hierarchical bootstrap, when applied sequentially over the levels of the hierarchical structure, keeps the Type-I error rate within the intended bound and retains more statistical power than summarizing methods. We conclude by demonstrating the effectiveness of the method in two real-world examples, first analyzing singing data in male Bengalese finches (Lonchura striata var. domestica) and second quantifying changes in behavior under optogenetic control in flies (Drosophila melanogaster).

Comparison of model- and design-based approaches to detect the treatment effect and covariate by treatment interactions in three-level models for multisite cluster-randomized trials

In this study, we evaluated the estimation of three important parameters for data collected in a multisite cluster-randomized trial (MS-CRT): the treatment effect, and the treatment by covariate interactions at Levels 1 and 2. The Level 1 and Level 2 interaction parameters are the coefficients for the products of the treatment indicator, with the covariate centered on its Level 2 expected value and with the Level 2 expected value centered on its Level 3 expected value, respectively. A comparison of a model-based approach to design-based approaches was performed using simulation studies. The results showed that both approaches produced similar treatment effect estimates and interaction estimates at Level 1, as well as similar Type I error rates and statistical power. However, the estimate of the Level 2 interaction coefficient for the product of the treatment indicator and an arithmetic mean of the Level 1 covariate was severely biased in most conditions. Therefore, applied researchers should be cautious when using arithmetic means to form a treatment by covariate interaction at Level 2 in MS-CRT data.

Small Sample Methods for Multilevel Modeling: A Colloquial Elucidation of REML and the Kenward-Roger Correction

Studies on small sample properties of multilevel models have become increasingly prominent in the methodological literature in response to the frequency with which small sample data appear in empirical studies. Simulation results generally recommend that empirical researchers employ restricted maximum likelihood estimation (REML) with a Kenward-Roger correction with small samples in frequentist contexts to minimize small sample bias in estimation and to prevent inflation of Type-I error rates. However, simulation studies focus on recommendations for best practice, and there is little to no explanation of why traditional maximum likelihood (ML) breaks down with smaller samples, what differentiates REML from ML, or how the Kenward-Roger correction remedies lingering small sample issues. Due to the complexity of these methods, most extant descriptions are highly mathematical and are intended to prove that the methods improve small sample performance as intended. Thus, empirical researchers have documentation that these methods are advantageous but still lack resources to help understand what the methods actually do and why they are needed. This tutorial explains why ML falters with small samples, how REML circumvents some issues, and how Kenward-Roger works. We do so without equations or derivations to support more widespread understanding and use of these valuable methods.