Estimating the reliability of repeatedly measured endpoints based on linear mixed-effects models. A tutorial

Meaningful changes in motor function in Duchenne muscular dystrophy (DMD): A multi-center study

Evaluations of treatment efficacy in Duchenne muscular dystrophy (DMD), a rare genetic disease that results in progressive muscle wasting, require an understanding of the 'meaningfulness' of changes in functional measures. We estimated the minimal detectable change (MDC) for selected motor function measures in ambulatory DMD, i.e., the minimal degree of measured change needed to be confident that true underlying change has occurred rather than transient variation or measurement error. MDC estimates were compared across multiple data sources, representing >1000 DMD patients in clinical trials and real-world clinical practice settings. Included patients were ambulatory, aged ≥4 to <18 years and receiving steroids. Minimal clinically important differences (MCIDs) for worsening were also estimated. Estimated MDC thresholds for >80% confidence in true change were 2.8 units for the North Star Ambulatory Assessment (NSAA) total score, 1.3 seconds for the 4-stair climb (4SC) completion time, 0.36 stairs/second for 4SC velocity and 36.3 meters for the 6-minute walk distance (6MWD). MDC estimates were similar across clinical trial and real-world data sources, and tended to be slightly larger than MCIDs for these measures. The identified thresholds can be used to inform endpoint definitions, or as benchmarks for monitoring individual changes in motor function in ambulatory DMD.

Minimal detectable change of gait and balance measures in older neurological patients: estimating the standard error of the measurement from before-after rehabilitation data thanks to the linear mixed-effects models

BACKGROUND

Tracking gait and balance impairment in time is paramount in the care of older neurological patients. The Minimal Detectable Change (MDC), built upon the Standard Error of the Measurement (SEM), is the smallest modification of a measure exceeding the measurement error. Here, a novel method based on linear mixed-effects models (LMMs) is applied to estimate the standard error of the measurement from data collected before and after rehabilitation and calculate the MDC of gait and balance measures.

METHODS

One hundred nine older adults with a gait impairment due to neurological disease (66 stroke patients) completed two assessment sessions before and after inpatient rehabilitation. In each session, two trials of the 10-meter walking test and the Timed Up and Go (TUG) test, instrumented with inertial sensors, have been collected. The 95% MDC was calculated for the gait speed, TUG test duration (TTD) and other measures from the TUG test, including the angular velocity peak (ωpeak) in the TUG test's turning phase. Random intercepts and slopes LMMs with sessions as fixed effects were used to estimate SEM. LMMs assumptions (residuals normality and homoscedasticity) were checked, and the predictor variable ln-transformed if needed.

RESULTS

The MDC of gait speed was 0.13 m/s. The TTD MDC, ln-transformed and then expressed as a percentage of the baseline value to meet LMMs' assumptions, was 15%, i.e. TTD should be < 85% of the baseline value to conclude the patient's improvement. ωpeak MDC, also ln-transformed and expressed as the baseline percentage change, was 25%.

CONCLUSIONS

LMMs allowed calculating the MDC of gait and balance measures even if the test-retest steady-state assumption did not hold. The MDC of gait speed, TTD and ωpeak from the TUG test with an inertial sensor have been provided. These indices allow monitoring of the gait and balance impairment, which is central for patients with an increased falling risk, such as neurological old persons.

TRIAL REGISTRATION

NA.

Reliability and accuracy of the torque applied to osteosynthesis screws by maxillofacial surgeons and residents

Applying the right torque to osteosynthesis screws is important for undisturbed bone healing. This study aimed to compare test-retest and intra-individual reliabilities of the torque applied to 1.5 mm and 2.0 mm osteosynthesis screws by residents and oral and maxillofacial surgeons (OMF-surgeons), to define the reference torque intervals, and to compare reference torque interval compliances. Five experienced OMF-surgeons and 20 residents, 5 of each 4 residency years, were included. Each participant inserted six 1.5 × 4 mm and six 2.0 × 6 mm screws into a preclinical model at two test moments 2 weeks apart (T1 and T2). Participants were blinded for the applied torque. Descriptive statistics, reference intervals, and intra-class correlation coefficients (ICC) were calculated. The OMF-surgeons complied more to the reference intervals (1.5 mm screws: 95% and 2.0 mm screws: 100%) than the residents (82% and 90%, respectively; P = 0.009 and P = 0.007) with the ICCs ranging between 0.85-0.95 and 0.45-0.97, respectively. The residents' accuracy and reliability were inadequate regarding the 1.5 mm screws but both measures improved at T2 for both screw types compared to T1, indicating a learning effect. Training residents and/or verifying the applied torque by experienced OMF-surgeons remains necessary to achieve high accuracy and reliability, particularly for 1.5 mm screws.

Mobile sonouroflowmetry using voiding sound and volume

Uroflowmetry (UF) is a common clinic-based non-invasive test to diagnose Lower Urinary Tract Dysfunction (LUTD). Accurate home-based uroflowmetry methods are needed to conveniently conduct repeated uroflowmetries when patients are physiologically ready to urinate. To this end, we propose and evaluate a novel mobile sonouroflowmetry (SUF) method that estimates the urinary flow rate from a sound signal recorded using a mobile phone. By linearly mapping the total sound energy to the total voided volume, the sound energy curve is transformed to a flow rate curve allowing the estimation of the flow rate over time. An evaluation using data from 44 healthy young men showed high similarity between the UF and SUF flow rates with a mixed-effects model correlation coefficient of 0.993 and a mean root mean square error of 2.37 ml/s. Maximum flow rates were estimated with an average absolute error of 2.41 ml/s. Future work on mobile uroflowmetry can use these results as an initial benchmark for flow rate estimation accuracy.

Pulse oximeter plethysmograph waveform and automated oscillometric sphygmomanometer for ankle-brachial index measurement

OBJECTIVES

There are limited non-invasive methods to assess lower extremity arterial injuries in the emergency department (ED) and pre-hospital setting. The ankle-brachial index (ABI) requires careful auscultation by Doppler, an approach made difficult in noisy environments. We sought to determine the agreement of the ABI measured using the pulse oximeter plethysmograph waveform (Pleth) with auscultation by Doppler in a controlled setting. A secondary outcome sought to examine the agreement of ABI by automated oscillometric sphygmomanometer (AOS) with Doppler.

METHODS

We measured blood pressure in the right upper and lower extremities of healthy volunteers using: (1) Doppler and manual sphygmomanometer; (2) Pleth and manual sphygmomanometer; and (3) AOS. The Bland-Altman approach to assessing agreement between methods was used comparing mean differences between ABI pairs to their means for Doppler versus Pleth and Doppler versus AOS. The intraclass correlation coefficient (ICC) from mixed effects models examined intra- and inter-rater reliability.

RESULTS

Among 100 participants with normal ABI the mean ABI (95%CI) were Doppler 1.11 (0.90-1.33), Pleth 1.10 (0.91-1.30), and AOS 1.10 (0.90-1.30). The ABI difference (95% CI for limits of agreement) were 0.01 (-0.20,0.18) for Doppler-Pleth and 0.02 (-0.26, 0.22) for Doppler-AOS. The ICC for the Doppler-Pleth comparison (ICC = 0.56, 95% CI 0.47-0.63) was greater than for the Doppler-AOS (ICC = 0.32, 95% CI 0.19-0.43).

CONCLUSIONS

The ABI measured using the Pleth has a high level of agreement with measurement by Doppler. The AOS and Doppler have good agreement with greater measurement variability. Pleth and AOS may be reasonable alternatives to Doppler for ABI.

Measuring Goal Setting in School-Aged Children: Studying the Effects of Demographic Variables in Regression-Based Norms

The aim of the study was to establish demographically representative norms for tasks measuring goal setting, and more specifically planning and reasoning in children. Three tasks were administered to n = 195 Ukrainian children aged 5.10 to 14.5 years old: the Spatial Working Memory (SWM), the Stockings of Cambridge (SOC) test, and the Naglieri Nonverbal Ability Test (NNAT). Main outcome per test was accuracy: i.e., the total number correct for the SOC and NNAT, and the total amount of incorrect responses for the SWM. Correlations among accuracy measures varied from − 0.51 to 0.60, indicating these tasks measure related but at the same time unique constructs. Higher age was associated with more accurate test performances on all outcome measures. On the NNAT, we found a curvilinear association between age and accuracy, indicating that younger children’s NNAT accuracy scores increased more with age compared with older children. We found a cubic age effect on accuracy for the SWM and SOC: i.e., test scores were relatively stable at younger and older ages, with a curvilinear increase in test scores in the other age groups. Demographically corrected norms were calculated and presented per test. These indicated that sex was not associated with accuracy scores on any of the tests. Last, a higher level of parental education (LPE) was associated with higher accuracy scores, but only on the NNAT. We conclude that demographic variables in norm analyses enhance insight in the scores and allow for application in clinical settings and research.

Measuring Visual, Spatial, and Visual Spatial Short-Term Memory in Schoolchildren: Studying the Influence of Demographic Factors and Regression-Based Normative Data

The study aims to establish demographically corrected norms for three computerized tasks measuring different aspects of visual short-term memory (VSTM) in Ukrainian schoolchildren. These tasks measure respectively visual STM (the Pattern Recognition Memory (PRM) test), spatial STM (the Spatial Span (SSP) task), and visual spatial STM (the Paired Associates Learning (PAL) task). All tasks were administered to n = 186 children aged 5.10 years to 14.5 years old to evaluate the influence of demographic variables. Relevant demographic factors that influence task scores (VSTM), i.e., age and level of parental education, are identified and in keeping with the current literature. No sex differences were found. Based on these data, regression-based, demographically corrected norms were established per task. This approach to constructing norms differs from how (worldwide) PRM, SSP, and/or PAL norms have been constructed traditionally. In the latter approach, norms are calculated for each age group separately and without correcting for level of parental education, whereas in the regression-based normative method, multiple regression models are used to compute the expected test scores of an individual (rather than the subgroup means that are used in the traditional approach). Consequently, the regression-based norms for the PRM, SSP, and PAL presented in this paper are individualized, taking into account the unique characteristics of the individual that is tested on these tasks. Last, the confidence intervals of the PRM scores of the Ukrainian schoolchildren and the western norm group largely overlapped, except for the youngest age group, which adds to the literature about cultural effects on cognition.

Lessons learned from IDeAl - 33 recommendations from the IDeAl-net about design and analysis of small population clinical trials

BACKGROUND

IDeAl (Integrated designs and analysis of small population clinical trials) is an EU funded project developing new statistical design and analysis methodologies for clinical trials in small population groups. Here we provide an overview of IDeAl findings and give recommendations to applied researchers.

METHOD

The description of the findings is broken down by the nine scientific IDeAl work packages and summarizes results from the project's more than 60 publications to date in peer reviewed journals. In addition, we applied text mining to evaluate the publications and the IDeAl work packages' output in relation to the design and analysis terms derived from in the IRDiRC task force report on small population clinical trials.

RESULTS

The results are summarized, describing the developments from an applied viewpoint. The main result presented here are 33 practical recommendations drawn from the work, giving researchers a comprehensive guidance to the improved methodology. In particular, the findings will help design and analyse efficient clinical trials in rare diseases with limited number of patients available. We developed a network representation relating the hot topics developed by the IRDiRC task force on small population clinical trials to IDeAl's work as well as relating important methodologies by IDeAl's definition necessary to consider in design and analysis of small-population clinical trials. These network representation establish a new perspective on design and analysis of small-population clinical trials.

CONCLUSION

IDeAl has provided a huge number of options to refine the statistical methodology for small-population clinical trials from various perspectives. A total of 33 recommendations developed and related to the work packages help the researcher to design small population clinical trial. The route to improvements is displayed in IDeAl-network representing important statistical methodological skills necessary to design and analysis of small-population clinical trials. The methods are ready for use.

Experimentally evoked same-sex sexual behaviour in pigeons: better to be in a female-female pair than alone

Same-sex sexual behaviour has been noted among social animals. However, because of the large number of observations necessary, data from controlled experiments are lacking. In this study, we performed experiments to evaluate the effects of male and female removal in colonies of the feral pigeon (Columba livia f. urbana). After the experimental removal of males, five long-lasting female-female pairs occurred. We found that those pairs could successfully raise offspring in a manner comparable to female-male pairs. Same-sex sexual behaviour and pairing in females is thus a better alternative to postponed breeding or breeding alone without the help of a partner. In contrast, in the case of female-removal experiments, same-sex pairing behaviour occurred in males as a temporary phenomenon with characteristic mutual aggression. Additionally, under a male-biased sex ratio, we observed father-son and father-daughter copulations. To the best of our knowledge, these results are the first obtained under controlled experimental conditions which demonstrate that the sex ratio of a population can shift the social structure and cause cooperative same-sex breeding behaviour to arise in a monogamous species.

Validating the Italian Version of the Disgust and Propensity Scale-Revised

The aim of this work is to evaluate the factor structure and psychometric properties of the Italian version of the Disgust Propensity and Sensitivity Scale-Revised (DPSS-R, 16 items) in two samples taken from the general population. In the first study, 285 participants completed the DPSS-R questionnaire through a web-based survey. Exploratory factor analysis for ordinal Likert-type data supported the existence of four underlying factors, reflecting self-focused disgust, disgust propensity, somatic anxiety and disgust sensitivity. In the second study, an independent sample of 293 participants was enrolled as a test set to validate the factor structure obtained in the exploratory phase. The factor solution was confirmed, but showed quite highly correlated latent factors. We fitted the model and tested whether or not the bifactor structure was better than the previous one (four correlated factors). Actually, we had evidence supporting the presence of a general factor, providing a measure of disgust susceptibility, along with the four specific factors previously defined. This result could be useful also from the clinical perspective since the DPSS-R questionnaire will be used in clinical context, where underlying factors may be related to different and specific psychopathological profiles. Finally, we examined and visualized the interrelationships among the four DPSS-R factors and the external scales (Anxiety Sensitivity, Disgust Scale and Padua) using a graphical model approach.