Enhancing Statistical Inference in Psychological Research via Prospective and Retrospective Design Analysis

Oral language interventions can improve language outcomes in children with neurodevelopmental disorders: A systematic review and meta-analysis

Background

Young people who fail to develop language as expected face significant challenges in all aspects of life. Unfortunately, language disorders are common, either as a distinct condition (e.g., Developmental Language Disorder) or as a part of another neurodevelopmental condition (e.g., autism). Finding ways to attenuate language problems through intervention has the potential to yield great benefits not only for the individual but also for society as a whole.

Objectives

This meta-analytic review examined the effect of oral language interventions for children with neurodevelopmental disorders.

Search Methods

The last electronic search was conducted in April 2022.

Selection Criteria

Intervention studies had to target language skills for children from 2 to 18 years of age with Developmental Language Disorder, autism, intellectual disability, Down syndrome, Fragile X syndrome, and Williams syndrome in randomised controlled trials or quasi-experimental designs. Control groups had to include business-as-usual, waiting list, passive or active conditions. However, we excluded studies in which the active control group received a different type, delivery, or dosage of another language intervention. Eligible interventions implemented explicit and structured activities (i.e., explicit instruction of vocabulary, narrative structure or grammatical rules) and/or implicit and broad activities (i.e., shared book reading, general language stimulation). The intervention studies had to assess language skills in receptive and/or expressive modalities.

Data Collection and Analysis

The search provided 8195 records after deduplication. Records were screened by title and abstract, leading to full-text examinations of 448 records. We performed Correlated and Hierarchical Effects models and ran a retrospective power analysis via simulation. Publication bias was assessed via p-curve and precision-effect estimate.

Main Results

We examined 38 studies, with 46 group comparisons and 108 effects comparing pre-/post-tests and eight studies, with 12 group comparisons and 21 effects at follow-up. The results showed a mean effect size of d = 0.27 at the post-test and d = 0.18 at follow-up. However, there was evidence of publication bias and overestimation of the mean effects. Effects from the meta-analysis were significantly related to these elements: (1) receptive vocabulary and omnibus receptive measures showed smaller effect sizes relative to expressive vocabulary, grammar, expressive and receptive discourse, and omnibus expressive tests; and (2) the length of the intervention, where longer sessions conducted over a longer period of time were more beneficial than brief sessions and short-term interventions. Neither moderators concerning participants' characteristics (children's diagnosis, diagnostic status, age, sex, and non-verbal cognitive ability and severity of language impairment), nor those regarding of the treatment components and implementation of the language interventions (intervention content, setting, delivery agent, session structure of the intervention or total number of sessions) reached significance. The same occurred to indicators of study quality. The risk of bias assessment showed that reporting quality for the studies examined in the review was poor.

Authors’ Conclusions

In sum, the current evidence base is promising but inconclusive. Pre-registration and replication of more robust and adequately powered trials, which include a wider range of diagnostic conditions, together with more long-term follow-up comparisons, are needed to drive evidence-based practice and policy.

Interdisciplinary and interprofessional communication intervention: How psychological safety fosters communication and increases patient safety

Background

Effective teamwork and communication are imperative for patient safety and quality care. Communication errors and human failures are considered the main source of patient harm. Thus, team trainings focusing on communication and creating psychologically safe environments are required. This can facilitate challenging communication and teamwork scenarios, prevent patient safety risks, and increase team performance perception. The sparse research concerning communication interventions calls for an understanding of psychological mechanisms. Therefore, this study investigated mechanisms of an interpersonal team intervention targeting communication and the relation of psychological safety to patient safety and team performance perception based on the applied input-process-output model of team effectiveness.

Methods

Before and after a 4-h communication intervention for multidisciplinary teams, a paper-pencil survey with N = 137 healthcare workers from obstetric units of two university hospitals was conducted. Changes after the intervention in perceived communication, patient safety risks, and team performance perception were analyzed via t-tests. To examine psychological mechanisms regarding psychological safety and communication behavior, mediation analyses were conducted.

Results

On average, perceived patient safety risks were lower after the intervention than before the intervention (M_T1 = 3.220, SD_T1 = 0.735; M_T2 = 2.887, SD_T2 = 0.902). This change was statistically significant (t (67) = 2.760, p =.007). However, no such effect was found for interpersonal communication and team performance perception. The results illustrate the mediating role of interpersonal communication between psychological safety and safety performances operationalized as perceived patient safety risks (α1∗β1 = -0.163, 95% CI [-0.310, -0.046]) and team performance perception (α1∗β1 = 0.189, 95% CI [0.044, 0.370]).

Discussion

This study demonstrates the psychological mechanisms of communication team training to foster safety performances and psychological safety as an important predecessor for interpersonal communication. Our results highlight the importance of teamwork for patient safety. Interpersonal and interprofessional team training represents a novel approach as it empirically brings together interpersonal communication and collaboration in the context of patient safety. Future research should work on follow-up measures in randomized-controlled trials to broaden an understanding of changes over time.

Isometric Knee Strength is Greater in Individuals Who Score Higher on Psychological Readiness to Return to Sport After Primary Anterior Cruciate Ligament Reconstruction

Background

Anterior cruciate ligament (ACL) injury is extremely common among athletes. Rate of second ACL injury due to surgical graft rupture or contralateral limb ACL injury is approximately 15-32%. Psychological readiness to return to sport (RTS) may be an important predictor of successful RTS outcomes. Psychological readiness can be quantified using the ACL Return to Sport after Injury (ACL-RSI) questionnaire, with higher scores demonstrating greater psychological readiness.

Purpose

The purpose of this study was to investigate differences in functional performance and psychological readiness to return to sport among athletes who have undergone primary ACL reconstruction (ACLR).

Study Design

Descriptive cohort study.

Methods

Eighteen athletes who had undergone primary ACLR were tested at time of RTS clearance. The cohort was divided into two groups, high score (HS) and low score (LS), based on median ACL-RSI score, and performance on static and dynamic postural stability testing, lower extremity isokinetic and isometric strength testing, and single leg hop testing was compared between the groups using an independent samples t-test.

Results

The median ACL-RSI score was 74.17. The average ACL-RSI score was 83.1±6.2 for the HS group and 61.8±8.0 for the LS group. High scorers on the ACL-RSI performed significantly better on isometric knee flexion as measured via handheld dynamometry (22.61% ±6.01 vs. 12.12% ±4.88, p=0.001) than the low score group.

Conclusion

The findings suggest that increased knee flexion strength may be important for psychological readiness to RTS after primary ACLR. Further research is indicated to explore this relationship, however, a continued emphasis on improving hamstring strength may be appropriate during rehabilitation following ACLR to positively impact psychological readiness for RTS.

Level of Evidence

III.

Entia Non Sunt Multiplicanda … Shall I look for clusters in my cognitive data?

Unsupervised clustering methods are increasingly being applied in psychology. Researchers may use such methods on multivariate data to reveal previously undetected sub-populations of individuals within a larger population. Realistic research scenarios in the cognitive science may not be ideally suited for a successful use of these methods, however, as they are characterized by modest effect sizes, limited sample sizes, and non-orthogonal indicators. This combination of characteristics even presents a high risk of detecting non-existing clusters. A systematic review showed that, among 191 studies published in 2016–2020 that used different clustering methods to classify human participants, the median sample size was only 322, and a median of 3 latent classes/clusters were detected. None of them concluded in favor of a one-cluster solution, potentially giving rise to an extreme publication bias. Dimensionality reduction techniques are almost never used before clustering. In a subsequent simulation study, we examined the performance of popular clustering techniques, including Gaussian mixture model, a partitioning, and a hierarchical agglomerative algorithm. We focused on their ability to detect the correct number of clusters, and on their classification accuracy. Under a reasoned set of scenarios that we considered plausible for the cognitive research, none of the methods adequately discriminates between one vs two true clusters. In addition, non-orthogonal indicators lead to a high risk of incorrectly detecting multiple clusters where none existed, even in the presence of only modest correlation (a frequent case in psychology). In conclusion, it is hard for researchers to be in a condition to achieve a valid unsupervised clustering for inferential purposes with a view to classifying individuals.

The Influence of 5-HTTLPR, BDNF Rs6265 and COMT Rs4680 Polymorphisms on Impulsivity in Bipolar Disorder: The Role of Gender

Impulsivity has been proposed as an endophenotype for bipolar disorder (BD); moreover, impulsivity levels have been shown to carry prognostic significance and to be quality-of-life predictors. To date, reports about the genetic determinants of impulsivity in mood disorders are limited, with no studies on BD individuals. Individuals with BD and healthy controls (HC) were recruited in the context of an observational, multisite study (GECOBIP). Subjects were genotyped for three candidate single-nucleotide polymorphisms (SNPs) (5-HTTLPR, COMT rs4680, BDNF rs6265); impulsivity was measured through the Italian version of the Barratt Impulsiveness Scale (BIS-11). A mixed-effects regression model was built, with BIS scores as dependent variables, genotypes of the three polymorphisms as fixed effects, and centers of enrollment as random effect. Compared to HC, scores for all BIS factors were higher among subjects with euthymic BD (adjusted β for Total BIS score: 5.35, p < 0.001). No significant interaction effect was evident between disease status (HC vs. BD) and SNP status for any polymorphism. Considering the whole sample, BDNF Met/Met homozygosis was associated with lower BIS scores across all three factors (adjusted β for Total BIS score: −10.2, p < 0.001). A significant 5-HTTLPR x gender interaction was found for the SS genotype, associated with higher BIS scores in females only (adjusted β for Total BIS score: 12.0, p = 0.001). Finally, COMT polymorphism status was not significantly associated with BIS scores. In conclusion, BD diagnosis did not influence the effect on impulsivity scores for any of the three SNPs considered. Only one SNP—the BDNF rs6265 Met/Met homozygosis—was independently associated with lower impulsivity scores. The 5-HTTLPR SS genotype was associated with higher impulsivity scores in females only. Further studies adopting genome-wide screening in larger samples are needed to define the genetic basis of impulsivity in BD.

Addressing context dependence in ecology

Context dependence is widely invoked to explain disparate results in ecology. It arises when the magnitude or sign of a relationship varies due to the conditions under which it is observed. Such variation, especially when unexplained, can lead to spurious or seemingly contradictory conclusions, which can limit understanding and our ability to transfer findings across studies, space, and time. Using examples from biological invasions, we identify two types of context dependence resulting from four sources: mechanistic context dependence arises from interaction effects; and apparent context dependence can arise from the presence of confounding factors, problems of statistical inference, and methodological differences among studies. Addressing context dependence is a critical challenge in ecology, essential for increased understanding and prediction.

#EEGManyLabs: Investigating the replicability of influential EEG experiments

There is growing awareness across the neuroscience community that the replicability of findings about the relationship between brain activity and cognitive phenomena can be improved by conducting studies with high statistical power that adhere to well-defined and standardised analysis pipelines. Inspired by recent efforts from the psychological sciences, and with the desire to examine some of the foundational findings using electroencephalography (EEG), we have launched #EEGManyLabs, a large-scale international collaborative replication effort. Since its discovery in the early 20th century, EEG has had a profound influence on our understanding of human cognition, but there is limited evidence on the replicability of some of the most highly cited discoveries. After a systematic search and selection process, we have identified 27 of the most influential and continually cited studies in the field. We plan to directly test the replicability of key findings from 20 of these studies in teams of at least three independent laboratories. The design and protocol of each replication effort will be submitted as a Registered Report and peer-reviewed prior to data collection. Prediction markets, open to all EEG researchers, will be used as a forecasting tool to examine which findings the community expects to replicate. This project will update our confidence in some of the most influential EEG findings and generate a large open access database that can be used to inform future research practices. Finally, through this international effort, we hope to create a cultural shift towards inclusive, high-powered multi-laboratory collaborations.

Dyslexia treatment studies: A systematic review and suggestions on testing treatment efficacy with small effects and small samples

Poor response to treatment is a defining characteristic of reading disorder. In the present systematic review and meta-analysis, we found that the overall average effect size for treatment efficacy was modest, with a mean standardized difference of 0.38. Small true effects, combined with the difficulty to recruit large samples, seriously challenge researchers planning to test treatment efficacy in dyslexia and potentially in other learning disorders. Nonetheless, most published studies claim effectiveness, generally based on liberal use of multiple testing. This inflates the risk that most statistically significant results are associated with overestimated effect sizes. To enhance power, we propose the strategic use of repeated measurements with mixed-effects modelling. This novel approach would enable us to estimate both individual parameters and population-level effects more reliably. We suggest assessing a reading outcome not once, but three times, at pre-treatment and three times at post-treatment. Such design would require only modest additional efforts compared to current practices. Based on this, we performed ad hoc a priori design analyses via simulation studies. Results showed that using the novel design may allow one to reach adequate power even with low sample sizes of 30–40 participants (i.e., 15–20 participants per group) for a typical effect size of d = 0.38. Nonetheless, more conservative assumptions are warranted for various reasons, including a high risk of publication bias in the extant literature. Our considerations can be extended to intervention studies of other types of neurodevelopmental disorders.

Using Mobile Devices in Teaching Large University Classes: How Does It Affect Exam Success?

The aim of this study was to examine the role of mobile-based student response systems in teaching to improve university students’ academic outcomes. Mobile devices can be useful tools for conveying content to large classes, with a potential impact on academic outcomes. This study involved a total of 294 undergraduates taking a psychology course. The course involved lessons in the classroom, which included answering quizzes (quiz activities) and activities such as preparing reports and laboratory experiences (out-of-class activities). Quizzes were administered using a mobile technology system. Data were collected on the motivational beliefs (theory of intelligence) and self-regulated learning strategies of students who voluntarily completed the online questionnaires. The results of the linear models showed that using the quizzes positively affected the performance in the final exams (involving closed and open questions). The same was true for the out-of-class activities. The motivation and strategy scores correlated moderately with out-of-class activities, but not with quiz activities. These results offer insight on the use of technology during lessons and other course-related activities to promote academic achievement.