The cost of large numbers of hypothesis tests on power, effect size and sample size

Improving rigor and reproducibility in western blot experiments with the blotRig analysis

Western blot is a popular biomolecular analysis method for measuring the relative quantities of independent proteins in complex biological samples. However, variability in quantitative western blot data analysis poses a challenge in designing reproducible experiments. The lack of rigorous quantitative approaches in current western blot statistical methodology may result in irreproducible inferences. Here we describe best practices for the design and analysis of western blot experiments, with examples and demonstrations of how different analytical approaches can lead to widely varying outcomes. To facilitate best practices, we have developed the blotRig tool for designing and analyzing western blot experiments to improve their rigor and reproducibility. The blotRig application includes functions for counterbalancing experimental design by lane position, batch management across gels, and analytics with covariates and random effects.

Improving rigor and reproducibility in western blot experiments with the blotRig analysis software

Western blot is a popular biomolecular analysis method for measuring the relative quantities of independent proteins in complex biological samples. However, variability in quantitative western blot data analysis poses a challenge in designing reproducible experiments. The lack of rigorous quantitative approaches in current western blot statistical methodology may result in irreproducible inferences. Here we describe best practices for the design and analysis of western blot experiments, with examples and demonstrations of how different analytical approaches can lead to widely varying outcomes. To facilitate best practices, we have developed the blotRig tool for designing and analyzing western blot experiments to improve their rigor and reproducibility. The blotRig application includes functions for counterbalancing experimental design by lane position, batch management across gels, and analytics with covariates and random effects.

Cytokine- and Vascular Endothelial Growth Factor-Related Gene-Based Genome-Wide Association Study of Low-Dose Ketamine Infusion in Patients with Treatment-Resistant Depression

BACKGROUND AND OBJECTIVE

Ketamine may work as an anti-inflammatory agent, and it increases the levels of vascular endothelial growth factor (VEGF) in patients with treatment-resistant depression. However, whether genes related to pro-inflammatory and anti-inflammatory cytokines and VEGF may predict the treatment response to ketamine remains unknown.Therefore the aim of this study was to analyze whether specific genes related to inflammatory processes and VEGF were associated with treatment response to low-dose ketamine in patients with treatment-resistant depression.

METHODS

Based on the genome data from our clinical trial, this study was a secondary analysis of candidate genes correlated with different timepoints of depressive symptoms. In total, 65 patients with treatment-resistant depression (n = 21 for ketamine 0.5 mg/kg, 20 for ketamine 0.2 mg/kg, and 24 for normal saline) were genotyped for 684,616 single nucleotide polymorphisms. Genes associated with 80 cytokines (i.e., interleukin [IL]-1, IL-6, tumor necrosis factor-α, and adiponectin) and VEGF (i.e., VEGF and VEGF receptors) were selected for the gene-based genome-wide association study on the antidepressant effect of a ketamine infusion.

RESULTS

Specific single nucleotide polymorphisms, including rs2540315 and rs75746675 in IL1R1 and rs79568085 in VEGFC, were related to the rapid (within 240 min) antidepressant effect of a ketamine infusion; specific single nucleotide polymorphisms, such as Affx-20131665 in PIGF and rs8179353, rs8179353, and rs8179353 in TNFRSF8, were associated with the sustained (up to 2 weeks) antidepressant effect of low-dose (combined 0.5 mg/kg and 0.2 mg/kg) ketamine.

CONCLUSIONS

Our findings further revealed that genes related to both anti-inflammatory and pro-inflammatory cytokines (i.e., IL-1, IL-2, IL-6, tumor necrosis factor-α, C-reactive protein, and adiponectin) and VEGF-FLK signaling predicted the treatment response to a ketamine infusion in patients with treatment-resistant depression. The synergic modulation of inflammatory and VEGF systems may contribute to the antidepressant effect of ketamine.

CLINICAL TRIAL REGISTRATION

University Hospital Medical Information Network Clinical Trials Registry (UMIN-CTR) number: UMIN000016985.

Lifestyle's influence on community-dwelling older adults' health: A mixed-methods study design

BACKGROUND

Aging often involves health problems and difficulties, such as physical and psychological impairments, isolation, and loneliness, causing social and existential consequences. Studies have explored aging from different perspectives. However, few studies have examined healthy older adults' genetic backgrounds, lifestyles, and meaning in life separately or in combination. This study aims to describe how healthy older adults experience aging, health, lifestyles, and meaning in life and explore potential genetic correlations.

METHODS AND DESIGN

The project will comprise three main parts: a quantitative section featuring the development and testing of a lifestyle questionnaire, a quantitative genetic analysis, and a qualitative interview study. Participants will be community-dwelling, healthy, older adults between 70 and 95 years of age. A sample size of 800 older adults will be invited to participate in seminars in collaboration with the national Swedish association Active Seniors. Data will be collected through lifestyle questionnaire, DNA extracted from saliva samples, and interviews. Based on questionnaire responses, profile groups will be created and compared statistically with variations in genetic backgrounds, providing the basis for recruiting participants to the qualitative interviews.

DISCUSSION

This study's expected outcome will be to gain knowledge about variations in genetic backgrounds correlated with individual experiences regarding aging, health, and meaning in life. This knowledge can improve the understanding of motivations for healthy lifestyle changes. The results can reveal potential implications for individual prerequisites to healthy aging and how health-promoting aging and lifestyle counseling can be adjusted to meet individual needs.

Design, analysis and reporting of multi-arm trials and strategies to address multiple testing

BACKGROUND

It is unclear how multiple treatment comparisons are managed in the analysis of multi-arm trials, particularly related to reducing type I (false positive) and type II errors (false negative).

METHODS

We conducted a cohort study of clinical-trial protocols that were approved by research ethics committees in the UK, Switzerland, Germany and Canada in 2012. We examined the use of multiple-testing procedures to control the overall type I error rate. We created a decision tool to determine the need for multiple-testing procedures. We compared the result of the decision tool to the analysis plan in the protocol. We also compared the pre-specified analysis plans in trial protocols to their publications.

RESULTS

Sixty-four protocols for multi-arm trials were identified, of which 50 involved multiple testing. Nine of 50 trials (18%) used a single-step multiple-testing procedures such as a Bonferroni correction and 17 (38%) used an ordered sequence of primary comparisons to control the overall type I error. Based on our decision tool, 45 of 50 protocols (90%) required use of a multiple-testing procedure but only 28 of the 45 (62%) accounted for multiplicity in their analysis or provided a rationale if no multiple-testing procedure was used. We identified 32 protocol-publication pairs, of which 8 planned a global-comparison test and 20 planned a multiple-testing procedure in their trial protocol. However, four of these eight trials (50%) did not use the global-comparison test. Likewise, 3 of the 20 trials (15%) did not perform the multiple-testing procedure in the publication. The sample size of our study was small and we did not have access to statistical-analysis plans for the included trials in our study.

CONCLUSIONS

Strategies to reduce type I and type II errors are inconsistently employed in multi-arm trials. Important analytical differences exist between planned analyses in clinical-trial protocols and subsequent publications, which may suggest selective reporting of analyses.

Structural Complexity of the Cerebellum and Cerebral Cortex is Reduced in Spinocerebellar Ataxia Type 2

BACKGROUND AND PURPOSE

Fractal dimension (FD) is an index of structural complexity of cortical gray matter (GM) and white matter (WM). Application of FD to pontocerebellar degeneration has revealed cerebellar changes. However, so far, possible concurrent cerebral changes and progression of changes in brain complexity have not been investigated.

METHODS

We computed FD of cerebellar and cerebral cortex and WM derived from longitudinal brain MRI of patients with spinocerebellar ataxia type 2 (SCA2), which is an inherited cause of pontocerebellar degeneration. Nine SCA2 patients and 16 age-matched healthy controls were examined twice (3.6 ± .7 and 3.3 ± 1.0 years apart, respectively) on the same 1.5T MR scanner with T1-weighted imaging. Cortical GM and WM of the cerebrum and cerebellum were segmented using FreeSurfer and FD of these segmentations were computed.

RESULTS

At baseline, FD values of cerebellar GM and WM were significantly (P < .001) lower in SCA2 patients (2.48 ± .04 for GM and 1.74 ± .09 for WM) than in controls (2.56 ± .02 for GM and 2.22 ± .19 for WM). Also, FD values of cerebral GM were significantly (P < .05) lower in SCA2 patients (2.39 ± .03) than in controls (2.43 ± .02). No significant differences were observed for FD of the cerebral WM. The rate of change of FD values was not significantly different between SCA2 patients and controls.

CONCLUSIONS

The structural complexity of the cerebellum and cerebral cortex is reduced in SCA2 patients. Fractal analysis seems not to be able to demonstrate progression of changes associated with degeneration in SCA2.

An introduction to multiplicity issues in clinical trials: the what, why, when and how

In clinical trials it is not uncommon to face a multiple testing problem which can have an impact on both type I and type II error rates, leading to inappropriate interpretation of trial results. Multiplicity issues may need to be considered at the design, analysis and interpretation stages of a trial. The proportion of trial reports not adequately correcting for multiple testing remains substantial. The purpose of this article is to provide an introduction to multiple testing issues in clinical trials, and to reduce confusion around the need for multiplicity adjustments. We use a tutorial, question-and-answer approach to address the key issues of why, when and how to consider multiplicity adjustments in trials. We summarize the relevant circumstances under which multiplicity adjustments ought to be considered, as well as options for carrying out multiplicity adjustments in terms of trial design factors including Population, Intervention/Comparison, Outcome, Time frame and Analysis (PICOTA). Results are presented in an easy-to-use table and flow diagrams. Confusion about multiplicity issues can be reduced or avoided by considering the potential impact of multiplicity on type I and II errors and, if necessary pre-specifying statistical approaches to either avoid or adjust for multiplicity in the trial protocol or analysis plan.

Paired proteomics, transcriptomics and miRNomics in non-small cell lung cancers: known and novel signaling cascades

High-throughput omics analyses are applied to elucidate molecular pathogenic mechanisms in cancer. Given restricted cohort sizes and contrasting large feature sets paired multi-omics analysis supports discovery of true positive deregulated signaling cascades. For lung cancer patients we measured from the same tissue biopsies proteomic- (6,183 proteins), transcriptomic- (34,687 genes) and miRNomic data (2,549 miRNAs). To minimize inter-individual variations case and control lung biopsies have been gathered from the same individuals.Considering single omics entities, 15 of 2,549 miRNAs (0.6%), 752 of 34,687 genes (2.2%) and 141 of 6,183 proteins (2.3%) were significantly deregulated. Multivariate analysis also revealed that effects in miRNA were smaller compared to genes and proteins indicating that expression changes of miRNAs might also have limited impact of pathogenicity. However, a new algorithm for modeling the complex mutual interactions of miRNAs and their target genes facilitated precise prediction of deregulation in cancer genes (92.3% accuracy, p=0.007). Lastly, deregulation of genes in cancer matched deregulation of proteins coded by the genes in 80% of cases.The resulting interaction network, which is based on quantitative analysis of the abundance of miRNAs, mRNAs and proteins each taken from the same lung cancer tissue and from the same autologous normal lung tissue confirms molecular pathological changes and further contributes to the discovery of altered signaling cascades in lung cancer.

Childhood adversity and cognitive functioning in patients with major depression

OBJECTIVE

Major depression is often accompanied by deficits in cognitive functioning and lowered executive functions. However, not all depressed patients show impairments in these domains. The aim of this study was to examine whether different kinds of childhood adversity might account for cognitive deficits in patients with major depression.

METHODS

Ninety-one patients with major depression (DSM-IV) and 40 healthy controls completed a neuropsychological test battery assessing memory, processing speed and executive functions. The Childhood Trauma Questionnaire (CTQ) was used to measure the severity and number of incidences of sexual, physical and emotional abuse and physical and emotional neglect.

RESULTS

Patients with major depression had a significantly higher number of traumas and reported more severe emotional abuse, emotional neglect and physical neglect than healthy controls. Patients performed less well in memory tasks, general knowledge and processing speed than healthy controls. Hierarchical regression analyses indicated that the overall number of traumas was significantly associated with poorer general knowledge, lower processing speed and impaired executive functions in patients with major depression. A second model including all CTQ-subscales simultaneously demonstrated an association between physical neglect and poorer verbal learning, and physical abuse and diminished executive functions.

CONCLUSION

A higher number of childhood adversities may influence general knowledge, processing speed and executive functions in patients with major depression. In addition, physical abuse and neglect seemed to be associated with verbal learning deficits and poorer executive functions.

A P-value model for theoretical power analysis and its applications in multiple testing procedures

Background

Power analysis is a critical aspect of the design of experiments to detect an effect of a given size. When multiple hypotheses are tested simultaneously, multiplicity adjustments to p-values should be taken into account in power analysis. There are a limited number of studies on power analysis in multiple testing procedures. For some methods, the theoretical analysis is difficult and extensive numerical simulations are often needed, while other methods oversimplify the information under the alternative hypothesis. To this end, this paper aims to develop a new statistical model for power analysis in multiple testing procedures.

Methods

We propose a step-function-based p-value model under the alternative hypothesis, which is simple enough to perform power analysis without simulations, but not too simple to lose the information from the alternative hypothesis. The first step is to transform distributions of different test statistics (e.g., t, chi-square or F) to distributions of corresponding p-values. We then use a step function to approximate each of the p-value’s distributions by matching the mean and variance. Lastly, the step-function-based p-value model can be used for theoretical power analysis.

Results

The proposed model is applied to problems in multiple testing procedures. We first show how the most powerful critical constants can be chosen using the step-function-based p-value model. Our model is then applied to the field of multiple testing procedures to explain the assumption of monotonicity of the critical constants. Lastly, we apply our model to a behavioral weight loss and maintenance study to select the optimal critical constants.

Conclusions

The proposed model is easy to implement and preserves the information from the alternative hypothesis.

Important options available--from start to finish--for translating proteomics results to clinical chemistry

In the realm of clinical chemistry, the field of clinical proteomics, that is, the application of proteomic methods for understanding mechanisms and enabling diagnosis, prediction, measurement of activity, and treatment response in disease, is first and foremost a discovery and research tool that feeds assay development downstream. Putative new assay candidates generated by proteomics discovery projects compete with well-established assays with known indications, well-described performance, and of known value in specific clinical settings. Careful attention to the many options available in the design, execution, and interpretation of clinical proteomics studies is thus necessary for translation into clinical practice. We here review and discuss important options associated with clinical proteomics endeavors stretching from the planning phases to the final use in clinical chemistry.

P-values in genomics: apparent precision masks high uncertainty

Scientists often interpret P-values as measures of the relative strength of statistical findings. This is common practice in large-scale genomic studies where P-values are used to choose which of numerous hypothesis test results should be pursued in subsequent research. In this study, we examine P-value variability to assess the degree of certainty P-values provide. We develop prediction intervals for the P-value in a replication study given the P-value observed in an initial study. The intervals depend on the initial value of P and the ratio of sample sizes between the initial and replication studies, but not on the underlying effect size or initial sample size. The intervals are valid for most large-sample statistical tests in any context, and can be used in the presence of single or multiple tests. While P-values are highly variable, future P-value variability can be explicitly predicted based on a P-value from an initial study. The relative size of the replication and initial study is an important predictor of the P-value in a subsequent replication study. We provide a handy calculator implementing these results and apply them to a study of Alzheimer's disease and recent findings of the Cross-Disorder Group of the Psychiatric Genomics Consortium. This study suggests that overinterpretation of very significant, but highly variable, P-values is an important factor contributing to the unexpectedly high incidence of non-replication. Formal prediction intervals can also provide realistic interpretations and comparisons of P-values associated with different estimated effect sizes and sample sizes.

Pro-inflammatory cytokines as predictors of antidepressant effects of exercise in major depressive disorder

Exercise is an efficacious treatment for major depressive disorder (MDD) and has independently been shown to have anti-inflammatory effects in non-depressed subjects. Patients with MDD have elevated inflammatory cytokines but it is not known if exercise affects inflammation in MDD patients and whether these changes are clinically relevant. In the TReatment with Exercise Augmentation for Depression (TREAD) study, participants who were partial responders to a selective serotonin reuptake inhibitor were randomized to receive one of two doses of exercise: 16 kilocalories per kilogram of body weight per week (KKW), or 4 KKW for 12 weeks. Blood samples were collected before initiation and again at the end of the 12-week exercise intervention. Serum was analyzed using a multiplexed ELISA for interferon-γ (IFN-γ), interleukin-1β (IL-1β), interleukin-6 (IL-6) and tumor necrosis factor-α (TNF-α). Higher baseline levels of TNF-α were associated with greater decrease in depression symptoms over the 12-week exercise period (P<0.0001). In addition, a significant positive correlation between change in IL-1β and change in depression symptom scores was observed (P=0.04). There were no significant changes in mean level of any cytokine following the 12-week intervention, and no significant relationship between exercise dose and change in mean cytokine level. Results suggest that high TNF-α may differentially predict better outcomes with exercise treatment as opposed to antidepressant medications for which high TNF-α is linked to poor response. Our results also confirm findings from studies of antidepressant medications that tie decreasing IL-1β to positive depression treatment outcomes.

Neurological imaging: statistics behind the pictures

Neurological imaging represents a powerful paradigm for investigation of brain structure, physiology and function across different scales. The diverse phenotypes and significant normal and pathological brain variability demand reliable and efficient statistical methodologies to model, analyze and interpret raw neurological images and derived geometric information from these images. The validity, reproducibility and power of any statistical brain map require appropriate inference on large cohorts, significant community validation, and multidisciplinary collaborations between physicians, engineers and statisticians.