A note on statistical repeatability and study design for high-throughput assays

Systematic data analysis pipeline for quantitative morphological cell phenotyping

Quantitative morphological phenotyping (QMP) is an image-based method used to capture morphological features at both the cellular and population level. Its interdisciplinary nature, spanning from data collection to result analysis and interpretation, can lead to uncertainties, particularly among those new to this actively growing field. High analytical specificity for a typical QMP is achieved through sophisticated approaches that can leverage subtle cellular morphological changes. Here, we outline a systematic workflow to refine the QMP methodology. For a practical review, we describe the main steps of a typical QMP; in each step, we discuss the available methods, their applications, advantages, and disadvantages, along with the R functions and packages for easy implementation. This review does not cover theoretical backgrounds, but provides several references for interested researchers. It aims to broaden the horizons for future phenome studies and demonstrate how to exploit years of endeavors to achieve more with less.

Employing multiple synchronous outcome samples per subject to improve study efficiency

Background

Accuracy can be improved by taking multiple synchronous samples from each subject in a study to estimate the endpoint of interest if sample values are not highly correlated. If feasible, it is useful to assess the value of this cluster approach when planning studies. Multiple assessments may be the only method to increase power to an acceptable level if the number of subjects is limited.

Methods

The main aim is to estimate the difference in outcome between groups of subjects by taking one or more synchronous primary outcome samples or measurements. A summary statistic from multiple samples per subject will typically have a lower sampling error. The number of subjects can be balanced against the number of synchronous samples to minimize the sampling error, subject to design constraints. This approach can include estimating the optimum number of samples given the cost per subject and the cost per sample.

Results

The accuracy improvement achieved by taking multiple samples depends on the intra-class correlation (ICC). The lower the ICC, the greater the benefit that can accrue. If the ICC is high, then a second sample will provide little additional information about the subject’s true value. If the ICC is very low, adding a sample can be equivalent to adding an extra subject. Benefits of multiple samples include the ability to reduce the number of subjects in a study and increase both the power and the available alpha. If, for example, the ICC is 35%, adding a second measurement can be equivalent to adding 48% more subjects to a single measurement study.

Conclusion

A study’s design can sometimes be improved by taking multiple synchronous samples. It is useful to evaluate this strategy as an extension of a single sample design. An Excel workbook is provided to allow researchers to explore the most appropriate number of samples to take in a given setting.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01414-7.

High-throughput screening for high-efficiency small-molecule biosynthesis

Systems metabolic engineering faces the formidable task of rewiring microbial metabolism to cost-effectively generate high-value molecules from a variety of inexpensive feedstocks for many different applications. Because these cellular systems are still too complex to model accurately, vast collections of engineered organism variants must be systematically created and evaluated through an enormous trial-and-error process in order to identify a manufacturing-ready strain. The high-throughput screening of strains to optimize their scalable manufacturing potential requires execution of many carefully controlled, parallel, miniature fermentations, followed by high-precision analysis of the resulting complex mixtures. This review discusses strategies for the design of high-throughput, small-scale fermentation models to predict improved strain performance at large commercial scale. Established and promising approaches from industrial and academic groups are presented for both cell culture and analysis, with primary focus on microplate- and microfluidics-based screening systems.

A Bayesian non-parametric mixed-effects model of microbial growth curves

Substantive changes in gene expression, metabolism, and the proteome are manifested in overall changes in microbial population growth. Quantifying how microbes grow is therefore fundamental to areas such as genetics, bioengineering, and food safety. Traditional parametric growth curve models capture the population growth behavior through a set of summarizing parameters. However, estimation of these parameters from data is confounded by random effects such as experimental variability, batch effects or differences in experimental material. A systematic statistical method to identify and correct for such confounding effects in population growth data is not currently available. Further, our previous work has demonstrated that parametric models are insufficient to explain and predict microbial response under non-standard growth conditions. Here we develop a hierarchical Bayesian non-parametric model of population growth that identifies the latent growth behavior and response to perturbation, while simultaneously correcting for random effects in the data. This model enables more accurate estimates of the biological effect of interest, while better accounting for the uncertainty due to technical variation. Additionally, modeling hierarchical variation provides estimates of the relative impact of various confounding effects on measured population growth.

Pet or pest? Stable isotope methods for determining the provenance of an invasive alien species

The illegal pet trade facilitates the global dispersal of invasive alien species (IAS), providing opportunities for new pests to establish in novel recipient environments. Despite the increasing threat of IAS to the environment and economy, biosecurity efforts often lack suitable, scientifically-based methods to make effective management decisions, such as identifying an established IAS population from a single incursion event. We present a proof-of-concept for a new application of a stable isotope technique to identify wild and captive histories of an invasive pet species. Twelve red-eared slider turtles (Trachemys scripta elegans) from historic Australian incursions with putative wild, captive and unknown origins were analysed to: (1) present best-practice methods for stable isotope sampling of T. s. elegans incursions; (2) effectively discriminate between wild and captive groups using stable isotope ratios; and (3) present a framework to expand the methodology for use on other IAS species. A sampling method was developed to obtain carbon (δ13C) and nitrogen (δ15N) stable isotope ratios from the keratin layer of the carapace (shells), which are predominantly influenced by dietary material and trophic level respectively. Both δ13C and δ15N exhibited the potential to distinguish between the wild and captive origins of the samples. Power simulations demonstrated that isotope ratios were consistent across the carapace and a minimum of eight individuals were required to effectively discriminate wild and captive groups, reducing overall sampling costs. Statistical classification effectively separated captive and wild groups by δ15N (captive: δ15N‰ ≥ 9.7‰, minimum of 96% accuracy). This study outlines a practical and accessible method for detecting IAS incursions, to potentially provide biosecurity staff and decision-makers with the tools to quickly identify and manage future IAS incursions.

A note on statistical repeatability and study design for high-throughput assays

Characterizing the technical precision of measurements is a necessary stage in the planning of experiments and in the formal sample size calculation for optimal design. Instruments that measure multiple analytes simultaneously, such as in high-throughput assays arising in biomedical research, pose particular challenges from a statistical perspective. The current most popular method for assessing precision of high-throughput assays is by scatterplotting data from technical replicates. Here, we question the statistical rationale of this approach from both an empirical and theoretical perspective, illustrating our discussion using four example data sets from different genomic platforms. We demonstrate that such scatterplots convey little statistical information of relevance and are potentially highly misleading. We present an alternative framework for assessing the precision of high-throughput assays and planning biomedical experiments. Our methods are based on repeatability-a long-established statistical quantity also known as the intraclass correlation coefficient. We provide guidance and software for estimation and visualization of repeatability of high-throughput assays, and for its incorporation into study design. © 2016 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.