Ten simple rules for effective computational research

Ten simple rules for managing laboratory information

Information is the cornerstone of research, from experimental (meta)data and computational processes to complex inventories of reagents and equipment. These 10 simple rules discuss best practices for leveraging laboratory information management systems to transform this large information load into useful scientific findings.

An ecologist's guide for studying DNA methylation variation in wild vertebrates

The field of molecular biology is advancing fast with new powerful technologies, sequencing methods and analysis software being developed constantly. Commonly used tools originally developed for research on humans and model species are now regularly used in ecological and evolutionary research. There is also a growing interest in the causes and consequences of epigenetic variation in natural populations. Studying ecological epigenetics is currently challenging, especially for vertebrate systems, because of the required technical expertise, complications with analyses and interpretation, and limitations in acquiring sufficiently high sample sizes. Importantly, neglecting the limitations of the experimental setup, technology and analyses may affect the reliability and reproducibility, and the extent to which unbiased conclusions can be drawn from these studies. Here, we provide a practical guide for researchers aiming to study DNA methylation variation in wild vertebrates. We review the technical aspects of epigenetic research, concentrating on DNA methylation using bisulfite sequencing, discuss the limitations and possible pitfalls, and how to overcome them through rigid and reproducible data analysis. This review provides a solid foundation for the proper design of epigenetic studies, a clear roadmap on the best practices for correct data analysis and a realistic view on the limitations for studying ecological epigenetics in vertebrates. This review will help researchers studying the ecological and evolutionary implications of epigenetic variation in wild populations.

Ten simple rules for providing bioinformatics support within a hospital

Bioinformatics has become a key aspect of the biomedical research programmes of many hospitals' scientific centres, and the establishment of bioinformatics facilities within hospitals has become a common practice worldwide. Bioinformaticians working in these facilities provide computational biology support to medical doctors and principal investigators who are daily dealing with data of patients to analyze. These bioinformatics analysts, although pivotal, usually do not receive formal training for this job. We therefore propose these ten simple rules to guide these bioinformaticians in their work: ten pieces of advice on how to provide bioinformatics support to medical doctors in hospitals. We believe these simple rules can help bioinformatics facility analysts in producing better scientific results and work in a serene and fruitful environment.

Meeting in the Middle: Towards Successful Multidisciplinary Bioimage Analysis Collaboration

With an increase in subject knowledge expertise required to solve specific biological questions, experts from different fields need to collaborate to address increasingly complex issues. To successfully collaborate, everyone involved in the collaboration must take steps to “meet in the middle.” We thus present a guide on truly cross-disciplinary work using bioimage analysis as a showcase, where it is required that the expertise of biologists, microscopists, data analysts, clinicians, engineers, and physicists meet. We discuss considerations and best practices from the perspective of both users and technology developers, while offering suggestions for working together productively and how this can be supported by institutes and funders. Although this guide uses bioimage analysis as an example, the guiding principles of these perspectives are widely applicable to other cross-disciplinary work.

Enhancing research informatics core user satisfaction through agile practices

OBJECTIVE

The Huntsman Cancer Institute Research Informatics Shared Resource (RISR), a software and database development core facility, sought to address a lack of published operational best practices for research informatics cores. It aimed to use those insights to enhance effectiveness after an increase in team size from 20 to 31 full-time equivalents coincided with a reduction in user satisfaction.

MATERIALS AND METHODS

RISR migrated from a water-scrum-fall model of software development to agile software development practices, which emphasize iteration and collaboration. RISR's agile implementation emphasizes the product owner role, which is responsible for user engagement and may be particularly valuable in software development that requires close engagement with users like in science.

RESULTS

All RISR's software development teams implemented agile practices in early 2020. All project teams are led by a product owner who serves as the voice of the user on the development team. Annual user survey scores for service quality and turnaround time recorded 9 months after implementation increased by 17% and 11%, respectively.

DISCUSSION

RISR is illustrative of the increasing size of research informatics cores and the need to identify best practices for maintaining high effectiveness. Agile practices may address concerns about the fit of software engineering practices in science. The study had one time point after implementing agile practices and one site, limiting its generalizability.

CONCLUSIONS

Agile software development may substantially increase a research informatics core facility's effectiveness and should be studied further as a potential best practice for how such cores are operated.

Quantification of increased MUC5AC expression in airway mucus of smoker using an automated image-based approach

Microscopic analysis of mucus quantity and composition is crucial in research and diagnostics on muco-obstructive diseases. Currently used image-based methods are unable to extract concrete numeric values of mucosal proteins, especially on the expression of the key mucosal proteins MUC5AC and MUC5B. Since their levels increase under pathologic conditions such as extensive exposure to cigarette smoke, it is imperative to quantify them to improve treatment strategies of pulmonary diseases. This study presents a simple, image-based, and high-processing computational method that allows determining the ratio of MUC5AC and MUC5B within the overall airway mucus while providing information on their spatial distribution. The presented pipeline was optimized for automated downstream analysis using a combination of bright field and immunofluorescence imaging suitable for tracheal and bronchial tissue samples, and air-liquid interface (ALI) cell cultures. To validate our approach, we compared tracheal tissue and ALI cell cultures of isolated primary normal human bronchial epithelial cells derived from smokers and nonsmokers. Our data indicated 18-fold higher levels of MUC5AC in submucosal glands of smokers covering about 8% of mucosal areas compared to <1% in nonsmoking individuals, confirming results of previous studies. We further identified a subpopulation of nonsmokers with slightly elevated glandular MUC5AC levels suggesting moderate exposure to second-hand smoke or fine particulate air pollution. Overall, this study demonstrates a novel, user-friendly and freely available tool for digital pathology and the analysis of therapeutic interventions tested in ALI cell cultures.

Responsible modelling: Unit testing for infectious disease epidemiology

Infectious disease epidemiology is increasingly reliant on large-scale computation and inference. Models have guided health policy for epidemics including COVID-19 and Ebola and endemic diseases including malaria and tuberculosis. Yet a coding bug may bias results, yielding incorrect conclusions and actions causing avoidable harm. We are ethically obliged to make our code as free of error as possible. Unit testing is a coding method to avoid such bugs, but it is rarely used in epidemiology. We demonstrate how unit testing can handle the particular quirks of infectious disease models and aim to increase the uptake of this methodology in our field.

Low availability of code in ecology: A call for urgent action

Access to analytical code is essential for transparent and reproducible research. We review the state of code availability in ecology using a random sample of 346 nonmolecular articles published between 2015 and 2019 under mandatory or encouraged code-sharing policies. Our results call for urgent action to increase code availability: only 27% of eligible articles were accompanied by code. In contrast, data were available for 79% of eligible articles, highlighting that code availability is an important limiting factor for computational reproducibility in ecology. Although the percentage of ecological journals with mandatory or encouraged code-sharing policies has increased considerably, from 15% in 2015 to 75% in 2020, our results show that code-sharing policies are not adhered to by most authors. We hope these results will encourage journals, institutions, funding agencies, and researchers to address this alarming situation.

Publication of the analytical code underlying a scientific study is increasingly expected or even mandated by journals, allowing others to reproduce the results. However, a survey of more than 300 recently published ecology papers finds the majority have no code publicly available, handicapping efforts to improve scientific transparency.

Shift in the paradigm towards next-generation microbiology

In this work, the position of contemporary microbiology is considered from the perspective of scientific success, and a list of historical points and lessons learned from the fields of medical microbiology, microbial ecology and systems biology is presented. In addition, patterns in the development of top-down research topics that emerged over time as well as overlapping ideas and personnel, which are the first signs of trans-domain research activities in the fields of metagenomics, metaproteomics, metatranscriptomics and metabolomics, are explored through analysis of the publication networks of 28 654 papers using the computer programme Pajek. The current state of affairs is defined, and the need for meta-analyses to leverage publication biases in the field of microbiology is put forward as a very important emerging field of microbiology, especially since microbiology is progressively dealing with multi-scale systems. Consequently, the need for cross-fertilisation with other fields/disciplines instead of ‘more microbiology’ is needed to advance the field of microbiology as such. The reader is directed to consider how novel technologies, the introduction of big data approaches and artificial intelligence have transformed microbiology into a multi-scale field and initiated a shift away from its history of mostly manual work and towards a largely technology-, data- and statistics-driven discipline that is often coupled with automation and modelling.

Twelve quick tips for designing sound dynamical models for bioprocesses

Because of the inherent complexity of bioprocesses, mathematical models are more and more used for process design, control, optimization, etc. These models are generally based on a set of biochemical reactions. Model equations are then derived from mass balance, coupled with empirical kinetics. Biological models are nonlinear and represent processes, which by essence are dynamic and adaptive. The temptation to embed most of the biology is high, with the risk that calibration would not be significant anymore. The most important task for a modeler is thus to ensure a balance between model complexity and ease of use. Since a model should be tailored to the objectives, which will depend on applications and environment, a universal model representing any possible situation is probably not the best option.

Comparing time series transcriptome data between plants using a network module finding algorithm

BACKGROUND

Comparative transcriptome analysis is the comparison of expression patterns between homologous genes in different species. Since most molecular mechanistic studies in plants have been performed in model species, including Arabidopsis and rice, comparative transcriptome analysis is particularly important for functional annotation of genes in diverse plant species. Many biological processes, such as embryo development, are highly conserved between different plant species. The challenge is to establish one-to-one mapping of the developmental stages between two species.

RESULTS

In this manuscript, we solve this problem by converting the gene expression patterns into co-expression networks and then apply network module finding algorithms to the cross-species co-expression network. We describe how such analyses are carried out using bash scripts for preliminary data processing followed by using the R programming language for module finding with a simulated annealing method. We also provide instructions on how to visualize the resulting co-expression networks across species.

CONCLUSIONS

We provide a comprehensive pipeline from installing software and downloading raw transcriptome data to predicting homologous genes and finding orthologous co-expression networks. From the example provided, we demonstrate the application of our method to reveal functional conservation and divergence of genes in two plant species.

Software engineering principles to improve quality and performance of R software

Today's computational researchers are expected to be highly proficient in using software to solve a wide range of problems ranging from processing large datasets to developing personalized treatment strategies from a growing range of options. Researchers are well versed in their own field, but may lack formal training and appropriate mentorship in software engineering principles. Two major themes not covered in most university coursework nor current literature are software testing and software optimization. Through a survey of all currently available Comprehensive R Archive Network packages, we show that reproducible and replicable software tests are frequently not available and that many packages do not appear to employ software performance and optimization tools and techniques. Through use of examples from an existing R package, we demonstrate powerful testing and optimization techniques that can improve the quality of any researcher's software.

Ten quick tips for getting the most scientific value out of numerical data

Most studies in the life sciences and other disciplines involve generating and analyzing numerical data of some type as the foundation for scientific findings. Working with numerical data involves multiple challenges. These include reproducible data acquisition, appropriate data storage, computationally correct data analysis, appropriate reporting and presentation of the results, and suitable data interpretation. Finding and correcting mistakes when analyzing and interpreting data can be frustrating and time-consuming. Presenting or publishing incorrect results is embarrassing but not uncommon. Particular sources of errors are inappropriate use of statistical methods and incorrect interpretation of data by software. To detect mistakes as early as possible, one should frequently check intermediate and final results for plausibility. Clearly documenting how quantities and results were obtained facilitates correcting mistakes. Properly understanding data is indispensable for reaching well-founded conclusions from experimental results. Units are needed to make sense of numbers, and uncertainty should be estimated to know how meaningful results are. Descriptive statistics and significance testing are useful tools for interpreting numerical results if applied correctly. However, blindly trusting in computed numbers can also be misleading, so it is worth thinking about how data should be summarized quantitatively to properly answer the question at hand. Finally, a suitable form of presentation is needed so that the data can properly support the interpretation and findings. By additionally sharing the relevant data, others can access, understand, and ultimately make use of the results. These quick tips are intended to provide guidelines for correctly interpreting, efficiently analyzing, and presenting numerical data in a useful way.

Lessons learnt on the analysis of large sequence data in animal genomics

The 'omics revolution has made a large amount of sequence data available to researchers and the industry. This has had a profound impact in the field of bioinformatics, stimulating unprecedented advancements in this discipline. Mostly, this is usually looked at from the perspective of human 'omics, in particular human genomics. Plant and animal genomics, however, have also been deeply influenced by next-generation sequencing technologies, with several genomics applications now popular among researchers and the breeding industry. Genomics tends to generate huge amounts of data, and genomic sequence data account for an increasing proportion of big data in biological sciences, due largely to decreasing sequencing and genotyping costs and to large-scale sequencing and resequencing projects. The analysis of big data poses a challenge to scientists, as data gathering currently takes place at a faster pace than does data processing and analysis, and the associated computational burden is increasingly taxing, making even simple manipulation, visualization and transferring of data a cumbersome operation. The time consumed by the processing and analysing of huge data sets may be at the expense of data quality assessment and critical interpretation. Additionally, when analysing lots of data, something is likely to go awry-the software may crash or stop-and it can be very frustrating to track the error. We herein review the most relevant issues related to tackling these challenges and problems, from the perspective of animal genomics, and provide researchers that lack extensive computing experience with guidelines that will help when processing large genomic data sets.

Ten quick tips for machine learning in computational biology

Machine learning has become a pivotal tool for many projects in computational biology, bioinformatics, and health informatics. Nevertheless, beginners and biomedical researchers often do not have enough experience to run a data mining project effectively, and therefore can follow incorrect practices, that may lead to common mistakes or over-optimistic results. With this review, we present ten quick tips to take advantage of machine learning in any computational biology context, by avoiding some common errors that we observed hundreds of times in multiple bioinformatics projects. We believe our ten suggestions can strongly help any machine learning practitioner to carry on a successful project in computational biology and related sciences.

Metaresearch for Evaluating Reproducibility in Ecology and Evolution

Recent replication projects in other disciplines have uncovered disturbingly low levels of reproducibility, suggesting that those research literatures may contain unverifiable claims. The conditions contributing to irreproducibility in other disciplines are also present in ecology. These include a large discrepancy between the proportion of "positive" or "significant" results and the average statistical power of empirical research, incomplete reporting of sampling stopping rules and results, journal policies that discourage replication studies, and a prevailing publish-or-perish research culture that encourages questionable research practices. We argue that these conditions constitute sufficient reason to systematically evaluate the reproducibility of the evidence base in ecology and evolution. In some cases, the direct replication of ecological research is difficult because of strong temporal and spatial dependencies, so here, we propose metaresearch projects that will provide proxy measures of reproducibility.

Robust cell tracking in epithelial tissues through identification of maximum common subgraphs

Tracking of cells in live-imaging microscopy videos of epithelial sheets is a powerful tool for investigating fundamental processes in embryonic development. Characterizing cell growth, proliferation, intercalation and apoptosis in epithelia helps us to understand how morphogenetic processes such as tissue invagination and extension are locally regulated and controlled. Accurate cell tracking requires correctly resolving cells entering or leaving the field of view between frames, cell neighbour exchanges, cell removals and cell divisions. However, current tracking methods for epithelial sheets are not robust to large morphogenetic deformations and require significant manual interventions. Here, we present a novel algorithm for epithelial cell tracking, exploiting the graph-theoretic concept of a 'maximum common subgraph' to track cells between frames of a video. Our algorithm does not require the adjustment of tissue-specific parameters, and scales in sub-quadratic time with tissue size. It does not rely on precise positional information, permitting large cell movements between frames and enabling tracking in datasets acquired at low temporal resolution due to experimental constraints such as phototoxicity. To demonstrate the method, we perform tracking on the Drosophila embryonic epidermis and compare cell-cell rearrangements to previous studies in other tissues. Our implementation is open source and generally applicable to epithelial tissues.

Where next for the reproducibility agenda in computational biology?

Background

The concept of reproducibility is a foundation of the scientific method. With the arrival of fast and powerful computers over the last few decades, there has been an explosion of results based on complex computational analyses and simulations. The reproducibility of these results has been addressed mainly in terms of exact replicability or numerical equivalence, ignoring the wider issue of the reproducibility of conclusions through equivalent, extended or alternative methods.

Results

We use case studies from our own research experience to illustrate how concepts of reproducibility might be applied in computational biology. Several fields have developed ‘minimum information’ checklists to support the full reporting of computational simulations, analyses and results, and standardised data formats and model description languages can facilitate the use of multiple systems to address the same research question. We note the importance of defining the key features of a result to be reproduced, and the expected agreement between original and subsequent results. Dynamic, updatable tools for publishing methods and results are becoming increasingly common, but sometimes come at the cost of clear communication. In general, the reproducibility of computational research is improving but would benefit from additional resources and incentives.

Conclusions

We conclude with a series of linked recommendations for improving reproducibility in computational biology through communication, policy, education and research practice. More reproducible research will lead to higher quality conclusions, deeper understanding and more valuable knowledge.

Computing Workflows for Biologists: A Roadmap

Extremely large datasets have become routine in biology. However, performing a computational analysis of a large dataset can be overwhelming, especially for novices. Here, we present a step-by-step guide to computing workflows with the biologist end-user in mind. Starting from a foundation of sound data management practices, we make specific recommendations on how to approach and perform computational analyses of large datasets, with a view to enabling sound, reproducible biological research.

A Framework for Global Collaborative Data Management for Malaria Research

Data generated during the course of research activities carried out by the International Centers of Excellence for Malaria Research (ICEMR) is heterogeneous, large, and multi-scaled. The complexity of federated and global data operations and the diverse uses planned for the data pose tremendous challenges and opportunities for collaborative research. In this article, we present the foundational principles for data management across the ICEMR Program, the logistics associated with multiple aspects of the data life cycle, and describe a pilot centralized web information system created in PlasmoDB to query a subset of this data. The paradigm proposed as a solution for the data operations in the ICEMR Program is widely applicable to large, multifaceted research projects, and could be reproduced in other contexts that require sophisticated data management.

Incorporating computational resources in a cancer research program

Recent technological advances have transformed cancer genetics research. These advances have served as the basis for the generation of a number of richly annotated datasets relevant to the cancer geneticist. In addition, many of these technologies are now within reach of smaller laboratories to answer specific biological questions. Thus, one of the most pressing issues facing an experimental cancer biology research program in genetics is incorporating data from multiple sources to annotate, visualize, and analyze the system under study. Fortunately, there are several computational resources to aid in this process. However, a significant effort is required to adapt a molecular biology-based research program to take advantage of these datasets. Here, we discuss the lessons learned in our laboratory and share several recommendations to make this transition effective. This article is not meant to be a comprehensive evaluation of all the available resources, but rather highlight those that we have incorporated into our laboratory and how to choose the most appropriate ones for your research program.

Improving functional magnetic resonance imaging reproducibility

BACKGROUND

The ability to replicate an entire experiment is crucial to the scientific method. With the development of more and more complex paradigms, and the variety of analysis techniques available, fMRI studies are becoming harder to reproduce.

RESULTS

In this article, we aim to provide practical advice to fMRI researchers not versed in computing, in order to make studies more reproducible. All of these steps require researchers to move towards a more open science, in which all aspects of the experimental method are documented and shared.

CONCLUSION

Only by sharing experiments, data, metadata, derived data and analysis workflows will neuroimaging establish itself as a true data science.

SlideToolkit: an assistive toolset for the histological quantification of whole slide images

The demand for accurate and reproducible phenotyping of a disease trait increases with the rising number of biobanks and genome wide association studies. Detailed analysis of histology is a powerful way of phenotyping human tissues. Nonetheless, purely visual assessment of histological slides is time-consuming and liable to sampling variation and optical illusions and thereby observer variation, and external validation may be cumbersome. Therefore, within our own biobank, computerized quantification of digitized histological slides is often preferred as a more precise and reproducible, and sometimes more sensitive approach. Relatively few free toolkits are, however, available for fully digitized microscopic slides, usually known as whole slides images. In order to comply with this need, we developed the slideToolkit as a fast method to handle large quantities of low contrast whole slides images using advanced cell detecting algorithms. The slideToolkit has been developed for modern personal computers and high-performance clusters (HPCs) and is available as an open-source project on github.com. We here illustrate the power of slideToolkit by a repeated measurement of 303 digital slides containing CD3 stained (DAB) abdominal aortic aneurysm tissue from a tissue biobank. Our workflow consists of four consecutive steps. In the first step (acquisition), whole slide images are collected and converted to TIFF files. In the second step (preparation), files are organized. The third step (tiles), creates multiple manageable tiles to count. In the fourth step (analysis), tissue is analyzed and results are stored in a data set. Using this method, two consecutive measurements of 303 slides showed an intraclass correlation of 0.99. In conclusion, slideToolkit provides a free, powerful and versatile collection of tools for automated feature analysis of whole slide images to create reproducible and meaningful phenotypic data sets.

Polyglot programming in applications used for genetic data analysis

Applications used for the analysis of genetic data process large volumes of data with complex algorithms. High performance, flexibility, and a user interface with a web browser are required by these solutions, which can be achieved by using multiple programming languages. In this study, I developed a freely available framework for building software to analyze genetic data, which uses C++, Python, JavaScript, and several libraries. This system was used to build a number of genetic data processing applications and it reduced the time and costs of development.