The Crisis of Reproducibility, the Denominator Problem and the Scientific Role of Multi-scale Modeling

A design specification for Critical Illness Digital Twins (CIDTs) to cure sepsis: responding to the National Academies of Sciences, Engineering and Medicine Report "Foundational Research Gaps and Future Directions for Digital Twins"

On December 15, 2023, The National Academies of Sciences, Engineering and Medicine (NASEM) released a report entitled: "Foundational Research Gaps and Future Directions for Digital Twins." The ostensible purpose of this report was to bring some structure to the burgeoning field of digital twins by providing a working definition and a series of research challenges that need to be addressed to allow this technology to fulfill its full potential. In the work presented herein we focus on five specific findings from the NASEM Report: 1) definition of a Digital Twin, 2) using "fit-for-purpose" guidance, 3) developing novel approaches to Verification, Validation and Uncertainty Quantification (VVUQ) of Digital Twins, 4) incorporating control as an explicit purpose for a Digital Twin and 5) using a Digital Twin to guide data collection and sensor development, and describe how these findings are addressed through the design specifications for a Critical Illness Digital Twin (CIDT) aimed at curing sepsis.

Efficacy of machine learning to identify clinical factors influencing levothyroxine dosage after total thyroidectomy

BACKGROUND

We employed Machine Learning (ML) to evaluate potential additional clinical factors influencing replacement dosage requirements of levothyroxine.

METHOD

This was a retrospective study of patients who underwent total or completion thyroidectomy with benign pathology. Patients who achieved an euthyroid state were included in three different ML models.

RESULTS

Of the 487 patients included, mean age was 54.1 ± 14.1 years, 86.0% were females, 39.0% were White, 53.0% Black, 2.7% Hispanic, 1.4% Asian, and 3.9% Other. The Extreme Gradient Boosting (XGBoost) model achieved the highest accuracy at 61.0% in predicting adequate dosage compared to 47.0% based on 1.6 mcg/kg/day (p < 0.05). The Poisson regression indicated non-Caucasian race (p < 0.05), routine alcohol use (estimate = 0.03, p = 0.02), and osteoarthritis (estimate = -0.10, p < 0.001) in addition to known factors such as age (estimate = -0.003, p < 0.001), sex (female, estimate = -0.06, p < 0.001), and weight (estimate = 0.01, p < 0.001) were associated with the dosing of levothyroxine.

CONCLUSIONS

Along with weight, sex, age, and BMI, ML algorithms indicated that race, ethnicity, lifestyle and comorbidity factors also may impact levothyroxine dosing in post-thyroidectomy patients with benign conditions.

Mice in translational neuroscience: What R we doing?

Animal models play a pivotal role in translational neuroscience but recurrent problems in data collection, analyses, and interpretation, lack of biomarkers, and a tendency to over-reliance on mice have marred neuroscience progress, leading to one of the highest attrition rates in drug translation. Global initiatives to improve reproducibility and model selection are being implemented. Notwithstanding, mice are still the preferred animal species to model human brain disorders even when the translation has been shown to be limited. Non-human primates are better positioned to provide relevant translational information because of their higher brain complexity and homology to humans. Among others, lack of resources and formal training, strict legislation, and ethical issues may impede broad access to large animals. We propose that instead of increasingly restrictive legislation, more resources for training, education, husbandry, and data sharing are urgently needed. The creation of multidisciplinary teams, in which veterinarians need to play a key role, would be critical to improve translational efficiency. Furthermore, it is not usually acknowledged by researchers and regulators the value of comparative studies in lower species, that are instrumental in toxicology, target identification, and mechanistic studies. Overall, we highlight here the need for a conceptual shift in neuroscience research and policies to reach the patients.

Drug Development Digital Twins for Drug Discovery, Testing and Repurposing: A Schema for Requirements and Development

There has been a great deal of interest in the concept, development and implementation of medical digital twins. This interest has led to wide ranging perceptions of what constitutes a medical digital twin. This Perspectives article will provide 1) a description of fundamental features of industrial digital twins, the source of the digital twin concept, 2) aspects of biology that challenge the implementation of medical digital twins, 3) a schematic program of how a specific medical digital twin project could be defined, and 4) an example description within that schematic program for a specific type of medical digital twin intended for drug discovery, testing and repurposing, the Drug Development Digital Twin (DDDT).

The Annoying Flaws of Gerontological Research

Gerontological research has accelerated dramatically in the last few decades. However, despite increased public interest, federal funding, an army of researchers, and many notable discoveries and high-impact publications, the goal of achieving even a modest extension of human lifespan seems to be as far away as ever or, at best, remains within the realm of lifestyle and diet optimization efforts. Humanity has already benefited from a lifespan revolution in the first half of 20^th Century, which was brought about by improved sanitation and hygiene, clean water, and our successful war on infectious diseases. Thanks to all these developments, in which gerontologists played no part, our expected lifespan increased by about 40% and our primary causes of death decidedly shifted from extrinsic to intrinsic causality. The next step is not that simple as it implies tackling intrinsic mechanisms of aging, and the lack of working human-specific antiaging solutions likely stems from flawed research strategies.

The Use of Artificial Neural Networks to Forecast the Behavior of Agent-Based Models of Pathophysiology: An Example Utilizing an Agent-Based Model of Sepsis

Introduction: Disease states are being characterized at finer and finer levels of resolution via biomarker or gene expression profiles, while at the same time. Machine learning (ML) is increasingly used to analyze and potentially classify or predict the behavior of biological systems based on such characterization. As ML applications are extremely data-intensive, given the relative sparsity of biomedical data sets ML training of artificial neural networks (ANNs) often require the use of synthetic training data. Agent-based models (ABMs) that incorporate known biological mechanisms and their associated stochastic properties are a potential means of generating synthetic data. Herein we present an example of ML used to train an artificial neural network (ANN) as a surrogate system used to predict the time evolution of an ABM focusing on the clinical condition of sepsis. Methods: The disease trajectories for clinical sepsis, in terms of temporal cytokine and phenotypic dynamics, can be interpreted as a random dynamical system. The Innate Immune Response Agent-based Model (IIRABM) is a well-established model that utilizes known cellular and molecular rules to simulate disease trajectories corresponding to clinical sepsis. We have utilized two distinct neural network architectures, Long Short-Term Memory and Multi-Layer Perceptron, to take a time sequence of five measurements of eleven IIRABM simulated serum cytokine concentrations as input and to return both the future cytokine trajectories as well as an aggregate metric representing the patient's state of health. Results: The ANNs predicted model trajectories with the expected amount of error, due to stochasticity in the simulation, and recognizing that the mapping from a specific cytokine profile to a state-of-health is not unique. The Multi-Layer Perceptron neural network, generated predictions with a more accurate forecasted trajectory cone. Discussion: This work serves as a proof-of-concept for the use of ANNs to predict disease progression in sepsis as represented by an ABM. The findings demonstrate that multicellular systems with intrinsic stochasticity can be approximated with an ANN, but that forecasting a specific trajectory of the system requires sequential updating of the system state to provide a rolling forecast horizon.

Enhancing Interrogation of Skeletal Muscle Samples for Informative Quantitative Data

Careful quantitative analysis of histological preparations of muscle samples is crucial to accurate investigation of myopathies in man and of interpretation of data from animals subjected to experimental or potentially therapeutic treatments. Protocols for measuring cell numbers are subject to problems arising from biases associated with preparative and analytical techniques. Prominent among these is the effect of polarized structure of skeletal muscle on sampling bias. It is also common in this tissue to collect data as ratios to convenient reference dominators, the fundamental bases of which are ill-defined, or unrecognized or not accurately assessable. Use of such 'floating' denominators raises a barrier to estimation of the absolute values that assume practical importance in medical research, where accurate comparison between different scenarios in different species is essential to the aim of translating preclinical research findings in animal models to clinical utility in Homo sapiens.This review identifies some of the underappreciated problems with current morphometric practice, some of which are exacerbated in skeletal muscle, and evaluates the extent of their intrusiveness into the of building an objective, accurate, picture of the structure of the muscle sample. It also contains recommendations for eliminating or at least minimizing these problems. Principal among these, would be the use of stereological procedures to avoid the substantial counting biases arising from inter-procedure differences in object size and section thickness.Attention is also drawn to the distortions of interpretation arising from use of undefined or inappropriate denominators.

Comparative Computational Modeling of the Bat and Human Immune Response to Viral Infection with the Comparative Biology Immune Agent Based Model

Given the impact of pandemics due to viruses of bat origin, there is increasing interest in comparative investigation into the differences between bat and human immune responses. The practice of comparative biology can be enhanced by computational methods used for dynamic knowledge representation to visualize and interrogate the putative differences between the two systems. We present an agent based model that encompasses and bridges differences between bat and human responses to viral infection: the comparative biology immune agent based model, or CBIABM. The CBIABM examines differences in innate immune mechanisms between bats and humans, specifically regarding inflammasome activity and type 1 interferon dynamics, in terms of tolerance to viral infection. Simulation experiments with the CBIABM demonstrate the efficacy of bat-related features in conferring viral tolerance and also suggest a crucial role for endothelial inflammasome activity as a mechanism for bat systemic viral tolerance and affecting the severity of disease in human viral infections. We hope that this initial study will inspire additional comparative modeling projects to link, compare, and contrast immunological functions shared across different species, and in so doing, provide insight and aid in preparation for future viral pandemics of zoonotic origin.

Truth and truthiness: evidence, experience and clinical judgement in surgery

The scientific basis of surgery, derived through observation and experiment, does not fully account for surgical expertise gained through experience. The evidence that supports surgical practice is limited, elusive, and subject to bias. Surgical judgment requires not only explicit, fact-based, knowledge but also tacit forms of understanding that are not easily teachable or testable.

Stakeholders' perspectives on research integrity training practices: a qualitative study

BACKGROUND

Even though research integrity (RI) training programs have been developed in the last decades, it is argued that current training practices are not always able to increase RI-related awareness within the scientific community. Defining and understanding the capacities and lacunas of existing RI training are becoming extremely important for developing up-to-date educational practices to tackle present-day challenges. Recommendations on how to implement RI education have been primarily made by selected people with specific RI-related expertise. Those recommendations were developed mainly without consulting a broader audience with no specific RI expertise. Moreover, the academic literature lacks qualitative studies on RI training practices. For these reasons, performing in-depth focus groups with non-RI expert stakeholders are of a primary necessity to understand and outline how RI education should be implemented.

METHODS

In this qualitative analysis, different focus groups were conducted to examine stakeholders' perspectives on RI training practices. Five stakeholders' groups, namely publishers and peer reviewers, researchers on RI, RI trainers, PhDs and postdoctoral researchers, and research administrators working within academia, have been identified to have a broader overview of state of the art.

RESULTS

A total of 39 participants participated in five focus group sessions. Eight training-related themes were highlighted during the focus group discussions. The training goals, timing and frequency, customisation, format and teaching approach, mentoring, compulsoriness, certification and evaluation, and RI-related responsibilities were discussed. Although confirming what was already proposed by research integrity experts in terms of timing, frequency, duration, and target audience in organising RI education, participants proposed other possible implementations strategies concerning the teaching approach, researchers' obligations, and development an evaluation-certification system.

CONCLUSIONS

This research aims to be a starting point for a better understanding of necessary, definitive, and consistent ways of structuring RI education. The research gives an overview of what has to be considered needed in planning RI training sessions regarding objectives, organisation, and teaching approach.

Utilizing the Heterogeneity of Clinical Data for Model Refinement and Rule Discovery Through the Application of Genetic Algorithms to Calibrate a High-Dimensional Agent-Based Model of Systemic Inflammation

Introduction: Accounting for biological heterogeneity represents one of the greatest challenges in biomedical research. Dynamic computational and mathematical models can be used to enhance the study and understanding of biological systems, but traditional methods for calibration and validation commonly do not account for the heterogeneity of biological data, which may result in overfitting and brittleness of these models. Herein we propose a machine learning approach that utilizes genetic algorithms (GAs) to calibrate and refine an agent-based model (ABM) of acute systemic inflammation, with a focus on accounting for the heterogeneity seen in a clinical data set, thereby avoiding overfitting and increasing the robustness and potential generalizability of the underlying simulation model. Methods: Agent-based modeling is a frequently used modeling method for multi-scale mechanistic modeling. However, the same properties that make ABMs well suited to representing biological systems also present significant challenges with respect to their construction and calibration, particularly with respect to the selection of potential mechanistic rules and the large number of associated free parameters. We have proposed that machine learning approaches (such as GAs) can be used to more effectively and efficiently deal with rule selection and parameter space characterization; the current work applies GAs to the challenge of calibrating a complex ABM to a specific data set, while preserving biological heterogeneity reflected in the range and variance of the data. This project uses a GA to augment the rule-set for a previously validated ABM of acute systemic inflammation, the Innate Immune Response ABM (IIRABM) to clinical time series data of systemic cytokine levels from a population of burn patients. The genome for the GA is a vector generated from the IIRABM's Model Rule Matrix (MRM), which is a matrix representation of not only the constants/parameters associated with the IIRABM's cytokine interaction rules, but also the existence of rules themselves. Capturing heterogeneity is accomplished by a fitness function that incorporates the sample value range ("error bars") of the clinical data. Results: The GA-enabled parameter space exploration resulted in a set of putative MRM rules and associated parameterizations which closely match the cytokine time course data used to design the fitness function. The number of non-zero elements in the MRM increases significantly as the model parameterizations evolve toward a fitness function minimum, transitioning from a sparse to a dense matrix. This results in a model structure that more closely resembles (at a superficial level) the structure of data generated by a standard differential gene expression experimental study. Conclusion: We present an HPC-enabled machine learning/evolutionary computing approach to calibrate a complex ABM to complex clinical data while preserving biological heterogeneity. The integration of machine learning, HPC, and multi-scale mechanistic modeling provides a pathway forward to more effectively representing the heterogeneity of clinical populations and their data.

Effect of chronic unpredictable mild stress on the expression profile of serotonin receptors in rats and mice: a meta-analysis

Chronic-stress-induced depression is recognized as a widespread public health concern. Selective serotonin reuptake inhibitors (SSRIs) have been the most common treatment for this illness. However, the role of 5-hydroxytryptamine (5-HT) receptor subtypes in stress-induced depression remains unclear. Evidence from Animal studies has reported a variety of results regarding the effects of chronic unpredictable mild stress (CUMS) on serotonin signaling pathways and 5-HT receptor subtypes. This divergence may rely on differences in protocols, methods, and studied pathways. Thus, the aim of this systematic review was to weigh the currently available findings regarding serotonin receptor changes in animal models of CUMS. Overall, our meta-analysis results showed the association of altered expression of 5-HT_1A receptors in the frontal cortex and 5-HT_2A receptors both in the whole cortex and the hypothalamus of rats following CUMS. Moreover, by using a qualitative-structured analysis and the application of risk-of-bias tools, we identified possible sources of data variation between the studied literature, which should be taken into account in future animal studies of chronic-stress induced depression.

Highly Versatile Cloud-Based Automation Solution for the Remote Design and Execution of Experiment Protocols during the COVID-19 Pandemic

There is an urgent need to accelerate the development and validation of both diagnostics and vaccines for COVID-19. These priorities are challenging both public and private sector research groups around the world and have shone a spotlight on both existing bottlenecks in the research workflows involved as well as on the implications of having to do much of this work remotely because of enforced social distancing and lockdown measures. The ability to respond quickly to rapidly evolving events, coupled with an emerging understanding of the disease and its pathology, as well as different mutations of the virus, necessitates a highly flexible liquid-handling automation solution that is amenable to rapid switching between different assay workflows and processes to be exploited tactically as needed. In addition, the use of cloud-based software imparts a unique benefit in enabling multiple research groups and remote technical staff around the world to have ready access to the same protocols in real-time without delays, down to the required level of detail, sharing methods and data (for example, in faster clinical trials). Informed by a recent use case, this article explores these issues alongside the recent development and deployment of an automation solution, whose unique approach in terms of both its cloud-native software and its highly modular hardware aligns especially well with achieving the challenge set by this new frontier in the bioanalytical laboratory.

CaliPro: A Calibration Protocol That Utilizes Parameter Density Estimation to Explore Parameter Space and Calibrate Complex Biological Models

Introduction

Mathematical and computational modeling have a long history of uncovering mechanisms and making predictions for biological systems. However, to create a model that can provide relevant quantitative predictions, models must first be calibrated by recapitulating existing biological datasets from that system. Current calibration approaches may not be appropriate for complex biological models because: 1) many attempt to recapitulate only a single aspect of the experimental data (such as a median trend) or 2) Bayesian techniques require specification of parameter priors and likelihoods to experimental data that cannot always be confidently assigned. A new calibration protocol is needed to calibrate complex models when current approaches fall short.

Methods

Herein, we develop CaliPro, an iterative, model-agnostic calibration protocol that utilizes parameter density estimation to refine parameter space and calibrate to temporal biological datasets. An important aspect of CaliPro is the user-defined pass set definition, which specifies how the model might successfully recapitulate experimental data. We define the appropriate settings to use CaliPro.

Results

We illustrate the usefulness of CaliPro through four examples including predator-prey, infectious disease transmission, and immune response models. We show that CaliPro works well for both deterministic, continuous model structures as well as stochastic, discrete models and illustrate that CaliPro can work across diverse calibration goals.

Conclusions

We present CaliPro, a new method for calibrating complex biological models to a range of experimental outcomes. In addition to expediting calibration, CaliPro may be useful in already calibrated parameter spaces to target and isolate specific model behavior for further analysis.

Interlaboratory comparison of humic substances compositional space as measured by Fourier transform ion cyclotron resonance mass spectrometry (IUPAC Technical Report)

Interlaboratory comparison on the determination of the molecular composition of humic substances (HS) was undertaken in the framework of IUPAC project 2016-015-2-600. The analysis was conducted using high resolution mass spectrometry, nominally, Fourier transform ion cyclotron resonance mass spectrometry (FTICR MS) with electrospray ionization. Six samples of HS from freshwater, soil, and leonardite were used for this study, including one sample of humic acids (HA) from coal (leonardite), two samples of soil HA (the sod-podzolic soil and chernozem), two samples of soil fulvic acids (FA) (the sod-podzolic soil and chernozem), and one sample of freshwater humic acids (the Suwannee River). The samples were analyzed on five different FTICR MS instruments using the routine conditions applied in each participating laboratory. The results were collected as mass lists, which were further assigned formulae for the determination of molecular composition. The similarity of the obtained data was evaluated using appropriate statistical metrics. The results have shown that direct comparison of discrete stoichiometries assigned to the mass lists obtained by the different laboratories yielded poor results with low values of the Jaccard similarity score – not exceeding 0.56 (not more than 56 % of the similar peaks). The least similarity was observed for the aromatics-rich HA samples from leonardite (coal) and the chernozem soil, which might be connected to difficulties in their ionization. The reliable similarity among the data obtained in this intercomparison study was achieved only by transforming a singular point (stoichiometry) in van Krevelen diagram into a sizeable pixel (a number of closely located stoichiometries), which can be calculated from the population density distribution. The conclusion was made that, so far, these are descriptors of occupation density distribution, which provide the metrics compliant with the data quality requirements, such as the reproducibility of the data measurements on different instruments.

Collecting and characterizing existing and freely accessible research integrity educational resources

In addition to effective training practices, well-structured educational resources are important for developing successful research integrity training programs. A considerable amount of educational material has been developed in the last years, but there is a necessity to find better ways to assess and categorize the already existing resources. We collected 237 freely available online RI educational resources with the aim to describe them in as much detail as possible using a set of well-defined criteria. We developed a grid that gives a full description, based on our 21 criteria, for each collected resource. Mainly videos and online RI training are present in our collection. Worldwide, resources are mainly from the US, whereas in Europe mainly from the UK. In the majority of the cases, the educational resources are not customized, presenting the big three (falsification, fabrication, and plagiarism) as the most addressed topics. Making RI educational resources easily accessible might help to increase awareness about the topic. Moreover, the characterization we provide might help researchers and students to deal with daily RI-related issues, to look for the right tool at the right time, and might help institutions and trainers to develop new trainings without the need to develop new tools.Abbreviations: CITI: Collaborative Institutional Training Initiative; COPE: Committee on Publication Ethics; ENERI: European Network of Research Ethics and Research Integrity; ENRIO: the European Network of Research Integrity Offices; EU: European Union; NIH: National Institutes of Health; NSF: National Science Foundation; NRIN: the Netherlands Research Integrity Network; ORI: the Office of Research Integrity; PPT: powerpoint; QRP: questionable research practice; RI: research integrity; RCR: responsible conduct of research.

Numerical Optimal Control of HIV Transmission in Octave/MATLAB

We provide easy and readable GNU Octave/MATLAB code for the simulation of mathematical models described by ordinary differential equations and for the solution of optimal control problems through Pontryagin’s maximum principle. For that, we consider a normalized HIV/AIDS transmission dynamics model based on the one proposed in our recent contribution (Silva, C.J.; Torres, D.F.M. A SICA compartmental model in epidemiology with application to HIV/AIDS in Cape Verde. Ecol. Complex. 2017, 30, 70–75), given by a system of four ordinary differential equations. An HIV initial value problem is solved numerically using the ode45 GNU Octave function and three standard methods implemented by us in Octave/MATLAB: Euler method and second-order and fourth-order Runge–Kutta methods. Afterwards, a control function is introduced into the normalized HIV model and an optimal control problem is formulated, where the goal is to find the optimal HIV prevention strategy that maximizes the fraction of uninfected HIV individuals with the least HIV new infections and cost associated with the control measures. The optimal control problem is characterized analytically using the Pontryagin Maximum Principle, and the extremals are computed numerically by implementing a forward-backward fourth-order Runge–Kutta method. Complete algorithms, for both uncontrolled initial value and optimal control problems, developed under the free GNU Octave software and compatible with MATLAB are provided along the article.

The Search for Clinically Useful Biomarkers of Complex Disease: A Data Analysis Perspective

Unmet clinical diagnostic needs exist for many complex diseases, which (it is hoped) will be solved by the discovery of metabolomics biomarkers. However, at present, no diagnostic tests based on metabolomics have yet been introduced to the clinic. This review is presented as a research perspective on how data analysis methods in metabolomics biomarker discovery may contribute to the failure of biomarker studies and suggests how such failures might be mitigated. The study design and data pretreatment steps are reviewed briefly in this context, and the actual data analysis step is examined more closely.