Topology observing 3D device reconstruction from continuous-sweep limited angle fluoroscopy

BACKGROUND

Minimally invasive procedures usually require navigating a microcatheter and guidewire through endoluminal structures such as blood vessels and airways to sites of the disease. For numerous clinical applications, two-dimensional (2D) fluoroscopy is the primary modality used for real-time image guidance during navigation. However, 2D imaging can pose challenges for navigation in complex structures. Real-time 3D visualization of devices within the anatomic context could provide considerable benefits for these procedures. Continuous-sweep limited angle (CLA) fluoroscopy has recently been proposed to provide a compromise between conventional rotational 3D acquisitions and real-time fluoroscopy.

PURPOSE

The purpose of this work was to develop and evaluate a noniterative 3D device reconstruction approach for CLA fluoroscopy acquisitions, which takes into account endoluminal topology to avoid impossible paths between disconnected branches.

METHODS

The algorithm relies on a static 3D roadmap (RM) of vessels or airways, which may be generated from conventional cone beam CT (CBCT) acquisitions prior to navigation. The RM is converted to a graph representation describing its topology. During catheter navigation, the device is segmented from the live 2D projection images using a deep learning approach from which the centerlines are extracted. Rays from the focal spot to detector pixels representing 2D device points are identified and intersections with the RM are computed. Based on the RM graph, a subset of line segments is selected as candidates to exclude device paths through disconnected branches of the RM. Depth localization for each point along the device is then performed by finding the point closest to the previous 3D reconstruction along the candidate segments. This process is repeated as the projection angle changes for each CLA image frame. The approach was evaluated in a phantom study in which a catheter and guidewire were navigated along five pathways within a complex vessel phantom. The result was compared to static cCBCT acquisitions of the device in the final position.

RESULTS

The average root mean squared 3D distance between CLA reconstruction and reference centerline was1.87 ± 0.30 $1.87 \pm 0.30$ mm. The Euclidean distance at the device tip was2.92 ± 2.35 $2.92 \pm 2.35$ mm. The correct pathway was identified during reconstruction in100 % $100\%$ of frames (n = 1475 $n=1475$ ). The percentage of 3D device points reconstructed inside the 3D roadmap was91.83 ± 2.52 % $91.83 \pm 2.52\%$ with an average distance of0.62 ± 0.30 $0.62 \pm 0.30$ mm between the device points outside the roadmap and the nearest point within the roadmap.

CONCLUSIONS

This study demonstrates the feasibility of reconstructing curvilinear devices such as catheters and guidewires during endoluminal procedures including intravascular and transbronchial interventions using a noniterative reconstruction approach for CLA fluoroscopy. This approach could improve device navigation in cases where the structure of vessels or airways is complex and includes overlapping branches.

MLL1 regulates cytokine-driven cell migration and metastasis

Cell migration is a critical contributor to metastasis. Cytokine production and its role in cancer cell migration have been traditionally associated with immune cells. We find that the histone methyltransferase Mixed-Lineage Leukemia 1 (MLL1) controls 3D cell migration via cytokines, IL-6, IL-8, and TGF-β1, secreted by the cancer cells themselves. MLL1, with its scaffold protein Menin, controls actin filament assembly via the IL-6/8/pSTAT3/Arp3 axis and myosin contractility via the TGF-β1/Gli2/ROCK1/2/pMLC2 axis, which together regulate dynamic protrusion generation and 3D cell migration. MLL1 also regulates cell proliferation via mitosis-based and cell cycle-related pathways. Mice bearing orthotopic MLL1-depleted tumors exhibit decreased lung metastatic burden and longer survival. MLL1 depletion leads to lower metastatic burden even when controlling for the difference in primary tumor growth rates. Combining MLL1-Menin inhibitor with paclitaxel abrogates tumor growth and metastasis, including preexistent metastasis. These results establish MLL1 as a potent regulator of cell migration and highlight the potential of targeting MLL1 in patients with metastatic disease.

Self-replicating artificial neural networks give rise to universal evolutionary dynamics

In evolutionary models, mutations are exogenously introduced by the modeler, rather than endogenously introduced by the replicator itself. We present a new deep-learning based computational model, the self-replicating artificial neural network (SeRANN). We train it to (i) copy its own genotype, like a biological organism, which introduces endogenous spontaneous mutations; and (ii) simultaneously perform a classification task that determines its fertility. Evolving 1,000 SeRANNs for 6,000 generations, we observed various evolutionary phenomena such as adaptation, clonal interference, epistasis, and evolution of both the mutation rate and the distribution of fitness effects of new mutations. Our results demonstrate that universal evolutionary phenomena can naturally emerge in a self-replicator model when both selection and mutation are implicit and endogenous. We therefore suggest that SeRANN can be applied to explore and test various evolutionary dynamics and hypotheses.

Background Subtraction Angiography with Deep Learning Using Multi-frame Spatiotemporal Angiographic Input

Catheter Digital Subtraction Angiography (DSA) is markedly degraded by all voluntary, respiratory, or cardiac motion artifact that occurs during the exam acquisition. Prior efforts directed toward improving DSA images with machine learning have focused on extracting vessels from individual, isolated 2D angiographic frames. In this work, we introduce improved 2D + t deep learning models that leverage the rich temporal information in angiographic timeseries. A total of 516 cerebral angiograms were collected with 8784 individual series. We utilized feature-based computer vision algorithms to separate the database into "motionless" and "motion-degraded" subsets. Motion measured from the "motion degraded" category was then used to create a realistic, but synthetic, motion-augmented dataset suitable for training 2D U-Net, 3D U-Net, SegResNet, and UNETR models. Quantitative results on a hold-out test set demonstrate that the 3D U-Net outperforms competing 2D U-Net architectures, with substantially reduced motion artifacts when compared to DSA. In comparison to single-frame 2D U-Net, the 3D U-Net utilizing 16 input frames achieves a reduced RMSE (35.77 ± 15.02 vs 23.14 ± 9.56, p < 0.0001; mean ± std dev) and an improved Multi-Scale SSIM (0.86 ± 0.08 vs 0.93 ± 0.05, p < 0.0001). The 3D U-Net also performs favorably in comparison to alternative convolutional and transformer-based architectures (U-Net RMSE 23.20 ± 7.55 vs SegResNet 23.99 ± 7.81, p < 0.0001, and UNETR 25.42 ± 7.79, p < 0.0001, mean ± std dev). These results demonstrate that multi-frame temporal information can boost performance of motion-resistant Background Subtraction Deep Learning algorithms, and we have presented a neuroangiography domain-specific synthetic affine motion augmentation pipeline that can be utilized to generate suitable datasets for supervised training of 3D (2d + t) architectures.

High Accuracy Protein Identification: Fusion of Solid-State Nanopore Sensing and Machine Learning

Proteins are arguably one of the most important class of biomarkers for health diagnostic purposes. Label-free solid-state nanopore sensing is a versatile technique for sensing and analyzing biomolecules such as proteins at single-molecule level. While molecular-level information on size, shape, and charge of proteins can be assessed by nanopores, the identification of proteins with comparable sizes remains a challenge. Here, solid-state nanopore sensing is combined with machine learning to address this challenge. The translocations of four similarly sized proteins is assessed using amplifiers with bandwidths (BWs) of 100 kHz and 10 MHz, the highest bandwidth reported for protein sensing, using nanopores fabricated in <10 nm thick silicon nitride membranes. F-values of up to 65.9% and 83.2% (without clustering of the protein signals) are achieved with 100 kHz and 10 MHz BW measurements, respectively, for identification of the four proteins. The accuracy of protein identification is further enhanced by classifying the signals into different clusters based on signal attributes, with F-value and specificity of up to 88.7% and 96.4%, respectively, for combinations of four proteins. The combined use of high bandwidth instruments, advanced clustering and machine learning methods allows label-free identification of proteins with high accuracy.

A marine cryptochrome with an inverse photo-oligomerization mechanism

Cryptochromes (CRYs) are a structurally conserved but functionally diverse family of proteins that can confer unique sensory properties to organisms. In the marine bristle worm Platynereis dumerilii, its light receptive cryptochrome L-CRY (PdLCry) allows the animal to discriminate between sunlight and moonlight, an important requirement for synchronizing its lunar cycle-dependent mass spawning. Using cryo-electron microscopy, we show that in the dark, PdLCry adopts a dimer arrangement observed neither in plant nor insect CRYs. Intense illumination disassembles the dimer into monomers. Structural and functional data suggest a mechanistic coupling between the light-sensing flavin adenine dinucleotide chromophore, the dimer interface, and the C-terminal tail helix, with a likely involvement of the phosphate binding loop. Taken together, our work establishes PdLCry as a CRY protein with inverse photo-oligomerization with respect to plant CRYs, and provides molecular insights into how this protein might help discriminating the different light intensities associated with sunlight and moonlight.

AmberTools

AmberTools is a free and open-source collection of programs used to set up, run, and analyze molecular simulations. The newer features contained within AmberTools23 are briefly described in this Application note.

A neurorobotics approach to behaviour selection based on human activity recognition

Behaviour selection has been an active research topic for robotics, in particular in the field of human-robot interaction. For a robot to interact autonomously and effectively with humans, the coupling between techniques for human activity recognition and robot behaviour selection is of paramount importance. However, most approaches to date consist of deterministic associations between the recognised activities and the robot behaviours, neglecting the uncertainty inherent to sequential predictions in real-time applications. In this paper, we address this gap by presenting an initial neurorobotics model that embeds, in a simulated robot, computational models of parts of the mammalian brain that resembles neurophysiological aspects of the basal ganglia-thalamus-cortex (BG-T-C) circuit, coupled with human activity recognition techniques. A robotics simulation environment was developed for assessing the model, where a mobile robot accomplished tasks by using behaviour selection in accordance with the activity being performed by the inhabitant of an intelligent home. Initial results revealed that the initial neurorobotics model is advantageous, especially considering the coupling between the most accurate activity recognition approaches and the computational models of more complex animals.

PepQuery2 democratizes public MS proteomics data for rapid peptide searching

We present PepQuery2, which leverages a new tandem mass spectrometry (MS/MS) data indexing approach to enable ultrafast, targeted identification of novel and known peptides in any local or publicly available MS proteomics datasets. The stand-alone version of PepQuery2 allows directly searching more than one billion indexed MS/MS spectra in the PepQueryDB or any public datasets from PRIDE, MassIVE, iProX, or jPOSTrepo, whereas the web version enables users to search datasets in PepQueryDB with a user-friendly interface. We demonstrate the utilities of PepQuery2 in a wide range of applications including detecting proteomic evidence for genomically predicted novel peptides, validating novel and known peptides identified using spectrum-centric database searching, prioritizing tumor-specific antigens, identifying missing proteins, and selecting proteotypic peptides for targeted proteomics experiments. By putting public MS proteomics data directly into the hands of scientists, PepQuery2 opens many new ways to transform these data into useful information for the broad research community.

Quantifying the movement, behaviour and environmental context of group-living animals using drones and computer vision

Methods for collecting animal behaviour data in natural environments, such as direct observation and biologging, are typically limited in spatiotemporal resolution, the number of animals that can be observed and information about animals' social and physical environments. Video imagery can capture rich information about animals and their environments, but image-based approaches are often impractical due to the challenges of processing large and complex multi-image datasets and transforming resulting data, such as animals' locations, into geographical coordinates. We demonstrate a new system for studying behaviour in the wild that uses drone-recorded videos and computer vision approaches to automatically track the location and body posture of free-roaming animals in georeferenced coordinates with high spatiotemporal resolution embedded in contemporaneous 3D landscape models of the surrounding area. We provide two worked examples in which we apply this approach to videos of gelada monkeys and multiple species of group-living African ungulates. We demonstrate how to track multiple animals simultaneously, classify individuals by species and age-sex class, estimate individuals' body postures (poses) and extract environmental features, including topography of the landscape and animal trails. By quantifying animal movement and posture while reconstructing a detailed 3D model of the landscape, our approach opens the door to studying the sensory ecology and decision-making of animals within their natural physical and social environments.

Correlated energy from radial density-energy relations

Here, we demonstrate that the radial distribution function can be mapped into a radial density-energy space and the relationship between the radial density and radial energy is linear for the ground and excited states of helium-like systems; the gradient of the resulting straight line delivers the energy of the state considered. To utilize this finding, a simple analytical expression for the total energy in terms of the density at the most probable nucleus-electron distance of the systems considered is derived using a fitting procedure.

An accessible and versatile deep learning-based sleep stage classifier

Manual sleep scoring for research purposes and for the diagnosis of sleep disorders is labor-intensive and often varies significantly between scorers, which has motivated many attempts to design automatic sleep stage classifiers. With the recent introduction of large, publicly available hand-scored polysomnographic data, and concomitant advances in machine learning methods to solve complex classification problems with supervised learning, the problem has received new attention, and a number of new classifiers that provide excellent accuracy. Most of these however have non-trivial barriers to use. We introduce the Greifswald Sleep Stage Classifier (GSSC), which is free, open source, and can be relatively easily installed and used on any moderately powered computer. In addition, the GSSC has been trained to perform well on a large variety of electrode set-ups, allowing high performance sleep staging with portable systems. The GSSC can also be readily integrated into brain-computer interfaces for real-time inference. These innovations were achieved while simultaneously reaching a level of accuracy equal to, or exceeding, recent state of the art classifiers and human experts, making the GSSC an excellent choice for researchers in need of reliable, automatic sleep staging.

Sign language recognition by means of common spatial patterns: An analysis

Currently there are around 466 million hard of hearing people and this amount is expected to grow in the coming years. Despite the efforts that have been made, there is a communication barrier between deaf and hard of hearing signers and non-signers in environments without an interpreter. Different approaches have been developed lately to try to deal with this issue. In this work, we present an Argentinian Sign Language (LSA) recognition system which uses hand landmarks extracted from videos of the LSA64 dataset in order to distinguish between different signs. Different features are extracted from the signals created with the hand landmarks values, which are first transformed by the Common Spatial Patterns (CSP) algorithm. CSP is a dimensionality reduction algorithm and it has been widely used for EEG systems. The features extracted from the transformed signals have been then used to feed different classifiers, such as Random Forest (RF), K-Nearest Neighbors (KNN) or Multilayer Perceptron (MLP). Several experiments have been performed from which promising results have been obtained, achieving accuracy values between 0.90 and 0.95 on a set of 42 signs.

An information-rich sampling technique over spatio-temporal CNN for classification of human actions in videos

We propose a novel video sampling scheme for human action recognition in videos, using Gaussian Weighing Function. Traditionally in deep learning-based human activity recognition approaches, either a few random frames or every k ^{t h} frame of the video is considered for training the 3D CNN, where k is a small positive integer, like 4, 5, or 6. This kind of sampling reduces the volume of the input data, which speeds-up the training network and also avoids overfitting to some extent, thus enhancing the performance of the 3D CNN model. In the proposed video sampling technique, consecutive k frames of a video are aggregated into a single frame by computing a Gaussian-weighted summation of the k frames. The resulting frame preserves the information in a better way than the conventional approaches and experimentally shown to perform better. In this paper, a 3-Dimensional deep CNN is proposed to extract the spatio-temporal features and follows Long Short-Term Memory (LSTM) to recognize human actions. The proposed 3D CNN architecture is capable of handling the videos where the camera is placed at a distance from the performer. Experiments are performed with KTH, WEIZMANN, and CASIA-B Human Activity and Gait datasets, whereby it is shown to outperform state-of-the-art deep learning based techniques. We achieve 95.78%, 95.27%, and 95.27% over the KTH, WEIZMANN, and CASIA-B human action and gait recognition datasets, respectively.

Leveraging Parameter Dependencies in High-Field Asymmetric Waveform Ion-Mobility Spectrometry and Size Exclusion Chromatography for Proteome-wide Cross-Linking Mass Spectrometry

Ion-mobility spectrometry shows great promise to tackle analytically challenging research questions by adding another separation dimension to liquid chromatography-mass spectrometry. The understanding of how analyte properties influence ion mobility has increased through recent studies, but no clear rationale for the design of customized experimental settings has emerged. Here, we leverage machine learning to deepen our understanding of field asymmetric waveform ion-mobility spectrometry for the analysis of cross-linked peptides. Knowing that predominantly m/z and then the size and charge state of an analyte influence the separation, we found ideal compensation voltages correlating with the size exclusion chromatography fraction number. The effect of this relationship on the analytical depth can be substantial as exploiting it allowed us to almost double unique residue pair detections in a proteome-wide cross-linking experiment. Other applications involving liquid- and gas-phase separation may also benefit from considering such parameter dependencies.

Predicting the zoonotic capacity of mammals to transmit SARS-CoV-2

Back and forth transmission of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) between humans and animals will establish wild reservoirs of virus that endanger long-term efforts to control COVID-19 in people and to protect vulnerable animal populations. Better targeting surveillance and laboratory experiments to validate zoonotic potential requires predicting high-risk host species. A major bottleneck to this effort is the few species with available sequences for angiotensin-converting enzyme 2 receptor, a key receptor required for viral cell entry. We overcome this bottleneck by combining species' ecological and biological traits with three-dimensional modelling of host-virus protein-protein interactions using machine learning. This approach enables predictions about the zoonotic capacity of SARS-CoV-2 for greater than 5000 mammals-an order of magnitude more species than previously possible. Our predictions are strongly corroborated by in vivo studies. The predicted zoonotic capacity and proximity to humans suggest enhanced transmission risk from several common mammals, and priority areas of geographic overlap between these species and global COVID-19 hotspots. With molecular data available for only a small fraction of potential animal hosts, linking data across biological scales offers a conceptual advance that may expand our predictive modelling capacity for zoonotic viruses with similarly unknown host ranges.

Comparing 2 Years of Empowering Families to Solve Real-World Problems with AI

Over the course of 2 years a global technology education nonprofit engaged ~ 20,000 under-resourced 3rd-8th grade students, parents and educators from 13 countries in a multi-week AI competition. Families worked together with the help of educators to identify meaningful problems in their communities and developed AI-prototypes to address them. Key findings included: (1) Identifying a high level of interest in underserved communities to develop and apply AI-literacy skills; (2) Determining curricular and program implementation elements that enable families to apply AI knowledge and skills to real problems; (3) Identifying effective methods of engaging industry mentors to support participants; (4) Measuring and identifying changes in self-efficacy and ability to apply AI-based tools to real-world problems; (5) Determining effective curricula around value-sensitive design and ethical innovation.

Pseudocell Tracer-A method for inferring dynamic trajectories using scRNAseq and its application to B cells undergoing immunoglobulin class switch recombination

Single cell RNA sequencing (scRNAseq) can be used to infer a temporal ordering of cellular states. Current methods for the inference of cellular trajectories rely on unbiased dimensionality reduction techniques. However, such biologically agnostic ordering can prove difficult for modeling complex developmental or differentiation processes. The cellular heterogeneity of dynamic biological compartments can result in sparse sampling of key intermediate cell states. To overcome these limitations, we develop a supervised machine learning framework, called Pseudocell Tracer, which infers trajectories in pseudospace rather than in pseudotime. The method uses a supervised encoder, trained with adjacent biological information, to project scRNAseq data into a low-dimensional manifold that maps the transcriptional states a cell can occupy. Then a generative adversarial network (GAN) is used to simulate pesudocells at regular intervals along a virtual cell-state axis. We demonstrate the utility of Pseudocell Tracer by modeling B cells undergoing immunoglobulin class switch recombination (CSR) during a prototypic antigen-induced antibody response. Our results revealed an ordering of key transcription factors regulating CSR to the IgG1 isotype, including the concomitant expression of Nfkb1 and Stat6 prior to the upregulation of Bach2 expression. Furthermore, the expression dynamics of genes encoding cytokine receptors suggest a poised IL-4 signaling state that preceeds CSR to the IgG1 isotype.

Comprehensive in silico analysis and molecular dynamics of the superoxide dismutase 1 (SOD1) variants related to amyotrophic lateral sclerosis

Amyotrophic Lateral Sclerosis (ALS) is the most frequent motor neuron disorder, with a significant social and economic burden. ALS remains incurable, and the only drugs approved for its treatments confers a survival benefit of a few months for the patients. Missense mutations in superoxide dismutase 1 (SOD1), a major cytoplasmic antioxidant enzyme, has been associated with ALS development, accounting for 23% of its familial cases and 7% of all sporadic cases. This work aims to characterize in silico the structural and functional effects of SOD1 protein variants. Missense mutations in SOD1 were compiled from the literature and databases. Twelve algorithms were used to predict the functional and stability effects of these mutations. ConSurf was used to estimate the evolutionary conservation of SOD1 amino-acids. GROMACS was used to perform molecular dynamics (MD) simulations of SOD1 wild-type and variants A4V, D90A, H46R, and I113T, which account for approximately half of all ALS-SOD1 cases in the United States, Europe, Japan, and United Kingdom, respectively. 233 missense mutations in SOD1 protein were compiled from the databases and literature consulted. The predictive analyses pointed to an elevated rate of deleterious and destabilizing predictions for the analyzed variants, indicating their harmful effects. The ConSurf analysis suggested that mutations in SOD1 mainly affect conserved and possibly functionally essential amino acids. The MD analyses pointed to flexibility and essential dynamics alterations at the electrostatic and metal-binding loops of variants A4V, D90A, H46R, and I113T that could lead to aberrant interactions triggering toxic protein aggregation. These alterations may have harmful implications for SOD1 and explain their association with ALS. Understanding the effects of SOD1 mutations on protein structure and function facilitates the design of further experiments and provides relevant information on the molecular mechanism of pathology, which may contribute to improvements in existing treatments for ALS.

Deep learning-based clustering approaches for bioinformatics

Clustering is central to many data-driven bioinformatics research and serves a powerful computational method. In particular, clustering helps at analyzing unstructured and high-dimensional data in the form of sequences, expressions, texts and images. Further, clustering is used to gain insights into biological processes in the genomics level, e.g. clustering of gene expressions provides insights on the natural structure inherent in the data, understanding gene functions, cellular processes, subtypes of cells and understanding gene regulations. Subsequently, clustering approaches, including hierarchical, centroid-based, distribution-based, density-based and self-organizing maps, have long been studied and used in classical machine learning settings. In contrast, deep learning (DL)-based representation and feature learning for clustering have not been reviewed and employed extensively. Since the quality of clustering is not only dependent on the distribution of data points but also on the learned representation, deep neural networks can be effective means to transform mappings from a high-dimensional data space into a lower-dimensional feature space, leading to improved clustering results. In this paper, we review state-of-the-art DL-based approaches for cluster analysis that are based on representation learning, which we hope to be useful, particularly for bioinformatics research. Further, we explore in detail the training procedures of DL-based clustering algorithms, point out different clustering quality metrics and evaluate several DL-based approaches on three bioinformatics use cases, including bioimaging, cancer genomics and biomedical text mining. We believe this review and the evaluation results will provide valuable insights and serve a starting point for researchers wanting to apply DL-based unsupervised methods to solve emerging bioinformatics research problems.

Multi-domain translation between single-cell imaging and sequencing data using autoencoders

The development of single-cell methods for capturing different data modalities including imaging and sequencing has revolutionized our ability to identify heterogeneous cell states. Different data modalities provide different perspectives on a population of cells, and their integration is critical for studying cellular heterogeneity and its function. While various methods have been proposed to integrate different sequencing data modalities, coupling imaging and sequencing has been an open challenge. We here present an approach for integrating vastly different modalities by learning a probabilistic coupling between the different data modalities using autoencoders to map to a shared latent space. We validate this approach by integrating single-cell RNA-seq and chromatin images to identify distinct subpopulations of human naive CD4+ T-cells that are poised for activation. Collectively, our approach provides a framework to integrate and translate between data modalities that cannot yet be measured within the same cell for diverse applications in biomedical discovery.

Precisely timed dopamine signals establish distinct kinematic representations of skilled movements

Brain dopamine is critical for normal motor control, as evidenced by its importance in Parkinson Disease and related disorders. Current hypotheses are that dopamine influences motor control by 'invigorating' movements and regulating motor learning. Most evidence for these aspects of dopamine function comes from simple tasks (e.g. lever pressing). Therefore, the influence of dopamine on motor skills requiring multi-joint coordination is unknown. To determine the effects of precisely timed dopamine manipulations on the performance of a complex, finely coordinated dexterous skill, we optogenetically stimulated or inhibited midbrain dopamine neurons as rats performed a skilled reaching task. We found that reach kinematics and coordination between gross and fine movements progressively changed with repeated manipulations. However, once established, rats transitioned abruptly between aberrant and baseline reach kinematics in a dopamine-dependent manner. These results suggest that precisely timed dopamine signals have immediate and long-term influences on motor skill performance, distinct from simply 'invigorating' movement.

Individual differences among deep neural network models

Deep neural networks (DNNs) excel at visual recognition tasks and are increasingly used as a modeling framework for neural computations in the primate brain. Just like individual brains, each DNN has a unique connectivity and representational profile. Here, we investigate individual differences among DNN instances that arise from varying only the random initialization of the network weights. Using tools typically employed in systems neuroscience, we show that this minimal change in initial conditions prior to training leads to substantial differences in intermediate and higher-level network representations despite similar network-level classification performance. We locate the origins of the effects in an under-constrained alignment of category exemplars, rather than misaligned category centroids. These results call into question the common practice of using single networks to derive insights into neural information processing and rather suggest that computational neuroscientists working with DNNs may need to base their inferences on groups of multiple network instances.

The Moran process on 2-chromatic graphs

Resources are rarely distributed uniformly within a population. Heterogeneity in the concentration of a drug, the quality of breeding sites, or wealth can all affect evolutionary dynamics. In this study, we represent a collection of properties affecting the fitness at a given location using a color. A green node is rich in resources while a red node is poorer. More colors can represent a broader spectrum of resource qualities. For a population evolving according to the birth-death Moran model, the first question we address is which structures, identified by graph connectivity and graph coloring, are evolutionarily equivalent. We prove that all properly two-colored, undirected, regular graphs are evolutionarily equivalent (where “properly colored” means that no two neighbors have the same color). We then compare the effects of background heterogeneity on properly two-colored graphs to those with alternative schemes in which the colors are permuted. Finally, we discuss dynamic coloring as a model for spatiotemporal resource fluctuations, and we illustrate that random dynamic colorings often diminish the effects of background heterogeneity relative to a proper two-coloring.

Heterogeneity in environmental conditions can have profound effects on long-term evolutionary outcomes in structured populations. We consider a population evolving on a colored graph, wherein the color of a node represents the resources at that location. Using a combination of analytical and numerical methods, we quantify the effects of background heterogeneity on a population’s dynamics. In addition to considering the notion of an “optimal” coloring with respect to mutant invasion, we also study the effects of dynamic spatial redistribution of resources as the population evolves. Although the effects of static background heterogeneity can be quite striking, these effects are often attenuated by the movement (or “flow”) of the underlying resources.

Active vision in immersive, 360° real-world environments

How do we construct a sense of place in a real-world environment? Real-world environments are actively explored via saccades, head turns, and body movements. Yet, little is known about how humans process real-world scene information during active viewing conditions. Here, we exploited recent developments in virtual reality (VR) and in-headset eye-tracking to test the impact of active vs. passive viewing conditions on gaze behavior while participants explored novel, real-world, 360° scenes. In one condition, participants actively explored 360° photospheres from a first-person perspective via self-directed motion (saccades and head turns). In another condition, photospheres were passively displayed to participants while they were head-restricted. We found that, relative to passive viewers, active viewers displayed increased attention to semantically meaningful scene regions, suggesting more exploratory, information-seeking gaze behavior. We also observed signatures of exploratory behavior in eye movements, such as quicker, more entropic fixations during active as compared with passive viewing conditions. These results show that active viewing influences every aspect of gaze behavior, from the way we move our eyes to what we choose to attend to. Moreover, these results offer key benchmark measurements of gaze behavior in 360°, naturalistic environments.

Automatic lung segmentation in routine imaging is primarily a data diversity problem, not a methodology problem

BACKGROUND

Automated segmentation of anatomical structures is a crucial step in image analysis. For lung segmentation in computed tomography, a variety of approaches exists, involving sophisticated pipelines trained and validated on different datasets. However, the clinical applicability of these approaches across diseases remains limited.

METHODS

We compared four generic deep learning approaches trained on various datasets and two readily available lung segmentation algorithms. We performed evaluation on routine imaging data with more than six different disease patterns and three published data sets.

RESULTS

Using different deep learning approaches, mean Dice similarity coefficients (DSCs) on test datasets varied not over 0.02. When trained on a diverse routine dataset (n = 36), a standard approach (U-net) yields a higher DSC (0.97 ± 0.05) compared to training on public datasets such as the Lung Tissue Research Consortium (0.94 ± 0.13, p = 0.024) or Anatomy 3 (0.92 ± 0.15, p = 0.001). Trained on routine data (n = 231) covering multiple diseases, U-net compared to reference methods yields a DSC of 0.98 ± 0.03 versus 0.94 ± 0.12 (p = 0.024).

CONCLUSIONS

The accuracy and reliability of lung segmentation algorithms on demanding cases primarily relies on the diversity of the training data, highlighting the importance of data diversity compared to model choice. Efforts in developing new datasets and providing trained models to the public are critical. By releasing the trained model under General Public License 3.0, we aim to foster research on lung diseases by providing a readily available tool for segmentation of pathological lungs.

Using machine learning models to predict the initiation of renal replacement therapy among chronic kidney disease patients

Starting renal replacement therapy (RRT) for patients with chronic kidney disease (CKD) at an optimal time, either with hemodialysis or kidney transplantation, is crucial for patient's well-being and for successful management of the condition. In this paper, we explore the possibilities of creating forecasting models to predict the onset of RRT 3, 6, and 12 months from the time of the patient's first diagnosis with CKD, using only the comorbidities data from National Health Insurance from Taiwan. The goal of this study was to see whether a limited amount of data (including comorbidities but not considering laboratory values which are expensive to obtain in low- and medium-income countries) can provide a good basis for such predictive models. On the other hand, in developed countries, such models could allow policy-makers better planning and allocation of resources for treatment. Using data from 8,492 patients, we obtained the area under the receiver operating characteristic curve (AUC) of 0.773 for predicting RRT within 12 months from the time of CKD diagnosis. The results also show that there is no additional advantage in focusing only on patients with diabetes in terms of prediction performance. Although these results are not as such suitable for adoption into clinical practice, the study provides a strong basis and a variety of approaches for future studies of forecasting models in healthcare.

Insights into herpesvirus assembly from the structure of the pUL7:pUL51 complex

Herpesviruses acquire their membrane envelopes in the cytoplasm of infected cells via a molecular mechanism that remains unclear. Herpes simplex virus (HSV)-1 proteins pUL7 and pUL51 form a complex required for efficient virus envelopment. We show that interaction between homologues of pUL7 and pUL51 is conserved across human herpesviruses, as is their association with trans-Golgi membranes. We characterized the HSV-1 pUL7:pUL51 complex by solution scattering and chemical crosslinking, revealing a 1:2 complex that can form higher-order oligomers in solution, and we solved the crystal structure of the core pUL7:pUL51 heterodimer. While pUL7 adopts a previously-unseen compact fold, the helix-turn-helix conformation of pUL51 resembles the cellular endosomal complex required for transport (ESCRT)-III component CHMP4B and pUL51 forms ESCRT-III-like filaments, suggesting a direct role for pUL51 in promoting membrane scission during virus assembly. Our results provide a structural framework for understanding the role of the conserved pUL7:pUL51 complex in herpesvirus assembly.

Aging induces aberrant state transition kinetics in murine muscle stem cells

Murine muscle stem cells (MuSCs) experience a transition from quiescence to activation that is required for regeneration, but it remains unknown if the trajectory and dynamics of activation change with age. Here, we use time-lapse imaging and single cell RNA-seq to measure activation trajectories and rates in young and aged MuSCs. We find that the activation trajectory is conserved in aged cells, and we develop effective machine-learning classifiers for cell age. Using cell-behavior analysis and RNA velocity, we find that activation kinetics are delayed in aged MuSCs, suggesting that changes in stem cell dynamics may contribute to impaired stem cell function with age. Intriguingly, we also find that stem cell activation appears to be a random walk-like process, with frequent reversals, rather than a continuous linear progression. These results support a view of the aged stem cell phenotype as a combination of differences in the location of stable cell states and differences in transition rates between them.

In silico analysis of the tryptophan hydroxylase 2 (TPH2) protein variants related to psychiatric disorders

The tryptophan hydroxylase 2 (TPH2) enzyme catalyzes the first step of serotonin biosynthesis. Serotonin is known for its role in several homeostatic systems related to sleep, mood, and food intake. As the reaction catalyzed by TPH2 is the rate-limiting step of serotonin biosynthesis, mutations in TPH2 have been associated with several psychiatric disorders (PD). This work undertakes an in silico analysis of the effects of genetic mutations in the human TPH2 protein. Ten algorithms were used to predict the functional and stability effects of the TPH2 mutations. ConSurf was used to estimate the evolutionary conservation of TPH2 amino acids. GROMACS was used to perform molecular dynamics (MD) simulations of TPH2 WT and P260S, R303W, and R441H, which had already been associated with the development of PD. Forty-six TPH2 variants were compiled from the literature. Among the analyzed variants, those occurring at the catalytic domain were shown to be more damaging to protein structure and function. The ConSurf analysis indicated that the mutations affecting the catalytic domain were also more conserved throughout evolution. The variants S364K and S383F were predicted to be deleterious by all the functional algorithms used and occurred at conserved positions, suggesting that they might be deleterious. The MD analyses indicate that the mutations P206S, R303W, and R441H affect TPH2 flexibility and essential mobility at the catalytic and oligomerization domains. The variants P206S, R303W, and R441H also exhibited alterations in dimer binding affinity and stability throughout the simulations. Thus, these mutations may impair TPH2 functional interactions and, consequently, its function, leading to the development of PD. Furthermore, we developed a database, SNPMOL (http://www.snpmol.org/), containing the results presented in this paper. Understanding the effects of TPH2 mutations on protein structure and function may lead to improvements in existing treatments for PD and facilitate the design of further experiments.

An artificial intelligence-based deep learning algorithm for the diagnosis of diabetic neuropathy using corneal confocal microscopy: a development and validation study

AIMS/HYPOTHESIS

Corneal confocal microscopy is a rapid non-invasive ophthalmic imaging technique that identifies peripheral and central neurodegenerative disease. Quantification of corneal sub-basal nerve plexus morphology, however, requires either time-consuming manual annotation or a less-sensitive automated image analysis approach. We aimed to develop and validate an artificial intelligence-based, deep learning algorithm for the quantification of nerve fibre properties relevant to the diagnosis of diabetic neuropathy and to compare it with a validated automated analysis program, ACCMetrics.

METHODS

Our deep learning algorithm, which employs a convolutional neural network with data augmentation, was developed for the automated quantification of the corneal sub-basal nerve plexus for the diagnosis of diabetic neuropathy. The algorithm was trained using a high-end graphics processor unit on 1698 corneal confocal microscopy images; for external validation, it was further tested on 2137 images. The algorithm was developed to identify total nerve fibre length, branch points, tail points, number and length of nerve segments, and fractal numbers. Sensitivity analyses were undertaken to determine the AUC for ACCMetrics and our algorithm for the diagnosis of diabetic neuropathy.

RESULTS

The intraclass correlation coefficients for our algorithm were superior to those for ACCMetrics for total corneal nerve fibre length (0.933 vs 0.825), mean length per segment (0.656 vs 0.325), number of branch points (0.891 vs 0.570), number of tail points (0.623 vs 0.257), number of nerve segments (0.878 vs 0.504) and fractals (0.927 vs 0.758). In addition, our proposed algorithm achieved an AUC of 0.83, specificity of 0.87 and sensitivity of 0.68 for the classification of participants without (n = 90) and with (n = 132) neuropathy (defined by the Toronto criteria).

CONCLUSIONS/INTERPRETATION

These results demonstrated that our deep learning algorithm provides rapid and excellent localisation performance for the quantification of corneal nerve biomarkers. This model has potential for adoption into clinical screening programmes for diabetic neuropathy.

DATA AVAILABILITY

The publicly shared cornea nerve dataset (dataset 1) is available at http://bioimlab.dei.unipd.it/Corneal%20Nerve%20Tortuosity%20Data%20Set.htm and http://bioimlab.dei.unipd.it/Corneal%20Nerve%20Data%20Set.htm.

The SFU Opinion and Comments Corpus: A Corpus for the Analysis of Online News Comments

We present the SFU Opinion and Comments Corpus (SOCC ), a collection of opinion articles and the comments posted in response to the articles. The articles include all the opinion pieces published in the Canadian newspaper The Globe and Mail in the 5-year period between 2012 and 2016, a total of 10,339 articles and 663,173 comments. SOCC is part of a project that investigates the linguistic characteristics of online comments. The corpus can be used to study a host of pragmatic phenomena. Among other aspects, researchers can explore: the connections between articles and comments; the connections of comments to each other; the types of topics discussed in comments; the nice (constructive) or mean (toxic) ways in which commenters respond to each other; how language is used to convey very specific types of evaluation; and how negation affects the interpretation of evaluative meaning in discourse. Our current focus is the study of constructiveness and evaluation in the comments. To that end, we have annotated a subset of the large corpus (1043 comments) with four layers of annotations: constructiveness, toxicity, negation and Appraisal (Martin and White, The language of evaluation, Palgrave, New York, 2005). This paper details our corpus, the data collection process, the characteristics of the corpus and describes the annotations. While our focus is comments posted in response to opinion news articles, the phenomena in this corpus are likely to be present in many commenting platforms: other news comments, comments and replies in fora such as Reddit, feedback on blogs, or YouTube comments.

DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning

Quantitative behavioral measurements are important for answering questions across scientific disciplines-from neuroscience to ecology. State-of-the-art deep-learning methods offer major advances in data quality and detail by allowing researchers to automatically estimate locations of an animal's body parts directly from images or videos. However, currently available animal pose estimation methods have limitations in speed and robustness. Here, we introduce a new easy-to-use software toolkit, DeepPoseKit, that addresses these problems using an efficient multi-scale deep-learning model, called Stacked DenseNet, and a fast GPU-based peak-detection algorithm for estimating keypoint locations with subpixel precision. These advances improve processing speed >2x with no loss in accuracy compared to currently available methods. We demonstrate the versatility of our methods with multiple challenging animal pose estimation tasks in laboratory and field settings-including groups of interacting individuals. Our work reduces barriers to using advanced tools for measuring behavior and has broad applicability across the behavioral sciences.

DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning

Quantitative behavioral measurements are important for answering questions across scientific disciplines-from neuroscience to ecology. State-of-the-art deep-learning methods offer major advances in data quality and detail by allowing researchers to automatically estimate locations of an animal's body parts directly from images or videos. However, currently available animal pose estimation methods have limitations in speed and robustness. Here, we introduce a new easy-to-use software toolkit, DeepPoseKit, that addresses these problems using an efficient multi-scale deep-learning model, called Stacked DenseNet, and a fast GPU-based peak-detection algorithm for estimating keypoint locations with subpixel precision. These advances improve processing speed >2x with no loss in accuracy compared to currently available methods. We demonstrate the versatility of our methods with multiple challenging animal pose estimation tasks in laboratory and field settings-including groups of interacting individuals. Our work reduces barriers to using advanced tools for measuring behavior and has broad applicability across the behavioral sciences.

DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning

Quantitative behavioral measurements are important for answering questions across scientific disciplines-from neuroscience to ecology. State-of-the-art deep-learning methods offer major advances in data quality and detail by allowing researchers to automatically estimate locations of an animal's body parts directly from images or videos. However, currently available animal pose estimation methods have limitations in speed and robustness. Here, we introduce a new easy-to-use software toolkit, DeepPoseKit, that addresses these problems using an efficient multi-scale deep-learning model, called Stacked DenseNet, and a fast GPU-based peak-detection algorithm for estimating keypoint locations with subpixel precision. These advances improve processing speed >2x with no loss in accuracy compared to currently available methods. We demonstrate the versatility of our methods with multiple challenging animal pose estimation tasks in laboratory and field settings-including groups of interacting individuals. Our work reduces barriers to using advanced tools for measuring behavior and has broad applicability across the behavioral sciences.

Enabling machine learning in X-ray-based procedures via realistic simulation of image formation

PURPOSE

Machine learning-based approaches now outperform competing methods in most disciplines relevant to diagnostic radiology. Image-guided procedures, however, have not yet benefited substantially from the advent of deep learning, in particular because images for procedural guidance are not archived and thus unavailable for learning, and even if they were available, annotations would be a severe challenge due to the vast amounts of data. In silico simulation of X-ray images from 3D CT is an interesting alternative to using true clinical radiographs since labeling is comparably easy and potentially readily available.

METHODS

We extend our framework for fast and realistic simulation of fluoroscopy from high-resolution CT, called DeepDRR, with tool modeling capabilities. The framework is publicly available, open source, and tightly integrated with the software platforms native to deep learning, i.e., Python, PyTorch, and PyCuda. DeepDRR relies on machine learning for material decomposition and scatter estimation in 3D and 2D, respectively, but uses analytic forward projection and noise injection to ensure acceptable computation times. On two X-ray image analysis tasks, namely (1) anatomical landmark detection and (2) segmentation and localization of robot end-effectors, we demonstrate that convolutional neural networks (ConvNets) trained on DeepDRRs generalize well to real data without re-training or domain adaptation. To this end, we use the exact same training protocol to train ConvNets on naïve and DeepDRRs and compare their performance on data of cadaveric specimens acquired using a clinical C-arm X-ray system.

RESULTS

Our findings are consistent across both considered tasks. All ConvNets performed similarly well when evaluated on the respective synthetic testing set. However, when applied to real radiographs of cadaveric anatomy, ConvNets trained on DeepDRRs significantly outperformed ConvNets trained on naïve DRRs ([Formula: see text]).

CONCLUSION

Our findings for both tasks are positive and promising. Combined with complementary approaches, such as image style transfer, the proposed framework for fast and realistic simulation of fluoroscopy from CT contributes to promoting the implementation of machine learning in X-ray-guided procedures. This paradigm shift has the potential to revolutionize intra-operative image analysis to simplify surgical workflows.

Autofocals: Evaluating gaze-contingent eyeglasses for presbyopes

As humans age, they gradually lose the ability to accommodate, or refocus, to near distances because of the stiffening of the crystalline lens. This condition, known as presbyopia, affects nearly 20% of people worldwide. We design and build a new presbyopia correction, autofocals, to externally mimic the natural accommodation response, combining eye tracker and depth sensor data to automatically drive focus-tunable lenses. We evaluated 19 users on visual acuity, contrast sensitivity, and a refocusing task. Autofocals exhibit better visual acuity when compared to monovision and progressive lenses while maintaining similar contrast sensitivity. On the refocusing task, autofocals are faster and, compared to progressives, also significantly more accurate. In a separate study, a majority of 23 of 37 users ranked autofocals as the best correction in terms of ease of refocusing. Our work demonstrates the superiority of autofocals over current forms of presbyopia correction and could affect the lives of millions.

Reconstructing faces from fMRI patterns using deep generative neural networks

Although distinct categories are reliably decoded from fMRI brain responses, it has proved more difficult to distinguish visually similar inputs, such as different faces. Here, we apply a recently developed deep learning system to reconstruct face images from human fMRI. We trained a variational auto-encoder (VAE) neural network using a GAN (Generative Adversarial Network) unsupervised procedure over a large data set of celebrity faces. The auto-encoder latent space provides a meaningful, topologically organized 1024-dimensional description of each image. We then presented several thousand faces to human subjects, and learned a simple linear mapping between the multi-voxel fMRI activation patterns and the 1024 latent dimensions. Finally, we applied this mapping to novel test images, translating fMRI patterns into VAE latent codes, and codes into face reconstructions. The system not only performed robust pairwise decoding (>95% correct), but also accurate gender classification, and even decoded which face was imagined, rather than seen.

In silico analysis of the V66M variant of human BDNF in psychiatric disorders: An approach to precision medicine

Brain-derived neurotrophic factor (BDNF) plays an important role in neurogenesis and synapse formation. The V66M is the most prevalent BDNF mutation in humans and impairs the function and distribution of BDNF. This mutation is related to several psychiatric disorders. The pro-region of BDNF, particularly position 66 and its adjacent residues, are determinant for the intracellular sorting and activity-dependent secretion of BDNF. However, it has not yet been fully elucidated. The present study aims to analyze the effects of the V66M mutation on BDNF structure and function. Here, we applied nine algorithms, including SIFT and PolyPhen-2, for functional and stability prediction of the V66M mutation. The complete theoretical model of BNDF was generated by Rosetta and validated by PROCHECK, RAMPAGE, ProSa, QMEAN and Verify-3D algorithms. Structural alignment was performed using TM-align. Phylogenetic analysis was performed using the ConSurf server. Molecular dynamics (MD) simulations were performed and analyzed using the GROMACS 2018.2 package. The V66M mutation was predicted as deleterious by PolyPhen-2 and SIFT in addition to being predicted as destabilizing by I-Mutant. According to SNPeffect, the V66M mutation does not affect protein aggregation, amyloid propensity, and chaperone binding. The complete theoretical structure of BDNF proved to be a reliable model. Phylogenetic analysis indicated that the V66M mutation of BDNF occurs at a non-conserved position of the protein. MD analyses indicated that the V66M mutation does not affect the BDNF flexibility and surface-to-volume ratio, but affects the BDNF essential motions, hydrogen-bonding and secondary structure particularly at its pre and pro-domain, which are crucial for its activity and distribution. Thus, considering that these parameters are determinant for protein interactions and, consequently, protein function; the alterations observed throughout the MD analyses may be related to the functional impairment of BDNF upon V66M mutation, as well as its involvement in psychiatric disorders.

A Comparison of Machine Learning and Deep Learning Techniques for Activity Recognition using Mobile Devices

We have compared the performance of different machine learning techniques for human activity recognition. Experiments were made using a benchmark dataset where each subject wore a device in the pocket and another on the wrist. The dataset comprises thirteen activities, including physical activities, common postures, working activities and leisure activities. We apply a methodology known as the activity recognition chain, a sequence of steps involving preprocessing, segmentation, feature extraction and classification for traditional machine learning methods; we also tested convolutional deep learning networks that operate on raw data instead of using computed features. Results show that combination of two sensors does not necessarily result in an improved accuracy. We have determined that best results are obtained by the extremely randomized trees approach, operating on precomputed features and on data obtained from the wrist sensor. Deep learning architectures did not produce competitive results with the tested architecture.

Unsupervised Medical Image Segmentation Based on the Local Center of Mass

Image segmentation is a critical step in numerous medical imaging studies, which can be facilitated by automatic computational techniques. Supervised methods, although highly effective, require large training datasets of manually labeled images that are labor-intensive to produce. Unsupervised methods, on the contrary, can be used in the absence of training data to segment new images. We introduce a new approach to unsupervised image segmentation that is based on the computation of the local center of mass. We propose an efficient method to group the pixels of a one-dimensional signal, which we then use in an iterative algorithm for two- and three-dimensional image segmentation. We validate our method on a 2D X-ray image, a 3D abdominal magnetic resonance (MR) image and a dataset of 3D cardiovascular MR images.

Adaptive QM/MM for Molecular Dynamics Simulations: 5. On the Energy-Conserved Permuted Adaptive-Partitioning Schemes

In combined quantum-mechanical/molecular-mechanical (QM/MM) dynamics simulations, the adaptive-partitioning (AP) schemes reclassify atoms on-the-fly as QM or MM in a smooth manner. This yields a mobile QM subsystem with contents that are continuously updated as needed. Here, we tailor the Hamiltonian adaptive many-body correction (HAMBC) proposed by Boreboom et al. [J. Chem. Theory Comput.2016, 12, 3441] to the permuted AP (PAP) scheme. The treatments lead to the HAMBC-PAP method (HPAP), which both conserves energy and produces accurate solvation structures in the test of "water-in-water" model system.

3-D Reconstruction in Canonical Co-Ordinate Space From Arbitrarily Oriented 2-D Images

Limited capture range, and the requirement to provide high quality initialization for optimization-based 2-D/3-D image registration methods, can significantly degrade the performance of 3-D image reconstruction and motion compensation pipelines. Challenging clinical imaging scenarios, which contain significant subject motion, such as fetal in-utero imaging, complicate the 3-D image and volume reconstruction process. In this paper, we present a learning-based image registration method capable of predicting 3-D rigid transformations of arbitrarily oriented 2-D image slices, with respect to a learned canonical atlas co-ordinate system. Only image slice intensity information is used to perform registration and canonical alignment, no spatial transform initialization is required. To find image transformations, we utilize a convolutional neural network architecture to learn the regression function capable of mapping 2-D image slices to a 3-D canonical atlas space. We extensively evaluate the effectiveness of our approach quantitatively on simulated magnetic resonance imaging (MRI), fetal brain imagery with synthetic motion and further demonstrate qualitative results on real fetal MRI data where our method is integrated into a full reconstruction and motion compensation pipeline. Our learning based registration achieves an average spatial prediction error of 7 mm on simulated data and produces qualitatively improved reconstructions for heavily moving fetuses with gestational ages of approximately 20 weeks. Our model provides a general and computationally efficient solution to the 2-D/3-D registration initialization problem and is suitable for real-time scenarios.

Interactive Medical Image Segmentation Using Deep Learning With Image-Specific Fine Tuning

Convolutional neural networks (CNNs) have achieved state-of-the-art performance for automatic medical image segmentation. However, they have not demonstrated sufficiently accurate and robust results for clinical use. In addition, they are limited by the lack of image-specific adaptation and the lack of generalizability to previously unseen object classes (a.k.a. zero-shot learning). To address these problems, we propose a novel deep learning-based interactive segmentation framework by incorporating CNNs into a bounding box and scribble-based segmentation pipeline. We propose image-specific fine tuning to make a CNN model adaptive to a specific test image, which can be either unsupervised (without additional user interactions) or supervised (with additional scribbles). We also propose a weighted loss function considering network and interaction-based uncertainty for the fine tuning. We applied this framework to two applications: 2-D segmentation of multiple organs from fetal magnetic resonance (MR) slices, where only two types of these organs were annotated for training and 3-D segmentation of brain tumor core (excluding edema) and whole brain tumor (including edema) from different MR sequences, where only the tumor core in one MR sequence was annotated for training. Experimental results show that: 1) our model is more robust to segment previously unseen objects than state-of-the-art CNNs; 2) image-specific fine tuning with the proposed weighted loss function significantly improves segmentation accuracy; and 3) our method leads to accurate results with fewer user interactions and less user time than traditional interactive segmentation methods.

NiftyPET: a High-throughput Software Platform for High Quantitative Accuracy and Precision PET Imaging and Analysis

We present a standalone, scalable and high-throughput software platform for PET image reconstruction and analysis. We focus on high fidelity modelling of the acquisition processes to provide high accuracy and precision quantitative imaging, especially for large axial field of view scanners. All the core routines are implemented using parallel computing available from within the Python package NiftyPET, enabling easy access, manipulation and visualisation of data at any processing stage. The pipeline of the platform starts from MR and raw PET input data and is divided into the following processing stages: (1) list-mode data processing; (2) accurate attenuation coefficient map generation; (3) detector normalisation; (4) exact forward and back projection between sinogram and image space; (5) estimation of reduced-variance random events; (6) high accuracy fully 3D estimation of scatter events; (7) voxel-based partial volume correction; (8) region- and voxel-level image analysis. We demonstrate the advantages of this platform using an amyloid brain scan where all the processing is executed from a single and uniform computational environment in Python. The high accuracy acquisition modelling is achieved through span-1 (no axial compression) ray tracing for true, random and scatter events. Furthermore, the platform offers uncertainty estimation of any image derived statistic to facilitate robust tracking of subtle physiological changes in longitudinal studies. The platform also supports the development of new reconstruction and analysis algorithms through restricting the axial field of view to any set of rings covering a region of interest and thus performing fully 3D reconstruction and corrections using real data significantly faster. All the software is available as open source with the accompanying wiki-page and test data.

Fully automated, deep learning segmentation of oxygen-induced retinopathy images

Oxygen-induced retinopathy (OIR) is a widely used model to study ischemia-driven neovascularization (NV) in the retina and to serve in proof-of-concept studies in evaluating antiangiogenic drugs for ocular, as well as nonocular, diseases. The primary parameters that are analyzed in this mouse model include the percentage of retina with vaso-obliteration (VO) and NV areas. However, quantification of these two key variables comes with a great challenge due to the requirement of human experts to read the images. Human readers are costly, time-consuming, and subject to bias. Using recent advances in machine learning and computer vision, we trained deep learning neural networks using over a thousand segmentations to fully automate segmentation in OIR images. While determining the percentage area of VO, our algorithm achieved a similar range of correlation coefficients to that of expert inter-human correlation coefficients. In addition, our algorithm achieved a higher range of correlation coefficients compared with inter-expert correlation coefficients for quantification of the percentage area of neovascular tufts. In summary, we have created an open-source, fully automated pipeline for the quantification of key values of OIR images using deep learning neural networks.

Invariant recognition drives neural representations of action sequences

Recognizing the actions of others from visual stimuli is a crucial aspect of human perception that allows individuals to respond to social cues. Humans are able to discriminate between similar actions despite transformations, like changes in viewpoint or actor, that substantially alter the visual appearance of a scene. This ability to generalize across complex transformations is a hallmark of human visual intelligence. Advances in understanding action recognition at the neural level have not always translated into precise accounts of the computational principles underlying what representations of action sequences are constructed by human visual cortex. Here we test the hypothesis that invariant action discrimination might fill this gap. Recently, the study of artificial systems for static object perception has produced models, Convolutional Neural Networks (CNNs), that achieve human level performance in complex discriminative tasks. Within this class, architectures that better support invariant object recognition also produce image representations that better match those implied by human and primate neural data. However, whether these models produce representations of action sequences that support recognition across complex transformations and closely follow neural representations of actions remains unknown. Here we show that spatiotemporal CNNs accurately categorize video stimuli into action classes, and that deliberate model modifications that improve performance on an invariant action recognition task lead to data representations that better match human neural recordings. Our results support our hypothesis that performance on invariant discrimination dictates the neural representations of actions computed in the brain. These results broaden the scope of the invariant recognition framework for understanding visual intelligence from perception of inanimate objects and faces in static images to the study of human perception of action sequences.

Regulatory network changes between cell lines and their tissues of origin

BACKGROUND

Cell lines are an indispensable tool in biomedical research and often used as surrogates for tissues. Although there are recognized important cellular and transcriptomic differences between cell lines and tissues, a systematic overview of the differences between the regulatory processes of a cell line and those of its tissue of origin has not been conducted. The RNA-Seq data generated by the GTEx project is the first available data resource in which it is possible to perform a large-scale transcriptional and regulatory network analysis comparing cell lines with their tissues of origin.

RESULTS

We compared 127 paired Epstein-Barr virus transformed lymphoblastoid cell lines (LCLs) and whole blood samples, and 244 paired primary fibroblast cell lines and skin samples. While gene expression analysis confirms that these cell lines carry the expression signatures of their primary tissues, albeit at reduced levels, network analysis indicates that expression changes are the cumulative result of many previously unreported alterations in transcription factor (TF) regulation. More specifically, cell cycle genes are over-expressed in cell lines compared to primary tissues, and this alteration in expression is a result of less repressive TF targeting. We confirmed these regulatory changes for four TFs, including SMAD5, using independent ChIP-seq data from ENCODE.

CONCLUSIONS

Our results provide novel insights into the regulatory mechanisms controlling the expression differences between cell lines and tissues. The strong changes in TF regulation that we observe suggest that network changes, in addition to transcriptional levels, should be considered when using cell lines as models for tissues.