Integrating Domain Knowledge Into Deep Networks for Lung Ultrasound With Applications to COVID-19

Lung ultrasound (LUS) is a cheap, safe and non-invasive imaging modality that can be performed at patient bed-side. However, to date LUS is not widely adopted due to lack of trained personnel required for interpreting the acquired LUS frames. In this work we propose a framework for training deep artificial neural networks for interpreting LUS, which may promote broader use of LUS. When using LUS to evaluate a patient's condition, both anatomical phenomena (e.g., the pleural line, presence of consolidations), as well as sonographic artifacts (such as A- and B-lines) are of importance. In our framework, we integrate domain knowledge into deep neural networks by inputting anatomical features and LUS artifacts in the form of additional channels containing pleural and vertical artifacts masks along with the raw LUS frames. By explicitly supplying this domain knowledge, standard off-the-shelf neural networks can be rapidly and efficiently finetuned to accomplish various tasks on LUS data, such as frame classification or semantic segmentation. Our framework allows for a unified treatment of LUS frames captured by either convex or linear probes. We evaluated our proposed framework on the task of COVID-19 severity assessment using the ICLUS dataset. In particular, we finetuned simple image classification models to predict per-frame COVID-19 severity score. We also trained a semantic segmentation model to predict per-pixel COVID-19 severity annotations. Using the combined raw LUS frames and the detected lines for both tasks, our off-the-shelf models performed better than complicated models specifically designed for these tasks, exemplifying the efficacy of our framework.

Data-driven algorithm for myelin water imaging: Probing subvoxel compartmentation based on identification of spatially global tissue features

PURPOSE

Multicomponent analysis of MRI T₂ relaxation time (mcT₂ ) is commonly used for estimating myelin content by separating the signal at each voxel into its underlying distribution of T₂ values. This voxel-based approach is challenging due to the large ambiguity in the multi-T₂ space and the low SNR of MRI signals. Herein, we present a data-driven mcT₂ analysis, which utilizes the statistical strength of identifying spatially global mcT₂ motifs in white matter segments before deconvolving the local signal at each voxel.

METHODS

Deconvolution is done using a tailored optimization scheme, which incorporates the global mcT₂ motifs without additional prior assumptions regarding the number of microscopic components. The end results of this process are voxel-wise myelin water fraction maps.

RESULTS

Validations are shown for computer-generated signals, uniquely designed subvoxel mcT₂ phantoms, and in vivo human brain. Results demonstrated excellent fitting accuracy, both for the numerical and the physical mcT₂ phantoms, exhibiting excellent agreement between calculated myelin water fraction and ground truth. Proof-of-concept in vivo validation is done by calculating myelin water fraction maps for white matter segments of the human brain. Interscan stability of myelin water fraction values was also estimated, showing good correlation between scans.

CONCLUSION

We conclude that studying global tissue motifs prior to performing voxel-wise mcT₂ analysis stabilizes the optimization scheme and efficiently overcomes the ambiguity in the T₂ space. This new approach can improve myelin water imaging and the investigation of microstructural compartmentation in general.

On Detection of Faint Edges in Noisy Images

A fundamental question for edge detection in noisy images is how faint can an edge be and still be detected. In this paper we offer a formalism to study this question and subsequently introduce computationally efficient multiscale edge detection algorithms designed to detect faint edges in noisy images. In our formalism we view edge detection as a search in a discrete, though potentially large, set of feasible curves. First, we derive approximate expressions for the detection threshold as a function of curve length and the complexity of the search space. We then present two edge detection algorithms, one for straight edges, and the second for curved ones. Both algorithms efficiently search for edges in a large set of candidates by hierarchically constructing difference filters that match the curves traced by the sought edges. We demonstrate the utility of our algorithms in both simulations and applications involving challenging real images. Finally, based on these principles, we develop an algorithm for fiber detection and enhancement. We exemplify its utility to reveal and enhance nerve axons in light microscopy images.

WIS-NeuroMath enables versatile high throughput analyses of neuronal processes

Automated analyses of neuronal morphology are important for quantifying connectivity and circuitry in vivo, as well as in high content imaging of primary neuron cultures. The currently available tools for quantification of neuronal morphology either are highly expensive commercial packages or cannot provide automated image quantifications at single cell resolution. Here, we describe a new software package called WIS-NeuroMath, which fills this gap and provides solutions for automated measurement of neuronal processes in both in vivo and in vitro preparations. Diverse image types can be analyzed without any preprocessing, enabling automated and accurate detection of neurites followed by their quantification in a number of application modules. A cell morphology module detects cell bodies and attached neurites, providing information on neurite length, number of branches, cell body area, and other parameters for each cell. A neurite length module provides a solution for images lacking cell bodies, such as tissue sections. Finally, a ganglion explant module quantifies outgrowth by identifying neurites at different distances from the ganglion. Quantification of a diverse series of preparations with WIS-NeuroMath provided data that were well matched with parallel analyses of the same preparations in established software packages such as MetaXpress or NeuronJ. The capabilities of WIS-NeuroMath are demonstrated in a range of applications, including in dissociated and explant cultures and histological analyses on thin and whole-mount sections. WIS-NeuroMath is freely available to academic users, providing a versatile and cost-effective range of solutions for quantifying neurite growth, branching, regeneration, or degeneration under different experimental paradigms.

Image segmentation by probabilistic bottom-up aggregation and cue integration

We present a bottom-up aggregation approach to image segmentation. Beginning with an image, we execute a sequence of steps in which pixels are gradually merged to produce larger and larger regions. In each step, we consider pairs of adjacent regions and provide a probability measure to assess whether or not they should be included in the same segment. Our probabilistic formulation takes into account intensity and texture distributions in a local area around each region. It further incorporates priors based on the geometry of the regions. Finally, posteriors based on intensity and texture cues are combined using “a mixture of experts” formulation. This probabilistic approach is integrated into a graph coarsening scheme, providing a complete hierarchical segmentation of the image. The algorithm complexity is linear in the number of the image pixels and it requires almost no user-tuned parameters. In addition, we provide a novel evaluation scheme for image segmentation algorithms, attempting to avoid human semantic considerations that are out of scope for segmentation algorithms. Using this novel evaluation scheme, we test our method and provide a comparison to several existing segmentation algorithms.

Efficient multilevel eigensolvers with applications to data analysis tasks

Multigrid solvers proved very efficient for solving massive systems of equations in various fields. These solvers are based on iterative relaxation schemes together with the approximation of the "smooth" error function on a coarser level (grid). We present two efficient multilevel eigensolvers for solving massive eigenvalue problems that emerge in data analysis tasks. The first solver, a version of classical algebraic multigrid (AMG), is applied to eigenproblems arising in clustering, image segmentation, and dimensionality reduction, demonstrating an order of magnitude speedup compared to the popular Lanczos algorithm. The second solver is based on a new, much more accurate interpolation scheme. It enables calculating a large number of eigenvectors very inexpensively.

Nearly automatic motion capture system for tracking octopus arm movements in 3D space

Tracking animal movements in 3D space is an essential part of many biomechanical studies. The most popular technique for human motion capture uses markers placed on the skin which are tracked by a dedicated system. However, this technique may be inadequate for tracking animal movements, especially when it is impossible to attach markers to the animal's body either because of its size or shape or because of the environment in which the animal performs its movements. Attaching markers to an animal's body may also alter its behavior. Here we present a nearly automatic markerless motion capture system that overcomes these problems and successfully tracks octopus arm movements in 3D space. The system is based on three successive tracking and processing stages. The first stage uses a recently presented segmentation algorithm to detect the movement in a pair of video sequences recorded by two calibrated cameras. In the second stage, the results of the first stage are processed to produce 2D skeletal representations of the moving arm. Finally, the 2D skeletons are used to reconstruct the octopus arm movement as a sequence of 3D curves varying in time. Motion tracking, segmentation and reconstruction are especially difficult problems in the case of octopus arm movements because of the deformable, non-rigid structure of the octopus arm and the underwater environment in which it moves. Our successful results suggest that the motion-tracking system presented here may be used for tracking other elongated objects.

Identification of novel pro-migratory, cancer-associated genes using quantitative, microscopy-based screening

Background

Cell migration is a highly complex process, regulated by multiple genes, signaling pathways and external stimuli. To discover genes or pharmacological agents that can modulate the migratory activity of cells, screening strategies that enable the monitoring of diverse migratory parameters in a large number of samples are necessary.

Methodology

In the present study, we describe the development of a quantitative, high-throughput cell migration assay, based on a modified phagokinetic tracks (PKT) procedure, and apply it for identifying novel pro-migratory genes in a cancer-related gene library. In brief, cells are seeded on fibronectin-coated 96-well plates, covered with a monolayer of carboxylated latex beads. Motile cells clear the beads, located along their migratory paths, forming tracks that are visualized using an automated, transmitted-light screening microscope. The tracks are then segmented and characterized by multi-parametric, morphometric analysis, resolving a variety of morphological and kinetic features.

Conclusions

In this screen we identified 4 novel genes derived from breast carcinoma related cDNA library, whose over-expression induces major alteration in the migration of the stationary MCF7 cells. This approach can serve for high throughput screening for novel ways to modulate cellular migration in pathological states such as tumor metastasis and invasion.

Shape representation and classification using the poisson equation

We present a novel approach that allows us to reliably compute many useful properties of a silhouette. Our approach assigns, for every internal point of the silhouette, a value reflecting the mean time required for a random walk beginning at the point to hit the boundaries. This function can be computed by solving Poisson's equation, with the silhouette contours providing boundary conditions. We show how this function can be used to reliably extract various shape properties including part structure and rough skeleton, local orientation and aspect ratio of different parts, and convex and concave sections of the boundaries. In addition to this, we discuss properties of the solution and show how to efficiently compute this solution using multigrid algorithms. We demonstrate the utility of the extracted properties by using them for shape classification and retrieval.

Hierarchy and adaptivity in segmenting visual scenes

Finding salient, coherent regions in images is the basis for many visual tasks, and is especially important for object recognition. Human observers perform this task with ease, relying on a system in which hierarchical processing seems to have a critical role. Despite many attempts, computerized algorithms have so far not demonstrated robust segmentation capabilities under general viewing conditions. Here we describe a new, highly efficient approach that determines all salient regions of an image and builds them into a hierarchical structure. Our algorithm, segmentation by weighted aggregation, is derived from algebraic multigrid solvers for physical systems, and consists of fine-to-coarse pixel aggregation. Aggregates of various sizes, which may or may not overlap, are revealed as salient, without predetermining their number or scale. Results using this algorithm are markedly more accurate and significantly faster (linear in data size) than previous approaches.

Atlas guided identification of brain structures by combining 3D segmentation and SVM classification

This study presents a novel automatic approach for the identification of anatomical brain structures in magnetic resonance images (MRI). The method combines a fast multiscale multi-channel three dimensional (3D) segmentation algorithm providing a rich feature vocabulary together with a support vector machine (SVM) based classifier. The segmentation produces a full hierarchy of segments, expressed by an irregular pyramid with only linear time complexity. The pyramid provides a rich, adaptive representation of the image, enabling detection of various anatomical structures at different scales. A key aspect of the approach is the thorough set of multiscale measures employed throughout the segmentation process which are also provided at its end for clinical analysis. These features include in particular the prior probability knowledge of anatomic structures due to the use of an MRI probabilistic atlas. An SVM classifier is trained based on this set of features to identify the brain structures. We validated the approach using a gold standard real brain MRI data set. Comparison of the results with existing algorithms displays the promise of our approach.

Ultrastructural localization of uranium biosorption in Penicillium digitatum by stem X-ray microanalysis

When Penicillium digitatum Saccardo cultures are exposed to aqueous solutions containing soluble uranium salts, considerable amounts of this element are accumulated in the fungal mycelium. The accumulated uranium is retained after thorough rinsing with distilled water but is removed by alkali carbonate solutions. Analysis of thick sections (0.5 microm) of the fungal hyphae with TEM, after incubation in UO(2)Cl(2) solutions of varying concentrations under both light and dark conditions, revealed conspicuous crystal-like deposits in UO(2)Cl(2)-exposed hyphae, but none in the control hyphae. Thick sections were necessary for crystal visualization. Using energy-dispersive X-ray analysis, uranium was detected as the only heavy element in these crystals. Uranium crystal biosorption was localized on the outside surface of the hyphal cell wall (following short exposures to relatively low uranium concentrations) or inside the cell wall (following long exposure to relatively high uranium concentrations). In some cases, crystal-like deposits of uranium salts were located on the outside surface as well as inside the cell.

Photoacoustic study of the green alga Trebouxia in the lichen Ramalina duriaei in vivo

Photosynthetic parameters of the lichen Ramalina duriael were investigated in vivo, by the photoacoustic method using intensity-modulated exciting light (frequency range 5-300 Hz). The photoacoustic signal in intact lichens was similar in its general characteristics to that obtained from intact leaves of higher plants (Poulet et al., Biochim. Biophys. Acta 724, 433-446, 1983). It included two components interpreted to be due to modulated heat and modulated oxygen evolution. The quantum yield of the oxygen evolution component was maximal in the red spectral region and exhibited the 'red drop' decrease at wavelengths larger than 680 nm, similar to observations in higher plants. In contrast to those however, there was a pronounced decrease in this yield in the region below about 600 nm, indicating that pigments absorbing at shorter wavelengths are inefficient energy transfer agents. Similar results were observed for the quantum yield spectrum of photochemical energy storage. Analysis of oxygen diffusion in the symbiont alga, from the modulation frequency dependence of the ratio of oxygen evolution to photothermal signal component is consistent with an average diffusion path of about 4 μm, compared to a smaller, 1 μm, average diffusion path obtained in green leaves.

Removal of Uranium(VI) from Solution by Fungal Biomass and Fungal Wall-Related Biopolymers

Penicillium digitatum mycelium can accumulate uranium from aqueous solutions of uranyl chloride. Azide present during the uptake tests does not inhibit the process. Killing the fungal biomass in boiling water or by treatment with alcohols, dimethyl sulfoxide, or potassium hydroxide increases the uptake capability to about 10,000 parts per million (dry weight). Formaldehyde killing does not enhance the uranium uptake. The inference that wall-binding sites were involved led to the testing of uranium uptake by chitin, cellulose, and cellulose derivatives in microcolumns. All were active, especially chitin.

Effect of water regime on carbon isotope composition of lichens

delta(13)C values of the lichens Ramalina duriaei and Teloschistes villosus collected in their natural habitat were repeatedly measured during 2 years. Results show variations in the stable carbon isotope ratios ((13)C/(12)C). Such variations are correlated to the seasonal rainfall, i.e. low values of delta(13)C of the lichens during the winter and high values of delta(13)C during the dry summer. Relatively low delta(13)C values were obtained also in laboratory experiments with lichens grown under controlled humid conditions and in lichens collected from humid habitats.The variations in carbon isotopes were associated with quantitative metabolic changes. Under humid conditions an increase was obtained in the total amount of the extracted water-soluble fraction of the plant tissues as well as in the relative content of soluble carbohydrates.Analysis of the possible factors which may cause such variations indicates that the quantity of precipitation and the exposure time to high humidity were the main environmental factors causing seasonal variations in the delta(13)C values of the lichens. Such variations are dependent mainly on enzymic reactions and are probably less influenced by purely physicochemical processes. In view of the data presented here the balance between carboxylation and decarboxylation reactions seems to be the major factor for the observed seasonal differences. Winter accumulation of (12)C enriched components causes an over-all decrease in delta(13)C. During the summer, those storage materials are respired with concomitant increase in the delta(13)C of the residual plant material.

Hyphal walls of isolated lichen fungi: autoradiographic localization of precursor incorporation and binding of fluorescein-conjugated lectins

The hyphal walls of three mycobionts, isolated from the lichens Xanthoria parietina, Tornabenia intricata and Sarcogyne sp. were investigated by two techniques: microautoradiography of fungal colonies exposed to radioactive carbohydrate precursors: and binding, in vivo, of fluorescein conjugated lectins to hyphal walls of such colonies. N-[3H] acetylglucosamine was readily incorporated into tips, young hyphal walls and septa of the three mycobionts and the free-living fungus Trichoderma viride, but not into Phytophthora citrophthora, indicating that chitin is a major component of the mycobionts' hyphal walls. All three mycobionts, but neither of the free-living fungi, incorporated [3H] mannose and [3H] mannitol into their hyphal walls. Fluorescein-conjugated wheat germ agglutinin was bound to the hyphal walls of the three mycobionts and T. viride, but not to the walls of P. citrophthora; the binding pattern was similar to the grain pattern obtained in autoradiographs after short N-[3H]acetylglucosamine labelling. As wheat germ agglutinin binds specifically to chitin oligomers, the lectin binding tests further confirmed that chitin is a mycobiont hyphal wall component. Binding characteristics of several fluorescein-conjugated lectins to the three mycobionts indicated that this technique can yield useful information concerning the chemical composition of hyphal wall surfaces.