The future of rapid and automated single-cell data analysis using reference mapping

As the number of single-cell datasets continues to grow rapidly, workflows that map new data to well-curated reference atlases offer enormous promise for the biological community. In this perspective, we discuss key computational challenges and opportunities for single-cell reference-mapping algorithms. We discuss how mapping algorithms will enable the integration of diverse datasets across disease states, molecular modalities, genetic perturbations, and diverse species and will eventually replace manual and laborious unsupervised clustering pipelines.

Mood Disorder Severity and Subtype Classification Using Multimodal Deep Neural Network Models

The subtype diagnosis and severity classification of mood disorder have been made through the judgment of verified assistance tools and psychiatrists. Recently, however, many studies have been conducted using biomarker data collected from subjects to assist in diagnosis, and most studies use heart rate variability (HRV) data collected to understand the balance of the autonomic nervous system on statistical analysis methods to perform classification through statistical analysis. In this research, three mood disorder severity or subtype classification algorithms are presented through multimodal analysis of data on the collected heart-related data variables and hidden features from the variables of time and frequency domain of HRV. Comparing the classification performance of the statistical analysis widely used in existing major depressive disorder (MDD), anxiety disorder (AD), and bipolar disorder (BD) classification studies and the multimodality deep neural network analysis newly proposed in this study, it was confirmed that the severity or subtype classification accuracy performance of each disease improved by 0.118, 0.231, and 0.125 on average. Through the study, it was confirmed that deep learning analysis of biomarker data such as HRV can be applied as a primary identification and diagnosis aid for mental diseases, and that it can help to objectively diagnose psychiatrists in that it can confirm not only the diagnosed disease but also the current mood status.

Deep learning in spatially resolved transcriptfomics: a comprehensive technical view

Spatially resolved transcriptomics (SRT) is a pioneering method for simultaneously studying morphological contexts and gene expression at single-cell precision. Data emerging from SRT are multifaceted, presenting researchers with intricate gene expression matrices, precise spatial details and comprehensive histology visuals. Such rich and intricate datasets, unfortunately, render many conventional methods like traditional machine learning and statistical models ineffective. The unique challenges posed by the specialized nature of SRT data have led the scientific community to explore more sophisticated analytical avenues. Recent trends indicate an increasing reliance on deep learning algorithms, especially in areas such as spatial clustering, identification of spatially variable genes and data alignment tasks. In this manuscript, we provide a rigorous critique of these advanced deep learning methodologies, probing into their merits, limitations and avenues for further refinement. Our in-depth analysis underscores that while the recent innovations in deep learning tailored for SRT have been promising, there remains a substantial potential for enhancement. A crucial area that demands attention is the development of models that can incorporate intricate biological nuances, such as phylogeny-aware processing or in-depth analysis of minuscule histology image segments. Furthermore, addressing challenges like the elimination of batch effects, perfecting data normalization techniques and countering the overdispersion and zero inflation patterns seen in gene expression is pivotal. To support the broader scientific community in their SRT endeavors, we have meticulously assembled a comprehensive directory of readily accessible SRT databases, hoping to serve as a foundation for future research initiatives.

Ursa: A Comprehensive Multiomics Toolbox for High-Throughput Single-Cell Analysis

The burgeoning amount of single-cell data has been accompanied by revolutionary changes to computational methods to map, quantify, and analyze the outputs of these cutting-edge technologies. Many are still unable to reap the benefits of these advancements due to the lack of bioinformatics expertise. To address this issue, we present Ursa, an automated single-cell multiomics R package containing 6 automated single-cell omics and spatial transcriptomics workflows. Ursa allows scientists to carry out post-quantification single or multiomics analyses in genomics, transcriptomics, epigenetics, proteomics, and immunomics at the single-cell level. It serves as a 1-stop analytic solution by providing users with outcomes to quality control assessments, multidimensional analyses such as dimension reduction and clustering, and extended analyses such as pseudotime trajectory and gene-set enrichment analyses. Ursa aims bridge the gap between those with bioinformatics expertise and those without by providing an easy-to-use bioinformatics package for scientists in hoping to accelerate their research potential. Ursa is freely available at https://github.com/singlecellomics/ursa.

Multimodal analysis of methylomics and fragmentomics in plasma cell-free DNA for multi-cancer early detection and localization

Despite their promise, circulating tumor DNA (ctDNA)-based assays for multi-cancer early detection face challenges in test performance, due mostly to the limited abundance of ctDNA and its inherent variability. To address these challenges, published assays to date demanded a very high-depth sequencing, resulting in an elevated price of test. Herein, we developed a multimodal assay called SPOT-MAS (screening for the presence of tumor by methylation and size) to simultaneously profile methylomics, fragmentomics, copy number, and end motifs in a single workflow using targeted and shallow genome-wide sequencing (~0.55×) of cell-free DNA. We applied SPOT-MAS to 738 non-metastatic patients with breast, colorectal, gastric, lung, and liver cancer, and 1550 healthy controls. We then employed machine learning to extract multiple cancer and tissue-specific signatures for detecting and locating cancer. SPOT-MAS successfully detected the five cancer types with a sensitivity of 72.4% at 97.0% specificity. The sensitivities for detecting early-stage cancers were 73.9% and 62.3% for stages I and II, respectively, increasing to 88.3% for non-metastatic stage IIIA. For tumor-of-origin, our assay achieved an accuracy of 0.7. Our study demonstrates comparable performance to other ctDNA-based assays while requiring significantly lower sequencing depth, making it economically feasible for population-wide screening.

Multimodal charting of molecular and functional cell states via in situ electro-sequencing

Paired mapping of single-cell gene expression and electrophysiology is essential to understand gene-to-function relationships in electrogenic tissues. Here, we developed in situ electro-sequencing (electro-seq) that combines flexible bioelectronics with in situ RNA sequencing to stably map millisecond-timescale electrical activity and profile single-cell gene expression from the same cells across intact biological networks, including cardiac and neural patches. When applied to human-induced pluripotent stem-cell-derived cardiomyocyte patches, in situ electro-seq enabled multimodal in situ analysis of cardiomyocyte electrophysiology and gene expression at the cellular level, jointly defining cell states and developmental trajectories. Using machine-learning-based cross-modal analysis, in situ electro-seq identified gene-to-electrophysiology relationships throughout cardiomyocyte development and accurately reconstructed the evolution of gene expression profiles based on long-term stable electrical measurements. In situ electro-seq could be applicable to create spatiotemporal multimodal maps in electrogenic tissues, potentiating the discovery of cell types and gene programs responsible for electrophysiological function and dysfunction.

Building a tissue-unbiased brain template of fiber orientation distribution and tractography with multimodal registration

PURPOSE

Brain templates provide an essential standard space for statistical analysis of brain structure and function. Despite recent advances, diffusion MRI still lacks a template of fiber orientation distribution (FOD) and tractography that is unbiased for both white and gray matter. Therefore, we aim to build up a set of such templates for better white-matter analysis and joint structural and functional analysis.

METHODS

We have developed a multimodal registration method to leverage the complementary information captured by T1 -weighted, T2 -weighted, and diffusion MRI, so that a coherent transformation is generated to register FODs into a common space and average them into a template. Consequently, the anatomically constrained fiber-tracking method was applied to the FOD template to generate a tractography template. Fiber-centered functional connectivity analysis was then performed as an example of the benefits of such an unbiased template.

RESULTS

Our FOD template preserves fine structural details in white matter and also, importantly, clear folding patterns in the cortex and good contrast in the subcortex. Quantitatively, our templates show better individual-template agreement at the whole-brain scale and segmentation scale. The tractography template aligns well with the gray matter, which led to fiber-centered functional connectivity showing high cross-group consistency.

CONCLUSION

We have proposed a novel methodology for building a tissue-unbiased FOD and anatomically constrained tractography template based on multimodal registration. Our templates provide a standard space and statistical platform for not only white-matter analysis but also joint structural and functional analysis, therefore filling an important gap in multimodal neuroimage analysis.

Nanoscale Multimodal Analysis of Sensitive Nanomaterials by Monochromated STEM-EELS in Low-Dose and Cryogenic Conditions

Scanning transmission electron microscopy coupled with electron energy loss spectroscopy (STEM-EELS) provides spatially resolved chemical information down to the atomic scale. However, studying radiation-sensitive specimens such as organic-inorganic composites remains extremely challenging. Here, we analyzed metal-organic framework nanoparticles (nanoMOFs) at low-dose (10 e^-/Ų) and liquid nitrogen temperatures, similar to cryo-TEM conditions usually employed for high-resolution imaging of biological specimens. Our results demonstrate that monochromated STEM-EELS enables damage-free analysis of nanoMOFs, providing in a single experiment, signatures of intact functional groups comparable with infrared, ultraviolet, and X-ray data, with an energy resolution down to 7 meV. The signals have been mapped at the nanoscale (<10 nm) for each of these energy spectral ranges, including the chemical features observed for high energy losses (X-ray range). By controlling beam irradiation and monitoring spectral changes, our work provides insights into the possible pathways of chemical reactions occurring under electron exposure. These results demonstrate the possibilities for characterizing at the nanoscale the chemistry of sensitive systems such as organic and biological materials.

Spatial RNA sequencing methods show high resolution of single cell in cancer metastasis and the formation of tumor microenvironment

Cancer metastasis often leads to death and therapeutic resistance. This process involves the participation of a variety of cell components, especially cellular and intercellular communications in the tumor microenvironment (TME). Using genetic sequencing technology to comprehensively characterize the tumor and TME is therefore key to understanding metastasis and therapeutic resistance. The use of spatial transcriptome sequencing enables the localization of gene expressions and cell activities in tissue sections. By examining the localization change as well as gene expression of these cells, it is possible to characterize the progress of tumor metastasis and TME formation. With improvements of this technology, spatial transcriptome sequencing technology has been extended from local regions to whole tissues, and from single sequencing technology to multimodal analysis combined with a variety of datasets. This has enabled the detection of every single cell in tissue slides, with high resolution, to provide more accurate predictive information for tumor treatments. In this review, we summarize the results of recent studies dealing with new multimodal methods and spatial transcriptome sequencing methods in tumors to illustrate recent developments in the imaging resolution of micro-tissues.

Walking through the city soundscape: an audio-visual analysis of sensory experience for people with psychosis

This article discusses how an aspect of urban environments - sound and noise - is experienced by people walking in the city; it particularly focuses on atypical populations such as people diagnosed with psychosis, who are reported to be particularly sensitive to noisy environments. Through an analysis of video-recordings of naturalistic activities in an urban context and of video-elicitations based on these recordings, the study details the way participants orient to sound and noise in naturalistic settings, and how sound and noise are reported and reexperienced during interviews. By bringing together urban context, psychosis and social interaction, this study shows that, thanks to video recordings and conversation analysis, it is possible to analyse in detail the multimodal organization of action (talk, gesture, gaze, walking bodies) and of the sensory experience(s) of aural factors, as well as the way this organization is affected by the ecology of the situation.

Prognostic factors of traumatic optic neuropathy based on multimodal analysis-Especially the influence of postoperative dressing change and optic nerve blood supply on prognosis

Objective

To investigate the critical prognostic factors of patients with traumatic optic neuropathy (TON) treated with endoscopic transnasal optic canal decompression (ETOCD) and to perform multimodal analysis based on imaging examinations of optical coherence tomography angiography (OCTA) and CT scan. Subsequently, a new prediction model was established.

Methods

The clinical data of 76 patients with TON who underwent decompression surgery with the endoscope-navigation system in the Department of Ophthalmology, Shanghai Ninth People's Hospital from January 2018 to December 2021 were retrospectively analyzed. The clinical data included demographic characteristics, reasons for injury, interval between injury and surgery, multimode imaging information of CT scan and OCTA, including orbital fracture, optical canal fractures, vessel density of optic disc and macula, and the times of postoperative dressing change. Binary logistic regression was used to establish a model for best corrected visual acuity (BCVA) after treatment as a predictor of TON outcome.

Results

Postoperative BCVA improved in 60.5% (46/76) patients and did not improve in 39.5% (30/76) patients. The times of postoperative dressing change had a significant impact on the prognosis. Other factors affecting the prognosis were microvessel density of the central optic disc, the cause of injury, and the microvessel density above the macula. The area under the raw current curves of the predictive model was 0.7596.

Conclusions

The times of dressing changes after the operation, i.e., continuous treatment, is the key factor affecting prognosis. The microvessel density in the center of the optic disc and superior macula, quantitatively analyzed by OCTA, is the prognostic factor of TON and may be used as a prognostic marker of TON.

Interpretation of Depression Detection Models via Feature Selection Methods

Given the prevalence of depression worldwide and its major impact on society, several studies employed artificial intelligence modelling to automatically detect and assess depression. However, interpretation of these models and cues are rarely discussed in detail in the AI community, but have received increased attention lately. In this study, we aim to analyse the commonly selected features using a proposed framework of several feature selection methods and their effect on the classification results, which will provide an interpretation of the depression detection model. The developed framework aggregates and selects the most promising features for modelling depression detection from 38 feature selection algorithms of different categories. Using three real-world depression datasets, 902 behavioural cues were extracted from speech behaviour, speech prosody, eye movement and head pose. To verify the generalisability of the proposed framework, we applied the entire process to depression datasets individually and when combined. The results from the proposed framework showed that speech behaviour features (e.g. pauses) are the most distinctive features of the depression detection model. From the speech prosody modality, the strongest feature groups were F0, HNR, formants, and MFCC, while for the eye activity modality they were left-right eye movement and gaze direction, and for the head modality it was yaw head movement. Modelling depression detection using the selected features (even though there are only 9 features) outperformed using all features in all the individual and combined datasets. Our feature selection framework did not only provide an interpretation of the model, but was also able to produce a higher accuracy of depression detection with a small number of features in varied datasets. This could help to reduce the processing time needed to extract features and creating the model.

Toward a General Framework for Multimodal Big Data Analysis

Multimodal Analytics in Big Data architectures implies compounded configurations of the data processing tasks. Each modality in data requires specific analytics that triggers specific data processing tasks. Scalability can be reached at the cost of an attentive calibration of the resources shared by the different tasks searching for a trade-off with the multiple requirements they impose. We propose a methodology to address multimodal analytics within the same data processing approach to get a simplified architecture that can fully exploit the potential of the parallel processing of Big Data infrastructures. Multiple data sources are first integrated into a unified knowledge graph (KG). Different modalities of data are addressed by specifying ad hoc views on the KG and producing a rewriting of the graph containing merely the data to be processed. Graph traversal and rule extraction are this way boosted. Using graph embeddings methods, the different ad hoc views can be transformed into low-dimensional representation following the same data format. This way a single machine learning procedure can address the different modalities, simplifying the architecture of our system. The experiments we executed demonstrate that our approach reduces the cost of execution and improves the accuracy of analytics.

The Particle Jako ("Like") in Spoken Czech: From Expressing Comparison to Mobilizing Affiliative Responses

This contribution investigates the use of the Czech particle jako (“like”/“as”) in naturally occurring conversations. Inspired by interactional research on unfinished or suspended utterances and on turn-final conjunctions and particles, the analysis aims to trace the possible development of jako from conjunction to a tag-like particle that can be exploited for mobilizing affiliative responses. Traditionally, jako has been described as conjunction used for comparing two elements or for providing a specification of a first element [“X (is) like Y”]. In spoken Czech, however, jako can be flexibly positioned within a speaking turn and does not seem to operate as a coordinating or hypotactic conjunction. As a result, prior studies have described jako as a polyfunctional particle. This article will try to shed light on the meaning of jako in spoken discourse by focusing on its apparent fuzzy or “filler” uses, i.e., when it is found in a mid-turn position in multi-unit turns and in the immediate vicinity of hesitations, pauses, and turn suspensions. Based on examples from mundane, video-recorded conversations and on a sequential and multimodal approach to social interaction, the analyses will first show that jako frequently frames discursive objects that co-participants should respond to. By using jako before a pause and concurrently adopting specific embodied displays, participants can more explicitly seek to mobilize responsive action. Moreover, as jako tends to cluster in multi-unit turns involving the formulation of subjective experience or stance, it can be shown to be specifically designed for mobilizing affiliative responses. Finally, it will be argued that the potential of jako to open up interactive turn spaces can be linked to the fundamental comparative semantics of the original conjunction.

Prospective Quantitative Neuroimaging Analysis of Putative Temporal Lobe Epilepsy

Purpose: A prospective study of individual and combined quantitative imaging applications for lateralizing epileptogenicity was performed in a cohort of consecutive patients with a putative diagnosis of mesial temporal lobe epilepsy (mTLE). Methods: Quantitative metrics were applied to MRI and nuclear medicine imaging studies as part of a comprehensive presurgical investigation. The neuroimaging analytics were conducted remotely to remove bias. All quantitative lateralizing tools were trained using a separate dataset. Outcomes were determined after 2 years. Of those treated, some underwent resection, and others were implanted with a responsive neurostimulation (RNS) device. Results: Forty-eight consecutive cases underwent evaluation using nine attributes of individual or combinations of neuroimaging modalities: 1) hippocampal volume, 2) FLAIR signal, 3) PET profile, 4) multistructural analysis (MSA), 5) multimodal model analysis (MMM), 6) DTI uncertainty analysis, 7) DTI connectivity, and 9) fMRI connectivity. Of the 24 patients undergoing resection, MSA, MMM, and PET proved most effective in predicting an Engel class 1 outcome (>80% accuracy). Both hippocampal volume and FLAIR signal analysis showed 76% and 69% concordance with an Engel class 1 outcome, respectively. Conclusion: Quantitative multimodal neuroimaging in the context of a putative mTLE aids in declaring laterality. The degree to which there is disagreement among the various quantitative neuroimaging metrics will judge whether epileptogenicity can be confined sufficiently to a particular temporal lobe to warrant further study and choice of therapy. Prediction models will improve with continued exploration of combined optimal neuroimaging metrics.

Quantifying skin sensitivity caused by mechanical insults: A review

BACKGROUND

Skin sensitivity (SS) is a commonly occurring response to a range of stimuli, including environmental conditions (e.g., sun exposure), chemical irritants (e.g., soaps and cosmetics), and mechanical forces (e.g., while shaving). From both industry and academia, many efforts have been taken to quantify the characteristics of SS in a standardised manner, but the study is hindered by the lack of an objective definition.

METHODS

A review of the scientific literature regarding different parameters attributed to the loss of skin integrity and linked with exhibition of SS was conducted. Articles included were screened for mechanical stimulation of the skin, with objective quantification of tissue responses using biophysical or imaging techniques. Additionally, studies where cohorts of SS and non-SS individuals were reported have been critiqued.

RESULTS

The findings identified that the structure and function of the stratum corneum and its effective barrier properties are closely associated with SS. Thus, an array of skin tissue responses has been selected for characterization of SS due to mechanical stimuli, including: transepidermal water loss, hydration, redness, temperature, and sebum index. Additionally, certain imaging tools allow quantification of the superficial skin layers, providing structural characteristics underlying SS.

CONCLUSION

This review proposes a multimodal approach for identification of SS, providing a means to characterise skin tissue responses objectively. Optical coherence tomography (OCT) has been suggested as a suitable tool for dermatological research with clinical applications. Such an approach would enhance the knowledge underlying the multifactorial nature of SS and aid the development of personalised solutions in medical and consumer devices.

Integrated analysis of multimodal single-cell data

The simultaneous measurement of multiple modalities represents an exciting frontier for single-cell genomics and necessitates computational methods that can define cellular states based on multimodal data. Here, we introduce "weighted-nearest neighbor" analysis, an unsupervised framework to learn the relative utility of each data type in each cell, enabling an integrative analysis of multiple modalities. We apply our procedure to a CITE-seq dataset of 211,000 human peripheral blood mononuclear cells (PBMCs) with panels extending to 228 antibodies to construct a multimodal reference atlas of the circulating immune system. Multimodal analysis substantially improves our ability to resolve cell states, allowing us to identify and validate previously unreported lymphoid subpopulations. Moreover, we demonstrate how to leverage this reference to rapidly map new datasets and to interpret immune responses to vaccination and coronavirus disease 2019 (COVID-19). Our approach represents a broadly applicable strategy to analyze single-cell multimodal datasets and to look beyond the transcriptome toward a unified and multimodal definition of cellular identity.

Recent advances in single-cell multimodal analysis to study immune cells

Recent advances in single-cell technologies have enabled the profiling of the genome, epigenome, transcriptome and proteome, along with temporal and spatial information of individual cells. These technologies have provided unique opportunities to understand mechanisms underpinning the immune system, such as characterizations of the molecular cell state, how the cell state evolves along its lineage and the impact of spatial location on cell state. In this review, we discuss how these mechanisms have been studied through recent advances in single-cell multimodal technologies.

Method Development for Multimodal Data Corpus Analysis of Expressive Instrumental Music Performance

Musical performance is a multimodal experience, for performers and listeners alike. This paper reports on a pilot study which constitutes the first step toward a comprehensive approach to the experience of music as performed. We aim at bridging the gap between qualitative and quantitative approaches, by combining methods for data collection. The purpose is to build a data corpus containing multimodal measures linked to high-level subjective observations. This will allow for a systematic inclusion of the knowledge of music professionals in an analytic framework, which synthesizes methods across established research disciplines. We outline the methods we are currently developing for the creation of a multimodal data corpus dedicated to the analysis and exploration of instrumental music performance from the perspective of embodied music cognition. This will enable the study of the multiple facets of instrumental music performance in great detail, as well as lead to the development of music creation techniques that take advantage of the cross-modal relationships and higher-level qualities emerging from the analysis of this multi-layered, multimodal corpus. The results of the pilot project suggest that qualitative analysis through stimulated recall is an efficient method for generating higher-level understandings of musical performance. Furthermore, the results indicate several directions for further development, regarding observational movement analysis, and computational analysis of coarticulation, chunking, and movement qualities in musical performance. We argue that the development of methods for combining qualitative and quantitative data are required to fully understand expressive musical performance, especially in a broader scenario in which arts, humanities, and science are increasingly entangled. The future work in the project will therefore entail an increasingly multimodal analysis, aiming to become as holistic as is music in performance.

Multimodal Imaging Mass Spectrometry: Next Generation Molecular Mapping in Biology and Medicine

Imaging mass spectrometry has become a mature molecular mapping technology that is used for molecular discovery in many medical and biological systems. While powerful by itself, imaging mass spectrometry can be complemented by the addition of other orthogonal, chemically informative imaging technologies to maximize the information gained from a single experiment and enable deeper understanding of biological processes. Within this review, we describe MALDI, SIMS, and DESI imaging mass spectrometric technologies and how these have been integrated with other analytical modalities such as microscopy, transcriptomics, spectroscopy, and electrochemistry in a field termed multimodal imaging. We explore the future of this field and discuss forthcoming developments that will bring new insights to help unravel the molecular complexities of biological systems, from single cells to functional tissue structures and organs.

Multimodal vibrational studies of drug uptake in vitro: Is the whole greater than the sum of their parts?

Herein, we investigated the use of multimodal Raman and infrared (IR) spectroscopic microscopy for the elucidation of drug uptake and subsequent cellular responses. Firstly, we compared different methods for the analysis of the combined data. Secondly, we evaluated whether the combined analysis provided enough benefits to justify the fusion of the data. A459 cells inoculated with doxorubicin (DOX) at different times were fixed and analysed using each technique. Raman spectroscopy provided high sensitivity to DOX and enabled an accurate estimation of the drug uptake at each time point, whereas IR provided a better insight into the resultant changes in the biochemical composition of the cell. In terms of benefits of data fusion, 2D correlation analysis allowed the study of the relationship between IR and Raman variables, whereas the joint analysis of IR and Raman enabled the correlation of the different variables to be monitored over time. In summary, the complementary nature of IR and Raman makes the combination of these vibrational techniques an appealing tool to follow drug kinetics and cellular response.

Modeling longitudinal imaging biomarkers with parametric Bayesian multi-task learning

Longitudinal imaging biomarkers are invaluable for understanding the course of neurodegeneration, promising the ability to track disease progression and to detect disease earlier than cross-sectional biomarkers. To properly realize their potential, biomarker trajectory models must be robust to both under-sampling and measurement errors and should be able to integrate multi-modal information to improve trajectory inference and prediction. Here we present a parametric Bayesian multi-task learning based approach to modeling univariate trajectories across subjects that addresses these criteria. Our approach learns multiple subjects' trajectories within a single model that allows for different types of information sharing, that is, coupling, across subjects. It optimizes a combination of uncoupled, fully coupled and kernel coupled models. Kernel-based coupling allows linking subjects' trajectories based on one or more biomarker measures. We demonstrate this using Alzheimer's Disease Neuroimaging Initiative (ADNI) data, where we model longitudinal trajectories of MRI-derived cortical volumes in neurodegeneration, with coupling based on APOE genotype, cerebrospinal fluid (CSF) and amyloid PET-based biomarkers. In addition to detecting established disease effects, we detect disease related changes within the insula that have not received much attention within the literature. Due to its sensitivity in detecting disease effects, its competitive predictive performance and its ability to learn the optimal parameter covariance from data rather than choosing a specific set of random and fixed effects a priori, we propose that our model can be used in place of or in addition to linear mixed effects models when modeling biomarker trajectories. A software implementation of the method is publicly available.

Evaluating the tumor biology of lung adenocarcinoma: A multimodal analysis

We evaluated the relationships among functional imaging modality such as PET-CT and DW-MRI and lung adenocarcinoma pathologic heterogeneity, extent of invasion depth, and tumor cellularity as a marker of tumor microenvironment.In total, 74 lung adenocarcinomas were prospectively included. All patients underwent 18F-fluorodeoxyglucose (FDG) PET-CT and MRI before curative surgery. Pathology revealed 68 stage I tumors, 3 stage II tumors, and 3 stage IIIA tumors. Comprehensive histologic subtyping was performed for all surgically resected tumors. Maximum standardized uptake value (SUVmax) and ADC values were correlated with pathologic grade, extent of invasion, solid tumor size, and tumor cellularity.Mean solid tumor size (low: 1.7 ± 3.0 mm, indeterminate: 13.9 ± 14.2 mm, and high grade: 30.3 ± 13.5 mm) and SUVmax (low: 1.5 ± 0.2, indeterminate: 3.5 ± 2.5, and high grade: 15.3 ± 0) had a significant relationship with pathologic grade based on 95% confidence intervals (P = .01 and P < .01, respectively). SUVmax showed a strong correlation with tumor cellularity (R = 0.713, P < .001), but was not correlated with extent of invasion (R = 0.387, P = .148). A significant and strong positive correlation was observed among SUVmax values and higher cellularity and pathologic grade. ADC did not exhibit a significant relationship with tumor cellularity.Intratumor heterogeneity quantification using a multimodal-multiparametric approach might be effective when tumor volume consists of a real tumor component as well as a non-tumorous stromal component.

Feature Extraction and Selection for Pain Recognition Using Peripheral Physiological Signals

In pattern recognition, the selection of appropriate features is paramount to both the performance and the robustness of the system. Over-reliance on machine learning-based feature selection methods can, therefore, be problematic; especially when conducted using small snapshots of data. The results of these methods, if adopted without proper interpretation, can lead to sub-optimal system design or worse, the abandonment of otherwise viable and important features. In this work, a deep exploration of pain-based emotion classification was conducted to better understand differences in the results of the related literature. In total, 155 different time domain and frequency domain features were explored, derived from electromyogram (EMG), skin conductance levels (SCL), and electrocardiogram (ECG) readings taken from the 85 subjects in response to heat-induced pain. To address the inconsistency in the optimal feature sets found in related works, an exhaustive and interpretable feature selection protocol was followed to obtain a generalizable feature set. Associations between features were then visualized using a topologically-informed chart, called Mapper, of this physiological feature space, including synthesis and comparison of results from previous literature. This topological feature chart was able to identify key sources of information that led to the formation of five main functional feature groups: signal amplitude and power, frequency information, nonlinear complexity, unique, and connecting. These functional groupings were used to extract further insight into observable autonomic responses to pain through a complementary statistical interaction analysis. From this chart, it was observed that EMG and SCL derived features could functionally replace those obtained from ECG. These insights motivate future work on novel sensing modalities, feature design, deep learning approaches, and dimensionality reduction techniques.

Integrated Multi-Assay Culture Model for Stem Cell Chondrogenic Differentiation

Recent osteochondral repair strategies highlight the promise of mesenchymal progenitors, an accessible stem cell source with osteogenic and chondrogenic potential, used in conjunction with biomaterials for tissue engineering. For this, regenerative medicine approaches require robust models to ensure selected cell populations can generate the desired cell type in a reproducible and measurable manner. Techniques for in vitro chondrogenic differentiation are well-established but largely qualitative, relying on sample staining and imaging. To facilitate the in vitro screening of pro-chondrogenic treatments, a 3D micropellet culture combined with three quantitative GAG assays has been developed, with a fourth parallel assay measuring sample content to enable normalisation. The effect of transforming growth factor beta (TGF-β) used to validate this culture format produced a measurable increase in proteoglycan production in the parallel assays, in both 2D and 3D culture configurations. When compared to traditional micropellets, the monolayer format appeared less able to detect changes in cell differentiation, however in-well 3D cultures displayed a significant differential response. Effects on collagen 2 expression confirmed these observations. Based on these results, a microplate format was optimised for 3D culture, in a high-throughput in-well configuration. This model showed improved sensitivity and confirmed the 3D micropellet in-well quantitative assays as an effective differentiation format compatible with streamlined, high-throughput chondrogenic screens.

Building Situation Awareness on the Move: Staff Monitoring Behavior in Clinic Corridors

We conducted a workplace research project on staff mobility in a Swiss hospital outpatient clinic that involved extensive fieldwork and video recordings. The article describes monitoring practices and routines that staff engage in as they walk through the corridors and in and out of the clinic's rooms. The staff perform checks on on-going activity, share their observations with colleagues, and take responsive action while engaged in away-oriented walk or in specific roaming, action-seeking, rallying, and patrolling walk. We argue that these behaviors are closely associated with building and sustaining situation awareness (SA) with regard to the status of the clinic's functioning. They contribute to the coordination of a spatially distributed team that rapidly accomplishes consequential and closely interrelated activities in constantly changing circumstances.

Multimodal analysis of in vivo resorbable CaP bone substitutes by combining histology, SEM, and microcomputed tomography data

This study introduced and demonstrated a new method to investigate the repair process of bone defects using micro- and macroporous beta-tricalcium phosphate (β-TCP) substitutes. Specifically, the new method combined and aligned histology, SEM, and preimplantation microcomputed tomography (mCT) data to accurately characterize tissue phases found in biopsies, and thus better understand the bone repair process. The results included (a) the exact fraction of ceramic remnants (CR); (b) the fraction of ceramic resorbed and substituted by bone (CSB); and (c) the fraction of ceramic resorbed and not substituted by bone (CNSB). The new method allowed in particular the detection and quantification of mineralized tissues within the 1-10 µm micropores of the ceramic ("micro-bone"). The utility of the new method was demonstrated by applying it on biopsies of two β-tricalcium phosphate bone substitute groups with two differing macropore sizes implanted in an ovine model for 6 weeks. The total bone deposition and ceramic resorption of the two substitute groups, having macropore sizes of 510 and 1220 μm, were 25.1 ± 8.1% and 67.5 ± 3.2%, and 24.4 ± 4.1% and 61.4 ± 6.5% for the group having the larger pore size. © 2017 Wiley Periodicals, Inc. J Biomed Mater Res Part B: Appl Biomater, 106B: 1567-1577, 2018.

ENFORCING CO-EXPRESSION IN MULTIMODAL REGRESSION FRAMEWORK

We consider the problem of multimodal data integration for the study of complex neurological diseases (e.g. schizophrenia). Among the challenges arising in such situation, estimating the link between genetic and neurological variability within a population sample has been a promising direction. A wide variety of statistical models arose from such applications. For example, Lasso regression and its multitask extension are often used to fit a multivariate linear relationship between given phenotype(s) and associated observations. Other approaches, such as canonical correlation analysis (CCA), are widely used to extract relationships between sets of variables from different modalities. In this paper, we propose an exploratory multivariate method combining these two methods. More Specifically, we rely on a 'CCA-type' formulation in order to regularize the classical multimodal Lasso regression problem. The underlying motivation is to extract discriminative variables that display are also co-expressed across modalities. We first evaluate the method on a simulated dataset, and further validate it using Single Nucleotide Polymorphisms (SNP) and functional Magnetic Resonance Imaging (fMRI) data for the study of schizophrenia.

VoxelStats: A MATLAB Package for Multi-Modal Voxel-Wise Brain Image Analysis

In healthy individuals, behavioral outcomes are highly associated with the variability on brain regional structure or neurochemical phenotypes. Similarly, in the context of neurodegenerative conditions, neuroimaging reveals that cognitive decline is linked to the magnitude of atrophy, neurochemical declines, or concentrations of abnormal protein aggregates across brain regions. However, modeling the effects of multiple regional abnormalities as determinants of cognitive decline at the voxel level remains largely unexplored by multimodal imaging research, given the high computational cost of estimating regression models for every single voxel from various imaging modalities. VoxelStats is a voxel-wise computational framework to overcome these computational limitations and to perform statistical operations on multiple scalar variables and imaging modalities at the voxel level. VoxelStats package has been developed in Matlab(®) and supports imaging formats such as Nifti-1, ANALYZE, and MINC v2. Prebuilt functions in VoxelStats enable the user to perform voxel-wise general and generalized linear models and mixed effect models with multiple volumetric covariates. Importantly, VoxelStats can recognize scalar values or image volumes as response variables and can accommodate volumetric statistical covariates as well as their interaction effects with other variables. Furthermore, this package includes built-in functionality to perform voxel-wise receiver operating characteristic analysis and paired and unpaired group contrast analysis. Validation of VoxelStats was conducted by comparing the linear regression functionality with existing toolboxes such as glim_image and RMINC. The validation results were identical to existing methods and the additional functionality was demonstrated by generating feature case assessments (t-statistics, odds ratio, and true positive rate maps). In summary, VoxelStats expands the current methods for multimodal imaging analysis by allowing the estimation of advanced regional association metrics at the voxel level.

Localization of function-specific segments of the primary motor pathway in children with Sturge-Weber syndrome: a multimodal imaging analysis

PURPOSE

To explore whether diffusion-weighted imaging (DWI) can localize specific segments of primary motor areas in children with Sturge-Weber syndrome (SWS), this study investigated the corticospinal tract (CST) between precentral gyrus (PCG) and posterior limb of internal capsule (PIC).

MATERIALS AND METHODS

DWI was performed on 32 healthy children and seven children with unilateral SWS affecting the sensorimotor area variably. A hierarchical dendrogram was applied to find PCG-segments uniquely connected to PIC-segments. The resulting PCG-clusters were used to image primary motor pathways in DWI and find metabolic abnormalities of primary motor areas in positron emission tomography (PET) scans.

RESULTS

In healthy children, five PCG-clusters were found to have unique CST courses, corresponding to CST segments of mouth/lip, fingers, and leg/ankle primary motor areas determined by functional magnetic resonance imaging (fMRI). In children with SWS, reduced streamlines in these PCG clusters were highly correlated with glucose-hypometabolism on PET (R(2) = 0.2312, P = 0.0032). Impaired CST segment corresponding to finger movements correlated with severity of hand motor deficit.

CONCLUSION

The presented method can detect impaired CST segments corresponding to specific motor functions in young children who cannot cooperate for fMRI. This approach can be clinically useful for a noninvasive presurgical evaluation of cortical motor areas in such children.

Canonical Correlation Analysis for Feature-Based Fusion of Biomedical Imaging Modalities and Its Application to Detection of Associative Networks in Schizophrenia

Typically data acquired through imaging techniques such as functional magnetic resonance imaging (fMRI), structural MRI (sMRI), and electroencephalography (EEG) are analyzed separately. However, fusing information from such complementary modalities promises to provide additional insight into connectivity across brain networks and changes due to disease. We propose a data fusion scheme at the feature level using canonical correlation analysis (CCA) to determine inter-subject covariations across modalities. As we show both with simulation results and application to real data, multimodal CCA (mCCA) proves to be a flexible and powerful method for discovering associations among various data types. We demonstrate the versatility of the method with application to two datasets, an fMRI and EEG, and an fMRI and sMRI dataset, both collected from patients diagnosed with schizophrenia and healthy controls. CCA results for fMRI and EEG data collected for an auditory oddball task reveal associations of the temporal and motor areas with the N2 and P3 peaks. For the application to fMRI and sMRI data collected for an auditory sensorimotor task, CCA results show an interesting joint relationship between fMRI and gray matter, with patients with schizophrenia showing more functional activity in motor areas and less activity in temporal areas associated with less gray matter as compared to healthy controls. Additionally, we compare our scheme with an independent component analysis based fusion method, joint-ICA that has proven useful for such a study and note that the two methods provide complementary perspectives on data fusion.