SpringLS: a deformable model representation to provide interoperability between meshes and level sets

Variational Level Set Evolution for Non-Rigid 3D Reconstruction From a Single Depth Camera

We present a framework for real-time 3D reconstruction of non-rigidly moving surfaces captured with a single RGB-D camera. Based on the variational level set method, it warps a given truncated signed distance field (TSDF) to a target TSDF via gradient flow without explicit correspondence search. We optimize an energy that contains a data term which steers towards voxel-wise alignment. To ensure geometrically consistent reconstructions, we develop and compare different strategies, namely an approximately Killing vector field regularizer, gradient flow in Sobolev space and newly devised accelerated optimization. The underlying TSDF evolution makes our approach capable of capturing rapid motions, topological changes and interacting agents, but entails loss of data association. To recover correspondences, we propose to utilize the lowest-frequency Laplacian eigenfunctions of the TSDFs, which encode inherent deformation patterns. For moderate motions we are able to obtain implicit associations via a term that imposes voxel-wise eigenfunction alignment. This is not sufficient for larger motions, so we explicitly estimate voxel correspondences via signature matching of lower-dimensional eigenfunction embeddings. We carry out qualitative and quantitative evaluation of our geometric reconstruction fidelity and voxel correspondence accuracy, demonstrating advantages over related techniques in handling topological changes and fast motions.

Spring level sets: a deformable model representation to provide interoperability between meshes and level sets

A new type of deformable model is presented that merges meshes and level sets into one representation to provide interoperability between methods designed for either. This includes the ability to circumvent the CFL time step restriction for methods that require large step sizes. The key idea is to couple a constellation of disconnected triangular surface elements (springls) with a level set that tracks the moving constellation. The target application for Spring Level Sets (SpringLS) is to implement comprehensive imaging pipelines that require a mixture of deformable model representations to achieve the best performance. We demonstrate how to implement key components of a comprehensive imaging pipeline with SpringLS, including image segmentation, registration, tracking, and atlasing.

Multi-object spring level sets (MUSCLE)

A new data structure is presented for geometrically modeling multi-objects. The model can exhibit elastic and fluid-like behavior to enable interpretability between tasks that require both deformable registration and active contour segmentation. The data structure consists of a label mask, distance field, and springls (a constellation of disconnected triangles). The representation has sub-voxel precision, is parametric, re-meshes, tracks point correspondences, and guarantees no self-intersections, air-gaps, or overlaps between adjacent structures. In this work, we show how to apply existing registration algorithms and active contour segmentation to the data structure; and as a demonstration, the data structure is used to segment cortical and subcortical structures (74 total) in the human brain.