Integrating Egocentric and Robotic Vision for Object Identification Using Siamese Networks and Superquadric Estimations in Partial Occlusion Scenarios

This paper introduces a novel method that enables robots to identify objects based on user gaze, tracked via eye-tracking glasses. This is achieved without prior knowledge of the objects' categories or their locations and without external markers. The method integrates a two-part system: a category-agnostic object shape and pose estimator using superquadrics and Siamese networks. The superquadrics-based component estimates the shapes and poses of all objects, while the Siamese network matches the object targeted by the user's gaze with the robot's viewpoint. Both components are effectively designed to function in scenarios with partial occlusions. A key feature of the system is the user's ability to move freely around the scenario, allowing dynamic object selection via gaze from any position. The system is capable of handling significant viewpoint differences between the user and the robot and adapts easily to new objects. In tests under partial occlusion conditions, the Siamese networks demonstrated an 85.2% accuracy in aligning the user-selected object with the robot's viewpoint. This gaze-based Human-Robot Interaction approach demonstrates its practicality and adaptability in real-world scenarios.

Reconstructing Superquadrics from Intensity and Color Images

The task of reconstructing 3D scenes based on visual data represents a longstanding problem in computer vision. Common reconstruction approaches rely on the use of multiple volumetric primitives to describe complex objects. Superquadrics (a class of volumetric primitives) have shown great promise due to their ability to describe various shapes with only a few parameters. Recent research has shown that deep learning methods can be used to accurately reconstruct random superquadrics from both 3D point cloud data and simple depth images. In this paper, we extended these reconstruction methods to intensity and color images. Specifically, we used a dedicated convolutional neural network (CNN) model to reconstruct a single superquadric from the given input image. We analyzed the results in a qualitative and quantitative manner, by visualizing reconstructed superquadrics as well as observing error and accuracy distributions of predictions. We showed that a CNN model designed around a simple ResNet backbone can be used to accurately reconstruct superquadrics from images containing one object, but only if one of the spatial parameters is fixed or if it can be determined from other image characteristics, e.g., shadows. Furthermore, we experimented with images of increasing complexity, for example, by adding textures, and observed that the results degraded only slightly. In addition, we show that our model outperforms the current state-of-the-art method on the studied task. Our final result is a highly accurate superquadric reconstruction model, which can also reconstruct superquadrics from real images of simple objects, without additional training.

Enhanced Processing of Regrind as Recycling Material in Single-Screw Extruders

Regrind processing poses challenges for single-screw extruders due to the irregularly shaped particles. For grooved feed zones, the output is lessened by the reduction of bulk density in comparison to virgin material. Simultaneously, the melt temperature increases, reducing the extruder's process window. Through experimental investigations on a test stand, a novel feed zone geometry (nominal diameter 35 mm) is developed. It aligns the regrind's specific throughput with that of virgin material. The regrind processing window is essentially increased. As the solids conveying in the novel feed zone cannot be simulated with existing methods, numerical simulations using the discrete element method are performed. Since plastic deformation occurs in the novel feed zone geometry, a new hysteresis contact model is developed. In addition to spheres, the regrind and virgin particles are modeled as superquadrics to better approximate the irregular shape. The new contact model's simulation results show excellent agreement with experimental compression tests. The throughput of the extruder simulations is considerably underestimated when using spheres to represent the real particles than when using irregularly shaped superquadrics. Corresponding advantages can be seen especially for virgin material.

Robust Tracing and Visualization of Heterogeneous Microvascular Networks

Advances in high-throughput imaging allow researchers to collect three-dimensional images of whole organ microvascular networks. These extremely large images contain networks that are highly complex, time consuming to segment, and difficult to visualize. In this paper, we present a framework for segmenting and visualizing vascular networks from terabyte-sized three-dimensional images collected using high-throughput microscopy. While these images require terabytes of storage, the volume devoted to the fiber network is ≈ 4 percent of the total volume size. While the networks themselves are sparse, they are tremendously complex, interconnected, and vary widely in diameter. We describe a parallel GPU-based predictor-corrector method for tracing filaments that is robust to noise and sampling errors common in these data sets. We also propose a number of visualization techniques designed to convey the complex statistical descriptions of fibers across large tissue sections-including commonly studied microvascular characteristics, such as orientation and volume.

Modeling the shape hierarchy for visually guided grasping

The monkey anterior intraparietal area (AIP) encodes visual information about three-dimensional object shape that is used to shape the hand for grasping. We modeled shape tuning in visual AIP neurons and its relationship with curvature and gradient information from the caudal intraparietal area (CIP). The main goal was to gain insight into the kinds of shape parameterizations that can account for AIP tuning and that are consistent with both the inputs to AIP and the role of AIP in grasping. We first experimented with superquadric shape parameters. We considered superquadrics because they occupy a role in robotics that is similar to AIP, in that superquadric fits are derived from visual input and used for grasp planning. We also experimented with an alternative shape parameterization that was based on an Isomap dimension reduction of spatial derivatives of depth (i.e., distance from the observer to the object surface). We considered an Isomap-based model because its parameters lacked discontinuities between similar shapes. When we matched the dimension of the Isomap to the number of superquadric parameters, the superquadric model fit the AIP data somewhat more closely. However, higher-dimensional Isomaps provided excellent fits. Also, we found that the Isomap parameters could be approximated much more accurately than superquadric parameters by feedforward neural networks with CIP-like inputs. We conclude that Isomaps, or perhaps alternative dimension reductions of visual inputs to AIP, provide a promising model of AIP electrophysiology data. Further work is needed to test whether such shape parameterizations actually provide an effective basis for grasp control.