Principal axis-based correspondence between multiple cameras for people tracking

Visual surveillance using multiple cameras has attracted increasing interest in recent years. Correspondence between multiple cameras is one of the most important and basic problems which visual surveillance using multiple cameras brings. In this paper, we propose a simple and robust method, based on principal axes of people, to match people across multiple cameras. The correspondence likelihood reflecting the similarity of pairs of principal axes of people is constructed according to the relationship between "ground-points" of people detected in each camera view and the intersections of principal axes detected in different camera views and transformed to the same view. Our method has the following desirable properties: 1) Camera calibration is not needed. 2) Accurate motion detection and segmentation are less critical due to the robustness of the principal axis-based feature to noise. 3) Based on the fused data derived from correspondence results, positions of people in each camera view can be accurately located even when the people are partially occluded in all views. The experimental results on several real video sequences from outdoor environments have demonstrated the effectiveness, efficiency, and robustness of our method.

Dysgnathia, orthognathic surgery and spinal posture

The aim of this study was to evaluate the spine by video rasterstereography before and after orthognathic surgery. Twenty-nine patients (17 patients with a skeletal class III, 7 patients with a skeletal class II, and 5 patients with mandibular asymmetry) were evaluated preoperatively and 1 year postoperatively. Video rasterstereography is a method of back surface measurement and shape analysis using the moire topography. Orthognathic surgery in cases of class III and asymmetry did not lead to significant changes in body posture. In class II patients it led to some changes in body posture, but without orthopaedic consequences. It is concluded that orthognathic surgery causes minimal or no change in body posture.

Adaptive support-weight approach for correspondence search

We present a new window-based method for correspondence search using varying support-weights. We adjust the support-weights of the pixels in a given support window based on color similarity and geometric proximity to reduce the image ambiguity. Our method outperforms other local methods on standard stereo benchmarks.

Scanning scene tunnel for city traversing

This paper proposes a visual representation named scene tunnel for capturing urban scenes along routes and visualizing them on the Internet. We scan scenes with multiple cameras or a fish-eye camera on a moving vehicle, which generates a real scene archive along streets that is more complete than previously proposed route panoramas. Using a translating spherical eye, properly set planes of scanning, and unique parallel-central projection, we explore the image acquisition of the scene tunnel from camera selection and alignment, slit calculation, scene scanning, to image integration. The scene tunnels cover high buildings, ground, and various viewing directions and have uniformed resolutions along the street. The sequentially organized scene tunnel benefits texture mapping onto the urban models. We analyze the shape characteristics in the scene tunnels for designing visualization algorithms. After combining this with a global panorama and forward image caps, the capped scene tunnels can provide continuous views directly for virtual or real navigation in a city. We render scene tunnel dynamically by view warping, fast transmission, and flexible interaction. The compact and continuous scene tunnel facilitates model construction, data streaming, and seamless route traversing on the Internet and mobile devices.

The shape of the anterior and posterior surface of the aging human cornea

PURPOSE

To determine the shape and astigmatism of the posterior corneal surface in a healthy population with age, using Scheimpflug photography corrected for distortion due to the geometry of the Scheimpflug imaging system and the refraction of the anterior corneal surface.

METHODS

Scheimpflug imaging was used to measure in six meridians the cornea of the right eye of 114 subjects, ranging in age from 18 to 65 years.

RESULTS

The average radius of the anterior corneal surface was 7.79+/-0.27 (SD) mm and the average radius of the posterior corneal surface was 6.53+/-0.25 (SD) mm. Both surfaces were found to be flatter horizontally than vertically. The cylindrical component of the posterior surface of 0.33 mm is twice that of the anterior surface (0.16 mm). The asphericity of both the anterior and the posterior surface was independent of the radius of curvature at the vertex, refractive error and gender. In contrast with that of the anterior corneal surface, the asphericity of the posterior corneal surface varied significantly between meridians. With age, the asphericity of both the anterior and the posterior corneal surface changes significantly, which results in a slight peripheral thinning of the cornea.

CONCLUSION

On average, the astigmatism of the posterior corneal surface (-0.305 D) compensates the astigmatism of the anterior corneal surface (0.99 D) with 31%. The results show that the effective refractive index is 1.329, which is lower than values commonly used. There is no correlation between the asphericity of the anterior and the posterior corneal surface. As a result, the shape of the anterior corneal surface provides no definitive basis for knowing the asphericity of the posterior surface.

Real-time range acquisition by adaptive structured light

The goal of this paper is to provide a "self-adaptive" system for real-time range acquisition. Reconstructions are based on a single frame structured light illumination. Instead of using generic, static coding that is supposed to work under all circumstances, system adaptation is proposed. This occurs on-the-fly and renders the system more robust against instant scene variability and creates suitable patterns at startup. A continuous trade-off between speed and quality is made. A weighted combination of different coding cues--based upon pattern color, geometry, and tracking--yields a robust way to solve the correspondence problem. The individual coding cues are automatically adapted within a considered family of patterns. The weights to combine them are based on the average consistency with the result within a small time-window. The integration itself is done by reformulating the problem as a graph cut. Also, the camera-projector configuration is taken into account for generating the projection patterns. The correctness of the range maps is not guaranteed, but an estimation of the uncertainty is provided for each part of the reconstruction. Our prototype is implemented using unmodified consumer hardware only and, therefore, is cheap. Frame rates vary between 10 and 25 fps, dependent on scene complexity.

Clinical experience with a 3D surface patient setup system for alignment of partial-breast irradiation patients

PURPOSE

To assess the utility of surface imaging on patient setup for accelerated partial-breast irradiation (APBI).

METHODS AND MATERIAL

A photogrammetry system was used in parallel to APBI setup by laser and portal imaging. Surface data were acquired after laser and port-film setup for 9 patients. Surfaces were analyzed in comparison to a reference surface from the first treatment session by use of rigid transformations. The surface model after laser setup was used in a simulated photogrammetry setup procedure. In addition, breathing data were acquired by surface acquisition at a frame rate of 7 Hz.

RESULTS

Mean 3D displacement was 7.3 mm (SD, 4.4 mm) and 7.6 mm (SD, 4.2 mm) for laser and port film, respectively. Simulated setup with the photogrammetry system yielded mean displacement of 1 mm (SD, 1.2 mm). Distance analysis resulted in mean distances of 3.7 mm (SD, 4.9 mm), 4.3 mm (SD, 5.6 mm), and 1.6 mm (SD, 2.4 mm) for laser, port film, and photogrammetry, respectively. Breathing motion at isocenter was smaller than 3.7 mm, with a mean of 1.9 mm (SD, 1.1 mm).

CONCLUSIONS

Surface imaging for PBI setup appears promising. Alignment of the 3D breast surface achieved by stereo-photogrammetry shows greater breast topology congruence than when patients are set up by laser or portal imaging. A correlation of breast surface and CTV must be quantitatively established.

Stereo matching with linear superposition of layers

In this paper, we address stereo matching in the presence of a class of non-Lambertian effects, where image formation can be modeled as the additive superposition of layers at different depths. The presence of such effects makes it impossible for traditional stereo vision algorithms to recover depths using direct color matching-based methods. We develop several techniques to estimate both depths and colors of the component layers. Depth hypotheses are enumerated in pairs, one from each layer, in a nested plane sweep. For each pair of depth hypotheses, matching is accomplished using spatial-temporal differencing. We then use graph cut optimization to solve for the depths of both layers. This is followed by an iterative color update algorithm which we proved to be convergent. Our algorithm recovers depth and color estimates for both synthetic and real image sequences.

Total hip replacement with a zirconium oxide ceramic femoral head: a randomised roentgen stereophotogrammetric study

We investigated the wear characteristics and clinical performance of four different total hip joint articulations in 114 patients. Wear and migration was measured by roentgenstereophotogrammetric analysis at five years or at the last follow-up. The mean annual wear was 0.11 mm for a stainless steel/Enduron articulation, 0.34 mm for stainless steel/Hylamer cup, 0.17 mm for zirconium oxide ceramic/Enduron and 0.40 mm for zirconium oxide ceramic/Hylamer. The difference between the groups was significant (p < 0.008) except for stainless steel/Hylamer vs zirconium oxide ceramic/Hylamer (p = 0.26). At present, 12 patients have undergone a revision procedure, four at five years and eight thereafter. No patient who received a stainless steel/Enduron articulation at their primary replacement required revision. Conflicting results have been reported about the performance of the zirconium oxide ceramic femoral head, but our findings suggest that it should not be used with a polymethylmethacrylate acetabular component. Hylamer has already been withdrawn from the market.

3-D shape estimation of DNA molecules from stereo cryo-electron micro-graphs using a projection-steerable snake

We introduce a three-dimensional (3-D) parametric active contour algorithm for the shape estimation of DNA molecules from stereo cryo-electron micrographs. We estimate the shape by matching the projections of a 3-D global shape model with the micrographs; we choose the global model as a 3-D filament with a B-spline skeleton and a specified radial profile. The active contour algorithm iteratively updates the B-spline coefficients, which requires us to evaluate the projections and match them with the micrographs at every iteration. Since the evaluation of the projections of the global model is computationally expensive, we propose a fast algorithm based on locally approximating it by elongated blob-like templates. We introduce the concept of projection-steerability and derive a projection-steerable elongated template. Since the two-dimensional projections of such a blob at any 3-D orientation can be expressed as a linear combination of a few basis functions, matching the projections of such a 3-D template involves evaluating a weighted sum of inner products between the basis functions and the micrographs. The weights are simple functions of the 3-D orientation and the inner-products are evaluated efficiently by separable filtering. We choose an internal energy term that penalizes the average curvature magnitude. Since the exact length of the DNA molecule is known a priori, we introduce a constraint energy term that forces the curve to have this specified length. The sum of these energies along with the image energy derived from the matching process is minimized using the conjugate gradients algorithm. We validate the algorithm using real, as well as simulated, data and show that it performs well.

Globally minimal surfaces by continuous maximal flows

In this paper, we address the computation of globally minimal curves and surfaces for image segmentation and stereo reconstruction. We present a solution, simulating a continuous maximal flow by a novel system of partial differential equations. Existing methods are either grid-biased (graph-based methods) or suboptimal (active contours and surfaces). The solution simulates the flow of an ideal fluid with isotropic velocity constraints. Velocity constraints are defined by a metric derived from image data. An auxiliary potential function is introduced to create a system of partial differential equations. It is proven that the algorithm produces a globally maximal continuous flow at convergence, and that the globally minimal surface may be obtained trivially from the auxiliary potential. The bias of minimal surface methods toward small objects is also addressed. An efficient implementation is given for the flow simulation. The globally minimal surface algorithm is applied to segmentation in 2D and 3D as well as to stereo matching. Results in 2D agree with an existing minimal contour algorithm for planar images. Results in 3D segmentation and stereo matching demonstrate that the new algorithm is robust and free from grid bias.

Matching 2.5D face scans to 3D models

The performance of face recognition systems that use two-dimensional images depends on factors such as lighting and subject's pose. We are developing a face recognition system that utilizes three-dimensional shape information to make the system more robust to arbitrary pose and lighting. For each subject, a 3D face model is constructed by integrating several 2.5D face scans which are captured from different views. 2.5D is a simplified 3D (x, y, z) surface representation that contains at most one depth value (z direction) for every point in the (x, y) plane. Two different modalities provided by the facial scan, namely, shape and texture, are utilized and integrated for face matching. The recognition engine consists of two components, surface matching and appearance-based matching. The surface matching component is based on a modified Iterative Closest Point (ICP) algorithm. The candidate list from the gallery used for appearance matching is dynamically generated based on the output of the surface matching component, which reduces the complexity of the appearance-based matching stage. Three-dimensional models in the gallery are used to synthesize new appearance samples with pose and illumination variations and the synthesized face images are used in discriminant subspace analysis. The weighted sum rule is applied to combine the scores given by the two matching components. Experimental results are given for matching a database of 200 3D face models with 598 2.5D independent test scans acquired under different pose and some lighting and expression changes. These results show the feasibility of the proposed matching scheme.

Computation of the three-dimensional medial surface dynamics of the vocal folds

To increase our understanding of pathological and healthy voice production, quantitative measurement of the medial surface dynamics of the vocal folds is significant, albeit rarely performed because of the inaccessibility of the vocal folds. Using an excised hemilarynx methodology, a new calibration technique, herein referred to as the linear approximate (LA) method, was introduced to compute the three-dimensional coordinates of fleshpoints along the entire medial surface of the vocal fold. The results were compared with results from the direct linear transform. An associated error estimation was presented, demonstrating the improved accuracy of the new method. A test on real data was reported including computation of quantitative measurements of vocal fold dynamics.

Symbolic signatures for deformable shapes

Recognizing classes of objects from their shape is an unsolved problem in machine vision that entails the ability of a computer system to represent and generalize complex geometrical information on the basis of a finite amount of prior data. A practical approach to this problem is particularly difficult to implement, not only because the shape variability of relevant object classes is generally large, but also because standard sensing devices used to capture the real world only provide a partial view of a scene, so there is partial information pertaining to the objects of interest. In this work, we develop an algorithmic framework for recognizing classes of deformable shapes from range data. The basic idea of our component-based approach is to generalize existing surface representations that have proven effective in recognizing specific 3D objects to the problem of object classes using our newly introduced symbolic-signature representation that is robust to deformations, as opposed to a numeric representation that is often tied to a specific shape. Based on this approach, we present a system that is capable of recognizing and classifying a variety of object shape classes from range data. We demonstrate our system in a series of large-scale experiments that were motivated by specific applications in scene analysis and medical diagnosis.

Evaluation of three pose estimation algorithms for model-based Roentgen stereophotogrammetric analysis

Model-based roentgen stereophotogrammetric analysis (RSA) uses a three-dimensional surface model of an implant in order to estimate accurately the pose of that implant from a stereo pair of roentgen images. The technique is based on minimization of the difference between the actually projected contour of an implant and the virtually projected contour of a model of that same implant. The advantage of model-based RSA over conventional marker-based RSA is that it is not necessary to attach markers to the implant. In this paper, three pose estimation algorithms for model-based RSA are evaluated. The algorithms were assessed on the basis of their sensitivities to noise in the actual contour, to the amount of drop-outs in the actual contour, to the number of points in the actual contour and to shrinkage or expansion of the actual contour. The algorithms that were studied are the iterative inverse perspective matching (IIPM) algorithm, an algorithm based on minimization of the difference (DIF) between the actual contour and the virtual contour, and an algorithm based on minimization of the non-overlapping area (NOA) between the actual and virtual contour. The results of the simulation and phantom experiments show that the NOA algorithm does not fulfil the high accuracy that is necessary for model-based RSA. The IIPM and DIF algorithms are robust to the different distortions, making model-based RSA a possible replacement for marker-based RSA.

Sensitivity of finite helical axis parameters to temporally varying realistic motion utilizing an idealized knee model

Various uses of the screw or helical axis have previously been reported in the literature in an attempt to quantify the complex displacements and coupled rotations of in vivo human knee kinematics. Multiple methods have been used by previous authors to calculate the axis parameters, and it has been theorized that the mathematical stability and accuracy of the finite helical axis (FHA) is highly dependent on experimental variability and rotation increment spacing between axis calculations. Previous research has not addressed the sensitivity of the FHA for true in vivo data collection, as required for gait laboratory analysis. This research presents a controlled series of experiments simulating continuous data collection as utilized in gait analysis to investigate the sensitivity of the three-dimensional finite screw axis parameters of rotation, displacement, orientation and location with regard to time step increment spacing, utilizing two different methods for spatial location. Six-degree-of-freedom motion parameters are measured for an idealized rigid body knee model that is constrained to a planar motion profile for the purposes of error analysis. The kinematic data are collected using a multicamera optoelectronic system combined with an error minimization algorithm known as the point cluster method. Rotation about the screw axis is seen to be repeatable, accurate and time step increment insensitive. Displacement along the axis is highly dependent on time step increment sizing, with smaller rotation angles between calculations producing more accuracy. Orientation of the axis in space is accurate with only a slight filtering effect noticed during motion reversal. Locating the screw axis by a projected point onto the screw axis from the mid-point of the finite displacement is found to be less sensitive to motion reversal than finding the intersection of the axis with a reference plane. A filtering effect of the spatial location parameters was noted for larger time step increments during periods of little or no rotation.

Muscle forces analysis in the shoulder mechanism during wheelchair propulsion

This study combines an ergometric wheelchair, a six-camera video motion capture system and a prototype computer graphics based musculoskeletal model (CGMM) to predict shoulder joint loading, muscle contraction force per muscle and the sequence of muscular actions during wheelchair propulsion, and also to provide an animated computer graphics model of the relative interactions. Five healthy male subjects with no history of upper extremity injury participated. A conventional manual wheelchair was equipped with a six-component load cell to collect three-dimensional forces and moments experienced by the wheel, allowing real-time measurement of hand/rim force applied by subjects during normal wheelchair operation. An Expert Vision™ six-camera video motion capture system collected trajectory data of markers attached on anatomical positions. The CGMM was used to simulate and animate muscle action by using an optimization technique combining observed muscular motions with physiological constraints to estimate muscle contraction forces during wheelchair propulsion. The CGMM provides results that satisfactorily match the predictions of previous work, disregarding minor differences which presumably result from differing experimental conditions, measurement technologies and subjects. Specifically, the CGMM shows that the supraspinatus, infraspinatus, anterior deltoid, pectoralis major and biceps long head are the prime movers during the propulsion phase. The middle and posterior deltoid and supraspinatus muscles are responsible for arm return during the recovery phase. CGMM modelling shows that the rotator cuff and pectoralis major play an important role during wheelchair propulsion, confirming the known risk of injury for these muscles during wheelchair propulsion. The CGMM successfully transforms six-camera video motion capture data into a technically useful and visually interesting animated video model of the shoulder musculoskeletal system. The CGMM further yields accurate estimates of muscular forces during motion, indicating that this prototype modelling and analysis technique will aid in study, analysis and therapy of the mechanics and underlying pathomechanics involved in various musculoskeletal overuse syndromes.

A 3-d surface reconstruction approach based on postnonlinear ICA model

Photometric stereo technique deals with the reconstruction of three-dimensional (3-D) shape of an object by using several images of the same surface taken from the same viewpoint but under illuminations from different directions. In this paper, we propose a new photometric stereo scheme based on a new reflectance model and the postnonlinear (PNL) independent components analysis (ICA) method. The proposed nonlinear reflectance model consists of diffuse components and specular components for modeling the surface reflectance of a stereo object in an image. Unlike the previous approaches, these two components are not separated and processed individually in the proposed model. An unsupervised learning adaptation algorithm is developed to estimate the reflectance model based on image intensities. In this algorithm, the PNL ICA method is used to obtain the surface normal on each point of an image. Then, the 3-D surface model is reconstructed based on the estimated surface normal on each point of image by using the enforcing integrability method. Two experiments are performed to assess the performance of the proposed approach. We test our algorithm on synthetically generated images for the reconstruction of surface of objects and on a number of real images captured from the Yale Face Database B. These testing images contain variability due to illumination and varying albedo in each point of surface of human faces. All the experimental results are compared to those of the existing photometric stereo approaches tested on the same images. The results clearly indicate the superiority of the proposed nonlinear reflectance model over the conventional Lambertian model and the other linear hybrid reflectance model.

Is image steganography natural?

Steganography is the art of secret communication. Its purpose is to hide the presence of information, using, for example, images as covers. We experimentally investigate if stego-images, bearing a secret message, are statistically "natural." For this purpose, we use recent results on the statistics of natural images and investigate the effect of some popular steganography techniques. We found that these fundamental statistics of natural images are, in fact, generally altered by the hidden "nonnatural" information. Frequently, the change is consistently biased in a given direction. However, for the class of natural images considered, the change generally falls within the intrinsic variability of the statistics, and, thus, does not allow for reliable detection, unless knowledge of the data hiding process is taken into account. In the latter case, significant levels of detection are demonstrated.

Mobile vs. fixed meniscal bearing in total knee replacement: a randomised radiostereometric study

52 knees scheduled for a total knee arthroplasty were randomised to either a fixed or a mobile polyethylene bearing. The design was identical in all parts. The knee systems used were the Rotaglide Total Knee System (RTK) and the Nuffield Total Knee System (NTK), both from the same manufacturer (Corin Medical Ltd., UK). All knees implanted were uncemented. The patients were followed for 2 years clinically and with radiostereometric analyses to assess migration over time and inducible displacement of the tibial component. Separate analysis of the mobility of the tibial insert in the knees with a mobile bearing was also made. The migration measured with RSA between the 1st and 2nd year expressed as maximum total point motion (MTPM) might predict the risk of loosening of the implant. There were no differences between the groups regarding clinical outcome (HSS Knee score), migration or inducible displacement during the 2 years follow-up. The movement between the tibial tray and the mobile meniscal insert expressed as maximum total point motion (MTPM) was 6.8+/-3.3 mm at the 1st year follow-up.

Stereo CT image compositing methods for lung nodule detection and characterization

RATIONALE AND OBJECTIVES

Stereographic display has been proposed as a possible method of improving performance in reading computed tomographic (CT) examinations acquired for lung cancer screening. Optimizing such displays is important given the large volume of image data that must be evaluated for each of these examinations. This study is designed to explore certain tradeoffs between rendering methods designed for the stereo display of CT images.

MATERIALS AND METHODS

Stereo CT image compositing methods, including distance-weighted averaging, distance-weighted maximum intensity projection (MIP), and conventional MIP, were applied to lung CT images and compared for lung nodule detection and characterization.

RESULTS

Using the Jonckheere test indicated a statistically significant (P < .01) increase in contrast among the three compositing methods. Wilcoxon-Mann-Whitney test showed significant differences in contrast between distance-weighted averaging and conventional MIP (P < .01) and between averaging and distance-weighted MIP (P < .05), but not between distance-weighted MIP and conventional MIP (P > .05). Conventional MIP compositing provided the highest image contrast, but produced ambiguities in local geometric detail and texture, whereas averaging resulted in the lowest contrast, but preserved geometric detail. Distance-weighted MIP partially recovered geometric information, which was lost in images composited by means of conventional MIP.

CONCLUSION

Our results indicate that distance-weighted MIP may be a better choice for nodule detection in stereo lung CT images for its high local contrast and partial preservation of geometric information, whereas compositing by means of distance-weighted averaging is preferable for nodule characterization. The relative clinical value of these compositing methods needs to be evaluated further.

3-D Reconstruction of Microcalcification Clusters Using Stereo Imaging: Algorithm and Mammographic Unit Calibration

The three-dimensional (3-D) shape of microcalcification clusters is an important indicator in early breast cancer detection. In fact, there is a relationship between the cluster topology and the type of lesion (malignant or benign). This paper presents a 3-D reconstruction method for such clusters using two 2-D views acquired during standard mammographic examinations. For this purpose, the mammographic unit was modeled using a camera with virtual optics. This model was used to calibrate the acquisition unit and then to reconstruct the clusters in the 3-D space after microcalcification segmentation and matching. The proposed model is hardware independent since it is suitable for digital mammographic units with different geometries and with various physical acquisition principles. Three-dimensional reconstruction results are presented here to prove the validity of the method. Tests were first performed using a phantom with a well-known geometry. The latter contained X-ray opaque glass balls representing microcalcifications. The positions of these balls were reconstructed with a 16.25-microm mean accuracy. This very high inherent algorithm accuracy is more than enough for a precise 3-D cluster representation. Further validation tests were carried out using a second phantom including a spherical cluster. This phantom was built with materials simulating the behavior of both mammary tissue and microcalcifications toward Xrays. The reconstructed shape was effectively spherical. Finally, reconstructions were carried out for real clusters and their results are also presented.

TORNADO: omnistereo video imaging with rotating optics

One of the key techniques for vision-based communication is omnidirectional stereo (omnistereo) imaging, in which stereoscopic images for an arbitrary horizontal direction are captured and presented according to the viewing direction of the observer. Although omnistereo models have been surveyed in several studies, few omnistereo sensors have actually been implemented. In this paper, a practical method for capturing omnistereo video sequences using rotating optics is proposed and evaluated. The rotating optics system consists of prism sheets, circular or linear polarizing films, and a hyperboloidal mirror. This system has two different modes of operation with regard to the separation of images for the left and right eyes. In the high-speed shutter mode, images are separated using postimage processing, while, in the low-speed shutter mode, the image separation is completed by optics. By capturing actual images, we confirmed the effectiveness of the methods.

Augmented virtuality based on stereoscopic reconstruction in multimodal image-guided neurosurgery: methods and performance evaluation

Displaying anatomical and physiological information derived from preoperative medical images in the operating room is critical in image-guided neurosurgery. This paper presents a new approach referred to as augmented virtuality (AV) for displaying intraoperative views of the operative field over three-dimensional (3-D) multimodal preoperative images onto an external screen during surgery. A calibrated stereovision system was set up between the surgical microscope and the binocular tubes. Three-dimensional surface meshes of the operative field were then generated using stereopsis. These reconstructed 3-D surface meshes were directly displayed without any additional geometrical transform over preoperative images of the patient in the physical space. Performance evaluation was achieved using a physical skull phantom. Accuracy of the reconstruction method itself was shown to be within 1 mm (median: 0.76 mm +/- 0.27), whereas accuracy of the overall approach was shown to be within 3 mm (median: 2.29 mm +/- 0.59), including the image-to-physical space registration error. We report the results of six surgical cases where AV was used in conjunction with augmented reality. AV not only enabled vision beyond the cortical surface but also gave an overview of the surgical area. This approach facilitated understanding of the spatial relationship between the operative field and the preoperative multimodal 3-D images of the patient.

Defocus morphing in real aperture images

A new concept called defocus morphing in real aperture images is introduced. View morphing is an existing example of shape-preserving image morphing based on the motion cue. It is proved that images can also be morphed based on the depth-related defocus cue. This illustrates that the morphing operation is not necessarily a geometric process alone; one can also perform a photometry-based morphing wherein the shape information is implicitly buried in the image intensity field. A theoretical understanding of the defocus morphing process is presented. It is shown mathematically that, given two observations of a three-dimensional scene for different camera parameter settings, we can obtain a virtual observation for any camera parameter setting through a simple nonlinear combination of these observations.