A fine-grained orthodontics segmentation model for 3D intraoral scan data

With the widespread application of digital orthodontics in the diagnosis and treatment of oral diseases, more and more researchers focus on the accurate segmentation of teeth from intraoral scan data. The accuracy of the segmentation results will directly affect the follow-up diagnosis of dentists. Although the current research on tooth segmentation has achieved promising results, the 3D intraoral scan datasets they use are almost all indirect scans of plaster models, and only contain limited samples of abnormal teeth, so it is difficult to apply them to clinical scenarios under orthodontic treatment. The current issue is the lack of a unified and standardized dataset for analyzing and validating the effectiveness of tooth segmentation. In this work, we focus on deformed teeth segmentation and provide a fine-grained tooth segmentation dataset (3D-IOSSeg). The dataset consists of 3D intraoral scan data from more than 200 patients, with each sample labeled with a fine-grained mesh unit. Meanwhile, 3D-IOSSeg meticulously classified every tooth in the upper and lower jaws. In addition, we propose a fast graph convolutional network for 3D tooth segmentation named Fast-TGCN. In the model, the relationship between adjacent mesh cells is directly established by the naive adjacency matrix to better extract the local geometric features of the tooth. Extensive experiments show that Fast-TGCN can quickly and accurately segment teeth from the mouth with complex structures and outperforms other methods in various evaluation metrics. Moreover, we present the results of multiple classical tooth segmentation methods on this dataset, providing a comprehensive analysis of the field. All code and data will be available at https://github.com/MIVRC/Fast-TGCN.

Automatic Detection of Tooth-Gingiva Trim Lines on Dental Surfaces

Detecting the tooth-gingiva trim line from a dental surface plays a critical role in dental treatment planning and aligner 3D printing. Existing methods treat this task as a segmentation problem, which is resolved with geometric deep learning based mesh segmentation techniques. However, these methods can only provide indirect results (i.e., segmented teeth) and suffer from unsatisfactory accuracy due to the incapability of making full use of high-resolution dental surfaces. To this end, we propose a two-stage geometric deep learning framework for automatically detecting tooth-gingiva trim lines from dental surfaces. Our framework consists of a trim line proposal network (TLP-Net) for predicting an initial trim line from the low-resolution dental surface as well as a trim line refinement network (TLR-Net) for refining the initial trim line with the information from the high-resolution dental surface. Specifically, our TLP-Net predicts the initial trim line by fusing the multi-scale features from a U-Net with a proposed residual multi-scale attention fusion module. Moreover, we propose feature bridge modules and a trim line loss to further improve the accuracy. The resulting trim line is then fed to our TLR-Net, which is a deep-based LDDMM model with the high-resolution dental surface as input. In addition, dense connections are incorporated into TLR-Net for improved performance. Our framework provides an automatic solution to trim line detection by making full use of raw high-resolution dental surfaces. Extensive experiments on a clinical dental surface dataset demonstrate that our TLP-Net and TLR-Net are superior trim line detection methods and outperform cutting-edge methods in both qualitative and quantitative evaluations.

A Tooth Segmentation Method Based on Multiple Geometric Feature Learning

Tooth segmentation is an important aspect of virtual orthodontic systems. In some existing studies using deep learning-based tooth segmentation methods, the feature learning of point coordinate information and normal vector information is not effectively distinguished. This will lead to the feature information of these two methods not producing complementary intermingling. To address this problem, a tooth segmentation method based on multiple geometric feature learning is proposed in this paper. First, the spatial transformation (T-Net) module is used to complete the alignment of dental model mesh features. Second, a multiple geometric feature learning module is designed to encode and enhance the centroid coordinates and normal vectors of each triangular mesh to highlight the differences between geometric features of different meshes. Finally, for local to global fusion features, feature downscaling and channel optimization are accomplished layer by layer using multilayer perceptron (MLP) and efficient channel attention (ECA). The experimental results show that our algorithm achieves better accuracy and efficiency of tooth segmentation and can assist dentists in their treatment work.

Two-Stream Graph Convolutional Network for Intra-Oral Scanner Image Segmentation

Precise segmentation of teeth from intra-oral scanner images is an essential task in computer-aided orthodontic surgical planning. The state-of-the-art deep learning-based methods often simply concatenate the raw geometric attributes (i.e., coordinates and normal vectors) of mesh cells to train a single-stream network for automatic intra-oral scanner image segmentation. However, since different raw attributes reveal completely different geometric information, the naive concatenation of different raw attributes at the (low-level) input stage may bring unnecessary confusion in describing and differentiating between mesh cells, thus hampering the learning of high-level geometric representations for the segmentation task. To address this issue, we design a two-stream graph convolutional network (i.e., TSGCN), which can effectively handle inter-view confusion between different raw attributes to more effectively fuse their complementary information and learn discriminative multi-view geometric representations. Specifically, our TSGCN adopts two input-specific graph-learning streams to extract complementary high-level geometric representations from coordinates and normal vectors, respectively. Then, these single-view representations are further fused by a self-attention module to adaptively balance the contributions of different views in learning more discriminative multi-view representations for accurate and fully automatic tooth segmentation. We have evaluated our TSGCN on a real-patient dataset of dental (mesh) models acquired by 3D intraoral scanners. Experimental results show that our TSGCN significantly outperforms state-of-the-art methods in 3D tooth (surface) segmentation.

Automatic recognition of landmarks on digital dental models

Fundamental principle in improving Dental and Orthodontic treatments is the ability to quantitatively assess and cross-compare their outcomes. Such assessments require calculating distances and angles from 3D coordinates of dental landmarks. The costly and repetitive task of hand-labelling dental models hinder studies requiring large sample size to penetrate statistical noise. We have developed techniques and a software implementing these techniques to map out automatically, 3D dental scans. This process is divided into consecutive steps - determining a model's orientation, separating and identifying the individual tooth and finding landmarks on each tooth - described in this paper. The examples to demonstrate the techniques, software and discussions on remaining issues are provided as well. The software is originally designed to automate Modified Huddard Bodemham (MHB) landmarking for assessing cleft lip/palate patients. Currently only MHB landmarks are supported, however it is extendable to any predetermined landmarks. The software, coupled with intra-oral scanning innovation, should supersede the arduous and error prone plaster model and calipers approach to Dental research, and provide a stepping-stone towards automation of routine clinical assessments such as "index of orthodontic treatment need" (IOTN).

TSegNet: An efficient and accurate tooth segmentation network on 3D dental model

Automatic and accurate segmentation of dental models is a fundamental task in computer-aided dentistry. Previous methods can achieve satisfactory segmentation results on normal dental models; however, they fail to robustly handle challenging clinical cases such as dental models with missing, crowding, or misaligned teeth before orthodontic treatments. In this paper, we propose a novel end-to-end learning-based method, called TSegNet, for robust and efficient tooth segmentation on 3D scanned point cloud data of dental models. Our algorithm detects all the teeth using a distance-aware tooth centroid voting scheme in the first stage, which ensures the accurate localization of tooth objects even with irregular positions on abnormal dental models. Then, a confidence-aware cascade segmentation module in the second stage is designed to segment each individual tooth and resolve ambiguities caused by aforementioned challenging cases. We evaluated our method on a large-scale real-world dataset consisting of dental models scanned before or after orthodontic treatments. Extensive evaluations, ablation studies and comparisons demonstrate that our method can generate accurate tooth labels robustly in various challenging cases and significantly outperforms state-of-the-art approaches by 6.5% of Dice Coefficient, 3.0% of F1 score in term of accuracy, while achieving 20 times speedup of computational time.

Tooth segmentation and gingival tissue deformation framework for 3D orthodontic treatment planning and evaluating

In this study, we propose an integrated tooth segmentation and gingival tissue deformation simulation framework used to design and evaluate the orthodontic treatment plan especially with invisible aligners. Firstly, the bio-characteristics information of the digital impression is analyzed quantitatively and demonstrated visually. With the derived information, the transitional regions of tooth-tooth and tooth-gingiva are extracted as the solution domain of the segmentation boundaries. Then, a boundary detection approach is proposed, which is used for the tooth segmentation and region division of the digital impression. After tooth segmentation, we propose the deformation simulation framework driven by energy function based on the biological deformation properties of gingival tissues. The correctness and availability of the proposed segmentation and gingival tissue deformation simulation framework are demonstrated with typical cases and qualitative analysis. Experimental results show that segmentation boundaries calculated by the proposed method are accurate, and local details of the digital impression under study are preserved well during deformation simulation. Qualitative analysis results of the gingival tissues' surface area and volume variations indicate that the proposed gingival tissue deformation simulation framework is consistent with the clinical gingival tissue deformation characteristics, and it can be used to predict the rationality of the treatment plan from both visual inspection and numerical simulation. The proposed tooth segmentation and gingival tissue deformation simulation framework is shown to be effective and has good practicability, but accurate quantitative analysis based on clinical results is still an open problem in this study. Combined with tooth rearrangement steps, it can be used to design the orthodontic treatment plan, and to output the data for production of invisible aligners. Graphical abstract.

Accelerating Chan-Vese model with cross-modality guided contrast enhancement for liver segmentation

Accurate and fast liver segmentation remains a challenging and important task for clinicians. Segmentation algorithms are slow and inaccurate due to noise and low quality images in computed tomography (CT) abdominal scans. Chan-Vese is an active contour based powerful and flexible method for image segmentation due to superior noise robustness. However, it is quite slow due to time-consuming partial differential equations, especially for large medical datasets. This can pose a problem for a real-time implementation of liver segmentation and hence, an efficient parallel implementation is highly desirable. Another important aspect is the contrast of CT liver images. Liver slices are sometimes very low in contrast which reduces the overall quality of liver segmentation. Hence, we implement cross-modality guided liver contrast enhancement as a pre-processing step to liver segmentation. GPU implementation of Chan-Vese improves average speedup by 99.811 (± 7.65) times and 14.647 (± 1.155) times with and without enhancement respectively in comparison with the CPU. Average dice, sensitivity and accuracy of liver segmentation are 0.656, 0.816 and 0.822 respectively on the original liver images and 0.877, 0.964 and 0.956 respectively on the enhanced liver images improving the overall quality of liver segmentation.

3D Intelligent Scissors for Dental Mesh Segmentation

Teeth segmentation is a crucial technologic component of the digital dentistry system. The limitations of the live-wire segmentation include two aspects: (1) computing the wire as the segmentation boundary is time-consuming and (2) a great deal of interactions for dental mesh is inevitable. For overcoming these disadvantages, 3D intelligent scissors for dental mesh segmentation based on live-wire is presented. Two tensor-based anisotropic metrics for making wire lie at valleys and ridges are defined, and a timesaving anisotropic Dijkstra is adopted. Besides, to improve with the smoothness of the path tracking back by the traditional Dijkstra, a 3D midpoint smoothing algorithm is proposed. Experiments show that the method is effective for dental mesh segmentation and the proposed tool outperforms in time complexity and interactivity.

3D Tooth Segmentation and Labeling Using Deep Convolutional Neural Networks

In this paper, we present a novel approach for 3D dental model segmentation via deep Convolutional Neural Networks (CNNs). Traditional geometry-based methods tend to receive undesirable results due to the complex appearance of human teeth (e.g., missing/rotten teeth, feature-less regions, crowding teeth, extra medical attachments, etc.). Furthermore, labeling of individual tooth is hardly enabled in traditional tooth segmentation methods. To address these issues, we propose to learn a generic and robust segmentation model by exploiting deep Neural Networks, namely NNs. The segmentation task is achieved by labeling each mesh face. We extract a set of geometry features as face feature representations. In the training step, the network is fed with those features, and produces a probability vector, of which each element indicates the probability a face belonging to the corresponding model part. To this end, we extensively experiment with various network structures, and eventually arrive at a 2-level hierarchical CNNs structure for tooth segmentation: one for teeth-gingiva labeling and the other for inter-teeth labeling. Further, we propose a novel boundary-aware tooth simplification method to significantly improve efficiency in the stage of feature extraction. After CNNs prediction, we do graph-based label optimization and further refine the boundary with an improved version of fuzzy clustering. The accuracy of our mesh labeling method exceeds that of the state-of-art geometry-based methods, reaching 99.06 percent measured by area which is directly applicable in orthodontic CAD systems. It is also robust to any possible foreign matters on model surface, e.g., air bubbles, dental accessories, and many more.

Dental hard tissue morphological segmentation with sparse representation-based classifier

In the field of dental image processing and analysis, automatic segmentation results of dental hard tissue can provide a useful reference for the clinical diagnosis and treatment process. However, the segmentation accuracy is greatly affected due to the limitation of imaging conditions in the oral environment, as well as the complexity of dental hard tissue topology. To further improve the precision of dental hard tissue segmentation, a novel algorithm was presented by using the sparse representation-based classifier and mathematical morphology operations. First, the captured dental image was preprocessed to eliminate the impact of imbalance local illumination. Then, the preliminary dental hard tissue areas were calculated as the initial marker regions based on color characteristics analysis, and the sparse representation-based classifier was applied sequentially to optimize the initial marker regions combined with certain morphological operations. Finally, a modified marker-controlled watershed transform was employed to segment dental hard tissue regions on the basis of the optimized marker regions, and the final results were obtained after homogeneous region merging. The experimental results show that our method has better adaptability and robustness than existing state-of-the-art methods. Graphical abstract.

Automatic tooth segmentation of dental mesh using a transverse plane

This paper proposes an automatic method to separate the gingiva and individual teeth from a dental mesh. We define a transverse plane that produces a cross-section of tooth lingual and labial surfaces, preserving the shape of individual teeth. The upper vertices from the transverse plane, which belong to the tooth, are projected onto the transverse plane, and partitioned into individual teeth. We apply region growing to the remaining non-segmented parts to determine the cluster the vertices belong to, and the proposed approach is fully automatic, i.e., segmentation does not require user interaction for feature point search or tooth boundary markers. The proposed segmentation method is applied to several dental mesh models to demonstrate its robustness.

Tooth separation from dental model using segmentation field

Tooth segmentation on dental model is an essential step of computer-aided-design systems for orthodontic virtual treatment planning. However, efficiently identifying cutting boundary to separate tooth from dental model still remains a challenge, due to various geometrical shapes of teeth, complex tooth arrangements and varying degrees of crowding problem. Most segmentation approaches presented before are not able to achieve a balance between fine segmentation results and simple operating procedure. In this article, we present a novel and efficient framework that achieves tooth segmentation based on the segmentation field. Specially, the candidate cutting boundaries are able to be detected from the concave regions with large variations of field data. The sensitivity to concave seams of segmentation field facilitates effective tooth partition, as well as avoids obtaining appropriate curvature threshold value, which is unreliable in some case. The experiments indicate that, our tooth segmentation algorithm is robust to different dental models with severe crowding problems and poor distinction.

Interactive Tooth Separation from Dental Model Using Segmentation Field

Tooth segmentation on dental model is an essential step of computer-aided-design systems for orthodontic virtual treatment planning. However, fast and accurate identifying cutting boundary to separate teeth from dental model still remains a challenge, due to various geometrical shapes of teeth, complex tooth arrangements, different dental model qualities, and varying degrees of crowding problems. Most segmentation approaches presented before are not able to achieve a balance between fine segmentation results and simple operating procedures with less time consumption. In this article, we present a novel, effective and efficient framework that achieves tooth segmentation based on a segmentation field, which is solved by a linear system defined by a discrete Laplace-Beltrami operator with Dirichlet boundary conditions. A set of contour lines are sampled from the smooth scalar field, and candidate cutting boundaries can be detected from concave regions with large variations of field data. The sensitivity to concave seams of the segmentation field facilitates effective tooth partition, as well as avoids obtaining appropriate curvature threshold value, which is unreliable in some case. Our tooth segmentation algorithm is robust to dental models with low quality, as well as is effective to dental models with different levels of crowding problems. The experiments, including segmentation tests of varying dental models with different complexity, experiments on dental meshes with different modeling resolutions and surface noises and comparison between our method and the morphologic skeleton segmentation method are conducted, thus demonstrating the effectiveness of our method.

Automatic Tooth Segmentation of Dental Mesh Based on Harmonic Fields

An important preprocess in computer-aided orthodontics is to segment teeth from the dental models accurately, which should involve manual interactions as few as possible. But fully automatic partition of all teeth is not a trivial task, since teeth occur in different shapes and their arrangements vary substantially from one individual to another. The difficulty is exacerbated when severe teeth malocclusion and crowding problems occur, which is a common occurrence in clinical cases. Most published methods in this area either are inaccurate or require lots of manual interactions. Motivated by the state-of-the-art general mesh segmentation methods that adopted the theory of harmonic field to detect partition boundaries, this paper proposes a novel, dental-targeted segmentation framework for dental meshes. With a specially designed weighting scheme and a strategy of a priori knowledge to guide the assignment of harmonic constraints, this method can identify teeth partition boundaries effectively. Extensive experiments and quantitative analysis demonstrate that the proposed method is able to partition high-quality teeth automatically with robustness and efficiency.

An enhanced method for registration of dental surfaces partially scanned by a 3D dental laser scanning

In this paper, we propose the fast and accurate registration method of partially scanned dental surfaces in a 3D dental laser scanning. To overcome the multiple point correspondence problems of conventional surface registration methods, we propose the novel depth map-based registration method to register 3D surface models. First, we convert a partially scanned 3D dental surface into a 2D image by generating the 2D depth map image of the surface model by applying a 3D rigid transformation into this model. Then, the image-based registration method using 2D depth map images accurately estimates the initial transformation between two consequently acquired surface models. To further increase the computational efficiency, we decompose the 3D rigid transformation into out-of-plane (i.e. x-, y-rotation, and z-translation) and in-plane (i.e. x-, y-translation, and z-rotation) transformations. For the in-plane transformation, we accelerate the transformation process by transforming the 2D depth map image instead of transforming the 3D surface model. For the more accurate registration of 3D surface models, we enhance iterative closest point (ICP) method for the subsequent fine registration. Our initial depth map-based registration well aligns each surface model. Therefore, our subsequent ICP method can accurately register two surface models since it is highly probable that the closest point pairs are the exact corresponding point pairs. The experimental results demonstrated that our method accurately registered partially scanned dental surfaces. Regarding the computational performance, our method delivered about 1.5 times faster registration than the conventional method. Our method can be successfully applied to the accurate reconstruction of 3D dental objects for orthodontic and prosthodontic treatment.

Interactive tooth partition of dental mesh base on tooth-target harmonic field

The accurate tooth partition of dental mesh is a crucial step in computer-aided orthodontics. However, tooth boundary identification is not a trivial task for tooth partition, since different shapes and their arrangements vary substantially among common clinical cases. Though curvature field is traditionally used for identifying boundaries, it is normally not reliable enough. Other methods may improve the accuracy, but require intensive user interaction. Motivated by state-of-the-art general interactive mesh segmentation methods, this paper proposes a novel tooth-target partition framework that employs harmonic fields to partition teeth accurately and effectively. In addition, a refining strategy is introduced to successfully segment teeth from the complicated dental model with indistinctive tooth boundaries on its lingual side surface, addressing an issue that had not been solved properly before. To utilise high-level information provided by the user, smart and intuitive user interfaces are also proposed with minimum interaction. In fact, most published interactive methods specifically designed for tooth partition are lacking efficient user interfaces. Extensive experiments and quantitative analyses show that our tooth partition method outperforms the state-of-the-art approaches in terms of accuracy, robustness and efficiency.