1
|
Abstract
Three-dimensional symmetry and coordination are important factors in facial aesthetics, and analysis of facial asymmetry is the basis for clinical diagnosis, treatment, and doctor–patient communication. With the development of three-dimensional measurement and data analysis technology, facial asymmetry analysis methods are mainly based on facial anatomic landmarks, original-mirror alignment algorithm, facial anthropometric mask, and artificial intelligence. This review summarizes the methods of three-dimensional facial asymmetry analysis, and current research progress in the field. The advantages and limitations of various methods are analyzed and discussed to provide a reference for oral clinical application.
Collapse
|
2
|
朱 玉, 许 晴, 赵 一, 张 磊, 付 子, 温 奥, 高 梓, 张 昀, 傅 湘, 王 勇. [Deep learning-assisted construction of three-demensional facial midsagittal plane]. BEIJING DA XUE XUE BAO. YI XUE BAN = JOURNAL OF PEKING UNIVERSITY. HEALTH SCIENCES 2022; 54:134-139. [PMID: 35165480 PMCID: PMC8860652] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Subscribe] [Scholar Register] [Received: 10/10/2021] [Indexed: 11/06/2023]
Abstract
OBJECTIVE To establish a deep learning algorithm that can accurately determine three-dimensional facial anatomical landmarks, multi-view stacked hourglass convolutional neural networks (MSH-CNN) and to construct three-dimensional facial midsagittal plane automatically based on MSH-CNN and weighted Procrustes analysis algorithm. METHODS One hundred subjects with no obvious facial deformity were collected in our oral clinic. Three-dimensional facial data were scanned by three-dimensional facial scanner. Experts annotated twenty-one facial landmarks and midsagittal plane of each data. Eighty three-dimensional facial data were used as training set, to train the MSH-CNN in this study. The overview of MSH-CNN network architecture contained multi-view rendering and training the MSH-CNN network. The three-dimensional facial data were rendered from ninety-six views that were fed to MSH-CNN and the output was one heatmap per landmark. The result of the twenty-one landmarks was accurately placed on the three-dimensional facial data after a three-dimensional view ray voting process. The remaining twenty three-dimensional facial data were used as test set. The trained MSH-CNN automatically determined twenty-one three-dimensional facial anatomical landmarks of each case of data, and calculated the distance between each MSH-CNN landmark and the expert landmark, which was defined as position error. The midsagittal plane of the twenty subjects' could be automatically constructed, using the MSH-CNN and Procrustes analysis algorithm. To evaluate the effect of midsagittal plane by automatic method, the angle between the midsagittal plane constructed by the automatic method and the expert annotated plane was calculated, which was defined as angle error. RESULTS For twenty subjects with no obvious facial deformity, the average angle error of the midsagittal plane constructed by MSH-CNN and weighted Procrustes analysis algorithm was 0.73°±0.50°, in which the average position error of the twenty-one facial landmarks automatically determined by MSH-CNN was (1.13±0.24) mm, the maximum position error of the orbital area was (1.31±0.54) mm, and the minimum position error of the nasal area was (0.79±0.36) mm. CONCLUSION This research combines deep learning algorithms and Procrustes analysis algorithms to realize the fully automated construction of the three-dimensional midsagittal plane, which initially achieves the construction effect of clinical experts. The obtained results constituted the basis for the independent intellectual property software development.
Collapse
Affiliation(s)
- 玉佳 朱
- 北京大学口腔医学院·口腔医院口腔医学数字化研究中心,国家口腔医学中心,国家口腔疾病临床医学研究中心,口腔数字化医疗技术和材料国家工程实验室,口腔数字医学北京市重点实验室,国家卫生健康委员会口 腔医学计算机应用工程技术研究中心,国家药品监督管理局口腔生物材料重点实验室,北京 100081Center of Digital Dentistry, Peking University School and Hospital of Stomatology & National Center of Stomatology & National Clinical Research Center for Oral Diseases & National Engineering Laboratory for Digital and Material Technology of Stomatology & Beijing Key Laboratory of Digital Stomatology & NHC Research Center of Engineering and Technology for Computerized Dentistry & NMPA Key Laboratory for Dental Materials, Beijing 100081, China
- 北京大学口腔医学院·口腔医院口腔修复科,北京 100081Department of Prosthodontics, Peking University School and Hospital of Stomatology, Beijing 100081, China
| | - 晴 许
- 北京邮电大学计算机学院(国家示范性软件学院),北京 100876School of Computer Science, Beijing University of Posts and Telecommunications(National Pilot Software Engineering School), Beijing 100876, China
| | - 一姣 赵
- 北京大学口腔医学院·口腔医院口腔医学数字化研究中心,国家口腔医学中心,国家口腔疾病临床医学研究中心,口腔数字化医疗技术和材料国家工程实验室,口腔数字医学北京市重点实验室,国家卫生健康委员会口 腔医学计算机应用工程技术研究中心,国家药品监督管理局口腔生物材料重点实验室,北京 100081Center of Digital Dentistry, Peking University School and Hospital of Stomatology & National Center of Stomatology & National Clinical Research Center for Oral Diseases & National Engineering Laboratory for Digital and Material Technology of Stomatology & Beijing Key Laboratory of Digital Stomatology & NHC Research Center of Engineering and Technology for Computerized Dentistry & NMPA Key Laboratory for Dental Materials, Beijing 100081, China
- 北京大学口腔医学院·口腔医院口腔修复科,北京 100081Department of Prosthodontics, Peking University School and Hospital of Stomatology, Beijing 100081, China
| | - 磊 张
- 北京大学口腔医学院·口腔医院口腔医学数字化研究中心,国家口腔医学中心,国家口腔疾病临床医学研究中心,口腔数字化医疗技术和材料国家工程实验室,口腔数字医学北京市重点实验室,国家卫生健康委员会口 腔医学计算机应用工程技术研究中心,国家药品监督管理局口腔生物材料重点实验室,北京 100081Center of Digital Dentistry, Peking University School and Hospital of Stomatology & National Center of Stomatology & National Clinical Research Center for Oral Diseases & National Engineering Laboratory for Digital and Material Technology of Stomatology & Beijing Key Laboratory of Digital Stomatology & NHC Research Center of Engineering and Technology for Computerized Dentistry & NMPA Key Laboratory for Dental Materials, Beijing 100081, China
- 北京大学口腔医学院·口腔医院口腔修复科,北京 100081Department of Prosthodontics, Peking University School and Hospital of Stomatology, Beijing 100081, China
| | - 子旺 付
- 北京邮电大学计算机学院(国家示范性软件学院),北京 100876School of Computer Science, Beijing University of Posts and Telecommunications(National Pilot Software Engineering School), Beijing 100876, China
| | - 奥楠 温
- 北京大学口腔医学院·口腔医院口腔医学数字化研究中心,国家口腔医学中心,国家口腔疾病临床医学研究中心,口腔数字化医疗技术和材料国家工程实验室,口腔数字医学北京市重点实验室,国家卫生健康委员会口 腔医学计算机应用工程技术研究中心,国家药品监督管理局口腔生物材料重点实验室,北京 100081Center of Digital Dentistry, Peking University School and Hospital of Stomatology & National Center of Stomatology & National Clinical Research Center for Oral Diseases & National Engineering Laboratory for Digital and Material Technology of Stomatology & Beijing Key Laboratory of Digital Stomatology & NHC Research Center of Engineering and Technology for Computerized Dentistry & NMPA Key Laboratory for Dental Materials, Beijing 100081, China
- 北京大学口腔医学院·口腔医院口腔修复科,北京 100081Department of Prosthodontics, Peking University School and Hospital of Stomatology, Beijing 100081, China
| | - 梓翔 高
- 北京大学口腔医学院·口腔医院口腔医学数字化研究中心,国家口腔医学中心,国家口腔疾病临床医学研究中心,口腔数字化医疗技术和材料国家工程实验室,口腔数字医学北京市重点实验室,国家卫生健康委员会口 腔医学计算机应用工程技术研究中心,国家药品监督管理局口腔生物材料重点实验室,北京 100081Center of Digital Dentistry, Peking University School and Hospital of Stomatology & National Center of Stomatology & National Clinical Research Center for Oral Diseases & National Engineering Laboratory for Digital and Material Technology of Stomatology & Beijing Key Laboratory of Digital Stomatology & NHC Research Center of Engineering and Technology for Computerized Dentistry & NMPA Key Laboratory for Dental Materials, Beijing 100081, China
- 北京大学口腔医学院·口腔医院口腔修复科,北京 100081Department of Prosthodontics, Peking University School and Hospital of Stomatology, Beijing 100081, China
| | - 昀 张
- 兰州市口腔医院特诊科,兰州 730000Department of Geriatric Dentistry, Lanzhou Stomatological Hospital, Lanzhou 730000, China
| | - 湘玲 傅
- 北京邮电大学计算机学院(国家示范性软件学院),北京 100876School of Computer Science, Beijing University of Posts and Telecommunications(National Pilot Software Engineering School), Beijing 100876, China
| | - 勇 王
- 北京大学口腔医学院·口腔医院口腔医学数字化研究中心,国家口腔医学中心,国家口腔疾病临床医学研究中心,口腔数字化医疗技术和材料国家工程实验室,口腔数字医学北京市重点实验室,国家卫生健康委员会口 腔医学计算机应用工程技术研究中心,国家药品监督管理局口腔生物材料重点实验室,北京 100081Center of Digital Dentistry, Peking University School and Hospital of Stomatology & National Center of Stomatology & National Clinical Research Center for Oral Diseases & National Engineering Laboratory for Digital and Material Technology of Stomatology & Beijing Key Laboratory of Digital Stomatology & NHC Research Center of Engineering and Technology for Computerized Dentistry & NMPA Key Laboratory for Dental Materials, Beijing 100081, China
- 北京大学口腔医学院·口腔医院口腔修复科,北京 100081Department of Prosthodontics, Peking University School and Hospital of Stomatology, Beijing 100081, China
| |
Collapse
|
3
|
朱 玉, 许 晴, 赵 一, 张 磊, 付 子, 温 奥, 高 梓, 张 昀, 傅 湘, 王 勇. [Deep learning-assisted construction of three-demensional facial midsagittal plane]. BEIJING DA XUE XUE BAO. YI XUE BAN = JOURNAL OF PEKING UNIVERSITY. HEALTH SCIENCES 2022; 54:134-139. [PMID: 35165480 PMCID: PMC8860652 DOI: 10.19723/j.issn.1671-167x.2022.01.021] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Subscribe] [Scholar Register] [Received: 10/10/2021] [Indexed: 06/14/2023]
Abstract
OBJECTIVE To establish a deep learning algorithm that can accurately determine three-dimensional facial anatomical landmarks, multi-view stacked hourglass convolutional neural networks (MSH-CNN) and to construct three-dimensional facial midsagittal plane automatically based on MSH-CNN and weighted Procrustes analysis algorithm. METHODS One hundred subjects with no obvious facial deformity were collected in our oral clinic. Three-dimensional facial data were scanned by three-dimensional facial scanner. Experts annotated twenty-one facial landmarks and midsagittal plane of each data. Eighty three-dimensional facial data were used as training set, to train the MSH-CNN in this study. The overview of MSH-CNN network architecture contained multi-view rendering and training the MSH-CNN network. The three-dimensional facial data were rendered from ninety-six views that were fed to MSH-CNN and the output was one heatmap per landmark. The result of the twenty-one landmarks was accurately placed on the three-dimensional facial data after a three-dimensional view ray voting process. The remaining twenty three-dimensional facial data were used as test set. The trained MSH-CNN automatically determined twenty-one three-dimensional facial anatomical landmarks of each case of data, and calculated the distance between each MSH-CNN landmark and the expert landmark, which was defined as position error. The midsagittal plane of the twenty subjects' could be automatically constructed, using the MSH-CNN and Procrustes analysis algorithm. To evaluate the effect of midsagittal plane by automatic method, the angle between the midsagittal plane constructed by the automatic method and the expert annotated plane was calculated, which was defined as angle error. RESULTS For twenty subjects with no obvious facial deformity, the average angle error of the midsagittal plane constructed by MSH-CNN and weighted Procrustes analysis algorithm was 0.73°±0.50°, in which the average position error of the twenty-one facial landmarks automatically determined by MSH-CNN was (1.13±0.24) mm, the maximum position error of the orbital area was (1.31±0.54) mm, and the minimum position error of the nasal area was (0.79±0.36) mm. CONCLUSION This research combines deep learning algorithms and Procrustes analysis algorithms to realize the fully automated construction of the three-dimensional midsagittal plane, which initially achieves the construction effect of clinical experts. The obtained results constituted the basis for the independent intellectual property software development.
Collapse
Affiliation(s)
- 玉佳 朱
- 北京大学口腔医学院·口腔医院口腔医学数字化研究中心,国家口腔医学中心,国家口腔疾病临床医学研究中心,口腔数字化医疗技术和材料国家工程实验室,口腔数字医学北京市重点实验室,国家卫生健康委员会口 腔医学计算机应用工程技术研究中心,国家药品监督管理局口腔生物材料重点实验室,北京 100081Center of Digital Dentistry, Peking University School and Hospital of Stomatology & National Center of Stomatology & National Clinical Research Center for Oral Diseases & National Engineering Laboratory for Digital and Material Technology of Stomatology & Beijing Key Laboratory of Digital Stomatology & NHC Research Center of Engineering and Technology for Computerized Dentistry & NMPA Key Laboratory for Dental Materials, Beijing 100081, China
- 北京大学口腔医学院·口腔医院口腔修复科,北京 100081Department of Prosthodontics, Peking University School and Hospital of Stomatology, Beijing 100081, China
| | - 晴 许
- 北京邮电大学计算机学院(国家示范性软件学院),北京 100876School of Computer Science, Beijing University of Posts and Telecommunications(National Pilot Software Engineering School), Beijing 100876, China
| | - 一姣 赵
- 北京大学口腔医学院·口腔医院口腔医学数字化研究中心,国家口腔医学中心,国家口腔疾病临床医学研究中心,口腔数字化医疗技术和材料国家工程实验室,口腔数字医学北京市重点实验室,国家卫生健康委员会口 腔医学计算机应用工程技术研究中心,国家药品监督管理局口腔生物材料重点实验室,北京 100081Center of Digital Dentistry, Peking University School and Hospital of Stomatology & National Center of Stomatology & National Clinical Research Center for Oral Diseases & National Engineering Laboratory for Digital and Material Technology of Stomatology & Beijing Key Laboratory of Digital Stomatology & NHC Research Center of Engineering and Technology for Computerized Dentistry & NMPA Key Laboratory for Dental Materials, Beijing 100081, China
- 北京大学口腔医学院·口腔医院口腔修复科,北京 100081Department of Prosthodontics, Peking University School and Hospital of Stomatology, Beijing 100081, China
| | - 磊 张
- 北京大学口腔医学院·口腔医院口腔医学数字化研究中心,国家口腔医学中心,国家口腔疾病临床医学研究中心,口腔数字化医疗技术和材料国家工程实验室,口腔数字医学北京市重点实验室,国家卫生健康委员会口 腔医学计算机应用工程技术研究中心,国家药品监督管理局口腔生物材料重点实验室,北京 100081Center of Digital Dentistry, Peking University School and Hospital of Stomatology & National Center of Stomatology & National Clinical Research Center for Oral Diseases & National Engineering Laboratory for Digital and Material Technology of Stomatology & Beijing Key Laboratory of Digital Stomatology & NHC Research Center of Engineering and Technology for Computerized Dentistry & NMPA Key Laboratory for Dental Materials, Beijing 100081, China
- 北京大学口腔医学院·口腔医院口腔修复科,北京 100081Department of Prosthodontics, Peking University School and Hospital of Stomatology, Beijing 100081, China
| | - 子旺 付
- 北京邮电大学计算机学院(国家示范性软件学院),北京 100876School of Computer Science, Beijing University of Posts and Telecommunications(National Pilot Software Engineering School), Beijing 100876, China
| | - 奥楠 温
- 北京大学口腔医学院·口腔医院口腔医学数字化研究中心,国家口腔医学中心,国家口腔疾病临床医学研究中心,口腔数字化医疗技术和材料国家工程实验室,口腔数字医学北京市重点实验室,国家卫生健康委员会口 腔医学计算机应用工程技术研究中心,国家药品监督管理局口腔生物材料重点实验室,北京 100081Center of Digital Dentistry, Peking University School and Hospital of Stomatology & National Center of Stomatology & National Clinical Research Center for Oral Diseases & National Engineering Laboratory for Digital and Material Technology of Stomatology & Beijing Key Laboratory of Digital Stomatology & NHC Research Center of Engineering and Technology for Computerized Dentistry & NMPA Key Laboratory for Dental Materials, Beijing 100081, China
- 北京大学口腔医学院·口腔医院口腔修复科,北京 100081Department of Prosthodontics, Peking University School and Hospital of Stomatology, Beijing 100081, China
| | - 梓翔 高
- 北京大学口腔医学院·口腔医院口腔医学数字化研究中心,国家口腔医学中心,国家口腔疾病临床医学研究中心,口腔数字化医疗技术和材料国家工程实验室,口腔数字医学北京市重点实验室,国家卫生健康委员会口 腔医学计算机应用工程技术研究中心,国家药品监督管理局口腔生物材料重点实验室,北京 100081Center of Digital Dentistry, Peking University School and Hospital of Stomatology & National Center of Stomatology & National Clinical Research Center for Oral Diseases & National Engineering Laboratory for Digital and Material Technology of Stomatology & Beijing Key Laboratory of Digital Stomatology & NHC Research Center of Engineering and Technology for Computerized Dentistry & NMPA Key Laboratory for Dental Materials, Beijing 100081, China
- 北京大学口腔医学院·口腔医院口腔修复科,北京 100081Department of Prosthodontics, Peking University School and Hospital of Stomatology, Beijing 100081, China
| | - 昀 张
- 兰州市口腔医院特诊科,兰州 730000Department of Geriatric Dentistry, Lanzhou Stomatological Hospital, Lanzhou 730000, China
| | - 湘玲 傅
- 北京邮电大学计算机学院(国家示范性软件学院),北京 100876School of Computer Science, Beijing University of Posts and Telecommunications(National Pilot Software Engineering School), Beijing 100876, China
| | - 勇 王
- 北京大学口腔医学院·口腔医院口腔医学数字化研究中心,国家口腔医学中心,国家口腔疾病临床医学研究中心,口腔数字化医疗技术和材料国家工程实验室,口腔数字医学北京市重点实验室,国家卫生健康委员会口 腔医学计算机应用工程技术研究中心,国家药品监督管理局口腔生物材料重点实验室,北京 100081Center of Digital Dentistry, Peking University School and Hospital of Stomatology & National Center of Stomatology & National Clinical Research Center for Oral Diseases & National Engineering Laboratory for Digital and Material Technology of Stomatology & Beijing Key Laboratory of Digital Stomatology & NHC Research Center of Engineering and Technology for Computerized Dentistry & NMPA Key Laboratory for Dental Materials, Beijing 100081, China
- 北京大学口腔医学院·口腔医院口腔修复科,北京 100081Department of Prosthodontics, Peking University School and Hospital of Stomatology, Beijing 100081, China
| |
Collapse
|