Deep Convolutional Neural Networks for Classifying Body Constitution Based on Face Image

Current Status of Objectification of Four Diagnostic Methods on Constitution Recognition of Chinese Medicine

Chinese medicine (CM) has thousands of years of experience in prevention of diseases. As for CM, people's constitution is closely related to their health status, thus recognition of CM constitution is the fundamental and core content of research on constitution types. With development of technologies such as sensors, artificial intelligence and big data, objectification of the four diagnostic methods of CM has gradually matured, bringing changes in the mindset and innovations in technical means for recognition of CM constitution. This paper presents a systematic review of the latest research trends in constitution recognition based on objectification of diagnostic methods in CM.

The Research and Development Thinking on the Status of Artificial Intelligence in Traditional Chinese Medicine

With the rapid development and application of artificial intelligence (AI) in medical field, the diagnostic ways of human health and the social medical structures have changed. Based on the concept of holism and the theory of syndrome differentiation and treatment, TCM realizes comprehensive informatization and intelligence with the help of AI technology in data mining, intelligent diagnosis and treatment, intelligent learning, and decision-making. Furthermore, the intelligent research of TCM technology will further promote the improvement in TCM diagnosis and treatment rules and the leaping development of TCM intelligent instruments. In this article, we performed a systematic review of scientific literature about TCM and AI. Moreover, the practical problems of TCM intellectualization, the current situation and demand of TCM, and the influence of AI in the TCM field are discussed by searching for literature using TCM scientific databases, reference lists, expert consultation, and targeted websites. Finally, we look forward to the application prospects of AI and propose a possible future direction of intelligent TCM in the current health-care system in China.

Traditional Chinese Medicine Constitution Identification Based on Objective Facial and Tongue Features: A Delphi Study and a Diagnostic Nomogram for Blood Stasis Constitution

Objective

The aim of this study was to systematically summarize and form an expert consensus on the theoretical experience of tongue and facial features for the identification of nine types of traditional Chinese medicine (TCM) constitution. Additionally, we sought to explore the feasibility of TCM constitution identification through objective tongue and facial features.

Methods

We used Delphi method to investigate the opinions of experts on facial and tongue feature items for identifying TCM constitution. We developed and validated a diagnostic nomogram for blood stasis constitution (BSC) based on objective facial and tongue features to demonstrate the reliability of expert consultation.

Results

Eleven experts participated in two rounds of expert consultation. The recovery rates of the two rounds of expert consultation were 100.0% and 90.9%. After the first round, 39 items were screened out from 147 initial items, and 2 items were supplemented by experts. In the second round, 7 items were eliminated, leaving 34 items for 8 types of TCM constitution. The coefficient of variation in the first round was 0.11–0.49 for the 147 items and 0.11–0.29 for the included items. The coefficient of variation in the second round was 0.10–0.27 for the 41 items and 0.10–0.20 for the included items. The W value was 0.548 (P < 0.001) in the first round and 0.240 (P < 0.001) in the second round. Based on expert consultation, we selected BSC as an example and developed and validated a diagnostic nomogram consisting of six indicators: sex, hair volume, lip color-dark purple, susceptibility-facial pigmentation/chloasma/ecchymosis, zygomatic texture-red blood streaks, and sublingual vein-varicose and dark purple. The nomogram showed good discrimination (AUC: 0.917 [95% confidence interval [CI], 0.877–0.956] for the primary dataset, 0.902 [95% CI, 0.828–0.976] for the validation dataset) and good calibration. Decision curve analysis demonstrated that the nomogram was clinically useful.

Conclusion

This is the first study to systematically summarize the existing knowledge and clinical experience to form an expert consensus on the tongue and facial features of nine types of TCM constitution. Our results will provide important prior knowledge and expert experience for future constitution identification research. Based on expert consultation, this study presents a nomogram for BSC that incorporates objective facial and tongue features, which can be conveniently used to facilitate the individualized identification of BSC.

Task-Coupling Elastic Learning for Physical Sign-based Medical Image Classification

Physical signs of patients indicate crucial evidence for diagnosing both location and nature of the disease, where there is a sequential relationship between the two tasks. Thus their joint learning can utilize intrinsic association by transferring related knowledge across relevant tasks. Choosing the right time to transfer is a critical problem for joint learning. However, how to dynamically adjust when tasks interact to capture the right time for transferring related knowledge is still an open issue. To this end, we propose a Task-Coupling Elastic Learning (TCEL) framework to model the task relatedness for classifying disease-location and disease-nature based on physical sign images. The main idea is to dynamically transfer relevant knowledge by progressively shifting task-coupling from loose to tight during the multi-stage training. In the early stage of training, we relax the constraints of modeling relations to focus more in learning the generic task-common features. In the later stage, the semantic guidance will be strengthened to learn the task-specific features. Specifically, a dynamic sequential module (DSM) is proposed to explicitly model the sequential relationship and enable multi-stage training. Moreover, to address the side effect of DSM, a new loss regularization is proposed. The extensive experiments on these two clinical datasets show the superiority of the proposed method over the baselines, and demonstrate the effectiveness of the proposed task-coupling elastic mechanism.

Computational Traditional Chinese Medicine diagnosis: A literature survey

BACKGROUND AND OBJECTIVE

Traditional Chinese Medicine (TCM) diagnosis is based on the theoretical principles and knowledge, where it is steeped in thousands of years of history to diagnose various types of diseases and syndromes. It can be generally divided into four main diagnostic approaches: 1. Inspection, 2. Auscultation and olfaction, 3. Inquiry, and 4. Palpation, which are widely used in TCM hospitals in China and around the world. With the development of intelligent computing technology in recent years, computational TCM diagnosis has grown rapidly.

METHODS

In this paper, we aim to systematically summarize the development of computational TCM diagnosis based on four diagnostic approaches, mainly focusing on digital acquisition devices, collected datasets, and computational detection approaches (algorithms). Furthermore, all related works of this field are compared and explored in detail.

RESULTS

This survey provides the principles, applications, and current progress in computing for readers and researchers in terms of computational TCM diagnosis. Moreover, the future development direction, prospect, and technological trend of computational TCM diagnosis will also be discussed in this study.

CONCLUSIONS

Recent computational TCM diagnosis works are compared in detail to show the pros/cons, where we provide some meaningful suggestions and opinions on the future research approaches in this area. This work is useful for disease detection in computational TCM diagnosis as well as health management in the smart healthcare area.

INDEX TERMS

Computational diagnosis, Traditional Chinese Medicine, survey, smart healthcare.

Personalized Body Constitution Inquiry Based on Machine Learning

Background

Body constitution (BC) is the abstract concept indicating the state of a person's health in Traditional Chinese Medicine (TCM). The doctor identifies the body constitution of the patient through inspection and inquiry. Previous research simulates doctors to identify BC types according to a patient's objective physical indicators. However, the lack of subjective feeling information can reduce the accuracy of the machine to imitate the doctor's diagnosis. The Constitution in Chinese Medicine Questionnaire (CCMQ) is used to collect subjective information but suffers from low acquisition efficiency.

Methods

This paper presents a personalized body constitution inquiry method based on a machine learning technique. It employs a random generator, a feature extractor, and a classifier to simulate the doctor inquiry and generate a personalized questionnaire. Specifically, the feature extractor evaluates and sorts the question of the constitution in the CCMQ based on the recognition results of the tongue coating image of patients. The sorted questions and relevant BC label are inputted into the classifier; the best questions are screened out for patients.

Results

The experimental results show that our method can select personalized questions from the CCMQ for the patients, significantly reducing the time and the number of questions to answer. It also improves the accuracy of recognizing BC. Compared with the CCMQ, patients had 68.3% fewer questions to answer and the time occupied by answering is reduced by 80.3%.

Conclusions

The proposed method can simulate the doctor's inquiry and pick out personalized questions for patients. It can act as auxiliary diagnosis tools to collect subjective patient feelings and help make further judgments on the patient's BC types.

Grouping attributes zero-shot learning for tongue constitution recognition

Traditional Chinese Medicine (TCM) considers that the personal constitution determines the occurrence trend and therapeutic effects of certain diseases, which can be recognized by machine learning through tongue images. However, current machine learning methods are confronted with two challenges. First, there are not some larger tongue image databases available. Second, they do not use the domain knowledge of TCM, so that the imbalance of constitution categories cannot be solved. Therefore, this paper proposes a new constitution recognition method based on the zero-shot learning with the knowledge of TCM. To further improve the performance, a new zero-shot learning method is proposed by grouping attributes and learning discriminant latent features, which can better solve the imbalance problem of constitution categories. Experimental results on our constructed databases validate the proposed methods.

Classification of Body Constitution Based on TCM Philosophy and Deep Learning

There is a growing demand for alternative or complementary medicine in health care disciplines that uses a non-invasive instrument to evaluate the health status of various organs inside the human body. In this regard, we proposed a real-time, non-invasive, and painless technique to assess an individual’s health condition. Our approach is based on the combination of iridology and the philosophy of traditional Chinese medicine (TCM). The iridology chart presents perfect symmetry between the left and right eyes, and such a unique representation reveals the body constitution based on TCM philosophy, which classifies the aforementioned body constitution into a combination of nine categories to describe the varieties of genomic traits. In addition, we applied a deep-learning method along with the combination of iridology and TCM to predict the possible physiological or psychological strength or weakness of the subjects and give advice to them about how to take care of their health according to the body constitution assessment. We used several pre-trained convolutional neural networks (CNNs, or ConvNet), such as a residual neural network (ResNet50), InceptionV3, and dense convolutional network (DenseNet201), to classify the body constitution using iris images. In the experiments, the CASIA-Iris-Thousand database was used to perform this task. The experimental results showed that the proposed iris-based health assessment method achieved an 82.9% accuracy.

Multilevel and Multiscale Feature Aggregation in Deep Networks for Facial Constitution Classification

Constitution classification is the basis and core content of TCM constitution research. In order to improve the accuracy of constitution classification, this paper proposes a multilevel and multiscale features aggregation method within the convolutional neural network, which consists of four steps. First, it uses the pretrained VGG16 as the basic network and then refines the network structure through supervised feature learning so as to capture local image features. Second, it extracts the image features of different layers from the fine-tuned VGG16 model, which are then dimensionally reduced by principal component analysis (PCA). Third, it uses another pretrained NASNetMobile network for supervised feature learning, where the previous layer features of the global average pooling layer are outputted. Similarly, these features are dimensionally reduced by PCA and then are fused with the features of different layers in VGG16 after the PCA. Finally, all features are aggregated with the fully connected layers of the fine-tuned VGG16, and then the constitution classification is performed. The conducted experiments show that using the multilevel and multiscale feature aggregation is very effective in the constitution classification, and the accuracy on the test dataset reaches 69.61%.