Weakly Supervised Deep Learning for Tooth-Marked Tongue Recognition

Development and Application of Traditional Chinese Medicine Using AI Machine Learning and Deep Learning Strategies

Traditional Chinese medicine (TCM) has been used for thousands of years and has been proven to be effective at treating many complicated illnesses with minimal side effects. The application and advancement of TCM are, however, constrained by the absence of objective measuring standards due to its relatively abstract diagnostic methods and syndrome differentiation theories. Ongoing developments in machine learning (ML) and deep learning (DL), specifically in computer vision (CV) and natural language processing (NLP), offer novel opportunities to modernize TCM by exploring the profound connotations of its theory. This review begins with an overview of the ML and DL methods employed in TCM; this is followed by practical instances of these applications. Furthermore, extensive discussions emphasize the mature integration of ML and DL in TCM, such as tongue diagnosis, pulse diagnosis, and syndrome differentiation treatment, highlighting their early successful application in the TCM field. Finally, this study validates the accomplishments and addresses the problems and challenges posed by the application and development of TCM powered by ML and DL. As ML and DL techniques continue to evolve, modern technology will spark new advances in TCM.

Artificial intelligence in tongue diagnosis: classification of tongue lesions and normal tongue images using deep convolutional neural network

OBJECTIVE

This study aims to classify tongue lesion types using tongue images utilizing Deep Convolutional Neural Networks (DCNNs).

METHODS

A dataset consisting of five classes, four tongue lesion classes (coated, geographical, fissured tongue, and median rhomboid glossitis), and one healthy/normal tongue class, was constructed using tongue images of 623 patients who were admitted to our clinic. Classification performance was evaluated on VGG19, ResNet50, ResNet101, and GoogLeNet networks using fusion based majority voting (FBMV) approach for the first time in the literature.

RESULTS

In the binary classification problem (normal vs. tongue lesion), the highest classification accuracy performance of 93,53% was achieved utilizing ResNet101, and this rate was increased to 95,15% with the application of the FBMV approach. In the five-class classification problem of tongue lesion types, the VGG19 network yielded the best accuracy rate of 83.93%, and the fusion approach improved this rate to 88.76%.

CONCLUSION

The obtained test results showed that tongue lesions could be identified with a high accuracy by applying DCNNs. Further improvement of these results has the potential for the use of the proposed method in clinic applications.

Tongue feature dataset construction and real-time detection

BACKGROUND

Tongue diagnosis in traditional Chinese medicine (TCM) provides clinically important, objective evidence from direct observation of specific features that assist with diagnosis. However, the current interpretation of tongue features requires a significant amount of manpower and time. TCM physicians may have different interpretations of features displayed by the same tongue. An automated interpretation system that interprets tongue features would expedite the interpretation process and yield more consistent results.

MATERIALS AND METHODS

This study applied deep learning visualization to tongue diagnosis. After collecting tongue images and corresponding interpretation reports by TCM physicians in a single teaching hospital, various tongue features such as fissures, tooth marks, and different types of coatings were annotated manually with rectangles. These annotated data and images were used to train a deep learning object detection model. Upon completion of training, the position of each tongue feature was dynamically marked.

RESULTS

A large high-quality manually annotated tongue feature dataset was constructed and analyzed. A detection model was trained with average precision (AP) 47.67%, 58.94%, 71.25% and 59.78% for fissures, tooth marks, thick and yellow coatings, respectively. At over 40 frames per second on a NVIDIA GeForce GTX 1060, the model was capable of detecting tongue features from any viewpoint in real time.

CONCLUSIONS/SIGNIFICANCE

This study constructed a tongue feature dataset and trained a deep learning object detection model to locate tongue features in real time. The model provided interpretability and intuitiveness that are often lacking in general neural network models and implies good feasibility for clinical application.

A Multi-stage Segmentation Method for Tongue Ecchymosis. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023;2023:1-4. [PMID: 38083193 DOI: 10.1109/embc40787.2023.10340439] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Tongue diagnosis is an important component of traditional Chinese medicine (TCM), in which tongue ecchymosis is the main diagnostic basis for the blood stasis syndrome of TCM. Most of the existing methods are unsupervised and cannot accurately segment tongue ecchymosis. In this paper, we propose a multi-stage segmentation method for tongue ecchymosis. We first employ an object detection model for rough localization of tongue ecchymosis, and then use the unsupervised clustering and the watershed transform for rough segmentation and fine segmentation of tongue ecchymosis respectively. To the best of our knowledge, we are the first to combine machine learning and deep learning to segment tongue ecchymosis. Experimental results show that the tongue ecchymoses obtained by our method are more similar to the real tongue ecchymoses compared with the existing methods, and the Intersection-over-Union (IoU) is improved by 0.12 compared with the latest method.Clinical Relevance-Tongue ecchymosis obtained by this paper is the main diagnostic basis for the blood stasis syndrome of TCM.

Intelligent Chinese Medicine: A New Direction Approach for Integrative Medicine in Diagnosis and Treatment of Cardiovascular Diseases

High mortality rates from cardiovascular diseases (CVDs) persist worldwide. Older people are at a higher risk of developing these diseases. Given the current high treatment cost for CVDs, there is a need to prevent CVDs and or develop treatment alternatives. Western and Chinese medicines have been used to treat CVDs. However, several factors, such as inaccurate diagnoses, non-standard prescriptions, and poor adherence behavior, lower the benefits of the treatments by Chinese medicine (CM). Artificial intelligence (AI) is increasingly used in clinical diagnosis and treatment, especially in assessing efficacy of CM in clinical decision support systems, health management, new drug research and development, and drug efficacy evaluation. In this study, we explored the role of AI in CM in the diagnosis and treatment of CVDs, and discussed application of AI in assessing the effect of CM on CVDs.

A survey of artificial intelligence in tongue image for disease diagnosis and syndrome differentiation

The rapid development of artificial intelligence technology has gradually extended from the general field to all walks of life, and intelligent tongue diagnosis is the product of a miraculous connection between this new discipline and traditional disciplines. We reviewed the deep learning methods and machine learning applied in tongue image analysis that have been studied in the last 5 years, focusing on tongue image calibration, detection, segmentation, and classification of diseases, syndromes, and symptoms/signs. Introducing technical evolutions or emerging technologies were applied in tongue image analysis; as we have noticed, attention mechanism, multiscale features, and prior knowledge were successfully applied in it, and we emphasized the value of combining deep learning with traditional methods. We also pointed out two major problems concerned with data set construction and the low reliability of performance evaluation that exist in this field based on the basic essence of tongue diagnosis in traditional Chinese medicine. Finally, a perspective on the future of intelligent tongue diagnosis was presented; we believe that the self-supervised method, multimodal information fusion, and the study of tongue pathology will have great research significance.