A deep learning study on osteosarcoma detection from histological images

Enhancing radiographic image interpretation: WARES-PRS model for knee bone tumour detection

The early diagnosis of tumour is significant in biomedical research field to lower the severity level and restrict the process extension from cancer. Moreover, the detection of early sign of cancer is undertaken with extensive research efforts that dedicated to the disclosure and recognition of tumours. However, the limited data size as well as diverse appearance of images lowered the detection performance and failed to detect complex stage of tumour. So to solve these issues, a Weighted Adaptive Random Ensemble Support Vector-based Partial Reinforcement Search (WARES-PRS) algorithm is proposed that detected bone lesions accurately and also predicted the severity level stage efficiently. Further, the detection is performed with varied stages to diminish the presence of noise and undertaken effective classification. The performance is validated with CNUH dataset that enhanced image pre-processing tasks. Despite the proposed method uncover the mutual relationships between each pixel's local texture and the overall image's global context. The detection and classification efficiency is validated with various measures and the experimental results revealed that the detection accuracy is enhanced for the proposed approach by 98.5%. The outcomes of our study have exhibited a substantial contribution to assisting physicians in the detection of knee bone tumours.

MICIL: Multiple-Instance Class-Incremental Learning for skin cancer whole slide images

Artificial intelligence (AI) agents encounter the problem of catastrophic forgetting when they are trained in sequentially with new data batches. This issue poses a barrier to the implementation of AI-based models in tasks that involve ongoing evolution, such as cancer prediction. Moreover, whole slide images (WSI) play a crucial role in cancer management, and their automated analysis has become increasingly popular in assisting pathologists during the diagnosis process. Incremental learning (IL) techniques aim to develop algorithms capable of retaining previously acquired information while also acquiring new insights to predict future data. Deep IL techniques need to address the challenges posed by the gigapixel scale of WSIs, which often necessitates the use of multiple instance learning (MIL) frameworks. In this paper, we introduce an IL algorithm tailored for analyzing WSIs within a MIL paradigm. The proposed Multiple Instance Class-Incremental Learning (MICIL) algorithm combines MIL with class-IL for the first time, allowing for the incremental prediction of multiple skin cancer subtypes from WSIs within a class-IL scenario. Our framework incorporates knowledge distillation and data rehearsal, along with a novel embedding-level distillation, aiming to preserve the latent space at the aggregated WSI level. Results demonstrate the algorithm's effectiveness in addressing the challenge of balancing IL-specific metrics, such as intransigence and forgetting, and solving the plasticity-stability dilemma.

Adapted Deep Ensemble Learning-Based Voting Classifier for Osteosarcoma Cancer Classification

The study utilizes osteosarcoma hematoxylin and the Eosin-stained image dataset, which is unevenly dispersed, and it raises concerns about the potential impact on the overall performance and reliability of any analyses or models derived from the dataset. In this study, a deep-learning-based convolution neural network (CNN) and adapted heterogeneous ensemble-learning-based voting classifier have been proposed to classify osteosarcoma. The proposed methods can also resolve the issue and develop unbiased learning models by introducing an evenly distributed training dataset. Data augmentation is employed to boost the generalization abilities. Six different pre-trained CNN models, namely MobileNetV1, Mo-bileNetV2, ResNetV250, InceptionV2, EfficientNetV2B0, and NasNetMobile, are applied and evaluated in frozen and fine-tuned-based phases. In addition, a novel CNN model and adapted heterogeneous ensemble-learning-based voting classifier developed from the proposed CNN model, fine-tuned NasNetMobile model, and fine-tuned Efficient-NetV2B0 model are also introduced to classify osteosarcoma. The proposed CNN model outperforms other pre-trained models. The Kappa score obtained from the proposed CNN model is 93.09%. Notably, the proposed voting classifier attains the highest Kappa score of 96.50% and outperforms all other models. The findings of this study have practical implications in telemedicine, mobile healthcare systems, and as a supportive tool for medical professionals.

Osteosarcoma Explorer: A Data Commons With Clinical, Genomic, Protein, and Tissue Imaging Data for Osteosarcoma Research

PURPOSE

Osteosarcoma research advancement requires enhanced data integration across different modalities and sources. Current osteosarcoma research, encompassing clinical, genomic, protein, and tissue imaging data, is hindered by the siloed landscape of data generation and storage.

MATERIALS AND METHODS

Clinical, molecular profiling, and tissue imaging data for 573 patients with pediatric osteosarcoma were collected from four public and institutional sources. A common data model incorporating standardized terminology was created to facilitate the transformation, integration, and load of source data into a relational database. On the basis of this database, a data commons accompanied by a user-friendly web portal was developed, enabling various data exploration and analytics functions.

RESULTS

The Osteosarcoma Explorer (OSE) was released to the public in 2021. Leveraging a comprehensive and harmonized data set on the backend, the OSE offers a wide range of functions, including Cohort Discovery, Patient Dashboard, Image Visualization, and Online Analysis. Since its initial release, the OSE has experienced an increasing utilization by the osteosarcoma research community and provided solid, continuous user support. To our knowledge, the OSE is the largest (N = 573) and most comprehensive research data commons for pediatric osteosarcoma, a rare disease. This project demonstrates an effective framework for data integration and data commons development that can be readily applied to other projects sharing similar goals.

CONCLUSION

The OSE offers an online exploration and analysis platform for integrated clinical, molecular profiling, and tissue imaging data of osteosarcoma. Its underlying data model, database, and web framework support continuous expansion onto new data modalities and sources.

A Novel Hybrid Approach for Classifying Osteosarcoma Using Deep Feature Extraction and Multilayer Perceptron

Osteosarcoma is the most common type of bone cancer that tends to occur in teenagers and young adults. Due to crowded context, inter-class similarity, inter-class variation, and noise in H&E-stained (hematoxylin and eosin stain) histology tissue, pathologists frequently face difficulty in osteosarcoma tumor classification. In this paper, we introduced a hybrid framework for improving the efficiency of three types of osteosarcoma tumor (nontumor, necrosis, and viable tumor) classification by merging different types of CNN-based architectures with a multilayer perceptron (MLP) algorithm on the WSI (whole slide images) dataset. We performed various kinds of preprocessing on the WSI images. Then, five pre-trained CNN models were trained with multiple parameter settings to extract insightful features via transfer learning, where convolution combined with pooling was utilized as a feature extractor. For feature selection, a decision tree-based RFE was designed to recursively eliminate less significant features to improve the model generalization performance for accurate prediction. Here, a decision tree was used as an estimator to select the different features. Finally, a modified MLP classifier was employed to classify binary and multiclass types of osteosarcoma under the five-fold CV to assess the robustness of our proposed hybrid model. Moreover, the feature selection criteria were analyzed to select the optimal one based on their execution time and accuracy. The proposed model achieved an accuracy of 95.2% for multiclass classification and 99.4% for binary classification. Experimental findings indicate that our proposed model significantly outperforms existing methods; therefore, this model could be applicable to support doctors in osteosarcoma diagnosis in clinics. In addition, our proposed model is integrated into a web application using the FastAPI web framework to provide a real-time prediction.

Deep Learning Approaches to Osteosarcoma Diagnosis and Classification: A Comparative Methodological Approach

BACKGROUND

Osteosarcoma is the most common primary malignancy of the bone, being most prevalent in childhood and adolescence. Despite recent progress in diagnostic methods, histopathology remains the gold standard for disease staging and therapy decisions. Machine learning and deep learning methods have shown potential for evaluating and classifying histopathological cross-sections.

METHODS

This study used publicly available images of osteosarcoma cross-sections to analyze and compare the performance of state-of-the-art deep neural networks for histopathological evaluation of osteosarcomas.

RESULTS

The classification performance did not necessarily improve when using larger networks on our dataset. In fact, the smallest network combined with the smallest image input size achieved the best overall performance. When trained using 5-fold cross-validation, the MobileNetV2 network achieved 91% overall accuracy.

CONCLUSIONS

The present study highlights the importance of careful selection of network and input image size. Our results indicate that a larger number of parameters is not always better, and the best results can be achieved on smaller and more efficient networks. The identification of an optimal network and training configuration could greatly improve the accuracy of osteosarcoma diagnoses and ultimately lead to better disease outcomes for patients.

Deep Learning-Based Objective and Reproducible Osteosarcoma Chemotherapy Response Assessment and Outcome Prediction

Osteosarcoma is the most common primary bone cancer, whose standard treatment includes pre-operative chemotherapy followed by resection. Chemotherapy response is used for prognosis and management of patients. Necrosis is routinely assessed after chemotherapy from histology slides on resection specimens, where necrosis ratio is defined as the ratio of necrotic tumor/overall tumor. Patients with necrosis ratio ≥90% are known to have a better outcome. Manual microscopic review of necrosis ratio from multiple glass slides is semiquantitative and can have intraobserver and interobserver variability. In this study, an objective and reproducible deep learning-based approach was proposed to estimate necrosis ratio with outcome prediction from scanned hematoxylin and eosin whole slide images (WSIs). To conduct the study, 103 osteosarcoma cases with 3134 WSIs were collected. Deep Multi-Magnification Network was trained to segment multiple tissue subtypes, including viable tumor and necrotic tumor at a pixel level and to calculate case-level necrosis ratio from multiple WSIs. Necrosis ratio estimated by the segmentation model highly correlates with necrosis ratio from pathology reports manually assessed by experts. Furthermore, patients were successfully stratified to predict overall survival with P = 2.4 × 10-6 and progression-free survival with P = 0.016. This study indicates that deep learning can support pathologists as an objective tool to analyze osteosarcoma from histology for assessing treatment response and predicting patient outcome.

An Intelligent Auxiliary Framework for Bone Malignant Tumor Lesion Segmentation in Medical Image Analysis

Bone malignant tumors are metastatic and aggressive, with poor treatment outcomes and prognosis. Rapid and accurate diagnosis is crucial for limb salvage and increasing the survival rate. There is a lack of research on deep learning to segment bone malignant tumor lesions in medical images with complex backgrounds and blurred boundaries. Therefore, we propose a new intelligent auxiliary framework for the medical image segmentation of bone malignant tumor lesions, which consists of a supervised edge-attention guidance segmentation network (SEAGNET). We design a boundary key points selection module to supervise the learning of edge attention in the model to retain fine-grained edge feature information. We precisely locate malignant tumors by instance segmentation networks while extracting feature maps of tumor lesions in medical images. The rich contextual-dependent information in the feature map is captured by mixed attention to better understand the uncertainty and ambiguity of the boundary, and edge attention learning is used to guide the segmentation network to focus on the fuzzy boundary of the tumor region. We implement extensive experiments on real-world medical data to validate our model. It validates the superiority of our method over the latest segmentation methods, achieving the best performance in terms of the Dice similarity coefficient (0.967), precision (0.968), and accuracy (0.996). The results prove the important contribution of the framework in assisting doctors to improve the accuracy of diagnosis and clinical efficiency.

Multi-modal wound classification using wound image and location by deep neural network

Wound classification is an essential step of wound diagnosis. An efficient classifier can assist wound specialists in classifying wound types with less financial and time costs and help them decide on an optimal treatment procedure. This study developed a deep neural network-based multi-modal classifier using wound images and their corresponding locations to categorize them into multiple classes, including diabetic, pressure, surgical, and venous ulcers. A body map was also developed to prepare the location data, which can help wound specialists tag wound locations more efficiently. Three datasets containing images and their corresponding location information were designed with the help of wound specialists. The multi-modal network was developed by concatenating the image-based and location-based classifier outputs with other modifications. The maximum accuracy on mixed-class classifications (containing background and normal skin) varies from 82.48 to 100% in different experiments. The maximum accuracy on wound-class classifications (containing only diabetic, pressure, surgical, and venous) varies from 72.95 to 97.12% in various experiments. The proposed multi-modal network also showed a significant improvement in results from the previous works of literature.

AI-Assisted Diagnosis and Decision-Making Method in Developing Countries for Osteosarcoma

Osteosarcoma is a malignant tumor derived from primitive osteogenic mesenchymal cells, which is extremely harmful to the human body and has a high mortality rate. Early diagnosis and treatment of this disease is necessary to improve the survival rate of patients, and MRI is an effective tool for detecting osteosarcoma. However, due to the complex structure and variable location of osteosarcoma, cancer cells are highly heterogeneous and prone to aggregation and overlap, making it easy for doctors to inaccurately predict the area of the lesion. In addition, in developing countries lacking professional medical systems, doctors need to examine mass of osteosarcoma MRI images of patients, which is time-consuming and inefficient, and may result in misjudgment and omission. For the sake of reducing labor cost and improve detection efficiency, this paper proposes an Attention Condenser-based MRI image segmentation system for osteosarcoma (OMSAS), which can help physicians quickly locate the lesion area and achieve accurate segmentation of the osteosarcoma tumor region. Using the idea of AttendSeg, we constructed an Attention Condenser-based residual structure network (ACRNet), which greatly reduces the complexity of the structure and enables smaller hardware requirements while ensuring the accuracy of image segmentation. The model was tested on more than 4000 samples from two hospitals in China. The experimental results demonstrate that our model has higher efficiency, higher accuracy and lighter structure for osteosarcoma MRI image segmentation compared to other existing models.

Auxiliary Segmentation Method of Osteosarcoma MRI Image Based on Transformer and U-Net

One of the most prevalent malignant bone tumors is osteosarcoma. The diagnosis and treatment cycle are long and the prognosis is poor. It takes a lot of time to manually identify osteosarcoma from osteosarcoma magnetic resonance imaging (MRI). Medical image processing technology has greatly alleviated the problems faced by medical diagnoses. However, MRI images of osteosarcoma are characterized by high noise and blurred edges. The complex features increase the difficulty of lesion area identification. Therefore, this study proposes an osteosarcoma MRI image segmentation method (OSTransnet) based on Transformer and U-net. This technique primarily addresses the issues of fuzzy tumor edge segmentation and overfitting brought on by data noise. First, we optimize the dataset by changing the precise spatial distribution of noise and the data-increment image rotation process. The tumor is then segmented based on the model of U-Net and Transformer with edge improvement. It compensates for the limitations of U-semantic Net by using channel-based transformers. Finally, we also add an edge enhancement module (BAB) and a combined loss function to improve the performance of edge segmentation. The method's accuracy and stability are demonstrated by the detection and training results based on more than 4,000 MRI images of osteosarcoma, which also demonstrate how well the method works as an adjunct to clinical diagnosis and treatment.

A Multimodal Auxiliary Classification System for Osteosarcoma Histopathological Images Based on Deep Active Learning

Histopathological examination is an important criterion in the clinical diagnosis of osteosarcoma. With the improvement of hardware technology and computing power, pathological image analysis systems based on artificial intelligence have been widely used. However, classifying numerous intricate pathology images by hand is a tiresome task for pathologists. The lack of labeling data makes the system costly and difficult to build. This study constructs a classification assistance system (OHIcsA) based on active learning (AL) and a generative adversarial network (GAN). The system initially uses a small, labeled training set to train the classifier. Then, the most informative samples from the unlabeled images are selected for expert annotation. To retrain the network, the final chosen images are added to the initial labeled dataset. Experiments on real datasets show that our proposed method achieves high classification performance with an AUC value of 0.995 and an accuracy value of 0.989 using a small amount of labeled data. It reduces the cost of building a medical system. Clinical diagnosis can be aided by the system's findings, which can also increase the effectiveness and verifiable accuracy of doctors.

Auxiliary Segmentation Method of Osteosarcoma in MRI Images Based on Denoising and Local Enhancement

Osteosarcoma is a bone tumor which is malignant. There are many difficulties when doctors manually identify patients’ MRI images to complete the diagnosis. The osteosarcoma in MRI images is very complex, making its recognition and segmentation resource-consuming. Automatic osteosarcoma area segmentation can solve these problems to a certain extent. However, existing studies usually fail to balance segmentation accuracy and efficiency. They are either sensitive to noise with low accuracy or time-consuming. So we propose an auxiliary segmentation method based on denoising and local enhancement. The method first optimizes the osteosarcoma images, including removing noise using the Edge Enhancement based Transformer for Medical Image Denoising (Eformer) and using a non-parameter method to localize and enhance the tumor region in MRI images. Osteosarcoma was then segmented by Deep Feature Aggregation for Real-Time Semantic Segmentation (DFANet). Our method achieves impressive segmentation accuracy. Moreover, it is efficient in both time and space. It can provide information about the location and extent of the osteosarcoma as a basis for further diagnosis.

IoMT-Based Osteosarcoma Cancer Detection in Histopathology Images Using Transfer Learning Empowered with Blockchain, Fog Computing, and Edge Computing

Bone tumors, such as osteosarcomas, can occur anywhere in the bones, though they usually occur in the extremities of long bones near metaphyseal growth plates. Osteosarcoma is a malignant lesion caused by a malignant osteoid growing from primitive mesenchymal cells. In most cases, osteosarcoma develops as a solitary lesion within the most rapidly growing areas of the long bones in children. The distal femur, proximal tibia, and proximal humerus are the most frequently affected bones, but virtually any bone can be affected. Early detection can reduce mortality rates. Osteosarcoma's manual detection requires expertise, and it can be tedious. With the assistance of modern technology, medical images can now be analyzed and classified automatically, which enables faster and more efficient data processing. A deep learning-based automatic detection system based on whole slide images (WSIs) is presented in this paper to detect osteosarcoma automatically. Experiments conducted on a large dataset of WSIs yielded up to 99.3% accuracy. This model ensures the privacy and integrity of patient information with the implementation of blockchain technology. Utilizing edge computing and fog computing technologies, the model reduces the load on centralized servers and improves efficiency.

Rethinking U-Net from an Attention Perspective with Transformers for Osteosarcoma MRI Image Segmentation

Osteosarcoma is one of the most common primary malignancies of bone in the pediatric and adolescent populations. The morphology and size of osteosarcoma MRI images often show great variability and randomness with different patients. In developing countries, with large populations and lack of medical resources, it is difficult to effectively address the difficulties of early diagnosis of osteosarcoma with limited physician manpower alone. In addition, with the proposal of precision medicine, existing MRI image segmentation models for osteosarcoma face the challenges of insufficient segmentation accuracy and high resource consumption. Inspired by transformer's self-attention mechanism, this paper proposes a lightweight osteosarcoma image segmentation architecture, UATransNet, by adding a multilevel guided self-aware attention module (MGAM) to the encoder-decoder architecture of U-Net. We successively perform dataset classification optimization and remove MRI image irrelevant background. Then, UATransNet is designed with transformer self-attention component (TSAC) and global context aggregation component (GCAC) at the bottom of the encoder-decoder architecture to perform integration of local features and global dependencies and aggregation of contexts to learned features. In addition, we apply dense residual learning to the convolution module and combined with multiscale jump connections, to improve the feature extraction capability. In this paper, we experimentally evaluate more than 80,000 osteosarcoma MRI images and show that our UATransNet yields more accurate segmentation performance. The IOU and DSC values of osteosarcoma are 0.922 ± 0.03 and 0.921 ± 0.04, respectively, and provide intuitive and accurate efficient decision information support for physicians.

Optimal Deep Stacked Sparse Autoencoder Based Osteosarcoma Detection and Classification Model

Osteosarcoma is a kind of bone cancer which generally starts to develop in the lengthy bones in the legs and arms. Because of an increase in occurrence of cancer and patient-specific treatment options, the detection and classification of cancer becomes a difficult process. The manual recognition of osteosarcoma necessitates expert knowledge and is time consuming. An earlier identification of osteosarcoma can reduce the death rate. With the development of new technologies, automated detection models can be exploited for medical image classification, thereby decreasing the expert’s reliance and resulting in timely identification. In recent times, an amount of Computer-Aided Detection (CAD) systems are available in the literature for the segmentation and detection of osteosarcoma using medicinal images. In this view, this research work develops a wind driven optimization with deep transfer learning enabled osteosarcoma detection and classification (WDODTL-ODC) method. The presented WDODTL-ODC model intends to determine the presence of osteosarcoma in the biomedical images. To accomplish this, the osteosarcoma model involves Gaussian filtering (GF) based on pre-processing and contrast enhancement techniques. In addition, deep transfer learning using a SqueezNet model is utilized as a featured extractor. At last, the Wind Driven Optimization (WDO) algorithm with a deep-stacked sparse auto-encoder (DSSAE) is employed for the classification process. The simulation outcome demonstrated that the WDODTL-ODC technique outperformed the existing models in the detection of osteosarcoma on biomedical images.

Osteosarcoma MRI Image-Assisted Segmentation System Base on Guided Aggregated Bilateral Network

Osteosarcoma is a primary malignant tumor. It is difficult to cure and expensive to treat. Generally, diagnosis is made by analyzing MRI images of patients. In the process of clinical diagnosis, the mainstream method is the still time-consuming and laborious manual screening. Modern computer image segmentation technology can realize the automatic processing of the original image of osteosarcoma and assist doctors in diagnosis. However, to achieve a better effect of segmentation, the complexity of the model is relatively high, and the hardware conditions in developing countries are limited, so it is difficult to use it directly. Based on this situation, we propose an osteosarcoma aided segmentation method based on a guided aggregated bilateral network (OSGABN), which improves the segmentation accuracy of the model and greatly reduces the parameter scale, effectively alleviating the above problems. The fast bilateral segmentation network (FaBiNet) is used to segment images. It is a high-precision model with a detail branch that captures low-level information and a lightweight semantic branch that captures high-level semantic context. We used more than 80,000 osteosarcoma MRI images from three hospitals in China for detection, and the results showed that our model can achieve an accuracy of around 0.95 and a params of 2.33 M.

Intelligent Segmentation Medical Assistance System for MRI Images of Osteosarcoma in Developing Countries

Osteosarcoma is the most common primary malignant bone tumor in children and adolescents. It has a high degree of malignancy and a poor prognosis in developing countries. The doctor manually explained that magnetic resonance imaging (MRI) suffers from subjectivity and fatigue limitations. In addition, the structure, shape, and position of osteosarcoma are complicated, and there is a lot of noise in MRI images. Directly inputting the original data set into the automatic segmentation system will bring noise and cause the model's segmentation accuracy to decrease. Therefore, this paper proposes an osteosarcoma MRI image segmentation system based on a deep convolution neural network, which solves the overfitting problem caused by noisy data and improves the generalization performance of the model. Firstly, we use Mean Teacher to optimize the data set. The noise data is put into the second round of training of the model to improve the robustness of the model. Then, we segment the image using a deep separable U-shaped network (SepUNet) and conditional random field (CRF). SepUnet can segment lesion regions of different sizes at multiple scales; CRF further optimizes the boundary. Finally, this article calculates the area of the tumor area, which provides a more intuitive reference for assisting doctors in diagnosis. More than 80000 MRI images of osteosarcoma from three hospitals in China were tested. The results show that the proposed method guarantees the balance of speed, accuracy, and cost under the premise of improving accuracy.

Automatic detection of osteosarcoma based on integrated features and feature selection using binary arithmetic optimization algorithm

Osteosarcoma is one of the most common malignant bone tumors mostly found in children and teenagers. Manual detection of osteosarcoma requires expertise and it is a labour-intensive process. If detected on time, the mortality rate can be reduced. With the advent of new technologies, automatic detection systems are used to analyse and classify medical images, which reduces the dependency on experts and leads to faster processing. In this paper, an automatic detection system: Integrated Features-Feature Selection Model for Classification (IF-FSM-C) to detect osteosarcoma from the high-resolution whole slide images (WSIs) is proposed. The novelty of the proposed approach is the use of integrated features obtained by fusion of features extracted using traditional handcrafted (HC) feature extraction techniques and deep learning models (DLMs) namely EfficientNet-B0 and Xception. To further improve the performance of the proposed system, feature selection (FS) is performed. Here, two binary variants of recently proposed Arithmetic Optimization Algorithm (AOA) known as BAOA-S and BAOA-V are proposed to perform FS. The selected features are given to a classifier that classifies the WSIs into Viable tumor (VT), Non-viable tumor (NVT) and non-tumor (NT). Experiments are performed to compare the performance of proposed IF-FSM-C to the classifiers which use HC or deep learning features alone as well as state-of-the-art methods for osteosarcoma detection. The best overall accuracy of 96.08% is obtained when integrated features extracted using HC techniques and Xception are used. The overall accuracy is enhanced to 99.54% after applying BAOA-S for FS. Further, the application of BAOA-S for FS reduces the number of features with the best model having only 188 features compared to 2118 features if no FS is applied.