1
|
Jaspers TJM, Boers TGW, Kusters CHJ, Jong MR, Jukema JB, de Groof AJ, Bergman JJ, de With PHN, van der Sommen F. Robustness evaluation of deep neural networks for endoscopic image analysis: Insights and strategies. Med Image Anal 2024; 94:103157. [PMID: 38574544 DOI: 10.1016/j.media.2024.103157] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2023] [Revised: 03/19/2024] [Accepted: 03/21/2024] [Indexed: 04/06/2024]
Abstract
Computer-aided detection and diagnosis systems (CADe/CADx) in endoscopy are commonly trained using high-quality imagery, which is not representative for the heterogeneous input typically encountered in clinical practice. In endoscopy, the image quality heavily relies on both the skills and experience of the endoscopist and the specifications of the system used for screening. Factors such as poor illumination, motion blur, and specific post-processing settings can significantly alter the quality and general appearance of these images. This so-called domain gap between the data used for developing the system and the data it encounters after deployment, and the impact it has on the performance of deep neural networks (DNNs) supportive endoscopic CAD systems remains largely unexplored. As many of such systems, for e.g. polyp detection, are already being rolled out in clinical practice, this poses severe patient risks in particularly community hospitals, where both the imaging equipment and experience are subject to considerable variation. Therefore, this study aims to evaluate the impact of this domain gap on the clinical performance of CADe/CADx for various endoscopic applications. For this, we leverage two publicly available data sets (KVASIR-SEG and GIANA) and two in-house data sets. We investigate the performance of commonly-used DNN architectures under synthetic, clinically calibrated image degradations and on a prospectively collected dataset including 342 endoscopic images of lower subjective quality. Additionally, we assess the influence of DNN architecture and complexity, data augmentation, and pretraining techniques for improved robustness. The results reveal a considerable decline in performance of 11.6% (±1.5) as compared to the reference, within the clinically calibrated boundaries of image degradations. Nevertheless, employing more advanced DNN architectures and self-supervised in-domain pre-training effectively mitigate this drop to 7.7% (±2.03). Additionally, these enhancements yield the highest performance on the manually collected test set including images with lower subjective quality. By comprehensively assessing the robustness of popular DNN architectures and training strategies across multiple datasets, this study provides valuable insights into their performance and limitations for endoscopic applications. The findings highlight the importance of including robustness evaluation when developing DNNs for endoscopy applications and propose strategies to mitigate performance loss.
Collapse
Affiliation(s)
- Tim J M Jaspers
- Department of Electrical Engineering, Video Coding & Architectures, Eindhoven University of Technology, Eindhoven, The Netherlands.
| | - Tim G W Boers
- Department of Electrical Engineering, Video Coding & Architectures, Eindhoven University of Technology, Eindhoven, The Netherlands
| | - Carolus H J Kusters
- Department of Electrical Engineering, Video Coding & Architectures, Eindhoven University of Technology, Eindhoven, The Netherlands
| | - Martijn R Jong
- Department of Gastroenterology and Hepatology, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, The Netherlands
| | - Jelmer B Jukema
- Department of Gastroenterology and Hepatology, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, The Netherlands
| | - Albert J de Groof
- Department of Gastroenterology and Hepatology, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, The Netherlands
| | - Jacques J Bergman
- Department of Gastroenterology and Hepatology, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, The Netherlands
| | - Peter H N de With
- Department of Electrical Engineering, Video Coding & Architectures, Eindhoven University of Technology, Eindhoven, The Netherlands
| | - Fons van der Sommen
- Department of Electrical Engineering, Video Coding & Architectures, Eindhoven University of Technology, Eindhoven, The Netherlands
| |
Collapse
|
2
|
Hung SC, Wang YT, Tseng MH. An Interpretable Three-Dimensional Artificial Intelligence Model for Computer-Aided Diagnosis of Lung Nodules in Computed Tomography Images. Cancers (Basel) 2023; 15:4655. [PMID: 37760624 PMCID: PMC10526230 DOI: 10.3390/cancers15184655] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2023] [Revised: 09/16/2023] [Accepted: 09/17/2023] [Indexed: 09/29/2023] Open
Abstract
Lung cancer is typically classified into small-cell carcinoma and non-small-cell carcinoma. Non-small-cell carcinoma accounts for approximately 85% of all lung cancers. Low-dose chest computed tomography (CT) can quickly and non-invasively diagnose lung cancer. In the era of deep learning, an artificial intelligence (AI) computer-aided diagnosis system can be developed for the automatic recognition of CT images of patients, creating a new form of intelligent medical service. For many years, lung cancer has been the leading cause of cancer-related deaths in Taiwan, with smoking and air pollution increasing the likelihood of developing the disease. The incidence of lung adenocarcinoma in never-smoking women has also increased significantly in recent years, resulting in an important public health problem. Early detection of lung cancer and prompt treatment can help reduce the mortality rate of patients with lung cancer. In this study, an improved 3D interpretable hierarchical semantic convolutional neural network named HSNet was developed and validated for the automatic diagnosis of lung cancer based on a collection of lung nodule images. The interpretable AI model proposed in this study, with different training strategies and adjustment of model parameters, such as cyclic learning rate and random weight averaging, demonstrated better diagnostic performance than the previous literature, with results of a four-fold cross-validation procedure showing calcification: 0.9873 ± 0.006, margin: 0.9207 ± 0.009, subtlety: 0.9026 ± 0.014, texture: 0.9685 ± 0.006, sphericity: 0.8652 ± 0.021, and malignancy: 0.9685 ± 0.006.
Collapse
Affiliation(s)
- Sheng-Chieh Hung
- Master Program in Medical Informatics, Chung Shan Medical University, Taichung 402, Taiwan;
| | - Yao-Tung Wang
- School of Medicine, Chung Shan Medical University, Taichung 402, Taiwan;
- Division of Pulmonary Medicine, Department of Internal Medicine, Chung Shan Medical University Hospital, Taichung 402, Taiwan
| | - Ming-Hseng Tseng
- Master Program in Medical Informatics, Chung Shan Medical University, Taichung 402, Taiwan;
- Department of Medical Informatics, Chung Shan Medical University, Taichung 402, Taiwan
- Information Technology Office, Chung Shan Medical University Hospital, Taichung 402, Taiwan
| |
Collapse
|
3
|
Jiang J, Jiang X, Xu L, Zhang Y, Zheng Y, Kong D. Noise-robustness test for ultrasound breast nodule neural network models as medical devices. Front Oncol 2023; 13:1177225. [PMID: 37427110 PMCID: PMC10325648 DOI: 10.3389/fonc.2023.1177225] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2023] [Accepted: 06/05/2023] [Indexed: 07/11/2023] Open
Abstract
Background Deep learning technology has been widely applied to medical image analysis. But due to the limitations of its own imaging principle, ultrasound image has the disadvantages of low resolution and high Speckle Noise density, which not only hinder the diagnosis of patients' conditions but also affect the extraction of ultrasound image features by computer technology. Objective In this study, we investigate the robustness of deep convolutional neural network (CNN) for classification, segmentation, and target detection of breast ultrasound image through random Salt & Pepper Noise and Gaussian Noise. Methods We trained and validated 9 CNN architectures in 8617 breast ultrasound images, but tested the models with noisy test set. Then, we trained and validated 9 CNN architectures with different levels of noise in these breast ultrasound images, and tested the models with noisy test set. Diseases of each breast ultrasound image in our dataset were annotated and voted by three sonographers based on their malignancy suspiciousness. we use evaluation indexes to evaluate the robustness of the neural network algorithm respectively. Results There is a moderate to high impact (The accuracy of the model decreased by about 5%-40%) on model accuracy when Salt and Pepper Noise, Speckle Noise, or Gaussian Noise is introduced to the images respectively. Consequently, DenseNet, UNet++ and Yolov5 were selected as the most robust model based on the selected index. When any two of these three kinds of noise are introduced into the image at the same time, the accuracy of the model will be greatly affected. Conclusions Our experimental results reveal new insights: The variation trend of accuracy with the noise level in Each network used for classification tasks and object detection tasks has some unique characteristics. This finding provides us with a method to reveal the black-box architecture of computer-aided diagnosis (CAD) systems. On the other hand, the purpose of this study is to explore the impact of adding noise directly to the image on the performance of neural networks, which is different from the existing articles on robustness in the field of medical image processing. Consequently, it provides a new way to evaluate the robustness of CAD systems in the future.
Collapse
Affiliation(s)
- Jiaxin Jiang
- School of Mathematical Sciences, Zhejiang University, Hangzhou, China
| | - Xiaoya Jiang
- Department of Mathematics, The Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong SAR, China
| | - Lei Xu
- Zhejiang Demetics Medical Technology Co., Ltd, Hangzhou, China
| | - Yan Zhang
- Department of Interventional Therapy, The Hwa Mei Hospital of the University of Chinese Academy of Sciences, Ningbo, China
| | - Yuwen Zheng
- School of Mathematical Sciences, Zhejiang University, Hangzhou, China
| | - Dexing Kong
- School of Mathematical Sciences, Zhejiang University, Hangzhou, China
| |
Collapse
|
4
|
A Series-Based Deep Learning Approach to Lung Nodule Image Classification. Cancers (Basel) 2023; 15:cancers15030843. [PMID: 36765801 PMCID: PMC9913559 DOI: 10.3390/cancers15030843] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2022] [Revised: 01/24/2023] [Accepted: 01/28/2023] [Indexed: 02/01/2023] Open
Abstract
Although many studies have shown that deep learning approaches yield better results than traditional methods based on manual features, CADs methods still have several limitations. These are due to the diversity in imaging modalities and clinical pathologies. This diversity creates difficulties because of variation and similarities between classes. In this context, the new approach from our study is a hybrid method that performs classifications using both medical image analysis and radial scanning series features. Hence, the areas of interest obtained from images are subjected to a radial scan, with their centers as poles, in order to obtain series. A U-shape convolutional neural network model is then used for the 4D data classification problem. We therefore present a novel approach to the classification of 4D data obtained from lung nodule images. With radial scanning, the eigenvalue of nodule images is captured, and a powerful classification is performed. According to our results, an accuracy of 92.84% was obtained and much more efficient classification scores resulted as compared to recent classifiers.
Collapse
|
5
|
|
6
|
Image Recognition of Pediatric Pneumonia Based on Fusion of Texture Features and Depth Features. COMPUTATIONAL AND MATHEMATICAL METHODS IN MEDICINE 2022; 2022:1973508. [PMID: 36060651 PMCID: PMC9439900 DOI: 10.1155/2022/1973508] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/15/2022] [Revised: 05/11/2022] [Accepted: 07/07/2022] [Indexed: 11/17/2022]
Abstract
Pneumonia is one of the diseases that seriously endangers human health, and it is also the leading cause of death of children under the age of five in China. The most commonly used imaging examination method for radiologists is mainly based on chest X-ray images. Still, imaging errors often result during imaging examinations due to objective factors such as visual fatigue and lack of experience. Therefore, this paper proposes a feature fusion model, FC-VGG, based on the fusion of texture features (local binary pattern LBP and directional gradient histogram HOG) and depth features. The model improves model performance by adding detailed information in texture features to the convolutional neural network while making the model more suitable for clinical use. We input the X-ray image with texture features into the modified VGG16 model, C-VGG, and then add the Add fusion method to C-VGG for feature fusion so that FC-VGG is obtained, so FC-VGG has texture features detailed information and abstract information of deep features. Through experiments, our model has achieved 92.19% accuracy in recognizing children's pneumonia images, 93.44% average precision, 92.19% average recall, and 92.81% average F1 coefficient, and the model performance exceeds existing deep learning models and traditional feature recognition algorithms.
Collapse
|
7
|
Hu X, Zhong Y, Huang Y, Shen C, Jia X. Improving small animal cone beam CT resolution by mitigating x-ray focal spot induced blurring via deconvolution. Phys Med Biol 2022; 67. [DOI: 10.1088/1361-6560/ac6b7a] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/27/2021] [Accepted: 04/28/2022] [Indexed: 11/12/2022]
Abstract
Abstract
Objective. Modern preclinical small animal radiation platforms utilize cone beam computerized tomography (CBCT) for image guidance and experiment planning purposes. The resolution of CBCT images is of particular importance for visualizing fine animal anatomical structures. One major cause of spatial resolution reduction is the finite size of the x-ray focal spot. In this work, we proposed a simple method to measure x-ray focal spot intensity map and a CBCT image domain deblurring model to mitigate the effect of focal spot-induced image blurring. Approach. We measured a projection image of a tungsten ball bearing using the flat panel detector of the CBCT platform. We built a forward blurring model of the projection image and derived the spot intensity map by deconvolving the measured projection image. Based on the measured spot intensity map, we derived a CBCT image domain blurring model for images reconstructed by the filtered backprojection algorithm. Based on this model, we computed image domain blurring kernel and improved the CBCT image resolution by deconvolving the CBCT image. Main results. We successfully measured the x-ray focal spot intensity map. The spot size characterized by full width at half maximum was ∼0.75 × 0.55 mm2 at 40 kVp. We computed image domain convolution kernels caused by the x-ray focal spot. A simulation study on noiseless projections was performed to evaluate the spatial resolution improvement exclusively by the focal spot kernel, and the modulation transfer function (MTF) at 50% was increased from 1.40 to 1.65 mm−1 for in-plane images and 1.05–1.32 mm−1 for cross-plane images. Experimental studies on a CT insert phantom and a plastinated mouse phantom demonstrated improved spatial resolution after image domain deconvolution, as indicated by visually improved resolution of fine structures. MTF at 50% was improved from 1.00 to 1.12 mm−1 for in-plane direction and from 0.72 to 0.84 mm−1 for cross-plane direction. Significance. The proposed method to mitigate blurring caused by finite x-ray spot size and improve CBCT image resolution is simple and effective.
Collapse
|
8
|
符 帅, 李 明, 边 兆, 马 建. [Performance of low-dose CT image reconstruction for detecting intracerebral hemorrhage: selection of dose, algorithms and their combinations]. NAN FANG YI KE DA XUE XUE BAO = JOURNAL OF SOUTHERN MEDICAL UNIVERSITY 2022; 42:223-231. [PMID: 35365446 PMCID: PMC8983357 DOI: 10.12122/j.issn.1673-4254.2022.02.08] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Subscribe] [Scholar Register] [Received: 10/14/2021] [Indexed: 06/14/2023]
Abstract
OBJECTIVE To investigate the performance of different low-dose CT image reconstruction algorithms for detecting intracerebral hemorrhage. METHODS Low-dose CT imaging simulation was performed on CT images of intracerebral hemorrhage at 30%, 25% and 20% of normal dose level (defined as 100% dose). Seven algorithms were tested to reconstruct low-dose CT images for noise suppression, including filtered back projection algorithm (FBP), penalized weighted least squares-total variation (PWLS-TV), non-local mean filter (NLM), block matching 3D (BM3D), residual encoding-decoding convolutional neural network (REDCNN), the FBP convolutional neural network (FBPConvNet) and image restoration iterative residual convolutional network (IRLNet). A deep learning-based model (CNN-LSTM) was used to detect intracerebral hemorrhage on normal dose CT images and low-dose CT images reconstructed using the 7 algorithms. The performance of different reconstruction algorithms for detecting intracerebral hemorrhage was evaluated by comparing the results between normal dose CT images and low-dose CT images. RESULTS At different dose levels, the low-dose CT images reconstructed by FBP had accuracies of detecting intracerebral hemorrhage of 82.21%, 74.61% and 65.55% at 30%, 25% and 20% dose levels, respectively. At the same dose level (30% dose), the images reconstructed by FBP, PWLS-TV, NLM, BM3D, REDCNN, FBPConvNet and IRLNet algorithms had accuracies for detecting intracerebral hemorrhage of 82.21%, 86.80%, 89.37%, 81.43%, 90.05%, 90.72% and 93.51%, respectively. The images reconstructed by IRLNet at 30%, 25% and 20% dose levels had accuracies for detecting intracerebral hemorrhage of 93.51%, 93.51% and 93.06%, respectively. CONCLUSION The performance of reconstructed low-dose CT images for detecting intracerebral hemorrhage is significantly affected by both dose and reconstruction algorithms. In clinical practice, choosing appropriate dose level and reconstruction algorithm can greatly reduce the radiation dose and ensure the detection performance of CT imaging for intracerebral hemorrhage.
Collapse
Affiliation(s)
- 帅 符
- 南方医科大学生物医学工程学院,广东 广州 510515School of Biomedical Engineering, Southern Medical University, Guangzhou 510515, China
- 广州市医用放射成像与检测技术重点实验室,广东 广州 510515Guangzhou Key Laboratory of Medical Radiation Imaging and Detection Technology, Guangzhou 510515, China
| | - 明强 李
- 琶洲实验室,广东 广州 510515Pazhou Lab, Guangzhou 510515, China
| | - 兆英 边
- 南方医科大学生物医学工程学院,广东 广州 510515School of Biomedical Engineering, Southern Medical University, Guangzhou 510515, China
- 广州市医用放射成像与检测技术重点实验室,广东 广州 510515Guangzhou Key Laboratory of Medical Radiation Imaging and Detection Technology, Guangzhou 510515, China
| | - 建华 马
- 南方医科大学生物医学工程学院,广东 广州 510515School of Biomedical Engineering, Southern Medical University, Guangzhou 510515, China
- 广州市医用放射成像与检测技术重点实验室,广东 广州 510515Guangzhou Key Laboratory of Medical Radiation Imaging and Detection Technology, Guangzhou 510515, China
| |
Collapse
|
9
|
Gao Y, Xiong J, Shen C, Jia X. Improving robustness of a deep learning-based lung-nodule classification model of CT images with respect to image noise. Phys Med Biol 2021; 66. [PMID: 34818638 DOI: 10.1088/1361-6560/ac3d16] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2021] [Accepted: 11/24/2021] [Indexed: 11/12/2022]
Abstract
Objective. Robustness is an important aspect to consider, when developing methods for medical image analysis. This study investigated robustness properties of deep neural networks (DNNs) for a lung nodule classification problem based on CT images and proposed a solution to improve robustness.Approach. We firstly constructed a class of four DNNs with different widths, each predicting an output label (benign or malignant) for an input CT image cube containing a lung nodule. These networks were trained to achieve Area Under the Curve of 0.891-0.914 on a testing dataset. We then added to the input CT image cubes noise signals generated randomly using a realistic CT image noise model based on a noise power spectrum at 100 mAs, and monitored the DNNs output change. We definedSAR5(%) to quantify the robustness of the trained DNN model, indicating that for 5% of CT image cubes, the noise can change the prediction results with a chance of at leastSAR5(%). To understand robustness, we viewed the information processing pipeline by the DNN as a two-step process, with the first step using all but the last layers to extract representations of the input CT image cubes in a latent space, and the second step employing the last fully-connected layer as a linear classifier to determine the position of the sample representations relative to a decision plane. To improve robustness, we proposed to retrain the last layer of the DNN with a Supporting Vector Machine (SVM) hinge loss function to enforce the desired position of the decision plane.Main results.SAR5ranged in 47.0%-62.0% in different DNNs. The unrobustness behavior may be ascribed to the unfavorable placement of the decision plane in the latent representation space, which made some samples be perturbed to across the decision plane and hence susceptible to noise. The DNN-SVM model improved robustness over the DNN model and reducedSAR5by 8.8%-21.0%.Significance. This study provided insights about the potential reason for the unrobustness behavior of DNNs and the proposed DNN-SVM model improved model robustness.
Collapse
Affiliation(s)
- Yin Gao
- Innovative Technology Of Radiotherapy Computations and Hardware (iTORCH) Laboratory, Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas, TX, United States of America
| | - Jennifer Xiong
- Innovative Technology Of Radiotherapy Computations and Hardware (iTORCH) Laboratory, Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas, TX, United States of America
| | - Chenyang Shen
- Innovative Technology Of Radiotherapy Computations and Hardware (iTORCH) Laboratory, Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas, TX, United States of America
| | - Xun Jia
- Innovative Technology Of Radiotherapy Computations and Hardware (iTORCH) Laboratory, Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas, TX, United States of America
| |
Collapse
|
10
|
Cui X, Chang S, Li C, Kong B, Tian L, Wang H, Huang P, Yang M, Wu Y, Li Z. DEAttack: A differential evolution based attack method for the robustness evaluation of medical image segmentation. Neurocomputing 2021. [DOI: 10.1016/j.neucom.2021.08.118] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
11
|
Al-Shabi M, Shak K, Tan M. 3D axial-attention for lung nodule classification. Int J Comput Assist Radiol Surg 2021; 16:1319-1324. [PMID: 34060010 DOI: 10.1007/s11548-021-02415-z] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2021] [Accepted: 05/20/2021] [Indexed: 11/27/2022]
Abstract
PURPOSE In recent years, Non-Local-based methods have been successfully applied to lung nodule classification. However, these methods offer 2D attention or limited 3D attention to low-resolution feature maps. Moreover, they still depend on a convenient local filter such as convolution as full 3D attention is expensive to compute and requires a big dataset, which might not be available. METHODS We propose to use 3D Axial-Attention, which requires a fraction of the computing power of a regular Non-Local network (i.e., self-attention). Unlike a regular Non-Local network, the 3D Axial-Attention network applies the attention operation to each axis separately. Additionally, we solve the invariant position problem of the Non-Local network by proposing to add 3D positional encoding to shared embeddings. RESULTS We validated the proposed method on 442 benign nodules and 406 malignant nodules, extracted from the public LIDC-IDRI dataset by following a rigorous experimental setup using only nodules annotated by at least three radiologists. Our results show that the 3D Axial-Attention model achieves state-of-the-art performance on all evaluation metrics, including AUC and Accuracy. CONCLUSIONS The proposed model provides full 3D attention, whereby every element (i.e., pixel) in the 3D volume space attends to every other element in the nodule effectively. Thus, the 3D Axial-Attention network can be used in all layers without the need for local filters. The experimental results show the importance of full 3D attention for classifying lung nodules.
Collapse
Affiliation(s)
- Mundher Al-Shabi
- Electrical and Computer Systems Engineering Discipline, School of Engineering, Monash University Malaysia, 47500, Bandar Sunway, Selangor, Malaysia.
| | - Kelvin Shak
- Electrical and Computer Systems Engineering Discipline, School of Engineering, Monash University Malaysia, 47500, Bandar Sunway, Selangor, Malaysia
| | - Maxine Tan
- Electrical and Computer Systems Engineering Discipline, School of Engineering, Monash University Malaysia, 47500, Bandar Sunway, Selangor, Malaysia
- School of Electrical and Computer Engineering, University of Oklahoma, Norman, OK, 73019, USA
| |
Collapse
|