1
|
Hassan A, Ghafoor M, Tariq SA, Zia T, Ahmad W. High Efficiency Video Coding (HEVC)-Based Surgical Telementoring System Using Shallow Convolutional Neural Network. J Digit Imaging 2019; 32:1027-1043. [PMID: 30980262 PMCID: PMC6841856 DOI: 10.1007/s10278-019-00206-2] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/22/2023] Open
Abstract
Surgical telementoring systems have gained lots of interest, especially in remote locations. However, bandwidth constraint has been the primary bottleneck for efficient telementoring systems. This study aims to establish an efficient surgical telementoring system, where the qualified surgeon (mentor) provides real-time guidance and technical assistance for surgical procedures to the on-spot physician (surgeon). High Efficiency Video Coding (HEVC/H.265)-based video compression has shown promising results for telementoring applications. However, there is a trade-off between the bandwidth resources required for video transmission and quality of video received by the remote surgeon. In order to efficiently compress and transmit real-time surgical videos, a hybrid lossless-lossy approach is proposed where surgical incision region is coded in high quality whereas the background region is coded in low quality based on distance from the surgical incision region. For surgical incision region extraction, state-of-the-art deep learning (DL) architectures for semantic segmentation can be used. However, the computational complexity of these architectures is high resulting in large training and inference times. For telementoring systems, encoding time is crucial; therefore, very deep architectures are not suitable for surgical incision extraction. In this study, we propose a shallow convolutional neural network (S-CNN)-based segmentation approach that consists of encoder network only for surgical region extraction. The segmentation performance of S-CNN is compared with one of the state-of-the-art image segmentation networks (SegNet), and results demonstrate the effectiveness of the proposed network. The proposed telementoring system is efficient and explicitly considers the physiological nature of the human visual system to encode the video by providing good overall visual impact in the location of surgery. The results of the proposed S-CNN-based segmentation demonstrated a pixel accuracy of 97% and a mean intersection over union accuracy of 79%. Similarly, HEVC experimental results showed that the proposed surgical region-based encoding scheme achieved an average bitrate reduction of 88.8% at high-quality settings in comparison with default full-frame HEVC encoding. The average gain in encoding performance (signal-to-noise) of the proposed algorithm is 11.5 dB in the surgical region. The bitrate saving and visual quality of the proposed optimal bit allocation scheme are compared with the mean shift segmentation-based coding scheme for fair comparison. The results show that the proposed scheme maintains high visual quality in surgical incision region along with achieving good bitrate saving. Based on comparison and results, the proposed encoding algorithm can be considered as an efficient and effective solution for surgical telementoring systems for low-bandwidth networks.
Collapse
Affiliation(s)
- Ali Hassan
- Department of Computer Science, COMSATS University, Islamabad, Pakistan
| | - Mubeen Ghafoor
- Department of Computer Science, COMSATS University, Islamabad, Pakistan
| | - Syed Ali Tariq
- Department of Computer Science, COMSATS University, Islamabad, Pakistan.
| | - Tehseen Zia
- Department of Computer Science, COMSATS University, Islamabad, Pakistan
| | - Waqas Ahmad
- Department of Information Systems and Technology, Mid Sweden University, Sundsvall, Sweden
| |
Collapse
|
2
|
Wu Y, Liu P, Gao Y, Jia K. Medical Ultrasound Video Coding with H.265/HEVC Based on ROI Extraction. PLoS One 2016; 11:e0165698. [PMID: 27814367 PMCID: PMC5096667 DOI: 10.1371/journal.pone.0165698] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2016] [Accepted: 10/07/2016] [Indexed: 11/21/2022] Open
Abstract
High-efficiency video compression technology is of primary importance to the storage and transmission of digital medical video in modern medical communication systems. To further improve the compression performance of medical ultrasound video, two innovative technologies based on diagnostic region-of-interest (ROI) extraction using the high efficiency video coding (H.265/HEVC) standard are presented in this paper. First, an effective ROI extraction algorithm based on image textural features is proposed to strengthen the applicability of ROI detection results in the H.265/HEVC quad-tree coding structure. Second, a hierarchical coding method based on transform coefficient adjustment and a quantization parameter (QP) selection process is designed to implement the otherness encoding for ROIs and non-ROIs. Experimental results demonstrate that the proposed optimization strategy significantly improves the coding performance by achieving a BD-BR reduction of 13.52% and a BD-PSNR gain of 1.16 dB on average compared to H.265/HEVC (HM15.0). The proposed medical video coding algorithm is expected to satisfy low bit-rate compression requirements for modern medical communication systems.
Collapse
Affiliation(s)
- Yueying Wu
- Beijing Advanced Innovation Center for Future Internet Technology, Beijing University of Technology, Beijing, China
- Beijing Laboratory of Advanced Information Networks, Beijing, China
- College of Electronic Information and Control Engineering, Beijing University of Technology, Beijing, China
| | - Pengyu Liu
- Beijing Advanced Innovation Center for Future Internet Technology, Beijing University of Technology, Beijing, China
- Beijing Laboratory of Advanced Information Networks, Beijing, China
- College of Electronic Information and Control Engineering, Beijing University of Technology, Beijing, China
- * E-mail: (PL); (KJ)
| | - Yuan Gao
- Beijing Advanced Innovation Center for Future Internet Technology, Beijing University of Technology, Beijing, China
- Beijing Laboratory of Advanced Information Networks, Beijing, China
- College of Electronic Information and Control Engineering, Beijing University of Technology, Beijing, China
| | - Kebin Jia
- Beijing Advanced Innovation Center for Future Internet Technology, Beijing University of Technology, Beijing, China
- Beijing Laboratory of Advanced Information Networks, Beijing, China
- College of Electronic Information and Control Engineering, Beijing University of Technology, Beijing, China
- * E-mail: (PL); (KJ)
| |
Collapse
|
3
|
Antoniou Z, Panayides AS, Pattichis MS, Stavrou S, Kyriacou E, Spanias A, Constantinides AG, Pattichis CS. Adaptive emergency scenery video communications using HEVC for responsive decision support in disaster incidents. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2016; 2015:173-6. [PMID: 26736228 DOI: 10.1109/embc.2015.7318328] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
This study proposes a unifying framework for m-Health video communication systems that provides for the joint optimization of video quality, bitrate demands, and encoding time. The framework is video modality and infrastructure independent and facilitates adaptation to the best available encoding mode that satisfies underlying technology and application imposed constraints. The scalability of the proposed algorithm is demonstrated using different HEVC encoding configurations and realistic modelling of 802.11× wireless infrastructure for emergency scenery and response videos. Extensive experimentation shows that a jointly optimal solution in the encoding time, bitrate, and video quality space is feasible.
Collapse
|
4
|
Avgousti S, Christoforou EG, Panayides AS, Voskarides S, Novales C, Nouaille L, Pattichis CS, Vieyres P. Medical telerobotic systems: current status and future trends. Biomed Eng Online 2016; 15:96. [PMID: 27520552 PMCID: PMC4983067 DOI: 10.1186/s12938-016-0217-7] [Citation(s) in RCA: 47] [Impact Index Per Article: 5.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2016] [Accepted: 08/02/2016] [Indexed: 01/27/2023] Open
Abstract
Teleoperated medical robotic systems allow procedures such as surgeries, treatments, and diagnoses to be conducted across short or long distances while utilizing wired and/or wireless communication networks. This study presents a systematic review of the relevant literature between the years 2004 and 2015, focusing on medical teleoperated robotic systems which have witnessed tremendous growth over the examined period. A thorough insight of telerobotics systems discussing design concepts, enabling technologies (namely robotic manipulation, telecommunications, and vision systems), and potential applications in clinical practice is provided, while existing limitations and future trends are also highlighted. A representative paradigm of the short-distance case is the da Vinci Surgical System which is described in order to highlight relevant issues. The long-distance telerobotics concept is exemplified through a case study on diagnostic ultrasound scanning. Moreover, the present review provides a classification into short- and long-distance telerobotic systems, depending on the distance from which they are operated. Telerobotic systems are further categorized with respect to their application field. For the reviewed systems are also examined their engineering characteristics and the employed robotics technology. The current status of the field, its significance, the potential, as well as the challenges that lie ahead are thoroughly discussed.
Collapse
Affiliation(s)
- Sotiris Avgousti
- Nursing Department, School of Health and Science, Cyprus University of Technology, 30 Archbishop Kyprianou Street, 3036 Limassol, Cyprus
| | - Eftychios G. Christoforou
- Department of Electrical and Computer Engineering, University of Cyprus, 75 Kalipoleos Street, P.O.BOX 20537, 1678 Nicosia, Cyprus
| | - Andreas S. Panayides
- Department of Electrical and Electronic Engineering, Imperial College, South Kensington Campus, London, SW7 2AZ UK
- Department of Computer Science, University of Cyprus, 75 Kalipoleos Street, P.O.BOX 20537, 1678 Nicosia, Cyprus
| | - Sotos Voskarides
- Department of Electrical Engineering, Computer Engineering and Informatics, Cyprus University of Technology, 30 Archbishop Kyprianou Street, 3036 Lemesos, Cyprus
| | - Cyril Novales
- Laboratoire PRISME-Universite d’Orleans, 63 Avenue de Lattre de Tassigny, 18020 Bourges, France
| | - Laurence Nouaille
- Laboratoire PRISME-Universite d’Orleans, 63 Avenue de Lattre de Tassigny, 18020 Bourges, France
| | - Constantinos S. Pattichis
- Department of Computer Science, University of Cyprus, 75 Kalipoleos Street, P.O.BOX 20537, 1678 Nicosia, Cyprus
| | - Pierre Vieyres
- Laboratoire PRISME-Universite d’Orleans, 63 Avenue de Lattre de Tassigny, 18020 Bourges, France
| |
Collapse
|