1
|
Ryu K, Kitaguchi D, Nakajima K, Ishikawa Y, Harai Y, Yamada A, Lee Y, Hayashi K, Kosugi N, Hasegawa H, Takeshita N, Kinugasa Y, Ito M. Deep learning-based vessel automatic recognition for laparoscopic right hemicolectomy. Surg Endosc 2024; 38:171-178. [PMID: 37950028 DOI: 10.1007/s00464-023-10524-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2023] [Accepted: 10/09/2023] [Indexed: 11/12/2023]
Abstract
BACKGROUND In laparoscopic right hemicolectomy (RHC) for right-sided colon cancer, accurate recognition of the vascular anatomy is required for appropriate lymph node harvesting and safe operative procedures. We aimed to develop a deep learning model that enables the automatic recognition and visualization of major blood vessels in laparoscopic RHC. MATERIALS AND METHODS This was a single-institution retrospective feasibility study. Semantic segmentation of three vessel areas, including the superior mesenteric vein (SMV), ileocolic artery (ICA), and ileocolic vein (ICV), was performed using the developed deep learning model. The Dice coefficient, recall, and precision were utilized as evaluation metrics to quantify the model performance after fivefold cross-validation. The model was further qualitatively appraised by 13 surgeons, based on a grading rubric to assess its potential for clinical application. RESULTS In total, 2624 images were extracted from 104 laparoscopic colectomy for right-sided colon cancer videos, and the pixels corresponding to the SMV, ICA, and ICV were manually annotated and utilized as training data. SMV recognition was the most accurate, with all three evaluation metrics having values above 0.75, whereas the recognition accuracy of ICA and ICV ranged from 0.53 to 0.57 for the three evaluation metrics. Additionally, all 13 surgeons gave acceptable ratings for the possibility of clinical application in rubric-based quantitative evaluations. CONCLUSION We developed a DL-based vessel segmentation model capable of achieving feasible identification and visualization of major blood vessels in association with RHC. This model may be used by surgeons to accomplish reliable navigation of vessel visualization.
Collapse
Affiliation(s)
- Kyoko Ryu
- Surgical Device Innovation, National Cancer Center Hospital East, Chiba, Japan
- Department of Colorectal Surgery, National Cancer Center Hospital East, Chiba, Japan
- Department of Gastrointestinal Surgery, Tokyo Medical and Dental University, Tokyo, Japan
| | - Daichi Kitaguchi
- Surgical Device Innovation, National Cancer Center Hospital East, Chiba, Japan
- Department of Colorectal Surgery, National Cancer Center Hospital East, Chiba, Japan
| | - Kei Nakajima
- Surgical Device Innovation, National Cancer Center Hospital East, Chiba, Japan
- Department of Colorectal Surgery, National Cancer Center Hospital East, Chiba, Japan
| | - Yuto Ishikawa
- Surgical Device Innovation, National Cancer Center Hospital East, Chiba, Japan
| | - Yuriko Harai
- Surgical Device Innovation, National Cancer Center Hospital East, Chiba, Japan
| | - Atsushi Yamada
- Surgical Device Innovation, National Cancer Center Hospital East, Chiba, Japan
| | - Younae Lee
- Surgical Device Innovation, National Cancer Center Hospital East, Chiba, Japan
| | - Kazuyuki Hayashi
- Surgical Device Innovation, National Cancer Center Hospital East, Chiba, Japan
| | - Norihito Kosugi
- Surgical Device Innovation, National Cancer Center Hospital East, Chiba, Japan
| | - Hiro Hasegawa
- Surgical Device Innovation, National Cancer Center Hospital East, Chiba, Japan
- Department of Colorectal Surgery, National Cancer Center Hospital East, Chiba, Japan
| | - Nobuyoshi Takeshita
- Surgical Device Innovation, National Cancer Center Hospital East, Chiba, Japan
- Department of Colorectal Surgery, National Cancer Center Hospital East, Chiba, Japan
| | - Yusuke Kinugasa
- Department of Gastrointestinal Surgery, Tokyo Medical and Dental University, Tokyo, Japan
| | - Masaaki Ito
- Surgical Device Innovation, National Cancer Center Hospital East, Chiba, Japan.
- Department of Colorectal Surgery, National Cancer Center Hospital East, Chiba, Japan.
- Division of Surgical Device Innovation, Department of Colorectal Surgery, National Cancer Center Hospital East, 6-5-1, Kashiwanoha, Kashiwa-Shi, Chiba, 277-8577, Japan.
| |
Collapse
|
2
|
Kitaguchi D, Harai Y, Kosugi N, Hayashi K, Kojima S, Ishikawa Y, Yamada A, Hasegawa H, Takeshita N, Ito M. Artificial intelligence for the recognition of key anatomical structures in laparoscopic colorectal surgery. Br J Surg 2023; 110:1355-1358. [PMID: 37552629 DOI: 10.1093/bjs/znad249] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2023] [Revised: 07/10/2023] [Accepted: 07/10/2023] [Indexed: 08/10/2023]
Abstract
Lay Summary
To prevent intraoperative organ injury, surgeons strive to identify anatomical structures as early and accurately as possible during surgery. The objective of this prospective observational study was to develop artificial intelligence (AI)-based real-time automatic organ recognition models in laparoscopic surgery and to compare its performance with that of surgeons. The time taken to recognize target anatomy between AI and both expert and novice surgeons was compared. The AI models demonstrated faster recognition of target anatomy than surgeons, especially novice surgeons. These findings suggest that AI has the potential to compensate for the skill and experience gap between surgeons.
Collapse
Affiliation(s)
- Daichi Kitaguchi
- Department for the Promotion of Medical Device Innovation, National Cancer Centre Hospital East, Chiba, Japan
- Department of Colorectal Surgery, National Cancer Centre Hospital East, Chiba, Japan
| | - Yuriko Harai
- Department for the Promotion of Medical Device Innovation, National Cancer Centre Hospital East, Chiba, Japan
| | - Norihito Kosugi
- Department for the Promotion of Medical Device Innovation, National Cancer Centre Hospital East, Chiba, Japan
| | - Kazuyuki Hayashi
- Department for the Promotion of Medical Device Innovation, National Cancer Centre Hospital East, Chiba, Japan
| | - Shigehiro Kojima
- Department for the Promotion of Medical Device Innovation, National Cancer Centre Hospital East, Chiba, Japan
| | - Yuto Ishikawa
- Department for the Promotion of Medical Device Innovation, National Cancer Centre Hospital East, Chiba, Japan
| | - Atsushi Yamada
- Department for the Promotion of Medical Device Innovation, National Cancer Centre Hospital East, Chiba, Japan
| | - Hiro Hasegawa
- Department for the Promotion of Medical Device Innovation, National Cancer Centre Hospital East, Chiba, Japan
- Department of Colorectal Surgery, National Cancer Centre Hospital East, Chiba, Japan
| | - Nobuyoshi Takeshita
- Department for the Promotion of Medical Device Innovation, National Cancer Centre Hospital East, Chiba, Japan
| | - Masaaki Ito
- Department for the Promotion of Medical Device Innovation, National Cancer Centre Hospital East, Chiba, Japan
- Department of Colorectal Surgery, National Cancer Centre Hospital East, Chiba, Japan
| |
Collapse
|
3
|
Huaulmé A, Harada K, Nguyen QM, Park B, Hong S, Choi MK, Peven M, Li Y, Long Y, Dou Q, Kumar S, Lalithkumar S, Hongliang R, Matsuzaki H, Ishikawa Y, Harai Y, Kondo S, Mitsuishi M, Jannin P. PEg TRAnsfer Workflow recognition challenge report: Do multimodal data improve recognition? Comput Methods Programs Biomed 2023; 236:107561. [PMID: 37119774 DOI: 10.1016/j.cmpb.2023.107561] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/19/2022] [Revised: 04/06/2023] [Accepted: 04/18/2023] [Indexed: 05/21/2023]
Abstract
BACKGROUND AND OBJECTIVE In order to be context-aware, computer-assisted surgical systems require accurate, real-time automatic surgical workflow recognition. In the past several years, surgical video has been the most commonly-used modality for surgical workflow recognition. But with the democratization of robot-assisted surgery, new modalities, such as kinematics, are now accessible. Some previous methods use these new modalities as input for their models, but their added value has rarely been studied. This paper presents the design and results of the "PEg TRAnsfer Workflow recognition" (PETRAW) challenge with the objective of developing surgical workflow recognition methods based on one or more modalities and studying their added value. METHODS The PETRAW challenge included a data set of 150 peg transfer sequences performed on a virtual simulator. This data set included videos, kinematic data, semantic segmentation data, and annotations, which described the workflow at three levels of granularity: phase, step, and activity. Five tasks were proposed to the participants: three were related to the recognition at all granularities simultaneously using a single modality, and two addressed the recognition using multiple modalities. The mean application-dependent balanced accuracy (AD-Accuracy) was used as an evaluation metric to take into account class balance and is more clinically relevant than a frame-by-frame score. RESULTS Seven teams participated in at least one task with four participating in every task. The best results were obtained by combining video and kinematic data (AD-Accuracy of between 93% and 90% for the four teams that participated in all tasks). CONCLUSION The improvement of surgical workflow recognition methods using multiple modalities compared with unimodal methods was significant for all teams. However, the longer execution time required for video/kinematic-based methods(compared to only kinematic-based methods) must be considered. Indeed, one must ask if it is wise to increase computing time by 2000 to 20,000% only to increase accuracy by 3%. The PETRAW data set is publicly available at www.synapse.org/PETRAW to encourage further research in surgical workflow recognition.
Collapse
Affiliation(s)
- Arnaud Huaulmé
- Univ Rennes, INSERM, LTSI - UMR 1099, Rennes, F35000, France.
| | - Kanako Harada
- Department of Mechanical Engineering, the University of Tokyo, Tokyo 113-8656, Japan
| | | | - Bogyu Park
- VisionAI hutom, Seoul, Republic of Korea
| | | | | | | | | | - Yonghao Long
- Department of Computer Science & Engineering, The Chinese University of Hong Kong, Hong Kong
| | - Qi Dou
- Department of Computer Science & Engineering, The Chinese University of Hong Kong, Hong Kong
| | | | | | - Ren Hongliang
- National University of Singapore, Singapore, Singapore; The Chinese University of Hong Kong, Hong Kong, Hong Kong
| | - Hiroki Matsuzaki
- National Cancer Center Japan East Hospital, Tokyo 104-0045, Japan
| | - Yuto Ishikawa
- National Cancer Center Japan East Hospital, Tokyo 104-0045, Japan
| | - Yuriko Harai
- National Cancer Center Japan East Hospital, Tokyo 104-0045, Japan
| | | | - Manoru Mitsuishi
- Department of Mechanical Engineering, the University of Tokyo, Tokyo 113-8656, Japan
| | - Pierre Jannin
- Univ Rennes, INSERM, LTSI - UMR 1099, Rennes, F35000, France.
| |
Collapse
|
4
|
Kojima S, Kitaguchi D, Igaki T, Nakajima K, Ishikawa Y, Harai Y, Yamada A, Lee Y, Hayashi K, Kosugi N, Hasegawa H, Ito M. Deep-learning-based semantic segmentation of autonomic nerves from laparoscopic images of colorectal surgery: an experimental pilot study. Int J Surg 2023; 109:813-820. [PMID: 36999784 PMCID: PMC10389575 DOI: 10.1097/js9.0000000000000317] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2022] [Accepted: 02/21/2023] [Indexed: 04/01/2023]
Abstract
BACKGROUND The preservation of autonomic nerves is the most important factor in maintaining genitourinary function in colorectal surgery; however, these nerves are not clearly recognisable, and their identification is strongly affected by the surgical ability. Therefore, this study aimed to develop a deep learning model for the semantic segmentation of autonomic nerves during laparoscopic colorectal surgery and to experimentally verify the model through intraoperative use and pathological examination. MATERIALS AND METHODS The annotation data set comprised videos of laparoscopic colorectal surgery. The images of the hypogastric nerve (HGN) and superior hypogastric plexus (SHP) were manually annotated under a surgeon's supervision. The Dice coefficient was used to quantify the model performance after five-fold cross-validation. The model was used in actual surgeries to compare the recognition timing of the model with that of surgeons, and pathological examination was performed to confirm whether the samples labelled by the model from the colorectal branches of the HGN and SHP were nerves. RESULTS The data set comprised 12 978 video frames of the HGN from 245 videos and 5198 frames of the SHP from 44 videos. The mean (±SD) Dice coefficients of the HGN and SHP were 0.56 (±0.03) and 0.49 (±0.07), respectively. The proposed model was used in 12 surgeries, and it recognised the right HGN earlier than the surgeons did in 50.0% of the cases, the left HGN earlier in 41.7% of the cases and the SHP earlier in 50.0% of the cases. Pathological examination confirmed that all 11 samples were nerve tissue. CONCLUSION An approach for the deep-learning-based semantic segmentation of autonomic nerves was developed and experimentally validated. This model may facilitate intraoperative recognition during laparoscopic colorectal surgery.
Collapse
Affiliation(s)
- Shigehiro Kojima
- Surgical Device Innovation
- Department of Colorectal Surgery, National Cancer Center Hospital East, Chiba
- Division of Frontier Surgery, The Institute of Medical Science, The University of Tokyo, Tokyo, Japan
| | - Daichi Kitaguchi
- Surgical Device Innovation
- Department of Colorectal Surgery, National Cancer Center Hospital East, Chiba
| | - Takahiro Igaki
- Surgical Device Innovation
- Department of Colorectal Surgery, National Cancer Center Hospital East, Chiba
| | - Kei Nakajima
- Surgical Device Innovation
- Department of Colorectal Surgery, National Cancer Center Hospital East, Chiba
| | | | | | | | | | | | | | - Hiro Hasegawa
- Surgical Device Innovation
- Department of Colorectal Surgery, National Cancer Center Hospital East, Chiba
| | - Masaaki Ito
- Surgical Device Innovation
- Department of Colorectal Surgery, National Cancer Center Hospital East, Chiba
| |
Collapse
|
5
|
Ota K, Harai Y, Unno H, Sakauchi S, Tomogane H. Corticosterone secretion in response to suckling at various stages of normal and prolonged lactation in rats. J Endocrinol 1974; 62:679-80. [PMID: 4412662 DOI: 10.1677/joe.0.0620679] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [What about the content of this article? (0)] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/10/2023]
|