1
|
Abad SA, Herzig N, Raitt D, Koltzenburg M, Wurdemann H. Bioinspired adaptable multiplanar mechano-vibrotactile haptic system. Nat Commun 2024; 15:7631. [PMID: 39261478 PMCID: PMC11390908 DOI: 10.1038/s41467-024-51779-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2023] [Accepted: 08/16/2024] [Indexed: 09/13/2024] Open
Abstract
Several gaps persist in haptic device development due to the multifaceted nature of the sense of touch. Existing gaps include challenges enhancing touch feedback fidelity, providing diverse haptic sensations, and ensuring wearability for delivering tactile stimuli to the fingertips. Here, we introduce the Bioinspired Adaptable Multiplanar Haptic system, offering mechanotactile/steady and vibrotactile pulse stimuli with adjustable intensity (up to 298.1 mN) and frequencies (up to 130 Hz). This system can deliver simultaneous stimuli across multiple fingertip areas. The paper includes a full characterisation of our system. As the device can play an important role in further understanding human touch, we performed human stimuli sensitivity and differentiation experiments to evaluate the capability of delivering mechano-vibrotactile, variable intensity, simultaneous, multiplanar and operator agnostic stimuli. Our system promises to accelerate the development of touch perception devices, providing painless, operator-independent data crucial for researching and diagnosing touch-related disorders.
Collapse
Affiliation(s)
- Sara-Adela Abad
- Department of Mechanical Engineering, University College London, London, UK.
- Faculty of Agriculture and Renewable Natural Resources, Universidad Nacional de Loja, Loja, Ecuador.
| | - Nicolas Herzig
- School of Engineering and Informatics, University of Sussex, Brighton, UK
| | - Duncan Raitt
- Department of Mechanical Engineering, University College London, London, UK
| | - Martin Koltzenburg
- Queen Square Institute of Neurology, University College London, London, UK
| | - Helge Wurdemann
- Department of Mechanical Engineering, University College London, London, UK
| |
Collapse
|
2
|
Zhang X, Zhang Y, Yang J, Du H. A prostate seed implantation robot system based on human-computer interactions: Augmented reality and voice control. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2024; 21:5947-5971. [PMID: 38872565 DOI: 10.3934/mbe.2024262] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2024]
Abstract
The technology of robot-assisted prostate seed implantation has developed rapidly. However, during the process, there are some problems to be solved, such as non-intuitive visualization effects and complicated robot control. To improve the intelligence and visualization of the operation process, a voice control technology of prostate seed implantation robot in augmented reality environment was proposed. Initially, the MRI image of the prostate was denoised and segmented. The three-dimensional model of prostate and its surrounding tissues was reconstructed by surface rendering technology. Combined with holographic application program, the augmented reality system of prostate seed implantation was built. An improved singular value decomposition three-dimensional registration algorithm based on iterative closest point was proposed, and the results of three-dimensional registration experiments verified that the algorithm could effectively improve the three-dimensional registration accuracy. A fusion algorithm based on spectral subtraction and BP neural network was proposed. The experimental results showed that the average delay of the fusion algorithm was 1.314 s, and the overall response time of the integrated system was 1.5 s. The fusion algorithm could effectively improve the reliability of the voice control system, and the integrated system could meet the responsiveness requirements of prostate seed implantation.
Collapse
Affiliation(s)
- Xinran Zhang
- Key Laboratory of Advanced Manufacturing and Intelligent Technology, Harbin University of Science and Technology, Harbin 150080, China
| | - Yongde Zhang
- Key Laboratory of Advanced Manufacturing and Intelligent Technology, Harbin University of Science and Technology, Harbin 150080, China
- Foshan Baikang Robot Technology Co., Ltd., Foshan 528237, China
| | - Jianzhi Yang
- Key Laboratory of Advanced Manufacturing and Intelligent Technology, Harbin University of Science and Technology, Harbin 150080, China
| | - Haiyan Du
- Key Laboratory of Advanced Manufacturing and Intelligent Technology, Harbin University of Science and Technology, Harbin 150080, China
| |
Collapse
|
3
|
Alkhayatt NM, Alzahrani HH, Ahmed S, Alotaibi BM, Alsaggaf RM, ALAlmuaysh AM, Alomair AA. Computer-assisted navigation in oral and maxillofacial surgery: A systematic review. Saudi Dent J 2024; 36:387-394. [PMID: 38525182 PMCID: PMC10960148 DOI: 10.1016/j.sdentj.2023.12.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2023] [Revised: 12/03/2023] [Accepted: 12/06/2023] [Indexed: 03/26/2024] Open
Abstract
Background The term "navigation" describes a device that can pinpoint critical anatomical features, the most direct path to the target, and the optimal surgical orientation. This study aimed to conduct a comprehensive literature search on computer-assisted navigation for use in oral and maxillofacial surgery. Methods Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement, relevant studies were retrieved from five electronic databases: Medline, Web of Science, PubMed, Google Scholar, and Saudi Digital Library (SDL). The central question was, "Does the computer-assisted navigation system improve the outcome of surgical procedures in the oral and maxillofacial region?" The Cochrane Risk of Bias 2 was used to determine the various types of bias. Results Post-traumatic midfacial reconstruction is one of the many fields that have benefited from the use of computer-assisted navigation because of its reliability. It can also be used to extricate difficult foreign entities from the operative zone. Locating critical anatomical components, communicating the surgical plan to the patient, and verifying surgical success can improve the function and appearance of patients with dentofacial abnormalities. In addition, it decreases the surgical error margin and duration. Conclusion Computer-assisted navigation is promising in surgical practice. The accuracy of surgery can be significantly enhanced by first planning the process in a virtual environment and then performing it under close supervision in real time. In addition, the time required for preoperative planning and surgery can be reduced by creating and improving software programs.
Collapse
Affiliation(s)
| | - Hadeel H Alzahrani
- College of Dentistry, Princess Nourah bint Abdulrahman University, Riyadh, Saudi Arabia
| | - Suhael Ahmed
- Department of Oral and Maxillofacial Surgery, College of Dentistry, Riyadh Elm University, Riyadh, Saudi Arabia
| | - Bassam M Alotaibi
- College of Dentistry, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia
| | | | | | | |
Collapse
|
4
|
Shi Y, Zhu P, Wang T, Mai H, Yeh X, Yang L, Wang J. Dynamic Virtual Fixture Generation Based on Intra-Operative 3D Image Feedback in Robot-Assisted Minimally Invasive Thoracic Surgery. SENSORS (BASEL, SWITZERLAND) 2024; 24:492. [PMID: 38257585 PMCID: PMC10820968 DOI: 10.3390/s24020492] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/12/2023] [Revised: 01/09/2024] [Accepted: 01/10/2024] [Indexed: 01/24/2024]
Abstract
This paper proposes a method for generating dynamic virtual fixtures with real-time 3D image feedback to facilitate human-robot collaboration in medical robotics. Seamless shared control in a dynamic environment, like that of a surgical field, remains challenging despite extensive research on collaborative control and planning. To address this problem, our method dynamically creates virtual fixtures to guide the manipulation of a trocar-placing robot arm using the force field generated by point cloud data from an RGB-D camera. Additionally, the "view scope" concept selectively determines the region for computational points, thereby reducing computational load. In a phantom experiment for robot-assisted port incision in minimally invasive thoracic surgery, our method demonstrates substantially improved accuracy for port placement, reducing error and completion time by 50% (p=1.06×10-2) and 35% (p=3.23×10-2), respectively. These results suggest that our proposed approach is promising in improving surgical human-robot collaboration.
Collapse
Affiliation(s)
- Yunze Shi
- ZJU-UIUC Institute, International Campus, Zhejiang University, Haining 314400, China; (Y.S.); (T.W.); (H.M.)
- School of Mechanical Engineering, Zhejiang University, Hangzhou 310058, China
| | - Peizhang Zhu
- Flexiv Ltd., Santa Clara, CA 95054, USA; (P.Z.); (X.Y.)
| | - Tengyue Wang
- ZJU-UIUC Institute, International Campus, Zhejiang University, Haining 314400, China; (Y.S.); (T.W.); (H.M.)
- School of Mechanical Engineering, Zhejiang University, Hangzhou 310058, China
| | - Haonan Mai
- ZJU-UIUC Institute, International Campus, Zhejiang University, Haining 314400, China; (Y.S.); (T.W.); (H.M.)
| | - Xiyang Yeh
- Flexiv Ltd., Santa Clara, CA 95054, USA; (P.Z.); (X.Y.)
| | - Liangjing Yang
- ZJU-UIUC Institute, International Campus, Zhejiang University, Haining 314400, China; (Y.S.); (T.W.); (H.M.)
- School of Mechanical Engineering, Zhejiang University, Hangzhou 310058, China
- Department of Mechanical Engineering, University of Illinois Urbana-Champaign, Urbana, IL 61801, USA
| | - Jingfan Wang
- Flexiv Ltd., Santa Clara, CA 95054, USA; (P.Z.); (X.Y.)
| |
Collapse
|
5
|
Ficuciello F, Villani A, Lisini Baldi T, Prattichizzo D. A Human Gesture Mapping Method to Control a Multi-Functional Hand for Robot-Assisted Laparoscopic Surgery: The MUSHA Case. Front Robot AI 2022; 8:741807. [PMID: 34993237 PMCID: PMC8725602 DOI: 10.3389/frobt.2021.741807] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2021] [Accepted: 10/29/2021] [Indexed: 11/13/2022] Open
Abstract
This work presents a novel technique to control multi-functional hand for robot-assisted laparoscopic surgery. We tested the technique using the MUSHA multi-functional hand, a robot-aided minimally invasive surgery tool with more degrees of freedom than the standard commercial end-effector of the da Vinci robot. Extra degrees of freedom require the development of a proper control strategy to guarantee high performance and avoid an increasing complexity of control consoles. However, developing reliable control algorithms while reducing the control side’s mechanical complexity is still an open challenge. In the proposed solution, we present a control strategy that projects the human hand motions into the robot actuation space. The human hand motions are tracked by a LeapMotion camera and mapped into the actuation space of the virtualized end-effector. The effectiveness of the proposed method was evaluated in a twofold manner. Firstly, we verified the Lyapunov stability of the algorithm, then an user study with 10 subjects assessed the intuitiveness and usability of the system.
Collapse
Affiliation(s)
- Fanny Ficuciello
- Department of Electrical Engineering and Information Technology, University of Naples Federico II, Napoli, Italy
| | - Alberto Villani
- Department of Information Engineering and Mathematics, University of Siena, Siena, Italy
| | - Tommaso Lisini Baldi
- Department of Information Engineering and Mathematics, University of Siena, Siena, Italy
| | - Domenico Prattichizzo
- Department of Information Engineering and Mathematics, University of Siena, Siena, Italy.,Department of Advanced Robotics, Istituto Italiano di Tecnologia, Genova, Italy
| |
Collapse
|
6
|
Zolotas M, Wonsick M, Long P, Padır T. Motion Polytopes in Virtual Reality for Shared Control in Remote Manipulation Applications. Front Robot AI 2021; 8:730433. [PMID: 34568439 PMCID: PMC8458706 DOI: 10.3389/frobt.2021.730433] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2021] [Accepted: 08/23/2021] [Indexed: 11/13/2022] Open
Abstract
In remote applications that mandate human supervision, shared control can prove vital by establishing a harmonious balance between the high-level cognition of a user and the low-level autonomy of a robot. Though in practice, achieving this balance is a challenging endeavor that largely depends on whether the operator effectively interprets the underlying shared control. Inspired by recent works on using immersive technologies to expose the internal shared control, we develop a virtual reality system to visually guide human-in-the-loop manipulation. Our implementation of shared control teleoperation employs end effector manipulability polytopes, which are geometrical constructs that embed joint limit and environmental constraints. These constructs capture a holistic view of the constrained manipulator’s motion and can thus be visually represented as feedback for users on their operable space of movement. To assess the efficacy of our proposed approach, we consider a teleoperation task where users manipulate a screwdriver attached to a robotic arm’s end effector. A pilot study with prospective operators is first conducted to discern which graphical cues and virtual reality setup are most preferable. Feedback from this study informs the final design of our virtual reality system, which is subsequently evaluated in the actual screwdriver teleoperation experiment. Our experimental findings support the utility of using polytopes for shared control teleoperation, but hint at the need for longer-term studies to garner their full benefits as virtual guides.
Collapse
Affiliation(s)
- Mark Zolotas
- Northeastern University, Boston, MA, United States
| | | | - Philip Long
- Northeastern University, Boston, MA, United States.,Irish Manufacturing Research, Dublin, Ireland
| | - Taşkın Padır
- Northeastern University, Boston, MA, United States
| |
Collapse
|
7
|
Jarota M. Artificial intelligence and robotisation in the EU - should we change OHS law? J Occup Med Toxicol 2021; 16:18. [PMID: 33952297 PMCID: PMC8097897 DOI: 10.1186/s12995-021-00301-7] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2020] [Accepted: 03/30/2021] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Technological progress in the twenty-first century offers real chances for economic development of the European Union (EU). The purpose of this publication is to analyse risks and threats relating to Occupational Health and Safety (OHS) considerations in the context of scientific and technological development. The article attempts the analysis of whether current legislation of the European Union enables good protection of workers' health in the performance of their duties using robots, artificial intelligence (AI). A feature of robotisation and AI may be new challenges in OHS protection. The analysis performed aims to determine whether threats posted by working with Artificial Intelligence are serious enough for the EU Legislator to focus on implementation of new OHS regulations. METHODS The analysis was carried out on the basis of current legal regulations related to the protection of employee's health in the European Union. The study used literature related to robotisation with artificial intelligence and health and safety at work in the working environment. RESULTS Given the new psychological and physical threats related to the use of AI robots, it is necessary to expand the EU legislation with general guidelines for the use of intelligent robots in the work environment. Indeed, such robots must be defined in the applicable legal framework. Employers should also define, as part of their internal regulations, the procedures for employee communication with artificial intelligence, and relevantly update their training in the OHS area. CONCLUSIONS The developments in AI-assisted robots come with inherent risks and threats to the working environment. New challenges create the need for adapting EU laws to changing reality. In order to structure European Union legislation on health and safety at work, these changes could be defined in a single piece of legislation covering robotics and AI after detailed analysis, dialogue, and debate.
Collapse
Affiliation(s)
- Maciej Jarota
- Department of Labour Law and Social Security Law, Institute of Law, John Paul II Catholic University of Lublin, Al. Raclawickie 14, 20-950, Lublin, Poland.
| |
Collapse
|
8
|
Yang Y, Yung KL, Hung TWR, Yu KM. Analyzing Liver Surface Indentation for In Vivo Refinement of Tumor Location in Minimally Invasive Surgery. Ann Biomed Eng 2021; 49:1402-1415. [PMID: 33258091 PMCID: PMC8058013 DOI: 10.1007/s10439-020-02698-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2020] [Accepted: 11/18/2020] [Indexed: 10/27/2022]
Abstract
Manual palpation to update the position of subsurface tumor(s) is a normal practice in open surgery, but is not possible through the small incisions of minimally invasive surgery (MIS). This paper proposes a method that has the potential to use a simple constant-force indenter and the existing laparoscopic camera for tumor location refinement in MIS. The indenter floats with organ movement to generate a static surface deformation on the soft tissue, resolving problems of previous studies that require complicated measurement of force and displacement during indentation. By analyzing the deformation profile, we can intraoperatively update the tumor's location in real-time. Indentation experiments were conducted on healthy and "diseased" porcine liver specimens to obtain the deformation surrounding the indenter site. An inverse finite element (FE) algorithm was developed to determine the optimal material parameters of the healthy liver tissue. With these parameters, a computational model of tumorous tissue was constructed to quantitatively evaluate the effects of the tumor location on the induced deformation. By relating the experimental data from the "diseased" liver specimen to the computational results, we estimated the radial distance between the tumor and the indenter, as well as the angular position of the tumor relative to the indenter.
Collapse
Affiliation(s)
- Yingqiao Yang
- Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, R606, 1 Yuk Road, Hung Hom, Kowloon, Hong Kong.
| | - Kai-Leung Yung
- Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, R606, 1 Yuk Road, Hung Hom, Kowloon, Hong Kong
| | - Tin Wai Robert Hung
- Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, R606, 1 Yuk Road, Hung Hom, Kowloon, Hong Kong
| | - Kai-Ming Yu
- Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, R606, 1 Yuk Road, Hung Hom, Kowloon, Hong Kong
| |
Collapse
|
9
|
Abstract
There is a need for semi-autonomous systems capable of performing both automated tasks and supervised maneuvers. When dealing with multiple robots or robots with high complexity (such as humanoids), we face the issue of effectively coordinating operators across robots. We build on our previous work to present a methodology for designing trajectories and policies for robots such that a few operators can supervise multiple robots. Specifically, we: (1) Analyze the complexity of the problem, (2) Design a procedure for generating policies allowing operators to oversee many robots, (3) Present a method for designing policies and robot trajectories to allow operators to oversee multiple robots, and (4) Include both simulation and hardware experiments demonstrating our methodologies.
Collapse
|
10
|
Yang Y, Li K, Sommer G, Yung KL, Holzapfel GA. Mechanical characterization of porcine liver properties for computational simulation of indentation on cancerous tissue. MATHEMATICAL MEDICINE AND BIOLOGY-A JOURNAL OF THE IMA 2020; 37:469-490. [PMID: 32424396 DOI: 10.1093/imammb/dqaa006] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/17/2019] [Revised: 04/06/2020] [Accepted: 04/06/2020] [Indexed: 11/13/2022]
Abstract
An accurate characterization of soft biological tissue properties is essential for a realistic simulation of surgical procedures. Unconfined uniaxial compression tests with specimens affixed to the fixtures are often performed to characterize the stress-stretch curves of soft biological tissues, with which the material parameters can be obtained. However, the constrained boundary condition causes non-uniform deformation during the uniaxial test, posing challenges for accurate measurement of tissue deformation. In this study, we measured the deformation locally at the middle of liver specimens and obtained the corresponding stress-stretch curves. Since the effect of the constrained boundary condition on the local deformation of specimen is minimized, the stress-stretch curves are thus more realistic. Subsequently, we fitted the experimental stress-stretch curves with several constitutive models and found that the first-order Ogden hyperelastic material model was most suitable for characterizing the mechanical properties of porcine liver tissues. To further verify the characterized material properties, we carried out indentation tests on porcine liver specimens and compared the experimental data with computational results by using finite element simulations. A good agreement was achieved. Finally, we constructed computational models of liver tissue with a tumor and investigated the effect of the tumor on the mechanical response of the tissue under indentation. The computational results revealed that the liver specimen with tumor shows a stiffer response if the distance between the tumor and the indenter is small.
Collapse
Affiliation(s)
- Yingqiao Yang
- Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, 1 Yuk Road, Hung Hom, Kowloon, Hong Kong
| | - Kewei Li
- Institute of Biomechanics, Graz University of Technology, Stremayrgasse 16-II, 8010 Graz, Austria
| | - Gerhard Sommer
- Institute of Biomechanics, Graz University of Technology, Stremayrgasse 16-II, 8010 Graz, Austria
| | - Kai-Leung Yung
- Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, 1 Yuk Road, Hung Hom, Kowloon, Hong Kong
| | - Gerhard A Holzapfel
- Institute of Biomechanics, Graz University of Technology, Stremayrgasse 16-II, 8010 Graz, Austria and Department of Structural Engineering, Norwegian University of Science and Technology (NTNU), 7491 Trondheim, Norway
| |
Collapse
|
11
|
Abstract
During traditional surgery, the surgeons' hands are in direct contact with organs, and surgeons rely on the sense of touch to perform surgery. In teleoperated robotic systems, all physical connections between the surgeon and both the robot and patient, are absent. The surgeon must estimate the force exerted on organs, based only on visual deformation of tissues he is pulling, pushing, gripping, or suturing. It is hard to imagine how to operate with no haptic sensations, and it is surprising that commercially available robots didn't include until now any Haptic Feedback, despite reports about tissue injury, and inability to perform complex manipulation. The sense of touch must be created by stimuli sensed by the surgeon. Haptic sensors are required to collect and send haptic information, and display them on the operator's side, creating telepresence, known as transparency. Multiple ways have been developed to improve transparency through force feedback and tactile feedback. However, this interferes with the stability of the closed-loop controlling interactions between master, robot and remote environment. Cutaneous feedback is more stable and less transparent; force feedback is more transparent and less stable. Thus, multimodal platforms of haptic feedback would try to find the best trade-off between both modalities.
Collapse
Affiliation(s)
| | - Jean-Michel El Rassi
- Department of Mechanical Engineering, Imperial College London, London, United Kingdom of Great Britain and Northern Ireland
| |
Collapse
|
12
|
A novel palpation-based method for tumor nodule quantification in soft tissue-computational framework and experimental validation. Med Biol Eng Comput 2020; 58:1369-1381. [PMID: 32279204 PMCID: PMC7211792 DOI: 10.1007/s11517-020-02168-y] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2019] [Accepted: 03/21/2020] [Indexed: 12/02/2022]
Abstract
Variation in mechanical properties is a useful marker for cancer in soft tissue and has been used in clinical diagnosis for centuries. However, to develop such methods as instrumented palpation, there remain challenges in using the mechanical response during palpation to quantify tumor load. This study proposes a computational framework of identification and quantification of cancerous nodules in soft tissue without a priori knowledge of its geometry, size, and depth. The methodology, using prostate tissue as an exemplar, is based on instrumented palpation performed at positions with various indentation depths over the surface of the relevant structure (in this case, the prostate gland). The profile of force feedback results is then compared with the benchmark in silico models to estimate the size and depth of the cancerous nodule. The methodology is first demonstrated using computational models and then validated using tissue-mimicking gelatin phantoms, where the depth and volume of the tumor nodule is estimated with good accuracy. The proposed framework is capable of quantifying a tumor nodule in soft tissue without a priori information about its geometry, thus presenting great promise in clinical palpation diagnosis for a wide variety of solid tumors including breast and prostate cancer. This study proposes a computational framework of quantification of cancerous nodules in soft tissue. The methodology is based on instrumental palpation performed at positions with various indentation depths. The profile of force feedback results is then compared with the benchmark in silico models to estimate the size and depth of the cancerous nodule. ![]()
Collapse
|
13
|
Qian L, Wu JY, DiMaio SP, Navab N, Kazanzides P. A Review of Augmented Reality in Robotic-Assisted Surgery. ACTA ACUST UNITED AC 2020. [DOI: 10.1109/tmrb.2019.2957061] [Citation(s) in RCA: 33] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
14
|
Kuntz A, Fu M, Alterovitz R. Planning High-Quality Motions for Concentric Tube Robots in Point Clouds via Parallel Sampling and Optimization. PROCEEDINGS OF THE ... IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS. IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS 2020; 2019:2205-2212. [PMID: 32355572 DOI: 10.1109/iros40897.2019.8968172] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
We present a method that plans motions for a concentric tube robot to automatically reach surgical targets inside the body while avoiding obstacles, where the patient's anatomy is represented by point clouds. Point clouds can be generated intra-operatively via endoscopic instruments, enabling the system to update obstacle representations over time as the patient anatomy changes during surgery. Our new motion planning method uses a combination of sampling-based motion planning methods and local optimization to efficiently handle point cloud data and quickly compute high quality plans. The local optimization step uses an interior point optimization method, ensuring that the computed plan is feasible and avoids obstacles at every iteration. This enables the motion planner to run in an anytime fashion, i.e., the method can be stopped at any time and the best solution found up until that point is returned. We demonstrate the method's efficacy in three anatomical scenarios, including two generated from endoscopic videos of real patient anatomy.
Collapse
Affiliation(s)
- Alan Kuntz
- Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| | - Mengyu Fu
- Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| | - Ron Alterovitz
- Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| |
Collapse
|
15
|
Chumnanvej S, Pattamarakha D, Sudsang T, Suthakorn J. Anatomical Workspace Study of Endonasal Endoscopic Transsphenoidal Approach. Open Med (Wars) 2019; 14:537-544. [PMID: 31667352 PMCID: PMC6814958 DOI: 10.1515/med-2019-0060] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2018] [Accepted: 04/23/2019] [Indexed: 12/19/2022] Open
Abstract
Purpose To determine the workspace through an anatomical dimensional study of the skull base to further facilitate the design of the robot for endonasal endoscopic transsphenoidal (EET) surgery. Methods There were 120 cases having a paranasal sinus CT scan in the database. The internal volumes of the nasal cavities (NC), the volumes of the sphenoid sinuses (SS), and the distance between the anterior nasal spine and base of the sellar (d-ANS-BS) were measured. Results The Pearson correlation coefficient (PCC) between the relevant distances and the volumes of the right NC was 0.32; between the relevant distances and the volumes of the left NC was 0.43; and between the relevant distances and volumes of NC was 0.41; with a statistically significant difference (p < 0.001). All PCCs had a statistically significant meaningful difference (p < 0.05). Conclusion The volume of NCs were significantly correlated with distances (p < 0.05). The safest and shortest distance to guide the robotic arm length in the EET approach could be represented by d-ANS-BS. This result was also used as primary information for further robotic design.
Collapse
Affiliation(s)
- Sorayouth Chumnanvej
- Neurosurgery Division, Department of Surgery, Faculty of Medicine Ramathibodi Hospital, Mahidol University, Bangkok, Thailand
| | - Duangkamol Pattamarakha
- Department of Diagnostic and Therapeutic Radiology, Faculty of Medicine Ramathibodi Hospital, Mahidol University, Bangkok, Thailand
| | - Thanwa Sudsang
- Department of Diagnostic and Therapeutic Radiology, Faculty of Medicine Ramathibodi Hospital, Mahidol University, Bangkok, Thailand
| | - Jackrit Suthakorn
- Center for Biomedical and Robotics Technology (BART LAB), Department of Biomedical Engineering, Faculty of Engineering, Mahidol University, Salaya, Thailand
- Phone: +662-441-4255; fax: +662-441-4254, ORCID id: - 0000-0003-1333-3982
| |
Collapse
|
16
|
Chumnanvej S, Pillai BM, Suthakorn J. Surgical Robotic Technology for Developing an Endonasal Endoscopic Transsphenoidal Surgery (EETS) Robotic System. Open Neurol J 2019. [DOI: 10.2174/1874205x01913010096] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/23/2023] Open
Abstract
Purpose:
Neurosurgical robots are acknowledged for their advantages in assisting neurosurgeons in enhancing their precision and accuracy. Here, the aim of this study is to report the first use as a robot-assisted Endonasal Endoscopic Transsphenoidal (EET) approach, applied to reach sphenoid sinus in a cadaver. The introduction of the seven tenets for the Endonasal Endoscopic Transsphenoidal approach will propel the feasibility of neurosurgical techniques and instruments for intrasellar lesions.
Methods:
Endonasal endoscopic transsphenoidal approach in a cadaver was performed under robot assistance with simple navigation system. This preliminary study reveals the accuracy and precision of the robot to reach a target at sphenoid sinus safely and within the shortest duration .
Results:
This robotic technology provided the foundation to support neurosurgeons when they are working in narrow and complicated surgical corridors with accuracy and precision.
Conclusion:
This article reveals the first robot-assisted Endonasal Endoscopic Transsphenoidal approach. This demonstrates the feasibility of the evolution and will augment neurosurgeons toward their limits of minimally invasive surgical techniques, manual dexterity, and spatial orientation. These tenets will be as state of the art and overcome the future challenges of Endonasal Endoscopic Transsphenoidal approach shortly.
Collapse
|
17
|
Remote Presence: Development and Usability Evaluation of a Head-Mounted Display for Camera Control on the da Vinci Surgical System. ROBOTICS 2019. [DOI: 10.3390/robotics8020031] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022] Open
Abstract
This paper describes the development of a new method to control the camera arm of a surgical robot and create a better sense of remote presence for the surgeon. The current surgical systems are entirely controlled by the surgeon, using hand controllers and foot pedals to manipulate either the instrument or the camera arms. The surgeon must pause the operation to move the camera arm to obtain a desired view and then resume the operation. The camera and tools cannot be moved simultaneously, leading to interrupted and unnatural movements. These interruptions can lead to medical errors and extended operation times. In our system, the surgeon controls the camera arm by his natural head movements while being immersed in a 3D-stereo view of the scene with a head-mounted display (HMD). The novel approach enables the camera arm to be maneuvered based on sensors of the HMD. We implemented this method on a da Vinci Standard Surgical System using the HTC Vive headset along with the Unity engine and the Robot Operating System framework. This paper includes the result of a subjective six-participant usability study that compares the workload of the traditional clutched camera control method against the HMD-based control. Initial results indicate that the system is usable, stable, and has a lower physical and mental workload when using the HMD control method.
Collapse
|
18
|
Torabi A, Khadem M, Zareinia K, Sutherland GR, Tavakoli M. Application of a Redundant Haptic Interface in Enhancing Soft-Tissue Stiffness Discrimination. IEEE Robot Autom Lett 2019. [DOI: 10.1109/lra.2019.2893606] [Citation(s) in RCA: 25] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
|
19
|
Haptic Glove and Platform with Gestural Control For Neuromorphic Tactile Sensory Feedback In Medical Telepresence †. SENSORS 2019; 19:s19030641. [PMID: 30717482 PMCID: PMC6386988 DOI: 10.3390/s19030641] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/11/2018] [Revised: 01/15/2019] [Accepted: 01/29/2019] [Indexed: 01/20/2023]
Abstract
Advancements in the study of the human sense of touch are fueling the field of haptics. This is paving the way for augmenting sensory perception during object palpation in tele-surgery and reproducing the sensed information through tactile feedback. Here, we present a novel tele-palpation apparatus that enables the user to detect nodules with various distinct stiffness buried in an ad-hoc polymeric phantom. The contact force measured by the platform was encoded using a neuromorphic model and reproduced on the index fingertip of a remote user through a haptic glove embedding a piezoelectric disk. We assessed the effectiveness of this feedback in allowing nodule identification under two experimental conditions of real-time telepresence: In Line of Sight (ILS), where the platform was placed in the visible range of a user; and the more demanding Not In Line of Sight (NILS), with the platform and the user being 50 km apart. We found that the entailed percentage of identification was higher for stiffer inclusions with respect to the softer ones (average of 74% within the duration of the task), in both telepresence conditions evaluated. These promising results call for further exploration of tactile augmentation technology for telepresence in medical interventions.
Collapse
|
20
|
An Augmented Reality Endoscope System for Ureter Position Detection. J Med Syst 2018; 42:138. [PMID: 29938379 DOI: 10.1007/s10916-018-0992-8] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2017] [Accepted: 06/12/2018] [Indexed: 01/30/2023]
Abstract
Iatrogenic injury of ureter in the clinical operation may cause the serious complication and kidney damage. To avoid such a medical accident, it is necessary to provide the ureter position information to the doctor. For the detection of ureter position, an ureter position detection and display system with the augmented ris proposed to detect the ureter that is covered by human tissue. There are two key issues which should be considered in this new system. One is how to detect the covered ureter that cannot be captured by the electronic endoscope and the other is how to display the ureter position that provides stable and high-quality images. Simultaneously, any delayed processing of the system should disturb the surgery. The aided hardware detection method and target detection algorithms are proposed in this system. To mark the ureter position, a surface-lighting plastic optical fiber (POF) with the encoded light-emitting diode (LED) light is used to indicate the ureter position. The monochrome channel filtering algorithm (MCFA) is proposed to locate the ureter region more precisely. The ureter position is extracted using the proposed automatic region growing algorithm (ARGA) that utilizes the statistical information of the monochrome channel for the selection of growing seed point. In addition, according to the pulse signal of encoded light, the recognition of bright and dark frames based on the aided hardware (BDAH) is proposed to expedite the processing speed. Experimental results demonstrate that the proposed endoscope system can identify 92.04% ureter region in average.
Collapse
|
21
|
Fukuda T, Tanaka Y, Fujiwara M, Sano A. DNN-Based Assistant in Laparoscopic Computer-Aided Palpation. Front Robot AI 2018; 5:71. [PMID: 33500950 PMCID: PMC7806085 DOI: 10.3389/frobt.2018.00071] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2018] [Accepted: 05/30/2018] [Indexed: 11/13/2022] Open
Abstract
Tactile sensory input of surgeons is severely limited in minimally invasive surgery, therefore manual palpation cannot be performed for intraoperative tumor detection. Computer-aided palpation, in which tactile information is acquired by devices and relayed to the surgeon, is one solution for overcoming this limitation. An important design factor is the method by which the acquired information is fed back to the surgeon. However, currently there is no systematic method for achieving this aim, and it is possible that a badly implemented feedback mechanism could adversely affect the performance of the surgeon. In this study, we propose an assistance algorithm for intraoperative tumor detection in laparoscopic surgery. Our scenario is that the surgeon manipulates a sensor probe, makes a decision based on the feedback provided from the sensor, while simultaneously, the algorithm analyzes the time series of the sensor output. Thus, the algorithm assists the surgeon in making decisions by providing independent detection results. A deep neural network model with three hidden layers was used to analyze the sensor output. We propose methods to input the time series of the sensor output to the model for real-time analysis, and to determine the criterion for detection by the model. This study aims to validate the feasibility of the algorithm by using data acquired in our previous psychophysical experiment. There, novice participants were asked to detect a phantom of an early-stage gastric tumor through visual feedback from the tactile sensor. In addition to the analysis of the accuracy, signal detection theory was employed to assess the potential detection performance of the model. The detection performance was compared with that of human participants. We conducted two types of validation, and found that the detection performance of the model was not significantly different from that of the human participants if the data from a known user was included in the model construction. The result supports the feasibility of the proposed algorithm for detection assistance in computer-aided palpation.
Collapse
Affiliation(s)
- Tomohiro Fukuda
- Department of Electrical and Mechanical Engineering, Graduate School of Engineering, Nagoya Institute of Technology, Nagoya, Japan
- Japan Society for the Promotion of Science, Tokyo, Japan
| | - Yoshihiro Tanaka
- Department of Electrical and Mechanical Engineering, Graduate School of Engineering, Nagoya Institute of Technology, Nagoya, Japan
| | - Michitaka Fujiwara
- Department of Gastroenterological Surgery, Graduate School of Medicine, Nagoya University, Nagoya, Japan
| | - Akihito Sano
- Department of Electrical and Mechanical Engineering, Graduate School of Engineering, Nagoya Institute of Technology, Nagoya, Japan
| |
Collapse
|
22
|
Shin J, Zhong Y, Oetomo D, Gu C. Random Weighting, Strong Tracking, and Unscented Kalman Filter for Soft Tissue Characterization. SENSORS (BASEL, SWITZERLAND) 2018; 18:E1650. [PMID: 29883430 PMCID: PMC5981475 DOI: 10.3390/s18051650] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/01/2018] [Revised: 05/17/2018] [Accepted: 05/17/2018] [Indexed: 11/25/2022]
Abstract
This paper presents a new nonlinear filtering method based on the Hunt-Crossley model for online nonlinear soft tissue characterization. This method overcomes the problem of performance degradation in the unscented Kalman filter due to contact model error. It adopts the concept of Mahalanobis distance to identify contact model error, and further incorporates a scaling factor in predicted state covariance to compensate identified model error. This scaling factor is determined according to the principle of innovation orthogonality to avoid the cumbersome computation of Jacobian matrix, where the random weighting concept is adopted to improve the estimation accuracy of innovation covariance. A master-slave robotic indentation system is developed to validate the performance of the proposed method. Simulation and experimental results as well as comparison analyses demonstrate that the efficacy of the proposed method for online characterization of soft tissue parameters in the presence of contact model error.
Collapse
Affiliation(s)
- Jaehyun Shin
- School of Engineering, RMIT University, Bundoora, VIC 3083, Australia.
| | - Yongmin Zhong
- School of Engineering, RMIT University, Bundoora, VIC 3083, Australia.
| | - Denny Oetomo
- Department of Mechanical Engineering, University of Melbourne, Parkville, VIC 3010, Australia.
| | - Chengfan Gu
- City of Whittlesea, Mill Park, VIC 3082, Australia.
| |
Collapse
|
23
|
Sorgini F, Massari L, D'Abbraccio J, Palermo E, Menciassi A, Petrovic PB, Mazzoni A, Carrozza MC, Newell FN, Oddo CM. Neuromorphic Vibrotactile Stimulation of Fingertips for Encoding Object Stiffness in Telepresence Sensory Substitution and Augmentation Applications. SENSORS (BASEL, SWITZERLAND) 2018; 18:E261. [PMID: 29342076 PMCID: PMC5795525 DOI: 10.3390/s18010261] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/16/2017] [Revised: 01/10/2018] [Accepted: 01/12/2018] [Indexed: 01/07/2023]
Abstract
We present a tactile telepresence system for real-time transmission of information about object stiffness to the human fingertips. Experimental tests were performed across two laboratories (Italy and Ireland). In the Italian laboratory, a mechatronic sensing platform indented different rubber samples. Information about rubber stiffness was converted into on-off events using a neuronal spiking model and sent to a vibrotactile glove in the Irish laboratory. Participants discriminated the variation of the stiffness of stimuli according to a two-alternative forced choice protocol. Stiffness discrimination was based on the variation of the temporal pattern of spikes generated during the indentation of the rubber samples. The results suggest that vibrotactile stimulation can effectively simulate surface stiffness when using neuronal spiking models to trigger vibrations in the haptic interface. Specifically, fractional variations of stiffness down to 0.67 were significantly discriminated with the developed neuromorphic haptic interface. This is a performance comparable, though slightly worse, to the threshold obtained in a benchmark experiment evaluating the same set of stimuli naturally with the own hand. Our paper presents a bioinspired method for delivering sensory feedback about object properties to human skin based on contingency-mimetic neuronal models, and can be useful for the design of high performance haptic devices.
Collapse
Affiliation(s)
- Francesca Sorgini
- Sant'Anna School of Advanced Studies, The BioRobotics Institute, 56025 Pisa, Italy.
| | - Luca Massari
- Sant'Anna School of Advanced Studies, The BioRobotics Institute, 56025 Pisa, Italy.
| | - Jessica D'Abbraccio
- Sant'Anna School of Advanced Studies, The BioRobotics Institute, 56025 Pisa, Italy.
| | - Eduardo Palermo
- Department of Mechanical and Aerospace Engineering, "Sapienza" University of Rome, 00185 Roma, Italy.
| | - Arianna Menciassi
- Sant'Anna School of Advanced Studies, The BioRobotics Institute, 56025 Pisa, Italy.
| | - Petar B Petrovic
- Production Engineering Department, Faculty of Mechanical Engineering, University of Belgrade, 11120 Belgrade, Serbia.
- Academy of Engineering Sciences of Serbia (AISS), 11120 Belgrade, Serbia.
| | - Alberto Mazzoni
- Sant'Anna School of Advanced Studies, The BioRobotics Institute, 56025 Pisa, Italy.
| | | | - Fiona N Newell
- School of Psychology and Institute of Neuroscience, Trinity College, 2 Dublin, Ireland.
| | - Calogero M Oddo
- Sant'Anna School of Advanced Studies, The BioRobotics Institute, 56025 Pisa, Italy.
| |
Collapse
|
24
|
Amirabdollahian F, Livatino S, Vahedi B, Gudipati R, Sheen P, Gawrie-Mohan S, Vasdev N. Prevalence of haptic feedback in robot-mediated surgery: a systematic review of literature. J Robot Surg 2017; 12:11-25. [PMID: 29196867 DOI: 10.1007/s11701-017-0763-4] [Citation(s) in RCA: 31] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2017] [Accepted: 11/07/2017] [Indexed: 01/27/2023]
Abstract
With the successful uptake and inclusion of robotic systems in minimally invasive surgery and with the increasing application of robotic surgery (RS) in numerous surgical specialities worldwide, there is now a need to develop and enhance the technology further. One such improvement is the implementation and amalgamation of haptic feedback technology into RS which will permit the operating surgeon on the console to receive haptic information on the type of tissue being operated on. The main advantage of using this is to allow the operating surgeon to feel and control the amount of force applied to different tissues during surgery thus minimising the risk of tissue damage due to both the direct and indirect effects of excessive tissue force or tension being applied during RS. We performed a two-rater systematic review to identify the latest developments and potential avenues of improving technology in the application and implementation of haptic feedback technology to the operating surgeon on the console during RS. This review provides a summary of technological enhancements in RS, considering different stages of work, from proof of concept to cadaver tissue testing, surgery in animals, and finally real implementation in surgical practice. We identify that at the time of this review, while there is a unanimous agreement regarding need for haptic and tactile feedback, there are no solutions or products available that address this need. There is a scope and need for new developments in haptic augmentation for robot-mediated surgery with the aim of improving patient care and robotic surgical technology further.
Collapse
Affiliation(s)
| | - Salvatore Livatino
- School of Engineering, University of Hertfordshire, Hatfield, AL10 9AB, UK
| | - Behrad Vahedi
- School of Engineering, University of Hertfordshire, Hatfield, AL10 9AB, UK
| | - Radhika Gudipati
- School of Computer Science, University of Hertfordshire, Hatfield, AL10 9AB, UK
| | - Patrick Sheen
- School of Engineering, University of Hertfordshire, Hatfield, AL10 9AB, UK
| | | | - Nikhil Vasdev
- Department of Urology, Hertfordshire and Bedfordshire Urological Cancer Centre, Lister Hospital, Stevenage, SG1 4AB, UK.,School of Life and Medical Sciences, University of Hertfordshire, Hatfield, Hertfordshire, AL10 9AB, UK
| |
Collapse
|
25
|
Suero EM, Hartung T, Westphal R, Hawi N, Liodakis E, Citak M, Krettek C, Stuebig T. Improving the human-robot interface for telemanipulated robotic long bone fracture reduction: Joystick device vs. haptic manipulator. Int J Med Robot 2017; 14. [PMID: 28948678 DOI: 10.1002/rcs.1863] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2017] [Revised: 08/02/2017] [Accepted: 08/22/2017] [Indexed: 01/03/2023]
Abstract
OBJECTIVES Intramedullary nailing is the treatment of choice for femoral shaft fractures. However, there are several problems associated with the technique, e.g. high radiation exposure and rotational malalignment. Experimental robotic assistance has been introduced to improve the quality of the reduction and to reduce the incidence of rotational malalignment. In the current study, we compare two devices for control of the fracture fragments during telemanipulated reduction. METHODS Ten male and ten female subjects were asked to participate as examiners in this experiment. A computer program was developed to render and manipulate CT-based renderings of femur fracture bone fragments. The user could manipulate the fragments using either a simple joystick device or a haptic manipulator. Each examiner performed telemanipulated reduction of 10 virtual fracture models of varying difficulty with each device (five in a 'training phase' and five in a 'testing phase'). Mixed models were used to test whether using the haptic device improved alignment accuracy and improved reduction times compared to using a joystick. RESULTS Reduction accuracy was not significantly different between devices in either the training phase or the testing phase (P > 0.05). Reduction time was significantly higher for the Phantom device than for the Joystick in the training phase (P < 0.0001), but it was no different in the testing phase (P = 0.865). High spatial ability with electronics had a significant effect on the alignment of fracture reduction and time to reduction. CONCLUSIONS The Joystick and the Phantom devices resulted in similarly accurate reductions, with the Joystick having an easier learning curve. The Phantom device offered no advantage over the Joystick for fracture telemanipulation. Considering the high cost of the Phantom device and the lack of a demonstrable advantage over the Joystick, its use is not justified for implementation in a fracture telemanipulation workflow. The Joystick remains as a low-cost and effective device for developing 3D fracture telemanipulation techniques.
Collapse
Affiliation(s)
- Eduardo M Suero
- Trauma Department, Hannover Medical School, Hannover, Germany
| | - Tristan Hartung
- Trauma Department, Hannover Medical School, Hannover, Germany
| | - Ralf Westphal
- Institute for Robotics and Process Control, Braunschweig University of Technology, Brunswick, Germany
| | - Nael Hawi
- Trauma Department, Hannover Medical School, Hannover, Germany
| | | | - Musa Citak
- Trauma Department, Hannover Medical School, Hannover, Germany
| | | | - Timo Stuebig
- Trauma Department, Hannover Medical School, Hannover, Germany
| |
Collapse
|
26
|
Encapsulation of Piezoelectric Transducers for Sensory Augmentation and Substitution with Wearable Haptic Devices. MICROMACHINES 2017; 8:mi8090270. [PMID: 30400460 PMCID: PMC6190144 DOI: 10.3390/mi8090270] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/30/2017] [Revised: 08/14/2017] [Accepted: 08/23/2017] [Indexed: 11/23/2022]
Abstract
The integration of polymeric actuators in haptic displays is widespread nowadays, especially in virtual reality and rehabilitation applications. However, we are still far from optimizing the transducer ability in conveying sensory information. Here, we present a vibrotactile actuator characterized by a piezoelectric disk embedded in a polydimethylsiloxane (PDMS) shell. An original encapsulation technique was performed to provide the stiff active element with a compliant cover as an interface towards the soft human skin. The interface stiffness, together with the new geometry, generated an effective transmission of vibrotactile stimulation and made the encapsulated transducer a performant component for the development of wearable tactile displays. The mechanical behavior of the developed transducer was numerically modeled as a function of the driving voltage and frequency, and the exerted normal forces were experimentally measured with a load cell. The actuator was then tested for the integration in a haptic glove in single-finger and bi-finger condition, in a 2-AFC tactile stimulus recognition test. Psychophysical results across all the tested sensory conditions confirmed that the developed integrated haptic system was effective in delivering vibrotactile information when the frequency applied to the skin is within the 200–700 Hz range and the stimulus variation is larger than 100 Hz.
Collapse
|
27
|
Li M, Konstantinova J, Xu G, He B, Aminzadeh V, Xie J, Wurdemann H, Althoefer K. Evaluation of stiffness feedback for hard nodule identification on a phantom silicone model. PLoS One 2017; 12:e0172703. [PMID: 28248996 PMCID: PMC5383005 DOI: 10.1371/journal.pone.0172703] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2016] [Accepted: 02/08/2017] [Indexed: 11/18/2022] Open
Abstract
Haptic information in robotic surgery can significantly improve clinical outcomes and help detect hard soft-tissue inclusions that indicate potential abnormalities. Visual representation of tissue stiffness information is a cost-effective technique. Meanwhile, direct force feedback, although considerably more expensive than visual representation, is an intuitive method of conveying information regarding tissue stiffness to surgeons. In this study, real-time visual stiffness feedback by sliding indentation palpation is proposed, validated, and compared with force feedback involving human subjects. In an experimental tele-manipulation environment, a dynamically updated color map depicting the stiffness of probed soft tissue is presented via a graphical interface. The force feedback is provided, aided by a master haptic device. The haptic device uses data acquired from an F/T sensor attached to the end-effector of a tele-manipulated robot. Hard nodule detection performance is evaluated for 2 modes (force feedback and visual stiffness feedback) of stiffness feedback on an artificial organ containing buried stiff nodules. From this artificial organ, a virtual-environment tissue model is generated based on sliding indentation measurements. Employing this virtual-environment tissue model, we compare the performance of human participants in distinguishing differently sized hard nodules by force feedback and visual stiffness feedback. Results indicate that the proposed distributed visual representation of tissue stiffness can be used effectively for hard nodule identification. The representation can also be used as a sufficient substitute for force feedback in tissue palpation.
Collapse
Affiliation(s)
- Min Li
- School of Mechanical Engineering, Xi’an Jiaotong University, Xi’an, Shaanxi, China
- State Key Laboratory for Manufacturing Systems Engineering, Xi’an Jiaotong University, Xi’an, Shaanxi, China
| | - Jelizaveta Konstantinova
- School of Engineering and Materials Science, Queen Mary University of London, London, United Kingdom
| | - Guanghua Xu
- School of Mechanical Engineering, Xi’an Jiaotong University, Xi’an, Shaanxi, China
- State Key Laboratory for Manufacturing Systems Engineering, Xi’an Jiaotong University, Xi’an, Shaanxi, China
| | - Bo He
- School of Mechanical Engineering, Xi’an Jiaotong University, Xi’an, Shaanxi, China
| | | | - Jun Xie
- School of Mechanical Engineering, Xi’an Jiaotong University, Xi’an, Shaanxi, China
- State Key Laboratory for Manufacturing Systems Engineering, Xi’an Jiaotong University, Xi’an, Shaanxi, China
| | - Helge Wurdemann
- Department of Mechanical Engineering, University College London, London, United Kingdom
| | - Kaspar Althoefer
- Faculty of Science & Engineering, Queen Mary University of London, London, United Kingdom
| |
Collapse
|
28
|
Xiong L, Chng CB, Chui CK, Yu P, Li Y. Shared control of a medical robot with haptic guidance. Int J Comput Assist Radiol Surg 2016; 12:137-147. [PMID: 27314590 DOI: 10.1007/s11548-016-1425-0] [Citation(s) in RCA: 21] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2015] [Accepted: 05/21/2016] [Indexed: 11/30/2022]
Abstract
PURPOSE Tele-operation of robotic surgery reduces the radiation exposure during the interventional radiological operations. However, endoscope vision without force feedback on the surgical tool increases the difficulty for precise manipulation and the risk of tissue damage. The shared control of vision and force provides a novel approach of enhanced control with haptic guidance, which could lead to subtle dexterity and better maneuvrability during MIS surgery. METHODS The paper provides an innovative shared control method for robotic minimally invasive surgery system, in which vision and haptic feedback are incorporated to provide guidance cues to the clinician during surgery. The incremental potential field (IPF) method is utilized to generate a guidance path based on the anatomy of tissue and surgical tool interaction. Haptic guidance is provided at the master end to assist the clinician during tele-operative surgical robotic task. RESULTS The approach has been validated with path following and virtual tumor targeting experiments. The experiment results demonstrate that comparing with vision only guidance, the shared control with vision and haptics improved the accuracy and efficiency of surgical robotic manipulation, where the tool-position error distance and execution time are reduced. CONCLUSIONS The validation experiment demonstrates that the shared control approach could help the surgical robot system provide stable assistance and precise performance to execute the designated surgical task. The methodology could also be implemented with other surgical robot with different surgical tools and applications.
Collapse
Affiliation(s)
- Linfei Xiong
- Center of Robotics and Intelligent Machine, Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Science, No. 266 Fangzhen Road, Beibei District, Chongqing, 400714, China.
| | - Chin Boon Chng
- Mechanical Engineering, National University of Singapore, Singapore, Singapore
| | - Chee Kong Chui
- Mechanical Engineering, National University of Singapore, Singapore, Singapore. .,Control and Mechatronics Laboratory, National University of Singapore, EA-04-06, 9 Engineering Drive 1, 117575, Singapore, Singapore.
| | - Peiwu Yu
- Department of General Surgery, Southwest Hospital, Chongqing, China
| | - Yao Li
- Center of Robotics and Intelligent Machine, Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Science, No. 266 Fangzhen Road, Beibei District, Chongqing, 400714, China
| |
Collapse
|
29
|
Enayati N, De Momi E, Ferrigno G. Haptics in Robot-Assisted Surgery: Challenges and Benefits. IEEE Rev Biomed Eng 2016; 9:49-65. [DOI: 10.1109/rbme.2016.2538080] [Citation(s) in RCA: 130] [Impact Index Per Article: 16.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
30
|
Helman SN, Schwedhelm T, Kadakia S, Wang Y, Schiff BA, Smith RV. Transoral Robotic Surgery in Oropharyngeal Carcinoma. Arch Pathol Lab Med 2015; 139:1389-97. [DOI: 10.5858/arpa.2014-0573-ra] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Context
The incidence of oropharyngeal squamous cell carcinoma has increased during the past decade and is related primarily to the human papillomavirus. This change in etiology, from tobacco and alcohol to human papillomavirus, has resulted in improved survival for the disease. In the United States, open resection had largely been replaced by concurrent chemotherapy and/or radiotherapy by the early 2000s. The advent of transoral surgery has led to an increase in surgery as the primary treatment for both early- and advanced-stage oropharyngeal squamous cell carcinoma because it has potential advantages over open surgery and nonsurgical modalities.
Objective
To provide an overview of transoral robotic surgery for oropharyngeal squamous cell carcinoma and contrast it with other surgical and nonsurgical modalities.
Data Sources
Articles from 2000 to 2014 were accessioned on PubMed and reviewed for utility by the primary authors.
Conclusions
Transoral surgery has become more commonly used as a minimally invasive approach to treat oropharyngeal tumors. Other strategies, including radiation, chemotherapy with radiation, and open surgery, are still important treatment approaches. The treatment options for an individual patient rely on multiple factors, including the tumor location and size, features of the tumor, and patient comorbidities. The continued study of these techniques is important to match the patient with the most appropriate treatment.
Collapse
Affiliation(s)
| | | | | | | | | | - Richard V. Smith
- From the Department of Otolaryngology/Head and Neck Surgery, New York Eye and Ear Infirmary–Mount Sinai Health System, New York, New York (Drs Helman and Kadakia); Albert Einstein College of Medicine, Bronx, New York (Mr Schwedhelm); and the Departments of Pathology (Dr Wang) and Otorhinolaryngology–Head and Neck Surgery (Drs Schiff and Smith), Montefiore Medical Center, Albert Einstein College o
| |
Collapse
|
31
|
Li M, Konstantinova J, Secco EL, Jiang A, Liu H, Nanayakkara T, Seneviratne LD, Dasgupta P, Althoefer K, Wurdemann HA. Using visual cues to enhance haptic feedback for palpation on virtual model of soft tissue. Med Biol Eng Comput 2015; 53:1177-86. [PMID: 26018755 DOI: 10.1007/s11517-015-1309-4] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2014] [Accepted: 05/04/2015] [Indexed: 10/23/2022]
Abstract
This paper explores methods that make use of visual cues aimed at generating actual haptic sensation to the user, namely pseudo-haptics. We propose a new pseudo-haptic feedback-based method capable of conveying 3D haptic information and combining visual haptics with force feedback to enhance the user's haptic experience. We focused on an application related to tumor identification during palpation and evaluated the proposed method in an experimental study where users interacted with a haptic device and graphical interface while exploring a virtual model of soft tissue, which represented stiffness distribution of a silicone phantom tissue with embedded hard inclusions. The performance of hard inclusion detection using force feedback only, pseudo-haptic feedback only, and the combination of the two feedbacks was compared with the direct hand touch. The combination method and direct hand touch had no significant difference in the detection results. Compared with the force feedback alone, our method increased the sensitivity by 5%, the positive predictive value by 4%, and decreased detection time by 48.7%. The proposed methodology has great potential for robot-assisted minimally invasive surgery and in all applications where remote haptic feedback is needed.
Collapse
Affiliation(s)
- Min Li
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, 710049, Shaanxi, China.
| | | | - Emanuele L Secco
- Department of Informatics, Kings College London, London, WC2R 2LS, UK.,Department of Mathematics and Computer Science, Hope University, Liverpool, UK
| | - Allen Jiang
- Department of Informatics, Kings College London, London, WC2R 2LS, UK
| | - Hongbin Liu
- Department of Informatics, Kings College London, London, WC2R 2LS, UK
| | | | - Lakmal D Seneviratne
- Department of Informatics, Kings College London, London, WC2R 2LS, UK.,College of Engineering, Khalifa University of Science, Technology and Research, Abu Dhabi, UAE
| | - Prokar Dasgupta
- Medical Research Council (MRC) Centre for Transplantation, King's College London, Kings Health Partners, Guys Hospital, London, SE1 9RT, UK
| | - Kaspar Althoefer
- Department of Informatics, Kings College London, London, WC2R 2LS, UK
| | - Helge A Wurdemann
- Department of Informatics, Kings College London, London, WC2R 2LS, UK
| |
Collapse
|
32
|
Nia Kosari S, Rydén F, Lendvay TS, Hannaford B, Chizeck HJ. Forbidden region virtual fixtures from streaming point clouds. Adv Robot 2014. [DOI: 10.1080/01691864.2014.962613] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/29/2022]
|
33
|
Image Guidance for All—TilePro Display of 3-Dimensionally Reconstructed Images in Robotic Partial Nephrectomy. Urology 2014; 84:237-42. [DOI: 10.1016/j.urology.2014.02.051] [Citation(s) in RCA: 31] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2013] [Revised: 01/02/2014] [Accepted: 02/08/2014] [Indexed: 11/23/2022]
|
34
|
|
35
|
Johnson PJ, Schmidt DE, Duvvuri U. Output control of da Vinci surgical system's surgical graspers. J Surg Res 2014; 186:56-62. [DOI: 10.1016/j.jss.2013.07.032] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2013] [Revised: 06/29/2013] [Accepted: 07/16/2013] [Indexed: 10/26/2022]
Affiliation(s)
- Paul J Johnson
- University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania.
| | | | | |
Collapse
|
36
|
Buchs NC, Volonte F, Pugin F, Toso C, Fusaglia M, Gavaghan K, Majno PE, Peterhans M, Weber S, Morel P. Augmented environments for the targeting of hepatic lesions during image-guided robotic liver surgery. J Surg Res 2013; 184:825-31. [PMID: 23684617 DOI: 10.1016/j.jss.2013.04.032] [Citation(s) in RCA: 41] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2013] [Revised: 04/08/2013] [Accepted: 04/17/2013] [Indexed: 12/19/2022]
Abstract
BACKGROUND Stereotactic navigation technology can enhance guidance during surgery and enable the precise reproduction of planned surgical strategies. Currently, specific systems (such as the CAS-One system) are available for instrument guidance in open liver surgery. This study aims to evaluate the implementation of such a system for the targeting of hepatic tumors during robotic liver surgery. MATERIAL AND METHODS Optical tracking references were attached to one of the robotic instruments and to the robotic endoscopic camera. After instrument and video calibration and patient-to-image registration, a virtual model of the tracked instrument and the available three-dimensional images of the liver were displayed directly within the robotic console, superimposed onto the endoscopic video image. An additional superimposed targeting viewer allowed for the visualization of the target tumor, relative to the tip of the instrument, for an assessment of the distance between the tumor and the tool for the realization of safe resection margins. RESULTS Two cirrhotic patients underwent robotic navigated atypical hepatic resections for hepatocellular carcinoma. The augmented endoscopic view allowed for the definition of an accurate resection margin around the tumor. The overlay of reconstructed three-dimensional models was also used during parenchymal transection for the identification of vascular and biliary structures. Operative times were 240 min in the first case and 300 min in the second. There were no intraoperative complications. CONCLUSIONS The da Vinci Surgical System provided an excellent platform for image-guided liver surgery with a stable optic and instrumentation. Robotic image guidance might improve the surgeon's orientation during the operation and increase accuracy in tumor resection. Further developments of this technological combination are needed to deal with organ deformation during surgery.
Collapse
Affiliation(s)
- Nicolas C Buchs
- Clinic for Visceral and Transplantation Surgery, Department of Surgery, University Hospital of Geneva, Geneva, Switzerland.
| | | | | | | | | | | | | | | | | | | |
Collapse
|