1
|
Ma CW, Cheng PS, Chan YS, Tipoe GL. Virtual reality: a technology to promote active learning of physiology for students across multiple disciplines. ADVANCES IN PHYSIOLOGY EDUCATION 2023; 47:594-603. [PMID: 37382503 DOI: 10.1152/advan.00172.2022] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/19/2022] [Revised: 06/27/2023] [Accepted: 06/28/2023] [Indexed: 06/30/2023]
Abstract
The usefulness of virtual reality (VR) technology in physiology education is largely unexplored. Although VR has the potential to enrich learning experience by enhancing the spatial awareness of students, it is unclear whether VR contributes to active learning of physiology. In the present study, we used a mixed-method research approach to investigate students' perceptions of physiology learning based on VR simulations. Quantitative and qualitative data indicate that the implementation of VR learning environments improves the quality of physiology education by promoting active learning in terms of interactive engagement, interest, problem-solving skills, and feedback. In the Technology-Enabled Active Learning Inventory, which consisted of 20 questions to which students responded along a 7-point Likert scale, the majority of students agreed that VR learning of physiology not only stimulated their curiosity (77%; P < 0.001) but also allowed them to obtain knowledge through diverse formats (76%; P < 0.001), participate in thought-provoking dialogue (72%; P < 0.001), and interact better with peers (72%; P < 0.001). Positive responses in the social, cognitive, behavioral, and evaluative domains of active learning were received from students across different disciplines, including medicine, Chinese medicine, biomedical sciences, and biomedical engineering. Their written feedback showed that VR enhanced their interest in physiology and facilitated the visualization of physiological processes to improve their learning. Overall, this study supports that the integration of VR technology into physiology courses can be an effective teaching strategy.NEW & NOTEWORTHY Virtual reality (VR) improves physiology education by promoting active learning in terms of interactive engagement, interest, problem-solving skills, and feedback. Positive responses toward multiple components of active learning were received from students across different disciplines. The majority of students agreed that VR learning of physiology not only stimulated their curiosity but also allowed them to obtain knowledge through diverse formats, participate in thought-provoking dialogue, and interact better with peers.
Collapse
Affiliation(s)
- Chun-Wai Ma
- School of Biomedical Sciences, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong, China
| | - Pui-Sum Cheng
- School of Biomedical Sciences, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong, China
| | - Ying-Shing Chan
- School of Biomedical Sciences, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong, China
| | - George Lim Tipoe
- School of Biomedical Sciences, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong, China
| |
Collapse
|
2
|
Gasteiger N, van der Veer SN, Wilson P, Dowding D. How, for Whom, and in Which Contexts or Conditions Augmented and Virtual Reality Training Works in Upskilling Health Care Workers: Realist Synthesis. JMIR Serious Games 2022; 10:e31644. [PMID: 35156931 PMCID: PMC8893595 DOI: 10.2196/31644] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2021] [Revised: 09/16/2021] [Accepted: 10/12/2021] [Indexed: 01/20/2023] Open
Abstract
Background Using traditional simulators (eg, cadavers, animals, or actors) to upskill health workers is becoming less common because of ethical issues, commitment to patient safety, and cost and resource restrictions. Virtual reality (VR) and augmented reality (AR) may help to overcome these barriers. However, their effectiveness is often contested and poorly understood and warrants further investigation. Objective The aim of this review is to develop, test, and refine an evidence-informed program theory on how, for whom, and to what extent training using AR or VR works for upskilling health care workers and to understand what facilitates or constrains their implementation and maintenance. Methods We conducted a realist synthesis using the following 3-step process: theory elicitation, theory testing, and theory refinement. We first searched 7 databases and 11 practitioner journals for literature on AR or VR used to train health care staff. In total, 80 papers were identified, and information regarding context-mechanism-outcome (CMO) was extracted. We conducted a narrative synthesis to form an initial program theory comprising of CMO configurations. To refine and test this theory, we identified empirical studies through a second search of the same databases used in the first search. We used the Mixed Methods Appraisal Tool to assess the quality of the studies and to determine our confidence in each CMO configuration. Results Of the 41 CMO configurations identified, we had moderate to high confidence in 9 (22%) based on 46 empirical studies reporting on VR, AR, or mixed simulation training programs. These stated that realistic (high-fidelity) simulations trigger perceptions of realism, easier visualization of patient anatomy, and an interactive experience, which result in increased learner satisfaction and more effective learning. Immersive VR or AR engages learners in deep immersion and improves learning and skill performance. When transferable skills and knowledge are taught using VR or AR, skills are enhanced and practiced in a safe environment, leading to knowledge and skill transfer to clinical practice. Finally, for novices, VR or AR enables repeated practice, resulting in technical proficiency, skill acquisition, and improved performance. The most common barriers to implementation were up-front costs, negative attitudes and experiences (ie, cybersickness), developmental and logistical considerations, and the complexity of creating a curriculum. Facilitating factors included decreasing costs through commercialization, increasing the cost-effectiveness of training, a cultural shift toward acceptance, access to training, and leadership and collaboration. Conclusions Technical and nontechnical skills training programs using AR or VR for health care staff may trigger perceptions of realism and deep immersion and enable easier visualization, interactivity, enhanced skills, and repeated practice in a safe environment. This may improve skills and increase learning, knowledge, and learner satisfaction. The future testing of these mechanisms using hypothesis-driven approaches is required. Research is also required to explore implementation considerations.
Collapse
Affiliation(s)
- Norina Gasteiger
- Division of Nursing, Midwifery and Social Work, University of Manchester, Manchester, United Kingdom.,Centre for Health Informatics, Division of Informatics, Imaging and Data Sciences, University of Manchester, Manchester, United Kingdom.,Division of Population Health, Health Services Research and Primary Care, University of Manchester, Manchester, United Kingdom
| | - Sabine N van der Veer
- Centre for Health Informatics, Division of Informatics, Imaging and Data Sciences, University of Manchester, Manchester, United Kingdom.,Manchester Academic Health Science Centre, University of Manchester, Manchester, United Kingdom
| | - Paul Wilson
- Division of Population Health, Health Services Research and Primary Care, University of Manchester, Manchester, United Kingdom
| | - Dawn Dowding
- Division of Nursing, Midwifery and Social Work, University of Manchester, Manchester, United Kingdom
| |
Collapse
|
3
|
Korzeniowski P, Chacon CS, Russell VR, Clarke SA, Bello F. Virtual Reality Simulator for Pediatric Laparoscopic Inguinal Hernia Repair. J Laparoendosc Adv Surg Tech A 2021; 31:1322-1330. [PMID: 34314635 DOI: 10.1089/lap.2020.0423] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/22/2023] Open
Abstract
Purpose: Due to the small space environment, the learning curve of pediatric laparoscopic procedures is steep and requires excellent procedural skills. These are mainly gained through an apprenticeship on real patients. Computer-based virtual reality (VR) simulators offer a safe, cost-effective, and configurable training environment free from ethical and patient safety issues. Materials and Methods: We have developed a prototype VR simulator for core manual skills training for pediatric laparoscopic hernia repair. The simulator currently consists of a hernia suturing task on a virtual nonanatomic trainer at a real pediatric scale. Results: A simulation realism validation study was carried out by obtaining subjective feedback (face and content validity) through a questionnaire from 36 pediatric surgeons. The overall simulation realism was on average marked 3.08 on a 5-point Likert scale (1: "very unrealistic" and 5: "very realistic"). The participants were most satisfied with the visual realism (3.33) and most critical about the behavior of virtual tissue. The simulator showed good content validity; its usefulness as a training tool for hernia repair, suturing in general, and in learning fundamental laparoscopic skills was marked 3.61, 3.64, and 3.89, respectively. Conclusions: VR simulation of pediatric laparoscopic procedures can contribute to surgical training and improve the educational experience without putting our youngest patients at risk. This simulator is a first prototype, and the initial results indicate that it provides promising foundations for further development. More formal and larger studies such as construct validity and transfer of skills are envisaged as the prototype is developed further.
Collapse
Affiliation(s)
- Przemyslaw Korzeniowski
- Department of Surgery and Cancer, Centre for Engagement and Simulation Science, Imperial College London, London, United Kingdom
| | - Carmen S Chacon
- Department of Pediatric Surgery, Chelsea Children's Hospital, Chelsea, London, United Kingdom
| | - Victoria R Russell
- Department of Surgery and Cancer, Centre for Engagement and Simulation Science, Imperial College London, London, United Kingdom
| | - Simon A Clarke
- Department of Pediatric Surgery, Chelsea Children's Hospital, Chelsea, London, United Kingdom
| | - Fernando Bello
- Department of Surgery and Cancer, Centre for Engagement and Simulation Science, Imperial College London, London, United Kingdom
| |
Collapse
|
4
|
Evaluation of Laparoscopy Virtual Reality Training on the Improvement of Trainees' Surgical Skills. ACTA ACUST UNITED AC 2021; 57:medicina57020130. [PMID: 33540817 PMCID: PMC7913105 DOI: 10.3390/medicina57020130] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2020] [Revised: 01/22/2021] [Accepted: 01/26/2021] [Indexed: 12/23/2022]
Abstract
Background and objectives: The primary objective was to evaluate the benefit of training with virtual reality simulation. The secondary objective was to describe the short-term skill acquisition obtained by simulation training and to determine the factors affecting its magnitude. Materials and Methods: We prospectively performed a three-stage evaluation: face, constructive, and predictive to evaluate the training with a laparoscopic simulator with haptic feedback. The participants (n = 63) were divided according to their level of experience into three groups: 16% residents; 46% specialists and 38% were consultants. Results: Face evaluation demonstrates the acceptance of the design and realism of the tasks; it showed a median score of eight (IQR 3) on a Likert scale and 54% of participants (n = 34) gave the tissue feedback a moderate rating. Constructive evaluation demonstrates the improvement of the participants in the training session and the ability of the designed task to distinguish the experienced from the inexperienced surgeon based on the performance score, at task I (transfer of pegs) and II (laparoscopic salpingectomy). There was an improvement in both tasks with a significant increase in score and reduction in time. The study showed that those with a high score at the pre-test recorded a high score post-test, showing a significant pair-wise comparison (Z) and correlation (p) showing a significant statistical significance (p < 0.001). The predictive evaluation demonstrates the beneficiary effect of training four weeks afterward on the practice of surgeons addressed with five questions. It showed an improvement regarding implementation into daily routine, performance of procedure, suturing, shortening of the operative time, and complication management. Conclusions: Virtual reality simulation established high ratings for both realism and training capacity, including clinical relevance, critical relevance, and maintaining training enthusiasm.
Collapse
|
5
|
Roy R, Warren E, Xu Y, Yow C, Madhurapantula RS, Orgel JPRO, Lister K. Functional Grading of a Transversely Isotropic Hyperelastic Model with Applications in Modeling Tricuspid and Mitral Valve Transition Regions. Int J Mol Sci 2020; 21:ijms21186503. [PMID: 32899559 PMCID: PMC7554844 DOI: 10.3390/ijms21186503] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2020] [Revised: 08/27/2020] [Accepted: 09/02/2020] [Indexed: 01/20/2023] Open
Abstract
Surgical simulators and injury-prediction human models require a combination of representative tissue geometry and accurate tissue material properties to predict realistic tool-tissue interaction forces and injury mechanisms, respectively. While biological tissues have been individually characterized, the transition regions between tissues have received limited research attention, potentially resulting in inaccuracies within simulations. In this work, an approach to characterize the transition regions in transversely isotropic (TI) soft tissues using functionally graded material (FGM) modeling is presented. The effect of nonlinearities and multi-regime nature of the TI model on the functional grading process is discussed. The proposed approach has been implemented to characterize the transition regions in the leaflet (LL), chordae tendinae (CT) and the papillary muscle (PM) of porcine tricuspid valve (TV) and mitral valve (MV). The FGM model is informed using high resolution morphological measurements of the collagen fiber orientation and tissue composition in the transition regions, and deformation characteristics predicted by the FGM model are numerically validated to experimental data using X-ray diffraction imaging. The results indicate feasibility of using the FGM approach in modeling soft-tissue transitions and has implications in improving physical representation of tissue deformation throughout the body using a scalable version of the proposed approach.
Collapse
Affiliation(s)
- Rajarshi Roy
- Corvid Technologies, Mooresville, NC 28117, USA; (Y.X.); (C.Y.); (K.L.)
- Correspondence: ; Tel.: +1-704-799-6944
| | | | - Yaoyao Xu
- Corvid Technologies, Mooresville, NC 28117, USA; (Y.X.); (C.Y.); (K.L.)
| | - Caleb Yow
- Corvid Technologies, Mooresville, NC 28117, USA; (Y.X.); (C.Y.); (K.L.)
| | - Rama S. Madhurapantula
- Department of Biology, Illinois Institute of Technology, Chicago, IL 60616, USA; (R.S.M.); (J.P.R.O.O.)
- Pritzker Institute of Biomedical Science and Engineering, Illinois Institute of Technology, Chicago, IL 60616, USA
| | - Joseph P. R. O. Orgel
- Department of Biology, Illinois Institute of Technology, Chicago, IL 60616, USA; (R.S.M.); (J.P.R.O.O.)
- Pritzker Institute of Biomedical Science and Engineering, Illinois Institute of Technology, Chicago, IL 60616, USA
- Department of Biomedical Engineering, Illinois Institute of Technology, Chicago, IL 60616, USA
| | - Kevin Lister
- Corvid Technologies, Mooresville, NC 28117, USA; (Y.X.); (C.Y.); (K.L.)
| |
Collapse
|
6
|
Elsayed M, Kadom N, Ghobadi C, Strauss B, Al Dandan O, Aggarwal A, Anzai Y, Griffith B, Lazarow F, Straus CM, Safdar NM. Virtual and augmented reality: potential applications in radiology. Acta Radiol 2020; 61:1258-1265. [PMID: 31928346 DOI: 10.1177/0284185119897362] [Citation(s) in RCA: 26] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
The modern-day radiologist must be adept at image interpretation, and the one who most successfully leverages new technologies may provide the highest value to patients, clinicians, and trainees. Applications of virtual reality (VR) and augmented reality (AR) have the potential to revolutionize how imaging information is applied in clinical practice and how radiologists practice. This review provides an overview of VR and AR, highlights current applications, future developments, and limitations hindering adoption.
Collapse
Affiliation(s)
- Mohammad Elsayed
- Department of Radiology and Imaging Sciences, Emory University School of Medicine, Atlanta, GA, USA
| | - Nadja Kadom
- Department of Radiology and Imaging Sciences, Emory University School of Medicine, Atlanta, GA, USA
| | - Comeron Ghobadi
- Department of Radiology, The University of Chicago Pritzker School of Medicine, IL, USA
| | - Benjamin Strauss
- Department of Radiology, The University of Chicago Pritzker School of Medicine, IL, USA
| | - Omran Al Dandan
- Department of Radiology, Imam Abdulrahman Bin Faisal University College of Medicine, Dammam, Eastern Province, Saudi Arabia
| | - Abhimanyu Aggarwal
- Department of Radiology, Eastern Virginia Medical School, Norfolk, VA, USA
| | - Yoshimi Anzai
- Department of Radiology and Imaging Sciences, University of Utah School of Medicine, Salt Lake City, Utah, USA
| | - Brent Griffith
- Department of Radiology, Henry Ford Health System, Detroit, MI, USA
| | - Frances Lazarow
- Department of Radiology, Eastern Virginia Medical School, Norfolk, VA, USA
| | - Christopher M Straus
- Department of Radiology, The University of Chicago Pritzker School of Medicine, IL, USA
| | - Nabile M Safdar
- Department of Radiology and Imaging Sciences, Emory University School of Medicine, Atlanta, GA, USA
| |
Collapse
|
7
|
Lohre R, Bois AJ, Athwal GS, Goel DP. Improved Complex Skill Acquisition by Immersive Virtual Reality Training: A Randomized Controlled Trial. J Bone Joint Surg Am 2020; 102:e26. [PMID: 31972694 DOI: 10.2106/jbjs.19.00982] [Citation(s) in RCA: 58] [Impact Index Per Article: 14.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/01/2023]
Abstract
BACKGROUND There has been limited literature on immersive virtual reality (VR) simulation in orthopaedic education. The purpose of this multicenter, blinded, randomized controlled trial was to determine the validity and efficacy of immersive VR training in orthopaedic resident education. METHODS Nineteen senior orthopaedic residents (resident group) and 7 consultant shoulder arthroplasty surgeons (expert group) participated in the trial comparing immersive VR with traditional learning using a technical journal article as a control. The examined task focused on achieving optimal glenoid exposure. Participants completed demographic questionnaires, knowledge tests, and a glenoid exposure on fresh-frozen cadavers while being examined by blinded shoulder arthroplasty surgeons. Training superiority was determined by the outcome measures of the Objective Structured Assessment of Technical Skills (OSATS) score, a developed laboratory metric, verbal answers, and time to task completion. RESULTS Immersive VR had greater realism and was superior in teaching glenoid exposure than the control (p = 0.01). The expert group outperformed the resident group on knowledge testing (p = 0.04). The immersive VR group completed the learning activity and knowledge tests significantly faster (p < 0.001) at a mean time (and standard deviation) of 11 ± 3 minutes than the control group at 20 ± 4 minutes, performing 3 to 5 VR repeats for a reduction in learning time of 570%. The immersive VR group completed the glenoid exposure significantly faster (p = 0.04) at a mean time of 14 ± 7 minutes than the control group at 21 ± 6 minutes, with superior OSATS instrument handling scores (p = 0.03). The immersive VR group scored equivalently in surprise verbal scores (p = 0.85) and written knowledge scores (p = 1.0). CONCLUSIONS Immersive VR demonstrated substantially improved translational technical and nontechnical skills acquisition over traditional learning in senior orthopaedic residents. Additionally, the results demonstrate the face, content, construct, and transfer validity for immersive VR. CLINICAL RELEVANCE This adequately powered, randomized controlled trial demonstrated how an immersive VR system can efficiently (570%) teach a complex surgical procedure and also demonstrate improved translational skill and knowledge acquisition when compared with a traditional learning method.
Collapse
Affiliation(s)
- Ryan Lohre
- Department of Orthopaedics, University of British Columbia, Vancouver, British Columbia, Canada
| | - Aaron J Bois
- Section of Orthopaedic Surgery, Department of Surgery, University of Calgary, Calgary, Alberta, Canada
| | - George S Athwal
- Roth McFarlane Hand and Upper Limb Center, Schulich School of Medicine and Dentistry, Western University, London, Ontario, Canada
| | - Danny P Goel
- Department of Orthopaedics, University of British Columbia, Vancouver, British Columbia, Canada
| | | |
Collapse
|
8
|
Novotny J, Miller WR, Luks FI, Merck D, Collins S, Laidlaw DH. Towards Placental Surface Vasculature Exploration in Virtual Reality. IEEE COMPUTER GRAPHICS AND APPLICATIONS 2020; 40:28-39. [PMID: 30582530 DOI: 10.1109/mcg.2018.2881985] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
We present a case study evaluating the potential for interactively identifying placental surface blood vessels using magnetic resonance imaging (MRI) scans in virtual reality (VR) environments. We visualized the MRI data using direct volume rendering in a high-fidelity CAVE-like VR system, allowing medical professionals to identify relevant placental vessels directly from volume visualizations in the VR system, without prior vessel segmentation. Participants were able to trace most of the observable vascular structure, and consistently identified blood vessels down to diameters of 1 mm, an important requirement in diagnosing vascular diseases. Qualitative feedback from our participants suggests that our VR visualization is easy to understand and allows intuitive data exploration, but complex user interactions remained a challenge. Using these observations, we discuss implications and requirements for spatial tracing user interaction methods in VR environments. We believe that VR MRI visualizations are the next step towards effective surgery planning for prenatal diseases.
Collapse
|
9
|
Chiu HY, Kang YN, Wang WL, Chen CC, Hsu W, Tseng MF, Wei PL. The Role of Active Engagement of Peer Observation in the Acquisition of Surgical Skills in Virtual Reality Tasks for Novices. JOURNAL OF SURGICAL EDUCATION 2019; 76:1655-1662. [PMID: 31130508 DOI: 10.1016/j.jsurg.2019.05.004] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/26/2019] [Revised: 04/05/2019] [Accepted: 05/11/2019] [Indexed: 05/26/2023]
Abstract
OBJECTIVE Peer-assisted learning has been regarded as an adjunct to teaching modalities. It remains inconclusive regarding the benefits of peer observation in skills learning. Hence, we investigated whether the active engagement (AE) of peer observation in addition to expert demonstration would facilitate the performance in the virtual reality (VR) tasks. SETTING/DESIGN The programs involved 4 VR tasks including basic (camera targeting), intermediate (energy dissection and energy switching), and advanced (suture sponge) tasks in the da Vinci Skills Simulators, which were set up in the operating room at Taipei Medical University Hospital. Fifty medical students participated in the study. The AE of the participants was defined as the total number of peer observations in addition to expert observation before their performance. We assessed the correlations between AE and surgical task performance using Pearson correlation and the concept of learning analytics. PARTICIPANTS Medical students (sixth-year students in Taiwan, equivalent to fourth-year students in the US system) from Taipei Medical University were recruited. RESULTS AE was correlated with the energy dissection task (r = 0.329, p = 0.02) and marginally associated with the energy switching task (r = 0.271, p = 0.057). However, AE was not correlated with either task scores for camera targeting (r = 0.096, p = 0.509) or task scores for suture sponge (r = -0.091, p = 0.529). CONCLUSIONS Our findings suggest that AE of peer observation may facilitate learning energy dissection task, which is an intermediate-level task, but not in other basic or advanced tasks in a VR context. The study highlights the potential effect of AE of peer observation on surgical learning based on a distinct level of tasks. Tasks that fit the learners' level are recommended. Nevertheless, the effectiveness of peer observation on surgical training still has to be explored to ensure favorable results and optimal learning outcomes.
Collapse
Affiliation(s)
- Hsin-Yi Chiu
- Division of Thoracic Surgery, Department of Surgery, Taipei Medical University Hospital, Taipei, Taiwan; Department of Medical Education, Taipei Medical University Hospital, Taipei, Taiwan; Department of Education and Humanities in Medicine, School of Medicine, Taipei Medical University, Taipei, Taiwan; Department of Surgery, School of Medicine, Taipei Medical University, Taipei, Taiwan; Department of Animal Science and Technology, National Taiwan University, Taipei, Taiwan
| | - Yi-No Kang
- Department of Medical Education, Taipei Medical University Hospital, Taipei, Taiwan; Department of Education and Humanities in Medicine, School of Medicine, Taipei Medical University, Taipei, Taiwan
| | - Wei-Lin Wang
- Division of Acute Care Surgery and Traumatology, Department of Surgery, Taipei Medical University Hospital, Taipei, Taiwan
| | - Chia-Che Chen
- Division of Acute Care Surgery and Traumatology, Department of Surgery, Taipei Medical University Hospital, Taipei, Taiwan; Division of Colorectal Surgery, Department of Surgery, Taipei Medical University Hospital, Taipei Medical University, Taipei, Taiwan
| | - Wayne Hsu
- Division of Acute Care Surgery and Traumatology, Department of Surgery, Taipei Medical University Hospital, Taipei, Taiwan
| | - Mei-Feng Tseng
- Center for General Education, School of Medicine, College of Medicine, Taipei Medical University, Taipei, Taiwan.
| | - Po-Li Wei
- Department of Surgery, School of Medicine, Taipei Medical University, Taipei, Taiwan; Division of Colorectal Surgery, Department of Surgery, Taipei Medical University Hospital, Taipei Medical University, Taipei, Taiwan; Department of Medical Research, Cancer Research Center and Translational Laboratory, Taipei Medical University Hospital, Taipei Medical University, Taipei, Taiwan; Graduate Institute of Cancer Biology and Drug Discovery, Taipei Medical University, Taipei, Taiwan.
| |
Collapse
|
10
|
Bisht B, Hope A, Paul MK. From papyrus leaves to bioprinting and virtual reality: history and innovation in anatomy. Anat Cell Biol 2019; 52:226-235. [PMID: 31598350 PMCID: PMC6773896 DOI: 10.5115/acb.18.213] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2018] [Revised: 03/26/2019] [Accepted: 03/28/2019] [Indexed: 12/03/2022] Open
Abstract
The human quest to master the anatomy and physiology of living systems started as early as 1600 BC, with documents from the Greeks, Indians, and Romans presenting the earliest systematic studies and advances. Following the fall of the Roman Empire, the progress slowed until the Renaissance renewed scientific interest in anatomy and physiology, ushering in an era of spectacular advances. Alongside the discoveries of modern science, innovations in media such as printing, photography and color reproduction, improved the accuracy of communicating science. Techniques for noninvasively viewing the human body, such as magnetic resonance imaging, opened up new ways of exploring and understanding anatomy, physiology, and disease pathogenesis. Advances in three-dimensional (3D)-technologies, including computer graphics and animation are directly linked to many advances in medicine and surgery. Anatomy education has come a long way from papyrus leaf inscriptions to computerized 3D modeling, holographic representation, and virtual reality-based software. The future presents unlimited options for studying and understanding anatomy as Google glasses, bioprinting, virtual reality, and allied technologies transform the world into a classroom. This review summarizes the journey of mankind to master anatomy and physiology.
Collapse
Affiliation(s)
- Bharti Bisht
- Division of Thoracic Surgery, David Geffen School of Medicine, UCLA, Los Angeles, CA, USA
| | - Ashley Hope
- Department of Molecular, Cell, and Developmental Biology, UCLA, Los Angeles, CA, USA
| | - Manash K Paul
- Division of Pulmonary and Critical Care Medicine, David Geffen School of Medicine, UCLA, Los Angeles, CA, USA
| |
Collapse
|
11
|
Topalli D, Cagiltay NE. Classification of Intermediate and Novice Surgeons' Skill Assessment Through Performance Metrics. Surg Innov 2019; 26:621-629. [PMID: 31167613 DOI: 10.1177/1553350619853112] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Background. Endoscopic surgeries have become an alternative for open procedures whenever possible. For such types of operations, surgeons are required to gain several skills, whose development needs hands-on practice. Accordingly, gaining these skills today is a challenge for surgical education programs. Despite the development of several technology-enhanced training environments, there are still problems to better integrate these technologies into educational programs. For an appropriate integration, it is critical to assess the skill levels and adapt the training content according to the trainees' requirements. In the literature, there exist several methods for assessing these skill levels. However, there are still problems in practice for objective and repetitive assessment. Methods. The present study aims to estimate the skill levels of participants in surgical training programs in an objective manner by collecting experimental data from residents in an endoscopic surgical simulation environment and gathering performance metrics. Results. It is shown that, by comparing the results of a number of classification algorithms for the best accuracy estimation and feature set, the "novice" and "intermediate" skill levels can be estimated with an accuracy of 86%. Conclusions. The outcomes help surgical educators and instructional system designers to better assess the skill levels of the trainees and guide them accordingly. In addition, objective assessments as highlighted in this study can be beneficial when designing technology-enhanced adaptive learning environments.
Collapse
|
12
|
Rossler KL, Sankaranarayanan G, Duvall A. Acquisition of Fire Safety Knowledge and Skills With Virtual Reality Simulation. Nurse Educ 2019; 44:88-92. [PMID: 29847356 PMCID: PMC6252293 DOI: 10.1097/nne.0000000000000551] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
BACKGROUND Prelicensure nursing students seeking to enter perioperative nursing need preparatory fire safety knowledge and skills training to participate as a member of an operating room (OR) team. PURPOSE This pilot study examined the effectiveness of the Virtual Electrosurgery Skill Trainer (VEST) on OR fire safety skills among prelicensure nursing students. METHODS An experimental pretest-posttest design was used in this study. Twenty nursing students were randomized to a control or an intervention group. Knowledge and skills acquisition of OR fire safety were assessed. RESULTS There were no statistically significant findings in knowledge for either group. Fisher exact test demonstrated significant relationships between the skills performance criteria of following emergency procedures for a fire and demonstrating PASS (pull-aim-squeeze-sweep) technique (P = .001). CONCLUSIONS Academic and hospital educators may consider incorporating virtual reality simulation to teach fire safety education or reinforce general fire safety practices to nursing students and novice nurses.
Collapse
Affiliation(s)
- Kelly L Rossler
- Author Affiliations: Assistant Professor (Dr Rossler), Lecturer (Dr Duvall), Louise Herrington School of Nursing, Baylor University; and Assistant Director (Dr Sankaranarayanan), Center for Evidence Based Simulation, Baylor University Medical Center, Dallas, Texas
| | | | | |
Collapse
|
13
|
Ganry L, Sigaux N, Ettinger KS, Salman SO, Fernandes RP. Modified GoPro Hero 6 and 7 for Intraoperative Surgical Recording-Transformation Into a Surgeon-Perspective Professional Quality Recording System. J Oral Maxillofac Surg 2019; 77:1703.e1-1703.e6. [PMID: 31009633 DOI: 10.1016/j.joms.2019.03.026] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2019] [Accepted: 03/18/2019] [Indexed: 11/27/2022]
Abstract
Recent innovations in wearable action cameras with high-definition video recording enable surgeons to use cameras for their surgical procedures. In this study, the GoPro HERO 6 (and 7) Black edition camera was modified step by step to allow for a completely wireless surgeon-perspective recording with a battery life and memory capacity never previously obtained with such a high level of digital video quality. With this system, a surgeon can record for more than 14 hours 26 minutes in 1,080 pixels at 60 frames per second without breaking scrub and capture the operating surgeon's direct view of the field. By modifying the newest generation of devices, the authors successfully eliminated all shortcomings of the prior generation of GoPro cameras for surgical recording. The modified GoPro HERO6 camera produced professional recording quality for a total cost lower than US$850. This is critically important, because video-based surgical training will continue to be a primary area of development in the future and represents a novel and effective way for young generations of surgeons to attain technical excellence and knowledge in surgery.
Collapse
Affiliation(s)
- Laurent Ganry
- Head and Neck Oncologic Surgery and Microvascular Reconstruction Fellow, Division of Head and Neck Surgery, Department of Oral and Maxillofacial Surgery, University of Florida College of Medicine-Jacksonville, Jacksonville, FL.
| | - Nicolas Sigaux
- Assisting Professor, Maxillofacial Surgery, Hospices Civils de Lyon, Centre Hospitalo-Universitaire Lyon-Sud, Université Claude Bernard Lyon I, Lyon; Plastic, Reconstructive and Esthetic Surgery, Hospices Civils de Lyon, Groupement Hospitalier Nord, Université Claude Bernard Lyon I, Lyon; Maxillofacial Surgery, Hospices Civils de Lyon, Université Claude Bernard Lyon I, Lyon, France
| | - Kyle S Ettinger
- Head and Neck Oncologic Surgery and Microvascular Reconstruction Fellow, Division of Head and Neck Surgery, Department of Oral and Maxillofacial Surgery, University of Florida College of Medicine-Jacksonville, Jacksonville, FL; Senior Associate Consultant, Mayo Clinic and Department of Surgery, Division of Oral and Maxillofacial Surgery, Mayo College of Medicine, Rochester, MN
| | - Salam O Salman
- Residency Program Director and Assistant Professor, Department of Oral and Maxillofacial Surgery, University of Florida Health-Jacksonville, Jacksonville, FL
| | - Rui P Fernandes
- Chief, Division of Head and Neck Surgery; Program Director, Head and Neck Oncologic Surgery and Microvascular Fellowship; Associate Department Chair and Professor, Department of Oral and Maxillofacial Surgery, University of Florida College of Medicine-Jacksonville, Jacksonville, FL
| |
Collapse
|
14
|
Vertemati M, Cassin S, Rizzetto F, Vanzulli A, Elli M, Sampogna G, Gallieni M. A Virtual Reality Environment to Visualize Three-Dimensional Patient-Specific Models by a Mobile Head-Mounted Display. Surg Innov 2019; 26:359-370. [PMID: 30632462 DOI: 10.1177/1553350618822860] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
INTRODUCTION With the availability of low-cost head-mounted displays (HMDs), virtual reality environments (VREs) are increasingly being used in medicine for teaching and clinical purposes. Our aim was to develop an interactive, user-friendly VRE for tridimensional visualization of patient-specific organs, establishing a workflow to transfer 3-dimensional (3D) models from imaging datasets to our immersive VRE. MATERIALS AND METHODS This original VRE model was built using open-source software and a mobile HMD, Samsung Gear VR. For its validation, we enrolled 33 volunteers: morphologists (n = 11), trainee surgeons (n = 15), and expert surgeons (n = 7). They tried our VRE and then filled in an original 5-point Likert-type scale 6-item questionnaire, considering the following parameters: ease of use, anatomy comprehension compared with 2D radiological imaging, explanation of anatomical variations, explanation of surgical procedures, preoperative planning, and experience of gastrointestinal/neurological disorders. Results in the 3 groups were statistically compared using analysis of variance. RESULTS Using cross-sectional medical imaging, the developed VRE allowed to visualize a 3D patient-specific abdominal scene in 1 hour. Overall, the 6 items were evaluated positively by all groups; only anatomy comprehension was statistically significant different among the 3 groups. CONCLUSIONS Our approach, based on open-source software and mobile hardware, proved to be a valid and well-appreciated system to visualize 3D patient-specific models, paving the way for a potential new tool for teaching and preoperative planning.
Collapse
Affiliation(s)
| | | | | | | | - Marco Elli
- 1 Università degli Studi di Milano, Milan, Italy
| | | | | |
Collapse
|
15
|
Yu M, Zhou R, Wang H, Zhao W. An evaluation for VR glasses system user experience: The influence factors of interactive operation and motion sickness. APPLIED ERGONOMICS 2019; 74:206-213. [PMID: 30487101 DOI: 10.1016/j.apergo.2018.08.012] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/13/2017] [Revised: 03/21/2018] [Accepted: 08/11/2018] [Indexed: 06/09/2023]
Abstract
As a new type of Virtual Reality (VR) headset, VR glasses rise rapidly in a number of areas. It's essential to understand the importance of user experience (UX) on VR glasses design. This study aimed to develop questionnaires for evaluating VR glasses' UX, as well as to investigate the relationship between various UX variables. With using lab-based usability tests, this study analyzed participants' self-reports and performance based on testing eight VR glasses and seven mobile applications. A nine-item questionnaire and a ten-item questionnaire were successfully developed to measure VR glasses systems' UX quality in terms of hardware and application, respectively. Within a proposed UX evaluation framework, the perceived UX quality relative to VR glasses hardware emerged as a core predictor in predicting interactive operation performance, whereas the application UX perception was a significant predictor of motion sickness.
Collapse
Affiliation(s)
- Mengli Yu
- School of Economics and Management, Beihang University, Beijing, China
| | - Ronggang Zhou
- School of Economics and Management, Beihang University, Beijing, China; Beijing Advanced Innovation Center for Big Data and Brain Computing, Beihang University, Beijing, China.
| | - Huiwen Wang
- School of Economics and Management, Beihang University, Beijing, China; Beijing Advanced Innovation Center for Big Data and Brain Computing, Beihang University, Beijing, China
| | - Weihua Zhao
- User and Market Research Department, China Mobile Research Institute, Beijing, China
| |
Collapse
|
16
|
Harrington CM, Kavanagh DO, Wright Ballester G, Wright Ballester A, Dicker P, Traynor O, Hill A, Tierney S. 360° Operative Videos: A Randomised Cross-Over Study Evaluating Attentiveness and Information Retention. JOURNAL OF SURGICAL EDUCATION 2018; 75:993-1000. [PMID: 29122571 DOI: 10.1016/j.jsurg.2017.10.010] [Citation(s) in RCA: 21] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/22/2017] [Revised: 10/09/2017] [Accepted: 10/13/2017] [Indexed: 05/28/2023]
Abstract
OBJECTIVE Although two-dimensional (2D) and three-dimensional videos have traditionally provided foundations for reviewing operative procedures, the recent 360º format may provide new dimensions to surgical education. This study sought to describe the production of a high quality 360º video for an index-operation (augmented with educational material), while evaluating for variances in attentiveness, information retention, and appraisal compared to 2D. DESIGN A 6-camera synchronised array (GoPro Omni, [California, United States]) was suspended inverted and recorded an elective laparoscopic cholecystectomy in 2016. A single-blinded randomised cross-over study was performed to evaluate this video in 360º vs 2D formats. Group A experienced the 360º video using Samsung (Suwon, South-Korea) GearVR virtual-reality headsets, followed by the 2D experience on a 75-inch television. Group B were reversed. Each video was probed at designated time points for engagement levels and task-unrelated images or thoughts. Alternating question banks were administered following each video experience. Feedback was obtained via a short survey at study completion. SETTING The New Academic and Education Building (NAEB) in Dublin, Royal College of Surgeons in Ireland, July 2017. PARTICIPANTS Preclinical undergraduate students from a medical university in Ireland. RESULTS Forty students participated with a mean age of 23.2 ± 4.5 years and equal sex involvement. The 360º video demonstrated significantly higher engagement (p < 0.01) throughout the experience and lower task-unrelated images or thoughts (p < 0.01). Significant variances in information retention between the 2 groups were absent (p = 0.143) but most (65%) reported the 360º video as their learning platform of choice. Mean appraisal levels for the 360º platform were positive with mean responses of >8/10 for the platform for learning, immersion, and entertainment. CONCLUSIONS This study describes the successful development and evaluation of a 360º operative video. This new video format demonstrated significant engagement and attentiveness benefits compared to traditional 2D formats. This requires further evaluation in the field of technology enhanced learning.
Collapse
Affiliation(s)
- Cuan M Harrington
- Department of Surgical Affairs, Royal College of Surgeons in Ireland, Dublin.
| | - Dara O Kavanagh
- Department of Surgical Affairs, Royal College of Surgeons in Ireland, Dublin
| | | | | | - Patrick Dicker
- Department of Epidemiology and Public Health Medicine, Royal College of Surgeons in Ireland, Dublin
| | - Oscar Traynor
- Department of Surgical Affairs, Royal College of Surgeons in Ireland, Dublin
| | - Arnold Hill
- Department of Surgery, Beaumont Hospital, Dublin, Ireland
| | - Sean Tierney
- Department of Surgical Affairs, Royal College of Surgeons in Ireland, Dublin
| |
Collapse
|
17
|
Ho CM, Wakabayashi G, Yeh CC, Hu RH, Sakaguchi T, Hasegawa Y, Takahara T, Nitta H, Sasaki A, Lee PH. Comprehensive evaluation of liver resection procedures: surgical mind development through cognitive task analysis. J Vis Surg 2018; 4:21. [PMID: 29445607 DOI: 10.21037/jovs.2018.01.08] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2017] [Accepted: 01/11/2018] [Indexed: 12/22/2022]
Abstract
Background Liver resection is a complex procedure for trainee surgeons. Cognitive task analysis (CTA) facilitates understanding and decomposing tasks that require a great proportion of mental activity from experts. Methods Using CTA and video-based coaching to compare liver resection by open and laparoscopic approaches, we decomposed the task of liver resection into exposure (visual field building), adequate tension made at the working plane (which may change three-dimensionally during the resection process), and target processing (intervention strategy) that can bridge the gap from the basic surgical principle. Results The key steps of highly-specialized techniques, including hanging maneuvers and looping of extra-hepatic hepatic veins, were shown on video by open and laparoscopic approaches. Conclusions Familiarization with laparoscopic anatomical orientation may help surgeons already skilled at open liver resection transit to perform laparoscopic liver resection smoothly. Facilities at hand (such as patient tolerability, advanced instruments, and trained teams of personnel) can influence surgical decision making. Application of the rationale and realizing the interplay between the surgical principles and the other paramedical factors may help surgeons in training to understand the mental abstractions of experienced surgeons, to choose the most appropriate surgical strategy effectively at will, and to minimize the gap.
Collapse
Affiliation(s)
- Cheng-Maw Ho
- Department of Surgery, National Taiwan University Hospital, Taipei.,College of Medicine, National Taiwan University, Taipei
| | - Go Wakabayashi
- Department of Surgery, Ageo Central General Hospital, Saitama, Japan
| | - Chi-Chuan Yeh
- Department of Surgery, National Taiwan University Hospital, Taipei.,Department of Medical Education, National Taiwan University Hospital, Taipei
| | - Rey-Heng Hu
- Department of Surgery, National Taiwan University Hospital, Taipei
| | - Takanori Sakaguchi
- Second Department of Surgery, Hamamatsu University School of Medicine, Hamamatsu, Japan
| | - Yasushi Hasegawa
- Department of Surgery, School of Medicine, Iwate Medical University, Iwate, Japan
| | - Takeshi Takahara
- Department of Surgery, School of Medicine, Iwate Medical University, Iwate, Japan
| | - Hiroyuki Nitta
- Department of Surgery, School of Medicine, Iwate Medical University, Iwate, Japan
| | - Akira Sasaki
- Department of Surgery, School of Medicine, Iwate Medical University, Iwate, Japan
| | - Po-Huang Lee
- Department of Surgery, National Taiwan University Hospital, Taipei
| |
Collapse
|
18
|
Huber T, Wunderling T, Paschold M, Lang H, Kneist W, Hansen C. Highly immersive virtual reality laparoscopy simulation: development and future aspects. Int J Comput Assist Radiol Surg 2017; 13:281-290. [PMID: 29151194 DOI: 10.1007/s11548-017-1686-2] [Citation(s) in RCA: 57] [Impact Index Per Article: 8.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2017] [Accepted: 11/06/2017] [Indexed: 12/12/2022]
Abstract
PURPOSE Virtual reality (VR) applications with head-mounted displays (HMDs) have had an impact on information and multimedia technologies. The current work aimed to describe the process of developing a highly immersive VR simulation for laparoscopic surgery. METHODS We combined a VR laparoscopy simulator (LapSim) and a VR-HMD to create a user-friendly VR simulation scenario. Continuous clinical feedback was an essential aspect of the development process. We created an artificial VR (AVR) scenario by integrating the simulator video output with VR game components of figures and equipment in an operating room. We also created a highly immersive VR surrounding (IVR) by integrating the simulator video output with a [Formula: see text] video of a standard laparoscopy scenario in the department's operating room. RESULTS Clinical feedback led to optimization of the visualization, synchronization, and resolution of the virtual operating rooms (in both the IVR and the AVR). Preliminary testing results revealed that individuals experienced a high degree of exhilaration and presence, with rare events of motion sickness. The technical performance showed no significant difference compared to that achieved with the standard LapSim. CONCLUSION Our results provided a proof of concept for the technical feasibility of an custom highly immersive VR-HMD setup. Future technical research is needed to improve the visualization, immersion, and capability of interacting within the virtual scenario.
Collapse
Affiliation(s)
- Tobias Huber
- Department of General, Visceral and Transplant Surgery, University Medicine of the Johannes Gutenberg-University Mainz, Mainz, Germany.
| | - Tom Wunderling
- Faculty of Computer Science, Otto-von-Guericke University Magdeburg, Magdeburg, Germany
| | - Markus Paschold
- Department of General, Visceral and Transplant Surgery, University Medicine of the Johannes Gutenberg-University Mainz, Mainz, Germany
| | - Hauke Lang
- Department of General, Visceral and Transplant Surgery, University Medicine of the Johannes Gutenberg-University Mainz, Mainz, Germany
| | - Werner Kneist
- Department of General, Visceral and Transplant Surgery, University Medicine of the Johannes Gutenberg-University Mainz, Mainz, Germany
| | - Christian Hansen
- Faculty of Computer Science, Otto-von-Guericke University Magdeburg, Magdeburg, Germany
| |
Collapse
|
19
|
Abdi E, Bouri M, Burdet E, Himidan S, Bleuler H. Positioning the endoscope in laparoscopic surgery by foot: Influential factors on surgeons' performance in virtual trainer. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2017; 2017:3944-3948. [PMID: 29060760 DOI: 10.1109/embc.2017.8037719] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
We have investigated how surgeons can use the foot to position a laparoscopic endoscope, a task that normally requires an extra assistant. Surgeons need to train in order to exploit the possibilities offered by this new technique and safely manipulate the endoscope together with the hands movements. A realistic abdominal cavity has been developed as training simulator to investigate this multi-arm manipulation. In this virtual environment, the surgeon's biological hands are modelled as laparoscopic graspers while the viewpoint is controlled by the dominant foot. 23 surgeons and medical students performed single-handed and bimanual manipulation in this environment. The results show that residents had superior performance compared to both medical students and more experienced surgeons, suggesting that residency is an ideal period for this training. Performing the single-handed task improves the performance in the bimanual task, whereas the converse was not true.
Collapse
|
20
|
|
21
|
Evans CH, Schenarts KD. Evolving Educational Techniques in Surgical Training. Surg Clin North Am 2016; 96:71-88. [DOI: 10.1016/j.suc.2015.09.005] [Citation(s) in RCA: 68] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|