Precision Education: The Future of Lifelong Learning in Medicine

ABSTRACT

The goal of medical education is to produce a physician workforce capable of delivering high-quality equitable care to diverse patient populations and communities. To achieve this aim amidst explosive growth in medical knowledge and increasingly complex medical care, a system of personalized and continuous learning, assessment, and feedback for trainees and practicing physicians is urgently needed. In this perspective, the authors build on prior work to advance a conceptual framework for such a system: precision education (PE).PE is a system that uses data and technology to transform lifelong learning by improving personalization, efficiency, and agency at the individual, program, and organization levels. PE "cycles" start with data inputs proactively gathered from new and existing sources, including assessments, educational activities, electronic medical records, patient care outcomes, and clinical practice patterns. Through technology-enabled analytics , insights are generated to drive precision interventions . At the individual level, such interventions include personalized just-in-time educational programming. Coaching is essential to provide feedback and increase learner participation and personalization. Outcomes are measured using assessment and evaluation of interventions at the individual, program, and organizational levels, with ongoing adjustment for repeated cycles of improvement. PE is rooted in patient, health system, and population data; promotes value-based care and health equity; and generates an adaptive learning culture.The authors suggest fundamental principles for PE, including promoting equity in structures and processes, learner agency, and integration with workflow (harmonization). Finally, the authors explore the immediate need to develop consensus-driven standards: rules of engagement between people, products, and entities that interact in these systems to ensure interoperability, data sharing, replicability, and scale of PE innovations.

Haptics: The Science of Touch As a Foundational Pathway to Precision Education and Assessment

ABSTRACT

Clinical touch is the cornerstone of the doctor-patient relationship and can impact patient experience and outcomes. In the current era, driven by an ever-increasing infusion of point-of-care technologies, physical exam skills have become undervalued. Moreover, touch and hands-on skills have been difficult to teach due to inaccurate assessments and difficulty with learning transfer through observation. In this article, the authors argue that haptics, the science of touch, provides a unique opportunity to explore new pathways to facilitate touch training. Furthermore, haptics can dramatically increase the density of touch-based assessments without increasing human rater burden-essential for realizing precision assessment. The science of haptics is reviewed, including the benefits of using haptics-informed language for objective structured clinical examinations. The authors describe how haptic devices and haptic language have and can be used to facilitate learning, communication, documentation and a much-needed reinvigoration of physical examination, and touch excellence at the point of care. The synergy of haptic devices, artificial intelligence, and virtual reality environments are discussed. The authors conclude with challenges of scaling haptic technology in medical education, such as cost and translational needs, and opportunities to achieve wider adoption of this transformative approach to precision education.

Sensor-Based Discovery of Search and Palpation Modes in the Clinical Breast Examination

PURPOSE

Successful implementation of precision education systems requires widespread adoption and seamless integration of new technologies with unique data streams that facilitate real-time performance feedback. This paper explores the use of sensor technology to quantify hands-on clinical skills. The goal is to shorten the learning curve through objective and actionable feedback.

METHOD

A sensor-enabled clinical breast examination (CBE) simulator was used to capture force and video data from practicing clinicians (N = 152). Force-by-time markers from the sensor data and a machine learning algorithm were used to parse physicians' CBE performance into periods of search and palpation and then these were used to investigate distinguishing characteristics of successful versus unsuccessful attempts to identify masses in CBEs.

RESULTS

Mastery performance from successful physicians showed stable levels of speed and force across the entire CBE and a 15% increase in force when in palpation mode compared with search mode. Unsuccessful physicians failed to search with sufficient force to detect deep masses ( F [5,146] = 4.24, P = .001). While similar proportions of male and female physicians reached the highest performance level, males used more force as noted by higher palpation to search force ratios ( t [63] = 2.52, P = .014).

CONCLUSIONS

Sensor technology can serve as a useful pathway to assess hands-on clinical skills and provide data-driven feedback. When using a sensor-enabled simulator, the authors found specific haptic approaches that were associated with successful CBE outcomes. Given this study's findings, continued exploration of sensor technology in support of precision education for hands-on clinical skills is warranted.

Surgical Education

This paper summarizes the proceedings of the joint European Surgical Association ESA/American Surgical Association symposium on Surgical Education that took place in Bordeaux, France, as part of the celebrations for 30 years of ESA scientific meetings. Three presentations on the use of quantitative metrics to understand technical decisions, coaching during training and beyond, and entrustable professional activities were presented by American Surgical Association members and discussed by ESA members in a symposium attended by members of both associations.

The Quantified Surgeon: A Glimpse Into the Future of Surgical Metrics and Outcomes

This paper summarizes key points of the 2023 Southeastern Surgical Congress Laws Lecture. The focus of the presentation was on the use of advanced engineering technology to quantify surgical mastery. New concepts relating to the visual-haptic loop, mastery and perception, and mastery and technical decisions were introduced and shown in an empirical fashion to have relevance in procedural outcomes in a simulated setting. The major takeaway point is that surgical mastery can be quantified using advanced engineering technology, and this process will help to shorten the learning curve.

Society of Black Academic Surgeons (SBAS) diversity, equity, and inclusion series: Microaggressions - Lessons Learned from Black Academic Surgeons

Background:

Microaggressions can target individuals based on a variety of differences and these can include sexual orientation, nationality, gender, or personal traits and are often disruptors in the healthcare setting.

Methods:

To address this issue, The Society of Black Academic Surgeons (SBAS) convened a series of presentations and a panel discussion by leaders from SBAS regarding the issue of microaggressions in the surgical workplace. This program was part of a monthly diversity, equity, and inclusion series produced by the Advances in Surgery Channel in alliance with the American College of Surgeons. Dr. Yewande Alimi addresses microaggressions in surgical training, Dr. Fabian Johnston talks about microaggressions in the black male physician, Dr. Lola Fayanju speaks to microaggressions and the black female surgeon, Dr. Carla Pugh discusses microaggressions in the surgical workplace, and Dr. Paris Butler presents on allyship, policies, and real solutions.

Results:

Specifically, through the lens of the Black surgeon experience, SBAS leaders candidly articulate and elaborate on microaggressions’ pervasiveness and the deleterious impact on the profession. Authentic opinions are rendered and constructive techniques to mitigate this challenge are provided. The concept of majority allyship is also introduced, and recommendations on how this can be operationalized is also examined.

Conclusions:

There are a lot of experiences that contribute to our understanding of microaggressions. We look forward to finding new ways to partner with our allies and continuing the conversation.

Do Individual Surgeon Preferences Affect Procedural Outcomes?

OBJECTIVES

Surgeon preferences such as instrument and suture selection and idiosyncratic approaches to individual procedure steps have been largely viewed as minor differences in the surgical workflow. We hypothesized that idiosyncratic approaches could be quantified and shown to have measurable effects on procedural outcomes.

METHODS

At the American College of Surgeons (ACS) Clinical Congress, experienced surgeons volunteered to wear motion tracking sensors and be videotaped while evaluating a loop of porcine intestines to identify and repair 2 preconfigured, standardized enterotomies. Video annotation was used to identify individual surgeon preferences and motion data was used to quantify surgical actions. χ 2 analysis was used to determine whether surgical preferences were associated with procedure outcomes (bowel leak).

RESULTS

Surgeons' (N=255) preferences were categorized into 4 technical decisions. Three out of the 4 technical decisions (repaired injuries together, double-layer closure, corner-stitches vs no corner-stitches) played a significant role in outcomes, P <0.05. Running versus interrupted did not affect outcomes. Motion analysis revealed significant differences in average operative times (leak: 6.67 min vs no leak: 8.88 min, P =0.0004) and work effort (leak-path length=36.86 cm vs no leak-path length=49.99 cm, P =0.001). Surgeons who took the riskiest path but did not leak had better bimanual dexterity (leak=0.21/1.0 vs no leak=0.33/1.0, P =0.047) and placed more sutures during the repair (leak=4.69 sutures vs no leak=6.09 sutures, P =0.03).

CONCLUSIONS

Our results show that individual preferences affect technical decisions and play a significant role in procedural outcomes. Future analysis in more complex procedures may make major contributions to our understanding of contributors to procedure outcomes.

Open surgery tool classification and hand utilization using a multi-camera system

PURPOSE

The goal of this work is to use multi-camera video to classify open surgery tools as well as identify which tool is held in each hand. Multi-camera systems help prevent occlusions in open surgery video data. Furthermore, combining multiple views such as a top-view camera covering the full operative field and a close-up camera focusing on hand motion and anatomy may provide a more comprehensive view of the surgical workflow. However, multi-camera data fusion poses a new challenge: A tool may be visible in one camera and not the other. Thus, we defined the global ground truth as the tools being used regardless their visibility. Therefore, tools that are out of the image should be remembered for extensive periods of time while the system responds quickly to changes visible in the video.

METHODS

Participants (n = 48) performed a simulated open bowel repair. A top-view and a close-up cameras were used. YOLOv5 was used for tool and hand detection. A high-frequency LSTM with a 1-second window at 30 frames per second (fps) and a low-frequency LSTM with a 40-second window at 3 fps were used for spatial, temporal, and multi-camera integration.

RESULTS

The accuracy and F1 of the six systems were: top-view (0.88/0.88), close-up (0.81,0.83), both cameras (0.9/0.9), high-fps LSTM (0.92/0.93), low-fps LSTM (0.9/0.91), and our final architecture the multi-camera classifier(0.93/0.94).

CONCLUSION

Since each camera in a multi-camera system may have a partial view of the procedure, we defined a 'global ground truth.' Defining this at the data labeling phase emphasized this requirement at the learning phase, eliminating the need for any heuristic decisions. By combining a system with a high fps and a low fps from the multiple camera array, we improved the classification abilities of the global ground truth.

Using open surgery simulation kinematic data for tool and gesture recognition

PURPOSE

The use of motion sensors is emerging as a means for measuring surgical performance. Motion sensors are typically used for calculating performance metrics and assessing skill. The aim of this study was to identify surgical gestures and tools used during an open surgery suturing simulation based on motion sensor data.

METHODS

Twenty-five participants performed a suturing task on a variable tissue simulator. Electromagnetic motion sensors were used to measure their performance. The current study compares GRU and LSTM networks, which are known to perform well on other kinematic datasets, as well as MS-TCN++, which was developed for video data and was adapted in this work for motion sensors data. Finally, we extended all architectures for multi-tasking.

RESULTS

In the gesture recognition task the MS-TCN++ has the highest performance with accuracy of [Formula: see text] and F1-Macro of [Formula: see text], edit distance of [Formula: see text] and F1@10 of [Formula: see text] In the tool usage recognition task for the right hand, MS-TCN++ performs the best in most metrics with an accuracy score of [Formula: see text], F1-Macro of [Formula: see text], F1@10 of [Formula: see text], and F1@25 of [Formula: see text]. The multi-task GRU performs best in all metrics in the left-hand case, with an accuracy of [Formula: see text], edit distance of [Formula: see text], F1-Macro of [Formula: see text], F1@10 of [Formula: see text], and F1@25 of [Formula: see text].

CONCLUSION

In this study, using motion sensor data, we automatically identified the surgical gestures and the tools used during an open surgery suturing simulation. Our methods may be used for computing more detailed performance metrics and assisting in automatic workflow analysis. MS-TCN++ performed better in gesture recognition as well as right-hand tool recognition, while the multi-task GRU provided better results in the left-hand case. It should be noted that our multi-task GRU network is significantly smaller and has achieved competitive results in the rest of the tasks as well.

Developing a longitudinal database of surgical skills performance for practicing surgeons: A formal feasibility and acceptance inquiry

BACKGROUND

We explored the feasibility and surgeons' perceptions of the utility of a longitudinal skills performance database.

METHODS

A 10-station surgical skills assessment center was established at a national scientific meeting. Skills assessment volunteers (n = 189) completed a survey including opinions on practicing surgeons' skills evaluation, ethics, and interest in a longitudinal database. A subset (n = 23) participated in a survey-related interview.

RESULTS

Nearly all participants reported interest in a longitudinal database and most believed there is an ethical obligation for such assessments to protect the public. Several interviewees specified a critical role for both formal and informal evaluation is to first create a safe and supportive environment.

CONCLUSIONS

Participants support the construction of longitudinal skills databases that allow information sharing and establishment of professional standards. In a constructive environment, structured peer feedback was deemed acceptable to enhance and diversify surgeon skills. Large scale skills testing is feasible and scientific meetings may be the ideal location.

SAGES consensus recommendations on an annotation framework for surgical video

BACKGROUND

The growing interest in analysis of surgical video through machine learning has led to increased research efforts; however, common methods of annotating video data are lacking. There is a need to establish recommendations on the annotation of surgical video data to enable assessment of algorithms and multi-institutional collaboration.

METHODS

Four working groups were formed from a pool of participants that included clinicians, engineers, and data scientists. The working groups were focused on four themes: (1) temporal models, (2) actions and tasks, (3) tissue characteristics and general anatomy, and (4) software and data structure. A modified Delphi process was utilized to create a consensus survey based on suggested recommendations from each of the working groups.

RESULTS

After three Delphi rounds, consensus was reached on recommendations for annotation within each of these domains. A hierarchy for annotation of temporal events in surgery was established.

CONCLUSIONS

While additional work remains to achieve accepted standards for video annotation in surgery, the consensus recommendations on a general framework for annotation presented here lay the foundation for standardization. This type of framework is critical to enabling diverse datasets, performance benchmarks, and collaboration.

Reassessing career pathways of surgical leaders: An examination of surgical leaders' early accomplishments

BACKGROUND

The American College of Surgeon (ACS), American Surgical Association (ASA), Association of Women Surgeons (AWS), and Society of Black Academic Surgeons (SBAS) partnered to gain insight into whether inequities found in surgical society presidents may be present earlier.

METHODS

ACS, ASA, AWS, and SBAS presidents' CVs were assessed for demographics and scholastic achievements at the time of first faculty appointment. Regression analyses controlling for age were performed to determine relative differences across societies.

RESULTS

66 of the 68 presidents' CVs were received and assessed (97% response rate). 50% of AWS future presidents were hired as Instructors rather than Assistant professors, compared to 29.4% of SBAS, 25% of ASA and 29.4% of ACS. The future ACS, ASA, and SBAS presidents had more total publications than the AWS presidents, but similar numbers of 1st and Sr. author publications.

CONCLUSION

Gender inequities in academic surgeon hiring practices and perceived scholastic success may be present at first hire.

How Wearable Technology Can Facilitate AI Analysis of Surgical Videos

Operative video has great potential to enable instant replays of critical surgical decisions for training and quality review. Recently, artificial intelligence (AI) has shown early promise as a method of enabling efficient video review, analysis, and segmentation. Despite the progress with AI analysis of surgical videos, more work needs to be done to improve the accuracy and efficiency of AI-driven video analysis. At a recent consensus conference held on July 10-11, 2020, 8 research teams shared their work using AI for surgical video analysis. Four of the teams showcased the utility of wearable technology in providing objective surgical metrics. Data from these technologies were shown to pinpoint important cognitive and motor actions during operative tasks and procedures. The results support the utility of wearable technology to facilitate efficient and accurate video analysis and segmentation.

The what? How? And Who? Of video based assessment

BACKGROUND

Currently, there is significant variability in the development, implementation and overarching goals of video review for assessment of surgical performance.

METHODS

This paper evaluates the current methods in which video review is used for evaluation of surgical performance and identifies which processes are critical for successful, widespread implementation of video-based assessment.

RESULTS

Despite the advances in video capture technology and growing interest in video-based assessment, there is a notable gap in the implementation and longitudinal use of formative and summative assessment using video.

CONCLUSION

Validity, scalability and discoverability are current but removable barriers to video-based assessment.

Does the location of short-arm cast univalve effect pressure of the three-point mould?

PURPOSE

Forearm and distal radius fractures are among the most common fractures in children. Many fractures are definitively treated with closed reduction and casting, however, the risk for re-displacement is high (7% to 39%). Proper cast application and the three-point moulding technique are modifiable factors that improve the ability of a cast to maintain the fracture reduction. Many providers univalve the cast to accommodate swelling. This study describes how the location of the univalve cut impacts the pressure at three-point mould sites for a typical dorsally displaced distal radius fracture.

METHODS

We placed nine force-sensing resistors on an arm model to collect pressure data at the three-point mould sites. Sensory inputs were sampled at 15 Hz. Cast padding and a three-point moulded short arm fibreglass cast was applied. The cast was then univalved on the dorsal, volar, radial or ulnar aspect. Pressure recordings were obtained throughout the procedure.

RESULTS

A total of 24 casts were analyzed. Casts univalved in the sagittal plane (dorsal or volar surface) retained up to 16% more pressure across the three moulding sites compared with casts univalved in the coronal plane (radial or ulnar border).

CONCLUSION

Maintaining pressure at the three-point mould prevents loss of reduction at the fracture site. This study shows that univalving the cast dorsally or volarly results in less pressure loss at moulding sites. This should improve the chances of maintaining fracture reductions when compared with radial or ulnar cuts in the cast. Sagittal plane univalving of forearm casts is recommended.

The Society of Black Academic Surgeons CV benchmarking initiative: Early career trends of academic surgical leaders

BACKGROUND

Surgeons from under-represented backgrounds are less likely to receive academic tenure and obtain leadership positions. Our objective was to query the curriculum vitaes (CVs) of SBAS leadership to develop a benchmarking tool to promote and guide careers in academic surgery.

METHODS

CVs from academic leaders were reviewed for academic productivity at early career stages-the first 5-and 10-years. Variables queried: peer-reviewed publications, grant funding, surgical societal involvement, invited lectureships and visiting professorships.

RESULTS

Of 20 CVs, 41 leadership positions including 13 SBAS Presidents were identified. At 5- and 10-years, respectively, the academic productivity increased: 20.6 and 52.3 publications; 4.7 and 9.7 grants; 18 and 42.6 lectures/professorships.

CONCLUSION

The CV benchmarking tool may be a useful framework for aspiring academic surgeons to track their progress relative to successful SBAS members. Creative strategies like these, paired with faculty mentorship and sponsorship are necessary to improve the ethnic diversity in academic surgery.

Modeling Framework Used to Analyze and Describe Junctional Tourniquet Skills

Medical educators have acknowledged the importance of simulation training in developing procedural skills. While simulation training in other disciplines has benefitted from evaluations of users' skill acquisition, the majority of medical training simulators continue to be developed from overly simplified descriptions of procedures, such as techniques prescribed by existing instructional material. Our objective was to use a modeling framework to characterize the skill of various users in applying junctional tourniquets in order to design an effective training simulator. We recorded 46 medical first responders performing training exercises applying a junctional tourniquet and used coded video and sensor data to identify the hierarchy of actions they performed in the process. The model provides several insights into trainee performance, such as the way in which advanced users perform more tasks in parallel, or areas where advanced users employ situational awareness to identify ways they can deviate from recommended protocol to improve outcomes. The model successfully identifies variations in tourniquet application technique that correlates with improvement on clinically relevant metrics including application speed, pressure applied, and tourniquet placement stability. This methodology can improve medical training simulations by indicating changes during the course of learning a new task, such as helpful deviations from instructional protocol.

Can Surgical Performance for Varying Experience be Measured from Hand Motions?

This study evaluates if hand movements, tracked using digital video, can quantify in-context surgical performance. Participants of varied experience completed simple interrupted suturing and running subcuticular suturing tasks. Marker-less motion tracking software traced the two-dimensional position of a region of the hand for every video frame. Four expert observers rated 219 short video clips of participants performing the task from 0 to 10 along the following visual analog scales: fluidity of motion, motion economy, tissue handling, and coordination. Expert ratings of attending surgeon hand motions (mean=7.5, sd=1.3) were significantly greater ( p<0.05) than medical students (mean=5.0, sd=1.9) and junior residents (mean=6.4, sd=1.5) for all rating scales. Significant differences ( p<0.02) in mean path length per cycle were also observed both between medical students (803 mm, sd=374) and senior residents (491 mm, sd=216), and attendings (424 mm, sd=250) and junior residents (609 mm, sd=187). These results suggest that substantial gains in performance are attained after the second year of residency and that hand kinematics can predict differences in expert ratings for simulated suturing tasks commensurate with experience – a necessary step to develop valid and automatic on-demand feedback tools.

Using epistemic network analysis to identify targets for educational interventions in trauma team communication

BACKGROUND

Epistemic Network Analysis (ENA) is a technique for modeling and comparing the structure of connections between elements in coded data. We hypothesized that connections among team discourse elements as modeled by ENA would predict the quality of team performance in trauma simulation.

METHODS

The Modified Non-technical Skills Scale for Trauma (T-NOTECHS) was used to score a simulation-based trauma team resuscitation. Sixteen teams of 5 trainees participated. Dialogue was coded using Verbal Response Modes (VRM), a speech classification system. ENA was used to model the connections between VRM codes. ENA models of teams with lesser T-NOTECHS scores (n = 9, mean = 16.98, standard deviation [SD] = 1.45) were compared with models of teams with greater T-NOTECHS scores (n = 7, mean = 21.02, SD = 1.09).

RESULTS

Teams had different patterns of connections among VRM speech form codes with regard to connections among questions and edifications (meanHIGH = 0.115, meanLOW = -0.089; t = 2.21; P = .046, Cohen d = 1.021). Greater-scoring groups had stronger connections between stating information and providing acknowledgments, confirmation, or advising. Lesser-scoring groups had a stronger connection between asking questions and stating information. Discourse data suggest that this pattern reflected increased uncertainty. Lesser-scoring groups also had stronger connections from edifications to disclosures (revealing thoughts, feelings, and intentions) and interpretations (explaining, judging, and evaluating the behavior of others).

CONCLUSION

ENA is a novel and valid method to assess communication among trauma teams. Differences in communication among higher- and lower-performing teams appear to result from the ways teams use questions. ENA allowed us to identify targets for improvement related to the use of questions and stating information by team members.

Improving diagnosis in healthcare: Local versus national adoption of recommended guidelines for the clinical breast examination

BACKGROUND

This study explores the long-term effectiveness of a newly developed clinical skills curriculum.

METHODS

Students (N = 40) were exposed to a newly developed, simulation-based, clinical breast exam (CBE) curriculum. The same students returned one year later to perform the CBE and were compared to a convenience sample of medical students (N = 15) attending a national conferences. All students were given a clinical vignette and performed the CBE. CBE techniques were video recorded. Chi-squared tests were used to assess differences in CBE technique.

RESULTS

Students exposed to a structured curriculum performed physical examination techniques more consistent with national guidelines than the random, national student sample. Structured curriculum students were more organized, likely to use two hands, a linear search pattern, and include the nipple-areolar complex during the CBE compared to national sample (p < 0.01).

CONCLUSIONS

Students exposed to a structured skills curriculum more consistently performed the CBE according to national guidelines. The variability in technique compared with the national sample of students calls for major improvements in adoption and implementation of structured skills curricula.

Residents' response to bleeding during a simulated robotic surgery task

BACKGROUND

The aim of this study was to assess performance measurement validity of our newly developed robotic surgery task trainer. We hypothesized that residents would exhibit wide variations in their intercohort performance as well as a measurable difference compared to surgeons in fellowship training.

MATERIALS AND METHODS

Our laboratory synthesized a model of a pelvic tumor that simulates unexpected bleeding. Surgical residents and fellows of varying specialties completed a demographic survey and were allowed 20 minutes to resect the tumor using the da Vinci robot and achieve hemostasis. At a standardized event in the simulation, venous bleeding began, and participants attempted hemostasis using suture ligation. A motion tracking system, using electromagnetic sensors, recorded participants' hand movements. A postparticipation Likert scale survey evaluated participants' assessment of the model's realism and usefulness.

RESULTS

Three of the seven residents (postgraduate year 2-5), and the fellow successfully resected the tumor in the allotted time. Residents showed high variability in performance and blood loss (125-700 mL) both within their cohort and compared to the fellow (150 mL blood). All participants rated the model as having high realism and utility for trainees.

CONCLUSIONS

The results support that our bleeding pelvic tumor simulator has the ability to discriminate resident performance in robotic surgery. The combination of motion, decision-making, and blood loss metrics offers a multilevel performance assessment, analyzing both technical and decision-making abilities.

Modeling Touch and Palpation Using Autoregressive Models

OBJECTIVE

The human haptic system uses a set of reproducible and subconscious hand maneuvers to identify objects. Similar subconscious maneuvers are used during medical palpation for screening and diagnosis. The goal of this work was to develop a mathematical model that can be used to describe medical palpation techniques.

METHODS

Palpation data were measured using a two-dimensional array of force sensors. A novel algorithm for estimating the hand position from force data was developed. The hand position data were then modeled using multivariate autoregressive models. Analysis of these models provided palpation direction and frequency as well as palpation type. The models were tested and validated using three different data sets: simulated data, a simplified experiment in which participant followed a known pattern, and breast simulator palpation data.

RESULTS

Simulated data showed that the minimal error in estimating palpation direction and frequency is achieved when the sampling frequency is five to ten times the palpation frequency. The classification accuracy was for the simplified experiment and for the breast simulator data.

CONCLUSION

Proper palpation is one of the vital components of many hands-on clinical examinations. In this study, an algorithm for characterizing medical palpation was developed. The algorithm measured palpation frequency and direction for the first time and provided classification of palpation type.

SIGNIFICANCE

These newly developed models can be used for quantifying and assessing clinical technique, and consequently, lead to improved performance in palpation-based exams. Furthermore, they provide a general tool for the study of human haptics.

Evaluating the role of simulation in healthcare innovation: recommendations of the Simnovate Medical Technologies Domain Group

Background

Innovation in healthcare is the practical application of new concepts, ideas, processes or technologies into clinical practice. Despite its necessity and potential to improve care in measurable ways, there are several issues related to patient safety, high costs, high failure rates and limited adoption by end-users. This mixed-method study aims to explore the role of simulation as a potential testbed for diminishing the risks, pitfalls and resources associated with development and implementation of medical innovations.

Methods

Subject-matter experts consisting of physicians, engineers, scientists and industry leaders participated in four semistructured teleconferences each lasting up to 2 hours each. Verbal data were transcribed verbatim, coded and categorised according to themes using grounded theory, and subsequently synthesised into a conceptual framework. Panelists were then invited to complete an online survey, ranking the (1) current use and (2) potential effectiveness of simulation-based technologies and techniques for evaluating and facilitating the product life cycle pathway. This was performed for each theme of the previously generated conceptual framework using a Likert scale of 1 (no effectiveness) to 9 (highest possible effectiveness) and then segregated according to various forms of simulation.

Results

Over 100 hours of data were collected and analysed. After 7 rounds of inductive data analysis, a conceptual framework of the product life cycle was developed. This framework helped to define and characterise the product development pathway. Agreement between reviewers for inclusion of items after the final round of analysis was 100%. A total of 7 themes were synthesised and categorised into 3 phases of the pathway: ‘design and development’, ‘implementation and value creation’ and ‘product launch’. Strong discrepancies were identified between the current and potential roles of simulation in each phase. Simulation was felt to have the strongest potential role for early prototyping, testing for safety and product quality and testing for product effectiveness and ergonomics.

Conclusions

Simulation has great potential to fulfil several unmet needs in healthcare innovation. This framework can be used to help guide innovators and channel resources appropriately. The ultimate goal is a structured, well-defined process that will result in a product development outcome that has the greatest potential to succeed.

A Simulation-based, cognitive assessment of resident decision making during complex urinary catheterization scenarios

BACKGROUND

This study explores general surgery residents' decision making skills in uncommon, complex urinary catheter scenarios.

METHODS

40 residents were presented with two scenarios. Scenario A was a male with traumatic urethral injury and scenario B was a male with complete urinary blockage. Residents verbalized whether they would catheterize the patient and described the workup and management of suspected pathologies. Residents' decision paths were documented and analyzed.

RESULTS

In scenario A, 45% of participants chose to immediately consult Urology. 47.5% named five diagnostic tests to decide if catheterization was safe. In scenario B, 27% chose to catheterize with a 16 French Coude. When faced with catheterization failure, participants randomly upsized or downsized catheters. Chi-square analysis revealed no measurable consensus amongst participants.

CONCLUSIONS

Residents need more training in complex decision making for urinary catheterization. The decision trees generated in this study provide a useful blueprint of residents' learning needs.

SUMMARY

Exploration of general surgery residents' decision making skills in uncommon, complex urinary catheter scenarios revealed major deficiencies. The resulting decision trees reveal residents' learning needs.

Errors in bladder catheterization: are residents ready for complex scenarios?

BACKGROUND

The aim of this study was to investigate whether junior surgical residents had successfully mastered bladder catheterization. Our hypothesis was that surgical residents would be overly confident in their abilities and underestimate the potential for case complexity.

MATERIALS AND METHODS

PGY 2-4 surgery residents (n = 44) were given 15 min. to complete three of four bladder catheterization simulations. Participants reported their mastery by rating confidence using a 5-point Likert scale. Multiple linear regression analysis was used to test predictors of procedure performance.

RESULTS

Participants made a total of 228 errors with an average of 5.1 errors (standard deviation = 2.6) per participant. The most common errors included not maintaining the sterile field (52.0%), failure to get urine return (20.3%), and inflating the catheter balloon before urine return (8.4%). Some residents committed the same error more than once. Presimulation confidence ratings ranged from "1" being not confident to "5" being extremely confident. Average presimulation confidence was 4.42 (range 1-5, standard deviation = 0.85). Sixteen (36%) residents ranked their presimulation confidence in problem-solving abilities as "moderately confident" or below, whereas 28 (64%) were "very confident" or above. The lower the resident's presimulation confidence in problem-solving, the more errors they committed during the simulation (beta = -0.33, t = -2.15, P = 0.04).

CONCLUSIONS

The residents did not perform as well as they anticipated when presented with more complicated bladder catheterization scenarios. Simulation can be used to identify and expose potential errors that may occur during complex presentations of basic procedures. This type of training and assessment may facilitate mastery.

Relationship Between Technical Errors and Decision-Making Skills in the Junior Resident

OBJECTIVE

The purpose of this study is to coevaluate resident technical errors and decision-making capabilities during placement of a subclavian central venous catheter (CVC). We hypothesize that there would be significant correlations between scenario-based decision-making skills and technical proficiency in central line insertion. We also predict residents would face problems in anticipating common difficulties and generating solutions associated with line placement.

DESIGN

Participants were asked to insert a subclavian central line on a simulator. After completion, residents were presented with a real-life patient photograph depicting CVC placement and asked to anticipate difficulties and generate solutions. Error rates were analyzed using chi-square tests and a 5% expected error rate. Correlations were sought by comparing technical errors and scenario-based decision-making skills.

SETTING

This study was performed at 7 tertiary care centers.

PARTICIPANTS

Study participants (N = 46) largely consisted of first-year research residents who could be followed longitudinally. Second-year research and clinical residents were not excluded.

RESULTS

In total, 6 checklist errors were committed more often than anticipated. Residents committed an average of 1.9 errors, significantly more than the 1 error, at most, per person expected (t(44) = 3.82, p < 0.001). The most common error was performance of the procedure steps in the wrong order (28.5%, p < 0.001). Some of the residents (24%) had no errors, 30% committed 1 error, and 46 % committed more than 1 error. The number of technical errors committed negatively correlated with the total number of commonly identified difficulties and generated solutions (r (33) = -0.429, p = 0.021, r (33) = -0.383, p = 0.044, respectively).

CONCLUSIONS

Almost half of the surgical residents committed multiple errors while performing subclavian CVC placement. The correlation between technical errors and decision-making skills suggests a critical need to train residents in both technique and error management.

Exploring Senior Residents' Intraoperative Error Management Strategies: A Potential Measure of Performance Improvement

OBJECTIVE

The study aim was to determine whether residents' error management strategies changed across 2 simulated laparoscopic ventral hernia (LVH) repair procedures after receiving feedback on their initial performance. We hypothesize that error detection and recovery strategies would improve during the second procedure without hands-on practice.

DESIGN

Retrospective review of participant procedural performances of simulated laparoscopic ventral herniorrhaphy. A total of 3 investigators reviewed procedure videos to identify surgical errors. Errors were deconstructed. Error management events were noted, including error identification and recovery.

SETTING

Residents performed the simulated LVH procedures during a course on advanced laparoscopy. Participants had 30 minutes to complete a LVH procedure. After verbal and simulator feedback, residents returned 24 hours later to perform a different, more difficult simulated LVH repair.

PARTICIPANTS

Senior (N = 7; postgraduate year 4-5) residents in attendance at the course participated in this study.

RESULTS

In the first LVH procedure, residents committed 121 errors (M = 17.14, standard deviation = 4.38). Although the number of errors increased to 146 (M = 20.86, standard deviation = 6.15) during the second procedure, residents progressed further in the second procedure. There was no significant difference in the number of errors committed for both procedures, but errors shifted to the late stage of the second procedure. Residents changed the error types that they attempted to recover (χ²₅=24.96, p<0.001). For the second procedure, recovery attempts increased for action and procedure errors, but decreased for strategy errors. Residents also recovered the most errors in the late stage of the second procedure (p < 0.001).

CONCLUSION

Residents' error management strategies changed between procedures following verbal feedback on their initial performance and feedback from the simulator. Errors and recovery attempts shifted to later steps during the second procedure. This may reflect residents' error management success in the earlier stages, which allowed further progression in the second simulation. Incorporating error recognition and management opportunities into surgical training could help track residents' learning curve and provide detailed, structured feedback on technical and decision-making skills.

Resident performance in complex simulated urinary catheter scenarios

BACKGROUND

Urinary catheter insertion is a common procedure performed in hospitals. Improper catheterization can lead to unnecessary catheter-associated urinary tract infections and urethral trauma, increasing patient morbidity. To prevent such complications, guidelines were created on how to insert and troubleshoot urinary catheters. As nurses have an increasing responsibility for catheter placement, resident responsibility has shifted to more complex scenarios. This study examines the clinical decision-making skills of surgical residents during simulated urinary catheter scenarios. We hypothesize that during urinary catheterization, residents will make inconsistent decisions relating to catheter choices and clinical presentations.

METHODS

Forty-five general surgery residents (postgraduate year 2-4) in Midwest training programs were presented with three of four urinary catheter scenarios of varying difficulty. Residents were allowed 15 min to complete the scenarios with five different urinary catheter choices. A chi-square test was performed to examine the relation between initial and subsequent catheter choices and to evaluate for consistency of decision-making for each scenario.

RESULTS

Eighty-two percent of residents performed scenario A; 49% performed scenario B; 64% performed scenario C, and 82% performed scenario D. For initial attempt for scenario A-C, the 16 French Foley catheter was the most common choice (38%, 54%, 50%, P's < 0.001), whereas for scenario D, the 16 French Coude was the most common choice (37%, P < 0.01). Residents were most likely to be successful in achieving urine output in the initial catheterization attempt (P < 0.001). Chi-square analyses showed no relationship between residents' first and subsequent catheter choices for each scenario (P's > 0.05).

CONCLUSIONS

Evaluation of clinical decision-making shows that initial catheter choice may have been deliberate based on patient background, as evidenced by the most popular choice in scenario D. Analyses of subsequent choices in each of the catheterization models reveal inconsistency. These findings suggest a possible lack of competence or training in clinical decision-making with regard to urinary catheter choices in residents.

Exploring hand coordination as a measure of surgical skill

BACKGROUND

The study aim was to identify residents' coordination between dominant and nondominant hands while grasping for sutures in a laparoscopic ventral hernia repair procedure simulation. We hypothesize residents will rely on their dominant and nondominant hands unequally while grasping for suture.

METHODS

Surgical residents had 15 min to complete the mesh securing and mesh tacking steps of a laparoscopic ventral hernia repair procedure. Procedure videos were coded for manual coordination events during the active suture grasping phase. Manual coordination events were defined as: active motion of dominant, nondominant, or both hands; and bimanual or unimanual manipulation of hands. A chi-square test was used to discriminate between coordination choices.

RESULTS

Thirty-six residents (postgraduate year, 1-5) participated in the study. Residents changed manual coordination types during active suture grasping 500 times, ranging between 5 and 24 events (M = 13.9 events, standard deviation [SD] = 4.4). Bimanual coordination was used most (40%) and required the most time on average (M = 20.6 s, SD = 27.2), while unimanual nondominant coordination was used least (2.2%; M = 7.9 s, SD = 6.9). Residents relied on their dominant and nondominant hands unequally (P < 0.001). During 24% of events, residents depended on their nondominant hand (n = 120), which was predominantly used to operate the suture passer device.

CONCLUSIONS

Residents appeared to actively coordinate both dominant and nondominant hands almost half of the time to complete suture grasping. Bimanual task durations took longer than other tasks on average suggesting these tasks were characteristically longer or switching hands required a greater degree of coordination. Future work is necessary to understand how task completion time and overall performance are affected by residents' hand utilization and switching between dominant and nondominant hands in surgical tasks.

Error tolerance: an evaluation of residents' repeated motor coordination errors

BACKGROUND

The study investigates the relationship between motor coordination errors and total errors using a human factors framework. We hypothesize motor coordination errors will correlate with total errors and provide validity evidence for error tolerance as a performance metric.

METHODS

Residents' laparoscopic skills were evaluated during a simulated laparoscopic ventral hernia repair for motor coordination errors when grasping for intra-abdominal mesh or suture. Tolerance was defined as repeated, failed attempts to correct an error and the time required to recover.

RESULTS

Residents (N = 20) committed an average of 15.45 (standard deviation [SD] = 4.61) errors and 1.70 (SD = 2.25) motor coordination errors during mesh placement. Total errors correlated with motor coordination errors (r[18] = .572, P = .008). On average, residents required 5.09 recovery attempts for 1 motor coordination error (SD = 3.15). Recovery approaches correlated to total error load (r[13] = .592, P = .02).

CONCLUSIONS

Residents' motor coordination errors and recovery approaches predict total error load. Error tolerance proved to be a valid assessment metric relating to overall performance.