Modeling Framework Used to Analyze and Describe Junctional Tourniquet Skills

Medical educators have acknowledged the importance of simulation training in developing procedural skills. While simulation training in other disciplines has benefitted from evaluations of users' skill acquisition, the majority of medical training simulators continue to be developed from overly simplified descriptions of procedures, such as techniques prescribed by existing instructional material. Our objective was to use a modeling framework to characterize the skill of various users in applying junctional tourniquets in order to design an effective training simulator. We recorded 46 medical first responders performing training exercises applying a junctional tourniquet and used coded video and sensor data to identify the hierarchy of actions they performed in the process. The model provides several insights into trainee performance, such as the way in which advanced users perform more tasks in parallel, or areas where advanced users employ situational awareness to identify ways they can deviate from recommended protocol to improve outcomes. The model successfully identifies variations in tourniquet application technique that correlates with improvement on clinically relevant metrics including application speed, pressure applied, and tourniquet placement stability. This methodology can improve medical training simulations by indicating changes during the course of learning a new task, such as helpful deviations from instructional protocol.

Can Surgical Performance for Varying Experience be Measured from Hand Motions?

This study evaluates if hand movements, tracked using digital video, can quantify in-context surgical performance. Participants of varied experience completed simple interrupted suturing and running subcuticular suturing tasks. Marker-less motion tracking software traced the two-dimensional position of a region of the hand for every video frame. Four expert observers rated 219 short video clips of participants performing the task from 0 to 10 along the following visual analog scales: fluidity of motion, motion economy, tissue handling, and coordination. Expert ratings of attending surgeon hand motions (mean=7.5, sd=1.3) were significantly greater ( p<0.05) than medical students (mean=5.0, sd=1.9) and junior residents (mean=6.4, sd=1.5) for all rating scales. Significant differences ( p<0.02) in mean path length per cycle were also observed both between medical students (803 mm, sd=374) and senior residents (491 mm, sd=216), and attendings (424 mm, sd=250) and junior residents (609 mm, sd=187). These results suggest that substantial gains in performance are attained after the second year of residency and that hand kinematics can predict differences in expert ratings for simulated suturing tasks commensurate with experience – a necessary step to develop valid and automatic on-demand feedback tools.

Using epistemic network analysis to identify targets for educational interventions in trauma team communication

BACKGROUND

Epistemic Network Analysis (ENA) is a technique for modeling and comparing the structure of connections between elements in coded data. We hypothesized that connections among team discourse elements as modeled by ENA would predict the quality of team performance in trauma simulation.

METHODS

The Modified Non-technical Skills Scale for Trauma (T-NOTECHS) was used to score a simulation-based trauma team resuscitation. Sixteen teams of 5 trainees participated. Dialogue was coded using Verbal Response Modes (VRM), a speech classification system. ENA was used to model the connections between VRM codes. ENA models of teams with lesser T-NOTECHS scores (n = 9, mean = 16.98, standard deviation [SD] = 1.45) were compared with models of teams with greater T-NOTECHS scores (n = 7, mean = 21.02, SD = 1.09).

RESULTS

Teams had different patterns of connections among VRM speech form codes with regard to connections among questions and edifications (meanHIGH = 0.115, meanLOW = -0.089; t = 2.21; P = .046, Cohen d = 1.021). Greater-scoring groups had stronger connections between stating information and providing acknowledgments, confirmation, or advising. Lesser-scoring groups had a stronger connection between asking questions and stating information. Discourse data suggest that this pattern reflected increased uncertainty. Lesser-scoring groups also had stronger connections from edifications to disclosures (revealing thoughts, feelings, and intentions) and interpretations (explaining, judging, and evaluating the behavior of others).

CONCLUSION

ENA is a novel and valid method to assess communication among trauma teams. Differences in communication among higher- and lower-performing teams appear to result from the ways teams use questions. ENA allowed us to identify targets for improvement related to the use of questions and stating information by team members.

Improving diagnosis in healthcare: Local versus national adoption of recommended guidelines for the clinical breast examination

BACKGROUND

This study explores the long-term effectiveness of a newly developed clinical skills curriculum.

METHODS

Students (N = 40) were exposed to a newly developed, simulation-based, clinical breast exam (CBE) curriculum. The same students returned one year later to perform the CBE and were compared to a convenience sample of medical students (N = 15) attending a national conferences. All students were given a clinical vignette and performed the CBE. CBE techniques were video recorded. Chi-squared tests were used to assess differences in CBE technique.

RESULTS

Students exposed to a structured curriculum performed physical examination techniques more consistent with national guidelines than the random, national student sample. Structured curriculum students were more organized, likely to use two hands, a linear search pattern, and include the nipple-areolar complex during the CBE compared to national sample (p < 0.01).

CONCLUSIONS

Students exposed to a structured skills curriculum more consistently performed the CBE according to national guidelines. The variability in technique compared with the national sample of students calls for major improvements in adoption and implementation of structured skills curricula.

Residents' response to bleeding during a simulated robotic surgery task

BACKGROUND

The aim of this study was to assess performance measurement validity of our newly developed robotic surgery task trainer. We hypothesized that residents would exhibit wide variations in their intercohort performance as well as a measurable difference compared to surgeons in fellowship training.

MATERIALS AND METHODS

Our laboratory synthesized a model of a pelvic tumor that simulates unexpected bleeding. Surgical residents and fellows of varying specialties completed a demographic survey and were allowed 20 minutes to resect the tumor using the da Vinci robot and achieve hemostasis. At a standardized event in the simulation, venous bleeding began, and participants attempted hemostasis using suture ligation. A motion tracking system, using electromagnetic sensors, recorded participants' hand movements. A postparticipation Likert scale survey evaluated participants' assessment of the model's realism and usefulness.

RESULTS

Three of the seven residents (postgraduate year 2-5), and the fellow successfully resected the tumor in the allotted time. Residents showed high variability in performance and blood loss (125-700 mL) both within their cohort and compared to the fellow (150 mL blood). All participants rated the model as having high realism and utility for trainees.

CONCLUSIONS

The results support that our bleeding pelvic tumor simulator has the ability to discriminate resident performance in robotic surgery. The combination of motion, decision-making, and blood loss metrics offers a multilevel performance assessment, analyzing both technical and decision-making abilities.

Modeling Touch and Palpation Using Autoregressive Models

OBJECTIVE

The human haptic system uses a set of reproducible and subconscious hand maneuvers to identify objects. Similar subconscious maneuvers are used during medical palpation for screening and diagnosis. The goal of this work was to develop a mathematical model that can be used to describe medical palpation techniques.

METHODS

Palpation data were measured using a two-dimensional array of force sensors. A novel algorithm for estimating the hand position from force data was developed. The hand position data were then modeled using multivariate autoregressive models. Analysis of these models provided palpation direction and frequency as well as palpation type. The models were tested and validated using three different data sets: simulated data, a simplified experiment in which participant followed a known pattern, and breast simulator palpation data.

RESULTS

Simulated data showed that the minimal error in estimating palpation direction and frequency is achieved when the sampling frequency is five to ten times the palpation frequency. The classification accuracy was for the simplified experiment and for the breast simulator data.

CONCLUSION

Proper palpation is one of the vital components of many hands-on clinical examinations. In this study, an algorithm for characterizing medical palpation was developed. The algorithm measured palpation frequency and direction for the first time and provided classification of palpation type.

SIGNIFICANCE

These newly developed models can be used for quantifying and assessing clinical technique, and consequently, lead to improved performance in palpation-based exams. Furthermore, they provide a general tool for the study of human haptics.

Evaluating the role of simulation in healthcare innovation: recommendations of the Simnovate Medical Technologies Domain Group

Background

Innovation in healthcare is the practical application of new concepts, ideas, processes or technologies into clinical practice. Despite its necessity and potential to improve care in measurable ways, there are several issues related to patient safety, high costs, high failure rates and limited adoption by end-users. This mixed-method study aims to explore the role of simulation as a potential testbed for diminishing the risks, pitfalls and resources associated with development and implementation of medical innovations.

Methods

Subject-matter experts consisting of physicians, engineers, scientists and industry leaders participated in four semistructured teleconferences each lasting up to 2 hours each. Verbal data were transcribed verbatim, coded and categorised according to themes using grounded theory, and subsequently synthesised into a conceptual framework. Panelists were then invited to complete an online survey, ranking the (1) current use and (2) potential effectiveness of simulation-based technologies and techniques for evaluating and facilitating the product life cycle pathway. This was performed for each theme of the previously generated conceptual framework using a Likert scale of 1 (no effectiveness) to 9 (highest possible effectiveness) and then segregated according to various forms of simulation.

Results

Over 100 hours of data were collected and analysed. After 7 rounds of inductive data analysis, a conceptual framework of the product life cycle was developed. This framework helped to define and characterise the product development pathway. Agreement between reviewers for inclusion of items after the final round of analysis was 100%. A total of 7 themes were synthesised and categorised into 3 phases of the pathway: ‘design and development’, ‘implementation and value creation’ and ‘product launch’. Strong discrepancies were identified between the current and potential roles of simulation in each phase. Simulation was felt to have the strongest potential role for early prototyping, testing for safety and product quality and testing for product effectiveness and ergonomics.

Conclusions

Simulation has great potential to fulfil several unmet needs in healthcare innovation. This framework can be used to help guide innovators and channel resources appropriately. The ultimate goal is a structured, well-defined process that will result in a product development outcome that has the greatest potential to succeed.

A Simulation-based, cognitive assessment of resident decision making during complex urinary catheterization scenarios

BACKGROUND

This study explores general surgery residents' decision making skills in uncommon, complex urinary catheter scenarios.

METHODS

40 residents were presented with two scenarios. Scenario A was a male with traumatic urethral injury and scenario B was a male with complete urinary blockage. Residents verbalized whether they would catheterize the patient and described the workup and management of suspected pathologies. Residents' decision paths were documented and analyzed.

RESULTS

In scenario A, 45% of participants chose to immediately consult Urology. 47.5% named five diagnostic tests to decide if catheterization was safe. In scenario B, 27% chose to catheterize with a 16 French Coude. When faced with catheterization failure, participants randomly upsized or downsized catheters. Chi-square analysis revealed no measurable consensus amongst participants.

CONCLUSIONS

Residents need more training in complex decision making for urinary catheterization. The decision trees generated in this study provide a useful blueprint of residents' learning needs.

SUMMARY

Exploration of general surgery residents' decision making skills in uncommon, complex urinary catheter scenarios revealed major deficiencies. The resulting decision trees reveal residents' learning needs.

Errors in bladder catheterization: are residents ready for complex scenarios?

BACKGROUND

The aim of this study was to investigate whether junior surgical residents had successfully mastered bladder catheterization. Our hypothesis was that surgical residents would be overly confident in their abilities and underestimate the potential for case complexity.

MATERIALS AND METHODS

PGY 2-4 surgery residents (n = 44) were given 15 min. to complete three of four bladder catheterization simulations. Participants reported their mastery by rating confidence using a 5-point Likert scale. Multiple linear regression analysis was used to test predictors of procedure performance.

RESULTS

Participants made a total of 228 errors with an average of 5.1 errors (standard deviation = 2.6) per participant. The most common errors included not maintaining the sterile field (52.0%), failure to get urine return (20.3%), and inflating the catheter balloon before urine return (8.4%). Some residents committed the same error more than once. Presimulation confidence ratings ranged from "1" being not confident to "5" being extremely confident. Average presimulation confidence was 4.42 (range 1-5, standard deviation = 0.85). Sixteen (36%) residents ranked their presimulation confidence in problem-solving abilities as "moderately confident" or below, whereas 28 (64%) were "very confident" or above. The lower the resident's presimulation confidence in problem-solving, the more errors they committed during the simulation (beta = -0.33, t = -2.15, P = 0.04).

CONCLUSIONS

The residents did not perform as well as they anticipated when presented with more complicated bladder catheterization scenarios. Simulation can be used to identify and expose potential errors that may occur during complex presentations of basic procedures. This type of training and assessment may facilitate mastery.

Relationship Between Technical Errors and Decision-Making Skills in the Junior Resident

OBJECTIVE

The purpose of this study is to coevaluate resident technical errors and decision-making capabilities during placement of a subclavian central venous catheter (CVC). We hypothesize that there would be significant correlations between scenario-based decision-making skills and technical proficiency in central line insertion. We also predict residents would face problems in anticipating common difficulties and generating solutions associated with line placement.

DESIGN

Participants were asked to insert a subclavian central line on a simulator. After completion, residents were presented with a real-life patient photograph depicting CVC placement and asked to anticipate difficulties and generate solutions. Error rates were analyzed using chi-square tests and a 5% expected error rate. Correlations were sought by comparing technical errors and scenario-based decision-making skills.

SETTING

This study was performed at 7 tertiary care centers.

PARTICIPANTS

Study participants (N = 46) largely consisted of first-year research residents who could be followed longitudinally. Second-year research and clinical residents were not excluded.

RESULTS

In total, 6 checklist errors were committed more often than anticipated. Residents committed an average of 1.9 errors, significantly more than the 1 error, at most, per person expected (t(44) = 3.82, p < 0.001). The most common error was performance of the procedure steps in the wrong order (28.5%, p < 0.001). Some of the residents (24%) had no errors, 30% committed 1 error, and 46 % committed more than 1 error. The number of technical errors committed negatively correlated with the total number of commonly identified difficulties and generated solutions (r (33) = -0.429, p = 0.021, r (33) = -0.383, p = 0.044, respectively).

CONCLUSIONS

Almost half of the surgical residents committed multiple errors while performing subclavian CVC placement. The correlation between technical errors and decision-making skills suggests a critical need to train residents in both technique and error management.

Exploring Senior Residents' Intraoperative Error Management Strategies: A Potential Measure of Performance Improvement

OBJECTIVE

The study aim was to determine whether residents' error management strategies changed across 2 simulated laparoscopic ventral hernia (LVH) repair procedures after receiving feedback on their initial performance. We hypothesize that error detection and recovery strategies would improve during the second procedure without hands-on practice.

DESIGN

Retrospective review of participant procedural performances of simulated laparoscopic ventral herniorrhaphy. A total of 3 investigators reviewed procedure videos to identify surgical errors. Errors were deconstructed. Error management events were noted, including error identification and recovery.

SETTING

Residents performed the simulated LVH procedures during a course on advanced laparoscopy. Participants had 30 minutes to complete a LVH procedure. After verbal and simulator feedback, residents returned 24 hours later to perform a different, more difficult simulated LVH repair.

PARTICIPANTS

Senior (N = 7; postgraduate year 4-5) residents in attendance at the course participated in this study.

RESULTS

In the first LVH procedure, residents committed 121 errors (M = 17.14, standard deviation = 4.38). Although the number of errors increased to 146 (M = 20.86, standard deviation = 6.15) during the second procedure, residents progressed further in the second procedure. There was no significant difference in the number of errors committed for both procedures, but errors shifted to the late stage of the second procedure. Residents changed the error types that they attempted to recover (χ²₅=24.96, p<0.001). For the second procedure, recovery attempts increased for action and procedure errors, but decreased for strategy errors. Residents also recovered the most errors in the late stage of the second procedure (p < 0.001).

CONCLUSION

Residents' error management strategies changed between procedures following verbal feedback on their initial performance and feedback from the simulator. Errors and recovery attempts shifted to later steps during the second procedure. This may reflect residents' error management success in the earlier stages, which allowed further progression in the second simulation. Incorporating error recognition and management opportunities into surgical training could help track residents' learning curve and provide detailed, structured feedback on technical and decision-making skills.

Resident performance in complex simulated urinary catheter scenarios

BACKGROUND

Urinary catheter insertion is a common procedure performed in hospitals. Improper catheterization can lead to unnecessary catheter-associated urinary tract infections and urethral trauma, increasing patient morbidity. To prevent such complications, guidelines were created on how to insert and troubleshoot urinary catheters. As nurses have an increasing responsibility for catheter placement, resident responsibility has shifted to more complex scenarios. This study examines the clinical decision-making skills of surgical residents during simulated urinary catheter scenarios. We hypothesize that during urinary catheterization, residents will make inconsistent decisions relating to catheter choices and clinical presentations.

METHODS

Forty-five general surgery residents (postgraduate year 2-4) in Midwest training programs were presented with three of four urinary catheter scenarios of varying difficulty. Residents were allowed 15 min to complete the scenarios with five different urinary catheter choices. A chi-square test was performed to examine the relation between initial and subsequent catheter choices and to evaluate for consistency of decision-making for each scenario.

RESULTS

Eighty-two percent of residents performed scenario A; 49% performed scenario B; 64% performed scenario C, and 82% performed scenario D. For initial attempt for scenario A-C, the 16 French Foley catheter was the most common choice (38%, 54%, 50%, P's < 0.001), whereas for scenario D, the 16 French Coude was the most common choice (37%, P < 0.01). Residents were most likely to be successful in achieving urine output in the initial catheterization attempt (P < 0.001). Chi-square analyses showed no relationship between residents' first and subsequent catheter choices for each scenario (P's > 0.05).

CONCLUSIONS

Evaluation of clinical decision-making shows that initial catheter choice may have been deliberate based on patient background, as evidenced by the most popular choice in scenario D. Analyses of subsequent choices in each of the catheterization models reveal inconsistency. These findings suggest a possible lack of competence or training in clinical decision-making with regard to urinary catheter choices in residents.

Exploring hand coordination as a measure of surgical skill

BACKGROUND

The study aim was to identify residents' coordination between dominant and nondominant hands while grasping for sutures in a laparoscopic ventral hernia repair procedure simulation. We hypothesize residents will rely on their dominant and nondominant hands unequally while grasping for suture.

METHODS

Surgical residents had 15 min to complete the mesh securing and mesh tacking steps of a laparoscopic ventral hernia repair procedure. Procedure videos were coded for manual coordination events during the active suture grasping phase. Manual coordination events were defined as: active motion of dominant, nondominant, or both hands; and bimanual or unimanual manipulation of hands. A chi-square test was used to discriminate between coordination choices.

RESULTS

Thirty-six residents (postgraduate year, 1-5) participated in the study. Residents changed manual coordination types during active suture grasping 500 times, ranging between 5 and 24 events (M = 13.9 events, standard deviation [SD] = 4.4). Bimanual coordination was used most (40%) and required the most time on average (M = 20.6 s, SD = 27.2), while unimanual nondominant coordination was used least (2.2%; M = 7.9 s, SD = 6.9). Residents relied on their dominant and nondominant hands unequally (P < 0.001). During 24% of events, residents depended on their nondominant hand (n = 120), which was predominantly used to operate the suture passer device.

CONCLUSIONS

Residents appeared to actively coordinate both dominant and nondominant hands almost half of the time to complete suture grasping. Bimanual task durations took longer than other tasks on average suggesting these tasks were characteristically longer or switching hands required a greater degree of coordination. Future work is necessary to understand how task completion time and overall performance are affected by residents' hand utilization and switching between dominant and nondominant hands in surgical tasks.

Error tolerance: an evaluation of residents' repeated motor coordination errors

BACKGROUND

The study investigates the relationship between motor coordination errors and total errors using a human factors framework. We hypothesize motor coordination errors will correlate with total errors and provide validity evidence for error tolerance as a performance metric.

METHODS

Residents' laparoscopic skills were evaluated during a simulated laparoscopic ventral hernia repair for motor coordination errors when grasping for intra-abdominal mesh or suture. Tolerance was defined as repeated, failed attempts to correct an error and the time required to recover.

RESULTS

Residents (N = 20) committed an average of 15.45 (standard deviation [SD] = 4.61) errors and 1.70 (SD = 2.25) motor coordination errors during mesh placement. Total errors correlated with motor coordination errors (r[18] = .572, P = .008). On average, residents required 5.09 recovery attempts for 1 motor coordination error (SD = 3.15). Recovery approaches correlated to total error load (r[13] = .592, P = .02).

CONCLUSIONS

Residents' motor coordination errors and recovery approaches predict total error load. Error tolerance proved to be a valid assessment metric relating to overall performance.