Competence in Decision Making: Setting Performance Standards for Critical Care

BACKGROUND

Health care professionals must be able to make frequent and timely decisions that can alter the illness trajectory of intensive care patients. A competence standard for this ability is difficult to establish yet assuring practitioners can make appropriate judgments is an important step in advancing patient safety. We hypothesized that simulation can be used effectively to assess decision-making competence. To test our hypothesis, we used a "standard-setting" method to derive cut scores (standards) for 16 simulated ICU scenarios targeted at decision-making skills and applied them to a cohort of critical care trainees.

METHODS

Panelists (critical care experts) reviewed digital audio-video performances of critical care trainees managing simulated critical care scenarios. Based on their collectively agreed-upon definition of "readiness" to make decisions in an ICU setting, each panelist made an independent judgment (ready, not ready) for a large number of recorded performances. The association between the panelists' judgments and the assessment scores was used to derive scenario-specific performance standards.

RESULTS

For all 16 scenarios, the aggregate panelists' ratings (ready/not ready for independent decision making) were positively associated with the performance scores, permitting derivation of performance standards for each scenario.

CONCLUSIONS

Minimum competence standards for high-stakes decision making can be established through standard-setting techniques. We effectively identified "front-line" providers who are, or are not, ready to make independent decisions in an ICU setting. Our approach may be used to assure stakeholders that clinicians are competent to make appropriate judgments. Further work is needed to determine whether our approach is effective in simulation-based assessments in other domains.

SAVE 2.0: Identifying and strengthening resident leadership skills through simulation based team training

BACKGROUND

The "Surgery for Abdomino-thoracic ViolencE (SAVE)" animate lab engages surgical residents in the management of penetrating injuries in a team setting. Senior residents, representing postgraduate year (PGY) 3-5, assume the role of team leader and facilitate the junior residents, PGY1-2, in operative management of simulated penetrating wounds. Residents completed five scenarios with increasing level of difficulty within set time limits. Senior residents were evaluated on their team's ability to "SAVE" their patient within the time allotted, as well as their communication and leadership skills.

METHODS

General, vascular, urology, and plastic surgery residents (n = 79) were divided into 25 teams of three to four residents by "resident scores" (R scores, the sum of the team members' PGY) to create balanced teams with comparable years of clinical experience. Residents completed assessments of their senior resident's leadership ability and style.

RESULTS

Evaluation of a resident's desired learning style changed across PGY with junior residents preferring more hands-on guidance compared with senior residents preferring only verbal correction. Resident leadership evaluations demonstrated that team leaders of varied resident years achieved the highest scores. Greater differences in the mismatch between autonomy provided to and desired by junior residents correlated to greater junior resident discomfort in expressing their opinion, confidence, and leadership ratings of senior residents. However, greater autonomy mismatch also correlated to more rapid time to task completion.

CONCLUSION

Different from our expectations, clinical experience alone did not define team leader success. Leadership is a powerful influence on the outcome of team performance and may be a skill, which can transcend overall clinical experience. A match between desired and provided resident autonomy and team cohesion may demonstrate a stronger effect on team success in stressful operative situations, such as trauma resuscitation. Enhancement of leadership skills early in residency training may represent an important focus for trauma surgery education.

Decision-making skills improve with critical care training: Using simulation to measure progress

PURPOSE

Health care professionals are expected to acquire decision-making skills during their training, but few methods are available to assess progress in acquiring these essential skills. The purpose of this study was to determine whether a simulation methodology could be used to assess whether decision-making skills improve during critical care training.

MATERIALS AND METHODS

Sixteen simulated scenarios were designed to assess a critical care provider's ability to make decisions in the care of a critical ill patient. Seventeen (17) critical care providers managed 8 of the scenarios early during their training and then managed a second set of 8 scenarios (T2) at the conclusion of their training.

RESULTS

Provider's mean global scenario scores (0-9) increased significantly fromT1 and T2 (5.64 ± 0.74) and (6.54 ± 0.64) with a large effect size (1.3). Acute care nurse practitioners and fellows achieved similar overall scores at the conclusion of their training (ACNP 6.43 ± 0.57; Fellows 6.64 ± 0.72).

CONCLUSIONS

These findings provide evidence to support the validity of a simulation-based method to assess progress in decision-making skills. A simulation methodology could be used to establish a performance standard that determined a provider's ability to make independent decisions.