Sources of non-compliance with clinical practice guidelines in trauma triage: a decision science study

Prospective validation of a hospital triage predictive model to decrease undertriage: an EAST multicenter study

Background

Tiered trauma team activation (TTA) allows systems to optimally allocate resources to an injured patient. Target undertriage and overtriage rates of <5% and <35% are difficult for centers to achieve, and performance variability exists. The objective of this study was to optimize and externally validate a previously developed hospital trauma triage prediction model to predict the need for emergent intervention in 6 hours (NEI-6), an indicator of need for a full TTA.

Methods

The model was previously developed and internally validated using data from 31 US trauma centers. Data were collected prospectively at five sites using a mobile application which hosted the NEI-6 model. A weighted multiple logistic regression model was used to retrain and optimize the model using the original data set and a portion of data from one of the prospective sites. The remaining data from the five sites were designated for external validation. The area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC) were used to assess the validation cohort. Subanalyses were performed for age, race, and mechanism of injury.

Results

14 421 patients were included in the training data set and 2476 patients in the external validation data set across five sites. On validation, the model had an overall undertriage rate of 9.1% and overtriage rate of 53.7%, with an AUROC of 0.80 and an AUPRC of 0.63. Blunt injury had an undertriage rate of 8.8%, whereas penetrating injury had 31.2%. For those aged ≥65, the undertriage rate was 8.4%, and for Black or African American patients the undertriage rate was 7.7%.

Conclusion

The optimized and externally validated NEI-6 model approaches the recommended undertriage and overtriage rates while significantly reducing variability of TTA across centers for blunt trauma patients. The model performs well for populations that traditionally have high rates of undertriage.

Level of evidence

2.

Randomized controlled trials in lung cancer surgery: How are we doing?

Objective

Randomized control trials are considered the highest level of evidence, yet the scalability and practicality of implementing randomized control trials in the thoracic surgical oncology space are not well described. The aim of this study is to understand what types of randomized control trials have been conducted in thoracic surgical oncology and ascertain their success rate in completing them as originally planned.

Methods

The ClinicalTrials.gov database was queried in April 2023 to identify registered randomized control trials performed in patients with lung cancer who underwent surgery (by any technique) as part of their treatment.

Results

There were 68 eligible randomized control trials; 33 (48.5%) were intended to examine different perioperative patient management strategies (eg, analgesia, ventilation, drainage) or to examine different intraoperative technical aspects (eg, stapling, number of ports, port placement, ligation). The number of randomized control trials was relatively stable over time until a large increase in randomized control trials starting in 2016. Forty-four of the randomized control trials (64.7%) were open-label studies, 43 (63.2%) were conducted in a single facility, 66 (97.1%) had 2 arms, and the mean number of patients enrolled per randomized control trial was 236 (SD, 187). Of 21 completed randomized control trials (31%), the average time to complete accrual was 1605 days (4.4 years) and average time to complete primary/secondary outcomes and adverse events collection was 2125 days (5.82 years).

Conclusions

Given the immense investment of resources that randomized control trials require, these findings suggest the need to scrutinize future randomized control trial proposals to assess the likelihood of successful completion. Future study is needed to understand the various contributing factors to randomized control trial success or failure.

The impact of referring hospital resources on interfacility overtriage: A population-based analysis

BACKGROUND

Nearly half of patients transferred from non-trauma to trauma centres have minor injuries. The transfer of patients with minor injuries to trauma centres is not associated with any known patient benefit and represents an opportunity to reduce healthcare costs and improve patient experience. In this study, we evaluated the relationship between hospital resources and overtriage, with the objective of identifying targets for system-level intervention.

METHODS

We conducted a population-based cohort study of adults, age ≥ 16, presenting with minor injuries to non-trauma centres in Ontario, Canada (2009-2020). The primary outcome was overtriage, defined as transfer to a trauma centre. Hierarchical logistic regression was used to evaluate the association between hospital resources and a patient's likelihood of being overtriaged, adjusting for case-mix.

RESULTS

amongst 165,302 patients with minor injuries, 15,641 (9.5 %) were transferred to a trauma centre (overtriage). Presence of a CT scanner, surgical support, or intensive care unit had no impact on a patient's likelihood of overtriage. Relative to community hospitals, presentation to a teaching hospital was independently associated with greater odds of overtriage (OR 2.97, 95 % CI: 1.26-7.00). Accounting for case-mix and resources, the median difference in a patient's odds of overtriage varied 3.7-fold across non-trauma centres (MOR 3.76).

CONCLUSIONS

There is significant variability in overtriage across non-trauma centres, even after adjusting for case-mix and hospital resources. These finding suggests that some centres have developed processes to minimize overtriage independent of available resources. Broad implementation of these processes may represent an opportunity for system-wide quality improvement.

Using a theory-based, customized video game as an educational tool to improve physicians' trauma triage decisions: study protocol for a randomized cluster trial

BACKGROUND

Transfer of severely injured patients to trauma centers, either directly from the field or after evaluation at non-trauma centers, reduces preventable morbidity and mortality. Failure to transfer these patients appropriately (i.e., under-triage) remains common, and occurs in part because physicians at non-trauma centers make diagnostic errors when evaluating the severity of patients' injuries. We developed Night Shift, a theory-based adventure video game, to recalibrate physician heuristics (intuitive judgments) in trauma triage and established its efficacy in the laboratory. We plan a type 1 hybrid effectiveness-implementation trial to determine whether the game changes physician triage decisions in real-life and hypothesize that it will reduce the proportion of patients under-triaged.

METHODS

We will recruit 800 physicians who work in the emergency departments (EDs) of non-trauma centers in the US and will randomize them to the game (intervention) or to usual education and training (control). We will ask those in the intervention group to play Night Shift for 2 h within 2 weeks of enrollment and again for 20 min at quarterly intervals. Those in the control group will receive only usual education (i.e., nothing supplemental). We will then assess physicians' triage practices for older, severely injured adults in the 1-year following enrollment, using Medicare claims, and will compare under-triage (primary outcome), 30-day mortality and re-admissions, functional independence, and over-triage between the two groups. We will evaluate contextual factors influencing reach, adoption, implementation, and maintenance with interviews of a subset of trial participants (n = 20) and of other key decision makers (e.g., patients, first responders, administrators [n = 100]).

DISCUSSION

The results of the trial will inform future efforts to improve the implementation of clinical practice guidelines in trauma triage and will provide deeper understanding of effective strategies to reduce diagnostic errors during time-sensitive decision making.

TRIAL REGISTRATION

ClinicalTrials.gov; NCT06063434 . Registered 26 September 2023.

The role of cue utilization in the detection of phishing emails

This study was designed to examine the roles of cue utilization, phishing features and time pressure in the detection of phishing emails. During two experiments, participants completed an email sorting task containing both phishing and genuine emails. Participants were allocated to either a high or low time pressure condition. Performance was assessed via detection sensitivity and response bias. Participants were classified with either higher or lower cue utilization and completed a measure of phishing knowledge. When participants were blind to the nature of the study (N = 191), participants with higher cue utilization were better able to discriminate phishing from genuine emails. However, they also recorded a stronger bias towards classifying emails as phishing, compared to participants with lower cue utilization. When notified of phishing base rates prior to the email sorting task (N = 191), participants with higher cue utilization were better able to discriminate phishing from genuine emails without recording an increase in rate of false alarms, compared to participants with lower cue utilization. Sensitivity increased with a reduction in time pressure, while response bias was influenced by the number of phishing-related features in each email. The outcomes support the proposition that cue-based processing of critical features is associated with an increase in the capacity of individuals to discriminate phishing from genuine emails, above and beyond phishing-related knowledge. From an applied perspective, these outcomes suggest that cue-based training may be beneficial for improving detection of phishing emails.

Testing the feasibility, acceptability, and preliminary effect of a novel deliberate practice intervention to reduce diagnostic error in trauma triage: a study protocol for a randomized pilot trial

BACKGROUND

Non-compliance with clinical practice guidelines in trauma remains common, in part because physicians make diagnostic errors when triaging injured patients. Deliberate practice, purposeful participation in a training task under the oversight of a coach, effectively changes behavior in procedural domains of medicine but has rarely been used to improve diagnostic skill. We plan a pilot parallel randomized trial to test the feasibility, acceptability, and preliminary effect of a novel deliberate practice intervention to reduce physician diagnostic errors in trauma triage.

METHODS

We will randomize a national convenience sample of physicians who work at non-trauma centers (n = 60) in a 1:1 ratio to a deliberate practice intervention or to a passive control. We will use a customized, theory-based serious video game as the basis of our training task, selected based on its behavior change techniques and game mechanics, along with a coaching manual to standardize the fidelity of the intervention delivery. The intervention consists of three 30-min sessions with content experts (coaches), conducted remotely, during which physicians (trainees) play the game and receive feedback on their diagnostic processes. We will assess (a) the fidelity with which the intervention is delivered by reviewing video recordings of the coaching sessions; (b) the acceptability of the intervention through surveys and semi-structured interviews, and (c) the effect of the intervention by comparing the performance of trainees and a control group of physicians on a validated virtual simulation. We hypothesize that trainees will make ≥ 25% fewer diagnostic errors on the simulation than control physicians, a large effect size. We additionally hypothesize that ≥ 90% of trainees will receive their intervention as planned.

CONCLUSIONS

The results of the trial will inform the decision to proceed with a future hybrid effectiveness-implementation trial of the intervention. It will also provide a deeper understanding of the challenges of using deliberate practice to modify the diagnostic skill of physicians.

TRIAL REGISTRATION

Clinical trials.gov ( NCT05168579 ); 23 December 2021.

Association Between Intoxication and Urgent Neurosurgical Procedures in Severe Traumatic Brain Injury: Results From the American College of Surgeons Trauma Quality Improvement Program

BACKGROUND

The probability of undergoing surgery after severe traumatic brain injury (TBI) varies significantly across studies and centers. However, causes of this variability are poorly understood. We hypothesized that intoxication may impact the probability of receiving an urgent neurosurgical procedure among patients with severe TBI.

METHODS

We performed a retrospective cohort study of adult patients admitted to a Level I or II trauma center in the United States or Canada with an isolated severe TBI (2012-2016). Data were derived from the Trauma Quality Improvement Program dataset. An urgent neurosurgical procedure was defined as a procedure that occurred within 24 hours of admission. Multivariable logistic regression was utilized to examine the independent effect of intoxication on a patient's likelihood of undergoing an urgent procedure, as well as the timing of the procedure.

RESULTS

Of the 33,646 patients with an isolated severe TBI, 11,313 (33.6%) were intoxicated. An urgent neurosurgical procedure was performed in 8,255 (24.5%) cases. Overall, there was no difference in the probability of undergoing an urgent procedure between patients who were and were not intoxicated (OR 0.99; 95% CI 0.94-1.06). While intoxication status had no impact on the probability of surgery among patients with the most severe TBI (head AIS 5: OR 1.06 [95% CI 0.98-1.15]), intoxicated patients on the lower spectrum of injury had lower odds of undergoing an urgent procedure (AIS 3: OR 0.80 [95% CI 0.66-0.97]). Among patients who underwent an urgent procedure, intoxication had no impact on timing.

CONCLUSION

Intoxication status was not associated with differences in the probability of undergoing an urgent neurosurgical procedure among all patients with a severe TBI. However, in patients with less severe TBI, intoxication status was associated with decreased likelihood of receiving an urgent intervention. This finding underscores the challenge in the management of intoxicated patients with TBI.

How do healthcare professionals make decisions concerning low-value care practices? Study protocol of a factorial survey experiment on de-implementation

BACKGROUND

A large number of practices used in health care lack evidence of effectiveness and may be unnecessary or even cause harm. As such, they should be de-implemented. While there are multiple actors involved in de-implementation of such low-value care (LVC) practices, ultimately, the decision to abandon a practice is often made by each health care professional. A recent scoping review identified 6 types of factors affecting the utilization vs. abandonment of LVC practices. These factors concern health care professionals, patients, outer context, inner context, processes, and the characteristics of LVC practice itself. However, it is unclear how professionals weigh these different factors in and how these determinants influence their decisions about abandoning LVC practices. This project aims to investigate how health care professionals account for various factors as they make decisions regarding de-implementation of LVC practices.

METHODS

This project will be carried out in two main steps. First, a factorial survey experiment (a vignette study) will be applied to empirically test the relevance of factors previously identified in the literature for health care professionals' decision-making about de-implementation. Second, interactive workshops with relevant stakeholders will be carried out to develop a framework for professionals' decision-making and to offer suggestions for interventions to support de-implementation of LVC practices.

DISCUSSION

The project has the potential to contribute to improved understanding of the decision-making involved in de-implementation of LVC practices. We will identify which factors are more important when they make judgments about utilizing versus abandoning LVC practices. The results will provide the basis for recommendations concerning appropriate interventions to support de-implementation decision-making processes.

Using the Knowledge-to-Action Framework to understand experiences of breastfeeding guideline implementation: A qualitative study

AIM

To examine the perceptions and experiences of health care professionals and mothers in relation to the implementation of a breastfeeding clinical practice guideline (CPG).

BACKGROUND

Breastfeeding CPG applications remain limited, and qualitative studies have indicated the need to overcome the perception by professionals of difficulties in applying recommendations.

METHODS

A qualitative study was conducted in a Spanish public hospital that implemented the Registered Nurses´ Association of Ontario breastfeeding CPG from 2012 through 2015. Between May and August 2017, 27 semi-structured interviews were conducted with managers, with professionals in maternity and paediatric departments and with mothers. Deductive content analysis was performed following the stages in the Knowledge-To-Action (KTA) Framework.

RESULTS

We obtained five main categories: (a) problem as opportunity; (b) adequate context and adapted recommendations; (c) extent of implementation; (d) impact of results; and (e) knowledge use normalization.

CONCLUSIONS

The KTA Framework assists understanding of the participation of the main actors in breastfeeding CPG implementation.

IMPLICATIONS FOR NURSING MANAGEMENT

The nature of the interventions and the participation of managers, different professionals and mothers in a multi-unit setting generate a complex implementation process that reveals key factors to be taken into account in future CPG implementations.

Disentangling the Relationship between Physician and Organizational Performance: A Signal Detection Approach

Background. In previous research, we employed a signal detection approach to measure the performance of general practitioners (GPs) when deciding about urgent referral for suspected lung cancer. We also explored associations between provider and organizational performance. We found that GPs from practices with higher referral positive predictive value (PPV; chance of referrals identifying cancer) were more reluctant to refer than those from practices with lower PPV. Here, we test the generalizability of our findings to a different cancer. Methods. A total of 252 GPs responded to 48 vignettes describing patients with possible colorectal cancer. For each vignette, respondents decided whether urgent referral to a specialist was needed. They then completed the 8-item Stress from Uncertainty scale. We measured GPs' discrimination (d') and response bias (criterion; c) and their associations with organizational performance and GP demographics. We also measured correlations of d' and c between the 2 studies for the 165 GPs who participated in both. Results. As in the lung study, organizational PPV was associated with response bias: in practices with higher PPV, GPs had higher criterion (b = 0.05 [0.03 to 0.07]; P < 0.001), that is, they were less inclined to refer. As in the lung study, female GPs were more inclined to refer than males (b = -0.17 [-0.30 to -0.105]; P = 0.005). In a mediation model, stress from uncertainty did not explain the gender difference. Only response bias correlated between the 2 studies (r = 0.39, P < 0.001). Conclusions. This study confirms our previous findings regarding the relationship between provider and organizational performance and strengthens the finding of gender differences in referral decision making. It also provides evidence that response bias is a relatively stable feature of GP referral decision making.

Judgment and Decision Making

The science of judgment and decision making involves three interrelated forms of research: analysis of the decisions people face, description of their natural responses, and interventions meant to help them do better. After briefly introducing the field's intellectual foundations, we review recent basic research into the three core elements of decision making: judgment, or how people predict the outcomes that will follow possible choices; preference, or how people weigh those outcomes; and choice, or how people combine judgments and preferences to reach a decision. We then review research into two potential sources of behavioral heterogeneity: individual differences in decision-making competence and developmental changes across the life span. Next, we illustrate applications intended to improve individual and organizational decision making in health, public policy, intelligence analysis, and risk management. We emphasize the potential value of coupling analytical and behavioral research and having basic and applied research inform one another.

Does Preexisting Practice Modify How Video Games Recalibrate Physician Heuristics in Trauma Triage?

BACKGROUND

A majority of severely injured patients fail to receive care at trauma centers (undertriage), in part, because of physician judgment. We previously developed two educational video games that reduced physicians' undertriage compared with control in two clinical trials. In this secondary analysis, we investigated heterogeneity of treatment effect of the interventions by assessing physicians' preexisting practice patterns in claims data. We hypothesized that physicians with high preexisting undertriage would benefit most from game-based training.

METHODS

Using Medicare claims records from 2010 to 2015, we measured physicians' preexisting triage practices before their participation in one of two trials conducted in 2016 and 2017. We categorized physicians as having received game-based training versus control and noted their postintervention simulation triage performance in the trials. We used multivariable linear regression models to assess the heterogeneity of game-based training effect among physicians with high and low preexisting undertriage.

RESULTS

Of the 394 eligible physicians from our trials, we identified 275 (70%) with claims for Medicare fee-for-service beneficiaries suffering severe injury between 2010 and 2015. On average, the physicians were 44 y old (SD 8.4) with 12 y (SD 8.2) of experience. We found significant interaction between preexisting practice and intervention efficacy (P = 0.04). Physicians with high undertriage before enrollment improved significantly with game-based training compared with the control (46% versus 63%, P < 0.001). Those with low preexisting undertriage did not (58% versus 56%, P = 0.76).

CONCLUSIONS

Using claims-based data, we found heterogeneity of treatment effect of interventions designed to recalibrate physician heuristics. Physicians with high preexisting undertriage benefited most from game-based training.

Variability in triage practices for critically ill cancer patients: A randomized controlled trial

PURPOSE

Intensive care triage practices and end-user interpretation of triage guidelines have rarely been assessed. We evaluated agreement between providers on the prioritization of patients for ICU admission using different triage guidelines.

MATERIALS AND METHODS

A multi-centered randomized study on providers from 18 different countries was conducted using clinical vignettes of oncological patients. The level of agreement between providers was measured using two different guidelines, with one being cancer specific.

RESULTS

Amongst 257 providers, 52.5% randomly received the Society of Critical Care Prioritization Model, and 47.5% received a cancer specific flowchart as a guide. In the Prioritization Model arm the average entropy was 1.193, versus 1.153 in the flowchart arm (P = .095) indicating similarly poor agreement. The Fleiss' kappa coefficients were estimated to be 0.2136 for the SCCMPM arm and 0.2457 for the flowchart arm, also similarly implying poor agreement.

CONCLUSIONS

The low agreement amongst practitioners on the prioritization of cancer patient cases for ICU admission existed using both general triage guidelines and guidelines tailored only to cancer patients. The lack of consensus on intensive care unit triage practices in the oncological population exposes a potential barrier to appropriate resource allocation that needs to be addressed.

A prospective evaluation of cervical spine immobilisation in low-risk trauma patients at a tertiary Emergency Department

BACKGROUND

In the Emergency Department cervical spine immobilisation precautions are frequently used. There is controversy in regard to the balance of risks and benefits of routine immobilisation in conscious patients.

METHODS

A prospective multi-methods evaluation in a tertiary trauma referral centre. The objectives were to investigate current practices and rate of concordance with established international guidelines. A provider survey focused on current knowledge, skills and attitudes and was disseminated to nurses, doctors and paramedics treating trauma patients. Additionally, clinical data were collected on a cohort of immobilised trauma patients. Demographic data were analysed using SPSS and content analysis was completed by manifest coding.

RESULTS

The response rate to the survey was 85.2%. Interdisciplinary providers included nurses (n=46), doctors (n=68) and paramedics (n=41). Content analysis revealed a range of themes for improving care. Themes identified included improved application of guidelines, tailored use of equipment in low-risk patients, improved access to radiology results, and staff education. The series of five case vignettes provided to participants revealed a high level of variance in intended approaches to immobilisation. In the cohort of trauma patients (n=54), the median age was 54 years and the most common mechanism of injury was falls (40.7%). Median time spent with immobilisation was 325min. Adherence to a recognised decision tool was 35/54 (64.8%). Precautions were initiated by paramedics in 42/54 (77.8%).

CONCLUSIONS

Despite widespread dissemination of guidelines, observed approaches to patient immobilisation appear to be highly variable in this trauma centre. Reducing variation for low-risk patients is likely to improve the patient journey and minimise the risk of prolonged immobilisation. Further assessment of the causes of variation could define goals for targeted translational change.

Defining the representativeness heuristic in trauma triage: A retrospective observational cohort study

BACKGROUND

Under-triage of severely injured patients presenting to non-trauma centers (failure to transfer to a trauma center) remains problematic despite quality improvement efforts. Insights from the behavioral science literature suggest that physician heuristics (intuitive judgments), and in particular the representativeness heuristic (pattern recognition), may contribute to under-triage. However, little is known about how the representativeness heuristic is instantiated in practice.

METHODS

A multi-disciplinary group of experts identified candidate characteristics of "representative" severe trauma cases (e.g., hypotension). We then reviewed the charts of patients with moderate-to-severe injuries who presented to nine non-trauma centers in western Pennsylvania from 2010-2014 to assess the association between the presence of those characteristics and triage decisions. We tested bivariate associations using χ2 and Fisher's Exact method and multivariate associations using random effects logistic regression.

RESULTS

We identified 235,605 injured patients with 3,199 patients (1%) having moderate-to-severe injuries. Patients had a median age of 78 years (SD 20.1) and mean Injury Severity Score of 10.9 (SD 3.3). Only 759 of these patients (24%) were transferred to a trauma center as recommended by the American College of Surgeons clinical practice guidelines. Representative characteristics occurred in 704 patients (22%). The adjusted odds of transfer were higher in the presence of representative characteristics compared to when they were absent (aOR 1.7, 95% CI: 1.4-2.0, p < 0.001).

CONCLUSIONS

Most moderate-to-severely injured patients present without the characteristics representative of severe trauma. Presence of these characteristics is associated with appropriate transfer, suggesting that modifying physicians' heuristics in trauma may improve triage patterns.

Referral Decision Making of General Practitioners: A Signal Detection Study

BACKGROUND

Signal detection theory (SDT) describes how respondents categorize ambiguous stimuli over repeated trials. It measures separately "discrimination" (ability to recognize a signal amid noise) and "criterion" (inclination to respond "signal" v. "noise"). This is important because respondents may produce the same accuracy rate for different reasons. We employed SDT to measure the referral decision making of general practitioners (GPs) in cases of possible lung cancer.

METHODS

We constructed 44 vignettes of patients for whom lung cancer could be considered and estimated their 1-year risk. Under UK risk-based guidelines, half of the vignettes required urgent referral. We recruited 216 GPs from practices across England. Practices differed in the positive predictive value (PPV) of their urgent referrals (chance of referrals identifying cancer) and the sensitivity (chance of cancer patients being picked up via urgent referral from their practice). Participants saw the vignettes online and indicated whether they would refer each patient urgently or not. We calculated each GP's discrimination ( d ') and criterion ( c) and regressed these on practice PPV and sensitivity, as well as on GP experience and gender.

RESULTS

Criterion was associated with practice PPV: as PPV increased, GPs' c also increased, indicating lower inclination to refer ( b = 0.06 [0.02-0.09]; P = 0.001). Female GPs were more inclined to refer than male GPs ( b = -0.20 [-0.40 to -0.001]; P = 0.049). Average discrimination was modest ( d' = 0.77), highly variable (range, -0.28 to 1.91), and not associated with practice referral performance.

CONCLUSIONS

High referral PPV at the organizational level indicates GPs' inclination to avoid false positives, not better discrimination. Rather than bluntly mandating increases in practice PPV via more referrals, it is necessary to increase discrimination by improving the evidence base for cancer referral decisions.

Using incentives to recruit physicians into behavioral trials: lessons learned from four studies

Objective

To describe lessons learned from the use of different strategies for recruiting physicians responsible for trauma triage, we summarize recruitment data from four behavioral trials run in the United States between 2010 and 2016.

Results

We ran a series of behavioral trials with the primary objective of understanding the influence of heuristics on physician decision making in trauma triage. Three studies were observational; one tested an intervention. The trials used different methods of recruitment (in-person vs. email), timing of the honorarium (pre-paid vs. conditional on completion), type of honorarium [a $100 gift card (monetary reward) vs. an iPad mini 2 (material incentive)], and study tasks (a vignette-based questionnaire, virtual simulation, and intervention plus virtual simulation). We recruited 989 physicians, asking each to complete a questionnaire or virtual simulation online. Recruitment and response rates were 80% in the study where we approached physicians in person, used a pre-paid material incentive, and required that they complete both an intervention plus a virtual simulation. They were 56% when we recruited physicians via email, used a monetary incentive conditional on completion of the task, and required that they complete a vignette-based questionnaire.

Trial registration clinicaltrials.gov; NCT02857348

Efficacy of educational video game versus traditional educational apps at improving physician decision making in trauma triage: randomized controlled trial

OBJECTIVE

To determine whether a behavioral intervention delivered through a video game can improve the appropriateness of trauma triage decisions in the emergency department of non-trauma centers.

DESIGN

Randomized clinical trial.

SETTING

Online intervention in national sample of emergency medicine physicians who make triage decisions at US hospitals.

PARTICIPANTS

368 emergency medicine physicians primarily working at non-trauma centers. A random sample (n=200) of those with primary outcome data was reassessed at six months.

INTERVENTIONS

Physicians were randomized in a 1:1 ratio to one hour of exposure to an adventure video game (Night Shift) or apps based on traditional didactic education (myATLS and Trauma Life Support MCQ Review), both on iPads. Night Shift was developed to recalibrate the process of using pattern recognition to recognize moderate-severe injuries (representativeness heuristics) through the use of stories to promote behavior change (narrative engagement). Physicians were randomized with a 2×2 factorial design to intervention (game v traditional education apps) and then to the experimental condition under which they completed the outcome assessment tool (low v high cognitive load). Blinding could not be maintained after allocation but group assignment was masked during the analysis phase.

MAIN OUTCOME MEASURES

Outcomes of a virtual simulation that included 10 cases; in four of these the patients had severe injuries. Participants completed the simulation within four weeks of their intervention. Decisions to admit, discharge, or transfer were measured. The proportion of patients under-triaged (patients with severe injuries not transferred to a trauma center) was calculated then (primary outcome) and again six months later, with a different set of cases (primary outcome of follow-up study). The secondary outcome was effect of cognitive load on under-triage.

RESULTS

149 (81%) physicians in the game arm and 148 (80%) in the traditional education arm completed the trial. Of these, 64/100 (64%) and 58/100 (58%), respectively, completed reassessment at six months. The mean age was 40 (SD 8.9), 283 (96%) were trained in emergency medicine, and 207 (70%) were ATLS (advanced trauma life support) certified. Physicians exposed to the game under-triaged fewer severely injured patients than those exposed to didactic education (316/596 (0.53) v 377/592 (0.64), estimated difference 0.11, 95% confidence interval 0.05 to 0.16; P<0.001). Cognitive load did not influence under-triage (161/308 (0.53) v 155/288 (0.54) in the game arm; 197/300 (0.66) v 180/292 (0.62) in the traditional educational apps arm; P=0.66). At six months, physicians exposed to the game remained less likely to under-triage patients (146/256 (0.57) v 172/232 (0.74), estimated difference 0.17, 0.09 to 0.25; P<0.001). No physician reported side effects. The sample might not reflect all emergency medicine physicians, and a small set of cases was used to assess performance.

CONCLUSIONS

Compared with apps based on traditional didactic education, exposure of physicians to a theoretically grounded video game improved triage decision making in a validated virtual simulation. Though the observed effect was large, the wide confidence intervals include the possibility of a small benefit, and the real world efficacy of this intervention remains uncertain.

TRIAL REGISTRATION

clinicaltrials.gov; NCT02857348 (initial study)/NCT03138304 (follow-up).

Deaths and high-risk trauma patients missed by standard trauma data sources

BACKGROUND

Trauma registries are used to evaluate and improve trauma care, yet potentially miss certain trauma deaths and high-risk patients. We estimated the number of missed deaths and high-risk trauma patients using commonly available sources of trauma data and resulting bias in quality metrics for field trauma triage.

METHODS

This was a preplanned secondary analysis of a population-based prospective cohort of injured patients transported by 44 emergency medical services agencies to 28 hospitals in seven Northwest counties from January 1, 2011 to December 31, 2011 and followed through hospitalization. We used a stratified probability sampling design for 17,633 patients, weighted to represent all 53,487 injured patients transported by emergency medical services. We compared patients meeting National Trauma Data Bank (NTDB) criteria (weighted n = 5,883), all injured patients presenting to major trauma centers (weighted n = 16,859), and all admitted patients (weighted n = 18,433), to the full sample. Outcomes included in-hospital mortality, Injury Severity Score (ISS) of 16 or higher, and critical resource use within 24 hours.

RESULTS

Among 53,487 injured patients, there were 520 emergency department and in-hospital deaths, 1,745 with ISS of 16 or higher, and 923 requiring early critical resources. Compared to the full cohort, the NTDB cohort missed 62.1% of deaths, 39.2% of patients with ISS of 16 or higher, and 23.8% requiring early critical resources, especially older adults injured by falls and admitted to nontrauma hospitals. The admission cohort missed the fewest patients-23.3% of deaths, 10.5% with an ISS of 16 or higher, and 13.1% requiring early resources. Compared to triage sensitivity in the full cohort (66.2%), sensitivity estimates ranged from 63.6% (all admissions) to 93.4% (NTDB). Compared to triage specificity in the full cohort (87.8%), estimates ranged from 36.4% (NTDB) to 77.3% (all admissions).

CONCLUSION

Common sources of trauma data miss substantial numbers of trauma deaths and high-risk trauma patients and can generate biased estimates for trauma system quality metrics.

LEVEL OF EVIDENCE

Epidemiologic, level III.

Role of Guideline Adherence in Improving Field Triage

OBJECTIVE

To compare the sensitivity of current field triage practices for identifying high-risk trauma patients to strict guideline adherence, including changes in triage specificity, ambulance transport patterns, and trauma center volumes.

METHODS

This was a pre-planned secondary analysis of an out-of-hospital prospective cohort of injured children and adults transported by 44 EMS agencies to 28 trauma and non-trauma hospitals in 7 Northwest U.S. counties from January 1, 2011 through December 31, 2011. Outcomes included Injury Severity Score (ISS) ≥16 (primary) and early critical resource use. Strict adherence of the triage guidelines was based on evidence in the EMS chart for patients meeting any current field triage criteria, calculated with and without strict interpretation of the age criterion (<15 or >55 years). Due to the probability sampling nature of the cohort, strata and weights were included in all analyses.

RESULTS

17,633 injured patients were transported by EMS (weighted to represent 53,487 transported patients), including 3.1% with ISS ≥16 and 1.7% requiring early critical resources. Field triage sensitivity for identifying patients with ISS ≥16 increased from the current 66.2% (95% CI 60.2-71.7%) to 87.3% (95% CI 81.9-91.2%) for strict adherence without age and to 91.0% (95% CI 86.4-94.2%) for strict adherence with age. Specificity decreased with increasing adherence, from 87.8% (current) to 47.6% (strict adherence without age) and 35.8% (strict adherence with age). Areas under the curve (AUC) were 0.78, 0.73, and 0.72, respectively. Results were similar for patients requiring early critical resources. We estimate the number of triage-positive patients transported each year by EMS to an individual major trauma center (on average) to increase from 1,331 (current) to 5,139 (strict adherence without age) and to 6,256 (strict adherence with age).

CONCLUSIONS

The low sensitivity of current triage practices would be expected to improve with strict adherence to current triage guidelines, with a commensurate decrease in triage specificity and an increase in the number of triage-positive patients transported to major trauma centers.

Testing a videogame intervention to recalibrate physician heuristics in trauma triage: study protocol for a randomized controlled trial

Background

Between 30 and 40 % of patients with severe injuries receive treatment at non-trauma centers (under-triage), largely because of physician decision making. Existing interventions to improve triage by physicians ignore the role that intuition (heuristics) plays in these decisions. One such heuristic is to form an initial impression based on representativeness (how typical does a patient appear of one with severe injuries). We created a video game (Night Shift) to recalibrate physician’s representativeness heuristic in trauma triage.

Methods

We developed Night Shift in collaboration with emergency medicine physicians, trauma surgeons, behavioral scientists, and game designers. Players take on the persona of Andy Jordan, an emergency medicine physician, who accepts a new job in a small town. Through a series of cases that go awry, they gain experience with the contextual cues that distinguish patients with minor and severe injuries (based on the theory of analogical encoding) and receive emotionally-laden feedback on their performance (based on the theory of narrative engagement). The planned study will compare the effect of Night Shift with that of an educational program on physician triage decisions and on physician heuristics. Psychological theory predicts that cognitive load increases reliance on heuristics, thereby increasing the under-triage rate when heuristics are poorly calibrated. We will randomize physicians (n = 366) either to play the game or to review an educational program, and will assess performance using a validated virtual simulation. The validated simulation includes both control and cognitive load conditions. We will compare rates of under-triage after exposure to the two interventions (primary outcome) and will compare the effect of cognitive load on physicians’ under-triage rates (secondary outcome). We hypothesize that: a) physicians exposed to Night Shift will have lower rates of under-triage compared to those exposed to the educational program, and b) cognitive load will not degrade triage performance among physicians exposed to Night Shift as much as it will among those exposed to the educational program.

Discussion

Serious games offer a new approach to the problem of poorly-calibrated heuristics in trauma triage. The results of this trial will contribute to the understanding of physician quality improvement and the efficacy of video games as behavioral interventions.

Trial registration

clinicaltrials.gov; NCT02857348; August 2, 2016.

Implementing research results in clinical practice- the experiences of healthcare professionals

Background

In healthcare research, results diffuse only slowly into clinical practice, and there is a need to bridge the gap between research and practice. This study elucidates how healthcare professionals in a hospital setting experience working with the implementation of research results.

Method

A descriptive design was chosen. During 2014, 12 interviews were carried out with healthcare professionals representing different roles in the implementation process, based on semi-structured interview guidelines. The analysis was guided by a directed content analysis approach.

Results

The initial implementation was non-formalized. In the decision-making and management process, the pattern among nurses and doctors, respectively, was found to be different. While nurses’ decisions tended to be problem-oriented and managed on a person-driven basis, doctors’ decisions were consensus-oriented and managed by autonomy. All, however, experienced a knowledge-based execution of the research results, as the implementation process ended.

Conclusion

The results illuminate the challenges involved in closing the evidence-practice gap, and may add to the growing body of knowledge on which basis actions can be taken to ensure the best care and treatment available actually reaches the patient.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-016-1292-y) contains supplementary material, which is available to authorized users.

Methods of Observing Variations in Physicians' Decisions: The Opportunities of Clinical Vignettes

To support their efforts to promote high quality and efficient care, policymakers need to better understand the key factors associated with variations in physicians' decisions, and in particular, physician deviations from evidence-based care. Clinical vignette survey instruments hold potential for research in this area as an approach that both allows for practical, large-scale study and overcomes the data quality challenges posed by analysis of clinical data. These surveys present respondents with a narrative description of a hypothetical patient case and solicit responses to one or more questions regarding the care of the patient. In this review, we describe various methods for measuring variations in physicians' decisions and highlight a range of design features researchers should consider when developing a clinical vignette survey. We conclude by identifying areas for future research.

Communicating scientific uncertainty

All science has uncertainty. Unless that uncertainty is communicated effectively, decision makers may put too much or too little faith in it. The information that needs to be communicated depends on the decisions that people face. Are they (i) looking for a signal (e.g., whether to evacuate before a hurricane), (ii) choosing among fixed options (e.g., which medical treatment is best), or (iii) learning to create options (e.g., how to regulate nanotechnology)? We examine these three classes of decisions in terms of how to characterize, assess, and convey the uncertainties relevant to each. We then offer a protocol for summarizing the many possible sources of uncertainty in standard terms, designed to impose a minimal burden on scientists, while gradually educating those whose decisions depend on their work. Its goals are better decisions, better science, and better support for science.

Assessing the validity of using serious game technology to analyze physician decision making

Background

Physician non-compliance with clinical practice guidelines remains a critical barrier to high quality care. Serious games (using gaming technology for serious purposes) have emerged as a method of studying physician decision making. However, little is known about their validity.

Methods

We created a serious game and evaluated its construct validity. We used the decision context of trauma triage in the Emergency Department of non-trauma centers, given widely accepted guidelines that recommend the transfer of severely injured patients to trauma centers. We designed cases with the premise that the representativeness heuristic influences triage (i.e. physicians make transfer decisions based on archetypes of severely injured patients rather than guidelines). We randomized a convenience sample of emergency medicine physicians to a control or cognitive load arm, and compared performance (disposition decisions, number of orders entered, time spent per case). We hypothesized that cognitive load would increase the use of heuristics, increasing the transfer of representative cases and decreasing the transfer of non-representative cases.

Findings

We recruited 209 physicians, of whom 168 (79%) began and 142 (68%) completed the task. Physicians transferred 31% of severely injured patients during the game, consistent with rates of transfer for severely injured patients in practice. They entered the same average number of orders in both arms (control (C): 10.9 [SD 4.8] vs. cognitive load (CL):10.7 [SD 5.6], p = 0.74), despite spending less time per case in the control arm (C: 9.7 [SD 7.1] vs. CL: 11.7 [SD 6.7] minutes, p<0.01). Physicians were equally likely to transfer representative cases in the two arms (C: 45% vs. CL: 34%, p = 0.20), but were more likely to transfer non-representative cases in the control arm (C: 38% vs. CL: 26%, p = 0.03).

Conclusions

We found that physicians made decisions consistent with actual practice, that we could manipulate cognitive load, and that load increased the use of heuristics, as predicted by cognitive theory.

Advance care planning norms may contribute to hospital variation in end-of-life ICU use: a simulation study

BACKGROUND

There is wide variation in end-of-life (EOL) intensive care unit (ICU) use among academic medical centers (AMCs). Our objective was to develop hypotheses regarding medical decision-making factors underlying this variation.

METHODS

This was a high-fidelity simulation experiment involving a critically and terminally ill elder, followed by a survey and debriefing cognitive interview and evaluated using triangulated quantitative-qualitative comparative analysis. The study was conducted in 2 AMCs in the same state and health care system with disparate EOL ICU use. Subjects were hospital-based physicians responsible for ICU admission decisions. Measurements included treatment plan, prognosis, diagnosis, qualitative case perceptions, and clinical reasoning.

RESULTS

Sixty-seven of 111 (60%) eligible physicians agreed to participate; 48 (72%) could be scheduled. There were no significant between-AMC differences in 3-month prognosis or treatment plan, but there were systematic differences in perceptions of the case. Case perceptions at the low-intensity AMC seemed to be influenced by the absence of a do-not-resuscitate order in the context of norms of universal code status discussion and documentation upon admission, whereas case perceptions at the high-intensity AMC seemed to be influenced by the patient's known metastatic gastric cancer in the context of norms of oncologists' avoiding code status discussions.

CONCLUSIONS

In this simulation study of 2 AMCs, hospital-based physicians had different perceptions of an identical case. We hypothesize that different advance care planning norms may have influenced their decision-making heuristics.

Validating a vignette-based instrument to study physician decision making in trauma triage

BACKGROUND

The evidence supporting the use of vignettes to study physician decision making comes primarily from the study of low-risk decisions and the demonstration of good agreement at the group level between vignettes and actual practice. The validity of using vignettes to predict decision making in more complex, high-risk contexts and at the individual level remains unknown.

METHODS

We had previously developed a vignette-based instrument to study physician decision making in trauma triage. Here, we measured the retest reliability, internal consistency, known-groups performance, and criterion validity of the instrument. Thirty-two emergency physicians, recruited at a national academic meeting, participated in reliability testing. Twenty-eight trauma surgeons, recruited using personal contacts, participated in known-groups testing. Twenty-eight emergency physicians, recruited from physicians working at hospitals for which we had access to medical records, participated in criterion validity testing. We measured rates of undertriage (the proportion of severely injured patients not transferred to trauma centers) and overtriage (the proportion of patients transferred with minor injuries) on the instrument. For physicians participating in criterion validity testing, we compared rates of triage on the instrument with rates in practice, based on chart review.

RESULTS

Physicians made similar transfer decisions for cases (κ = 0.42, P < 0.01) on 2 administrations of the instrument. Responses were internally consistent (Kuder-Richardson, 0.71-0.91). Surgeons had lower rates of undertriage than emergency physicians (13% v. 70%, P < 0.01). No correlation existed between individual rates of under- or overtriage on the vignettes and in practice (r = -0.17, P = 0.4; r = -0.03, P = 0.85).

CONCLUSIONS

The instrument developed to assess trauma triage decision making performed reliably and detected known group differences. However, it did not predict individual physician performance.