Implementation of the Canadian CT Head Rule and Its Association With Use of Computed Tomography Among Patients With Head Injury

Overutilization of head computed tomography in cases of mild traumatic brain injury: a systematic review and meta-analysis

Head computed tomography (CT) is the preferred imaging modality for mild traumatic brain injury (mTBI). The routine use of head CT in low-risk individuals with mild TBI offers no clinical benefit but also causes notable health and financial burden. Despite the availability of related guidelines, studies have reported considerable rate of non-indicated head CT requests. The objectives were to provide an overall estimate for the head CT overutilization rate and to identify the factors contributing to the overuse. A systematic review of PubMed, Scopus, Web of Science, and Embase databases was conducted up to November 2023, following PRISMA and MOOSE guidelines. Two reviewers independently selected eligible articles and extracted data. Quality assessment was performed using a bias risk tool, and a random-effects model was used for data synthesis. Fourteen studies, encompassing 28,612 patients, were included, with 27,809 undergoing head CT scans. Notably, 75% of the included studies exhibited a moderate to high risk of bias. The overutilization rate for pediatric and adult patients was 27% (95% CI: 5-50%) and 32% (95% CI: 21-44%), respectively. An alternative rate, focusing on low-risk pediatric patients, was 54% (95% CI: 20-89%). Overutilization rates showed no significant difference between teaching and non-teaching hospitals. Patients with mTBI from falls or assaults were less likely to receive non-indicated scans. There was no significant association between physician specialty or seniority and overuse, nor between patients' age or sex and the likelihood of receiving a non-indicated scan. Approximately one-third of head CT scans in mTBI cases are avoidable, underscoring the necessity for quality improvement programs to reduce unnecessary imaging and its associated burdens.

The role of information systems in emergency department decision-making-a literature review

OBJECTIVES

Healthcare providers employ heuristic and analytical decision-making to navigate the high-stakes environment of the emergency department (ED). Despite the increasing integration of information systems (ISs), research on their efficacy is conflicting. Drawing on related fields, we investigate how timing and mode of delivery influence IS effectiveness. Our objective is to reconcile previous contradictory findings, shedding light on optimal IS design in the ED.

MATERIALS AND METHODS

We conducted a systematic review following PRISMA across PubMed, Scopus, and Web of Science. We coded the ISs' timing as heuristic or analytical, their mode of delivery as active for automatic alerts and passive when requiring user-initiated information retrieval, and their effect on process, economic, and clinical outcomes.

RESULTS

Our analysis included 83 studies. During early heuristic decision-making, most active interventions were ineffective, while passive interventions generally improved outcomes. In the analytical phase, the effects were reversed. Passive interventions that facilitate information extraction consistently improved outcomes.

DISCUSSION

Our findings suggest that the effectiveness of active interventions negatively correlates with the amount of information received during delivery. During early heuristic decision-making, when information overload is high, physicians are unresponsive to alerts and proactively consult passive resources. In the later analytical phases, physicians show increased receptivity to alerts due to decreased diagnostic uncertainty and information quantity. Interventions that limit information lead to positive outcomes, supporting our interpretation.

CONCLUSION

We synthesize our findings into an integrated model that reveals the underlying reasons for conflicting findings from previous reviews and can guide practitioners in designing ISs in the ED.

What works for and what hinders deimplementation of low-value care in emergency medicine practice? A scoping review

OBJECTIVES

Low-value care can harm patients and healthcare systems. Despite a decade of global endeavours, low value care has persisted. Identification of barriers and enablers is essential for effective deimplementation of low-value care. This scoping review is an evidence summary of barriers, enablers and features of effective interventions for deimplementation of low-value care in emergency medicine practice worldwide.

DESIGN

A mixed-methods scoping review was conducted using the Arksey and O'Malley framework.

DATA SOURCES

Medline, CINAHL, Embase, EMCare, Scopus and grey literature were searched from inception to 5 December 2022.

ELIGIBILITY CRITERIA

Primary studies which employed qualitative, quantitative or mixed-methods approaches to explore deimplementation of low-value care in an EM setting and reported barriers, enablers or interventions were included. Reviews, protocols, perspectives, comments, opinions, editorials, letters to editors, news articles, books, chapters, policies, guidelines and animal studies were excluded. No language limits were applied.

DATA EXTRACTION AND SYNTHESIS

Study selection, data collection and quality assessment were performed by two independent reviewers. Barriers, enablers and interventions were mapped to the domains of the Theoretical Domains Framework. The Mixed Methods Appraisal Tool was used for quality assessment.

RESULTS

The search yielded 167 studies. A majority were quantitative studies (90%, 150/167) that evaluated interventions (86%, 143/167). Limited provider abilities, diagnostic uncertainty, lack of provider insight, time constraints, fear of litigation, and patient expectations were the key barriers. Enablers included leadership commitment, provider engagement, provider training, performance feedback to providers and shared decision-making with patients. Interventions included one or more of the following facets: education, stakeholder engagement, audit and feedback, clinical decision support, nudge, clinical champions and training. Multifaceted interventions were more likely to be effective than single-faceted interventions. Effectiveness of multifaceted interventions was influenced by fidelity of the intervention facets. Use of behavioural change theories such as the Theoretical Domains Framework in the published studies appeared to enhance the effectiveness of interventions to deimplement low-value care.

CONCLUSION

High-fidelity, multifaceted interventions that incorporated education, stakeholder engagement, audit/feedback and clinical decision support, were administered daily and lasted longer than 1 year were most effective in achieving deimplementation of low-value care in emergency departments. This review contributes the best available evidence to date, but further rigorous, theory-informed, qualitative and mixed-methods studies are needed to supplement the growing body of evidence to effectively deimplement low-value care in emergency medicine practice.

Maximizing the Clinical Value of Blood-Based Biomarkers for Mild Traumatic Brain Injury

Mild traumatic brain injury (TBI) and concussion can have serious consequences that develop over time with unpredictable levels of recovery. Millions of concussions occur yearly, and a substantial number result in lingering symptoms, loss of productivity, and lower quality of life. The diagnosis may not be made for multiple reasons, including due to patient hesitancy to undergo neuroimaging and inability of imaging to detect minimal damage. Biomarkers could fill this gap, but the time needed to send blood to a laboratory for analysis made this impractical until point-of-care measurement became available. A handheld blood test is now on the market for diagnosis of concussion based on the specific blood biomarkers glial fibrillary acidic protein (GFAP) and ubiquitin carboxyl terminal hydrolase L1 (UCH-L1). This paper discusses rapid blood biomarker assessment for mild TBI and its implications in improving prediction of TBI course, avoiding repeated head trauma, and its potential role in assessing new therapeutic options. Although we focus on the Abbott i-STAT TBI plasma test because it is the first to be FDA-cleared, our discussion applies to any comparable test systems that may become available in the future. The difficulties in changing emergency department protocols to include new technology are addressed.

Improving the Management of Adults With Mild Traumatic Brain Injury: An Initiative to Reduce Unnecessary Computed Tomographic Scans in the Emergency Department

The overuse of computed tomographic (CT) scans for patients who present to the emergency department (ED) after mild traumatic brain injury (mTBI) has been well-documented. The Canadian Computed Tomography Head Rule (CCHR) is a validated tool to guide ED providers in determining the need for emergent CT of mTBI patients. The purpose of this project was to reduce radiation exposure and ED length of stay by using the CCHR to decrease unnecessary CT scans in adults with TBI. Cost of care was also estimated. The CCHR implementation strategy included an education program for ED staff. The use of the CCHR was promoted throughout the intervention period. The outcomes measured were the number of CT scans ordered, ED length of stay, and the cost of avoidable CT scans. Data were collected through medical record reviews completed by the project leader and were evaluated using the independent samples t test. A total of 600 medical records were reviewed. There was a significant difference between adherence to the CCHR before (M = 64.6%) and after provider education (M = 74.3%). The percentage of CT scans that could have been avoided significantly decreased from baseline (M = 0.63) after provider education (M = 0.46). Length of stay for mTBI patients who were managed based on the CCHR (M = 184.9) was significantly less than the length of stay for those who were not (M = 260.1). The cost of avoidable scans was decreased by 37% over the course of the project. There were no incidents of missed diagnosis found. By increasing awareness of the CCHR and promoting its use, the number of head CT scans ordered, cost of care, and ED length of stay for patients who present after mTBI were significantly improved.

Applicability of machine learning technique in the screening of patients with mild traumatic brain injury

Even though the demand of head computed tomography (CT) in patients with mild traumatic brain injury (TBI) has progressively increased worldwide, only a small number of individuals have intracranial lesions that require neurosurgical intervention. As such, this study aims to evaluate the applicability of a machine learning (ML) technique in the screening of patients with mild TBI in the Regional University Hospital of Maringá, Paraná state, Brazil. This is an observational, descriptive, cross-sectional, and retrospective study using ML technique to develop a protocol that predicts which patients with an initial diagnosis of mild TBI should be recommended for a head CT. Among the tested models, he linear extreme gradient boosting was the best algorithm, with the highest sensitivity (0.70 ± 0.06). Our predictive model can assist in the screening of mild TBI patients, assisting health professionals to manage the resource utilization, and improve the quality and safety of patient care.

Justification of Indication for Cranial CT Imaging after Mild Traumatic Brain Injury According to the Current National Guidelines

The primary aim was to evaluate the compliance of cranial CT indication with the national guideline-based decision rules in patients after mTBI. The secondary aim was to determine the incidence of CT pathologies among justified and unjustified CT scans and to investigate the diagnostic value of these decision rules. This is a retrospective, single-center study on 1837 patients (mean age = 70.7 years) referred to a clinic of oral and maxillofacial surgery following mTBI over a five-year period. The current national clinical decision rules and recommendations for mTBI were retrospectively applied to calculate the incidence of unjustified CT imaging. The intracranial pathologies among the justified and unjustified CT scans were presented using descriptive statistical analysis. The performance of the decision rules was ascertained by calculating the sensitivity, specificity, and predictive values. A total of 123 intracerebral lesions were radiologically detected in 102 (5.5%) of the study patients. Most (62.1%) of the CT scans strictly complied with the guideline recommendations, and 37.8% were not justified and likely avoidable. A significantly higher incidence of intracranial pathology was observed in patients with justified CT scans compared with patients with unjustified CT scans (7.9% vs. 2.5%, p < 0.0001). Patients with loss of consciousness, amnesia, seizures, cephalgia, somnolence, dizziness, nausea, and clinical signs of cranial fractures presented pathologic CT findings more frequently (p < 0.05). The decision rules identified CT pathologies with 92.28% sensitivity and 39.08% specificity. To conclude, compliance with the national decision rules for mTBI was low, and more than a third of the CT scans performed were identified as "likely avoidable". A higher incidence of pathologic CT findings was detected in patients with justified cranial CT imaging. The investigated decision rules showed a high sensitivity but low specificity for predicting CT pathologies.

Validation of the Canadian CT Head Rule and the New Orleans Criteria for Mild Traumatic Brain Injury in Ethiopia

BACKGROUND

Traumatic brain injury (TBI) is a major public health problem worldwide. Although computed tomography (CT) scans are often used for TBI workup, clinicians in low-income countries are limited by fewer radiographic resources. The Canadian CT Head Rule (CCHR) and the New Orleans Criteria (NOC) are widely used screening tools to rule out clinically important brain injury without CT imaging. Although these tools are well validated in studies from upper- and middle-income countries, it is important to study these tools in low-income countries. This study sought to validate the CCHR and NOC in a tertiary teaching hospital population in Addis Ababa, Ethiopia.

METHODS

This single-center retrospective cohort study included patients older than 13 years presenting from December 2018 to July 2021 with a head injury and a Glasgow Coma Scale score of 13-15. Retrospective chart review collected demographic, clinical, radiographic, and hospital course variables. Proportion tables were constructed to determine the sensitivity and specificity of these tools.

RESULTS

A total of 193 patients were included. Both tools showed 100% sensitivity for identifying patients requiring neurosurgical intervention and abnormal CT scans. The specificity for the CCHR was 41.5% and 26.5% for the NOC. Male gender, falling accidents, and headaches had the strongest association with abnormal CT findings.

CONCLUSIONS

The NOC and the CCHR are highly sensitive screening tools that can help rule out clinically important brain injury in mild TBI patients without a head CT in an urban Ethiopian population. Their implementation in this low-resource setting may help spare a significant number of CT scans.

Re-Aiming Equity Evaluation in Clinical Decision Support: A Scoping Review of Equity Assessments in Surgical Decision Support Systems

OBJECTIVE

We critically evaluated the surgical literature to explore the prevalence and describe how equity assessments occur when using clinical decision support systems.

BACKGROUND

Clinical decision support (CDS) systems are increasingly used to facilitate surgical care delivery. Despite formal recommendations to do so, equity evaluations are not routinely performed on CDS systems and underrepresented populations are at risk of harm and further health disparities. We explored surgical literature to determine frequency and rigor of CDS equity assessments and offer recommendations to improve CDS equity by appending existing frameworks.

METHODS

We performed a scoping review up to Augus 25, 2021 using PubMed and Google Scholar for the following search terms: clinical decision support, implementation, RE-AIM, Proctor, Proctor's framework, equity, trauma, surgery, surgical. We identified 1415 citations and 229 abstracts met criteria for review. A total of 84 underwent full review after 145 were excluded if they did not assess outcomes of an electronic CDS tool or have a surgical use case.

RESULTS

Only 6% (5/84) of surgical CDS systems reported equity analyses, suggesting that current methods for optimizing equity in surgical CDS are inadequate. We propose revising the RE-AIM framework to include an Equity element (RE 2 -AIM) specifying that CDS foundational analyses and algorithms are performed or trained on balanced datasets with sociodemographic characteristics that accurately represent the CDS target population and are assessed by sensitivity analyses focused on vulnerable subpopulations.

CONCLUSION

Current surgical CDS literature reports little with respect to equity. Revising the RE-AIM framework to include an Equity element (RE 2 -AIM) promotes the development and implementation of CDS systems that, at minimum, do not worsen healthcare disparities and possibly improve their generalizability.

Prevalence of Computed Tomography Overuse for Mild Head Injury in Adults

INTRODUCTION

The Canadian CT Head Rule (CCHR) is one of many established guidelines for assessing the need for computed tomography (CT) imaging in patients with minor head injuries. Adhering to such criteria would promote the appropriate use of CT imaging, lower healthcare expenses, and prevent harmful radiation exposure. There is no current literature assessing the overuse of CT imaging for minor head injuries in the Kingdom of Bahrain. This study aims to evaluate CT overuse in adult patients with minor head trauma.  Methods: The study was conducted at the Bahrain Defense Force Hospital over 12 months from January to December 2021. All adult patients (>14 years) who sustained a minor head injury and were referred to the emergency department for CT brain imaging were included in the study. Patients presenting for other reasons or suffering from moderate to severe head injuries were excluded. CT reports were retrieved for analysis. The CCHR was used as a reference.  Results: A total of 486 CT scans were performed. Loss of consciousness was the most common symptom on presentation (n = 74 cases). Only 12.1% of CT scans reported positive findings. The prevalence of CT overuse was highest in patients aged 21-30 years. Patients presenting with loss of consciousness showed a high overuse of CT imaging, accounting for 20.3% of all cases. Only 77.4% of cases met the CCHR criteria and 22.6% were defined as overuse, with 95% confidence interval (0.189, 0.266).  Conclusion: When referring to the CCHR, CT imaging for a minor head injury in adults was overused in 22.6% of cases. Further research will be required to reveal the underlying reasons for these findings along with interventions to reduce future overuse.

The use of head computerized tomography in patients with GCS 15 following trauma: Less is more

INTRODUCTION

Computerized tomography (CT) imaging is a standard part of traumatic brain injury (TBI) evaluation but not all patients require it after mild head injury. Given the increasing incidence of TBI in the United States, there is an urgent need to better characterize CT head imaging utilization in evaluating trauma patients, especially patients at low risk of requiring intervention, such as those presenting with a normal GCS.

METHODS

We analyzed the 2017-2019 National Trauma Databank using ICD-10 codes to identify patients who received a head CT. We used Abbreviated Injury Scale (AIS) scores to identify patients with a moderate to severe head injury defined as an AIS severity ≥ 3. Procedural TBI management was defined as having an intracranial monitor or operative decompression. We used a modified Poisson modeling to identify risk factors for a moderate/severe TBI and risk factors for undergoing procedural management among patients with head CT and GCS 15.

RESULTS

Of 2,850,036 patients, 1,502,039 (52.7%) had a head CT. Among patients who had a head CT, 1,078,093 patients (74.9%) had a GCS 15 on arrival. Of this group, only 16.6% (n = 176,431) had a moderate/severe head injury. For those with moderate/severe head injury, 6.0% (n = 10,544/176,431) of patients underwent procedural head injury management. Risk factors for undergoing procedural head injury management included: isolated head injury (RR 2.43, 95% CI 2.34, 2.53), male sex (RR 1.73, 95% CI 1.67, 1.80), age > 50 years (RR 1.39 95% CI 1.32, 1.47), falls (RR 1.28, 95% CI 1.22, 1.35), and the use of anti-coagulation (RR 1.16, 95% CI 1.11, 1.21).

CONCLUSION

Few patients had moderate/severe head injury when presenting with a GCS 15. However, patients ≥ 50 years, men, and those who suffered falls were at higher risk. Anti-coagulation use was not associated with moderate/severe head injury but did increase the risk of procedural TBI management. Given the cost and associated radiation, reducing CT utilization for younger patients while using a more liberal head CT strategy for high-risk patients may provide substantial patient value.

Development of clinical-guideline-based mobile application and its effect on head CT scan utilization in neurology and neurosurgery departments

Background

There is little evidence regarding the adoption and intention of using mobile apps by health care professionals (HCP) and the effectiveness of using mobile apps among physicians is still unclear. To address this challenge, the current study seeks two objectives: developing and implementing a head CT scan appropriateness criteria mobile app (HAC app), and investigating the effect of HAC app on CT scan order.

Methods

A one arm intervention quasi experimental study with before/after analysis was conducted in neurology & neurosurgery (N&N) departments at the academic hospital. We recruited all residents' encounters to N&N departments with head CT scan to examine the effect of HAC app on residents' CT scan utilization. The main outcome measure was CT scan order per patient for seven months at three points, before the intervention, during the intervention, after cessation of the intervention -post-intervention follow-up. Data for CT scan utilization were collected by reviewing medical records and then analyzed using descriptive statistics, Kruskal-Wallis, and Mann-Whitney tests. A focus group discussion with residents was performed to review and digest residents' experiences during interaction with the HAC app.

Results

Sixteen residents participated in this study; a total of 415 N&N encounters with CT scan order, pre-intervention 127 (30.6%), intervention phase 187 (45.1%), and 101 (24.3%) in the post-intervention follow-up phase were included in this study. Although total CT scan utilization was statistically significant during three-time points of the study (P = 0.027), no significant differences were found for CT utilization after cessation of the intervention (P = 1).

Conclusion

The effect of mobile devices on residents' CT scan ordering behavior remains open to debate since the changes were not long-lasting. Further studies based on real interactive experiences with mobile devices is advisable before it can be recommended for widespread use by HCP.

Effectiveness of Clinical Decision Support Systems on the Appropriate Use of Imaging for Central Nervous System Injuries: A Systematic Review

BACKGROUND

One of the best practices for timely and efficient diagnoses of central nervous system (CNS) trauma and complex diseases is imaging. However, rates of imaging for CNS are high and impose a lot of costs to health care facilities in addition to exposing patients with negative impact of ionizing radiation.

OBJECTIVES

This study aimed to systematically review the effects and features of clinical decision support systems (CDSSs) for the appropriate use of imaging for CNS injuries.

METHOD

We searched MEDLINE, SCOPUS, Web of Science, and Cochrane without time period restriction. We included experimental and quasiexperimental studies that assessed the effectiveness of CDSSs designed for the appropriate use of imaging for CNS injuries in any clinical setting, including primary, emergency, and specialist care. The outcomes were categorized based on imaging-related, physician-related, and patient-related groups.

RESULT

A total of 3,223 records were identified through the online literature search. Of the 55 potential papers for the full-text review, 11 eligible studies were included. Reduction of CNS imaging proportion varied from 2.6 to 40% among the included studies. Physician-related outcomes, including guideline adherence, diagnostic yield, and knowledge, were reported in five studies, and all demonstrated positive impact of CDSSs. Four studies had addressed patient-related outcomes, including missed or delayed diagnosis, as well as length of stay. These studies reported a very low rate of missed diagnosis due to the cancellation of computed tomography (CT) examine according to the CDSS recommendations.

CONCLUSION

This systematic review reports that CDSSs decrease the utilization of CNS CT scan, while increasing physicians' adherence to the rules. However, the possible harm of CDSSs to patients was not well addressed by the included studies and needs additional investigation. The actual effect of CDSSs on appropriate imaging would be realized when the saved cost of examinations is compared with the cost of missed diagnosis.

Pitch and Rhythm Perception and Verbal Short-Term Memory in Acute Traumatic Brain Injury

Music perception deficits are common following acquired brain injury due to stroke, epilepsy surgeries, and aneurysmal clipping. Few studies have examined these deficits following traumatic brain injury (TBI), resulting in an under-diagnosis in this population. We aimed to (1) compare TBI patients to controls on pitch and rhythm perception during the acute phase; (2) determine whether pitch and rhythm perception disorders co-occur; (3) examine lateralization of injury in the context of pitch and rhythm perception; and (4) determine the relationship between verbal short-term memory (STM) and pitch and rhythm perception. Music perception was examined using the Scale and Rhythm tests of the Montreal Battery of Evaluation of Amusia, in association with CT scans to identify lesion laterality. Verbal short-term memory was examined using Digit Span Forward. TBI patients had greater impairment than controls, with 43% demonstrating deficits in pitch perception, and 40% in rhythm perception. Deficits were greater with right hemisphere damage than left. Pitch and rhythm deficits co-occurred 31% of the time, suggesting partly dissociable networks. There was a dissociation between performance on verbal STM and pitch and rhythm perception 39 to 42% of the time (respectively), with most individuals (92%) demonstrating intact verbal STM, with impaired pitch or rhythm perception. The clinical implications of music perception deficits following TBI are discussed.

System Level Informatics to Improve Triage Practices for Sickle Cell Disease Vaso-Occlusive Crisis: A Cluster Randomized Controlled Trial

BACKGROUND

National Heart Lung and Blood Institute guidelines for the treatment of vaso-occlusive crisis among people with sickle cell disease in the emergency department recommend assigning an emergency severity index of 2 at triage. However, patients with sickle cell disease often do not receive guideline-concordant care at triage. To address this gap, a decision support tool was developed, in the form of a text banner on the triage page in the electronic health record system, visible to triage nurses.

METHODS

A prospective quality improvement initiative was designed where the emergency severity index clinical decision support tool was deployed to a stratified random sample of emergency department triage nurses to receive the banner (n = 24) or not to receive the banner (n = 27), reminding them to assign the patient to emergency severity index category 2. The acceptability of the emergency severity index clinical decision support tool was evaluated with the Ottawa Acceptability of Decision Rules Instrument. Descriptive and bivariate (chi-square test) statistics were used to characterize the study's primary outcome, proportion of visits assigned an emergency severity index of 2 or higher. A generalized linear mixed model with clustering at the level of the triage nurse was performed to test the association between the banner intervention and triage practices.

RESULTS

A total of 384 ED visits were included for analysis. Before study initiation, the percentage of sickle cell disease patients' visits with the proper emergency severity index assignment at triage was 37.04%. After initiation, the proportion of sickle cell disease patients' visits with an emergency severity index of 2 or higher triaged by nurses in the intervention group was markedly higher in the intervention group than in the control group (64.95% vs 35.05%; χ² = 8.79, P ≤ .003). Accounting for clustering by nurse, the odds ratio for proper triage emergency severity index assignment was 3.22 (95% confidence interval 1.17-8.85; P ≤ .02) for the intervention versus control. Surveyed triage nurses reported the emergency severity index clinical decision support tool to be moderately acceptable (nurses' mean Ottawa Acceptability of Decision Rules Instrument scores ranged from 4.13 to 4.90 on the 6-point scale; n = 11). There were no differences in ED experience outcomes including time to first analgesic or length of stay between the control and intervention groups.

CONCLUSION

Substantial improvements in triage guideline concordance were achieved and sustained without direct nursing education.

Decision support for computed tomography in the emergency department: a multicenter cluster-randomized controlled trial

OBJECTIVES

Clinical decision support may facilitate evidence-based imaging, but most studies to date examining the impact of decision support have used non-randomized designs which limit the conclusions that can be drawn from them. This randomized trial examines if decision support can reduce computed tomography (CT) utilization for patients with mild traumatic brain injuries and suspected pulmonary embolism in the emergency department. This study was funded by a competitive public research grant and registered on ClinicalTrials.gov (NCT02410941).

METHODS

Emergency physicians at five urban sites were assigned to voluntary decision support for CT imaging of patients with either head injuries or suspected pulmonary embolism using a cluster-randomized design over a 1-year intervention period. The co-primary outcomes were CT head and CT pulmonary angiography utilization. CT pulmonary angiography diagnostic yield (proportion of studies diagnostic for acute pulmonary embolism) was a secondary outcome.

RESULTS

A total of 225 physicians were randomized and studied over a 2-year baseline and 1-year intervention period. Physicians interacted with the decision support in 38.0% and 45.0% of eligible head injury and suspected pulmonary embolism cases, respectively. A mixed effects logistic regression model demonstrated no significant impact of decision support on head CT utilization (OR 0.93, 95% CI 0.79-1.10, p = 0.31), CT pulmonary angiography utilization (OR 0.98, 95% CI 0.88-1.11, p = 0.74) or diagnostic yield (OR 1.23, 95% CI 0.96-1.65, p = 0.10). However, overall CT pulmonary diagnostic yield (17.7%) was almost three times higher than that reported by a recent large US study, suggesting that selective imaging was already being employed.

CONCLUSION

Voluntary decision support addressing many commonly cited barriers to evidence-based imaging did not significantly reduce CT utilization or improve diagnostic yield but was limited by low rates of participation and high baseline rates of selective imaging. Demonstrating value to clinicians through interventions that improve workflow is likely necessary to meaningfully change imaging practices.

Diagnostic Accuracy and Failure Mode Analysis of a Deep Learning Algorithm for the Detection of Cervical Spine Fractures

BACKGROUND AND PURPOSE

Artificial intelligence decision support systems are a rapidly growing class of tools to help manage ever-increasing imaging volumes. The aim of this study was to evaluate the performance of an artificial intelligence decision support system, Aidoc, for the detection of cervical spinal fractures on noncontrast cervical spine CT scans and to conduct a failure mode analysis to identify areas of poor performance.

MATERIALS AND METHODS

This retrospective study included 1904 emergent noncontrast cervical spine CT scans of adult patients (60 [SD, 22] years, 50.3% men). The presence of cervical spinal fracture was determined by Aidoc and an attending neuroradiologist; discrepancies were independently adjudicated. Algorithm performance was assessed by calculation of the diagnostic accuracy, and a failure mode analysis was performed.

RESULTS

Aidoc and the neuroradiologist's interpretation were concordant in 91.5% of cases. Aidoc correctly identified 67 of 122 fractures (54.9%) with 106 false-positive flagged studies. Diagnostic performance was calculated as the following: sensitivity, 54.9% (95% CI, 45.7%-63.9%); specificity, 94.1% (95% CI, 92.9%-95.1%); positive predictive value, 38.7% (95% CI, 33.1%-44.7%); and negative predictive value, 96.8% (95% CI, 96.2%-97.4%). Worsened performance was observed in the detection of chronic fractures; differences in diagnostic performance were not altered by study indication or patient characteristics.

CONCLUSIONS

We observed poor diagnostic accuracy of an artificial intelligence decision support system for the detection of cervical spine fractures. Many similar algorithms have also received little or no external validation, and this study raises concerns about their generalizability, utility, and rapid pace of deployment. Further rigorous evaluations are needed to understand the weaknesses of these tools before widespread implementation.

Diagnostic Accuracy and Failure Mode Analysis of a Deep Learning Algorithm for the Detection of Intracranial Hemorrhage

OBJECTIVE

To determine the institutional diagnostic accuracy of an artificial intelligence (AI) decision support systems (DSS), Aidoc, in diagnosing intracranial hemorrhage (ICH) on noncontrast head CTs and to assess the potential generalizability of an AI DSS.

METHODS

This retrospective study included 3,605 consecutive, emergent, adult noncontrast head CT scans performed between July 1, 2019, and December 30, 2019, at our institution (51% female subjects, mean age of 61 ± 21 years). Each scan was evaluated for ICH by both a certificate of added qualification certified neuroradiologist and Aidoc. We determined the diagnostic accuracy of the AI model and performed a failure mode analysis with quantitative CT radiomic image characterization.

RESULTS

Of the 3,605 scans, 349 cases of ICH (9.7% of studies) were identified. The neuroradiologist and Aidoc interpretations were concordant in 96.9% of cases and the overall sensitivity, specificity, positive predictive value, and negative predictive value were 92.3%, 97.7%, 81.3%, and 99.2%, respectively, with positive predictive values unexpectedly lower than in previously reported studies. Prior neurosurgery, type of ICH, and number of ICHs were significantly associated with decreased model performance. Quantitative image characterization with CT radiomics failed to reveal significant differences between concordant and discordant studies.

DISCUSSION

This study revealed decreased diagnostic accuracy of an AI DSS at our institution. Despite extensive evaluation, we were unable to identify the source of this discrepancy, raising concerns about the generalizability of these tools with indeterminate failure modes. These results further highlight the need for standardized study design to allow for rigorous and reproducible site-to-site comparison of emerging deep learning technologies.

Effect of Using an Age-adjusted D-dimer to Assess for Pulmonary Embolism in Community Emergency Departments

BACKGROUND

The objective of this study was to evaluate the effect of changing the laboratory-reported D-dimer reference intervals to age-adjusted reference intervals on the use of advanced chest imaging and 30-day adverse events among emergency department (ED) encounters.

METHODS

A retrospective interrupted time-series analysis of ED encounters for patients > 50 years evaluated for suspected pulmonary embolism (PE) from April 2014 to April 2016. The primary outcome was use of advanced diagnostic imaging, and the secondary outcome was 30-day mortality or PE diagnosis. Secondary analyses also quantified delayed PE diagnoses pre- and postintervention. A generalized estimating equation segmented logistic regression model, adjusting for patient and facility characteristics, was used to determine changes in odds of diagnostic imaging and 30-day mortality or PE diagnoses.

RESULTS

A total of 10,534 (5,153 pre- and 5,381 postimplementation) ED encounters were included. Advanced imaging was obtained in 35.9% of pre- versus 33% of postimplementation encounters. Age-adjusted D-dimer (AADD) showed a small and nonsignificant decrease in month-to-month trends of advanced chest imaging postimplementation (odds ratio [OR] = 0.98, 95% confidence interval [CI] = 0.96 to 1.00). Use of advanced imaging in patients with D-dimer values lower than 500 ng/mL fibrinogen-equivalent units (FEU) was similar in the preintervention (5.8%) and postintervention (6.8%) periods. However, imaging was obtained in 30% of patients postintervention with a D-dimer result less than AADD reference interval , but more than the historical 500 ng/mL FEU reference interval. Implementing an AADD threshold demonstrated no change in the rate of 30-day adverse events (missed PE or mortality).

CONCLUSION

Changing the laboratory-reported D-dimer reference intervals for evaluation of PE was not associated with reduction in advanced chest imaging and did not increase 30-day adverse events. However, there was substantial noncompliance with the age-adjusted reference intervals in the postintervention period likely blunting the impact of this intervention.

Embedded Research in the Learning Healthcare System: Ongoing Challenges and Recommendations for Researchers, Clinicians, and Health System Leaders

Embedded research is an innovative means to improve performance in the learning healthcare system (LHS). However, few descriptions of successful embedded research programs have been published. In this perspective, we describe the Care Improvement Research Team, a mature partnership between researchers and clinicians at Kaiser Permanente Southern California. The program supports a core team of researchers and staff with dedicated resources to partner with health system leaders and practicing clinicians, using diverse methods to identify and rectify gaps in clinical practice. For example, recent projects helped clinicians to provide better care by reducing prescribing of unnecessary antibiotics for acute sinusitis and by preventing readmissions among the elderly. Embedded in operational workgroups, the team helps formulate research questions and enhances the rigor and relevance of data collection and analysis. A recent business-case analysis cited savings to the organization of over $10 million. We conclude that embedded research programs can play a key role in fulfilling the promise of the LHS. Program success depends on dedicated funding, robust data systems, and strong relationships between researchers and clinical stakeholders. Embedded researchers must be responsive to health system priorities and timelines, while clinicians should embrace researchers as partners in problem solving.

Adapting two American Decision Aids for Mild Traumatic Brain Injury to the Canadian Context Using the Nominal Group Technique

BACKGROUND

Decision aids are patient-focused tools that have the potential to reduce the overuse of head computed tomography (CT) scans.

OBJECTIVE

The objective of this study was to create a consensus among Canadian mild traumatic brain injury and emergency medicine experts on modifications required to adapt two American decision aids about head CT use for adult and paediatric mild traumatic brain injury to the Canadian context.

METHODS

We invited 21 Canadian stakeholders and the two authors of the American decision aids to a Nominal Group Technique consensus meeting to generate suggestions for adapting the decision aids. This method encourages idea generation and sharing between team members. Each idea was discussed and then prioritised using a voting system. We collected data using videotaping, writing material and online collaborative writing tools. The modifications proposed were analysed using a qualitative thematic content analysis.

RESULTS

Twenty-one participants took part in the meeting, including researchers and clinician researchers (n = 9; 43%), patient partners (n = 3; 14%) and decision makers (n = 2; 10%). A total of 84 ideas were generated. Participants highlighted the need to clarify the purpose of the decision aids, the nature of the problem being addressed and the target population. The tools require sociocultural adaptations, better identification of their target population, better description of head CT utility, advantages and related risks, modification of the visual and written representation of the risk of brain injury and head CT use, and locally adapted, patient follow-up plans.

CONCLUSIONS

This study based on a Nominal Group Technique identified several adaptations for two American decision aids about head CT use for mild traumatic brain injury to support their use in Canada's different healthcare, social, cultural and legal context. These adaptations concerned the target users of the decision aids, the information presented, and how the benefits and risks were communicated in the decision aids. Future steps include prototyping the two adapted decision aids, conducting formative evaluations with actual emergency department patients and clinicians, and measuring the impact of the adapted tools on CT scan use.

Rates of Intracranial Hemorrhage in Mild Head Trauma Patients Presenting to Emergency Department and Their Management: A Comparison of Direct Oral Anticoagulant Drugs with Vitamin K Antagonists

Background and objectives: Anticoagulants are thought to increase the risks of traumatic intracranial injury and poor clinical outcomes after blunt head trauma. The safety of using direct oral anticoagulants (DOACs) compared to vitamin K antagonists (VKAs) after intracranial hemorrhage (ICH) is unclear. This study aims to compare the incidence of post-traumatic ICH following mild head injury (MHI) and to assess the need for surgery, mortality rates, emergency department (ED) revisit rates, and the volume of ICH. Materials and Methods: This is a retrospective, single-center observational study on all patients admitted to our emergency department for mild head trauma from 1 January 2016, to 31 December 2018. We enrolled 234 anticoagulated patients, of which 156 were on VKAs and 78 on DOACs. Patients underwent computed tomography (CT) scans on arrival (T0) and after 24 h (T24). The control group consisted of patients not taking anticoagulants, had no clotting disorders, and who reported an MHI in the same period. About 54% in the control group had CTs performed. Results: The anticoagulated groups were comparable in baseline parameters. Patients on VKA developed ICH more frequently than patients on DOACs and the control group at 17%, 5.13%, and 7.5%, respectively. No significant difference between the two groups was noted in terms of surgery, intrahospital mortality rates, ED revisit rates, and the volume of ICH. Conclusions: Patients with mild head trauma on DOAC therapy had a similar prevalence of ICH to that of the control group. Meanwhile, patients on VKA therapy had about twice the ICH prevalence than that on the control group or patients on DOAC, which remained after correcting for age. No significant difference in the need for surgery was determined; however, this result must take into account the very small number of patients needing surgery.

Effect of the implementation of a new guideline for minor head injury on computed tomography-ratio and hospitalizations in the Netherlands

OBJECTIVE

A new nationwide guideline for minor head injury was introduced in the Netherlands in 2010. The effect on computed tomography (CT) ratio and hospital admission ratio after introduction of the guideline is unknown. The aim was to reduce these numbers as part of cost-effective health care. Therefore, we assessed the effect on these variables after introduction of the guideline.

METHODS

We used an interrupted time-series study design. Data selection was done 3 years before (2007-2009) and several years after (2012, 2014, 2015) introduction of the guideline.

RESULTS

Data collection was performed for 3880 patients. Introduction of the new guideline was associated with an increase in CT ratio from 24.6% before to 55% after introduction (P < 0.001). This increase is the result of both the new guideline and a secular trend. Besides this, hospital admissions increased from 14.7 to 23.4% (P < 0.001) during the study period. This increase was less clearly associated with the new guideline. After introduction of the guideline there was no significant difference in (intra)cranial traumatic findings (2.6% vs. 3.4%; P = 0.13) and neurosurgical interventions (0.1% vs. 0.2%; P = 0.50).

CONCLUSION

Between 2007 and 2015, a marked increase in CT ratio and hospital admissions has been observed. The increase in CT ratio seems to be caused both by the new guideline and by a secular trend to perform more CT scans. Adaptations to the guideline should be considered to improve patient care and cost-effectiveness in patients with minor head injury.

Checklist for Head Injury Management Evaluation Study (CHIMES): a quality improvement initiative to reduce imaging utilisation for head injuries in the emergency department

Over 90% of patients with head trauma seen in emergency departments (EDs) are diagnosed with minor head injuries. Over-utilisation of CT scans results in unnecessary exposure to radiation and increases healthcare utilisation. Using recommendations from the Choosing Wisely Canada (CWC) campaign and quality improvement (QI) methodology, we aimed to reduce the CT scan rate for head injuries by 10% over a 6-month period.

Baseline CT scan rates were determined through a 27-month retrospective cohort review. We used stakeholder engagement and provider surveys to develop our driver diagram and Plan-Do-Study-Act (PDSA) cycles, which included (1) improving provider knowledge about the CWC campaign recommendations; (2) testing, refining and implementing a modified Canadian CT Head Rule checklist; (3) developing CWC-themed head injury–specific patient handouts; and (4) feedback on CT scan group ordering rates to providers. Our primary outcome measure was the number of CT scans performed for patients with head injuries. Process measures included the number of checklists completed and ED length of stay (LOS). Our balancing measure was return ED visits within 72 hours (with or without admission).

Baseline CT scan rates prior to our interventions was 46.1%. Our QI initiative resulted in a ‘shift’ in the Statistical Process Control chart of the weekly CT scan rates, associated with the first and second PDSA cycles, resulting in a 13.9% reduction in CT rates during the initial 3 months, and a sustained reduction of 8% at 16 months (p<0.05). Mean ED LOS for all patients with head injuries decreased by 1.5 min (p=0.74). 33% of checklists were completed. 72-hour return visits did not change significantly (p=0.68).

Through provider and patient education, and the creation of a user-friendly evidence-based tool, our local QI initiative was successful in achieving long-term reduction in CT rates for patients presenting to EDs with head injuries.

Implementation of Evidence-Based Practice for Benign Paroxysmal Positional Vertigo in the Emergency Department: A Stepped-Wedge Randomized Trial

STUDY OBJECTIVE

We evaluated a strategy to increase use of the test (Dix-Hallpike's test [DHT]) and treatment (canalith repositioning maneuver [CRM]) for benign paroxysmal positional vertigo in emergency department (ED) dizziness visits.

METHODS

We conducted a stepped-wedge randomized trial in 6 EDs. The population was visits with dizziness as a principal reason for the visit. The intervention included educational sessions and decision aid materials. Outcomes were DHT or CRM documentation (primary), head computed tomography (CT) use, length of stay, admission, and 90-day stroke events. The analysis was multilevel logistic regression with intervention, month, and hospital as fixed effects and provider as a random effect. We assessed fidelity with monitoring intervention use and semistructured interviews.

RESULTS

We identified 7,635 dizziness visits during 18 months. The DHT or CRM was documented in 1.5% of control visits (45/3,077; 95% confidence interval 1% to 1.9%) and 3.5% of intervention visits (159/4,558; 95% confidence interval 3% to 4%; difference 2%, 95% confidence interval 1.3% to 2.7%). Head CT use was lower in intervention visits compared with control visits (44.0% [1,352/3,077] versus 36.9% [1,682/4,558]). No differences were observed in admission or 90-day subsequent stroke risk. In fidelity evaluations, providers who used the materials typically reported positive clinical experiences but provider engagement was low at facilities without an emergency medicine residency program.

CONCLUSION

These findings provide evidence that an implementation strategy of a benign paroxysmal positional vertigo-focused approach to ED dizziness visits can be successful and safe in promoting evidence-based care. Absolute rates of DHT and CRM use, however, were still low, which relates in part to our broad inclusion criteria for dizziness visits.

Computerized Clinical Decision Support System for Emergency Department-Initiated Buprenorphine for Opioid Use Disorder: User-Centered Design

BACKGROUND

Emergency departments (EDs) frequently care for individuals with opioid use disorder (OUD). Buprenorphine (BUP) is an effective treatment option for patients with OUD that can safely be initiated in the ED. At present, BUP is rarely initiated as a part of routine ED care. Clinical decision support (CDS) could accelerate adoption of ED-initiated BUP into routine emergency care.

OBJECTIVE

This study aimed to design and formatively evaluate a user-centered decision support tool for ED initiation of BUP for patients with OUD.

METHODS

User-centered design with iterative prototype development was used. Initial observations and interviews identified workflows and information needs. The design team and key stakeholders reviewed prototype designs to ensure accuracy. A total of 5 prototypes were evaluated and iteratively refined based on input from 26 attending and resident physicians.

RESULTS

Early feedback identified concerns with the initial CDS design: an alert with several screens. The timing of the alert led to quick dismissal without using the tool. User feedback on subsequent iterations informed the development of a flexible tool to support clinicians with varied levels of experience with the intervention by providing both one-click options for direct activation of care pathways and user-activated support for critical decision points. The final design resolved challenging navigation issues through targeted placement, color, and design of the decision support modules and care pathways. In final testing, users expressed that the tool could be easily learned without training and was reasonable for use during routine emergency care.

CONCLUSIONS

A user-centered design process helped designers to better understand users' needs for a Web-based clinical decision tool to support ED initiation of BUP for OUD. The process identified varying needs across user experience and familiarity with the protocol, leading to a flexible design supporting both direct care pathways and user-initiated decision support.

Effect of a HEART Care Pathway on Chest Pain Management Within an Integrated Health System

STUDY OBJECTIVE

We describe the association of implementing a History, ECG, Age, Risk Factors, and Troponin (HEART) care pathway on use of hospital care and noninvasive stress testing, as well as 30-day patient outcomes in community emergency departments (EDs).

METHODS

We performed a prospective interrupted-time-series study of adult encounters for patients evaluated for suspected acute coronary syndrome. The primary outcome was hospitalization or observation, noninvasive stress testing, or both within 30 days. The secondary outcome was 30-day all-cause mortality or acute myocardial infarction. A generalized estimating equation segmented logistic regression model was used to compare the odds of the primary outcome before and after HEART implementation. All models were adjusted for patient and facility characteristics and fit with physicians as a clustering variable.

RESULTS

A total of 65,393 ED encounters (before, 30,522; after, 34,871) were included in the study. Overall, 33.5% (before, 35.5%; after, 31.8%) of ED chest pain encounters resulted in hospitalization or observation, noninvasive stress testing, or both. Primary adjusted results found a significant decrease in the primary outcome postimplementation (odds ratio 0.984; 95% confidence interval [CI] 0.974 to 0.995). This resulted in an absolute adjusted month-to-month decrease of 4.39% (95% CI 3.72% to 5.07%) after 12 months' follow-up, with a continued trend downward. There was no difference in 30-day mortality or myocardial infarction (0.6% [before] versus 0.6% [after]; odds ratio 1.02; 95% CI 0.97 to 1.08).

CONCLUSION

Implementation of a HEART pathway in the ED evaluation of patients with chest pain resulted in less inpatient care and noninvasive cardiac testing and was safe. Using HEART to risk stratify chest pain patients can improve the efficiency and quality of care.

Scope and Influence of Electronic Health Record-Integrated Clinical Decision Support in the Emergency Department: A Systematic Review

STUDY OBJECTIVE

As electronic health records evolve, integration of computerized clinical decision support offers the promise of sorting, collecting, and presenting this information to improve patient care. We conducted a systematic review to examine the scope and influence of electronic health record-integrated clinical decision support technologies implemented in the emergency department (ED).

METHODS

A literature search was conducted in 4 databases from their inception through January 18, 2018: PubMed, Scopus, the Cumulative Index of Nursing and Allied Health, and Cochrane Central. Studies were included if they examined the effect of a decision support intervention that was implemented in a comprehensive electronic health record in the ED setting. Standardized data collection forms were developed and used to abstract study information and assess risk of bias.

RESULTS

A total of 2,558 potential studies were identified after removal of duplicates. Of these, 42 met inclusion criteria. Common targets for clinical decision support intervention included medication and radiology ordering practices, as well as more comprehensive systems supporting diagnosis and treatment for specific disease entities. The majority of studies (83%) reported positive effects on outcomes studied. Most studies (76%) used a pre-post experimental design, with only 3 (7%) randomized controlled trials.

CONCLUSION

Numerous studies suggest that clinical decision support interventions are effective in changing physician practice with respect to process outcomes such as guideline adherence; however, many studies are small and poorly controlled. Future studies should consider the inclusion of more specific information in regard to design choices, attempt to improve on uncontrolled before-after designs, and focus on clinically relevant outcomes wherever possible.

Tablet-Based Patient-Centered Decision Support for Minor Head Injury in the Emergency Department: Pilot Study

Background

The Concussion or Brain Bleed app is a clinician- and patient-facing electronic tool to guide decisions about head computed tomography (CT) use in patients presenting to the emergency department (ED) with minor head injury. This app integrates a patient decision aid and clinical decision support (using the Canadian CT Head Rule, CCHR) at the bedside on a tablet computer to promote conversations around individualized risk and patients’ specific concerns within the ED context.

Objective

The objective of this study was to describe the use of the Concussion or Brain Bleed app in a high-volume ED and to establish preliminary efficacy estimates on patient experience, clinician experience, health care utilization, and patient safety. These data will guide the planning of a larger multicenter trial testing the effectiveness of the Concussion or Brain Bleed app.

Methods

We conducted a prospective pilot study of adult (age 18-65 years) patients presenting to the ED after minor head injury who were identified by participating clinicians as low risk by the CCHR. The primary outcome was patient knowledge regarding the injury, risks, and CT use. Secondary outcomes included patient satisfaction, decisional conflict, trust in physician, clinician acceptability, system usability, Net Promoter scores, head CT rate, and patient safety at 7 days.

Results

We enrolled 41 patients cared for by 29 different clinicians. Patient knowledge increased after the use of the app (questions correct out of 9: pre-encounter, 3.3 vs postencounter, 4.7; mean difference 1.4, 95% CI 0.8-2.0). Patients reported a mean of 11.7 (SD 13.5) on the Decisional Conflict Scale and 92.5 (SD 12.0) in the Trust in Physician Scale (both scales range from 0 to 100). Most patients were satisfied with the app’s clarity of information (35, 85%), helpfulness of information (36, 88%), and amount of information (36, 88%). In the 41 encounters, most clinicians thought the information was somewhat or extremely helpful to the patient (35, 85%), would want to use something similar for other decisions (27, 66%), and would recommend the app to other providers (28, 68%). Clinicians reported a mean system usability score of 85.1 (SD 15; scale from 0 to 100 with 85 in the “excellent” acceptability range). The total Net Promoter Score was 36.6 (on a scale from –100 to 100). A total of 7 (17%) patients received a head CT in the ED. No patients had a missed clinically important brain injury at 7 days.

Conclusions

An app to help patients assess the utility of CT imaging after head injury in the ED increased patient knowledge. Nearly all clinicians reported the app to be helpful to patients. The high degree of patient satisfaction, clinician acceptability, and system usability support rigorous testing of the app in a larger multicenter trial.