Automated tracking of emergency department abdominal CT findings during the COVID-19 pandemic using natural language processing

Orthopedic Surgery Volume Trends During the COVID-19 Pandemic and Postvaccination Era: Implications for Healthcare Planning

BACKGROUND

The Coronavirus Disease 2019 (COVID-19) pandemic decreased surgical volumes, but prior studies have not investigated recovery through 2022, or analyzed specific procedures or cases of urgency within orthopedic surgery. The aims of this study were to (1) quantify the declines in orthopedic surgery volume during and after the pandemic peak, (2) characterize surgical volume recovery during the postvaccination period, and (3) characterize recovery in the 1-year postvaccine release period.

METHODS

We conducted a retrospective cohort study of 27,476 orthopedic surgeries from January 2019 to December 2022 at one urban academic quaternary referral center. We reported trends over the following periods: baseline pre-COVID-19 period (1/6/2019 to 1/4/2020), COVID-19 peak (3/15/2020 to 5/16/2020), post-COVID-19 peak (5/17/2020 to 1/2/2021), postvaccine release (1/3/2021 to 1/1/2022), and 1-year postvaccine release (1/2/2022 to 12/30/2022). Comparisons were performed with 2 sample t-tests.

RESULTS

Pre-COVID-19 surgical volume fell by 72% at the COVID-19 peak, especially impacting elective procedures (P < .001) and both hip and knee joint arthroplasty (P < .001) procedures. Nonurgent (P = .024) and urgent or emergency (P = .002) cases also significantly decreased. Postpeak recovery before the vaccine saw volumes rise to 92% of baseline, which further rose to 96% and 94% in 2021 and 2022, respectively. While elective procedures surpassed the baseline in 2022, nonurgent and urgent or emergency surgeries remained low.

CONCLUSIONS

The COVID-19 pandemic substantially reduced orthopedic surgical volumes, which have still not fully recovered through 2022, particularly nonelective procedures. The differential recovery within an orthopedic surgery program may result in increased morbidity and can serve to inform department-level operational recovery.

Using the Textual Content of Radiological Reports to Detect Emerging Diseases: A Proof-of-Concept Study of COVID-19

Changes in the content of radiological reports at population level could detect emerging diseases. Herein, we developed a method to quantify similarities in consecutive temporal groupings of radiological reports using natural language processing, and we investigated whether appearance of dissimilarities between consecutive periods correlated with the beginning of the COVID-19 pandemic in France. CT reports from 67,368 consecutive adults across 62 emergency departments throughout France between October 2019 and March 2020 were collected. Reports were vectorized using time frequency-inverse document frequency (TF-IDF) analysis on one-grams. For each successive 2-week period, we performed unsupervised clustering of the reports based on TF-IDF values and partition-around-medoids. Next, we assessed the similarities between this clustering and a clustering from two weeks before according to the average adjusted Rand index (AARI). Statistical analyses included (1) cross-correlation functions (CCFs) with the number of positive SARS-CoV-2 tests and advanced sanitary index for flu syndromes (ASI-flu, from open-source dataset), and (2) linear regressions of time series at different lags to understand the variations of AARI over time. Overall, 13,235 chest CT reports were analyzed. AARI was correlated with ASI-flu at lag = + 1, + 5, and + 6 weeks (P = 0.0454, 0.0121, and 0.0042, respectively) and with SARS-CoV-2 positive tests at lag = - 1 and 0 week (P = 0.0057 and 0.0001, respectively). In the best fit, AARI correlated with the ASI-flu with a lag of 2 weeks (P = 0.0026), SARS-CoV-2-positive tests in the same week (P < 0.0001) and their interaction (P < 0.0001) (adjusted R2 = 0.921). Thus, our method enables the automatic monitoring of changes in radiological reports and could help capturing disease emergence.

How the COVID-19 pandemic affected the severity and clinical presentation of diverticulitis

BACKGROUND/AIMS

Single-institution studies showed that patients presented with more severe diverticulitis and underwent more emergency operations during the coronavirus disease 2019 (COVID-19) pandemic. Therefore, we studied this trend using nationwide data from the American College of Surgeons National Surgical Quality Improvement Program database.

METHODS

Patients (n = 23,383) who underwent a colectomy for diverticulitis in 2018 (control year) and 2020 (pandemic year) were selected. We compared these groups for differences in disease severity, comorbidities, perioperative factors, and complications.

RESULTS

During the pandemic, colonic operations for diverticulitis decreased by 13.14%, but the rates of emergency operations (17.31% vs. 20.04%, P< 0.001) and cases with a known abscess/perforation (50.11% vs. 54.55%, P< 0.001) increased. Likewise, the prevalence of comorbidities, such as congestive heart failure, acute renal failure, systemic inflammatory response syndrome, and septic shock, were higher during the pandemic (P< 0.05). During this same period, significantly more patients were classified under American Society of Anesthesiologists classes 3, 4, and 5, suggesting their preoperative health states were more severe and life-threatening. Correspondingly, the average operation time was longer (P< 0.001) and complications, such as organ space surgical site infection, wound disruption, pneumonia, acute renal failure, septic shock, and myocardial infarction, increased (P< 0.05) during the pandemic.

CONCLUSIONS

During the pandemic, surgical volume decreased, but the clinical presentation of diverticulitis became more severe. Due to resource reallocation and possibly patient fear of seeking medical attention, diverticulitis was likely underdiagnosed, and cases that would have been elective became emergent. This underscores the importance of monitoring patients at risk for diverticulitis and intervening when criteria for surgery are met.

Evaluating GPT as an Adjunct for Radiologic Decision Making: GPT-4 Versus GPT-3.5 in a Breast Imaging Pilot

OBJECTIVE

Despite rising popularity and performance, studies evaluating the use of large language models for clinical decision support are lacking. Here, we evaluate ChatGPT (Generative Pre-trained Transformer)-3.5 and GPT-4's (OpenAI, San Francisco, California) capacity for clinical decision support in radiology via the identification of appropriate imaging services for two important clinical presentations: breast cancer screening and breast pain.

METHODS

We compared ChatGPT's responses to the ACR Appropriateness Criteria for breast pain and breast cancer screening. Our prompt formats included an open-ended (OE) and a select all that apply (SATA) format. Scoring criteria evaluated whether proposed imaging modalities were in accordance with ACR guidelines. Three replicate entries were conducted for each prompt, and the average of these was used to determine final scores.

RESULTS

Both ChatGPT-3.5 and ChatGPT-4 achieved an average OE score of 1.830 (out of 2) for breast cancer screening prompts. ChatGPT-3.5 achieved a SATA average percentage correct of 88.9%, compared with ChatGPT-4's average percentage correct of 98.4% for breast cancer screening prompts. For breast pain, ChatGPT-3.5 achieved an average OE score of 1.125 (out of 2) and a SATA average percentage correct of 58.3%, as compared with an average OE score of 1.666 (out of 2) and a SATA average percentage correct of 77.7%.

DISCUSSION

Our results demonstrate the eventual feasibility of using large language models like ChatGPT for radiologic decision making, with the potential to improve clinical workflow and responsible use of radiology services. More use cases and greater accuracy are necessary to evaluate and implement such tools.

Transformer versus traditional natural language processing: how much data is enough for automated radiology report classification?

OBJECTIVES

Current state-of-the-art natural language processing (NLP) techniques use transformer deep-learning architectures, which depend on large training datasets. We hypothesized that traditional NLP techniques may outperform transformers for smaller radiology report datasets.

METHODS

We compared the performance of BioBERT, a deep-learning-based transformer model pre-trained on biomedical text, and three traditional machine-learning models (gradient boosted tree, random forest, and logistic regression) on seven classification tasks given free-text radiology reports. Tasks included detection of appendicitis, diverticulitis, bowel obstruction, and enteritis/colitis on abdomen/pelvis CT reports, ischemic infarct on brain CT/MRI reports, and medial and lateral meniscus tears on knee MRI reports (7,204 total annotated reports). The performance of NLP models on held-out test sets was compared after training using the full training set, and 2.5%, 10%, 25%, 50%, and 75% random subsets of the training data.

RESULTS

In all tested classification tasks, BioBERT performed poorly at smaller training sample sizes compared to non-deep-learning NLP models. Specifically, BioBERT required training on approximately 1,000 reports to perform similarly or better than non-deep-learning models. At around 1,250 to 1,500 training samples, the testing performance for all models began to plateau, where additional training data yielded minimal performance gain.

CONCLUSIONS

With larger sample sizes, transformer NLP models achieved superior performance in radiology report binary classification tasks. However, with smaller sizes (<1000) and more imbalanced training data, traditional NLP techniques performed better.

ADVANCES IN KNOWLEDGE

Our benchmarks can help guide clinical NLP researchers in selecting machine-learning models according to their dataset characteristics.

Effect of the COVID-19 pandemic on emergency department utilization of computed tomography scans of appendicitis and diverticulitis

PURPOSE

Investigating the effect of the COVID-19 lockdown on adult patient visits, computed tomography (CT) abdominal scans, and presentations of appendicitis and diverticulitis, to emergency departments (ED) in St. John's NL.

METHODS

A retrospective quantitative analysis was applied, using ED visits and Canadian Triage and Acuity Scale (CTAS) scores. mPower (Nuance Communications, UK) identified CT abdominal scan reports, which were categorized into (1) normal/other, (2) appendicitis, or (3) diverticulitis. Time intervals included pre-lockdown (January-February), lockdown (March-June), and post-lockdown (July-August). Data from 2018 to 2019 (January-August) were used to generate expected patient volumes for 2020, and pre- and post-lockdown were included to control for other variables outside the lockdown.

RESULTS

Chi-squared goodness of fit tested for deviations from predicted means for 2018-2019. Compared to expectations, daily ED visits from January to August 2020 showed a significant (p < 0.001) decrease in patient volumes independent of gender, age, and CTAS scores. During and post-lockdown, CT abdominal scans did not drop in proportion to patient volume. Appendicitis presentations remained indifferent to lockdown, while diverticulitis presentations appeared to wane, with no difference in combined complicated cases in comparison to what was expected.

CONCLUSION

During lockdown, significantly fewer patients presented to the ED. The proportion of ordered CT abdominal scans increased significantly per person seen, without change in CTAS scores. Considering combined pathology cases increased during the lockdown, ED physicians were warranted in increasing abdominal imaging as patients did not avoid the ED. This may have resulted from a change in clinical practice where the uncertainty of COVID-19 increased CT scan usage.

Trends in oncological imaging during the COVID-19 pandemic through the vaccination era

BACKGROUND

This study examines the impact that the COVID-19 pandemic has had on computed tomography (CT)-based oncologic imaging utilization.

METHODS

We retrospectively analyzed cancer-related CT scans during four time periods: pre-COVID (1/5/20-3/14/20), COVID peak (3/15/20-5/2/20), post-COVID peak (5/3/20-12/19/20), and vaccination period (12/20/20-10/30/21). We analyzed CTs by imaging indication, setting, and hospital type. Using percentage decrease computation and Student's t-test, we calculated the change in mean number of weekly cancer-related CTs for all periods compared to the baseline pre-COVID period. This study was performed at a single academic medical center and three affiliated hospitals.

RESULTS

During the COVID peak, mean CTs decreased (-43.0%, p < 0.001), with CTs for (1) cancer screening, (2) initial workup, (3) cancer follow-up, and (4) scheduled surveillance of previously treated cancer dropping by 81.8%, 56.3%, 31.7%, and 45.8%, respectively (p < 0.001). During the post-COVID peak period, cancer screenings and initial workup CTs did not return to prepandemic imaging volumes (-11.4%, p = 0.028; -20.9%, p = 0.024). The ED saw increases in weekly CTs compared to prepandemic levels (+31.9%, p = 0.008), driven by increases in cancer follow-up CTs (+56.3%, p < 0.001). In the vaccination period, cancer screening CTs did not recover to baseline (-13.5%, p = 0.002) and initial cancer workup CTs doubled (+100.0%, p < 0.001). The ED experienced increased cancer-related CTs (+75.9%, p < 0.001), driven by cancer follow-up CTs (+143.2%, p < 0.001) and initial workups (+46.9%, p = 0.007).

CONCLUSIONS AND RELEVANCE

The pandemic continues to impact cancer care. We observed significant declines in cancer screening CTs through the end of 2021. Concurrently, we observed a 2× increase in initial cancer workup CTs and a 2.4× increase in cancer follow-up CTs in the ED during the vaccination period, suggesting a boom of new cancers and more cancer examinations associated with emergency level acute care.

Trends in Academic Productivity Among Radiologists During the COVID-19 Pandemic

PURPOSE

There is a scarcity of literature examining changes in radiologist research productivity during the COVID-19 pandemic. The current study aimed to investigate changes in academic productivity as measured by publication volume before and during the COVID-19 pandemic.

METHODS

This single-center, retrospective cohort study included the publication data of 216 researchers consisting of associate professors, assistant professors, and professors of radiology. Wilcoxon's signed-rank test was used to identify changes in publication volume between the 1-year-long defined prepandemic period (publications between May 1, 2019, and April 30, 2020) and COVID-19 pandemic period (May 1, 2020, to April 30, 2021).

RESULTS

There was a significantly increased mean annual volume of publications in the pandemic period (5.98, SD = 7.28) compared with the prepandemic period (4.98, SD = 5.53) (z = -2.819, P = .005). Subset analysis demonstrated a similar (17.4%) increase in publication volume for male researchers when comparing the mean annual prepandemic publications (5.10, SD = 5.79) compared with the pandemic period (5.99, SD = 7.60) (z = -2.369, P = .018). No statistically significant changes were found in similar analyses with the female subset.

DISCUSSION

Significant increases in radiologist publication volume were found during the COVID-19 pandemic compared with the year before. Changes may reflect an overall increase in academic productivity in response to clinical and imaging volume ramp down.

Institutional Surgical Response and Associated Volume Trends Throughout the COVID-19 Pandemic and Postvaccination Recovery Period

Importance

The COVID-19 pandemic is associated with decreased surgical procedure volumes, but existing studies have not investigated this association beyond the end of 2020, analyzed changes during the post-vaccine release period, or quantified these changes by patient acuity.

Objective

To quantify changes in the volume of surgical procedures at a 1017-bed academic quaternary care center from January 6, 2019, to December 31, 2021.

Design, Setting, and Participants

In this cohort study, 129 596 surgical procedure volumes were retrospectively analyzed during 4 periods: pre-COVID-19 (January 6, 2019, to January 4, 2020), COVID-19 peak (March 15, 2020, to May 2, 2020), post-COVID-19 peak (May 3, 2020, to January 2, 2021), and post-vaccine release (January 3, 2021, to December 31, 2021). Surgery volumes were analyzed by subspecialty and case class (elective, emergent, nonurgent, urgent). Statistical analysis was by autoregressive integrated moving average modeling.

Main Outcomes and Measures

The primary outcome of this study was the change in weekly surgical procedure volume across the 4 COVID-19 periods.

Results

A total of 129 596 records of surgical procedures were reviewed. During the COVID-19 peak, overall weekly surgical procedure volumes (mean [SD] procedures per week, 406.00 [171.45]; 95% CI, 234.56-577.46) declined 44.6% from pre-COVID-19 levels (mean [SD] procedures per week, 732.37 [12.70]; 95% CI, 719.67-745.08; P < .001). This weekly volume decrease occurred across all surgical subspecialties. During the post-COVID peak period, overall weekly surgical volumes (mean [SD] procedures per week, 624.31 [142.45]; 95% CI, 481.85-766.76) recovered to only 85.8% of pre-COVID peak volumes (P < .001). This insufficient recovery was inconsistent across subspecialties and case classes. During the post-vaccine release period, although some subspecialties experienced recovery to pre-COVID-19 volumes, others continued to experience declines.

Conclusions and Relevance

This quaternary care institution effectively responded to the pressures of the COVID-19 pandemic by substantially decreasing surgical procedure volumes during the peak of the pandemic. However, overall surgical procedure volumes did not fully recover to pre-COVID-19 levels well into 2021, with inconsistent recovery rates across subspecialties and case classes. These declines suggest that delays in surgical procedures may result in potentially higher morbidity rates in the future. The differential recovery rates across subspecialties may inform institutional focus for future operational recovery.