Validation of an administrative claims coding algorithm for serious opioid overdose: A medical chart review

Perioperative opioid prescribing and iatrogenic opioid use disorder and overdose: a state-of-the-art narrative review

BACKGROUND/IMPORTANCE

Considerable attention has been paid to identifying and mitigating perioperative opioid-related harms. However, rates of postsurgical opioid use disorder (OUD) and overdose, along with associated risk factors, have not been clearly defined.

OBJECTIVE

Evaluate the evidence connecting perioperative opioid prescribing with postoperative OUD and overdose, compare these data with evidence from the addiction literature, discuss the clinical impact of these conditions, and make recommendations for further study.

EVIDENCE REVIEW

State-of-the-art narrative review.

FINDINGS

Nearly all evidence is from large retrospective studies of insurance claims and Veterans Health Administration (VHA) data. Incidence rates of new OUD within the first year after surgery ranged from 0.1% to 0.8%, while rates of overdose events ranged from 0.01% to 0.8%. Higher rates were seen among VHA patients, which may reflect differences in data completeness and/or risk factors. Identified risk factors included those related to substance use (preoperative opioid use; non-opioid substance use disorders; preoperative sedative, anxiolytic, antidepressant, and gabapentinoid use; and postoperative new persistent opioid use (NPOU)); demographic attributes (chiefly male sex, younger age, white race, and Medicaid or no insurance coverage); psychiatric comorbidities such as depression, bipolar disorder, and PTSD; and certain medical and surgical factors. Several challenges related to the use of administrative claims data were identified; there is a need for more granular retrospective studies and, ideally, prospective cohorts to assess postoperative OUD and overdose incidence with greater accuracy.

CONCLUSIONS

Retrospective data suggest an incidence of new postoperative OUD and overdose of up to 0.8% during the first year after surgery, but prospective studies are lacking.

Evaluating Large Language Models in Extracting Cognitive Exam Dates and Scores

Importance

Large language models (LLMs) are crucial for medical tasks. Ensuring their reliability is vital to avoid false results. Our study assesses two state-of-the-art LLMs (ChatGPT and LlaMA-2) for extracting clinical information, focusing on cognitive tests like MMSE and CDR.

Objective

Evaluate ChatGPT and LlaMA-2 performance in extracting MMSE and CDR scores, including their associated dates.

Methods

Our data consisted of 135,307 clinical notes (Jan 12th, 2010 to May 24th, 2023) mentioning MMSE, CDR, or MoCA. After applying inclusion criteria 34,465 notes remained, of which 765 underwent ChatGPT (GPT-4) and LlaMA-2, and 22 experts reviewed the responses. ChatGPT successfully extracted MMSE and CDR instances with dates from 742 notes. We used 20 notes for fine-tuning and training the reviewers. The remaining 722 were assigned to reviewers, with 309 each assigned to two reviewers simultaneously. Inter-rater-agreement (Fleiss' Kappa), precision, recall, true/false negative rates, and accuracy were calculated. Our study follows TRIPOD reporting guidelines for model validation.

Results

For MMSE information extraction, ChatGPT (vs. LlaMA-2) achieved accuracy of 83% (vs. 66.4%), sensitivity of 89.7% (vs. 69.9%), true-negative rates of 96% (vs 60.0%), and precision of 82.7% (vs 62.2%). For CDR the results were lower overall, with accuracy of 87.1% (vs. 74.5%), sensitivity of 84.3% (vs. 39.7%), true-negative rates of 99.8% (98.4%), and precision of 48.3% (vs. 16.1%). We qualitatively evaluated the MMSE errors of ChatGPT and LlaMA-2 on double-reviewed notes. LlaMA-2 errors included 27 cases of total hallucination, 19 cases of reporting other scores instead of MMSE, 25 missed scores, and 23 cases of reporting only the wrong date. In comparison, ChatGPT's errors included only 3 cases of total hallucination, 17 cases of wrong test reported instead of MMSE, and 19 cases of reporting a wrong date.

Conclusions

In this diagnostic/prognostic study of ChatGPT and LlaMA-2 for extracting cognitive exam dates and scores from clinical notes, ChatGPT exhibited high accuracy, with better performance compared to LlaMA-2. The use of LLMs could benefit dementia research and clinical care, by identifying eligible patients for treatments initialization or clinical trial enrollments. Rigorous evaluation of LLMs is crucial to understanding their capabilities and limitations.

Comparison of different sustained-release opioids and acute respiratory conditions in patients with cancer and chronic kidney disease

STUDY OBJECTIVE

Few data are available on the association between the use of oxycodone in patients with chronic kidney disease (CKD) and acute respiratory conditions. The aim of this study was to investigate whether oxycodone is associated with an increased risk of acute respiratory conditions in patients with cancer and CKD compared with other opioids.

DESIGN AND SETTING

The data were obtained from a claims database in Japan. Patients with cancer and CKD who had received sustained-release opioids, including oral oxycodone, oral morphine, or transdermal fentanyl, between April 2014 and May 2021 were selected. The primary outcome was defined as an acute respiratory condition. Data for age and sex, morphine equivalent daily dose, concomitant use of specified medications, comorbidities defined based on the modified Charlson comorbidity index, substance use disorder, and lung cancer or metastatic lung cancer were investigated as covariates. Distribution of acute respiratory conditions was compared among the three sustained-release opioid groups using the log-rank test. Estimates of the incidence of acute respiratory conditions were compared among the groups using a Cox proportional hazards model with time-varying variables.

MAIN RESULTS

A significant difference in the distribution of acute respiratory conditions was found among the three groups (p < 0.01). Cox regression analysis showed a significantly higher risk of acute respiratory conditions with morphine (hazard ratio [HR]: 3.04, 95% confidence interval [CI]: 1.07-8.65, p = 0.04) compared with oxycodone but no significant difference in risk with oxycodone (HR 0.67, 95% CI: 0.32-1.38, p = 0.27) compared with fentanyl.

CONCLUSIONS

The findings suggest that the risk of acute respiratory conditions may be lower in patients with CKD who use oxycodone for cancer pain than in those who use morphine. Additionally, no difference in the risk of acute respiratory conditions was found between oxycodone and fentanyl use.

Health Care Utilization and Costs in Systemic Therapies for Metastatic Melanoma from 2016 to 2020

BACKGROUND

Widespread implementation of immune checkpoint inhibitors (ICI) and targeted therapies for metastatic melanoma has led to a decline in melanoma-related mortality but increased healthcare costs. We aimed to determine how healthcare utilization varied by systemic, non-adjuvant melanoma treatment from 2016 to 2020.

PATIENTS AND METHODS

Adults with presumed stage IV metastatic melanoma receiving systemic therapy from 2016 to 2020 were identified in Optum, a nationwide commercial claims database. Treatment groups were nivolumab, pembrolizumab, ipilimumab+nivolumab (combination-ICI), or BRAF+MEK inhibitor (BRAFi+MEKi) therapy. Outcomes included hospitalizations, days hospitalized, emergency room (ER) visits, outpatient visits, and healthcare costs per patient per month (pppm). Multivariable regression models were used to analyze whether cost and utilization outcomes varied by treatment group, with nivolumab as reference.

RESULTS

Among 2018 adult patients with metastatic melanoma identified, mean (SD) age was 67 (15) years. From 2016 to 2020, nivolumab surpassed pembrolizumab as the most prescribed systemic melanoma therapy while combination-ICI and BRAFi+MEKi therapies remained stable. Relative to nivolumab, all other therapies were associated with increased total healthcare costs (combination-ICI: β = $47 600 pppm, 95%CI $42 200-$53 100; BRAFi+MEKi: β = $3810, 95%CI $365-$7260; pembrolizumab: β = $6450, 95%CI $4420-$8480). Combination-ICI and BRAFi+MEKi therapies were associated with more inpatient hospital days.

CONCLUSIONS

Amid the evolving landscape of systemic therapy for advanced melanoma, nivolumab monotherapy emerged as the most used and least costly systemic treatment from 2016 to 2020. Its sharp increase in use in 2018 and lower costs relative to pembrolizumab may in part be due to earlier adoption of less frequent dosing intervals.

Association of Common Pediatric Surgeries With New Onset Chronic Pain in Patients 0-21 Years of Age in the United States

Chronic pain (CP) is a major public health issue. While new onset CP is known to occur frequently after some pediatric surgeries, its incidence after the most common pediatric surgeries is unknown. This retrospective cohort study used insurance claims data from 2002 to 2017 for patients 0 to 21 years of age. The primary outcome was CP 90 to 365 days after each of the 20 most frequent surgeries in 5 age categories (identified using CP ICD codes). Multivariable logistic regression identified surgeries and risk factors associated with CP after surgery. A total of 424,590 surgical patients aged 0 to 21 were included, 22,361 of whom developed CP in the 90 to 365 days after surgery. The incidences of CP after surgery were: 1.1% in age group 0 to 1 years; 3.0% in 2 to 5 years; 5.6% in 6 to 11 years; 10.1% in 12 to 18 years; 9.9% in 19 to 21 years. Some surgeries and patient variables were associated with CP. Approximately 1 in 10 adolescents who underwent the most common surgeries developed CP, as did a striking percentage of children in other age groups. Given the long-term consequences of CP, resources should be allocated toward identification of high-risk pediatric patients and strategies to prevent CP after surgery. PERSPECTIVE: This study identifies the incidences of and risk factors for chronic pain after common surgeries in patients 0 to 21 years of age. Our findings suggest that resources should be allocated toward the identification of high-risk pediatric patients and strategies to prevent CP after surgery.

Sensitivity of health records for self-reported nonfatal drug and alcohol overdose

BACKGROUND AND OBJECTIVES

Public health surveillance for overdose sometimes depends on nonfatal drug overdoses recorded in health records. However, the proportion of total overdoses identified through health record systems is unclear. Comparison of overdoses from health records to those that are self-reported may provide insight on the proportion of nonfatal overdoses that are not identified.

METHODS

We conducted a cohort study linking survey data on overdose from a national survey of Veterans to United States Department of Veterans Affairs (VA) health records, including community care paid for by VA. Self-reported overdose in the prior 3 years was compared to diagnostic codes for overdoses and substance use disorders in the same time period.

RESULTS

The sensitivity of diagnostic codes for overdose, compared to self-report as a reference standard for this analysis, varied by substance: 28.1% for alcohol, 23.1% for sedatives, 12.0% for opioids, and 5.5% for cocaine. There was a notable concordance between substance use disorder diagnoses and self-reported overdose (sensitivity range 17.9%-90.6%).

DISCUSSION AND CONCLUSIONS

Diagnostic codes in health records may not identify a substantial proportion of drug overdoses. A health record diagnosis of substance use disorder may offer a stronger inference regarding the size of the population at risk. Alternatively, screening for self-reported overdose in routine clinical care could enhance overdose surveillance and targeted intervention.

SCIENTIFIC SIGNIFICANCE

This study suggests that diagnostic codes for overdose are insensitive. These findings support consideration of alternative approaches to overdose surveillance in public health.

Automatically identifying opioid use disorder in non-cancer patients on chronic opioid therapy

Background: Using the International Classification of Diseases (ICD) codes alone to record opioid use disorder (OUD) may not completely document OUD in the electronic health record (EHR). We developed and evaluated natural language processing (NLP) approaches to identify OUD from the clinal note. We explored the concordance between ICD-coded and NLP-identified OUD.Methods: We studied EHRs from 13,654 (female: 8223; male: 5431) adult non-cancer patients who received chronic opioid therapy (COT) and had at least one clinical note between 2013 and 2018. Of eligible patients, we randomly selected 10,218 (75%) patients as the training set and the remaining 3436 patients (25%) as the test dataset for NLP approaches.Results: We generated 539 terms representing OUD mentions in clinical notes (e.g., "opioid use disorder," "opioid abuse," "opioid dependence," "opioid overdose") and 73 terms representing OUD medication treatments. By domain expert manual review for the test dataset, our NLP approach yielded high performance: 98.5% for precision, 100% for recall, and 99.2% for F-measure. The concordance of these NLP and ICD identified OUD was modest (Kappa = 0.63).Conclusions: Our NLP approach can accurately identify OUD patients from clinical notes. The combined use of ICD diagnostic code and NLP approach can improve OUD identification.

ChartSweep: A HIPAA-compliant Tool to Automate Chart Review for Plastic Surgery Research

Retrospective chart review (RCR) is the process of manual patient data review to answer research questions. Large and heterogeneous datasets make the RCR process time-consuming, with potential to introduce errors. The authors therefore designed and developed ChartSweep to expedite the RCR process while remaining faithful to its methodological rigor. ChartSweep is an open-source tool that can be customized for use with any electronic health record system. ChartSweep was developed by the authors to extract information from electronic health records using the Python coding language. As proof-of-concept, the tool was tested in three studies: RCR1-Identification of subjects who underwent radiofrequency ablation in a cohort of patients who had undergone headache surgery (n = 172); RCR2-Identification of patients with a diagnosis of thoracic outlet syndrome in patients who underwent peripheral neuroplasty (n = 806); RCR3-Identification of patients with a history of implant illness or breast implant-associated anaplastic large cell lymphoma in patients who had undergone implant-based breast augmentation or reconstruction (n = 1133). Inter-rater reliability was assessed. ChartSweep reduced the time required to conduct RCR1 by 1315 minutes (21.9 hours), RCR2 by 1664 minutes (27.7 hours), and RCR3 by 2215 minutes (36.9 hours). Inter-rater reliability was uncompromised (k = 1.00). Open-source Python libraries as leveraged by ChartSweep significantly accelerate the RCR process in plastic surgery research. Quality of data review is not compromised. Further analyses with larger, heterogeneous study populations are required to further validate ChartSweep as a research tool.

Opioid Stewardship Program and Postoperative Adverse Events

Background

A 6-month opioid use educational program consisting of webinars on pain assessment, postoperative and multimodal pain opioid management, safer opioid use, and preventing addiction coupled with on-site coaching and monthly assessments reports was implemented in 31 hospitals. The authors hypothesized the intervention would measurably reduce and/or prevent opioid-related harm among adult hospitalized patients compared to 33 nonintervention hospitals.

Methods

Outcomes were extracted from medical records for 12 months before and after the intervention start date. Opioid adverse events, evaluated by opioid overdose, wrong substance given or taken in error, naloxone administration, and acute postoperative respiratory failure causing prolonged ventilation were the primary outcomes. Opioid use in adult patients undergoing elective hip or knee arthroplasty or colorectal procedures was also assessed. Differences-in-differences were compared between intervention and nonintervention hospitals.

Results

Before the intervention, the incidence ± SD of opioid overdose, wrong substance given, or substance taken in error was 1 ± 0.5 per 10,000 discharges, and naloxone use was 117 ± 13 per 10,000 patients receiving opioids. The incidence of respiratory failure was 42 ± 10 per 10,000 surgical discharges. A difference-in-differences of –0.2 (99% CI, –1.1 to 0.6, P = 0.499) per 10,000 in opioid overdose, wrong substance given, or substance taken in error and –13.6 (99% CI, –29.0 to 0.0, P = 0.028) per 10,000 in respiratory failure was observed postintervention in the intervention hospitals; however, naloxone administration increased by 15.2 (99% CI, 3.8 to 30.0, P = 0.011) per 10,000. Average total daily opioid use, as well as the fraction of patients receiving daily opioid greater than 90 mg morphine equivalents was not different between the intervention and nonintervention hospitals.

Conclusions

A 6-month opioid educational intervention did not reduce opioid adverse events or alter opioid use in hospitalized patients. The authors’ findings suggest that despite opioid and multimodal analgesia awareness, limited-duration educational interventions do not substantially change the hospital use of opioid analgesics.

Editor’s Perspective

What We Already Know about This Topic

What This Article Tells Us That Is New