1
|
Naderalvojoud B, Curtin CM, Yanover C, El-Hay T, Choi B, Park RW, Tabuenca JG, Reeve MP, Falconer T, Humphreys K, Asch SM, Hernandez-Boussard T. Towards global model generalizability: independent cross-site feature evaluation for patient-level risk prediction models using the OHDSI network. J Am Med Inform Assoc 2024; 31:1051-1061. [PMID: 38412331 PMCID: PMC11031239 DOI: 10.1093/jamia/ocae028] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2023] [Revised: 01/26/2024] [Accepted: 02/01/2024] [Indexed: 02/29/2024] Open
Abstract
BACKGROUND Predictive models show promise in healthcare, but their successful deployment is challenging due to limited generalizability. Current external validation often focuses on model performance with restricted feature use from the original training data, lacking insights into their suitability at external sites. Our study introduces an innovative methodology for evaluating features during both the development phase and the validation, focusing on creating and validating predictive models for post-surgery patient outcomes with improved generalizability. METHODS Electronic health records (EHRs) from 4 countries (United States, United Kingdom, Finland, and Korea) were mapped to the OMOP Common Data Model (CDM), 2008-2019. Machine learning (ML) models were developed to predict post-surgery prolonged opioid use (POU) risks using data collected 6 months before surgery. Both local and cross-site feature selection methods were applied in the development and external validation datasets. Models were developed using Observational Health Data Sciences and Informatics (OHDSI) tools and validated on separate patient cohorts. RESULTS Model development included 41 929 patients, 14.6% with POU. The external validation included 31 932 (UK), 23 100 (US), 7295 (Korea), and 3934 (Finland) patients with POU of 44.2%, 22.0%, 15.8%, and 21.8%, respectively. The top-performing model, Lasso logistic regression, achieved an area under the receiver operating characteristic curve (AUROC) of 0.75 during local validation and 0.69 (SD = 0.02) (averaged) in external validation. Models trained with cross-site feature selection significantly outperformed those using only features from the development site through external validation (P < .05). CONCLUSIONS Using EHRs across four countries mapped to the OMOP CDM, we developed generalizable predictive models for POU. Our approach demonstrates the significant impact of cross-site feature selection in improving model performance, underscoring the importance of incorporating diverse feature sets from various clinical settings to enhance the generalizability and utility of predictive healthcare models.
Collapse
Affiliation(s)
| | - Catherine M Curtin
- Department of Surgery, Veterans Affairs Palo Alto Health Care System, Palo Alto, CA 94304, United States
| | - Chen Yanover
- KI Research Institute, Kfar Malal, 4592000, Israel
| | - Tal El-Hay
- KI Research Institute, Kfar Malal, 4592000, Israel
| | - Byungjin Choi
- Department of Biomedical Informatics, Ajou University Graduate School of Medicine, Suwon, 16499, Korea
| | - Rae Woong Park
- Department of Biomedical Informatics, Ajou University Graduate School of Medicine, Suwon, 16499, Korea
| | - Javier Gracia Tabuenca
- Institute for Molecular Medicine Finland (FIMM), HiLIFE, University of Helsinki, Helsinki, 00014, Finland
| | - Mary Pat Reeve
- Institute for Molecular Medicine Finland (FIMM), HiLIFE, University of Helsinki, Helsinki, 00014, Finland
| | - Thomas Falconer
- Department of Biomedical Informatics, Columbia University, New York, NY 10032, United States
| | - Keith Humphreys
- Department of Psychiatry and the Behavioral Sciences, Stanford University, Stanford, CA 94305, United States
- Center for Innovation to Implementation, Veterans Affairs Palo Alto Health Care System, Palo Alto, CA 94304, United States
| | - Steven M Asch
- Department of Medicine, Stanford University, Stanford, CA 94305, United States
- Center for Innovation to Implementation, Veterans Affairs Palo Alto Health Care System, Palo Alto, CA 94304, United States
| | | |
Collapse
|
2
|
Rosen DM, Alcock MM, Palmer GM. Opioids for acute pain management in children. Anaesth Intensive Care 2022; 50:81-94. [PMID: 35172638 DOI: 10.1177/0310057x211065769] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Opioids are integral to multimodal analgesic regimens in children with moderate to severe acute pain. Throughout normal childhood there are marked changes in physiology, and social and psychological development that influence the perception and expression of pain, the pharmacology of opioids, and how they are used. A multidimensional pain assessment is key to guiding appropriate opioid prescribing. Most of the commonly used opioids in adults are used in children, with the increasing exception of codeine (as a result of regulatory change), and are generally well tolerated. Patient groups at increased risk of ventilatory impairment include neonates and those with obstructive sleep apnoea, severe neurodevelopmental conditions, trisomy 21, and severe epilepsy. Slow-release opioids are not recommended for general use, but may be used in select populations, for example, following scoliosis surgery, major trauma or burns. Prescribing and administration errors are a major issue in paediatrics generally; the potential consequences of opioid prescribing or administration errors are serious, particularly following hospital discharge. Opioids prescribed at discharge are frequently in excess of a child's analgesic requirements; three to five days supply appears sufficient for the majority of common paediatric operations. Discharge opioid prescriptions have been linked to long-term opioid use in adolescents with risk factors. Misuse of prescription opioids by adolescents is also concerning, with prevalence estimates ranging from 1.1% to 20%. Caregivers have a tendency to underdose opioids in their children; caregiver education may improve appropriate administration. Caregivers must also be provided with instructions on safe storage and disposal of unused opioids.
Collapse
Affiliation(s)
- Derek M Rosen
- Department of Anaesthesia and Pain Management, Queensland Children's Hospital, Brisbane, Queensland, Australia.,University of Queensland, Queensland, Brisbane, Australia
| | - Mark M Alcock
- Department of Anaesthesia and Pain Management, Queensland Children's Hospital, Brisbane, Queensland, Australia.,University of Queensland, Queensland, Brisbane, Australia
| | - Greta M Palmer
- Department of Anaesthesia and Pain Management, Royal Children's Hospital, Melbourne, Victoria, Australia.,University of Melbourne, Melbourne, Victoria, Australia
| |
Collapse
|
3
|
Ayad AE, Salman OH, Ibrahim AMF, Al-Taher WAM, Mishriky AM, Pergolizzi JV, Viswanath O, Urits I, Rekatsina M, Peppin JF, Paladini A, Varrassi G. A Response to: Letter to the Editor regarding "Influences of Gender on Intravenous Nalbuphine Actions After Major Abdominal Surgery: A Multicenter Study". Pain Ther 2021; 10:1783-1786. [PMID: 34431072 PMCID: PMC8586306 DOI: 10.1007/s40122-021-00305-5] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2021] [Accepted: 08/06/2021] [Indexed: 11/28/2022] Open
Affiliation(s)
- Amany E Ayad
- Department of Anesthesia ICU and Pain, Cairo University, Cairo, 11566, Egypt
| | - Ossama H Salman
- Department of Anesthesiology, South Valley University, Qena, 83511, Egypt
| | | | | | - Adel M Mishriky
- Department of Community Medicine, Suez Canal University, Ismailia, 41511, Egypt
| | | | - Omar Viswanath
- Department of Anesthesiology, University of Arizona, Phoenix, AZ, 85003, USA
| | - Ivan Urits
- Department of Anesthesia, Critical Care and Pain Medicine, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MS, USA
| | | | - John F Peppin
- College of Osteopathic Medicine, Pikeville University, c, KT, USA
| | | | | |
Collapse
|
4
|
Szarmach J, Cubała WJ, Włodarczyk A, Gałuszko-Węgielnik M. Somatic Comorbidities and Cardiovascular Safety in Ketamine Use for Treatment-Resistant Depression. ACTA ACUST UNITED AC 2021; 57:medicina57030274. [PMID: 33809766 PMCID: PMC8002231 DOI: 10.3390/medicina57030274] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2021] [Revised: 03/12/2021] [Accepted: 03/15/2021] [Indexed: 01/15/2023]
Abstract
Background and Objectives: There is evidence for ketamine efficacy in treatment-resistant depression (TRD). Several safety and tolerability concerns arise that some psychotropic agents may provide blood pressure or/and heart rate alterations. The aim of this study is to review blood pressure measurements in course of the treatment with ketamine on treatment refractory inpatients with somatic comorbidities in the course of MDD and BP. Materials and Methods: The study population of 49 patients comprised MDD and BP subjects treated with ketamine registered in the naturalistic observational protocol of treatment-resistant mood disorders (NCT04226963). Results: The conducted analysis showed that among people suffering from hypertension there is a higher increase in systolic blood pressure (RR) after infusion 2 (p = 0.004) than among people who do not suffer from hypertension. Patients with hypertension have a higher increase in diastolic RR compared to those not suffering from hypertension (p = 0,038). Among the subjects with diabetes mellitus, significant differences occurred for infusions 2 (p = 0.020), 7 (p = 0.020), and 8 (p = 0.035) for heart rate (HR), compared to subjects without diabetes mellitus. A higher increase in diastolic RR was noted in the group of subjects suffering from diabetes mellitus (p = 0.010) compared to those who did not. In the hyperlipidemic patients studied, a significantly greater decrease in HR after infusion 5 (p = 0.031) and systolic RR after infusion 4 (p = 0.036) was noted compared to nonpatients. People after a stroke had significantly higher increases in diastolic RR after infusions 4 (p = 0.021) and 6 (p = 0.001) than those who did not have a stroke. Patients suffering from epilepsy had a significantly greater decrease in systolic RR after the 8th infusion (p = 0.017) compared to those without epilepsy. Limitations: The study may be underpowered due to the small sample size. The observations apply to inhomogeneous TRD population in a single-site with no blinding and are limited to the acute administration. Conclusions: This study supports evidence for good safety and tolerability profile for short-term IV ketamine use in TRD treatment. However, risk mitigation measures are to be considered in patients with metabolic and cardiovascular comorbidities.
Collapse
|
5
|
Bergese SD, Brzezinski M, Hammer GB, Beard TL, Pan PH, Mace SE, Berkowitz RD, Cochrane K, Wase L, Minkowitz HS, Habib AS. ATHENA: A Phase 3, Open-Label Study Of The Safety And Effectiveness Of Oliceridine (TRV130), A G-Protein Selective Agonist At The µ-Opioid Receptor, In Patients With Moderate To Severe Acute Pain Requiring Parenteral Opioid Therapy. J Pain Res 2019; 12:3113-3126. [PMID: 31814753 PMCID: PMC6861532 DOI: 10.2147/jpr.s217563] [Citation(s) in RCA: 39] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2019] [Accepted: 10/19/2019] [Indexed: 01/15/2023] Open
Abstract
Background Pain management with conventional opioids can be challenging due to dose-limiting adverse events (AEs), some of which may be related to the simultaneous activation of β-arrestin (a signaling pathway associated with opioid-related AEs) and G-protein pathways. The investigational analgesic oliceridine is a G-protein-selective agonist at the µ-opioid receptor with less recruitment of β-arrestin. The objective of this phase 3, open-label, multi-center study was to evaluate the safety and tolerability, of IV oliceridine for moderate to severe acute pain in a broad, real-world patient population, including postoperative surgical patients and non-surgical patients with painful medical conditions. Methods Adult patients with a score ≥4 on 11-point NRS for pain intensity received IV oliceridine either by bolus or PCA; multimodal analgesia was permitted. Safety was assessed using AE reports, study discontinuations, clinical laboratory and vital sign measures. Results A total of 768 patients received oliceridine. The mean age (SD) was 54.1 (16.1) years, with 32% ≥65 years of age. Most patients were female (65%) and Caucasian (78%). Surgical patients comprised the majority of the study population (94%), most common being orthopedic (30%), colorectal (15%) or gynecologic (15%) procedures. Multimodal analgesia was administered to 84% of patients. Oliceridine provided a rapid reduction in NRS pain score by 2.2 ± 2.3 at 30 mins from a score of 6.3 ± 2.1 (at baseline) which was maintained to the end of treatment. No deaths or significant cardiorespiratory events were reported. The incidence of AEs leading to early discontinuation and serious AEs were 2% and 3%, respectively. Nausea (31%), constipation (11%), and vomiting (10%) were the most common AEs. AEs were mostly of mild (37%) or moderate (25%) severity and considered possibly or probably related to oliceridine in 33% of patients. Conclusion Oliceridine IV for the management of moderate to severe acute pain was generally safe and well tolerated in the patients studied. ClinicalTrialsgov identifier NCT02656875.
Collapse
Affiliation(s)
- Sergio D Bergese
- School of Medicine, Stony Brook University, Stony Brook, NY, USA
| | - Marek Brzezinski
- School of Medicine, University of California San Francisco, VA Medical Center, San Francisco, CA, USA
| | | | - Timothy L Beard
- Clinical Research, Summit Medical Group/Bend Memorial Clinic, Bend, OR, USA
| | - Peter H Pan
- Wake Forest School of Medicine, Winston-Salem, NC, USA
| | - Sharon E Mace
- Cleveland Clinic Lerner College of Medicine at Case Western Reserve University, Cleveland, OH, USA
| | | | | | | | | | | |
Collapse
|