1
|
Guillen-Grima F, Guillen-Aguinaga S, Guillen-Aguinaga L, Alas-Brun R, Onambele L, Ortega W, Montejo R, Aguinaga-Ontoso E, Barach P, Aguinaga-Ontoso I. Evaluating the Efficacy of ChatGPT in Navigating the Spanish Medical Residency Entrance Examination (MIR): Promising Horizons for AI in Clinical Medicine. Clin Pract 2023; 13:1460-1487. [PMID: 37987431 PMCID: PMC10660543 DOI: 10.3390/clinpract13060130] [Citation(s) in RCA: 11] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2023] [Revised: 11/15/2023] [Accepted: 11/16/2023] [Indexed: 11/22/2023] Open
Abstract
The rapid progress in artificial intelligence, machine learning, and natural language processing has led to increasingly sophisticated large language models (LLMs) for use in healthcare. This study assesses the performance of two LLMs, the GPT-3.5 and GPT-4 models, in passing the MIR medical examination for access to medical specialist training in Spain. Our objectives included gauging the model's overall performance, analyzing discrepancies across different medical specialties, discerning between theoretical and practical questions, estimating error proportions, and assessing the hypothetical severity of errors committed by a physician. MATERIAL AND METHODS We studied the 2022 Spanish MIR examination results after excluding those questions requiring image evaluations or having acknowledged errors. The remaining 182 questions were presented to the LLM GPT-4 and GPT-3.5 in Spanish and English. Logistic regression models analyzed the relationships between question length, sequence, and performance. We also analyzed the 23 questions with images, using GPT-4's new image analysis capability. RESULTS GPT-4 outperformed GPT-3.5, scoring 86.81% in Spanish (p < 0.001). English translations had a slightly enhanced performance. GPT-4 scored 26.1% of the questions with images in English. The results were worse when the questions were in Spanish, 13.0%, although the differences were not statistically significant (p = 0.250). Among medical specialties, GPT-4 achieved a 100% correct response rate in several areas, and the Pharmacology, Critical Care, and Infectious Diseases specialties showed lower performance. The error analysis revealed that while a 13.2% error rate existed, the gravest categories, such as "error requiring intervention to sustain life" and "error resulting in death", had a 0% rate. CONCLUSIONS GPT-4 performs robustly on the Spanish MIR examination, with varying capabilities to discriminate knowledge across specialties. While the model's high success rate is commendable, understanding the error severity is critical, especially when considering AI's potential role in real-world medical practice and its implications for patient safety.
Collapse
Affiliation(s)
- Francisco Guillen-Grima
- Department of Health Sciences, Public University of Navarra, 31008 Pamplona, Spain; (S.G.-A.); (L.G.-A.); (R.A.-B.)
- Healthcare Research Institute of Navarra (IdiSNA), 31008 Pamplona, Spain
- Department of Preventive Medicine, Clinica Universidad de Navarra, 31008 Pamplona, Spain
- CIBER in Epidemiology and Public Health (CIBERESP), Institute of Health Carlos III, 46980 Madrid, Spain
| | - Sara Guillen-Aguinaga
- Department of Health Sciences, Public University of Navarra, 31008 Pamplona, Spain; (S.G.-A.); (L.G.-A.); (R.A.-B.)
| | - Laura Guillen-Aguinaga
- Department of Health Sciences, Public University of Navarra, 31008 Pamplona, Spain; (S.G.-A.); (L.G.-A.); (R.A.-B.)
- Department of Nursing, Kystad Helse-og Velferdssenter, 7026 Trondheim, Norway
| | - Rosa Alas-Brun
- Department of Health Sciences, Public University of Navarra, 31008 Pamplona, Spain; (S.G.-A.); (L.G.-A.); (R.A.-B.)
| | - Luc Onambele
- School of Health Sciences, Catholic University of Central Africa, Yaoundé 1100, Cameroon;
| | - Wilfrido Ortega
- Department of Surgery, Medical and Social Sciences, University of Alcala de Henares, 28871 Alcalá de Henares, Spain;
| | - Rocio Montejo
- Department of Obstetrics and Gynecology, Institute of Clinical Sciences, University of Gothenburg, 413 46 Gothenburg, Sweden;
- Department of Obstetrics and Gynecology, Sahlgrenska University Hospital, 413 46 Gothenburg, Sweden
| | | | - Paul Barach
- Jefferson College of Population Health, Philadelphia, PA 19107, USA;
- School of Medicine, Thomas Jefferson University, Philadelphia, PA 19107, USA
- Interdisciplinary Research Institute for Health Law and Science, Sigmund Freud University, 1020 Vienna, Austria
- Department of Surgery, Imperial College, London SW7 2AZ, UK
| | - Ines Aguinaga-Ontoso
- Department of Health Sciences, Public University of Navarra, 31008 Pamplona, Spain; (S.G.-A.); (L.G.-A.); (R.A.-B.)
- Healthcare Research Institute of Navarra (IdiSNA), 31008 Pamplona, Spain
| |
Collapse
|
2
|
Uguz F, Sharma V, Boyce P, Clark CT, Galbally M, Koukopoulos A, Marsh W, Stevens A, Viguera A. Prophylactic Management of Women With Bipolar Disorder During Pregnancy and the Perinatal Period: Clinical Scenario-Based Practical Recommendations From A Group of Perinatal Psychiatry Authors. J Clin Psychopharmacol 2023; 43:434-452. [PMID: 37683233 DOI: 10.1097/jcp.0000000000001740] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 08/29/2023]
Abstract
ABSTRACT Many women with bipolar disorder experience episodes of illness or relapses over the perinatal period, especially in the immediate postpartum period. Risks associated with treated/untreated psychopathologies and fetal exposure to bipolar medications make the management of bipolar disorder during these periods challenging for clinicians and patients. In light of the available effectiveness and reproductive safety data, the current clinical update based on the opinions of a group of international perinatal psychiatry authors recommends general considerations and specific management strategies for each possible clinical scenario, including mixed features, predominant polarity, diagnosis of subtypes of bipolar disorder, severity of previous episodes, and risk of recurrence of mood episodes.
Collapse
Affiliation(s)
- Faruk Uguz
- From the Department of Psychiatry, Meram Faculty of Medicine, Necmettin Erbakan University, Konya, Turkey
| | - Verinder Sharma
- Schulich School of Medicine and Dentistry, The University of Western Ontario; Lawson Health Research Institute; Parkwood Institute Mental Health, Perinatal Mental Health Clinic, London Health Sciences Center, London, Ontario, Canada
| | - Philip Boyce
- Westmead Institute of Medical Research, Faculty of Medicine and Health, University of Sydney, Sydney, Australia
| | - Crystal T Clark
- Department of Psychiatry and Behavioral Sciences and Obstetrics and Gynecology, Northwestern Feinberg School of Medicine, Chicago, IL
| | - Megan Galbally
- Department of Psychiatry, School of Clinical Sciences, Monash University, Melbourne, Australia
| | - Alexia Koukopoulos
- University Hospital Policlinico Umberto I, La Sapienza University of Rome; Centro Lucio Bini, Rome, Italy
| | - Wendy Marsh
- Department of Psychiatry, University of Massachusetts Medical School/UMass Memorial Medical Center, Worcester, MA.
| | - Anja Stevens
- Centre for Bipolar Disorders, Dimence Group, Deventer, the Netherlands; Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam Public Health Research Institute, Psychiatry, the Netherlands
| | - Adele Viguera
- Cleveland Clinic, Cleveland Clinic Neurological Institute, Cleveland, OH
| |
Collapse
|
3
|
Uguz F, Kirkas A. Olanzapine and quetiapine in the prevention of a new mood episode in women with bipolar disorder during the postpartum period: a retrospective cohort study. BRAZILIAN JOURNAL OF PSYCHIATRY 2021; 43:617-620. [PMID: 33825764 PMCID: PMC8639011 DOI: 10.1590/1516-4446-2020-1629] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Received: 10/23/2020] [Accepted: 01/27/2021] [Indexed: 11/22/2022]
Abstract
OBJECTIVE To examine whether olanzapine and quetiapine are useful in the prevention of a new mood episode during the postpartum period. METHODS Data on 23 patients (n=14 for olanzapine and n=9 for quetiapine) with bipolar disorder who met the criteria for this study were retrospectively gathered. The diagnosis of bipolar disorder was determined by means of the DSM-IV. RESULTS The mean follow-up period was 33.95±12.07 weeks. Six (26.1%) of 23 patients experienced recurrent mood episodes during the postpartum period. Four of these six patients were taking olanzapine and two were taking quetiapine. Patients with recurrent mood episodes had a significantly stronger family history of bipolar disorder, higher number of past episodes, and earlier onset and longer duration of illness compared to patients without recurrent mood episodes. CONCLUSION Monotherapy with olanzapine or quetiapine can be considered as an alternative to mood stabilizers in preventing the development of new mood episodes after childbirth.
Collapse
Affiliation(s)
- Faruk Uguz
- Department of Psychiatry, Meram Faculty of Medicine, Necmettin Erbakan University, Konya, Turkey
| | - Aysegul Kirkas
- Department of Psychiatry, Meram Faculty of Medicine, Necmettin Erbakan University, Konya, Turkey
| |
Collapse
|
4
|
Uguz F. Pharmacological prevention of mood episodes in women with bipolar disorder during the perinatal period: A systematic review of current literature. Asian J Psychiatr 2020; 52:102145. [PMID: 32516746 DOI: 10.1016/j.ajp.2020.102145] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/11/2020] [Revised: 04/27/2020] [Accepted: 04/28/2020] [Indexed: 01/28/2023]
Abstract
OBJECTIVE This review examined the efficacy of mood stabilizers and antipsychotics in patients with bipolar disorder during pregnancy and the postpartum period. METHODS PubMed was searched for reports between 01 January 1996 and 31 December 2019 by using combinations of key words bipolar disorder, pregnancy, postpartum period, puerperium, prophylaxis, mood stabilizers, antipsychotics, lithium, lamotrigine, valproate, carbamazepine, oxcarbazepine, olanzapine, risperidone, quetiapine, aripiprazole, haloperidol, and chlorpromazine. RESULTS The present reports included a total of 256 patients using lithium (n = 143), lamotrigine (n = 73), valproate (n = 17), olanzapine (n = 17), quetiapine (n = 4) and haloperidol (n = 1) during pregnancy or the postpartum period. Recurrence rates in pregnant patients using lithium (n = 79) and lamotrigine (n = 17) were 22.7 % and 41.2 %, respectively. According to very limited data, none of the patients using valproate (n = 2), quetiapine (n = 3) or olanzapine (n = 6) experienced a new episode during pregnancy. A recurrence was reported in 12 (70.6 %) of 17 patients using valproate during the postpartum period. The same recurrence rates in patients using lithium (n = 123), lamotrigine (n = 63), olanzapine (n = 17) and quetiapine (n = 3) were 20.3 %, 7.9 %, 11.7 %, and 33.3 %, respectively. CONCLUSIONS This review suggests that lithium, lamotrigine and olanzapine seem to be effective in preventing new mood episodes in patients with bipolar disorder during the perinatal period.
Collapse
Affiliation(s)
- Faruk Uguz
- Department of Psychiatry, Meram Faculty of Medicine, Necmettin Erbakan University, Konya, Turkey.
| |
Collapse
|
5
|
Badhan RKS, Macfarlane H. Quetiapine dose optimisation during gestation: a pharmacokinetic modelling study. J Pharm Pharmacol 2020; 72:670-681. [DOI: 10.1111/jphp.13236] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2019] [Accepted: 01/13/2020] [Indexed: 12/14/2022]
Abstract
Abstract
Objectives
The second-generation antipsychotic quetiapine has been demonstrated to undergo gestation-related changes in pharmacokinetics. This study applied pharmacokinetic modelling principles to investigate the mechanism of these changes and to propose new dosing strategies to counteract these changes.
Methods
A pharmacokinetic modelling approach was implemented using virtual population groups. Changes in quetiapine trough plasma concentration during gestation were quantified across all trimesters, and dose adjustment strategies were applied to counteract these changes by targeting a therapeutic range of 50–500 ng/ml throughout gestation.
Key findings
The application of the model during gestation predicted a decrease in trough concentration. A maximum decrease of 58% was predicted during trimester 2, and being associated with a statistically significant decrease in oral clearance at gestation week 25, 204 l/h ± 100.8 l/h compared with non-pregnant subjects, 121.9 l/h ± 51.8 l/h. A dosing optimisation strategy identified that dose increases to 500–700 mg twice daily would result in 32–55% of subjects possessing trough concentration in excess of 50 ng/ml.
Conclusions
Quetiapine doses in pregnancy should be increased to 500–700 mg twice daily to counteract a concomitant increase in metabolic clearance, increase in volume of distribution and decrease in plasma protein binding.
Collapse
Affiliation(s)
- Raj K S Badhan
- Medicines Optimisation Research Group, Aston Pharmacy School, Aston University, Birmingham, UK
| | - Hannah Macfarlane
- Medicines Optimisation Research Group, Aston Pharmacy School, Aston University, Birmingham, UK
| |
Collapse
|
6
|
Stevens AWMM, Goossens PJJ, Knoppert-van der Klein EAM, Draisma S, Honig A, Kupka RW. Risk of recurrence of mood disorders during pregnancy and the impact of medication: A systematic review. J Affect Disord 2019; 249:96-103. [PMID: 30769297 DOI: 10.1016/j.jad.2019.02.018] [Citation(s) in RCA: 32] [Impact Index Per Article: 6.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/22/2018] [Revised: 01/30/2019] [Accepted: 02/05/2019] [Indexed: 12/31/2022]
Abstract
BACKGROUND Mood disorders can be difficult to treat during pregnancy. There is still lack of evidence whether pregnancy influences their natural course and whether continuation of pharmacotherapy, despite potential risks for the unborn child, is beneficial in preventing recurrence of mood episodes during pregnancy. METHODS Systematic review conducted according to the PRISMA guidelines, searching Pubmed, PsycINFO, Embase and Cochrane databases up till January 9th, 2018. Recurrence rates and various measures of risk were calculated. RESULTS Out of 1387 articles from an initial search 22 studies met the inclusion criteria. Included studies reported a wide variation in the recurrence rate of bipolar disorder and major depressive disorder during pregnancy (BD: mean = 19%, range = 4%-73%; MDD: mean = 8%, range = 1%-75%). Observational data showed a relative risk reduction of maintenance therapy during pregnancy of 66% in women with BD and 54% for women with MDD, a significant difference (95% CI 9.4-14.6; p < 0.001). LIMITATIONS heterogeneous samples, study designs, and reported outcomes in included studies. CONCLUSIONS Despite the importance of the topic there is a paucity of evidence on recurrence rates of mood episodes during pregnancy among women with MDD or BD. Unlike the impact of the postpartum period, it is still uncertain whether the course of mood disorders is influenced by pregnancy. Non-randomized studies show that maintenance pharmacotherapy during pregnancy in women with mood disorders significantly (p < 0.01) reduces the risk of recurrence.
Collapse
Affiliation(s)
- Anja W M M Stevens
- Dimence Mental Health, Center for Bipolar Disorders, Deventer, the Netherlands; Amsterdam UMC, Vrije Universiteit Amsterdam, Psychiatry, Amsterdam Public Health Research Institute, Amsterdam, the Netherlands.
| | - Peter J J Goossens
- Dimence Mental Health, Center for Bipolar Disorders, Deventer, the Netherlands; University Centre for Nursing and Midwifery, Department of Public Health, Faculty of Medicine and Health Sciences, Ghent University, Ghent, Belgium
| | | | - Stasja Draisma
- Amsterdam UMC, Vrije Universiteit Amsterdam, Psychiatry, Amsterdam Public Health research Institute, GGZ inGeest Specialized Mental Health Care, Research and Innovation Amsterdam, the Netherlands
| | - Adriaan Honig
- Department of Psychiatry OLVG/msterdam UMC, Vrije Universiteit Amsterdam, Psychiatry, Amsterdam Public Health Research Institute, Amsterdam, the Netherlands
| | - Ralph W Kupka
- Amsterdam UMC, Vrije Universiteit Amsterdam, Psychiatry, Amsterdam Public Health Research Institute, Amsterdam, the Netherlands
| |
Collapse
|
7
|
Abstract
PURPOSE OF REVIEW The management of bipolar disorder during pregnancy requires difficult treatment decisions be made by both women and their clinicians. There is little consensus on management despite the high prevalence of bipolar disorder in reproductive-aged women. In this review, we have summarized the available literature and discuss the balancing of risks associated with treatment decisions. RECENT FINDINGS Cohort studies have shown a high relapse rate in women with bipolar disorder who discontinue mood-stabilizing medications. The risks of fetal medication exposure have been assessed in multiple database studies. Management decisions of bipolar disorder in pregnancy have been made difficult by inconsistencies in study outcomes. There were many confounding factors in the studies of medication discontinuation relapse risk. Inconsistencies in the findings of fetal risks from mood-stabilizing medications have further complicated management decisions. Larger studies are needed to clarify the risks of bipolar disorder relapse in pregnancy with and without treatment.
Collapse
|
8
|
Abstract
The study of perinatal mental health (mental health during pregnancy and postpartum) is a complex field of study that is of major importance both for the mental and physical health of new mothers and for the neurobehavioral development and long-term functioning of the children they bear. In this review, we cover the most recent additions to this rapidly evolving field. Notable advances include further illumination of the epidemiological patterns and clinical manifestations of perinatal mood disruption; new efficacy data on treatment and prevention; clarifications of the respective contributions of maternal mental illness and psychotropic medication to outcomes of pregnancy, birth, and child development; and updated expert guidelines for screening.
Collapse
Affiliation(s)
- Thalia Robakis
- Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, CA, USA
| | - Eugenia Jernick
- Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, CA, USA
| | - Katherine Williams
- Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, CA, USA
| |
Collapse
|