Evaluating the performance of ChatGPT in clinical pharmacy: A comparative study of ChatGPT and clinical pharmacists

Poor performance of ChatGPT in clinical rule-guided dose interventions in hospitalized patients with renal dysfunction

PURPOSE

Clinical decision support systems (CDSS) are used to identify drugs with potential need for dose modification in patients with renal impairment. ChatGPT holds the potential to be integrated in the electronic health record (EHR) system to give such dosing advices. In this study, we aim to evaluate the performance of ChatGPT in clinical rule-guided dose interventions in hospitalized patients with renal impairment.

METHODS

This cross-sectional study was performed at Tergooi Medical Center, the Netherlands. CDSS alerts regarding renal dysfunction were collected from the electronic health record (EHR) during a 2-week period and were presented to ChatGPT and an expert panel. Alerts were presented with and without patient variables. To evaluate the performance, suggested medication interventions were compared.

RESULTS

In total, 172 CDDS alerts were generated for 80 patients. Indecisive responses by ChatGPT to alerts were excluded. For alerts presented without patient variables, ChatGPT provided "correct and identical" responses to 19.9%, "correct and different" responses to 26.7%, and "incorrect responses to 53.4% of the alerts. For alerts including patient variables, ChatGPT provided "correct and identical" responses to 16.7%, "correct and different" responses to 16.0%, and "incorrect responses to 67.3% of the alerts. Accuracy was better for newer drugs such as direct oral anticoagulants.

CONCLUSION

The performance of ChatGPT in clinical rule-guided dose interventions in hospitalized patients with renal dysfunction was poor. Based on these results, we conclude that ChatGPT, in its current state, is not appropriate for automatic integration into our EHR to handle CDSS alerts related to renal dysfunction.

Assessing the applicability and appropriateness of ChatGPT in answering clinical pharmacy questions

OBJECTIVES

Clinical pharmacists rely on different scientific references to ensure appropriate, safe, and cost-effective drug use. Tools based on artificial intelligence (AI) such as ChatGPT (Generative Pre-trained Transformer) could offer valuable support. The objective of this study was to assess ChatGPT's capacity to correctly respond to clinical pharmacy questions asked by healthcare professionals in our university hospital.

MATERIAL AND METHODS

ChatGPT's capacity to respond correctly to the last 100 consecutive questions recorded in our clinical pharmacy database was assessed. Questions were copied from our FileMaker Pro database and pasted into ChatGPT March 14 version online platform. The generated answers were then copied verbatim into an Excel file. Two blinded clinical pharmacists reviewed all the questions and the answers given by the software. In case of disagreements, a third blinded pharmacist intervened to decide.

RESULTS

Documentation-related issues (n=36) and drug administration mode (n=30) were preponderantly recorded. Among 69 applicable questions, the rate of correct answers varied from 30 to 57.1% depending on questions type with a global rate of 44.9%. Regarding inappropriate answers (n=38), 20 were incorrect, 18 gave no answers and 8 were incomplete with 8 answers belonging to 2 different categories. No better answers than the pharmacists were observed.

CONCLUSIONS

ChatGPT demonstrated a mitigated performance in answering clinical pharmacy questions. It should not replace human expertise as a high rate of inappropriate answers was highlighted. Future studies should focus on the optimization of ChatGPT for specific clinical pharmacy questions and explore the potential benefits and limitations of integrating this technology into clinical practice.

Performance of ChatGPT on Factual Knowledge Questions Regarding Clinical Pharmacy

ChatGPT is a language model that was trained on a large dataset including medical literature. Several studies have described the performance of ChatGPT on medical exams. In this study, we examine its performance in answering factual knowledge questions regarding clinical pharmacy. Questions were obtained from a Dutch application that features multiple-choice questions to maintain a basic knowledge level for clinical pharmacists. In total, 264 clinical pharmacy-related questions were presented to ChatGPT and responses were evaluated for accuracy, concordance, quality of the substantiation, and reproducibility. Accuracy was defined as the correctness of the answer, and results were compared to the overall score by pharmacists over 2022. Responses were marked concordant if no contradictions were present. The quality of the substantiation was graded by two independent pharmacists using a 4-point scale. Reproducibility was established by presenting questions multiple times and on various days. ChatGPT yielded accurate responses for 79% of the questions, surpassing pharmacists' accuracy of 66%. Concordance was 95%, and the quality of the substantiation was deemed good or excellent for 73% of the questions. Reproducibility was consistently high, both within day and between days (>92%), as well as across different users. ChatGPT demonstrated a higher accuracy and reproducibility to factual knowledge questions related to clinical pharmacy practice than pharmacists. Consequently, we posit that ChatGPT could serve as a valuable resource to pharmacists. We hope the technology will further improve, which may lead to enhanced future performance.

Search Engines and Generative Artificial Intelligence Integration: Public Health Risks and Recommendations to Safeguard Consumers Online

BACKGROUND

The online pharmacy market is growing, with legitimate online pharmacies offering advantages such as convenience and accessibility. However, this increased demand has attracted malicious actors into this space, leading to the proliferation of illegal vendors that use deceptive techniques to rank higher in search results and pose serious public health risks by dispensing substandard or falsified medicines. Search engine providers have started integrating generative artificial intelligence (AI) into search engine interfaces, which could revolutionize search by delivering more personalized results through a user-friendly experience. However, improper integration of these new technologies carries potential risks and could further exacerbate the risks posed by illicit online pharmacies by inadvertently directing users to illegal vendors.

OBJECTIVE

The role of generative AI integration in reshaping search engine results, particularly related to online pharmacies, has not yet been studied. Our objective was to identify, determine the prevalence of, and characterize illegal online pharmacy recommendations within the AI-generated search results and recommendations.

METHODS

We conducted a comparative assessment of AI-generated recommendations from Google's Search Generative Experience (SGE) and Microsoft Bing's Chat, focusing on popular and well-known medicines representing multiple therapeutic categories including controlled substances. Websites were individually examined to determine legitimacy, and known illegal vendors were identified by cross-referencing with the National Association of Boards of Pharmacy and LegitScript databases.

RESULTS

Of the 262 websites recommended in the AI-generated search results, 47.33% (124/262) belonged to active online pharmacies, with 31.29% (82/262) leading to legitimate ones. However, 19.04% (24/126) of Bing Chat's and 13.23% (18/136) of Google SGE's recommendations directed users to illegal vendors, including for controlled substances. The proportion of illegal pharmacies varied by drug and search engine. A significant difference was observed in the distribution of illegal websites between search engines. The prevalence of links leading to illegal online pharmacies selling prescription medications was significantly higher (P=.001) in Bing Chat (21/86, 24%) compared to Google SGE (6/92, 6%). Regarding the suggestions for controlled substances, suggestions generated by Google led to a significantly higher number of rogue sellers (12/44, 27%; P=.02) compared to Bing (3/40, 7%).

CONCLUSIONS

While the integration of generative AI into search engines offers promising potential, it also poses significant risks. This is the first study to shed light on the vulnerabilities within these platforms while highlighting the potential public health implications associated with their inadvertent promotion of illegal pharmacies. We found a concerning proportion of AI-generated recommendations that led to illegal online pharmacies, which could not only potentially increase their traffic but also further exacerbate existing public health risks. Rigorous oversight and proper safeguards are urgently needed in generative search to mitigate consumer risks, making sure to actively guide users to verified pharmacies and prioritize legitimate sources while excluding illegal vendors from recommendations.

Utilizing ChatGPT in Telepharmacy

BACKGROUND

ChatGPT is an artificial intelligence-powered chatbot that has demonstrated capabilities in numerous fields, including medical and healthcare sciences. This study evaluates the potential for ChatGPT application in telepharmacy, the delivering of pharmaceutical care via means of telecommunications, through assessing its interactions, adherence to instructions, and ability to role-play as a pharmacist while handling a series of life-like scenario questions.

METHODS

Two versions (ChatGPT 3.5 and 4.0, OpenAI) were assessed using two independent trials each. ChatGPT was instructed to act as a pharmacist and answer patient inquiries, followed by a set of 20 assessment questions. Then, ChatGPT was instructed to stop its act, provide feedback and list its sources for drug information. The responses to the assessment questions were evaluated in terms of accuracy, precision and clarity using a 4-point Likert-like scale.

RESULTS

ChatGPT demonstrated the ability to follow detailed instructions, role-play as a pharmacist, and appropriately handle all questions. ChatGPT was able to understand case details, recognize generic and brand drug names, identify drug side effects, interactions, prescription requirements and precautions, and provide proper point-by-point instructions regarding administration, dosing, storage and disposal. The overall means of pooled scores were 3.425 (0.712) and 3.7 (0.61) for ChatGPT 3.5 and 4.0, respectively. The rank distribution of scores was not significantly different (P>0.05). None of the answers could be considered directly harmful or labeled as entirely or mostly incorrect, and most point deductions were due to other factors such as indecisiveness, adding immaterial information, missing certain considerations, or partial unclarity. The answers were similar in length across trials and appropriately concise. ChatGPT 4.0 showed superior performance, higher consistency, better character adherence and the ability to report various reliable information sources. However, it only allowed an input of 40 questions every three hours and provided inaccurate feedback regarding the number of assessed patients, compared to 3.5 which allowed unlimited input but was unable to provide feedback.

CONCLUSIONS

Integrating ChatGPT in telepharmacy holds promising potential; however, a number of drawbacks are to be overcome in order to function effectively.