Leveraging code-free deep learning for pill recognition in clinical settings: A multicenter, real-world study of performance across multiple platforms

BACKGROUND

Preventable patient harm, particularly medication errors, represent significant challenges in healthcare settings. Dispensing the wrong medication is often associated with mix-up of lookalike and soundalike drugs in high workload environments. Replacing manual dispensing with automated unit dose and medication dispensing systems to reduce medication errors is not always feasible in clinical facilities experiencing high patient turn-around or frequent dose changes. Artificial intelligence (AI) based pill recognition tools and smartphone applications could potentially aid healthcare workers in identifying pills in situations where more advanced dispensing systems are not implemented.

OBJECTIVE

Most of the published research on pill recognition focuses on theoretical aspects of model development using traditional coding and deep learning methods. The use of code-free deep learning (CFDL) as a practical alternative for accessible model development, and implementation of such models in tools intended to aid decision making in clinical settings, remains largely unexplored. In this study, we sought to address this gap in existing literature by investigating whether CFDL is a viable approach for developing pill recognition models using a custom dataset, followed by a thorough evaluation of the model across various deployment scenarios, and in multicenter clinical settings. Furthermore, we aimed to highlight challenges and propose solutions to achieve optimal performance and real-world applicability of pill recognition models, including when deployed on smartphone applications.

METHODS

A pill recognition model was developed utilizing Microsoft Azure Custom Vision platform and a large custom training dataset of 26,880 images captured from the top 30 most dispensed solid oral dosage forms (SODFs) at the three participating hospitals. A comprehensive internal and external testing strategy was devised, model's performance was investigated through the online API, and offline using exported TensorFlow Lite model running on a Windows PC and on Android, using a tailor-made testing smartphone application. Additionally, model's calibration, degree of reliance on color features and device dependency was thoroughly evaluated. Real-world performance was assessed using images captured by hospital pharmacists at three participating clinical centers.

RESULTS

The pill recognition model showed high performance in Microsoft Azure Custom Vision platform with 98.7 % precision, 95.1 % recall, and 98.2 % mean average precision (mAP), with thresholds set to 50 %. During internal testing utilizing the online API, the model reached 93.7 % precision, 88.96 % recall, 90.81 % F1-score and 87.35 % mAP. Testing the offline TensorFlow Lite model on Windows PC showed a slight performance reduction, with 91.16 % precision, 83.82 % recall, 86.18 % F1-score and 82.55 % mAP. Performance of the model running offline on the Android application was further reduced to 86.50 % precision, 75.00 % recall, 77.83 % F1-score and 69.24 % mAP. During external clinical testing through the online API an overall precision of 83.10 %, recall of 71.39 %, and F1-score of 75.76 % was achieved.

CONCLUSION

Our study demonstrates that using a CFDL approach is a feasible and cost-effective method for developing AI-based pill recognition systems. Despite the limitations encountered, our model performed well, particularly when accessed through the online API. The use of CFDL facilitates interdisciplinary collaboration, resulting in human-centered AI models with enhanced real-world applicability. We suggest that rather than striving to build a universally applicable pill recognition system, models should be tailored to the medications in a regional formulary or needs of a specific clinic, which can in turn lead to improved performance in real-world deployment in these locations. Parallel to focusing on model development, it is crucial to employ a human centered approach by training the end users on how to properly interact with the AI based system to maximize benefits. Future research is needed on refining pill recognition models for broader adaptability. This includes investigating image pre-processing and optimization techniques to enhance offline performance and operation on handheld devices. Moreover, future studies should explore methods to overcome limitations of CFDL development to enhance the robustness of models and reduce overfitting. Collaborative efforts between researchers in this domain and sharing of best practices are vital to improve pill recognition systems, ultimately enhancing patient safety and healthcare outcomes.

Search Engines and Generative Artificial Intelligence Integration: Public Health Risks and Recommendations to Safeguard Consumers Online

BACKGROUND

The online pharmacy market is growing, with legitimate online pharmacies offering advantages such as convenience and accessibility. However, this increased demand has attracted malicious actors into this space, leading to the proliferation of illegal vendors that use deceptive techniques to rank higher in search results and pose serious public health risks by dispensing substandard or falsified medicines. Search engine providers have started integrating generative artificial intelligence (AI) into search engine interfaces, which could revolutionize search by delivering more personalized results through a user-friendly experience. However, improper integration of these new technologies carries potential risks and could further exacerbate the risks posed by illicit online pharmacies by inadvertently directing users to illegal vendors.

OBJECTIVE

The role of generative AI integration in reshaping search engine results, particularly related to online pharmacies, has not yet been studied. Our objective was to identify, determine the prevalence of, and characterize illegal online pharmacy recommendations within the AI-generated search results and recommendations.

METHODS

We conducted a comparative assessment of AI-generated recommendations from Google's Search Generative Experience (SGE) and Microsoft Bing's Chat, focusing on popular and well-known medicines representing multiple therapeutic categories including controlled substances. Websites were individually examined to determine legitimacy, and known illegal vendors were identified by cross-referencing with the National Association of Boards of Pharmacy and LegitScript databases.

RESULTS

Of the 262 websites recommended in the AI-generated search results, 47.33% (124/262) belonged to active online pharmacies, with 31.29% (82/262) leading to legitimate ones. However, 19.04% (24/126) of Bing Chat's and 13.23% (18/136) of Google SGE's recommendations directed users to illegal vendors, including for controlled substances. The proportion of illegal pharmacies varied by drug and search engine. A significant difference was observed in the distribution of illegal websites between search engines. The prevalence of links leading to illegal online pharmacies selling prescription medications was significantly higher (P=.001) in Bing Chat (21/86, 24%) compared to Google SGE (6/92, 6%). Regarding the suggestions for controlled substances, suggestions generated by Google led to a significantly higher number of rogue sellers (12/44, 27%; P=.02) compared to Bing (3/40, 7%).

CONCLUSIONS

While the integration of generative AI into search engines offers promising potential, it also poses significant risks. This is the first study to shed light on the vulnerabilities within these platforms while highlighting the potential public health implications associated with their inadvertent promotion of illegal pharmacies. We found a concerning proportion of AI-generated recommendations that led to illegal online pharmacies, which could not only potentially increase their traffic but also further exacerbate existing public health risks. Rigorous oversight and proper safeguards are urgently needed in generative search to mitigate consumer risks, making sure to actively guide users to verified pharmacies and prioritize legitimate sources while excluding illegal vendors from recommendations.

Prevalence of Poisoned Google Search Results of Erectile Dysfunction Medications Redirecting to Illegal Internet Pharmacies: Data Analysis Study

Background

Illegal online pharmacies function as affiliate networks, in which search engine results pages (SERPs) are poisoned by several links redirecting site visitors to unlicensed drug distribution pages upon clicking on the link of a legitimate, yet irrelevant domain. This unfair online marketing practice is commonly referred to as search redirection attack, a most frequently used technique in the online illegal pharmaceutical marketplace.

Objective

This study is meant to describe the mechanism of search redirection attacks in Google search results in relation to erectile dysfunction medications in European countries and also to determine the local and global scales of this problem.

Methods

The search engine query results regarding 4 erectile dysfunction medications were documented using Google. The search expressions were “active ingredient” and “buy” in the language of 12 European countries, including Hungary. The final destination website legitimacy was checked at LegitScript, and the estimated number of monthly unique visitors was obtained from SEMrush traffic analytics. Compromised links leading to international illegal medicinal product vendors via redirection were analyzed using Gephi graph visualization software.

Results

Compromised links redirecting to active online pharmacies were present in search query results of all evaluated countries. The prevalence was highest in Spain (62/160, 38.8%), Hungary (52/160, 32.5%), Italy (46/160, 28.8%), and France (37/160, 23.1%), whereas the lowest was in Finland (12/160, 7.5%), Croatia (10/160, 6.3%), and Bulgaria (2/160, 1.3%), as per data recorded in November 2020. A decrease in the number of compromised sites linking visitors to illegitimate medicine sellers was observed in the Hungarian data set between 2019 and 2021, from 41% (33/80) to 5% (4/80), respectively. Out of 1920 search results in the international sample, 380 (19.79%) search query results were compromised, with the majority (n=342, 90%) of links redirecting individuals to 73 international illegal medicinal product vendors. Most of these illegal online pharmacies (41/73, 56%) received only 1 or 2 compromised links, whereas the top 3 domains with the highest in-degree link value received more than one-third of all incoming links. Traffic analysis of 35 pharmacy specific domains, accessible via compromised links in search engine queries, showed a total of 473,118 unique visitors in November 2020.

Conclusions

Although the number of compromised links in SERPs has shown a decreasing tendency in Hungary, an analysis of the European search query data set points to the global significance of search engine poisoning. Our research illustrates that search engine poisoning is a constant threat, as illegitimate affiliate networks continue to flourish while uncoordinated interventions by authorities and individual stakeholders remain insufficient. Ultimately, without a dedicated and comprehensive effort on the part of search engine providers for effectively monitoring and moderating SERPs, they may never be entirely free of compromised links leading to illegal online pharmacy networks.

Effects of Thymus vulgaris L., Cinnamomum verum J.Presl and Cymbopogon nardus (L.) Rendle Essential Oils in the Endotoxin-induced Acute Airway Inflammation Mouse Model

Thyme (TO), cinnamon (CO), and Ceylon type lemongrass (LO) essential oils (EOs) are commonly used for inhalation. However, their effects and mechanisms on inflammatory processes are not well-documented, and the number of in vivo data that would be important to determine their potential benefits or risks is low. Therefore, we analyzed the chemical composition and investigated the activity of TO, CO, and LO on airway functions and inflammatory parameters in an acute pneumonitis mouse model. The components of commercially available EOs were measured by gas chromatography-mass spectrometry. Airway inflammation was induced by intratracheal endotoxin administration in mice. EOs were inhaled during the experiments. Airway function and hyperresponsiveness were determined by unrestrained whole-body plethysmography on conscious animals. Myeloperoxidase (MPO) activity was measured by spectrophotometry from lung tissue homogenates, from which semiquantitative histopathological scores were assessed. The main components of TO, CO, and LO were thymol, cinnamaldehyde, and citronellal, respectively. We provide here the first evidence that TO and CO reduce inflammatory airway hyperresponsiveness and certain cellular inflammatory parameters, so they can potentially be considered as adjuvant treatments in respiratory inflammatory conditions. In contrast, Ceylon type LO inhalation might have an irritant effect (e.g., increased airway hyperresponsiveness and MPO activity) on the inflamed airways, and therefore should be avoided.

Comparison the effect of 3 point valgus stress knee support and lateral wedge insoles in medial compartment knee osteoarthritis

BACKGROUND

There are many opinions and controversies regard the effect of lateral wedge insoles and valgus stress 3point knee braces in treatment of medial compartment knee osteoarthritis (OA). In this study we compared the effect of lateral wedge insoles and 3 point knee supports in treatment of medial compartment knee OA.

METHOD

Sixty patients (35-65 years), with knee pain and genu varum and moderate to severe medial compartment DJD were divided into three groups. The first group received a custom molded 3 point valgus stress knee support. Lateral wedge insoles were applied for the second group and the third group served as control. All groups were followed for 9 months according to pain severity, walking distance, and radiologic changes.

RESULTS

Pain reduced significantly in both lateral wedge and knee brace groups compared to control group with more significant reduction in the brace group. The walking distance was significantly longer only in the brace group. There was more pronounced effect of brace in patients with severe DJD in walking distance compared to moderate DJD, but not in severity of pain.

CONCLUSION

Three point valgus stress knee support had more significant effect on pain reduction, walking distance and also radiologic improvement of patients with moderate to severe medial compartment DJD compared to lateral wedge insoles and could even reverse radiologic findings.

Sympathetic skin response (SSR) in Crohn disease: a pilot study

Peripheral neuropathy is one of the most frequently reported neurological extra-intestinal manifestations of the Crohn disease. Former studies have showed sympathetic neuropathy as early manifestation of CD by physical tests. The aim of this study is to examine sympathetic neuropathy in CD by using sympathetic skin response (SSR). We performed SSR on 6 Patients (4 male, 2 female) that were diagnosed as CD. Their age was between 26 till 68 years with the mean of (43.3 +/- 17.1) years. The duration of their disease was at least 4 years (mean: 9.0 +/- 8.4) and none of the patients had any symptoms or signs related to autonomic nervous system dysfunction. Hand and foot latencies in CD patients were prolonged relative to controls and the difference of hand latencies was statistically significant. Two patients demonstrated total abnormal results. In conclusion, there is some sub-clinical sympathetic dysfunction in CD patient. This suggests that SSR can be useful in detecting early autonomic changes in these patients.

The diagnostic value of ultrasonography in patients with electrophysiologicaly confirmed carpal tunnel syndrome

OBJECTIVE

To evaluate the diagnostic value of ultrasonography in patients with electrophysiologically confirmed carpal tunnel syndrome.

DESIGN

A prospective ultrasonographic study of 70 wrists with electrophysiologically confirmed carpal tunnel syndrome and of 80 normal wrists. Receiver-operating-characteristics curves for the ultrasonographic measurements of median nerve were plotted to identify the most optimal cutoff values.

RESULTS

The ultrasonographic measurements of median nerves were found to be increased significantly in patients with carpal tunnel syndrome when compared with controls, particularly in terms of cross-sectional area (P <0.001). According to receiver-operating-characteristics curve results, the most optimal cutoff value for the cross-sectional area of the median nerve was obtained at the level of middle carpal tunnel, which was 9.3 mm2, with a sensitivity of 80% and specificity of 77.5%.

CONCLUSION

Ultrasonographic examination of the median nerve seems to be a promising method in the diagnosis of carpal tunnel syndrome, evaluating the morphologic changes of the median nerve in patients with clinical signs and symptoms. Further studies with wider series are needed to confirm our preliminary results.