1
|
Madanay F, Tu K, Campagna A, Davis JK, Doerstling SS, Chen F, Ubel PA. Classification of Patients' Judgments of Their Physicians in Web-Based Written Reviews Using Natural Language Processing: Algorithm Development and Validation. J Med Internet Res 2024; 26:e50236. [PMID: 39088259 PMCID: PMC11327625 DOI: 10.2196/50236] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2023] [Revised: 03/21/2024] [Accepted: 04/29/2024] [Indexed: 08/02/2024] Open
Abstract
BACKGROUND Patients increasingly rely on web-based physician reviews to choose a physician and share their experiences. However, the unstructured text of these written reviews presents a challenge for researchers seeking to make inferences about patients' judgments. Methods previously used to identify patient judgments within reviews, such as hand-coding and dictionary-based approaches, have posed limitations to sample size and classification accuracy. Advanced natural language processing methods can help overcome these limitations and promote further analysis of physician reviews on these popular platforms. OBJECTIVE This study aims to train, test, and validate an advanced natural language processing algorithm for classifying the presence and valence of 2 dimensions of patient judgments in web-based physician reviews: interpersonal manner and technical competence. METHODS We sampled 345,053 reviews for 167,150 physicians across the United States from Healthgrades.com, a commercial web-based physician rating and review website. We hand-coded 2000 written reviews and used those reviews to train and test a transformer classification algorithm called the Robustly Optimized BERT (Bidirectional Encoder Representations from Transformers) Pretraining Approach (RoBERTa). The 2 fine-tuned models coded the reviews for the presence and positive or negative valence of patients' interpersonal manner or technical competence judgments of their physicians. We evaluated the performance of the 2 models against 200 hand-coded reviews and validated the models using the full sample of 345,053 RoBERTa-coded reviews. RESULTS The interpersonal manner model was 90% accurate with precision of 0.89, recall of 0.90, and weighted F1-score of 0.89. The technical competence model was 90% accurate with precision of 0.91, recall of 0.90, and weighted F1-score of 0.90. Positive-valence judgments were associated with higher review star ratings whereas negative-valence judgments were associated with lower star ratings. Analysis of the data by review rating and physician gender corresponded with findings in prior literature. CONCLUSIONS Our 2 classification models coded interpersonal manner and technical competence judgments with high precision, recall, and accuracy. These models were validated using review star ratings and results from previous research. RoBERTa can accurately classify unstructured, web-based review text at scale. Future work could explore the use of this algorithm with other textual data, such as social media posts and electronic health records.
Collapse
Affiliation(s)
- Farrah Madanay
- Sanford School of Public Policy, Duke University, Durham, NC, United States
- Center for Bioethics and Social Sciences in Medicine, University of Michigan Medical School, Ann Arbor, MI, United States
| | - Karissa Tu
- Fuqua School of Business, Duke University, Durham, NC, United States
- School of Medicine, University of Washington, Seattle, WA, United States
| | - Ada Campagna
- Center for Advanced Hindsight, Duke University, Durham, NC, United States
- Department of Sociology, University of California, Los Angeles, Los Angeles, CA, United States
| | - J Kelly Davis
- Fuqua School of Business, Duke University, Durham, NC, United States
- Department of Population Health Sciences, Duke University School of Medicine, Durham, NC, United States
| | | | | | - Peter A Ubel
- Sanford School of Public Policy, Duke University, Durham, NC, United States
- Fuqua School of Business, Duke University, Durham, NC, United States
| |
Collapse
|
2
|
Helbich M, Zeng Y, Sarker A. Area-level Measures of the Social Environment: Operationalization, Pitfalls, and Ways Forward. Curr Top Behav Neurosci 2024. [PMID: 38453766 DOI: 10.1007/7854_2024_464] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/09/2024]
Abstract
People's mental health is intertwined with the social environment in which they reside. This chapter explores approaches for quantifying the area-level social environment, focusing specifically on socioeconomic deprivation and social fragmentation. We discuss census data and administrative units, egocentric and ecometric approaches, neighborhood audits, social media data, and street view-based assessments. We close the chapter by discussing possible paths forward from associations between social environments and health to establishing causality, including longitudinal research designs and time-series social environmental indices.
Collapse
Affiliation(s)
- Marco Helbich
- Department of Human Geography and Spatial Planning, Faculty of Geosciences, Utrecht University, Utrecht, The Netherlands.
- Health and Quality of Life in a Green and Sustainable Environment Research Group, Strategic Research and Innovation Program for the Development of MU - Plovdiv, Medical University of Plovdiv, Plovdiv, Bulgaria.
- Environmental Health Division, Research Institute at Medical University of Plovdiv, Medical University of Plovdiv, Plovdiv, Bulgaria.
| | - Yi Zeng
- Department of Human Geography and Spatial Planning, Faculty of Geosciences, Utrecht University, Utrecht, The Netherlands
| | - Abeed Sarker
- Emory University School of Medicine, Atlanta, GA, USA
| |
Collapse
|
3
|
Yang YC, Xie A, Kim S, Hair J, Al-Garadi M, Sarker A. Automatic Detection of Twitter Users Who Express Chronic Stress Experiences via Supervised Machine Learning and Natural Language Processing. Comput Inform Nurs 2023; 41:717-724. [PMID: 36445331 PMCID: PMC10510804 DOI: 10.1097/cin.0000000000000985] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Abstract
Americans bear a high chronic stress burden, particularly during the COVID-19 pandemic. Although social media have many strengths to complement the weaknesses of conventional stress measures, including surveys, they have been rarely utilized to detect individuals self-reporting chronic stress. Thus, this study aimed to develop and evaluate an automatic system on Twitter to identify users who have self-reported chronic stress experiences. Using the Twitter public streaming application programming interface, we collected tweets containing certain stress-related keywords (eg, "chronic," "constant," "stress") and then filtered the data using pre-defined text patterns. We manually annotated tweets with (without) self-report of chronic stress as positive (negative). We trained multiple classifiers and tested them via accuracy and F1 score. We annotated 4195 tweets (1560 positives, 2635 negatives), achieving an inter-annotator agreement of 0.83 (Cohen's kappa). The classifier based on Bidirectional Encoder Representation from Transformers performed the best (accuracy of 83.6% [81.0-86.1]), outperforming the second best-performing classifier (support vector machines: 76.4% [73.5-79.3]). The past tweets from the authors of positive tweets contained useful information, including sources and health impacts of chronic stress. Our study demonstrates that users' self-reported chronic stress experiences can be automatically identified on Twitter, which has a high potential for surveillance and large-scale intervention.
Collapse
|
4
|
Raza S, Schwartz B, Lakamana S, Ge Y, Sarker A. A framework for multi-faceted content analysis of social media chatter regarding non-medical use of prescription medications. BMC DIGITAL HEALTH 2023; 1:29. [PMID: 37680768 PMCID: PMC10483682 DOI: 10.1186/s44247-023-00029-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Accepted: 07/17/2023] [Indexed: 09/09/2023]
Abstract
Background Substance use, including the non-medical use of prescription medications, is a global health problem resulting in hundreds of thousands of overdose deaths and other health problems. Social media has emerged as a potent source of information for studying substance use-related behaviours and their consequences. Mining large-scale social media data on the topic requires the development of natural language processing (NLP) and machine learning frameworks customized for this problem. Our objective in this research is to develop a framework for conducting a content analysis of Twitter chatter about the non-medical use of a set of prescription medications. Methods We collected Twitter data for four medications-fentanyl and morphine (opioids), alprazolam (benzodiazepine), and Adderall® (stimulant), and identified posts that indicated non-medical use using an automatic machine learning classifier. In our NLP framework, we applied supervised named entity recognition (NER) to identify other substances mentioned, symptoms, and adverse events. We applied unsupervised topic modelling to identify latent topics associated with the chatter for each medication. Results The quantitative analysis demonstrated the performance of the proposed NER approach in identifying substance-related entities from data with a high degree of accuracy compared to the baseline methods. The performance evaluation of the topic modelling was also notable. The qualitative analysis revealed knowledge about the use, non-medical use, and side effects of these medications in individuals and communities. Conclusions NLP-based analyses of Twitter chatter associated with prescription medications belonging to different categories provide multi-faceted insights about their use and consequences. Our developed framework can be applied to chatter about other substances. Further research can validate the predictive value of this information on the prevention, assessment, and management of these disorders.
Collapse
Affiliation(s)
- Shaina Raza
- Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
- Vector Institute for Artificial Intelligence, Toronto, ON, Canada
| | - Brian Schwartz
- Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
| | - Sahithi Lakamana
- Department of Biomedical Informatics, School of Medicine, Emory University, Atlanta, GA, USA
| | - Yao Ge
- Department of Biomedical Informatics, School of Medicine, Emory University, Atlanta, GA, USA
| | - Abeed Sarker
- Department of Biomedical Informatics, School of Medicine, Emory University, Atlanta, GA, USA
| |
Collapse
|
5
|
Sarker A, Lakamana S, Guo Y, Ge Y, Leslie A, Okunromade O, Gonzalez-Polledo E, Perrone J, McKenzie-Brown AM. #ChronicPain: Automated Building of a Chronic Pain Cohort from Twitter Using Machine Learning. HEALTH DATA SCIENCE 2023; 3:0078. [PMID: 38333075 PMCID: PMC10852024 DOI: 10.34133/hds.0078] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/03/2023] [Accepted: 06/12/2023] [Indexed: 02/10/2024]
Abstract
Background Due to the high burden of chronic pain, and the detrimental public health consequences of its treatment with opioids, there is a high-priority need to identify effective alternative therapies. Social media is a potentially valuable resource for knowledge about self-reported therapies by chronic pain sufferers. Methods We attempted to (a) verify the presence of large-scale chronic pain-related chatter on Twitter, (b) develop natural language processing and machine learning methods for automatically detecting self-disclosures, (c) collect longitudinal data posted by them, and (d) semiautomatically analyze the types of chronic pain-related information reported by them. We collected data using chronic pain-related hashtags and keywords and manually annotated 4,998 posts to indicate if they were self-reports of chronic pain experiences. We trained and evaluated several state-of-the-art supervised text classification models and deployed the best-performing classifier. We collected all publicly available posts from detected cohort members and conducted manual and natural language processing-driven descriptive analyses. Results Interannotator agreement for the binary annotation was 0.82 (Cohen's kappa). The RoBERTa model performed best (F1 score: 0.84; 95% confidence interval: 0.80 to 0.89), and we used this model to classify all collected unlabeled posts. We discovered 22,795 self-reported chronic pain sufferers and collected over 3 million of their past posts. Further analyses revealed information about, but not limited to, alternative treatments, patient sentiments about treatments, side effects, and self-management strategies. Conclusion Our social media based approach will result in an automatically growing large cohort over time, and the data can be leveraged to identify effective opioid-alternative therapies for diverse chronic pain types.
Collapse
Affiliation(s)
- Abeed Sarker
- Department of Biomedical Informatics, School of Medicine, Emory University, Atlanta, GA, USA
| | - Sahithi Lakamana
- Department of Biomedical Informatics, School of Medicine, Emory University, Atlanta, GA, USA
| | - Yuting Guo
- Department of Biomedical Informatics, School of Medicine, Emory University, Atlanta, GA, USA
| | - Yao Ge
- Department of Biomedical Informatics, School of Medicine, Emory University, Atlanta, GA, USA
| | - Abimbola Leslie
- Department of Radiology, Robert Larner College of Medicine, University of Vermont, Burlington, VT, USA
| | - Omolola Okunromade
- Department of Health Policy and Community Health, Jiann-Ping Hsu College of Public Health, Georgia Southern University, Statesboro, GA, USA
| | | | - Jeanmarie Perrone
- Department of Emergency Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | | |
Collapse
|
6
|
Guo Y, Kim S, Warren E, Yang YC, Lakamana S, Sarker A. Automatic Detection of Intimate Partner Violence Victims from Social Media for Proactive Delivery of Support. AMIA JOINT SUMMITS ON TRANSLATIONAL SCIENCE PROCEEDINGS. AMIA JOINT SUMMITS ON TRANSLATIONAL SCIENCE 2023; 2023:254-260. [PMID: 37351791 PMCID: PMC10283132] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Subscribe] [Scholar Register] [Indexed: 06/24/2023]
Abstract
Social media platforms are increasingly being used by intimate partner violence (IPV) victims to share experiences and seek support. If such information is automatically curated, it may be possible to conduct social media based surveillance and even design interventions over such platforms. In this paper, we describe the development of a supervised classification system that automatically characterizes IPV-related posts on the social network Reddit. We collected data from four IPV-related subreddits and manually annotated the data to indicate whether a post is a self-report of IPV or not. Using the annotated data (N=289), we trained, evaluated, and compared supervised machine learning systems. A transformer-based classifier, RoBERTa, obtained the best classification performance with overall accuracy of 78% and IPV-self-report class 𝐹1 -score of 0.67. Post-classification error analyses revealed that misclassifications often occur for posts that are very long or are non-first-person reports of IPV. Despite the relatively small annotated data, our classification methods obtained promising results, indicating that it may be possible to detect and, hence, provide support to IPV victims over Reddit.
Collapse
Affiliation(s)
- Yuting Guo
- Department of Biomedical Informatics, School of Medicine, Emory University, Atlanta, GA, United States
| | - Sangmi Kim
- School of Nursing, Emory University, Atlanta, GA, United States
| | - Elise Warren
- Rollins School of Public Health, Emory University, Atlanta, GA, United States
| | - Yuan-Chi Yang
- Department of Biomedical Informatics, School of Medicine, Emory University, Atlanta, GA, United States
| | - Sahithi Lakamana
- Department of Biomedical Informatics, School of Medicine, Emory University, Atlanta, GA, United States
| | - Abeed Sarker
- Department of Biomedical Informatics, School of Medicine, Emory University, Atlanta, GA, United States
- Department of Biomedical Engineering, Georgia Institute of Technology and Emory University, Atlanta, GA, United States
| |
Collapse
|
7
|
Bremer W, Plaisance K, Walker D, Bonn M, Love JS, Perrone J, Sarker A. Barriers to opioid use disorder treatment: A comparison of self-reported information from social media with barriers found in literature. Front Public Health 2023; 11:1141093. [PMID: 37151596 PMCID: PMC10158842 DOI: 10.3389/fpubh.2023.1141093] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2023] [Accepted: 03/21/2023] [Indexed: 05/09/2023] Open
Abstract
Introduction Medications such as buprenorphine and methadone are effective for treating opioid use disorder (OUD), but many patients face barriers related to treatment and access. We analyzed two sources of data-social media and published literature-to categorize and quantify such barriers. Methods In this mixed methods study, we analyzed social media (Reddit) posts from three OUD-related forums (subreddits): r/suboxone, r/Methadone, and r/naltrexone. We applied natural language processing to identify posts relevant to treatment barriers, categorized them into insurance- and non-insurance-related, and manually subcategorized them into fine-grained topics. For comparison, we used substance use-, OUD- and barrier-related keywords to identify relevant articles from PubMed published between 2006 and 2022. We searched publications for language expressing fear of barriers, and hesitation or disinterest in medication treatment because of barriers, paying particular attention to the affected population groups described. Results On social media, the top three insurance-related barriers included having no insurance (22.5%), insurance not covering OUD treatment (24.7%), and general difficulties of using insurance for OUD treatment (38.2%); while the top two non-insurance-related barriers included stigma (47.6%), and financial difficulties (26.2%). For published literature, stigma was the most prominently reported barrier, occurring in 78.9% of the publications reviewed, followed by financial and/or logistical issues to receiving medication treatment (73.7%), gender-specific barriers (36.8%), and fear (31.5%). Conclusion The stigma associated with OUD and/or seeking treatment and insurance/cost are the two most common types of barriers reported in the two sources combined. Harm reduction efforts addressing barriers to recovery may benefit from leveraging multiple data sources.
Collapse
Affiliation(s)
- Whitney Bremer
- Department of Biomedical Informatics, School of Medicine, Emory University, Atlanta, GA, United States
- Department of Biomedical Informatics, School of Medicine, College of Engineering and Applied Sciences, Stony Brook University, Stony Brook, NY, United States
| | - Karma Plaisance
- Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, GA, United States
| | - Drew Walker
- Department of Behavioral, Social and Health Education Sciences, Rollins School of Public Health, Emory University, Atlanta, GA, United States
| | - Matthew Bonn
- Canadian Association of People Who Use Drugs, Dartmouth, NS, Canada
| | - Jennifer S. Love
- Department of Emergency Medicine, Icahn School of Medicine at Mount Sinai, New York, NY, United States
| | - Jeanmarie Perrone
- Department of Emergency Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States
| | - Abeed Sarker
- Department of Biomedical Informatics, School of Medicine, Emory University, Atlanta, GA, United States
| |
Collapse
|
8
|
Singh T, Roberts K, Cohen T, Cobb N, Franklin A, Myneni S. Discerning conversational context in online health communities for personalized digital behavior change solutions using Pragmatics to Reveal Intent in Social Media (PRISM) framework. J Biomed Inform 2023; 140:104324. [PMID: 36842490 PMCID: PMC10206862 DOI: 10.1016/j.jbi.2023.104324] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2022] [Revised: 02/18/2023] [Accepted: 02/21/2023] [Indexed: 02/28/2023]
Abstract
BACKGROUND Online health communities (OHCs) have emerged as prominent platforms for behavior modification, and the digitization of online peer interactions has afforded researchers with unique opportunities to model multilevel mechanisms that drive behavior change. Existing studies, however, have been limited by a lack of methods that allow the capture of conversational context and socio-behavioral dynamics at scale, as manifested in these digital platforms. OBJECTIVE We develop, evaluate, and apply a novel methodological framework, Pragmatics to Reveal Intent in Social Media (PRISM), to facilitate granular characterization of peer interactions by combining multidimensional facets of human communication. METHODS We developed and applied PRISM to analyze peer interactions (N = 2.23 million) in QuitNet, an OHC for tobacco cessation. First, we generated a labeled set of peer interactions (n = 2,005) through manual annotation along three dimensions: communication themes (CTs), behavior change techniques (BCTs), and speech acts (SAs). Second, we used deep learning models to apply our qualitative codes at scale. Third, we applied our validated model to perform a retrospective analysis. Finally, using social network analysis (SNA), we portrayed large-scale patterns and relationships among the aforementioned communication dimensions embedded in peer interactions in QuitNet. RESULTS Qualitative analysis showed that the themes of social support and behavioral progress were common. The most used BCTs were feedback and monitoring and comparison of behavior, and users most commonly expressed their intentions using SAs-expressive and emotion. With additional in-domain pre-training, bidirectional encoder representations from Transformers (BERT) outperformed other deep learning models on the classification tasks. Content-specific SNA revealed that users' engagement or abstinence status is associated with the prevalence of various categories of BCTs and SAs, which also was evident from the visualization of network structures. CONCLUSIONS Our study describes the interplay of multilevel characteristics of online communication and their association with individual health behaviors.
Collapse
Affiliation(s)
- Tavleen Singh
- School of Biomedical Informatics, The University of Texas Health Science Center, Houston, TX, USA.
| | - Kirk Roberts
- School of Biomedical Informatics, The University of Texas Health Science Center, Houston, TX, USA
| | - Trevor Cohen
- Biomedical Informatics and Medical Education, The University of Washington, Seattle, WA, USA
| | - Nathan Cobb
- Georgetown University Medical Center, Washington, DC, USA
| | - Amy Franklin
- School of Biomedical Informatics, The University of Texas Health Science Center, Houston, TX, USA
| | - Sahiti Myneni
- School of Biomedical Informatics, The University of Texas Health Science Center, Houston, TX, USA
| |
Collapse
|
9
|
Can accurate demographic information about people who use prescription medications nonmedically be derived from Twitter? Proc Natl Acad Sci U S A 2023; 120:e2207391120. [PMID: 36787355 PMCID: PMC9974473 DOI: 10.1073/pnas.2207391120] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/15/2023] Open
Abstract
Traditional substance use (SU) surveillance methods, such as surveys, incur substantial lags. Due to the continuously evolving trends in SU, insights obtained via such methods are often outdated. Social media-based sources have been proposed for obtaining timely insights, but methods leveraging such data cannot typically provide fine-grained statistics about subpopulations, unlike traditional approaches. We address this gap by developing methods for automatically characterizing a large Twitter nonmedical prescription medication use (NPMU) cohort (n = 288,562) in terms of age-group, race, and gender. Our natural language processing and machine learning methods for automated cohort characterization achieved 0.88 precision (95% CI:0.84 to 0.92) for age-group, 0.90 (95% CI: 0.85 to 0.95) for race, and 94% accuracy (95% CI: 92 to 97) for gender, when evaluated against manually annotated gold-standard data. We compared automatically derived statistics for NPMU of tranquilizers, stimulants, and opioids from Twitter with statistics reported in the National Survey on Drug Use and Health (NSDUH) and the National Emergency Department Sample (NEDS). Distributions automatically estimated from Twitter were mostly consistent with the NSDUH [Spearman r: race: 0.98 (P < 0.005); age-group: 0.67 (P < 0.005); gender: 0.66 (P = 0.27)] and NEDS, with 34/65 (52.3%) of the Twitter-based estimates lying within 95% CIs of estimates from the traditional sources. Explainable differences (e.g., overrepresentation of younger people) were found for age-group-related statistics. Our study demonstrates that accurate subpopulation-specific estimates about SU, particularly NPMU, may be automatically derived from Twitter to obtain earlier insights about targeted subpopulations compared to traditional surveillance approaches.
Collapse
|
10
|
Heeney C, Malden S, Sheikh A. Finding the 'sweet spot' between customisation and workflows when optimising ePrescribing systems: a multisite qualitative study. BMJ Open 2022; 12:e062391. [PMID: 36576184 PMCID: PMC9723895 DOI: 10.1136/bmjopen-2022-062391] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/09/2022] Open
Abstract
OBJECTIVES The introduction of ePrescribing systems offers the potential to improve the safety, quality and efficiency of prescribing, medication management decisions and patient care. However, an ePrescribing system will require some customisation and configuration to capture a range of workflows in particular hospital settings. This can be part of an optimisation strategy, which aims at avoiding workarounds that lessen anticipated safety and efficiency benefits. This paper aims to identify ePrescribing optimisation strategies that can be translated into hospitals in different national settings. We will explore the views of professionals on the impact of configuration and customisation on workflow. DESIGN This paper draws on 54 qualitative interviews with clinicians, pharmacists and informatics professionals with experience of optimising ePrescribing systems in eight hospital sites and one health system, in four different countries. Interview transcripts were analysed using an inductive thematic analysis. SETTING Secondary and tertiary care hospitals in the UK, USA and mainland Europe. PARTICIPANTS Fifty-four healthcare workers with expertise in clinical informatics. RESULTS Five identified themes following thematic analysis showed that optimisation of ePrescribing systems can involve configuration and/or customisation. This can be a strategy to combat workarounds and to respond to local policy, safety protocols and workflows for particular patient populations. However, it can result in sites taking on responsibility for training and missing out on vendor updates. Working closely with vendors and other users can mitigate the need for extensive system modification and produce better outcomes. CONCLUSIONS Modifying an ePrescribing system remains key to enhancing patient safety, and better captured workflow remains key to optimisation. However, we found evidence of an increasingly cautious approach to both customisation and configuration among system users. This has led to users seeking to make less changes to the system.
Collapse
Affiliation(s)
- Catherine Heeney
- Centre for Medical Informatics, Usher Institute, The University of Edinburgh, Edinburgh, UK
| | - Stephen Malden
- Advanced Care Research Centre, Usher Institute, The University of Edinburgh, Edinburgh, UK
| | - Aziz Sheikh
- Division of Community Health Sciences, Usher Institute, The University of Edinburgh, Edinburgh, UK
| |
Collapse
|
11
|
Kumpik DP, Santos-Rodriguez R, Selwood J, Coulthard E, Twomey N, Craddock I, Ben-Shlomo Y. A longitudinal observational study of home-based conversations for detecting early dementia: protocol for the CUBOId TV task. BMJ Open 2022; 12:e065033. [PMID: 36418120 PMCID: PMC9684963 DOI: 10.1136/bmjopen-2022-065033] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
INTRODUCTION Limitations in effective dementia therapies mean that early diagnosis and monitoring are critical for disease management, but current clinical tools are impractical and/or unreliable, and disregard short-term symptom variability. Behavioural biomarkers of cognitive decline, such as speech, sleep and activity patterns, can manifest prodromal pathological changes. They can be continuously measured at home with smart sensing technologies, and permit leveraging of interpersonal interactions for optimising diagnostic and prognostic performance. Here we describe the ContinUous behavioural Biomarkers Of cognitive Impairment (CUBOId) study, which explores the feasibility of multimodal data fusion for in-home monitoring of mild cognitive impairment (MCI) and early Alzheimer's disease (AD). The report focuses on a subset of CUBOId participants who perform a novel speech task, the 'TV task', designed to track changes in ecologically valid conversations with disease progression. METHODS AND ANALYSIS CUBOId is a longitudinal observational study. Participants have diagnoses of MCI or AD, and controls are their live-in partners with no such diagnosis. Multimodal activity data were passively acquired from wearables and in-home fixed sensors over timespans of 8-25 months. At two time points participants completed the TV task over 5 days by recording audio of their conversations as they watched a favourite TV programme, with further testing to be completed after removal of the sensor installations. Behavioural testing is supported by neuropsychological assessment for deriving ground truths on cognitive status. Deep learning will be used to generate fused multimodal activity-speech embeddings for optimisation of diagnostic and predictive performance from speech alone. ETHICS AND DISSEMINATION CUBOId was approved by an NHS Research Ethics Committee (Wales REC; ref: 18/WA/0158) and is sponsored by University of Bristol. It is supported by the National Institute for Health Research Clinical Research Network West of England. Results will be reported at conferences and in peer-reviewed scientific journals.
Collapse
Affiliation(s)
- Daniel Paul Kumpik
- Department of Engineering Mathematics, University of Bristol, Bristol, UK
| | | | - James Selwood
- Bristol Medical School, University of Bristol, Bristol, UK
- Department of Population Health Sciences, University of Bristol, Bristol, UK
| | - Elizabeth Coulthard
- Bristol Medical School, University of Bristol, Bristol, UK
- Department of Translational Health Sciences, University of Bristol, Bristol, UK
| | - Niall Twomey
- Department of Electrical and Electronic Engineering, University of Bristol, Bristol, UK
| | - Ian Craddock
- Department of Electrical and Electronic Engineering, University of Bristol, Bristol, UK
| | - Yoav Ben-Shlomo
- Department of Population Health Sciences, University of Bristol, Bristol, UK
| |
Collapse
|
12
|
Liu W, Zhang X, Lv H, Li J, Liu Y, Yang Z, Weng X, Lin Y, Song H, Wang Z. Using a classification model for determining the value of liver radiological reports of patients with colorectal cancer. Front Oncol 2022; 12:913806. [DOI: 10.3389/fonc.2022.913806] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2022] [Accepted: 11/04/2022] [Indexed: 11/22/2022] Open
Abstract
BackgroundMedical imaging is critical in clinical practice, and high value radiological reports can positively assist clinicians. However, there is a lack of methods for determining the value of reports.ObjectiveThe purpose of this study was to establish an ensemble learning classification model using natural language processing (NLP) applied to the Chinese free text of radiological reports to determine their value for liver lesion detection in patients with colorectal cancer (CRC).MethodsRadiological reports of upper abdominal computed tomography (CT) and magnetic resonance imaging (MRI) were divided into five categories according to the results of liver lesion detection in patients with CRC. The NLP methods including word segmentation, stop word removal, and n-gram language model establishment were applied for each dataset. Then, a word-bag model was built, high-frequency words were selected as features, and an ensemble learning classification model was constructed. Several machine learning methods were applied, including logistic regression (LR), random forest (RF), and so on. We compared the accuracy between priori choosing pertinent word strings and our machine language methodologies.ResultsThe dataset of 2790 patients included CT without contrast (10.2%), CT with/without contrast (73.3%), MRI without contrast (1.8%), and MRI with/without contrast (14.6%). The ensemble learning classification model determined the value of reports effectively, reaching 95.91% in the CT with/without contrast dataset using XGBoost. The logistic regression, random forest, and support vector machine also achieved good classification accuracy, reaching 95.89%, 95.04%, and 95.00% respectively. The results of XGBoost were visualized using a confusion matrix. The numbers of errors in categories I, II and V were very small. ELI5 was used to select important words for each category. Words such as “no abnormality”, “suggest”, “fatty liver”, and “transfer” showed a relatively large degree of positive correlation with classification accuracy. The accuracy based on string pattern search method model was lower than that of machine learning.ConclusionsThe learning classification model based on NLP was an effective tool for determining the value of radiological reports focused on liver lesions. The study made it possible to analyze the value of medical imaging examinations on a large scale.
Collapse
|
13
|
Thakur N. MonkeyPox2022Tweets: A Large-Scale Twitter Dataset on the 2022 Monkeypox Outbreak, Findings from Analysis of Tweets, and Open Research Questions. Infect Dis Rep 2022; 14:855-883. [PMID: 36412745 PMCID: PMC9680479 DOI: 10.3390/idr14060087] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/23/2022] [Revised: 10/13/2022] [Accepted: 11/08/2022] [Indexed: 11/16/2022] Open
Abstract
The mining of Tweets to develop datasets on recent issues, global challenges, pandemics, virus outbreaks, emerging technologies, and trending matters has been of significant interest to the scientific community in the recent past, as such datasets serve as a rich data resource for the investigation of different research questions. Furthermore, the virus outbreaks of the past, such as COVID-19, Ebola, Zika virus, and flu, just to name a few, were associated with various works related to the analysis of the multimodal components of Tweets to infer the different characteristics of conversations on Twitter related to these respective outbreaks. The ongoing outbreak of the monkeypox virus, declared a Global Public Health Emergency (GPHE) by the World Health Organization (WHO), has resulted in a surge of conversations about this outbreak on Twitter, which is resulting in the generation of tremendous amounts of Big Data. There has been no prior work in this field thus far that has focused on mining such conversations to develop a Twitter dataset. Furthermore, no prior work has focused on performing a comprehensive analysis of Tweets about this ongoing outbreak. To address these challenges, this work makes three scientific contributions to this field. First, it presents an open-access dataset of 556,427 Tweets about monkeypox that have been posted on Twitter since the first detected case of this outbreak. A comparative study is also presented that compares this dataset with 36 prior works in this field that focused on the development of Twitter datasets to further uphold the novelty, relevance, and usefulness of this dataset. Second, the paper reports the results of a comprehensive analysis of the Tweets of this dataset. This analysis presents several novel findings; for instance, out of all the 34 languages supported by Twitter, English has been the most used language to post Tweets about monkeypox, about 40,000 Tweets related to monkeypox were posted on the day WHO declared monkeypox as a GPHE, a total of 5470 distinct hashtags have been used on Twitter about this outbreak out of which #monkeypox is the most used hashtag, and Twitter for iPhone has been the leading source of Tweets about the outbreak. The sentiment analysis of the Tweets was also performed, and the results show that despite a lot of discussions, debate, opinions, information, and misinformation, on Twitter on various topics in this regard, such as monkeypox and the LGBTQI+ community, monkeypox and COVID-19, vaccines for monkeypox, etc., "neutral" sentiment was present in most of the Tweets. It was followed by "negative" and "positive" sentiments, respectively. Finally, to support research and development in this field, the paper presents a list of 50 open research questions related to the outbreak in the areas of Big Data, Data Mining, Natural Language Processing, and Machine Learning that may be investigated based on this dataset.
Collapse
Affiliation(s)
- Nirmalya Thakur
- Department of Computer Science, Emory University, Atlanta, GA 30322, USA
| |
Collapse
|
14
|
Chang SW, Zitko P, Passi-Solar Á, Danaei G, Margozzini P. Changes in cardiovascular mortality in Chile during the COVID-19 pandemic. Heart 2022; 108:1716-1721. [PMID: 35105666 PMCID: PMC8822542 DOI: 10.1136/heartjnl-2021-320082] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/03/2021] [Accepted: 12/30/2021] [Indexed: 11/04/2022] Open
Abstract
OBJECTIVES The COVID-19 pandemic has impacted regular cardiovascular healthcare access and delivery. Service utilisation has declined, and excess cardiovascular mortality has been reported in several countries. We aim to estimate excess cardiovascular deaths in Chile during 2020. METHODS We collected mortality data from the Chilean Department of Statistics and Health Information and refined them using the maps of Global Burden of Disease Studies in 2017 and 2010. We conducted a time series analysis using quasi-Poisson distribution to predict cardiovascular mortality in 2020 and compared it with observed numbers and calculated attributable fractions (AFs) with 95% uncertainty intervals, as a whole and by sex, age group and type of cardiovascular disease. RESULTS During 2015-2020, 173 283 cardiovascular deaths were recorded, with 28 141 deaths in 2020. The observation in 2020 was lower than our projection in the overall data (-4.0% (-5.0% to -2.8%)) and in male (-11.7% (-13% to -10.3%)). However, positive AFs were noted among female (5.0% (3.2% to 6.8%)), people in age group 80-89 years (11.0% (8.6% to 13.5%)) and people who died from hypertensive heart diseases (18.9% (14.7% to 23.5%)). CONCLUSIONS Less overall cardiovascular deaths were observed in 2020 compared with our projection, possibly associated with competing risks from COVID-19 infection in men. Nonetheless, excess cardiovascular deaths were observed among women, people in the age group 80-89 years and people who died from hypertensive heart diseases suggesting possible negative cardiovascular impacts brought by the pandemic on these vulnerable groups.
Collapse
Affiliation(s)
- Sheng-Wei Chang
- Global Health and Population, Harvard TH Chan School of Public Health, Boston, Massachusetts, USA
| | - Pedro Zitko
- Department of Public Health, School of Medicine, Pontificia Universidad Catolica de Chile, Santiago, Chile
- Health Services & Population Research, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK
| | - Álvaro Passi-Solar
- Department of Public Health, School of Medicine, Pontificia Universidad Catolica de Chile, Santiago, Chile
- Department of Epidemiology and Public Health, University College London, London, UK
| | - Goodarz Danaei
- Global Health and Population, Harvard TH Chan School of Public Health, Boston, Massachusetts, USA
| | - Paula Margozzini
- Global Health and Population, Harvard TH Chan School of Public Health, Boston, Massachusetts, USA
- Department of Public Health, School of Medicine, Pontificia Universidad Catolica de Chile, Santiago, Chile
| |
Collapse
|
15
|
Böhnke J, Rübsamen N, Mast M, Rathert H, Karch A, Jack T, Wulff A. Prediction models for SIRS, sepsis and associated organ dysfunctions in paediatric intensive care: study protocol for a diagnostic test accuracy study. BMJ Paediatr Open 2022; 6:10.1136/bmjpo-2022-001618. [PMID: 36645795 PMCID: PMC9621157 DOI: 10.1136/bmjpo-2022-001618] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/22/2022] [Accepted: 10/07/2022] [Indexed: 11/06/2022] Open
Abstract
INTRODUCTION Systemic inflammatory response syndrome (SIRS), sepsis and associated organ dysfunctions are life-threating conditions occurring at paediatric intensive care units (PICUs). Early recognition and treatment within the first hours of onset are critical. However, time pressure, lack of personnel resources, and the need for complex age-dependent diagnoses impede an accurate and timely diagnosis by PICU physicians. Data-driven prediction models integrated in clinical decision support systems (CDSS) could facilitate early recognition of disease onset. OBJECTIVES To estimate the sensitivity and specificity of previously developed prediction models (index tests) for the detection of SIRS, sepsis and associated organ dysfunctions in critically ill children up to 12 hours before reference standard diagnosis is possible. METHODS AND ANALYSIS We conduct a monocentre, prospective diagnostic test accuracy study. Clinicians in the PICU of the tertiary care centre Hannover Medical School, Germany, continuously screen and recruit patients until the adaptive sample size (originally intended sample size of 500 patients) is enrolled. Eligible are children (0-17 years, all sexes) who stay in the PICU for ≥12 hours and for whom an informed consent is given. All eligible patients are independently assessed for SIRS, sepsis and organ dysfunctions using corresponding predictive and knowledge-based CDSS models. The knowledge-based CDSS models serve as imperfect reference standards. The assessments are used to estimate the sensitivities and specificities of each predictive model using a clustered nonparametric approach (main analysis). Subgroup analyses ('age groups', 'sex' and 'age groups by sex') are predefined. ETHICS AND DISSEMINATION This study obtained ethics approval from the Hannover Medical School Ethics Committee (No. 10188_BO_SK_2022). Results will be disseminated as peer-reviewed publications, at scientific conferences, and to patients in an appropriate dissemination approach. TRIAL REGISTRATION NUMBER This study was registered with the German Clinical Trial Register (DRKS00029071) on 2022-05-23. PROTOCOL VERSION 10188_BO_SK_2022_V.2.0-20220330_4_Studienprotokoll.
Collapse
Affiliation(s)
- Julia Böhnke
- Institute of Epidemiology and Social Medicine, University of Münster, Münster, Germany
| | - Nicole Rübsamen
- Institute of Epidemiology and Social Medicine, University of Münster, Münster, Germany
| | - Marcel Mast
- Peter L. Reichertz Institute for Medical Informatics, TU Braunschweig and Hannover Medical School, Hannover, Germany
| | - Henning Rathert
- Department of Pediatric Cardiology and Intensive Care Medicine, Hannover Medical School, Hannover, Germany
| | | | - André Karch
- Institute of Epidemiology and Social Medicine, University of Münster, Münster, Germany
| | - Thomas Jack
- Department of Pediatric Cardiology and Intensive Care Medicine, Hannover Medical School, Hannover, Germany
| | - Antje Wulff
- Peter L. Reichertz Institute for Medical Informatics, TU Braunschweig and Hannover Medical School, Hannover, Germany.,Big Data in Medicine, Department of Health Services Research, Carl von Ossietzky University of Oldenburg, Oldenburg, Germany
| |
Collapse
|
16
|
Kabir MK, Islam M, Kabir ANB, Haque A, Rhaman MK. Detection of Depression Severity Using Bengali Social Media Posts on Mental Health: Study Using Natural Language Processing Techniques. JMIR Form Res 2022; 6:e36118. [PMID: 36169989 PMCID: PMC9557762 DOI: 10.2196/36118] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2022] [Revised: 05/16/2022] [Accepted: 07/20/2022] [Indexed: 11/13/2022] Open
Abstract
Background
There are a myriad of language cues that indicate depression in written texts, and natural language processing (NLP) researchers have proven the ability of machine learning and deep learning approaches to detect these cues. However, to date, these approaches bridging NLP and the domain of mental health for Bengali literature are not comprehensive. The Bengali-speaking population can express emotions in their native language in greater detail.
Objective
Our goal is to detect the severity of depression using Bengali texts by generating a novel Bengali corpus of depressive posts. We collaborated with mental health experts to generate a clinically sound labeling scheme and an annotated corpus to train machine learning and deep learning models.
Methods
We conducted a study using Bengali text-based data from blogs and open source platforms. We constructed a procedure for annotated corpus generation and extraction of textual information from Bengali literature for predictive analysis. We developed our own structured data set and designed a clinically sound labeling scheme with the help of mental health professionals, adhering to the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) during the process. We used 5 machine learning models for detecting the severity of depression: kernel support vector machine (SVM), random forest, logistic regression K-nearest neighbor (KNN), and complement naive Bayes (NB). For the deep learning approach, we used long short-term memory (LSTM) units and gated recurrent units (GRUs) coupled with convolutional blocks or self-attention layers. Finally, we aimed for enhanced outcomes by using state-of-the-art pretrained language models.
Results
The independent recurrent neural network (RNN) models yielded the highest accuracies and weighted F1 scores. GRUs, in particular, produced 81% accuracy. The hybrid architectures could not surpass the RNNs in terms of performance. Kernel SVM with term frequency–inverse document frequency (TF-IDF) embeddings generated 78% accuracy on test data. We used validation and training loss curves to observe and report the performance of our architectures. Overall, the number of available data remained the limitation of our experiment.
Conclusions
The findings from our experimental setup indicate that machine learning and deep learning models are fairly capable of assessing the severity of mental health issues from texts. For the future, we suggest more research endeavors to increase the volume of Bengali text data, in particular, so that modern architectures reach improved generalization capability.
Collapse
Affiliation(s)
| | - Maisha Islam
- Department of Computer Science, Brac University, Dhaka, Bangladesh
| | | | - Adiba Haque
- Department of Computer Science, Brac University, Dhaka, Bangladesh
| | - Md Khalilur Rhaman
- Department of Computer Science and Engineering, Brac University, Dhaka, Bangladesh
| |
Collapse
|
17
|
Harmonized system code prediction of import and export commodities based on Hybrid Convolutional Neural Network with Auxiliary Network. Knowl Based Syst 2022. [DOI: 10.1016/j.knosys.2022.109836] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
|
18
|
Guo Y, Ge Y, Yang YC, Al-Garadi MA, Sarker A. Comparison of Pretraining Models and Strategies for Health-Related Social Media Text Classification. Healthcare (Basel) 2022; 10:healthcare10081478. [PMID: 36011135 PMCID: PMC9408372 DOI: 10.3390/healthcare10081478] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2022] [Revised: 07/29/2022] [Accepted: 08/02/2022] [Indexed: 11/24/2022] Open
Abstract
Pretrained contextual language models proposed in the recent past have been reported to achieve state-of-the-art performances in many natural language processing (NLP) tasks, including those involving health-related social media data. We sought to evaluate the effectiveness of different pretrained transformer-based models for social media-based health-related text classification tasks. An additional objective was to explore and propose effective pretraining strategies to improve machine learning performance on such datasets and tasks. We benchmarked six transformer-based models that were pretrained with texts from different domains and sources—BERT, RoBERTa, BERTweet, TwitterBERT, BioClinical_BERT, and BioBERT—on 22 social media-based health-related text classification tasks. For the top-performing models, we explored the possibility of further boosting performance by comparing several pretraining strategies: domain-adaptive pretraining (DAPT), source-adaptive pretraining (SAPT), and a novel approach called topic specific pretraining (TSPT). We also attempted to interpret the impacts of distinct pretraining strategies by visualizing document-level embeddings at different stages of the training process. RoBERTa outperformed BERTweet on most tasks, and better than others. BERT, TwitterBERT, BioClinical_BERT and BioBERT consistently underperformed. For pretraining strategies, SAPT performed better or comparable to the off-the-shelf models, and significantly outperformed DAPT. SAPT + TSPT showed consistently high performance, with statistically significant improvement in three tasks. Our findings demonstrate that RoBERTa and BERTweet are excellent off-the-shelf models for health-related social media text classification, and extended pretraining using SAPT and TSPT can further improve performance.
Collapse
Affiliation(s)
- Yuting Guo
- Department of Biomedical Informatics, Emory University, Atlanta, GA 30322, USA
- Correspondence:
| | - Yao Ge
- Department of Biomedical Informatics, Emory University, Atlanta, GA 30322, USA
| | - Yuan-Chi Yang
- Department of Biomedical Informatics, Emory University, Atlanta, GA 30322, USA
| | - Mohammed Ali Al-Garadi
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN 37240, USA
| | - Abeed Sarker
- Department of Biomedical Informatics, Emory University, Atlanta, GA 30322, USA
| |
Collapse
|
19
|
Sarker A, Al-Garadi MA, Ge Y, Nataraj N, Jones CM, Sumner SA. Signals of increasing co-use of stimulants and opioids from online drug forum data. Harm Reduct J 2022; 19:51. [PMID: 35614501 PMCID: PMC9131693 DOI: 10.1186/s12954-022-00628-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2022] [Accepted: 05/10/2022] [Indexed: 11/21/2022] Open
Abstract
Background Despite recent rises in fatal overdoses involving multiple substances, there is a paucity of knowledge about stimulant co-use patterns among people who use opioids (PWUO) or people being treated with medications for opioid use disorder (PTMOUD). A better understanding of the timing and patterns in stimulant co-use among PWUO based on mentions of these substances on social media can help inform prevention programs, policy, and future research directions. This study examines stimulant co-mention trends among PWUO/PTMOUD on social media over multiple years. Methods We collected publicly available data from 14 forums on Reddit (subreddits) that focused on prescription and illicit opioids, and medications for opioid use disorder (MOUD). Collected data ranged from 2011 to 2020, and we also collected timelines comprising past posts from a sample of Reddit users (Redditors) on these forums. We applied natural language processing to generate lexical variants of all included prescription and illicit opioids and stimulants and detect mentions of them on the chosen subreddits. Finally, we analyzed and described trends and patterns in co-mentions. Results Posts collected for 13,812 Redditors showed that 12,306 (89.1%) mentioned at least 1 opioid, opioid-related medication, or stimulant. Analyses revealed that the number and proportion of Redditors mentioning both opioids and/or opioid-related medications and stimulants steadily increased over time. Relative rates of co-mentions by the same Redditor of heroin and methamphetamine, the substances most commonly co-mentioned, decreased in recent years, while co-mentions of both fentanyl and MOUD with methamphetamine increased. Conclusion Our analyses reflect increasing mentions of stimulants, particularly methamphetamine, among PWUO/PTMOUD, which closely resembles the growth in overdose deaths involving both opioids and stimulants. These findings are consistent with recent reports suggesting increasing stimulant use among people receiving treatment for opioid use disorder. These data offer insights on emerging trends in the overdose epidemic and underscore the importance of scaling efforts to address co-occurring opioid and stimulant use including harm reduction and comprehensive healthcare access spanning mental-health services and substance use disorder treatment. Supplementary Information The online version contains supplementary material available at 10.1186/s12954-022-00628-2.
Collapse
Affiliation(s)
- Abeed Sarker
- Department of Biomedical Informatics, School of Medicine, Emory University, 101 Woodruff Circle, Suite 4101, Atlanta, GA, 30322, USA.
| | - Mohammed Ali Al-Garadi
- Department of Biomedical Informatics, School of Medicine, Emory University, 101 Woodruff Circle, Suite 4101, Atlanta, GA, 30322, USA
| | - Yao Ge
- Department of Biomedical Informatics, School of Medicine, Emory University, 101 Woodruff Circle, Suite 4101, Atlanta, GA, 30322, USA
| | - Nisha Nataraj
- National Center for Injury Prevention and Control, Centers for Disease Control and Prevention, Atlanta, GA, 30341, USA
| | - Christopher M Jones
- National Center for Injury Prevention and Control, Centers for Disease Control and Prevention, Atlanta, GA, 30341, USA
| | - Steven A Sumner
- National Center for Injury Prevention and Control, Centers for Disease Control and Prevention, Atlanta, GA, 30341, USA
| |
Collapse
|
20
|
Clinical Trial Classification of SNS24 Calls with Neural Networks. FUTURE INTERNET 2022. [DOI: 10.3390/fi14050130] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022] Open
Abstract
SNS24, the Portuguese National Health Contact Center, is a telephone and digital public service that provides clinical services. SNS24 plays an important role in the identification of users’ clinical situations according to their symptoms. Currently, there are a number of possible clinical algorithms defined, and selecting the appropriate clinical algorithm is very important in each telephone triage episode. Decreasing the duration of the phone calls and allowing a faster interaction between citizens and SNS24 service can further improve the performance of the telephone triage service. In this paper, we present a study using deep learning approaches to build classification models, aiming to support the nurses with the clinical algorithm’s choice. Three different deep learning architectures, namely convolutional neural network (CNN), recurrent neural network (RNN), and transformers-based approaches are applied across a total number of 269,654 call records belonging to 51 classes. The CNN, RNN, and transformers-based model each achieve an accuracy of 76.56%, 75.88%, and 78.15% over the test set in the preliminary experiments. Models using the transformers-based architecture are further fine-tuned, achieving an accuracy of 79.67% with Adam and 79.72% with SGD after learning rate fine-tuning; an accuracy of 79.96% with Adam and 79.76% with SGD after epochs fine-tuning; an accuracy of 80.57% with Adam after the batch size fine-tuning. Analysis of similar clinical symptoms is carried out using the fine-tuned neural network model. Comparisons are done over the labels predicted by the neural network model, the support vector machines model, and the original labels from SNS24. These results suggest that using deep learning is an effective and promising approach to aid the clinical triage of the SNS24 phone call services.
Collapse
|
21
|
Ebbehoj A, Thunbo MØ, Andersen OE, Glindtvad MV, Hulman A. Transfer learning for non-image data in clinical research: A scoping review. PLOS DIGITAL HEALTH 2022; 1:e0000014. [PMID: 36812540 PMCID: PMC9931256 DOI: 10.1371/journal.pdig.0000014] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/05/2021] [Accepted: 12/15/2021] [Indexed: 01/14/2023]
Abstract
BACKGROUND Transfer learning is a form of machine learning where a pre-trained model trained on a specific task is reused as a starting point and tailored to another task in a different dataset. While transfer learning has garnered considerable attention in medical image analysis, its use for clinical non-image data is not well studied. Therefore, the objective of this scoping review was to explore the use of transfer learning for non-image data in the clinical literature. METHODS AND FINDINGS We systematically searched medical databases (PubMed, EMBASE, CINAHL) for peer-reviewed clinical studies that used transfer learning on human non-image data. We included 83 studies in the review. More than half of the studies (63%) were published within 12 months of the search. Transfer learning was most often applied to time series data (61%), followed by tabular data (18%), audio (12%) and text (8%). Thirty-three (40%) studies applied an image-based model to non-image data after transforming data into images (e.g. spectrograms). Twenty-nine (35%) studies did not have any authors with a health-related affiliation. Many studies used publicly available datasets (66%) and models (49%), but fewer shared their code (27%). CONCLUSIONS In this scoping review, we have described current trends in the use of transfer learning for non-image data in the clinical literature. We found that the use of transfer learning has grown rapidly within the last few years. We have identified studies and demonstrated the potential of transfer learning in clinical research in a wide range of medical specialties. More interdisciplinary collaborations and the wider adaption of reproducible research principles are needed to increase the impact of transfer learning in clinical research.
Collapse
Affiliation(s)
- Andreas Ebbehoj
- Department of Endocrinology and Internal Medicine, Aarhus University Hospital, Denmark
- Department of Clinical Medicine, Aarhus University, Denmark
| | | | | | | | - Adam Hulman
- Steno Diabetes Center Aarhus, Aarhus University Hospital, Denmark
- * E-mail:
| |
Collapse
|
22
|
A Fine-Tuned BERT-Based Transfer Learning Approach for Text Classification. JOURNAL OF HEALTHCARE ENGINEERING 2022; 2022:3498123. [PMID: 35013691 PMCID: PMC8742153 DOI: 10.1155/2022/3498123] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/14/2021] [Revised: 11/25/2021] [Accepted: 12/03/2021] [Indexed: 01/10/2023]
Abstract
Text Classification problem has been thoroughly studied in information retrieval problems and data mining tasks. It is beneficial in multiple tasks including medical diagnose health and care department, targeted marketing, entertainment industry, and group filtering processes. A recent innovation in both data mining and natural language processing gained the attention of researchers from all over the world to develop automated systems for text classification. NLP allows categorizing documents containing different texts. A huge amount of data is generated on social media sites through social media users. Three datasets have been used for experimental purposes including the COVID-19 fake news dataset, COVID-19 English tweet dataset, and extremist-non-extremist dataset which contain news blogs, posts, and tweets related to coronavirus and hate speech. Transfer learning approaches do not experiment on COVID-19 fake news and extremist-non-extremist datasets. Therefore, the proposed work applied transfer learning classification models on both these datasets to check the performance of transfer learning models. Models are trained and evaluated on the accuracy, precision, recall, and F1-score. Heat maps are also generated for every model. In the end, future directions are proposed.
Collapse
|
23
|
Kalyan KS, Rajasekharan A, Sangeetha S. AMMU: A survey of transformer-based biomedical pretrained language models. J Biomed Inform 2021; 126:103982. [PMID: 34974190 DOI: 10.1016/j.jbi.2021.103982] [Citation(s) in RCA: 25] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2021] [Revised: 12/12/2021] [Accepted: 12/20/2021] [Indexed: 01/04/2023]
Abstract
Transformer-based pretrained language models (PLMs) have started a new era in modern natural language processing (NLP). These models combine the power of transformers, transfer learning, and self-supervised learning (SSL). Following the success of these models in the general domain, the biomedical research community has developed various in-domain PLMs starting from BioBERT to the latest BioELECTRA and BioALBERT models. We strongly believe there is a need for a survey paper that can provide a comprehensive survey of various transformer-based biomedical pretrained language models (BPLMs). In this survey, we start with a brief overview of foundational concepts like self-supervised learning, embedding layer and transformer encoder layers. We discuss core concepts of transformer-based PLMs like pretraining methods, pretraining tasks, fine-tuning methods, and various embedding types specific to biomedical domain. We introduce a taxonomy for transformer-based BPLMs and then discuss all the models. We discuss various challenges and present possible solutions. We conclude by highlighting some of the open issues which will drive the research community to further improve transformer-based BPLMs. The list of all the publicly available transformer-based BPLMs along with their links is provided at https://mr-nlp.github.io/posts/2021/05/transformer-based-biomedical-pretrained-language-models-list/.
Collapse
|
24
|
Ali SI, Jung SW, Bilal HSM, Lee SH, Hussain J, Afzal M, Hussain M, Ali T, Chung T, Lee S. Clinical Decision Support System Based on Hybrid Knowledge Modeling: A Case Study of Chronic Kidney Disease-Mineral and Bone Disorder Treatment. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2021; 19:226. [PMID: 35010486 PMCID: PMC8750681 DOI: 10.3390/ijerph19010226] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/14/2021] [Accepted: 12/20/2021] [Indexed: 11/30/2022]
Abstract
Clinical decision support systems (CDSSs) represent the latest technological transformation in healthcare for assisting clinicians in complex decision-making. Several CDSSs are proposed to deal with a range of clinical tasks such as disease diagnosis, prescription management, and medication ordering. Although a small number of CDSSs have focused on treatment selection, areas such as medication selection and dosing selection remained under-researched. In this regard, this study represents one of the first studies in which a CDSS is proposed for clinicians who manage patients with end-stage renal disease undergoing maintenance hemodialysis, almost all of whom have some manifestation of chronic kidney disease-mineral and bone disorder (CKD-MBD). The primary objective of the system is to aid clinicians in dosage prescription by levering medical domain knowledge as well existing practices. The proposed CDSS is evaluated with a real-world hemodialysis patient dataset acquired from Kyung Hee University Hospital, South Korea. Our evaluation demonstrates overall high compliance based on the concordance metric between the proposed CKD-MBD CDSS recommendations and the routine clinical practice. The concordance rate of overall medication dosing selection is 78.27%. Furthermore, the usability aspects of the system are also evaluated through the User Experience Questionnaire method to highlight the appealing aspects of the system for clinicians. The overall user experience dimension scores for pragmatic, hedonic, and attractiveness are 1.53, 1.48, and 1.41, respectively. A service reliability for the Cronbach's alpha coefficient greater than 0.7 is achieved using the proposed system, whereas a dependability coefficient of the value 0.84 reveals a significant effect.
Collapse
Affiliation(s)
- Syed Imran Ali
- Department of Computer Science and Engineering, Kyung Hee University, Yongin-si 17104, Korea; (S.I.A.); (H.S.M.B.)
| | - Su Woong Jung
- Department of Internal Medicine, Division of Nephrology, Kyung Hee University Hospital at Gangdong, Seoul 05278, Korea;
| | - Hafiz Syed Muhammad Bilal
- Department of Computer Science and Engineering, Kyung Hee University, Yongin-si 17104, Korea; (S.I.A.); (H.S.M.B.)
- Department of Computing, SEECS, NUST University, Islamabad 44000, Pakistan
| | - Sang-Ho Lee
- Department of Internal Medicine, Division of Nephrology, Kyung Hee University Hospital at Gangdong, Seoul 05278, Korea;
| | - Jamil Hussain
- Department of Data Science, Sejong University, Seoul 30019, Korea;
| | - Muhammad Afzal
- Department of Software, Sejong University, Seoul 30019, Korea; (M.A.); (M.H.)
| | - Maqbool Hussain
- Department of Software, Sejong University, Seoul 30019, Korea; (M.A.); (M.H.)
| | - Taqdir Ali
- BC Children’s Hospital, University of British Columbia, Vancouver, BC V6H 3N1, Canada;
| | - Taechoong Chung
- Department of Computer Science and Engineering, Kyung Hee University, Yongin-si 17104, Korea; (S.I.A.); (H.S.M.B.)
| | - Sungyoung Lee
- Department of Computer Science and Engineering, Kyung Hee University, Yongin-si 17104, Korea; (S.I.A.); (H.S.M.B.)
| |
Collapse
|
25
|
Yang YC, Al-Garadi MA, Love JS, Perrone J, Sarker A. Automatic gender detection in Twitter profiles for health-related cohort studies. JAMIA Open 2021; 4:ooab042. [PMID: 34169232 PMCID: PMC8220305 DOI: 10.1093/jamiaopen/ooab042] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2021] [Revised: 04/27/2021] [Accepted: 05/04/2021] [Indexed: 11/17/2022] Open
Abstract
Objective Biomedical research involving social media data is gradually moving from population-level to targeted, cohort-level data analysis. Though crucial for biomedical studies, social media user’s demographic information (eg, gender) is often not explicitly known from profiles. Here, we present an automatic gender classification system for social media and we illustrate how gender information can be incorporated into a social media-based health-related study. Materials and Methods We used a large Twitter dataset composed of public, gender-labeled users (Dataset-1) for training and evaluating the gender detection pipeline. We experimented with machine learning algorithms including support vector machines (SVMs) and deep-learning models, and public packages including M3. We considered users’ information including profile and tweets for classification. We also developed a meta-classifier ensemble that strategically uses the predicted scores from the classifiers. We then applied the best-performing pipeline to Twitter users who have self-reported nonmedical use of prescription medications (Dataset-2) to assess the system’s utility. Results and Discussion We collected 67 181 and 176 683 users for Dataset-1 and Dataset-2, respectively. A meta-classifier involving SVM and M3 performed the best (Dataset-1 accuracy: 94.4% [95% confidence interval: 94.0–94.8%]; Dataset-2: 94.4% [95% confidence interval: 92.0–96.6%]). Including automatically classified information in the analyses of Dataset-2 revealed gender-specific trends—proportions of females closely resemble data from the National Survey of Drug Use and Health 2018 (tranquilizers: 0.50 vs 0.50; stimulants: 0.50 vs 0.45), and the overdose Emergency Room Visit due to Opioids by Nationwide Emergency Department Sample (pain relievers: 0.38 vs 0.37). Conclusion Our publicly available, automated gender detection pipeline may aid cohort-specific social media data analyses (https://bitbucket.org/sarkerlab/gender-detection-for-public).
Collapse
Affiliation(s)
- Yuan-Chi Yang
- Department of Biomedical Informatics, School of Medicine, Emory University, Atlanta, Georgia, USA
| | - Mohammed Ali Al-Garadi
- Department of Biomedical Informatics, School of Medicine, Emory University, Atlanta, Georgia, USA
| | - Jennifer S Love
- Department of Emergency Medicine, School of Medicine, Oregon Health & Science University, Portland, Oregon, USA
| | - Jeanmarie Perrone
- Department of Emergency Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Abeed Sarker
- Department of Biomedical Informatics, School of Medicine, Emory University, Atlanta, Georgia, USA.,Department of Biomedical Engineering, Georgia Institute of Technology and Emory University, Atlanta, Georgia, USA
| |
Collapse
|
26
|
The Impact of COVID-19 on Consumers' Psychological Behavior Based on Data Mining for Online User Comments in the Catering Industry in China. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2021; 18:ijerph18084178. [PMID: 33920875 PMCID: PMC8071308 DOI: 10.3390/ijerph18084178] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/08/2021] [Revised: 04/08/2021] [Accepted: 04/09/2021] [Indexed: 12/28/2022]
Abstract
The outbreak of COVID-19 in late 2019 has had a huge impact on people’s daily life. Many restaurant businesses have been greatly affected by it. Consumers’ preferences for catering industry in China have changed, such as environmental hygiene, variety of dishes, and service methods. Therefore, the analysis of consumer preference differences and changes before and after the epidemic can not only provide emergency strategies for the catering industry but further improve the catering industry’s ability to deal with public health emergencies. This paper takes five cities in China as representatives to explore the impact of COVID-19 on China’s catering industry. Based on catering review data from August 2019 to April 2020, this paper first carries out Latent Dirichlet Allocation (LDA) topic analysis and SNOWNLP (A Python library for processing Chinese text) sentiment analysis. Then this paper compares the results of topic classification and sentiment analysis before and after the epidemic. Furthermore, differences and changes of consumer preferences are obtained and preferences of consumers under COVID-19 are analyzed and forecasted. The results of LDA thematic analysis before the outbreak of COVID-19 show that consumers tend to punch in cyber celebrity restaurants and pay more attention to the taste of dishes, whereas after it consumers pay more attention to the changes of dishes, dining environment as well as epidemic prevention. The number of packages and takeout was also increasing. However, the waiting time is constantly considered by consumers before and after COVID-19. Firstly, to our surprise, final outcome of emotional analysis showed that consumers’ emotional state was more positive after the epidemic than before. COVID-19 has changed the lifestyle of consumers, consumption concepts, and consumption habits. Therefore, businesses also need to take positive and flexible measures to actively get feedback from consumers to adjust dishes and business methods. Secondly, the psychological attitude of catering consumers is relatively positive during the epidemic period, which indicates that consumers have great confidence in the recovery and development of the catering industry. Businesses can comply with consumers’ psychology and combine consumption vouchers with restaurant discounts to promote consumers’ consumption. Finally, the environment and service play more and more important effect on consumers’ emotional scores at present, which indicates that dining state and comfortable mealtime environment are becoming increasingly valuable. Therefore, businesses need to improve service standards.
Collapse
|