Toward automatic evaluation of medical abstracts: The current value of sentiment analysis and machine learning for classification of the importance of PubMed abstracts of randomized trials for stroke

The 3Ranker: An AI-based Algorithm for Finding Non-animal Alternative Methods

The search for existing non-animal alternative methods for use in experiments is currently challenging because of the lack of both comprehensive structured databases and balanced keyword-based search strategies to mine unstructured textual databases. In this paper we describe 3Ranker, which is a fast, keyword-independent algorithm for finding non-animal alternative methods for use in biomedical research. The 3Ranker algorithm was created by using a machine learning approach, consisting of a Random Forest model built on a dataset of 35 million abstracts and constructed with weak supervision, followed by iterative model improvement with expert curated data. We found a satisfactory trade-off between sensitivity and specificity, with Area Under the Curve (AUC) values ranging from 0.85-0.95. Trials showed that the AI-based classifier was able to identify articles that describe potential alternatives to animal use, among the thousands of articles returned by generic PubMed queries on dermatitis and Parkinson's disease. Application of the classification models on time series data showed the earlier implementation and acceptance of Three Rs principles in the area of cosmetics and skin research, as compared to the area of neurodegenerative disease research. The 3Ranker algorithm is freely available at www.open3r.org; the future goal is to expand this framework to cover multiple research domains and to enable its broad use by researchers, policymakers, funders and ethical review boards, in order to promote the replacement of animal use in research wherever possible.

Social influence makes outlier opinions in online reviews offer more helpful information

Identifying helpful information from large-scale online reviews has become a core issue in studies on harnessing wisdom-of-crowds. We investigated whether online reviews expressing dissenting opinions (i.e., outlier reviews) can provide helpful information. Using statistical and simulation methods with a large-scale dataset, we found that, compared with other online reviews, outlier reviews were deemed more helpful because they provided more sufficient, neutral, and concise information. To interpret these results, we considered that in collective behaviours, a prevalent social psychological process-conformity (i.e., changing one's behaviour in response to pressure from others)-pressured reviewers expressing dissenting opinions. This motivated them to provide more convincing evidence (i.e., sufficient, neutral, and concise information). This study offers a simple yet effective approach for eliciting helpful information from many online reviews and deepens the understanding of the mechanism underlying collective online behaviour. Specifically, conformity was considered to cause biases in the collective behaviour of humans; however, this study revealed that conformity can elicit valuable outcomes in collective behaviour.

The use of artificial intelligence for automating or semi-automating biomedical literature analyses: a scoping review

OBJECTIVE

Evidence-based medicine (EBM) is a decision-making process based on the conscious and judicious use of the best available scientific evidence. However, the exponential increase in the amount of information currently available likely exceeds the capacity of human-only analysis. In this context, artificial intelligence (AI) and its branches such as machine learning (ML) can be used to facilitate human efforts in analyzing the literature to foster EBM. The present scoping review aimed to examine the use of AI in the automation of biomedical literature survey and analysis with a view to establishing the state-of-the-art and identifying knowledge gaps.

MATERIALS AND METHODS

Comprehensive searches of the main databases were performed for articles published up to June 2022 and studies were selected according to inclusion and exclusion criteria. Data were extracted from the included articles and the findings categorized.

RESULTS

The total number of records retrieved from the databases was 12,145, of which 273 were included in the review. Classification of the studies according to the use of AI in evaluating the biomedical literature revealed three main application groups, namely assembly of scientific evidence (n=127; 47%), mining the biomedical literature (n=112; 41%) and quality analysis (n=34; 12%). Most studies addressed the preparation of systematic reviews, while articles focusing on the development of guidelines and evidence synthesis were the least frequent. The biggest knowledge gap was identified within the quality analysis group, particularly regarding methods and tools that assess the strength of recommendation and consistency of evidence.

CONCLUSION

Our review shows that, despite significant progress in the automation of biomedical literature surveys and analyses in recent years, intense research is needed to fill knowledge gaps on more difficult aspects of ML, deep learning and natural language processing, and to consolidate the use of automation by end-users (biomedical researchers and healthcare professionals).

Validating GAN-BioBERT: A Methodology for Assessing Reporting Trends in Clinical Trials

Background

The aim of this study was to validate a three-class sentiment classification model for clinical trial abstracts combining adversarial learning and the BioBERT language processing model as a tool to assess trends in biomedical literature in a clearly reproducible manner. We then assessed the model's performance for this application and compared it to previous models used for this task.

Methods

Using 108 expert-annotated clinical trial abstracts and 2,000 unlabeled abstracts this study develops a three-class sentiment classification algorithm for clinical trial abstracts. The model uses a semi-supervised model based on the Bidirectional Encoder Representation from Transformers (BERT) model, a much more advanced and accurate method compared to previously used models based upon traditional machine learning methods. The prediction performance was compared to those previous studies.

Results

The algorithm was found to have a classification accuracy of 91.3%, with a macro F1-Score of 0.92, significantly outperforming previous studies used to classify sentiment in clinical trial literature, while also making the sentiment classification finer grained with greater reproducibility.

Conclusion

We demonstrate an easily applied sentiment classification model for clinical trial abstracts that significantly outperforms previous models with greater reproducibility and applicability to large-scale study of reporting trends.

Opinion mining for national security: techniques, domain applications, challenges and research opportunities

BACKGROUND

Opinion mining, or sentiment analysis, is a field in Natural Language Processing (NLP). It extracts people's thoughts, including assessments, attitudes, and emotions toward individuals, topics, and events. The task is technically challenging but incredibly useful. With the explosive growth of the digital platform in cyberspace, such as blogs and social networks, individuals and organisations are increasingly utilising public opinion for their decision-making. In recent years, significant research concerning mining people's sentiments based on text in cyberspace using opinion mining has been explored. Researchers have applied numerous opinions mining techniques, including machine learning and lexicon-based approach to analyse and classify people's sentiments based on a text and discuss the existing gap. Thus, it creates a research opportunity for other researchers to investigate and propose improved methods and new domain applications to fill the gap.

METHODS

In this paper, a structured literature review has been done by considering 122 articles to examine all relevant research accomplished in the field of opinion mining application and the suggested Kansei approach to solve the challenges that occur in mining sentiments based on text in cyberspace. Five different platforms database were systematically searched between 2015 and 2021: ACM (Association for Computing Machinery), IEEE (Advancing Technology for Humanity), SCIENCE DIRECT, SpringerLink, and SCOPUS.

RESULTS

This study analyses various techniques of opinion mining as well as the Kansei approach that will help to enhance techniques in mining people's sentiment and emotion in cyberspace. Most of the study addressed methods including machine learning, lexicon-based approach, hybrid approach, and Kansei approach in mining the sentiment and emotion based on text. The possible societal impacts of the current opinion mining technique, including machine learning and the Kansei approach, along with major trends and challenges, are highlighted.

CONCLUSION

Various applications of opinion mining techniques in mining people's sentiment and emotion according to the objective of the research, used method, dataset, summarized in this study. This study serves as a theoretical analysis of the opinion mining method complemented by the Kansei approach in classifying people's sentiments based on text in cyberspace. Kansei approach can measure people's impressions using artefacts based on senses including sight, feeling and cognition reported precise results for the assessment of human emotion. Therefore, this research suggests that the Kansei approach should be a complementary factor including in the development of a dictionary focusing on emotion in the national security domain. Also, this theoretical analysis will act as a reference to researchers regarding the Kansei approach as one of the techniques to improve hybrid approaches in opinion mining.