Machine learning techniques for the automation of literature reviews and systematic reviews in EFSA

Evaluating algorithms of decision tree, support vector machine and regression for anode side catalyst data in proton exchange membrane water electrolysis

Nowadays, due to the various type of problems stemmed from using chemical compounds and fossil fuels which have widely influence on whole environment including acid rain, polar ice melting and etc., number of researches have been leading on replacing the nonrenewable energy sources with renewable ones in order to produce clean fuels. Among these, hydrogen emerges as a quintessential clean fuel, garnering substantial attention for its potential to be synthesized from the electric power generated by renewable sources like nuclear and solar energies. This is achieved through the employment of a proton exchange membrane water electrolysis (PEMWE) system, widely recognized as one of the most proficient and economically viable technologies for effecting the separation of H2O into H+ and OH-. In this study, the important affecting parameters on the anode side of catalyst in PEMWE and analyzed them by machine-learning (ML) algorithms through developing a data science (DS) procedure were discussed. Various machine learning models were subjected to comparison, wherein the Decision Tree models, specifically those configured with maximum depths of 3 and 4, emerged as the optimal choices, attaining a perfect 100% accuracy across both Dataset 1 and Dataset 2. Moreover, notable enhancements in accuracy values were observed for the Support Vector Machine (SVM) model, registering increments from 0.79 to 0.82 for Dataset 1 and 2, respectively. In stark contrast, the remaining models experienced a decrement in their accuracy scores. This phenomenon underscores the pivotal role played by the data generation process in rendering the models more faithful to real-world scenarios.

Search, identification, and curation of cell and gene therapy product regulations using augmented intelligent systems

Background

Manually keeping up-to-date with regulations such as directives, guidance, laws, and ordinances related to cell and gene therapy is a labor-intensive process. We used machine learning (ML) algorithms to create an augmented intelligent system to optimize systematic screening of global regulations to improve efficiency and reduce overall labor and missed regulations.

Methods

Combining Boolean logic and artificial intelligence (i.e., augmented intelligence) for the search process, ML algorithms were used to identify and suggest relevant cell and gene therapy regulations. Suggested regulations were delivered to a landing page for further subject matter expert (SME) tagging of words/phrases to provide system relevance on functional words. Ongoing learning from the repository regulations continued to increase system reliability and performance. The automated ability to train and retrain the system allows for continued refinement and improvement of system accuracy. Automated daily searches for applicable regulations in global databases provide ongoing opportunities to update the repository.

Results

Compared to manual searching, which required 3-4 SMEs to review ~115 regulations, the current system performance, with continuous system learning, requires 1 full-time equivalent to process approximately 9,000 regulations/day. Currently, system performance has 86% overall accuracy, a recommend recall of 87%, and a reject recall of 84%. A conservative search strategy is intentionally used to permit SMEs to assess low-recommended regulations in order to prevent missing any applicable regulations.

Conclusion

Compared to manual searches, our custom automated search system greatly improves the management of cell and gene therapy regulations and is efficient, cost effective, and accurate.

Machine learning computational tools to assist the performance of systematic reviews: A mapping review

BACKGROUND

Within evidence-based practice (EBP), systematic reviews (SR) are considered the highest level of evidence in that they summarize the best available research and describe the progress in a determined field. Due its methodology, SR require significant time and resources to be performed; they also require repetitive steps that may introduce biases and human errors. Machine learning (ML) algorithms therefore present a promising alternative and a potential game changer to speed up and automate the SR process. This review aims to map the current availability of computational tools that use ML techniques to assist in the performance of SR, and to support authors in the selection of the right software for the performance of evidence synthesis.

METHODS

The mapping review was based on comprehensive searches in electronic databases and software repositories to obtain relevant literature and records, followed by screening for eligibility based on titles, abstracts, and full text by two reviewers. The data extraction consisted of listing and extracting the name and basic characteristics of the included tools, for example a tool's applicability to the various SR stages, pricing options, open-source availability, and type of software. These tools were classified and graphically represented to facilitate the description of our findings.

RESULTS

A total of 9653 studies and 585 records were obtained from the structured searches performed on selected bibliometric databases and software repositories respectively. After screening, a total of 119 descriptions from publications and records allowed us to identify 63 tools that assist the SR process using ML techniques.

CONCLUSIONS

This review provides a high-quality map of currently available ML software to assist the performance of SR. ML algorithms are arguably one of the best techniques at present for the automation of SR. The most promising tools were easily accessible and included a high number of user-friendly features permitting the automation of SR and other kinds of evidence synthesis reviews.

Future perspectives for advancing regulatory science of nanotechnology-enabled health products

The identification of regulatory challenges for nanotechnology-enabled health products, followed by discussions with the involved stakeholders, is the first step towards a strategic planning of how such challenges can be successfully addressed in the future. In order to better understand whether the identified regulatory needs are sector-specific for health products or might also hinder the progress in other domains, the REFINE consortium reached out to communities representing other sectors that also exploit the potential of nanotechnology, i.e. industrial chemicals, food and cosmetics. Through a series of trans-sectorial workshops, REFINE partners identified common as well as sector-specific challenges and discussed possible ways forward. Potential solutions lie in a more strengthen collaboration between regulatory and research communities resulting in a targeted production and exploitation of academic data for the regulatory decision-making. Furthermore, a coordinated use of knowledge sharing platforms and databases, trans-sectorial standardisation activities and harmonisation of regulatory activities between geographical regions are possible ways forward, in line with the upcoming European political initiatives such as the Chemical Strategy for Sustainability (CSS). Finally, we also discuss the perspectives for further development and sustainability of methods and tools developed in the REFINE project.

Toxic effects of nanomaterials for health applications: How automation can support a systematic review of the literature?

Systematic reviews of the scientific literature can be an important source of information supporting the daily work of the regulators in their decision making, particularly in areas of innovative technologies where the regulatory experience is still limited. Significant research activities in the field of nanotechnology resulted in a huge number of publications in the last decades. However, even if the published data can provide relevant information, scientific articles are often of diverse quality, and it is nearly impossible to manually process and evaluate such amount of data in a systematic manner. In this feasibility study, we investigated to what extent open-access automation tools can support a systematic review of toxic effects of nanomaterials for health applications reported in the scientific literature. In this study, we used a battery of available tools to perform the initial steps of a systematic review such as targeted searches, data curation and abstract screening. This work was complemented with an in-house developed tool that allowed us to extract specific sections of the articles such as the materials and methods part or the results section where we could perform subsequent text analysis. We ranked the articles according to quality criteria based on the reported nanomaterial characterisation and extracted most frequently described toxic effects induced by different types of nanomaterials. Even if further demonstration of the reliability and applicability of automation tools is necessary, this study demonstrated the potential to leverage information from the scientific literature by using automation systems in a tiered strategy.

Applying machine classifiers to update searches: Analysis from two case studies

Manual screening of citation records could be reduced by using machine classifiers to remove records of very low relevance. This seems particularly feasible for update searches, where a machine classifier can be trained from past screening decisions. However, feasibility is unclear for broad topics. We evaluate the performance and implementation of machine classifiers for update searches of public health research using two case studies. The first study evaluates the impact of using different sets of training data on classifier performance, comparing recall and screening reduction with a manual screening ‘gold standard’. The second study uses screening decisions from a review to train a classifier that is applied to rank the update search results. A stopping threshold was applied in the absence of a gold standard. Time spent screening titles and abstracts of different relevancy‐ranked records was measured. Results: Study one: Classifier performance varies according to the training data used; all custom‐built classifiers had a recall above 93% at the same threshold, achieving screening reductions between 41% and 74%. Study two: applying a classifier provided a solution for tackling a large volume of search results from the update search, and screening volume was reduced by 61%. A tentative estimate indicates over 25 h screening time was saved. In conclusion, custom‐built machine classifiers are feasible for reducing screening workload from update searches across a range of public health interventions, with some limitation on recall. Key considerations include selecting a training dataset, agreeing stopping thresholds and processes to ensure smooth workflows.

Principle-based recommendations for big data and machine learning in food safety: the P-SAFETY model

Big data and Machine learning Techniques are reshaping the way in which food safety risk assessment is conducted. The ongoing ‘datafication’ of food safety risk assessment activities and the progressive deployment of probabilistic models in their practices requires a discussion on the advantages and disadvantages of these advances. In particular, the low level of trust in EU food safety risk assessment framework highlighted in 2019 by an EU-funded survey could be exacerbated by novel methods of analysis. The variety of processed data raises unique questions regarding the interplay of multiple regulatory systems alongside food safety legislation. Provisions aiming to preserve the confidentiality of data and protect personal information are juxtaposed to norms prescribing the public disclosure of scientific information. This research is intended to provide guidance for data governance and data ownership issues that unfold from the ongoing transformation of the technical and legal domains of food safety risk assessment. Following the reconstruction of technological advances in data collection and analysis and the description of recent amendments to food safety legislation, emerging concerns are discussed in light of the individual, collective and social implications of the deployment of cutting-edge Big Data collection and analysis techniques. Then, a set of principle-based recommendations is proposed by adapting high-level principles enshrined in institutional documents about Artificial Intelligence to the realm of food safety risk assessment. The proposed set of recommendations adopts Safety, Accountability, Fairness, Explainability, Transparency as core principles (SAFETY), whereas Privacy and data protection are used as a meta-principle.

Iterative guided machine learning-assisted systematic literature reviews: a diabetes case study

BACKGROUND

Systematic Reviews (SR), studies of studies, use a formal process to evaluate the quality of scientific literature and determine ensuing effectiveness from qualifying articles to establish consensus findings around a hypothesis. Their value is increasing as the conduct and publication of research and evaluation has expanded and the process of identifying key insights becomes more time consuming. Text analytics and machine learning (ML) techniques may help overcome this problem of scale while still maintaining the level of rigor expected of SRs.

METHODS

In this article, we discuss an approach that uses existing examples of SRs to build and test a method for assisting the SR title and abstract pre-screening by reducing the initial pool of potential articles down to articles that meet inclusion criteria. Our approach differs from previous approaches to using ML as a SR tool in that it incorporates ML configurations guided by previously conducted SRs, and human confirmation on ML predictions of relevant articles during multiple iterative reviews on smaller tranches of citations. We applied the tailored method to a new SR review effort to validate performance.

RESULTS

The case study test of the approach proved a sensitivity (recall) in finding relevant articles during down selection that may rival many traditional processes and show ability to overcome most type II errors. The study achieved a sensitivity of 99.5% (213 out of 214) of total relevant articles while only conducting a human review of 31% of total articles available for review.

CONCLUSIONS

We believe this iterative method can help overcome bias in initial ML model training by having humans reinforce ML models with new and relevant information, and is an applied step towards transfer learning for ML in SR.

On the use of computer-assistance to facilitate systematic mapping

The volume of published academic research is growing rapidly and this new era of "big literature" poses new challenges to evidence synthesis, pushing traditional, manual methods of evidence synthesis to their limits. New technology developments, including machine learning, are likely to provide solutions to the problem of information overload and allow scaling of systematic maps to large and even vast literatures. In this paper, we outline how systematic maps lend themselves well to automation and computer-assistance. We believe that it is a major priority to consolidate efforts to develop and validate efficient, rigorous and robust applications of these novel technologies, ensuring the challenges of big literature do not prevent the future production of systematic maps.

Integration of computational tools, data analysis and social science into food safety risk assessment

The EU-FORA Fellowship Programme 'Integration of tools and social science into food safety risk assessments' was proposed by the Food Standards Agency (FSA), the government department responsible for food safety in the UK. The working programme was organised into four modules, covering different areas of risk assessment, including microbiological risk assessment, chemical risk assessment, exposure assessment, risk prioritisation and the integration of risk assessment with social science. During this period, the fellow had the unique opportunity to gain experience in different fields of risk assessment, namely how to conduct a systematic review, to assess the risk of microbiological and chemical hazards, to make use of modelling tools for exposure assessment and risk prioritisation, to write scientific reports for committees and networks at the national level and to understand the role of social science in risk assessment. In addition, the fellow was able to attend several meetings, seminars, courses and workshops that helped him to gain further insight in the field of food science. The complete programme enabled a fast learning curve that allowed the fellow to have an overview of the different tools that can be employed in the wide field of food safety risk assessment, in order to acquire skills and competences that can be used in his future career.