The efficiency of machine learning-assisted platform for article screening in systematic reviews in orthopaedics

Integrating AI and Assistive Technologies in Healthcare: Insights from a Narrative Review of Reviews

The integration of artificial intelligence (AI) into assistive technologies is an emerging field with transformative potential, aimed at enhancing autonomy and quality of life for individuals with disabilities and aging populations. This overview of reviews, utilizing a standardized checklist and quality control procedures, examines recent advancements and future implications in this domain. The search for articles for the review was finalized by 15 December 2024. Nineteen review studies were selected through a systematic process identifying prevailing themes, opportunities, challenges, and recommendations regarding the integration of AI in assistive technologies. First, AI is increasingly central to improving mobility, healthcare diagnostics, and cognitive support, enabling personalized and adaptive solutions for users. The integration of AI into traditional assistive technologies, such as smart wheelchairs and exoskeletons, enhances their performance, creating more intuitive and responsive devices. Additionally, AI is improving the inclusion of children with autism spectrum disorders, promoting social interaction and cognitive development through innovative devices. The review also identifies significant opportunities and challenges. AI-powered assistive technologies offer enormous potential to increase independence, reduce reliance on external support, and improve communication for individuals with cognitive disorders. However, challenges such as personalization, digital literacy among the elderly, and privacy concerns in healthcare contexts need to be addressed. Notably, AI itself is expanding the concept of assistive technology, shifting from traditional tools to intelligent systems capable of learning and adapting to individual needs. This evolution represents a fundamental change in assistive technology, emphasizing dynamic, adaptive systems over static solutions. Finally, the study emphasizes the growing economic investment in this sector, forecasting significant market growth, with AI-driven assistive devices poised to transform the landscape. Despite challenges such as high development costs and regulatory hurdles, opportunities for innovation and affordability remain. This review underscores the importance of addressing challenges related to standardization, accessibility, and ethical considerations to ensure the successful integration of AI into assistive technologies, fostering greater inclusivity and improved quality of life for users globally.

Assessing the article screening efficiency of artificial intelligence for Systematic Reviews

OBJECTIVES

Artificial intelligence (AI) tools utilizing machine learning (ML) have gained increasing utility in medicine and academia as a means of enhancing efficiency. ASReview is one such AI program designed to streamline the systematic review process through the automated prioritization of relevant articles for screening. This study examined the screening efficiency of ASReview when conducting systematic reviews and the potential factors that could influence its efficiency.

METHODS

Six distinct topics within the field of periodontics were searched in PubMed and Web of Science to obtain articles for screening within ASReview. Through a "training" process, relevant and irrelevant articles were manually incorporated to develop "prior knowledge" and facilitate ML optimization. Screening was then conducted following ASReview's algorithmically-generated relevance rankings. Screening efficiency was evaluated based on the normalized number of articles not requiring detailed review and on the total time expenditure.

RESULTS

Across the six topics, an average of 60.2 % of articles did not warrant extensive screening, given that all relevant articles were discovered within the first 39.8 % of publication reviewed. No significant variations in efficiencies were observed with differing methods of assembling prior knowledge articles or via modifications in article ratios and numbers.

CONCLUSIONS

On average, ASReview conferred a 60.2 % improvement in screening efficiency, largely attributed to its dynamic ML capabilities. While advanced technologies like ASReview promise enhanced efficiencies, the accurate human discernment of article relevancy and quality remains indispensable when training these AI tools.

CLINICAL SIGNIFICANCE

Using ASReview has the potential to save approximately 60 % of time and effort required for screening articles.

An exploration of available methods and tools to improve the efficiency of systematic review production: a scoping review

BACKGROUND

Systematic reviews (SRs) are time-consuming and labor-intensive to perform. With the growing number of scientific publications, the SR development process becomes even more laborious. This is problematic because timely SR evidence is essential for decision-making in evidence-based healthcare and policymaking. Numerous methods and tools that accelerate SR development have recently emerged. To date, no scoping review has been conducted to provide a comprehensive summary of methods and ready-to-use tools to improve efficiency in SR production.

OBJECTIVE

To present an overview of primary studies that evaluated the use of ready-to-use applications of tools or review methods to improve efficiency in the review process.

METHODS

We conducted a scoping review. An information specialist performed a systematic literature search in four databases, supplemented with citation-based and grey literature searching. We included studies reporting the performance of methods and ready-to-use tools for improving efficiency when producing or updating a SR in the health field. We performed dual, independent title and abstract screening, full-text selection, and data extraction. The results were analyzed descriptively and presented narratively.

RESULTS

We included 103 studies: 51 studies reported on methods, 54 studies on tools, and 2 studies reported on both methods and tools to make SR production more efficient. A total of 72 studies evaluated the validity (n = 69) or usability (n = 3) of one method (n = 33) or tool (n = 39), and 31 studies performed comparative analyses of different methods (n = 15) or tools (n = 16). 20 studies conducted prospective evaluations in real-time workflows. Most studies evaluated methods or tools that aimed at screening titles and abstracts (n = 42) and literature searching (n = 24), while for other steps of the SR process, only a few studies were found. Regarding the outcomes included, most studies reported on validity outcomes (n = 84), while outcomes such as impact on results (n = 23), time-saving (n = 24), usability (n = 13), and cost-saving (n = 3) were less often evaluated.

CONCLUSION

For title and abstract screening and literature searching, various evaluated methods and tools are available that aim at improving the efficiency of SR production. However, only few studies have addressed the influence of these methods and tools in real-world workflows. Few studies exist that evaluate methods or tools supporting the remaining tasks. Additionally, while validity outcomes are frequently reported, there is a lack of evaluation regarding other outcomes.

Artificial intelligence in systematic reviews: promising when appropriately used

BACKGROUND

Systematic reviews provide a structured overview of the available evidence in medical-scientific research. However, due to the increasing medical-scientific research output, it is a time-consuming task to conduct systematic reviews. To accelerate this process, artificial intelligence (AI) can be used in the review process. In this communication paper, we suggest how to conduct a transparent and reliable systematic review using the AI tool 'ASReview' in the title and abstract screening.

METHODS

Use of the AI tool consisted of several steps. First, the tool required training of its algorithm with several prelabelled articles prior to screening. Next, using a researcher-in-the-loop algorithm, the AI tool proposed the article with the highest probability of being relevant. The reviewer then decided on relevancy of each article proposed. This process was continued until the stopping criterion was reached. All articles labelled relevant by the reviewer were screened on full text.

RESULTS

Considerations to ensure methodological quality when using AI in systematic reviews included: the choice of whether to use AI, the need of both deduplication and checking for inter-reviewer agreement, how to choose a stopping criterion and the quality of reporting. Using the tool in our review resulted in much time saved: only 23% of the articles were assessed by the reviewer.

CONCLUSION

The AI tool is a promising innovation for the current systematic reviewing practice, as long as it is appropriately used and methodological quality can be assured.

PROSPERO REGISTRATION NUMBER

CRD42022283952.