1
|
Tóth B, Berek L, Gulácsi L, Péntek M, Zrubka Z. Automation of systematic reviews of biomedical literature: a scoping review of studies indexed in PubMed. Syst Rev 2024; 13:174. [PMID: 38978132 PMCID: PMC11229257 DOI: 10.1186/s13643-024-02592-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/27/2023] [Accepted: 06/20/2024] [Indexed: 07/10/2024] Open
Abstract
BACKGROUND The demand for high-quality systematic literature reviews (SRs) for evidence-based medical decision-making is growing. SRs are costly and require the scarce resource of highly skilled reviewers. Automation technology has been proposed to save workload and expedite the SR workflow. We aimed to provide a comprehensive overview of SR automation studies indexed in PubMed, focusing on the applicability of these technologies in real world practice. METHODS In November 2022, we extracted, combined, and ran an integrated PubMed search for SRs on SR automation. Full-text English peer-reviewed articles were included if they reported studies on SR automation methods (SSAM), or automated SRs (ASR). Bibliographic analyses and knowledge-discovery studies were excluded. Record screening was performed by single reviewers, and the selection of full text papers was performed in duplicate. We summarized the publication details, automated review stages, automation goals, applied tools, data sources, methods, results, and Google Scholar citations of SR automation studies. RESULTS From 5321 records screened by title and abstract, we included 123 full text articles, of which 108 were SSAM and 15 ASR. Automation was applied for search (19/123, 15.4%), record screening (89/123, 72.4%), full-text selection (6/123, 4.9%), data extraction (13/123, 10.6%), risk of bias assessment (9/123, 7.3%), evidence synthesis (2/123, 1.6%), assessment of evidence quality (2/123, 1.6%), and reporting (2/123, 1.6%). Multiple SR stages were automated by 11 (8.9%) studies. The performance of automated record screening varied largely across SR topics. In published ASR, we found examples of automated search, record screening, full-text selection, and data extraction. In some ASRs, automation fully complemented manual reviews to increase sensitivity rather than to save workload. Reporting of automation details was often incomplete in ASRs. CONCLUSIONS Automation techniques are being developed for all SR stages, but with limited real-world adoption. Most SR automation tools target single SR stages, with modest time savings for the entire SR process and varying sensitivity and specificity across studies. Therefore, the real-world benefits of SR automation remain uncertain. Standardizing the terminology, reporting, and metrics of study reports could enhance the adoption of SR automation techniques in real-world practice.
Collapse
Affiliation(s)
- Barbara Tóth
- Doctoral School of Innovation Management, Óbuda University, Bécsi út 96/B, Budapest, 1034, Hungary
| | - László Berek
- Doctoral School for Safety and Security, Óbuda University, Bécsi út 96/B, Budapest, 1034, Hungary
- University Library, Óbuda University, Bécsi út 96/B, Budapest, 1034, Hungary
| | - László Gulácsi
- HECON Health Economics Research Center, University Research, and Innovation Center, Óbuda University, Bécsi út 96/B, Budapest, 1034, Hungary
| | - Márta Péntek
- HECON Health Economics Research Center, University Research, and Innovation Center, Óbuda University, Bécsi út 96/B, Budapest, 1034, Hungary
| | - Zsombor Zrubka
- HECON Health Economics Research Center, University Research, and Innovation Center, Óbuda University, Bécsi út 96/B, Budapest, 1034, Hungary.
| |
Collapse
|
2
|
Guo Q, Jiang G, Zhao Q, Long Y, Feng K, Gu X, Xu Y, Li Z, Huang J, Du L. Rapid review: A review of methods and recommendations based on current evidence. J Evid Based Med 2024; 17:434-453. [PMID: 38512942 DOI: 10.1111/jebm.12594] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/26/2023] [Accepted: 02/28/2024] [Indexed: 03/23/2024]
Abstract
Rapid review (RR) could accelerate the traditional systematic review (SR) process by simplifying or omitting steps using various shortcuts. With the increasing popularity of RR, numerous shortcuts had emerged, but there was no consensus on how to choose the most appropriate ones. This study conducted a literature search in PubMed from inception to December 21, 2023, using terms such as "rapid review" "rapid assessment" "rapid systematic review" and "rapid evaluation". We also scanned the reference lists and performed citation tracking of included impact studies to obtain more included studies. We conducted a narrative synthesis of all RR approaches, shortcuts and studies assessing their effectiveness at each stage of RRs. Based on the current evidence, we provided recommendations on utilizing certain shortcuts in RRs. Ultimately, we identified 185 studies focusing on summarizing RR approaches and shortcuts, or evaluating their impact. There was relatively sufficient evidence to support the use of the following shortcuts in RRs: limiting studies to those published in English-language; conducting abbreviated database searches (e.g., only searching PubMed/MEDLINE, Embase, and CENTRAL); omitting retrieval of grey literature; restricting the search timeframe to the recent 20 years for medical intervention and the recent 15 years for reviewing diagnostic test accuracy; conducting a single screening by an experienced screener. To some extent, the above shortcuts were also applicable to SRs. This study provided a reference for future RR researchers in selecting shortcuts, and it also presented a potential research topic for methodologists.
Collapse
Affiliation(s)
- Qiong Guo
- Innovation Institute for Integration of Medicine and Engineering, West China Hospital, Sichuan University, Chengdu, P. R. China
- West China Medical Publishers, West China Hospital, Sichuan University, Chengdu, P. R. China
| | - Guiyu Jiang
- West China School of Public Health, Sichuan University, Chengdu, P. R. China
| | - Qingwen Zhao
- West China School of Public Health, Sichuan University, Chengdu, P. R. China
| | - Youlin Long
- Innovation Institute for Integration of Medicine and Engineering, West China Hospital, Sichuan University, Chengdu, P. R. China
- Chinese Evidence-Based Medicine Center, West China Hospital, Sichuan University, Chengdu, P. R. China
| | - Kun Feng
- Innovation Institute for Integration of Medicine and Engineering, West China Hospital, Sichuan University, Chengdu, P. R. China
- Chinese Evidence-Based Medicine Center, West China Hospital, Sichuan University, Chengdu, P. R. China
| | - Xianlin Gu
- Innovation Institute for Integration of Medicine and Engineering, West China Hospital, Sichuan University, Chengdu, P. R. China
- Chinese Evidence-Based Medicine Center, West China Hospital, Sichuan University, Chengdu, P. R. China
| | - Yihan Xu
- Innovation Institute for Integration of Medicine and Engineering, West China Hospital, Sichuan University, Chengdu, P. R. China
- Chinese Evidence-Based Medicine Center, West China Hospital, Sichuan University, Chengdu, P. R. China
- Center for education of medical humanities, West China Hospital, Sichuan University, Chengdu, P. R. China
| | - Zhengchi Li
- Center for education of medical humanities, West China Hospital, Sichuan University, Chengdu, P. R. China
| | - Jin Huang
- Innovation Institute for Integration of Medicine and Engineering, West China Hospital, Sichuan University, Chengdu, P. R. China
| | - Liang Du
- Innovation Institute for Integration of Medicine and Engineering, West China Hospital, Sichuan University, Chengdu, P. R. China
- West China Medical Publishers, West China Hospital, Sichuan University, Chengdu, P. R. China
- Chinese Evidence-Based Medicine Center, West China Hospital, Sichuan University, Chengdu, P. R. China
| |
Collapse
|
3
|
Hirt J, Ewald H, Briel M, Schandelmaier S. Searching a methods topic: practical challenges and implications for search design. J Clin Epidemiol 2024; 166:111201. [PMID: 37914105 DOI: 10.1016/j.jclinepi.2023.10.017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2022] [Revised: 10/23/2023] [Accepted: 10/24/2023] [Indexed: 11/03/2023]
Affiliation(s)
- Julian Hirt
- Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland; International Graduate Academy, Institute of Health and Nursing Science, Medical Faculty, Martin Luther University Halle-Wittenberg, Halle (Saale), Germany; Department of Health, Institute of Nursing Science, Eastern Switzerland University of Applied Sciences, St.Gallen, Switzerland
| | - Hannah Ewald
- University Medical Library, University of Basel, Basel, Switzerland
| | - Matthias Briel
- Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland; Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Canada
| | - Stefan Schandelmaier
- Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland; MTA-PTE Lendület "Momentum" Evidence in Medicine Research Group, Medical School, University of Pécs, Pécs, Hungary.
| |
Collapse
|
4
|
Haby MM, Barreto JOM, Kim JYH, Peiris S, Mansilla C, Torres M, Guerrero-Magaña DE, Reveiz L. What are the best methods for rapid reviews of the research evidence? A systematic review of reviews and primary studies. Res Synth Methods 2024; 15:2-20. [PMID: 37696668 DOI: 10.1002/jrsm.1664] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2023] [Revised: 07/09/2023] [Accepted: 08/07/2023] [Indexed: 09/13/2023]
Abstract
Rapid review methodology aims to facilitate faster conduct of systematic reviews to meet the needs of the decision-maker, while also maintaining quality and credibility. This systematic review aimed to determine the impact of different methodological shortcuts for undertaking rapid reviews on the risk of bias (RoB) of the results of the review. Review stages for which reviews and primary studies were sought included the preparation of a protocol, question formulation, inclusion criteria, searching, selection, data extraction, RoB assessment, synthesis, and reporting. We searched 11 electronic databases in April 2022, and conducted some supplementary searching. Reviewers worked in pairs to screen, select, extract data, and assess the RoB of included reviews and studies. We included 15 systematic reviews, 7 scoping reviews, and 65 primary studies. We found that several commonly used shortcuts in rapid reviews are likely to increase the RoB in the results. These include restrictions based on publication date, use of a single electronic database as a source of studies, and use of a single reviewer for screening titles and abstracts, selecting studies based on the full-text, and for extracting data. Authors of rapid reviews should be transparent in reporting their use of these shortcuts and acknowledge the possibility of them causing bias in the results. This review also highlights shortcuts that can save time without increasing the risk of bias. Further research is needed for both systematic and rapid reviews on faster methods for accurate data extraction and RoB assessment, and on development of more precise search strategies.
Collapse
Affiliation(s)
- Michelle M Haby
- Science and Knowledge Unit, Evidence and Intelligence for Action in Health Department, Pan American Health Organization, Washington, DC, USA
- Department of Chemical and Biological Sciences, University of Sonora, Hermosillo, Mexico
- Melbourne School of Population and Global Health, The University of Melbourne, Parkville, Victoria, Australia
| | | | - Jenny Yeon Hee Kim
- Science and Knowledge Unit, Evidence and Intelligence for Action in Health Department, Pan American Health Organization, Washington, DC, USA
| | - Sasha Peiris
- Science and Knowledge Unit, Evidence and Intelligence for Action in Health Department, Pan American Health Organization, Washington, DC, USA
| | - Cristián Mansilla
- McMaster Health Forum, McMaster University, Hamilton, Ontario, Canada
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada
| | - Marcela Torres
- Science and Knowledge Unit, Evidence and Intelligence for Action in Health Department, Pan American Health Organization, Washington, DC, USA
| | - Diego Emmanuel Guerrero-Magaña
- Doctoral Program in Chemical and Biological Sciences and Health, Department of Chemical and Biological Sciences, University of Sonora, Hermosillo, Mexico
| | - Ludovic Reveiz
- Science and Knowledge Unit, Evidence and Intelligence for Action in Health Department, Pan American Health Organization, Washington, DC, USA
| |
Collapse
|
5
|
McDonald S, Hill K, Li HZ, Turner T. Evidence surveillance for a living clinical guideline: Case study of the Australian stroke guidelines. Health Info Libr J 2023. [PMID: 37942888 DOI: 10.1111/hir.12515] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2023] [Revised: 07/26/2023] [Accepted: 10/26/2023] [Indexed: 11/10/2023]
Abstract
BACKGROUND Continual evidence surveillance is an integral feature of living guidelines. The Australian Stroke Guidelines include recommendations on 100 clinical topics and have been 'living' since 2018. OBJECTIVES To describe the approach for establishing and evaluating an evidence surveillance system for the living Australian Stroke Guidelines. METHODS We developed a pragmatic surveillance system based on an analysis of the searches for the 2017 Stroke Guidelines and evaluated its reliability by assessing the potential impact on guideline recommendations. Search retrieval and screening workload are monitored monthly, together with the frequency of changes to the guideline recommendations. RESULTS Evidence surveillance was guided by practical considerations of efficiency and sustainability. A single PubMed search covering all guideline topics, limited to systematic reviews and randomised trials, is run monthly. The search retrieves about 400 records a month of which a sixth are triaged to the guideline panels for further consideration. Evaluations with Epistemonikos and the Cochrane Stroke Trials Register demonstrated the robustness of adopting this more restrictive approach. Collaborating with the guideline team in designing, implementing and evaluating the surveillance is essential for optimising the approach. CONCLUSION Monthly evidence surveillance for a large living guideline is feasible and sustainable when applying a pragmatic approach.
Collapse
Affiliation(s)
- Steve McDonald
- Cochrane Australia, School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia
| | - Kelvin Hill
- Stroke Services, Stroke Foundation, Melbourne, Australia
| | - Heidi Z Li
- Stroke Services, Stroke Foundation, Melbourne, Australia
| | - Tari Turner
- Cochrane Australia, School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia
| |
Collapse
|
6
|
Kataoka Y, Taito S, Yamamoto N, So R, Tsutsumi Y, Anan K, Banno M, Tsujimoto Y, Wada Y, Sagami S, Tsujimoto H, Nihashi T, Takeuchi M, Terasawa T, Iguchi M, Kumasawa J, Ichikawa T, Furukawa R, Yamabe J, Furukawa TA. An open competition involving thousands of competitors failed to construct useful abstract classifiers for new diagnostic test accuracy systematic reviews. Res Synth Methods 2023; 14:707-717. [PMID: 37337729 DOI: 10.1002/jrsm.1649] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2022] [Revised: 06/05/2023] [Accepted: 06/07/2023] [Indexed: 06/21/2023]
Abstract
There are currently no abstract classifiers, which can be used for new diagnostic test accuracy (DTA) systematic reviews to select primary DTA study abstracts from database searches. Our goal was to develop machine-learning-based abstract classifiers for new DTA systematic reviews through an open competition. We prepared a dataset of abstracts obtained through database searches from 11 reviews in different clinical areas. As the reference standard, we used the abstract lists that required manual full-text review. We randomly splitted the datasets into a train set, a public test set, and a private test set. Competition participants used the training set to develop classifiers and validated their classifiers using the public test set. The classifiers were refined based on the performance of the public test set. They could submit as many times as they wanted during the competition. Finally, we used the private test set to rank the submitted classifiers. To reduce false exclusions, we used the Fbeta measure with a beta set to seven for evaluating classifiers. After the competition, we conducted the external validation using a dataset from a cardiology DTA review. We received 13,774 submissions from 1429 teams or persons over 4 months. The top-honored classifier achieved a Fbeta score of 0.4036 and a recall of 0.2352 in the external validation. In conclusion, we were unable to develop an abstract classifier with sufficient recall for immediate application to new DTA systematic reviews. Further studies are needed to update and validate classifiers with datasets from other clinical areas.
Collapse
Affiliation(s)
- Yuki Kataoka
- Department of Internal Medicine, Kyoto Min-iren Asukai Hospital, Kyoto, Japan
- Scientific Research Works Peer Support Group (SRWS-PSG), Osaka, Japan
- Section of Clinical Epidemiology, Department of Community Medicine, Kyoto University Graduate School of Medicine, Kyoto, Japan
- Department of Healthcare Epidemiology, Kyoto University Graduate School of Medicine/School of Public Health, Kyoto, Japan
| | - Shunsuke Taito
- Scientific Research Works Peer Support Group (SRWS-PSG), Osaka, Japan
- Division of Rehabilitation, Department of Clinical Practice and Support, Hiroshima University Hospital, Hiroshima, Japan
| | - Norio Yamamoto
- Scientific Research Works Peer Support Group (SRWS-PSG), Osaka, Japan
- Department of Orthopedic Surgery, Miyamoto Orthopedic Hospital, Okayama, Japan
- Department of Epidemiology, Graduate School of Medicine, Dentistry and Pharmaceutical Sciences, Okayama University, Okayama, Japan
| | - Ryuhei So
- Scientific Research Works Peer Support Group (SRWS-PSG), Osaka, Japan
- Department of Psychiatry, Okayama Psychiatric Medical Center, Okayama, Japan
- CureApp, Inc., Tokyo, Japan
| | - Yusuke Tsutsumi
- Scientific Research Works Peer Support Group (SRWS-PSG), Osaka, Japan
- Department of Healthcare Epidemiology, Kyoto University Graduate School of Medicine/School of Public Health, Kyoto, Japan
- Department of Emergency Medicine, National Hospital Organization Mito Medical Center, Ibaraki, Japan
| | - Keisuke Anan
- Scientific Research Works Peer Support Group (SRWS-PSG), Osaka, Japan
- Division of Respiratory Medicine, Saiseikai Kumamoto Hospital, Kumamoto, Japan
- Department of Healthcare Epidemiology, Graduate School of Medicine and Public Health, Kyoto University, Kyoto, Japan
| | - Masahiro Banno
- Scientific Research Works Peer Support Group (SRWS-PSG), Osaka, Japan
- Department of Psychiatry, Seichiryo Hospital, Nagoya, Japan
- Department of Psychiatry, Nagoya University Graduate School of Medicine, Nagoya, Japan
| | - Yasushi Tsujimoto
- Scientific Research Works Peer Support Group (SRWS-PSG), Osaka, Japan
- Oku Medical Clinic, Osaka, Japan
- Department of Health Promotion and Human Behavior, Kyoto University Graduate School of Medicine/School of Public Health, Kyoto University, Kyoto, Japan
| | - Yoshitaka Wada
- Scientific Research Works Peer Support Group (SRWS-PSG), Osaka, Japan
- Department of Rehabilitation Medicine, School of Medicine, Fujita Health University, Toyoake, Japan
| | - Shintaro Sagami
- Center for Advanced IBD Research and Treatment, Kitasato University Kitasato Institute Hospital, Tokyo, Japan
- Department of Gastroenterology and Hepatology, Kitasato University Kitasato Institute Hospital, Tokyo, Japan
| | - Hiraku Tsujimoto
- Hospital Care Research Unit, Hyogo Prefectural Amagasaki General Medical Center, Amagasaki, Japan
| | - Takashi Nihashi
- Department of Radiology, Komaki City Hospital, Komaki, Japan
| | - Motoki Takeuchi
- Department of Emergency and General Internal Medicine, Fujita Health University School of Medicine, Toyoake, Japan
| | - Teruhiko Terasawa
- Section of General Internal Medicine, Department of Emergency and General Internal Medicine, Fujita Health University School of Medicine, Toyoake, Japan
| | - Masahiro Iguchi
- Department of Neurology, Fukushima Medical University, Fukushima, Japan
| | - Junji Kumasawa
- Human Health Sciences, Kyoto University Graduate School of Medicine, Kyoto, Japan
- Department of Critical Care Medicine, Sakai City Medical Center, Sakai, Japan
| | | | | | | | - Toshi A Furukawa
- Department of Health Promotion and Human Behavior, Kyoto University Graduate School of Medicine/School of Public Health, Kyoto University, Kyoto, Japan
| |
Collapse
|
7
|
Oliveira Dos Santos Á, Sergio da Silva E, Machado Couto L, Valadares Labanca Reis G, Silva Belo V. The use of artificial intelligence for automating or semi-automating biomedical literature analyses: a scoping review. J Biomed Inform 2023; 142:104389. [PMID: 37187321 DOI: 10.1016/j.jbi.2023.104389] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2023] [Revised: 04/11/2023] [Accepted: 05/08/2023] [Indexed: 05/17/2023]
Abstract
OBJECTIVE Evidence-based medicine (EBM) is a decision-making process based on the conscious and judicious use of the best available scientific evidence. However, the exponential increase in the amount of information currently available likely exceeds the capacity of human-only analysis. In this context, artificial intelligence (AI) and its branches such as machine learning (ML) can be used to facilitate human efforts in analyzing the literature to foster EBM. The present scoping review aimed to examine the use of AI in the automation of biomedical literature survey and analysis with a view to establishing the state-of-the-art and identifying knowledge gaps. MATERIALS AND METHODS Comprehensive searches of the main databases were performed for articles published up to June 2022 and studies were selected according to inclusion and exclusion criteria. Data were extracted from the included articles and the findings categorized. RESULTS The total number of records retrieved from the databases was 12,145, of which 273 were included in the review. Classification of the studies according to the use of AI in evaluating the biomedical literature revealed three main application groups, namely assembly of scientific evidence (n=127; 47%), mining the biomedical literature (n=112; 41%) and quality analysis (n=34; 12%). Most studies addressed the preparation of systematic reviews, while articles focusing on the development of guidelines and evidence synthesis were the least frequent. The biggest knowledge gap was identified within the quality analysis group, particularly regarding methods and tools that assess the strength of recommendation and consistency of evidence. CONCLUSION Our review shows that, despite significant progress in the automation of biomedical literature surveys and analyses in recent years, intense research is needed to fill knowledge gaps on more difficult aspects of ML, deep learning and natural language processing, and to consolidate the use of automation by end-users (biomedical researchers and healthcare professionals).
Collapse
Affiliation(s)
| | - Eduardo Sergio da Silva
- Federal University of São João del-Rei, Campus Centro-Oeste Dona Lindu, Divinópolis, Minas Gerais, Brazil.
| | - Letícia Machado Couto
- Federal University of São João del-Rei, Campus Centro-Oeste Dona Lindu, Divinópolis, Minas Gerais, Brazil.
| | | | - Vinícius Silva Belo
- Federal University of São João del-Rei, Campus Centro-Oeste Dona Lindu, Divinópolis, Minas Gerais, Brazil.
| |
Collapse
|
8
|
Forsgren E, Wallström S, Feldthusen C, Zechner N, Sawatzky R, Öhlén J. The use of text-mining software to facilitate screening of literature on centredness in health care. Syst Rev 2023; 12:73. [PMID: 37120578 PMCID: PMC10148558 DOI: 10.1186/s13643-023-02242-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/02/2023] [Accepted: 04/20/2023] [Indexed: 05/01/2023] Open
Abstract
Research evidence supporting the implementation of centredness in health care is not easily accessible due to the sheer amount of literature available and the diversity in terminology and conceptualisations used. The use of text-mining functions to semi-automate the process of screening and collating citations for a review is a way of tackling the vast amount of research citations available today. There are several programmes that use text-mining functions to facilitate screening and data extraction for systematic reviews. However, the suitability of these programmes for reviews on broad topics of research, as well as the general uptake by researchers, is unclear. This commentary has a dual aim, which consists in outlining the challenges of screening literature in fields characterised by vague and overlapping conceptualisations, and to exemplify this by exploratory use of text-mining in the context of a scoping review on centredness in health care.
Collapse
Affiliation(s)
- Emma Forsgren
- Institute of Health and Care Sciences, Sahlgrenska Academy, University of Gothenburg, Box 457, SE-405 30, Gothenburg, Sweden.
- University of Gothenburg Centre for Person-Centred Care (GPCC), Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden.
| | - Sara Wallström
- Institute of Health and Care Sciences, Sahlgrenska Academy, University of Gothenburg, Box 457, SE-405 30, Gothenburg, Sweden
- University of Gothenburg Centre for Person-Centred Care (GPCC), Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
- Department of Forensic Psychiatry, Sahlgrenska University Hospital, Gothenburg, Sweden
| | - Caroline Feldthusen
- Institute of Health and Care Sciences, Sahlgrenska Academy, University of Gothenburg, Box 457, SE-405 30, Gothenburg, Sweden
- University of Gothenburg Centre for Person-Centred Care (GPCC), Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
- Department of Occupational Therapy and Physiotherapy, Sahlgrenska University Hospital, Gothenburg, Sweden
| | - Niklas Zechner
- Department of Swedish, Multilingualism, Language Technology, University of Gothenburg, Gothenburg, Sweden
| | - Richard Sawatzky
- Institute of Health and Care Sciences, Sahlgrenska Academy, University of Gothenburg, Box 457, SE-405 30, Gothenburg, Sweden
- University of Gothenburg Centre for Person-Centred Care (GPCC), Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
- School of Nursing, Trinity Western University, Langley, British Columbia, Canada
- Centre for Health Evaluation and Outcome Sciences, Providence Health Care, Vancouver, British Columbia, Canada
| | - Joakim Öhlén
- Institute of Health and Care Sciences, Sahlgrenska Academy, University of Gothenburg, Box 457, SE-405 30, Gothenburg, Sweden
- University of Gothenburg Centre for Person-Centred Care (GPCC), Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
- Palliative Centre, Sahlgrenska University Hospital, Gothenburg, Sweden
| |
Collapse
|
9
|
Muller AE, Berg RC, Meneses-Echavez JF, Ames HMR, Borge TC, Jardim PSJ, Cooper C, Rose CJ. The effect of machine learning tools for evidence synthesis on resource use and time-to-completion: protocol for a retrospective pilot study. Syst Rev 2023; 12:7. [PMID: 36650579 PMCID: PMC9843684 DOI: 10.1186/s13643-023-02171-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/22/2022] [Accepted: 01/06/2023] [Indexed: 01/18/2023] Open
Abstract
BACKGROUND Machine learning (ML) tools exist that can reduce or replace human activities in repetitive or complex tasks. Yet, ML is underutilized within evidence synthesis, despite the steadily growing rate of primary study publication and the need to periodically update reviews to reflect new evidence. Underutilization may be partially explained by a paucity of evidence on how ML tools can reduce resource use and time-to-completion of reviews. METHODS This protocol describes how we will answer two research questions using a retrospective study design: Is there a difference in resources used to produce reviews using recommended ML versus not using ML, and is there a difference in time-to-completion? We will also compare recommended ML use to non-recommended ML use that merely adds ML use to existing procedures. We will retrospectively include all reviews conducted at our institute from 1 August 2020, corresponding to the commission of the first review in our institute that used ML. CONCLUSION The results of this study will allow us to quantitatively estimate the effect of ML adoption on resource use and time-to-completion, providing our organization and others with better information to make high-level organizational decisions about ML.
Collapse
Affiliation(s)
| | | | | | | | | | | | - Chris Cooper
- Bristol Medical School, University of Bristol, Bristol, UK
- Department of Clinical, Educational and Health Psychology, University College London, London, UK
| | | |
Collapse
|