3
|
Frank RA, Salameh JP, Islam N, Yang B, Murad MH, Mustafa R, Leeflang M, Bossuyt PM, Takwoingi Y, Whiting P, Dawit H, Kang SK, Ebrahimzadeh S, Levis B, Hutton B, McInnes MDF. How to Critically Appraise and Interpret Systematic Reviews and Meta-Analyses of Diagnostic Accuracy: A User Guide. Radiology 2023; 307:e221437. [PMID: 36916896 PMCID: PMC10140638 DOI: 10.1148/radiol.221437] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2022] [Revised: 10/25/2022] [Accepted: 10/28/2022] [Indexed: 03/16/2023]
Abstract
Systematic reviews of diagnostic accuracy studies can provide the best available evidence to inform decisions regarding the use of a diagnostic test. In this guide, the authors provide a practical approach for clinicians to appraise diagnostic accuracy systematic reviews and apply their results to patient care. The first step is to identify an appropriate systematic review with a research question matching the clinical scenario. The user should evaluate the rigor of the review methods to evaluate its credibility (Did the review use clearly defined eligibility criteria, a comprehensive search strategy, structured data collection, risk of bias and applicability appraisal, and appropriate meta-analysis methods?). If the review is credible, the next step is to decide whether the diagnostic performance is adequate for clinical use (Do sensitivity and specificity estimates exceed the threshold that makes them useful in clinical practice? Are these estimates sufficiently precise? Is variability in the estimates of diagnostic accuracy across studies explained?). Diagnostic accuracy systematic reviews that are judged to be credible and provide diagnostic accuracy estimates with sufficient certainty and relevance are the most useful to inform patient care. This review discusses comparative, noncomparative, and emerging approaches to systematic reviews of diagnostic accuracy using a clinical scenario and examples based on recent publications.
Collapse
Affiliation(s)
| | | | - Nayaar Islam
- From the Department of Radiology, University of Ottawa, The Ottawa
Hospital Civic Campus, 1053 Carling Ave, Room c159, Ottawa, ON, Canada K1Y 4E9
(R.A.F., M.D.F.M.); Faculty of Health Sciences, Queen’s University,
Kingston, Ontario, Canada (J.P.S.); Clinical Epidemiology Program, Ottawa
Hospital Research Institute, University of Ottawa, Ottawa, Canada (N.I., M.H.M.,
H.D., S.E., B.H.); Julius Center for Health Sciences and Primary Care,
University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
(B.Y.); Evidence-Based Practice Center, Mayo Clinic, Rochester, Minn (M.H.M.);
Department of Medicine, Division of Nephrology and Hypertension, University of
Kansas Medical Center, Kansas City, Mo (R.M.); Department of Epidemiology and
Data Science, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
(M.L., P.M.B.); Amsterdam Public Health, Amsterdam, the Netherlands (P.M.B.);
Institute of Applied Health Research, University of Birmingham, Birmingham, UK
(Y.T.); NIHR Birmingham Biomedical Research Centre, University Hospitals
Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
(Y.T.); Population Health Sciences, Bristol Medical School, University of
Bristol, Bristol, UK (P.W.); Department of Radiology, NYU Langone Health, New
York, NY (S.K.K.); and Centre for Clinical Epidemiology, Lady Davis Institute
for Medical Research, Jewish General Hospital, Montréal, Canada
(B.L.)
| | - Bada Yang
- From the Department of Radiology, University of Ottawa, The Ottawa
Hospital Civic Campus, 1053 Carling Ave, Room c159, Ottawa, ON, Canada K1Y 4E9
(R.A.F., M.D.F.M.); Faculty of Health Sciences, Queen’s University,
Kingston, Ontario, Canada (J.P.S.); Clinical Epidemiology Program, Ottawa
Hospital Research Institute, University of Ottawa, Ottawa, Canada (N.I., M.H.M.,
H.D., S.E., B.H.); Julius Center for Health Sciences and Primary Care,
University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
(B.Y.); Evidence-Based Practice Center, Mayo Clinic, Rochester, Minn (M.H.M.);
Department of Medicine, Division of Nephrology and Hypertension, University of
Kansas Medical Center, Kansas City, Mo (R.M.); Department of Epidemiology and
Data Science, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
(M.L., P.M.B.); Amsterdam Public Health, Amsterdam, the Netherlands (P.M.B.);
Institute of Applied Health Research, University of Birmingham, Birmingham, UK
(Y.T.); NIHR Birmingham Biomedical Research Centre, University Hospitals
Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
(Y.T.); Population Health Sciences, Bristol Medical School, University of
Bristol, Bristol, UK (P.W.); Department of Radiology, NYU Langone Health, New
York, NY (S.K.K.); and Centre for Clinical Epidemiology, Lady Davis Institute
for Medical Research, Jewish General Hospital, Montréal, Canada
(B.L.)
| | - Mohammad Hassan Murad
- From the Department of Radiology, University of Ottawa, The Ottawa
Hospital Civic Campus, 1053 Carling Ave, Room c159, Ottawa, ON, Canada K1Y 4E9
(R.A.F., M.D.F.M.); Faculty of Health Sciences, Queen’s University,
Kingston, Ontario, Canada (J.P.S.); Clinical Epidemiology Program, Ottawa
Hospital Research Institute, University of Ottawa, Ottawa, Canada (N.I., M.H.M.,
H.D., S.E., B.H.); Julius Center for Health Sciences and Primary Care,
University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
(B.Y.); Evidence-Based Practice Center, Mayo Clinic, Rochester, Minn (M.H.M.);
Department of Medicine, Division of Nephrology and Hypertension, University of
Kansas Medical Center, Kansas City, Mo (R.M.); Department of Epidemiology and
Data Science, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
(M.L., P.M.B.); Amsterdam Public Health, Amsterdam, the Netherlands (P.M.B.);
Institute of Applied Health Research, University of Birmingham, Birmingham, UK
(Y.T.); NIHR Birmingham Biomedical Research Centre, University Hospitals
Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
(Y.T.); Population Health Sciences, Bristol Medical School, University of
Bristol, Bristol, UK (P.W.); Department of Radiology, NYU Langone Health, New
York, NY (S.K.K.); and Centre for Clinical Epidemiology, Lady Davis Institute
for Medical Research, Jewish General Hospital, Montréal, Canada
(B.L.)
| | - Reem Mustafa
- From the Department of Radiology, University of Ottawa, The Ottawa
Hospital Civic Campus, 1053 Carling Ave, Room c159, Ottawa, ON, Canada K1Y 4E9
(R.A.F., M.D.F.M.); Faculty of Health Sciences, Queen’s University,
Kingston, Ontario, Canada (J.P.S.); Clinical Epidemiology Program, Ottawa
Hospital Research Institute, University of Ottawa, Ottawa, Canada (N.I., M.H.M.,
H.D., S.E., B.H.); Julius Center for Health Sciences and Primary Care,
University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
(B.Y.); Evidence-Based Practice Center, Mayo Clinic, Rochester, Minn (M.H.M.);
Department of Medicine, Division of Nephrology and Hypertension, University of
Kansas Medical Center, Kansas City, Mo (R.M.); Department of Epidemiology and
Data Science, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
(M.L., P.M.B.); Amsterdam Public Health, Amsterdam, the Netherlands (P.M.B.);
Institute of Applied Health Research, University of Birmingham, Birmingham, UK
(Y.T.); NIHR Birmingham Biomedical Research Centre, University Hospitals
Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
(Y.T.); Population Health Sciences, Bristol Medical School, University of
Bristol, Bristol, UK (P.W.); Department of Radiology, NYU Langone Health, New
York, NY (S.K.K.); and Centre for Clinical Epidemiology, Lady Davis Institute
for Medical Research, Jewish General Hospital, Montréal, Canada
(B.L.)
| | - Mariska Leeflang
- From the Department of Radiology, University of Ottawa, The Ottawa
Hospital Civic Campus, 1053 Carling Ave, Room c159, Ottawa, ON, Canada K1Y 4E9
(R.A.F., M.D.F.M.); Faculty of Health Sciences, Queen’s University,
Kingston, Ontario, Canada (J.P.S.); Clinical Epidemiology Program, Ottawa
Hospital Research Institute, University of Ottawa, Ottawa, Canada (N.I., M.H.M.,
H.D., S.E., B.H.); Julius Center for Health Sciences and Primary Care,
University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
(B.Y.); Evidence-Based Practice Center, Mayo Clinic, Rochester, Minn (M.H.M.);
Department of Medicine, Division of Nephrology and Hypertension, University of
Kansas Medical Center, Kansas City, Mo (R.M.); Department of Epidemiology and
Data Science, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
(M.L., P.M.B.); Amsterdam Public Health, Amsterdam, the Netherlands (P.M.B.);
Institute of Applied Health Research, University of Birmingham, Birmingham, UK
(Y.T.); NIHR Birmingham Biomedical Research Centre, University Hospitals
Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
(Y.T.); Population Health Sciences, Bristol Medical School, University of
Bristol, Bristol, UK (P.W.); Department of Radiology, NYU Langone Health, New
York, NY (S.K.K.); and Centre for Clinical Epidemiology, Lady Davis Institute
for Medical Research, Jewish General Hospital, Montréal, Canada
(B.L.)
| | - Patrick M. Bossuyt
- From the Department of Radiology, University of Ottawa, The Ottawa
Hospital Civic Campus, 1053 Carling Ave, Room c159, Ottawa, ON, Canada K1Y 4E9
(R.A.F., M.D.F.M.); Faculty of Health Sciences, Queen’s University,
Kingston, Ontario, Canada (J.P.S.); Clinical Epidemiology Program, Ottawa
Hospital Research Institute, University of Ottawa, Ottawa, Canada (N.I., M.H.M.,
H.D., S.E., B.H.); Julius Center for Health Sciences and Primary Care,
University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
(B.Y.); Evidence-Based Practice Center, Mayo Clinic, Rochester, Minn (M.H.M.);
Department of Medicine, Division of Nephrology and Hypertension, University of
Kansas Medical Center, Kansas City, Mo (R.M.); Department of Epidemiology and
Data Science, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
(M.L., P.M.B.); Amsterdam Public Health, Amsterdam, the Netherlands (P.M.B.);
Institute of Applied Health Research, University of Birmingham, Birmingham, UK
(Y.T.); NIHR Birmingham Biomedical Research Centre, University Hospitals
Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
(Y.T.); Population Health Sciences, Bristol Medical School, University of
Bristol, Bristol, UK (P.W.); Department of Radiology, NYU Langone Health, New
York, NY (S.K.K.); and Centre for Clinical Epidemiology, Lady Davis Institute
for Medical Research, Jewish General Hospital, Montréal, Canada
(B.L.)
| | - Yemisi Takwoingi
- From the Department of Radiology, University of Ottawa, The Ottawa
Hospital Civic Campus, 1053 Carling Ave, Room c159, Ottawa, ON, Canada K1Y 4E9
(R.A.F., M.D.F.M.); Faculty of Health Sciences, Queen’s University,
Kingston, Ontario, Canada (J.P.S.); Clinical Epidemiology Program, Ottawa
Hospital Research Institute, University of Ottawa, Ottawa, Canada (N.I., M.H.M.,
H.D., S.E., B.H.); Julius Center for Health Sciences and Primary Care,
University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
(B.Y.); Evidence-Based Practice Center, Mayo Clinic, Rochester, Minn (M.H.M.);
Department of Medicine, Division of Nephrology and Hypertension, University of
Kansas Medical Center, Kansas City, Mo (R.M.); Department of Epidemiology and
Data Science, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
(M.L., P.M.B.); Amsterdam Public Health, Amsterdam, the Netherlands (P.M.B.);
Institute of Applied Health Research, University of Birmingham, Birmingham, UK
(Y.T.); NIHR Birmingham Biomedical Research Centre, University Hospitals
Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
(Y.T.); Population Health Sciences, Bristol Medical School, University of
Bristol, Bristol, UK (P.W.); Department of Radiology, NYU Langone Health, New
York, NY (S.K.K.); and Centre for Clinical Epidemiology, Lady Davis Institute
for Medical Research, Jewish General Hospital, Montréal, Canada
(B.L.)
| | - Penny Whiting
- From the Department of Radiology, University of Ottawa, The Ottawa
Hospital Civic Campus, 1053 Carling Ave, Room c159, Ottawa, ON, Canada K1Y 4E9
(R.A.F., M.D.F.M.); Faculty of Health Sciences, Queen’s University,
Kingston, Ontario, Canada (J.P.S.); Clinical Epidemiology Program, Ottawa
Hospital Research Institute, University of Ottawa, Ottawa, Canada (N.I., M.H.M.,
H.D., S.E., B.H.); Julius Center for Health Sciences and Primary Care,
University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
(B.Y.); Evidence-Based Practice Center, Mayo Clinic, Rochester, Minn (M.H.M.);
Department of Medicine, Division of Nephrology and Hypertension, University of
Kansas Medical Center, Kansas City, Mo (R.M.); Department of Epidemiology and
Data Science, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
(M.L., P.M.B.); Amsterdam Public Health, Amsterdam, the Netherlands (P.M.B.);
Institute of Applied Health Research, University of Birmingham, Birmingham, UK
(Y.T.); NIHR Birmingham Biomedical Research Centre, University Hospitals
Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
(Y.T.); Population Health Sciences, Bristol Medical School, University of
Bristol, Bristol, UK (P.W.); Department of Radiology, NYU Langone Health, New
York, NY (S.K.K.); and Centre for Clinical Epidemiology, Lady Davis Institute
for Medical Research, Jewish General Hospital, Montréal, Canada
(B.L.)
| | - Haben Dawit
- From the Department of Radiology, University of Ottawa, The Ottawa
Hospital Civic Campus, 1053 Carling Ave, Room c159, Ottawa, ON, Canada K1Y 4E9
(R.A.F., M.D.F.M.); Faculty of Health Sciences, Queen’s University,
Kingston, Ontario, Canada (J.P.S.); Clinical Epidemiology Program, Ottawa
Hospital Research Institute, University of Ottawa, Ottawa, Canada (N.I., M.H.M.,
H.D., S.E., B.H.); Julius Center for Health Sciences and Primary Care,
University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
(B.Y.); Evidence-Based Practice Center, Mayo Clinic, Rochester, Minn (M.H.M.);
Department of Medicine, Division of Nephrology and Hypertension, University of
Kansas Medical Center, Kansas City, Mo (R.M.); Department of Epidemiology and
Data Science, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
(M.L., P.M.B.); Amsterdam Public Health, Amsterdam, the Netherlands (P.M.B.);
Institute of Applied Health Research, University of Birmingham, Birmingham, UK
(Y.T.); NIHR Birmingham Biomedical Research Centre, University Hospitals
Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
(Y.T.); Population Health Sciences, Bristol Medical School, University of
Bristol, Bristol, UK (P.W.); Department of Radiology, NYU Langone Health, New
York, NY (S.K.K.); and Centre for Clinical Epidemiology, Lady Davis Institute
for Medical Research, Jewish General Hospital, Montréal, Canada
(B.L.)
| | - Stella K. Kang
- From the Department of Radiology, University of Ottawa, The Ottawa
Hospital Civic Campus, 1053 Carling Ave, Room c159, Ottawa, ON, Canada K1Y 4E9
(R.A.F., M.D.F.M.); Faculty of Health Sciences, Queen’s University,
Kingston, Ontario, Canada (J.P.S.); Clinical Epidemiology Program, Ottawa
Hospital Research Institute, University of Ottawa, Ottawa, Canada (N.I., M.H.M.,
H.D., S.E., B.H.); Julius Center for Health Sciences and Primary Care,
University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
(B.Y.); Evidence-Based Practice Center, Mayo Clinic, Rochester, Minn (M.H.M.);
Department of Medicine, Division of Nephrology and Hypertension, University of
Kansas Medical Center, Kansas City, Mo (R.M.); Department of Epidemiology and
Data Science, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
(M.L., P.M.B.); Amsterdam Public Health, Amsterdam, the Netherlands (P.M.B.);
Institute of Applied Health Research, University of Birmingham, Birmingham, UK
(Y.T.); NIHR Birmingham Biomedical Research Centre, University Hospitals
Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
(Y.T.); Population Health Sciences, Bristol Medical School, University of
Bristol, Bristol, UK (P.W.); Department of Radiology, NYU Langone Health, New
York, NY (S.K.K.); and Centre for Clinical Epidemiology, Lady Davis Institute
for Medical Research, Jewish General Hospital, Montréal, Canada
(B.L.)
| | - Sanam Ebrahimzadeh
- From the Department of Radiology, University of Ottawa, The Ottawa
Hospital Civic Campus, 1053 Carling Ave, Room c159, Ottawa, ON, Canada K1Y 4E9
(R.A.F., M.D.F.M.); Faculty of Health Sciences, Queen’s University,
Kingston, Ontario, Canada (J.P.S.); Clinical Epidemiology Program, Ottawa
Hospital Research Institute, University of Ottawa, Ottawa, Canada (N.I., M.H.M.,
H.D., S.E., B.H.); Julius Center for Health Sciences and Primary Care,
University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
(B.Y.); Evidence-Based Practice Center, Mayo Clinic, Rochester, Minn (M.H.M.);
Department of Medicine, Division of Nephrology and Hypertension, University of
Kansas Medical Center, Kansas City, Mo (R.M.); Department of Epidemiology and
Data Science, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
(M.L., P.M.B.); Amsterdam Public Health, Amsterdam, the Netherlands (P.M.B.);
Institute of Applied Health Research, University of Birmingham, Birmingham, UK
(Y.T.); NIHR Birmingham Biomedical Research Centre, University Hospitals
Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
(Y.T.); Population Health Sciences, Bristol Medical School, University of
Bristol, Bristol, UK (P.W.); Department of Radiology, NYU Langone Health, New
York, NY (S.K.K.); and Centre for Clinical Epidemiology, Lady Davis Institute
for Medical Research, Jewish General Hospital, Montréal, Canada
(B.L.)
| | - Brooke Levis
- From the Department of Radiology, University of Ottawa, The Ottawa
Hospital Civic Campus, 1053 Carling Ave, Room c159, Ottawa, ON, Canada K1Y 4E9
(R.A.F., M.D.F.M.); Faculty of Health Sciences, Queen’s University,
Kingston, Ontario, Canada (J.P.S.); Clinical Epidemiology Program, Ottawa
Hospital Research Institute, University of Ottawa, Ottawa, Canada (N.I., M.H.M.,
H.D., S.E., B.H.); Julius Center for Health Sciences and Primary Care,
University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
(B.Y.); Evidence-Based Practice Center, Mayo Clinic, Rochester, Minn (M.H.M.);
Department of Medicine, Division of Nephrology and Hypertension, University of
Kansas Medical Center, Kansas City, Mo (R.M.); Department of Epidemiology and
Data Science, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
(M.L., P.M.B.); Amsterdam Public Health, Amsterdam, the Netherlands (P.M.B.);
Institute of Applied Health Research, University of Birmingham, Birmingham, UK
(Y.T.); NIHR Birmingham Biomedical Research Centre, University Hospitals
Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
(Y.T.); Population Health Sciences, Bristol Medical School, University of
Bristol, Bristol, UK (P.W.); Department of Radiology, NYU Langone Health, New
York, NY (S.K.K.); and Centre for Clinical Epidemiology, Lady Davis Institute
for Medical Research, Jewish General Hospital, Montréal, Canada
(B.L.)
| | - Brian Hutton
- From the Department of Radiology, University of Ottawa, The Ottawa
Hospital Civic Campus, 1053 Carling Ave, Room c159, Ottawa, ON, Canada K1Y 4E9
(R.A.F., M.D.F.M.); Faculty of Health Sciences, Queen’s University,
Kingston, Ontario, Canada (J.P.S.); Clinical Epidemiology Program, Ottawa
Hospital Research Institute, University of Ottawa, Ottawa, Canada (N.I., M.H.M.,
H.D., S.E., B.H.); Julius Center for Health Sciences and Primary Care,
University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
(B.Y.); Evidence-Based Practice Center, Mayo Clinic, Rochester, Minn (M.H.M.);
Department of Medicine, Division of Nephrology and Hypertension, University of
Kansas Medical Center, Kansas City, Mo (R.M.); Department of Epidemiology and
Data Science, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
(M.L., P.M.B.); Amsterdam Public Health, Amsterdam, the Netherlands (P.M.B.);
Institute of Applied Health Research, University of Birmingham, Birmingham, UK
(Y.T.); NIHR Birmingham Biomedical Research Centre, University Hospitals
Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
(Y.T.); Population Health Sciences, Bristol Medical School, University of
Bristol, Bristol, UK (P.W.); Department of Radiology, NYU Langone Health, New
York, NY (S.K.K.); and Centre for Clinical Epidemiology, Lady Davis Institute
for Medical Research, Jewish General Hospital, Montréal, Canada
(B.L.)
| | - Matthew D. F. McInnes
- From the Department of Radiology, University of Ottawa, The Ottawa
Hospital Civic Campus, 1053 Carling Ave, Room c159, Ottawa, ON, Canada K1Y 4E9
(R.A.F., M.D.F.M.); Faculty of Health Sciences, Queen’s University,
Kingston, Ontario, Canada (J.P.S.); Clinical Epidemiology Program, Ottawa
Hospital Research Institute, University of Ottawa, Ottawa, Canada (N.I., M.H.M.,
H.D., S.E., B.H.); Julius Center for Health Sciences and Primary Care,
University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
(B.Y.); Evidence-Based Practice Center, Mayo Clinic, Rochester, Minn (M.H.M.);
Department of Medicine, Division of Nephrology and Hypertension, University of
Kansas Medical Center, Kansas City, Mo (R.M.); Department of Epidemiology and
Data Science, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
(M.L., P.M.B.); Amsterdam Public Health, Amsterdam, the Netherlands (P.M.B.);
Institute of Applied Health Research, University of Birmingham, Birmingham, UK
(Y.T.); NIHR Birmingham Biomedical Research Centre, University Hospitals
Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
(Y.T.); Population Health Sciences, Bristol Medical School, University of
Bristol, Bristol, UK (P.W.); Department of Radiology, NYU Langone Health, New
York, NY (S.K.K.); and Centre for Clinical Epidemiology, Lady Davis Institute
for Medical Research, Jewish General Hospital, Montréal, Canada
(B.L.)
| |
Collapse
|
4
|
Veroniki AA, Tricco AC, Watt J, Tsokani S, Khan PA, Soobiah C, Negm A, Doherty-Kirby A, Taylor P, Lunny C, McGowan J, Little J, Mallon P, Moher D, Wong S, Dinnes J, Takwoingi Y, Saxinger L, Chan A, Isaranuwatchai W, Lander B, Meyers A, Poliquin G, Straus SE. Rapid antigen-based and rapid molecular tests for the detection of SARS-CoV-2: a rapid review with network meta-analysis of diagnostic test accuracy studies. BMC Med 2023; 21:110. [PMID: 36978074 PMCID: PMC10049780 DOI: 10.1186/s12916-023-02810-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/12/2022] [Accepted: 02/27/2023] [Indexed: 03/30/2023] Open
Abstract
BACKGROUND The global spread of COVID-19 created an explosion in rapid tests with results in < 1 hour, but their relative performance characteristics are not fully understood yet. Our aim was to determine the most sensitive and specific rapid test for the diagnosis of SARS-CoV-2. METHODS Design: Rapid review and diagnostic test accuracy network meta-analysis (DTA-NMA). ELIGIBILITY CRITERIA Randomized controlled trials (RCTs) and observational studies assessing rapid antigen and/or rapid molecular test(s) to detect SARS-CoV-2 in participants of any age, suspected or not with SARS-CoV-2 infection. INFORMATION SOURCES Embase, MEDLINE, and Cochrane Central Register of Controlled Trials, up to September 12, 2021. OUTCOME MEASURES Sensitivity and specificity of rapid antigen and molecular tests suitable for detecting SARS-CoV-2. Data extraction and risk of bias assessment: Screening of literature search results was conducted by one reviewer; data abstraction was completed by one reviewer and independently verified by a second reviewer. Risk of bias was not assessed in the included studies. DATA SYNTHESIS Random-effects meta-analysis and DTA-NMA. RESULTS We included 93 studies (reported in 88 articles) relating to 36 rapid antigen tests in 104,961 participants and 23 rapid molecular tests in 10,449 participants. Overall, rapid antigen tests had a sensitivity of 0.75 (95% confidence interval 0.70-0.79) and specificity of 0.99 (0.98-0.99). Rapid antigen test sensitivity was higher when nasal or combined samples (e.g., combinations of nose, throat, mouth, or saliva samples) were used, but lower when nasopharyngeal samples were used, and in those classified as asymptomatic at the time of testing. Rapid molecular tests may result in fewer false negatives than rapid antigen tests (sensitivity: 0.93, 0.88-0.96; specificity: 0.98, 0.97-0.99). The tests with the highest sensitivity and specificity estimates were the Xpert Xpress rapid molecular test by Cepheid (sensitivity: 0.99, 0.83-1.00; specificity: 0.97, 0.69-1.00) among the 23 commercial rapid molecular tests and the COVID-VIRO test by AAZ-LMB (sensitivity: 0.93, 0.48-0.99; specificity: 0.98, 0.44-1.00) among the 36 rapid antigen tests we examined. CONCLUSIONS Rapid molecular tests were associated with both high sensitivity and specificity, while rapid antigen tests were mainly associated with high specificity, according to the minimum performance requirements by WHO and Health Canada. Our rapid review was limited to English, peer-reviewed published results of commercial tests, and study risk of bias was not assessed. A full systematic review is required. REVIEW REGISTRATION PROSPERO CRD42021289712.
Collapse
Affiliation(s)
- Areti Angeliki Veroniki
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, 209 Victoria Street, East Building, Toronto, ON, M5B 1T8, Canada.
- Institute for Health Policy, Management, and Evaluation, University of Toronto, Toronto, ON, Canada.
| | - Andrea C Tricco
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, 209 Victoria Street, East Building, Toronto, ON, M5B 1T8, Canada
- Epidemiology Division & Institute of Health Policy, Management, and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
- Queen's Collaboration for Health Care Quality: A JBI Centre of Excellence, Kingston, Canada
| | - Jennifer Watt
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, 209 Victoria Street, East Building, Toronto, ON, M5B 1T8, Canada
| | - Sofia Tsokani
- School of Education, University of Ioannina, Ioannina, Greece
| | - Paul A Khan
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, 209 Victoria Street, East Building, Toronto, ON, M5B 1T8, Canada
| | - Charlene Soobiah
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, 209 Victoria Street, East Building, Toronto, ON, M5B 1T8, Canada
- Institute for Health Policy, Management, and Evaluation, University of Toronto, Toronto, ON, Canada
| | - Ahmed Negm
- University of Alberta, Edmonton, AB, Canada
| | - Amanda Doherty-Kirby
- Patient Partner, Strategy for Patient Oriented-Research Evidence Alliance (SPOR EA), Toronto, Canada
| | - Paul Taylor
- Patient Partner, Strategy for Patient Oriented-Research Evidence Alliance (SPOR EA), Toronto, Canada
| | - Carole Lunny
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, 209 Victoria Street, East Building, Toronto, ON, M5B 1T8, Canada
| | - Jessie McGowan
- University of Ottawa/Université d'Ottawa, Ottawa, ON, Canada
| | - Julian Little
- University of Ottawa/Université d'Ottawa, Ottawa, ON, Canada
| | | | - David Moher
- Ottawa Hospital Research Institute/Institut de Recherche de L'Hôpital d'Ottawa, Ottawa, ON, Canada
| | - Sabrina Wong
- University of British Columbia, Vancouver, BC, Canada
| | | | | | | | | | | | - Bryn Lander
- Health Canada (Ottawa)/Santé Canada (Ottawa), Ottawa, ON, Canada
| | - Adrienne Meyers
- Public Health Agency of Canada/Agence de La Santé Publique du Canada, Ottawa, ON, Canada
| | - Guillaume Poliquin
- Public Health Agency of Canada/Agence de La Santé Publique du Canada, Ottawa, ON, Canada
| | - Sharon E Straus
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, 209 Victoria Street, East Building, Toronto, ON, M5B 1T8, Canada
- Institute for Health Policy, Management, and Evaluation, University of Toronto, Toronto, ON, Canada
- Department of Geriatric Medicine, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
5
|
Fox T, Geppert J, Dinnes J, Scandrett K, Bigio J, Sulis G, Hettiarachchi D, Mathangasinghe Y, Weeratunga P, Wickramasinghe D, Bergman H, Buckley BS, Probyn K, Sguassero Y, Davenport C, Cunningham J, Dittrich S, Emperador D, Hooft L, Leeflang MM, McInnes MD, Spijker R, Struyf T, Van den Bruel A, Verbakel JY, Takwoingi Y, Taylor-Phillips S, Deeks JJ. Antibody tests for identification of current and past infection with SARS-CoV-2. Cochrane Database Syst Rev 2022; 11:CD013652. [PMID: 36394900 PMCID: PMC9671206 DOI: 10.1002/14651858.cd013652.pub2] [Citation(s) in RCA: 27] [Impact Index Per Article: 13.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
BACKGROUND The diagnostic challenges associated with the COVID-19 pandemic resulted in rapid development of diagnostic test methods for detecting SARS-CoV-2 infection. Serology tests to detect the presence of antibodies to SARS-CoV-2 enable detection of past infection and may detect cases of SARS-CoV-2 infection that were missed by earlier diagnostic tests. Understanding the diagnostic accuracy of serology tests for SARS-CoV-2 infection may enable development of effective diagnostic and management pathways, inform public health management decisions and understanding of SARS-CoV-2 epidemiology. OBJECTIVES To assess the accuracy of antibody tests, firstly, to determine if a person presenting in the community, or in primary or secondary care has current SARS-CoV-2 infection according to time after onset of infection and, secondly, to determine if a person has previously been infected with SARS-CoV-2. Sources of heterogeneity investigated included: timing of test, test method, SARS-CoV-2 antigen used, test brand, and reference standard for non-SARS-CoV-2 cases. SEARCH METHODS The COVID-19 Open Access Project living evidence database from the University of Bern (which includes daily updates from PubMed and Embase and preprints from medRxiv and bioRxiv) was searched on 30 September 2020. We included additional publications from the Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre) 'COVID-19: Living map of the evidence' and the Norwegian Institute of Public Health 'NIPH systematic and living map on COVID-19 evidence'. We did not apply language restrictions. SELECTION CRITERIA We included test accuracy studies of any design that evaluated commercially produced serology tests, targeting IgG, IgM, IgA alone, or in combination. Studies must have provided data for sensitivity, that could be allocated to a predefined time period after onset of symptoms, or after a positive RT-PCR test. Small studies with fewer than 25 SARS-CoV-2 infection cases were excluded. We included any reference standard to define the presence or absence of SARS-CoV-2 (including reverse transcription polymerase chain reaction tests (RT-PCR), clinical diagnostic criteria, and pre-pandemic samples). DATA COLLECTION AND ANALYSIS We use standard screening procedures with three reviewers. Quality assessment (using the QUADAS-2 tool) and numeric study results were extracted independently by two people. Other study characteristics were extracted by one reviewer and checked by a second. We present sensitivity and specificity with 95% confidence intervals (CIs) for each test and, for meta-analysis, we fitted univariate random-effects logistic regression models for sensitivity by eligible time period and for specificity by reference standard group. Heterogeneity was investigated by including indicator variables in the random-effects logistic regression models. We tabulated results by test manufacturer and summarised results for tests that were evaluated in 200 or more samples and that met a modification of UK Medicines and Healthcare products Regulatory Agency (MHRA) target performance criteria. MAIN RESULTS We included 178 separate studies (described in 177 study reports, with 45 as pre-prints) providing 527 test evaluations. The studies included 64,688 samples including 25,724 from people with confirmed SARS-CoV-2; most compared the accuracy of two or more assays (102/178, 57%). Participants with confirmed SARS-CoV-2 infection were most commonly hospital inpatients (78/178, 44%), and pre-pandemic samples were used by 45% (81/178) to estimate specificity. Over two-thirds of studies recruited participants based on known SARS-CoV-2 infection status (123/178, 69%). All studies were conducted prior to the introduction of SARS-CoV-2 vaccines and present data for naturally acquired antibody responses. Seventy-nine percent (141/178) of studies reported sensitivity by week after symptom onset and 66% (117/178) for convalescent phase infection. Studies evaluated enzyme-linked immunosorbent assays (ELISA) (165/527; 31%), chemiluminescent assays (CLIA) (167/527; 32%) or lateral flow assays (LFA) (188/527; 36%). Risk of bias was high because of participant selection (172, 97%); application and interpretation of the index test (35, 20%); weaknesses in the reference standard (38, 21%); and issues related to participant flow and timing (148, 82%). We judged that there were high concerns about the applicability of the evidence related to participants in 170 (96%) studies, and about the applicability of the reference standard in 162 (91%) studies. Average sensitivities for current SARS-CoV-2 infection increased by week after onset for all target antibodies. Average sensitivity for the combination of either IgG or IgM was 41.1% in week one (95% CI 38.1 to 44.2; 103 evaluations; 3881 samples, 1593 cases), 74.9% in week two (95% CI 72.4 to 77.3; 96 evaluations, 3948 samples, 2904 cases) and 88.0% by week three after onset of symptoms (95% CI 86.3 to 89.5; 103 evaluations, 2929 samples, 2571 cases). Average sensitivity during the convalescent phase of infection (up to a maximum of 100 days since onset of symptoms, where reported) was 89.8% for IgG (95% CI 88.5 to 90.9; 253 evaluations, 16,846 samples, 14,183 cases), 92.9% for IgG or IgM combined (95% CI 91.0 to 94.4; 108 evaluations, 3571 samples, 3206 cases) and 94.3% for total antibodies (95% CI 92.8 to 95.5; 58 evaluations, 7063 samples, 6652 cases). Average sensitivities for IgM alone followed a similar pattern but were of a lower test accuracy in every time slot. Average specificities were consistently high and precise, particularly for pre-pandemic samples which provide the least biased estimates of specificity (ranging from 98.6% for IgM to 99.8% for total antibodies). Subgroup analyses suggested small differences in sensitivity and specificity by test technology however heterogeneity in study results, timing of sample collection, and smaller sample numbers in some groups made comparisons difficult. For IgG, CLIAs were the most sensitive (convalescent-phase infection) and specific (pre-pandemic samples) compared to both ELISAs and LFAs (P < 0.001 for differences across test methods). The antigen(s) used (whether from the Spike-protein or nucleocapsid) appeared to have some effect on average sensitivity in the first weeks after onset but there was no clear evidence of an effect during convalescent-phase infection. Investigations of test performance by brand showed considerable variation in sensitivity between tests, and in results between studies evaluating the same test. For tests that were evaluated in 200 or more samples, the lower bound of the 95% CI for sensitivity was 90% or more for only a small number of tests (IgG, n = 5; IgG or IgM, n = 1; total antibodies, n = 4). More test brands met the MHRA minimum criteria for specificity of 98% or above (IgG, n = 16; IgG or IgM, n = 5; total antibodies, n = 7). Seven assays met the specified criteria for both sensitivity and specificity. In a low-prevalence (2%) setting, where antibody testing is used to diagnose COVID-19 in people with symptoms but who have had a negative PCR test, we would anticipate that 1 (1 to 2) case would be missed and 8 (5 to 15) would be falsely positive in 1000 people undergoing IgG or IgM testing in week three after onset of SARS-CoV-2 infection. In a seroprevalence survey, where prevalence of prior infection is 50%, we would anticipate that 51 (46 to 58) cases would be missed and 6 (5 to 7) would be falsely positive in 1000 people having IgG tests during the convalescent phase (21 to 100 days post-symptom onset or post-positive PCR) of SARS-CoV-2 infection. AUTHORS' CONCLUSIONS Some antibody tests could be a useful diagnostic tool for those in whom molecular- or antigen-based tests have failed to detect the SARS-CoV-2 virus, including in those with ongoing symptoms of acute infection (from week three onwards) or those presenting with post-acute sequelae of COVID-19. However, antibody tests have an increasing likelihood of detecting an immune response to infection as time since onset of infection progresses and have demonstrated adequate performance for detection of prior infection for sero-epidemiological purposes. The applicability of results for detection of vaccination-induced antibodies is uncertain.
Collapse
Affiliation(s)
- Tilly Fox
- Department of Clinical Sciences, Liverpool School of Tropical Medicine, Liverpool, UK
| | - Julia Geppert
- Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, UK
| | - Jacqueline Dinnes
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Katie Scandrett
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Jacob Bigio
- Research Institute of the McGill University Health Centre, Montreal, Canada
- McGill International TB Centre, Montreal, Canada
| | - Giorgia Sulis
- Department of Epidemiology, Biostatistics and Occupational Health, McGill University, Montreal, Canada
| | - Dineshani Hettiarachchi
- Department of Anatomy Genetics and Biomedical Informatics, Faculty of Medicine, University of Colombo, Colombo, Sri Lanka
| | - Yasith Mathangasinghe
- Department of Anatomy Genetics and Biomedical Informatics, Faculty of Medicine, University of Colombo, Colombo, Sri Lanka
- Australian Regenerative Medicine Institute, Monash University, Clayton, Australia
| | - Praveen Weeratunga
- Department of Clinical Medicine, Faculty of Medicine, University of Colombo, Colombo, Sri Lanka
| | | | | | - Brian S Buckley
- Cochrane Response, Cochrane, London, UK
- Department of Surgery, University of the Philippines, Manila, Philippines
| | | | | | - Clare Davenport
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Jane Cunningham
- Global Malaria Programme, World Health Organization, Geneva, Switzerland
| | | | | | - Lotty Hooft
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht , Netherlands
| | - Mariska Mg Leeflang
- Epidemiology and Data Science, Amsterdam UMC location University of Amsterdam, Amsterdam, Netherlands
- Amsterdam Public Health, Amsterdam, Netherlands
| | | | - René Spijker
- Medical Library, Amsterdam UMC, University of Amsterdam, Amsterdam Public Health, Amsterdam, Netherlands
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | - Thomas Struyf
- Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
| | - Ann Van den Bruel
- Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
| | - Jan Y Verbakel
- Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
| | - Yemisi Takwoingi
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Sian Taylor-Phillips
- Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, UK
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Jonathan J Deeks
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| |
Collapse
|