Operator Influence on Blinded Diagnostic Accuracy of Point-of-Care Antigen Testing for Group A Streptococcal Pharyngitis

Improving diagnostic performance of coronavirus disease 2019 rapid antigen testing through computer-based feedback training using open-source experimental psychology software

INTRODUCTION

Rapid antigen testing (RAT) results are visually read as whether colored line is present or absent. The subjective interpretation potentially misses detecting weak lines due to lower analyte concentration in samples tested, requiring training. Although routine test experience has improved the result readout skills, it consumes time and resources. Therefore, we created a computer-based feedback training method using open-source experimental psychology software, wherein participants accumulate RAT result readout experience by repeatedly responding positive/negative to randomly presented pictures showing RAT results; then, they receive feedback on their answers as correct or incorrect and are asked to stare at the pictures again with the knowledge of correct answer. This study aimed to examine the training effects in improving the skills, using coronavirus disease 2019 (COVID-19) RAT.

METHODS

Twenty-two medical technologists were randomly divided into two groups: the feedback-training and test-experience groups. Using several pictures showing positive and negative results of COVID-19 RAT, after examination of their initial result readout skills, feedback-training group received the feedback training, whereas test-experience group performed an equal number of tests without feedback to accumulate test experience, and their skills were examined again. The ratio of "positive" answers to the pictures showing positive results (i.e., hit rate) was statistically analyzed.

RESULTS

The feedback-training group showed a significantly higher hit rate after their training, whereas the test-experience group did not. The feedback training effects were manifested in weak line detection.

CONCLUSIONS

This computer-based feedback training method can be an effective tool for improving RAT result readout skills.

Rapid antigen detection and molecular tests for group A streptococcal infections for acute sore throat: systematic reviews and economic evaluation

BACKGROUND

Sore throat is a common condition caused by an infection of the airway. Most cases are of a viral nature; however, a number of these infections may be caused by the group A Streptococcus bacterium. Most viral and bacterial sore throat infections resolve spontaneously within a few weeks. Point-of-care testing in primary care has been recognised as an emerging technology for aiding targeted antibiotic prescribing for sore throat in cases that do not spontaneously resolve.

OBJECTIVE

Systematically review the evidence for 21 point-of-care tests for detecting group A Streptococcus bacteria and develop a de novo economic model to compare the cost-effectiveness of point-of-care tests alongside clinical scoring tools with the cost-effectiveness of clinical scoring tools alone for patients managed in primary care and hospital settings.

DATA SOURCES

Multiple electronic databases were searched from inception to March 2019. The following databases were searched in November and December 2018 and searches were updated in March 2019: MEDLINE [via OvidSP (Health First, Rockledge, FL, USA)], MEDLINE In-Process & Other Non-Indexed Citations (via OvidSP), MEDLINE Epub Ahead of Print (via OvidSP), MEDLINE Daily Update (via OvidSP), EMBASE (via OvidSP), Cochrane Database of Systematic Reviews [via Wiley Online Library (John Wiley & Sons, Inc., Hoboken, NJ, USA)], Cochrane Central Register of Controlled Trials (CENTRAL) (via Wiley Online Library), Database of Abstracts of Reviews of Effects (DARE) (via Centre for Reviews and Dissemination), Health Technology Assessment database (via the Centre for Reviews and Dissemination), Science Citation Index and Conference Proceedings [via the Web of Science™ (Clarivate Analytics, Philadelphia, PA, USA)] and the PROSPERO International Prospective Register of Systematic Reviews (via the Centre for Reviews and Dissemination).

REVIEW METHODS

Eligible studies included those of people aged ≥ 5 years presenting with sore throat symptoms, studies comparing point-of-care testing with antibiotic-prescribing decisions, studies of test accuracy and studies of cost-effectiveness. Quality assessment of eligible studies was undertaken. Meta-analysis of sensitivity and specificity was carried out for tests with sufficient data. A decision tree model estimated costs and quality-adjusted life-years from an NHS and Personal Social Services perspective.

RESULTS

The searches identified 38 studies of clinical effectiveness and three studies of cost-effectiveness. Twenty-six full-text articles and abstracts reported on the test accuracy of point-of-care tests and/or clinical scores with biological culture as a reference standard. In the population of interest (patients with Centor/McIsaac scores of ≥ 3 points or FeverPAIN scores of ≥ 4 points), point estimates were 0.829 to 0.946 for sensitivity and 0.849 to 0.991 for specificity. There was considerable heterogeneity, even for studies using the same point-of-care test, suggesting that is unlikely that any single study will have accurately captured a test's true performance. There is some randomised controlled trial evidence to suggest that the use of rapid antigen detection tests may help to reduce antibiotic-prescribing rates. Sensitivity and specificity estimates for each test in each age group and care setting combination were obtained using meta-analyses where appropriate. Any apparent differences in test accuracy may not be attributable to the tests, and may have been caused by known differences in the studies, latent characteristics or chance. Fourteen of the 21 tests reviewed were included in the economic modelling, and these tests were not cost-effective within the current National Institute for Health and Care Excellence's cost-effectiveness thresholds. Uncertainties in the cost-effectiveness estimates included model parameter inputs and assumptions that increase the cost of testing, and the penalty for antibiotic overprescriptions.

LIMITATIONS

No information was identified for the elderly population or pharmacy setting. It was not possible to identify which test is the most accurate owing to the paucity of evidence.

CONCLUSIONS

The systematic review and the cost-effectiveness models identified uncertainties around the adoption of point-of-care tests in primary and secondary care settings. Although sensitivity and specificity estimates are promising, we have little information to establish the most accurate point-of-care test. Further research is needed to understand the test accuracy of point-of-care tests in the proposed NHS pathway and in comparable settings and patient groups.

STUDY REGISTRATION

The protocol of the review is registered as PROSPERO CRD42018118653.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 24, No. 31. See the NIHR Journals Library website for further project information.