Diagnostic tests: understanding results, assessing utility, and predicting performance

Evaluation of in-house dengue real-time PCR assays in West Java, Indonesia

Dengue is an infectious disease caused by infection of dengue virus (DENV) transmitted by Aedes aegypti and Aedes albopictus. In Indonesia, dengue commonly occurs with an increasing incidence rate annually. It is known that early detection of dengue infection is one of the keys to controlling this disease outbreak. Rapid and accurate early detection to diagnose dengue can be achieved by molecular tests, one of which is through a real-time PCR method. However, real-time PCR assay for dengue developed based on Indonesian DENV sequences has not been available. Therefore, we developed in-house dengue real-time PCR (SYBR- and TaqMan-based) assays and evaluated those assays in routine clinical testing in the community. These assays target the 3' UTR region of the four DENV serotypes and was found to be specific for DENV. The most sensitive assay was the TaqMan assay with the LOD95% of 482 copy/ml, followed by the SYBR assay with the LOD95% of 14,398 copy/ml. We recruited dengue suspected patients from three primary health care services in West Java, Indonesia to represent the community testing setting. Dengue infection was examined using the two in-house real-time PCR assays along with NS1, IgM, and IgG rapid diagnostic tests (RDT). In total, as many as 74 clinical specimens of dengue suspected patients were included in this study. Among those patients, 21 were positive for TaqMan assay, 17 were positive for SYBR assay, nine were positive for NS1 test, six were positive for both IgG and IgM tests, and 22 were positive for IgG test only. Compared with our in-house TaqMan assay, the sensitivity of NS1 test, IgM test, and IgG test were 42.86%, 14.29%, and 28.57% respectively. Among these three RDT tests, NS1 showed 100% specificity. Thus, our study confirmed that NS1 test showed high specificity, indicating that a positive result of NS1 can be confidently considered a dengue case. However, NS1, IgM, and IgG tests with RDT are not enough to diagnose a dengue case. We suggest applying the high sensitivity and specificity rRT-PCR test as the gold standard for early detection and antibody test as a follow-up test for rRT-PCR negative cases.

Real-world artificial intelligence-based interpretation of fundus imaging as part of an eyewear prescription renewal protocol

OBJECTIVE

A real-world evaluation of the diagnostic accuracy of the Opthai® software for artificial intelligence-based detection of fundus image abnormalities in the context of the French eyewear prescription renewal protocol (RNO).

METHODS

A single-center, retrospective review of the sensitivity and specificity of the software in detecting fundus abnormalities among consecutive patients seen in our ophthalmology center in the context of the RNO protocol from July 28 through October 22, 2021. We compared abnormalities detected by the software operated by ophthalmic technicians (index test) to diagnoses confirmed by the ophthalmologist following additional examinations and/or consultation (reference test).

RESULTS

The study included 2056 eyes/fundus images of 1028 patients aged 6-50years. The software detected fundus abnormalities in 149 (7.2%) eyes or 107 (10.4%) patients. After examining the same fundus images, the ophthalmologist detected abnormalities in 35 (1.7%) eyes or 20 (1.9%) patients. The ophthalmologist did not detect abnormalities in fundus images deemed normal by the software. The most frequent diagnoses made by the ophthalmologist were glaucoma suspect (0.5% of eyes), peripapillary atrophy (0.44% of eyes), and drusen (0.39% of eyes). The software showed an overall sensitivity of 100% (95% CI 0.879-1.00) and an overall specificity of 94.4% (95% CI 0.933-0.953). The majority of false-positive software detections (5.6%) were glaucoma suspect, with the differential diagnosis of large physiological optic cups. Immediate OCT imaging by the technician allowed diagnosis by the ophthalmologist without separate consultation for 43/53 (81%) patients.

CONCLUSION

Ophthalmic technicians can use this software for highly-sensitive screening for fundus abnormalities that require evaluation by an ophthalmologist.

Safe and valid? A systematic review of the psychometric properties of culturally adapted depression scales for use among Indigenous populations

Background

Implementing culturally sensitive psychometric measures of depression may be an effective strategy to improve acceptance, response rate, and reliability of psychological assessment among Indigenous populations. However, the psychometric properties of depression scales after cultural adaptation remain unclear.

Methods

We screened the Ovid Medline, PubMed, Embase, Global Health, PsycInfo, and CINAHL databases through three levels of search terms: Depression, Psychometrics, and Indigenous, following the PRISMA guidelines. We assessed metrics for reliability (including Cronbach's alpha), validity (including fit indices), and clinical utility (including predictive value).

Results

Across 31 studies included the review, 13 different depression scales were adapted through language or content modification. Sample populations included Indigenous from the Americas, Asia, Africa, and Oceania. Most cultural adaptations had strong psychometric properties; however, few and inconsistent properties were reported. Where available, alphas, inter-rater and test-retest reliability, construct validity, and incremental validity often indicated increased cultural sensitivity of adapted scales. There were mixed results for clinical utility, criterion validity, cross-cultural validity, sensitivity, specificity, area under the receiver operating characteristic curve, predictive value, and likelihood ratio.

Conclusions

Modifications to increase cultural relevance have the potential to improve fit and acceptance of a scale by the Indigenous population, however, these changes may decrease specificity and negative predictive value. There is an urgent need for suitable tools that are useful and reliable for identifying Indigenous individuals for clinical treatment of depression. This awaits future work for optimal specificity and validated cut-off points that take into account the high prevalence of depression in these populations.

Diagnostic Performance of Various Ultrasound Risk Stratification Systems for Benign and Malignant Thyroid Nodules: A Meta-Analysis

BACKGROUND

To evaluate the diagnostic performance of ultrasound risk-stratification systems for the discrimination of benign and malignant thyroid nodules and to determine the optimal cutoff values of individual risk-stratification systems.

METHODS

PubMed, Embase, SCOPUS, Web of Science, and Cochrane library databases were searched up to August 2022. Sensitivity and specificity data were collected along with the characteristics of each study related to ultrasound risk stratification systems.

RESULTS

Sixty-seven studies involving 76,512 thyroid nodules were included in this research. The sensitivity, specificity, diagnostic odds ratios, and area under the curves by K-TIRADS (4), ACR-TIRADS (TR5), ATA (high suspicion), EU-TIRADS (5), and Kwak-TIRADS (4b) for malignancy risk stratification of thyroid nodules were 92.5%, 63.5%, 69.8%, 70.6%, and 95.8%, respectively; 62.8%, 89.6%, 87.2%, 83.9%, and 63.8%, respectively; 20.7111, 16.8442, 15.7398, 12.2986, and 38.0578, respectively; and 0.792, 0.882, 0.859, 0.843, and 0.929, respectively.

CONCLUSION

All ultrasound-based risk-stratification systems had good diagnostic performance. Although this study determined the best cutoff values in individual risk-stratification systems based on statistical assessment, clinicians could adjust or alter cutoff values based on the clinical purpose of the ultrasound and the reciprocal changes in sensitivity and specificity.

Diagnostic Performance of ACR and Kwak TI-RADS for Benign and Malignant Thyroid Nodules: An Update Systematic Review and Meta-Analysis

(1) Background: To determine the optimal cut-off values of two risk stratification systems to discriminate malignant thyroid nodules and to compare the diagnostic performance; (2) Methods: True and false positive and negative data were collected, and methodological quality was assessed for forty-six studies involving 39,085 patients; (3) Results: The highest area under the receiver operating characteristic (ROC) curve (AUC) of ACR and Kwak TI-RADS were 0.875 and 0.884. Based on the optimal sensitivity and specificity, the highest accuracy values of ROC curves or diagnostic odds ratios (DOR) were taken as the cut-off values for TR4 (moderate suspicious) and 4B. The sensitivity, specificity, DOR, and AUC by ACR (TR4) and Kwak TI-RADS (4B) for malignancy risk stratification of thyroid nodules were 94.3% and 96.4%; 52.2% and 53.7%; 17.5185 and 31.8051; 0.786 and 0.884, respectively. There were no significant differences in diagnostic accuracy in any of the direction comparisons of the two systems; (4) Conclusions: ACR and Kwak TI-RADS had good diagnostic performances (AUCs > 85%). Although we determined the best cut-off values in individual risk stratification systems based on statistical assessment, clinicians can adjust the optimal cut-off value according to the clinical purpose of the ultrasonography because raising or lowering cut-points leads to reciprocal changes in sensitivity and specificity.

Performance of the onstructural 1 Antigen Rapid Test for detecting all four DENV serotypes in clinical specimens from Bangkok, Thailand

BACKGROUND

Dengue is an arboviral disease that has a large effect on public health in subtropical and tropical countries. Rapid and accurate detection of dengue infection is necessary for diagnosis and disease management. We previously developed highly sensitive immunochromatographic devices, the TKK 1st and TKK 2nd kits, based on dengue virus (DENV) nonstructural protein 1 detection. However, these TKK kits were evaluated mainly using DENV type 2 clinical specimens collected in Bangladesh, and further validation using clinical specimens of other serotypes was needed.

METHODS

In the present study, one of the TKK kits, TKK 2nd, was evaluated using 10 DENV-1, 10 DENV-2, 4 DENV-3, 16 DENV-4, and 10 zika virus-infected clinical specimens collected in Bangkok, Thailand.

RESULTS

The TKK 2nd kit successfully detected all four DENV serotypes in patient serum specimens and did not show any cross-reactivities against zika virus serum specimens. The IgM and/or IgG anti-DENV antibodies were detected in seven serum specimens, but did not seem to affect the results of antigen detection in the TKK 2nd kit.

CONCLUSION

The results showed that the TKK 2nd kit successfully detected all four DENV serotypes in clinical specimens and confirmed the potential of the kit for dengue diagnosis in endemic countries.

Rapid diagnostic tests for the detection of recent dengue infections: An evaluation of six kits on clinical specimens

Introduction

Early and rapid confirmation of dengue infections strengthens disease surveillance program and are critical to the success of vector control measures. Rapid diagnostics tests (RDTs) are increasingly used to confirm recent dengue infections due to their ease of use and short turnaround time for results. Several studies undertaken in dengue-endemic Southeast Asia have reported the performance of RDTs against enzyme-linked immunosorbent assay (ELISA), reverse transcriptase polymerase chain reaction (RT-PCR) and virus isolation methods. However, few studies have compared multiple RDTs for the detection of dengue NS1 antigen and IgM antibody in a single combo cassette. We evaluated six RDTs in Singapore for their utility in routine clinical testing to detect recent dengue infections.

Methods

The evaluation comprised two phases. The first phase sought to determine each RDT’s specificity to dengue NS1 and IgM using zika and chikungunya virus supernatant and zika convalescent samples. RDTs that cross-reacted with zika or chikungunya were not further tested in phase 2. The second phase sought to determine the sensitivity and specificity of the remaining RDTs to dengue NS1 and IgM using pre-characterised dengue specimens and non-dengue/chikungunya febrile clinical specimens.

Results

None of the RDTs cross-reacted with zika IgM in Phase 1. Truquick and Quickprofile cross reacted with zika and chikungunya viruses and were not evaluated thereafter. Standard Q had the highest dengue NS1 and IgM sensitivity at 87.0% and 84.3% respectively whereas Bioline (68.5%) and Multisure (58.3%) had the lowest dengue NS1 and IgM sensitivity respectively. Combining dengue NS1/IgM detection results greatly improved the RDT ability to detect recent dengue infection; Standard Q had the highest sensitivity at 99.1% while Multisure had the lowest at 92.6%. All the RDTs were highly specific for dengue NS1 and IgM (96.7% to 100%). All the RDTs had high positive predictive values (98.4% to 100%) for NS1, IgM and combined NS1/IgM parameters whereas Standard Q had the highest negative predictive values at 68.2% (NS1), 63.8% (IgM) and 96.8% (NS1/IgM). For the RDTs, detection of NS1 declined from acute to convalescent phase of illness whereas IgM detection rate gradually increased over time.

Conclusion

In our study, several RDTs were evaluated for their diagnostic accuracy and capability in detecting recent dengue infection. Standard Q demonstrated a high degree of diagnostic accuracy and capability in the detection of NS1 and IgM biomarkers. RDTs can provide rapid and accurate confirmation of recent dengue infections and augment dengue surveillance and control programmes. Further studies are required to assess the usefulness of these RDTs in other epidemiology settings.

Efficient and rapid assessment of multiple aspects of frailty using the Kyoto Frailty Scale, developed from the Edmonton Frail Scale

[Purpose] Global aging has led to a dramatic increase in the number of frail people, who are likely to become bedridden. Since frailty can be partially reversed, early intervention would be beneficial for patients, family members, and clinicians. This study was designed to develop a screening tool for an accurate and comprehensive assessment of frailty by modulating the Edmonton Frail Scale (EFS). [Participants and Methods] The EFS, covering multiple domains, is one of the major diagnostic tools for frailty. Frail and non-frail participants (n=67) were evaluated for each diagnostic item of the EFS to identify the most efficient combination of questions by evaluating its sensitivity and specificity. [Results] The Kyoto Frailty Scale (KFS) was developed as a rapid frailty scale, based on the EFS. The KFS comprises nine questions about health status, polypharmacy, hospitalization, living with a reliable caregiver, shopping, transportation, housework, money management, and forgetting to take medicine. The KFS has an excellent negative predictive value (100%) for screening frailty and a positive predictive value (97%) for screening prefrailty and frailty if we regard KFS ≥4 as a test positive. [Conclusion] The KFS permits clinician to rapidly and accurately screen for frailty and prefrailty, or exclude frailty.