Variation of a test's sensitivity and specificity with disease prevalence

External validation of the preHEART score and comparison with current clinical risk scores for prehospital risk assessment in patients with suspected NSTE-ACS

BACKGROUND

Emergency Medical Services (EMS) studies have shown that prehospital risk stratification and triage decisions in patients with suspected non-ST-elevation acute coronary syndrome (NSTE-ACS) can be improved using clinical risk scores with point-of-care (POC) troponin. In current EMS studies, three different clinical risk scores are used in patients suspected of NSTE-ACS: the prehospital History, ECG, Age, Risk and Troponin (preHEART) score, History, ECG, Age, Risk and Troponin (HEART) score and Troponin-only Manchester Acute Coronary Syndromes (T-MACS). The preHEART score lacks external validation and there exists no prospective comparative analysis of the different risk scores within the prehospital setting. The aim of this analysis is to externally validate the preHEART score and compare the diagnostic performance of the these three clinical risk scores and POC-troponin.

METHODS

Prespecified analysis from a prospective, multicentre, cohort study in patients with suspected NSTE-ACS who were transported to an ED between April 2021 and December 2022 in the Netherlands. Risk stratification is performed by EMS personnel using preHEART, HEART, T-MACS and POC-troponin. The primary end point was the hospital diagnosis of NSTE-ACS. The diagnostic performance was expressed as area under the receiver operating characteristic (AUROC), sensitivity, specificity, negative predictive value (NPV) and positive predictive value (PPV).

RESULTS

A total of 823 patients were included for external validation of the preHEART score, final hospital diagnosis of NSTE-ACS was made in 29% (n=235). The preHEART score classified 27% as low risk, with a sensitivity of 92.8% (95% CI 88.7 to 95.7) and NPV of 92.3% (95% CI 88.3 to 95.1). The preHEART classified 9% of the patients as high risk, with a specificity of 98.5% (95% CI 97.1 to 99.3) and PPV of 87.7% (95% CI 78.3 to 93.4). Data for comparing clinical risk scores and POC-troponin were available in 316 patients. No difference was found between the preHEART score and HEART score (AUROC 0.83 (95% CI 0.78 to 0.87) vs AUROC 0.80 (95% CI 0.74 to 0.85), p=0.19), and both were superior compared with T-MACS (AUROC 0.72 (95% CI 0.66 to 0.79), p≤0.001 and p=0.03, respectively) and POC-troponin measurement alone (AUROC 0.71 (95% CI 0.64 to 0.78), p<0.001 and p=0.01, respectively).

CONCLUSION

On external validation, the preHEART demonstrates good overall diagnostic performance as a prehospital risk stratification tool. Both the preHEART and HEART scores have better overall diagnostic performance compared with T-MACS and sole POC-troponin measurement. These data support the implementation of clinical risk scores in prehospital clinical pathways.

TRIAL REGISTRATION NUMBER

NCT05243485.

Performance of the Vetscan Imagyst in point-of-care detection of Giardia duodenalis in canine fecal samples

Giardia duodenalis is a common parasite of the gastrointestinal tract of dogs, with an especially high prevalence in dogs <1-y-old. Methods for detecting G. duodenalis are point-of-care (POC) tests such as lateral-flow tests or fecal flotation. The Vetscan Imagyst (Zoetis) is a new POC device for the detection of G. duodenalis in fecal samples using zinc sulfate flotation, automated slide scanning, and image recognition with artificial intelligence. Vetscan results are the number of Giardia cysts per coverslip. We compared the performance of the Vetscan and another POC test (SNAP Giardia test; Idexx) with a direct immunofluorescence assay (IFA) performed in a specialized parasitology laboratory as the reference test. We included 164 dogs <19-mo-old. We used pooled fecal samples from 3 defecations gained within 2-3 d and tested the repeatability of the Vetscan by triplicate measurement. Compared to IFA, Vetscan had a diagnostic sensitivity of 88.4% and specificity of 98.1%; SNAP had a diagnostic sensitivity of 74.4% and specificity of 98.1%. A variation coefficient of 67.0% was determined for the Vetscan results. The performance of the Vetscan is acceptable for the qualitative evaluation of fecal samples (Giardia positive or negative), and the device can be used by untrained personnel. Given its high variation coefficient, we do not recommend the Vetscan for monitoring the number of cysts.

A Comparative Analysis of Two Commonly Used FDA-Approved Immunoassays for Fentanyl Detection

BACKGROUND

Given the opioid epidemic, fentanyl screening in urine has become increasingly important. Immunoassays remain the most common screening methodology due to the high throughput and ease of integration into automated chemistry systems. The fentanyl ARK II from Ark Diagnostics is a widely used immunoassay, while a novel fentanyl assay called FEN2 by Lin-Zhi has become available on the Roche platform. Here, we evaluate and compare their performance.

METHODS

Four hundred and thirty-four urine samples were analyzed for fentanyl across the Lin-Zhi FEN2 and ARK II assays on the Cobas c502 platform. Samples were analyzed immediately upon request for drug of abuse screening or frozen for subsequent analysis. For confirmation testing, a liquid chromatography-tandem mass spectrometry (LC-MS/MS) method with a limit of detection of 1 ng/mL for fentanyl/norfentanyl was used. Any sample with either fentanyl or norfentanyl above the LC-MS/MS cutoff was deemed positive.

RESULTS

The ARK II had 11 false negatives and 7 false positives, while the Lin-Zhi FEN2 had 12 false negatives and 2 false positives. This resulted in ARK II having a sensitivity and specificity of 90.4% and 97.8% respectively, while Lin-Zhi FEN2 had a sensitivity and specificity of 89.5% and 99.4%.

CONCLUSIONS

Both the ARK II and Lin-Zhi FEN2 immunoassays detected fentanyl well. Overall, the Lin-Zhi assay had slightly better specificity than ARK II, in our data set. While some discrepant results were observed between the 2 immunoassay systems, most occurred near the immunoassay detection cutoffs.

A Real-World Assessment of Stage I Lung Cancer Through Electronic Nose Technology

INTRODUCTION

Electronic nose (E-nose) technology has reported excellent sensitivity and specificity in the setting of lung cancer screening. However, the performance of E-nose specifically for early-stage tumors remains unclear. Therefore, the aim of our study was to assess the diagnostic performance of E-nose technology in clinical stage I lung cancer.

METHODS

This phase IIc trial (NCT04734145) included patients diagnosed with a single greater than or equal to 50% solid stage I nodule. Exhalates were prospectively collected from January 2020 to August 2023. Blinded bioengineers analyzed the exhalates, using E-nose technology to determine the probability of malignancy. Patients were stratified into three risk groups (low-risk, [<0.2]; moderate-risk, [≥0.2-0.7]; high-risk, [≥0.7]). The primary outcome was the diagnostic performance of E-nose versus histopathology (accuracy and F1 score). The secondary outcome was the clinical performance of the E-nose versus clinicoradiological prediction models.

RESULTS

Based on the predefined cutoff (<0.20), E-nose agreed with histopathologic results in 86% of cases, achieving an F1 score of 92.5%, based on 86 true positives, two false negatives, and 12 false positives (n = 100). E-nose would refer fewer patients with malignant nodules to observation (low-risk: 2 versus 9 and 11, respectively; p = 0.028 and p = 0.011) than would the Swensen and Brock models and more patients with malignant nodules to treatment without biopsy (high-risk: 27 versus 19 and 6, respectively; p = 0.057 and p < 0.001).

CONCLUSIONS

In the setting of clinical stage I lung cancer, E-nose agrees well with histopathology. Accordingly, E-nose technology can be used in addition to imaging or as part of a "multiomics" platform.

Identifying and predicting heat stress events for grazing dairy cows using rumen temperature boluses

Heat stress events in dairy cows are associated with behavioral and physiological changes such as seeking shade, increased respiration rate and body temperature, reduced milk production, and psychological distress. Knowledge of the relationship between weather and animal responses to heat stress enables automated alerts using forecast weather, aiding early provision of shade or other mitigation practices. While numerous heat stress indices for cattle have been developed, these have limitations for cows exposed to wind and solar radiation (i.e., predominantly grazing outdoors or managed on pasture). To develop a predictive model for heat stress events in pasture-based dairy systems, rumen temperature data from smaXtec (smaXtec animal care GmbH, Graz, Austria) rumen boluses in 443 cows on 3 dairy farms in Northland, New Zealand, were used to identify heat stress events and these were matched with automated weather station data collected on or near the farm. Heat stress rate (HSR) was defined as the percentage of cows within an age-breed group having a rumen temperature greater than 3 standard deviations above an individual cow's mean and heat stress events were defined as HSR >25%. Single and multiple linear regression models, including published heat stress indices, were generally able to predict a high proportion of heat stress events (sensitivity 34%-68%), but were insufficiently discriminating, predicting also a high number of false positives (precision only 9%-27%). A machine learning algorithm, cubist, was the best performing model, predicting 79% of heat stress events with a precision of 52% for this dataset. Our proof-of-concept study demonstrates the potential of this approach, using climate data to predict and forecast heat stress events in pasture-based dairy systems. Further work should test the cubist model using independent data, refine dataset construction, investigate the value of including known animal variables such as cow age or breed, and incorporate other measures of heat stress such as respiration rate.

Ultrasound screening of paediatric non-alcoholic fatty liver disease (NAFLD): A critical literature review

INTRODUCTION

Paediatric NAFLD is an increasing global health concern, which can be effectively managed with early detection. Screening, using accurate, affordable, and accessible tests is recommended, however, there is currently no consensus on the most appropriate tests. Although ultrasound techniques are widely used, their performance against reference tests have not been fully assessed.

METHODS

A literature search of related databases for peer-reviewed original articles published from January 2010-March 2024 was conducted. Appropriate tools were used to systematise and document the search results and selected studies were quality assessed and critically appraised. Extracted data was subjected to thematic analysis and narrative synthesis.

RESULTS

Eighteen articles met the inclusion criteria. B-mode and Quantitative ultrasound techniques were compared against MR spectroscopy, MRI-PDFF and Liver biopsy.

CONCLUSION

Liver echogenicity and Steato-scores were the B-mode methods used. The former was less effective, with a maximum reported sensitivity of 70%. The latter reached up to 100% sensitivity, and >80% specificity. Ultrasound performed better with moderate-severe steatosis. There was not enough evidence to support steatosis grading, possibly due to small sample sizes and lack of established cut-off values. QUS (Quantitative Ultrasound)) methods including Continuous Attenuation Parameter (CAP), Attenuation Coefficient (AC), Ultrasound derived fat fraction (UDFF), Tissue Scatter Imaging (TSI) Hepato-Renal Index (HRI), Heterogeneity Index (HIA), Computer Assisted Ultrasound (CAUS) and Picture Archiving and Communication System (PACS-based Image analysis performed better than B-mode methods. Although QUS demonstrated excellent performance, with sensitivity and specificity of up to 100%, this will require further verification before implementation in practice.

PRACTICE IMPLICATIONS

Ultrasound techniques can effectively be used for paediatric NAFLD screening, especially in higher-risk subjects. The steato-scores method is currently recommendable for this, with excellent potential for the use of QUS, after cut-off values and validation requirements have been addressed.

Outcome-based analytical performance specifications: current status and future challenges

Analytical performance specifications (APS) based on outcomes refer to how 'good' the analytical performance of a test needs to be to do more good than harm to the patient. Analytical performance of a measurand affects its clinical performance. Without first setting clinical performance requirements, it is difficult to define how good analytically the test needs to be to meet medical needs. As testing is indirectly linked to health outcomes through clinical decisions on patient management, often simulation-based studies are used to assess the impact of analytical performance on the probability of clinical outcomes which is then translated to Model 1b APS according to the Milan consensus. This paper discusses the related key definitions, concepts and considerations that should assist in finding the most appropriate methods for deriving Model 1b APS. We review the advantages and limitations of published methods and discuss the criteria for transferability of Model 1b APS to different settings. We consider that the definition of the clinically acceptable misclassification rate is central to Model 1b APS. We provide some examples and guidance on a more systematic approach for first defining the clinical performance requirements for tests and we also highlight a few ideas to tackle the future challenges associated with providing outcome-based APS for laboratory testing.

Test accuracy of rapid diagnostic tests and reverse-transcription polymerase chain reaction against virus isolation in cell culture for assessing SARS-CoV-2 infectivity: Systematic review and meta-analysis

We aimed to assess the performance of Ag-RDT and RT-qPCR with regard to detecting infectious SARS-CoV-2 in cell cultures, as their diagnostic test accuracy (DTA) compared to virus isolation remains largely unknown. We searched three databases up to 15 December 2021 for DTA studies. The bivariate model was used to synthesise the estimates. Risk of bias was assessed using QUADAS-2/C. Twenty studies (2605 respiratory samples) using cell culture and at least one molecular test were identified. All studies were at high or unclear risk of bias in at least one domain. Three comparative DTA studies reported results on Ag-RDT and RT-qPCR against cell culture. Two studies evaluated RT-qPCR against cell culture only. Fifteen studies evaluated Ag-RDT against cell culture as reference standard in RT-qPCR-positive samples. For Ag-RDT, summary sensitivity was 93% (95% CI 78; 98%) and specificity 87% (95% CI 70; 95%). For RT-qPCR, summary sensitivity (continuity-corrected) was 98% (95% CI 95; 99%) and specificity 45% (95% CI 28; 63%). In studies relying on RT-qPCR-positive subsamples (n = 15), the summary sensitivity of Ag-RDT was 93% (95% CI 92; 93%) and specificity 63% (95% CI 63; 63%). Ag-RDT show moderately high sensitivity, detecting most but not all samples demonstrated to be infectious based on virus isolation. Although RT-qPCR exhibits high sensitivity across studies, its low specificity to indicate infectivity raises the question of its general superiority in all clinical settings. Study findings should be interpreted with caution due to the risk of bias, heterogeneity and the imperfect reference standard for infectivity.

The Japanese Esophageal Society classification for prediction of superficial esophageal squamous cell neoplasia invasion depth: Validation in a Western population

BACKGROUND

The Japan Esophageal Society proposed the JES microvessel classification to assess eligibility of early esophageal squamous cell neoplasia (ESCN) for endoscopic resection based on intrapapillary capillary loop assessment. We aimed to assess its diagnostic reproducibility and accuracy in Western ESCN patients.

METHODS

Intrapapillary capillary loops on endoscopic images of Western ESCN lesions (n = 113) collected between 2010 and 2022 were assessed by nine endoscopists, including three Japanese expert endoscopists, three Western expert endoscopists, and three residents-in-training, and graded according to the JES microvessel classification where microvessel type A corresponds with normality or low-grade intraepithelial neoplasia, and microvessel types B1, B2, and B3 correspond with high-grade intraepithelial neoplasia or invasion into the lamina propria, muscularis mucosae or superficial submucosa, and deep submucosa, respectively. Outcomes included overall accuracy in predicting ESCN invasion depth and interobserver agreement.

RESULTS

Good interobserver agreement was observed among expert endoscopists (Krippendorf's alpha 0.64, 95% CI 0.57-0.70), while agreement was moderate among residents-in-training (Krippendorf's alpha 0.58, 95% CI 0.52-0.72). Overall accuracy of the JES microvessel classification was 53% (95% CI 42-63), 52% (95% CI 41-62), and 44% (95% CI 34-55) for Japanese endoscopists, Western endoscopists, and residents-in-training, respectively. Sensitivity and specificity for vessel type A, B1, B2, and B3 across assessors were 0%-50% and 89%-100%, 55%-64% and 66%-77%, 42%-71% and 60%-76%, and 10%-24% and 92%-97%, respectively. Negative predictive value ranged between 80% and 85% for B3 vessels.

CONCLUSION

Overall accuracy of the JES microvessel classification in Western ESCN patients is low, though absence of B3 vessels as assessed by experienced endoscopists may predict superficial ESCN amenable to endoscopic resection.

TRIAL REGISTRY

www.trialregister.nl; NL8897 (6-9-2020).

Comment on "Does the 5-2-1 criteria identify patients with advanced Parkinson's disease? Real-world screening accuracy and burden of 5-2-1-positive patients in 7 countries"

The 5-2-1 criteria are intended to help general neurologists identify patients with advanced Parkinson's disease who may benefit from treatment optimisation, such as with a device-aided therapy. Although the 5-2-1 criteria claim to address an unmet need, we urge readers to cautiously interpret the results of this validation study.

Comparison of six screening methods for sarcopenia among rural community-dwelling older adults: a diagnostic accuracy study

OBJECTIVE

The objective of this analysis was to determine the diagnostic efficacy of the Ishii test, SarSA-Mod, SARC-F, SARC-Calf, SARC-F+AC, and SARC-Calf+AC for screening for sarcopenia among rural community-dwelling older adults.

METHODS

The AWGS 2019 diagnostic criteria was a diagnostic reference for sarcopenia. There were six screening tools whose accuracy was determined through the use of metrics, including specificity, sensitivity, negative and positive predictive values, and the receiver operating characteristic (ROC) curve.

RESULTS

The study included 551 participants (304 women, age 70.9 ± 4.9 years). The prevalence of sarcopenia was 44.5% in men and 39.1% in women. In males, the sensitivity/specificity of the Ishii test, SarSA-Mod, SARC-F, SARC-Calf, SARC-F+AC, and SARC-Calf+AC screening sarcopenia were 87.3%/65.7%, 98.2%/21.9%, 6.4%/98.5%, 28.2%/91.2%, 33.6%/83.9%, and 84.6%/43.8%, and in females, they were 68.1%/82.2, 100%/23.2%, 16.0%/90.3%, 35.3%/84.3%, 58.8%/61.1%, and 89.9%/42.2%, respectively. In males, the area under the curves of the Ishii test, SarSA-Mod, SARC-F, SARC-Calf, SARC-F+AC, and SARC-Calf+AC were 0.846 (95% CI 0.795-0.889), 0.800 (95% CI 0.745-0.848), 0.581 (95% CI 0.516-0.643), 0.706 (95% CI 0.645-0.762), 0.612 (95% CI 0.548-0.673), and 0.707 (95% CI 0.646-0.763), respectively, and in females, they were 0.824 (95% CI 0.776-0.865), 0.845 (95% CI 0.799-0.883), 0.581 (95% CI 0.524-0.637), 0.720 (95% CI 0.666-0.770), 0.632 (95% CI 0.575-0.686), and 0.715 (95% CI 0.661-0.765), respectively.

CONCLUSION

Our findings demonstrate that the overall accuracy of the Ishii test was best among the six screening tools for sarcopenia screening in rural community-dwelling older adults.

Association of Contrast Sensitivity With Eye Disease and Vision-Related Quality of Life

PURPOSE

To investigate contrast sensitivity (CS) as a screening tool to detect eye disease and assess its association with both eye disease and vision-related quality of life.

DESIGN

Cross-sectional study.

METHODS

Setting and population: Adults receiving care from a free clinic and a Federally Qualified Health Center in Michigan.

MAIN OUTCOME MEASURES

Screening positive for eye disease and Visual Function Questionnaire (VFQ) score.

OBSERVATION

Participants received a vision exam reviewed via telemedicine for disease, completed a demographic survey, and the 9-item VFQ. The ability of CS to predict eye disease was explored and area under the curve (AUC) is reported. Logistic and linear regression were used to investigate the continuous effect of CS on the probability of screening positive for eye disease and VFQ score, respectively, adjusting for age and visual acuity.

RESULTS

1159 included participants were, on average, 54.9 ± 14.5 years old, 62% identified as female, 34% as White, 54% as Black, 10% as Hispanic/Latino, and reported mean VFQ score of 79.7 ± 15.3. CS ranged from 0.00 to 1.95 log units (mean = 1.54 ± 0.24), 21% of eyes had glaucoma, 19% cataract, 6% DR, and 2% AMD. AUCs were 0.53 to 0.73. A 0.3 log unit decrease in better eye CS was associated with increased odds of glaucoma (odds ratio [OR] = 1.35, confidence interval [CI] = 1.09-1.67), cataract (OR = 1.35, CI = 1.05-1.72), DR (OR = 2.05, CI = 1.51-2.77), and AMD (OR = 2.08, CI = 1.10-3.91). A 0.3 log unit increase in better eye CS was associated with a 5.9 unit increase in VFQ.

CONCLUSION

While CS alone is not sufficient to identify people with eye disease, it is an important measure of visual function that can add value to comprehensive eye screening.

A scoping review of scores or grading systems for pelvic venous disorders

BACKGROUND

Pelvic venous disorders (PeVD) encompass a variety of conditions linked to chronic pelvic pain in women. However, PeVD remain underdiagnosed due to the absence of universally accepted diagnostic criteria. The complexity of PeVD classifications across specialties leads to delays in treatment. This scoping review aims to fill a gap in PeVD diagnosis and management by identifying all existing scoring or grading systems to lay the foundation for standardized clinical scoring tools for PeVD.

METHODS

This scoping review was undertaken according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping reviews. Online databases were searched up to April 2023. Studies implementing a scoring or grading system for patients with confirmed or suspected PeVD were included. Scores or grading systems were classified into four main categories based on their use in the study: screening, diagnosis, measure of disease severity, and measure of response to treatment.

RESULTS

Of the 2976 unique records identified, 82 were reviewed in full, and 20 were included in this study. The publication dates ranged from 1984 to 2023 (median, 2018; interquartile range, 2003-2022). A total of 21 scores and/or grading systems were identified. Of these 21 scores, 10 (47.6%) were clinical scores, and 10 (47.6%) were scores based on radiological findings; one study included a score that used both clinical and radiological findings. The identified scores were used in various settings. Of the 21 scores, 2 (9.52%) were used for screening in a tertiary care setting; 3 (14.3%) were used to establish the PeVD diagnosis; 8 (38.1%) were used to assess disease severity; and 8 (38.1%) were used as measures of response to treatment. Of the eight scores assessing disease severity, four (50.0%) assessed the degree of dilatation of pelvic veins and four (50%) assessed the severity of reflux. Only three of the scores were validated.

CONCLUSIONS

This scoping review identified a range of scoring and grading systems for PeVD. We note a lack of a validated scoring system, both clinical and radiological, for screening and assessment of disease severity. This is an important first step in developing validated disease-specific scoring systems for patient screening, appropriate referral, assessment of symptom severity, and assessment of the response to treatment.

External Validation of the 2023 Duke-International Society for Cardiovascular Infectious Diseases Diagnostic Criteria for Infective Endocarditis

BACKGROUND

The 2023 Duke-International Society of Cardiovascular Infectious Diseases (ISCVID) criteria for infective endocarditis (IE) were introduced to improve classification of IE for research and clinical purposes. External validation studies are required.

METHODS

We studied consecutive patients with suspected IE referred to the IE team of Amsterdam University Medical Center (from October 2016 to March 2021). An international expert panel independently reviewed case summaries and assigned a final diagnosis of "IE" or "not IE," which served as the reference standard, to which the "definite" Duke-ISCVID classifications were compared. We also evaluated accuracy when excluding cardiac surgical and pathologic data ("clinical" criteria). Finally, we compared the 2023 Duke-ISCVID with the 2000 modified Duke criteria and the 2015 and 2023 European Society of Cardiology (ESC) criteria.

RESULTS

A total of 595 consecutive patients with suspected IE were included: 399 (67%) were adjudicated as having IE; 111 (19%) had prosthetic valve IE, and 48 (8%) had a cardiac implantable electronic device IE. The 2023 Duke-ISCVID criteria were more sensitive than either the modified Duke or 2015 ESC criteria (84.2% vs 74.9% and 80%, respectively; P < .001) without significant loss of specificity. The 2023 Duke-ISCVID criteria were similarly sensitive but more specific than the 2023 ESC criteria (94% vs 82%; P < .001). The same pattern was seen for the clinical criteria (excluding surgical/pathologic results). New modifications in the 2023 Duke-ISCVID criteria related to "major microbiological" and "imaging" criteria had the most impact.

CONCLUSIONS

The 2023 Duke-ISCVID criteria represent a significant advance in the diagnostic classification of patients with suspected IE.

The efficacy of modified contrast-enhanced ultrasound Liver Imaging Reporting and Data System (CEUS LI-RADS) using Sonazoid in diagnosis of hepatocellular carcinoma: a systematic review and meta-analysis

Background

The contrast-enhanced ultrasound Liver Imaging Reporting and Data System (CEUS LI-RADS) is an algorithm for the diagnosis of hepatocellular carcinoma (HCC) in high-risk populations. Previous studies have shown the algorithm to have high specificity and moderate sensitivity. Nevertheless, it is designated for utilization solely with blood pool contrast agents. Sonazoid, a contrast agent that combines blood pools and Kupffer cells properties, has recently gained approval for marketing in an increased number of countries. Enhanced sensitivity in diagnosing HCC may be achieved through the distinctive Kupffer phase (KP) exhibited by Sonazoid. Certain academics have suggested the modified CEUS LI-RADS using Sonazoid. The main criteria of mild and late (≥60 seconds) washout in CEUS LI-RADS LR-5 were replaced by KP (>10 minutes) defects as the primary criteria. The purpose of this research was to evaluate the effectiveness of the modified CEUS LI-RADS using Sonazoid in diagnosing HCC.

Methods

Original studies on Sonazoid and CEUS LI-RADS were searched in the PubMed, Embase, Cochrane Library, and Web of Science databases until 13 July 2023, with no restrictions on language. We enrolled studies that applied Sonazoid for CEUS in patients at high risk of HCC and modified CEUS LI-RADS for the diagnosis of intrahepatic nodules. Meta-analyses, evaluations, case studies, correspondences, remarks, and summaries of conferences were excluded. Additionally, studies that fell outside the scope of this study and contained data on the same patients were also excluded. We evaluated the quality of research by employing the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool. A bivariate mixed effects model was utilized to conduct a meta-analysis, summarizing the sensitivity and specificity in the diagnosis of HCC. The investigation of potential factors contributing to study heterogeneity was conducted using meta-regression analysis.

Results

Out of the 103 studies screened, 6 studies (835 lesions) were included in the final results. Modified CEUS LR-5 exhibited a sensitivity of 0.77 [95% confidence interval (CI): 0.70-0.82; I2=71.98%; P=0.00] and a specificity of 0.88 (95% CI: 0.83-0.92; I2=0.00; P=0.47) for HCC diagnosis, with heterogeneity in sensitivity. The presence of heterogeneity in the study was found to have a significant association with factors such as the study design, the number of image reviewers, the proportion of cirrhosis, the proportion of other non-HCC malignancies (OM) cases, and the type of reference standard (P≤0.05).

Conclusions

The modified CEUS LI-RADS LR-5 categorization demonstrates a reasonable level of sensitivity 0.77, but an insufficient level of specificity 0.88 when diagnosing HCC. KP defects cannot be used as a primary feature in the diagnosis of HCC by CEUS LI-RADS, perhaps as an ancillary feature.

Association between early lactate-related variables and 6-month neurological outcome in out-of-hospital cardiac arrest patients

INTRODUCTION

The role of lactate measurement in out-of-hospital cardiac arrest (OHCA) survivors remains controversial. We assessed the association between early lactate-related variables, OHCA characteristics, and long-term neurological outcome.

METHODS

In OHCA patients who received targeted temperature management, lactate levels were measured at 0, 12, and 24 h after the return of spontaneous circulation. We calculated lactate clearance and time-weighted cumulative lactate (TWCL), which represent the area under the time-lactate curve. The area under the receiver operating characteristic curve (AUC) and the adjusted odds ratios (AORs) of lactate-related variables for predicting 6-month poor outcome (Cerebral Performance Category 3-5) were evaluated. Interactions between lactate variables and characteristics of OHCA were evaluated by a multivariable logistic model with interaction terms and subgroup analysis.

RESULTS

A total of 347 OHCA patients were included. After adjustment, higher lactate levels at the three time points were associated with a poor outcome (AOR 1.10 [95% CI, 1.03-1.18], AOR 1.15 [95% CI, 1.02-1.29], and AOR 1.36 [95% CI, 1.15-1.60], respectively), while TWCL was the only lactate kinetics variable associated with a poor outcome (AOR 1.29 [95% CI, 1.12-1.49]). We identified several interactions between lactate-related variables and OHCA characteristics. In particular, the AUC of TWCL was excellent in cases of noncardiac etiology (AUC 0.92 [95% CI, 0.86-0.96] but only moderate in cardiac etiology (AUC 0.69 [95% CI, 0.62-0.75]).

CONCLUSIONS

Early lactate levels, especially at 24 h, and TWCL were independent predictors of neurologic outcome in these patients, whereas lactate clearance was not. The prognostic ability of lactate-related variables varied depending on the OHCA characteristics.

Analysis of agreement between measures of subjective cognitive impairment and probable dementia in the National Health and Aging Trends Study

BACKGROUND

Subjective cognitive impairment (SCI) measures in population-based surveys offer potential for dementia surveillance, yet their validation against established dementia measures is lacking.

METHODS

We assessed agreement between SCI and a validated probable dementia algorithm in a random one-third sample (n = 1936) of participants in the 2012 National Health and Aging Trends Study (NHATS).

RESULTS

SCI was more prevalent than probable dementia (12.2% vs 8.4%). Agreement between measures was 90.0% and of substantial strength. Misclassification rates were higher among older and less-educated subgroups due to higher prevalence of false-positive misclassification but did not vary by sex or race and ethnicity.

DISCUSSION

SCI sensitivity (63.4%) and specificity (92.5%) against dementia were comparable with similar metrics for the NHATS probable dementia measure against the "gold-standard" Aging, Demographics, and Memory Study-based dementia criteria, implying that population-based surveys may afford cost-effective opportunities for dementia surveillance to assess risk and inform policy.

HIGHLIGHTS

The prevalence of subjective cognitive impairment (SCI) is generally higher than that of a validated measure of probable dementia, particularly within the youngest age group, females, Whites, and persons with a college or higher degree. Percent agreement between SCI and a validated measure of probable dementia was 90.0% and of substantial strength (prevalence- and bias-adjusted kappa, 0.80). Agreement rates were higher in older and less-educated subgroups, driven by the higher prevalence of false-positive disagreement, but did not vary significantly by sex or race and ethnicity. SCI's overall sensitivity and specificity were 63.4% and 92.5%, respectively, against a validated measure of probable dementia, suggesting utility as a low-cost option for dementia surveillance. Heterogeneity in agreement quality across subpopulations warrants caution in its use for subgroup analyses.

Can eHealth programs for cardiac arrhythmias be scaled-up by using the KardiaMobile algorithm?

Background

Remote monitoring devices for atrial fibrillation are known to positively contribute to the diagnostic process and therapy compliance. However, automatic algorithms within devices show varying sensitivity and specificity, so manual double-checking of electrocardiographic (ECG) recordings remains necessary.

Objective

The purpose of this study was to investigate the validity of the KardiaMobile algorithm within the Dutch telemonitoring program (HartWacht).

Methods

This retrospective study determined the diagnostic accuracy of the algorithm using assessments by a telemonitoring team as reference. The sensitivity, specificity, negative predictive value (NPV), positive predictive value (PPV), and F1 scores were determined.

Results

A total of 2298 patients (59.5% female; median age 57 ± 15 years) recorded 86,816 ECGs between April 2019 and January 2021. The algorithm showed sensitivity of 0.956, specificity 0.985, PPV 0.996, NPV 0.847, and F1 score 0.976 for the detection of sinus rhythm. A total of 29 false-positive outcomes remained uncorrected within the same patients. The algorithm showed sensitivity of 0.989, specificity 0.953, PPV 0.835, NPV 0.997, and F1 score 0.906 for detection of atrial fibrillation. A total of 2 false-negative outcomes remained uncorrected.

Conclusion

Our research showed high validity of the algorithm for the detection of both sinus rhythm and, to a lesser extent, atrial fibrillation. This finding suggests that the algorithm could function as a standalone instrument particularly for detection of sinus rhythm.

Contrast-enhanced point of care ultrasound for the evaluation of stable blunt abdominal trauma by the emergency physician: A prospective diagnostic study

Objectives

Clinical examination alone cannot reliably rule out significant traumatic abdominal injury. Computed tomography (CT) has become the primary method for evaluating blunt abdominal trauma and clinicians rely heavily on it to rule out abdominal injury. Ultrasound examination may miss significant abdominal injury particularly in stable patients. The use of a contrast agent improves ultrasound sensitivity to visceral abdominal injuries. The objective of this diagnostic study is to compare bedside contrast enhanced ultrasound (CEUS) performed by emergency physicians to CT in hemodynamically stable adults for the assessment of blunt abdominal trauma and evaluate CEUS accuracy outcomes.

Methods

Hemodynamically stable patients with blunt trauma were prospectively enrolled in the trauma bay. After initial evaluation, we included patients at risk of abdominal injury and for whom an abdominal CT was planned by the trauma leader. Ultrasonography was performed prospectively and at the bedside by the emergency physician followed by abdominal CT used as a reference standard.

Results

Thirty-three patients were enrolled in the study; among them, 52% showed positive traumatic findings in abdominal CT scans, and 42% were diagnosed with solid organ lesions. Compared to CT, a focused abdominal sonography (FOCUS) examination, looking for free fluid or perirenal hematoma, showed limited performance for traumatic findings with a sensitivity of 65% (95% confidence interval [CI]: 38%-86%), a specificity of 75% (95% CI: 48%-93%), a negative likelihood ratio (NLR) of 0.47 (95% CI: 0.23-0.95), and a positive likelihood ratio (PLR) of 2.59 (95% CI: 1.03-6.48). When combining FOCUS with CEUS, the sensitivity of the sonography increased to 94% (95% CI: 71%-100%) with a specificity of 75% (95% CI: 48%-93%). The PLR was 3.76 (95% CI: 1.6-8.87) and the NLR was 0.08 (95% CI: 0.01-0.54). In our population, abdominal sonography with contrast failed to identify a single positive abdominal CT with a grade 1 kidney injury.

Conclusions

A FOCUS examination shows limited sensitivity and specificity to detect positive abdominal CT in stable adults with abdominal trauma. With the addition of contrast and careful inspection of solid organs, abdominal sonography with contrast performed by the emergency physician improves the ability to rule out traumatic findings on abdominal CT. CEUS performed by emergency physicians may miss injuries, especially in the absence of free fluid, in cases of low-grade injuries, simultaneous injuries, or poor-quality examinations.

Citizens can help to map putative transmission sites for snail-borne diseases

INTRODUCTION

Schistosomiasis and fasciolosis are snail-borne diseases of great medical and veterinary health importance. The World Health Organization recommends complementing drug treatment with snail control and community involvement for disease elimination, but there is a general lack of snail experts and hence snail distribution data. Therefore, we adopted a citizen science approach and involved citizens in the monitoring of medically and veterinary important snail taxa.

MATERIALS AND METHODS

Snail data was collected weekly by 25 trained citizen scientists (CSs) at 76 sites around southern Lake Albert (Uganda) for 20 months. At each site, snails were searched for 30 minutes, sorted, target snail hosts identified to genus level, counted and data submitted through a smartphone application. The quality of this data was assessed by comparing it to monthly data collected by an 'expert' malacologist using the same sampling protocol. Generalised binomial logistic and linear mixed-effects models were used to analyse the variables for agreement between the CSs and expert.

FINDINGS

The binary agreement in presence/absence of Biomphalaria, Bulinus and Radix snails reported by the expert and CSs ranged between 70% and 86% (900 reports) with an average of 17% false negatives (sites wrongly defined as snail-free). The agreement for Biomphalaria and Radix increased with snail abundance, and false negatives decreased when the number of snails collected by citizens was aggregated per month. Site type significantly predicted binary agreement, which was lowest at lake sites (55%) and highest at spring sites (99%) with variations across genera. Similar temporal trends in snail abundance were recorded despite the expert reporting higher abundance. However, the relative abundance was consistent across site types. The match between the sites with highest Biomphalaria spp. abundance identified by CSs and expert was consistently high (~84.1%) and increased over time.

CONCLUSIONS AND RECOMMENDATIONS

Our results demonstrate the potential of citizen science to map putative schistosomiasis transmission sites. We therefore argue that this inclusive, powerful and cost-effective approach can be more sustainable than top-down monitoring and intervention campaigns.

On the Dependence of the Critical Success Index (CSI) on Prevalence

The critical success index (CSI) is an established metric used in meteorology to verify the accuracy of weather forecasts. It is defined as the ratio of hits to the sum of hits, false alarms, and misses. Translationally, CSI has gained popularity as a unitary outcome measure in various clinical situations where large numbers of true negatives may influence the interpretation of other, more traditional, outcome measures, such as specificity (Spec) and negative predictive value (NPV), or when unified interpretation of positive predictive value (PPV) and sensitivity (Sens) is needed. The derivation of CSI from measures including PPV has prompted questions as to whether and how CSI values may vary with disease prevalence (P), just as PPV estimates are dependent on P, and hence whether CSI values are generalizable between studies with differing prevalences. As no detailed study of the relation of CSI to prevalence has been undertaken hitherto, the dataset of a previously published test accuracy study of a cognitive screening instrument was interrogated to address this question. Three different methods were used to examine the change in CSI across a range of prevalences, using both the Bayes formula and equations directly relating CSI to Sens, PPV, P, and the test threshold (Q). These approaches showed that, as expected, CSI does vary with prevalence, but the dependence differs according to the method of calculation that is adopted. Bayesian rescaling of both Sens and PPV generates a concave curve, suggesting that CSI will be maximal at a particular prevalence, which may vary according to the particular dataset.

Performance of Echinococcosis Serology is Associated with Disease Endemicity, Hydatid Cyst Location, Eosinophilia, Imaging Finding, and Treatment

BACKGROUND

We aimed to assess echinococcosis serology performance for diagnosing cystic echinococcosis (CE) in children living in CE-endemic vs. non-endemic populations, and in different clinical settings.

METHODS

A retrospective cohort study, assessing children with ELISA test for echinococcosis, 2005-2021. Sensitivity, specificity, positive and negative predictive values (PPV and NPV) were calculated comparing CE-endemic vs. non-endemic populations, cases with/without eosinophilia, and cases with/without CE-suggestive imaging findings. Additionally, we examined the association between serology titers/levels (values) and clinical characteristics.

RESULTS

Of 273 cases, 66 (24%) were confirmed as CE. Overall, 97% of CE were in Bedouin children, and the pre-test probability was 28% vs. 9% (p < 0.001) in CE-endemic vs. non-endemic population, respectively. Sensitivity was higher in hepatic than extra-hepatic CE (74% vs. 47%). Overall specificity was 86%. PPVs were higher in CE-endemic population compared to non-endemic (66% vs. 22%), while NPVs were higher in non-endemic population (100% vs. 87%). Eosinophilia was associated with lower specificity (73% vs. 94%) and PPV (47% vs. 78%). Typical imaging was associated with higher specificity (94% vs. 82%) and PPV (91% vs. 11%), while NPVs were lower in typical imaging cases (77% vs. 98%). Higher titer levels (above median) were associated with typical imaging (76% vs. 49%), higher PPV (79% vs. 43%), albendazole treatment (100% vs. 56%), surgery (60% vs. 19%), and new imaging finding (75% vs. 0%).

CONCLUSIONS

Echinococcosis serology performance was impacted by disease endemicity, and by various clinical characteristics. These findings may assist physicians in the interpretation of echinococcosis serology results.

Performance of Lung-RADS in different target populations: a systematic review and meta-analysis

OBJECTIVES

Multiple lung cancer screening studies reported the performance of Lung CT Screening Reporting and Data System (Lung-RADS), but none systematically evaluated its performance across different populations. This systematic review and meta-analysis aimed to evaluate the performance of Lung-RADS (versions 1.0 and 1.1) for detecting lung cancer in different populations.

METHODS

We performed literature searches in PubMed, Web of Science, Cochrane Library, and Embase databases on October 21, 2022, for studies that evaluated the accuracy of Lung-RADS in lung cancer screening. A bivariate random-effects model was used to estimate pooled sensitivity and specificity, and heterogeneity was explored in stratified and meta-regression analyses.

RESULTS

A total of 31 studies with 104,224 participants were included. For version 1.0 (27 studies, 95,413 individuals), pooled sensitivity was 0.96 (95% confidence interval [CI]: 0.90-0.99) and pooled specificity was 0.90 (95% CI: 0.87-0.92). Studies in high-risk populations showed higher sensitivity (0.98 [95% CI: 0.92-0.99] vs. 0.84 [95% CI: 0.50-0.96]) and lower specificity (0.87 [95% CI: 0.85-0.88] vs. 0.95 (95% CI: 0.92-0.97]) than studies in general populations. Non-Asian studies tended toward higher sensitivity (0.97 [95% CI: 0.91-0.99] vs. 0.91 [95% CI: 0.67-0.98]) and lower specificity (0.88 [95% CI: 0.85-0.90] vs. 0.93 [95% CI: 0.88-0.96]) than Asian studies. For version 1.1 (4 studies, 8811 individuals), pooled sensitivity was 0.91 (95% CI: 0.83-0.96) and specificity was 0.81 (95% CI: 0.67-0.90).

CONCLUSION

Among studies using Lung-RADS version 1.0, considerable heterogeneity in sensitivity and specificity was noted, explained by population type (high risk vs. general), population area (Asia vs. non-Asia), and cancer prevalence.

CLINICAL RELEVANCE STATEMENT

Meta-regression of lung cancer screening studies using Lung-RADS version 1.0 showed considerable heterogeneity in sensitivity and specificity, explained by the different target populations, including high-risk versus general populations, Asian versus non-Asian populations, and populations with different lung cancer prevalence.

KEY POINTS

• High-risk population studies showed higher sensitivity and lower specificity compared with studies performed in general populations by using Lung-RADS version 1.0. • In non-Asian studies, the diagnostic performance of Lung-RADS version 1.0 tended to be better than in Asian studies. • There are limited studies on the performance of Lung-RADS version 1.1, and evidence is lacking for Asian populations.

Diagnostic accuracy of oral glucose tolerance tests, fasting plasma glucose and haemoglobin A1c for type 2 diabetes in women with polycystic ovary syndrome: A systematic review and meta-analysis

AIMS

To inform international guidelines, a systematic review and meta-analysis was conducted to assess the performance of diagnostic methods for type 2 diabetes in women with polycystic ovary syndrome (PCOS).

METHODS

An updated systematic search was conducted on five databases from 2017 until October 2023 and combined with prior searches (from inception). Meta-analyses of diagnostic accuracy tests were conducted.

RESULTS

Nine studies comprising 2628 women with PCOS were included. Against the oral glucose tolerance test, a haemoglobin A1C (HbA1c) ≥ 6.5% had a pooled sensitivity of 50.00% (95% confidence interval (CI): 35.53-64.47), specificity of 99.86% (95%CI: 99.49-99.98), and positive and negative predictive values of 92.59% (95%CI: 75.27-98.09) and 98.27% (95%CI: 97.73-98.68), respectively, with an accuracy of 98.17% (95%CI: 97.34-98.79). Fasting plasma glucose values ≥ 7.0 mmol/L had a pooled sensitivity of 58.14% (95%CI: 42.13-72.99), specificity of 92.59% (95%CI: 75.35-98.08), positive and negative predictive values of 92.59% (95%CI: 75.35-98.08) and 99.09% (95%CI: 98.71-99.36), respectively, and an accuracy of 99.00% (95%CI: 98.46-99.39) against the oral glucose tolerance test.

CONCLUSIONS

To our knowledge, this is the first systematic review assessing the performance of diagnostic methods for type 2 diabetes in women with PCOS. We demonstrate that using a cut-off for HbA1c of ≥6.5% in this population may result in misdiagnosis of half of the women with type 2 diabetes. Our results directly informed the recommendations of the 2023 International PCOS Guideline, suggesting that the oral glucose tolerance test is the optimal method for screening and diagnosing type 2 diabetes in women with PCOS and is superior to fasting plasma glucose and HbA1c.

Prevalence of Microbial Isolates Cultured from Endometrial Swab Samples Collected from United Kingdom Thoroughbred Mares from 2014 to 2020

Determining whether endometrial microbial isolates are pathogens, contaminants, or even part of the "normal" microbiome is extremely complex, particularly given the absence of "gold standard" tests for endometritis. Population-level benchmarking and temporal monitoring can provide novel insights and a wider context to improve understanding. This study aimed to (i) estimate the prevalence of endometrial isolates from swabs of Thoroughbred broodmares in Newmarket, UK between 2014 and 2020; and (ii) evaluate the effects of year, mare age, and cytology findings on isolate prevalence. Generalised linear mixed models with a logit link, both null models and models using year of sampling, mare age, or cytology findings as predictors, were fitted to estimate isolate prevalence. Over the 7-year period, data were available from 18,996 endometrial-swab samples from 6050 mares on 290 premises. The overall isolate prevalence was 35.5% (95% confidence interval (CI) 33.0-37.9), and this varied significantly between years. The most prevalent isolates were β-hemolytic Streptococcus (17.9; 95% CI: 17-19) and E. coli (10.3%; 95% CI: 9.0-11.6). Isolate prevalence increased with mare age except for E. coli isolates, and with increasing category of cytology findings except for α-hemolytic Streptococcus isolates. The results provide novel estimates of isolate prevalence and highlight knowledge gaps around potential complexities in the interpretation of findings.

Bayesian diagnostic test evaluation and true prevalence estimation of malnutrition in gastric cancer patients

BACKGROUND & AIMS

Malnutrition is prevalent among gastric cancer (GC) patients, necessitating early assessment of nutritional status to guide monitoring and interventions for improved outcomes. We aim to evaluate the accuracy and prognostic capability of three nutritional tools in GC patients, providing insights for clinical implementation.

METHODS

The present study is an analysis of data from 1308 adult GC patients recruited in a multicenter from July 2013 to July 2018. Nutritional status was assessed using Nutritional Risk Screening 2002 (NRS-2002), Patient-Generated Subjective Global Assessment (PG-SGA) and Global Leadership Initiative on Malnutrition (GLIM) criteria. Bayesian latent class model (LCM) estimated the malnutrition prevalence of GC patients, sensitivity and specificity of nutritional tools. Cox regression model analyzed the relationship between nutritional status and overall survival (OS) in GC patients.

RESULTS

Among 1308 GC patients, NRS-2002, PG-SGA, and GLIM identified 50.46%, 76.76%, and 68.81% as positive, respectively. Bayesian LCM analysis revealed that PG-SGA had the highest sensitivity (0.96) for malnutrition assessment, followed by GLIM criteria (0.78) and NRS-2002 (0.65). Malnutrition or being at risk of malnutrition were identified as independent prognostic factors for OS. Use any of these tools improved survival prediction in TNM staging system.

CONCLUSION

PG-SGA is the most reliable tool for diagnosing malnutrition in GC patients, whereas NRS-2002 is suitable for nutritional screening in busy clinical practice. Given the lower sensitivity of NRS-2002, direct utilization of GLIM for nutritional assessment may be necessary. Each nutritional tool should be associated with a specific course of action, although further research is needed.

French community grid for the evaluation of radiological artificial intelligence solutions (DRIM France Artificial Intelligence Initiative)

PURPOSE

The purpose of this study was to validate a national descriptive and analytical grid for artificial intelligence (AI) solutions in radiology.

MATERIALS AND METHODS

The RAND-UCLA Appropriateness Method was chosen by expert radiologists from the DRIM France IA group for this statement paper. The study, initiated by the radiology community, involved seven steps including literature review, template development, panel selection, pre-panel meeting survey, data extraction and analysis, second and final panel meeting, and data reporting.

RESULTS

The panel consisted of seven software vendors, three for bone fracture detection using conventional radiology and four for breast cancer detection using mammography. A consensus was reached on various aspects, including general target, main objective, certification marking, integration, expression of results, forensic aspects and cybersecurity, performance and scientific validation, description of the company and economic details, possible usage scenarios in the clinical workflow, database, specific objectives and targets of the AI tool.

CONCLUSION

The study validates a descriptive and analytical grid for radiological AI solutions consisting of ten items, using breast cancer and bone fracture as an experimental guide. This grid would assist radiologists in selecting relevant and validated AI solutions. Further developments of the grid are needed to include other organs and tasks.

A meta-analysis of the performance of ultrasound, hepatobiliary scintigraphy, CT and MRI in the diagnosis of acute cholecystitis

PURPOSE

To evaluate the recently reported relative diagnostic accuracy of US, CT, MRI, and cholescintigraphy for diagnosing acute cholecystitis.

METHODS

2 radiologists independently performed systematic electronic searches for articles published between 2000 and 2021 and applied inclusion/exclusion criteria. 2 different radiologists extracted data from the articles and scored each with a methodological quality tool. Pooled estimates of sensitivity and specificity were calculated with a bivariate linear mixed model. A second analysis made head-to-head comparisons (US vs. CT, US vs. cholescintigraphy). Factors were also analyzed for potential confounding effects on diagnostic accuracy.

RESULTS

Of 6121 initial titles, 22 were included. The prevalence of cholecystitis varied widely across studies (9.4-98%). Pooled sensitivity and specificity estimates were 69% (confidence limit [CL] 62-76%) and 79% (CL 71-86%) for US, 91% (CL 86-94%) and 63% (CL 51-74%) for cholescintigraphy, 78% (CL 69-84%) and 81% (CL 71-88%) for CT, and 91% (CL 78-97%) and 93% (CL 70-99%) for MRI. Regarding head-to-head comparisons, the sensitivity of CT (87.6%, CL 70-96%) was significantly higher than US (66.8%, CL 43-84%), while specificities (81.7% with CL 54-95% for US, 91.9% with CL 67-99% for CT) were similar. The sensitivity of cholescintigraphy (87.4%, CL 76-94%) was significantly greater than US (61.6%, CL 44-77%), while the specificity of US (82%, CL 65-92%) was significantly higher than cholescintigraphy (68%, CL 47-84%).

CONCLUSION

Recent data suggests that CT may have a higher sensitivity than US for diagnosing acute cholecystitis, with similar specificity. Cholescintigraphy remains a highly sensitive modality with lower specificity than previously reported. MRI remains under studied, but with promising results.

Epidemiology of Pathogenic Retroviruses and Domestic Cat Hepadnavirus in Community and Client-Owned Cats in Hong Kong

Understanding the local epidemiology of feline leukaemia virus (FeLV) and feline immunodeficiency virus (FIV) in Hong Kong will inform retrovirus prevention strategies. Domestic cat hepadnavirus (DCH), a novel hepatitis-B-like virus, is commonly detected among client-owned cats in Hong Kong, but community cats have not been studied. The aims of this study were to investigate the frequency and potential risk factors for (i) FeLV and FIV among community and client-owned cats and (ii) perform molecular detection of DCH among community cats in Hong Kong. Blood samples from 713 cats were obtained from client-owned (n = 415, residual diagnostic) and community cats (n = 298, at trap-neuter-return). Point-of-care (POC) testing for FeLV antigen and feline immunodeficiency virus (FIV) anti-p15 and p24 antibodies was performed. FeLV-positive samples were progressed to p27 sandwich enzyme-linked immunosorbent assay. Whole blood DNA was tested with qPCRs for FeLV U3 and gag, and nested PCRs where additional information was required. DCH qPCR was performed on a subset of community cats (n = 193). A single, regressive, FeLV infection was detected in a client-owned cat (1/415 FeLV U3 qPCR positive, 0.2%, 95% CI 0.0-1.3%). Five/415 client-owned cats tested presumably false FeLV-antigen positive (qPCR negative). No markers of FeLV infection were detected in community cats (0/298; 0%). FIV seroprevalence was much higher in community cats (46/298, 15.4%) than in client-owned cats (13/415, 3.1%) (p < 0.001). Mixed breed was a risk factor for FIV infection in client-owned cats. Neither sex nor age were associated with FIV infection. DCH DNA was detected in 34/193 (17.6%) community cats (median viral load 6.32 × 103 copies/reaction). FeLV infection is rare in Hong Kong, negatively impacting the positive predictive value of diagnostic tests. FeLV-antigen testing remains the screening test of choice, but confirmation of a positive result using FeLV qPCR is essential. FIV infection is common in community cats and the absence of a sex predisposition, seen previously in cats managed similarly, raises questions about virus-transmission dynamics in these groups. DCH infection is very common in Hong Kong, both in client-owned and community cats, highlighting the importance of understanding the pathogenic potential of this virus for cats.

Lessons learned from applying established cut-off values of questionnaires to detect somatic symptom disorders in primary care: a cross-sectional study

Introduction

Based on two diagnostic accuracy studies in high-prevalence settings, two distinctly different combinations of cut-off values have been recommended to identify persons at risk for somatic symptom disorder (SSD) with the combination of the Patient-Health Questionnaire-15 (PHQ-15) and the Somatic Symptom Disorder-B Criteria Scale (SSD-12). We investigated whether the reported sensitivity and specificity of both recommended cut-off combinations are transferable to primary care.

Methods

In a cross-sectional study, 420 unselected adult primary care patients completed PHQ-15 and SSD-12. Patients scoring ≥9 and ≥ 23 (recommended cut-off combination #1) or ≥ 8 and ≥ 13 (recommended cut-off combination #2) were considered test-positive for SSD, respectively. To assess the validity of the reported sensitivity and specificity in different low- to high-prevalence settings, we compared correspondingly expected proportions of test positives to the proportion observed in our sample.

Results

Based on combination #1, 38 participants (9%) were found to be test positive, far fewer than expected, based on the reported values for sensitivity and specificity (expected minimum frequency 30% with a true prevalence ≥1%). This can only be explained by a lower sensitivity and higher specificity in primary care. For combination #2, 98 participants (23%) were test positive, a finding consistent with a true prevalence of SSD of 15% or lower.

Discussion

Our analyzes strongly suggest that the sensitivity and specificity estimates reported for combination #1 are not applicable to unselected primary care patients and that the cut-off for the SSD (≥23) is too strict. Cut-off combination #2 seems more applicable but still needs to be tested in studies that compare screening findings by questionnaires with validated diagnostic interviews as reference standards in primary care populations.

The use of low dose CT scouts for MR safety screening: A multi-reader evaluation

INTRODUCTION

Plain film radiographs are recommended to assist in MRI safety screening of patients with unknown medical histories, especially in an emergency setting where patients might be unable to answer a safety questionnaire. This study assesses the performance of CT scout images, which have low radiation dose and are faster and easier to acquire compared to plain film radiographs, in finding and naming a range of head and body implants.

METHODS

A retrospective analysis of 40 CT Head and Neck (HN) scout images and 40 CT Chest, Abdomen and Pelvis (CAP) scout images was undertaken. A subset of these were chosen to include a range of common internal implants not identifiable externally to the patient. The images were assessed by three readers with varying levels of clinical experience in MRI who were asked to find and name any implants seen.

RESULTS

Collectively, all readers reached a sensitivity of 85 % in finding internal implants, regardless of their clinical experience or experience in reviewing CT images, and a minimum specificity of 95 %. Implants were correctly named in 74 % of the images presented.

CONCLUSION

CT scout images were able to reveal most of the implants included. However, clinical experience in reviewing the images enhances a reader's ability to identify the type of implant.

IMPLICATIONS FOR PRACTICE

In an emergency setting, imaging can be critical in the management of patients presenting with acute illnesses. In the unconscious or unresponsive patient, the use of CT scouts, where this is the only option available, could provide valuable MRI safety information prior to a scan, improving access to the MRI scan in a timely manner.

The Diagnostic Accuracy of Transvaginal Ultrasound for Detection of Ureteral Involvement in Deep Infiltrating Endometriosis: A Systematic Review and Meta-Analysis

OBJECTIVE

The aim of this study is to determine the accuracy of transvaginal ultrasound (TVU) for the diagnosis of ureteral involvement in women with deep infiltrating endometriosis (DIE).

METHODS

The meta-analysis included primary studies comparing the use of TVU for diagnosing endometriotic involvement of the ureter, using laparoscopic surgery and histological diagnosis as the reference standard. Search was performed in several databases (Scopus, Web of Science, and PubMed/MEDLINE). The studies' quality and bias risk were assessed using the Quality Assessment of Diagnostic Accuracy Study-2 (QUADAS-2). Diagnostic performance was estimated by assessing pooled sensitivity and specificity.

RESULTS

A total of 496 citations were found. Six articles were ultimately selected for this systematic review and meta-analysis after the inclusion and exclusion criteria were applied. Pooled sensitivity and specificity were 0.81 (95% CI: 0.42-0.96), 1.00 (95% CI: 0.93-1.00). The heterogeneity observed was high for both sensitivity and specificity. Overall risk of bias was low.

CONCLUSION

TVU is a valuable tool for the pre-operative identification of ureteral involvement by DIE.

GRADE guidance 37: rating imprecision in a body of evidence on test accuracy

OBJECTIVES

To provide guidance on rating imprecision in a body of evidence assessing the accuracy of a single test. This guide will clarify when Grading of Recommendations Assessment, Development and Evaluation (GRADE) users should consider rating down the certainty of evidence by one or more levels for imprecision in test accuracy.

STUDY DESIGN AND SETTING

A project group within the GRADE working group conducted iterative discussions and presentations at GRADE working group meetings to produce this guidance.

RESULTS

Before rating the certainty of evidence, GRADE users should define the target of their certainty rating. GRADE recommends setting judgment thresholds defining what they consider a very accurate, accurate, inaccurate, and very inaccurate test. These thresholds should be set after considering consequences of testing and effects on people-important outcomes. GRADE's primary criterion for judging imprecision in test accuracy evidence is considering confidence intervals (i.e., CI approach) of absolute test accuracy results (true and false, positive, and negative results in a cohort of people). Based on the CI approach, when a CI appreciably crosses the predefined judgment threshold(s), one should consider rating down certainty of evidence by one or more levels, depending on the number of thresholds crossed. When the CI does not cross judgment threshold(s), GRADE suggests considering the sample size for an adequately powered test accuracy review (optimal or review information size [optimal information size (OIS)/review information size (RIS)]) in rating imprecision. If the combined sample size of the included studies in the review is smaller than the required OIS/RIS, one should consider rating down by one or more levels for imprecision.

CONCLUSION

This paper extends previous GRADE guidance for rating imprecision in single test accuracy systematic reviews and guidelines, with a focus on the circumstances in which one should consider rating down one or more levels for imprecision.

Evaluation of the accuracy of a multi-infection screening test based on a multiplex immunoassay targeting imported diseases common in migrant populations

BACKGROUND

We aimed to evaluate the performance of a novel multiplex serological assay, able to simultaneously detect IgG of six infections, as a screening tool for imported diseases in migrants.

METHODS

Six panels of 40 (n = 240) anonymized serum samples with confirmed infections were used as positive controls to assess the multiplex assay's sensitivity. One panel of 40 sera from non-infected subjects was used to estimate the seropositivity cutoffs, and 32 non-infected sera were used as negative controls to estimate each serology's sensitivity and specificity. The multi-infection screening test was validated in a prospective cohort of 48 migrants from endemic areas. The sensitivity of the Luminex assay was calculated as the proportion of positive results over all positive samples identified by reference tests. The specificity was calculated using 32 negative samples. Uncertainty was quantified with 95 % confidence intervals using receiver operating characteristic analyses.

RESULTS

The sensitivity/specificity were 100 %/100 % for HIV (gp41 antigen), 97.5 %/100 % for Hepatitis B virus (HBV-core antigen), 100 %/100 % for Hepatitis C virus (HCV-core antigen), 92.5 %/90.6 % for strongyloidiasis [31-kDa recombinant antigen (NIE)], 97.5 %/100 % for schistosomiasis (combined serpin Schistosoma mansoni and S.haematobium antigens) and 95 %/90.6 % for Chagas disease [combined Trypanosoma cruzi kinetoplastid membrane protein-11 (KMP11) and paraflagellar rod proteins 2 (PFR2) antigens]. In the migrant cohort, antibody response to the combination of the T.cruzi antigens correctly identified 100 % individuals, whereas HBV-core antigen correctly identified 91.7 % and Strongyloides-NIE antigen 86.4 %.

CONCLUSIONS

We developed a new, robust and accurate 8-plex Luminex assay that could facilitate the implementation of screening programmes targeting migrant populations.

Validating the predictive ability of the 2MACE score for major adverse cardiovascular events in patients with atrial fibrillation: results from phase II/III of the GLORIA-AF registry

The 2MACE score was specifically developed as a risk-stratification tool in atrial fibrillation (AF) to predict cardiovascular outcomes. We evaluated the predictive ability of the 2MACE score in the GLORIA-AF registry. All eligible patients from phase II/III of the prospective global GLORIA-AF registry were included. Major adverse cardiac events (MACEs) were defined as the composite outcome of stroke, myocardial infarction and cardiovascular death. Cox proportional hazards were used to examine the relationship between the 2MACE score and study outcomes. Predictive capability of the 2MACE score was investigated using receiver-operating characteristic curves. A total of 25,696 patients were included (mean age 71 years, female 44.9%). Over 3 years, 1583 MACEs were recorded. Patients who had MACE were older, with more cardiovascular risk factors and were less likely to be managed using a rhythm-control strategy. The median 2MACE score in the MACE and non-MACE groups were 2 (IQR 1-3) and 1 (IQR 0-2), respectively (p < 0.001). The 2MACE score was positively associated with an increase in the risk of MACE, with a score of ≥ 2 providing the best combination of sensitivity (69.6%) and specificity (51.6%), HR 2.47 (95% CI, 2.21-2.77). The 2MACE score had modest predictive performance for MACE in patients with AF (AUC 0.655 (95% CI, 0.641-0.669)). Our analysis in this prospective global registry demonstrates that the 2MACE score can adequately predict the risk of MACE (defined as myocardial infarction, CV death and stroke) in patients with AF. Clinical trial registration: http://www.clinicaltrials.gov . Unique identifiers: NCT01468701, NCT01671007 and NCT01937377.

Bayesian hierarchical models incorporating study-level covariates for multivariate meta-analysis of diagnostic tests without a gold standard with application to COVID-19

When evaluating a diagnostic test, it is common that a gold standard may not be available. One example is the diagnosis of SARS-CoV-2 infection using saliva sampling or nasopharyngeal swabs. Without a gold standard, a pragmatic approach is to postulate a "reference standard," defined as positive if either test is positive, or negative if both are negative. However, this pragmatic approach may overestimate sensitivities because subjects infected with SARS-CoV-2 may still have double-negative test results even when both tests exhibit perfect specificity. To address this limitation, we propose a Bayesian hierarchical model for simultaneously estimating sensitivity, specificity, and disease prevalence in the absence of a gold standard. The proposed model allows adjusting for study-level covariates. We evaluate the model performance using an example based on a recently published meta-analysis on the diagnosis of SARS-CoV-2 infection and extensive simulations. Compared with the pragmatic reference standard approach, we demonstrate that the proposed Bayesian method provides a more accurate evaluation of prevalence, specificity, and sensitivity in a meta-analytic framework.

A comparative systematic review and meta-analysis on the diagnostic accuracy of non-invasive tests for Helicobacter pylori detection in elderly patients

Background

Helicobacter pylori (H. pylori) infection, a type I carcinogen, affects approximately 50% of the global population, correlating with various gastric pathologies. Notably, diagnostic sensitivities of non-invasive methods, such as the stool antigen test (HpSA), Serology, and Urea Breath Test (UBT), have been suggested to be less effective in older age groups. This study systematically reviews and meta-analyzes the diagnostic accuracy of these tests within the elderly population.

Methods

A comprehensive literature search was performed across multiple databases, including PubMed, Medline, and Web of Science, up to July 2023. Data were pooled and analyzed using random-effects models. Sensitivity, specificity, and Diagnostic Odds Ratios (DOR) were computed for the tests. Heterogeneity and risk of bias were assessed.

Results

Eight studies involving diverse geographic locations and totaling between 46 and 1,441 participants per study were included. The pooled sensitivity and specificity for HpSA were 72.5 and 94.7%, for Serology 83.7 and 73.3%, and for UBT 96.4 and 88.3%, respectively. DOR for UBT, HpSA, and Serology were 94.5, 47.9, and 14.2, respectively. High levels of heterogeneity were observed across the studies.

Conclusion

UBT and HpSA proved effective for diagnosing H. pylori in those over 60, while serology showed lower specificity. Despite methodological variations in available studies, these non-invasive tests offer reliable alternatives, especially for older patients who recently undergone endoscopy or without an indication for it, warranting consideration by healthcare practitioners.

Estimation of the performance of two real-time polymerase chain reaction assays for detection of Staphylococcus aureus, Streptococcus agalactiae, and Streptococcus dysgalactiae in pooled milk samples in a field study

The early detection of major mastitis pathogens is crucial for the udder health management of dairy herds. Testing of pooled milk samples, either individual test-day cow samples (TDCS) or aseptically collected pre-milk quarter samples (PMQS) may provide an easy to use and cost-effective group level screening tool. Therefore, the aim of this study was (1) to evaluate the sensitivity (Se) and specificity (Sp) of 2 commercial multiplex real-time PCR test kits applied to pooled milk samples using a Bayesian latent class analysis and (2) to estimate the probability of detection in relation to the pool size and the number of cows positively tested by bacteriological culture (BC) within a pool. Pools of 10, 20 and 50 cows were assembled from 1,912 test-day samples and 7,336 PMQS collected from a total of 2,045 cows from 2 commercial dairy farms. Two commercial quantitative real-time PCR kits were applied to detect Staphylococcus aureus, Streptococcus agalactiae, and Streptococcus dysgalactiae in the pooled samples, and a BC was applied to PMQS yielding a cumulative pool result. A pool was considered BC-positive if it contained at least one BC-positive PMQS. Pathogens were more frequently detected in the PMQS pools than in the TDCS pools. Pools of 10 cows showed the highest probability of detection irrespective of sample type or type of PCR kit compared with larger pool sizes. Estimation with a Bayesian latent class analysis resulted in a median Se in PMQS pools of 10 cows for Staph. aureus of 63.3% for PCR kit I, 78.1% for PCR kit II, and 95.5% for BC; the Sp values were 97.0%, 97.6%, and 89.1%, respectively. The estimated median Se for Strep. species for PCR kits ranged between 77.5 and 85.6% and for BC between 73.7% and 79.2%; the median Sp values ranged between 93.6 and 99.2% for PCR kits, and between 96.9% and 97.4% for BC. In addition, the probability of detection increased with an increasing number of BC-positive cows per pool. To achieve a probability of detection of 90%, the estimated number of positive cows in PMQS pools of 10 cows for kit I was 4.1 for Staph. aureus, 1.5 for Strep. agalactiae, and 1.3 for Strep. dysgalactiae; for the equivalent TDCS pools and pathogens, 6.9, 1.9, and 2.0 positive cows were required, respectively. For Kit II and PMQS pools, the number of positive cows required was 2.8 for Staph. aureus, 1.4 for Strep. agalactiae, and 1.2 for Strep. dysgalactiae; for the equivalent TDCS pools and pathogens, 5.3, 1.8, and 2.0 positive cows were required, respectively. In conclusion, the type of samples used for pooling, the pool size and the number of infected cows per pool determine the probability of detecting an infection with major mastitis pathogens within a pool by PCR testing.

Sensitivity and specificity of the ICDAS II system and bitewing radiographs for detecting occlusal caries using the Spectra™ caries detection system as the reference test in children

BACKGROUND

Most studies assessing the sensitivity and specificity of caries detection methods have been performed in vitro using the histological method as the gold standard showing inconsistent values. The aim of this study was to compare the sensitivity and specificity in detecting occlusal caries using the International Caries Detection and Assessment System (ICDAS II) with the radiographic method (RM), while using the Spectra™ Caries Detection System (SCDS) as the reference test.

METHODS

One hundred sixty children, ages 7-12 years, participated in the study. Five zones in the occlusal surfaces of 859 primary and 632 first permanent molars were examined visually using ICDAS-II, the RM using bitewing radiographs and SCDS. The descriptive statistics of sensitivity and specificity were calculated and compared.

RESULTS

For all molars combined and for primary molars only, the sensitivity of ICDAS II was higher for detecting total caries (p < 0.001), caries in enamel (p < 0.001), and caries in dentin (p = 0.016), but it was not different for detecting caries in the dentin of permanent first molars (p = 0.214), and primary second molars (p = 0.761). The specificity of RM was higher for detecting total caries, caries in enamel for all molars combined and for permanent first molars (p < 0.001). For caries in dentin, the specificity of ICDAS II was higher for all molars combined and for primary molars only (p < 0.001). For total caries in primary molars only, and caries in dentin of permanent first molars only, the specificity was not different (p = 0.156 and p = 0.181 respectively).

CONCLUSIONS

The sensitivity and specificity of ICDAS II and RM changes depending on whether the carious lesion compromises the enamel or dentin, and if the caries detection is performed in the primary molars or permanent first molars.

Approximate likelihood and pseudo-likelihood inference in meta-analysis of diagnostic accuracy studies accounting for disease prevalence and study design

Bivariate random-effects models represent a recommended approach for meta-analysis of diagnostic test accuracy, jointly modeling study-specific sensitivity and specificity. As the severity of the disease status can vary across studies, a proper analysis should account for the dependence of the accuracy measures on the disease prevalence. To this aim, trivariate generalized linear mixed-effects models have been proposed in the literature, although computational difficulties strongly limit their applicability. In addition, the attention has been mainly paid to cohort studies, where the study-specific disease prevalence can be estimated from, while information from case-control studies is often neglected. To overcome such limits, this article introduces a trivariate approximate normal model, which accounts for disease prevalence along with accuracy measures in cohort studies and sensitivity and specificity in case-control studies. The model represents an extension of the bivariate normal mixed-effects model originally developed for meta-analysis not accounting for disease prevalence, under an approximate normal within-study distribution for the logit of estimated sensitivity and specificity. The components of the approximate within-study covariance matrix are derived and the likelihood function is obtained in closed-form. The approximate likelihood approach is compared to that based on the exact within-study distribution and to its modifications following a pseudo-likelihood strategy aimed at reducing the computational effort. The comparison is based on simulation studies in a variety of scenarios, and illustrated in a meta-analysis about the accuracy of a test to diagnose fungal infection and a meta-analysis of a noninvasive test to detect colorectal cancer.

Prospective Laboratory Evaluation of the cobas Plasma Separation Card for HIV and Treponema pallidum Antibody Analysis

BACKGROUND

The cobas Plasma Separation Card (PSC; Roche Diagnostics) was developed for HIV viral load testing. This study evaluates the performance of HIV and Treponema pallidum (Tp) antibody (Ab) detection on PSCs as an alternative to dried blood spots (DBSs).

METHODS

EDTA whole blood samples were collected from HIV-positive (n = 100), HIV-negative (n = 50), Tp-positive (n = 100), and Tp-negative patients (n = 50) and spotted on DBS and PSC. Antibody detection performance was evaluated for HIV Ab using the Genscreen ULTRA HIV Ag-Ab test (Bio-Rad) and for Tp Ab using the Syphilis Total Ab test (Bio-Rad). Plasma was used as a reference specimen.

RESULTS

Receiver operating characteristic curve analysis for DBS and PSC generated areas under the curve (AUC + 95% confidence interval) of 0.985 (0.960-1.000) and 0.987 (0.973-1.000) for HIV Ab and 1.000 (1.000-1.000) and 0.996 (0.983-1.000) for Tp Ab, respectively. Receiver operating characteristic areas under the curve were not significantly different between DBS and PSC for HIV or TP Ab. At selected cutoff values rendering at least 99% sensitivity for HIV Ab detection, the specificity was 96% on DBS and 68% on PSC. For Tp Ab detection at 90% sensitivity, 100% specificity is reached on both DBS and PSC (exceeding the required 95%). However, the median quantitative HIV and Tp Ab signal of positive samples significantly decreased in PSC compared with DBS and plasma.

CONCLUSIONS

Although receiver operating characteristic analysis does not seem to indicate significant differences in performance between DBS and PSC, the significant reduction in quantitative Ab detection signal dictates card composition optimization before its use for HIV and Tp Ab detection can be advised.

Diagnostic accuracy of superb microvascular imaging for detecting intraplaque neovascularization: a systematic review and meta-analysis

Background

Atherosclerotic plaques can cause carotid artery stenosis, and "vulnerable plaques" can even lead to ischemic stroke. The objective of this study was to assess the accuracy of superb microvascular imaging (SMI) for the detection of carotid intraplaque neovascularization (IPN) in patients with atherosclerotic plaques.

Methods

We searched the Cochrane Library, Embase, Medline, and Wanfang databases until January 17, 2023. We included original studies with information on diagnostic accuracy of SMI for the evaluation of carotid IPN. The primary outcome was the accuracy of SMI for detecting carotid IPN. A meta-analysis was performed to estimate the accuracy of each parameter. We used the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) to assess the risk of bias for each included article. Meta-regression was performed to determine items that may have contributed to heterogeneity in the sensitivity or specificity of the test.

Results

This meta-analysis included 20 studies with 1,589 carotid plaques in 1,225 patients. The analysis showed a sensitivity and specificity of SMI for detecting IPN of 93% [95% confidence interval (CI): 87-96%] and 80% (95% CI: 71-87%), respectively. The risk of bias across the QUADAS-2 domains was low. Only the proportion of dyslipidemia influenced the estimates of sensitivity and specificity.

Conclusions

This review suggests that SMI has a good diagnostic performance for detecting carotid IPN. The very high sensitivity with excellent post-test probability indicated that SMI can be recommended to screen for carotid IPN among patients with carotid plaques.

Reproducibility matters: intra- and inter-sample variation of the point-of-care circulating cathodic antigen test in two Schistosoma mansoni endemic areas in Uganda

Over 240 million people are infected with schistosomiasis. Detecting Schistosoma mansoni eggs in stool using Kato-Katz thick smears (Kato-Katzs) is highly specific but lacks sensitivity. The urine-based point-of-care circulating cathodic antigen test (POC-CCA) has higher sensitivity, but issues include specificity, discrepancy between batches and interpretation of trace results. A semi-quantitative G-score and latent class analyses making no assumptions about trace readings have helped address some of these issues. However, intra-sample and inter-sample variation remains unknown for POC-CCAs. We collected 3 days of stool and urine from 349 and 621 participants, from high- and moderate-endemicity areas, respectively. We performed duplicate Kato-Katzs and one POC-CCA per sample. In the high-endemicity community, we also performed three POC-CCA technical replicates on one urine sample per participant. Latent class analysis was performed to estimate the relative contribution of intra- (test technical reproducibility) and inter-sample (day-to-day) variation on sensitivity and specificity. Within-sample variation for Kato-Katzs was higher than between-sample, with the opposite true for POC-CCAs. A POC-CCA G3 threshold most accurately assesses individual infections. However, to reach the WHO target product profile of the required 95% specificity for prevalence and monitoring and evaluation, a threshold of G4 is needed, but at the cost of reducing sensitivity. This article is part of the theme issue 'Challenges and opportunities in the fight against neglected tropical diseases: a decade from the London Declaration on NTDs'.

Challenges in the real world use of classification accuracy metrics: From recall and precision to the Matthews correlation coefficient

The accuracy of a classification is fundamental to its interpretation, use and ultimately decision making. Unfortunately, the apparent accuracy assessed can differ greatly from the true accuracy. Mis-estimation of classification accuracy metrics and associated mis-interpretations are often due to variations in prevalence and the use of an imperfect reference standard. The fundamental issues underlying the problems associated with variations in prevalence and reference standard quality are revisited here for binary classifications with particular attention focused on the use of the Matthews correlation coefficient (MCC). A key attribute claimed of the MCC is that a high value can only be attained when the classification performed well on both classes in a binary classification. However, it is shown here that the apparent magnitude of a set of popular accuracy metrics used in fields such as computer science medicine and environmental science (Recall, Precision, Specificity, Negative Predictive Value, J, F1, likelihood ratios and MCC) and one key attribute (prevalence) were all influenced greatly by variations in prevalence and use of an imperfect reference standard. Simulations using realistic values for data quality in applications such as remote sensing showed each metric varied over the range of possible prevalence and at differing levels of reference standard quality. The direction and magnitude of accuracy metric mis-estimation were a function of prevalence and the size and nature of the imperfections in the reference standard. It was evident that the apparent MCC could be substantially under- or over-estimated. Additionally, a high apparent MCC arose from an unquestionably poor classification. As with some other metrics of accuracy, the utility of the MCC may be overstated and apparent values need to be interpreted with caution. Apparent accuracy and prevalence values can be mis-leading and calls for the issues to be recognised and addressed should be heeded.

Diabetes Mellitus Impacts on the Performance of Hepatocellular Carcinoma Risk Scores in Chronic Hepatitis B Patients

BACKGROUND & AIMS

We examined whether changing clinical characteristics and presence of diabetes mellitus (DM) impact the performance of hepatocellular carcinoma (HCC) risk scores.

METHODS

Adult patients with chronic hepatitis B (CHB) on ≥6 months of entecavir/tenofovir treatment between January 2005 and March 2020 were identified using a territory-wide electronic database in Hong Kong. DM was defined by antidiabetic agents, hemoglobin A1c ≥6.5%, fasting glucose ≥7 mmol/L, and/or diagnosis codes. PAGE-B, modified PAGE-B (mPAGE-B), and aMAP scores were assessed by area under the time-dependent receiver operating characteristic curves (AUROCs) and compared with CAMD and REAL-B scores with DM as a component.

RESULTS

Of 48,706 patients, 2792, 11,563, 15,471, and 18,880 started entecavir/tenofovir treatment between 2005-2008, 2009-2012, 2013-2016, and 2017-2020, respectively; DM prevalence rose from 15.5% in 2005-2008 to 24.3% in 2017-2020. AUROCs were comparable across the 4 periods in the 5 HCC risk scores (AUROCs ranged between 0.75 and 0.81). At a median follow-up of 4.4 years, 1512 non-diabetic (4.0%) and 645 (6.2%) diabetic patients developed HCC. AUROCs of all 5 scores were lower in diabetic patients than in non-diabetic patients (AUROCs ranged between 0.67-0.71 vs 0.78-0.82; all P < .001). REAL-B score achieved an AUROC of 0.71 in diabetic and 0.82 in non-diabetic patients. Both diabetic and non-diabetic patients in the low-risk group by REAL-B score had a low HCC incidence below the threshold of cost-effective HCC surveillance, ie, 0.2% annually.

CONCLUSIONS

REAL-B score is accurate and preferred in entecavir/tenofovir-treated CHB patients because of the increasing prevalence of DM.

Diagnostic accuracy of intravoxel incoherent motion (IVIM) and dynamic contrast-enhanced (DCE) MRI to differentiate benign from malignant breast lesions: A systematic review and meta-analysis

PURPOSE

Magnetic resonance imaging (MRI) can reduce the need for unnecessary invasive diagnostic tests by nearly half. In this meta-analysis, we investigated the diagnostic accuracy of intravoxel incoherent motion modeling (IVIM) and dynamic contrast-enhanced (DCE) MRI in differentiating benign from malignant breast lesions.

METHOD

We systematically searched PubMed, EMBASE, and Scopus. We included English articles reporting diagnostic accuracy for both sequences in differentiating benign from malignant breast lesions. Articles were assessed by quality assessment of diagnostic accuracy studies-2 (QUADAS-2) questionnaire. We used a bivariate effects model for standardized mean difference (SMD) analysis and diagnostic test accuracy analysis.

RESULTS

Ten studies with 537 patients and 707 (435 malignant and 272 benign) lesions were included. The D, f, Ktrans, and Kep mean values significantly differ between benign and malignant lesions. The pooled sensitivity (95 % confidence interval) and specificity were 86.2 % (77.9 %-91.7 %) and 70.3 % (56.5 %-81.1 %) for IVIM, and 93.8 % (85.3 %-97.5 %) and 68.1 % (52.7 %-80.4 %) for DCE, respectively. Combined IVIM and DCE depicted the highest area under the curve of 0.94, with a sensitivity and specificity of 91.8 % (82.8 %-96.3 %) and 87.6 % (73.8 %-94.7 %), respectively.

CONCLUSIONS

Combined IVIM and DCE had the highest diagnostic accuracy, and multiparametric MRI may help reduce unnecessary benign breast biopsy.

Evaluation of a point-of-care calcium device in bovine plasma and serum

Hypocalcemia is a common metabolic disease in dairy cows, and it is defined as total calcium (tCa) blood concentration <2.0 mmol/L. The alternatives for the gold standard test to measure tCa in bovine blood are limited. Therefore, our objective was to compare the performance of the calcium (Ca) point-of-care compact analyzer (POC; ARKRAY Inc.) device with the gold standard method to measure bovine blood tCa concentration. Blood samples (n = 151) from dairy cows were collected within 24 h postpartum from multiparous and primiparous dairy cows for serum and plasma. Then, serum and plasma were stored at -80°C until further analyses with the gold standard method on an automatic analyzer (Cobas C501 analyzer; Roche Diagnostics) and the POC device. The tCa blood concentration was measured in the laboratory in plasma and serum samples using both methods within 10 mo of sample collection. Correlation coefficients (Spearman), coefficients of variation (CV, %), sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy, Passing and Bablok regression, and Bland-Altman agreement test were performed between the gold standard and the POC device. The range and median tCa plasma concentrations measured with the POC device were 1.1 to 2.8 mmol/L and 2.4 mmol/L, respectively. The range and median tCa serum concentrations measured with the POC device were 1.1 to 2.7 mmol/L and 2.3 mmol/L, respectively. The tCa blood concentrations range and median with the gold standard were 1.1 to 2.6 mmol/L and 2.3 mmol/L. The hypocalcemia prevalence of our study population was 11.2%. The CV were 1.89% and 0.55% for low and high tCa in plasma samples measured with the POC, respectively. The CV were 2.57% and 1.58% for low and high tCa in serum, respectively. The Spearman correlation coefficient showed a strong correlation between the gold standard and the POC device for both serum and plasma tCa concentration. The sensitivity of the POC device for both plasma (41.1%) and serum (64.7%) Ca was poor. However, the specificity of the POC device was perfect in plasma (99.2%) and serum (99.2%). The PPV in plasma and serum were 87.5% and 91.6%, respectively. Negative predicted values were 93.0% and 95.6% in plasma and serum. The mean (95% CI) difference between the gold standard and the POC device in plasma and serum were 0.35 (-0.52, 1.23) mmol/L and 0.19 (-0.53, 0.92) mmol/L, respectively. Finally, we observed a strong correlation between the POC device and the gold standard method for tCa plasma and serum. However, the clinical application of the POC device should be carefully considered because its ability to detect cows with hypocalcemia in serum or plasma samples was poor. However, the device performed better than previously analyzed POC devices and needs further improvement to be a valuable tool for the dairy industry.

Sex Differences in Poststroke Cognitive Impairment: A Multicenter Study in 2343 Patients With Acute Ischemic Stroke

BACKGROUND

Poststroke cognitive impairment (PSCI) occurs in about half of stroke survivors. Cumulative evidence indicates that functional outcomes of stroke are worse in women than men. Yet it is unknown whether the occurrence and characteristics of PSCI differ between men and women.

METHODS

Individual patient data from 9 cohorts of patients with ischemic stroke were harmonized and pooled through the Meta-VCI-Map consortium (n=2343, 38% women). We included patients with visible symptomatic infarcts on computed tomography/magnetic resonance imaging and cognitive assessment within 15 months after stroke. PSCI was defined as impairment in ≥1 cognitive domains on neuropsychological assessment. Logistic regression analyses were performed to compare men to women, adjusted for study cohort, to obtain odds ratios for PSCI and individual cognitive domains. We also explored sensitivity and specificity of cognitive screening tools for detecting PSCI, according to sex (Mini-Mental State Examination, 4 cohorts, n=1814; Montreal Cognitive Assessment, 3 cohorts, n=278).

RESULTS

PSCI was found in 51% of both women and men. Men had a lower risk of impairment of attention and executive functioning (men: odds ratio, 0.76 [95% CI, 0.61-0.96]), and language (men: odds ratio, 0.67 [95% CI, 0.45-0.85]), but a higher risk of verbal memory impairment (men: odds ratio, 1.43 [95% CI, 1.17-1.75]). The sensitivity of Mini-Mental State Examination (<25) for PSCI was higher for women (0.53) than for men (0.27; P=0.02), with a lower specificity for women (0.80) than men (0.96; P=0.01). Sensitivity and specificity of Montreal Cognitive Assessment (<26.) for PSCI was comparable between women and men (0.91 versus 0.86; P=0.62 and 0.29 versus 0.28; P=0.86, respectively).

CONCLUSIONS

Sex was not associated with PSCI occurrence but affected domains differed between men and women. The latter may explain why sensitivity of the Mini-Mental State Examination for detecting PSCI was higher in women with a lower specificity compared with men. These sex differences need to be considered when screening for and diagnosing PSCI in clinical practice.

Cervical cancer screening and treatment for PLWHIV: experiences from an innovative program in Nigeria

BACKGROUND

We evaluated cervical cancer program for women living with HIV (WLHIV) to determine program screening rate, primary case finder screening accuracy and treatment and post-treatment screening rate among screen-positive patients.

METHODS

A ten-month review of cervical cancer program data among WLHIV aged 15-49 years on HIV care across forty-one comprehensive ART sites, supported by APIN (a PEPFAR implementing partner) for cervical cancer screening and treatment in Nigeria, was conducted from October 2020 to July 2021. Initial screening was done using visual inspection with acetic acid (VIA) followed by a gynaecologist expert review through a program-designed software named AVIVA, as a confirmatory test. Associations were measured between the primary case finder screening accuracy and study covariates at p-value of 0.05.

RESULTS

About 10,289 asymptomatic women aged 15-49 years living with HIV were screened for cervical cancer by primary case finders using VIA-based screening test. About 732 (7.1%) had a positive screening test suggestive of precancerous lesions or cervical cancer. Three hundred and fifteen (43.0%) of VIA positive women had treatment using thermal ablation and less than one-third (21.6%) of those treated came back for post-treatment screening test. Primary case finder screening sensitivity, specificity, positive predictive and negative predictive accuracy using gynaecologist review as confirmatory test were 60.8%, 71.5%, 41.7% and 84.5% respectively. Overall screening accuracy was 68.8%.

CONCLUSION AND RECOMMENDATIONS

This innovative approach to cervical cancer screening among WLHIV yielded modest results in preventing program error and wastages. Wider deployment of expert-based reviews of VIA though AVIVA software might be a veritable approach to improve screening accuracy in low resource settings.