Test-retest variability of plasma biomarkers in Alzheimer's disease and its effects on clinical prediction models

INTRODUCTION

The effect of random error on the performance of blood-based biomarkers for Alzheimer's disease (AD) must be determined before clinical implementation.

METHODS

We measured test-retest variability of plasma amyloid beta (Aβ)42/Aβ40, neurofilament light (NfL), glial fibrillary acidic protein (GFAP), and phosphorylated tau (p-tau)217 and simulated effects of this variability on biomarker performance when predicting either cerebrospinal fluid (CSF) Aβ status or conversion to AD dementia in 399 non-demented participants with cognitive symptoms.

RESULTS

Clinical performance was highest when combining all biomarkers. Among single-biomarkers, p-tau217 performed best. Test-retest variability ranged from 4.1% (Aβ42/Aβ40) to 25% (GFAP). This variability reduced the performance of the biomarkers (≈ΔAUC [area under the curve] -1% to -4%) with the least effects on models with p-tau217. The percent of individuals with unstable predicted outcomes was lowest for the multi-biomarker combination (14%).

DISCUSSION

Clinical prediction models combining plasma biomarkers-particularly p-tau217-exhibit high performance and are less effected by random error. Individuals with unstable predicted outcomes ("gray zone") should be recommended for further tests.

Quantification of the Dynamic Visual Acuity Space at Real-World Luminances and Contrasts: The VA-CAL Test

Purpose

Best-corrected visual acuity (BCVA) is assessed at a single standardized luminance with maximum optotype contrast, not reflecting the constantly changing daily-life viewing conditions. For a more realistic estimation of visual performance at varying object contrasts (Cs) and ambient luminances (ALs), we developed a new VA test, VA-CAL.

Methods

Landolt-C-rings between 18% and 95% Weber contrast, were presented at 1 m distance (8 Alternative Forced Choice) on a 5.7 degree field in the middle of a frosted glass screen (66 degrees), back-lit by 3060 LEDs (generating ambient luminances between 0-10,000 cd/m²). Visual acuity (VA) was measured in 14 normally sighted participants twice for 8 conditions of ambient luminance and 6 conditions of contrast using a QUEST staircase procedure.

Results

VA improved continuously up to an ambient luminance of 3000 to 5000 cd/m² (best mean VA ± SEM: -0.47 ± 0.03 logMAR at C = 95%, AL = 3000 cd/m²), followed by a decline of VA at higher luminances with good test-retest variability. As expected, reduced contrast leads to a lower VA (worst mean VA ± SEM: -0.03 ± 0.03 logMAR at C = 18%, AL = 0 cd/m²). A 3D plot of these data shows the VA space (VAS) extending between the contrast and luminance axes, which describes the dynamics of VA continuously changing under varying everyday life conditions.

Conclusions

VA-CAL, an automated device and procedure, allows for simultaneous evaluation of VA at various contrast-luminance combinations, thus providing a more comprehensive assessment of spatial vision problems not seen with standard BCVA tests.

Translational Relevance

The new BCVA test VA-CAL incorporates a range of everyday contrast and ambient luminance conditions for a more realistic description of visual performance.

Reproducibility of graph measures at the subject level using resting-state fMRI

INTRODUCTION

Graph metrics have been proposed as potential biomarkers for diagnosis in clinical work. However, before it can be applied in a clinical setting, their reproducibility should be evaluated.

METHODS

This study systematically investigated the effect of two denoising pipelines and different whole-brain network constructions on reproducibility of subject-specific graph measures. We used the multi-session fMRI dataset from the Brain Genomics Superstruct Project consisting of 69 healthy young adults.

RESULTS

In binary networks, the test-retest variability for global measures was large at low density irrespective of the denoising strategy or the type of correlation. Weighted networks showed very low test-retest values (and thus a good reproducibility) for global graph measures irrespective of the strategy used. Comparing the test-retest values for different strategies, there were significant main effects of the type of correlation (Pearson correlation vs. partial correlation), the (partial) correlation value (absolute vs. positive vs. negative), and weight calculation (based on the raw (partial) correlation values vs. based on transformed Z-values). There was also a significant interaction effect between type of correlation and weight calculation. Similarly as for the binary networks, there was no main effect of the denoising pipeline.

CONCLUSION

Our results demonstrated that normalized global graph measures based on a weighted network using the absolute (partial) correlation as weight were reproducible. The denoising pipeline and the granularity of the whole-brain parcellation used to define the nodes were not critical for the reproducibility of normalized graph measures.

The Test-Retest Variability of the COMPlog System in Participants with Induced Non-Normal Visual Acuity

Aim:

The aim of this study was to determine and compare test-retest variability (TRV) of the computerised visual acuity (VA) COMPlog system on participants with normal vision and non-normal vision induced by bangerter foils (BFs).

Methods:

Twenty adult volunteers with VA of 0.100 logMAR or better in each eye and no eye conditions were included. Monocular VA data using the COMPlog system under five conditions—with plain Plano glasses (visually normal condition) and four pairs of Plano glasses with BF strengths of 0.6, 0.3, 0.2 and 0.1 (induced non-normal vision conditions)—were collected on two separate visits. To reduce bias, the eye tested and order of the BFs assessed were randomised. Data comparison was analysed using 2-factor ANOVA and paired t-tests and Bland Altman analysis to assess TRV.

Results:

Mean VA score from the two visits was –0.072 ± 0.1 logMAR for Plano, 0.106 ± 0.1 logMAR for BF 0.6, 0.428 ± 0.1 logMAR for BF 0.3, 0.662 ± 0.09 logMAR for BF 0.2 and 0.850 ± 0.08 logMAR for BF 0.1. As BF density increased, VA score significantly worsened (p < 0.0001). Overall mean VA score from the first and second visit was 0.410 ± 0.4 logMAR and 0.379 ± 0.4 logMAR, respectively. This improvement was significant (p < 0.009). The 95% limits of agreement of the VA scores between testing conditions had a range of ±0.120 to ±0.220 logMAR.

Conclusions:

Increase in BF strength led to a worsened VA score. However, the COMPlog TRV under the visually normal and induced non-normal vision conditions were within a similar range (±0.120 to ±0.220 logMAR). VA significantly improved on the second visit, suggesting a possible learning effect, which could have a clinical impact.

Test-retest variability for quantitative two-dimensional and Doppler measurements in the fetus

OBJECTIVES

Echocardiography is used to quantitatively characterize cardiovascular function in fetuses with cardiac abnormalities and inform decisions for fetal or perinatal interventions. It is clinically important to understand the reproducibility of these measures, particularly between testers. While studies have reported intra-observer variability and inter-observer variability, little is known about test-retest variability for these measures. We hypothesized that even in a high volume echocardiography laboratory, quantitative measurements will demonstrate higher test-retest variability compared with inter-observer variability and intra-observer variability of the same measurements.

METHODS

Prospective study of uncomplicated, singleton pregnancies to evaluate fetal measures of cardiovascular function obtained by echocardiography. One sonographer obtained predefined variables, and then, a second sonographer obtained the same variables 15 minutes after the first sonographer. Separate data acquisitions were obtained by the two sonographers to evaluate test-retest variability. Intra-observer variability and inter-observer variability were also evaluated.

RESULTS

Thirty fetuses between 17- and 36-week gestation were enrolled. Time-based variables had the best intra-observer agreement and inter-observer agreement (1.2%-7.4%), while 2D (7.5%-10%), M-mode (4.9%-10.1%), and velocity-time integral (VTI; 2.6%-13.8%) measurements had poorer agreement. For all variables, test-retest agreement was worse (3%-32.1%), particularly for measurement of myocardial performance index (MPI; 19.7%-21.1%), cardiac output estimation (27.2%-27.9%), and VTI-based indices (14.7%-32.1%).

CONCLUSIONS

In a laboratory highly experienced in quantitative fetal echocardiography, intra-observer agreement and inter-observer agreement are good for most quantitative parameters. However, test-retest agreement is fair or poor for several variables, notably the MPI, cardiac output estimation, and VTI-based indices. Understanding how these measures vary between separate acquisitions is important for clinical interpretation and decision making.

18F-JNJ-64413739, a Novel PET Ligand for the P2X7 Ion Channel: Radiation Dosimetry, Kinetic Modeling, Test-Retest Variability, and Occupancy of the P2X7 Antagonist JNJ-54175446

The P2X7 receptor (P2X7R) is an adenosine triphosphate-gated ion channel that is predominantly expressed on microglial cells in the central nervous system. We report the clinical qualification of P2X7-specific PET ligand ¹⁸F-JNJ-64413739 in healthy volunteers, including dosimetry, kinetic modeling, test-retest variability, and blocking by the P2X7 antagonist JNJ-54175446. Methods: Whole-body dosimetry was performed in 3 healthy male subjects by consecutive whole-body PET/CT scanning, estimation of the normalized cumulated activity, and calculation of the effective dose using OLINDA (v1.1). Next, 5 healthy male subjects underwent a 120-min dynamic ¹⁸F-JNJ-64413739 PET/MRI scan with arterial blood sampling to determine the appropriate kinetic model. For this purpose, 1- and 2-tissue compartment models and Logan graphic analysis (LGA) were evaluated for estimating regional volumes of distribution (V_T). PET/MRI scanning was repeated in 4 of these subjects to evaluate medium-term test-retest variability (interscan interval, 26-97 d). For the single-dose occupancy study, 8 healthy male subjects underwent baseline and postdose ¹⁸F-JNJ-64413739 PET/MRI scans 4-6 h after the administration of a single oral dose of JNJ-54175446 (dose range, 5-300 mg). P2X7 occupancies were estimated using a Lassen plot and regional baseline and postdose V_T Results: The average (mean ± SD) effective dose was 22.0 ± 1.0 μSv/MBq. The 2-tissue compartment model was the most appropriate kinetic model, with LGA showing very similar results. Regional 2-tissue compartment model V_T values were about 3 and were rather homogeneous across all brain regions, with slightly higher estimates for the thalamus, striatum, and brain stem. Between-subject V_T variability was relatively high, with cortical V_T showing an approximate 3-fold range across subjects. As for time stability, the acquisition time could be reduced to 90 min. The average regional test-retest variability values were 10.7% ± 2.2% for 2-tissue compartment model V_T and 11.9% ± 2.2% for LGA V_T P2X7 occupancy approached saturation for single doses of JNJ-54175446 higher than 50 mg, and no reference region could be identified. Conclusion: ¹⁸F-JNJ-64413739 is a suitable PET ligand for the quantification of P2X7R expression in the human brain. It can be used to provide insight into P2X7R expression in health and disease, to evaluate target engagement by P2X7 antagonists, and to guide dose selection.

Prediction of Glucose Tolerance without an Oral Glucose Tolerance Test

INTRODUCTION

Impaired glucose tolerance (IGT) is diagnosed by a standardized oral glucose tolerance test (OGTT). However, the OGTT is laborious, and when not performed, glucose tolerance cannot be determined from fasting samples retrospectively. We tested if glucose tolerance status is reasonably predictable from a combination of demographic, anthropometric, and laboratory data assessed at one time point in a fasting state.

METHODS

Given a set of 22 variables selected upon clinical feasibility such as sex, age, height, weight, waist circumference, blood pressure, fasting glucose, HbA1c, hemoglobin, mean corpuscular volume, serum potassium, fasting levels of insulin, C-peptide, triglyceride, non-esterified fatty acids (NEFA), proinsulin, prolactin, cholesterol, low-density lipoprotein, HDL, uric acid, liver transaminases, and ferritin, we used supervised machine learning to estimate glucose tolerance status in 2,337 participants of the TUEF study who were recruited before 2012. We tested the performance of 10 different machine learning classifiers on data from 929 participants in the test set who were recruited after 2012. In addition, reproducibility of IGT was analyzed in 78 participants who had 2 repeated OGTTs within 1 year.

RESULTS

The most accurate prediction of IGT was reached with the recursive partitioning method (accuracy = 0.78). For all classifiers, mean accuracy was 0.73 ± 0.04. The most important model variable was fasting glucose in all models. Using mean variable importance across all models, fasting glucose was followed by NEFA, triglycerides, HbA1c, and C-peptide. The accuracy of predicting IGT from a previous OGTT was 0.77.

CONCLUSION

Machine learning methods yield moderate accuracy in predicting glucose tolerance from a wide set of clinical and laboratory variables. A substitution of OGTT does not currently seem to be feasible. An important constraint could be the limited reproducibility of glucose tolerance status during a subsequent OGTT.

Value of posterior keratometry in the assessment of surgically induced astigmatic change in cataract surgery

PURPOSE

To investigate the value of posterior keratometry in the assessment of surgically induced astigmatic change (AC) in cataract surgery, with particular emphasis on the influence of test-retest variability.

METHODS

Seventy-seven eyes of 77 cataract patients scheduled for routine cataract surgery were enrolled. All patients received a 2.2-mm self-sealing scleral incision (n = 24), single-plane clear corneal incision (SPCCI; n = 29) or biplanar clear corneal incision (BPCCI; n = 24). Measurements of anterior and posterior corneal astigmatism were performed with a rotating Scheimpflug camera (Pentacam HR) preoperatively and postoperatively. Two repeated readings were taken preoperatively to assess the role of the test-retest effect. Astigmatic change (AC) was analysed according to the polar value method.

RESULTS

On the anterior corneal surface, SPCCIs and BPCCIs caused a statistically significant mean flattening of the incisional meridian of 0.37 and 0.27 dioptres (D), respectively. Scleral incisions on average did not cause AC, although steepening, flattening or torque beyond the test-retest effect was observed in individual cases. On the posterior surface, mean power changes in the incisional meridian were below 0.1 D for all incisions, and these changes were of the same order of magnitude as the test-retest effect.

CONCLUSION

Surgically induced AC of the posterior corneal surface after cataract surgery is of negligible clinical relevance. Moreover, it is of the same order of magnitude as the test-retest variability of the measurement device and therefore cannot (yet) be reliably assessed.

Development and Validation of a Smartphone-Based Visual Acuity Test (Peek Acuity) for Clinical Practice and Community-Based Fieldwork

IMPORTANCE

Visual acuity is the most frequently performed measure of visual function in clinical practice and most people worldwide living with visual impairment are living in low- and middle-income countries.

OBJECTIVE

To design and validate a smartphone-based visual acuity test that is not dependent on familiarity with symbols or letters commonly used in the English language.

DESIGN, SETTING, AND PARTICIPANTS

Validation study conducted from December 11, 2013, to March 4, 2014, comparing results from smartphone-based Peek Acuity to Snellen acuity (clinical normal) charts and the Early Treatment Diabetic Retinopathy Study (ETDRS) logMAR chart (reference standard). This study was nested within the 6-year follow-up of the Nakuru Eye Disease Cohort in central Kenya and included 300 adults aged 55 years and older recruited consecutively.

MAIN OUTCOMES AND MEASURES

Outcome measures were monocular logMAR visual acuity scores for each test: ETDRS chart logMAR, Snellen acuity, and Peek Acuity. Peek Acuity was compared, in terms of test-retest variability and measurement time, with the Snellen acuity and ETDRS logMAR charts in participants' homes and temporary clinic settings in rural Kenya in 2013 and 2014.

RESULTS

The 95% CI limits for test-retest variability of smartphone acuity data were ±0.033 logMAR. The mean differences between the smartphone-based test and the ETDRS chart and the smartphone-based test and Snellen acuity data were 0.07 (95% CI, 0.05-0.09) and 0.08 (95% CI, 0.06-0.10) logMAR, respectively, indicating that smartphone-based test acuities agreed well with those of the ETDRS and Snellen charts. The agreement of Peek Acuity and the ETDRS chart was greater than the Snellen chart with the ETDRS chart (95% CI, 0.05-0.10; P = .08). The local Kenyan community health care workers readily accepted the Peek Acuity smartphone test; it required minimal training and took no longer than the Snellen test (77 seconds vs 82 seconds; 95% CI, 71-84 seconds vs 73-91 seconds, respectively; P = .13).

CONCLUSIONS AND RELEVANCE

The study demonstrated that the Peek Acuity smartphone test is capable of accurate and repeatable acuity measurements consistent with published data on the test-retest variability of acuities measured using 5-letter-per-line retroilluminated logMAR charts.

Test-retest variability of multifocal electroretinography in normal volunteers and short-term variability in hydroxychloroquine users

PURPOSE

To determine measurement variability of N1P1 amplitudes and the R1/R2 ratio in normal subjects and hydroxychloroquine users without retinopathy.

DESIGN

Retrospective, observational study.

SUBJECTS

Normal subjects (n=21) and 44 patients taking hydroxychloroquine (n=44) without retinopathy.

METHODS

Multifocal electroretinography (mfERG) was performed twice in one session in the 21 normal subjects and twice within 1 year in the hydroxychloroquine users, during which time no clinical change in macular status occurred.

MAIN OUTCOME MEASURES

N1P1 amplitudes of rings R1-R5, the R1/R2 ratio, and coefficients of repeatability (COR) for these measurements.

RESULTS

Values for N1P1 amplitudes in hydroxychloroquine users were reduced compared with normal subjects by the known effect of age, but R1/R2 was not affected by age. The COR for R1-R5 ranged from 43% to 52% for normal subjects and from 43% to 59% for hydroxychloroquine users; for R1/R2 the COR was 29% in normal subjects and 45% in hydroxychloroquine users.

CONCLUSION

mfERG measurements show high test-retest variability, limiting the ability of a single mfERG test to influence a decision to stop hydroxychloroquine; corroborative evidence with a different ancillary test is recommended in a suspicious case.

Reliability of a modified logMAR distant visual acuity chart for routine clinical use

BACKGROUND/AIM

Acuity charts that follow the principle of logarithmic size progression (logMAR charts) are considered to be the gold standard for the assessment of distant vision. But it is not well accepted for routine eye examinations due to increased testing time and the complexity of scoring. This study was designed to check whether a modified logMAR chart with three optotypes would provide a reliable acuity assessment compared to standard logMAR charts for routine eye examination.

MATERIALS AND METHODS

Two versions of modified and standard logMAR charts were designed, constructed, and used to assess the visual acuity of 50 individuals drawn from a typical out-patient population. Timed test-retest variability and limit of agreement (95% confidence limit of agreement) of the modified chart are compared to the standard logMAR chart using Bland-Altman method. A comparison of the testing time was carried out using paired t-test.

RESULTS

The test-retest variability of the charts was comparable, with 95% confidence limit of the mean difference being ±0.08 for standard logMAR and ±0.10 for modified logMAR. Both the versions of modified logMAR charts produced the results which agreed well with those of the standard logMAR charts. The mean testing time required to complete the acuity measurements with the modified chart was significantly lesser compared to the standard chart (P < 0.001).

CONCLUSION

The outcomes of the current study demonstrates that the modified logMAR chart with three optotypes offers a comparable result to the standard logMAR charts for assessing distant visual acuity in routine clinical examination set up with a much lesser testing time.