A pipeline for the fully automated estimation of continuous reference intervals using real-world data

Reference intervals are essential for interpreting laboratory test results. Continuous reference intervals precisely capture physiological age-specific dynamics that occur throughout life, and thus have the potential to improve clinical decision-making. However, established approaches for estimating continuous reference intervals require samples from healthy individuals, and are therefore substantially restricted. Indirect methods operating on routine measurements enable the estimation of one-dimensional reference intervals, however, no automated approach exists that integrates the dependency on a continuous covariate like age. We propose an integrated pipeline for the fully automated estimation of continuous reference intervals expressed as a generalized additive model for location, scale and shape based on discrete model estimates using an indirect method (refineR). The results are free of subjective user-input, enable conversion of test results into z-scores and can be integrated into laboratory information systems. Comparison of our results to established and validated reference intervals from the CALIPER and PEDREF studies and manufacturers' package inserts shows good agreement of reference limits, indicating that the proposed pipeline generates high-quality results. In conclusion, the developed pipeline enables the generation of high-precision percentile charts and continuous reference intervals. It represents the first parameter-less and fully automated solution for the indirect estimation of continuous reference intervals.

Age-and sex-specific reference intervals of total cholesterol, LDL cholesterol, HDL cholesterol and non-HDL cholesterol. Comparison of two algorithms for the indirect estimation of reference intervals

Objectives

Reference intervals of total cholesterol concentrations in plasma and of their fractions low-density lipoprotein (LDL)-, high-density lipoprotein (HDL)-and non-HDL concentrations are seldom studied with respect to the relevance of age and sex. Therefore, the effect of age and sex on the reference intervals was reinvestigated with 2 indirect procedures.

Methods

As an indirect approach, the truncated minimum chi-square method was applied. All analyses were performed by computer programs available. The script published on the homepage of the German Society of Clinical Chemistry and Laboratory Medicine (DGKL) allows to derive a continuous age dependency of reference intervals together with their confidence and equivalence limits. The results of this approach were compared with those obtained by an indirect method developed more recently, the refineR algorithm.

Results

In the present study, the upper reference limits of total cholesterol varied from 5.1 to 7.8 mmol/L (197–302 mg/dL) depending on various biological variables (as age, sex, inpatients versus outpatients). These upper limits increased with age. Differences between sexes can be neglected except for the age above 80 years. The pattern of reference limits of LDL cholesterol and non-HDL cholesterol paralleled those of total cholesterol. The reference limits of HDL cholesterol were higher in women than in men but were independent of age.

Conclusions

Reference limits for the concentrations of total cholesterol and their fractions LDL-, HDL-and non-HDL concentrations should be stratified for age and sex.

Estimation of Reference Intervals from Routine Data Using the refineR Algorithm-A Practical Guide

BACKGROUND

Accurate reference intervals are essential for the interpretation of laboratory test results. Typically, they are determined by the central 95% range of test results from a predefined reference population. As these direct studies can face practical and ethical challenges, indirect methods using routine measurements offer an alternative approach.

METHODS

We provide step-by-step guidance on how to apply an indirect method in practice using refineR, the most recently published indirect method, and showcase the application by evaluating real-world data of 12 prespecified analytes. Measurements were retrieved from ARUP Laboratories' data warehouse, and were obtained from routine patient testing on cobas c502 or e602 analyzers. Test results were prefiltered and cleaned and, if necessary, physiologically partitioned prior to estimating reference intervals using refineR. Estimated reference intervals were then compared to established intervals provided by the manufacturer.

RESULTS

For most analytes, the reference intervals estimated by refineR were comparable to those provided by the manufacturer, shown by overlapping confidence intervals at both reference limits, or only the upper or lower limit. For thyroid-stimulating hormone, refineR estimated higher reference limits, while estimates for prealbumin were lower compared to the established reference interval.

CONCLUSIONS

We applied the refineR algorithm to a variety of real-world data sets resulting in reference intervals similar to intervals previously established by direct methods. We further provide practical guidance and a code example on how to apply an indirect method in a real-world scenario facilitating their access and thus their use in laboratory settings.

RIbench: A Proposed Benchmark for the Standardized Evaluation of Indirect Methods for Reference Interval Estimation

BACKGROUND

Indirect methods leverage real-world data for the estimation of reference intervals. These constitute an active field of research, and several methods have been developed recently. So far, no standardized tool for evaluation and comparison of indirect methods exists.

METHODS

We provide RIbench, a benchmarking suite for quantitative evaluation of any existing or novel indirect method. The benchmark contains simulated test sets for 10 biomarkers mimicking routine measurements of a mixed distribution of non-pathological (reference) values and pathological values. The non-pathological distributions represent 4 common distribution types: normal, skewed, heavily skewed, and skewed-and-shifted. To identify strengths and weaknesses of indirect methods, test sets have varying sample sizes and pathological distributions differ in location, extent of overlap, and fraction. For performance evaluation, we use an overall benchmark score and sub-scores derived from absolute z-score deviations between estimated and true reference limits. We illustrate the application of RIbench by evaluating and comparing the Hoffmann method and 4 modern indirect methods -TML (Truncated-Maximum-Likelihood), kosmic, TMC (Truncated-Minimum-Chi-Square), and refineR- against one another and against a nonparametric direct method (n = 120).

RESULTS

For the modern indirect methods, pathological fraction and sample size had a strong influence on the results: With a pathological fraction up to 20% and a minimum sample size of 5000, most methods achieved results comparable or superior to the direct method.

CONCLUSIONS

We present RIbench, an open-source R-package, for the systematic evaluation of existing and novel indirect methods. RIbench can serve as a tool for enhancement of indirect methods, improving the estimation of reference intervals.

How to approach clinically discordant FT4 results when changing testing platforms: real-world evidence

PURPOSE

Measurement of thyroid-stimulating hormone (TSH) and free thyroxine (FT4) is important for assessing thyroid dysfunction. After changing assay manufacturer, high FT4 versus TSH levels were reported at Ente Ospedaliero Cantonale (EOC; Bellinzona, Switzerland).

METHODS

Exploratory analysis used existing TSH and FT4 measurements taken at EOC during routine clinical practice (February 2018-April 2020) using Elecsys® TSH and Elecsys FT4 III immunoassays on cobas® 6000 and cobas 8000 analyzers (Roche Diagnostics). Reference intervals (RIs) were estimated using both direct and indirect (refineR algorithm) methods.

RESULTS

In samples with normal TSH levels, 90.9% of FT4 measurements were within the normal range provided by Roche (12-22 pmol/L). For FT4 measurements, confidence intervals (CIs) for the lower end of the RI obtained using direct and indirect methods were lower than estimated values in the method sheet; the estimated value of the upper end of the RI (UEoRI) in the method sheet was within the CI for the UEoRI using the direct method but not the indirect method. CIs for the direct and indirect methods overlapped at both ends of the RI. The most common cause of increased FT4 with normal TSH was identified in a subset of patients as use of thyroxine therapy (72.6%).

CONCLUSIONS

It is important to verify RIs for FT4 in the laboratory population when changing testing platforms; indirect methods may constitute a convenient tool for this. Applying specific RIs for selected subpopulations should be considered to avoid misinterpretations and inappropriate clinical actions.

Age- and sex-specific pediatric reference intervals for neutrophil-to-lymphocyte ratio, lymphocyte-to-monocyte ratio, and platelet-to-lymphocyte ratio

INTRODUCTION

Neutrophil-to-lymphocyte ratio (NLR), platelet-to-lymphocyte ratio (PLR), and lymphocyte-to-monocyte ratio (LMR) are emerging biomarkers for systemic inflammation and have been shown to predict morbidity and mortality for several diseases. However, lack of pediatric reference intervals (RIs) prevents their comprehensive use in patient care and medical research.

MATERIAL AND METHODS

We calculated reference intervals and corresponding confidence intervals for NLR, PLR, and LMR from birth to 18 years using a data-mining approach: We analyzed 232 746 blood counts from 60 685 patients performed during patient care and excluded patients with elevated C-reactive protein and procalcitonin. Test results were separated according to age and sex, and the distribution of physiological ratios was estimated using an indirect approach (refineR). Additionally, we investigated the ratios' diagnostic benefit for different inflammatory diseases (acute appendicitis, asthma, Bell's palsy, Henoch-Schonlein purpura, and cystic fibrosis) using the newly obtained reference intervals.

RESULTS

We estimated age- and sex-specific reference intervals from birth to adulthood for NLR, PLR, and LMR. Analyses in pediatric inflammatory diseases showed that PLR and LMR were poor markers to detect the examined inflammatory diseases, while NLR was significantly increased in patients with appendicitis and asthma.

CONCLUSION

We provide pediatric reference intervals for NLR, PLR, and LMR to improve the interpretation of these biomarkers in children.

refineR: A Novel Algorithm for Reference Interval Estimation from Real-World Data

Reference intervals are essential for the interpretation of laboratory test results in medicine. We propose a novel indirect approach to estimate reference intervals from real-world data as an alternative to direct methods, which require samples from healthy individuals. The presented refineR algorithm separates the non-pathological distribution from the pathological distribution of observed test results using an inverse approach and identifies the model that best explains the non-pathological distribution. To evaluate its performance, we simulated test results from six common laboratory analytes with a varying location and fraction of pathological test results. Estimated reference intervals were compared to the ground truth, an alternative indirect method (kosmic), and the direct method (N = 120 and N = 400 samples). Overall, refineR achieved the lowest mean percentage error of all methods (2.77%). Analyzing the amount of reference intervals within ± 1 total error deviation from the ground truth, refineR (82.5%) was inferior to the direct method with N = 400 samples (90.1%), but outperformed kosmic (70.8%) and the direct method with N = 120 (67.4%). Additionally, reference intervals estimated from pediatric data were comparable to published direct method studies. In conclusion, the refineR algorithm enables precise estimation of reference intervals from real-world data and represents a viable complement to the direct method.