Effect of sample size and the traditional parametric, nonparametric, and robust methods on the establishment of reference intervals: Evidence from real world data

Development of next-generation reference interval models to establish reference intervals based on medical data: current status, algorithms and future consideration

Evidence derived from laboratory medicine plays a pivotal role in the diagnosis, treatment monitoring, and prognosis of various diseases. Reference intervals (RIs) are indispensable tools for assessing test results. The accuracy of clinical decision-making relies directly on the appropriateness of RIs. With the increase in real-world studies and advances in computational power, there has been increased interest in establishing RIs using big data. This approach has demonstrated cost-effectiveness and applicability across diverse scenarios, thereby enhancing the overall suitability of the RI to a certain extent. However, challenges persist when tests results are influenced by age and sex. Reliance on a single RI or a grouping of RIs based on age and sex can lead to erroneous interpretation of results with significant implications for clinical decision-making. To address this issue, the development of next generation of reference interval models has arisen at an historic moment. Such models establish a curve relationship to derive continuously changing reference intervals for test results across different age and sex categories. By automatically selecting appropriate RIs based on the age and sex of patients during result interpretation, this approach facilitates clinical decision-making and enhances disease diagnosis/treatment as well as health management practices. Development of next-generation reference interval models use direct or indirect sampling techniques to select reference individuals and then employed curve fitting methods such as splines, polynomial regression and others to establish continuous models. In light of these studies, several observations can be made: Firstly, to date, limited interest has been shown in developing next-generation reference interval models, with only a few models currently available. Secondly, there are a wide range of methods and algorithms for constructing such models, and their diversity may lead to confusion. Thirdly, the process of constructing next-generation reference interval models can be complex, particularly when employing indirect sampling techniques. At present, normative documents pertaining to the development of next-generation reference interval models are lacking. In summary, this review aims to provide an overview of the current state of development of next-generation reference interval models by defining them, highlighting inherent advantages, and addressing existing challenges. It also describes the process, advanced algorithms for model building, the tools required and the diagnosis and validation of models. Additionally, a discussion on the prospects of utilizing big data for developing next-generation reference interval models is presented. The ultimate objective is to equip clinical laboratories with the theoretical framework and practical tools necessary for developing and optimizing next-generation reference interval models to establish next-generation reference intervals while enhancing the use of medical data resources to facilitate precision medicine.

Current status and challenges in establishing reference intervals based on real-world data

Reference intervals (RIs) are the cornerstone for evaluation of test results in clinical practice and are invaluable in judging patient health and making clinical decisions. Establishing RIs based on clinical laboratory data is a branch of real-world data mining research. Compared to the traditional direct method, this indirect approach is highly practical, widely applicable, and low-cost. Improving the accuracy of RIs requires not only the collection of sufficient data and the use of correct statistical methods, but also proper stratification of heterogeneous subpopulations. This includes the establishment of age-specific RIs and taking into account other characteristics of reference individuals. Although there are many studies on establishing RIs by indirect methods, it is still very difficult for laboratories to select appropriate statistical methods due to the lack of formal guidelines. This review describes the application of real-world data and an approach for establishing indirect reference intervals (iRIs). We summarize the processes for establishing iRIs using real-world data and analyze the principle and applicable scope of the indirect method model in detail. Moreover, we compare different methods for constructing growth curves to establish age-specific RIs, in hopes of providing laboratories with a reference for establishing specific iRIs and giving new insight into clinical laboratory RI research. (201 words).

Utilization of five data mining algorithms combined with simplified preprocessing to establish reference intervals of thyroid-related hormones for non-elderly adults

BACKGROUND

Despite the extensive research on data mining algorithms, there is still a lack of a standard protocol to evaluate the performance of the existing algorithms. Therefore, the study aims to provide a novel procedure that combines data mining algorithms and simplified preprocessing to establish reference intervals (RIs), with the performance of five algorithms assessed objectively as well.

METHODS

Two data sets were derived from the population undergoing a physical examination. Hoffmann, Bhattacharya, Expectation Maximum (EM), kosmic, and refineR algorithms combined with two-step data preprocessing respectively were implemented in the Test data set to establish RIs for thyroid-related hormones. Algorithm-calculated RIs were compared with the standard RIs calculated from the Reference data set in which reference individuals were selected following strict inclusion and exclusion criteria. Objective assessment of the methods is implemented by the bias ratio (BR) matrix.

RESULTS

RIs of thyroid-related hormones are established. There is a high consistency between TSH RIs established by the EM algorithm and the standard TSH RIs (BR = 0.063), although EM algorithms seems to perform poor on other hormones. RIs calculated by Hoffmann, Bhattacharya, and refineR methods for free and total triiodo-thyronine, free and total thyroxine respectively are close and match the standard RIs.

CONCLUSION

An effective approach for objectively evaluating the performance of the algorithm based on the BR matrix is established. EM algorithm combined with simplified preprocessing can handle data with significant skewness, but its performance is limited in other scenarios. The other four algorithms perform well for data with Gaussian or near-Gaussian distribution. Using the appropriate algorithm based on the data distribution characteristics is recommended.

Personalized reference intervals: from theory to practice

Using laboratory test results for diagnosis and monitoring requires a reliable reference to which the results can be compared. Currently, most reference data is derived from the population, and patients in this context are considered members of a population group rather than individuals. However, such reference data has limitations when used as the reference for an individual. A patient's test results preferably should be compared with their own, individualized reference intervals (RI), i.e. a personalized RI (prRI).The prRI is based on the homeostatic model and can be calculated using an individual's previous test results obtained in a steady-state situation and estimates of analytical (CV_A) and biological variation (BV). BV used to calculate the prRI can be obtained from the population (within-subject biological variation, CV_I) or an individual's own data (within-person biological variation, CV_P). Statistically, the prediction interval provides a useful tool to calculate the interval (i.e. prRI) for future observation based on previous measurements. With the development of information technology, the data of millions of patients is stored and processed in medical laboratories, allowing the implementation of personalized laboratory medicine. PrRI for each individual should be made available as part of the laboratory information system and should be continually updated as new test results become available.In this review, we summarize the limitations of population-based RI for the diagnosis and monitoring of disease, provide an outline of the prRI concept and different approaches to its determination, including statistical considerations for deriving prRI, and discuss aspects which must be further investigated prior to implementation of prRI in clinical practice.

An innovative approach based on real-world big data mining for calculating the sample size of the reference interval established using transformed parametric and non-parametric methods

Background

Currently, the direct method is the main approach for establishment of reference interval (RI). However, only a handful of studies have described the effects of sample size on establishment of RI and estimation of sample size. We describe a novel approach for estimation of the sample size when establishing RIs using the transformed parametric and non-parametric methods.

Methods

A total of 3,697 healthy participants were enrolled in this study. We adopted a two-layer nested loop sample size estimation method to determine the effects of sample size on RI, using thyroid-related hormone as an example. The sample size was selected as the calculation result when the width of the confidence interval (CI) of the upper and lower limit of the RI were both stably < 0.2 times the width of RI. Then, we calculated the sample size for establishing RIs via transformed parametric and non-parametric methods for thyroid-related hormones.

Results

Sample sizes for thyroid stimulating hormone (TSH), as required by parametric and non-parametric methods to establish RIs were 239 and 850, respectively. Sample sizes required by the transformed parametric method for free triiodothyronine (FT3), free thyroxine (FT4), total triiodothyronine (TT3) and total thyroxine (TT4) were all less than 120, while those required by the non-parametric method were more than 120.

Conclusion

We describe a novel approach for estimating sample sizes for establishment of RI. A corresponding open-source code has been developed and is available for applications. The established method is suitable for most analytes, with evidence based on thyroid-related hormones indicating that different sample sizes are required to establish RIs using different methods for analytes with different variations.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01751-1.

Big data and reference intervals

Although reference intervals (RIs) play an important role in clinical diagnosis, there remain significant differences with respect to race, gender, age and geographic location. Accordingly, the Clinical Laboratory Standards Institute (CLSI) EP28-A3c has recommended that clinical laboratories establish RIs appropriate to their subject population. Unfortunately, the traditional and direct approach to establish RIs relies on the recruitment of a sufficient number of healthy individuals of various age groups, collection and testing of large numbers of specimens and accurate data interpretation. The advent of the big data era has, however, created a unique opportunity to "mine" laboratory information. Unfortunately, this indirect method lacks standardization, consensus support and CLSI guidance. In this review we provide a historical perspective, comprehensively assess data processing and statistical methods, and post-verification analysis to validate this big data approach in establishing laboratory specific RIs.