Current status and challenges in establishing reference intervals based on real-world data

Reference intervals (RIs) are the cornerstone for evaluation of test results in clinical practice and are invaluable in judging patient health and making clinical decisions. Establishing RIs based on clinical laboratory data is a branch of real-world data mining research. Compared to the traditional direct method, this indirect approach is highly practical, widely applicable, and low-cost. Improving the accuracy of RIs requires not only the collection of sufficient data and the use of correct statistical methods, but also proper stratification of heterogeneous subpopulations. This includes the establishment of age-specific RIs and taking into account other characteristics of reference individuals. Although there are many studies on establishing RIs by indirect methods, it is still very difficult for laboratories to select appropriate statistical methods due to the lack of formal guidelines. This review describes the application of real-world data and an approach for establishing indirect reference intervals (iRIs). We summarize the processes for establishing iRIs using real-world data and analyze the principle and applicable scope of the indirect method model in detail. Moreover, we compare different methods for constructing growth curves to establish age-specific RIs, in hopes of providing laboratories with a reference for establishing specific iRIs and giving new insight into clinical laboratory RI research. (201 words).

Thyroid Function across the Lifespan: Do Age-Related Changes Matter?

Circulating concentrations of thyrotropin (TSH) and thyroxine (T4) are tightly regulated. Each individual has setpoints for TSH and free T4 which are genetically determined, and subject to environmental and epigenetic influence. Pituitary-thyroid axis setpoints are probably established in utero, with maturation of thyroid function continuing until late gestation. From neonatal life (characterized by a surge of TSH and T4 secretion) through childhood and adolescence (when free triiodothyronine levels are higher than in adults), thyroid function tests display complex, dynamic patterns which are sexually dimorphic. In later life, TSH increases with age in healthy older adults without an accompanying fall in free T4, indicating alteration in TSH setpoint. In view of this, and evidence that mild subclinical hypothyroidism in older people has no health impact, a strong case can be made for implementation of age-related TSH reference ranges in adults, as is routine in children.

Big data and reference intervals

Although reference intervals (RIs) play an important role in clinical diagnosis, there remain significant differences with respect to race, gender, age and geographic location. Accordingly, the Clinical Laboratory Standards Institute (CLSI) EP28-A3c has recommended that clinical laboratories establish RIs appropriate to their subject population. Unfortunately, the traditional and direct approach to establish RIs relies on the recruitment of a sufficient number of healthy individuals of various age groups, collection and testing of large numbers of specimens and accurate data interpretation. The advent of the big data era has, however, created a unique opportunity to "mine" laboratory information. Unfortunately, this indirect method lacks standardization, consensus support and CLSI guidance. In this review we provide a historical perspective, comprehensively assess data processing and statistical methods, and post-verification analysis to validate this big data approach in establishing laboratory specific RIs.

Development of nation-wide reference intervals using an indirect method and harmonized assays

OBJECTIVES

For many years, clinical laboratories have either verified or estimated reference intervals (RI) for laboratory tests. Those calculations have largely been performed by direct sampling analysis of ostensibly healthy individuals or by post-analysis biochemical screening. Recently however, indirect calculations have come to the forefront as an IFCC endorsed method by using normal and abnormal patient data.

DESIGN AND METHODS

Using a large database of patient test results from Laboratory Corporation of America, age and gender based RIs, inclusive of neonatal, pediatric, and geriatric populations, were determined using a modified indirect method of Hoffmann, and represent a diverse population distributed across the United States from a nation-wide system of laboratories and is unbiased with respect to age, gender, race or geography.

RESULTS

The tabulation of RIs using big data by an indirect method represent 72 M patient test results. The table includes 266 individual analytes consisting of approximately 2,700 age categories, including tests across multiple medical disciplines.

CONCLUSIONS

To our knowledge, this is the largest collection of RIs that were calculated by an indirect method representing clinical chemistry, endocrinology, coagulation, and hematology analytes that have been derived with very powerful "Ns" for each age bracket. This process provides more robust RIs and allows for the determination of pediatric and geriatric RIs that would otherwise be difficult to obtain using traditional direct RI determinations.

Percentile transformation and recalibration functions allow harmonization of thyroid-stimulating hormone (TSH) immunoassay results

Background The comparability of thyroid-stimulating hormone (TSH) results cannot be easily obtained using SI-traceable reference measurement procedures (RPMs) or reference materials, whilst harmonization is more feasible. The aim of this study was to identify and validate a new approach for the harmonization of TSH results. Methods Percentile normalization was applied to 125,419 TSH results, obtained from seven laboratories using three immunoassays (Access 3rd IS Thyrotropin, Beckman Coulter Diagnostics; Architect System, Abbott Diagnostics and Elecsys, Roche Diagnostics). Recalibration equations (RCAL) were derived by robust regressions using bootstrapped distribution. Two datasets, the first of 119 EQAs, the second of 610, 638 and 639 results from Access, Architect and Elecsys TSH results, respectively, were used to validate RCAL. A dataset of 142,821 TSH values was used to derive reference intervals (RIs) after applying RCAL. Results Access, Abbott and Elecsys TSH distributions were significantly different (p < 0.001). RCAL intercepts and slopes were -0.003 and 0.984 for Access, 0.032 and 1.041 for Architect, -0.031 and 1.003 for Elecsys, respectively. Validation using EQAs showed that before and after RCAL, the coefficients of variation (CVs) or among-assay results decreased from 10.72% to 8.16%. The second validation dataset was used to test RCALs. The median of between-assay differences ranged from -0.0053 to 0.1955 mIU/L of TSH. Elecsys recalibrated to Access (and vice-versa) showed non-significant difference. TSH RI after RCAL resulted in 0.37-5.11 mIU/L overall, 0.49-4.96 mIU/L for females and 0.40-4.92 mIU/L for males. A significant difference across age classes was identified. Conclusions Percentile normalization and robust regression are valuable tools for deriving RCALs and harmonizing TSH values.

Review of potentials and limitations of indirect approaches for estimating reference limits/intervals of quantitative procedures in laboratory medicine

Reference intervals (RIs) can be determined by direct and indirect procedures. Both approaches identify a reference population from which the RIs are defined. The crucial difference between direct and indirect methods is that direct methods select particular individuals after individual anamnesis and medical examination have confirmed the absence of pathological conditions. These individuals form a reference subpopulation. Indirect methods select a reference subpopulation in which the individuals are not identified. They isolate a reference population from a mixed population of patients with pathological and non-pathological conditions by statistical reasoning.

At present, the direct procedure internationally recommended is the “gold standard”. It has, however, the disadvantage of high expenses which cannot easily be afforded by most medical laboratories. Therefore, laboratories adopt RIs established by direct methods from external sources requiring a high responsibility for transference problems which are usually neglected by most laboratories. These difficulties can be overcome by indirect procedures which can easily be performed by most laboratories without causing economic problems.

The present review focuses on indirect approaches. Various procedures are presented with their benefits and limitations. Preliminary simulation studies indicate that more recently developed concepts are superior to older approaches.

Potential value and impact of data mining and machine learning in clinical diagnostics

Data mining involves the use of mathematical sciences, statistics, artificial intelligence, and machine learning to determine the relationships between variables from a large sample of data. It has previously been shown that data mining can improve the prediction and diagnostic precision of type 2 diabetes mellitus. A few studies have applied machine learning to assess hypertension and metabolic syndrome-related biomarkers, as well as refine the assessment of cardiovascular disease risk. Machine learning methods have also been applied to assess new biomarkers and survival outcomes in patients with renal diseases to predict the development of chronic kidney disease, disease progression, and renal graft survival. In the latter, random forest methods were found to be the best for the prediction of chronic kidney disease. Some studies have investigated the prognosis of nonalcoholic fatty liver disease and acute liver failure, as well as therapy response prediction in patients with viral disorders, using decision tree models. Machine learning techniques, such as Sparse High-Order Interaction Model with Rejection Option, have been used for diagnosing Alzheimer's disease. Data mining techniques have also been applied to identify the risk factors for serious mental illness, such as depression and dementia, and help to diagnose and predict the quality of life of such patients. In relation to child health, some studies have determined the best algorithms for predicting obesity and malnutrition. Machine learning has determined the important risk factors for preterm birth and low birth weight. Published studies of patients with cancer and bacterial diseases are limited and should perhaps be addressed more comprehensively in future studies. Herein, we provide an in-depth review of studies in which biochemical biomarker data were analyzed using machine learning methods to assess the risk of several common diseases, in order to summarize the potential applications of data mining methods in clinical diagnosis. Data mining techniques have now been increasingly applied to clinical diagnostics, and they have the potential to support this field.

The role of laboratory medicine in the diagnosis of the hyperthyroidism

Hyperthyroidism is a clinical condition characterized by inappropriately high synthesis and secretion of thyroid hormones by the thyroid gland. It has multiple aetiologies, manifestations and potential therapies. Graves' disease is the most common form of hyperthyroidism, due to the production of autoantibodies against thyrotropin receptor, capable of over-stimulating thyroid function. A reliable diagnosis of hyperthyroidism can be established on clinical grounds, followed by the evaluation of serum thyroid function tests (thyrotropin first and then free thyroxine, adding the measurement of free triiodothyronine in selected specific situations). The recent guidelines of both the American and European Thyroid Associations have strongly recommended the measurement of thyrotropin receptor autoantibodies for the accurate diagnosis and management of Graves' disease. If autoantibody test is negative, a radioiodine uptake should be performed. Considering the most recent laboratory improvements, binding assays can be considered the best first solution for the measurement of thyrotropin receptor autoantibodies in diagnosis and management of overt cases of Graves' disease. In fact, they have a satisfactory clinical sensitivity and specificity (97.4% and 99.2%, respectively) being performed in clinical laboratories on automated platforms together with the other thyroid function tests. In this setting, the bioassays should be reserved for fine and complex diagnoses and for particular clinical conditions where it is essential to document the transition from stimulating to blocking activity or vice versa (e.g. pregnancy and post-partum, related thyroid eye disease, Hashimoto's thyroiditis with extrathyroidal manifestations, unusual cases after LT4 therapy for hypothyroidism or after antithyroid drug treatment for Graves' disease). Undoubtedly, technological advances will help improve laboratory diagnostics of hyperthyroidism. Nevertheless, despite future progress, the dialogue between clinicians and laboratory will continue to be crucial for an adequate knowledge and interpretation of the laboratory tests and, therefore, for an accurate diagnosis and correct management of the patient.

Indirect Reference Intervals: Harnessing the Power of Stored Laboratory Data

Reference intervals are relied upon by clinicians when interpreting their patients' test results. Therefore, laboratorians directly contribute to patient care when they report accurate reference intervals. The traditional approach to establishing reference intervals is to perform a study on healthy volunteers. However, the practical aspects of the staff time and cost required to perform these studies make this approach difficult for clinical laboratories to routinely use. Indirect methods for deriving reference intervals, which utilise patient results stored in the laboratory's database, provide an alternative approach that is quick and inexpensive to perform. Additionally, because large amounts of patient data can be used, the approach can provide more detailed reference interval information when multiple partitions are required, such as with different age-groups. However, if the indirect approach is to be used to derive accurate reference intervals, several considerations need to be addressed. The laboratorian must assess whether the assay and patient population were stable over the study period, whether data 'clean-up' steps should be used prior to data analysis and, often, how the distribution of values from healthy individuals should be modelled. The assumptions and potential pitfalls of the particular indirect technique chosen for data analysis also need to be considered. A comprehensive understanding of all aspects of the indirect approach to establishing reference intervals allows the laboratorian to harness the power of the data stored in their laboratory database and ensure the reference intervals they report are accurate.

Indirect methods for reference interval determination - review and recommendations

Reference intervals are a vital part of the information supplied by clinical laboratories to support interpretation of numerical pathology results such as are produced in clinical chemistry and hematology laboratories. The traditional method for establishing reference intervals, known as the direct approach, is based on collecting samples from members of a preselected reference population, making the measurements and then determining the intervals. An alternative approach is to perform analysis of results generated as part of routine pathology testing and using appropriate statistical techniques to determine reference intervals. This is known as the indirect approach. This paper from a working group of the International Federation of Clinical Chemistry (IFCC) Committee on Reference Intervals and Decision Limits (C-RIDL) aims to summarize current thinking on indirect approaches to reference intervals. The indirect approach has some major potential advantages compared with direct methods. The processes are faster, cheaper and do not involve patient inconvenience, discomfort or the risks associated with generating new patient health information. Indirect methods also use the same preanalytical and analytical techniques used for patient management and can provide very large numbers for assessment. Limitations to the indirect methods include possible effects of diseased subpopulations on the derived interval. The IFCC C-RIDL aims to encourage the use of indirect methods to establish and verify reference intervals, to promote publication of such intervals with clear explanation of the process used and also to support the development of improved statistical techniques for these studies.

A decade of thyroidology

Significant scientific progress has been achieved in the past decade in thyroidology driven by scholarly enquiry, unmet patient needs, and investment by the pharmaceutical and diagnostics industry. In this review, nine publications have been selected for their impact in pushing the frontiers of knowledge and understanding. They include new perspectives in the diagnosis, pathophysiology, epidemiology and management of thyroid cancer, understanding of thyroid hormone physiology, and new treatments for Graves' orbitopathy.

Effects of sex, age, sampling time, and season on thyroid-stimulating hormone concentrations: A retrospective study

BACKGROUND

Measuring thyroid-stimulating hormone (TSH) is essential for diagnosing and monitoring thyroid diseases. The aim of this study was to evaluate the effect of sex, age, sampling time and season on TSH in a large Chinese population and to determine which factor had the greatest impact on TSH measurement results.

METHODS

Data were obtained from the laboratory information system from September 1, 2013 to August 31, 2016. A total of 80150 TSH measurements of outpatients were enrolled in this study. TSH was measured using a Siemens ADVIA Centaur XP automatic chemiluminescence immunoassay analyzer. Linear regression models were used to assess the association between log-transformed TSH concentrations and sex, age, sampling time and season.

RESULTS

The serum TSH concentrations in women were significantly higher than in men. In all subjects, serum TSH concentrations increased by 0.005 μIU/mL for each year of age. TSH concentrations showed circannual variation during the 3 consecutive years of data collection and decreased during the summer while increased during the winter. The serum TSH concentrations decreased from 7 a.m. to 1 p.m. while increased from 1 p.m. to 4 p.m. The same trend was observed in TSH concentrations for sampling time stratified by sex. Linear regression revealed that sampling time might be the major factor affecting serum TSH concentrations.

CONCLUSION

Sex, age, season, and sampling time significantly affected serum TSH concentrations. Age-related alteration in serum TSH concentrations was observed in this study. Sampling time was the major factor affecting serum TSH concentrations.

A multicenter study for the evaluation of the reference interval for TSH in Italy (ELAS TSH Italian Study)

Background

The aims of this study were: (1) to calculate reliable thyroid stimulating hormone (TSH) reference intervals using laboratory databases; (2) to evaluate the relationship between TSH, sex and age values in different large Italian populations.

Methods

The TSH values stored in the laboratory information system of clinical laboratories of four Italian city hospitals, including 146,801 TSH measurements (with the respective age and sex data of individuals) were taken in consideration. Assuming a log-normal distribution, to log-transformed TSH values were applied the Dixon’s iterative principle in order to exclude the outliers. At the end of this iterative process 142,821 log-transformed TSH results remained. The four clinical laboratories measured serum TSH concentrations using the same TSH immunoassay method (Access TSH 3rd IS, using UniCel DxI platform).

Results

The TSH reference interval calculated in the present study (0.362–5.280 mIU/L) is similar to that suggested by the manufacturer for the Access TSH 3rd IS assay (0.45–5.33 mIU/L). TSH values in females were significantly higher than in males (females: mean=2.06 mIU/L; standard deviation [SD]=1.26 mIU/L; n=101,243; males: mean=1.92 mIU/L; SD=1.19 mIU/L; n=41,578; p<0.0001). Moreover, a negative linear relationship was observed between TSH throughout all interval age values (from 0 to 105 years).

Conclusions

The results of the present multicenter study confirm that data mining techniques can be used to calculate clinically useful reference intervals for TSH. From a pathophysiological point of view, our results suggest that some Northern populations of Italy might still suffer some harmful effects on the thyroid gland due to mild to moderate iodine intake deficiency. Specific clinical trials are needed to confirm these results.

Biological variables influencing the estimation of reference limits

Reference limits (RLs) are required to evaluate laboratory results for medical decisions. The establishment of RL depends on the pre-analytical and the analytical conditions. Furthermore, biological characteristics of the sub-population chosen to provide the reference samples may influence the RL. The most important biological preconditions are gender, age, chronobiological influences, posture, regional and ethnic effects. The influence of these components varies and is often neglected. Therefore, a list of biological variables is collected from the literature and their influence on the estimation of RL is discussed. Biological preconditions must be specified if RL are reported as well for directly as for indirectly estimated RL. The influence of biological variables is especially important if RL established by direct methods are compared with those derived from indirect techniques. Even if these factors are not incorporated into the estimation of RL, their understanding can assist the interpretation of laboratory results of an individual.

Parathyroid hormone: Data mining for age-related reference intervals in adults

OBJECTIVE

Age-related changes in parathyroid hormone (PTH) have been previously documented in adults. However, because of the limitations of traditional approaches to establishing reference intervals, age-related reference intervals have not been defined. We sought to use a data mining approach to derive age-related PTH reference intervals.

DESIGN AND PARTICIPANTS

Results from patients undergoing PTH testing over a 4-year period were extracted from the database of a private pathology laboratory in New South Wales, Australia. Patients were included in the study if they were 18 years or older and had simultaneous determination of PTH, serum calcium, estimated glomerular filtration rate and 25-hydroxyvitamin D (25-OHD). Patients with abnormalities of serum calcium or renal function were excluded.

MEASUREMENTS

Bhattacharya analysis of log-transformed data was used to derive age-related PTH reference intervals across adulthood.

RESULTS

Results were available for 33 652 subjects. Among patients with optimal 25-OHD status, older age was associated with higher PTH concentrations. Age-related reference intervals were derived and showed a 63% increase in the upper and lower reference limits between the youngest (18-29 years of age) and the oldest (80 years of age or older) age partitions. The appropriateness of using a single reference interval for patients of all ages was evaluated against objective criteria and was found to be unsatisfactory.

CONCLUSIONS

Data mining was demonstrated to be a useful tool for establishing age-related PTH reference intervals. The technique demonstrated that increasing age is associated with higher PTH concentrations and that age-related reference intervals are important for accurate result interpretation.