Meta-analytic estimation of measurement variability and assessment of its impact on decision-making: the case of perioperative haemoglobin concentration monitoring

Current Status of Measurement Accuracy for Total Hemoglobin Concentration in the Clinical Context

OBJECTIVE

The main objective of this investigation is to provide data about the accuracy of total hemoglobin concentration measurements with respect to clinical settings, and to devices within the categories of point-of-care and reference systems. In particular, tolerance of hemoglobin concentrations below 9 g/dL that have become common in clinical practice today determines the need to demonstrate the limits of measurement accuracy in patient care.

METHODS

Samples extracted from six units of heparinized human blood with total hemoglobin concentrations ranging from 3 to 18 g/dL were assigned to the test devices in a random order. The pool of test devices comprised blood gas analyzers, an automatic hematology analyzer, a laboratory reference method, and the point-of-care system HemoCue. To reduce the pre-analytic error, each sample was measured three times. Due to the characteristics of the tested devices and methods, we selected the mean values of the data from all these devices, measured at the corresponding total hemoglobin concentrations, as the reference.

MAIN RESULTS

The measurement results of the test devices overlap within strict limits (R² = 0.999). Only the detailed analysis provides information about minor but systematic deviations. In the group of clinically relevant devices, which are involved in patient blood management decisions, the relative differences were within the limit of +/- 5 % for values down to 3 g/dL.

CONCLUSIONS

A clinically relevant change of +/- 0.5 g/dL of total hemoglobin concentration can be detected with all selected devices and methods. Compliance with more stringent definitions-these are the relative differences of 5 % in relation to the corresponding reference values and the clinically adapted thresholds in the format of a tolerance level analysis-was achieved by the clinical devices assessed here.

Diagnostic accuracy of point-of-care devices for detection of anemia in community settings in India

Background

Accurate diagnosis of anemia by community workers using a point-of-care device is a challenge. The objective of the study was to establish the diagnostic accuracy of point-of-care devices for detecting anemia in community settings.

Methods

It was diagnostic accuracy study with cross-sectional design on adult patients attending the outpatient department of rural/ urban health centres of Medical colleges from India. The index tests were HemoCue, TrueHb, Massimo’s device and spectroscopic device, compared against autoanalyzer (gold standard). Accuracy was expressed by sensitivity, specificity, likelihood ratios, predictive values, area under the curve (AUC) and levels of agreement. For the diagnostic accuracy component, 1407 participants were recruited with a minimum of 600 for each device. An additional 200 participants were considered to elucidate the performance of devices in different weather conditions.

Results

HemoCue and TrueHb performed better than Massimo and spectroscopic devices. Detection of anemia by technicians was similar between TrueHb and HemoCue (AUC 0.92 v/s 0.90, p > 0.05). Community workers performed better with Hemocue for detecting anemia compared to TrueHb (AUC 0.92 v/s 0.90, p < 0.05). For detection of severe anemia, accuracy of TrueHb was significantly better with technicians (AUC 0.91 v/s 0.70; p < 0.05) and community workers (AUC 0.91 v/s 0.73; p < 0.05).

HemoCue showed a bias or mean difference (95%CI) of 0.47 g/dl (0.42, 0.52) for all values, and 0.92 g/dl (0.82, 1.03) for severe anemia. For TrueHb, it was − 0.28 g/dl (− 0.37, − 0.20) for all readings, and 0.06 g/dl (− 0.52, 0.63) for severe anemia. TrueHb appeared to be more consistent across different weather conditions, although it overestimated Hb in extreme cold weather conditions.

Conclusion

For detection of anemia, True Hb and HemoCue were comparable. For severe anemia, True Hb seemed to be a better and feasible point-of-care device for detecting anemia in the community settings.

Predictive probability of success using surrogate endpoints

The predictive probability of success of a future clinical trial is a key quantitative tool for decision-making in drug development. It is derived from prior knowledge and available evidence, and the latter typically comes from the accumulated data on the clinical endpoint of interest in previous clinical trials. However, a surrogate endpoint could be used as primary endpoint in early development and, usually, no or limited data are collected on the clinical endpoint of interest. We propose a general, reliable, and broadly applicable methodology to predict the success of a future trial from surrogate endpoints, in a way that makes the best use of all the available evidence. The predictions are based on an informative prior, called surrogate prior, derived from the results of past trials on one or several surrogate endpoints. If available, in a Bayesian framework, this prior could be combined with data from past trials on the clinical endpoint of interest. Two methods are proposed to address a potential discordance between the surrogate prior and the data on the clinical endpoint. We investigate the patterns of behavior of the predictions in a comprehensive simulation study, and we present an application to the development of a drug in Multiple Sclerosis. The proposed methodology is expected to support decision-making in many different situations, since the use of predictive markers is important to accelerate drug developments and to select promising drug candidates, better and earlier.

Sample size re-estimation incorporating prior information on a nuisance parameter

Prior information is often incorporated informally when planning a clinical trial. Here, we present an approach on how to incorporate prior information, such as data from historical clinical trials, into the nuisance parameter-based sample size re-estimation in a design with an internal pilot study. We focus on trials with continuous endpoints in which the outcome variance is the nuisance parameter. For planning and analyzing the trial, frequentist methods are considered. Moreover, the external information on the variance is summarized by the Bayesian meta-analytic-predictive approach. To incorporate external information into the sample size re-estimation, we propose to update the meta-analytic-predictive prior based on the results of the internal pilot study and to re-estimate the sample size using an estimator from the posterior. By means of a simulation study, we compare the operating characteristics such as power and sample size distribution of the proposed procedure with the traditional sample size re-estimation approach that uses the pooled variance estimator. The simulation study shows that, if no prior-data conflict is present, incorporating external information into the sample size re-estimation improves the operating characteristics compared to the traditional approach. In the case of a prior-data conflict, that is, when the variance of the ongoing clinical trial is unequal to the prior location, the performance of the traditional sample size re-estimation procedure is in general superior, even when the prior information is robustified. When considering to include prior information in sample size re-estimation, the potential gains should be balanced against the risks.

Meta-analysis of variance: an illustration comparing the effects of two dietary interventions on variability in weight

New meta-analysis methods from evolutionary biology allow us to ask how treatments affect variability, as opposed to just the average. Using these methods we demonstrate that low carbohydrate ad libitum diets may have more variable outcomes than calorie restricted diets.

Meta-analysis, which drives evidence-based practice, typically focuses on the average response of subjects to a treatment. For instance in nutritional research the difference in average weight of participants on different diets is typically used to draw conclusions about the relative efficacy of interventions. As a result of their focus on the mean, meta-analyses largely overlook the effects of treatments on inter-subject variability. Recent tools from the study of biological evolution, where inter-individual variability is one of the key ingredients for evolution by natural selection, now allow us to study inter-subject variability using established meta-analytic models. Here we use meta-analysis to study how low carbohydrate (LC) ad libitum diets and calorie restricted diets affect variance in mass. We find that LC ad libitum diets may have a more variable outcome than diets that prescribe a reduced calorie intake. Our results suggest that whilst LC diets are effective in a large proportion of the population, for a subset of individuals, calorie restricted diets may be more effective. There is evidence that LC ad libitum diets rely on appetite suppression to drive weight loss. Extending this hypothesis, we suggest that between-individual variability in protein appetite may drive the trends that we report. A priori identification of an individual’s target intake for protein may help define the most effective dietary intervention to prescribe for weight loss.