Measurement error is often neglected in medical literature: a systematic review

Cross-institution natural language processing for reliable clinical association studies: a methodological exploration

OBJECTIVES

Natural language processing (NLP) of clinical notes in electronic medical records is increasingly used to extract otherwise sparsely available patient characteristics, to assess their association with relevant health outcomes. Manual data curation is resource intensive and NLP methods make these studies more feasible. However, the methodology of using NLP methods reliably in clinical research is understudied. The objective of this study is to investigate how NLP models could be used to extract study variables (specifically exposures) to reliably conduct exposure-outcome association studies.

STUDY DESIGN AND SETTING

In a convenience sample of patients admitted to the intensive care unit of a US academic health system, multiple association studies are conducted, comparing the association estimates based on NLP-extracted vs. manually extracted exposure variables. The association studies varied in NLP model architecture (Bidirectional Encoder Decoder from Transformers, Long Short-Term Memory), training paradigm (training a new model, fine-tuning an existing external model), extracted exposures (employment status, living status, and substance use), health outcomes (having a do-not-resuscitate/intubate code, length of stay, and in-hospital mortality), missing data handling (multiple imputation vs. complete case analysis), and the application of measurement error correction (via regression calibration).

RESULTS

The study was conducted on 1,174 participants (median [interquartile range] age, 61 [50, 73] years; 60.6% male). Additionally, up to 500 discharge reports of participants from the same health system and 2,528 reports of participants from an external health system were used to train the NLP models. Substantial differences were found between the associations based on NLP-extracted and manually extracted exposures under all settings. The error in association was only weakly correlated with the overall F1 score of the NLP models.

CONCLUSION

Associations estimated using NLP-extracted exposures should be interpreted with caution. Further research is needed to set conditions for reliable use of NLP in medical association studies.

Causal Assessment of Income Inequality on Self-Rated Health and All-Cause Mortality: A Systematic Review and Meta-Analysis

Policy Points Income is thought to impact a broad range of health outcomes. However, whether income inequality (how unequal the distribution of income is in a population) has an additional impact on health is extensively debated. Studies that use multilevel data, which have recently increased in popularity, are necessary to separate the contextual effects of income inequality on health from the effects of individual income on health. Our systematic review found only small associations between income inequality and poor self-rated health and all-cause mortality. The available evidence does not suggest causality, although it remains methodologically flawed and limited, with very few studies using natural experimental approaches or examining income inequality at the national level.

CONTEXT

Whether income inequality has a direct effect on health or is only associated because of the effect of individual income has long been debated. We aimed to understand the association between income inequality and self-rated health (SRH) and all-cause mortality (mortality) and assess if these relationships are likely to be causal.

METHODS

We searched Medline, ISI Web of Science, Embase, and EconLit (PROSPERO: CRD42021252791) for studies considering income inequality and SRH or mortality using multilevel data and adjusting for individual-level socioeconomic position. We calculated pooled odds ratios (ORs) for poor SRH and relative risk ratios (RRs) for mortality from random-effects meta-analyses. We critically appraised included studies using the Risk of Bias in Nonrandomized Studies - of Interventions tool. We assessed certainty of evidence using the Grading of Recommendations Assessment, Development and Evaluation framework and causality using Bradford Hill (BH) viewpoints.

FINDINGS

The primary meta-analyses included 2,916,576 participants in 38 cross-sectional studies assessing SRH and 10,727,470 participants in 14 cohort studies of mortality. Per 0.05-unit increase in the Gini coefficient, a measure of income inequality, the ORs and RRs (95% confidence intervals) for SRH and mortality were 1.06 (1.03-1.08) and 1.02 (1.00-1.04), respectively. A total of 63.2% of SRH and 50.0% of mortality studies were at serious risk of bias (RoB), resulting in very low and low certainty ratings, respectively. For SRH and mortality, we did not identify relevant evidence to assess the specificity or, for SRH only, the experiment BH viewpoints; evidence for strength of association and dose-response gradient was inconclusive because of the high RoB; we found evidence in support of temporality and plausibility.

CONCLUSIONS

Increased income inequality is only marginally associated with SRH and mortality, but the current evidence base is too methodologically limited to support a causal relationship. To address the gaps we identified, future research should focus on income inequality measured at the national level and addressing confounding with natural experiment approaches.

Improving accuracy in diagnosing aortic stenosis severity: An in-depth analysis of echocardiographic measurement error through literature review and simulation study

AIMS

The present guidelines advise replacing the aortic valve for individuals with severe aortic stenosis (AS) based on various echocardiographic parameters. Accurate measurements are essential to avoid misclassification and unnecessary interventions. The objective of this study was to evaluate the influence of measurement error on the echocardiographic evaluation of the severity of AS.

METHODS AND RESULTS

A systematic review was performed to examine whether measurement errors are reported in studies focusing on the prognostic value of peak aortic jet velocity (Vmax ), mean pressure gradient (MPG), and effective orifice area (EOA) in asymptomatic patients with AS. Out of the 37 studies reviewed, 17 (46%) acknowledged the existence of measurement errors, but none of them utilized methods to address them. Secondly, the magnitude of potential errors was collected from available literature for use in clinical simulations. Interobserver variability ranged between 0.9% and 8.3% for Vmax and MPG but was higher for EOA (range 7.7%-12.7%), indicating lower reliability. Assuming a circular left ventricular outflow tract area led to a median underestimation of EOA by 23% compared to planimetry by other modalities. A clinical simulation resulted in the reclassification of 42% of patients, shifting them from a diagnosis of severe AS to moderate AS.

CONCLUSIONS

Measurement errors are underreported in studies on echocardiographic assessment of AS severity. These errors can lead to misclassification and misdiagnosis. Clinicians and scientists should be aware of the implications for accurate clinical decision-making and assuring research validity.

Linking Acute Physiology to Outcomes in the ICU: Challenges and Solutions for Research

ICU clinicians rely on bedside physiological measurements to inform many routine clinical decisions. Because deranged physiology is usually associated with poor clinical outcomes, it is tempting to hypothesize that manipulating and intervening on physiological parameters might improve outcomes for patients. However, testing these hypotheses through mathematical models of the relationship between physiology and outcomes presents a number of important methodological challenges. These models reflect the theories of the researcher and can therefore be heavily influenced by one's assumptions and background beliefs. Model building must therefore be approached with great care and forethought, because failure to consider relevant sources of measurement error, confounding, coupling, and time dependency or failure to assess the direction of causality for associations of interest before modeling may give rise to spurious results. This paper outlines the main challenges in analyzing and interpreting these models and offers potential solutions to address these challenges.

Inter-Rater Reliability of Scoring Systems for Abomasal Lesions in Quebec Veal Calves

The objective of this study was to determine the inter-rater reliability of current scoring systems used to detect abomasal lesions in veal calves. In addition, macroscopic lesions were compared with corresponding histological lesions. For this, 76 abomasa were retrieved from veal calves in a slaughterhouse in Quebec and scored by four independent raters using current scoring systems. The localisations of the lesions were separated into pyloric, fundic, or torus pyloricus areas. Lesions were classified into three different types, i.e., erosions, ulcers, and scars. To estimate the inter-rater reliability, the coefficient type 1 of Gwet's agreement and Fleiss κ were used for the presence or absence of a lesion, and the intra-class correlation coefficient was used for the number of lesions. All veal calves had at least one abomasal lesion detected. Most lesions were erosions, and most of them were located in the pyloric area. Overall, a poor to very good inter-rater agreement was seen for the pyloric area and the torus pyloricus regarding the presence or absence of a lesion (Fleiss κ: 0.00-0.34; Gwet's AC1: 0.12-0.83), although a higher agreement was observed when combining all lesions in the pyloric area (Fleiss κ: 0.09-0.12; Gwet's AC1: 0.43-0.93). For the fundic area, a poor to very good agreement was also observed (Fleiss κ: 0.17-0.70; Gwet's AC1: 0.90-0.97). Regarding the inter-rater agreement for the number of lesions, a poor to moderate agreement was found (ICC: 0.11-0.73). When using the scoring system developed in the European Welfare Quality Protocol, a poor single random rater agreement (ICC: 0.42; 95% CI: 0.31-0.56) but acceptable average random rater agreement (ICC: 0.75; 95% CI: 0.64-0.83) was determined. Microscopic scar lesions were often mistaken as ulcers macroscopically. These results show that the scoring of abomasal lesions is challenging and highlight the need for a reliable scoring system. A fast, simple, and reliable scoring system would allow for large scale studies which investigate possible risk factors and hopefully help to prevent these lesions, which can compromise veal calves' health and welfare.

Adjusting for misclassification of an exposure in an individual participant data meta-analysis

A common problem in the analysis of multiple data sources, including individual participant data meta-analysis (IPD-MA), is the misclassification of binary variables. Misclassification may lead to biased estimators of model parameters, even when the misclassification is entirely random. We aimed to develop statistical methods that facilitate unbiased estimation of adjusted and unadjusted exposure-outcome associations and between-study heterogeneity in IPD-MA, where the extent and nature of exposure misclassification may vary across studies. We present Bayesian methods that allow misclassification of binary exposure variables to depend on study- and participant-level characteristics. In an example of the differential diagnosis of dengue using two variables, where the gold standard measurement for the exposure variable was unavailable for some studies which only measured a surrogate prone to misclassification, our methods yielded more accurate estimates than analyses naive with regard to misclassification or based on gold standard measurements alone. In a simulation study, the evaluated misclassification model yielded valid estimates of the exposure-outcome association, and was more accurate than analyses restricted to gold standard measurements. Our proposed framework can appropriately account for the presence of binary exposure misclassification in IPD-MA. It requires that some studies supply IPD for the surrogate and gold standard exposure, and allows misclassification to follow a random effects distribution across studies conditional on observed covariates (and outcome). The proposed methods are most beneficial when few large studies that measured the gold standard are available, and when misclassification is frequent.

Assessment of Natural Language Processing of Electronic Health Records to Measure Goals-of-Care Discussions as a Clinical Trial Outcome

IMPORTANCE

Many clinical trial outcomes are documented in free-text electronic health records (EHRs), making manual data collection costly and infeasible at scale. Natural language processing (NLP) is a promising approach for measuring such outcomes efficiently, but ignoring NLP-related misclassification may lead to underpowered studies.

OBJECTIVE

To evaluate the performance, feasibility, and power implications of using NLP to measure the primary outcome of EHR-documented goals-of-care discussions in a pragmatic randomized clinical trial of a communication intervention.

DESIGN, SETTING, AND PARTICIPANTS

This diagnostic study compared the performance, feasibility, and power implications of measuring EHR-documented goals-of-care discussions using 3 approaches: (1) deep-learning NLP, (2) NLP-screened human abstraction (manual verification of NLP-positive records), and (3) conventional manual abstraction. The study included hospitalized patients aged 55 years or older with serious illness enrolled between April 23, 2020, and March 26, 2021, in a pragmatic randomized clinical trial of a communication intervention in a multihospital US academic health system.

MAIN OUTCOMES AND MEASURES

Main outcomes were natural language processing performance characteristics, human abstractor-hours, and misclassification-adjusted statistical power of methods of measuring clinician-documented goals-of-care discussions. Performance of NLP was evaluated with receiver operating characteristic (ROC) curves and precision-recall (PR) analyses and examined the effects of misclassification on power using mathematical substitution and Monte Carlo simulation.

RESULTS

A total of 2512 trial participants (mean [SD] age, 71.7 [10.8] years; 1456 [58%] female) amassed 44 324 clinical notes during 30-day follow-up. In a validation sample of 159 participants, deep-learning NLP trained on a separate training data set from identified patients with documented goals-of-care discussions with moderate accuracy (maximal F1 score, 0.82; area under the ROC curve, 0.924; area under the PR curve, 0.879). Manual abstraction of the outcome from the trial data set would require an estimated 2000 abstractor-hours and would power the trial to detect a risk difference of 5.4% (assuming 33.5% control-arm prevalence, 80% power, and 2-sided α = .05). Measuring the outcome by NLP alone would power the trial to detect a risk difference of 7.6%. Measuring the outcome by NLP-screened human abstraction would require 34.3 abstractor-hours to achieve estimated sensitivity of 92.6% and would power the trial to detect a risk difference of 5.7%. Monte Carlo simulations corroborated misclassification-adjusted power calculations.

CONCLUSIONS AND RELEVANCE

In this diagnostic study, deep-learning NLP and NLP-screened human abstraction had favorable characteristics for measuring an EHR outcome at scale. Adjusted power calculations accurately quantified power loss from NLP-related misclassification, suggesting that incorporation of this approach into the design of studies using NLP would be beneficial.

Nondifferential Treatment Misclassification Biases Toward the Null? Not a Safe Bet for Active Comparator Studies

Active comparator studies are increasingly common, particularly in pharmacoepidemiology. In such studies, the parameter of interest is a contrast (difference or ratio) in the outcome risks between the treatment of interest and the selected active comparator. While it may appear treatment is dichotomous, treatment is actually polytomous as there are at least 3 levels: no treatment, the treatment of interest, and the active comparator. Because misclassification may occur between any of these groups, independent nondifferential treatment misclassification may not be toward the null (as expected with a dichotomous treatment). In this work, we describe bias from independent nondifferential treatment misclassification in active comparator studies with a focus on misclassification that occurs between each active treatment and no treatment. We derive equations for bias in the estimated outcome risks, risk difference, and risk ratio, and we provide bias correction equations that produce unbiased estimates, in expectation. Using data obtained from US insurance claims data, we present a hypothetical comparative safety study of antibiotic treatment to illustrate factors that influence bias and provide an example probabilistic bias analysis using our derived bias correction equations.

Socioeconomic status and depressive symptoms: An individual-participant data meta-analysis on range restriction and measurement in the United States

INTRODUCTION

The association between socioeconomic status (SES) and depressive symptoms is well documented, yet less attention has been paid to the methodological factors contributing to between-study variability. We examined the moderating role of range restriction and the depressive-symptom measurement instrument used in estimating the correlation between components of SES and depressive symptoms.

METHODS

We conducted an individual participant data meta-analysis of nationally-representative, public-access datasets in the United States. We identified 123 individual datasets with a total of 1,655,991 participants (56.8 % female, mean age = 40.33).

RESULTS

The presence of range restriction was associated with larger correlations between income and depressive symptoms and with smaller correlations between years of education and depressive symptoms. The measurement instrument of depressive symptoms moderated the association for income, years of education, and occupational status/prestige. The Center for Epidemiological Studies-Depression scale consistently produced larger correlations. Higher measurement reliability was also associated with larger correlations.

LIMITATIONS

This study was not a comprehensive review of all measurement instruments of depressive symptoms, focused on datasets from the United States, and did not examine the moderating role of sample characteristics.

DISCUSSION

Methodological characteristics, including range restriction of SES and instrument of depressive symptoms, meaningfully influence the observed magnitude of association between SES and depressive symptoms. Clinicians and researchers designing future studies should consider which instrument of depressive symptoms is suitable for their purpose and population.

Measurement Error and Misclassification in Orthopedics: When Study Subjects are Categorized in the Wrong Exposure or Outcome Groups

Datasets available for orthopedic research often contain measurement and misclassification errors due to errors in data collection or missing data. These errors can have different effects on the study results. Measurement error refers to inaccurate measurement of continuous variables (eg, body mass index), whereas misclassification refers to assigning subjects in the wrong exposure and/or outcome groups (eg, obesity categories). Misclassification of any type can result in underestimation or overestimation of the association between exposures and outcomes. In this article, we offer practical guidelines to avoid, identify, and account for measurement and misclassification errors. We also provide an illustrative example on how to perform a validation study to address misclassification based on real-world orthopedic data. Please visit the followinghttps://youtu.be/9-ekW2NnWrsor videos that explain the highlights of the article in practical terms.

Timing errors and temporal uncertainty in clinical databases-A narrative review

A firm concept of time is essential for establishing causality in a clinical setting. Review of critical incidents and generation of study hypotheses require a robust understanding of the sequence of events but conducting such work can be problematic when timestamps are recorded by independent and unsynchronized clocks. Most clinical models implicitly assume that timestamps have been measured accurately and precisely, but this custom will need to be re-evaluated if our algorithms and models are to make meaningful use of higher frequency physiological data sources. In this narrative review we explore factors that can result in timestamps being erroneously recorded in a clinical setting, with particular focus on systems that may be present in a critical care unit. We discuss how clocks, medical devices, data storage systems, algorithmic effects, human factors, and other external systems may affect the accuracy and precision of recorded timestamps. The concept of temporal uncertainty is introduced, and a holistic approach to timing accuracy, precision, and uncertainty is proposed. This quantitative approach to modeling temporal uncertainty provides a basis to achieve enhanced model generalizability and improved analytical outcomes.

Mathematical characterization of dose uncertainty effects on functions summarizing findings of community noise attitudinal surveys

Previous Monte Carlo simulations have quantified the extent to which dose (sound level) uncertainty in community noise dose-response surveys can bias the shape of inferred dose-response functions. The present work extends the prior findings to create a mathematical model of the biasing effect. The exact effect on any particular data set depends on additional attributes (situational variables) beyond dose uncertainty itself. Several variables and their interaction effects are accounted for in the model. The model produced identical results to the prior Monte Carlo simulations and thereby demonstrated the same slope reduction effect. This model was further exercised to demonstrate the nature and extent of situational variable interaction effects related to the range of doses employed and their distribution across the range. One manifestation was a false asymptotic behavior in the observed dose-response relationship. The mathematical model provides a means to not only predict dose uncertainty effects but also to serve as a foundation for correcting for such effects in regression analyses of transportation noise dose-response relationships.

Measuring Memory for Treatment Using Patient Conceptualizations of Clinical Vignettes: A Pilot Psychometric Study in the Context of Cognitive Therapy for Depression

Background

Patient memory for psychological treatment contents is a promising transdiagnostic mechanism of change, but there is little consensus concerning its measurement. We conducted a pilot psychometric investigation of the Conceptualization Task, a novel measure of patient memory for treatment.

Methods

Data were from a trial comparing cognitive therapy-as-usual to cognitive therapy plus the Memory Support Intervention (MSI) for adults with depression (N = 171). For the Conceptualization Task, patients read clinical vignettes and provided written responses to assess three facets of conceptualization: identifying contributing factors to psychopathology, making intervention recommendations, and providing a rationale for recommendations. Higher scores were given to responses reflecting accurate memory for the theoretical model and change strategies used in treatment.

Results

The Conceptualization Task showed excellent inter-rater reliability and sensitivity to change during treatment, but only fair test–retest reliability and insufficient internal consistency. Findings supported discriminant validity with measures of education, IQ, and general memory functioning, but not convergent validity with existing measures of patient memory for treatment. Criterion validity analyses showed that some aspects of the Conceptualization Task were associated with therapist use of memory support strategies from the MSI and treatment outcome. However, findings were mixed, effect sizes were small, and some results did not remain statistically significant after correcting for multiple comparisons.

Conclusions

Further refinement and testing is needed before the Conceptualization Task may be used to assess the patient memory for treatment contents.

The Measurement Error Elephant in the Room: Challenges and Solutions to Measurement Error in Epidemiology

Measurement error, although ubiquitous, is uncommonly acknowledged and rarely assessed or corrected in epidemiologic studies. This review offers a straightforward guide to common problems caused by measurement error in research studies and a review of several accessible bias-correction methods for epidemiologists and data analysts. Although most correction methods require criterion validation including a gold standard, there are also ways to evaluate the impact of measurement error and potentially correct for it without such data. Technical difficulty ranges from simple algebra to more complex algorithms that require expertise, fine tuning, and computational power. However, at all skill levels, software packages and methods are available and can be used to understand the threat to inferences that arises from imperfect measurements.

Measurement Properties of Clinically Accessible Movement Assessment Tools for Analyzing Single-Leg Squats and Step-Downs: A Systematic Review

CONTEXT

Poor lower-extremity biomechanics are predictive of increased risk of injury. Clinicians analyze the single-leg squat (SLS) and step-down (SD) with rubrics and 2D assessments to identify these poor lower-extremity biomechanics. However, evidence on measurement properties of movement assessment tools is not strongly outlined. Measurement properties must be established before movement assessment tools are recommended for clinical use.

OBJECTIVE

The purpose of this study was to systematically review the evidence on measurement properties of rubrics and 2D assessments used to analyze an SLS and SD.

EVIDENCE ACQUISITION

The search strategy was developed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-analysis guidelines. The search was performed in PubMed, SPORTDiscus, and Web of Science databases. The COnsensus-based Standards for the selection of health Measurement INstruments multiphase procedure was used to extract relevant data, evaluate methodological quality of each study, score the results of each movement assessment, and synthesize the evidence.

EVIDENCE SYNTHESIS

A total of 44 studies were included after applying eligibility criteria. Reliability and construct validity of knee frontal plane projection angle was acceptable, but criterion validity was unacceptable. Reliability of the Chmielewski rubric was unacceptable. Content validity of the knee-medial-foot and pelvic drop rubrics was acceptable. The remaining rubrics and 2D measurements had inconclusive or conflicting results regarding reliability and validity.

CONCLUSIONS

Knee frontal plane projection angle is reliable for analyzing the SLS and SD; however, it does not serve as a substitute for 3D motion analysis. The Chmielewski rubric is not recommended for assessing the SLS or SD as it may be unreliable. Most movement assessment tools yield indeterminate results. Within the literature, standardized names, procedures, and reporting of movement assessment tool reliability and validity are inconsistent.

Sensitivity analysis for random measurement error using regression calibration and simulation-extrapolation

Objective

Sensitivity analysis for random measurement error can be applied in the absence of validation data by means of regression calibration and simulation-extrapolation. These have not been compared for this purpose.

Study design and setting

A simulation study was conducted comparing the performance of regression calibration and simulation-extrapolation for linear and logistic regression. The performance of the two methods was evaluated in terms of bias, mean squared error (MSE) and confidence interval coverage, for various values of reliability of the error-prone measurement (0.05-0.91), sample size (125-4000), number of replicates (2-10), and R-squared (0.03-0.75). It was assumed that no validation data were available about the error-free measures, while correct information about the measurement error variance was available.

Results

Regression calibration was unbiased while simulation-extrapolation was biased: median bias was 0.8% (interquartile range (IQR): -0.6;1.7%), and -19.0% (IQR: -46.4;-12.4%), respectively. A small gain in efficiency was observed for simulation-extrapolation (median MSE: 0.005, IQR: 0.004;0.006) versus regression calibration (median MSE: 0.006, IQR: 0.005;0.009). Confidence interval coverage was at the nominal level of 95% for regression calibration, and smaller than 95% for simulation-extrapolation (median coverage: 85%, IQR: 73;93%). The application of regression calibration and simulation-extrapolation for a sensitivity analysis was illustrated using an example of blood pressure and kidney function.

Conclusion

Our results support the use of regression calibration over simulation-extrapolation for sensitivity analysis for random measurement error.

Evaluation of a suggested novel method to adjust BMI calculated from self-reported weight and height for measurement error

OBJECTIVE

In 2019, Ward et al. proposed a method to adjust BMI calculated from self-reported weight and height for bias relative to measured data. They did not evaluate the adjusted values relative to measured BMI values for the same individuals.

METHODS

A large data set (n = 37,439) with both measured and self-reported weight and height was randomly divided into two groups. The proposed method was used to adjust the BMI values in one group to the measured data from the other group. The adjusted values were then compared with the measured values for the same individuals.

RESULTS

Before adjustment, 24.9% were incorrectly classified relative to measured BMI categories, including 7.9% in too high a category; after adjustment, 24.3% were incorrectly classified, with 12.8% in too high a category. The variance of the difference was unchanged. The adjustments reduced some errors and introduced new errors. At an individual level, results were unpredictable.

CONCLUSIONS

The suggested method has little effect on misclassification, can introduce new errors, and could magnify errors associated with factors, such as age, race, educational level, or other characteristics. State-level estimates and projections of obesity prevalence from values adjusted by this method may be incorrect.

An examination of dose uncertainty and dose distribution effects on community noise attitudinal survey outcomes

Social survey data sets of large numbers of individual respondents' opinions are generally viewed as supporting reliable inferences of relationships between the prevalence of noise-induced annoyance and noise exposure levels. The current analyses identify conditions under which noise dose distributions and acoustic measurement uncertainty lead to appreciable mis-estimation of the slopes of empirical dose-response relationships with respect to those of true slopes in exposure ranges of interest. These findings were revealed by Monte Carlo methods for creating simulated data sets with varying exposure ranges and degrees of dose uncertainty. These simulated data sets support quantitative comparisons of dose-response relationships between empirical outcomes and known (assumed) relationships. The effect of noise dose uncertainty is appreciable for dose uncertainties with standard deviations greater than about 2 decibels. Limited dose ranges as well as haystack-shaped (non-uniform) dose distributions magnify the biasing effect of dose uncertainty on the slopes of observed relationships. Narrow exposure ranges can also create a false asymptotic behavior in the relationship. These phenomena are well documented in the non-acoustic literature.

Sampling Strategies for Internal Validation Samples for Exposure Measurement-Error Correction: A Study of Visceral Adipose Tissue Measures Replaced by Waist Circumference Measures

Statistical correction for measurement error in epidemiologic studies is possible, provided that information about the measurement error model and its parameters are available. Such information is commonly obtained from a randomly sampled internal validation sample. It is however unknown whether randomly sampling the internal validation sample is the optimal sampling strategy. We conducted a simulation study to investigate various internal validation sampling strategies in conjunction with regression calibration. Our simulation study showed that for an internal validation study sample of 40% of the main study’s sample size, stratified random and extremes sampling had a small efficiency gain over random sampling (10% and 12% decrease on average over all scenarios, respectively). The efficiency gain was more pronounced in smaller validation samples of 10% of the main study’s sample size (i.e., a 31% and 36% decrease on average over all scenarios, for stratified random and extremes sampling, respectively). To mitigate the bias due to measurement error in epidemiologic studies, small efficiency gains can be achieved for internal validation sampling strategies other than random, but only when measurement error is nondifferential. For regression calibration, the gain in efficiency is, however, at the cost of a higher percentage bias and lower coverage.

Mecor: An R package for measurement error correction in linear regression models with a continuous outcome

Measurement error in a covariate or the outcome of regression models is common, but is often ignored, even though measurement error can lead to substantial bias in the estimated covariate-outcome association. While several texts on measurement error correction methods are available, these methods remain seldomly applied. To improve the use of measurement error correction methodology, we developed mecor, an R package that implements measurement error correction methods for regression models with a continuous outcome. Measurement error correction requires information about the measurement error model and its parameters. This information can be obtained from four types of studies, used to estimate the parameters of the measurement error model: an internal validation study, a replicates study, a calibration study and an external validation study. In the package mecor, regression calibration methods and a maximum likelihood method are implemented to correct for measurement error in a continuous covariate in regression analyses. Additionally, methods of moments methods are implemented to correct for measurement error in the continuous outcome in regression analyses. Variance estimation of the corrected estimators is provided in closed form and using the bootstrap.

Measurement error in meta-analysis (MEMA)-A Bayesian framework for continuous outcome data subject to non-differential measurement error

Ideally, a meta-analysis will summarize data from several unbiased studies. Here we look into the less than ideal situation in which contributing studies may be compromised by non-differential measurement error in the exposure variable. Specifically, we consider a meta-analysis for the association between a continuous outcome variable and one or more continuous exposure variables, where the associations may be quantified as regression coefficients of a linear regression model. A flexible Bayesian framework is developed which allows one to obtain appropriate point and interval estimates with varying degrees of prior knowledge about the magnitude of the measurement error. We also demonstrate how, if individual-participant data (IPD) are available, the Bayesian meta-analysis model can adjust for multiple participant-level covariates, these being measured with or without measurement error.

Examining the robustness of observational associations to model, measurement and sampling uncertainty with the vibration of effects framework

BACKGROUND

The results of studies on observational associations may vary depending on the study design and analysis choices as well as due to measurement error. It is important to understand the relative contribution of different factors towards generating variable results, including low sample sizes, researchers' flexibility in model choices, and measurement error in variables of interest and adjustment variables.

METHODS

We define sampling, model and measurement uncertainty, and extend the concept of vibration of effects in order to study these three types of uncertainty in a common framework. In a practical application, we examine these types of uncertainty in a Cox model using data from the National Health and Nutrition Examination Survey. In addition, we analyse the behaviour of sampling, model and measurement uncertainty for varying sample sizes in a simulation study.

RESULTS

All types of uncertainty are associated with a potentially large variability in effect estimates. Measurement error in the variable of interest attenuates the true effect in most cases, but can occasionally lead to overestimation. When we consider measurement error in both the variable of interest and adjustment variables, the vibration of effects are even less predictable as both systematic under- and over-estimation of the true effect can be observed. The results on simulated data show that measurement and model vibration remain non-negligible even for large sample sizes.

CONCLUSION

Sampling, model and measurement uncertainty can have important consequences for the stability of observational associations. We recommend systematically studying and reporting these types of uncertainty, and comparing them in a common framework.

Data quality assessment and subsampling strategies to correct distributional bias in prevalence studies

Background

Healthcare-associated infections (HAIs) represent a major Public Health issue. Hospital-based prevalence studies are a common tool of HAI surveillance, but data quality problems and non-representativeness can undermine their reliability.

Methods

This study proposes three algorithms that, given a convenience sample and variables relevant for the outcome of the study, select a subsample with specific distributional characteristics, boosting either representativeness (Probability and Distance procedures) or risk factors’ balance (Uniformity procedure). A “Quality Score” (QS) was also developed to grade sampled units according to data completeness and reliability.

The methodologies were evaluated through bootstrapping on a convenience sample of 135 hospitals collected during the 2016 Italian Point Prevalence Survey (PPS) on HAIs.

Results

The QS highlighted wide variations in data quality among hospitals (median QS 52.9 points, range 7.98–628, lower meaning better quality), with most problems ascribable to ward and hospital-related data reporting. Both Distance and Probability procedures produced subsamples with lower distributional bias (Log-likelihood score increased from 7.3 to 29 points). The Uniformity procedure increased the homogeneity of the sample characteristics (e.g., − 58.4% in geographical variability).

The procedures selected hospitals with higher data quality, especially the Probability procedure (lower QS in 100% of bootstrap simulations). The Distance procedure produced lower HAI prevalence estimates (6.98% compared to 7.44% in the convenience sample), more in line with the European median.

Conclusions

The QS and the subsampling procedures proposed in this study could represent effective tools to improve the quality of prevalence studies, decreasing the biases that can arise due to non-probabilistic sample collection.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01277-y.

Combining self-reported and objectively measured survey data to improve hypertension prevalence estimates: Portuguese experience

BACKGROUND

Accurate data on hypertension is essential to inform decision-making. Hypertension prevalence may be underestimated by population-based surveys due to misclassification of health status by participants. Therefore, adjustment for misclassification bias is required when relying on self-reports. This study aims to quantify misclassification bias in self-reported hypertension prevalence and prevalence ratios in the Portuguese component of the European Health Interview Survey (INS2014), and illustrate application of multiple imputation (MIME) for bias correction using measured high blood pressure data from the first Portuguese health examination survey (INSEF).

METHODS

We assumed that objectively measured hypertension status was missing for INS2014 participants (n = 13,937) and imputed it using INSEF (n = 4910) as auxiliary data. Self-reported, objectively measured and MIME-corrected hypertension prevalence and prevalence ratios (PR) by sex, age group and education were estimated. Bias in self-reported and MIME-corrected estimates were computed using objectively measured INSEF data as a gold-standard.

RESULTS

Self-reported INS2014 data underestimated hypertension prevalence in all population subgroups, with misclassification bias ranging from 5.2 to 18.6 percentage points (pp). After MIME-correction, prevalence estimates increased and became closer to objectively measured ones, with bias reduction to 0 pp - 5.7 pp. Compared to objectively measured INSEF, self-reported INS2014 data considerably underestimated prevalence ratio by sex (PR = 0.8, 95CI = [0.7, 0.9] vs. PR = 1.2, 95CI = [1.1, 1.4]). MIME successfully corrected direction of association with sex in bivariate (PR = 1.1, 95CI = [1.0, 1.3]) and multivariate analyses (PR = 1.2, 95CI = [1.0, 1.3]). Misclassification bias in hypertension prevalence ratios by education and age group were less pronounced and did not require correction in multivariate analyses.

CONCLUSIONS

Our results highlight the importance of misclassification bias analysis in self-reported hypertension. Multiple imputation is a feasible approach to adjust for misclassification bias in prevalence estimates and exposure-outcomes associations in survey data.

Calibrating validation samples when accounting for measurement error in intervention studies

Many lifestyle intervention trials depend on collecting self-reported outcomes, such as dietary intake, to assess the intervention's effectiveness. Self-reported outcomes are subject to measurement error, which impacts treatment effect estimation. External validation studies measure both self-reported outcomes and accompanying biomarkers, and can be used to account for measurement error. However, in order to account for measurement error using an external validation sample, an assumption must be made that the inferences are transportable from the validation sample to the intervention trial of interest. This assumption does not always hold. In this paper, we propose an approach that adjusts the validation sample to better resemble the trial sample, and we also formally investigate when bias due to poor transportability may arise. Lastly, we examine the performance of the methods using simulation, and illustrate them using PREMIER, a lifestyle intervention trial measuring self-reported sodium intake as an outcome, and OPEN, a validation study measuring both self-reported diet and urinary biomarkers.

Propensity score trimming mitigates bias due to covariate measurement error in inverse probability of treatment weighted analyses: A plasmode simulation

BACKGROUND

Inverse probability of treatment weighting (IPTW) may be biased by influential observations, which can occur from misclassification of strong exposure predictors.

METHODS

We evaluated bias and precision of IPTW estimators in the presence of a misclassified confounder and assessed the effect of propensity score (PS) trimming. We generated 1000 plasmode cohorts of size N = 10 000, sampled with replacement from 6063 NHANES respondents (1999-2014) age 40 to 79 with labs and no statin use. We simulated statin exposure as a function of demographics and CVD risk factors; and outcomes as a function of 10-year CVD risk score and statin exposure (rate ratio [RR] = 0.5). For 5% of the people in selected populations (eg, all patients, exposed, those with outcomes), we randomly misclassified a confounder that strongly predicted exposure. We fit PS models and estimated RRs using IPTW and 1:1 PS matching, with and without asymmetric trimming.

RESULTS

IPTW bias was substantial when misclassification was differential by outcome (RR range: 0.38-0.63) and otherwise minimal (RR range: 0.51-0.53). However, trimming reduced bias for IPTW, nearly eliminating it at 5% trimming (RR range: 0.49-0.52). In one scenario, when the confounder was misclassified for 5% of those with outcomes (0.3% of cohort), untrimmed IPTW was more biased and less precise (RR = 0.37 [SE(logRR) = 0.21]) than matching (RR = 0.50 [SE(logRR) = 0.13]). After 1% trimming, IPTW estimates were unbiased and more precise (RR = 0.49 [SE(logRR) = 0.12]) than matching (RR = 0.51 [SE(logRR) = 0.14]).

CONCLUSIONS

Differential misclassification of a strong predictor of exposure resulted in biased and imprecise IPTW estimates. Asymmetric trimming reduced bias, with more precise estimates than matching.

Clinical relevance assessment of animal preclinical research (RAA) tool: development and explanation

Background

Only a small proportion of preclinical research (research performed in animal models prior to clinical trials in humans) translates into clinical benefit in humans. Possible reasons for the lack of translation of the results observed in preclinical research into human clinical benefit include the design, conduct, and reporting of preclinical studies. There is currently no formal domain-based assessment of the clinical relevance of preclinical research. To address this issue, we have developed a tool for the assessment of the clinical relevance of preclinical studies, with the intention of assessing the likelihood that therapeutic preclinical findings can be translated into improvement in the management of human diseases.

Methods

We searched the EQUATOR network for guidelines that describe the design, conduct, and reporting of preclinical research. We searched the references of these guidelines to identify further relevant publications and developed a set of domains and signalling questions. We then conducted a modified Delphi-consensus to refine and develop the tool. The Delphi panel members included specialists in evidence-based (preclinical) medicine specialists, methodologists, preclinical animal researchers, a veterinarian, and clinical researchers. A total of 20 Delphi-panel members completed the first round and 17 members from five countries completed all three rounds.

Results

This tool has eight domains (construct validity, external validity, risk of bias, experimental design and data analysis plan, reproducibility and replicability of methods and results in the same model, research integrity, and research transparency) and a total of 28 signalling questions and provides a framework for researchers, journal editors, grant funders, and regulatory authorities to assess the potential clinical relevance of preclinical animal research.

Conclusion

We have developed a tool to assess the clinical relevance of preclinical studies. This tool is currently being piloted.

Challenges and solutions in prognostic prediction models in spinal disorders

Methodological shortcomings in prognostic modeling for patients with spinal disorders are highly common. This general commentary discusses methodological challenges related to the specific nature of this field. Five specific methodological challenges in prognostic modeling for patients with spinal disorders are presented with their potential solutions, as related to the choice of study participants, purpose of studies, limitations in measurements of outcomes and predictors, complexity of recovery predictions, and confusion of prognosis and treatment response. Large studies specifically designed for prognostic model research are needed, using standard baseline measurement sets, clearly describing participants' recruitment and accounting and correcting for measurement limitations.

Importance of incorporating quantitative imaging biomarker technical performance characteristics when estimating treatment effects

BACKGROUND/AIMS

Quantitative imaging biomarkers have the potential to detect change in disease early and noninvasively, providing information about the diagnosis and prognosis of a patient, aiding in monitoring disease, and informing when therapy is effective. In clinical trials testing new therapies, there has been a tendency to ignore the variability and bias in quantitative imaging biomarker measurements. Unfortunately, this can lead to underpowered studies and incorrect estimates of the treatment effect. We illustrate the problem when non-constant measurement bias is ignored and show how treatment effect estimates can be corrected.

METHODS

Monte Carlo simulation was used to assess the coverage of 95% confidence intervals for the treatment effect when non-constant bias is ignored versus when the bias is corrected for. Three examples are presented to illustrate the methods: doubling times of lung nodules, rates of change in brain atrophy in progressive multiple sclerosis clinical trials, and changes in proton-density fat fraction in trials for patients with nonalcoholic fatty liver disease.

RESULTS

Incorrectly assuming that the measurement bias is constant leads to 95% confidence intervals for the treatment effect with reduced coverage (<95%); the coverage is especially reduced when the quantitative imaging biomarker measurements have good precision and/or there is a large treatment effect. Estimates of the measurement bias from technical performance validation studies can be used to correct the confidence intervals for the treatment effect.

CONCLUSION

Technical performance validation studies of quantitative imaging biomarkers are needed to supplement clinical trial data to provide unbiased estimates of the treatment effect.

Introduction to statistical simulations in health research

In health research, statistical methods are frequently used to address a wide variety of research questions. For almost every analytical challenge, different methods are available. But how do we choose between different methods and how do we judge whether the chosen method is appropriate for our specific study? Like in any science, in statistics, experiments can be run to find out which methods should be used under which circumstances. The main objective of this paper is to demonstrate that simulation studies, that is, experiments investigating synthetic data with known properties, are an invaluable tool for addressing these questions. We aim to provide a first introduction to simulation studies for data analysts or, more generally, for researchers involved at different levels in the analyses of health data, who (1) may rely on simulation studies published in statistical literature to choose their statistical methods and who, thus, need to understand the criteria of assessing the validity and relevance of simulation results and their interpretation; and/or (2) need to understand the basic principles of designing statistical simulations in order to efficiently collaborate with more experienced colleagues or start learning to conduct their own simulations. We illustrate the implementation of a simulation study and the interpretation of its results through a simple example inspired by recent literature, which is completely reproducible using the R-script available from online supplemental file 1.

Approaches to addressing missing values, measurement error, and confounding in epidemiologic studies

OBJECTIVES

Epidemiologic studies often suffer from incomplete data, measurement error (or misclassification), and confounding. Each of these can cause bias and imprecision in estimates of exposure-outcome relations. We describe and compare statistical approaches that aim to control all three sources of bias simultaneously.

STUDY DESIGN AND SETTING

We illustrate four statistical approaches that address all three sources of bias, namely, multiple imputation for missing data and measurement error, multiple imputation combined with regression calibration, full information maximum likelihood within a structural equation modeling framework, and a Bayesian model. In a simulation study, we assess the performance of the four approaches compared with more commonly used approaches that do not account for measurement error, missing values, or confounding.

RESULTS

The results demonstrate that the four approaches consistently outperform the alternative approaches on all performance metrics (bias, mean squared error, and confidence interval coverage). Even in simulated data of 100 subjects, these approaches perform well.

CONCLUSION

There can be a large benefit of addressing measurement error, missing values, and confounding to improve the estimation of exposure-outcome relations, even when the available sample size is relatively small.

A weighting method for simultaneous adjustment for confounding and joint exposure-outcome misclassifications

Joint misclassification of exposure and outcome variables can lead to considerable bias in epidemiological studies of causal exposure-outcome effects. In this paper, we present a new maximum likelihood based estimator for marginal causal effects that simultaneously adjusts for confounding and several forms of joint misclassification of the exposure and outcome variables. The proposed method relies on validation data for the construction of weights that account for both sources of bias. The weighting estimator, which is an extension of the outcome misclassification weighting estimator proposed by Gravel and Platt (Weighted estimation for confounded binary outcomes subject to misclassification. Stat Med 2018; 37: 425–436), is applied to reinfarction data. Simulation studies were carried out to study its finite sample properties and compare it with methods that do not account for confounding or misclassification. The new estimator showed favourable large sample properties in the simulations. Further research is needed to study the sensitivity of the proposed method and that of alternatives to violations of their assumptions. The implementation of the estimator is facilitated by a new R function (ipwm) in an existing R package (mecor).

Current clinical methods of measurement of respiratory rate give imprecise values

Background

Respiratory rate is a basic clinical measurement used for illness assessment. Errors in measuring respiratory rate are attributed to observer and equipment problems. Previous studies commonly report rate differences ranging from 2 to 6 breaths·min^-1 between observers.

Methods

To study why repeated observations should vary so much, we conducted a virtual experiment, using continuous recordings of breathing from acutely ill patients. These records allowed each breathing cycle to be precisely timed. We made repeated random measures of respiratory rate using different sample durations of 30, 60 and 120 s. We express the variation in these repeated rate measurements for the different sample durations as the interquartile range of the values obtained for each subject. We predicted what values would be found if a single measure, taken from any patient, were repeated and inspected boundary values of 12, 20 or 25 breaths·min^-1, used by the UK National Early Warning Score, for possible mis-scoring.

Results

When the sample duration was nominally 30 s, the mean interquartile range of repeated estimates was 3.4 breaths·min^-1. For the 60 s samples, the mean interquartile range was 3 breaths·min^-1, and for the 120 s samples it was 2.5 breaths·min^-1. Thus, repeat clinical counts of respiratory rate often differ by >3 breaths·min^-1. For 30 s samples, up to 40% of National Early Warning Scores could be misclassified.

Conclusions

Early warning scores will be unreliable when short sample durations are used to measure respiratory rate. Precision improves with longer sample duration, but this may be impractical unless better measurement methods are used.

When predictive analytics goes wrong: what can healthcare learn from Formula 1?

Predictive analytics refers to technology that uses patterns in large datasets to predict future events and inform decisions. This article considers the challenges of this technology and how these should be considered, before incorporating this technology into healthcare settings.

Toward a framework for the design, implementation, and reporting of methodology scoping reviews

BACKGROUND AND OBJECTIVE

In view of the growth of published articles, there is an increasing need for studies that summarize scientific research. An increasingly common review is a "methodology scoping review," which provides a summary of existing analytical methods, techniques and software that have been proposed or applied in research articles to address an analytical problem or further an analytical approach. However, guidelines for their design, implementation, and reporting are limited.

METHODS

Drawing on the experiences of the authors, which were consolidated through a series of face-to-face workshops, we summarize the challenges inherent in conducting a methodology scoping review and offer suggestions of best practice to promote future guideline development.

RESULTS

We identified three challenges of conducting a methodology scoping review. First, identification of search terms; one cannot usually define the search terms a priori, and the language used for a particular method can vary across the literature. Second, the scope of the review requires careful consideration because new methodology is often not described (in full) within abstracts. Third, many new methods are motivated by a specific clinical question, where the methodology may only be documented in supplementary materials. We formulated several recommendations that build upon existing review guidelines. These recommendations ranged from an iterative approach to defining search terms through to screening and data extraction processes.

CONCLUSION

Although methodology scoping reviews are an important aspect of research, there is currently a lack of guidelines to standardize their design, implementation, and reporting. We recommend a wider discussion on this topic.

STRATOS guidance document on measurement error and misclassification of variables in observational epidemiology: Part 2-More complex methods of adjustment and advanced topics

We continue our review of issues related to measurement error and misclassification in epidemiology. We further describe methods of adjusting for biased estimation caused by measurement error in continuous covariates, covering likelihood methods, Bayesian methods, moment reconstruction, moment-adjusted imputation, and multiple imputation. We then describe which methods can also be used with misclassification of categorical covariates. Methods of adjusting estimation of distributions of continuous variables for measurement error are then reviewed. Illustrative examples are provided throughout these sections. We provide lists of available software for implementing these methods and also provide the code for implementing our examples in the Supporting Information. Next, we present several advanced topics, including data subject to both classical and Berkson error, modeling continuous exposures with measurement error, and categorical exposures with misclassification in the same model, variable selection when some of the variables are measured with error, adjusting analyses or design for error in an outcome variable, and categorizing continuous variables measured with error. Finally, we provide some advice for the often met situations where variables are known to be measured with substantial error, but there is only an external reference standard or partial (or no) information about the type or magnitude of the error.

STRATOS guidance document on measurement error and misclassification of variables in observational epidemiology: Part 1-Basic theory and simple methods of adjustment

Measurement error and misclassification of variables frequently occur in epidemiology and involve variables important to public health. Their presence can impact strongly on results of statistical analyses involving such variables. However, investigators commonly fail to pay attention to biases resulting from such mismeasurement. We provide, in two parts, an overview of the types of error that occur, their impacts on analytic results, and statistical methods to mitigate the biases that they cause. In this first part, we review different types of measurement error and misclassification, emphasizing the classical, linear, and Berkson models, and on the concepts of nondifferential and differential error. We describe the impacts of these types of error in covariates and in outcome variables on various analyses, including estimation and testing in regression models and estimating distributions. We outline types of ancillary studies required to provide information about such errors and discuss the implications of covariate measurement error for study design. Methods for ascertaining sample size requirements are outlined, both for ancillary studies designed to provide information about measurement error and for main studies where the exposure of interest is measured with error. We describe two of the simpler methods, regression calibration and simulation extrapolation (SIMEX), that adjust for bias in regression coefficients caused by measurement error in continuous covariates, and illustrate their use through examples drawn from the Observing Protein and Energy (OPEN) dietary validation study. Finally, we review software available for implementing these methods. The second part of the article deals with more advanced topics.

Testing repeatability, measurement error and species differentiation when using geometric morphometrics on complex shapes: a case study of Patagonian lizards of the genus Liolaemus (Squamata: Liolaemini)

The repeatability of findings is the key factor behind scientific reliability, and the failure to reproduce scientific findings has been termed the ‘replication crisis’. Geometric morphometrics is an established tool in evolutionary biology. However, different operators (and/or different methods) could act as large sources of variation in the data obtained. Here, we investigated inter-operator error in geometric morphometric protocols on complex shapes of Liolaemus lizards, as well as measurement error in three taxa varying in their difficulty of digitalization. We also examined the potential for these protocols to discriminate among complex shapes in closely related species. We found a wide range of inter-operator error, contributing between 19.5% and 60% to the total variation. Moreover, measurement error increased with the complexity of the quantified shape. All protocols were able to discriminate between species, but the use of more landmarks did not imply better performance. We present evidence that complex shapes reduce repeatability, highlighting the need to explore different sources of variation that could lead to such low repeatability. Lastly, we suggest some recommendations to improve the repeatability and reliability of geometric morphometrics results.

Relying on repeated biospecimens to reduce the effects of classical-type exposure measurement error in studies linking the exposome to health

The exposome calls for assessing numerous exposures, typically using biomarkers with varying amounts of measurement error, which can be assumed to be of classical type. We evaluated the impact of classical-type measurement error on the performance of exposome-health studies, and the efficiency of two measurement error correction methods relying on the collection of repeated biospecimens: within-subject biospecimens pooling and regression calibration. In a simulation study, we generated 237 exposures from a realistic correlation matrix, with various amounts of classical-type measurement error, and a continuous health outcome linearly influenced by exposures. Measurement error decreased the sensitivity to identify exposures influencing health from a value of 75% down to 46%, increased false discovery proportion from 26% to 49% and increased attenuation bias in the slope of true predictors from 45% to 66%. Assuming that repeated biospecimens were available, within-subject pooling and regression calibration improved sensitivity (which increased to 63%), false discovery proportion (down to 37%) and bias (down to 49%) compared to an error-prone study with a single biospecimen per subject. Performances were poorer for the exposures with the largest amount of measurement error, and increased with the number of available biospecimens. Relying on repeated biospecimens only for the exposures with the largest amount of measurement error provided similar performance improvement. Exposome studies relying on spot exposure biospecimens suffer from decreased performances if some biomarkers suffer from measurement error due to their temporal variability; performances can be improved by collecting repeated biospecimens per subject, in particular for non persistent chemicals.

Regression on imperfect class labels derived by unsupervised clustering

Outcome regressed on class labels identified by unsupervised clustering is custom in many applications. However, it is common to ignore the misclassification of class labels caused by the learning algorithm, which potentially leads to serious bias of the estimated effect parameters. Due to their generality we suggest to address the problem by use of regression calibration or the misclassification simulation and extrapolation method. Performance is illustrated by simulated data from Gaussian mixture models, documenting a reduced bias and improved coverage of confidence intervals when adjusting for misclassification with either method. Finally, we apply our method to data from a previous study, which regressed overall survival on class labels derived from unsupervised clustering of gene expression data from bone marrow samples of multiple myeloma patients.

Reflection on modern methods: five myths about measurement error in epidemiological research

Epidemiologists are often confronted with datasets to analyse which contain measurement error due to, for instance, mistaken data entries, inaccurate recordings and measurement instrument or procedural errors. If the effect of measurement error is misjudged, the data analyses are hampered and the validity of the study's inferences may be affected. In this paper, we describe five myths that contribute to misjudgments about measurement error, regarding expected structure, impact and solutions to mitigate the problems resulting from mismeasurements. The aim is to clarify these measurement error misconceptions. We show that the influence of measurement error in an epidemiological data analysis can play out in ways that go beyond simple heuristics, such as heuristics about whether or not to expect attenuation of the effect estimates. Whereas we encourage epidemiologists to deliberate about the structure and potential impact of measurement error in their analyses, we also recommend exercising restraint when making claims about the magnitude or even direction of effect of measurement error if not accompanied by statistical measurement error corrections or quantitative bias analysis. Suggestions for alleviating the problems or investigating the structure and magnitude of measurement error are given.

Accounting for measurement error to assess the effect of air pollution on omic signals

Studies on the effects of air pollution and more generally environmental exposures on health require measurements of pollutants, which are affected by measurement error. This is a cause of bias in the estimation of parameters relevant to the study and can lead to inaccurate conclusions when evaluating associations among pollutants, disease risk and biomarkers. Although the presence of measurement error in such studies has been recognized as a potential problem, it is rarely considered in applications and practical solutions are still lacking. In this work, we formulate Bayesian measurement error models and apply them to study the link between air pollution and omic signals. The data we use stem from the "Oxford Street II Study", a randomized crossover trial in which 60 volunteers walked for two hours in a traffic-free area (Hyde Park) and in a busy shopping street (Oxford Street) of London. Metabolomic measurements were made in each individual as well as air pollution measurements, in order to investigate the association between short-term exposure to traffic related air pollution and perturbation of metabolic pathways. We implemented error-corrected models in a classical framework and used the flexibility of Bayesian hierarchical models to account for dependencies among omic signals, as well as among different pollutants. Models were implemented using traditional Markov Chain Monte Carlo (MCMC) simulative methods as well as integrated Laplace approximation. The inclusion of a classical measurement error term resulted in variable estimates of the association between omic signals and traffic related air pollution measurements, where the direction of the bias was not predictable a priori. The models were successful in including and accounting for different correlation structures, both among omic signals and among different pollutant exposures. In general, more associations were identified when the correlation among omics and among pollutants were modeled, and their number increased when a measurement error term was additionally included in the multivariate models (particularly for the associations between metabolomics and NO2).

An approach to explore for a sweet spot in randomized trials

OBJECTIVE

The objective of the study was to demonstrate how a conventional randomized trial can be analyzed through a stratified or a matched approach to identify a potential sweet spot where observed differences might be accentuated in the mid range of disease severity.

DESIGN AND SETTING

We review a landmark randomized trial of heart failure patients that tested whether implantable defibrillators reduce mortality (n = 2,521).

RESULTS

Overall, 22% (182/829) of the patients in the defibrillator group died compared with 29% (484/1,692) of patients in the control group. Proportional hazards analysis yielded a modest 25% survival benefit (hazard ratio = 0.75, 95% confidence interval: 0.63 to 0.89). Stratified analysis of the trial yielded a larger 52% survival benefit for those in the middle quintile of disease severity (hazard ratio = 0.48, 95% confidence interval: 0.29 to 0.79). In contrast, little of the survival benefit was explained by patients with the greatest disease severity (hazard ratio = 0.89, 95% confidence interval: 0.69 to 1.15). The discrepancy between crude and stratified analyses could be visualized by graphical displays and replicated with matched comparisons.

CONCLUSION

Our approach for analyzing a randomized trial could help identify a potential sweet spot of an accentuated treatment effect.

Exploratory data analysis for pre and post 24/7/365 attending radiologist coverage support in an emergency department: fundamentals of data science

OBJECTIVE

To present a detailed exploratory data analysis for critically investigating the patterns in medical doctor (MD) to disposition time, pre and post 24/7/365 attending radiologist coverage, for patients presenting to an emergency department (ED).

MATERIALS AND METHODS

The process involved presenting several modeling techniques. To share an understanding of concepts and techniques, we used proportions, medians, and means, Mann-Whitney U test, Kaplan-Meier's (KM) survival analysis, linear and log-linear regression, log-ranked test, Cox proportional hazards model, Weibull parametric survival models and tertile analysis. Retrospective chart review was conducted to obtain a data set which was used to determine the trends in MD to disposition time. Data comprised of patients who had visited the emergency department (ED) during two distinct time periods and whose imaging studies were read by an attending emergency and trauma radiologist.

RESULTS

Median provided more insight into the data as compared with the mean. The Mann-Whitney U test was appropriate to evaluate MD to disposition time, but provided limited information. The Kaplan-Meier (KM) was able to offer more insight into the data since it did not assume an underlying model and that is the reason why it was appropriate. However, KM had limited ability to handle measured confounders and was unable to describe the magnitude of difference between curves. The Cox proportional hazards semi-parametric model or some other parametric model such as the Weibull could handle multiple measured confounders and described the magnitude of difference between two (survival) groups in the data set. However, both methods assumed underlying models that may not apply to the data set such as the one used in this study. Linear regression was unlikely to be appropriate due to the shape of survival time distributions, but log transforming the outcome could address the distribution issue. Nearly all the results of the KM subgroup analyses were consistent with the results of the log-transformed linear regression subgroup analyses and the interpretation of the results was the same for both.

CONCLUSION

Different statistical procedures may be applied to conduct exploratory subgroup analysis for a data set from a pre and post 24/7/365 attending coverage model. This could guide potential areas of further research to compare trends in MD to disposition time in ED. Pattern analysis provides evidence for various stakeholders to rethink the discourse about trends in MD to disposition time, pre and post 24/7/365 attending coverage. Graphical Illustration: The role of Emergency and Trauma Radiology in an Emergency Department.