Evaluating individualized treatment effect predictions: A model-based perspective on discrimination and calibration assessment

In recent years, there has been a growing interest in the prediction of individualized treatment effects. While there is a rapidly growing literature on the development of such models, there is little literature on the evaluation of their performance. In this paper, we aim to facilitate the validation of prediction models for individualized treatment effects. The estimands of interest are defined based on the potential outcomes framework, which facilitates a comparison of existing and novel measures. In particular, we examine existing measures of discrimination for benefit (variations of the c-for-benefit), and propose model-based extensions to the treatment effect setting for discrimination and calibration metrics that have a strong basis in outcome risk prediction. The main focus is on randomized trial data with binary endpoints and on models that provide individualized treatment effect predictions and potential outcome predictions. We use simulated data to provide insight into the characteristics of the examined discrimination and calibration statistics under consideration, and further illustrate all methods in a trial of acute ischemic stroke treatment. The results show that the proposed model-based statistics had the best characteristics in terms of bias and accuracy. While resampling methods adjusted for the optimism of performance estimates in the development data, they had a high variance across replications that limited their accuracy. Therefore, individualized treatment effect models are best validated in independent data. To aid implementation, a software implementation of the proposed methods was made available in R.

Optimizing Autologous Stem Cell Transplantation in Multiple Myeloma: The Impact of Intensive Chemomobilization

Most transplant-eligible multiple myeloma (MM) patients undergo autologous peripheral blood stem cell collection (PBSC) using G-CSF with on-demand plerixafor (G ± P). Chemomobilization (CM) can be used as a salvage regimen after G ± P failure or for debulking residual tumor burden ahead of autologous peripheral blood stem cell transplantation (ASCT). Prior studies utilizing cyclophosphamide-based CM have not shown long-term benefits. At our center, intensive CM (ICM) using a PACE- or HyperCVAD-based regimen has been used to mitigate "excessive" residual disease based on plasma cell (PC) burden or MM-related biomarkers. Given the lack of efficacy of non-ICM, we sought to determine the impact of ICM on event-free survival (EFS), defined as death, progressive disease, or unplanned treatment escalation. We performed a retrospective study of newly diagnosed MM patients who collected autologous PBSCs with the intent to proceed immediately to ASCT at our center between 7/2020 and 2/2023. Patients were excluded if they underwent a tandem autologous or sequential autologous-allogeneic transplant, had primary PC leukemia, received non-ICM treatment (i.e., cyclophosphamide and/or etoposide), or had previously failed G ± P mobilization. To appropriately evaluate the impact of ICM among those who potentially could have received it, we utilized a propensity score matching (PSM) approach whereby ICM patients were compared to a cohort of non-CM patients matched on pre-ASCT factors most strongly associated with the receipt of ICM. Of 451 patients identified, 61 (13.5%) received ICM (PACE-based, n = 45; hyper-CVAD-based, n = 16). Post-ICM/pre-ASCT, 11 patients (18%) required admission for neutropenic fever and/or infection. Among 51 evaluable patients, the overall response rate was 31%; however, 46 of 55 evaluable patients (84%) saw a reduction in M-spike and/or involved free light chains. Among those evaluated with longitudinal peripheral blood flow cytometry (n = 8), 5 patients (63%) cleared circulating blood PCs post-ICM. Compared to patients mobilized with non-CM, ICM patients collected a slightly greater median number of CD34+ cells (10.8 versus 10.2 × 10⁶/kg, P = .018). The median follow-up was 30.6 months post-ASCT. In a PSM multivariable analysis, ICM was associated with significantly improved EFS (hazard ratio [HR] 0.30, 95% CI 0.14 to 0.67, P = .003), but not improved OS (HR 0.38, 95% CI 0.10 to 1.44, P = .2). ICM was associated with longer post-ASCT inpatient duration (+4.1 days, 95% CI, 2.4 to 5.8, P < .001), more febrile days (+0.96 days, 95% CI 0.50 to 1.4, P < .001), impaired platelet engraftment (HR 0.23, 95% CI 0.06 to 0.87, P = .031), more bacteremia (OR 3.41, 95% CI 1.20 to 9.31, P = .018), and increased antibiotic usage (cefepime: +2.3 doses, 95% CI 0.39 to 4.1, P = .018; vancomycin: +1.0 doses, 95% CI 0.23 to 1.8, P = .012). ICM was independently associated with improved EFS in a matched analysis involving MM patients with excessive disease burden at pre-ASCT workup. This benefit came at the cost of longer inpatient duration, more febrile days, greater incidence of bacteremia, and increased antibiotic usage in the immediate post-ASCT setting. Our findings suggest that ICM could be considered for a subset of MM patients, but its use must be weighed carefully against additional toxicity.

Optimal refinement of strata to balance covariates

What is the best way to split one stratum into two to maximally reduce the within-stratum imbalance in many covariates? We formulate this as an integer program and approximate the solution by randomized rounding of a linear program. A linear program may assign a fraction of a person to each refined stratum. Randomized rounding views fractional people as probabilities, assigning intact people to strata using biased coins. Randomized rounding is a well-studied theoretical technique for approximating the optimal solution of certain insoluble integer programs. When the number of people in a stratum is large relative to the number of covariates, we prove the following new results: (i) randomized rounding to split a stratum does very little randomizing, so it closely resembles the linear programming relaxation without splitting intact people; (ii) the linear relaxation and the randomly rounded solution place lower and upper bounds on the unattainable integer programming solution; and because of (i), these bounds are often close, thereby ratifying the usable randomly rounded solution. We illustrate using an observational study that balanced many covariates by forming matched pairs composed of 2016 patients selected from 5735 using a propensity score. Instead, we form 5 propensity score strata and refine them into 10 strata, obtaining excellent covariate balance while retaining all patients. An R package optrefine at CRAN implements the method. Supplementary materials are available online.

Impact of English proficiency on use of Glasgow Coma Scale in geriatric patients with traumatic brain injury

Background

The relationship between English proficiency (EP), Glasgow Coma Scale (GCS) and traumatic brain injury (TBI) is not well characterized. We aimed to understand the impact of limited English proficiency (LEP) on the evaluation and outcomes of TBI.

Methods

Retrospective comparative study in a single institution of patients aged ⪰65 who presented to the emergency department after a fall with head strike between January 2018 and December 2021. TBI was defined as documented loss of consciousness or intracranial hemorrhage (ICH). Relationships between EP, GCS, and TBI were analyzed with multivariable and propensity score-matched models.

Results

Of the 2905 included, 1233 (42%) had LEP. Most LEP patients were Asian (60%) while the majority of EP patients were non-Hispanic Caucasians (72%). In a univariate analysis, LEP had higher incidence of decreased GCS and was strongly correlated with risk of TBI (OR 1.47, CI 1.26 to 1.71). After adjusting for multiple covariates including race, LEP did not have a significantly increased risk for GCS score <13 (OR 1.66, CI 0.99 to 2.76) or increased risk of TBI. In the matched analysis, LEP had a small but significantly higher risk of GCS score <13 (OR 1.03, CI 1.02 to 1.05) without an increased risk in TBI. Decreased GCS remained strongly correlated with presence of ICH in LEP patients in the adjusted model (OR 1.39, CI 1.30 to 1.50).

Conclusions

LEP correlated with lower GCS in geriatric patients with TBI. This association weakened after adjusting for factors like race, suggesting racial disparities may have more influence than language differences. Moreover, GCS remained effective for predicting ICH in LEP individuals, highlighting its value with suitable translation resources.

Level of evidence

This is a Level III evidence restrospective comparative study.

Causal Effects of High Stress Assessed Via Interviews on Mental and Physical Health: Toward Computer Agent-Driven Stress Assessment

OBJECTIVES

This study investigated the causal effect of high stress assessment via an interview on the mental and physical health of workers 1 month later.

METHODS

Stress assessment interviews and feedback were conducted with 50 Japanese workers. In addition to the interviewer, two occupational health professionals assessed participants' stress based on recordings. The average treatment effect was estimated by propensity score matching.

RESULTS

High stress, according to the interview-based assessment, had a significant negative causal effect on self-reported well-being 1 month later (95% confidence interval: -3.02, -1.10). In addition, no effect of high stress on stress load, mental and physical symptoms, or burnout was observed.

CONCLUSIONS

This study provides important insights into the prognosis of individuals who were assessed through interviews to have high stress. The findings are expected to help automate stress assessments using computer agents.

Treatment for Insomnia Symptoms is Associated with Reduced Depression Among Older Adults: A Propensity Score Matching Approach

OBJECTIVES

The study aimed to investigate the effect of utilization of treatment for insomnia symptoms on the prevalence of major depressive disorder among older adults in India.

METHODS

We used the data from the Longitudinal Ageing Study in India (LASI), 2017-18. The sample included 10,911 older individuals who reported insomnia symptoms. The propensity score matching (PSM) approach was used to compare the depressive disorder among those who received vs. not received treatment.

RESULTS

Only 5.7% of older adults reporting insomnia symptoms received treatment. On average, prevalence of depressive disorder among men and women who received treatment for insomnia symptoms was lesser by 0.79 and 0.33 points, respectively, than those who did not receive treatment. In the matched sample, treatment for insomnia symptoms was significantly associated with lesser prevalence of depression for both older men (β= -0.68, p < .001) and older women (β= -0.62, p < .001).

CONCLUSIONS

The current findings suggest that treatment for insomnia symptoms can reduce the risk of depressive disorder among older adults and the effects are higher among older men than women.

Effects of an Animation-Based Comprehensive Sexuality Education Package: A 2-Year Repeated Intervention Study

BACKGROUND

Preadolescents, who are in a transitional phase of development, may experience higher exposure to heterosexual interactions while facing higher risks regarding misinformation in sexual knowledge and unsafe engagement in sexual activities. There is a deficiency in the availability of qualified educators and age-appropriate teaching materials for sexuality education in China.

METHODS

We implemented an animation-based comprehensive sexuality education package among preadolescents aged 9 to 12 years from eight schools in Anhui, China. The first round of intervention included 1,835 participants, lasting 2 months from September to November 2020. A total of 374 participants, accounting for 52% of the intervention group, received a second round of intervention in September 2021. Participants completed immediate follow-up assessment and 1-year follow-up assessment to assess changes in their sexual knowledge, attitudes, and other outcomes. Propensity score matching and difference-in-difference analysis were performed to determine the short- and long-term impacts.

RESULTS

Significant improvements were observed for both sexual knowledge and sexual attitudes in the immediate follow-up. There was no significant effect on pornography-seeking behavior or awareness of experiencing sexual abuse. After 1 year, the effect was sustained for sexual knowledge, but slightly declined for sexual attitudes. The second intervention significantly improved sexual knowledge; however, no significant change in sexual attitudes, pornography-seeking behavior, or awareness of experiencing sexual abuse was observed.

CONCLUSIONS

Our comprehensive sexuality education package was effective in improving sexual knowledge both immediately and 1 year after the intervention. Repeated intervention can be an effective strategy for promoting preadolescent health development regarding comprehensive sexuality education.

Impact of weather and climate advisories on agricultural outcomes in Pakistan

The earth is warming, and the frequency of extreme weather events have been rapidly growing globally with unprecedented consequences to farming communities. In principle, weather and climate information services (WCIS) can help farmers better manage their activities and farm level outcomes by supporting their decision-making with relevant and usable information to address the potential impacts of expected changing climate conditions. But, in practice, can the availability and use of WCIS help improve agricultural outcomes given the weather and climate related uncertainties? To understand the use and impact of WCIS in the cotton-wheat cropping areas of Pakistan, we conducted a multistage stratified clustered random sample of 612 farm households in the provinces of Punjab and Sindh. Over 55% of the farm households in the sample indicated that they used WCIS provided by the Pakistan Meteorological Department and other sources for making their agricultural decisions. Our analysis, however, suggests that the impact of using WCIS on major farm outcomes (i.e. farm revenue, costs, profits, and input usage) is not statistically significant when compared with those farm households not using WCIS (null result). This result is robust to different estimation techniques (i.e. ordinary least squares, instrumental variable approach, and propensity score matching method). From the focus group discussions, we gathered that farmers perceived WCIS as less reliable, often unclear, and difficult to understand as this information is not translated and transmitted in local languages. Addressing these issues can help reduce the impact of climate and weather variability on farm outcomes in those provinces as well as in Pakistan more generally. Our study suggests that, under uncertainty, emphasis should be on WCIS that farmers can rely on for making farming related decisions.

Impact of the matching algorithm on the treatment effect estimate: A neutral comparison study

Propensity score matching is increasingly being used in the medical literature. Choice of matching algorithms, reporting quality, and estimands are oftentimes not discussed. We evaluated the impact of propensity score matching algorithms, based on a recent clinical dataset, with three commonly used outcomes. The resulting estimands for different strengths of treatment effects were compared in a neutral comparison study and based on a thoroughly designed simulation study. Different algorithms yielded different levels of balance after matching. Along with full matching and genetic matching with replacement, good balance was achieved with nearest neighbor matching with caliper but thereby more than one fifth of the treated units were discarded. Average marginal treatment effect estimates were least biased with genetic or nearest neighbor matching, both with replacement and full matching. Double adjustment yielded conditional treatment effects that were closer to the true values, throughout. The choice of the matching algorithm had an impact on covariate balance after matching as well as treatment effect estimates. In comparison, genetic matching with replacement yielded better covariate balance than all other matching algorithms. A literature review in the British Medical Journal including its subjournals revealed frequent use of propensity score matching; however, the use of different matching algorithms before treatment effect estimation was only reported in one out of 21 studies. Propensity score matching is a methodology for causal treatment effect estimation from observational data; however, the methodological difficulties and low reporting quality in applied medical research need to be addressed.

Testing weak nulls in matched observational studies

We develop sensitivity analyses for the sample average treatment effect in matched observational studies while allowing unit-level treatment effects to vary. The methods may be applied to studies using any optimal without-replacement matching algorithm. In contrast to randomized experiments and to paired observational studies, we show for general matched designs that over a large class of test statistics, any procedure bounding the worst-case expectation while allowing for arbitrary effect heterogeneity must be unnecessarily conservative if treatment effects are actually constant across individuals. We present a sensitivity analysis which bounds the worst-case expectation while allowing for effect heterogeneity, and illustrate why it is generally conservative if effects are constant. An alternative procedure is presented that is asymptotically sharp if treatment effects are constant, and that is valid for testing the sample average effect under additional restrictions which may be deemed benign by practitioners. Simulations demonstrate that this alternative procedure results in a valid sensitivity analysis for the weak null hypothesis under a host of reasonable data-generating processes. The procedures allow practitioners to assess robustness of estimated sample average treatment effects to hidden bias while allowing for effect heterogeneity in matched observational studies.

Oversampling and replacement strategies in propensity score matching: a critical review focused on small sample size in clinical settings

Background

Propensity score matching is a statistical method that is often used to make inferences on the treatment effects in observational studies. In recent years, there has been widespread use of the technique in the cardiothoracic surgery literature to evaluate to potential benefits of new surgical therapies or procedures. However, the small sample size and the strong dependence of the treatment assignment on the baseline covariates that often characterize these studies make such an evaluation challenging from a statistical point of view. In such settings, the use of propensity score matching in combination with oversampling and replacement may provide a solution to these issues by increasing the initial sample size of the study and thus improving the statistical power that is needed to detect the effect of interest. In this study, we review the use of propensity score matching in combination with oversampling and replacement in small sample size settings.

Methods

We performed a series of Monte Carlo simulations to evaluate how the sample size, the proportion of treated, and the assignment mechanism affect the performances of the proposed approaches. We assessed the performances with overall balance, relative bias, root mean squared error and nominal coverage. Moreover, we illustrate the methods using a real case study from the cardiac surgery literature.

Results

Matching without replacement produced estimates with lower bias and better nominal coverage than matching with replacement when 1:1 matching was considered. In contrast to that, matching with replacement showed better balance, relative bias, and root mean squared error than matching without replacement for increasing levels of oversampling. The best nominal coverage was obtained by using the estimator that accounts for uncertainty in the matching procedure on sets of units obtained after matching with replacement.

Conclusions

The use of replacement provides the most reliable treatment effect estimates and that no more than 1 or 2 units from the control group should be matched to each treated observation. Moreover, the variance estimator that accounts for the uncertainty in the matching procedure should be used to estimate the treatment effect.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01454-z.

Comparison of statistical methodologies used to estimate the treatment effect on time-to-event outcomes in observational studies

The use of real-world data became more and more popular in the pharmaceutical industry. The impact of real-world evidence is now well emphasized by the regulatory authorities. Indeed, the analysis of this type of data can play a key role for treatment efficacy and safety. The aim of this work is to assess various methods and give guidance on the comparisons of drugs, mostly with respect to time-to-event data, in non-randomized studies with potentially confounding variables. For that purpose, several statistical methodologies are compared based on simulation studies. These methodologies belong to family classes of methods that are widely used for this type of problem: regression, matching, weighting and subclassification methods. The evaluation criteria used to compare methods performances are the relative bias, the mean square error, the coverage probability and the width of the confidence interval. In this paper, we consider different scenarios of dataset features in order to study the effect of the sample size, the number of covariates and the magnitude of the treatment effect on the statistical methodologies performances. These statistical analyses are conducted within a proportional hazard model framework. Furthermore, we highlight the advantage of using techniques to identify relevant covariates for time-to-event outcomes by comparing two variable selection methods under a frequentist and a Bayesian inference. Based on simulation results, recommendations on each of the family of methods are provided to guide decision making.

Empirical use of causal inference methods to evaluate survival differences in a real-world registry vs those found in randomized clinical trials

With heighted interest in causal inference based on real-world evidence, this empirical study sought to understand differences between the results of observational analyses and long-term randomized clinical trials. We hypothesized that patients deemed "eligible" for clinical trials would follow a different survival trajectory from those deemed "ineligible" and that this factor could partially explain results. In a large observational registry dataset, we estimated separate survival trajectories for hypothetically trial-eligible vs ineligible patients under both coronary artery bypass surgery (CABG) and percutaneous coronary intervention (PCI). We also explored whether results would depend on the causal inference method (inverse probability of treatment weighting vs optimal full propensity matching) or the approach to combine propensity scores from multiple imputations (the "across" vs "within" approaches). We found that, in this registry population of PCI/CABG multivessel patients, 32.5% would have been eligible for contemporaneous RCTs, suggesting that RCTs enroll selected populations. Additionally, we found treatment selection bias with different distributions of propensity scores between PCI and CABG patients. The different methodological approaches did not result in different conclusions. Overall, trial-eligible patients appeared to demonstrate at least marginally better survival than ineligible patients. Treatment comparisons by eligibility depended on disease severity. Among trial-eligible three-vessel diseased and trial-ineligible two-vessel diseased patients, CABG appeared to have at least a slight advantage with no treatment difference otherwise. In conclusion, our analyses suggest that RCTs enroll highly selected populations, and our findings are generally consistent with RCTs but less pronounced than major registry findings.

The Effect of National Family Caregiver Support Program Services on Caregiver Burden

Extant evidence on the effectiveness of caregiver programs in alleviating caregiver burden is mixed, underscoring the need for further investigations. This study evaluated the effect of the National Family Caregiver Support Program (NFCSP) educational services and respite care on caregiver burden. We used survey data from caregivers assigned to program (n = 491) or comparison (n = 417) group based on their reported use of NFCSP services. Adjusted difference-in-differences (DiD) analysis found an increase in mean burden scores for both groups from baseline to 6 or 12 months. Among program caregivers receiving ≥4 hr of NFCSP respite care per week (n = 307) and matched comparisons (n = 370), burden scores decreased slightly for program caregivers (-0.095 points), but increased for comparison caregivers (+0.145 points). The DiD (0.239 points) was not statistically significant. More research is needed to determine the minimum amount of respite care needed to positively impact caregiver burden.

A Bayesian procedure for estimating the causal effects of nursing home bed-hold policy

Nursing home bed-hold policies provide continuity of care for Medicaid beneficiaries by paying nursing homes to reserve beds so residents can return to their facility of occupancy following an acute hospitalization. In 2001, Michigan implemented bed-hold policies in nursing homes. We investigated the impact of these policies on mortality and hospitalizations using 1999-2004 quarterly data from nursing homes in Michigan and nursing homes in 11 states that did not implement such policies. Synthetic Control has been used to estimate the effects of policies by accounting for changes over time unrelated to the intervention. Synthetic Control is intended for scalar continuous outcome at each period, and assumes a single treated unit and multiple control units. We propose a Bayesian procedure to overcome these limitations. It imputes the outcomes of nursing homes in Michigan if they were not exposed to the policy by matching to non-exposed nursing homes that are associated with the exposed ones in the pre-policy period. Because sampling from a Bayesian model is computationally challenging, we describe an approximation procedure that can be implemented using existing software. Our approach can be applied to other studies that examine the impact of policies.

After the gun: examining police visits and intimate partner violence following incidents involving a firearm

Laws have been enacted to keep firearms out of the hands of abusers. In this study, we examined one such effort-removal of a firearm at the scene of intimate partner violence (IPV)-to assess the subsequent occurrence and number of IPV incidents responded to by police and subsequent risk of injury to the victim. Using the 28,977 IPV calls in one large U.S. city to which officers responded during the 2013 calendar year, we identified 220 first-time incidents in which offenders used (i.e., brandished, pistol whipped, shot) a pistol, revolver, rifle, or shotgun. Officers reported removing a firearm from 52 (24%) of the offenders. After using full propensity score matching to control for potential confounders, logistic and Poisson regressions were used to assess differences between those from whom a firearm was removed and those whose firearm was not removed. Firearm removal at the scene of an IPV incident appears to increase the likelihood of subsequent IPV reports to police and suggestive evidence that subsequent injury to the victim might increase as well. The offender shifting from threats with a firearm to physical violence and a change (an increase as well as a decrease) in victim willingness to summon police may account for the findings.

Statistical evaluation of drug efficacy for bridging study in companion diagnostic test trials

Personalized medicine is an area of growing attention in medical research and practice. A market-ready companion diagnostic test (CDx) is used in personalized medicine for identifying the best treatment for an individual patient. Unfortunately, development of CDx may lag behind the development of the drug, and consequently we use a clinical trial assay (CTA) to enroll patients into the drug pivotal clinical trial instead. Thus, when CDx becomes available, a bridging study will be required to assess the drug efficacy in the CDx intended use (CDx IU) population. Due to missingness of the CDx results that could be associated with randomization, one challenge we face in a bridging study is covariate imbalance between treatment arms for the subpopulation with both positive CDx and CTA. In this paper, we evaluate the performance of two methods in bridging studies under a causal inference framework. Particularly, we aim to use the propensity score method with doubly robust estimation and optimal matching to address the challenge. We extend under a current framework on drug efficacy estimation in the CDx IU population, using data from both the bridging study and the CTA drug pivotal clinical trial. Both approaches are discussed in the context of a randomized bridging study, and a targeted design clinical trial with simulations, followed by analyzing simulated data that mimic a real ongoing clinic trial.

Examining the Consequences of the "Prevalent Life Events" of Arrest and Incarceration among an Urban African-American Cohort

The life course perspective has traditionally examined prevalent adult life events, such as marriage and employment, and their potential to redirect offending trajectories. However, for African Americans, the life events of arrest and incarceration are becoming equally prevalent in young adulthood. Therefore, it is critical to understand how these "standard" criminal justice practices, which are designed to deter as well as punish, affect deviance among this population. This study evaluates the long-term consequences of criminal justice intervention on substance use and offending into midlife among an African American community cohort using propensity score matching and multivariate regression analyses. The results largely point to a criminogenic effect of criminal justice intervention on midlife deviance with a particularly strong effect of young adult arrest on rates of violent and property arrest counts into midlife. The theoretical and policy implications of the findings are discussed.

A Systematic Review of Propensity Score Methods in the Social Sciences

The use of propensity scores in psychological and educational research has been steadily increasing in the last 2 to 3 years. However, there are some common misconceptions about the use of different estimation techniques and conditioning choices in the context of propensity score analysis. In addition, reporting practices for propensity score analyses often lack important details that allow other researchers to confidently judge the appropriateness of reported analyses and potentially to replicate published findings. In this article we conduct a systematic literature review of a large number of published articles in major areas of social science that used propensity scores up until the fall of 2009. We identify common errors in estimation, conditioning, and reporting of propensity score analyses and suggest possible solutions.

Matching methods for causal inference: A review and a look forward

When estimating causal effects using observational data, it is desirable to replicate a randomized experiment as closely as possible by obtaining treated and control groups with similar covariate distributions. This goal can often be achieved by choosing well-matched samples of the original treated and control groups, thereby reducing bias due to the covariates. Since the 1970's, work on matching methods has examined how to best choose treated and control subjects for comparison. Matching methods are gaining popularity in fields such as economics, epidemiology, medicine, and political science. However, until now the literature and related advice has been scattered across disciplines. Researchers who are interested in using matching methods-or developing methods related to matching-do not have a single place to turn to learn about past and current research. This paper provides a structure for thinking about matching methods and guidance on their use, coalescing the existing research (both old and new) and providing a summary of where the literature on matching methods is now and where it should be headed.

ERROR-FREE MILESTONES IN ERROR PRONE MEASUREMENTS

A predictor variable or dose that is measured with substantial error may possess an error-free milestone, such that it is known with negligible error whether the value of the variable is to the left or right of the milestone. Such a milestone provides a basis for estimating a linear relationship between the true but unknown value of the error-free predictor and an outcome, because the milestone creates a strong and valid instrumental variable. The inferences are nonparametric and robust, and in the simplest cases, they are exact and distribution free. We also consider multiple milestones for a single predictor and milestones for several predictors whose partial slopes are estimated simultaneously. Examples are drawn from the Wisconsin Longitudinal Study, in which a BA degree acts as a milestone for sixteen years of education, and the binary indicator of military service acts as a milestone for years of service.

Interval estimation for treatment effects using propensity score matching

In causal studies without random assignment of treatment, causal effects can be estimated using matched treated and control samples, where matches are obtained using estimated propensity scores. Propensity score matching can reduce bias in treatment effect estimators in cases where the matched samples have overlapping covariate distributions. Despite its application in many applied problems, there is no universally employed approach to interval estimation when using propensity score matching. In this article, we present and evaluate approaches to interval estimation when using propensity score matching.

Maternal employment and child development: a fresh look using newer methods

The employment rate for mothers with young children has increased dramatically over the past 25 years. Estimating the effects of maternal employment on children's development is challenged by selection bias and the missing data endemic to most policy research. To address these issues, this study uses propensity score matching and multiple imputation. The authors compare outcomes across 4 maternal employment patterns: no work in first 3 years postbirth, work only after 1st year, part-time work in 1st year, and full-time work in 1st year. Our results demonstrate small but significant negative effects of maternal employment on children's cognitive outcomes for full-time employment in the 1st year postbirth as compared with employment postponed until after the 1st year. Multiple imputation yields noticeably different estimates as compared with a complete case approach for many measures. Differences between results from propensity score approaches and regression modeling are often minimal.