Do health care quality improvement policies work for all? Distributional effects by baseline quality in South Africa

Health care quality improvement (QI) initiatives are being implemented by a number of low- and middle-income countries. However, there is concern that these policies may not reduce, or may even worsen, inequities in access to high-quality care. Few studies have examined the distributional impact of QI programmes. We study the Ideal Clinic Realization and Maintenance program implemented in health facilities in South Africa, assessing whether the effects of the program are sensitive to previous quality performance. Implementing difference-in-difference-in-difference and changes-in-changes approaches we estimate the effect of the program on quality across the distribution of past facility quality performance. We find that the largest gains are realized by facilities with higher baseline quality, meaning this policy may have led to a worsening of pre-existing inequity in health care quality. Our study highlights that the full consequences of QI programmes cannot be gauged solely from examination of the mean impact.

A second evidence factor for a second control group

In an observational study of the effects caused by a treatment, a second control group is used in an effort to detect bias from unmeasured covariates, and the investigator is content if no evidence of bias is found. This strategy is not entirely satisfactory: two control groups may differ significantly, yet the difference may be too small to invalidate inferences about the treatment, or the control groups may not differ yet nonetheless fail to provide a tangible strengthening of the evidence of a treatment effect. Is a firmer conclusion possible? Is there a way to analyze a second control group such that the data might report measurably strengthened evidence of cause and effect, that is, insensitivity to larger unmeasured biases? Evidence factor analyses are not commonly used with a second control group: most analyses compare the treated group to each control group, but analyses of that kind are partially redundant; so, they do not constitute evidence factors. An alternative analysis is proposed here, one that does yield two evidence factors, and with a carefully designed test statistic, is capable of extracting strong evidence from the second factor. The new technical work here concerns the development of a test statistic with high design sensitivity and high Bahadur efficiency in a sensitivity analysis for the second factor. A study of binge drinking as a cause of high blood pressure is used as an illustration.

Effectiveness of community hospital post-acute care on mortality, re-admission, institutionalization, and activation of a home care programme in Emilia-Romagna region, Italy

BACKGROUND

In Italy, there is scant evidence on the impact of Community Hospitals (CHs) on clinical outcomes.

AIMS

To assess the effectiveness of CHs versus long-term care hospital or inpatient rehabilitation facilities on mortality, re-admission, institutionalization, and activation of a home care programme in the Emilia-Romagna Region (ERR-Italy) after acute hospitalisation.

METHODS

We implemented a cohort study drawing upon the ERR Administrative Healthcare Database System and including hospital episodes of ERR residents subject ≥ 65 years, discharged from a public or private hospital with a medical diagnosis to a CH or to usual care between 2017 and 2019. To control for confounding, we applied a propensity score matching.

RESULTS

Patients transferred to CHs had a significantly lower risk of dying but an increased risk of being readmitted to community or acute hospital within 30/90 days from discharge. The hazard of institutionalisation within 30/90 days was significantly lower in the whole population of the CH exposed group but not among patients with cardiac or respiratory chronic diseases or diabetes. The activation of a home care program within 90 days was slightly higher for those who were transferred to a CH.

DISCUSSION

The findings of our study show mixed effects on outcomes of patients transferred to CHs compared to those who followed the post-acute usual care and should be taken with cautious as could be affected by the so-called 'confounding by indication'.

CONCLUSIONS

The study contributes to the intermediate care available evidence from a region with a well-established care provision through CHs.

The clinical impact of cirrhosis on the postoperative outcomes of patients undergoing bariatric surgery: propensity score-matched analysis of 2011-2017 US hospitals

Objectives: Since there is increasing number of patients with cirrhosis who require the bariatric procedure due to obesity and obesity-related nonalcoholic steatohepatitis fibrosis, we evaluate the effect of cirrhosis on post-bariatric surgery outcomes.Methods: 2011-2017 National Inpatient Sample was used to isolate bariatric cases, which were stratified by cirrhosis; controls were propensity-score matched to cases and compared to endpoints: mortality, length of stay (LOS), costs, and postoperative complications.Results: From 190,753 patients undergoing bariatric surgery, there were 957 with cirrhosis and 957 matched controls. There was no difference in mortality (0.94 vs 0.52% p = 0.42, OR 1.81 95%CI 0.60-5.41); however, cirrhosis patients had higher LOS (3.36 vs 2.89d p = 0.002), costs ($68,671 vs $61,301 p < 0.001), and bleeding (2.09 vs 0.72% p < 0.001, OR 2.95 95%CI 1.89-4.61). In multivariate, there was no difference in mortality (p = 0.330, aOR 1.73 95%CI 0.58-5.19). In subgroup comparison of cirrhosis patients, those with decompensated cirrhosis had higher mortality (7.69 vs 0.94% p < 0.001, OR 8.78 95%CI 3.41-22.59).Conclusion: The results of this study show compensated cirrhosis does not pose an increased risk toward post-bariatric surgery mortality; however, hepatic decompensation increases the postsurgical risks.

Ensuring exchangeability in data-based priors for a Bayesian analysis of clinical trials

In many orphan diseases and pediatric indications, the randomized controlled trials may be infeasible because of their size, duration, and cost. Leveraging information on the control through a prior can potentially reduce sample size. However, unless an objective prior is used to impose complete ignorance for the parameter being estimated, it results in biased estimates and inflated type-I error. Hence, it is essential to assess both the confirmatory and supplementary knowledge available during the construction of the prior to avoid "cherry-picking" advantageous information. For this purpose, propensity score methods are employed to minimize selection bias by weighting supplemental control subjects according to their similarity in terms of pretreatment characteristics to the subjects in the current trial. The latter can be operationalized through a proposed measure of overlap in propensity-score distributions. In this paper, we consider single experimental arm in the current trial and the control arm is completely borrowed from the supplemental data. The simulation experiments show that the proposed method reduces prior and data conflict and improves the precision of the of the average treatment effect.

Assessing the impact of unmeasured confounders for credible and reliable real-world evidence

PURPOSE

We review statistical methods for assessing the possible impact of bias due to unmeasured confounding in real world data analysis and provide detailed recommendations for choosing among the methods.

METHODS

By updating an earlier systematic review, we summarize modern statistical best practices for evaluating and correcting for potential bias due to unmeasured confounding in estimating causal treatment effect from non-interventional studies.

RESULTS

We suggest a hierarchical structure for assessing unmeasured confounding. First, for initial sensitivity analyses, we strongly recommend applying a recently developed method, the E-value, that is straightforward to apply and does not require prior knowledge or assumptions about the unmeasured confounder(s). When some such knowledge is available, the E-value could be supplemented by the rule-out or array method at this step. If these initial analyses suggest results may not be robust to unmeasured confounding, subsequent analyses could be conducted using more specialized statistical methods, which we categorize based on whether they require access to external data on the suspected unmeasured confounder(s), internal data, or no data. Other factors for choosing the subsequent sensitivity analysis methods are also introduced and discussed, including the types of unmeasured confounders and whether the subsequent sensitivity analysis is intended to provide a corrected causal treatment effect.

CONCLUSION

Various analytical methods have been proposed to address unmeasured confounding, but little research has discussed a structured approach to select appropriate methods in practice. In providing practical suggestions for choosing appropriate initial and, potentially, more specialized subsequent sensitivity analyses, we hope to facilitate the widespread reporting of such sensitivity analyses in non-interventional studies. The suggested approach also has the potential to inform pre-specification of sensitivity analyses before executing the analysis, and therefore increase the transparency and limit selective study reporting.

Pharyngeal dysfunction associated with early and late onset sleep disordered breathing in children

OBJECTIVE

To compare the frequency and type of diagnoses associated with pharyngeal dysfunction (PD) in children presenting with early versus late onset sleep disordered breathing (SDB).

METHODS

This was a retrospective, cross-sectional study. A consecutive series of children ≤3 years old who underwent management for SDB were retrospectively identified from a prospectively kept surgical database. The early onset group was compared with two separate late onset (≥4years old) groups. Diagnoses associated with PD included gastroesophageal reflux disease (GERD), swallowing dysfunction, prematurity, asthma, and obesity. Distribution of PD diagnoses, airway lesions, syndromic conditions, pulse oximetry scores, and endoscopic pattern of airway obstruction were compared.

RESULTS

73 patients with early onset SDB were identified (51 boys, mean age 2.25 ± 0.64 years, range 1.75-3 years) and compared with two groups of later onset SDB consisting of 75 and 72 patients with mean ages of 7.58 ± 2.40 years and 8.04 ± 3.34 years respectively (range 4-16 years). The early onset SDB group had a higher prevalence of PD diagnoses compared to the later onset group with 35 of 73 patients being diagnosed compared to 41 of 147 children (p = 0.01). Early onset SDB patients were more likely to have GERD or swallowing dysfunction (p < 0.01) while later onset patients more commonly presented with associated asthma or obesity (p < 0.01). There was no statistically significant difference in airway lesions between groups.

CONCLUSION

Early-onset SDB is associated with conditions causing PD more often than later-onset SDB. Identifying these conditions and optimizing their management may impact outcomes in treating pediatric SDB.

Advance Care Planning Claims and Health Care Utilization Among Seriously Ill Patients Near the End of Life

IMPORTANCE

Although advance care planning is known to increase patient and caregiver satisfaction, its association with health care utilization is not well understood.

OBJECTIVE

To examine the association between billed advance care planning encounters and subsequent health care utilization among seriously ill patients.

DESIGN, SETTING, AND PARTICIPANTS

This retrospective cohort study conducted from October 1, 2015, to May 31, 2018, used a national commercial insurance claims database to retrieve data from 18 484 Medicare Advantage members 65 years or older who had a claim that contained a serious illness diagnosis.

EXPOSURE

A claim that contained an advance care planning billing code between October 1, 2016, and November 30, 2017.

MAIN OUTCOMES AND MEASURES

Receipt of intensive therapies, hospitalization, emergency department use, hospice use, costs, and death during the 6-month follow-up period.

RESULTS

The final study sample included 18 484 seriously ill patients (mean [SD] age, 79.7 [7.9] years; 10 033 [54.3%] female), 864 (4.7%) of whom had a billed advanced care planning encounter between October 1, 2016, and November 30, 2017. In analyses adjusted for patient characteristics and a propensity score for advance care planning, the presence of a billed advance care planning encounter was associated with a higher likelihood of hospice enrollment (incidence rate ratio [IRR], 2.52; 95% CI, 2.22-2.86) and mortality (hazard ratio, 2.27; 95% CI, 1.79-2.88) compared with no billed advance care planning encounter. Although patients with billed advance care planning encounters were also more likely to be hospitalized (IRR, 1.37; 95% CI, 1.26-1.49), including in the intensive care unit (IRR, 1.25; 95% CI, 1.08-1.45), they were less likely to receive any intensive therapies (IRR, 0.85; 95% CI, 0.78-0.92), such as chemotherapy (IRR, 0.65; 95% CI, 0.55-0.78). Similar results were observed in a propensity score-matched analysis (99% matched) and in a decedent analysis of patients who died during the 6-month follow-up period.

CONCLUSIONS AND RELEVANCE

Patients with billed advance care planning encounters were more likely than those without these encounters to receive hospice services and less likely to receive any intensive therapies, such as chemotherapy. However, they were also hospitalized more frequently than patients without billed advance care planning encounters. Although these findings were robust to multiple analytic methods, the results may be attributable to residual confounding because of a higher unmeasured severity of illness in the advance care planning group. Additional evidence appears to be needed to understand the effect of advance care planning on these outcomes.

Evaluating Missouri's Handgun Purchaser Law: A Bracketing Method for Addressing Concerns About History Interacting with Group

In the comparative interrupted time series design (also called the method of difference-in-differences), the change in outcome in a group exposed to treatment in the periods before and after the exposure is compared with the change in outcome in a control group not exposed to treatment in either period. The standard difference-in-difference estimator for a comparative interrupted time series design will be biased for estimating the causal effect of the treatment if there is an interaction between history in the after period and the groups; for example, there is a historical event besides the start of the treatment in the after period that benefits the treated group more than the control group. We present a bracketing method for bounding the effect of an interaction between history and the groups that arises from a time-invariant unmeasured confounder having a different effect in the after period than the before period. The method is applied to a study of the effect of the repeal of Missouri's permit-to-purchase handgun law on its firearm homicide rate. We estimate that the effect of the permit-to-purchase repeal on Missouri's firearm homicide rate is bracketed between 0.9 and 1.3 homicides per 100,000 people, corresponding to a percentage increase of 17% to 27% (95% confidence interval: 0.6, 1.7 or 11%, 35%). A placebo study provides additional support for the hypothesis that the repeal has a causal effect of increasing the rate of state-wide firearm homicides.

How much do conditional cash transfers increase the utilization of maternal and child health care services? New evidence from Janani Suraksha Yojana in India

Janani Suraksha Yojana (safe motherhood scheme, or JSY) provides cash incentives to marginal pregnant women in India conditional on having mainly institutional delivery. Using the fourth round of district level household survey (DLHS-4), we have estimated its effects on both intended and unintended outcomes. Our estimates of average treatment effect on the treated (ATT) from propensity score matching are remarkably higher than those found in previous prominent studies using the second and third rounds of the survey (DLHS-2 and DLHS-3). When we apply fuzzy regression discontinuity design exploiting the second birth order, our estimates of local average treatment effect (LATE) are much higher than that of ATT. For example, due to JSY, institutional delivery increases by around 16 percentage points according to ATT estimate but about 23 percentage points according to LATE estimate.

Unmeasured Confounding in Observational Studies with Multiple Treatment Arms: Comparing Emergency Department Mortality of Severe Trauma Patients by Trauma Center Level

Comparing emergency department mortality across different levels of trauma care (nontrauma centers, level I and II centers) is important in evaluating regionalized care. Patient population characteristics differ across different levels of trauma care and it is essential to adjust for baseline covariates to make valid comparisons. Propensity score matching has been established as a more robust method to infer causal relationship in observational studies than conventional regression adjustment. We designed and implemented a three group matching methodology. First, we conducted optimal pair matching between the treatment group (nontrauma centers) and the first control group (level I trauma centers); second, we conducted optimal pair matching between the nontrauma centers and the second control group (level II trauma centers); the final step was to link the two sets of matched pairs by the common treatment subjects to form matched triplets. We then implemented a sensitivity analysis with three treatment arms, Lu's imputation based method, to assess the impact due to potential unmeasured confounding. The results showed that if the most severe adult trauma patients treated in nontrauma centers were to be treated in level I or II trauma centers, the odds of emergency department death would be reduced dramatically (odds ratio [OR] = 0.61 and 0.49). There was no difference between being treated in level I or II trauma centers. The sensitivity analysis revealed that the positive association between being treated at level I or II trauma centers and the reduced odds of mortality would remain present even in the presence of strong unmeasured confounding.

Evidence from a natural experiment that malaria parasitemia is pathogenic in retinopathy-negative cerebral malaria

Cerebral malaria (CM) can be classified as retinopathy-positive or retinopathy-negative, based on the presence or absence of characteristic retinal features. While malaria parasites are considered central to the pathogenesis of retinopathy-positive CM, their contribution to retinopathy-negative CM is largely unknown. One theory is that malaria parasites are innocent bystanders in retinopathy-negative CM and the etiology of the coma is entirely non-malarial. Because hospitals in malaria-endemic areas often lack diagnostic facilities to identify non-malarial causes of coma, it has not been possible to evaluate the contribution of malaria infection to retinopathy-negative CM. To overcome this barrier, we studied a natural experiment involving genetically inherited traits, and find evidence that malaria parasitemia does contribute to the pathogenesis of retinopathy-negative CM. A lower bound for the fraction of retinopathy-negative CM that would be prevented if malaria parasitemia were to be eliminated is estimated to be 0.93 (95% confidence interval: 0.68, 1).

DOI:http://dx.doi.org/10.7554/eLife.23699.001

Malaria is a life-threatening disease caused by a parasite that is transferred between people by infected mosquitoes. Most infected individuals suffer flu-like symptoms, but in rare cases malaria can affect the brain, resulting in brain damage, coma or death.

The World Health Organization defines a person as suffering from cerebral malaria if the person is in a coma, has malaria parasites in his or her blood, and has no known alternative cause of the coma. Patients suffering from cerebral malaria are categorized based on whether they have damage to the back of the eyes known as retinopathy. It had previously been found that children who died of “retinopathy-positive” cerebral malaria (i.e. those who had retinopathy) had malaria parasites stuck in small vessels in their brains, which likely caused the coma. By contrast, children who died of “retinopathy-negative” cerebral malaria lacked this parasitic condition, and often also had other infections that can cause a coma, such as meningitis or sepsis.

Because hospitals in many of the areas most affected by malaria often lack the ability to identify what – other than malaria – caused a coma, it was not clear whether malaria parasites influence how retinopathy-negative cerebral malaria develops.

People with certain genetic variants – such as those that underlie sickle cell disease – are protected against the symptoms of malaria infections, and so these variants should also protect against cerebral malaria cases caused by the parasites. Small et al. therefore looked through data that had been collected over several years from people who had been admitted to a hospital in Malawi for cerebral malaria. This revealed that the genetically inherited sickle cell trait is highly protective against retinopathy-negative (as well as retinopathy-positive) cerebral malaria. Therefore, malaria parasites do play a role in a substantial proportion of cases of retinopathy-negative cerebral malaria.

Although Small et al. provide evidence that malaria parasites play a role in retinopathy-negative cerebral malaria, they may not be the only cause of the coma. In the future, the absence of retinopathy could be used as a sign to look for additional factors that contribute to the coma. Currently, all cerebral malaria patients are treated in the same way. Understanding how malaria parasites interact with other illnesses to produce a coma could lead to the development of targeted treatment plans for retinopathy-negative patients.

DOI:http://dx.doi.org/10.7554/eLife.23699.002

The State of Applied Econometrics: Causality and Policy Evaluation

In this paper, we discuss recent developments in econometrics that we view as important for empirical researchers working on policy evaluation questions. We focus on three main areas, in each case, highlighting recommendations for applied work. First, we discuss new research on identification strategies in program evaluation, with particular focus on synthetic control methods, regression discontinuity, external validity, and the causal interpretation of regression methods. Second, we discuss various forms of supplementary analyses, including placebo analyses as well as sensitivity and robustness analyses, intended to make the identification strategies more credible. Third, we discuss some implications of recent advances in machine learning methods for causal effects, including methods to adjust for differences between treated and control units in high-dimensional settings, and methods for identifying and estimating heterogenous treatment effects.

Economic Assimilation and Skill Acquisition: Evidence From the Occupational Sorting of Childhood Immigrants

We study the economic assimilation of childhood immigrants to the United States. The linguistic distance between English and the predominant language in one's country of birth interacted with age at arrival is shown to be closely connected to occupational sorting in adulthood. By applying big-data techniques to occupations' detailed skill requirements, we provide evidence that childhood immigrants from English-distant countries who arrived after the primary school years reveal comparative advantages in tasks distinct from those for which (close to) Anglophone immigrants are better suited. Meanwhile, those who arrive at younger ages specialize in a bundle of skills very similar to that supplied by observationally equivalent workers. These patterns emerge even after we net out the effects of formal education. Such findings are compatible with the existence of different degrees of complementarity between relative English-learning potential at arrival and the acquisition of multiple capabilities demanded in the U.S. labor market (math/logic, socioemotional, physical, and communication skills). Consistent with the investment-complementarity argument, we show that linguistic distance and age at arrival also play a significant role on the choice of college major within this population.

Evaluating the Health Impact of Large-Scale Public Policy Changes: Classical and Novel Approaches

Large-scale public policy changes are often recommended to improve public health. Despite varying widely-from tobacco taxes to poverty-relief programs-such policies present a common dilemma to public health researchers: how to evaluate their health effects when randomized controlled trials are not possible. Here, we review the state of knowledge and experience of public health researchers who rigorously evaluate the health consequences of large-scale public policy changes. We organize our discussion by detailing approaches to address three common challenges of conducting policy evaluations: distinguishing a policy effect from time trends in health outcomes or preexisting differences between policy-affected and -unaffected communities (using difference-in-differences approaches); constructing a comparison population when a policy affects a population for whom a well-matched comparator is not immediately available (using propensity score or synthetic control approaches); and addressing unobserved confounders by utilizing quasi-random variations in policy exposure (using regression discontinuity, instrumental variables, or near-far matching approaches).

A comparison of alternative strategies for choosing control populations in observational studies

Various approaches have been used to select control groups in observational studies: (1) from within the intervention area; (2) from a convenience sample, or randomly chosen areas; (3) from areas matched on area-level characteristics; and (4) nationally. The consequences of the decision are rarely assessed but, as we show, it can have complex impacts on confounding at both the area and individual levels. We began by reanalyzing data collected for an evaluation of a rapid response service on rates of unplanned hospital admission. Balance on observed individual-level variables was better with external than local controls, after matching. Further, when important prognostic variables were omitted from the matching algorithm, imbalances on those variables were also minimized using external controls. Treatment effects varied markedly depending on the choice of control area, but in the case study the variation was minimal after adjusting for the characteristics of areas. We used simulations to assess relative bias and means-squared error, as this could not be done in the case study. A particular feature of the simulations was unexplained variation in the outcome between areas. We found that the likely impact of unexplained variation for hospital admissions dwarfed the benefits of better balance on individual-level variables, leading us to prefer local controls in this instance. In other scenarios, in which there was less unexplained variation in the outcome between areas, bias and mean-squared error were optimized using external controls. We identify some general considerations relevant to the choice of control population in observational studies.

The impact of obstetric unit closures on maternal and infant pregnancy outcomes

OBJECTIVE

To define the association between large-scale obstetric unit closures and relative changes in maternal and neonatal outcomes.

DATA SOURCES/STUDY SETTING

Birth and death certificates were linked to maternal and neonatal hospital discharge records for all births between January 1, 1995 and June 30, 2005 in Philadelphia, which experienced the closure of 9 of 19 obstetric units between 1997 and 2005, and five surrounding counties and eight urban counties that did not experience a similar reduction in obstetric units.

DESIGN

A before-and-after study design with an untreated control group compared changes in perinatal outcomes in Philadelphia to five surrounding control counties and eight urban control counties after controlling for case mix differences and secular trends (N = 3,140,782).

RESULTS

Relative to the preclosure years, the difference in neonatal mortality (odds ratio (OR) 1.49, 95 percent CI 1.12-2.00) and all perinatal mortality (OR 1.53, 95 percent CI 1.14-2.04) increased for Philadelphia residents compared with both control groups between 1997 and 1999. After 2000, there was no statistically significant change in any outcome in Philadelphia county compared with the preclosure epoch.

CONCLUSIONS

Obstetric unit closures were initially associated with adverse changes in perinatal outcomes, but these outcomes ameliorated over time.

Using the Propensity Score Method to Estimate Causal Effects

Evidence-based management requires management scholars to draw causal inferences. Researchers generally rely on observational data sets and regression models where the independent variables have not been exogenously manipulated to estimate causal effects; however, using such models on observational data sets can produce a biased effect size of treatment intervention. This article introduces the propensity score method (PSM)—which has previously been widely employed in social science disciplines such as public health and economics—to the management field. This research reviews the PSM literature, develops a procedure for applying the PSM to estimate the causal effects of intervention, elaborates on the procedure using an empirical example, and discusses the potential application of the PSM in different management fields. The implementation of the PSM in the management field will increase researchers’ ability to draw causal inferences using observational data sets.

Using Multiple Control Groups and Matching to Address Unobserved Biases in Comparative Effectiveness Research: An Observational Study of the Effectiveness of Mental Health Parity

Studies of large policy interventions typically do not involve randomization. Adjustments, such as matching, can remove the bias due to observed covariates, but residual confounding remains a concern. In this paper we introduce two analytical strategies to bolster inferences of the effectiveness of policy interventions based on observational data. First, we identify how study groups may differ and then select a second comparison group on this source of difference. Second, we match subjects using a strategy that finely balances the distributions of key categorical covariates and stochastically balances on other covariates. An observational study of the effect of parity on the severely ill subjects enrolled in the Federal Employees Health Benefits (FEHB) Program illustrates our methods.

An Introduction to Propensity Score Methods for Reducing the Effects of Confounding in Observational Studies

The propensity score is the probability of treatment assignment conditional on observed baseline characteristics. The propensity score allows one to design and analyze an observational (nonrandomized) study so that it mimics some of the particular characteristics of a randomized controlled trial. In particular, the propensity score is a balancing score: conditional on the propensity score, the distribution of observed baseline covariates will be similar between treated and untreated subjects. I describe 4 different propensity score methods: matching on the propensity score, stratification on the propensity score, inverse probability of treatment weighting using the propensity score, and covariate adjustment using the propensity score. I describe balance diagnostics for examining whether the propensity score model has been adequately specified. Furthermore, I discuss differences between regression-based methods and propensity score-based methods for the analysis of observational data. I describe different causal average treatment effects and their relationship with propensity score analyses.

Heterogeneous treatment effects: what does a regression estimate?

Regressions that control for confounding factors are the workhorse of evaluation research. When treatment effects are heterogeneous, however, the workhorse regression leads to estimated treatment effects that lack behavioral interpretations even when the selection on observables assumption holds. Regressions that use propensity scores as weights and regressions based on random coefficients or hierarchical models provide alternative estimators that have clear behavioral interpretations. Assuming selection on the observables and heterogeneous treatment effects, this article (a) shows what is identified as the treatment effect in the workhorse model, (b) shows what is identified as the treatment effect by propensity score models and models based on random coefficients/ hierarchical models, and (c) provides advice for evaluators.

Matching methods for causal inference: A review and a look forward

When estimating causal effects using observational data, it is desirable to replicate a randomized experiment as closely as possible by obtaining treated and control groups with similar covariate distributions. This goal can often be achieved by choosing well-matched samples of the original treated and control groups, thereby reducing bias due to the covariates. Since the 1970's, work on matching methods has examined how to best choose treated and control subjects for comparison. Matching methods are gaining popularity in fields such as economics, epidemiology, medicine, and political science. However, until now the literature and related advice has been scattered across disciplines. Researchers who are interested in using matching methods-or developing methods related to matching-do not have a single place to turn to learn about past and current research. This paper provides a structure for thinking about matching methods and guidance on their use, coalescing the existing research (both old and new) and providing a summary of where the literature on matching methods is now and where it should be headed.

ANCOVA versus change from baseline had more power in randomized studies and more bias in nonrandomized studies

BACKGROUND AND OBJECTIVE

For inferring a treatment effect from the difference between a treated and untreated group on a quantitative outcome measured before and after treatment, current methods are analysis of covariance (ANCOVA) of the outcome with the baseline as covariate, and analysis of variance (ANOVA) of change from baseline. This article compares both methods on power and bias, for randomized and nonrandomized studies.

METHODS

The methods are compared by writing both as a regression model and as a repeated measures model, and are applied to a nonrandomized study of preventing depression.

RESULTS

In randomized studies both methods are unbiased, but ANCOVA has more power. If treatment assignment is based on the baseline, only ANCOVA is unbiased. In nonrandomized studies with preexisting groups differing at baseline, the two methods cannot both be unbiased, and may contradict each other. In the study of depression, ANCOVA suggests absence, but ANOVA of change suggests presence, of a treatment effect. The methods differ because ANCOVA assumes absence of a baseline difference.

CONCLUSION

In randomized studies and studies with treatment assignment depending on the baseline, ANCOVA must be used. In nonrandomized studies of preexisting groups, ANOVA of change seems less biased than ANCOVA, but two control groups and two baseline measurements are recommended.

Use of inhaled corticosteroids and risk of fractures

Treatment with systemic corticosteroids is known to increase the risk of fractures but little is known of the fracture risks associated with inhaled corticosteroids. A retrospective cohort study was conducted using a large UK primary care database (the General Practice Research Database [GPRD]). Inhaled corticosteroid users aged 18 years or older were compared with matched control patients and to a group of noncorticosteroid bronchodilator users. Patients with concomitant use of systemic corticosteroids were excluded. The study comprised 170,818 inhaled corticosteroid users, 108,786 bronchodilator users, and 170,818 control patients. The average age was 45.1 years in the inhaled corticosteroid, 49.3 years in the bronchodilator, and 45.2 years in the control groups. In the inhaled corticosteroid cohort, 54.5% were female. The relative rates (RRs) of nonvertebral, hip, and vertebral fractures during inhaled corticosteroid treatment compared with control were 1.15 (95% CI, 1.10-1.20), 1.22 (95% CI, 1.04-1.43), and 1.51 (95% CI, 1.22-1.85), respectively. No differences were found between the inhaled corticosteroid and bronchodilator groups (nonvertebral fracture RR = 1.00; 95% CI, 0.94-1.06). The rates of nonvertebral fractures among users of budesonide (RR = 0.95; 95% CI, 0.85-1.07) and fluticasone propionate (RR = 1.03; 95% CI, 0.71-1.49) were similar to the rate determined for users of beclomethasone dipropionate. We conclude that users of inhaled corticosteroids have an increased risk of fracture, particularly at the hip and spine. However, this excess risk may be related more to the underlying respiratory disease than to inhaled corticosteroid.

Misunderstanding analysis of covariance

Despite numerous technical treatments in many venues, analysis of covariance (ANCOVA) remains a widely misused approach to dealing with substantive group differences on potential covariates, particularly in psychopathology research. Published articles reach unfounded conclusions, and some statistics texts neglect the issue. The problem with ANCOVA in such cases is reviewed. In many cases, there is no means of achieving the superficially appealing goal of "correcting" or "controlling for" real group differences on a potential covariate. In hopes of curtailing misuse of ANCOVA and promoting appropriate use, a nontechnical discussion is provided, emphasizing a substantive confound rarely articulated in textbooks and other general presentations, to complement the mathematical critiques already available. Some alternatives are discussed for contexts in which ANCOVA is inappropriate or questionable.

A bibliography and comments on the use of statistical models in epidemiology in the 1980s

This paper reviews developments in statistical modelling in epidemiology in the 1980's, with emphasis on cohort and case-control studies. The central roles of the logistic and proportional hazard models are highlighted, and it is shown how these models lead to a deeper understanding of classical designs and methods of analysis as well as to efficient new designs and analytical procedures. The important area of model misspecification is discussed, including the problems of omitted latent structure, mis-modelling of available measurements, missing data and errors in measurements. Various designs motivated by the logistic model are illustrated numerically, and designs based on the proportional hazards model are discussed, as are papers on sample size determination. There are brief introductions to the literature on other topics, including attributable risk, disease clustering, family studies and genetics, analysis of disease incidence data, infectious disease, longitudinal data, screening and miscellaneous related topics in statistics. An extensive bibliography is indexed according to the outline of the paper.

Alternative parameterization of polychotomous models: theory and application to matched case-control studies

A method is proposed for transforming a class of models having an outcome variable with more than two levels into an equivalent binary model. The polychotomous logistic model is used to demonstrate the method. The equivalency to a simple logistic regression model after some data transformation (augmentation) is shown. The method is applied to the data from two case-control studies each with two control groups, and further applications are indicated.