Amplification of Sensitivity Analysis in Matched Observational Studies

A sensitivity analysis displays the increase in uncertainty that attends an inference when a key assumption is relaxed. In matched observational studies of treatment effects, a key assumption in some analyses is that subjects matched for observed covariates are comparable, and this assumption is relaxed by positing a relevant covariate that was not observed and not controlled by matching. What properties would such an unobserved covariate need to have to materially alter the inference about treatment effects? For ease of calculation and reporting, it is convenient that the sensitivity analysis be of low dimension, perhaps indexed by a scalar sensitivity parameter, but for interpretation in specific contexts, a higher dimensional analysis may be of greater relevance. An amplification of a sensitivity analysis is defined as a map from each point in a low dimensional sensitivity analysis to a set of points, perhaps a 'curve,' in a higher dimensional sensitivity analysis such that the possible inferences are the same for all points in the set. Possessing an amplification, an investigator may calculate and report the low dimensional analysis, yet have available the interpretations of the higher dimensional analysis.

ERROR-FREE MILESTONES IN ERROR PRONE MEASUREMENTS

A predictor variable or dose that is measured with substantial error may possess an error-free milestone, such that it is known with negligible error whether the value of the variable is to the left or right of the milestone. Such a milestone provides a basis for estimating a linear relationship between the true but unknown value of the error-free predictor and an outcome, because the milestone creates a strong and valid instrumental variable. The inferences are nonparametric and robust, and in the simplest cases, they are exact and distribution free. We also consider multiple milestones for a single predictor and milestones for several predictors whose partial slopes are estimated simultaneously. Examples are drawn from the Wisconsin Longitudinal Study, in which a BA degree acts as a milestone for sixteen years of education, and the binary indicator of military service acts as a milestone for years of service.

Time to send the preemie home? Additional maturity at discharge and subsequent health care costs and outcomes

OBJECTIVE

To determine whether longer stays of premature infants allowing for increased physical maturity result in subsequent postdischarge cost savings that help counterbalance increased inpatient costs.

DATA SOURCES

One thousand four hundred and two premature infants born in the Northern California Kaiser Permanente Medical Care Program between 1998 and 2002.

STUDY DESIGN/METHODS

Using multivariate matching with a time-dependent propensity score we matched 701 "Early" babies to 701 "Late" babies (developmentally similar at the time the earlier baby was sent home but who were discharged on average 3 days later) and assessed subsequent costs and clinical outcomes.

PRINCIPAL FINDINGS

Late babies accrued inpatient costs after the Early baby was already home, yet costs after discharge through 6 months were virtually identical across groups, as were clinical outcomes. Overall, after the Early baby went home, the Late-Early cost difference was $5,016 (p<.0001). A sensitivity analysis suggests our conclusions would not easily be altered by failure to match on some unmeasured covariate.

CONCLUSIONS

In a large integrated health care system, if a baby is ready for discharge (as defined by the typical criteria), staying longer increased inpatient costs but did not reduce postdischarge costs nor improve postdischarge clinical outcomes.

Hospital teaching intensity, patient race, and surgical outcomes

OBJECTIVES

To determine if the lower mortality often observed in teaching-intensive hospitals is because of lower complication rates or lower death rates after complications (failure to rescue) and whether the benefits at these hospitals accrue equally to white and black patients, since black patients receive a disproportionate share of their care at teaching-intensive hospitals.

DESIGN

A retrospective study of patient outcomes and teaching intensity using logistic regression models, with and without adjusting for hospital fixed and random effects.

SETTING

Three thousand two hundred seventy acute care hospitals in the United States.

PATIENTS

Medicare claims on general, orthopedic, and vascular surgery admissions in the United States for 2000-2005 (N = 4,658,954 unique patients).

MAIN OUTCOME MEASURES

Thirty-day mortality, in-hospital complications, and failure to rescue (the probability of death following complications).

RESULTS

Combining all surgeries, compared with nonteaching hospitals, patients at very major teaching hospitals demonstrated a 15% lower odds of death (P < .001), no difference in complications, and a 15% lower odds of death after complications (failure to rescue) (P < .001). These relative benefits associated with higher resident-to-bed ratio were not experienced by black patients, for whom the odds of mortality and failure to rescue were similar at teaching and nonteaching hospitals, a pattern that is significantly different from that of white patients (P < .001).

CONCLUSIONS

Survival after surgery is higher at hospitals with higher teaching intensity. Improved survival is because of lower mortality after complications (better failure to rescue) and generally not because of fewer complications. However, this better survival and failure to rescue at teaching-intensive hospitals is seen for white patients, not for black patients.

Development of a measure of family-centred care for resource-poor South African settings: the experience of using a modified version of the MPOC-20

BACKGROUND

The Measure of Processes of Care (MPOC) is a widely used tool to assess parents' self-reported experiences of family-centred behaviours of paediatric rehabilitation services. It has never been used in resource-constrained settings or in a cross-cultural environment where cultural and language differences may complicate effective implementation of family-centred services. In this study, the MPOC-20 was used as the starting point for the development of a measure of family-centred care in disadvantaged South African settings. The objective was to establish to what extent the MPOC-20 needed to be adapted for these settings.

METHODS

After modifying MPOC-20 through focus groups, the adapted scale was translated into six local languages. Trained interviewers administered the scale to a convenience sample of 267 caregivers of children aged between 1 and 18 years with a diagnosis of cerebral palsy living in poorly resourced areas in two provinces in South Africa.

RESULTS

The modified MPOC-20 was neither reliable nor valid in the new setting. Cronbach's alpha for each of the sub-scales varied between 0.30 and 0.66 while for the test-retest reliability, the Intraclass Correlation Coefficients were between 0.51 and 0.61. The first two criteria for item convergent validity were not met. Repeated multi-trait scaling identified eight items that when combined into a scale [named the MPOC-8(SA)] had acceptable reliability and validity. Factor analysis of the MPOC-8(SA) yielded two factors: an interpersonal factor and an informational factor.

CONCLUSIONS

Although extreme caution has to be used when using measures created in one socio-cultural setting in a different context, the MPOC-20 provides a useful starting point for the development of a measure of family-centred care in a poor resourced setting. Caregivers in different settings have more in common than they have differences. However, the process of asking the questions and the words used to capture caregivers' experiences needs to be different.

An Algorithm for Optimal Tapered Matching, With Application to Disparities in Survival

In a tapered matched comparison, one group of individuals, called the focal group, is compared to two or more nonoverlapping matched comparison groups constructed from one population in such a way that successive comparison groups increasingly resemble the focal group. An optimally tapered matching solves two problems simultaneously: it optimally divides the single comparison population into nonoverlapping comparison groups and optimally pairs members of the focal group with members of each comparison group. We show how to use the optimal assignment algorithm in a new way to solve the optimally tapered matching problem, with implementation in R. This issue often arises in studies of groups Defined by race, gender, or other categorizations such that equitable public policy might require an understanding of the mechanisms that produce disparate outcomes, where certain specific mechanisms would be judged illegitimate, necessitating reform. In particular, we use data from Medicare and the SEER Program of the National Cancer Institute as part of an ongoing study of black-white disparities in survival among women with endometrial cancer.

Combining group-based trajectory modeling and propensity score matching for causal inferences in nonexperimental longitudinal data

A central theme of research on human development and psychopathology is whether a therapeutic intervention or a turning-point event, such as a family break-up, alters the trajectory of the behavior under study. This article describes and applies a method for using observational longitudinal data to make more transparent causal inferences about the impact of such events on developmental trajectories. The method combines 2 distinct lines of research: work on the use of finite mixture modeling to analyze developmental trajectories and work on propensity score matching. The propensity scores are used to balance observed covariates and the trajectory groups are used to control pretreatment measures of response. The trajectory groups also aid in characterizing classes of subjects for which no good matches are available. The approach is demonstrated with an analysis of the impact of gang membership on violent delinquency based on data from a large longitudinal study conducted in Montréal, Canada.

Sensitivity analysis for m-estimates, tests, and confidence intervals in matched observational studies

Huber's m-estimates use an estimating equation in which observations are permitted a controlled level of influence. The family of m-estimates includes least squares and maximum likelihood, but typical applications give extreme observations limited weight. Maritz proposed methods of exact and approximate permutation inference for m-tests, confidence intervals, and estimators, which can be derived from random assignment of paired subjects to treatment or control. In contrast, in observational studies, where treatments are not randomly assigned, subjects matched for observed covariates may differ in terms of unobserved covariates, so differing outcomes may not be treatment effects. In observational studies, a method of sensitivity analysis is developed for m-tests, m-intervals, and m-estimates: it shows the extent to which inferences would be altered by biases of various magnitudes due to nonrandom treatment assignment. The method is developed for both matched pairs, with one treated subject matched to one control, and for matched sets, with one treated subject matched to one or more controls. The method is illustrated using two studies: (i) a paired study of damage to DNA from exposure to chromium and nickel and (ii) a study with one or two matched controls comparing side effects of two drug regimes to treat tuberculosis. The approach yields sensitivity analyses for: (i) m-tests with Huber's weight function and other robust weight functions, (ii) the permutational t-test which uses the observations directly, and (iii) various other procedures such as the sign test, Noether's test, and the permutation distribution of the efficient score test for a location family of distributions. Permutation inference with covariance adjustment is briefly discussed.

Mortality among patients in VA hospitals in the first 2 years following ACGME resident duty hour reform

CONTEXT

Limitations in duty hours for physicians-in-training in the United States were established by the Accreditation Council for Graduate Medical Education (ACGME) and implemented on July 1, 2003. The association of these changes with mortality among hospitalized patients has not been well established.

OBJECTIVE

To determine whether the change in duty hour regulations was associated with relative changes in mortality in hospitals of different teaching intensity within the US Veterans Affairs (VA) system.

DESIGN, SETTING, AND PATIENTS

An observational study of all unique patients (N = 318 636) admitted to acute-care VA hospitals (N = 131) using interrupted time series analysis with data from July 1, 2000, to June 30, 2005. All patients had principal diagnoses of acute myocardial infarction (AMI), congestive heart failure, gastrointestinal bleeding, or stroke or a diagnosis related group classification of general, orthopedic, or vascular surgery. Logistic regression was used to examine the change in mortality for patients in more vs less teaching-intensive hospitals before (academic years 2000-2003) and after (academic years 2003-2005) duty hour reform, adjusting for patient comorbidities, common time trends, and hospital site.

MAIN OUTCOME MEASURE

All-location mortality within 30 days of hospital admission.

RESULTS

In postreform year 1, no significant relative changes in mortality were observed for either medical or surgical patients. In postreform year 2, the odds of mortality decreased significantly in more teaching-intensive hospitals for medical patients only. Comparing a hospital having a resident-to-bed ratio of 1 with a hospital having a resident-to-bed ratio of 0, the odds of mortality were reduced for patients with AMI (odds ratio [OR], 0.48; 95% confidence interval [CI], 0.33-0.71), for the 4 medical conditions together (OR, 0.74; 95% CI, 0.61-0.89), and for the 3 medical conditions excluding AMI (OR, 0.79; 95% CI, 0.63-0.98). Compared with hospitals in the 25th percentile of teaching intensity, there was an absolute improvement in mortality from prereform year 1 to postreform year 2 of 0.70 percentage points (11.1% relative decrease) and 0.88 percentage points (13.9% relative decrease) in hospitals in the 75th and 90th percentile of teaching intensity, respectively, for the combined medical conditions.

CONCLUSIONS

The ACGME duty hour reform was associated with significant relative improvement in mortality for patients with 4 common medical conditions in more teaching-intensive VA hospitals in postreform year 2. No associations were identified for surgical patients.

Mortality among hospitalized Medicare beneficiaries in the first 2 years following ACGME resident duty hour reform

CONTEXT

The Accreditation Council for Graduate Medical Education (ACGME) implemented duty hour regulations for physicians-in-training throughout the United States on July 1, 2003. The association of duty hour reform with mortality among patients in teaching hospitals nationally has not been well established.

OBJECTIVE

To determine whether the change in duty hour regulations was associated with relative changes in mortality among Medicare patients in hospitals of different teaching intensity.

DESIGN, SETTING, AND PATIENTS

An observational study of all unique Medicare patients (N = 8 529 595) admitted to short-term, acute-care, general US nonfederal hospitals (N = 3321) using interrupted time series analysis with data from July 1, 2000, to June 30, 2005. All Medicare patients had principal diagnoses of acute myocardial infarction, congestive heart failure, gastrointestinal bleeding, or stroke or a diagnosis related group classification of general, orthopedic, or vascular surgery. Logistic regression was used to examine the change in mortality for patients in more vs less teaching-intensive hospitals before (academic years 2000-2003) and after (academic years 2003-2005) duty hour reform, adjusting for patient comorbidities, common time trends, and hospital site.

MAIN OUTCOME MEASURE

All-location mortality within 30 days of hospital admission.

RESULTS

In medical and surgical patients, no significant relative increases or decreases in the odds of mortality for more vs less teaching-intensive hospitals were observed in either postreform year 1 (combined medical conditions group: odds ratio [OR], 1.03; 95% confidence interval [CI], 0.98-1.07; and combined surgical categories group: OR, 1.05; 95% CI, 0.98-1.12) or postreform year 2 (combined medical conditions group: OR, 1.03; 95% CI, 0.99-1.08; and combined surgical categories group: OR, 1.01; 95% CI, 0.95-1.08) compared with the prereform years. The only condition for which there was a relative increase in mortality in more teaching-intensive hospitals postreform was stroke, but this association preceded the onset of duty hour reform. Compared with nonteaching hospitals, the most teaching-intensive hospitals had an absolute change in mortality from prereform year 1 to postreform year 2 of 0.42 percentage points (4.4% relative increase) for patients in the combined medical conditions group and 0.05 percentage points (2.3% relative increase) for patients in the combined surgical categories group, neither of which were statistically significant.

CONCLUSION

The ACGME duty hour reform was not associated with either significant worsening or improvement in mortality for Medicare patients in the first 2 years after implementation.

Confidence Intervals for Uncommon but Dramatic Responses to Treatment

A small literature discusses locally most powerful rank tests when only a fraction of treated subjects respond to treatment. The ranks used in these tests are very different from conventional ranks, being relatively flat for low responses and then rising steeply, and the associated tests are much more powerful than conventional rank tests when, indeed, only a small fraction of treated subjects exhibit dramatic responses. Because the tests are derived from considerations of local power, they do not yield a plausible family of models for effect, and therefore they do not yield confidence intervals for the magnitude of effect formed by inverting the tests. There is a similarity between these tests and another family of tests, originally motivated by different considerations involving peak performance in small subsets. Exploiting this similarity, a method for obtaining confidence statements is proposed. In the case of observational studies, sensitivity to unobserved bias from nonrandom assignment of treatments is also examined. Two examples are used as illustrations: (i) a study of smoking during pregnancy and its effects on birth weight, in which smokers are matched to six controls, and (ii) a matched pair study of damage to DNA among workers at aluminum production plants.

Does Ovarian Cancer Treatment and Survival Differ by the Specialty Providing Chemotherapy?

Purpose

Chemotherapy for ovarian cancer is usually administered by medical oncologists (MOs) or gynecologic oncologists (GOs). GOs perform a broad spectrum of surgical and medical activities while managing a limited number of diseases; MOs specialize in the administration of chemotherapy but manage a broad array of diseases. We asked whether survival, treatment, and toxicity differed according to the type of specialist providing the chemotherapy after surgery.

Patients and Methods

Using Surveillance, Epidemiology, and End Results (SEER) -Medicare data for patients ≥ 65 years old from 1991 through 2001 from eight SEER sites, we identified 344 patients with ovarian cancer who were treated with chemotherapy by a GO after surgery. Using optimal matching and propensity scores based on 36 characteristics, we matched these patients to 344 similar patients who were operated on and staged by the same type of surgeon but who received chemotherapy from an MO.

Results

MOs administered chemotherapy over more weeks than did the GOs (16.5 v 12.1 weeks, respectively; P < .0023), and MO patients had substantially more weeks that included chemotherapy-associated adverse events than GO patients (16.2 v 8.9 weeks, respectively; P < .0001). However, there was no difference in 5-year survival rate between the GO and MO groups (35% v 34%, respectively; P = .45).

Conclusion

GO- and MO-treated patients who were closely matched on prognostic characteristics experienced very different rates of chemotherapy-associated adverse events and very different chemotherapy treatment styles by specialty type; however, their survival was virtually identical.

Estimating anesthesia and surgical procedure times from medicare anesthesia claims

BACKGROUND

Procedure times are important variables that often are included in studies of quality and efficiency. However, due to the need for costly chart review, most studies are limited to single-institution analyses. In this article, the authors describe how well the anesthesia claim from Medicare can estimate chart times.

METHODS

The authors abstracted information on time of induction and entrance to the recovery room ("anesthesia chart time") from the charts of 1,931 patients who underwent general and orthopedic surgical procedures in Pennsylvania. The authors then merged the associated bills from claims data supplied from Medicare (Part B data) that included a variable denoting the time in minutes for the anesthesia service. The authors also investigated the time from incision to closure ("surgical chart time") on a subset of 1,888 patients.

RESULTS

Anesthesia claim time from Medicare was highly predictive of anesthesia chart time (Kendall's rank correlation tau = 0.85, P < 0.0001, median absolute error = 5.1 min) but somewhat less predictive of surgical chart time (Kendall's tau = 0.73, P < 0.0001, median absolute error = 13.8 min). When predicting chart time from Medicare bills, variables reflecting procedure type, comorbidities, and hospital type did not significantly improve the prediction, suggesting that errors in predicting the chart time from the anesthesia bill time are not related to these factors; however, the individual hospital did have some influence on these estimates.

CONCLUSIONS

Anesthesia chart time can be well estimated using Medicare claims, thereby facilitating studies with vastly larger sample sizes and much lower costs of data collection.

Influence of patient and hospital characteristics on anesthesia time in medicare patients undergoing general and orthopedic surgery

BACKGROUND

Procedure time is a clinically important variable that is often analyzed when studying quality and efficiency. Norms for procedure length have not been reported from Medicare data sets, nor has the influence of patient and hospital characteristics on procedure time been estimated using Medicare data.

METHODS

The authors obtained Medicare claims on all patients aged 65-85 years who underwent general surgical and orthopedic surgical procedures in Pennsylvania. Anesthesia procedure time was estimated from anesthesia time units bills supplied from Medicare on 20 common general and orthopedic surgery procedures, and models to determine the influence of hospital and patient characteristics were developed.

RESULTS

Of the 77,638 patients, 31,472 had general surgery and 46,166 underwent orthopedic procedures. The median anesthesia time for general surgery was 133 min, and for orthopedic surgery it was 146 min. After adjusting for principal procedure, hospital, and physiologic severity, covariates associated with increased anesthesia time included: multiple procedure on same day + 18.3 min (P < 0.0001); transfer-in + 6.7 min (P = 0.0002); black race + 5.5 (P < 0.0001); coagulation disorders + 4.9 (P = 0.0012); and paraplegia + 4.5 (P = 0.0006). Lower-income black patients had significantly longer procedure times than lower-income white patients (+ 7 min; P < 0.0001). Among the 15 hospitals with the largest black surgical populations, 5 hospitals had statistically significant procedure lengths for black versus white patients, ranging from + 9 to + 16 min.

CONCLUSIONS

In addition to variation by patient comorbidities and procedure, anesthesia procedure time varies with hospital, medical history, and sociodemographic characteristics.

Variation in chemotherapy utilization in ovarian cancer: the relative contribution of geography

OBJECTIVE

This study investigates geographic variation in chemotherapy utilization for ovarian cancer in both absolute and relative terms and examines area characteristics associated with this variation.

DATA SOURCES

Surveillance, Epidemiology, and End Results (SEER) Medicare data from 1990 to 2001 for Medicare patients over 65 with a diagnosis of ovarian cancer between 1990 and 1999. Chemotherapy within a year of diagnosis was identified by Medicare billing codes. The hospital referral region (HRR) represents the geographic unit of analysis.

STUDY DESIGN

A logit model predicting the probability of receiving chemotherapy by each of the 39 HRRs. Control variables included medical characteristics (patient age, stage, year of diagnosis, and comorbidities) and socioeconomic characteristics (race, income, and education). The variation among HRRs was tested by the chi2 statistic, and the relative contribution was measured by the omega statistic. HHR market characteristic are then used to explain HRR-level variation.

PRINCIPAL FINDINGS

The average chemotherapy rate was 56.6 percent, with a range by HRR from 33 percent to 67 percent. There were large and significant differences in chemotherapy use between HRRs, reflected by a chi2 for HRR of 146 (df = 38, p < .001). HRR-level variation in chemotherapy use can be partially explained by higher chemotherapy rates in HRRs with a higher percentage of hospitals with oncology services. However, an omega analysis indicates that, by about 15 to one, the variation between patients in use of chemotherapy reflects variations in patient characteristics rather than unexplained variation among HRRs.

CONCLUSIONS

While absolute levels of chemotherapy variation between geographic areas are large and statistically significant, this analysis suggests that the role of geography in determining who gets chemotherapy is small relative to individual medical characteristics. Nevertheless, while variation by medical characteristics can be medically justified, the same cannot be said for geographic variation. Our finding that density of oncology hospitals predicts chemotherapy use suggests that provider supply is positively correlated with geographic variation.

Combining propensity score matching and group-based trajectory analysis in an observational study

In a nonrandomized or observational study, propensity scores may be used to balance observed covariates and trajectory groups may be used to control baseline or pretreatment measures of outcome. The trajectory groups also aid in characterizing classes of subjects for whom no good matches are available and to define substantively interesting groups between which treatment effects may vary. These and related methods are illustrated using data from a Montreal-based study. The effects on subsequent violence of gang joining at age 14 are studied while controlling for measured characteristics of boys prior to age 14. The boys are divided into trajectory groups based on violence from ages 11 to 13. Within trajectory group, joiners are optimally matched to a variable number of controls using propensity scores, Mahalanobis distances, and a combinatorial optimization algorithm. Use of variable ratio matching results in greater efficiency than pair matching and also greater bias reduction than matching at a fixed ratio. The possible impact of failing to adjust for an important but unmeasured covariate is examined using sensitivity analysis.