Ayres Sensory Integration Therapy for a Child With Rett Syndrome: A Case Report

Rett syndrome (RTT) is a neurodevelopmental disorder characterized by severe dyspraxia, hand stereotypies, and sensory processing issues for which there is no known treatment. This case describes a child with classic RTT and the child's responses to an Ayres Sensory Integration (ASI) treatment intervention (36 one-hour sessions, 3 per week). We coded and analyzed 36 detailed treatment notes to answer the following questions: What strategies and factors facilitated or interfered with participation in the intervention? What critical elements of treatment documentation might detect small changes in praxis and participation? How do patterns of motor or praxis milestones that emerge over time relate to this child's level of participation? We observed an increase in participation when the therapist incorporated elements of neurodevelopmental treatment (NDT) and motor learning theory- treatment strategies commonly used with children who have neuromotor conditions. This increase in participation in the ASI intervention emerged at approximately the same time that the therapist documented acquisition of new motor and praxis skills. We observed the importance of using: lateral movement activities to develop weight-shifting and bilateral coordination, rotary play to increase trunk rotation and improve postural transitions, and rhythm to promote continuing or initiating actions. The documentation of the specific amounts of assistance and prompting needed during treatment sessions was an important tool for tracking small yet meaningful responses to treatment. This case illustrates a novel use of ASI intervention supplemented with strategies that developed foundational skills, and the emergence of praxis and participation in the therapeutic intervention. We suggest further research is needed to determine efficacy of ASI for other children with this rare disorder.

High-Deductible Health Plans Paired With Health Savings Accounts Increased Medication Cost Burden Among Individuals With Bipolar Disorder

Objective: High-deductible health plans paired with health savings accounts (HSA-HDHPs) require substantial out-of-pocket spending for most services, including medications. We examined effects of HSA-HDHPs on medication out-of-pocket spending and use among people with bipolar disorder. Methods: This quasi-experimental study used claims data for January 2003 through December 2014. We studied a national sample of 348 members with bipolar disorder (defined based on International Classification of Diseases, 9th Revision), aged 12 to 64 years, who were continuously enrolled for 1 year in a low-deductible plan (≤ $500) then 1 year in an HSA-HDHP (≥ $1,000) after an employer-mandated switch. HSA-HDHP members were matched to 4,087 contemporaneous controls who remained in low-deductible plans. Outcome measures included out-of-pocket spending and use of bipolar disorder medications, non-bipolar psychotropics, and all other medications. Results: Mean pre-to-post out-of-pocket spending per person for bipolar disorder medications increased by 149.7% among HSA-HDHP versus control members (95% confidence interval [CI], 109.9% to 189.5%). Specifically, out-of-pocket spending increased for antipsychotics (220.9% [95% CI, 150.0% to 291.8%]) and anticonvulsants (109.6% [95% CI, 67.3% to 152.0%]). Both higher-income and lower-income HSA-HDHP members experienced increases in out-of-pocket spending for bipolar disorder medications (135.2% [95% CI, 86.4% to 184.0%] and 164.5% [95% CI, 100.9% to 228.1%], respectively). We did not detect statistically significant changes in use of bipolar disorder medications, non-bipolar psychotropics, or all other medications in this study population of HSA-HDHP members. Conclusions: HSA-HDHP members with bipolar disorder experienced substantial increases in out-of-pocket burdens for medications essential for their functioning and well-being. Although HSA-HDHPs were not associated with detectable reductions in medication use, high out-of-pocket costs could cause financial strain for lower-income enrollees.

Affordability of Medical Care Among Medicare Enrollees

Importance

Cost-sharing requirements can discourage patients from seeking care and impose financial hardship. The Medicare program serves many older and disabled individuals with multimorbidity and limited resources, but little has been known about the affordability of care in this population.

Objective

To examine the affordability of medical care among Medicare enrollees, in terms of the prevalence of delaying medical care because of costs and having problems paying medical bills, and risk factors for these outcomes.

Design Setting and Participants

Cross-sectional analyses conducted from November 1, 2019, to October 15, 2021, used logistic regression to compare the probability of outcomes by demographic and health characteristics. Data were obtained from the 2017 nationally representative Medicare Current Beneficiary Survey (response rate, 61.7%), with respondents representing 53 million community-dwelling Medicare enrollees.

Main Outcomes and Measures

New questions about medical care affordability were included in the 2017 Medicare Current Beneficiary Survey: difficulty paying medical bills, ongoing medical debt, and contact by collection agencies. A companion survey question asked whether individuals had delayed seeking medical care because of worries about costs.

Results

Respondents included 10 974 adults aged 65 years or older and 2197 aged 18 to 64 years; 54.2% of all respondents were women. The weighted proportions of Medicare enrollees with annual incomes below $25 000K were 30.7% in the older population and 67.4% in the younger group. Self-reported prevalence of delaying care because of cost was 8.3% (95% CI, 7.4%-9.1%) among enrollees aged 65 years or older, 25.2% (95% CI, 21.8%-28.6%) among enrollees younger than 65 years, and 10.9% (95% CI, 9.9%-11.9%) overall. Similarly, 7.4% (95% CI, 6.6%-8.2%) of older enrollees had problems paying medical bills, compared with 29.8% (95% CI, 25.6%-34.1%) among those younger than 65 years and 10.8% (95% CI, 9.8%-11.9%) overall. Regarding specific payment problems, 7.9% (95% CI, 7.0%-8.9%) of enrollees overall experienced ongoing medical debt, contact by a collection agency, or both. In adjusted analyses, older adults with incomes $15 000 to $25 000 per year had odds of delaying care more than twice as high as those with incomes greater than $50 000 (odds ratio, 2.47; 95% CI, 1.82-3.39), and their odds of problems paying medical bills were more than 3 times as high (odds ratio, 3.37; 95% CI, 2.81-5.21). Older adults with 4 to 10 chronic conditions were more than twice as likely to have problems paying medical bills as those with 0 or 1 condition.

Conclusions and Relevance

The findings of this study suggest that unaffordability of medical care is common among Medicare enrollees, especially those with lower incomes, or worse health, or who qualify for Medicare based on disability. Policy reforms, such as caps on patient spending, are needed to reduce Medical cost burdens on the most vulnerable enrollees.

Impact of High-Deductible Health Plans on Medication Use Among Individuals With Bipolar Disorder

OBJECTIVE

High-deductible health plans (HDHPs) require substantial out-of-pocket spending for most services, although medications may be subject to traditional copayment arrangements. This study examined effects of HDHPs on medication out-of-pocket spending and use and quality of care among individuals with bipolar disorder.

METHODS

This quasi-experimental study used claims data (2003-2014) for a national sample of 3,532 members with bipolar disorder, ages 12-64, continuously enrolled for 1 year in a low-deductible plan (≤$500) and then for 1 year in an HDHP (≥$1,000) after an employer-mandated switch. HDHP members were matched to 18,923 contemporaneous individuals in low-deductible plans (control group). Outcome measures were out-of-pocket spending and use of bipolar disorder medications, psychotropics for other disorders, and all other medications and appropriate laboratory monitoring for psychotropics.

RESULTS

Relative to the control group, annual out-of-pocket spending per person for bipolar disorder medications increased 20.8% among HDHP members (95% confidence interval [CI]=14.9%-26.7%), and the absolute increase was $36 (95% CI=$25.9-$45.2). Specifically, out-of-pocket spending increased for antipsychotics (27.1%; 95% CI=17.4%-36.7%) and anticonvulsants (19.2%; 95% CI=11.9%-26.6%) but remained stable for lithium (-3.7%; 95% CI=-12.2% to 4.8%). No statistically significant changes were detected in use of bipolar disorder medications, other psychotropics, or all other medications or in appropriate laboratory monitoring for bipolar disorder medications.

CONCLUSIONS

HDHP members with bipolar disorder experienced a moderate increase in out-of-pocket spending for medications but preserved bipolar disorder medication use. Findings may reflect individuals' perceptions of the importance of these medications for their functioning and well-being.

Experiences of health care costs among people with employer-sponsored insurance and bipolar disorder

BACKGROUND

Cost-sharing disproportionately affects people with chronic illnesses needing more care. Our qualitative study examined lived experiences navigating insurance benefits and treatment for bipolar disorder, which requires ongoing access to behavioral specialists and psychotropic medications.

METHODS

Forty semi-structured telephone interviews with individuals with bipolar disorder and employer-sponsored health insurance, or their family caregivers, explored health care needs, coverage details, out-of-pocket (OOP) costs, and perspectives on value. An iterative analytic approach identified salient themes.

RESULTS

Most individuals in our sample faced an annual insurance deductible, from $350-$10,000. OOP costs for specialist visits ranged from $0-$450 and for monthly psychotropic medications from $0-$1650. Acute episodes and care for comorbidities, including medication side effects, added to cost burdens. Medication nonadherence due to OOP costs was rare; respondents frequently pointed to the necessity of medications: "whatever it takes to get those"; "it's a life or death situation." Respondents also prioritized visits to psychiatrist prescribers, though visits were maximally spaced because of cost. Psychotherapy was often deemed unaffordable and forgone, despite perceived need. Interviewees cited limited networks and high out-of-network costs as barriers to specialists. Cost-sharing sometimes led to debt, skimping on nonbehavioral care or other necessities, exacerbated or prolonged mood symptoms, and stress at home.

LIMITATIONS

Volunteer respondents may not fully represent the target population.

CONCLUSIONS

Many people with bipolar disorder in US employer-sponsored plans experience undertreatment, hardship, and adverse health consequences due to high cost-sharing. More nuanced insurance benefit designs should accommodate the needs of individuals with complex conditions.

Cost-related Medication Nonadherence and Its Risk Factors Among Medicare Beneficiaries

BACKGROUND

Unaffordability of medications is a barrier to effective treatment. Cost-related nonadherence (CRN) is a crucial, widely used measure of medications access.

OBJECTIVES

Our study examines the current national prevalence of and risk factors for CRN (eg, not filling, skipping or reducing doses) and companion measures in the US Medicare population.

RESEARCH DESIGN

Survey-weighted analyses included logistic regression and trends 2006-2016.

SUBJECTS

Main analyses used the 2016 Medicare Current Beneficiary Survey. Our study sample of 12,625 represented 56 million community-dwelling beneficiaries.

MEASURES

Additional outcome measures were spending less on other necessities in order to pay for medicines and use of drug cost reduction strategies such as requesting generics.

RESULTS

In 2016, 34.5% of enrollees under 65 years with disability and 14.4% of those 65 years and older did not take their medications as prescribed due to high costs; 19.4% and 4.7%, respectively, experienced going without other essentials to pay for medicines. Near-poor older beneficiaries with incomes $15-25K had 50% higher odds of CRN (vs. >$50K), but beneficiaries with incomes <$15K, more likely to be eligible for the Part D Low-Income Subsidy, did not have significantly higher risk. Three indicators of worse health (general health status, functional limits, and count of conditions) were all independently associated with higher risk of CRN.

CONCLUSIONS

Changes in the risk profile for CRN since Part D reflect the effectiveness of targeted policies. The persistent prevalence of CRN and associated risks for sicker people in Medicare demonstrate the consequences of high cost-sharing for prescription fills.

Increases in Suicide Deaths Among Adolescents and Young Adults Following US Food and Drug Administration Antidepressant Boxed Warnings and Declines in Depression Care

Objective

Studies show decreased depression diagnosis, psychotherapy, and medications and increased suicide attempts following US Food and Drug Administration antidepressant warnings regarding suicidality risk among youth. Effects on care spilled over to older adults. This study investigated whether suicide deaths increased following the warnings and declines in depression care.

Methods

We conducted an interrupted time series study of validated death data (1990–2017) to estimate changes in trends of US suicide deaths per 100,000 adolescents (ages 10–19) and young adults (ages 20–24) after the warnings, controlling for baseline trends.

Results

Before the warnings (1990–2002), suicide deaths decreased markedly. After the warnings (2005–2017) and abrupt declines in treatment, this downward trend reversed. There was an immediate increase of 0.49 suicides per 100,000 adolescents, 95% confidence interval [CI]: 0.12, 0.86) and a trend increase of 0.03 suicides per 100,000 adolescents per year (95% CI: 0.026, 0.031). Similarly, there was an immediate increase of 2.07 suicides per 100,000 young adults (95% CI: 1.04, 3.10) and a trend increase of 0.05 suicides per 100,000 young adults per year (95% CI: 0.04, 0.06). Assuming baseline trends continued, there may have been 5958 excess suicides nationally by 2010 among yearly cohorts of 43 million adolescents and 21 million young adults.

Conclusions

We observed increases in suicide deaths among youth following the warnings and declines in depression care. Alternative explanations were explored, including substance use, economic recessions, smart phone use, and unintentional injury deaths. Additional factors may have contributed to continued increases in youth suicide during the last decade. Combined with previous research on declining treatment, these results call for re‐evaluation of the antidepressant warnings.

Previous research showed that depression care declined following the US FDA antidepressant warnings regarding suicidality for adolescents and young adults.

In this interrupted time series analysis using 28 years of nationwide death certificates, we found youth suicide deaths increased after the FDA antidepressant warnings and reductions in depression care.

We recommend that the FDA err on the side of caution and consider replacing the boxed warning with less severe warnings that still communicate information on possible drug risks without endangering essential, first‐line treatments of depression in youth.

Risk Factors Associated With Food Insecurity in the Medicare Population

This analysis of survey data presents national estimates of food insecurity prevalence within the Medicare population.

Sensory Integration and Functional Reaching in Children With Rett Syndrome/Rett-Related Disorders

Background

The loss of functional hand skills is a primary characteristic of Rett syndrome. Stereotypies, dyspraxia, and other sensory processing issues severely limit the individual's ability to reach toward and sustain grasp on objects. This loss of functional reach and grasp severely limits their ability to participate in self-help, play, and school-related activities. We proposed that Ayres Sensory Integration (ASI) treatment would improve sensory processing and motor planning, which would lay the sensory-motor groundwork for improving grasp of objects, an important first step in developing functional hand use.

Objective

We examined effects of ASI treatment on rate of reaching and grasping for children with Rett syndrome/Rett-related disorders.

Methods

We used an interrupted time series design to measure changes in outcome variables occurring after intervention initiation and cessation. We analyzed daily video observations during baseline, intervention, and post-intervention periods, over a span of 7 months.

Results

During baseline, rate of grasping declined moderately. There was a 15% increase in grasping from the end of baseline to end of the post-intervention period. There was no significant change in rate of reaching.

Conclusions

This study provides preliminary data showing very small improvements in hand grasp of children with Rett syndrome following ASI treatment; larger studies in diverse settings are needed to establish the effectiveness of this approach. This study shows that an interrupted time series research design provides a valid template for evaluating interventions for children with rare disorders.

Vulnerable And Less Vulnerable Women In High-Deductible Health Plans Experienced Delayed Breast Cancer Care

The effects of high-deductible health plans (HDHPs) on breast cancer diagnosis and treatment among vulnerable populations are unknown. We examined time to first breast cancer diagnostic testing, diagnosis, and chemotherapy among a group of women whose employers switched their insurance coverage from health plans with low deductibles ($500 or less) to plans with high deductibles ($1,000 or more) between 2004 and 2014. Primary subgroups of interest comprised 54,403 low-income and 76,776 high-income women continuously enrolled in low-deductible plans for a year and then up to four years in HDHPs. Matched controls had contemporaneous low-deductible enrollment. Low-income women in HDHPs experienced relative delays of 1.6 months to first breast imaging, 2.7 months to first biopsy, 6.6 months to incident early-stage breast cancer diagnosis, and 8.7 months to first chemotherapy. High-income HDHP members had shorter delays that did not differ significantly from those of their low-income counterparts. HDHP members living in metropolitan, nonmetropolitan, predominantly white, and predominantly nonwhite areas also experienced delayed breast cancer care. Policies may be needed to reduce out-of-pocket spending obligations for breast cancer care.

Counter-Point: Staying Honest When Policy Changes Backfire

Despite the good intentions of the Food and Drug Administration (FDA), many drug warnings are ineffective or have unintended consequences, particularly if the media exaggerates the messages and scares the public. The controversial 2003 to 2004 FDA warnings on youth suicidality associated with antidepressant use are a case in point. In a 10-year interrupted time series (ITS) analysis in 11 health plans, we found that the warnings and hyped media coverage led to substantial reductions in antidepressant use (declines in antidepressant use and overall care corroborated in several studies), and small, visible increases in emergency room and inpatient poisonings with psychotropic drugs. In a gross misunderstanding of the method, Dr Stone calls ITS, "an intuition based upon false analogies, fallacious assumptions and analytical error." We demonstrate visually using published studies that the ITS method is one of the oldest (hundreds of years) and strongest quasi-experimental study designs, and that the alternative data analyses proposed by Dr Stone do not have rates (denominators), nor baselines, so the measures of change are invalid.

Effect of High-Deductible Insurance on High-Acuity Outcomes in Diabetes: A Natural Experiment for Translation in Diabetes (NEXT-D) Study

OBJECTIVE

High-deductible health plans (HDHPs) are now the predominant commercial health insurance benefit in the U.S. We sought to determine the effects of HDHPs on emergency department and hospital care, adverse outcomes, and total health care expenditures among patients with diabetes.

RESEARCH DESIGN AND METHODS

We applied a controlled interrupted time-series design to study 23,493 HDHP members with diabetes, aged 12-64, insured through a large national health insurer from 2003 to 2012. HDHP members were enrolled for 1 year in a low-deductible (≤$500) plan, followed by 1 year in an HDHP (≥$1,000 deductible) after an employer-mandated switch. Patients transitioning to HDHPs were matched to 192,842 contemporaneous patients whose employers offered only low-deductible coverage. HDHP members from low-income neighborhoods (n = 8,453) were a subgroup of interest. Utilization measures included emergency department visits, hospitalizations, and total (health plan plus member out-of-pocket) health care expenditures. Proxy health outcome measures comprised high-severity emergency department visit expenditures and high-severity hospitalization days.

RESULTS

After the HDHP transition, emergency department visits declined by 4.0% (95% CI -7.8, -0.1), hospitalizations fell by 5.6% (-10.8, -0.5), direct (nonemergency department-based) hospitalizations declined by 11.1% (-16.6, -5.6), and total health care expenditures dropped by 3.8% (-4.3, -3.4). Adverse outcomes did not change in the overall HDHP cohort, but members from low-income neighborhoods experienced 23.5% higher (18.3, 28.7) high-severity emergency department visit expenditures and 27.4% higher (15.5, 39.2) high-severity hospitalization days.

CONCLUSIONS

After an HDHP switch, direct hospitalizations declined by 11.1% among patients with diabetes, likely driving 3.8% lower total health care expenditures. Proxy adverse outcomes were unchanged in the overall HDHP population with diabetes, but members from low-income neighborhoods experienced large, concerning increases in high-severity emergency department visit expenditures and hospitalization days.

Breast Cancer Diagnosis and Treatment After High-Deductible Insurance Enrollment

Purpose High-deductible health plans (HDHPs) require substantial out-of-pocket spending and might delay crucial health services. Breast cancer treatment delays of as little as 2 months are associated with adverse outcomes. Methods We used a controlled prepost design with survival analysis to assess timing of breast cancer care events among 273,499 women age 25 to 64 years without evidence of breast cancer before inclusion. Women were included if continuously enrolled for 1 year in a low-deductible ($0 to $500) plan followed by up to 4 years in a HDHP (at least $1,000 deductible) after an employer-mandated switch. Study inclusion was on a rolling basis, and members were followed between 2003 and 2012. The comparison group comprised 2.4 million contemporaneously matched women whose employers offered only low-deductible plans. Measures were times to first diagnostic breast imaging (diagnostic mammogram, breast ultrasound, or breast magnetic resonance imaging), breast biopsy, incident early-stage breast cancer diagnosis, and breast cancer chemotherapy. Outcomes were analyzed by using Cox models and adjusted for age-group, morbidity score, poverty level, US region, index date, and employer size. Results After the index date, HDHP members experienced delays in receipt of diagnostic imaging (adjusted hazard ratio [aHR], 0.95; 95% CI, 0.94 to 0.96), biopsy (aHR, 0.92; 95% CI, 0.89 to 0.95), early-stage breast cancer diagnosis (aHR, 0.83; 0.78 to 0.90), and chemotherapy initiation (aHR, 0.79; 95% CI, 0.72 to 0.86) compared with the control group. Conclusion Women switched to HDHPs experienced delays in diagnostic breast imaging, breast biopsy, early-stage breast cancer diagnosis, and chemotherapy initiation. Additional research should determine whether such delays cause adverse health outcomes, and policymakers should consider selectively reducing out-of-pocket costs for key breast cancer services.

Effects of Transitioning to Medicare Part D on Access to Drugs for Medical Conditions among Dual Enrollees with Cancer

OBJECTIVES

To evaluate the impact of transitioning from Medicaid to Medicare Part D drug coverage on the use of noncancer treatments among dual enrollees with cancer.

METHODS

We leveraged a representative 5% national sample of all fee-for-service dual enrollees in the United States (2004-2007) to evaluate the impact of the removal of caps on the number of reimbursable prescriptions per month (drug caps) under Part D on 1) prevalence and 2) average days' supply dispensed for antidepressants, antihypertensives, and lipid-lowering agents overall and by race (white and black).

RESULTS

The removal of drug caps was associated with increased use of lipid-lowering medications (days' supply 3.63; 95% confidence interval [CI] 1.57-5.70). Among blacks in capped states, we observed increased use of lipid-lowering therapy (any use 0.08 percentage points; 95% CI 0.05-0.10; and days' supply 4.01; 95% CI 2.92-5.09) and antidepressants (days' supply 2.20; 95% CI 0.61-3.78) and increasing trends in antihypertensive use (any use 0.01 percentage points; 95% CI 0.004-0.01; and days' supply 1.83; 95% CI 1.25-2.41). The white-black gap in the use of lipid-lowering medications was immediately reduced (-0.09 percentage points; 95% CI -0.15 to -0.04). We also observed a reversal in trends toward widening white-black differences in antihypertensive use (level -0.08 percentage points; 95% CI -0.12 to -0.05; and trend -0.01 percentage points; 95% CI -0.02 to -0.01) and antidepressant use (-0.004 percentage points; 95% CI -0.01 to -0.0004).

CONCLUSIONS

Our findings suggest that the removal of drug caps under Part D had a modest impact on the treatment of hypercholesterolemia overall and may have reduced white-black gaps in the use of lipid-lowering and antidepressant therapies.

The impact of a physician detailing and sampling program for generic atorvastatin: an interrupted time series analysis

BACKGROUND

In 2011, Manitoba implemented a province-wide program of physician detailing and free sampling for generic atorvastatin to increase use of this generic statin. We examined the impact of this unique combined program of detailing and sampling for generic atorvastatin on the use and cost of statin medicines, market share of generic atorvastatin, the choice of starting statin for new users, and switching from a branded statin to generic atorvastatin.

METHODS

We conducted a retrospective study of Manitoba insurance claims data for all continuously enrolled patients who filled one or more prescriptions for a statin between 2008 and 2013. Data were linked to physician-level data on the number of detailing visits and sample provision. We used interrupted time series analyses to assess policy-related changes in the use and cost of statin medicines, market share of generic atorvastatin, the choice of starting statin for new users, and switching from a branded statin to generic atorvastatin.

RESULTS

The detailing program reached 31% (651/2103) of physicians who prescribed a statin during the study period. Collectively, these physicians prescribed 61% of statins dispensed in the province. Free sample cards were provided to 61% (394/651) of the detailed physicians. The program did not change the level or trend in the overall statin use rate and the total cost of statins or increase the number of patients switching from another branded statin to generic atorvastatin. We found the program had a small impact on atorvastatin's market share of new prescriptions, with a level increase of 2.6%.

CONCLUSIONS

Though physician detailers were skilled at targeting high-prescribing physicians, a combined program of detailing visits and sample provision for generic atorvastatin did not lower overall statin costs or lead to switching from branded statins to the generic. The preceding introduction of generic atorvastatin appeared sufficient to modify prescribing patterns and decrease costs.

Impact of Massachusetts Health Reform on Enrollment Length and Health Care Utilization in the Unsubsidized Individual Market

OBJECTIVE

To evaluate the impact of the 2006 Massachusetts health reform, the model for the Affordable Care Act, on short-term enrollment and utilization in the unsubsidized individual health insurance market.

DATA SOURCE

Seven years of administrative and claims data from Harvard Pilgrim Health Care.

RESEARCH DESIGN

We employed pre-post survival analysis and an interrupted time series design to examine changes in enrollment length, utilization patterns, and use of elective procedures (discretionary inpatient surgeries and infertility treatment) among nonelderly adult enrollees before (n = 6,912) and after (n = 29,207) the MA reform.

PRINCIPAL FINDINGS

The probability of short-term enrollment dropped immediately after the reform. Rates of inpatient encounters (HR = 0.83, 95 percent CI: 0.74, 0.93), emergency department encounters (HR = 0.85, 95 percent CI: 0.80, 0.91), and discretionary inpatient surgeries (HR = 0.66 95 percent CI: 0.45, 0.97) were lower in the postreform period, whereas the rate of ambulatory visits was somewhat higher (HR = 1.04, 95 percent CI: 1.00, 1.07). The rate of infertility treatment was higher after the reform (HR = 1.61, 95 percent CI: 1.33, 1.97), driven by women in individual (vs. family) plans. The reform was not associated with increased utilization among short-term enrollees.

CONCLUSIONS

MA health reform was associated with a decrease in short-term enrollment and changes in utilization patterns indicative of reduced adverse selection in the unsubsidized individual market. Adverse selection may be a problem for specific, high-cost treatments.

False Dichotomies and Health Policy Research Designs: Randomized Trials Are Not Always the Answer

Some medical scientists argue that only data from randomized controlled trials (RCTs) are trustworthy. They claim data from natural experiments and administrative data sets are always spurious and cannot be used to evaluate health policies and other population-wide phenomena in the real world. While many acknowledge biases caused by poor study designs, in this article we argue that several valid designs using administrative data can produce strong findings, particularly the interrupted time series (ITS) design. Many policy studies neither permit nor require an RCT for cause-and-effect inference. Framing our arguments using Campbell and Stanley's classic research design monograph, we show that several "quasi-experimental" designs, especially interrupted time series (ITS), can estimate valid effects (or non-effects) of health interventions and policies as diverse as public insurance coverage, speed limits, hospital safety programs, drug abuse regulation and withdrawal of drugs from the market. We further note the recent rapid uptake of ITS and argue for expanded training in quasi-experimental designs in medical and graduate schools and in post-doctoral curricula.

The Impact of the Massachusetts Behavioral Health Child Screening Policy on Service Utilization

OBJECTIVE

In 2008, Massachusetts Medicaid implemented a pediatric behavioral health (BH) screening mandate. This study conducted a population-level, longitudinal policy analysis to determine the impact of the policy on ambulatory, emergency, and inpatient BH care in comparison with use of these services in California, where no similar policy exists.

METHODS

With Medicaid Analytic Extract (MAX) data, an interrupted time-series analysis with control series design was performed to assess changes in service utilization in the 18 months (January 2008-June 2009) after a BH screening policy was implemented in Massachusetts and to compare service utilization with California's. Outcomes included population rates of BH screening, BH-related outpatient visits, BH-related emergency department visits, BH-related hospitalizations, and psychotropic drug use. Medicaid-eligible children from January 1, 2006, to December 31, 2009, with at least ten months of Medicaid eligibility who were older than 4.5 years and younger than 18 years were included.

RESULTS

Compared with rates in California, Massachusetts rates of BH screening and BH-related outpatient visits rose significantly after Massachusetts implemented its screening policy. BH screening rose about 13 per 1,000 youths per month during the first nine months, and BH-related outpatient visits rose to about 4.5 per 1,000 youths per month (p<.001). Although BH-related emergency department visits, hospitalization and psychotropic drug use increased, there was no difference between the states in rate of increase.

CONCLUSIONS

The goal of BH screening is to identify previously unidentified children with BH issues and provide earlier treatment options. The short-term outcomes of the Massachusetts policy suggest that screening at preventive care visits led to more BH-related outpatient visits among vulnerable children.

Impact of Copayment Changes on Children's Albuterol Inhaler Use and Costs after the Clean Air Act Chlorofluorocarbon Ban

OBJECTIVE

To examine changes in children's albuterol use and out-of-pocket (OOP) costs in response to increased copayments after the Food and Drug Administration banned inhalers with chlorofluorocarbon (CFC) propellants.

SETTING

Four health maintenance organizations (HMOs), two that increased copayments for albuterol inhalers that went from generic CFC-containing to branded CFC-free versions, and two that retained generic copayments for CFC-free inhalers (controls). We included children with asthma aged 4-17 years with commercial coverage from 2007 to 2010.

DESIGN

Interrupted time series with comparison series.

DATA

We obtained enrollee and plan characteristics from enrollment files, and utilization data from pharmacy and medical claims; OOP expenditures were extracted from pharmacy claims for two HMOs with cost data available.

FINDINGS

There were no significant differences in albuterol use between the group with increased cost-sharing and controls with respect to changes after the policy change. There was a postpolicy increase of $6.11 OOP per month per child using albuterol among those with increased cost-sharing versus $0.36 in controls; the difference between groups was significant (p < .01).

CONCLUSIONS

Increased copayments for brand-name CFC-free albuterol after the CFC ban did not lead to a decrease in children's albuterol use, but it led to a modest increase in OOP costs.

Medication affordability gains following Medicare Part D are eroding among elderly with multiple chronic conditions

Elderly Americans, especially those with multiple chronic conditions, face difficulties paying for prescriptions, which results in worse adherence to and discontinuation of therapy, called cost-related medication nonadherence. Medicare Part D, implemented in January 2006, was supposed to address issues of affordability for prescriptions. We investigated whether the gains in medication affordability attributable to Part D persisted during the six years that followed its implementation. Overall, we found continued incremental improvements in medication affordability in the period 2007-09 that eroded during the period 2009-11. Among elderly beneficiaries with four or more chronic conditions, we observed an increase in the prevalence of cost-related nonadherence from 14.4 percent in 2009 to 17.0 percent in 2011, reversing previous downward trends. Similarly, the prevalence among the sickest elderly of forgoing basic needs to purchase medicines decreased from 8.7 percent in 2007 to 6.8 percent in 2009 but rose to 10.2 percent in 2011. Our findings highlight the need for targeted policy efforts to alleviate the persistent burden of drug treatment costs on this vulnerable population.

Missing clinical and behavioral health data in a large electronic health record (EHR) system

OBJECTIVE

Recent massive investment in electronic health records (EHRs) was predicated on the assumption of improved patient safety, research capacity, and cost savings. However, most US health systems and health records are fragmented and do not share patient information. Our study compared information available in a typical EHR with more complete data from insurance claims, focusing on diagnoses, visits, and hospital care for depression and bipolar disorder.

METHODS

We included insurance plan members aged 12 and over, assigned throughout 2009 to a large multispecialty medical practice in Massachusetts, with diagnoses of depression (N = 5140) or bipolar disorder (N = 462). We extracted insurance claims and EHR data from the primary care site and compared diagnoses of interest, outpatient visits, and acute hospital events (overall and behavioral) between the 2 sources.

RESULTS

Patients with depression and bipolar disorder, respectively, averaged 8.4 and 14.0 days of outpatient behavioral care per year; 60% and 54% of these, respectively, were missing from the EHR because they occurred offsite. Total outpatient care days were 20.5 for those with depression and 25.0 for those with bipolar disorder, with 45% and 46% missing, respectively, from the EHR. The EHR missed 89% of acute psychiatric services. Study diagnoses were missing from the EHR's structured event data for 27.3% and 27.7% of patients.

CONCLUSION

EHRs inadequately capture mental health diagnoses, visits, specialty care, hospitalizations, and medications. Missing clinical information raises concerns about medical errors and research integrity. Given the fragmentation of health care and poor EHR interoperability, information exchange, and usability, priorities for further investment in health IT will need thoughtful reconsideration.

Effect of Pediatric Behavioral Health Screening and Colocated Services on Ambulatory and Inpatient Utilization

OBJECTIVE

The study sought to determine the impact of a pediatric behavioral health screening and colocation model on utilization of behavioral health care.

METHODS

In 2003, Cambridge Health Alliance, a Massachusetts public health system, introduced behavioral health screening and colocation of social workers sequentially within its pediatric practices. An interrupted time-series study was conducted to determine the impact on behavioral health care utilization in the 30 months after model implementation compared with the 18 months prior. Specifically, the change in trends of ambulatory, emergency, and inpatient behavioral health utilization was examined. Utilization data for 11,223 children ages ≥4 years 9 months to <18 years 3 months seen from 2003 to 2008 contributed to the study.

RESULTS

In the 30 months after implementation of pediatric behavioral health screening and colocation, there was a 20.4% cumulative increase in specialty behavioral health visit rates (trend of .013% per month, p=.049) and a 67.7% cumulative increase in behavioral health primary care visit rates (trend of .019% per month, p<.001) compared with the expected rates predicted by the 18-month preintervention trend. In addition, behavioral health emergency department visit rates increased 245% compared with the expected rate (trend .01% per month, p=.002).

CONCLUSIONS

After the implementation of a behavioral health screening and colocation model, more children received behavioral health treatment. Contrary to expectations, behavioral health emergency department visits also increased. Further study is needed to determine whether this is an effect of how care was organized for children newly engaged in behavioral health care or a reflection of secular trends in behavioral health utilization or both.

Did Medicare Part D Affect National Trends in Health Outcomes or Hospitalizations? A Time-Series Analysis

BACKGROUND

Medicare Part D increased economic access to medications, but its effect on population-level health outcomes and use of other medical services remains unclear.

OBJECTIVE

To examine changes in health outcomes and medical services in the Medicare population after implementation of Part D.

DESIGN

Population-level longitudinal time-series analysis with generalized linear models.

SETTING

Community.

PATIENTS

Nationally representative sample of Medicare beneficiaries (n = 56,293 [unweighted and unique]) from 2000 to 2010.

MEASUREMENTS

Changes in self-reported health status, limitations in activities of daily living (ADLs) (ADLs and instrumental ADLs), emergency department visits and hospital admissions (prevalence, counts, and spending), and mortality. Medicare claims data were used for confirmatory analyses.

RESULTS

Five years after Part D implementation, no clinically or statistically significant reductions in the prevalence of fair or poor health status or limitations in ADLs or instrumental ADLs, relative to historical trends, were detected. Compared with trends before Part D, no changes in emergency department visits, hospital admissions or days, inpatient costs, or mortality after Part D were seen. Confirmatory analyses were consistent.

LIMITATIONS

Only total population-level outcomes were studied. Self-reported measures may lack sensitivity.

CONCLUSION

Five years after implementation, and contrary to previous reports, no evidence was found of Part D's effect on a range of population-level health indicators among Medicare enrollees. Further, there was no clear evidence of gains in medical care efficiencies.

Reducing the Use of H2-Receptor Antagonists in the Long-Term-Care Setting

OBJECTIVES

To examine the patterns of H2 blocker use in the long-term-care setting and to assess the effect of educational interventions designed to improve H2 blocker utilization patterns.

DESIGN

Time-series quasi-experimental study and retrospective chart review.

SETTING

A large academically-oriented long-term-care facility.

PATIENTS

Institutionalized elderly patients with a mean age of 88 years receiving H2 blocker therapy.

INTERVENTIONS

Two interventions involving group discussions with the medical staff, supporting educational materials, and physician-specific listings of patients receiving H2 blockers were employed sequentially over a 32-month period.

RESULTS

Each intervention resulted in substantial reductions in medication use (59.6% and 32.1%, respectively). Indications for H2 blocker use were determined retrospectively for patients identified as receiving therapy prior to the interventions (n = 110). Forty-one percent were found to be receiving therapy for reasons unsubstantiated by the medical literature. These patients were more likely to be discontinued from therapy than those receiving therapy for substantiated indications (P less than 0.01), consistent with the primary focus of the educational interventions.

CONCLUSIONS

These results suggest that the excessive use of H2 blocker therapy in the long-term care setting responds to educational interventions with therapeutically appropriate reductions in utilization. Repeated interventions are necessary to maintain such reductions over time although there may be some reduction in the effectiveness of the intervention with repetition.

Asthma Treatments and Mental Health Visits After a Food and Drug Administration Label Change for Leukotriene Inhibitors

PURPOSE

In 2009, the US Food and Drug Administration (FDA) mandated a label change for leukotriene inhibitors (LTIs) to include neuropsychiatric adverse events (eg, depression and suicidality) as a precaution. This study investigated how this label change affected the use of LTIs and other asthma controller medications, mental health visits, and suicide attempts.

METHODS

We analyzed data (2005-2010) from 5 large health plans in the US Population-Based Effectiveness in Asthma and Lung Diseases (PEAL) Network. The study cohort included children and adolescents (n = 30,000), young adults (n = 20,000), and adults (n = 90,000) with asthma. We used interrupted time series to examine changes in rates of LTI dispensings, non-LTI dispensings, mental health visits, and suicide attempts (using a validated algorithm based on a combination of diagnoses of injury or poisoning and psychiatric conditions).

FINDINGS

The label change was associated with abrupt reductions in LTI use among all age groups (relative reductions of 8.3%, 15.1%, and 6.0% among adolescents, young adults, and adults, respectively, compared with expected rates at 1 year after the warnings). Although we detected immediate offset increases in non-LTI asthma medication use, these increases were not sustained among adolescents and young adults. There were small increases in mental health visits among LTI users.

IMPLICATIONS

The FDA label change for LTIs communicated possible risk of neuropsychiatric events. Communication and enhanced awareness may have increased reporting of mental health symptoms among young adults and adults. It is important to assess intended and unintended consequences of FDA warnings and label changes.

Effects of eliminating drug caps on racial differences in antidepressant use among dual enrollees with diabetes and depression

PURPOSE

Black patients with diabetes are at greater risk of underuse of antidepressants even when they have equal access to health insurance. This study aimed to evaluate the impact of removing a significant financial barrier to prescription medications (drug caps) on existing black-white disparities in antidepressant treatment rates among patients with diabetes and comorbid depression.

METHODS

We used an interrupted time series with comparison series design and a 5% representative sample of all fee-for-service Medicare and Medicaid dual enrollees to evaluate the removal of drug caps on monthly antidepressant treatment rates. We evaluated the impact of drug cap removal on racial gaps in treatment by modeling the month-to-month white-black difference in use within age strata (younger than 65 years of age or 65 years of age or older). We compared adult dual enrollees with diabetes and comorbid depression living in states with strict drug caps (n = 221) and those without drug caps (n = 1133) before the policy change. Our primary outcome measures were the proportion of patients with any antidepressant use per month and the mean standardized monthly doses (SMDs) of antidepressants per month.

FINDINGS

The removal of drug caps in strict drug cap states was associated with a sudden increase in the proportion of patients treated for depression (4 percentage points; 95% CI, 0.03-0.05, P < 0.0001) and in the intensity of antidepressant use (SMD: 0.05; 95% CI, 0.03-0.07, P < 0.001). Although antidepressant treatment rates increased for both white and black patients, the white-black treatment gap increased immediately after Part D (0.04 percentage points; 95% CI, 0.01-0.08) and grew over time (0.04 percentage points per month; 95% CI, 0.002-0.01; P < 0.001).

IMPLICATIONS

Policies that remove financial barriers to medications may increase depression treatment rates among patients with diabetes overall while exacerbating treatment disparities. Tailored outreach may be needed to address nonfinancial barriers to mental health services use among black patients with diabetes.

Changes in drug coverage generosity and untreated serious mental illness: transitioning from Medicaid to Medicare Part D

IMPORTANCE

More than 1 in 5 disabled people with dual Medicare-Medicaid enrollment have schizophrenia or a bipolar disorder (ie, a serious mental illness). The effect of their transition from Medicaid drug coverage, which varies in generosity across states, to the Medicare Part D drug benefit is unknown. Many thousands make this transition annually.

OBJECTIVES

To determine the effect of transitioning from Medicaid drug benefits to Medicare Part D on medication use by patients with a serious mental illness and to determine the influence of Medicaid drug caps.

DESIGN, SETTING, AND PARTICIPANTS

In time-series analysis of continuously enrolled patient cohorts (2004-2007), we estimated changes in medication use before and after transitioning to Part D, comparing states that capped monthly prescription fills with states with no prescription limits. We used Medicaid and Medicare claims from a 5% national sample of community-dwelling, nonelderly disabled dual enrollees with schizophrenia (n = 5554) or bipolar disorder (n = 3675).

MAIN OUTCOMES AND MEASURES

Psychotropic treatments included antipsychotics for schizophrenia and antipsychotics, anticonvulsants, and lithium for bipolar disorder. We measured monthly rates of untreated illness, intensity of treatment, and overall prescription medication use.

RESULTS

Prior to Part D, the prevalence of untreated illness among patients with a bipolar disorder was 30.0% in strict-cap states and 23.8% in no-cap states. In strict-cap states, the proportion of untreated patients decreased by 17.2% (relatively) 1 year after Part D, whereas there was no change in the proportion of untreated patients in no-cap states. For patients with schizophrenia, the untreated rate (20.6%) did not change in strict-cap states, yet it increased by 23.3% (from 11.6%) in no-cap states. Overall medication use increased substantially after Part D in strict-cap states: prescription fills were 35.5% higher among patients with a bipolar disorder and 17.7% higher than predicted among schizophrenic patients; overall use in no-cap states was unchanged in both cohorts.

CONCLUSIONS AND RELEVANCE

The effects of transitioning from Medicaid to Medicare Part D on essential treatment of serious mental illness vary by state. Transition to Part D in states with strict drug benefit limits may reduce rates of untreated illness among patients with bipolar disorders, who have high levels of overall medication use. Access to antipsychotic treatment may decrease after Part D for patients with a serious mental illness living in states with relatively generous uncapped Medicaid coverage.

Potential bias of instrumental variable analyses for observational comparative effectiveness research

Instrumental variable analysis is an increasingly popular method in comparative effectiveness research (CER). In theory, the instrument controls for unobserved and observed patient characteristics that affect the outcome. However, the results of instrumental variable analyses in observational settings may be biased if the instrument and outcome are related through an unadjusted third variable: an "instrument-outcome confounder." The authors identified published CER studies that used instrumental variable analysis and searched the literature for potential confounders of the most common instrument-outcome pairs. Of the 187 studies identified, 114 used 1 or more of the 4 most common instrument categories: distance to facility, regional variation, facility variation, and physician variation. Of these, 65 used mortality as an outcome. Potential unadjusted instrument-outcome confounders were observed in all studies, including patient race, socioeconomic status, clinical risk factors, health status, and urban or rural residency; facility and procedure volume; and co-occurring treatments. Only 4 (6%) instrumental variable CER studies considered potential instrument-outcome confounders outside the study data. Many effect estimates may be biased by the failure to adjust for instrument-outcome confounding. The authors caution against overreliance on instrumental variable studies for CER.

National prevalence of receipt of antidepressant prescriptions by persons without a psychiatric diagnosis

OBJECTIVE

The study addressed recent concerns regarding increasing prescription of antidepressant drugs to patients with no recorded psychiatric diagnosis.

METHODS

Records from ten large integrated health systems in the Mental Health Research Network were used to examine diagnoses received by 1,011,946 health plan members who filled at least one antidepressant prescription in 2010.

RESULTS

Among individuals filling antidepressant prescriptions, psychiatric diagnoses recorded during the year were depressive disorders (48%), anxiety disorders (27%), bipolar disorders (3%), and attention deficit disorders (3%). The proportion of those filling prescriptions who had no psychiatric diagnosis was 39%, which fell to 27% after the analysis excluded prescriptions for antidepressants often prescribed for nonpsychiatric indications (tricyclic antidepressants, trazodone, and bupropion).

CONCLUSIONS

Prescription of antidepressants to patients without an appropriate diagnosis appears to be less common than previously reported.

Changes in antidepressant use by young people and suicidal behavior after FDA warnings and media coverage: quasi-experimental study

OBJECTIVE

To investigate if the widely publicized warnings in 2003 from the US Food and Drug Administration about a possible increased risk of suicidality with antidepressant use in young people were associated with changes in antidepressant use, suicide attempts, and completed suicides among young people.

DESIGN

Quasi-experimental study assessing changes in outcomes after the warnings, controlling for pre-existing trends.

SETTING

Automated healthcare claims data (2000-10) derived from the virtual data warehouse of 11 health plans in the US Mental Health Research Network.

PARTICIPANTS

Study cohorts included adolescents (around 1.1 million), young adults (around 1.4 million), and adults (around 5 million).

MAIN OUTCOME MEASURES

Rates of antidepressant dispensings, psychotropic drug poisonings (a validated proxy for suicide attempts), and completed suicides.

RESULTS

Trends in antidepressant use and poisonings changed abruptly after the warnings. In the second year after the warnings, relative changes in antidepressant use were -31.0% (95% confidence interval -33.0% to -29.0%) among adolescents, -24.3% (-25.4% to -23.2%) among young adults, and -14.5% (-16.0% to -12.9%) among adults. These reflected absolute reductions of 696, 1216, and 1621 dispensings per 100,000 people among adolescents, young adults, and adults, respectively. Simultaneously, there were significant, relative increases in psychotropic drug poisonings in adolescents (21.7%, 95% confidence interval 4.9% to 38.5%) and young adults (33.7%, 26.9% to 40.4%) but not among adults (5.2%, -6.5% to 16.9%). These reflected absolute increases of 2 and 4 poisonings per 100,000 people among adolescents and young adults, respectively (approximately 77 additional poisonings in our cohort of 2.5 million young people). Completed suicides did not change for any age group.

CONCLUSIONS

Safety warnings about antidepressants and widespread media coverage decreased antidepressant use, and there were simultaneous increases in suicide attempts among young people. It is essential to monitor and reduce possible unintended consequences of FDA warnings and media reporting.

Effects of Medicare Part D on guideline-concordant pharmacotherapy for bipolar I disorder among dual beneficiaries

OBJECTIVE

In January 2006 insurance coverage for medications shifted from Medicaid to Medicare Part D private drug plans for the six million individuals enrolled in both programs. Dual beneficiaries faced new formularies and utilization management policies. It is unclear whether Part D, compared with Medicaid, relaxed or tightened psychiatric medication management, which could affect receipt of recommended pharmacotherapy, and emergency department use related to treatment discontinuities. This study examined the impact of the transition from Medicaid to Part D on guideline-concordant pharmacotherapy for bipolar I disorder and emergency department use.

METHODS

Using interrupted-time-series analysis and Medicaid and Medicare administrative data from 2004 to 2007, the authors analyzed the effect of the coverage transition on receipt of guideline-concordant antimanic medication, guideline-discordant antidepressant monotherapy, and emergency department visits for a nationally representative continuous cohort of 1,431 adults with diagnosed bipolar I disorder.

RESULTS

Sixteen months after the transition to Part D, the proportion of the population with any recommended use of antimanic drugs was an estimated 3.1 percentage points higher than expected once analyses controlled for baseline trends. The monthly proportion of beneficiaries with seven or more days of antidepressant monotherapy was 2.1 percentage points lower than expected. The number of emergency department visits per month temporarily increased by 19% immediately posttransition.

CONCLUSIONS

Increased receipt of guideline-concordant pharmacotherapy for bipolar I disorder may reflect relatively less restrictive management of antimanic medications under Part D. The clinical significance of the change is unclear, given the small effect sizes. However, increased emergency department visits merit attention for the Medicaid beneficiaries who continue to transition to Part D.

Impact of Medicare's payment policy on mediastinitis following coronary artery bypass graft surgery in US hospitals

BACKGROUND

The Centers for Medicare and Medicaid Services (CMS) implemented a policy in October 2008 to eliminate additional Medicare payment for mediastinitis following coronary artery bypass graft (CABG) surgery.

OBJECTIVE

To evaluate the impact of this policy on mediastinitis rates, using Medicare claims and National Healthcare Safety Network (NHSN) prospective surveillance data.

METHODS

We used an interrupted time series design to compare mediastinitis rates before and after the policy, adjusted for secular trends. Billing rates came from Medicare inpatient claims following 638,761 CABG procedures in 1,234 US hospitals (January 2006-September 2010). Prospective surveillance rates came from 151 NHSN hospitals in 29 states performing 94,739 CABG procedures (January 2007-September 2010). Logistic regression mixed-effects models estimated trends for mediastinitis rates.

RESULTS

We found a sudden drop in coding for index admission mediastinitis at the time of policy implementation (odds ratio, 0.36 [95% confidence interval (CI), 0.23-0.57]) and a decreasing trend in coding for index admission mediastinitis in the postintervention period compared with the preintervention period (ratio of slopes, 0.83 [95% CI, 0.74-0.95]). However, we saw no impact of the policy on infection rates as measured using NHSN data. Our results were not affected by changes in patient risk over time, heterogeneity in hospital demographics, or timing of hospital participation in NHSN.

CONCLUSIONS

The CMS policy of withholding additional Medicare payment for mediastinitis on the basis of claims-based evidence of infection was associated with changes in coding for infections but not with changes in actual infection rates during the first 2 years after policy implementation.

A critical review of methods to evaluate the impact of FDA regulatory actions

PURPOSE

To conduct a synthesis of the literature on methods to evaluate the impacts of FDA regulatory actions and identify best practices for future evaluations.

METHODS

We searched MEDLINE for manuscripts published between January 1948 and August 2011 that included terms related to FDA, regulatory actions, and empirical evaluation; the review additionally included FDA-identified literature. We used a modified Delphi method to identify preferred methodologies. We included studies with explicit methods to address threats to validity and identified designs and analytic methods with strong internal validity that have been applied to other policy evaluations.

RESULTS

We included 18 studies out of 243 abstracts and papers screened. Overall, analytic rigor in prior evaluations of FDA regulatory actions varied considerably; less than a quarter of studies (22%) included control groups. Only 56% assessed changes in the use of substitute products/services, and 11% examined patient health outcomes. Among studies meeting minimal criteria of rigor, 50% found no impact or weak/modest impacts of FDA actions and 33% detected unintended consequences. Among those studies finding significant intended effects of FDA actions, all cited the importance of intensive communication efforts. There are preferred methods with strong internal validity that have yet to be applied to evaluations of FDA regulatory actions.

CONCLUSIONS

Rigorous evaluations of the impact of FDA regulatory actions have been limited and infrequent. Several methods with strong internal validity are available to improve trustworthiness of future evaluations of FDA policies.