A Monte Carlo Investigation of Methods for Controlling Type I Errors with Specification Searches in Structural Equation Modeling

A standard strategy in structural equation modeling is to conduct multiple Lagrange multiplier (LM) tests after rejection of an initial model. Controlling for Type 1 error across these tests minimizes the likelihood of including unnecessary additional parameters in the model. Three methods for controlling Type I errors are evaluated using simulated data for factor analytic models: the standard approach which involves testing each parameter at the .05 level, a Bonferroni approach, and a simultaneous test procedure (STP). In the first part of the study, all samples were generated from a population in which all null hypotheses associated with the LM tests were correct. Three factors were manipu1,~ted: factor weights, sample size, and number of parameters in the specification search. The standard and the STP approaches yielded overly liberal and overly conservative familywise error rates, respectively, while the Bonferroni approach yielded error rates closer to the nominal level. In the second part of the study, data were generated in which one or more null hypotheses associated with the LM test were incorrect, and the number of parameters in the search was manipulated. Again the Bonferroni method was the best approach in controlling familywise: error rate, particularly when the alpha level was adjusted for the number of parameters evaluated at each step.

Contraceptive implants: long acting and provider dependent contraception raises concerns about freedom of choice

David Bromham's editorial on contraceptive implants ignores the wider issues to voice concern that trial by media could limit contraceptive choice by jeopardising research into new methods. However, it is more beneficial to the public for points of conflict to be debated openly. Furthermore, the impetus for research into new contraceptive technology is driven by profit and political motives and is only marginally affected by the media. Implanted contraceptives may increase the choice of contraceptive methods, but they put control of fertility increasingly into the hands of the medical profession. Herein lies their greatest problem: their potential to increase providers' control over clients' choice. There is the danger that certain groups of women may be targeted for their use: in the United States the coercive use of Norplant for mothers receiving welfare benefit has been suggested. Long acting contraceptives are a contraceptive of choice only when they are available without pressure, as part of a wider menu; when instant removal on request is guaranteed; and when there is an open and free flow of information and opinions between users, health professionals, and special interest groups.

The use of the arthroscope as an adjunct in the resection of a chondroblastoma of the femoral head

We present a case of the use of an arthroscope as an adjunct in the resection of a tumor (chondroblastoma) of the femoral head. Use of the endoscope in this case allowed preservation of the femoral head and neck. Although the arthroscope has been used intraarticularly for the resection of tumors, to our knowledge it has not been used endosteally.

One peer review organization's experience in developing hospital peer groups

The Health Care Quality Improvement Initiative is moving Medicare's quality improvement activities from review and action on individual cases to the analysis of patterns of care. A primary source for pattern analysis is the mortality data presented in the Medicare Hospital Information release. One of the requirements set forth by the Health Care Financing Administration is that peer review organizations classify hospitals into peer groups in order to compare mortality rates within and among groups. It is hoped that this type of analysis will lead to a better understanding of the relationship between process and outcome for a variety of medical conditions. This report describes the experience of one peer review organization in establishing hospital peer groups.

The cost effectiveness of auranofin: results of a randomized clinical trial

In a 6-month randomized trial at 14 sites, the cost effectiveness of auranofin (AF) treatment for patients with rheumatoid arthritis was gauged in comparison with placebo. Measures of global health and of impacts on daily life suggest that the benefits of disease modification outweigh adverse effects after 4 and 6 months of treatment (p less than 0.01), with negligible differences between placebo and treated patients after 1 and 2 months. Additional medical costs directly associated with AF treatment amounted to $778/patient annually. Observed differences in less direct medical costs, help received, and earnings were not statistically significant.

Cost-effectiveness of total joint arthroplasty in osteoarthritis

Although total joint replacement (TJR) is a major advance in the treatment of patients with osteoarthritis, its cost-effectiveness has been questioned. We report the results of a study of the costs and benefits of TJR in consecutive osteoarthritis patients, 6 months after the surgery. Health status was measured by the Index of Well-Being. Costs of services for arthritis were determined by interview and billing records. Six months after TJR, significant improvements were seen in global health and in functional status. The average cost of care for the 6 months prior to TJR was $933. The average cost during the 6 months beginning with the TJR was $22,730 per patient--due almost entirely to costs of surgery. In general, the surgery did not change work status, probably because the mean age of the patients was 66.4 years. There were large effectiveness/cost differentials (the larger the effectiveness/cost differential, the higher the degree of cost-effectiveness [CE]). At 6 months, for all patients, the CE was associated with initial health status. The highest CE was observed in 10 patients who initially had the poorest health. TJR is more cost-effective for patients with the most to gain and less effective for those with better preoperative health status.

Willingness to pay and accept risks to cure chronic disease

Measurements of disease burden focus most often on economic outputs--neglecting effects on quality of life. More comprehensive quantification is based on what people would pay or risk to avoid illness. Many, however, find it difficult to respond thoughtfully to hypothetical questions about what they would pay or risk. With response rates frequently under 50 per cent, the practicality of these methods has been of concern. In this study, specially trained interviewers asked 247 subjects with rheumatoid arthritis how much of their income they would pay and how large a mortal risk they would accept to achieve a hypothetical cure. Ninety-eight per cent of the subjects estimated their maximum acceptable risk (MAR) at an average 27 per cent chance of immediate death. Eighty-four per cent gave plausible responses to the willingness-to-pay (WTP) questions, with a mean WTP of 22 per cent of household income. The aspect of disease most strongly associated with WTP was impairment in activities of daily living; measured pain was most associated with MAR. The response rates achieved indicate the overall feasibility of these methods; the associations of WTP and MAR with other variables suggest systematic consideration of personal circumstances.

A randomized controlled trial of quality assurance in sixteen ambulatory care practices

A crossover randomized controlled trial of cycles of quality assurance in 16 primary care (8 medical, 8 pediatric) group practices was conducted. Of four medical and four pediatric tasks important to patient outcome, two were randomly assigned to experimental intervention (a quality assurance cycle), and two were also measured and used as blinded controls for each medical or pediatric group practice. Task performance was measured in each group for 12 months prior to, 9 months during, and 9 months after the experimental intervention, using as a performance score the percentage of evaluation criteria failed of those applicable to a case. As a result of quality assurance intervention, quality of performance was significantly improved in two of the tasks (P less than 0.0001, with 6.7, and 9.8 percentage points improvement), and marginally improved in one task (P = 0.06, 5.7 percentage points improvement). Surprisingly, tasks with lower perceived effect on patient health (low physician motivation) had greater improvement in quality. Unimproved tasks were associated with the perceived need for delivery system changes beyond the immediate control of the individual practitioner.

Resource requirements for evaluating ambulatory health care

We implemented the most frequently used form of quality assurance activity: abstracting information on the quality of patient care from medical records and communicating findings to providers in 16 ambulatory care groups. Site providers accepted the evaluation criteria, agreed that deficiencies in care were detected, and, for some medical tasks, effected improvements in care. Direct costs in 1980 dollars for the quality assurance cycle including data system development were $46 per evaluated case. Per-case costs varied considerably among tasks, decreased with larger numbers of cases and as experience grew, and were reduced through computerization. Measured costs were high due to: a demanding research design; our extended accounting of direct, indirect, and induced costs; and the substantial resource requirements of rigorously performed evaluations.

Evaluation of a computer-assisted medication refill reminder system for improving patient compliance

Computer-generated refill reminder notices were mailed to patients receiving continual medication for cardiovascular diseases to measure improved compliance and to discover whether a computer-assisted program was economically viable. Guidelines were established to define compliance. A computer-assisted compliance intervention program did not significantly improve the rate at which patients had their prescriptions filled "on time" and the mean compliance rate for both experimental and control groups was greater than 79%. Also discussed were cost and compliance strategy implications and the receptiveness of patients to the reminder program.

Quality assurance in eight adult medicine group practices

Four evaluations of ambulatory medical care tasks were developed for use in quality assurance. The evaluations used medical records data and explicit criteria incorporating branching logic. They were implemented in eight general medicine provider groups in two teaching hospitals and six related health centers. Agreement with criteria among 316 provider responses to questionnaires varied from 57% to 100%. The percentage of cases with one or more variation from evaluation criteria, confirmed on peer review to have a deficiency in care, ranged by task from 6% to 42%, with substantial variation between sites. Physician reviewers from each site varied in leniency. Numbers of actions taken to correct deficiencies ranged by site and task from zero to six. Multisite evaluations revealed differences in performance and efforts to improve that are not apparent when each site conducts its own evaluations. More uniformly effective and impartial quality assurance is needed to correct some important deficiencies in care observed in this study.

Feasibility of willingness-to-pay measurement in chronic arthritis

Quantification of nonmonetary aspects of disease is a major challenge for economic analysts. Using the amounts of money recipients are willing to pay for nonmonetary benefits has theoretical appeal, but it has proven difficult to implement. Difficulties encountered include noncomprehension by subjects, misrepresentation of preferences, extraneous determinants of answers, and ethical concerns. In a preliminary exploration of feasibility, 184 patients with osteoarthritis and rheumatoid arthritis were asked their willingness to pay (WTP) for hypothetical complete cure. With minimal pressure put on the patients to respond, 27 percent gave plausible answers. People with more schooling, with paid employment, or who were having more treatments for their arthritis were more likely to respond. Patients were willing on average to pay 17 percent of family income for arthritis cure. Methods for measuring WTP are being strengthened and may soon play an important role in health services research.

A method for evaluating performance of ambulatory pediatric tasks

Four evaluations of ambulatory pediatric tasks were used for quality assurance in eight pediatric group practices situated in two teaching hospitals and six related health centers. The evaluations used criteria incorporating branching logic to judge the quality of care revealed in data abstracted from medical records. Performance was evaluated for follow-up of positive urine cultures, assessment and follow-up of otitis media, initial assessment for gastroenteritis, and "well child" care for infants. A computerized data system was developed to process evaluation data and produce easily read reports. This work is part of a controlled trial of the feasibility, cost, and effectiveness of quality assurance as a means to improve patient care, but this preliminary report concerns only the principles for design of the evaluations and their use in quality assurance. Acceptance of evaluations by site providers was high: of 203 provider responses to a survey, only four reported disagreement with the criteria. Rates of cases "variant" from criteria and found on peer review to represent deficiencies in care, when averaged across sites, ranged by task from 1% to 47% of cases evaluated. In most sites, providers planned and implemented actions to correct these deficiencies. It is noted that improvements in care may increase costs of care.

Physician perceptions of medical malpractice and defensive medicine

Using judgments obtained in interviews with 33 Massachusetts physicians, the annual statewide volume of expenditures incurred for defensive medical reasons in 1982 was estimated to be $1.0 billion, 12% of all medical care expenditures. Estimates for the nation were $37 billion, 14% of expenditures. Nationally, 180,000 cesarean deliveries were thought to be performed for defensive motives. In their own institutions, respondents judged 43% of all skull x-rays following injury to be medically justified, 30% to be defensive medicine, 16% to be placebos, and 11% to be physician misjudgments. In considering the economic and noneconomic costs of medical malpractice procedures, the dollar costs of insurance were considered most serious, followed closely by defensive medicine, unfairness, and poorer relations with patients. Thirty-two percent of the responsibility for the negative aspects of malpractice processes was assigned to lawyers, 21% to physicians, 18% to legislatures and courts, 16% to patients, and 13% to insurance companies.

The cost of quality assurance in medicine

Cost analysis has been frequently neglected in program evaluations but is currently of high relevance in policy decisions on quality assurance in medicine. The Ambulatory Care Medical Audit Demonstration (ACMAD) Project implemented and evaluated a program of medical record-based quality assurance in eleven sites for nine medical topics. Total direct costs for the project were $1.22 million over five years; indirect costs, $694,000. A computerized data system enabled disaggregation of the cost data by person, timing, type of work, project phase, health topic, health center, and research or operational nature. Of the costs incurred, 79% were for operational reasons, with 21% incurred for research reasons. Costs per audited case were 31% higher in hospitals than in neighborhood health centers. Audit topics of low per-case costs tended to have automated case findings, straightforward and limited abstracting, little need to examine multiple visits, and a low proportion of case-found patients ineligible for audit.

Health versus money. Value judgments in the perspective of decision analysis

An important, but largely uninvestigated, value trade-off balances marginal nonhealth consumption against marginal medical care. Benefit-cost analysts have traditionally, if not fully satisfactorily, dealt with this issue by valuing health gains by their effects on productivity. Cost-effectiveness analysts compare monetary and health effects and leave their relative valuations to decision makers. A decision-analytic model using the satisfaction or utility gained from nonhealth consumption and the level of health enables one to calculate willingness to pay--a theoretically superior way of assigning monetary values to effects for benefit-cost analysis-and to determine minimally acceptable cost-effectiveness ratios. Examples show how a decision-analytic model of utility can differentiate medical actions so essential that failure to take them would be considered negligent from actions so expensive as to be unjustifiable, and can help to determine optimal legal arrangements for compensation for medical malpractice.

Electronic fetal monitoring and clinical practice. A survey of obstetric opinion

Interviews with 12 obstetricians recognized for their scientific and clinical contributions in the use of electronic fetal monitoring (EFM) revealed notable areas of agreement and disagreement in the interpretation and use of these methods. In reviewing 14 abnormal fetal heart rate (FHR) patterns, the obstetricians displayed an average pairwise agreement of 68% in classifying the patterns as "innocuous," "nonreassuring," or "ominous." When these patterns persisted after corrective treatment, average pairwise agreement was 69% in deciding between continued monitoring and immediate delivery. With the additional option of scalp blood pH sampling, average agreement was 59%. For the set of FHR patterns studied, scalp blood pH sampling was recommended more often to confirm conservative management of labor than to verify the need to intervene. The obstetricians may be classified by their degrees of (1) alarm and (2) interventionism, and by their (3) frequency of and (4) motivation for scalp sampling. Associations among these four dimensions of behavior were limited.

Decision-analytic determination of study size. The case of electronic fetal monitoring

There is uncertainty about the efficacy of electronic fetal monitoring (EFM) in lowering perinatal mortality among births at low prospective risk. A randomized controlled trial offers the greatest promise of reducing this uncertainty. Haphazard methods of evaluating decision making have prevailed in the past and have led to study sizes too small to estimate effects on perinatal mortality. Statistical methods can determine the study size necessary to meet statistical parameters. Choice of these parameters is, however, somewhat arbitrary. Decision-analytic methods calculate the expected value of information (EVI) as the likely worth of future decision guidance. The optimal size, cost, and focus for an evaluation study can then be taken as those maximizing the net EVI after consideration of study cost. This methodology indicates that, in evaluating EFM, two randomly-assigned groups of roughly 180,000 births each should be studied. This would achieve net expected societal benefits estimated at $118 million at a cost of roughly $22 million. The optimal study size is somewhat sensitive to analytic parameters. If feasible, a superior dynamic strategy is to allow study findings to determine the ultimate study size.

Cost-effectiveness of screening for hypo- and hyperthyroidism in India

The development of relatively inexpensive radioimmunoassay techniques opens new possibilities for widespread screening for hypo- and hyperthyroidism in developing regions where iodine deficiency is great. From component cost analysis, it appears that radioimmunoassays of thyroid hormones can be performed in India for as little as 2.4 rupees (29 cents) per test. Cost-effectiveness analysis indicates that screening for hypothyroidism in iodine-deficient areas in India is, of alternative detection strategies, the most cost-effective: Cases are found at an average cost of 40 rupees ($4.80) per case. Cost breakdowns and considerations of convenience and acceptability indicate that filter paper methods may soon be preferred to whole-blood assays, especially for screening. Cost-effectiveness findings for these screening strategies and for reasonable modifications of them should be compared with the estimated cost-effectiveness of iodine supplementation in determining optimal health policy toward subclinical thyroid disease.