Can administrative data be used to compare postoperative complication rates across hospitals?

BACKGROUND

Several quality assessment systems use administrative data to identify postoperative complications, with uncertain validity.

OBJECTIVES

To determine how accurately postoperative complications are reported in administrative data, whether accuracy varies systematically across hospitals, and whether serious complications are more consistently reported.

DESIGN

Retrospective cohort.

SUBJECTS

Nine hundred ninety-one randomly sampled adults who underwent elective lumbar diskectomies at 30 nonfederal acute care hospitals in California in 1990 to 1991. Hospitals with especially low or high risk-adjusted complication rates, and patients who experienced complications, were over sampled.

MEASURES

Postoperative complications were specified by reviewing medical literature and consulting clinical experts; each complication was mapped to ICD-9-CM. Hospital-reported complications were compared with our independent recoding of the same records.

RESULTS

The weighted sensitivity, specificity, and positive and negative predictive values for reported complications were 35%, 98%, 82%, and 84%, respectively. The weighted sensitivity was 30% for serious, 40% for minor, and 10% for questionable complications. It varied from 21% among hospitals with fewer complications than expected to 45% among hospitals with more complications than expected. Only reoperation, bacteremia/sepsis, postoperative infection, and deep vein thrombosis were reported with at least 60% sensitivity. Half of the difference in risk-adjusted complication rates between low and high outlier hospitals was attributable to reporting variation.

CONCLUSIONS

ICD-9-CM complications were underreported among diskectomy patients, especially at hospitals with low risk-adjusted complication rates. The validity of using coded complications to compare provider performance is questionable, even with careful efforts to identify serious events, although these results must be confirmed using more recent data.

[The use of randomisation in clinical studies in rehabilitation medicine: basics and practical aspects]

New therapies in rehabilitation medicine have to be evaluated with clinical trials. For drug approval the methodology of clinical trials is standardized world wide and the results of these studies are widely accepted. This standard should be achieved in clinical trials in rehabilitation research, too. One of the standards is the existence of a control group, comparing the effect of the new intervention against controls. In addition, the investigational and control groups must be equal in terms of the structure of possible confounders. Randomisation is the best possibility to distribute the patients to the therapy-groups, confounders will be equally distributed by chance. Other procedures for assignment to the study groups can result in confounding and lead into biased results. In spite of these advantages, randomisation is not generally accepted in rehabilitation research up to now. There are some reservations, mostly ethical, organisational and methodological ones. However, randomised clinical trials should be conducted in rehabilitation research in order to obtain more convincing results. Our intention is to bring some input in this debate and to present basics and practical aspects of randomisation.

Research governance in health and social care

In all aspects of our professional, personal and social life we will be faced with issues that need clear authority. Governance is about having a well-defined agreed set of standards by which we work. The purpose of this article is to raise the level of awareness of research governance in health and social care in the UK and to encourage colleagues to look at their local research governance framework critically.

Why we should not seek individual informed consent for participation in health services research

Ethics committees now require that individuals give informed consent to much health services research, in the same way as for clinical research. This is misguided. Existing ethical guidelines do not help us decide how to seek consent in these cases, and have allowed managerial experimentation to remain largely unchecked. Inappropriate requirements for individual consent can institutionalise health inequalities and reduce access to services for vulnerable groups. This undermines the fundamental purpose of the National Health Service (NHS), and ignores our rights and duties as its members, explored here. Alternative forms of community consent should be actively pursued.

Mental competence, caregivers, and the process of consent: research involving Alzheimer's patients or others with decreasing mental capacity

Alzheimer's disease and other forms of dementia are among the fastest growing health problems in America. Dementia incidence tends to increase with age, and the elderly are the fastest growing segment of the population. Medical and social sciences research on dementia involving demented patients is both ongoing and necessary. However, as noted in a report of the Office for Human Subjects Research, “while research with intellectually impaired people generates valuable … data, it also provides significant ethical challenges.

Life course health development: an integrated framework for developing health, policy, and research

The life course health development (LCHD) framework organizes research from several fields into a conceptual approach explaining how individual and population health develops and how developmental trajectories are determined by interactions between biological and environmental factors during the lifetime. This approach thus provides a construct for interpreting how people's experiences in the early years of life influence later health conditions and functional status. By focusing on the relationship between experiences and the biology of development, the LCHD framework offers a better understanding of how diseases occur. By suggesting new strategies for health measurement, service delivery, and research, as well as for improving health outcomes, this framework also supports health care-purchasing strategies to develop health throughout life and to build human health capital.

Enhancing protections for human participants in clinical and health services research: a continuing process

The authors provide brief information about the Department of Veterans Affairs (VA) and the Veterans Health Administration, to establish an understanding of this large, multifaceted organization. They then offer an overview of the VA's Office of Research and Development, with particular emphasis on recent and ongoing efforts to enhance protection of the rights of human participants in clinical research. The authors discuss various policies and regulations intended to protect human participants in any type of VA research. The protection of human research participants must be a major concern for all researchers. Patients should be well informed about the scope of the research and what their participation entails, and there should be no question as to the voluntariness of their consent. Making sure that these standards are met for each and every veteran who participates in VA research is of the utmost importance.

Lack of correlation between facility-based standardized rates of transplantation and mortality

The standardized mortality ratio (SMR) has been used to provide information about adjusted survival outcomes at dialysis facilities. There has been concern that high rates of transplantation could unjustly lead to unfavorable SMR profiles for individual dialysis units because healthier patients would be removed from dialysis therapy, leaving less healthy patients in the dialysis pool. We correlated 1999 overall adjusted SMR and 1999 standardized transplantation ratio (STR) weighted for mortality patient count and count of first transplantations of patients younger than 65 years. A total of 2,362 facilities were included in analyses. We found no correlation between rates of transplantation (by STR) and overall mortality profile (by SMR) based on Pearson's correlation coefficients (r), either unweighted, weighted by number of patients included in the 1999 mortality calculation (SMR), or weighted by number of patients included in the 1999 transplantation calculation (r = -0.016, r = -0.015, and r = -0.015, respectively; P > 0.40 for each). Sensitivity analyses using SMR and STR over 3- and 3.5-year periods (January 1997 to June 2000) also showed no correlation between SMR and STR, respectively. We conclude that reported standardized rates for transplantation do not correlate with those reported for mortality by dialysis facilities.

Economic evaluation of epilepsy treatment: a review of the literature

This article provides an overview of methods used and findings from economic analyses in epilepsy. Cost-effectiveness studies have evaluated different drugs for monotherapy and add-on therapy, and compared alternative treatment modalities for refractory epilepsy. The methodological characteristics of these studies are examined, and their results are compared and interpreted. Health outcome measures are defined and data sources described. Methods for assessing the direct and indirect costs, and/or cost savings, with a treatment's use, are explored. Directions for future research are identified and discussed.

Current opinion: consensus statement on intrauterine contraception

Forty-five experts from around the world attended a 1-day seminar in September 2001 in Chapel Hill, North Carolina, USA, to identify ways that they might collaborate to overcome unnecessary barriers to the use of intrauterine devices (IUDs). Seminar participants formed working groups that produced at least three specific recommendations relating to: training/performance improvement; service delivery improvement; general public information; and clinical and programmatic research. Key recommendations included: integrating reproductive health knowledge and skills into curricula for all healthcare professionals; reviewing and reinforcing with providers evidence-based guidelines for IUD use; encouraging evidence-based review of the IUD label and package insert; and conducting further research about IUD client eligibility, potential health benefits, acceptability among clients and providers, and use by HIV-infected women. At the meeting's conclusion, a number of participants, representing the fields of research, policy, communications, donors, women's advocacy, and medicine, expressed an interest in refining and acting upon the recommendations. Hosted by Family Health International, the meeting was supported by the Mellon Foundation.

Assessing efficacy of complementary medicine: adding qualitative research methods to the "Gold Standard"

Randomized controlled trials (RCTs) have an important place in the assessment of the efficacy of complementary and alternative medicine (CAM). However, they address only one, limited, question, namely whether an intervention has-statistically-an effect. They do not address why the intervention works, how participants are experiencing the intervention, and/or how they give meaning to these experiences. Therefore, we argue that the addition of qualitative research methods to RCTs can greatly enhance understanding of CAM interventions. Qualitative research can assist in understanding the meaning of an intervention to patients as well as patients' beliefs about the treatment and expectations of the outcome. Qualitative research also assists in understanding the impact of the context and the process of the intervention. Finally, qualitative research is helpful in developing appropriate outcome measures for CAM interventions. Greater understanding of CAM interventions has the potential to improve health care delivery.

Searching literature databases for health care economic evaluations: how systematic can we afford to be?

BACKGROUND

As the health care economic-evaluation literature continues to grow, a need for sound methods to conduct systematic reviews of the existing evidence is emerging. So far, reviews of economic evaluations have relied upon noncomprehensive sources and have adopted simplistic search methods, both likely to lead to biased results.

OBJECTIVE

To provide evidence of the performance of alternative approaches for identifying published health care economic evaluations and to illustrate what forms of bias may affect systematic reviews of such studies.

METHODS

The sensitivity and specificity of alternative search strategies were tested for the period January to March 1997, using seven major medical and social science literature databases, one economic evaluation database and a published bibliography. Studies were selected blindly by pairs of reviewers (agreement 94.1%-96.5%), using a two-stage procedure.

RESULTS

By limiting the scope of a review to Medline and by using appropriate search strategies, researchers may significantly reduce the number of nonrelevant references retrieved by their electronic searches (sensitivity 72%, specificity 75%, compared with more extensive strategies), which require exclusion by manual screening. The yield of searches based on specialized bibliographies and databases may be different because of variations in selection criteria, coverage and time lag for inclusion of references.

CONCLUSIONS

Medline is the key source for reviews of economic evaluations. Researchers may select from the search strategies proposed in this paper the one that offers an optimal balance between sensitivity and specificity in relation to the aims of their review. Manual searches and searches of databases other than Medline have a limited incremental yield. The sensitivity of all search strategies increases when tighter methodological standards are set, but more research is needed on methods for identifying methodologically sound studies.

Ethics of qualitative research: are there special issues for health services research?

BACKGROUND

An increasing volume of qualitative research and articles about qualitative methods has been published recently in medical journals. However, compared with the extensive debate in social sciences literature, there has been little consideration in medical journals of the ethical issues surrounding qualitative research. A possible explanation for this lack of discussion is that it is assumed commonly that qualitative research is unlikely to cause significant harm to participants. There are no agreed guidelines for judging the ethics of qualitative research proposals and there is some evidence that medical research ethics committees have difficulty making these judgements.

OBJECTIVES

Our aim was to consider the ethical issues which arise when planning and carrying out qualitative research into health and health care, and to offer a framework within which health services researchers can consider these issues.

RESULTS

Four potential risks to research participants are discussed: anxiety and distress; exploitation; misrepresentation; and identification of the participant in published papers, by themselves or others. Recommended strategies for reducing the risk of harm include ensuring scientific soundness, organizing follow-up care where appropriate, considering obtaining consent as a process, ensuring confidentiality and taking a reflexive stance towards analysis.

CONCLUSIONS

While recognizing the reservations held about strict ethical guidelines for qualitative research, we argue for further debate of these issues so that the health services research community can move towards the adoption of agreed standards of good practice. In addition, we suggest that empirical research is desirable in order to quantify the actual risks to participants in qualitative studies.

Quality of clinical reports on behavioral interventions for hypertension

PURPOSE

The purpose of this study was to provide clinicians with the confidence to employ behavioral interventions for hypertension and to guide researchers in the development and reporting of studies.

METHODS

We systematically reviewed English language articles (1970-1999) describing behavioral interventions for hypertension, evaluating aspects of design, analysis, reporting of results, and factors that were associated with higher quality of these studies.

RESULTS

Of 100 articles, 49 were randomized controlled trials (RCT), 33 were observational studies with control groups, and 18 were observational studies without control groups; mean (SE) quality scores were 69.2 (1.6), 57.6 (5.3), and 60.3 (2.2), respectively. RCTs were more likely than observational studies to attain high scores in descriptions of appropriateness of control group, inclusion and exclusion criteria, study population, and the intervention protocol. In multivariate analysis, date of publication, reported funding source, and intervention type were independently associated with greater quality scores: 7.4 [95% CI: 0.03, 14.7] points greater for articles published 1990-1999 vs 1970-1979, 6.5 [95% CI: 1.4, 11.6] points greater for articles reporting government funding vs those not reporting funding sources, and 8.6 [95% CI: 0.3, 17.1], 12.9 [95% CI: 3.4, 22.4], and 14.2 [4.1, 24.4] points greater for articles examining patient education/support, change in delivery system, and mass health campaigns vs articles examining patient reminders, respectively.

CONCLUSIONS

While quality has improved over time, there is considerable room for improvement. Investigators should pay particular attention to description of study population and allocation of subjects, the use of standardized outcomes reporting, and appropriate statistical analysis.

Do patient consent procedures affect participation rates in health services research?

BACKGROUND

Few studies have examined the effects of Institutional Review Board (IRB) requirements to contact potential research participants.

OBJECTIVE

To examine the association between requirements to contact potential research subjects and participation rates in a multisite health services research study. RESEARCH DESIGN, SUBJECTS: Prospective observational study of survey participation by 2673 individuals with diabetes and 1974 individuals with congestive heart failure treated at 15 clinical sites in the United States that had implemented a quality improvement intervention.

MAIN OUTCOME MEASURES

Telephone survey response rates.

RESULTS

Of 15 IRBs, seven required sites to obtain authorization from participants to release contact information to the study team. Five required oral and two required written advance permission. The response rate was 58% (913/1571) at sites where no advance permission was required, 39% (989/2530) from sites that required oral advance permission and 27% (145/546, P <0.001) at sites requiring written advance permission. Although 85% of eligible participants contacted directly by the study team consented to complete the survey, only 43% of individuals at sites requiring written advance permission allowed the release of contact information to the study team.

CONCLUSIONS

Many potential participants did not grant advance permission to be contacted by the study team. Requiring advance permission reduced participation rates, especially at sites requiring written authorization.

Evidence-based practice for mere mortals: the role of informatics and health services research

The poor translation of evidence into practice is a well-known problem. Hopes are high that information technology can help make evidence-based practice feasible for mere mortal physicians. In this paper, we draw upon the methods and perspectives of clinical practice, medical informatics, and health services research to analyze the gap between evidence and action, and to argue that computing systems for bridging this gap should incorporate both informatics and health services research expertise. We discuss 2 illustrative systems--trial banks and a web-based system to develop and disseminate evidence-based guidelines (alchemist)--and conclude with a research and training agenda.

Benefits and drawbacks of including consumer reviewers in the scientific merit review of breast cancer research

BACKGROUND

This study assessed participant opinions about inclusion of breast cancer survivors as lay representatives in a scientific and technical merit review of proposals for the 1995 Department of Defense Breast Cancer Research Program (DOD BCRP).

METHODS

The evaluation employed a prepanel and postpanel survey design, which was intended to elicit feedback about attitudes, perceptions, and beliefs toward collaborative consumer and scientist participation in scientific merit review. Qualitative methods were used to describe the consumers' and scientists' responses, to explore the significance of this interaction, and to gain an understanding of the benefits and disadvantages of bringing these participants together.

RESULTS

Both groups were initially troubled about the consumers' lack of scientific background and questioned their qualifications and preparation for participation in a scientific panel. In particular, consumers were concerned that their judgments would not be taken seriously by scientists, a concern somewhat lessened by participation. After the meeting, scientists viewed the consumers as hard-working, dedicated survivors and advocates and endorsed the presence of carefully chosen lay panel members. Scientists were troubled that consumers potentially would have an impact on voting and on the subsequent scoring of proposals, a concern that was not validated by quantitative findings.

CONCLUSIONS

As a result of these data, the DOD BCRP continues to embrace clarify the nature of collaborative participation in scientific merit review.

Health services research: what is it and what does it offer?

Over the last 20 years, clinical medicine has witnessed rapid expansion in its underlying evidence base, greater demand for accountability in clinicians' use of limited resources and increasing societal expectation for health care that confers proven benefit at reasonable cost to all eligible recipients. Health services research, also referred to as the clinical evaluative sciences, has grown in response to the need for objective empirical analysis of the modern health system's ability to deliver effective, efficient, equitable and safe care and to further the health and well-being of whole populations. In this article we provide an overview of the aims, methods and outputs of this burgeoning new discipline.