Nursing intervention studies require outcome measures that are sensitive to change: Part One

In selecting an outcome measure for a study evaluating a nursing intervention, the criterion of sensitivity to change should predominate. Researchers commonly justify their choice of outcome measures for experiments based on such psychometric criteria as high internal consistency reliability and patterns of correlations reflecting convergent and discriminant validity. Although such criteria are appropriate for measures to assess individual differences, they are insufficient when the measures will be used for intervention studies. Researchers may need to develop new measures that are tailored for experimental studies if existing measures are valid mainly for the assessment of individual differences. In this first portion of a two-part article, three of seven factors to consider in selecting an outcome measure for an intervention study are outlined and recommendations for application of principles of reliability and validity in the context of sensitivity to change are given.

Assessing providers of coronary revascularization: a method for peer review organizations

OBJECTIVES

Current methods to evaluate quality of care are usually limited to reviews of individual cases or comparisons of hospital mortality rates. We present an alternative method that compares complication rates adjusted for patient characteristics.

METHODS

Detailed clinical data that were specifically designed for quality comparisons of providers of revascularization procedures were abstracted from the medical records of 1998 Medicare patients, in 16 hospitals, who had coronary artery bypass surgery and 2091 patients, in 16 hospitals, who had angioplasty. Providers were ranked on the basis of an unadjusted risk, a risk adjusted for detailed clinical information, and a risk adjusted only for patient comorbidities.

RESULTS

Complication rates differed significantly and substantially among the hospitals. Clinical adjustment changed the hospital rankings for the bypass surgery hospitals, but not for the angioplasty hospitals. Adjustment for comorbidities did not affect hospital rankings for either procedure.

CONCLUSIONS

When sample sizes are limited, adverse outcome rates may be a more sensitive measure of quality of care than mortality rates. Rates that are unadjusted or adjusted only for comorbidities may be inadequate for evaluating some providers of bypass surgery.

Conceptual and methodological issues in nurse case management research

Conceptual and methodological issues facing researchers who are studying the process and outcomes of nurse case management are identified. The evolution of a research program designed to understand and evaluate one model of nurse case management is described as an example of responding to these issues. The development of a body of knowledge around nurse case management is essential to assure the expansion of this role and to justify reimbursement through current and evolving insurance mechanisms.

Program outcome and QA: a new approach to evaluation

Health care providers and managers are familiar with the limitations of current quality assurance (QA) practices, which do little to indicate the overall effectiveness of health care programs. This article discusses a comprehensive method of evaluating program effectiveness, efficiency, cost, client satisfaction, and adherence to standards. The authors present the Brockville Psychiatric Hospital's experience in evaluating a new program with data collected from a multi-faceted QA perspective, and suggest that combining program evaluation with QA could become the standard for future program assessments.

The relation between experience and outcome in heart transplantation

BACKGROUND

Current policies related to organ transplantation in the United States are designed to ensure that centers and physicians with experience in transplantation perform these procedures. It is essential to confirm the validity of such policies, since they may limit access to transplantation services.

METHODS

To determine the relation between experience with heart transplantation and mortality after the procedure, we merged data from the registry of the International Society for Heart and Lung Transplantation with data from a survey that provided additional information about patients and transplantation centers. Our study included 1123 patients who received a heart transplant at one of 56 hospitals in the United States from 1984 through 1986. We used univariate and bivariate techniques, as well as logistic regression, to analyze our data.

RESULTS

We observed an institutional learning curve for heart transplantation. Patients who received one of a center's first five transplants had higher mortality rates than patients who received a subsequent transplant (20 percent vs. 12 percent; P = 0.002; relative risk = 2.2; 95 percent confidence interval, 1.6 to 3.4). In addition, we found a correlation between the training of key personnel on the transplantation team and mortality at new transplantation centers. For example, new centers staffed by cardiologists with previous training in heart transplantation had lower mortality rates among heart-transplant recipients than centers without experienced cardiologists (7 percent vs. 16 percent; P = 0.001; relative risk = 2.7; 95 percent confidence interval, 1.3 to 5.9). By contrast, the previous training of the surgeons who performed transplantations was not related to the mortality rate associated with the procedure.

CONCLUSIONS

Experience with heart transplantation is associated with a better outcome for patients after that procedure. Opportunities exist to refine transplantation policies on the basis of the experience of a center and its transplantation team and to develop similar policies for other forms of organ transplantation.

An initial assessment of the Veterans Affairs occurrence screening program

The Department of Veterans Affairs (VA) established a computerized occurrence screening program in its medical centers in October 1988. Data collected from these hospitals suggest that occurrence screening has been a useful component of the VA's overall quality assurance effort; opportunities for improvement were found in 4.2% of all occurrences. When asked to name the three most effective criteria, there was strong agreement among participating hospitals--50% or more of the facilities ranked readmissions, death, and admission within three days of an unscheduled outpatient visit as the most effective. A majority of facilities (56%) named occurrence screening as one of the more effective elements in identifying quality-of-care issues. These findings must be balanced against important limitations of the occurrence screening process; however, guided by the data it collected, the VA has recently made several changes in its occurrence screening program to address these limitations.

A new measure of quality of life in depression: testing the reliability and construct validity of the QLDS

Our previous paper described the development of a new quality of life scale for use with people suffering from depression; the Quality of Life in Depression Scale (QLDS). This paper reports on the testing of the scale for reliability and construct validity. Reliability was assessed by giving the questionnaire to the same set of patients on two occasions 2 weeks apart. This test-retest technique yielded a correlation of 0.94, with high internal consistency at both time 1 and time 2. A test of split-half reliability also indicated very high reliability. Construct validity was measured by comparing scores on the QLDS with those on an established scale of well-being from the same group of patients. The results gave a correlation between the two measures of 0.79, giving a satisfactory validity. It is concluded that the QLDS is a reliable and valid measure which is easy to use and acceptable to patients. Further tests of discriminative, concurrent and criterion validity are planned.

Maternal health and the measurement trap

This paper focuses on the contribution of measurement-related factors to the neglect of maternal health in resource allocation for programmes and in public health research. As the recent interest in maternal health has now progressed beyond the need for information primarily for the purpose of advocacy, measurement-related factors have emerged as powerful constraints on programme action. Three outstanding needs for information can be identified: first, to establish the levels and trends of specific maternal health outcomes; secondly, to identify the characteristics and determinants of health outcomes; and thirdly, to monitor and evaluate the effectiveness of programmes designed to influence health outcomes. In order to meet these needs, the emphasis placed on operational research by the current major initiatives in maternal health must be complemented by an equivalent emphasis on methodological studies. The call for improved information by international and national agencies should be made in unison with the call for action. Inadequate information is a reality that has to be faced throughout the world, but particularly in developing countries. The quality, quantity and scope of health-related data are the elements of this inadequacy and may be discussed in terms of four factors: the indicators, the data sources, the measurement techniques, and the conceptual framework. In this paper, the neglect of maternal health and the lack of information are shown to be self-reinforcing and constitute a measurement trap sprung by these four factors. Dismantling this trap has revealed a weak conceptual framework to lie at the very centre. Maternal health has tended to be conceptualized as a discrete, negative state, characterized by physical rather than social or mental manifestations, and by a narrow time-perspective focusing on pregnancy, delivery and the puerperium. The need to broaden this perspective and to develop equally broad operational definitions represent important steps forward that must be taken.

Clinical versus administrative data bases for CABG surgery. Does it matter?

This study compared the ability of a clinical and administrative data base in New York State to predict in-hospital mortality and to assess hospital performance for coronary artery bypass graft surgery. The results indicated that the clinical data base, the Cardiac Surgery Reporting System, is substantially better at predicting case-specific mortality than the administrative data base, the Statewide Planning and Research Cooperative System. Also, correlations between hospital mortality rates that are risk-adjusted using the two systems were only moderately high (0.75 to 0.80). The addition of new risk factors from the Statewide Planning and Research Cooperative System improved the predictive power of both systems but did not diminish the difference in effectiveness of the two systems. The three unique clinical risk factors in the Cardiac Surgery Reporting System (ejection fraction, reoperation, and more than 90% narrowing of the left main trunk) seemed to account for much of the difference in effectiveness of the two systems.

The QLDS: a scale for the measurement of quality of life in depression

It has been apparent for some time that utilisation of health services is more influenced by how people feel than by their 'objective' symptoms. Research has also shown that compliance with treatment regimens is dependent on the impact of that treatment on the patient's well-being. Such findings highlight the need to assess the patient's view of his or her progress through the health care system. This has led to a growing interest in quality of life assessment and, in particular, to the impact of illness and its treatment on the quality of life of the patient. Perhaps surprisingly, interest in quality of life measures for outcome assessment in Psychiatry has been slower to develop than in other specialties. While a number of clinician- and patient-completed measures of the severity of depressive symptoms exist, no measure of the quality of life of depressed patients is available. This paper describes the development and testing of the Quality of Life in Depression Scale (QLDS), a new measure designed to fill this gap. The theoretical basis for the instrument is that life gains its quality from the ability and capacity of the individual to satisfy his or her needs. The items included in the QLDS were derived from 30 qualitative interviews with depressed or recently recovered patients. Further interviews were held with depressed patients to establish the face and content validity of the instrument. Respondents had little trouble completing the questionnaire and found the content to be relevant to their experience of depression. The QLDS represents an important advance in our ability to understand the impact of depression on quality of life.

Patient satisfaction as a measure of quality in the care of the elderly

This article examines the relevance of obtaining patients' views in the measurement of quality in nursing care. The literature on the selection of instruments that measure quality in the care of older people is reviewed.

The health assessment questionnaire 1992: status and review

Over 100 papers describing and utilizing the Stanford Health Assessment Questionnaire (HAQ) have been published since 1980. A brief overview of the HAQ is presented along with a guide to the accumulated literature. The topics covered include: studies using the disability, pain, economic, and drug side effect dimensions of the HAQ; reliability and validity studies; applications to various rheumatic diseases; language adaptations; modifications and derivative scales; studies correlating the HAQ with sociodemographic, health status, laboratory, and physical measures; and randomized controlled trials and observational studies using the HAQ. A few comments regarding future directions for research are also presented.

Quality of life and policy analysis in arthritis

Problems in the American health care system have stimulated interest in cost-effectiveness methodologies. However, there is little consensus on how to define a common unit of health outcome. Many measures used in policy studies consider only mortality and do not fully capture the significant impact of disease-related dysfunction. The impact of conditions, such as osteoarthritis, that have little impact on mortality rates but substantial impact on functioning and well-being may be underestimated in these analyses. In this article, we propose a measurement and policy model that is based on a theoretical conceptualization of health outcome. The model considers the impact of disease and its treatment in terms of both morbidity and mortality. The value of the model for clinical trials, population assessments, and policy analysis is reviewed. A public policy application of the model in Oregon is briefly described.

Scoring methods and application of the activity record (ACTRE) for patients with musculoskeletal disorders

The National Institutes of Health (NIH) Activity Record (ACTRE) has been used to document daily activities in patients with musculoskeletal disorders. Quantification of the amount of time spent resting and physically active, the intensity of pain and fatigue associated with patterns of activities, and motivational considerations are possible with the ACTRE. Scoring has been streamlined to permit identification of the amount of pain, fatigue, and motivational factors as they relate to activity patterns (e.g., rest and physical activity). The ACTRE provides a performance-based, quantifiable measure of daily activity.

Infection control indicators in critical care settings

The Joint Commission on Accreditation of Healthcare Organizations has developed and is currently testing eight infection control indicators in hundreds of hospitals. The indicator set focuses on surgical wound infections, postoperative pneumonia, use of urinary catheterization, ventilator-associated pneumonia, primary bloodstream infection, endometritis, adequacy of surveillance, and employee measles immunization. Critical care unit staff members may be involved in using indicator rates. Following the trend rates should suggest organizational systems and processes of care in critical care settings that may warrant further examination and improvement.

Do self-reported arthritis symptom (RADAR) and health status (AIMS2) data provide duplicative or complementary information?

This study assessed whether self-report measures of symptoms and functional health status provide unique outcome information, or whether functional status assessments primarily serve as a proxy for self-reported arthritis symptoms. Symptom scores of 138 individuals with rheumatoid arthritis (RA) collected with the Rapid Assessment of Disease Activity in Rheumatology (RADAR) measure were compared with same-day functional health scores collected with the recently revised Arthritis Impact Measurement Scales (AIMS2). Correlational and factor analyses revealed that self-assessed arthritis symptoms, physical function and work impact, psychological status, and social health each made independent contributions to outcome. Satisfaction with health status was shown not to be independent of symptoms, functional capacity, or psychological status. It is important to document that self-reported symptom and health status information, when collected concurrently, provides complementary rather than duplicative information.

Nursing's next advance: an internal classification for nursing practice

An International Classification of Nursing Practice (ICNP) is needed to support the processes of nursing practice and advance the knowledge necessary for cost-effective delivery of quality nursing care. Below, the authors present their case for developing such a system that will provide nursing with a nomenclature, a language and a classification that can be used to describe and organize nursing data. It is their belief that this landmark project is achievable and that ICN should lead the work in collaboration with its member associations, the World Health Organization and key national, international, governmental and nongovernmental groups. But to ensure that the system will be adaptable across borders, nurses and organizations are being encouraged to share their ideas and research on such a system.