Quality-of-care appraisal in primary care: a quantitative method

Maternal-infant health outcomes and nursing practice in a remote First Nations community in northern Canada

This article reports those findings related to maternal-infant health outcomes of an ethnographic study that explored nursing practice, continuity of care, and health outcomes in one remote First Nations community in northern Canada. Use of multiple data sources within an ethnographic design ensured that quantitative health outcomes data were interpreted within a contextualized understanding of the remote First Nations community.The sample comprised the charts of 65 mothers and 63 infants randomly selected for retrospective chart review. The findings suggest suboptimal maternal-infant health outcomes on several of the health indicator criteria identified for the purposes of this study. The authors discuss long-term sequelae of prenatal and infant health in terms of diabetes and other chronic health conditions in First Nations populations.They explore the implications of these findings in relation to nurses' preparation to offer prenatal and infant primary care in remote First Nations communities.

A Self-Monitoring Method of Resident Care Quality Assurance in Long Term Care Facilities

Quality assurance packages for common health problems are being developed for use in long term care settings. Each package includes a criteria map outlining explicit process and outcome criteria, a reviewer's manual and data summary forms. The packages were developed for use by direct care providers in long term care facilities. Established criteria are applied to records on resident care in order to identify areas which require remedial actions. A feasibility study to assess the use of the packages in eleven nursing homes is reported.

Evidence-based medicine, medical decision analysis, and pathology

Recent advances in molecular pathology and other technologies such as proteomics present pathologists with the challenge of integrating the new information generated with high-throughput methods with current diagnostic models based mostly on histopathology and clinicopathologic correlations. Parallel developments in the field of medical informatics and bioinformatics provide the technical and mathematical methods to approach these problems in a rational manner. However, it remains unclear whether pathologists or other medical specialists will become primarily responsible for the development and maintenance of these multivariate and multidisciplinary diagnostic and prognostic models that are hoped to provide more accurate, individualized patient-based information. Evidence-based medicine (EBM) and medical decision analysis (MDA) are relatively new disciplines that use quantitative methods to assess the value of information, differentiate fact from myth, and integrate so-called best evidence into multivariate models for the assessment of prognosis, response to therapy, selection of laboratory tests, and other complex problems that influence individual patient care. We review from an epistemological viewpoint the current approach to information in pathology and describe some of the concepts developed by the practitioners of EBM and MDA.

Assessing primary care content: four conditions common in community health center practice

Under managed care, community health center (CHC) care patterns will be increasingly subject to outside scrutiny. This article discusses results of medical records reviews assessing quality of care at CHCs for acute otitis media, diabetes, asthma, and hypertension. As a group, these safety net providers meet or exceed prevailing practice across other health care settings; however, there is substantial variation among sites. Regression analyses indicate that the individual CHC used by a patient is the most consistent determinant of whether a patient receives recommended care. Drawing on these results, the article explores approaches for improving care and discusses the implications for performance measurement among CHCs and other safety net providers.

Assessment of general practitioners by video observation of communicative and medical performance in daily practice: issues of validity, reliability and feasibility

OBJECTIVES

To develop a video assessment method for General Practitioners (GPs) by analysing issues of validity, reliability and feasibility of observation of videotaped regular consultations.

DESIGN

In a cross-sectional study consultations of 93 GPs were video recorded in the practice during 1 week. The GPs registered consultation and patient data in a logbook; 16 consultations per GP were selected using preset criteria. The quality of communicative and medical performance of these consultations was assessed by GP observers with a validated instrument. The validity of the procedure was evaluated by checking the content of each GP's sample using specific sample criteria. Selection bias was estimated by multiple regression analysis, with sample characteristics as independent variables and scores on communication and medical performance as dependent variables. The influence of observation on GPs and patients was assessed by a questionnaire. Generalizability theory was used to estimate reliability. Feasibility was assessed by conducting a questionnaire, by keeping accounts, and by checking the technical quality of the videotaped consultations.

SETTING

Universities of Nijmegen and Maastricht, The Netherlands.

SUBJECTS

General Practitioners (GPs).

RESULTS

The domain of general practice was well covered in the samples; content validity was satisfactory. With regard to the sample characteristics, only the total duration of consultations appeared to correlate significantly with both the score on communication and the score on medical performance. A majority (71%) of GPs reported not being influenced by the observation, except in the first cases, and recognizing their usual daily performance in the videotaped consultations. An acceptable level of reliability was reached after 2.5 hours of observation, i.e. 12 cases by a single observer. The method was well accepted by both GPs and patients. The costs were pound250 per GP.

CONCLUSIONS

Video assessment of GPs in daily practice according to the procedures described is a valid and reliable method, one which is useful for education and quality improvement. There is a trade-off between feasibility on one hand and validity, reliability and credibility on the other hand. Compared to investments in observation methods in standardized settings, the costs of video observation of GPs' actual performance are acceptable.

Profiling primary care physicians for a new managed care network

We developed methods for comparing physicians who would be selected to participate in a major employer's self-insurance program. These methods used insurance claims data to identify and profile physicians according to deviations from prevailing practice and outcome patterns, after considering differences in case-mix and severity of illness among the patients treated by those providers. The discussion notes the usefulness and limitations of claims data for this and other purposes. We also comment on policy implications and the relationships between our methods and health care reform strategies designed to influence overall health care costs.

Consistency in performance among primary care practitioners

OBJECTIVES

The authors studied the consistency of performance of individual physicians to evaluate the identification of outlier practitioners as a strategy for improving patient care.

METHODS

The authors used a data base containing information on 430 practitioners caring for 6,090 patients in 16 group practices. The authors analyzed inter- and intraphysician differences in performance on the basis of review criteria for 8 patient care guidelines. These criteria allowed for a variety of acceptable clinical strategies, incorporated decision tree logic, and included input from participating practitioners. The authors took steps to maximize validity and controlled for potentially confounding characteristics of patients and practitioners. The authors identified outliers, evaluated the significance of differences between outliers and nonoutliers, and studied variations in performance across cases and guidelines in conformance with guidelines.

RESULTS

The authors identified a few statistically significant outliers. Correlations for performance across cases seen by a given physician were low. The highest positive correlation for performance between any pair of guidelines was 0.32.

CONCLUSIONS

The performance of a given practitioner is highly variable from patient to patient and from guideline to guideline. Thus, strategies focusing solely on substandard outliers will miss opportunities to improve performance.

Quality of mental health service care: the forgotten pathway from process to outcome

The validity of the concept of outcome depends on a relationship between routine treatment and later health status. Outcome evaluations and audits are very rare in psychiatry. A substantial expansion in epidemiologically based, naturalistic, observational, process-outcome data collection in routine psychiatric practice is essential in order to identify treatment allocation biases and other reasons for unexpected outcomes. Identified causes of undertreatment should lead to locally agreed detailed clinical guidelines. Experimental evaluation should take place in routine clinical practice settings, with change in both process and outcome as the objective. Ultimately, the results of both experimental and observational outcome studies on representative service users should converge, permitting outcomes to be the ultimate arbitrator of quality.

Reasons for contact in family practice. An Icelandic multicentre study on content of practice

To establish data on the patient's reasons for a contact, as a part of data on content of Icelandic family practice, a prospective practice audit was made of 16 Icelandic health centres with computerized contact data from 1 January to 31 December 1988. The study comprised 16 community health centres in Iceland and their target population, 12 rural and four urban. The reasons for contact in the study group are analysed. A total of 284348 reasons for contact were analysed; 36-39% were for symptoms and 44-50% were initiated by health professionals. The latter included renewal of prescriptions, which comprised 17-18% of all reasons for contact. Musculoskeletal symptoms were the most common symptomatic complaint, 6.6-7.3% of all reasons for contact. The five most often stated symptoms were: rash, cough, cold, lower limb symptoms, and fever. A "reason for contact" record increases the understanding of the patient's presenting complaint, as well as the patient's agenda in each contact. This record gives an opportunity to follow the presenting complaint in the continuous process of care i. e. reason for contact diagnosis, management, and follow-up. We are reminded that common things are common in family practice; nevertheless more research is needed to understand the process of care.

Obstetrical attitudes and practices before and after the Canadian Consensus Conference Statement on Cesarean Birth

This paper describes one aspect of a research program aimed at reducing the incidence of cesarean section in Ontario for women with a previous cesarean section or a breech presentation. Using data from multiple sources--surveys of obstetricians, and hospital administrators, and hospital record statistics, the authors attempt to assess the response of obstetricians to pressure to change their practice. This pressure comes principally from the Canadian Consensus Conference Statement on Cesarean Birth, released in June 1986 and subsequently endorsed by a number of professional organizations. The Statement provides clear guidelines for the management of labour in women with previous cesarean section or a breech presentation. The findings present a number of interpretive challenges. Based on their response to hypothetical cases obstetricians are favourably disposed to considering a trial of labour for women with previous cesarean section and breech presentation. However, both their reported practices, as well as hospital statistics indicate the continued high prevalence of cesarean section, though there is a small decline in cesareans for previous cesarean section. There was no evidence that hospitals lacked appropriate facilities for a trial of labour or had unduly restricted formal policies. Furthermore, although awareness of and agreement with the Consensus Statement recommendations was high, when questioned on the actual details of the recommendations, obstetrician's recall was surprisingly low. Respondents tended to err in the direction of choosing more conservative measures than those recommended by the Statement.(ABSTRACT TRUNCATED AT 250 WORDS)

Medical care classification systems in the ambulatory care environment: an evaluative framework

An essential need of a Quality Assurance Program (QAP) in an ambulatory care setting is accurate and reliable information characterizing the encounter between patient and provider. This information includes identification of the patient's reason for visit, the provider's diagnostic impressions, and procedures performed. Such data can be used to maintain an informational index on the nature of patient care. Based upon this index, a wide variety of special studies in the assessment and assurance of the quality of care can be conducted. A primary consideration in the development of such an index is the choice of a Medical Care Classification System (MCCS) to be used to code encounter-related data. The objective of this paper is to provide a methodology and evaluative framework in which available Medical Care Classification Systems can be assessed as to which one best meets the needs of the quality assurance programs within ambulatory care settings.

Assessing peer review in the quest for improved medical services

The increased concern about the quality of medical services evidenced by, inter alia, the growing attention to quality of Peer Review Organizations. The purchasing and implementation of sophisticated medical data systems by hospitals, and the growing clamor from private health insurers and employers about the rapidly rising costs of health services has made determining the effectiveness of medical interventions a priority subject for many authorities in the field of medical care assessment. At the risk of oversimplification, the view that a greater focus on quality of health services is overdue has begun to energize healthcare institutions, the health professions, payers, and the general public. The objective of this paper and those that will follow is to examine medical peer review. Medical peer review involves peer appraisal in at least two stages: in criteria and standard setting (1, 2), and in determining whether criteria and standards have been met by practitioners in the rendering of services (2). This first article describes some difficulties with the information available to facilitate effective medical peer review, and examines the processes that provide the knowledge base upon which medical peer review depends.

Criteria map: potential for skin breakdown--a quality assurance tool for use in any setting

An interagency group of nurses from five Hamilton, Ontario-area hospitals and community agencies developed a criteria map with a branching format to evaluate the potential for skin breakdown and to link patient characteristics to care decisions and outcomes of care. An existing criteria map originally intended for use solely in long term care settings was modified and tested by the group for additional use in acute care and community settings. The criteria map includes a reviewer's instruction manual and data collection summary forms. The map was tested for feasibility and interrater reliability and was proven to be a versatile, easy-to-use QA tool that can stimulate change in clinical and organizational practices.

Toward an applied technology for quality measurement in health care

Cost containment, financial incentives to conserve resources, the growth of for-profit hospitals, an aggressive malpractice environment, and demands from purchasers are among the forces today increasing the need for improved methods that measure quality in health care. At the same time, increasingly sophisticated databases and the existence of managed care systems yield new opportunities to observe and correct quality problems. Research on targets of measurement (structure, process, and outcome) and methods of measurement (implicit, explicit, and sentinel methods) has not yet produced managerially useful applied technology for quality measurement in real-world settings. Such an applied technology would have to be cheaper, faster, more flexible, better reported, and more multidimensional than the majority of current research on quality assurance. In developing a new applied technology for the measurement of health care quality, quantitative disciplines have much to offer, such as decision support systems, criteria based on rigorous decision analyses, utility theory, tools for functional status measurement, and advances in operations research.

Consultation letters as a method for assessing in-training performance in a department of medicine

This study examined the feasibility of using outpatient consultation letters as a means of assessing the clinical performance of clerks and house staff in a large teaching hospital. A total of 222 assessments were obtained based on samples of consultation problems from two clinical services. The method involved a two-stage rating scale based on the performance areas of Data Base, Problem Formulation, Diagnostic Tests, Management and Overall Assessment. The technique was investigated in terms of the objectivity, reliability, validity and efficiency of the performance ratings. Product-moment coefficients were used to assess inter- and intrarater agreement in scoring the consultation letters. These analyses identified a consistent difficulty in assessing the category of Data Base. For the remaining performance categories, interrater coefficients in the range .19-.76 and intrarater coefficients in the range .23-.96 were obtained. Other preliminary findings indicated good potential for a reliable and valid assessment. The method is technically feasible, and is based on a novel and reasonably unobtrusive approach to assessing trainee performance in a clinical setting.

Measurement of actions of care-providers in long-term care

While measurement of the actions of providers (process) are a vital part of assessing the quality of care in long-term care settings, instruments that are appropriate to such measurement in program evaluation, auditing staff performance, and evaluation of continuing education programs are often unavailable. Long-term care providers are therefore faced with the challenge of either selecting an instrument from the few available or constructing a new one. This article describes several scientifically acceptable measurement properties for instruments measuring the actions of long-term care providers and their application to instruments reported in the long-term care literature. Only five of the 23 instruments reviewed met four or more of the seven measurement properties.

Primary care of adolescents. Issues in program development and implementation

Recent reviews of adolescent primary care urge the development of a dynamic program evaluation strategy. An action research evaluation model that combines service, training, and research to foster the development and implementation of primary care services is presented. The strategy includes description of patient populations, determination of patient health care needs, specification of service objectives, assessment of health care resources, and evaluation of service procedures and outcomes. Elements of the action research strategy are applied to evaluate two current issues in adolescent primary care: family-oriented versus adolescent-limited services and "new morbidity".

Factors influencing consumer receptivity to the nurse practitioner. A systems analysis

This paper describes a methodology to delineate factors associated with consumer acceptance of the medical nurse practitioner (MNP). The MNP is a category of new health practitioners with proficiency in medical/health care functions traditionally performed only by physicians. A process model approach was developed to study the significance of selected sociodemographic cognitive, attitudinal, and clinical/medical factors that are expected to predispose consumer acceptance of the MNP. The survey population consists of predominantly elderly and indigent ambulatory patients to an inner-city primary care clinic. The data were collected from 156 primary care patients before and after the introduction of the MNP program. The results clearly demonstrate consumer support for the new health practitioner concept. This finding is further substantiated in the "after" study. From an analysis of symptoms experience, symptom severity, and type of symptoms, symptoms experience emerged as the strongest indicator of consumer receptivity to MNP concept. Additionally, exposure to MNP caused the consumer to become more aware of proper consumption and increased the demand for physician care and specialty care where appropriate.

A randomized trial of continuing medical education

To determine whether continuing medical education affects the quality of clinical care, we randomly allocated 16 Ontario family physicians to receive or not receive continuing-education packages covering clinical problems commonly confronted in general practice. Over 4500 episodes of care, provided before and after study physicians received continuing education, were compared with preset clinical criteria and classified according to quality. Although objective tests confirmed that the study physicians learned from the packages, there was little effect on the overall quality of care. When the topics were of relatively great interest to the physicians, the control group (who did not receive the packages) showed as much improvement as did the study group. When the topics were not preferred, however, the documented quality of care provided by study physicians rose (P less than 0.05) and differed from that provided by control physicians (P = 0.01). Finally, there was no spillover effect on clinical problems not directly covered by the program. In view of the trend toward mandatory continuing education and the resources expended, it is time to reconsider whether it works.

Comparing nurse-practitioners and physicians: a stimulation study on processes of care

The question "where is the 'nurse' in the nurse-practitioner" is addressed by comparing processes of care employed by nurse-practitioners (NP) and physicians (MD) in the overall management of commonly occurring problems in primary care practice. Subjects in a nonrandom sample of 15 NPs and 11 MDs independently described their proposed management of two pediatric and two adult hypothetical cases. NP and MD responses were compared to identify differences in the inclusion of explicit process criteria items in the following categories: problem assessment, diagnostic plan, therapeutic plan, health education, follow-up. The differences between NPs and MDs were almost entirely in the NPs' more frequent inclusion of expressive "caring" functions which have traditionally been the focus of nursing practice. The extent to which "care" as well as "cure" process variables affect outcome measures is discussed with regard to testing the effectiveness of the "care" component of health services, regardless of the professional identity of the provider.

A comparison of behaviour on simulated patients and patient management problems

This paper describes a study in which students were faced with a series of problems presented as patient management problems and as simulated patients (individuals trained to accurately portray a clinical problem). The subjects were sixty-five final-year medical students in a clinical clerkship in family medicine. Four clinical problems were used--each was developed in the PMP and simulated patient format. Each student completed one PMP and one simulated patient encounter (SPE) during the 2nd week of the 8-week clerkship, and a second PMP and SPE in the 7th week of the clerkship. Performance on the two formats was compared by determining the number of options, and the number of critical options (weighted +1 or +2 by a criterion panel) elicited in each section of the problem--history, physical examination, investigations and management. Students were found to elicit significantly more options in the PMP in all sections of the problem, an increase of from 20 to 150%. This difference due to format was of similar magnitude to the difference between problems and the proportion of observed variance in response due to the differences between formats and cases was consistently greater than the variance due to systematic differences between students. The findings of this study are consistent with previous studies comparing performance on PMPs to oral examinations and medical records, and raise some concern about the use of PMPs as a measure of competence in certification and licensure decisions.

Protocol-based computer reminders, the quality of care and the non-perfectability of man

To determine whether clinical errors can be reduced by prospective computer suggestions about the management of simple clinical events, I studied the responses of nine physicians to computer suggestions generated by 390 protocols in a controlled crossover design. These protocols dealt primarily with conditions managed (e.g., elevated blood pressure) or caused (e.g., liver toxicity) by drugs. Physicians responded to 51 per cent of 327 events when given, and 22 per cent of 385 events when not given computer suggestions. Neither level of postgraduate training (first-year postgraduate or third-year post-graduate) nor the order in which physicians served as study and control subjects had statistically significant overall effect on the results. It appears that the prospective reminders do reduce errors, and that many of these errors are probably due to man's limitations as a data processor rather than to correctable human deficiencies.

Physician management in primary care

Minimal explicit consensus criteria in the management of patients with four indicator conditions were established by an ad hoc committee of primary care physicians practicing in different locations. These criteria were then applied to the practices of primary care physicians located in a single community by abstracting medical records and obtaining questionnaire data about patients with the indicator conditions. A standardized management score for each physician was used as the dependent variable in stepwise regression analysis with physician/practice and patient/disease characteristics as the candidate independent variables. For all physicians combined, the mean management scores were high, ranging from .78 to .93 for the four conditions. For two of the conditions, care of the normal infant and pregnant woman, the management scores were better for pediatricians and obstetricians respectively than for family physicians. For the other two conditions, adult onset diabetes and congestive heart failure, there were no differences between the management scores of family physicians and internists. Patient/disease characteristics did not contribute significantly to explaining the variation in the standardized management scores.