Reducing waste from incomplete or unusable reports of biomedical research

Research publication can both communicate and miscommunicate. Unless research is adequately reported, the time and resources invested in the conduct of research is wasted. Reporting guidelines such as CONSORT, STARD, PRISMA, and ARRIVE aim to improve the quality of research reports, but all are much less adopted and adhered to than they should be. Adequate reports of research should clearly describe which questions were addressed and why, what was done, what was shown, and what the findings mean. However, substantial failures occur in each of these elements. For example, studies of published trial reports showed that the poor description of interventions meant that 40-89% were non-replicable; comparisons of protocols with publications showed that most studies had at least one primary outcome changed, introduced, or omitted; and investigators of new trials rarely set their findings in the context of a systematic review, and cited a very small and biased selection of previous relevant trials. Although best documented in reports of controlled trials, inadequate reporting occurs in all types of studies-animal and other preclinical studies, diagnostic studies, epidemiological studies, clinical prediction research, surveys, and qualitative studies. In this report, and in the Series more generally, we point to a waste at all stages in medical research. Although a more nuanced understanding of the complex systems involved in the conduct, writing, and publication of research is desirable, some immediate action can be taken to improve the reporting of research. Evidence for some recommendations is clear: change the current system of research rewards and regulations to encourage better and more complete reporting, and fund the development and maintenance of infrastructure to support better reporting, linkage, and archiving of all elements of research. However, the high amount of waste also warrants future investment in the monitoring of and research into reporting of research, and active implementation of the findings to ensure that research reports better address the needs of the range of research users.

Relationship Between Clerical Burden and Characteristics of the Electronic Environment With Physician Burnout and Professional Satisfaction

OBJECTIVE

To evaluate associations between the electronic environment, clerical burden, and burnout in US physicians.

PARTICIPANTS AND METHODS

Physicians across all specialties in the United States were surveyed between August and October 2014. Physicians provided information regarding use of electronic health records (EHRs), computerized physician order entry (CPOE), and electronic patient portals. Burnout was measured using validated metrics.

RESULTS

Of 6375 responding physicians in active practice, 5389 (84.5%) reported that they used EHRs. Of 5892 physicians who indicated that CPOE was relevant to their specialty, 4858 (82.5%) reported using CPOE. Physicians who used EHRs and CPOE had lower satisfaction with the amount of time spent on clerical tasks and higher rates of burnout on univariate analysis. On multivariable analysis, physicians who used EHRs (odds ratio [OR]=0.67; 95% CI, 0.57-0.79; P<.001) or CPOE (OR=0.72; 95% CI, 0.62-0.84; P<.001) were less likely to be satisfied with the amount of time spent on clerical tasks after adjusting for age, sex, specialty, practice setting, and hours worked per week. Use of CPOE was also associated with a higher risk of burnout after adjusting for these same factors (OR=1.29; 95% CI, 1.12-1.48; P<.001). Use of EHRs was not associated with burnout in adjusted models controlling for CPOE and other factors.

CONCLUSION

In this large national study, physicians' satisfaction with their EHRs and CPOE was generally low. Physicians who used EHRs and CPOE were less satisfied with the amount of time spent on clerical tasks and were at higher risk for professional burnout.

Dual-energy X-ray absorptiometry interpretation and reporting in children and adolescents: the revised 2013 ISCD Pediatric Official Positions

The International Society for Clinical Densitometry Official Revised Positions on reporting of densitometry results in children represent current expert recommendations to assist health care providers determine which skeletal sites should be measured, which, if any, adjustments should be made, reference databases to be used, and the elements to include in a dual-energy X-ray absorptiometry report. The recommended scanning sites remain the total body less head and the posterior-anterior spine. Other sites such as the proximal femur, lateral distal femur, lateral vertebral assessment, and forearm are discussed but are only recommended for specific pediatric populations. Different methods of interpreting bone density scans in children with short stature or growth delay are presented. The use of bone mineral apparent density and height-adjusted Z-scores are recommended as suitable size adjustment techniques. The validity of appropriate reference databases and technical considerations to consider when upgrading software and hardware remain unchanged. Updated reference data sets for all contemporary bone densitometers are listed. The inclusion of relevant demographic and health information, technical details of the scan, Z-scores, and the wording "low bone mass or bone density" for Z-scores less than or equal to -2.0 standard deviation are still recommended for clinical practice. The rationale and evidence for the development of the Official Positions are provided. Changes in the grading of quality of evidence, strength of recommendation, and worldwide applicability represent a change in current evidence and/or differences in opinion of the expert panelists used to validate the position statements for the 2013 Position Development Conference.

Guidelines for reporting morbidity and mortality after cardiac valvular operations. The American Association for Thoracic Surgery, Ad Hoc Liaison Committee for Standardizing Definitions of Prosthetic Heart Valve Morbidity

At the request of the Councils of The Society of Thoracic Surgeons (STS) and The American Association for Thoracic Surgery (AATS) the Ad Hoc Liaison Committee for Standardizing Definitions of Prosthetic Heart Valve Morbidity "revisited" the "Guidelines" published in September 1988 [1-3]. The purpose of the review was to update and clarify definitions within the guidelines and to consider recommendations made by others [4, 5]. The variety of cardiac valvular procedures has expanded since 1988; therefore, in this document the term "operated valve" indicates prosthetic and bioprosthetic heart valves of all types: operated or repaired native valves and allograft and autograft valves. The term "operated valve" includes any cardiac valve altered by a surgeon during an operation. Much morbidity and mortality is a direct consequence of the interaction between the patient and operated valve(s), although patient variables (e.g., age, degree of coronary arterial disease, follow-up care) may be more responsible for outcomes than an operated valve. However, no set of guidelines can identify all possible patient factors that may affect morbidity and mortality. General agreement regarding the following definitions of terms and suggestions for reporting data do not preclude more detailed analyses or constructive recommendations and investigators are encouraged to identify relevant patient factors in addition to factors related to operated valves.

Minimum information requested in the annotation of biochemical models (MIRIAM)

Most of the published quantitative models in biology are lost for the community because they are either not made available or they are insufficiently characterized to allow them to be reused. The lack of a standard description format, lack of stringent reviewing and authors' carelessness are the main causes for incomplete model descriptions. With today's increased interest in detailed biochemical models, it is necessary to define a minimum quality standard for the encoding of those models. We propose a set of rules for curating quantitative models of biological systems. These rules define procedures for encoding and annotating models represented in machine-readable form. We believe their application will enable users to (i) have confidence that curated models are an accurate reflection of their associated reference descriptions, (ii) search collections of curated models with precision, (iii) quickly identify the biological phenomena that a given curated model or model constituent represents and (iv) facilitate model reuse and composition into large subcellular models.

A Guide to Field Notes for Qualitative Research: Context and Conversation

Field notes are widely recommended in qualitative research as a means of documenting needed contextual information. With growing use of data sharing, secondary analysis, and metasynthesis, field notes ensure rich context persists beyond the original research team. However, while widely regarded as essential, there is not a guide to field note collection within the literature to guide researchers. Using the qualitative literature and previous research experience, we provide a concise guide to collection, incorporation, and dissemination of field notes. We provide a description of field note content for contextualization of an entire study as well as individual interviews and focus groups. In addition, we provide two "sketch note" guides, one for study context and one for individual interviews or focus groups for use in the field. Our guides are congruent with many qualitative and mixed methodologies and ensure contextual information is collected, stored, and disseminated as an essential component of ethical, rigorous qualitative research.

Accuracy of diagnostic coding for Medicare patients under the prospective-payment system

Reimbursement of hospitals by Medicare under the prospective-payment system is based on patients' diagnoses as coded at discharge. During the period October 1984 through March 1985, we studied the accuracy of the coding for diagnosis-related groups (DRGs) in hospitals receiving Medicare reimbursement. We used a two-stage cluster method to sample 7050 medical records from 239 hospitals that were stratified according to size. Using blinded techniques with reliability checks, medical-record specialists reabstracted the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes to assign correct DRGs to discharged patients. The correct DRGs were then compared with those originally assigned by the physician and the hospital administration. The study revealed an error rate of 20.8 percent in DRG coding. Errors were distributed equally between physicians and hospitals. Small hospitals had significantly higher error rates. Previous studies had found that errors occurred randomly, so that half the errors benefited the hospital financially and half penalized the hospital. The present study found that a statistically significant 61.7 percent of coding errors favored the hospital. These errors caused the average hospital's case-mix index--a measure of the complexity of illness of the hospital's patients--to increase by 1.9 percent. As a result, hospitals received higher net reimbursement from Medicare than was supportable by the medical records. We conclude that "creep" does occur in the coding of DRGs, resulting in overpayment to hospitals for patients covered by Medicare.

Validity of procedure codes in International Classification of Diseases, 9th revision, clinical modification administrative data

BACKGROUND

Administrative hospital discharge data are widely used to assess quality of care in patients undergoing certain procedures. However, little is known about the validity of administrative coding of procedure data. We conducted a detailed chart review to evaluate the accuracy and completeness of information on procedures in administrative data.

METHODS

We randomly selected 1200 hospital separations in the period April 1, 1996, to March 31, 1997, from administrative discharge data of 3 acute adult hospitals in Calgary, Alberta, Canada. Each separation record in administrative data contains up to 10 procedure coding fields. The corresponding medical charts were reviewed for recording presence or absence of procedures. We then determined sensitivity to quantify the accuracy of coding presence of procedures in administrative data when these are present in the chart data (criterion standard).

RESULTS

The agreement between the 2 databases varied greatly across 35 procedures studied. The sensitivity ranged from 0% to 94%. Of 6 major procedures studied, validity of coding was generally good, with 5 procedures having coding sensitivity of 69% and over and only 1 (lysis of peritoneal adhesion) with a low sensitivity of 41%. In contrast, many minor procedures had low sensitivities. Of 29 minor procedures studied, sensitivity was lower than 50% for 15 procedures, between 50% and 79% for 10, and 80% and over for 4.

CONCLUSION

Validity of information on procedures in administrative discharge data appears to be related to type of procedures. Major procedures that are usually performed in operating rooms are reasonably well-coded. Meanwhile, minor procedures that are routinely performed on wards or in radiology departments are generally undercoded.

Standardized reporting of clinical practice guidelines: a proposal from the Conference on Guideline Standardization

Despite enormous energies invested in authoring clinical practice guidelines, the quality of individual guidelines varies considerably. The Conference on Guideline Standardization (COGS) was convened in April 2002 to define a standard for guideline reporting that would promote guideline quality and facilitate implementation. Twenty-three people with expertise and experience in guideline development, dissemination, and implementation participated. A list of candidate guideline components was assembled from the Institute of Medicine Provisional Instrument for Assessing Clinical Guidelines, the National Guideline Clearinghouse, the Guideline Elements Model, and other published guideline models. In a 2-stage modified Delphi process, panelists first rated their agreement with the statement that "[Item name] is a necessary component of practice guidelines" on a 9-point scale. An individualized report was prepared for each panelist; the report summarized the panelist's rating for each item and the median and dispersion of rankings of all the panelists. In a second round, panelists separately rated necessity for validity and necessity for practical application. Items achieving a median rank of 7 or higher on either scale, with low disagreement index, were retained as necessary guideline components. Representatives of 22 organizations active in guideline development reviewed the proposed items and commented favorably. Closely related items were consolidated into 18 topics to create the COGS checklist. This checklist provides a framework to support more comprehensive documentation of practice guidelines. Most organizations that are active in guideline development found the component items to be comprehensive and to fit within their existing development methods.

The qualitative research audit trail: a complex collection of documentation

A qualitative study typically involves a large volume of researcher-generated data, including notes about the context of the study, methodological decisions, data analysis procedures, and self-awareness of the researcher. Such data are important in many aspects of the study, particularly in the development of an audit trail to substantiate trustworthiness. Unfortunately, there is little information available to assist researchers in generating the needed documentation. In this article, we discuss the types of data that contribute to credible investigations. Strategies for maintaining effective records in qualitative studies are included, along with examples from our own research.

Physician recognition of cognitive impairment: evaluating the need for improvement

OBJECTIVES

To assess physician recognition of dementia and cognitive impairment, compare recognition with documentation, and identify physician and patient factors associated with recognition.

DESIGN

Survey of physicians and review of medical records.

SETTING

Health maintenance organization in southern California.

PARTICIPANTS

Seven hundred twenty-nine physicians who provided care for women participating in a cohort study of memory (Women's Memory Study).

MEASUREMENTS

Percentage of patients with dementia or cognitive impairment (using the Telephone Interview of Cognitive Status supplemented by the Telephone Dementia Questionnaire) recognized by physicians. Relationship between physician recognition and patient characteristics and physician demographics, practice characteristics, training, knowledge, and attitudes about dementia.

RESULTS

Physicians (n=365) correctly identified 81% of patients with dementia and 44% of patients with cognitive impairment without definite dementia. Medical records documented cognitive impairment in 83% of patients with dementia and 26% of patients with cognitive impairment without definite dementia. In a multivariable model, physicians with geriatric credentials (defined as geriatric fellowship experience and/or the certificate of added qualifications) recognized cognitive impairment more often than did those without (risk ratio (RR)=1.56, 95% confidence interval (CI)=1.04-1.66). Physicians were more likely to recognize cognitive impairment in patients with a history of depression treatment (RR=1.3, 95% CI=1.03-1.45) or stroke (RR=1.37, 95% CI=1.04-1.45) and less likely to recognize impairment in patients with cognitive impairment without definite dementia than in those with dementia (RR=0.46, 95% CI=0.23-0.72) and in patients with a prior hospitalization for myocardial infarction (RR=0.37, 95% CI=0.09-0.88) or cancer (RR=0.49, 95% CI=0.18-0.90).

CONCLUSION

Medical record documentation reflects physician recognition of dementia, yet physicians are aware of, but have not documented, many patients with milder cognitive impairment. Physicians are unaware of cognitive impairment in more than 40% of their cognitively impaired patients. Additional geriatrics training may promote recognition, but systems solutions are needed to improve recognition critical to provision of emerging therapies for early dementia.

Use of Short Peripheral Intravenous Catheters: Characteristics, Management, and Outcomes Worldwide

BACKGROUND

Peripheral intravenous catheter (PIVC) use in health care is common worldwide. Failure of PIVCs is also common, resulting in premature removal and replacement.

OBJECTIVE

To investigate the characteristics, management practices, and outcomes of PIVCs internationally.

SETTING/PATIENTS

Cross-sectional study. Hospitalized patients from rural, regional, and metropolitan areas internationally.

MEASUREMENTS

Hospital, device, and inserter characteristics were collected along with assessment of the catheter insertion site. PIVC use in different geographic regions was compared.

RESULTS

We reviewed 40,620 PIVCs in 51 countries. PIVCs were used primarily for intravenous medication (n = 28,571, 70%) and predominantly inserted in general wards (n = 22,167, 55%). Two-thirds of all devices were placed in non-recommended sites such as the hand, wrist, or antecubital veins. Nurses inserted most PIVCs (n = 28,575, 71%); although there was wide regional variation (26% to 97%). The prevalence of iIn this study, we found that many PIVCs were placed in areas of flexion, were symptomatic or idle, had suboptimal dressings, or lacked adequate documentation. This suggests inconsistency between recommended management guidelines for PIVCs and current practice.dle PIVCs was 14% (n = 5,796). Overall, 10% (n = 4,204) of PIVCs were painful to the patient or otherwise symptomatic of phlebitis; a further 10% (n = 3,879) had signs of PIVC malfunction; and 21% of PIVC dressings were suboptimal (n = 8,507). Over one-third of PIVCs (n = 14,787, 36%) had no documented daily site assessment and half (n = 19,768, 49%) had no documented date and time of insertion.

CONCLUSIONS

In this study, we found that many PIVCs were placed in areas of flexion, were symptomatic or idle, had suboptimal dressings, or lacked adequate documentation. This suggests inconsistency between recommended management guidelines for PIVCs and current practice.

Portfolios and assessment of competence: a review of the literature

BACKGROUND

The literature review presented here was conducted as part of an English National Board for Nursing, Midwifery and Health Visiting funded project to evaluate the use of portfolios in the assessment of learning and competence. Initial consideration of the topic revealed the need to clarify the terminology and approaches used to assess competence using portfolios, and therefore the literature review was conducted to inform the study.

AIMS

To clarify definitions, theoretical bases and approaches to competence and the use of portfolios in the assessment of learning and competence in nursing education.

METHODS

A comprehensive literature review was conducted using the CINAHL and MEDLINE databases and the keywords competenc*, portfolios and nursing. Articles published in the period 1989-2001 in English were obtained and their reference lists scrutinized to identify additional references. Twenty articles were found using a combination of the keywords competenc* and portfolios, and 52 using the combination portfolios and nurse education. Articles were included in the review if they focused on the use of portfolios in nursing, and those concerned with professional or transitional portfolios were excluded. This article will analyse definitions of and approaches to competence and its measurement and to portfolios and their use as discussed in the articles identified.

RESULTS

Three approaches to competence were identified, each with its appropriate forms of assessment. With regard to portfolios, a number of definitions were again found, but there was a consensus that the theoretical basis of their use is theories of adult learning. A number of reasons for and advantages and disadvantages of their use were found, as well as varying ideas about what a portfolios should consist of and how it should be assessed.

CONCLUSION

A holistic approach to competence seems to be compatible with the use of portfolios to assess competence in nursing students, but the concept and its implementation is still evolving. A variety of assessment methods are needed for assessment and portfolios appear to have the potential to integrate these. Reflection is an essential component of a portfolio, as are the student-teacher relationship and explicit guidelines for constructing the portfolio. Issues of rigour in assessment of portfolios need to be addressed, but the assessor's professional judgement will inevitably enter into this assessment.

Development, standardization, and testing of a lexicon for reporting contrast-enhanced breast magnetic resonance imaging studies

The purpose of this study was to develop, standardize, and test reproducibility of a lexicon for reporting contrast-enhanced breast magnetic resonance imaging (MRI) examinations. To standardize breast MRI lesion description and reporting, seven radiologists with extensive breast MRI experience developed consensus on technical detail, clinical history, and terminology reporting to describe kinetic and architectural features of lesions detected on contrast-enhanced breast MR images. This lexicon adapted American College of Radiology Breast Imaging and Data Reporting System terminology for breast MRI reporting, including recommendations for reporting clinical history, technical parameters for breast MRI, descriptions for general breast composition, morphologic and kinetic characteristics of mass lesions or regions of abnormal enhancement, and overall impression and management recommendations. To test morphology reproducibility, seven radiologists assessed morphology characteristics of 85 contrast-enhanced breast MRI studies. Data from each independent reader were used to compute weighted and unweighted kappa (kappa) statistics for interobserver agreement among readers. The MR lexicon differentiates two lesion types, mass and non-mass-like enhancement based on morphology and geographical distribution, with descriptors of shape, margin, and internal enhancement. Lexicon testing showed substantial agreement for breast density (kappa = 0.63) and moderate agreement for lesion type (kappa = 0.57), mass margins (kappa = 0.55), and mass shape (kappa = 0.42). Agreement was fair for internal enhancement characteristics. Unweighted kappa statistics showed highest agreement for the terms dense in the breast composition category, mass in lesion type, spiculated and smooth in mass margins, irregular in mass shape, and both dark septations and rim enhancement for internal enhancement characteristics within a mass. The newly developed breast MR lexicon demonstrated moderate interobserver agreement. While breast density and lesion type appear reproducible, other terms require further refinement and testing to lead to a uniform standard language and reporting system for breast MRI. J. Magn. Reson. Imaging 2001;13:889-895.

The validity of death certificates: routine validation of death certification and its effects on mortality statistics

The 3478 death certificates (7.1% of all annual death certificates) of this study comprise those national death certificates in 1995 submitted for validation to the panel representing both medical and nosological expertise. As such, it is highly selected and represents, from the nosological point of view, the most inconsistently filled-in portion of Finnish death certificates. The routine validation procedure is essentially based on exploitation of the extra medical information, i.e. the case history, on the Finnish death certificate form. Altogether, 2813 (80.9%) out of 3478 certificates could be adjusted at the primary panel session; the rest required further clarification. The re-assignment of cause of death by the panel and the impact of panel adjustments on the national mortality statistics is assessed here by comparing the initial death certification and the finally registered underlying cause of death grouped into ICD-9 major categories with special reference to the subcategories of neoplasm, cardiovascular disease (HVD) and unnatural death. A statistically significant decline (p<0.0001) in deaths, both in the category of symptoms, signs and ill-defined conditions and in the pulmonary circulation disease subcategory of HVD with 37.6 and 35.1%, respectively, was observed. The decrease of 11.1% in the benign or NUD neoplasm subcategory and the increase of 8.6 and 7.0% in the categories of endocrine disease, and musculo-skeletal and connective tissue disease, respectively, are essential observations as to the quality of the cause of death register. The effect on the HVD major category was practically nil. At the HVD-subcategorial level, a decrease of 14.0% for diseases of the veins and lymphatics and other circulatory diseases and an increase of 3.5% for hypertensive diseases (HYP) were the two next most obvious alterations to the diseases of the pulmonary circulation, but were without statistical significance. For ischaemic heart disease and other subcategories, the effects were minor. The unnatural deaths as a whole increased in the final statistics with only 0.9%. In the study data, categorial changes ranged from the decrease of 75.2% for symptoms, signs and ill-defined conditions to the increase of 77.3% for endocrine diseases. In conclusion, the Finnish death certificate form, death certification practices and cause of death validation procedure seem to serve the coding of causes of death for mortality statistics appropriately. The results of the study form a relevant reference background to evaluation of epidemiological studies on mortality.

A proposed framework for the description of plant metabolomics experiments and their results

The study of the metabolite complement of biological samples, known as metabolomics, is creating large amounts of data, and support for handling these data sets is required to facilitate meaningful analyses that will answer biological questions. We present a data model for plant metabolomics known as ArMet (architecture for metabolomics). It encompasses the entire experimental time line from experiment definition and description of biological source material, through sample growth and preparation to the results of chemical analysis. Such formal data descriptions, which specify the full experimental context, enable principled comparison of data sets, allow proper interpretation of experimental results, permit the repetition of experiments and provide a basis for the design of systems for data storage and transmission. The current design and example implementations are freely available (http://www.armet.org/). We seek to advance discussion and community adoption of a standard for metabolomics, which would promote principled collection, storage and transmission of experiment data.

Good enough practices in scientific computing

Computers are now essential in all branches of science, but most researchers are never taught the equivalent of basic lab skills for research computing. As a result, data can get lost, analyses can take much longer than necessary, and researchers are limited in how effectively they can work with software and data. Computing workflows need to follow the same practices as lab projects and notebooks, with organized data, documented steps, and the project structured for reproducibility, but researchers new to computing often don't know where to start. This paper presents a set of good computing practices that every researcher can adopt, regardless of their current level of computational skill. These practices, which encompass data management, programming, collaborating with colleagues, organizing projects, tracking work, and writing manuscripts, are drawn from a wide variety of published sources from our daily lives and from our work with volunteer organizations that have delivered workshops to over 11,000 people since 2010.

Symptoms experienced in the last six months of life in patients with end-stage heart failure

BACKGROUND

Despite today's modern therapy, the advancement of chronic heart failure (CHF) has not been curbed (Dying from heart failure: lessons from palliative care, BMJ, 317, 1998, 961-962, Editorial). Consequently, the suffering in patients with end-stage CHF is still present. The knowledge on the frequency of symptoms in these patients is scarce.

AIMS

The aim of this study was to achieve a deeper understanding of patients with end-stage CHF and to describe symptoms in these patients during the last 6 months of life by examining documentation in medical records.

METHODS

The study adopted a descriptive, retrospective design using record reviews to collect data, in which 80 medical records of patients hospitalised for CHF in 1995 were reviewed. Descriptive statistics and Student's t-tests were performed.

RESULTS

Breathlessness was the most common symptom (men 90%, women 86%). The largest gender difference was on limitation in physical activity (men=77%, women=37%, total=49%, P=0.001).

CONCLUSION

(1) Twenty-one symptoms were observed, implicating that, patients with CHF in later stages of the disease experience a wide range of symptoms. (2) Despite the fact that several symptoms were documented by both nurses and physicians, it is quite remarkable that symptom-controlling measures were only provided sparingly. (3) A palliative care approach may benefit patients suffering from end-stage CHF.

Coronary Artery Disease - Reporting and Data System (CAD-RADS): An Expert Consensus Document of SCCT, ACR and NASCI: Endorsed by the ACC

The intent of CAD-RADS - Coronary Artery Disease Reporting and Data System is to create a standardized method to communicate findings of coronary CT angiography (coronary CTA) in order to facilitate decision-making regarding further patient management. The suggested CAD-RADS classification is applied on a per-patient basis and represents the highest-grade coronary artery lesion documented by coronary CTA. It ranges from CAD-RADS 0 (Zero) for the complete absence of stenosis and plaque to CAD-RADS 5 for the presence of at least one totally occluded coronary artery and should always be interpreted in conjunction with the impression found in the report. Specific recommendations are provided for further management of patients with stable or acute chest pain based on the CAD-RADS classification. The main goal of CAD-RADS is to standardize reporting of coronary CTA results and to facilitate communication of test results to referring physicians along with suggestions for subsequent patient management. In addition, CAD-RADS will provide a framework of standardization that may benefit education, research, peer-review and quality assurance with the potential to ultimately result in improved quality of care.

Hawthorne effect: implications for prehospital research

STUDY OBJECTIVE

The phenomenon of altered behavior or performance resulting from awareness of being a part of an experimental study has been termed the "Hawthorne effect." Prehospital studies generally involve paramedics or are designed to use data collected by paramedics. Our objective was to determine whether paramedic performance, as measured by frequency of documentation, can be modified by (1) written notification of the importance of documentation, (2) written notification of a research project involving paramedic documentation, or (3) written notification of a quality-improvement audit of paramedic documentation.

DESIGN

Prospective, sequential intervention study with five study phases.

SETTING

Urban, all-advanced life support public utility model emergency medical services system with 55,000 emergency calls per year.

PARTICIPANTS

One hundred forty-five paramedics who completed all ambulance run reports from August 1992 to May 1993.

RESULTS

A total of 30,828 run reports was entered into the study. Baseline undocumented parameters ranged from 3.7% to 6.5%. Compared with baseline, a memo to heighten awareness (phase 2) did not alter documentation (P > or = .08). A medication study memo (phase 3) improved medication documentation (P = .0005) and allergies documentation (P = .037). A quality-improvement audit memo (phase 4) improved documentation of all parameters (P < or = .001).

CONCLUSION

The Hawthorne effect occurs in prehospital research. It does not require direct observation, nor does it require direct feedback. However, it may require a perceived demand for performance. The Hawthorne effect must be considered in the design of prehospital studies and interpretation of data collected by paramedics.

A 10-year review of quality improvement monitoring in pain management: recommendations for standardized outcome measures

Quality measurement in health care is complex and in a constant state of evolution. Different approaches are necessary depending on the purpose of the measurement (e.g., accountability, research, improvement). Recent changes in health care accreditation standards are driving increased attention to measurement of the quality of pain management for improvement purposes. The purpose of this article is to determine what indicators are being used for pain quality improvement, compare results across studies, and provide specific recommendations to simplify and standardize future measurement of quality for hospital-based pain management initiatives. Pain management quality improvement monitoring experience and data from 1992 to 2001 were analyzed from 20 studies performed at eight large hospitals in the United States. Hospitals included: the University of Wisconsin Hospital and Clinics, Madison; Texas Medical Center, Houston; McAllen Medical Center, McAllen, TX; San Francisco General Hospital, San Francisco; Rush-Presbyterian-St. Luke's Medical Center and Northwestern Memorial Hospital, Chicago, IL; Memorial Sloan Kettering Cancer Center, New York; and Kaiser Sunnyside Medical Center of Kaiser Permanente Northwest, Clackamas, OR. Analyses of data led to consensus on six quality indicators for hospital-based pain management. These indicators include: the intensity of pain is documented with a numeric or descriptive rating scale; pain intensity is documented at frequent intervals; pain is treated by a route other than intramuscular; pain is treated with regularly administered analgesics, and when possible, a multimodal approach is used; pain is prevented and controlled to a degree that facilitates function and quality of life; and patients are adequately informed and knowledgeable about pain management. Although there are no perfect measures of quality, longitudinal data support the validity of a core set of indicators that could be used to obtain benchmark data for quality improvement in pain management in the hospital setting.

The impact of the principles of evidence interpretation on the structure and content of statements

The Forensic Science Service (FSS) has devoted appreciable effort to developing the application of the principles of evidence interpretation. Much of the work has been reported in previous papers in this journal, in particular those that develop a model for Case Assessment and Interpretation (CAI). The principles of interpretation are restated and the implications for structure and content of statements are described.