Damage control laparotomy utilization rates are highly variable among Level I trauma centers: Pragmatic, Randomized Optimal Platelet and Plasma Ratios findings

BACKGROUND

Damage control laparotomy (DCL) is intended to limit deleterious effects from trauma-induced coagulopathy. DCL has been associated with mortality reduction, but may increase complications including sepsis, abscess, respiratory failure, hernia, and gastrointestinal fistula. We hypothesized that (1) DCL incidence would vary between institutions; (2) mortality rates would vary with DCL rates; (3) standard DCL criteria of pH, international normalized ratio, temperature and major intra-abdominal vascular injury would not adequately capture all patients.

METHODS

Trauma patients at 12 Level 1 North American trauma centers were randomized based on transfusion ratios as described in the Pragmatic, Randomized Optimal Platelet and Plasma Ratios trial. We analyzed outcomes after emergent laparotomy using a mixed-effects logistic model comparing DCL versus definitive surgical management with random effect for study site. Primary outcomes were 24-hour and 30-day mortality.

RESULTS

Three hundred twenty-nine patients underwent emergent laparotomy: 213 (65%) DCL and 116 (35%) definitive surgical management. DCL rates varied between institutions (33-83%), (p = 0.002). Median Injury Severity Score (ISS) was higher in the DCL group, 29 (interquartile range, 13-34) versus 21 (interquartile range, 22-41) (p < 0.001). Twenty-four-hour mortality was 19% with DCL versus 4% (p < 0.001); 30-day mortality was 28% with DCL versus 19% (p < 0.001). In a mixed-effects model, ISS and major intra-abdominal vascular injury were correlates of DCL (odds ratio [OR], 1.05; 95% confidence interval [CI], 1.02-1.07 and OR, 2.7; 95% CI, 1.4-5.2). DCL was not associated with 30-day mortality (OR, 2.33; 95% CI, 0.97-5.60). Correlates included ISS (OR, 1.06; 95% CI, 1.02-1.09), PRBCs in 24 hours (OR, 1.10; 95% CI, 1.03-1.18), and age (OR, 1.04; 95% CI, 1.01-1.06). No significant mortality difference was detected between institutions (p = 0.63). Sepsis and VAP occurred more frequently with DCL (p < 0.05). Eighty percent (135/213) of DCL patients met standard criteria.

CONCLUSION

Although DCL utilization varied significantly between institutions, there was no significant mortality difference between centers. This finding suggests tempering DCL use may not decrease mortality, but could decrease related complications.

LEVEL OF EVIDENCE

Therapeutic study, level III.

Evolution of European Union legislation on emergency research

AIM

Emergency research is necessary to prevent exposure of patients to unvalidated clinical practice (nonmaleficence), and to improve the dismal prognosis of disorders requiring emergent treatment such as cardiac arrest (beneficence). Regulations that govern clinical research should conform to bioethical principles of respect for nonmaleficence, beneficence, autonomy, and justice. Our objectives are to review the evolution of European Union (EU) legislation on emergency research, and to identify potentially remaining problems.

DATA SOURCES

EU legislative sources on clinical research and medical literature describing the impact of EU Regulations on emergency research.

RESULTS

Article 5 of EU Directive 2001/20/EC required consent before enrolment in a research study to ensure the autonomy of potentially incapacitated research subjects. However, obtaining such consent is often impossible in emergency situations. Directive 2001/20/EC was criticized for potentially preventing emergency research. Several EU Member States addressed this problem by permitting deferred consent. International ethical guidelines supporting deferred consent were also cited by Good Clinical Practice Directive 2005/28/EC. However, Directive 2001/20/EC was not revised to achieve harmonization of EU emergency research, thus resulting in ongoing "ambiguity" as regards to emergency research legitimacy. This will be definitively addressed by applying EU Regulation No. 536/2014 and repealing Directive 2001/20/EC. The new EU Regulation permits using deferred consent under clearly specified conditions, and may foster emergency research that evaluates interventions posing minimal risk relative to standard practice.

CONCLUSIONS

Legislation related to emergency research in Europe has evolved to increase concordance with bioethical principles so as to increase evidence-based improvements in emergency care.

Transfusion of plasma, platelets, and red blood cells in a 1:1:1 vs a 1:1:2 ratio and mortality in patients with severe trauma: the PROPPR randomized clinical trial

IMPORTANCE

Severely injured patients experiencing hemorrhagic shock often require massive transfusion. Earlier transfusion with higher blood product ratios (plasma, platelets, and red blood cells), defined as damage control resuscitation, has been associated with improved outcomes; however, there have been no large multicenter clinical trials.

OBJECTIVE

To determine the effectiveness and safety of transfusing patients with severe trauma and major bleeding using plasma, platelets, and red blood cells in a 1:1:1 ratio compared with a 1:1:2 ratio.

DESIGN, SETTING, AND PARTICIPANTS

Pragmatic, phase 3, multisite, randomized clinical trial of 680 severely injured patients who arrived at 1 of 12 level I trauma centers in North America directly from the scene and were predicted to require massive transfusion between August 2012 and December 2013.

INTERVENTIONS

Blood product ratios of 1:1:1 (338 patients) vs 1:1:2 (342 patients) during active resuscitation in addition to all local standard-of-care interventions (uncontrolled).

MAIN OUTCOMES AND MEASURES

Primary outcomes were 24-hour and 30-day all-cause mortality. Prespecified ancillary outcomes included time to hemostasis, blood product volumes transfused, complications, incidence of surgical procedures, and functional status.

RESULTS

No significant differences were detected in mortality at 24 hours (12.7% in 1:1:1 group vs 17.0% in 1:1:2 group; difference, -4.2% [95% CI, -9.6% to 1.1%]; P = .12) or at 30 days (22.4% vs 26.1%, respectively; difference, -3.7% [95% CI, -10.2% to 2.7%]; P = .26). Exsanguination, which was the predominant cause of death within the first 24 hours, was significantly decreased in the 1:1:1 group (9.2% vs 14.6% in 1:1:2 group; difference, -5.4% [95% CI, -10.4% to -0.5%]; P = .03). More patients in the 1:1:1 group achieved hemostasis than in the 1:1:2 group (86% vs 78%, respectively; P = .006). Despite the 1:1:1 group receiving more plasma (median of 7 U vs 5 U, P < .001) and platelets (12 U vs 6 U, P < .001) and similar amounts of red blood cells (9 U) over the first 24 hours, no differences between the 2 groups were found for the 23 prespecified complications, including acute respiratory distress syndrome, multiple organ failure, venous thromboembolism, sepsis, and transfusion-related complications.

CONCLUSIONS AND RELEVANCE

Among patients with severe trauma and major bleeding, early administration of plasma, platelets, and red blood cells in a 1:1:1 ratio compared with a 1:1:2 ratio did not result in significant differences in mortality at 24 hours or at 30 days. However, more patients in the 1:1:1 group achieved hemostasis and fewer experienced death due to exsanguination by 24 hours. Even though there was an increased use of plasma and platelets transfused in the 1:1:1 group, no other safety differences were identified between the 2 groups.

TRIAL REGISTRATION

clinicaltrials.gov Identifier: NCT01545232.

International variation in policies and practices related to informed consent in acute cardiovascular research: Results from a 44 country survey

BACKGROUND

Research in an emergency setting such as that with an acute cardiovascular event is challenging because the window of opportunity to treat may be short and may preclude time to obtain informed consent from the patient or their representative. Some perceive that requiring informed consent in emergency situations has limited improvements in care. Vulnerable populations including minorities or residents of low-income countries are at greatest risk of need for resuscitation. Lack of enrollment of such patients would increase uncertainties in treatment benefit or harm in those at greater risk of need for resuscitation. We sought to assess international variation in policies and procedures related to exception from informed consent (EFIC) or deferred consent for emergency research.

METHODS

A brief survey instrument was developed and modified by consensus among the investigators. Included were multiple choice and open-ended responses. The survey included an illustrative example of a hypothetical randomized study. Elicited information included the possibility of conducting such a study in the respondent's country, as well as approvals required to conduct the study. The population of interest was emergency physicians or other practitioners of acute cardiovascular event research.

RESULTS

Usable responses were obtained from 44 countries (76% of surveyed). Community opposition to EFIC was noted in 6 (14%) countries. Emergency Medical Services (EMS) providers in 8 (20%) countries were judged unable or unwilling to participate. A majority of countries (36, 82%) required approval by a Research Ethics Committee or similar. Government approval was required in 25 (57%) countries.

CONCLUSION

There is international variation in practices and policies related to consent for emergency research. There is an ongoing need to converge regulations based on the usefulness of multinational emergency research to benefit both affluent and disadvantaged populations.

Pragmatic Randomized Optimal Platelet and Plasma Ratios (PROPPR) Trial: design, rationale and implementation

BACKGROUND

Forty percent of in-hospital deaths among injured patients involve massive truncal haemorrhage. These deaths may be prevented with rapid haemorrhage control and improved resuscitation techniques. The Pragmatic Randomized Optimal Platelet and Plasma Ratios (PROPPR) Trial was designed to determine if there is a difference in mortality between subjects who received different ratios of FDA approved blood products. This report describes the design and implementation of PROPPR.

STUDY DESIGN

PROPPR was designed as a randomized, two-group, Phase III trial conducted in subjects with the highest level of trauma activation and predicted to have a massive transfusion. Subjects at 12 North American level 1 trauma centres were randomized into one of two standard transfusion ratio interventions: 1:1:1 or 1:1:2, (plasma, platelets, and red blood cells). Clinical data and serial blood samples were collected under Exception from Informed Consent (EFIC) regulations. Co-primary mortality endpoints of 24h and 30 days were evaluated.

RESULTS

Between August 2012 and December 2013, 680 patients were randomized. The overall median time from admission to randomization was 26min. PROPPR enrolled at higher than expected rates with fewer than expected protocol deviations.

CONCLUSION

PROPPR is the largest randomized study to enrol severely bleeding patients. This study showed that rapidly enrolling and successfully providing randomized blood products to severely injured patients in an EFIC study is feasible. PROPPR was able to achieve these goals by utilizing a collaborative structure and developing successful procedures and design elements that can be part of future trauma studies.

Predictive Analytics In Healthcare: Medications as a Predictor of Medical Complexity

Children with special healthcare needs (CSHCN) require health and related services that exceed those required by most hospitalized children. A small but growing and important subset of the CSHCN group includes medically complex children (MCCs). MCCs typically have comorbidities and disproportionately consume healthcare resources. To enable strategic planning for the needs of MCCs, simple screens to identify potential MCCs rapidly in a hospital setting are needed. We assessed whether the number of medications used and the class of those medications correlated with MCC status. Retrospective analysis of medication data from the inpatients at Seattle Children's Hospital found that the numbers of inpatient and outpatient medications significantly correlated with MCC status. Numerous variables based on counts of medications, use of individual medications, and use of combinations of medications were considered, resulting in a simple model based on three different counts of medications: outpatient and inpatient drug classes and individual inpatient drug names. The combined model was used to rank the patient population for medical complexity. As a result, simple, objective admission screens for predicting the complexity of patients based on the number and type of medications were implemented.

Global health and emergency care: a resuscitation research agenda--part 1

At the 2013 Academic Emergency Medicine global health consensus conference, a breakout session on a resuscitation research agenda was held. Two articles focusing on cardiac arrest and trauma resuscitation are the result of that discussion. This article describes the burden of disease and outcomes, issues in resuscitation research, and global trends in resuscitation research funding priorities. Globally, cardiovascular disease and trauma cause a high burden of disease that receives a disproportionately smaller research investment. International resuscitation research faces unique ethical challenges. It needs reliable baseline statistics regarding quality of care and outcomes; data linkages between providers; reliable and comparable national databases; and an effective, efficient, and sustainable resuscitation research infrastructure to advance the field. Research in resuscitation in low- and middle-income countries is needed to understand the epidemiology, infrastructure and systems context, level of training needed, and potential for cost-effective care to improve outcomes. Research is needed on low-cost models of population-based research, ways to disseminate information to the developing world, and finding the most cost-effective strategies to improve outcomes.

Combining Multiple Markers to Improve the Longitudinal Rate of Progression-Application to Clinical Trials on the Early Stage of Alzheimer's Disease

Clinical trials on early stage Alzheimer's disease (AD) are reaching a bottleneck because none of the current disease markers changes appreciably early in the disease process and therefore a huge sample is required to adequately power such trials. We propose a method to combine multiple markers so that the longitudinal rate of progression can be improved. The criterion is to maximize the probability that the combined marker will be decreased over time (assuming a negative mean slope for each marker). We propose estimates to the weights of markers in the optimum combination and a confidence interval estimate to the combined rate of progression through the maximum likelihood estimates and a bootstrap procedure. We conduct simulations to assess the performance of our estimates and compare our approach with the first principal component from a principal component analysis. The proposed method is applied to a real world sample of individuals with preclinical AD to combine measures from two cognitive domains. The combined cognitive marker is finally used to design future clinical trials on preclinical AD, demonstrating a significant improvement in reducing the sample sizes needed to power such trials when compared with individual markers alone.

Designing clinical trials to test disease-modifying agents: application to the treatment trials of Alzheimer's disease

BACKGROUND

Therapeutic trials of disease-modifying agents on Alzheimer's disease (AD) require novel designs and analyses involving switch of treatments for at least a portion of subjects enrolled. Randomized start and randomized withdrawal designs are two examples of such designs. Crucial design parameters such as sample size and the time of treatment switch are important to understand in designing such clinical trials.

PURPOSE

The purpose of this article is to provide methods to determine sample sizes and time of treatment switch as well as optimum statistical tests of treatment efficacy for clinical trials of disease-modifying agents on AD.

METHODS

A general linear mixed effects model is proposed to test the disease-modifying efficacy of novel therapeutic agents on AD. This model links the longitudinal growth from both the placebo arm and the treatment arm at the time of treatment switch for these in the delayed treatment arm or early withdrawal arm and incorporates the potential correlation on the rate of cognitive change before and after the treatment switch. Sample sizes and the optimum time for treatment switch of such trials as well as optimum test statistic for the treatment efficacy are determined according to the model.

RESULTS

Assuming an evenly spaced longitudinal design over a fixed duration, the optimum treatment switching time in a randomized start or a randomized withdrawal trial is half way through the trial. With the optimum test statistic for the treatment efficacy and over a wide spectrum of model parameters, the optimum sample size allocations are fairly close to the simplest design with a sample size ratio of 1:1:1 among the treatment arm, the delayed treatment or early withdrawal arm, and the placebo arm. The application of the proposed methodology to AD provides evidence that much larger sample sizes are required to adequately power disease-modifying trials when compared with those for symptomatic agents, even when the treatment switch time and efficacy test are optimally chosen.

LIMITATIONS

The proposed method assumes that the only and immediate effect of treatment switch is on the rate of cognitive change.

CONCLUSIONS

Crucial design parameters for the clinical trials of disease-modifying agents on AD can be optimally chosen. Government and industry officials as well as academia researchers should consider the optimum use of the clinical trials design for disease-modifying agents on AD in their effort to search for the treatments with the potential to modify the underlying pathophysiology of AD.

IPM: An integrated protein model for false discovery rate estimation and identification in high-throughput proteomics

In high-throughput mass spectrometry proteomics, peptides and proteins are not simply identified as present or not present in a sample, rather the identifications are associated with differing levels of confidence. The false discovery rate (FDR) has emerged as an accepted means for measuring the confidence associated with identifications. We have developed the Systematic Protein Investigative Research Environment (SPIRE) for the purpose of integrating the best available proteomics methods. Two successful approaches to estimating the FDR for MS protein identifications are the MAYU and our current SPIRE methods. We present here a method to combine these two approaches to estimating the FDR for MS protein identifications into an integrated protein model (IPM). We illustrate the high quality performance of this IPM approach through testing on two large publicly available proteomics datasets. MAYU and SPIRE show remarkable consistency in identifying proteins in these datasets. Still, IPM results in a more robust FDR estimation approach and additional identifications, particularly among low abundance proteins. IPM is now implemented as a part of the SPIRE system.

Design and initial characterization of the SC-200 proteomics standard mixture

High-throughput (HTP) proteomics studies generate large amounts of data. Interpretation of these data requires effective approaches to distinguish noise from biological signal, particularly as instrument and computational capacity increase and studies become more complex. Resolving this issue requires validated and reproducible methods and models, which in turn requires complex experimental and computational standards. The absence of appropriate standards and data sets for validating experimental and computational workflows hinders the development of HTP proteomics methods. Most protein standards are simple mixtures of proteins or peptides, or undercharacterized reference standards in which the identity and concentration of the constituent proteins is unknown. The Seattle Children's 200 (SC-200) proposed proteomics standard mixture is the next step toward developing realistic, fully characterized HTP proteomics standards. The SC-200 exhibits a unique modular design to extend its functionality, and consists of 200 proteins of known identities and molar concentrations from 6 microbial genomes, distributed into 10 molar concentration tiers spanning a 1,000-fold range. We describe the SC-200's design, potential uses, and initial characterization. We identified 84% of SC-200 proteins with an LTQ-Orbitrap and 65% with an LTQ-Velos (false discovery rate = 1% for both). There were obvious trends in success rate, sequence coverage, and spectral counts with protein concentration; however, protein identification, sequence coverage, and spectral counts vary greatly within concentration levels.

A Unified Approach of Meta-Analysis: Application to an Antecedent Biomarker Study in Alzheimer's Disease

This article provides a unified methodology of meta-analysis that synthesizes medical evidence by using both available individual patient data (IPD) and published summary statistics within the framework of likelihood principle. Most up-to-date scientific evidence on medicine is crucial information not only to consumers but also to decision makers, and can only be obtained when existing evidence from the literature and the most recent individual patient data are optimally synthesized. We propose a general linear mixed effects model to conduct meta-analyses when individual patient data are only available for some of the studies and summary statistics have to be used for the rest of the studies. Our approach includes both the traditional meta-analyses in which only summary statistics are available for all studies and the other extreme case in which individual patient data are available for all studies as special examples. We implement the proposed model with statistical procedures from standard computing packages. We provide measures of heterogeneity based on the proposed model. Finally, we demonstrate the proposed methodology through a real life example studying the cerebrospinal fluid biomarkers to identify individuals with high risk of developing Alzheimer's disease when they are still cognitively normal.

The United States of America and scientific research

To gauge the current commitment to scientific research in the United States of America (US), we compared federal research funding (FRF) with the US gross domestic product (GDP) and industry research spending during the past six decades. In order to address the recent globalization of scientific research, we also focused on four key indicators of research activities: research and development (R&D) funding, total science and engineering doctoral degrees, patents, and scientific publications. We compared these indicators across three major population and economic regions: the US, the European Union (EU) and the People's Republic of China (China) over the past decade. We discovered a number of interesting trends with direct relevance for science policy. The level of US FRF has varied between 0.2% and 0.6% of the GDP during the last six decades. Since the 1960s, the US FRF contribution has fallen from twice that of industrial research funding to roughly equal. Also, in the last two decades, the portion of the US government R&D spending devoted to research has increased. Although well below the US and the EU in overall funding, the current growth rate for R&D funding in China greatly exceeds that of both. Finally, the EU currently produces more science and engineering doctoral graduates and scientific publications than the US in absolute terms, but not per capita. This study's aim is to facilitate a serious discussion of key questions by the research community and federal policy makers. In particular, our results raise two questions with respect to: a) the increasing globalization of science: “What role is the US playing now, and what role will it play in the future of international science?”; and b) the ability to produce beneficial innovations for society: “How will the US continue to foster its strengths?”

Biomarkers of sensitivity and exposure in Washington state pesticide handlers

Organophosphate (OP) and N-methyl-carbamate (CB) insecticides are widely used in agriculture in the US and abroad. These compounds - which inhibit acetylcholinestersase (AChE) enzyme activity - continue to be responsible for a high proportion of pesticide poisonings among US agricultural workers. It is possible that some individuals may be especially susceptible to health effects related to OP/CB exposure. The paraoxonase (PON1) enzyme metabolizes the highly toxic oxon forms of some OPs, and an individual's PON1 status may be an important determinant of his or her sensitivity to these chemicals. This chapter discusses methods used to characterize the PON1 status of individuals and reviews previous epidemiologic studies that have evaluated PON1-related sensitivity to OPs in relation to various health endpoints. It also describes an ongoing longitudinal study among OP-exposed agricultural pesticide handlers who are participating in a recently implemented cholinesterase monitoring program in Washington State. This study will evaluate handlers' PON1 status as a hypothesized determinant of butyrylcholinesterase (BuChE) inhibition. Such studies will be useful to determine how regulatory risk assessments might account for differences in PON1-related OP sensitivity when characterizing inter-individual variability in risk related to OP exposure. Recent work assessing newer and more sensitive biomarkers of OP exposure is also discussed briefly in this chapter.

Occupational determinants of serum cholinesterase inhibition among organophosphate-exposed agricultural pesticide handlers in Washington State

OBJECTIVE

To identify potential risk factors for serum cholinesterase (BuChE) inhibition among agricultural pesticide handlers exposed to organophosphate (OP) and N-methyl-carbamate (CB) insecticides.

METHODS

We conducted a longitudinal study among 154 agricultural pesticide handlers who participated in the Washington State cholinesterase monitoring program in 2006 and 2007. BuChE inhibition was analysed in relation to reported exposures before and after adjustment for potential confounders using linear regression. ORs estimating the risk of BuChE depression (>20% from baseline) were also calculated for selected exposures based on unconditional logistic regression analyses.

RESULTS

An overall decrease in mean BuChE activity was observed among study participants at the time of follow-up testing during the OP/CB spray season relative to pre-season baseline levels (mean decrease of 5.6%, p<0.001). Score for estimated cumulative exposure to OP/CB insecticides in the past 30 days was a significant predictor of BuChE inhibition (beta=-1.74, p<0.001). Several specific work practices and workplace conditions were associated with greater BuChE inhibition, including mixing/loading pesticides and cleaning spray equipment. Factors that were protective against BuChE inhibition included full-face respirator use, wearing chemical-resistant boots and storing personal protective equipment in a locker at work.

CONCLUSIONS

Despite existing regulations, agricultural pesticide handlers continue to be exposed to OP/CB insecticides at levels resulting in BuChE inhibition. These findings suggest that modifying certain work practices could potentially reduce BuChE inhibition. Replication from other studies will be valuable.

Serum cholinesterase inhibition in relation to paraoxonase-1 (PON1) status among organophosphate-exposed agricultural pesticide handlers

BACKGROUND

Animal studies have demonstrated that low paraoxonase-1 (PON1) status (i.e., low catalytic efficiency and/or low plasma PON1 activity) is associated with neurotoxic effects after exposure to several organophosphate (OP) insecticides. However, few human studies have investigated associations between PON1 status and intermediate end points, such as serum cholinesterase [butyrylcholinesterase (BuChE)] inhibition, among OP-exposed individuals.

OBJECTIVES

We evaluated the relation between plasma PON1 status and BuChE inhibition among OP-exposed agricultural pesticide handlers.

METHODS

Agricultural pesticide handlers in Washington State were recruited during the 2006 and 2007 spray seasons when they were seen for follow-up ChE testing by collaborating medical providers as part of a statewide monitoring program. Blood samples were collected from 163 participants and tested for PON1 status based on plasma PON1 activity [arylesterase (AREase)] and PON1 Q192R genotype. We evaluated percent change in BuChE activity from baseline level in relation to PON1 status.

RESULTS

We observed significantly greater BuChE inhibition among QQ homozygotes relative to RR homozygotes (p = 0.036). Lower levels of plasma PON1 activity were significantly associated with greater BuChE inhibition (p = 0.004). These associations remained after adjustment for year, days since baseline test, age, and OP exposure in the last 30 days.

CONCLUSIONS

We found that both low PON1 catalytic efficiency (i.e., the Q192 alloform) and low plasma PON1 activity were associated with BuChE inhibition among OP-exposed agricultural pesticide handlers. Corroborative findings from future studies with prospective collection of blood samples for PON1 testing, more sensitive markers of OP-related effects, and larger sample sizes are needed.

Consortium to Establish a Registry for Alzheimer's Disease (CERAD): the first twenty years

The Consortium to Establish a Registry for Alzheimer's Disease (CERAD) was funded by the National Institute on Aging in 1986 to develop standardized, validated measures for the assessment of Alzheimer's disease (AD). The present report describes the measures that CERAD developed during its first decade and their continued use in their original and translated forms. These measures include clinical, neuropsychological, neuropathologic, and behavioral assessments of AD and also assessment of family history and parkinsonism in AD. An approach to evaluating neuroimages did not meet the standards desired. Further evaluations that could not be completed because of lack of funding (but where some materials are available) include evaluation of very severe AD and of service use and need by patient and caregiver. The information that was developed in the U.S. and abroad permits standardized assessment of AD in clinical practice, facilitates epidemiologic studies, and provides information valuable for individual and public health planning. CERAD materials and data remain available for those wishing to use them.

Item response theory facilitated cocalibrating cognitive tests and reduced bias in estimated rates of decline

OBJECTIVE

To cocalibrate the Mini-Mental State Examination, the Modified Mini-Mental State, the Cognitive Abilities Screening Instrument, and the Community Screening Instrument for Dementia using item response theory (IRT) to compare screening cut points used to identify cases of dementia from different studies, to compare measurement properties of the tests, and to explore the implications of these measurement properties on longitudinal studies of cognitive functioning over time.

STUDY DESIGN AND SETTING

We used cross-sectional data from three large (n>1000) community-based studies of cognitive functioning in the elderly. We used IRT to cocalibrate the scales and performed simulations of longitudinal studies.

RESULTS

Screening cut points varied quite widely across studies. The four tests have curvilinear scaling and varied levels of measurement precision, with more measurement error at higher levels of cognitive functioning. In longitudinal simulations, IRT scores always performed better than standard scoring, whereas a strategy to account for varying measurement precision had mixed results.

CONCLUSION

Cocalibration allows direct comparison of cognitive functioning in studies using any of these four tests. Standard scoring appears to be a poor choice for analysis of longitudinal cognitive testing data. More research is needed into the implications of varying levels of measurement precision.

A note on the false discovery rate and inconsistent comparisons between experiments

MOTIVATION

The false discovery rate (FDR) has been widely adopted to address the multiple comparisons issue in high-throughput experiments such as microarray gene-expression studies. However, while the FDR is quite useful as an approach to limit false discoveries within a single experiment, like other multiple comparison corrections it may be an inappropriate way to compare results across experiments. This article uses several examples based on gene-expression data to demonstrate the potential misinterpretations that can arise from using FDR to compare across experiments. Researchers should be aware of these pitfalls and wary of using FDR to compare experimental results. FDR should be augmented with other measures such as p-values and expression ratios. It is worth including standard error and variance information for meta-analyses and, if possible, the raw data for re-analyses. This is especially important for high-throughput studies because data are often re-used for different objectives, including comparing common elements across many experiments. No single error rate or data summary may be appropriate for all of the different objectives.

Experiment-specific estimation of peptide identification probabilities using a randomized database

Determining the error rate for peptide and protein identification accurately and reliably is necessary to enable evaluation and crosscomparisons of high throughput proteomics experiments. Currently, peptide identification is based either on preset scoring thresholds or on probabilistic models trained on datasets that are often dissimilar to experimental results. The false discovery rates (FDR) and peptide identification probabilities for these preset thresholds or models often vary greatly across different experimental treatments, organisms, or instruments used in specific experiments. To overcome these difficulties, randomized databases have been used to estimate the FDR. However, the cumulative FDR may include low probability identifications when there are a large number of peptide identifications and exclude high probability identifications when there are few. To overcome this logical inconsistency, this study expands the use of randomized databases to generate experiment-specific estimates of peptide identification probabilities. These experiment-specific probabilities are generated by logistic and Loess regression models of the peptide scores obtained from original and reshuffled database matches. These experiment-specific probabilities are shown to very well approximate "true" probabilities based on known standard protein mixtures across different experiments. Probabilities generated by the earlier Peptide_Prophet and more recent LIPS models are shown to differ significantly from this study's experiment-specific probabilities, especially for unknown samples. The experiment-specific probabilities reliably estimate the accuracy of peptide identifications and overcome potential logical inconsistencies of the cumulative FDR. This estimation method is demonstrated using a Sequest database search, LIPS model, and a reshuffled database. However, this approach is generally applicable to any search algorithm, peptide scoring, and statistical model when using a randomized database.

Evaluating the quality of longitudinal statistical applications in original publications on Alzheimer's disease

BACKGROUND/AIMS

To evaluate the quality of longitudinal statistical applications in published studies on Alzheimer's disease (AD).

METHODS

A 21-item instrument, the Quality of Longitudinal AD Studies (QLADS), was developed by the research team (4 biostatisticians, 1 neuroepidemiologist, and 1 neurologist). All items were extensively discussed within the team for content validity. After pilot testing on 5 publications, the instrument was revised and tested for reliability with a sample of 40 published longitudinal AD studies randomly sampled from MEDLINE.

RESULTS

Item-specific test-retest reliability coefficients for QLADS ranged from 0.53 to 1.00 with the associated standard error (SE) ranging from 0.02 to 0.13. The test-retest reliability for the overall score over the 21 items was high (intraclass correlation coefficient (ICC) = 0.94, 95% CI 0.90, 0.97). Item-specific inter-rater reliability coefficients for QLADS ranged from 0.46 to 1.00 with the associated SE ranging from 0.07 to 0.18. The inter-rater reliability for the overall score was also high (ICC = 0.87, 95% CI 0.77, 0.93).

CONCLUSIONS

This study indicates that the quality of longitudinal statistical applications in AD publications can be reliably assessed.

A parametric comparison of diagnostic accuracy with three ordinal diagnostic groups

Many medical diagnostic studies involve three ordinal diagnostic groups in which the diagnostic accuracy can be summarized by the volume or partial volume under a Receiver Operating Characteristic (ROC) surface. We study in this paper the statistical comparison of diagnostic accuracy from multiple diagnostic tests when three ordinal diagnostic groups are involved. Under the assumption that the multiple diagnostic tests follow a multivariate normal distribution within each diagnostic group, we provide the asymptotic variance and covariance for the maximum likelihood estimates of the volumes under the ROC surfaces from multiple diagnostic tests and propose statistical tests to test whether the diagnostic accuracy as measured by the volume under the ROC surface is the same for multiple diagnostic tests. We also propose a confidence interval estimate to the difference of two volumes under two ROC surfaces. Our approach depends crucially on the assumptions of normal distributions on diagnostic tests, which might not be robust when such assumptions are violated. Finally, we apply our proposed methodology to a real data set of 118 subjects to compare the diagnostic accuracy of early stage Alzheimer's disease (AD) from multiple neuropsychological tests.

Development of BIATECH-54 standard mixtures for assessment of protein identification and relative expression

Mixtures of known proteins have been very useful in the assessment and validation of methods for high-throughput (HTP) MS (MS/MS) proteomics experiments. However, these test mixtures have generally consisted of few proteins at near equal concentration or of a single protein at varied concentrations. Such mixtures are too simple to effectively assess the validity of error rates for protein identification and differential expression in HTP MS/MS studies. This work aimed at overcoming these limitations and simulating studies of complex biological samples. We introduced a pair of 54-protein standard mixtures of variable concentrations with up to a 1000-fold dynamic range in concentration and up to ten-fold expression ratios with additional negative controls (infinite expression ratios). These test mixtures comprised 16 off-the-shelf Sigma-Aldrich proteins and 38 Shewanella oneidensis proteins produced in-house. The standard proteins were systematically distributed into three main concentration groups (high, medium, and low) and then the concentrations were varied differently for each mixture within the groups to generate different expression ratios. The mixtures were analyzed with both low mass accuracy LCQ and high mass accuracy FT-LTQ instruments. In addition, these 54 standard proteins closely follow the molecular weight distributions of both bacterial and human proteomes. As a result, these new standard mixtures allow for a much more realistic assessment of approaches for protein identification and label-free differential expression than previous mixtures. Finally, methodology and experimental design developed in this work can be readily applied in future to development of more complex standard mixtures for HTP proteomics studies.

Assessing statistical applications in publications on Alzheimer's disease

BACKGROUND/AIMS

To evaluate statistical applications in publications on Alzheimer's disease (AD).

METHODS

Three instruments/checklists were developed: Assessment of Statistical Reporting (ASR; 44 items), Survey of Statistical Designs (SSD; 10 items), and Survey of Statistical Methods (SSM; 7 items). After a pilot testing on 5 AD publications, the instruments/checklists were revised and tested for reliability with a sample of 30 AD articles and for validity with another sample of 10 AD articles from MEDLINE.

RESULTS

Item-specific test-retest and interrater reliability for ASR ranged from 0.29 to 1.0 with the associated standard errors (SEs) ranging from 0.01 to 0.31. The test-retest reliability (intraclass correlation coefficient = 0.94, 95% CI: 0.88-0.97) and the interrater reliability (intraclass correlation coefficient = 0.84, 95% CI: 0.69-0.92) for the overall score of ASR were high. The correlational validity of the ASR with a published checklist was also high (r = 0.74, SE = 0.24). The item-specific test-retest reliability in SSD and SSM ranged from 0.58 to 1.00 with the associated SEs ranging from 0.01 to 0.32. The item-specific interrater reliability in SSD and SSM ranged from 0.17 to 1.00 with the associated SEs ranging from 0.01 to 0.22.

CONCLUSIONS

This study suggested that it was feasible to assess statistical applications in AD publications.

The epidemiology of narcolepsy

Much has been learned about the pathophysiology of narcolepsy over the last several decades. It is likely that hypocretin-producing cells in the lateral hypothalamus are selectively destroyed in genetically susceptible individuals carrying 1 or more alleles of HLA DQB1*0602. Despite advances, the causes of narcolepsy and how to prevent it remain elusive. Classic epidemiology aims not only to enumerate occurrence of disease in populations, but also to identify etiologic risk factors. This review details what the application of classic epidemiology has taught us so far about narcolepsy and suggests directions for future studies to clarify its etiology. The prevalence of narcolepsy with cataplexy has been examined in many studies and falls between 25 and 50 per 100,000 people. Information on incidence is limited, with 1 study finding the incidence of narcolepsy with cataplexy to be 0.74 per 100,000 person-years. The search for etiologic risk factors has yet to yield important associations. Factors most thoroughly examined include body mass index, immune responses, and stressful life events. Such associations may reflect a consequence rather than a cause of disease. As with other diseases characterized by selective cell loss, such as Parkinson disease or type 1 diabetes mellitus, narcolepsy is likely caused by environmental exposures before the age of onset in genetically susceptible individuals. Matching efforts in these other diseases and using large well-designed epidemiologic studies of narcolepsy, investigators must intensify the search for these exposures, focusing on the first 2 decades of life. Identification of modifiable risk factors will help to prevent this disease.