The central role of receiver operating characteristic (ROC) curves in evaluating tests for the early detection of cancer

Discovery and validation of graft-versus-host disease biomarkers

Allogeneic hematopoietic stem cell transplantation (allo-HSCT) is the most effective tumor immunotherapy available. Although allo-HSCT provides beneficial graft-versus-tumor effects, acute GVHD (aGVHD) is the primary source of morbidity and mortality after HSCT. Diagnosis of aGVHD is typically based on clinical symptoms in one or more of the main target organs (skin, liver, gastrointestinal tract) and confirmed by biopsy. However, currently available diagnostic and staging tools often fail to identify patients at higher risk of GVHD progression, unresponsiveness to therapy, or death. In addition, there are shortcomings in the prediction of GVHD before clinical signs develop, indicating the urgent need for noninvasive and reliable laboratory tests. Through the continuing evolution of proteomics technologies seen in recent years, plasma biomarkers have been identified and validated as promising diagnostic tools for GVHD and prognostic tools for nonrelapse mortality. These biomarkers may facilitate timely and selective therapeutic intervention but should be more widely validated and incorporated into a new grading system for risk stratification of patients and better-customized treatment. This review identifies biomarkers for detecting GVHD, summarizes current information on aGVHD biomarkers, proposes future prospects for the blinded evaluation of these biomarkers, and discusses the need for biomarkers of chronic GVHD.

Assessing the clinical utility of biomarkers in medicine

Biomarkers in medicine have gained immense scientific and clinical interest in recent years. Biomarkers are potentially useful in the contexts of primary, secondary and tertiary prevention. Some of the characteristics of an ideal biomarker include that they are safe and easy to measure, are associated with acceptable costs (including those of the follow-up tests), and there is scientific evidence to suggest that biomarker use/modification influences disease outcomes. Additionally, variation in biomarker levels with gender and ethnicity should be elucidated, and the biomarker should have 'good performance characteristics' (i.e., sensitivity, specificity, positive- and negative-predictive values and positive- and negative-likelihood ratios). Risk prediction scores can combine information from several different biomarkers in order to estimate an individual's risk of developing an outcome, such as disease or death. Three commonly employed methods to test if a biomarker will add to traditional risk prediction models are model discrimination, model calibration and risk reclassification. 'Multimarker' strategies serve to integrate information from multiple biomarkers into risk prediction but may be limited by the presence of highly correlated biomarkers, economic costs and selection bias of biomarker candidates in a particular study sample. In the future, integration of biomarkers identified using emerging technologies from the 'omics fields (including genomics, proteomics, metabolomics, lipomics, ribomics and pharmacogenomics) may be useful for the 'personalization' of treatment/disease prevention.

An Extension of the Receiver Operating Characteristic Curve and AUC-Optimal Classification

While most proposed methods for solving classification problems focus on minimization of the classification error rate, we are interested in the receiver operating characteristic (ROC) curve, which provides more information about classification performance than the error rate does. The area under the ROC curve (AUC) is a natural measure for overall assessment of a classifier based on the ROC curve. We discuss a class of concave functions for AUC maximization in which a boosting-type algorithm including RankBoost is considered, and the Bayesian risk consistency and the lower bound of the optimum function are discussed. A procedure derived by maximizing a specific optimum function has high robustness, based on gross error sensitivity. Additionally, we focus on the partial AUC, which is the partial area under the ROC curve. For example, in medical screening, a high true-positive rate to the fixed lower false-positive rate is preferable and thus the partial AUC corresponding to lower false-positive rates is much more important than the remaining AUC. We extend the class of concave optimum functions for partial AUC optimality with the boosting algorithm. We investigated the validity of the proposed method through several experiments with data sets in the UCI repository.

Potential of computer-aided diagnosis to improve CT lung cancer screening

The development of low-dose spiral computed tomography (CT) has rekindled hope that effective lung cancer screening might yet be found. Screening is justified when there is evidence that it will extend lives at reasonable cost and acceptable levels of risk. A screening test should detect all extant cancers while avoiding unnecessary workups. Thus optimal screening modalities have both high sensitivity and specificity. Due to the present state of technology, radiologists must opt to increase sensitivity and rely on follow-up diagnostic procedures to rule out the incurred false positives. There is evidence in published reports that computer-aided diagnosis technology may help radiologists alter the benefit-cost calculus of CT sensitivity and specificity in lung cancer screening protocols. This review will provide insight into the current discussion of the effectiveness of lung cancer screening and assesses the potential of state-of-the-art computer-aided design developments.

MET phosphorylation predicts poor outcome in small cell lung carcinoma and its inhibition blocks HGF-induced effects in MET mutant cell lines

Background:

Small cell lung carcinoma (SCLC) has poor prognosis and remains orphan from targeted therapy. MET is activated in several tumour types and may be a promising therapeutic target.

Methods:

To evaluate the role of MET in SCLC, MET gene status and protein expression were evaluated in a panel of SCLC cell lines. The MET inhibitor PHA-665752 was used to study effects of pathway inhibition in basal and hepatocyte growth factor (HGF)-stimulated conditions. Immunohistochemistry for MET and p-MET was performed in human SCLC samples and association with outcome was assessed.

Results:

In MET mutant SCLC cells, HGF induced MET phosphorylation, increased proliferation, invasiveness and clonogenic growth. PHA-665752 blocked MET phosphorylation and counteracted HGF-induced effects. In clinical samples, total MET and p-MET overexpression were detected in 54% and 43% SCLC tumours (n=77), respectively. MET phosphorylation was associated with poor median overall survival (132 days) vs p-MET negative cases (287 days)(P<0.001). Phospho-MET retained its prognostic value in a multivariate analysis.

Conclusions:

MET activation resulted in a more aggressive phenotype in MET mutant SCLC cells and its inhibition by PHA-665752 reversed this phenotype. In patients with SCLC, MET activation was associated with worse prognosis, suggesting a role in the adverse clinical behaviour in this disease.

Multimarker panels in sepsis

Biomarkers differentiate between 2 or more biologic states. The complexity of diseases like sepsis makes it unlikely that any single marker will allow for precise disease specification. Combining several biomarkers into a single classification rule should help to improve their accuracy and, therefore, their usefulness. This article reviews several studies using multimarker panels, and highlights the potential of more sophisticated diagnostic and prognostic techniques in future multimarker panels. More complex algorithms should accelerate the adoption of multimarker panels into the routine management of patients with sepsis, provided that clinicians understand the multimarker approach.

A min-max combination of biomarkers to improve diagnostic accuracy

Diagnostic accuracy can be improved considerably by combining multiple biomarkers. Although the likelihood ratio provides optimal solution to combination of biomarkers, the method is sensitive to distributional assumptions which are often difficult to justify. Alternatively simple linear combinations can be considered whose empirical solution may encounter intensive computation when the number of biomarkers is relatively large. Moreover, the optimal linear combinations derived under multivariate normality may suffer substantial loss of efficiency if the distributions are apart from normality. In this paper, we propose a new approach that linearly combines the minimum and maximum values of the biomarkers. Such combination only involves searching for a single combination coefficient that maximizes the area under the receiver operating characteristic (ROC) curves and is thus computation-effective. Simulation results show that the min-max combination may yield larger partial or full area under the ROC curves and is more robust against distributional assumptions. The methods are illustrated using the growth-related hormones data from the Growth and Maturation in Children with Autism or Autistic Spectrum Disorder Study (Autism/ASD Study).

High levels of PROM1 (CD133) transcript are a potential predictor of poor prognosis in medulloblastoma

The surface marker PROM1 is considered one of the most important markers of tumor-initiating cells, and its expression is believed to be an adverse prognostic factor in gliomas and in other malignancies. To date, to our knowledge, no specific studies of its expression in medulloblastoma series have been performed. The aims of our study were to evaluate the expression profile of the PROM1 gene in medulloblastoma and to assess its possible role as a prognostic factor. The PROM1 gene expression was evaluated by quantitative- polymerase chain reaction on 45 medulloblastoma samples by using specific dye-labeled probe systems. A significantly higher expression of PROM1 was found both in patients with poorer prognosis (P= .007) and in those with metastasis (P= .03). Kaplan-Meier analysis showed that both overall survival (OS) and progression-free survival (PFS) were shorter in patients with higher PROM1 mRNA levels than in patients with lower expression, even when the desmoplastic cases were excluded (P= .0004 and P= .002, for OS and PFS for all cases, respectively; P= .002 and P= .008 for OS and PFS for nondesmoplastic cases, respectively). Cox regression model demonstrated that PROM1 expression is an independent prognostic factor (hazard ratio, 4.56; P= .008). The result was validated on an independent cohort of 42 cases by microarray-based analysis (P= .019). This work suggests that high mRNA levels of PROM1 are associated with poor outcome in pediatric medulloblastoma. Furthermore, high PROM1 expression levels seem to increase the likelihood of metastases. Such results need to be confirmed in larger prospective series to possibly incorporate PROM1 gene expression into risk classification systems to be used in the clinical setting.

Urinary glycoprotein biomarker discovery for bladder cancer detection using LC/MS-MS and label-free quantification

BACKGROUND

Cancers of the urinary bladder are the fifth most commonly diagnosed malignancy in the United States. Early clinical diagnosis of bladder cancer remains a major challenge, and the development of noninvasive methods for detection and surveillance is desirable for both patients and health care providers.

APPROACH

To identify urinary proteins with potential clinical utility, we enriched and profiled the glycoprotein component of urine samples by using a dual-lectin affinity chromatography and liquid chromatography/tandem mass spectrometry platform.

RESULTS

From a primary sample set obtained from 54 cancer patients and 46 controls, a total of 265 distinct glycoproteins were identified with high confidence, and changes in glycoprotein abundance between groups were quantified by a label-free spectral counting method. Validation of candidate biomarker alpha-1-antitrypsin (A1AT) for disease association was done on an independent set of 70 samples (35 cancer cases) by using an ELISA. Increased levels of urinary A1AT glycoprotein were indicative of the presence of bladder cancer (P < 0.0001) and augmented voided urine cytology results. A1AT detection classified bladder cancer patients with a sensitivity of 74% and specificity of 80%.

SUMMARY

The described strategy can enable higher resolution profiling of the proteome in biological fluids by reducing complexity. Application of glycoprotein enrichment provided novel candidates for further investigation as biomarkers for the noninvasive detection of bladder cancer.

Systems biology and cancer: promises and perils

Systems biology uses systems of mathematical rules and formulas to study complex biological phenomena. In cancer research there are three distinct threads in systems biology research: modeling biology or biophysics with the goal of establishing plausibility or obtaining insights, modeling based on statistics, bioinformatics, and reverse engineering with the goal of better characterizing the system, and modeling with the goal of clinical predictions. Using illustrative examples we discuss these threads in the context of cancer research.

In Patients Younger Than Age 55 Years With Lymph Node–Negative Breast Cancer, Proliferation by Mitotic Activity Index Is Prognostically Superior to Adjuvant!

Purpose

In breast cancer, different tools are used for prognostication and adjuvant systemic therapy selection. We compared the accuracy of the online program Adjuvant!, the Norwegian Breast Cancer Group (NBCG) guidelines, and the proliferation factor mitotic activity index (MAI) in patients with lymph node (LN) –negative disease (pN0).

Patients and Methods

Adjuvant! and MAI thresholds were set to 90% to 95% breast cancer–specific survival (BCSS) rates. These thresholds were 95% for Adjuvant!, 3 for MAI, and as follows for NBCG: pT1 grade 1 + pT1a-b grade 2 to 3; all pN0M0 and estrogen receptor/progesterone receptor positive versus all others. In 516 patients younger than age 55 years (T1-3N0M0) without adjuvant systemic therapy, univariable and multivariable 10-year BCSS rates were estimated.

Results

Median follow-up time was 118 months. The concordance between MAI and Adjuvant! or NBCG was fair (κ = 0.35 and κ = 0.29, respectively). Adjuvant!, NBCG, and MAI were all prognostically significant (P ≤ .001). In the univariable analysis, the 10-year BCSS of MAI less than 3 versus ≥ 3 was 95% v 71%, respectively, with a hazard ratio of 7.0. In multivariable analysis, MAI was superior to Adjuvant! and NBCG. The 10-year survival of Adjuvant! ≥ 95% versus less than 95% was 91% v 74%, respectively, but stratification by MAI identified subgroups with different prognosis. Similar results occurred for NBCG and MAI. Adjuvant! and NBCG were not prognostic to each other.

Conclusion

MAI is superior to Adjuvant! and NBCG in prognostication of patients with LN-negative breast cancer younger than age 55 years.

Diagnostic accuracy and receiver-operating characteristics curve analysis in surgical research and decision making

In surgical research, the ability to correctly classify one type of condition or specific outcome from another is of great importance for variables influencing clinical decision making. Receiver-operating characteristic (ROC) curve analysis is a useful tool in assessing the diagnostic accuracy of any variable with a continuous spectrum of results. In order to rule a disease state in or out with a given test, the test results are usually binary, with arbitrarily chosen cut-offs for defining disease versus health, or for grading of disease severity. In the postgenomic era, the translation from bench-to-bedside of biomarkers in various tissues and body fluids requires appropriate tools for analysis. In contrast to predetermining a cut-off value to define disease, the advantages of applying ROC analysis include the ability to test diagnostic accuracy across the entire range of variable scores and test outcomes. In addition, ROC analysis can easily examine visual and statistical comparisons across tests or scores. ROC is also favored because it is thought to be independent from the prevalence of the condition under investigation. ROC analysis is used in various surgical settings and across disciplines, including cancer research, biomarker assessment, imaging evaluation, and assessment of risk scores.With appropriate use, ROC curves may help identify the most appropriate cutoff value for clinical and surgical decision making and avoid confounding effects seen with subjective ratings. ROC curve results should always be put in perspective, because a good classifier does not guarantee the expected clinical outcome. In this review, we discuss the fundamental roles, suggested presentation, potential biases, and interpretation of ROC analysis in surgical research.

PSGR and PCA3 as biomarkers for the detection of prostate cancer in urine

BACKGROUND

Several studies have demonstrated the usefulness of monitoring an RNA transcript in urine, such as PCA3, for prostate cancer (PCa) diagnosis. PCa screening would benefit from additional biomarkers of higher specificity and could be used in conjunction with prostate-specific antigen (PSA) testing, in order to better determine biopsy candidates.

METHODS

We used urine sediments after prostate massage (PM) from 215 consecutive patients, who presented for prostate biopsy. We tested whether prostate-specific G-protein coupled receptor (PSGR), a biomarker previously described to be over-expressed in PCa tissue, could also be detected by quantitative real-time PCR in post-PM urine sediment. We combined these findings with prostate cancer gene 3 (PCA3), the current gold standard for PCa diagnosis in urine, to test if a combination of both biomarkers could improve the sensitivity of PCA3 alone.

RESULTS

By univariate analysis we found that PSGR and PCA3 were significant predictors of PCa. Receiver operator characteristic curve analysis and its multivariate extension, multivariate ROC (MultiROC), were used to assess the outcome predictive values of the individual and the paired biomarkers. We obtained the following area under the curve values: PSA (0.602), PSGR (0.681), PCA3 (0.656), and PSGRvPCA3 (0.729). Then, we tested whether a combination of PSGR and PCA3 could improve specificity by fixing the sensitivity at 95%. We obtained specificities of 15% (PSGR), 17% (PCA3), and 34% (PSGRvPCA3).

CONCLUSIONS

A multiplexed model including PSGR and PCA3 improves the specificity for the detection of PCa, especially in the area of high sensitivity. This could be clinically useful for determining which patients should undergo biopsy.

Type IV collagen levels are elevated in the serum of patients with peritoneal dissemination of gastric cancer

Type III procollagen (amino-terminal propeptide of procollagen type III) and type IV collagen are considered to be reliable serum markers for monitoring the progression of liver fibrosis. The peritoneal dissemination of gastric cancer is also characterised by abundant collagen deposition in the peritoneum. The present study was performed to investigate the potential of serum type III procollagen and IV collagen as biomarkers for peritoneal dissemination in gastric cancer. The study population consisted of 117 patients with gastric cancer: 32 patients had peritoneal dissemination which was pathologically diagnosed by laparotomy or laparoscopic examination, while 85 patients (45/40, early/advanced gastric cancer) had no peritoneal dissemination. We measured the serum levels of type III procollagen and type IV collagen in comparison to the commonly accepted tumor markers carcinoembryonic (CEA), carbohydrate antigen (CA)19-9 and CA125. The median type III procollagen levels showed no significant differences between the two groups, whereas the median type IV collagen levels were significantly (201 ng/ml) higher in patients with than in those without peritoneal dissemination (early/advanced gastric cancer, 124/136 ng/ml) (P<0.05). In receiver operating characteristic (ROC) curve analysis, type IV collagen had the largest area under the curve (0.83), followed by CA125 (0.72), CA19-9 (0.64), CEA (0.59) and type III procollagen (0.48). Type IV collagen was an independent marker (P<0.0001, odds ratio 15.7) for predicting peritoneal dissemination along with CA125 (P=0.0086, odds ratio 9.4) based on multivariate logistic regression. In conclusion, serum type IV collagen levels may be significant in the early detection and management of patients with peritoneal dissemination of gastric cancer.

Estimating the diagnostic likelihood ratio of a continuous marker

The diagnostic likelihood ratio function, DLR, is a statistical measure used to evaluate risk prediction markers. The goal of this paper is to develop new methods to estimate the DLR function. Furthermore, we show how risk prediction markers can be compared using rank-invariant DLR functions. Various estimators are proposed that accommodate cohort or case-control study designs. Performances of the estimators are compared using simulation studies. The methods are illustrated by comparing a lung function measure and a nutritional status measure for predicting subsequent onset of major pulmonary infection in children suffering from cystic fibrosis. For continuous markers, the DLR function is mathematically related to the slope of the receiver operating characteristic (ROC) curve, an entity used to evaluate diagnostic markers. We show that our methodology can be used to estimate the slope of the ROC curve and illustrate use of the estimated ROC derivative in variance and sample size calculations for a diagnostic biomarker study.

A boosting method for maximizing the partial area under the ROC curve

BACKGROUND

The receiver operating characteristic (ROC) curve is a fundamental tool to assess the discriminant performance for not only a single marker but also a score function combining multiple markers. The area under the ROC curve (AUC) for a score function measures the intrinsic ability for the score function to discriminate between the controls and cases. Recently, the partial AUC (pAUC) has been paid more attention than the AUC, because a suitable range of the false positive rate can be focused according to various clinical situations. However, existing pAUC-based methods only handle a few markers and do not take nonlinear combination of markers into consideration.

RESULTS

We have developed a new statistical method that focuses on the pAUC based on a boosting technique. The markers are combined componentially for maximizing the pAUC in the boosting algorithm using natural cubic splines or decision stumps (single-level decision trees), according to the values of markers (continuous or discrete). We show that the resulting score plots are useful for understanding how each marker is associated with the outcome variable. We compare the performance of the proposed boosting method with those of other existing methods, and demonstrate the utility using real data sets. As a result, we have much better discrimination performances in the sense of the pAUC in both simulation studies and real data analysis.

CONCLUSIONS

The proposed method addresses how to combine the markers after a pAUC-based filtering procedure in high dimensional setting. Hence, it provides a consistent way of analyzing data based on the pAUC from maker selection to marker combination for discrimination problems. The method can capture not only linear but also nonlinear association between the outcome variable and the markers, about which the nonlinearity is known to be necessary in general for the maximization of the pAUC. The method also puts importance on the accuracy of classification performance as well as interpretability of the association, by offering simple and smooth resultant score plots for each marker.

Prognostic immunophenotypic biomarker studies in diffuse large B cell lymphoma with special emphasis on rational determination of cut-off scores

A number of biomarkers, particularly proteins that contribute to prognosis in diffuse large B cell lymphoma (DLBCL), have been identified. However, translation into accepted standards to predict survival has not yet been accomplished, primarily due to contradictory reports in the literature resulting from, among other factors, arbitrary methodologies used to set cut-off values for determining positivity. Some of these problems might be resolved by application of rational statistical methods for determination of cut-off scores. Herein, we critically address issues on in situ phenotypic prognostic tumor-related biomarkers in DLBCL with a particular and practical emphasis on tools for cut-off level determination, especially receiver operating characteristic curve analysis. Moreover, we candidly illustrate the application of these tools for efficient disease-specific survival prognostication on a tissue microarray collective of 240 primary DLBCL using the common prognostic biomarkers Bcl-2, Bcl-6, CD10, FOXP1, MUM1, and Cyclin E. Comparison of the results relative to disease-specific survival unequivocally showed the superior discriminatory power of the cut-off levels calculated by receiver operating curves and the Youden's index, compared to arbitrary cut-off values from the literature, advocating fundamental application of rational methods for determination of clinically relevant prognostic biomarkers' cut-off scores.

Clinical significance of CEA and CA19-9 in postoperative follow-up of colorectal cancer

BACKGROUND

We evaluated the efficiency of CEA and CA19-9 as tools for diagnosing recurrence in the postoperative surveillance of colorectal cancer.

MATERIALS AND METHODS

A total of 227 patients who underwent curative resection for colorectal cancer between 1999 and 2003 at our hospital received complete follow-up according to the schedule determined prospectively. Using receiver operating characteristic (ROC) analysis, performance of postoperative values of CEA or CA19-9 for detecting recurrence was assessed.

RESULTS

The sensitivity (1.000) and specificity (0.978) of the postoperative values of CEA in the high preoperative CEA group were very high. Even in the normal preoperative CEA group, the area under the curve (AUC) of the ROC curve of CEA (0.740, 95% confidence interval [95% CI], 0.628-0.852) was significantly larger than 0.5 (P < 0.001). The postoperative values of CA19-9 showed high sensitivity (0.833) and specificity (0.900) in the high preoperative CA19-9 group, while the AUC of the ROC curve of the normal preoperative group was as small as 0.510 (95% CI, 0.376-0.644). In the high preoperative CA19-9 group, however, there was no significant difference between the AUC of CA19-9 (0.904, 95% CI, 0.786-1.000) and that of CEA (0.869, 95% CI, 0.744-0.994) (P = 0.334).

CONCLUSIONS

The measurement of CEA is an efficient way to detect recurrence. The efficiency of measuring CA19-9 for the purpose of detecting recurrence is low, especially in patients with a normal level of preoperative CA19-9. Even in patients with a high preoperative level of CA19-9, CEA might be able to fill the role of CA19-9.

Next generation sequencing of serum circulating nucleic acids from patients with invasive ductal breast cancer reveals differences to healthy and nonmalignant controls

Circulating nucleic acids (CNA) isolated from serum or plasma are increasingly recognized as biomarkers for cancers. Recently developed next generation sequencing provides high numbers of DNA sequences to detect the trace amounts of unique serum biomarkers associated with breast carcinoma. Serum CNA of 38 women with ductal carcinoma was extracted and sequenced on a 454/Roche high-throughput GS-FLX platform and compared with healthy controls and patients with other medical conditions. Repetitive elements present in CNA were detected and classified, and each repetitive element was normalized based on total sequence count or repeat count. Multivariate regression models were calculated using an information-theoretical approach and multimodel inference. A total of 423,150 and 953,545 sequences for the cancer patients and controls, respectively, were obtained. Data from 26 patients with stages II to IV tumors and from 67 apparently healthy female controls were used as the training data set. Using a bootstrap method to avoid sampling bias, a five-parameter model was developed. When this model was applied to a validation data set consisting of patients with tumor stage I (n = 10) compared with healthy and nonmalignant disease controls (n = 87; 1,261,561 sequences) a sensitivity of 70% at a specificity of 100% was obtained. At a diagnostic specificity level of 95%, a sensitivity of 90% was calculated. Identification of specific breast cancer-related CNA sequences provides the basis for the development of a serum-based routine laboratory test for breast cancer screening and monitoring.

Putting risk prediction in perspective: relative utility curves

Risk prediction models based on medical history or results of tests are increasingly common in the cancer literature. An important use of these models is to make treatment decisions on the basis of estimated risk. The relative utility curve is a simple method for evaluating risk prediction in a medical decision-making framework. Relative utility curves have three attractive features for the evaluation of risk prediction models. First, they put risk prediction into perspective because relative utility is the fraction of the expected utility of perfect prediction obtained by the risk prediction model at the optimal cut point. Second, they do not require precise specification of harms and benefits because relative utility is plotted against a summary measure of harms and benefits (ie, the risk threshold). Third, they are easy to compute from standard tables of data found in many articles on risk prediction. An important use of relative utility curves is to evaluate the addition of a risk factor to the risk prediction model. To illustrate an application of relative utility curves, an analysis was performed on previously published data involving the addition of breast density to a risk prediction model for invasive breast cancer.

Prostate specific antigen for early detection of prostate cancer: longitudinal study

OBJECTIVE

To evaluate if prostate specific antigen test attains validity standards required for screening in view of recent prostate cancer screening trial results.

DESIGN

Case-control study nested in longitudinal cohort.

SETTING

Västerbotten Intervention Project cohort, Umeå, Sweden.

PARTICIPANTS

540 cases and 1034 controls matched for age and date of blood draw.

MAIN OUTCOME MEASURE

Validity of prostate specific antigen for prediction of subsequent prostate cancer diagnosis by record linkage to cancer registry.

RESULTS

Blood samples were drawn on average 7.1 (SD 3.7) years before diagnosis. The area under the curve for prostate specific antigen was 0.84 (95% confidence interval 0.82 to 0.86). At prostate specific antigen cut-off values of 3, 4, and 5 ng/ml, sensitivity estimates were 59%, 44%, and 33%, and specificity estimates were 87%, 92%, and 95%. The positive likelihood ratio commonly considered to "rule in disease" is 10; in this study the positive likelihood ratios were 4.5, 5.5, and 6.4 for prostate specific antigen cut-off values of 3, 4, and 5 ng/ml. The negative likelihood ratio commonly considered to "rule out disease" is 0.1; in this study the negative likelihood ratios were 0.47, 0.61, and 0.70 for prostate specific antigen cut-off values of 3, 4, and 5 ng/ml. For a cut-off of 1.0 ng/ml, the negative likelihood ratio was 0.08.

CONCLUSIONS

No single cut-off value for prostate specific antigen concentration attained likelihood ratios formally required for a screening test. Prostate specific antigen concentrations below 1.0 ng/ml virtually ruled out a prostate cancer diagnosis during the follow-up. Additional biomarkers for early detection of prostate cancer are needed before population based screening for prostate cancer should be introduced.

Raman signature from brain hippocampus could aid Alzheimer's disease diagnosis

Micro-Raman spectroscopy (MRS) is used for the first time to our knowledge to investigate brain hippocampus tissue from Alzheimer's disease (AD) infected rats. In situ Raman analysis of tissue sections provides distinct spectra useful for distinguishing AD from normal state. The biochemical changes of brain hippocampus tissue including the deposit of beta-amyloid (Abeta) protein, the increase of cholesterol, and hyperphosphorylated tau are observed through MRS when AD occurs. A more convincing multi-Raman criterion based on single Raman peaks, and further in combination with statistical analysis of the entire Raman spectrum, is found capable of classifying brain hippocampus tissues with different pathological features. This study demonstrates the brain hippocampus is an important candidate for considering the early pathological state of AD, and Raman signatures from the brain hippocampus could aid AD diagnosis. In addition, Raman results undoubtedly confirm simultaneous changes of cholesterol and Abeta in the progression of AD.

Overview of risk prediction models in cardiovascular disease research

Many risk prediction models have been developed for cardiovascular diseases in different countries during the past three decades. However, there has not been consistent agreement regarding how to appropriately assess a risk prediction model, especially when new markers are added to an established risk prediction model. Researchers often use the area under the receiver operating characteristic curve (ROC) to assess the discriminatory ability of a risk prediction model. However, recent studies suggest that this method has serious limitations and cannot be the sole approach to evaluate the usefulness of a new marker in clinical and epidemiological studies. To overcome the shortcomings of this traditional method, new assessment methods have been proposed. The aim of this article is to overview various risk prediction models for cardiovascular diseases, to describe the receiver operating characteristic curve method and discuss some new assessment methods proposed recently. Some of the methods were illustrated with figures from a cardiovascular disease study in Australia.

MassMatrix: a database search program for rapid characterization of proteins and peptides from tandem mass spectrometry data

MassMatrix is a program that matches tandem mass spectra with theoretical peptide sequences derived from a protein database. The program uses a mass accuracy sensitive probabilistic score model to rank peptide matches. The MS/MS search software was evaluated by use of a high mass accuracy dataset and its results compared with those from MASCOT, SEQUEST, X!Tandem, and OMSSA. For the high mass accuracy data, MassMatrix provided better sensitivity than MASCOT, SEQUEST, X!Tandem, and OMSSA for a given specificity and the percentage of false positives was 2%. More importantly all manually validated true positives corresponded to a unique peptide/spectrum match. The presence of decoy sequence and additional variable PTMs did not significantly affect the results from the high mass accuracy search. MassMatrix performs well when compared with MASCOT, SEQUEST, X!Tandem, and OMSSA with regard to search time. MassMatrix was also run on a distributed memory clusters and achieved search speeds of approximately 100,000 spectra per hour when searching against a complete human database with eight variable modifications. The algorithm is available for public searches at (http://www.massmatrix.net).

Adjusting for covariate effects on classification accuracy using the covariate-adjusted receiver operating characteristic curve

Recent scientific and technological innovations have produced an abundance of potential markers that are being investigated for their use in disease screening and diagnosis. In evaluating these markers, it is often necessary to account for covariates associated with the marker of interest. Covariates may include subject characteristics, expertise of the test operator, test procedures or aspects of specimen handling. In this paper, we propose the covariate-adjusted receiver operating characteristic curve, a measure of covariate-adjusted classification accuracy. Nonparametric and semiparametric estimators are proposed, asymptotic distribution theory is provided and finite sample performance is investigated. For illustration we characterize the age-adjusted discriminatory accuracy of prostate-specific antigen as a biomarker for prostate cancer.

Prognostic relevance of androgen receptor detection in operable breast cancer

BACKGROUND AND OBJECTIVES

Androgen receptor (AR) is relevant for prognostication in breast cancer. Different determination methods and cut-off levels hamper interpretation and comparisons of studies. Long-term prognostic evaluation of different AR assays in patients comprising operable breast cancers is scarce.

METHODS

AR was evaluated in 120 primary tumors using the dextran-coated charcoal method (charc-AR), and quantitative immunohistochemistry (IHC) on whole sections (WS) and tissue microarrays (TMA). Nuclear and cytoplasmic-AR localization was determined, and the prognostic importance of AR assays was assessed. Comparisons and correlations with the mitotic activity index (MAI), estrogen receptor (ERalpha), progesterone receptor (PR), HER-2, and histological grade (WHO I-III) were made.

RESULTS

Nuclear-AR in WS, but not charc-AR, strongly correlated with MAI (P = 0.001). However, prognostic information appeared in univariate survival analyses only. Nuclear-AR in TMA was not prognostic. Charc-AR was independent prognostic in node positives both for relapse free survival (RFS) and breast cancer specific survival (BCSS). Both charc-AR and IHC cytoplasmic-AR provided independent prognostic survival information for BCSS in women <55 years.

CONCLUSION

Methods that can detect AR localized in the cytoplasm yield important prognostic information, and further studies in patients with operable breast cancer are warranted.

Looking back at prospective studies

Gur's perspective article raises important points about analytic methods and the clinical inferences drawn from retrospective statistical analyses of prospective studies. Specifically, he associates three problems with the scientific methods of retrospective analyses: (1) using the parametric receiver-operating characteristic (ROC) curve and the area under the ROC curve (AUC) as a performance measure, (2) using Bonferroni adjustments to account for multiple comparisons, and (3) failing to evaluate the variability of results across sites and observers. Gur demonstrates these problems with a case study: a recent paper analyzing the Digital Mammographic Imaging Screening Trial (DMIST) (1). The issues he raises are not specific to either retrospective study designs or secondary exploratory analyses of large studies but are important issues to consider in many design settings. I address each of these issues in the following and relate them to the information provided by DMIST papers.

Multiplex analysis of cytokines as biomarkers that differentiate benign and malignant thyroid diseases

Thyroid cancer incidence is increasing, and its diagnosis can be challenging. Fine needle biopsy, the principal clinical tool to make a tissue diagnosis, leads to inconclusive diagnoses in up to 30% of the cases, leading to surgery. Advances in proteomics are improving abilities to diagnose malignant conditions using small samples of tissue or body fluids. We hypothesized that analysis of serum growth factors would uncover diagnostically informative differences between benign and malignant thyroid conditions. Using xMAP profiling, we evaluated concentrations of 19 cytokines, chemokines, and growth factors. We used sera from 23 patients with cancer (Malignant group), 24 patients with benign nodular thyroid disease (Benign group), and 23 healthy subjects (Normal group). In univariate analysis, five factors (epithelial growth factor, hepatocyte growth factor, Interleukins-5 and -8, and regulated upon activation, normally T-expressed and presumably secreted (RANTES) distinguished subjects with thyroid disease from the Normal group. In multivariate analysis, the set {Interleukin-8, hepatocyte growth factor, monocyte-induced γ interferon, interleukin-12 p40} achieved noteworthy discrimination between Benign and Malignant groups (area under the receiver operating characteristics curve was 0.81 (95% confidence interval: 0.65-0.90)). Multiplex panels of serum biomarkers may be promising tools to diagnose cancer in patients presenting with evidence of nodular thyroid disease.

Not proper ROC curves as new tool for the analysis of differentially expressed genes in microarray experiments

Most microarray experiments are carried out with the purpose of identifying genes whose expression varies in relation with specific conditions or in response to environmental stimuli. In such studies, genes showing similar mean expression values between two or more groups are considered as not differentially expressed, even if hidden subclasses with different expression values may exist. In this paper we propose a new method for identifying differentially expressed genes, based on the area between the ROC curve and the rising diagonal (ABCR). ABCR represents a more general approach than the standard area under the ROC curve (AUC), because it can identify both proper (i.e., concave) and not proper ROC curves (NPRC). In particular, NPRC may correspond to those genes that tend to escape standard selection methods.

RESULTS

We assessed the performance of our method using data from a publicly available database of 4026 genes, including 14 normal B cell samples (NBC) and 20 heterogeneous lymphomas (namely: 9 follicular lymphomas and 11 chronic lymphocytic leukemias). Moreover, NBC also included two sub-classes, i.e., 6 heavily stimulated and 8 slightly or not stimulated samples. We identified 1607 differentially expressed genes with an estimated False Discovery Rate of 15%. Among them, 16 corresponded to NPRC and all escaped standard selection procedures based on AUC and t statistics. Moreover, a simple inspection to the shape of such plots allowed to identify the two subclasses in either one class in 13 cases (81%).

CONCLUSION

NPRC represent a new useful tool for the analysis of microarray data.

Biomarker evaluation and comparison using the controls as a reference population

The classification accuracy of a continuous marker is typically evaluated with the receiver operating characteristic (ROC) curve. In this paper, we study an alternative conceptual framework, the "percentile value." In this framework, the controls only provide a reference distribution to standardize the marker. The analysis proceeds by analyzing the standardized marker in cases. The approach is shown to be equivalent to ROC analysis. Advantages are that it provides a framework familiar to a broad spectrum of biostatisticians and it opens up avenues for new statistical techniques in biomarker evaluation. We develop several new procedures based on this framework for comparing biomarkers and biomarker performance in different populations. We develop methods that adjust such comparisons for covariates. The methods are illustrated on data from 2 cancer biomarker studies.

The leaf ionome as a multivariable system to detect a plant's physiological status

The contention that quantitative profiles of biomolecules contain information about the physiological state of the organism has motivated a variety of high-throughput molecular profiling experiments. However, unbiased discovery and validation of biomolecular signatures from these experiments remains a challenge. Here we show that the Arabidopsis thaliana (Arabidopsis) leaf ionome, or elemental composition, contains such signatures, and we establish statistical models that connect these multivariable signatures to defined physiological responses, such as iron (Fe) and phosphorus (P) homeostasis. Iron is essential for plant growth and development, but potentially toxic at elevated levels. Because of this, shoot Fe concentrations are tightly regulated and show little variation over a range of Fe concentrations in the environment, making them a poor probe of a plant's Fe status. By evaluating the shoot ionome in plants grown under different Fe nutritional conditions, we have established a multivariable ionomic signature for the Fe response status of Arabidopsis. This signature has been validated against known Fe-response proteins and allows the high-throughput detection of the Fe status of plants with a false negative/positive rate of 18%/16%. A "metascreen" of previously collected ionomic data from 880 Arabidopsis mutants and natural accessions for this Fe response signature successfully identified the known Fe mutants frd1 and frd3. A similar approach has also been taken to identify and use a shoot ionomic signature associated with P homeostasis. This study establishes that multivariable ionomic signatures of physiological states associated with mineral nutrient homeostasis do exist in Arabidopsis and are in principle robust enough to detect specific physiological responses to environmental or genetic perturbations.

Metachronous cancer development in patients with sporadic colorectal adenomas-multivariate risk model with independent and combined value of hTERT and survivin

BACKGROUND AND AIMS

Accurate, long-term risk predictors for colorectal cancer development in patients with sporadic adenomas are lacking. We sought to validate biomarkers predictive of metachronous colorectal cancer (mCRC) in patients with sporadic colorectal adenomas, using 374 consecutive patients from a large defined population.

MATERIALS AND METHODS

Risk evaluation was performed for patient and adenoma risk factors (morphometric longest nuclear axis and immunohistochemical markers survivin, human telomerase reverse transcriptase (hTERT), beta-catenin, p16INK4a, p21CIP1, and cyclin D1). Diagnostic accuracy was assessed by receiver-operating characteristics curve analysis, and uni- and multivariate survival analysis was performed.

RESULTS/FINDINGS

Of the 374 patients, 26 (7%) developed mCRC with a median of 5.6 years (range 2-19) from index adenoma. Independent risk factors included age greater than or equal to 60 years, proximal location, multiplicity (greater than or equal to three adenomas), and high-grade neoplasia, with high-grade intraepithelial neoplasia and proximal location as the strongest on multivariate analysis (hazard ratio [HR] of 4.1 and 5.2, respectively; both p< 0.05). The molecular markers hTERT (HR 11.3, 95% confidence interval [CI] 3.9-33.1; p < 0.001) and survivin (HR 7.0, 95% CI 2.4-20.5; p < 0.001) were independent predictors for mCRC, and proximal location (4 of 16 = 25% with mCRC) was the only clinical one. The value of hTERT and survivin were retained in the validation set. Survivin and hTERT together yielded high mCRC risk when both were positive (15 of 51 = 29%; HR 14.3, 5.6-36.5), modest with one positive (survivin 4 of 90 = 4.4%; hTERT 4 of 60 = 6.7%), and no risk with both negative (0 of 144 = 0%).

INTERPRETATION/CONCLUSION

hTERT and survivin are the best risk predictors for long-term, mCRC development in patients with sporadic colorectal adenomas.

Evaluating the ROC performance of markers for future events

Receiver operating characteristic (ROC) curves play a central role in the evaluation of biomarkers and tests for disease diagnosis. Predictors for event time outcomes can also be evaluated with ROC curves, but the time lag between marker measurement and event time must be acknowledged. We discuss different definitions of time-dependent ROC curves in the context of real applications. Several approaches have been proposed for estimation. We contrast retrospective versus prospective methods in regards to assumptions and flexibility, including their capacities to incorporate censored data, competing risks and different sampling schemes. Applications to two datasets are presented.

Serological immunoreactivity against colon cancer proteome varies upon disease progression

Sera from colon carcinoma patients were used to identify tumor-associated antigens (TAAs) by screening tumor proteome resolved by 2D electrophoresis. A panel of six TAAs eliciting a serological immune response in colorectal cancer was identified, showing a modification in antigen recognition by B cells in patients as a function of colon cancer progression. The expression of these proteins was either confined or increased in tumor as compared to normal mucosa.

Statistical evaluation of prognostic versus diagnostic models: beyond the ROC curve

BACKGROUND

Diagnostic and prognostic or predictive models serve different purposes. Whereas diagnostic models are usually used for classification, prognostic models incorporate the dimension of time, adding a stochastic element.

CONTENT

The ROC curve is typically used to evaluate clinical utility for both diagnostic and prognostic models. This curve assesses how well a test or model discriminates, or separates individuals into two classes, such as diseased and nondiseased. A strong risk predictor, such as lipids for cardiovascular disease, may have limited impact on the area under the curve, called the AUC or c-statistic, even if it alters predicted values. Calibration, measuring whether predicted probabilities agree with observed proportions, is another component of model accuracy important to assess. Reclassification can directly compare the clinical impact of two models by determining how many individuals would be reclassified into clinically relevant risk strata. For example, adding high-sensitivity C-reactive protein and family history to prediction models for cardiovascular disease using traditional risk factors moves approximately 30% of those at intermediate risk levels, such as 5%-10% or 10%-20% 10-year risk, into higher or lower risk categories, despite little change in the c-statistic. A calibration statistic can asses how well the new predicted values agree with those observed in the cross-classified data.

SUMMARY

Although it is useful for classification, evaluation of prognostic models should not rely solely on the ROC curve, but should assess both discrimination and calibration. Risk reclassification can aid in comparing the clinical impact of two models on risk for the individual, as well as the population.

Matching in studies of classification accuracy: implications for analysis, efficiency, and assessment of incremental value

In case-control studies evaluating the classification accuracy of a marker, controls are often matched to cases with respect to factors associated with the marker and disease status. In contrast with matching in epidemiologic etiology studies, matching in the classification setting has not been rigorously studied. In this article, we consider the implications of matching in terms of the choice of statistical analysis, efficiency, and assessment of the incremental value of the marker over the matching covariates. We find that adjustment for the matching covariates is essential, as unadjusted summaries of classification accuracy can be biased. In many settings, matching is the most efficient covariate-dependent sampling scheme, and we provide an expression for the optimal matching ratio. However, we also show that matching greatly complicates estimation of the incremental value of the marker. We recommend that matching be carefully considered in the context of these findings.

DBDigger: reorganized proteomic database identification that improves flexibility and speed

Database search identification algorithms, such as Sequest and Mascot, constitute powerful enablers for proteomic tandem mass spectrometry. We introduce DBDigger, an algorithm that reorganizes the database identification process to remove a problematic bottleneck. Typically such algorithms determine which candidate sequences can be compared to each spectrum. Instead, DBDigger determines which spectra can be compared to each candidate sequence, enabling the software to generate candidate sequences only once for each HPLC separation rather than for each spectrum. This reorganization also reduces the number of times a spectrum must be predicted for a particular candidate sequence and charge state. As a result, DBDigger can accelerate some database searches by more than an order of magnitude. In addition, the software offers features to reduce the performance degradation introduced by posttranslational modification (PTM) searching. DBDigger allows researchers to specify the sequence context in which each PTM is possible. In the case of CNBr digests, for example, modified methionine residues can be limited to occur only at the C-termini of peptides. Use of "context-dependent" PTM searching reduces the performance penalty relative to traditional PTM searching. We characterize the performance possible with DBDigger, showcasing MASPIC, a new statistical scorer. We describe the implementation of these innovations in the hope that other researchers will employ them for rapid and highly flexible proteomic database search.

Clinical proteomics: A need to define the field and to begin to set adequate standards

The aim of this manuscript is to initiate a constructive discussion about the definition of clinical proteomics, study requirements, pitfalls and (potential) use. Furthermore, we hope to stimulate proposals for the optimal use of future opportunities and seek unification of the approaches in clinical proteomic studies. We have outlined our collective views about the basic principles that should be considered in clinical proteomic studies, including sample selection, choice of technology and appropriate quality control, and the need for collaborative interdisciplinary efforts involving clinicians and scientists. Furthermore, we propose guidelines for the critical aspects that should be included in published reports. Our hope is that, as a result of stimulating discussion, a consensus will be reached amongst the scientific community leading to guidelines for the studies, similar to those already published for mass spectrometric sequencing data. We contend that clinical proteomics is not just a collection of studies dealing with analysis of clinical samples. Rather, the essence of clinical proteomics should be to address clinically relevant questions and to improve the state-of-the-art, both in diagnosis and in therapy of diseases.

Relationships between the density of different indicator organisms on sheep and beef carcasses and in frozen beef and sheep meat

AIM

To describe the relationship between the concentration of different indicator bacteria in red meat.

METHODS AND RESULTS

Enumeration data for aerobic plate count (APC), Enterobacteriaceae, coliforms and Escherichia coli biotype I were analysed from an Australia-wide survey of beef carcasses, sheep carcasses, frozen beef and frozen sheep meat. In all commodities, there was only low-to-moderate rank correlation (0.16-0.47) between concentration of APC and concentration of each Gram-negative indicator. Rank correlations between counts of Gram-negative indicators were much higher (0.47-0.92) especially when nondetections were excluded from analysis (0.78-0.94). Receiver-operator characteristics analysis showed that detection of coliforms can predict the presence of E. coli biotype I with almost 100% sensitivity but fails to predict absence in 2.7-8.5% of samples not containing E. coli biotype I.

CONCLUSIONS

Enumeration of coliforms is a useful adjunct to enumeration of E. coli biotype I or Enterobacteriaceae in red meat. The density of coliforms or Enterobacteriaceae can be used to predict the presence or absence of E. coli biotype I, although when the latter is at low prevalence errors in positive test prediction can be large.

SIGNIFICANCE AND IMPACT OF THE STUDY

A quantitative basis is provided for comparing the concentration of different indicator bacteria measured in the production, regulation and trade of red meat.

Systems biology, metabolic modelling and metabolomics in drug discovery and development

Unlike signalling pathways, metabolic networks are subject to strict stoichiometric constraints. Metabolomics amplifies changes in the proteome, and represents more closely the phenotype of an organism. Recent advances enable the production (and computer-readable encoding as SBML) of metabolic network models reconstructed from genome sequences, as well as experimental measurements of much of the metabolome. There is increasing convergence between the number of human metabolites estimated via genomics ( approximately 3000) and the number measured experimentally. It is thus both timely, and now possible, to bring these two approaches together as an integrated (if distributed) whole to help understand the genesis of metabolic biomarkers, the progress of disease, and the modes of action, efficacy, off-target effects and toxicity of pharmaceutical drugs.

Incorporating monotonicity into the evaluation of a biomarker

In the assessment of clinical utility of biomarkers, case-control studies are often undertaken based on existing serum samples. A common assumption made in these studies is that higher levels of the biomarker are associated with increased disease risk. In this article, we consider methods of analysis in which monotonicity is incorporated in associating the biomarker and the clinical outcome. We consider the roles of discrimination versus association and assess methods for both goals. In addition, we propose a semiparametric isotonic regression model for binary data and describe a simple estimation procedure as well as attendant inferential procedures. We apply the various methodologies to data from a prostate cancer study involving a serum biomarker.

Pharmacogenomic predictor of sensitivity to preoperative chemotherapy with paclitaxel and fluorouracil, doxorubicin, and cyclophosphamide in breast cancer

PURPOSE

We developed a multigene predictor of pathologic complete response (pCR) to preoperative weekly paclitaxel and fluorouracil-doxorubicin-cyclophosphamide (T/FAC) chemotherapy and assessed its predictive accuracy on independent cases.

PATIENTS AND METHODS

One hundred thirty-three patients with stage I-III breast cancer were included. Pretreatment gene expression profiling was performed with oligonecleotide microarrays on fine-needle aspiration specimens. We developed predictors of pCR from 82 cases and assessed accuracy on 51 independent cases.

RESULTS

Overall pCR rate was 26% in both cohorts. In the training set, 56 probes were identified as differentially expressed between pCR versus residual disease, at a false discovery rate of 1%. We examined the performance of 780 distinct classifiers (set of genes + prediction algorithm) in full cross-validation. Many predictors performed equally well. A nominally best 30-probe set Diagonal Linear Discriminant Analysis classifier was selected for independent validation. It showed significantly higher sensitivity (92% v 61%) than a clinical predictor including age, grade, and estrogen receptor status. The negative predictive value (96% v 86%) and area under the curve (0.877 v 0.811) were nominally better but not statistically significant. The combination of genomic and clinical information yielded a predictor not significantly different from the genomic predictor alone. In 31 samples, RNA was hybridized in replicate with resulting predictions that were 97% concordant.

CONCLUSION

A 30-probe set pharmacogenomic predictor predicted pCR to T/FAC chemotherapy with high sensitivity and negative predictive value. This test correctly identified all but one of the patients who achieved pCR (12 of 13 patients) and all but one of those who were predicted to have residual disease had residual cancer (27 of 28 patients).

Combining predictors for classification using the area under the receiver operating characteristic curve

No single biomarker for cancer is considered adequately sensitive and specific for cancer screening. It is expected that the results of multiple markers will need to be combined in order to yield adequately accurate classification. Typically, the objective function that is optimized for combining markers is the likelihood function. In this article, we consider an alternative objective function-the area under the empirical receiver operating characteristic curve (AUC). We note that it yields consistent estimates of parameters in a generalized linear model for the risk score but does not require specifying the link function. Like logistic regression, it yields consistent estimation with case-control or cohort data. Simulation studies suggest that AUC-based classification scores have performance comparable with logistic likelihood-based scores when the logistic regression model holds. Analysis of data from a proteomics biomarker study shows that performance can be far superior to logistic regression derived scores when the logistic regression model does not hold. Model fitting by maximizing the AUC rather than the likelihood should be considered when the goal is to derive a marker combination score for classification or prediction.

Evaluating markers for the early detection of cancer: overview of study designs and methods

BACKGROUND

The field of cancer biomarker development has been evolving rapidly. New developments both in the biologic and statistical realms are providing increasing opportunities for evaluation of markers for both early detection and diagnosis of cancer.

PURPOSE

To review the major conceptual and methodological issues in cancer biomarker evaluation, with an emphasis on recent developments in statistical methods together with practical recommendations.

METHODS

We organized this review by type of study: preliminary performance, retrospective performance, prospective performance and cancer screening evaluation.

RESULTS

For each type of study, we discuss methodologic issues, provide examples and discuss strengths and limitations.

CONCLUSION

Preliminary performance studies are useful for quickly winnowing down the number of candidate markers; however their results may not apply to the ultimate target population, asymptomatic subjects. If stored specimens from cohort studies with clinical cancer endpoints are available, retrospective studies provide a quick and valid way to evaluate performance of the markers or changes in the markers prior to the onset of clinical symptoms. Prospective studies have a restricted role because they require large sample sizes, and, if the endpoint is cancer on biopsy, there may be bias due to overdiagnosis. Cancer screening studies require very large sample sizes and long follow-up, but are necessary for evaluating the marker as a trigger of early intervention.

Joint National Cancer Institute-Food and Drug Administration Workshop on Research Strategies, Study Designs, and Statistical Approaches to Biomarker Validation for Cancer Diagnosis and Detection

Cancer remains the second leading cause of death in the United States, in spite of tremendous advances made in therapeutic and diagnostic strategies. Successful cancer treatment depends on improved methods to detect cancers at early stages when they can be treated more effectively. Biomarkers for early detection of cancer enable screening of asymptomatic populations and thus play a critical role in cancer diagnosis. However, the approaches for validating biomarkers have yet to be addressed clearly. In an effort to delineate the ambiguities related to biomarker validation and related statistical considerations, the National Cancer Institute, in collaboration with the Food and Drug Administration, conducted a workshop in July 2004 entitled "Research Strategies, Study Designs, and Statistical Approaches to Biomarker Validation for Cancer Diagnosis and Detection." The main objective of this workshop was to review basic considerations underpinning the study designs, statistical methodologies, and novel approaches necessary to rapidly advance the clinical application of cancer biomarkers. The current commentary describes various aspects of statistical considerations and study designs for cancer biomarker validation discussed in this workshop.