Proteomics research to discover markers: what can we learn from Netflix?

BACKGROUND

Research in the field of proteomics to discover markers for detection of cancer has produced disappointing results, with few markers gaining US Food and Drug Administration approval, and few claims borne out when subsequently tested in rigorous studies. What is the role of better mathematical or statistical analysis in improving the situation?

CONTENT

This article examines whether a recent successful Netflix-sponsored competition using mathematical analysis to develop a prediction model for movie ratings of individual subscribers can serve to improve studies of markers in the field of proteomics. Netflix developed a database of movie preferences of individual subscribers using a longitudinal cohort research design. Groups of researchers then competed to develop better ways to analyze the data. Against this background, the strengths and weaknesses of research design are reviewed, contrasting the Netflix design with that of studies of biomarkers to detect cancer. Such biomarker studies generally have less-strong design, lower numbers of outcomes, and greater difficulty in even just measuring predictors and outcomes, so the fundamental data that will be used in mathematical analysis tend to be much weaker than in other kinds of research.

CONCLUSIONS

If the fundamental data that will be analyzed are not strong, then better analytic methods have limited use in improving the situation. Recognition of this situation is an important first step toward improving the quality of clinical research about markers to detect cancer.

Leveraging biospecimen resources for discovery or validation of markers for early cancer detection

Validation of early detection cancer biomarkers has proven to be disappointing when initial promising claims have often not been reproducible in diagnostic samples or did not extend to prediagnostic samples. The previously reported lack of rigorous internal validity (systematic differences between compared groups) and external validity (lack of generalizability beyond compared groups) may be effectively addressed by utilizing blood specimens and data collected within well-conducted cohort studies. Cohort studies with prediagnostic specimens (eg, blood specimens collected prior to development of clinical symptoms) and clinical data have recently been used to assess the validity of some early detection biomarkers. With this background, the Division of Cancer Control and Population Sciences (DCCPS) and the Division of Cancer Prevention (DCP) of the National Cancer Institute (NCI) held a joint workshop in August 2013. The goal was to advance early detection cancer research by considering how the infrastructure of cohort studies that already exist or are being developed might be leveraged to include appropriate blood specimens, including prediagnostic specimens, ideally collected at periodic intervals, along with clinical data about symptom status and cancer diagnosis. Three overarching recommendations emerged from the discussions: 1) facilitate sharing of existing specimens and data, 2) encourage collaboration among scientists developing biomarkers and those conducting observational cohort studies or managing healthcare systems with cohorts followed over time, and 3) conduct pilot projects that identify and address key logistic and feasibility issues regarding how appropriate specimens and clinical data might be collected at reasonable effort and cost within existing or future cohorts.

Effectiveness and economic impact of screening for colorectal cancer by mass fecal occult blood testing

OBJECTIVES

Fecal occult blood testing has been shown to reduce mortality from colorectal cancer in large randomized, controlled trials conducted in the United States, Denmark, and the United Kingdom, and mathematical simulation modeling found it to be cost-effective relative to other health care services. Before making a concerted effort to implement mass fecal occult blood testing based on this evidence alone, however, we considered it prudent to critically re-evaluate the effectiveness and economic impact of screening in the US population as a whole.

METHODS

To assess the effectiveness of screening, we projected published outcomes from each of the three large randomized controlled trials of fecal occult blood testing to the US population, as if each clinical trial had been done in the population as a whole. We then determined the resource costs of detection and treatment that would be associated with the outcomes predicted from each trial.

RESULTS

More than 1 million colorectal cancers could be expected to arise over 10 yr in the cohort of US residents eligible to enter a screening program in 1997, and trial outcomes indicate that > or = 60% of these cancers would be fatal. If the 60-67% compliance rate of the population-based randomized controlled trials were achieved, a fecal occult blood testing program would detect 30% of known colorectal cancers and save 100,000 lives over 10 yr. Screening would incur total costs of $3-4 billion over 10 yr, or $2,500 per life-year saved.

CONCLUSIONS

Mass fecal occult blood testing is cost-effective, and, although not inexpensive, many would consider the total cost acceptable. Even with a concerted effort to achieve compliance, however, the effectiveness of fecal occult blood testing would be limited to saving the lives of < or = 15% of those who otherwise would die from their cancer in the first 10 yr after beginning mass screening. The limitations of fecal occult blood testing suggest the need to further evaluate the role of endoscopy in screening, and to develop more effective, noninvasive screening tools.

An efficient strategy for evaluating new non-invasive screening tests for colorectal cancer: the guiding principles

OBJECTIVE

New screening tests for colorectal cancer (CRC) are rapidly emerging. Conducting trials with mortality reduction as the end point supporting their adoption is challenging. We re-examined the principles underlying evaluation of new non-invasive tests in view of technological developments and identification of new biomarkers.

DESIGN

A formal consensus approach involving a multidisciplinary expert panel revised eight previously established principles.

RESULTS

Twelve newly stated principles emerged. Effectiveness of a new test can be evaluated by comparison with a proven comparator non-invasive test. The faecal immunochemical test is now considered the appropriate comparator, while colonoscopy remains the diagnostic standard. For a new test to be able to meet differing screening goals and regulatory requirements, flexibility to adjust its positivity threshold is desirable. A rigorous and efficient four-phased approach is proposed, commencing with small studies assessing the test's ability to discriminate between CRC and non-cancer states (phase I), followed by prospective estimation of accuracy across the continuum of neoplastic lesions in neoplasia-enriched populations (phase II). If these show promise, a provisional test positivity threshold is set before evaluation in typical screening populations. Phase III prospective studies determine single round intention-to-screen programme outcomes and confirm the test positivity threshold. Phase IV studies involve evaluation over repeated screening rounds with monitoring for missed lesions. Phases III and IV findings will provide the real-world data required to model test impact on CRC mortality and incidence.

CONCLUSION

New non-invasive tests can be efficiently evaluated by a rigorous phased comparative approach, generating data from unbiased populations that inform predictions of their health impact.

To biopsy or not to biopsy in suspected herpes simplex encephalitis: a quantitative analysis

The authors performed a decision analysis to determine whether a patient suspected to have herpes simplex encephalitis (HSE) should undergo a brain biopsy or be treated empirically with medical therapy. In most cases, empiric treatment with acyclovir would be slightly favored; brain biopsy was not essential in management. However, brain biopsy was found useful for patients who had low CSF glucose at the time of initial lumbar puncture; such patients may have a very high risk to have other treatable conditions such as tuberculosis, brain abscess, toxoplasmosis, or cryptococcosis. The results of the analysis suggest that even with the advent of safe antiviral drug therapy such as acyclovir, brain biopsy is useful in a well-defined subset of patients with possible HSE. The rationale, however, is not to confirm HSE but rather to detect other treatable conditions.

Assessment of serum proteomics to detect large colon adenomas

A noninvasive blood test that could reliably detect early colorectal cancer or large adenomas would provide an important advance in colon cancer screening. The purpose of this study was to determine whether a serum proteomics assay could discriminate between persons with and without a large (> or =1 cm) colon adenoma. To avoid problems of "bias" that have affected many studies about molecular markers for diagnosis, specimens were obtained from a previously conducted study of colorectal cancer etiology in which bloods had been collected before the presence or absence of neoplasm had been determined by colonoscopy, helping to assure that biases related to differences in sample collection and handling would be avoided. Mass spectra of 65 unblinded serum samples were acquired using a nanoelectrospray ionization source on a QSTAR-XL mass spectrometer. Classification patterns were developed using the ProteomeQuest algorithm, performing measurements twice on each specimen, and then applied to a blinded validation set of 70 specimens. After removing 33 specimens that had discordant results, the "test group" comprised 37 specimens that had never been used in training. Although in the primary analysis, no discrimination was found, a single post hoc analysis, done after hemolyzed specimens had been removed, showed a sensitivity of 78%, a specificity of 53%, and an accuracy of 63% (95% confidence interval, 53-72%). The results of this study, although preliminary, suggest that further study of serum proteomics, in a larger number of appropriate specimens, could be useful. They also highlight the importance of understanding sources of "noise" and "bias" in studies of proteomics assays.

Promises and limitations of biomarkers

In many "omics" fields, extraordinary promises have been made about the ability of biomarkers for detecting early cancer, for predicting prognosis, and for predicting response to therapy. Yet very few markers are brought to clinical practice, and many are not even found to be reproducible. This essay discusses chance and bias as threats to validity that can explain the huge disconnect between promise and product, along with approaches to address those threats.

The practices and attitudes of primary care nurse practitioners and physician assistants with respect to colorectal cancer screening

OBJECTIVES

Although nurse practitioners and physician assistants form a large and growing portion of the primary care workforce, little is known about their colorectal cancer screening practices. The aim of this study was to assess the colorectal cancer screening practices, training, and attitudes of nurse practitioners and physician assistants practicing primary care medicine.

METHODS

All nurse practitioners (827) and physician assistants (1178) licensed by the Medical Board of the State of North Carolina were surveyed by mail. Both groups were further divided into primary care versus non-primary care by self-described roles. Self-reported practices, training, and attitudes with respect to colorectal cancer screening were elicited.

RESULTS

Response rates were 71.4% and 61.2%, for nurse practitioners and physician assistants respectively. A total of 51.3% of nurse practitioners and 50.3% of physician assistants described themselves as adult primary care providers. No primary care nurse practitioners and only 3.8% of primary care physician assistants performed screening flexible sigmoidoscopy. However, 76% of primary care physician assistants and 69% of primary care nurse practitioners reported recommending screening flexible sigmoidoscopy. A total of 95% primary care physician assistants and 92% of primary care nurse practitioners reported performing fecal occult blood testing. Only 9.4% of physician assistants and 2.8% of nurse practitioners received any formal instruction in flexible sigmoidoscopy while in their training. Additionally, 41.4% of primary care physician assistants and 27.7% of primary care nurse practitioners reported that they would be interested in obtaining formal training in flexible sigmoidoscopy.

CONCLUSIONS

Physician assistants and nurse practitioners are motivated, willing and underutilized groups with respect to CRC screening. Efforts to increase education and training of these professionals may improve the availability of CRC screening modalities.

Screening colonoscopy in balance. Issues of implementation

Although colonoscopy is currently considered an option for colon cancer screening, its choice as a preferred screening test is not supported by data, recommendations, or cost-effectiveness analysis. The use of colonoscopy in postpolypectomy surveillance constitutes a huge potential effort that should be appropriately anticipated and managed. Noninvasive methods for colon cancer screening may reduce the need for colonoscopy in the future.

Achieving quality in flexible sigmoidoscopy screening for colorectal cancer

BACKGROUND

Sigmoidoscopy screening, which can dramatically reduce colorectal cancer mortality, is supported increasingly by physicians and payers, and is likely to be performed more frequently in the future. As more physicians and nonphysician medical personnel learn how to perform this procedure, and with attention to quality standards, the overall impact of sigmoidoscopy screening may improve. This review describes elements that characterize high-quality examinations and identifies resources for in-depth information on performing flexible sigmoidoscopy.

METHODS

The domains of quality were identified from textbooks, articles, and the professional opinions of gastroenterologists and primary care physicians. Information was obtained from MEDLINE, bibliographies in recent articles, medical professional organizations, equipment manufacturers' representatives, and focus groups of primary care physicians.

RESULTS

Nine domains of quality are identified and discussed: training, logistical start-up, patient interaction, bowel preparation, examination technique, lesion recognition, complications, reporting, and processing (equipment cleaning and disinfection).

CONCLUSIONS

Persons learning how to perform and to implement flexible sigmoidoscopy may use this information to help ensure the quality of screening examinations.

Is Helicobacter pylori a cause of duodenal ulcer? A methodologic critique of current evidence

INTRODUCTION

Helicobacter pylori (HP) in the gastric antrum has been strongly associated with both duodenal ulcer (DU) and chronic active gastritis (CAG). The relationship between HP and DU has been interpreted as causal by many observers. An alternate hypothesis is that HP coincidently colonizes CAG, which is independently associated with DU by some yet-unknown mechanism.

PURPOSE

To assess the extent to which a causal relationship between HP and DU has been demonstrated, we performed a methodologic critique of published clinical studies. We carried out a literature search to identify clinical studies that included at least 25 subjects. Of the eight studies we identified, six used a cross-sectional design and two used a prospective cohort design. We applied methodologic criteria to assess causation: strength of association, biologic gradient, temporality, and experiment.

METHODS

A strong association between HP and DU was demonstrated in all eight studies. Biologic gradient and temporality were not assessed in any study. In the two experimental studies of therapy, loss of antral HP was associated with a decreased rate of DU relapse; however, we did not interpret this as sufficient to support causality because the effect may have been due to a direct mucosal action rather than eradication of HP.

CONCLUSION

We conclude that published evidence does not establish HP as a cause of DU. One approach to address causality would be an observational cohort study of ulcer relapse to assess the temporal relationships between HP, CAG, and DU.

What can we conclude from the randomized controlled trials of fecal occult blood test screening?

The results of three published randomized controlled trials of fecal occult blood testing (FOBT) provide unequivocal proof of the principle that screening reduces mortality from colorectal cancer (CRC). However, several interesting questions remain in interpreting and applying the results of the clinical trials, including: how well does FOBT screening work (i.e. how much can CRC mortality be reduced), how does it work, when is it worthwhile and worthwhile doing, and how can technique be optimized? The answers to these questions have important practical and clinical implications.

Understanding differences in the guidelines for colorectal cancer screening

In this issue of Gastroenterology, the editors present 2 expert perspectives on the colorectal screening guidelines published in the May 2008 issue of the journal.

Cultivating cohort studies for observational translational research

BACKGROUND

"Discovery" research about molecular markers for diagnosis, prognosis, or prediction of response to therapy has frequently produced results that were not reproducible in subsequent studies. What are the reasons, and can observational cohorts be cultivated to provide strong and reliable answers to those questions? Experimental

METHODS

Selected examples are used to illustrate: (i) what features of research design provide strength and reliability in observational studies about markers of diagnosis, prognosis, and response to therapy? (ii) How can those design features be cultivated in existing observational cohorts, for example, within randomized controlled clinical trial (RCT), other existing observational research studies, or practice settings like health maintenance organization (HMOs)?

RESULTS

Examples include a study of RNA expression profiles of tumor tissue to predict prognosis of breast cancer, a study of serum proteomics profiles to diagnose ovarian cancer, and a study of stool-based DNA assays to screen for colon cancer. Strengths and weaknesses of observational study design features are discussed, along with lessons about how features that help assure strength might be "cultivated" in the future.

CONCLUSIONS AND IMPACT

By considering these examples and others, it may be possible to develop a process of "cultivating cohorts" in ongoing RCTs, observational cohort studies, and practice settings like HMOs that have strong features of study design. Such an effort could produce sources of data and specimens to reliably answer questions about the use of molecular markers in diagnosis, prognosis, and response to therapy.

What is the role of iFOBT in screening for colorectal cancer?

Policy makers will need to consider if it has one, not only as an adjunct to gFOBT screening, but also as a primary screening test