Prediction of pre-eclampsia-related complications in women with suspected or confirmed pre-eclampsia: development and internal validation of clinical prediction model

OBJECTIVE

A model that can predict reliably the risk of pre-eclampsia (PE)-related pregnancy complications does not exist. The aim of this study was to develop and validate internally a clinical prediction model to predict the risk of a composite outcome of PE-related maternal and fetal complications within 7, 14 and 30 days of testing in women with suspected or confirmed PE.

METHODS

The data for this study were derived from a prospective, multicenter, observational cohort study on women with a singleton pregnancy and suspected or confirmed PE at 20 to < 37 weeks' gestation. For the development of the prediction model, the possible contribution of clinical and standard laboratory variables, as well as the biomarkers soluble fms-like tyrosine kinase-1 (sFlt-1), placental growth factor (PlGF) and their ratio, in the prediction of a composite outcome of PE-related complications, consisting of maternal and fetal adverse events within 7, 14 and 30 days, was explored using multivariable competing-risks regression analysis. The discriminative ability of the model was assessed using the concordance (c-) statistic. A bootstrap validation procedure with 500 replications was used to correct the estimate of the prediction model performance for optimism and to compute a shrinkage factor for the regression coefficients to correct for overfitting.

RESULTS

Among 384 women with suspected or confirmed PE, 96 (25%) had an adverse PE-related outcome at any time after hospital admission. Important predictors of adverse PE-related outcome included sFlt-1/PlGF ratio, gestational age at the time of biomarker measurement and protein-to-creatinine ratio as continuous variables. The c-statistics (corrected for optimism) for developing a PE-related complication within 7, 14 and 30 days were 0.89, 0.88 and 0.87, respectively. There was limited overfitting, as indicated by a shrinkage factor of 0.91.

CONCLUSIONS

We propose a simple clinical prediction model with good discriminative performance to predict PE-related complications. Determination of its usefulness in clinical practice awaits further investigation and external validation. © 2020 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of International Society of Ultrasound in Obstetrics and Gynecology.

Occurrence of metachronous basal cell carcinomas: a prognostic model

BACKGROUND

A third of patients with a first basal cell carcinoma (BCC) will develop subsequent (metachronous) BCCs.

OBJECTIVES

To study the prognostic effect of the number of previous BCC diagnosis dates a patient has experienced to derive a prediction model to assess the risk of metachronous BCCs that may inform individualized decision making on surveillance.

METHODS

We considered participants of north-western European ancestry from a prospective population-based cohort study (Rotterdam Study). After linkage with the Dutch Pathology Registry, 1077 patients with a first BCC were included. Candidate predictors for metachronous BCCs included patient, lifestyle and tumour characteristics. The prognostic model was developed with Fine and Gray regression analysis to account for competing risk of death. We used bootstrapping to correct for within-patient correlation and statistical optimism in predictive performance.

RESULTS

Second to fifth BCCs occurred in 293, 122, 58 and 36 patients, with median follow-up times of 3·0, 2·1, 1·7 and 1·8 years after the previous BCC, respectively. The risk of a new BCC was higher for patients with more metachronous BCCs. Having more than one BCC at diagnosis was another strong predictor of metachronous BCCs. Discriminative ability of the model was reasonable with an optimism-corrected c-index of 0·70 at 3 years.

CONCLUSIONS

The number of previous BCC diagnosis dates was a strong prognostic factor and should be considered when predicting the risk of metachronous BCCs. When the number of previous BCC diagnosis dates is combined with other readily available characteristics into a prognostic model, patients at high risk of a new BCC can be identified.

Improving prediction models with new markers: a comparison of updating strategies

Background

New markers hold the promise of improving risk prediction for individual patients. We aimed to compare the performance of different strategies to extend a previously developed prediction model with a new marker.

Methods

Our motivating example was the extension of a risk calculator for prostate cancer with a new marker that was available in a relatively small dataset. Performance of the strategies was also investigated in simulations. Development, marker and test sets with different sample sizes originating from the same underlying population were generated. A prediction model was fitted using logistic regression in the development set, extended using the marker set and validated in the test set. Extension strategies considered were re-estimating individual regression coefficients, updating of predictions using conditional likelihood ratios (LR) and imputation of marker values in the development set and subsequently fitting a model in the combined development and marker sets. Sample sizes considered for the development and marker set were 500 and 100, 500 and 500, and 100 and 500 patients. Discriminative ability of the extended models was quantified using the concordance statistic (c-statistic) and calibration was quantified using the calibration slope.

Results

All strategies led to extended models with increased discrimination (c-statistic increase from 0.75 to 0.80 in test sets). Strategies estimating a large number of parameters (re-estimation of all coefficients and updating using conditional LR) led to overfitting (calibration slope below 1). Parsimonious methods, limiting the number of coefficients to be re-estimated, or applying shrinkage after model revision, limited the amount of overfitting. Combining the development and marker set using imputation of missing marker values approach led to consistently good performing models in all scenarios. Similar results were observed in the motivating example.

Conclusion

When the sample with the new marker information is small, parsimonious methods are required to prevent overfitting of a new prediction model. Combining all data with imputation of missing marker values is an attractive option, even if a relatively large marker data set is available.

Electronic supplementary material

The online version of this article (doi:10.1186/s12874-016-0231-2) contains supplementary material, which is available to authorized users.

Does ignoring clustering in multicenter data influence the performance of prediction models? A simulation study

Clinical risk prediction models are increasingly being developed and validated on multicenter datasets. In this article, we present a comprehensive framework for the evaluation of the predictive performance of prediction models at the center level and the population level, considering population-averaged predictions, center-specific predictions, and predictions assuming an average random center effect. We demonstrated in a simulation study that calibration slopes do not only deviate from one because of over- or underfitting of patterns in the development dataset, but also as a result of the choice of the model (standard versus mixed effects logistic regression), the type of predictions (marginal versus conditional versus assuming an average random effect), and the level of model validation (center versus population). In particular, when data is heavily clustered (ICC 20%), center-specific predictions offer the best predictive performance at the population level and the center level. We recommend that models should reflect the data structure, while the level of model validation should reflect the research question.

A simulation study of sample size demonstrated the importance of the number of events per variable to develop prediction models in clustered data

OBJECTIVES

This study aims to investigate the influence of the amount of clustering [intraclass correlation (ICC) = 0%, 5%, or 20%], the number of events per variable (EPV) or candidate predictor (EPV = 5, 10, 20, or 50), and backward variable selection on the performance of prediction models.

STUDY DESIGN AND SETTING

Researchers frequently combine data from several centers to develop clinical prediction models. In our simulation study, we developed models from clustered training data using multilevel logistic regression and validated them in external data.

RESULTS

The amount of clustering was not meaningfully associated with the models' predictive performance. The median calibration slope of models built in samples with EPV = 5 and strong clustering (ICC = 20%) was 0.71. With EPV = 5 and ICC = 0%, it was 0.72. A higher EPV related to an increased performance: the calibration slope was 0.85 at EPV = 10 and ICC = 20% and 0.96 at EPV = 50 and ICC = 20%. Variable selection sometimes led to a substantial relative bias in the estimated predictor effects (up to 118% at EPV = 5), but this had little influence on the model's performance in our simulations.

CONCLUSION

We recommend at least 10 EPV to fit prediction models in clustered data using logistic regression. Up to 50 EPV may be needed when variable selection is performed.

Failure of introduction of food allergens after negative oral food challenge tests in children

One of the purposes to perform an oral food challenge (FC) test is to avoid unnecessary elimination of food allergens. In case of a negative FC test result, the food can be introduced. It is, however, unknown if patients act according to the outcome of the test. This study evaluates the rate of introduction of peanut, hazelnut, cow's milk or hen's egg allergens after a negative FC test. We investigated the introduction rate of children (0-18 years) with a negative FC test visiting the Department of Allergology, Erasmus Medical Centre Rotterdam from 2008 till 2013 and the factors that influence the rate of introduction. Patients were asked to complete a comprehensive questionnaire about their FC test. In total, 157 (38% girls, mean age during challenge 6.9 years) participated in the study. Of these FC tests, 104 (56%) were followed by a successful introduction, 30 (16%) by a partly introduction (traces or processed foods) and 52 (28%) by a failed introduction. Peanut and hazelnut showed a statistically significant lower successful introduction rate. Age, gender, symptoms during FC test, dietary advice and time period to introduction significantly influenced the rate of introduction. One fourth of the children with failure of introducing foods experienced symptoms during the introduction.

CONCLUSION

More than one quarter of all children with a negative FC test result did not introduce the food. The FC test in its current form does not achieve its objective for this group of children.

Impact of adding therapeutic recommendations to risk assessments from a prediction model for postoperative nausea and vomiting

BACKGROUND

In a large cluster-randomized trial on the impact of a prediction model, presenting the calculated risk of postoperative nausea and vomiting (PONV) on-screen (assistive approach) increased the administration of risk-dependent PONV prophylaxis by anaesthetists. This change in therapeutic decision-making did not improve the patient outcome; that is, the incidence of PONV. The present study aimed to quantify the effects of adding a specific therapeutic recommendation to the predicted risk (directive approach) on PONV prophylaxis decision-making and the incidence of PONV.

METHODS

A prospective before-after study was conducted in 1483 elective surgical inpatients. The before-period included care-as-usual and the after-period included the directive risk-based (intervention) strategy. Risk-dependent effects on the administered number of prophylactic antiemetics and incidence of PONV were analysed by mixed-effects regression analysis.

RESULTS

During the intervention period anaesthetists administered 0.5 [95% confidence intervals (CIs): 0.4-0.6] more antiemetics for patients identified as being at greater risk of PONV. This directive approach led to a reduction in PONV [odds ratio (OR): 0.60, 95% CI: 0.43-0.83], with an even greater reduction in PONV in high-risk patients (OR: 0.45, 95% CI: 0.28-0.72).

CONCLUSIONS

Anaesthetists administered more prophylactic antiemetics when a directive approach was used for risk-tailored intervention compared with care-as-usual. In contrast to the previously studied assistive approach, the increase in PONV prophylaxis now resulted in a lower PONV incidence, particularly in high-risk patients. When one aims for a truly 'PONV-free hospital', a more liberal use of prophylactic antiemetics must be accepted and lower-risk thresholds should be set for the actionable recommendations.

Prognostic value of age for chronic postoperative inguinal pain

PURPOSE

Chronic postoperative inguinal pain (CPIP) is considered the most common and serious long-term problem after inguinal hernia repair. Young age has been described as a risk factor for developing chronic pain after several surgical procedures. Our aim was to assess if age has prognostic value on CPIP.

METHODS

The database of a randomized trial; the LEVEL trial, 669 patients, TEP versus Lichtenstein, was used for analysis. Data on incidence and intensity of preoperative pain, postoperative pain and CPIP at 1 year were collected. The association of age with incidence and intensity of pain was assessed with regression analysis. Further, hernia type and surgical technique were studied in combination with age and CPIP as possible risk factors on CPIP over age alone.

RESULTS

Younger patients (18-40 years) presented more often with CPIP than middle-aged patients (40-60 years) and elderly (>60 years); 43 vs. 29 vs. 19 %; overall 27 %. Younger and middle-aged patients had more frequently preoperative pain; 54 vs. 55 vs. 41 % and intensity of pain was higher during the first three postoperative days (VAS on day 1: 5.5 vs. 4.5 vs. 3.9 and on day 3: 3.8 vs. 2.9 vs. 2.6). Indirect-type hernias were seen more often in younger patients (77 vs. 51 vs. 48 %) and were not related to CPIP or with surgical technique.

CONCLUSIONS

Almost one out of three patients experiences CPIP. The younger the patient, the higher the risk of CPIP. Hernia type and surgical technique did not influence CPIP.

Comparison of two models predicting IVF success; the effect of time trends on model performance

STUDY QUESTION

How well does the recently developed UK model predicting the success rate of IVF treatment (the 2011 Nelson model) perform in comparison with a UK model developed in the early 1990s (the Templeton model)?

SUMMARY ANSWER

Both models showed similar performance, after correction for the increasing success rate over time of IVF.

WHAT IS KNOWN ALREADY

For counselling couples undergoing IVF treatment it is of paramount importance to be able to predict success. Several prediction models for the chance of success after IVF treatment have been developed. So far, the Templeton model has been recommended as the best approach after having been validated in several independent patient data sets. The Nelson model, developed in 2011 and characterized by the largest development sample containing the most recently treated couples, may well perform better.

STUDY DESIGN, SIZE, DURATION

We tested both models in couples that were included in a national cohort study carried out in the Netherlands between the beginning of January 2002 and the end of December 2004.

PARTICIPANTS/MATERIALS, SETTING, METHODS

We analysed the IVF cycles of Dutch couples with primary infertility (n = 5176). The chance of success was calculated using the two UK models that had been developed using the information collected in the Human Fertilisation and Embryology Authority database. Women were treated in 1991-1994 (Templeton) or 2003-2007 (Nelson). The outcome of success for both UK models is the occurrence of a live birth after IVF but the outcome in the Dutch data is an ongoing pregnancy. In order to make the outcomes compatible, we used a factor to convert the chance of live birth to ongoing pregnancy and use the overall terms 'success or no success after IVF'. The discriminative ability and the calibration of both models were assessed, the latter before and after adjustment for time trends in IVF success rates.

MAIN RESULTS AND THE ROLE OF CHANCE

The two models showed a similarly limited degree of discriminative ability on the tested data (area under the receiver operating characteristic curve 0.597 for the Templeton model and 0.590 for the Nelson model). The Templeton model underestimated the success rate (observed 21% versus predicted 14%); the Nelson model overestimated the success rate (observed 21% versus predicted 29%). When the models were adjusted for the changing success rates over time, the calibration of both models considerably improved (Templeton observed 21% versus predicted 20%; Nelson observed 21% versus predicted 24%).

LIMITATIONS, REASONS FOR CAUTION

We could only test the models in couples with primary infertility because detailed information on secondary infertile couples was lacking in the Dutch data. This shortcoming may have negatively influenced the performance of the Nelson model.

WIDER IMPLICATIONS OF THE FINDINGS

The changes in success rates over time should be taken into account when assessing prediction models for estimating the success rate of IVF treatment. In patients with primary infertility, the choice to use the Templeton or Nelson model is arbitrary.

Internal validation of risk models in clustered data: a comparison of bootstrap schemes

Internal validity of a risk model can be studied efficiently with bootstrapping to assess possible optimism in model performance. Assumptions of the regular bootstrap are violated when the development data are clustered. We compared alternative resampling schemes in clustered data for the estimation of optimism in model performance. A simulation study was conducted to compare regular resampling on only the patient level with resampling on only the cluster level and with resampling sequentially on both the cluster and patient levels (2-step approach). Optimism for the concordance index and calibration slope was estimated. Resampling of only patients or only clusters showed accurate estimates of optimism in model performance. The 2-step approach overestimated the optimism in model performance. If the number of centers or intraclass correlation coefficient was high, resampling of clusters showed more accurate estimates than resampling of patients. The 3 bootstrap schemes also were applied to empirical data that were clustered. The results presented in this paper support the use of resampling on only the clusters for estimation of optimism in model performance when data are clustered.

Progression to microalbuminuria in type 1 diabetes: development and validation of a prediction rule

AIMS/HYPOTHESIS

Microalbuminuria is common in type 1 diabetes and is associated with an increased risk of renal and cardiovascular disease. We aimed to develop and validate a clinical prediction rule that estimates the absolute risk of microalbuminuria.

METHODS

Data from the European Diabetes Prospective Complications Study (n = 1115) were used to develop the prediction rule (development set). Multivariable logistic regression analysis was used to assess the association between potential predictors and progression to microalbuminuria within 7 years. The performance of the prediction rule was assessed with calibration and discrimination (concordance statistic [c-statistic]) measures. The rule was validated in three other diabetes studies (Pittsburgh Epidemiology of Diabetes Complications [EDC] study, Finnish Diabetic Nephropathy [FinnDiane] study and Coronary Artery Calcification in Type 1 Diabetes [CACTI] study).

RESULTS

Of patients in the development set, 13% were microalbuminuric after 7 years. Glycosylated haemoglobin, AER, WHR, BMI and ever smoking were found to be the most important predictors. A high-risk group (n = 87 [8%]) was identified with a risk of progression to microalbuminuria of 32%. Predictions showed reasonable discriminative ability, with c-statistic of 0.71. The rule showed good calibration and discrimination in EDC, FinnDiane and CACTI (c-statistic 0.71, 0.79 and 0.79, respectively).

CONCLUSIONS/INTERPRETATION

We developed and validated a clinical prediction rule that uses relatively easily obtainable patient characteristics to predict microalbuminuria in patients with type 1 diabetes. This rule can help clinicians to decide on more frequent check-ups for patients at high risk of microalbuminuria in order to prevent long-term chronic complications.

Validation, updating and impact of clinical prediction rules: a review

OBJECTIVE

To provide an overview of the research steps that need to follow the development of diagnostic or prognostic prediction rules. These steps include validity assessment, updating (if necessary), and impact assessment of clinical prediction rules.

STUDY DESIGN AND SETTING

Narrative review covering methodological and empirical prediction studies from primary and secondary care.

RESULTS

In general, three types of validation of previously developed prediction rules can be distinguished: temporal, geographical, and domain validations. In case of poor validation, the validation data can be used to update or adjust the previously developed prediction rule to the new circumstances. These update methods differ in extensiveness, with the easiest method a change in model intercept to the outcome occurrence at hand. Prediction rules -- with or without updating -- showing good performance in (various) validation studies may subsequently be subjected to an impact study, to demonstrate whether they change physicians' decisions, improve clinically relevant process parameters, patient outcome, or reduce costs. Finally, whether a prediction rule is implemented successfully in clinical practice depends on several potential barriers to the use of the rule.

CONCLUSION

The development of a diagnostic or prognostic prediction rule is just a first step. We reviewed important aspects of the subsequent steps in prediction research.

A new diagnostic rule for deep vein thrombosis: safety and efficiency in clinically relevant subgroups

BACKGROUND

Recently, a new, simple diagnostic rule was introduced to enable GPs to safely refrain from referring a considerable proportion of the patients suspected of having deep vein thrombosis (DVT). The rule (which includes seven patient history and physical examination items plus the result of a D-dimer test) discriminates 'very low' risk patients (not to be referred) from patients with an increased risk of DVT (to be referred). However, the rule's 'efficiency' (proportion of patients designated by the rule as very low risk) and safety (DVT prevalence among these very low risk patients) may change according to patient characteristics.

OBJECTIVE

To test the rule's safety and efficiency in clinically relevant subgroups; i.e. across three age groups, in men and women, and in patients with and without a history of DVT, separately.

METHODS

We retrospectively analysed data of 2086 primary care patients suspected of DVT, in whom all rule items and the result of the reference ('gold') standard (compression ultrasonography) were collected.

RESULTS

The rule's efficiency decreased with age from 38.1% in the relatively young (<50 years) compared to 9.8% in patients aged > or =70 years. The percentage of DVT among the very low risk patients was <1.5% in all subgroups. The low efficiency in the elderly could be improved without compromising the safety by increasing the D-dimer threshold.

CONCLUSION

The rule can safely exclude DVT in primary care patients suspected of DVT, irrespective of age, gender and history of DVT.

Polytomous logistic regression analysis could be applied more often in diagnostic research

OBJECTIVE

Physicians commonly consider the presence of all differential diagnoses simultaneously. Polytomous logistic regression modeling allows for simultaneous estimation of the probability of multiple diagnoses. We discuss and (empirically) illustrate the value of this method for diagnostic research.

STUDY DESIGN AND SETTING

We used data from a study on the diagnosis of residual retroperitoneal mass histology in patients presenting with nonseminomatous testicular germ cell tumor. The differential diagnoses include benign tissue, mature teratoma, and viable cancer. Probabilities of each diagnosis were estimated with a polytomous logistic regression model and compared with the probabilities estimated from two consecutive dichotomous logistic regression models.

RESULTS

We provide interpretations of the odds ratios derived from the polytomous regression model and present a simple score chart to facilitate calculation of predicted probabilities from the polytomous model. For both modeling methods, we show the calibration plots and receiver operating characteristics curve (ROC) areas comparing each diagnostic outcome category with the other two. The ROC areas for benign tissue, mature teratoma, and viable cancer were similar for both modeling methods, 0.83 (95% confidence interval [CI]=0.80-0.85) vs. 0.83 (95% CI=0.80-0.85), 0.78 (95% CI=0.75-0.81) vs. 0.78 (95% CI=0.75-0.81), and 0.66 (95% CI=0.61-0.71) vs. 0.64 (95% CI=0.59-0.69), for polytomous and dichotomous regression models, respectively.

CONCLUSION

Polytomous logistic regression is a useful technique to simultaneously model predicted probabilities of multiple diagnostic outcome categories. The performance of a polytomous prediction model can be assessed similarly to a dichotomous logistic regression model, and predictions by a polytomous model can be made with a user-friendly method. Because the simultaneous consideration of the presence of multiple (differential) conditions serves clinical practice better than consideration of the presence of only one target condition, polytomous logistic regression could be applied more often in diagnostic research.

Assessing the applicability of scoring systems for predicting postoperative nausea and vomiting

We have validated two scoring systems for predicting postoperative nausea and vomiting, derived by Apfel et al. and Koivuranta et al. from 1388 adult inpatients undergoing a wide range of surgical procedures. The predictive accuracy of the scoring systems was evaluated in terms of the ability to discriminate between patients with and without postoperative nausea and vomiting (discrimination) and agreement between observed and predicted outcomes (calibration). Discrimination and calibration were less than expected based on previous reports, with both scoring systems providing risk predictions that were too extreme. The area under the ROC curve was 0.63 for Apfel et al.'s scoring system and 0.66 for Koivuranta et al.'s scoring system. Neither of the scoring systems provided a risk threshold for administering anti-emetic prophylaxis that yielded satisfying results in terms of predictive values, sensitivity and specificity. Hence, in their original forms, the scoring systems do not guarantee accurate prediction of the risk of postoperative nausea and vomiting in other patient populations. Koivuranta et al.'s scoring system appears to be more robust across different populations.

External validity of a prediction rule for residual mass histology in testicular cancer: an evaluation for good prognosis patients

We assessed the external validity of a prediction rule for nonseminomatous testicular cancer patients. The rule was developed to predict the probability of retroperitoneal metastases being benign (only necrosis/fibrosis) after chemotherapy treatment. Patients with a high probability of benign residual masses might be offered surveillance as opposed to patients with a low probability, who should undergo retroperitoneal lymph node dissection (RPLND). We compared the observed histology with the predicted probability in 105 patients with good prognosis germ cell cancer who underwent RPLND between 1995 and 1998. We found that predicted probabilities higher than 5% were in good agreement with the observed frequencies of benign masses. The area under the receiver operating characteristic curve was 0.76, suggesting that the rule could reasonably discriminate between benign masses and tumour. However, nearly all predicted probabilities (n=101) were lower than 70%, which might be considered as the lowest value at which surveillance offers a reasonable alternative to RPLND. Further, 35% of patients currently under surveillance (84 out of 241) had predicted probabilities lower than 70%. In conclusion, the clinical relevance of the prediction rule was limited for the patients who underwent RPLND; use of the rule would change the policy from RPLND to surveillance in only a few. On the other hand, the rule might support selection of patients for RPLND, who currently are under surveillance.

Residual mass histology in testicular cancer: development and validation of a clinical prediction rule

After chemotherapy for metastatic non-seminomatous testicular cancer, surgical resection is a generally accepted treatment to remove remnants of the initial metastases, since residual tumour may still be present (mature teratoma or viable cancer cells). In this paper, we review the development and external validation of a logistic regression model to predict the absence of residual tumour. Three sources of information were used. A quantitative review identified six relevant predictors from 19 published studies (996 resections). Second, a development data set included individual data of 544 patients from six centres. This data set was used to assess the predictive relationships of five continuous predictors, which resulted in dichotomization for two, and a log, square root, and linear transformation for three other predictors. The multiple logistic regression coefficients were reduced with a shrinkage factor (0.95) to improve calibration, based on a bootstrapping procedure. Third, a validation data set included 172 more recently treated patients. The model showed adequate calibration and good discrimination in the development and in the validation sample (areas under the ROC curve 0.83 and 0.82). This study illustrates that a careful modelling strategy may result in an adequate predictive model. Further study of model validity may stimulate application in clinical practice.

Internal validation of predictive models: efficiency of some procedures for logistic regression analysis

The performance of a predictive model is overestimated when simply determined on the sample of subjects that was used to construct the model. Several internal validation methods are available that aim to provide a more accurate estimate of model performance in new subjects. We evaluated several variants of split-sample, cross-validation and bootstrapping methods with a logistic regression model that included eight predictors for 30-day mortality after an acute myocardial infarction. Random samples with a size between n = 572 and n = 9165 were drawn from a large data set (GUSTO-I; n = 40,830; 2851 deaths) to reflect modeling in data sets with between 5 and 80 events per variable. Independent performance was determined on the remaining subjects. Performance measures included discriminative ability, calibration and overall accuracy. We found that split-sample analyses gave overly pessimistic estimates of performance, with large variability. Cross-validation on 10% of the sample had low bias and low variability, but was not suitable for all performance measures. Internal validity could best be estimated with bootstrapping, which provided stable estimates with low bias. We conclude that split-sample validation is inefficient, and recommend bootstrapping for estimation of internal validity of a predictive logistic regression model.

Validation of a prediction model and its predictors for the histology of residual masses in nonseminomatous testicular cancer

PURPOSE

We validated a prediction model for histology of residual retroperitoneal masses, either benign or tumor, in patients treated with chemotherapy for metastatic nonseminomatous testicular cancer.

MATERIALS AND METHODS

We studied 276 patients treated with chemotherapy before retroperitoneal lymph node dissection at Indiana University Medical Center between 1985 and 1999. A previously developed prediction model was modified to provide predictions for the Indiana population based on 5 predictors. For these predictors, including teratomatous elements in the primary tumor, pre-chemotherapy tumor markers (alpha-fetoprotein and human chorionic gonadotropin), size of the residual mass and reduction in mass size, univariate and multivariate odds ratios were determined. The modified model was evaluated by calculating the concordance statistic and studying model reliability.

RESULTS

All odds ratios from univariate and multivariate analyses were in the expected directions. The modified model had good discriminative ability (concordance statistic 0.79). However, the predicted probabilities for benign tissue were generally too high due to the low prevalence of benign tissue (76 of 276 cases or 28%).

CONCLUSIONS

This study confirms the predictive ability of formerly identified predictors for the histology of residual retroperitoneal masses in testicular cancer. However, the previously developed prognostic model must be adjusted for the local overall ratio of benign versus tumor histology to provide reliable predictions in the Indiana population.

Severe hepatic dysfunction after adenovirus-mediated transfer of the herpes simplex virus thymidine kinase gene and ganciclovir administration

The use of so-called 'suicide' genes to activate prodrugs has been effective in animal models for several solid tumor types and is now in phase I and II clinical trials. We have exploited adenovirus vectors (Ad) for transfer and expression of the herpes simplex virus thymidine kinase (HSVtk) gene to render rat colorectal liver metastases sensitive to the anti-herpetic agent ganciclovir (GCV). The efficacy and toxicity of this enzyme-prodrug combination were tested after in situ transduction of rat colorectal tumor cells and after intraportal administration of the vector Ad.CMV.TK. Our results demonstrate the validity of the approach but reveal that hepatic expression of HSVtk, both in tumor bearing and in tumor-free rats, provokes severe liver dysfunction and mortality upon GCV administration. These data show, that in contrast to the common assumption, normally non-mitotic tissues too, can be affected by adenovirus-mediated HSVtk transfer and subsequent GCV treatment. Given the hepatotropic nature of systemically administered adenovirus type 2- and 5-derived vectors, it will be essential to monitor liver functions of patients included in all gene therapy trials involving adenoviral vectors with the HSVtk gene.

Gastric mucosal plasminogen activators in Helicobacter pylori infection

Long-term H. pylori associated gastritis is recognized as a pathogenic factor in gastric carcinogenesis. In gastric carcinomas the amount and activity of the tissue-type plasminogen activator (t-PA) have been reported to be decreased, whereas those of the urokinase-type plasminogen activator (u-PA) were increased, contributing to the neoplastic and invasive process. The present study was performed to determine t-PA and u-PA levels and activity in gastric mucosa from 102 patients and to investigate whether these levels are influenced by H. pylori infection. The antigen concentration and activity of t-PA and u-PA in corpus mucosa were low (P < 0.001) compared with those in antral mucosa, although for the u-PA activity this did not reach statistical significance. In H. pylori-associated antral gastritis the mucosal t-PA antigen concentration and activity were found to be decreased (P < 0.001) compared with normal mucosa, whereas in H. pylori-associated pangastritis the corpus t-PA levels were not affected. The antigen concentration and activity of u-PA were found to be significantly (P < 0.005) increased, both in H. pylori-associated gastritis of antrum and corpus mucosa. Levels of u-PA in histologically normal corpus mucosa of patients with an H. pylori-associated antral gastritis were also found to be increased (P < 0.05). In conclusion, the alterations in the plasminogen activator profile found in H. pylori-associated gastritis, ie, a decrease in t-PA and an increase in u-PA, show a similar tendency as the previously found alterations in gastric carcinomas, which provides additional support for the possible involvement of H. pylori-associated gastritis in the pathogenesis of gastric carcinoma.