A clinical prediction rule for ambulation outcomes after traumatic spinal cord injury: a longitudinal cohort study

BACKGROUND

Traumatic spinal cord injury is a serious disorder in which early prediction of ambulation is important to counsel patients and to plan rehabilitation. We developed a reliable, validated prediction rule to assess a patient's chances of walking independently after such injury.

METHODS

We undertook a longitudinal cohort study of adult patients with traumatic spinal cord injury, with early (within the first 15 days after injury) and late (1-year follow-up) clinical examinations, who were admitted to one of 19 European centres between July, 2001, and June, 2008. A clinical prediction rule based on age and neurological variables was derived from the international standards for neurological classification of spinal cord injury with a multivariate logistic regression model. Primary outcome measure 1 year after injury was independent indoor walking based on the Spinal Cord Independence Measure. Model performances were quantified with respect to discrimination (area under receiver-operating-characteristics curve [AUC]). Temporal validation was done in a second group of patients from July, 2008, to December, 2009.

FINDINGS

Of 1442 patients with spinal cord injury, 492 had available outcome measures. A combination of age (<65 vs ≥65 years), motor scores of the quadriceps femoris (L3), gastrocsoleus (S1) muscles, and light touch sensation of dermatomes L3 and S1 showed excellent discrimination in distinguishing independent walkers from dependent walkers and non-walkers (AUC 0·956, 95% CI 0·936-0·976, p<0·0001). Temporal validation in 99 patients confirmed excellent discriminating ability of the prediction rule (AUC 0·967, 0·939-0·995, p<0·0001).

INTERPRETATION

Our prediction rule, including age and four neurological tests, can give an early prognosis of an individual's ability to walk after traumatic spinal cord injury, which can be used to set rehabilitation goals and might improve the ability to stratify patients in interventional trials.

FUNDING

Internationale Stiftung für Forschung in Paraplegie.

Unpredictable bias when using the missing indicator method or complete case analysis for missing confounder values: an empirical example

OBJECTIVE

Missing indicator method (MIM) and complete case analysis (CC) are frequently used to handle missing confounder data. Using empirical data, we demonstrated the degree and direction of bias in the effect estimate when using these methods compared with multiple imputation (MI).

STUDY DESIGN AND SETTING

From a cohort study, we selected an exposure (marital status), outcome (depression), and confounders (age, sex, and income). Missing values in "income" were created according to different patterns of missingness: missing values were created completely at random and depending on exposure and outcome values. Percentages of missing values ranged from 2.5% to 30%.

RESULTS

When missing values were completely random, MIM gave an overestimation of the odds ratio, whereas CC and MI gave unbiased results. MIM and CC gave under- or overestimations when missing values depended on observed values. Magnitude and direction of bias depended on how the missing values were related to exposure and outcome. Bias increased with increasing percentage of missing values.

CONCLUSION

MIM should not be used in handling missing confounder data because it gives unpredictable bias of the odds ratio even with small percentages of missing values. CC can be used when missing values are completely random, but it gives loss of statistical power.

Missing covariate data in medical research: to impute is better than to ignore

OBJECTIVE

We compared popular methods to handle missing data with multiple imputation (a more sophisticated method that preserves data).

STUDY DESIGN AND SETTING

We used data of 804 patients with a suspicion of deep venous thrombosis (DVT). We studied three covariates to predict the presence of DVT: d-dimer level, difference in calf circumference, and history of leg trauma. We introduced missing values (missing at random) ranging from 10% to 90%. The risk of DVT was modeled with logistic regression for the three methods, that is, complete case analysis, exclusion of d-dimer level from the model, and multiple imputation.

RESULTS

Multiple imputation showed less bias in the regression coefficients of the three variables and more accurate coverage of the corresponding 90% confidence intervals than complete case analysis and dropping d-dimer level from the analysis. Multiple imputation showed unbiased estimates of the area under the receiver operating characteristic curve (0.88) compared with complete case analysis (0.77) and when the variable with missing values was dropped (0.65).

CONCLUSION

As this study shows that simple methods to deal with missing data can lead to seriously misleading results, we advise to consider multiple imputation. The purpose of multiple imputation is not to create data, but to prevent the exclusion of observed data.

Increased oxytocin concentrations and prosocial feelings in humans after ecstasy (3,4-methylenedioxymethamphetamine) administration

MDMA (3,4-methylenedioxymethamphetamine or "ecstasy") is a recreationally used drug with remarkable and characteristic prosocial effects. In spite of abundant attention in the scientific literature, the mechanism of its prosocial effects has not been elucidated in humans. Recently, research in animals has suggested that the neuropeptide oxytocin may induce these effects. In a double blind, randomized, crossover, and placebo-controlled study in 15 healthy volunteers we assessed blood oxytocin and MDMA concentrations and subjective prosocial effects after oral administration of 100 mg MDMA or placebo. MDMA induced a robust increase of blood oxytocin concentrations and an increase of subjective prosocial feelings. Within subjects, the variations in these feelings were significantly and positively correlated with variation in oxytocin levels, and the correlations between these feelings and oxytocin were significantly stronger than those between these feelings and blood MDMA levels. MDMA induces oxytocin release in humans, which may be involved in the characteristic prosocial effects of ecstasy.

Dealing with missing predictor values when applying clinical prediction models

BACKGROUND

Prediction models combine patient characteristics and test results to predict the presence of a disease or the occurrence of an event in the future. In the event that test results (predictor) are unavailable, a strategy is needed to help users applying a prediction model to deal with such missing values. We evaluated 6 strategies to deal with missing values.

METHODS

We developed and validated (in 1295 and 532 primary care patients, respectively) a prediction model to predict the risk of deep venous thrombosis. In an application set (259 patients), we mimicked 3 situations in which (1) an important predictor (D-dimer test), (2) a weaker predictor (difference in calf circumference), and (3) both predictors simultaneously were missing. The 6 strategies to deal with missing values were (1) ignoring the predictor, (2) overall mean imputation, (3) subgroup mean imputation, (4) multiple imputation, (5) applying a submodel including only the observed predictors as derived from the development set, or (6) the "one-step-sweep" method. We compared the model's discriminative ability (expressed by the ROC area) with the true ROC area (no missing values) and the model's estimated calibration slope and intercept with the ideal values of 1 and 0, respectively.

RESULTS

Ignoring the predictor led to the worst and multiple imputation to the best discrimination. Multiple imputation led to calibration intercepts closest to the true value. The effect of the strategies on the slope differed between the 3 scenarios.

CONCLUSIONS

Multiple imputation is preferred if a predictor value is missing.

The Voice Handicap of Student-Teachers and Risk Factors Perceived to Have a Negative Influence on the Voice

A cross-sectional questionnaire survey was performed. The objectives of the study were to assess the psychosocial impact of current voice complaints as perceived by student-teachers with voice complaints in comparison with student-teachers without voice complaints, and to observe the pattern of risk factors in relation to their voice handicap. Subjects in the general population without a voice-demanding profession were selected as a reference group for limited comparison with the total group of student-teachers (future professional voice users). The respondents to the questionnaires were anonymous. Among the student-teachers, 17.2% reported current voice complaints in comparison with 9.7% of the reference group, and the odds ratio was 1.94, which showed the relative risk. Student-teachers had significantly greater total Voice Handicap Index (VHI) scores than the reference group (P = 0.034). The VHI subscale scores were not significantly different (P > 0.05). Student-teachers who reported current voice complaints had a significantly higher total VHI and subscale scores than student teachers without voice complaints (P < 0.001). Of the student-teachers without voice complaints, 17.0% had VHI scores greater than the 75th percentile. These persons may be neglecting their voice handicap and probably represent the false-negative cases in the estimation of voice complaints. Logistic regression analysis of each of the given risk factors with the VHI as the independent variable showed that the perceived negative influence of the given risk factors on their voices was significantly greater with increasing VHI scores across the VHI range. A significant correlation was observed between the number of perceived risk factors and increasing VHI scores across the VHI range. An increased awareness of risk factors in relation to their voice handicap would serve to motivate student-teachers to change factors that contributed to their voice problem. Attention to all risk factors, which the subjects perceive to be a risk, would aid in effective management of their voice handicap.

Validating and updating a prediction rule for serious bacterial infection in patients with fever without source

AIM

To externally validate and update a previously developed rule for predicting the presence of serious bacterial infections in children with fever without apparent source.

METHODS

Patients, 1-36 mo, presenting with fever without source, were prospectively enrolled. Serious bacterial infection included bacterial meningitis, sepsis, bacteraemia, pneumonia, urinary tract infection, bacterial gastroenteritis, osteomyelitis/ethmoiditis. The generalizability of the original rule was determined. Subsequently, the prediction rule was updated using all available data of the patients with fever without source (1996-1998 and 2000-2001, n = 381) using multivariable logistic regression.

RESULTS

the generalizability of the rule appeared insufficient in the new patients (n = 150). In the updated rule, independent predictors from history and examination were duration of fever, vomiting, ill clinical appearance, chest-wall retractions and poor peripheral circulation (ROC area (95%CI): 0.69 (0.63-0.75)). Additional independent predictors from laboratory were serum white blood cell count and C-reactive protein, and in urinalysis > or = 70 white bloods (ROC area (95%CI): 0.83 (0.78-0.88).

CONCLUSIONS

A previously developed prediction rule for predicting the presence of serious bacterial infection in children with fever without apparent source was updated. Its clinical score can be used as a first screening tool. Additional laboratory testing may specify the individual risk estimate (range: 4-54%) further.

Imputation of missing values is superior to complete case analysis and the missing-indicator method in multivariable diagnostic research: A clinical example

BACKGROUND AND OBJECTIVES

To illustrate the effects of different methods for handling missing data--complete case analysis, missing-indicator method, single imputation of unconditional and conditional mean, and multiple imputation (MI)--in the context of multivariable diagnostic research aiming to identify potential predictors (test results) that independently contribute to the prediction of disease presence or absence.

METHODS

We used data from 398 subjects from a prospective study on the diagnosis of pulmonary embolism. Various diagnostic predictors or tests had (varying percentages of) missing values. Per method of handling these missing values, we fitted a diagnostic prediction model using multivariable logistic regression analysis.

RESULTS

The receiver operating characteristic curve area for all diagnostic models was above 0.75. The predictors in the final models based on the complete case analysis, and after using the missing-indicator method, were very different compared to the other models. The models based on MI did not differ much from the models derived after using single conditional and unconditional mean imputation.

CONCLUSION

In multivariable diagnostic research complete case analysis and the use of the missing-indicator method should be avoided, even when data are missing completely at random. MI methods are known to be superior to single imputation methods. For our example study, the single imputation methods performed equally well, but this was most likely because of the low overall number of missing values.

Review: a gentle introduction to imputation of missing values

In most situations, simple techniques for handling missing data (such as complete case analysis, overall mean imputation, and the missing-indicator method) produce biased results, whereas imputation techniques yield valid results without complicating the analysis once the imputations are carried out. Imputation techniques are based on the idea that any subject in a study sample can be replaced by a new randomly chosen subject from the same source population. Imputation of missing data on a variable is replacing that missing by a value that is drawn from an estimate of the distribution of this variable. In single imputation, only one estimate is used. In multiple imputation, various estimates are used, reflecting the uncertainty in the estimation of this distribution. Under the general conditions of so-called missing at random and missing completely at random, both single and multiple imputations result in unbiased estimates of study associations. But single imputation results in too small estimated standard errors, whereas multiple imputation results in correctly estimated standard errors and confidence intervals. In this article we explain why all this is the case, and use a simple simulation study to demonstrate our explanations. We also explain and illustrate why two frequently used methods to handle missing data, i.e., overall mean imputation and the missing-indicator method, almost always result in biased estimates.

Voice Complaints, Risk Factors for Voice Problems and History of Voice Problems in Relation to Puberty in Female Student Teachers

The aim of the study was to estimate voice complaints, risk factors for voice complaints and history of voice problems in student teachers before they embarked on their professional teaching career. A cross-sectional questionnaire survey was performed among female student teachers. The response rate was 72% and 457 questionnaires were analyzed. Voice complaints at the moment and/or during the past year were reported by 39.6% subjects. Subjects with voice complaints had significantly higher VHI scores than subjects without voice complaints. In comparison to subjects without voice complaints, overall, subjects with voice complaints reported more frequently that vocal loading factors, physical factors, environmental factors and psychological factors had a negative influence on their voice. Subjects with voice complaints reported more frequently a history of voice complaints during puberty and before puberty in comparison to subjects without voice complaints. Voice complaints in student teachers apparently had a multifactorial genesis and with roots during puberty or before puberty. Logistic regression analysis revealed that intensive voice use, emotions and history of voice complaints during puberty were the most discriminating set of risk factors for voice complaints. Subjects with voice complaints in comparison to those without voice complaints reported more frequently that they would develop a voice problem due to future teaching and that future teaching would have a negative influence on their voice. Around three quarters of subjects with and without voice complaints reported that attention paid to their voice during their training was sufficient. However, subjects with voice complaints were observed to report the need for a refresher course on voice use more frequently than those without voice complaints. The findings call for more intensive voice training for student teachers to cope with the vocal, physical and psychological demands of the teaching profession. Authorities should take responsibility to monitor and improve working conditions of student teachers and teachers.

Prediction of pressure ulcer development in hospitalized patients: a tool for risk assessment

OBJECTIVES

To identify independent predictors for development of pressure ulcers in hospitalized patients and to develop a simple prediction rule for pressure ulcer development.

DESIGN

The Prevention and Pressure Ulcer Risk Score Evaluation (prePURSE) study is a prospective cohort study in which patients are followed up once a week until pressure ulcer occurrence, discharge from hospital, or length of stay over 12 weeks. Data were collected between January 1999 and June 2000.

SETTING

Two large hospitals in the Netherlands.

PARTICIPANTS

Adult patients admitted to the surgical, internal, neurological and geriatric wards for more than 5 days were eligible. A consecutive sample of 1536 patients was visited, 1431 (93%) of whom agreed to participate. Complete follow up data were available for 1229 (80%) patients.

MAIN OUTCOME MEASURES

Occurrence of a pressure ulcer grade 2 or worse during admission to hospital.

RESULTS

Independent predictors of pressure ulcers were age, weight at admission, abnormal appearance of the skin, friction and shear, and planned surgery in coming week. The area under the curve of the final prediction rule was 0.70 after bootstrapping. At a cut off score of 20, 42% of the patient weeks were identified as at risk for pressure ulcer development, thus correctly identifying 70% of the patient weeks in which a pressure ulcer occurred.

CONCLUSION

A simple clinical prediction rule based on five patient characteristics may help to identify patients at increased risk for pressure ulcer development and in need of preventive measures.

Can Screening for Retinopathy of Prematurity Be Reduced?

BACKGROUND

As screening for retinopathy of prematurity (ROP) is costly, time-consuming for the ophthalmologist and discomforting for the neonate, the minimum number of infants should be screened for ROP, without missing infants with severe ROP, at risk for threshold ROP.

OBJECTIVES

To develop a diagnostic screening guideline for ROP that would safely reduce the number of ROP screening funduscopies in our department.

METHODS

Data of 275 infants admitted between 1996 and 2000 and screened for ROP according to our Dutch National guideline were studied. Significant risk factors for ROP were calculated, using logistic regression analysis and used to develop a guideline. The discriminative power of the guideline was evaluated using the area under the curve for the receiver operating characteristic curve.

RESULTS

Significant risk factors for ROP were: gestational age, birth weight and number of erythrocyte transfusions within the first 4 weeks of life. The combination of these 3 factors resulted in the highest area under the curve: 0.793. Using these 3 factors, a diagnostic screening guideline for ROP was developed: if birth weight + 2 x (gestational age - 20) - 6 x erythrocyte transfusion value within the first 4 weeks of life >or=34, no screening for ROP is necessary. Using this guideline, 22.2% of the infants of the study group could have been excluded from screening; 3.8% of the infants with ROP stages 1-2 would have been missed.

CONCLUSION

In our department, ROP screening can be safely reduced using our diagnostic screening guideline.

Penalized maximum likelihood estimation to directly adjust diagnostic and prognostic prediction models for overoptimism: a clinical example

BACKGROUND AND OBJECTIVE

There is growing interest in developing prediction models. The accuracy of such models when applied in new patient samples is commonly lower than estimated from the development sample. This may be because of differences between the samples and/or because the developed model was overfitted (too optimistic). Various methods, including bootstrapping techniques exist for afterwards shrinking the regression coefficients and the model's discrimination and calibration for overoptimism. Penalized maximum likelihood estimation (PMLE) is a more rigorous method because adjustment for overfitting is directly built into the model development, instead of relying on shrinkage afterwards. PMLE has been described mainly in the statistical literature and is rarely applied to empirical data. Using empirical data, we illustrate the use of PMLE to develop a prediction model.

METHODS

The accuracy of the final PMLE model will be contrasted with the final models derived by ordinary stepwise logistic regression without and with shrinkage afterwards. The potential advantages and disadvantages of PMLE over the other two strategies are discussed.

RESULTS

PMLE leads to smaller prediction errors, provides for model reduction to a user-defined degree, and may differently shrink each predictor for overoptimism without sacrificing much discriminative accuracy of the model.

CONCLUSION

PMLE is an easily applicable and promising method to directly adjust clinical prediction models for overoptimism.

External validation is necessary in prediction research:

BACKGROUND AND OBJECTIVES

Prediction models tend to perform better on data on which the model was constructed than on new data. This difference in performance is an indication of the optimism in the apparent performance in the derivation set. For internal model validation, bootstrapping methods are recommended to provide bias-corrected estimates of model performance. Results are often accepted without sufficient regard to the importance of external validation. This report illustrates the limitations of internal validation to determine generalizability of a diagnostic prediction model to future settings.

METHODS

A prediction model for the presence of serious bacterial infections in children with fever without source was derived and validated internally using bootstrap resampling techniques. Subsequently, the model was validated externally.

RESULTS

In the derivation set (n=376), nine predictors were identified. The apparent area under the receiver operating characteristic curve (95% confidence interval) of the model was 0.83 (0.78-0.87) and 0.76 (0.67-0.85) after bootstrap correction. In the validation set (n=179) the performance was 0.57 (0.47-0.67).

CONCLUSION

For relatively small data sets, internal validation of prediction models by bootstrap techniques may not be sufficient and indicative for the model's performance in future patients. External validation is essential before implementing prediction models in clinical practice.

The incidence of visually impaired children with retinopathy of prematurity and their concomitant disabilities

PURPOSE

To determine, in relation to two previous studies, the incidence of visual impairment (VI) caused by retinopathy of prematurity (ROP) in children born in the Netherlands between 1994 and 2000 and their associated nonvisual disabilities.

METHODS

From all institutes, schools, and ambulatory services for the partially sighted and blind, data on ROP infants born between 1994 and 2000 were retrieved. Apart from neonatal and ophthalmologic data, other data were gathered concerning the following concomitant disabilities: behavioral abnormalities, epilepsy, hearing deficit, developmental delay, and distinct neurological handicaps. Birth rate data were obtained from the Central Bureau of Statistics for the Netherlands.

RESULTS

Data on 51 of 54 infants (94.4%) were studied. There was a statistically insignificant decrease in the incidence of visually impaired children caused by ROP in the period between 1994 and 2000 compared with the period between 1986 and 1994. Neonatal and ophthalmologic data of the infants were comparable for the study period and the previous period. The incidence of behavioral abnormalities in infants with ROP-related sequelae increased significantly in the study period compared with the previous one (46.9% v 21.8%, P <.05. The percentage of infants who received treatment for severe ROP did not change significantly (56.9% v 43.9% in the previous period).

CONCLUSIONS

This study shows a statistically insignificant decrease in the number of infants with visual impairment caused by ROP in the Netherlands in the period between 1994 and 2000 as well as a significant increase in the incidence of behavioral abnormalities in these children. Still, 43.1% of the infants did not receive acute-phase ROP treatment.