Risk assessment for endometrial cancer in women with abnormal vaginal bleeding: Results from the prospective IETA-1 cohort study

OBJECTIVE

To investigate the association between personal history, anthropometric features and lifestyle characteristics and endometrial malignancy in women with abnormal vaginal bleeding.

METHODS

Prospective observational cohort assessed by descriptive and multivariable logistic regression analyses. Three features-age, body mass index (BMI; calculated as weight in kilograms divided by the square of height in meters), and nulliparity-were defined a priori for baseline risk assessment of endometrial malignancy. The following variables were tested for added value: intrauterine contraceptive device, bleeding pattern, age at menopause, coexisting diabetes/hypertension, physical exercise, fat distribution, bra size, waist circumference, smoking/drinking habits, family history, use of hormonal/anticoagulant therapy, and sonographic endometrial thickness. We calculated adjusted odds ratio, optimism-corrected area under the receiver operating characteristic curve (AUC), R² , and Akaike's information criterion.

RESULTS

Of 2417 women, 155 (6%) had endometrial malignancy or endometrial intraepithelial neoplasia. In women with endometrial cancer median age was 67 years (interquartile range [IQR] 56-75 years), median parity was 2 (IQR 0-10), and median BMI was 28 (IQR 25-32). Age, BMI, and parity produced an AUC of 0.82. Other variables marginally affected the AUC, adding endometrial thickness substantially increased the AUC in postmenopausal women.

CONCLUSION

Age, parity, and BMI help in the assessment of endometrial cancer risk in women with abnormal uterine bleeding. Other patient information adds little, whereas sonographic endometrial thickness substantially improves assessment.

The Risk of Endometrial Malignancy and Other Endometrial Pathology in Women with Abnormal Uterine Bleeding: An Ultrasound-Based Model Development Study by the IETA Group

OBJECTIVES

The aim of this study was to develop a model that can discriminate between different etiologies of abnormal uterine bleeding.

DESIGN

The International Endometrial Tumor Analysis 1 study is a multicenter observational diagnostic study in 18 bleeding clinics in 9 countries. Consecutive women with abnormal vaginal bleeding presenting for ultrasound examination (n = 2,417) were recruited. The histology was obtained from endometrial sampling, D&C, hysteroscopic resection, hysterectomy, or ultrasound follow-up for >1 year.

METHODS

A model was developed using multinomial regression based on age, body mass index, and ultrasound predictors to distinguish between: (1) endometrial atrophy, (2) endometrial polyp or intracavitary myoma, (3) endometrial malignancy or atypical hyperplasia, (4) proliferative/secretory changes, endometritis, or hyperplasia without atypia and validated using leave-center-out cross-validation and bootstrapping. The main outcomes are the model's ability to discriminate between the four outcomes and the calibration of risk estimates.

RESULTS

The median age in 2,417 women was 50 (interquartile range 43-57). 414 (17%) women had endometrial atrophy; 996 (41%) had a polyp or myoma; 155 (6%) had an endometrial malignancy or atypical hyperplasia; and 852 (35%) had proliferative/secretory changes, endometritis, or hyperplasia without atypia. The model distinguished well between malignant and benign histology (c-statistic 0.88 95% CI: 0.85-0.91) and between all benign histologies. The probabilities for each of the four outcomes were over- or underestimated depending on the centers.

LIMITATIONS

Not all patients had a diagnosis based on histology. The model over- or underestimated the risk for certain outcomes in some centers, indicating local recalibration is advisable.

CONCLUSIONS

The proposed model reliably distinguishes between four histological outcomes. This is the first model to discriminate between several outcomes and is the only model applicable when menopausal status is uncertain. The model could be useful for patient management and counseling, and aid in the interpretation of ultrasound findings. Future research is needed to externally validate and locally recalibrate the model.

ROC curves for clinical prediction models part 1. ROC plots showed no added value above the AUC when evaluating the performance of clinical prediction models

OBJECTIVES

Receiver operating characteristic (ROC) curves show how well a risk prediction model discriminates between patients with and without a condition. We aim to investigate how ROC curves are presented in the literature and discuss and illustrate their potential limitations.

STUDY DESIGN AND SETTING

We conducted a pragmatic literature review of contemporary publications that externally validated clinical prediction models. We illustrated limitations of ROC curves using a testicular cancer case study and simulated data.

RESULTS

Of 86 identified prediction modeling studies, 52 (60%) presented ROC curves without thresholds and one (1%) presented an ROC curve with only a few thresholds. We illustrate that ROC curves in their standard form withhold threshold information have an unstable shape even for the same area under the curve (AUC) and are problematic for comparing model performance conditional on threshold. We compare ROC curves with classification plots, which show sensitivity and specificity conditional on risk thresholds.

CONCLUSION

ROC curves do not offer more information than the AUC to indicate discriminative ability. To assess the model's performance for decision-making, results should be provided conditional on risk thresholds. Therefore, if discriminatory ability must be visualized, classification plots are attractive.

Regression shrinkage methods for clinical prediction models do not guarantee improved performance: Simulation study

When developing risk prediction models on datasets with limited sample size, shrinkage methods are recommended. Earlier studies showed that shrinkage results in better predictive performance on average. This simulation study aimed to investigate the variability of regression shrinkage on predictive performance for a binary outcome. We compared standard maximum likelihood with the following shrinkage methods: uniform shrinkage (likelihood-based and bootstrap-based), penalized maximum likelihood (ridge) methods, LASSO logistic regression, adaptive LASSO, and Firth's correction. In the simulation study, we varied the number of predictors and their strength, the correlation between predictors, the event rate of the outcome, and the events per variable. In terms of results, we focused on the calibration slope. The slope indicates whether risk predictions are too extreme (slope < 1) or not extreme enough (slope > 1). The results can be summarized into three main findings. First, shrinkage improved calibration slopes on average. Second, the between-sample variability of calibration slopes was often increased relative to maximum likelihood. In contrast to other shrinkage approaches, Firth's correction had a small shrinkage effect but showed low variability. Third, the correlation between the estimated shrinkage and the optimal shrinkage to remove overfitting was typically negative, with Firth's correction as the exception. We conclude that, despite improved performance on average, shrinkage often worked poorly in individual datasets, in particular when it was most needed. The results imply that shrinkage methods do not solve problems associated with small sample size or low number of events per variable.

The potential value of activin B and fibronectin for the triage of pregnancies of unknown location and prediction of first trimester viability

Aim

We have assessed the potential predictive ability of the biomarkers activin B and fibronectin (FN1) alone and when added to established markers for triaging patients as being at low or high risk of ectopic pregnancy (EP). We also assessed their use as predictors of viability at 12 weeks gestation.

Methods

Exploratory secondary analysis of a prospective study including all women classified as a pregnancy of known location (PUL) based on transvaginal ultrasonography between January and December 2007 at the early pregnancy unit of St Georges' Hospital (London). We used multinomial logistic regression to assess the diagnostic potential of the biomarkers to triage PUL at high risk of complications (EP or persistent PUL), and standard binary logistic regression to predict first trimester viability at 12 weeks.

Results

For discriminating high-risk (n = 16) from low-risk PUL (n = 93), the area under the receiver operating characteristic curve (AUC) was 0.75 (95% confidence interval 0.60-0.85) for activin B and 0.55 (0.41-0.68) for FN1. Adding activin B to a multinomial logistic regression model incorporating β-hCG ratio and initial progesterone yielded odds ratios of 0.16 (0.05-0.55) for failing vs high-risk PUL and 0.29 (0.07-1.19) for intrauterine vs high-risk PUL and increased the model's AUC from 0.84 to 0.89. At a risk threshold of 5% for high-risk PUL, sensitivity increased from 84% to 87% and specificity from 48% to 64%. For discriminating viable (n = 28) from non-viable (n = 81) pregnancies at 12 weeks, both markers had an AUC of 0.54.

Conclusions

Our results suggested that activin B may be a promising marker to improve PUL triage in addition to established markers.

Doctors' perception of support and the processes involved in complaints investigations and how these relate to welfare and defensive practice: a cross-sectional survey of the UK physicians

OBJECTIVE

How adverse outcomes and complaints are managed may significantly impact on physician well-being and practice. We aimed to investigate how depression, anxiety and defensive medical practice are associated with doctors actual and perceived support, behaviour of colleagues and process issues regarding how complaints investigations are carried out.

DESIGN

A survey study. Respondents were classified into three groups: no complaint, recent/current complaint (within 6 months) or past complaint. Each group completed specific surveys.

SETTING

British Medical Association (BMA) members were invited to complete an online survey.

PARTICIPANTS

95 636 members of the BMA were asked to participate. 7926 (8.3%) completed the survey, of whom 1780 (22.5%) had no complaint, 3889 (49.1%) had a past complaint and 2257 (28.5%) had a recent/current complaint. We excluded those with no complaints leaving 6144 in the final sample.

PRIMARY OUTCOMES MEASURES

We measured anxiety and depression using the Generalised Anxiety Disorder Scale 7 and Physical Health Questionnaire 9. Defensive practice was assessed using a new measure for avoidance and hedging.

RESULTS

Most felt supported by colleagues (61%), only 31% felt supported by management. Not following process (56%), protracted timescales (78%), vexatious complaints (49%), feeling bullied (39%) or victimised for whistleblowing (20%), and using complaints to undermine (31%) were reported. Perceived support by management (relative risk (RR) depression: 0.77, 95% CI 0.71 to 0.83; RR anxiety: 0.80, 95% CI 0.74 to 0.87), speaking to colleagues (RR depression: 0.64, 95% CI 0.48 to 0.84 and RR anxiety: 0.69, 95% CI 0.51 to 0.94, respectively), fair/accurate documentation (RR depression: 0.80, 95% CI 0.75 to 0.86; RR anxiety: 0.81, 95% CI 0.75 to 0.87), and being informed about rights (RR depression 0.96 (0.89 to 1.03) and anxiety 0.94 (0.87 to 1.02), correlated positively with well-being and reduced defensive practice. Doctors worried most about professional humiliation following a complaint investigation (80%).

CONCLUSION

Poor process, prolonged timescales and vexatious use of complaints systems are associated with decreased psychological welfare and increased defensive practice. In contrast, perceived support from colleagues and management is associated with a reduction in these effects.

Erratum to: Methods for evaluating medical tests and biomarkers

[This corrects the article DOI: 10.1186/s41512-016-0001-y.].

Doctors' experiences and their perception of the most stressful aspects of complaints processes in the UK: an analysis of qualitative survey data

OBJECTIVES

To examine doctors' experiences of complaints, including which aspects are most stressful. We also investigated how doctors felt complaints processes could be improved.

DESIGN AND METHODS

A qualitative study based on a cross-sectional survey of members of the British Medical Association (BMA). We asked the following: (1) Try to summarise as best as you can your experience of the complaints process and how it made you feel. (2) What were the most stressful aspects of the complaint? (3) What would you improve in the complaints system?

PARTICIPANTS

We sent the survey to 95 636 doctors, and received 10 930 (11.4%) responses. Of these, 6146 had a previous, recent or current complaint and 3417 (31.3%) of these respondents answered questions 1 and 2. We randomly selected 1000 answers for analysis, and included 100 using the saturation principle. Of this cohort, 93 responses for question 3 were available.

MAIN RESULTS

Doctors frequently reported feeling powerless, emotionally distressed, and experiencing negative feelings towards both those managing complaints and the complainants themselves. Many felt unsupported, fearful of the consequences and that the complaint was unfair. The most stressful aspects were the prolonged duration and unpredictability of procedures, managerial incompetence, poor communication and perceiving that processes are biased in favour of complainants. Many reported practising defensively or considering changing career after a complaint, and few found any positive outcomes from complaints investigations. Physicians suggested procedures should be more transparent, competently managed, time limited, and that there should be an open dialogue with complainants and policies for dealing with vexatious complaints. Some felt more support for doctors was needed.

CONCLUSIONS

Complaints seriously impact on doctors' psychological wellbeing, and are associated with defensive practise. This is not beneficial to patient care. To improve procedures, doctors propose they are simplified, time limited and more transparent.

A calibration hierarchy for risk models was defined: from utopia to empirical data

OBJECTIVE

Calibrated risk models are vital for valid decision support. We define four levels of calibration and describe implications for model development and external validation of predictions.

STUDY DESIGN AND SETTING

We present results based on simulated data sets.

RESULTS

A common definition of calibration is "having an event rate of R% among patients with a predicted risk of R%," which we refer to as "moderate calibration." Weaker forms of calibration only require the average predicted risk (mean calibration) or the average prediction effects (weak calibration) to be correct. "Strong calibration" requires that the event rate equals the predicted risk for every covariate pattern. This implies that the model is fully correct for the validation setting. We argue that this is unrealistic: the model type may be incorrect, the linear predictor is only asymptotically unbiased, and all nonlinear and interaction effects should be correctly modeled. In addition, we prove that moderate calibration guarantees nonharmful decision making. Finally, results indicate that a flexible assessment of calibration in small validation data sets is problematic.

CONCLUSION

Strong calibration is desirable for individualized decision support but unrealistic and counter productive by stimulating the development of overly complex models. Model development and external validation should focus on moderate calibration.