How to use likelihood ratios to interpret evidence from randomized trials

OBJECTIVE

The likelihood ratio is a method for assessing evidence regarding two simple statistical hypotheses. Its interpretation is simple - for example, a value of 10 means that the first hypothesis is 10 times as strongly supported by the data as the second. A method is shown for deriving likelihood ratios from published trial reports.

STUDY DESIGN

The likelihood ratio compares two hypotheses in light of data: that a new treatment is effective, at a specified level (alternate hypothesis: for instance, the hazard ratio equals 0.7), and that it is not (null hypothesis: the hazard ratio equals 1). The result of the trial is summarised by the test statistic z (ie, the estimated treatment effect divided by its standard error). The expected value of z is 0 under the null hypothesis, and A under the alternate hypothesis. The logarithm of the likelihood ratio is given by z·A - A²/2. The values of A and z can be derived from the alternate hypothesis used for sample size computation, and from the observed treatment effect and its standard error or confidence interval.

RESULTS

Examples are given of trials that yielded strong or moderate evidence in favor of the alternate hypothesis, and of a trial that favored the null hypothesis. The resulting likelihood ratios are applied to initial beliefs about the hypotheses to obtain posterior beliefs.

CONCLUSIONS

The likelihood ratio is a simple and easily understandable method for assessing evidence in data about two competing a priori hypotheses.

Patient satisfaction and survey response in 717 hospital surveys in Switzerland: a cross-sectional study

BACKGROUND

The association between patient satisfaction and survey response is only partly understood. In this study, we describe the association between average satisfaction and survey response rate across hospital surveys, and model the association between satisfaction and propensity to respond for individual patients.

METHODS

Secondary analysis of patient responses (166'014 respondents) and of average satisfaction scores and response rates obtained in 717 annual patient satisfaction surveys conducted between 2011 and 2015 at 164 Swiss hospitals. The satisfaction score was the average of 5 items scored between 0 and 10. The association between satisfaction and response propensity in individuals was modeled as the function that predicted best the observed response rates across surveys.

RESULTS

Among the 717 surveys, response rates ranged from 16.1 to 80.0% (pooled average 49.8%), and average satisfaction scores ranged from 8.36 to 9.79 (pooled mean 9.15). At the survey level, the mean satisfaction score and response rate were correlated (r = 0.61). This correlation held for all subgroups of surveys, except for the 5 large university hospitals. The estimated individual response propensity function was "J-shaped": the probability of responding was lowest (around 20%) for satisfaction scores between 3 and 7, increased sharply to about 70% for those maximally satisfied, and increased slightly for the least satisfied. Average satisfaction scores projected for 100% participation were lower than observed average scores.

CONCLUSIONS

The most satisfied patients were the most likely to participate in a post-hospitalization satisfaction survey. This tendency produces an upward bias in observed satisfaction scores, and a positive correlation between average satisfaction and response rate across surveys.

Symptoms of osteoarthritis influence mental and physical health differently before and after joint replacement surgery: A prospective study

Background

Patient-reported outcomes are increasingly used in evaluations of joint replacement surgery, but it is unclear if symptoms of osteoarthritis (i.e., pain and dysfunction) influence health perceptions similarly before and after surgery.

Methods

In this prospective study based on a hospital-based arthroplasty registry, patients with primary total hip or knee arthroplasty (THA, N = 990, and TKA, N = 907) completed the WOMAC Pain and Function scales, and the SF12 Physical and Mental Component Scores (PCS and MCS), before surgery and one year later. Associations between WOMAC and SF12 scales were examined using mixed linear regression models.

Results

All patient-reported outcomes improved following total joint arthroplasty, but the associations between symptom scales and global health perceptions were altered. Mental health scores at a given level of pain or function were lower after surgery than before, by about 4–5 points, a clinically meaningful and statistically significant difference. In contrast, the associations between WOMAC scales and the PCS remained stable. These findings were observed in both cohorts of patients.

Conclusions

After total joint arthroplasty, mental health scores were lower than would have been expected given the symptomatic improvement. This suggests that relationships between patient-reported outcomes are context-dependent, and that care should be exerted when interpreting changes in patient-reported outcomes over time.

Internists' and intensivists' roles in intensive care admission decisions: a qualitative study

BACKGROUND

Intensive care Unit (ICU) admission decisions involve collaboration between internists and intensivists. Clear perception of each other's roles is a prerequisite for good collaboration. The objective was to explore how internists and intensivists perceive their roles during admission decisions.

METHODS

Individual in-depth interviews with 12 intensivists and 12 internists working at a Swiss teaching hospital. Interviews were analyzed using a thematic approach.

RESULTS

Roles could be divided into practical roles and identity roles. Internist and intensivists had the same perception of each other's practical roles. Internists' practical roles were: recognizing signs of severity when the patient becomes acutely ill, calling the intensivist at the right moment, having the relevant information about the patient and having determined the goals of care. Intensivists' practical roles were: assessing the patient on the ward, giving expert advice, making quick decisions, managing access to the ICU, having the final decision power and, sometimes, deciding whether or not to limit treatment. In complex situations, perceived flaws in performing practical roles could create tensions between the doctors. Intensivists' identity roles included those of leader, gatekeeper, life-death decision maker, and supporting colleague doctors (consultant, senior and helper). These roles could be perceived as emotionally burdensome. Internists' identity roles were those of leader and partner.

CONCLUSIONS

Despite a common perception of each other's practical roles, tensions can arise between internists and intensivists in complex situations of ICU admission decisions. Training in communication skills and interprofessional education interventions aimed at a better understanding of each other roles would improve collaboration.

Clinical and radiographic predictors of acute compartment syndrome in the treatment of tibial plateau fractures: a retrospective cohort study

Background

The aim of the study was to evaluate the relation between demographic, injury-related, clinical and radiological factors of patients with tibial plateau fractures and the development of acute compartment syndrome.

Methods

All consecutive adult patients with intra-articular tibial plateau fractures admitted in our urban academic medical centre between January 2005 and December 2009 were included in this retrospective cohort study. The main outcome measurement was the development of acute compartment syndrome.

Results

The charts of 265 patients (mean age 48.6 years) sustaining 269 intra-articular tibial plateau fractures were retrospectively reviewed. Acute compartment syndrome occurred in 28 fractures (10.4%). Four patients presented bilateral tibial plateau fractures; of them, 2 had unilateral, but none had bilateral acute compartment syndrome. Non-contiguous tibia fracture or knee dislocation and higher AO/OTA classification (type 41-C) were statistically significantly associated with the development of acute compartment syndrome in multivariable regression analysis, while younger age (<45 years), male sex, higher Schatzker grade (IV-V-VI), higher tibial widening ratio (≥1.05) and higher femoral displacement ratio (≥0.08) were significantly associated in the analysis adjusted for age and sex.

Conclusions

Two parameters related to the occurrence of ACS in tibial plateau fractures were highlighted in this study: the presence of a non-contiguous tibia fracture or knee dislocation, and higher AO/OTA classification. They may be especially useful when clinical findings are difficult to assess (doubtful clinical signs, obtunded, sedated or intubated patients), and should rise the suspicion level of the treating surgeon. In these cases, regular clinical examinations and/or intra-compartmental pressure measurements should be performed before and after surgery, even if acute compartment syndrome seemed unlikely during initial assessment. However, larger studies are mandatory to confirm and refine both factors in predicting the occurrence of acute compartment syndrome.

Thinker, Soldier, Scribe: cross-sectional study of researchers' roles and author order in the Annals of Internal Medicine

OBJECTIVE

How researchers' contributions relate to author order on the byline remains unclear. We sought to identify researchers' contributions associated with author order, and to explore the existence of author profiles.

DESIGN

Observational study.

SETTING

Published record.

PARTICIPANTS

1139 authors of 119 research articles published in 2015 in the Annals of Internal Medicine.

PRIMARY OUTCOMES

Presence or absence of 10 contributions, reported by each author, published in the journal.

RESULTS

On average, first authors reported 7.1 contributions, second authors 5.2, middle authors 4.0, penultimate authors 4.5 and last authors 6.4 (p<0.001). The first author made the greatest contributions to drafting the article, designing the study, analysing and interpreting the data, and providing study materials or patients. The second author contributed to data analysis as well and to drafting the article. The last author was most involved in obtaining the funding, critically revising the article, designing the study and providing support. Factor analysis yielded three author profiles-Thinker (study design, revision of article, obtaining funding), Soldier (providing material or patients, providing administrative and logistical support, collecting data) and Scribe (analysis and interpretation of data, drafting the article, statistical expertise). These profiles do not strictly correspond to byline position.

CONCLUSIONS

First, second and last authors of research articles made distinct contributions to published research. Three authorship profiles can be used to summarise author contributions. These findings shed light on the organisation of clinical research teams and may help researchers discuss, plan and report authorship in a more transparent way.

Kappa statistic to measure agreement beyond chance in free-response assessments

Background

The usual kappa statistic requires that all observations be enumerated. However, in free-response assessments, only positive (or abnormal) findings are notified, but negative (or normal) findings are not. This situation occurs frequently in imaging or other diagnostic studies. We propose here a kappa statistic that is suitable for free-response assessments.

Method

We derived the equivalent of Cohen’s kappa statistic for two raters under the assumption that the number of possible findings for any given patient is very large, as well as a formula for sampling variance that is applicable to independent observations (for clustered observations, a bootstrap procedure is proposed). The proposed statistic was applied to a real-life dataset, and compared with the common practice of collapsing observations within a finite number of regions of interest.

Results

The free-response kappa is computed from the total numbers of discordant (b and c) and concordant positive (d) observations made in all patients, as 2d/(b + c + 2d). In 84 full-body magnetic resonance imaging procedures in children that were evaluated by 2 independent raters, the free-response kappa statistic was 0.820. Aggregation of results within regions of interest resulted in overestimation of agreement beyond chance.

Conclusions

The free-response kappa provides an estimate of agreement beyond chance in situations where only positive findings are reported by raters.

Determinants of the calibration of SAPS II and SAPS 3 mortality scores in intensive care: a European multicenter study

Background

The aim of the Simplified Acute Physiology Score (SAPS) II and SAPS 3 is to predict the mortality of patients admitted to intensive care units (ICUs). Previous studies have suggested that the calibration of these scores may vary across countries, centers, and/or characteristics of patients. In the present study, we aimed to assess determinants of the calibration of these scores.

Methods

We assessed the calibration of the SAPS II and SAPS 3 scores among 5266 patients admitted to ICUs during a 4-week period at 120 centers in 17 European countries. We obtained calibration curves, Brier scores, and standardized mortality ratios. Points attributed to SAPS items were reevaluated and compared with those of the original scores. Finally, we tested associations between the calibration and center characteristics.

Results

The mortality was overestimated by both scores: The standardized mortality ratios were 0.75 (95% CI 0.71–0.79) for the SAPS II score and 0.91 (95% CI 0.86–0.96) for the SAPS 3 score. This overestimation was partially explained by changes in associations between some items of the scores and mortality, especially the heart rate, Glasgow Coma Scale score, and diagnosis of AIDS for SAPS II. The calibration of both scores was better in countries with low health expenditures. The between-center variability in calibration curves was much greater than expected by chance.

Conclusions

Both scores overestimate current mortality among European ICU patients. The magnitude of the miscalibration of SAPS II and SAPS 3 scores depends not only on patient characteristics but also on center characteristics. Furthermore, much between-center variability in calibration remains unexplained by these factors.

Trial registration

ClinicalTrials.gov identifier: NCT01422070. Registered 19 August 2011.

Electronic supplementary material

The online version of this article (doi:10.1186/s13054-017-1673-6) contains supplementary material, which is available to authorized users.

Public Support for Social Financing of Health Care in Switzerland

The purpose of this study was to identify factors associated with the public's preference for financing health care according to people's ability to pay. The authors compared voters' support in 26 Swiss cantons for a legislative proposal to replace regionally rated health insurance premiums (current system) with premiums proportional to income and wealth, and co-financed through the value added tax. The vote took place in May 2003, and the initiative was rejected, with only 27 percent of support nationwide. However, support varied more than threefold, from 13 to 44 percent, among cantons. In multivariate analysis, support was most strongly correlated with the approval rate of the 1994 law on health insurance, which strengthened solidarity between the sick and the healthy. More modest associations were seen between support for the initiative and the health insurance premium of 2003, and proportions of elderly and urban residents in the population. Hence support for more social financing of health care was best explained by past preference for a social health insurance system in the local community.

Health utility indexes in patients with acute coronary syndromes

Background

Acute coronary syndromes (ACS) have been associated with lower health utilities (HUs) compared with the general population. Given the prognostic improvements after ACS with the implementation of coronary angiography (eg, percutaneous coronary intervention (PCI)), contemporary HU values derived from patient-reported outcomes are needed.

Methods

We analysed data of 1882 patients with ACS 1 year after coronary angiography in a Swiss prospective cohort. We used the EuroQol five-dimensional questionnaire (EQ-5D) and visual analogue scale (VAS) to derive HU indexes. We estimated the effects of clinical factors on HU using a linear regression model and compared the observed HU with the average values of individuals of the same sex and age in the general population.

Results

Mean EQ-5D HU 1-year after coronary angiography for ACS was 0.82 (±0.16) and mean VAS was 0.77 (±0.18); 40.9% of participants exhibited the highest utility values. Compared with population controls, the mean EQ-5D HU was similar (expected mean 0.82, p=0.58) in patients with ACS, but the mean VAS was slightly lower (expected mean 0.79, p<0.001). Patients with ACS who are younger than 60 years had lower HU than the general population (<0.001). In patients with ACS, significant differences were found according to the gender, education and employment status, diabetes, obesity, heart failure, recurrent ischaemic or incident bleeding event and participation in cardiac rehabilitation (p<0.01).

Conclusions

At 1 year, patients with ACS with coronary angiography had HU indexes similar to a control population. Subgroup analyses based on patients' characteristics and further disease-specific instruments could provide better sensitivity for detecting smaller variations in health-related quality of life.

Patient enrollment and logistical problems top the list of difficulties in clinical research: a cross-sectional survey

Background

Many medical research projects encounter difficulties. The objective of this study was to assess the self-reported frequency of difficulties encountered by medical researchers while conducting research and to identify factors associated with their occurrence.

Methods

The authors conducted a cross-sectional survey in 2010 among principal investigators of 996 study protocols approved by the Research Ethics Committee in Geneva, Switzerland, between 2001 and 2005. The authors asked principal investigators to rate the level of difficulty (1: none, to 5: very great) encountered across the research process.

Results

588 questionnaires were sent back (participation rate 59.0 %). 391 (66.5 %) studies were completed at the time of the survey. Investigators reported that the most frequent difficulties were related to patient enrollment (44.3 %), data collection (26.7 %), data analysis and interpretation (21.5 %), collaboration with caregivers (21.0 %), study design (20.4 %), publication in peer-reviewed journal (20.2 %), hiring of competent study personnel (20.2 %), and getting funding (19.2 %). On average, investigators reported 2.8 difficulties per project (SD 2.8, range 0 to 12). In multivariable analysis, the number of difficulties was higher for studies initiated by public sponsors (vs. private), single center studies (vs. multicenter), and studies about treatment, diagnosis or prognosis (i.e., clinical vs. other studies).

Conclusions

Medical researchers reported substantial logistical difficulties in conducting clinical research.

Electronic supplementary material

The online version of this article (doi:10.1186/s12874-016-0151-1) contains supplementary material, which is available to authorized users.

Normality and Sample Size Do Not Matter for the Selection of an Appropriate Statistical Test for Two-Group Comparisons

Abstract. Many applied researchers are taught to use the t-test when distributions appear normal and/or sample sizes are large and non-parametric tests otherwise, and fear inflated error rates if the “wrong” test is used. In a simulation study (four tests: t-test, Mann-Whitney test, Robust t-test, Permutation test; seven sample sizes between 2 × 10 and 2 × 500; four distributions: normal, uniform, log-normal, bimodal; under the null and alternate hypotheses), we show that type 1 errors are well controlled in all conditions. The t-test is most powerful under the normal and the uniform distributions, the Mann-Whitney test under the lognormal distribution, and the robust t-test under the bimodal distribution. Importantly, even the t-test was more powerful under asymmetric distributions than under the normal distribution for the same effect size. It appears that normality and sample size do not matter for the selection of a test to compare two groups of same size and variance. The researcher can opt for the test that fits the scientific hypothesis the best, without fear of poor test performance.

Doctors' decisions when faced with contradictory patient advance directives and health care proxy opinion: a randomized vignette-based study

CONTEXT

Sometimes a written advance directive contradicts the opinion of a health care proxy. How this affects doctors' decision making is unknown.

OBJECTIVES

To quantify the influence of contradictory instructions on doctors' decisions.

METHODS

All the generalists and internists in French-speaking Switzerland were mailed the questionnaire. Respondents (43.5%) evaluated three vignettes that described medical decisions for incapacitated patients. Each vignette was produced in four versions: one with an advance directive, one with a proxy opinion, one with both, and one with neither (control). In the first vignette, the directive and proxy agreed on the recommendation to forgo a medical intervention; in the second, the advance directive opposed, but the proxy favored the intervention; and in the third, the roles were reversed. Each doctor received one version of each vignette, attributed at random. The outcome variables were the doctor's decision to forgo the medical intervention and the rating of the decision as difficult.

RESULTS

Written advance directives and proxy opinions significantly influenced doctors' decision making. When both were available and concordant, they reinforced each other (odds ratio [OR] of forgoing intervention 35.7, P < 0.001 compared with no instruction). When the directive and proxy disagreed, the resulting effect was to forgo the intervention (ORs 2.1 and 2.2 for the two discordant vignettes, both P < 0.001). Discordance between instructions was associated with increased odds of doctors rating the decision as difficult (both ORs 2.0, P ≤ 0.001).

CONCLUSION

Contradictions between advance directives and proxy opinions result in a weak preference for abstention from treatment and increase the difficulty of the decision.

Comparison of noninferiority margins reported in protocols and publications showed incomplete and inconsistent reporting

OBJECTIVES

To compare noninferiority margins defined in study protocols and trial registry records with margins reported in subsequent publications.

STUDY DESIGN AND SETTING

Comparison of protocols of noninferiority trials submitted 2001 to 2005 to ethics committees in Switzerland and The Netherlands with corresponding publications and registry records. We searched MEDLINE via PubMed, the Cochrane Controlled Trials Register (Cochrane Library issue 01/2012), and Google Scholar in September 2013 to identify published reports, and the International Clinical Trials Registry Platform of the World Health Organization in March 2013 to identify registry records. Two readers recorded the noninferiority margin and other data using a standardized data-abstraction form.

RESULTS

The margin was identical in study protocol and publication in 43 (80%) of 54 pairs of study protocols and articles. In the remaining pairs, reporting was inconsistent (five pairs, 9%), or the noninferiority margin was either not reported in the publication (five pairs, 9%) or not defined in the study protocol (one pair). The confidence interval or the exact P-value required to judge whether the result was compatible with noninferior, inferior, or superior efficacy was reported in 43 (80%) publications. Complete and consistent reporting of both noninferiority margin and confidence interval (or exact P-value) was present in 39 (72%) protocol-publication pairs. Twenty-nine trials (54%) were registered in trial registries, but only one registry record included the noninferiority margin.

CONCLUSION

The reporting of noninferiority margins was incomplete and inconsistent with study protocols in a substantial proportion of published trials, and margins were rarely reported in trial registries.

Validation of a 15-item care-related regret coping scale for health-care professionals (RCS-HCP)

OBJECTIVES

Coping with difficult care-related situations is a common challenge for health-care professionals. How these professionals deal with the regrets they may experience following one of the many decisions and interventions they must make every day can have an impact on their own health and quality of life, and also on their patient care practices. To identify professionals most at need for extra support, development and validation of a tool measuring coping style are needed.

METHODS

We performed a survey of physicians and nurses of a French-speaking University hospital; 469 health-care professionals responded to the survey, and 175 responded to the same survey one-month later. Regret was assessed with the regret coping scale developed for this study, self-report questions on the frequency of regretted situations and the intensity of regret. Construct validity was assessed using measures of health-care professionals' quality of life (including job and life satisfaction, and self-reported health) as well as sleep problems and depression.

RESULTS

Based on factor analysis and item response analysis, the initial 31-item scale was shortened to 15 items, which measured three types of strategies: problem-focused strategies (i.e., trying to find solutions, talking to colleagues) and two types of emotion-focused strategies, A (i.e., self-blame, rumination) and B (e.g., acceptance, emotional distance). All subscales showed high internal consistency (α >0.85). Overall, as expected, problem-focused and emotion-focused B strategies correlated with higher quality of life, fewer sleep problems and less depression, and emotion-focused A strategies showed the opposite pattern.

CONCLUSIONS

The regret coping scale (RCS-HCP) is a valid and reliable measure of coping abilities of hospital-based health-care professionals.

Sample size for pre-tests of questionnaires

PURPOSE

To provide guidance regarding the desirable size of pre-tests of psychometric questionnaires, when the purpose of the pre-test is to detect misunderstandings, ambiguities, or other difficulties participants may encounter with instrument items (called «problems»).

METHODS

We computed (a) the power to detect a problem for various levels of prevalence and various sample sizes, (b) the required sample size to detect problems for various levels of prevalence, and (c) upper confidence limits for problem prevalence in situations where no problems were detected.

RESULTS

As expected, power increased with problem prevalence and with sample size. If problem prevalence was 0.05, a sample of 10 participants had only a power of 40 % to detect the problem, and a sample of 20 achieved a power of 64 %. To achieve a power of 80 %, 32 participants were necessary if the prevalence of the problem was 0.05, 16 participants if prevalence was 0.10, and 8 if prevalence was 0.20. If no problems were observed in a given sample, the upper limit of a two-sided 90 % confidence interval reached 0.26 for a sample size of 10, 0.14 for a sample size of 20, and 0.10 for a sample of 30 participants.

CONCLUSIONS

Small samples (5-15 participants) that are common in pre-tests of questionaires may fail to uncover even common problems. A default sample size of 30 participants is recommended.

What should be the basis for compulsory and optional health insurance premiums? Opinions of Swiss doctors

PRINCIPLES

Little is known about doctors' opinions on how to finance health services. In Switzerland, mandatory basic health insurance currently uses regional flat fees that are unrelated to health and ability to pay, and optional complementary insurance uses risk-based premiums. Our objective was to assess Swiss physicians' opinions on what should determine health insurance premiums.

METHODS

We surveyed doctors in the canton of Geneva, Switzerland, about the desirable funding mechanism for mandatory health insurance and complementary health insurance. The proposed determinants of insurance premiums were current health and past medical history, lifestyle, healthcare costs in the previous year, genetic susceptibility to disease, regional average healthcare costs, household income, and wealth and demographic characteristics.

RESULTS

Among the 1,516 respondents, only a few (<5%) believed that the mandatory health insurance premium should depend on health risk (health status, previous costs, genetics, and age and sex). More than 30% of respondents supported premiums based on lifestyle (34.6%), regional average health expenditures (31.2%), and household income and wealth (39.6%). For complementary health insurance, most respondents supported premiums based on lifestyle (74.6%) and on health risk (46.4%), but surprisingly also on household income and wealth (44.9%) and regional average health expenditures (39.4%). The characteristic most influencing the answers was the medical specialty.

CONCLUSION

Doctors' opinions about healthcare financing mechanisms varied considerably, for both mandatory and complementary health insurance. Lifestyle was a surprisingly frequent choice, even though this criterion is not currently used in Switzerland. Ability to pay was not supported by the majority.

Effect of numbering of return envelopes on participation, explicit refusals, and bias: experiment and meta-analysis

BACKGROUND

Tracing mail survey responses is useful for the management of reminders but may cause concerns about anonymity among prospective participants. We examined the impact of numbering return envelopes on the participation and the results of a survey on a sensitive topic among hospital staff.

METHODS

In a survey about regrets associated with providing healthcare conducted among hospital-based doctors and nurses, two randomly drawn subsamples were provided numbered (N = 1100) and non-numbered (N = 500) envelopes for the return of completed questionnaires. Participation, explicit refusals, and item responses were compared. We also conducted a meta-analysis of the effect of questionnaire/envelope numbering on participation in health surveys.

RESULTS

The participation rate was lower in the "numbered" group than in the "non-numbered" group (30.3% vs. 35.0%, p = 0.073), the proportion of explicit refusals was higher in the "numbered" group (23.1% vs 17.5%, p = 0.016), and the proportion of those who never returned the questionnaire was similar (46.6% vs 47.5%, p = 0.78). The means of responses differed significantly for 12 of 105 items (11.4%), which did not differ significantly from the expected frequency of type 1 errors, i.e., 5% (permutation test, p = 0.078). The meta-analysis of 7 experimental surveys (including this one) indicated that numbering is associated with a 2.4% decrease in the survey response rate (95% confidence interval 0.3% to 4.4%).

CONCLUSIONS

Numbered return envelopes may reduce the response rate and increase explicit refusals to participate in a sensitive survey. Reduced participation was confirmed by a meta-analysis of randomized health surveys. There was no strong evidence of bias.

Impact of advance directives and a health care proxy on doctors' decisions: a randomized trial

CONTEXT

Advance directives or proxy designations are widely recommended, but how they affect doctors' decision making is not well known.

OBJECTIVES

The aim of this study was to quantify the influence of advance directives and proxy opinions on doctors' decisions.

METHODS

We mailed to all the generalists and internists in French-speaking Switzerland (N = 1962) three vignettes describing difficult decisions involving incapacitated patients. In each case, the advance directive requested that further care be withheld. One vignette tested the impact of a written advance directive vs. a proxy. Another compared the impact of a handwritten directive vs. a formalized document. The third vignette compared the impact of a family member vs. a doctor as a proxy. Each vignette was prepared in three or four versions, including a control version in which no directive or proxy was present. Vignettes were randomly allocated to respondents. We used logistic regression to predict the decision to forgo a medical intervention.

RESULTS

Compared with the control condition, the odds of forgoing a medical intervention were increased by the written advance directive (odds ratio [OR] 7.3; P < 0.001), the proxy (OR 7.9; P < 0.001), and the combination of the two (OR 35.7; P < 0.001). The handwritten directive had the same impact (OR 13.3) as the formalized directive (OR 13.8). The effect of proxy opinion was slightly stronger when provided by a doctor (OR 11.3) rather than by family (OR 7.8).

CONCLUSION

Advance directives and proxy opinions are equally effective in influencing doctors' decisions, but having both has the strongest effect. The format of the advance directive and the identity of the proxy have little influence on decisions.

Internal consistency, factor structure and construct validity of the French version of the Hospital Survey on Patient Safety Culture

OBJECTIVE

To assess the psychometric properties of the French-language version of the Hospital Survey on Patient Safety Culture (HSOPSC).

METHODS

Data were obtained from a staff survey at a Swiss multisite hospital. We computed descriptive statistics and internal consistency coefficients, then conducted a confirmatory and exploratory factor analysis, and performed construct validity tests.

RESULTS

1171 staff members participated (response rate 74%). The internal consistency coefficients of the 12 dimension scores ranged from 0.57 to 0.86 (median 0.73). Confirmatory factor analysis indicated a reasonable but not perfect fit of the hypothesised measurement model (root mean square error of approximation 0.043, comparative fit index 0.89). Exploratory data analysis suggested 10 dimensions instead of 12, grouping items from teamwork across hospital units with those of hospital handoffs and transitions, and items from communication openness with those of feedback and communication about error. However, the loading pattern was clean: 41 of 42 main loadings exceeded 0.40, and only 3 of 378 cross-loadings exceeded 0.30. All 10 process scores were higher among respondents who rated the global safety grade as 'excellent' or 'very good' rather than 'good', 'fair' or 'poor' (effect sizes 0.41-0.79, all p<0.001), but score differences between those who have and have not reported an incident in the past year were weak or inconsistent with theory.

DISCUSSION

The French version of the HSOPSC did not perform as well as the original in standard psychometric analyses.

Self-rated health: analysis of distances and transitions between response options

PURPOSE

We explored health differences between population groups who describe their health as excellent, very good, good, fair, or poor.

METHODS

We used data from a population-based survey which included self-rated health (SRH) and three global measures of health: the SF36 general health score (computed from the 4 items other than SRH), the EQ-5D health utility, and a visual analogue health thermometer. We compared health characteristics of respondents across the five health ratings.

RESULTS

Survey respondents (N = 1.844, 49.2 % response) rated their health as excellent (12.2 %), very good (39.1 %), good (41.9 %), fair (6.0 %), or poor (0.9 %). The means of global health assessments were not equidistant across these five groups, for example, means of the health thermometer were 95.8 (SRH excellent), 88.8 (SRH very good), 76.6 (SRH good), 49.7 (SRH fair), and 33.5 (SRH poor, p < 0.001). Recoding the SRH to reflect these mean values substantially improved the variance explained by the SRH, for example, the linear r (2) increased from 0.50 to 0.56 for the health thermometer if the SRH was coded as poor = 1, fair = 2, good = 3.7, very good = 4.5, and excellent = 5. Furthermore, transitions between response options were not explained by the same health-related characteristics of the respondents.

CONCLUSIONS

The adjectival SRH is not an evenly spaced interval scale. However, it can be turned into an interval variable if the ratings are recoded in proportion to the underlying construct of health. Possible improvements include the addition of a rating option between good and fair or the use of a numerical scale instead of the classic adjectival scale.

Citation bias favoring statistically significant studies was present in medical research

OBJECTIVE

Statistically significant studies may be cited more than negative studies on the same topic. We aimed to assess here whether such citation bias is present across the medical literature.

STUDY DESIGN AND SETTING

We conducted a cohort study of the association between statistical significance and citations. We selected all therapeutic intervention studies included in meta-analyses published between January and March 2010 in the Cochrane database, and retrieved citation counts of all individual studies using ISI Web of Knowledge. The association between the statistical significance of each study and the number of citations it received between 2008 and 2010 was assessed in mixed Poisson models.

RESULTS

We identified 89 research questions addressed in 458 eligible articles. Significant studies were cited twice as often as nonsignificant studies (multiplicative effect of significance: 2.14, 95% confidence interval: 1.38-3.33). This association was partly because of the higher impact factor of journals where significant studies are published (adjusted multiplicative effect of significance: 1.14, 95% confidence interval: 0.87-1.51).

CONCLUSION

A citation bias favoring significant results occurs in medical research. As a consequence, treatments may seem more effective to the readers of medical literature than they really are.

Meta-analysis of two-arm studies: Modeling the intervention effect from survival probabilities

Pooling the hazard ratios is not always feasible in meta-analyses of two-arm survival studies, because the measure of the intervention effect is not systematically reported. An alternative approach proposed by Moodie et al. is to use the survival probabilities of the included studies, all collected at a single point in time: the intervention effect is then summarised as the pooled ratio of the logarithm of survival probabilities (which is an estimator of the hazard ratios when hazards are proportional). In this article, we propose a generalization of this method. By using survival probabilities at several points in time, this generalization allows a flexible modeling of the intervention over time. The method is applicable to partially proportional hazards models, with the advantage of not requiring the specification of the baseline survival. As in Moodie et al.'s method, the study-level factors modifying the survival functions can be ignored as long as they do not modify the intervention effect. The procedures of estimation are presented for fixed and random effects models. Two illustrative examples are presented.

Testing the WHO Hand Hygiene Self-Assessment Framework for usability and reliability

BACKGROUND

The World Health Organization (WHO) Hand Hygiene Self-Assessment Framework (HHSAF) was conceived as a structured self-assessment tool to provide a situation analysis of hand hygiene resources, promotion and practices within healthcare facilities.

AIM

To perform usability pretesting and reliability testing of the HHSAF.

METHODS

The HHSAF draft was developed in consultation with experts to reflect key elements of the WHO Multimodal Hand Hygiene Improvement Strategy. Forty-two facilities were invited to pretest the draft HHSAF and complete a feedback survey. For reliability testing, two users in each facility completed the HHSAF independently. The reliability of each indicator, component subtotal and the overall score were estimated using the variance components model. After each phase, the tool was re-examined and modified as appropriate.

FINDINGS

Twenty-seven indicators were selected during drafting. Twenty-six facilities in 19 countries completed pretesting (62% response rate), with total scores ranging from 35 to 480 (mean 262). The HHSAF took less than 2 h to complete for 21 facilities. Most agreed that the HHSAF was 'easy to use' (23/26) and 'useful for establishing facility status with regard to hand hygiene promotion' (24/26). Complete reliability responses were received from 41 facilities in 16 countries. Reliability for the total score for the HHSAF and the subtotal of each of the five components ranged from 0.54 to 0.86. Seven indicators had poor reliability; these were examined for potential flaws and modified accordingly.

CONCLUSION

This process confirmed the usability and reliability of this tool for the promotion of hand hygiene in health care.

Sensitivity, specificity and likelihood ratios of PCR in the diagnosis of syphilis: a systematic review and meta-analysis

OBJECTIVE

To systematically review and estimate pooled sensitivity and specificity of the polymerase chain reaction (PCR) technique compared to recommended reference tests in the diagnosis of suspected syphilis at various stages and in various biological materials.

DESIGN

Systematic review and meta-analysis.

DATA SOURCES

Search of three electronic bibliographic databases from January 1990 to January 2012 and the abstract books of five congresses specialized in the infectious diseases' field (1999-2011). Search key terms included syphilis, Treponema pallidum or neurosyphilis and molecular amplification, polymerase chain reaction or PCR.

REVIEW METHODS

We included studies that used both reference tests to diagnose syphilis plus PCR and we presented pooled estimates of PCR sensitivity, specificity, and positive and negative likelihood ratios (LR) per syphilis stages and biological materials.

RESULTS

Of 1160 identified abstracts, 69 were selected and 46 studies used adequate reference tests to diagnose syphilis. Sensitivity was highest in the swabs from primary genital or anal chancres (78.4%; 95% CI: 68.2-86.0) and in blood from neonates with congenital syphilis (83.0%; 55.0-95.2). Most pooled specificities were ∼95%, except those in blood. A positive PCR is highly informative with a positive LR around 20 in ulcers or skin lesions. In the blood, the positive LR was <10.

CONCLUSIONS

The pooled values of LR showed that T. pallidum PCR was more efficient to confirm than to exclude syphilis diagnosis in lesions. PCR is a useful diagnostic tool in ulcers, especially when serology is still negative and in medical settings with a high prevalence of syphilis.

Increasing occurrence of atypical femoral fractures associated with bisphosphonate use

BACKGROUND

Current evidence suggests that there is an association between bisphosphonate therapy and atypical femoral fractures, but the extent of this risk remains unclear.

METHODS

Between 1999 and 2010, a total of 477 patients 50 years and older were hospitalized with a subtrochanteric or femoral shaft fracture at a single university medical center. Admission radiographs and medical and treatment records were examined, and patients were classified as having atypical or classic femoral fractures. A random sample of 200 healthy individuals without femoral fracture were also identified. Multivariate logistic regression was used to assess the association of bisphosphonate use and atypical femoral fracture, and the incidence rates of each type of fracture over time were calculated.

RESULTS

Thirty-nine patients with atypical fractures and 438 patients with classic fractures were identified. Of the patients with atypical fractures, 32 (82.1%) had been treated with bisphosphonates compared with 28 (6.4%) in the classic fractures group (odds ratios [OR], 66.9; 95% CI, 27.1-165.1) and 11.5% in the group without fracture (OR, 35.2; 95% CI, 13.9-88.8). Bisphosphonate use was associated with a 47% reduction in risk of classic fracture (OR, 0.5; 95% CI, 0.3-0.9). Considering the duration of use, the ORs (95% CIs) for atypical fractures were 35.1 (10.0-123.6) for less than 2 years, 46.9 (14.2-154.4) for 2 to 5 years, 117.1 (34.2-401.7) for 5 to 9 years, and 175.7 (30.0-1027.6) for more than 9 years compared with no use. A contralateral fracture occurred in 28.2% of atypical cases and in 0.9% of classic cases (OR, 42.6; 95% CI, 12.8-142.4). The incidence rate of atypical fractures was low (32 cases per million person-years) and increased by 10.7% per year on average.

CONCLUSIONS

Atypical femoral fractures were associated with bisphosphonate use; longer duration of treatment resulted in augmented risk. The incidence of atypical fractures increased over a 12-year period, but the absolute number of such fractures is very small.

Sensitivity and predictive value of 15 PubMed search strategies to answer clinical questions rated against full systematic reviews

Background

Clinicians perform searches in PubMed daily, but retrieving relevant studies is challenging due to the rapid expansion of medical knowledge. Little is known about the performance of search strategies when they are applied to answer specific clinical questions.

Objective

To compare the performance of 15 PubMed search strategies in retrieving relevant clinical trials on therapeutic interventions.

Methods

We used Cochrane systematic reviews to identify relevant trials for 30 clinical questions. Search terms were extracted from the abstract using a predefined procedure based on the population, interventions, comparison, outcomes (PICO) framework and combined into queries. We tested 15 search strategies that varied in their query (PIC or PICO), use of PubMed’s Clinical Queries therapeutic filters (broad or narrow), search limits, and PubMed links to related articles. We assessed sensitivity (recall) and positive predictive value (precision) of each strategy on the first 2 PubMed pages (40 articles) and on the complete search output.

Results

The performance of the search strategies varied widely according to the clinical question. Unfiltered searches and those using the broad filter of Clinical Queries produced large outputs and retrieved few relevant articles within the first 2 pages, resulting in a median sensitivity of only 10%–25%. In contrast, all searches using the narrow filter performed significantly better, with a median sensitivity of about 50% (all P < .001 compared with unfiltered queries) and positive predictive values of 20%–30% (P < .001 compared with unfiltered queries). This benefit was consistent for most clinical questions. Searches based on related articles retrieved about a third of the relevant studies.

Conclusions

The Clinical Queries narrow filter, along with well-formulated queries based on the PICO framework, provided the greatest aid in retrieving relevant clinical trials within the 2 first PubMed pages. These results can help clinicians apply effective strategies to answer their questions at the point of care.

Pain as an important predictor of psychosocial health in patients with rheumatoid arthritis

OBJECTIVE

To examine the evolution of psychosocial aspects of health-related quality of life in rheumatoid arthritis (RA) patients, and to identify their predictors.

METHODS

All patients within a Swiss RA cohort and a US RA cohort who completed a Short Form 36 (SF-36) scale at least twice within a 4-year period were included. The primary outcome was psychosocial health as measured by the mental component summary (MCS) score of the SF-36. The evolution of this outcome over time was analyzed using structural equation models, which distinguish between the stable, the variable, and the measurement error components of the outcome's variance.

RESULTS

A total of 15,282 patients (48,323 observations) were included. MCS scores were mostly stable over time (between 69% and 75% of the variance was not due to measurement error). The variable component of the SF-36 was mostly due to fluctuations at the moment of measurement and not to a global time trend of psychosocial health. Pain was the most important predictor of both the stable and variable components of psychosocial health, explaining ∼44% of the observed psychosocial health variance.

CONCLUSION

This large cohort study demonstrates that pain is the most important predictor of a patient's psychosocial health in RA patients. This suggests that physicians should place greater emphasis on pain management.

Doctors and patients' susceptibility to framing bias: a randomized trial

BACKGROUND

Framing of risk influences the perceptions of treatment benefit.

OBJECTIVE

To determine which risk framing format corresponds best to comprehensive multi-faceted information, and to compare framing bias in doctors and in patients.

DESIGN

Randomized mail surveys.

PARTICIPANTS

One thousand four hundred and thirty-one doctors (56% response rate) and 1121 recently hospitalized patients (65% response rate).

INTERVENTION

Respondents were asked to interpret the results of a hypothetical clinical trial comparing an old and a new drug. They were randomly assigned to the following framing formats: absolute survival (new drug: 96% versus old drug: 94%), absolute mortality (4% versus 6%), relative mortality reduction (reduction by a third) or all three (fully informed condition). The new drug was reported to cause more side-effects.

MAIN MEASURE

Rating of the new drug as more effective than the old drug.

RESULTS

The proportions of doctors who rated the new drug as more effective varied by risk presentation format (abolute survival 51.8%, absolute mortality 68.3%, relative mortality reduction 93.8%, and fully informed condition 69.8%, p < 0.001). In patients these proportions were similar (abolute survival 51.7%, absolute mortality 66.8%, relative mortality reduction 89.3%, and fully informed condition 71.2%, p < 0.001). In both doctors (p = 0.72) and patients (p = 0.23) the fully informed condition was similar to the absolute risk format, but it differed significantly from the other conditions (all p < 0.01). None of the differences between doctors and patients were significant (all p > 0.1). In comparison to the fully informed condition, the odds ratio of greater perceived effectiveness was 0.45 for absolute survival (p < 0.001), 0.89 for absolute mortality (p = 0.29), and 4.40 for relative mortality reduction (p < 0.001).

CONCLUSIONS

Framing bias affects doctors and patients similarly. Describing clinical trial results as absolute risks is the least biased format, for both doctors and patients. Presenting several risk formats (on both absolute and relative scales) should be encouraged.

Use of brain natriuretic peptide to detect previously unknown left ventricular dysfunction in patients with acute exacerbation of chronic obstructive pulmonary disease

BACKGROUND

Up to 30% of patients with chronic obstructive pulmonary disease (COPD) simultaneously suffer from often-unrecognised chronic heart failure (HF). Their timely identification is challenging as both conditions share similar clinical presentations.

OBJECTIVE

To assess the performance of BNP in detecting left ventricular systolic dysfunction in patients with no history of HF admitted for acute exacerbation of COPD in a regional second-care hospital.

METHODS

Retrospective medical records analysis of all patients hospitalised between January 2003 and May 2009 with the final diagnosis of acute exacerbation of COPD, and who had undergone BNP dosage at admission followed by an echocardiography.

RESULTS

Among the 57 patients included, 13 had left ventricular systolic dysfunction. There was a statistically significant difference of mean BNP values between patients with or without systolic dysfunction (mean 689 pg/ml vs. 340 pg/ml, p = 0.007). For the detection of systolic dysfunction, a BNP level inferior to 100 pg/ml yielded a sensitivity of 92% and a negative predictive value of 91%, whereas BNP higher than 500 yielded a sensitivity of 80% and a positive predictive value of 47%. In a multivariate logistic regression analysis, a BNP value ≥500 (odds ratio 8.5, 95% confidence interval 1.9 to 38.2, p = 0.005) and history of coronary heart disease (odds ratio 5.9, 95% confidence interval 1.01 to 34.7, p = 0.048) remained as independent and mutually adjusted predictors of left ventricular systolic dysfunction.

CONCLUSIONS

Our study confirms that BNP can help physicians in identifying heart failure in patients suffering from an acute exacerbation of COPD.

Survival predictors for second-line chemotherapy in Caucasian patients with metastatic gastric cancer

PRINCIPLES

There are very limited data suggesting a benefit for second-line chemotherapy in advanced gastric cancer. Therefore, the number of patients who receive further treatment after failure of first-line chemotherapy varies considerably, ranging from 14% to 75%. In the absence of a demonstrated survival benefit of second-line chemotherapy, appropriate selection of patients based on survival predictors is essential. However, no clinico-pathologic parameters are currently widely adopted in clinical practice. We looked exclusively at Caucasian patients with metastatic gastric cancer treated with second-line chemotherapy to see if we could establish prognostic factors for survival.

METHODS

This study retrospectively evaluated 43 Caucasian patients with metastatic gastric cancer treated with second-line chemotherapy at the Geneva University Hospital. Prognostic values of clinico-pathologic parameters were analysed by Cox regression for overall survival (OS).

RESULTS

Univariate analysis found three variables to be associated with survival: progression-free survival (PFS) at first-line chemotherapy of more than 26 weeks (hazard ratio (HR) = 0.33, confidence interval (CI) 95% 0.16-0.65, p = 0.002), previous curative surgery (HR = 0.51, CI 95% 0.27-0.96, p = 0.04) and carcinoma embryonic antigen (CEA) >6.5 μg/l (HR = 1.97, CI 95% 1.06-3.65, p = 0.03).

CONCLUSIONS

In line with published data, sensitivity to previous chemotherapy identifies Caucasian patients who will survive the longest following second-line chemotherapy. A low tumour burden and previous curative gastrectomy also seem to have a positive prognostic value.

Regrets associated with providing healthcare: qualitative study of experiences of hospital-based physicians and nurses

Background

Regret is an unavoidable corollary of clinical practice. Physicians and nurses perform countless clinical decisions and actions, in a context characterised by time pressure, information overload, complexity and uncertainty.

Objective

To explore feelings associated with regretted clinical decisions or interventions of hospital-based physicians and nurses and to examine how these regrets are coped with.

Method

Qualitative study of a volunteer sample of 12 physicians and 13 nurses from Swiss University Hospitals using semi-structured interviews and thematic analysis

Results

All interviewees reported at least one intense regret, which sometimes led to sleep problems, or taking sickness leave. Respondents also reported an accumulation effect of small and large regrets, which sometimes led to quitting one's unit or choosing another specialty. Respondents used diverse ways of coping with regrets, including changing their practices and seeking support from peers and family but also suppression of thoughts related to the situation and ruminations on the situation. Another coping strategy was acceptance of one's limits and of medicine's limits. Physicians reported that they avoided sharing with close colleagues because they felt they could lose their credibility.

Conclusions

Since regret seems related to both positive and negative consequences, it is important to learn more about regret coping among healthcare providers and to determine whether training in coping strategies could help reduce negative consequences such as sleep problems, absenteeism, or turnover.

Numbering questionnaires had no impact on the response rate and only a slight influence on the response content of a patient safety culture survey: a randomized trial

OBJECTIVES

In self-completed surveys, anonymous questionnaires are sometimes numbered so as to avoid sending reminders to initial nonrespondents. This number may be perceived as a threat to confidentiality by some respondents, which may reduce the response rate, or cause social desirability bias. In this study, we evaluated whether using nonnumbered vs. numbered questionnaires influenced the response rate and the response content.

STUDY DESIGN AND SETTING

During a patient safety culture survey, we randomized participants into two groups: one received an anonymous nonnumbered questionnaire and the other a numbered questionnaire. We compared the survey response rates and distributions of the responses for the 42-questionnaire items across the two groups.

RESULTS

Response rates were similar in the two groups (nonnumbered, 75.2%; numbered, 72.8%; difference, 2.4%; P=0.28). Five of the 42 questions had statistically significant differences in distributions, but these differences were small. Unexpectedly, in all five instances, the patient safety culture ratings were more favorable in the nonnumbered group.

CONCLUSION

Numbering of mailed questionnaires had no impact on the response rate. Numbering influenced significantly the response content of several items, but these differences were small and ran against the hypothesis of social desirability bias.

Does information about risks and benefits improve the decision-making process in cancer screening - randomized study

BACKGROUND

Whether the provision of evidence-based information improves satisfaction with decision-making is unclear.

OBJECTIVE

To examine whether information about risks and benefits of cancer screening leads to a higher satisfaction with the decision that was made.

METHODS

Randomized mail survey in the general population, among 2333 adults aged 30-60 years. The survey included a hypothetical cancer screening scenario that included varying amounts of information about benefits and risks of screening (factorial randomized design). The decision process was evaluated by a 6 item scale, with scores between 0 (lowest score) and 100 (highest score).

RESULTS

Substantial proportions of respondents "completely agreed" that the decision reflected what was most important to them (61.2%), were satisfied with their decision (56.0%), were certain of their decision (54.1%), thought that the best choice for them was obvious (53.5%) and that the decision was easy to make (44.1%). The Cronbach alpha coefficient of the scale was 0.88, the mean score was 82.5, and the standard deviation 17.5. Providing information about benefits increased the decision evaluation score only modestly (+1.1, p=0.11); in contrast, providing information about risks sharply reduced the score (-5.1, p<0.001). Those who refused the screening test had lower scores than those who accepted the screening test (69.2 versus 85.6, p<0.001).

CONCLUSIONS

Contrary to expectations, informing potential participants about the risks of cancer screening lowered their assessment of the decision process.

Desire for autonomy in health care decisions: a general population survey

OBJECTIVE

To examine factors associated with desire for autonomy in health care decisions in the general population.

METHODS

Mailed survey of 2348 residents of Geneva, Switzerland. Participants answered questions on a scale measuring their desire for autonomy in health care decisions. The scale was scored between 0 (lowest desire for autonomy) and 100 (highest desire for autonomy).

RESULTS

On average the respondents favoured shared or active involvement in medical decisions (mean score 62.0, SD 20.9), but attitudes varied considerably. In the multivariate model, factors associated with a higher desire for autonomy included female gender, younger age, higher education, living alone, reporting an excellent global health and - a new observation compared to previous studies - having made several medical decisions in the past 6 months.

CONCLUSIONS

The attitudes of the general public appear to be consistent with the model of shared decision making. However, people vary considerably in their desire for autonomy.

PRACTICE IMPLICATIONS

An explicit assessment of each individual's desire for autonomy may improve the decision-making process. Such an assessment should be repeated regularly, as familiarity with medical decisions may increase the desire for autonomy.

Does prevalence matter to physicians in estimating post-test probability of disease? A randomized trial

BACKGROUND

The probability of a disease following a diagnostic test depends on the sensitivity and specificity of the test, but also on the prevalence of the disease in the population of interest (or pre-test probability). How physicians use this information is not well known.

OBJECTIVE

To assess whether physicians correctly estimate post-test probability according to various levels of prevalence and explore this skill across respondent groups.

DESIGN

Randomized trial.

PARTICIPANTS

Population-based sample of 1,361 physicians of all clinical specialties.

INTERVENTION

We described a scenario of a highly accurate screening test (sensitivity 99% and specificity 99%) in which we randomly manipulated the prevalence of the disease (1%, 2%, 10%, 25%, 95%, or no information).

MAIN MEASURES

We asked physicians to estimate the probability of disease following a positive test (categorized as <60%, 60-79%, 80-94%, 95-99.9%, and >99.9%). Each answer was correct for a different version of the scenario, and no answer was possible in the "no information" scenario. We estimated the proportion of physicians proficient in assessing post-test probability as the proportion of correct answers beyond the distribution of answers attributable to guessing.

KEY RESULTS

Most respondents in each of the six groups (67%-82%) selected a post-test probability of 95-99.9%, regardless of the prevalence of disease and even when no information on prevalence was provided. This answer was correct only for a prevalence of 25%. We estimated that 9.1% (95% CI 6.0-14.0) of respondents knew how to assess correctly the post-test probability. This proportion did not vary with clinical experience or practice setting.

CONCLUSIONS

Most physicians do not take into account the prevalence of disease when interpreting a positive test result. This may cause unnecessary testing and diagnostic errors.

Performance of logistic regression modeling: beyond the number of events per variable, the role of data structure

OBJECTIVE

Logistic regression is commonly used in health research, and it is important to be sure that the parameter estimates can be trusted. A common problem occurs when the outcome has few events; in such a case, parameter estimates may be biased or unreliable. This study examined the relation between correctness of estimation and several data characteristics: number of events per variable (EPV), number of predictors, percentage of predictors that are highly correlated, percentage of predictors that were non-null, size of regression coefficients, and size of correlations.

STUDY DESIGN

Simulation studies.

RESULTS

In many situations, logistic regression modeling may pose substantial problems even if the number of EPV exceeds 10. Moreover, the number of EPV is not the only element that impacts on the correctness of parameter estimation. High regression coefficients and high correlations between the predictors may cause large problems in the estimation process. Finally, power is generally very low, even at 20 EPV.

CONCLUSION

There is no single rule based on EPV that would guarantee an accurate estimation of logistic regression parameters. Instead, the number of predictors, probable size of the regression coefficients based on previous literature, and correlations among the predictors must be taken into account as guidelines to determine the necessary sample size.

Patient-reported conformity of informed consent procedures and participation in clinical research

BACKGROUND

There is growing evidence that the quality of informed consent in clinical research is often sub-optimal.

AIMS

To explore the conformity of patient recruitment with recommended informed consent procedures among patients who were invited to participate in a clinical study at a general teaching hospital, and to examine the association between consent procedures and the patients' decision to participate.

DESIGN AND METHODS

All inpatients discharged during a 1-month period were invited to complete a mailed survey in which they reported whether they were invited to participate in a study and whether 13 recommended elements of informed consent actually occurred.

RESULTS

Among 1303 respondents, 265 (20.3%) reported that they had been invited to participate in a study, and 191 (72.1%) accepted. While the majority of potential participants were fully informed about practical issues related to the study (e.g. what their participation would consist in), <50% were informed of possible risks or benefits, and only 20% about the origin of the study funds. Only 60% reported satisfactory answers to items assessing the overall information process (e.g. explanations were easy to understand). Older and sicker patients reported lower levels of conformity with informed consent procedures, as did patients who refused to participate in a study.

CONCLUSION

Our results confirm that informed consent procedures fail to meet standards for many patients. In particular, consent information should be adapted to the needs of older and sicker patients. Improving the quality of informed consent may increase patients' participation in clinical research.

What doctors think about the impact of managed care tools on quality of care, costs, autonomy, and relations with patients

Background

How doctors perceive managed care tools and incentives is not well known. We assessed doctors' opinions about the expected impact of eight managed care tools on quality of care, control of health care costs, professional autonomy and relations with patients.

Methods

Mail survey of doctors (N = 1546) in Geneva, Switzerland. Respondents were asked to rate the impact of 8 managed care tools on 4 aspects of care on a 5-level scale (1 very negative, 2 rather negative, 3 neutral, 4 rather positive, 5 very positive). For each tool, we obtained a mean score from the 4 separate impacts.

Results

Doctors had predominantly negative opinions of the impact of managed care tools: use of guidelines (mean score 3.18), gate-keeping (2.76), managed care networks (2.77), second opinion requirement (2.65), pay for performance (1.90), pay by salary (2.24), selective contracting (1.56), and pre-approval of expensive treatments (1.77). Estimated impacts on cost control were positive or neutral for most tools, but impacts on professional autonomy were predominantly negative. Primary care doctors held more positive opinions than doctors in other specialties, and psychiatrists were in general the most critical. Older doctors had more negative opinions, as well as those in private practice.

Conclusions

Doctors perceived most managed care tools to have a positive impact on the control of health care costs but a negative impact on medical practice. Tools that are controlled by the profession were better accepted than those that are imposed by payers.

What differences are detected by superiority trials or ruled out by noninferiority trials? A cross-sectional study on a random sample of two-hundred two-arms parallel group randomized clinical trials

BACKGROUND

The smallest difference to be detected in superiority trials or the largest difference to be ruled out in noninferiority trials is a key determinant of sample size, but little guidance exists to help researchers in their choice. The objectives were to examine the distribution of differences that researchers aim to detect in clinical trials and to verify that those differences are smaller in noninferiority compared to superiority trials.

METHODS

Cross-sectional study based on a random sample of two hundred two-arm, parallel group superiority (100) and noninferiority (100) randomized clinical trials published between 2004 and 2009 in 27 leading medical journals. The main outcome measure was the smallest difference in favor of the new treatment to be detected (superiority trials) or largest unfavorable difference to be ruled out (noninferiority trials) used for sample size computation, expressed as standardized difference in proportions, or standardized difference in means. Student t test and analysis of variance were used.

RESULTS

The differences to be detected or ruled out varied considerably from one study to the next; e.g., for superiority trials, the standardized difference in means ranged from 0.007 to 0.87, and the standardized difference in proportions from 0.04 to 1.56. On average, superiority trials were designed to detect larger differences than noninferiority trials (standardized difference in proportions: mean 0.37 versus 0.27, P = 0.001; standardized difference in means: 0.56 versus 0.40, P = 0.006). Standardized differences were lower for mortality than for other outcomes, and lower in cardiovascular trials than in other research areas.

CONCLUSIONS

Superiority trials are designed to detect larger differences than noninferiority trials are designed to rule out. The variability between studies is considerable and is partly explained by the type of outcome and the medical context. A more explicit and rational approach to choosing the difference to be detected or to be ruled out in clinical trials may be desirable.

Interpretation of evidence in data by untrained medical students: a scenario-based study

BACKGROUND

To determine which approach to assessment of evidence in data - statistical tests or likelihood ratios - comes closest to the interpretation of evidence by untrained medical students.

METHODS

Empirical study of medical students (N = 842), untrained in statistical inference or in the interpretation of diagnostic tests. They were asked to interpret a hypothetical diagnostic test, presented in four versions that differed in the distributions of test scores in diseased and non-diseased populations. Each student received only one version. The intuitive application of the statistical test approach would lead to rejecting the null hypothesis of no disease in version A, and to accepting the null in version B. Application of the likelihood ratio approach led to opposite conclusions - against the disease in A, and in favour of disease in B. Version C tested the importance of the p-value (A: 0.04 versus C: 0.08) and version D the importance of the likelihood ratio (C: 1/4 versus D: 1/8).

RESULTS

In version A, 7.5% concluded that the result was in favour of disease (compatible with p value), 43.6% ruled against the disease (compatible with likelihood ratio), and 48.9% were undecided. In version B, 69.0% were in favour of disease (compatible with likelihood ratio), 4.5% against (compatible with p value), and 26.5% undecided. Increasing the p value from 0.04 to 0.08 did not change the results. The change in the likelihood ratio from 1/4 to 1/8 increased the proportion of non-committed responses.

CONCLUSIONS

Most untrained medical students appear to interpret evidence from data in a manner that is compatible with the use of likelihood ratios.

Meta-analysis of binary outcomes from two-by-two tables when the length of follow-up varies and hazards are proportional

Odds ratios (ORs) and relative risks (RRs) are sensitive to the length of follow-up. In meta-analyses, pooling such results from studies with different lengths of follow-up may lead to an artificial heterogeneity and discrepancy caused by the choice of the summary index. In this article, we explore the utility of a meta-analysis method that uses the ratio of logarithms of survival probability as the measure of association, and that avoids the problems mentioned above when hazards are proportional. Meta-analyses of ORs, RRs and ratios of logarithms of survival probability are compared through a simulation study, in which data are simulated from a proportional hazard model and the length of follow-up varies across studies using realistic patterns of variability. Results show that the heterogeneity increases with the variability of length of follow-up for OR and RR, but not for the ratio of the logarithms of survival probability. A published meta-analysis is used to illustrate the method.

General population reference values for the French version of the EuroQol EQ-5D health utility instrument

OBJECTIVE

To provide reference values for the French version of the EQ-5D and verify its capacity to discriminate between subgroups.

METHODS

General population mail survey in French-speaking Switzerland that included the EQ-5D instrument (five items rated on three levels as no problem, moderate problem, severe problem, and a visual analog health scale between 0 and 100) and descriptive variables.

RESULTS

Questionnaires were returned by 1956 adults (response rate 52.1%). Three of the five items had important "ceiling" effects (proportions with no problem: mobility 92.1%, self-care 97.6%, usual activities 91.2%), the other two less so (pain/discomfort 54.3%, anxiety/depression 68.1%). Four health states represented the majority of the population: no health problem (41.8%), moderate pain/discomfort only (21.0%), moderate anxiety/depression only (11.5%), moderate pain/discomfort and moderate anxiety/depression only (13.2%). The mean health utility was 0.83 (SD 0.15) on a scale between 0 and 1 and the mean visual analog score 81.7 (SD 15.5); the two were correlated (Pearson r 0.63, P < 0.001). Health utility scores were lower among women, older respondents, those with basic education, users of health services, and those with lower self-reported health status. The pattern was similar for the visual analog score except that women reported slightly higher ratings than men.

CONCLUSION

The EQ-5D performed as expected in a French-speaking general population sample. Reference values by sex and age group may facilitate the interpretation of results obtained in clinical settings.

Measuring Physicians’ and Medical Students’ Attitudes Toward Caring for Immigrant Patients

It is generally believed that culturally competent clinical practice depends in part on the development of positive attitudes toward the care of immigrant patients. However, few tools exist to measure such attitudes in physicians. The authors operationalized ‘‘culturally competent attitudes’’ to include a high level of interest in caring for immigrant patients, an acceptance of the responsibility of doctors and hospitals to adapt to immigrant patients’ needs, and the opinion that understanding the patient’s psychosocial context is particularly important when caring for immigrant patients. The authors then assessed these attitudes and opinions among a sample of 619 Geneva doctors and medical students using a self-administered questionnaire and explored their association to respondents’ personal characteristics and professional experience. The authors found that both personal characteristics and professional experience were associated with attitudes toward caring for immigrant patients. In particular, the perceived importance of understanding the psychosocial context when caring for migrants was higher among medical students, women, Swiss nationals, those with greater interest in caring for immigrant patients and those who had received training in cultural competence. However, it is unclear whether cultural competence training and clinical context lead to the development of more positive attitudes or whether medical students and physicians who already have positive attitudes are more likely to participate in such training.

Impact of information about risks and benefits of cancer screening on intended participation

BACKGROUND

Providing comprehensive information about the risks and benefits of cancer screening is ethically necessary, but information about risks may decrease participation. This study explored the impact of information on intended participation using a randomised factorial design.

METHODS

We conducted a mail survey of 2333 adults living in Geneva, Switzerland. Each participant was given one randomly chosen version of a scenario that described a hypothetical cancer screening test, and was asked whether he or she would accept to undergo screening. The versions varied in terms of the amount of information about risks and benefits.

RESULTS

Respondents who received information about risks associated with screening were more likely to refuse participation (odds ratio 2.6 (95% confidence interval (CI) 2.0-3.5)) than those who received minimal information. In contrast, information about benefits had no impact on intended participation (odds ratio 1.0 (95% CI 0.8-1.2)). The impact of information about risks was significantly stronger in women than in men, in respondents who were in poorer health, who have had a doctor visit in the past 6months, those who have had a cancer screening test in the past 3years, and those who reported a high desire for autonomy in medical decisions.

CONCLUSIONS

Informing potential participants about the risks of screening may reduce participation rates. Enhanced information about the benefits of screening does not counterbalance this effect.

The association between body mass index and patients' experiences with inpatient care

OBJECTIVE

To explore the association between patients' body mass index (BMI) and their experiences with inpatient care.

DESIGN

Cross-sectional. Mail survey.

SETTING

University Hospital of Geneva.

PARTICIPANTS

Questionnaires were mailed to 2385 eligible adult patients, 6 weeks after discharge (response rate = 69%).

MAIN OUTCOME MEASURES

Patients' experiences with care were measured using the Picker inpatient survey questionnaire. BMI was calculated using self-reported height and weight. Main dependent variables were the global Picker patient experience (PPE-15) score and nine dimension-specific problem scores, scored from 0 (no reported problems) to 1 (all items coded as problems). We used linear regressions, adjusting for age, gender, education, subjective health, smoking and hospitalization, to assess the association between patients' BMI and their experiences with inpatient care.

RESULTS

Of the patients, 4.8% were underweight, 50.8% had normal weight, 30.3% were overweight and 14.1% were obese. Adjusted analysis shows that compared with normal weight, obesity was significantly associated with fewer problematic items in the surgery-related information domain, and being underweight or overweight was associated with more problematic items in the involvement of family/friends domain. The global PPE-15 score was significantly higher (more problems) for underweight patients.

CONCLUSIONS

Underweight patients, but not obese patients, reported more problems during hospitalization.