Scientific inbreeding and same-team replication: type D personality as an example

Scientists' perception of pilot study quality was influenced by statistical significance and study design

OBJECTIVES

Preliminary studies play a key role in developing large-scale interventions but may be held to higher or lower scientific standards during the peer review process because of their preliminary study status.

STUDY DESIGN AND SETTING

Abstracts from 5 published obesity prevention preliminary studies were systematically modified to generate 16 variations of each abstract. Variations differed by 4 factors: sample size (n = 20 vs. n = 150), statistical significance (P < 0.05 vs. P > 0.05), study design (single group vs. randomized 2 groups), and preliminary study status (presence/absence of pilot language). Using an online survey, behavioral scientists were provided with a randomly selected variation of each of the 5 abstracts and blinded to the existence of other variations. Respondents rated each abstract on aspects of study quality.

RESULTS

Behavioral scientists (n = 271, 79.7% female, median age 34 years) completed 1,355 abstract ratings. Preliminary study status was not associated with perceived study quality. Statistically significant effects were rated as more scientifically significant, rigorous, innovative, clearly written, warranted further testing, and had more meaningful results. Randomized designs were rated as more rigorous, innovative, and meaningful.

CONCLUSION

Findings suggest reviewers place a greater value on statistically significant findings and randomized control design and may overlook other important study characteristics.

Influence of pilot and small trials in meta-analyses of behavioral interventions: a meta-epidemiological study

BACKGROUND

Pilot/feasibility or studies with small sample sizes may be associated with inflated effects. This study explores the vibration of effect sizes (VoE) in meta-analyses when considering different inclusion criteria based upon sample size or pilot/feasibility status.

METHODS

Searches were to identify systematic reviews that conducted meta-analyses of behavioral interventions on topics related to the prevention/treatment of childhood obesity from January 2016 to October 2019. The computed summary effect sizes (ES) were extracted from each meta-analysis. Individual studies included in the meta-analyses were classified into one of the following four categories: self-identified pilot/feasibility studies or based upon sample size but not a pilot/feasibility study (N ≤ 100, N > 100, and N > 370 the upper 75th of sample size). The VoE was defined as the absolute difference (ABS) between the re-estimations of summary ES restricted to study classifications compared to the originally reported summary ES. Concordance (kappa) of statistical significance of summary ES between the four categories of studies was assessed. Fixed and random effects models and meta-regressions were estimated. Three case studies are presented to illustrate the impact of including pilot/feasibility and N ≤ 100 studies on the estimated summary ES.

RESULTS

A total of 1602 effect sizes, representing 145 reported summary ES, were extracted from 48 meta-analyses containing 603 unique studies (avg. 22 studies per meta-analysis, range 2-108) and included 227,217 participants. Pilot/feasibility and N ≤ 100 studies comprised 22% (0-58%) and 21% (0-83%) of studies included in the meta-analyses. Meta-regression indicated the ABS between the re-estimated and original summary ES where summary ES ranged from 0.20 to 0.46 depending on the proportion of studies comprising the original ES were either mostly small (e.g., N ≤ 100) or mostly large (N > 370). Concordance was low when removing both pilot/feasibility and N ≤ 100 studies (kappa = 0.53) and restricting analyses only to the largest studies (N > 370, kappa = 0.35), with 20% and 26% of the originally reported statistically significant ES rendered non-significant. Reanalysis of the three case study meta-analyses resulted in the re-estimated ES rendered either non-significant or half of the originally reported ES.

CONCLUSIONS

When meta-analyses of behavioral interventions include a substantial proportion of both pilot/feasibility and N ≤ 100 studies, summary ES can be affected markedly and should be interpreted with caution.

The iatrogenic opioid crisis: An example of 'institutional corruption of pharmaceuticals'?

RATIONALE

Prescribed opioids are major contributors to the international public health opioid crisis. Such widespread iatrogenic harms usually result from collective decision failures of healthcare organizations rather than solely of individual organizations or professionals. Findings from a system-wide safety analysis of the iatrogenic opioid crisis that includes roles of pertinent healthcare organizations may help avoid or mitigate similar future iatrogenic consequences. In this retrospective exploratory study, we report such an analysis.

METHODS

The study population encompassed the entire age spectrum and included those in whom opioids prescribed for chronic pain (unrelated to malignancy) were associated with death or morbidity. Root cause analysis, incorporating recent suggestions for improvement, was used to identify possible contributory factors from the literature. Based on their mandated roles and potential influences to prevent or mitigate the iatrogenic crisis, relevant organizations were grouped and stratified from most to least influential.

RESULTS

The analysis identified a chain of multiple interrelated causal factors within and between organizations. The most influential organizations were pharmaceutical, political, and drug regulatory; next: experts and their related societies, and publications. Less influential: accreditation, professional licensing and regulatory, academic and healthcare funding bodies. Collectively, their views and decisions influenced prescribing practices of frontline healthcare professionals and advocacy groups. Financial associations between pharmaceutical and most other organizations/groups were common. Ultimately, patients were adversely affected. There was a complex association with psychosocial variables.

LIMITATIONS

The analysis suggests associations not causality.

CONCLUSION

The iatrogenic crisis has multiple intricately linked roots. The major catalyst: pervasive pharma-linked financial conflicts of interest (CoIs) involving most other healthcare organizations. These extensive financial CoIs were likely triggers for a cascade of erroneous decisions and actions that adversely affected patients. The actions and decisions of pharma ranged from unethical to illegal. The iatrogenic opioid crisis may exemplify 'institutional corruption of pharmaceuticals'.

Vital personality scores and healthy aging: Life-course associations and familial transmission

OBJECTIVES

Personality traits are linked with healthy aging, but it is not clear how these associations come to manifest across the life-course and across generations. To study this question, we tested a series of hypotheses about (a) personality-trait prediction of markers of healthy aging across the life-course, (b) developmental origins, stability and change of links between personality and healthy aging across time, and (c) intergenerational transmission of links between personality and healthy aging. For our analyses we used a measure that aggregates the contributions of Big 5 personality traits to healthy aging: a "vital personality" score.

METHODS

Data came from two population-based longitudinal cohort studies, one based in New Zealand and the other in the UK, comprising over 6000 study members across two generations, and spanning an age range from birth to late life.

RESULTS

Our analyses revealed three main findings: first, individuals with higher vital personality scores engaged in fewer health-risk behaviors, aged slower, and lived longer. Second, individuals' vital personality scores were preceded by differences in early-life temperament and were relatively stable across adulthood, but also increased from young adulthood to midlife. Third, individuals with higher vital personality scores had children with similarly vital partners, promoted healthier behaviors in their children, and had children who grew up to have more vital personality scores themselves, for genetic and environmental reasons.

CONCLUSION

Our study shows how the health benefits associated with personality accrue throughout the life-course and across generations.

Replications, Comparisons, Sampling and the Problem of Representativeness in Animal Cognition Research

Animal cognition research often involves small and idiosyncratic samples. This can constrain the generalizability and replicability of a study's results and prevent meaningful comparisons between samples. However, there is little consensus about what makes a strong replication or comparison in animal research. We apply a resampling definition of replication to answer these questions in Part 1 of this article, and, in Part 2, we focus on the problem of representativeness in animal research. Through a case study and a simulation study, we highlight how and when representativeness may be an issue in animal behavior and cognition research and show how the representativeness problems can be viewed through the lenses of, i) replicability, ii) generalizability and external validity, iii) pseudoreplication and, iv) theory testing. Next, we discuss when and how researchers can improve their ability to learn from small sample research through, i) increasing heterogeneity in experimental design, ii) increasing homogeneity in experimental design, and, iii) statistically modeling variation. Finally, we describe how the strongest solutions will vary depending on the goals and resources of individual research programs and discuss some barriers towards implementing them.

Transdiagnostic individualized clinically-based risk calculator for the automatic detection of individuals at-risk and the prediction of psychosis: external replication in 2,430,333 US patients

The real-world impact of psychosis prevention is reliant on effective strategies for identifying individuals at risk. A transdiagnostic, individualized, clinically-based risk calculator to improve this has been developed and externally validated twice in two different UK healthcare trusts with convincing results. The prognostic performance of this risk calculator outside the UK is unknown. All individuals who accessed primary or secondary health care services belonging to the IBM® MarketScan® Commercial Database between January 2015 and December 2017, and received a first ICD-10 index diagnosis of nonorganic/nonpsychotic mental disorder, were included. According to the risk calculator, age, gender, ethnicity, age-by-gender, and ICD-10 cluster diagnosis at index date were used to predict development of any ICD-10 nonorganic psychotic disorder. Because patient-level ethnicity data were not available city-level ethnicity proportions were used as proxy. The study included 2,430,333 patients with a mean follow-up of 15.36 months and cumulative incidence of psychosis at two years of 1.43%. There were profound differences compared to the original development UK database in terms of case-mix, psychosis incidence, distribution of baseline predictors (ICD-10 cluster diagnoses), availability of patient-level ethnicity data, follow-up time and availability of specialized clinical services for at-risk individuals. Despite these important differences, the model retained accuracy significantly above chance (Harrell's C = 0.676, 95% CI: 0.672-0.679). To date, this is the largest international external replication of an individualized prognostic model in the field of psychiatry. This risk calculator is transportable on an international scale to improve the automatic detection of individuals at risk of psychosis.

Changing the logic of replication: A case from infant studies

Among infant researchers there is growing concern regarding the widespread practice of undertaking studies that have small sample sizes and employ tests with low statistical power (to detect a wide range of possible effects). For many researchers, issues of confidence may be partially resolved by relying on replications. Here, we bring further evidence that the classical logic of confirmation, according to which the result of a replication study confirms the original finding when it reaches statistical significance, could be usefully abandoned. With real examples taken from the infant literature and Monte Carlo simulations, we show that a very wide range of possible replication results would in a formal statistical sense constitute confirmation as they can be explained simply due to sampling error. Thus, often no useful conclusion can be derived from a single or small number of replication studies. We suggest that, in order to accumulate and generate new knowledge, the dichotomous view of replication as confirmatory/disconfirmatory can be replaced by an approach that emphasizes the estimation of effect sizes via meta-analysis. Moreover, we discuss possible solutions for reducing problems affecting the validity of conclusions drawn from meta-analyses in infant research.

High impact nutrition and dietetics journals' use of publication procedures to increase research transparency

BACKGROUND

The rigor and integrity of the published research in nutrition studies has come into serious question in recent years. Concerns focus on the use of flexible data analysis practices and selective reporting and the failure of peer review journals to identify and correct these practices. In response, it has been proposed that journals employ editorial procedures designed to improve the transparency of published research.

OBJECTIVE

The present study examines the adoption of editorial procedures designed to improve the reporting of empirical studies in the field of nutrition and dietetics research.

DESIGN

The instructions for authors of 43 journals included in Quartiles 1 and 2 of the Clarivate Analytics' 2018 Journal Citation Report category Nutrition and Dietetics were reviewed. For journals that published original research, conflict of interest disclosure, recommendation of reporting guidelines, registration of clinical trials, registration of other types of studies, encouraging data sharing, and use of the Registered Reports were assessed. For journals that only published reviews, all of the procedures except clinical trial registration were assessed.

RESULTS

Thirty-three journals published original research and 10 published only reviews. Conflict of interest disclosure was required by all 33 original research journals. Use of guidelines, trial registration and encouragement of data sharing were mentioned by 30, 27 and 25 journals, respectively. Registration of other studies was required by eight and none offered Registered Reports as a publication option at the time of the review. All 10 review journals required conflict of interest disclosure, four recommended data sharing and three the use of guidelines. None mentioned the other two procedures.

CONCLUSIONS

While nutrition journals have adopted a number of procedures designed to improve the reporting of research findings, their limited effects likely result from the mechanisms through which they influence analytic flexibility and selective reporting and the extent to which they are properly implemented and enforced by journals.

Evidence gap on antihyperglycemic pharmacotherapy in frail older adults : A systematic review

Background

Although antihyperglycemic pharmacotherapy in frail older adults with type 2 diabetes mellitus (T2DM) is challenging, recommendations from international guidelines are mainly based on indirect evidence from trials not including frail participants.

Objective

This systematic review investigated the effectiveness and safety of pharmacotherapy in frail older adults with T2DM.

Material and methods

Randomized (RCT) and non-randomized prospective clinical trials (non-RCT) were searched in three electronic databases (Medline, Embase, Central) up to October 2018. Trials in older adults with T2DM who were assessed as significantly or severely impaired by defined cut-off scores of assessment instruments on frailty, activities of daily living or physical functional impairment were included.

Results

Two reviewers independently screened 17,391 references for inclusion and assessed risk of bias with ROBINS‑I. Five non-RCTs and no RCT were identified. Treatment of T2DM without insulin compared to insulin could be associated with increased improvement in cardiac functions in patients with cardiac resynchronization therapy and with decreased falls in frail older women. While better glycemic control with low variability and low HbA1c (hemoglobin A1c) values (<7%) was associated with better maintenance of physical function in community-dwelling older persons, higher HbA1c values (8–8.9%) were associated with a reduction in the composite outcome of death or functional decline in community-dwelling diabetic older adults with need for skilled assistance. Due to serious risk of bias in all studies, results should be considered with caution.

Conclusion

Well-designed, large-scale RCTs including this important group of patients are required to assess the effectiveness and safety of pharmacotherapy and HbA1c targets.

Electronic supplementary material

The online version of this article (10.1007/s00391-020-01724-3) contains supplementary material, which is available to authorized users.

Identification and evaluation of risk of generalizability biases in pilot versus efficacy/effectiveness trials: a systematic review and meta-analysis

BACKGROUND

Preliminary evaluations of behavioral interventions, referred to as pilot studies, predate the conduct of many large-scale efficacy/effectiveness trial. The ability of a pilot study to inform an efficacy/effectiveness trial relies on careful considerations in the design, delivery, and interpretation of the pilot results to avoid exaggerated early discoveries that may lead to subsequent failed efficacy/effectiveness trials. "Risk of generalizability biases (RGB)" in pilot studies may reduce the probability of replicating results in a larger efficacy/effectiveness trial. We aimed to generate an operational list of potential RGBs and to evaluate their impact in pairs of published pilot studies and larger, more well-powered trial on the topic of childhood obesity.

METHODS

We conducted a systematic literature review to identify published pilot studies that had a published larger-scale trial of the same or similar intervention. Searches were updated and completed through December 31st, 2018. Eligible studies were behavioral interventions involving youth (≤18 yrs) on a topic related to childhood obesity (e.g., prevention/treatment, weight reduction, physical activity, diet, sleep, screen time/sedentary behavior). Extracted information included study characteristics and all outcomes. A list of 9 RGBs were defined and coded: intervention intensity bias, implementation support bias, delivery agent bias, target audience bias, duration bias, setting bias, measurement bias, directional conclusion bias, and outcome bias. Three reviewers independently coded for the presence of RGBs. Multi-level random effects meta-analyses were performed to investigate the association of the biases to study outcomes.

RESULTS

A total of 39 pilot and larger trial pairs were identified. The frequency of the biases varied: delivery agent bias (19/39 pairs), duration bias (15/39), implementation support bias (13/39), outcome bias (6/39), measurement bias (4/39), directional conclusion bias (3/39), target audience bias (3/39), intervention intensity bias (1/39), and setting bias (0/39). In meta-analyses, delivery agent, implementation support, duration, and measurement bias were associated with an attenuation of the effect size of - 0.325 (95CI - 0.556 to - 0.094), - 0.346 (- 0.640 to - 0.052), - 0.342 (- 0.498 to - 0.187), and - 0.360 (- 0.631 to - 0.089), respectively.

CONCLUSIONS

Pre-emptive avoidance of RGBs during the initial testing of an intervention may diminish the voltage drop between pilot and larger efficacy/effectiveness trials and enhance the odds of successful translation.

Cognitive Bias of Interpretation in Type D Personality: Associations with Physiological Indices of Arousal

It is argued that the Type D person experiences increased distress when encountering social situations, which results in altered stress responding. However, little is known about how Type D individuals behave or feel during social interactions. Using a within-subjects design, the present study examined the physiological arousal of Type D and non-Type D individuals when rating how they would deal with a number of hypothetical social situations. Results showed that across all social situations, Type D individuals experienced greater levels of discomfort. In clearly negative and ambiguously neutral situations, this was paired with higher pulse rate. This study shows that the Type D individual not only reports greater discomfort but also experiences changes physiologically that support the role of the cardiovascular system as a potential psychosomatic pathway to disease.

Outpatient Readmission in Rheumatology: A Machine Learning Predictive Model of Patient's Return to the Clinic

Our objective is to develop and validate a predictive model based on the random forest algorithm to estimate the readmission risk to an outpatient rheumatology clinic after discharge. We included patients from the Hospital Clínico San Carlos rheumatology outpatient clinic, from 1 April 2007 to 30 November 2016, and followed-up until 30 November 2017. Only readmissions between 2 and 12 months after the discharge were analyzed. Discharge episodes were chronologically split into training, validation, and test datasets. Clinical and demographic variables (diagnoses, treatments, quality of life (QoL), and comorbidities) were used as predictors. Models were developed in the training dataset, using a grid search approach, and performance was compared using the area under the receiver operating characteristic curve (AUC-ROC). A total of 18,662 discharge episodes were analyzed, out of which 2528 (13.5%) were followed by outpatient readmissions. Overall, 38,059 models were developed. AUC-ROC, sensitivity, and specificity of the reduced final model were 0.653, 0.385, and 0.794, respectively. The most important variables were related to follow-up duration, being prescribed with disease-modifying anti-rheumatic drugs and corticosteroids, being diagnosed with chronic polyarthritis, occupation, and QoL. We have developed a predictive model for outpatient readmission in a rheumatology setting. Identification of patients with higher risk can optimize the allocation of healthcare resources.

Transdiagnostic Risk Calculator for the Automatic Detection of Individuals at Risk and the Prediction of Psychosis: Second Replication in an Independent National Health Service Trust

BACKGROUND

The benefits of indicated primary prevention among individuals at Clinical High Risk for Psychosis (CHR-P) are limited by the difficulty in detecting these individuals. To overcome this problem, a transdiagnostic, clinically based, individualized risk calculator has recently been developed and subjected to a first external validation in 2 different catchment areas of the South London and Maudsley (SLaM) NHS Trust.

METHODS

Second external validation of real world, real-time electronic clinical register-based cohort study. All individuals who received a first ICD-10 index diagnosis of nonorganic and nonpsychotic mental disorder within the Camden and Islington (C&I) NHS Trust between 2009 and 2016 were included. The model previously validated included age, gender, ethnicity, age by gender, and ICD-10 index diagnosis to predict the development of any ICD-10 nonorganic psychosis. The model's performance was measured using Harrell's C-index.

RESULTS

This study included a total of 13702 patients with an average age of 40 (range 16-99), 52% were female, and most were of white ethnicity (64%). There were no CHR-P or child/adolescent services in the C&I Trust. The C&I and SLaM Trust samples also differed significantly in terms of age, gender, ethnicity, and distribution of index diagnosis. Despite these significant differences, the original model retained an acceptable predictive performance (Harrell's C of 0.73), which is comparable to that of CHR-P tools currently recommended for clinical use.

CONCLUSIONS

This risk calculator may pragmatically support an improved transdiagnostic detection of at-risk individuals and psychosis prediction even in NHS Trusts in the United Kingdom where CHR-P services are not provided.

A Systems Approach to Understanding and Improving Research Integrity

Concern about the integrity of empirical research has arisen in recent years in the light of studies showing the vast majority of publications in academic journals report positive results, many of these results are false and cannot be replicated, and many positive results are the product of data dredging and the application of flexible data analysis practices coupled with selective reporting. While a number of potential solutions have been proposed, the effects of these are poorly understood and empirical evaluation of each would take many years. We propose that methods from the systems sciences be used to assess the effects, both positive and negative, of proposed solutions to the problem of declining research integrity such as study registration, Registered Reports, and open access to methods and data. In order to illustrate the potential application of systems science methods to the study of research integrity, we describe three broad types of models: one built on the characteristics of specific academic disciplines; one a diffusion of research norms model conceptualizing researchers as susceptible, "infected" and recovered; and one conceptualizing publications as a product produced by an industry comprised of academics who respond to incentives and disincentives.

Predictors for independent external validation of cardiovascular risk clinical prediction rules: Cox proportional hazards regression analyses

BACKGROUND

Clinical prediction rules (CPRs) should be externally validated by independent researchers. Although there are many cardiovascular CPRs, most have not been externally validated. It is not known why some CPRs are externally validated by independent researchers and others are not.

METHODS

We analyzed cardiovascular risk CPRs included in a systematic review. Independent external validations were identified by forward citation searches of derivation studies. Time between the publication of a cardiovascular CPR and the first independent external validation was calculated. We assessed Kaplan-Meier estimates of the probability to have an independent external validation. Using univariable Cox regression, we explored whether characteristics of derivation (design, location, sample size, number of predictors, presentation format, validation in derivation), reporting (participants, predictors, outcomes, performance measure, information for risk calculation), and publication (journal impact factor) are associated with time to the first independent external validation.

RESULTS

Of 125 cardiovascular risk CPRs, 29 had an independent external validation. The median follow-up was 118 months (95% CI, 99-130). The 25th percentile of event time was 122 months (95% CI, 91-299). Cardiovascular risk CPRs from the USA were 4.15 times (95% CI, 1.89-9.13) more likely to have an independent external validation. Increasing the sample size of derivation by ten times was associated with a 2.32-fold (95% CI, 1.37-3.91) increase in the probability of having an independent external validation. CPRs presented with an internal validation tend to get an independent external validation sooner (HR = 1.73, 95% CI, 0.77-3.93). CPRs reporting all the information necessary for calculating individual risk were 2.65 (95% CI, 1.01-6.96) times more likely to have an independent external validation. Publishing a cardiovascular risk CPR in a journal that has one unit higher impact factor was associated with a 6% (95% CI, 3-9) higher likelihood of an independent external validation.

CONCLUSIONS

The probability for cardiovascular risk CPRs to get an independent external validation was low even many years after their derivations. Authors of new cardiovascular risk CPRs should consider using adequate sample size, conducting an internal validation, and reporting all the information needed for risk calculation as these features were associated with an independent external validation.

Development and validation of a multivariate predictive model for rheumatoid arthritis mortality using a machine learning approach

We developed and independently validated a rheumatoid arthritis (RA) mortality prediction model using the machine learning method Random Survival Forests (RSF). Two independent cohorts from Madrid (Spain) were used: the Hospital Clínico San Carlos RA Cohort (HCSC-RAC; training; 1,461 patients), and the Hospital Universitario de La Princesa Early Arthritis Register Longitudinal study (PEARL; validation; 280 patients). Demographic and clinical-related variables collected during the first two years after disease diagnosis were used. 148 and 21 patients from HCSC-RAC and PEARL died during a median follow-up time of 4.3 and 5.0 years, respectively. Age at diagnosis, median erythrocyte sedimentation rate, and number of hospital admissions showed the higher predictive capacity. Prediction errors in the training and validation cohorts were 0.187 and 0.233, respectively. A survival tree identified five mortality risk groups using the predicted ensemble mortality. After 1 and 7 years of follow-up, time-dependent specificity and sensitivity in the validation cohort were 0.79–0.80 and 0.43–0.48, respectively, using the cut-off value dividing the two lower risk categories. Calibration curves showed overestimation of the mortality risk in the validation cohort. In conclusion, we were able to develop a clinical prediction model for RA mortality using RSF, providing evidence for further work on external validation.

Replication initiatives will not salvage the trustworthiness of psychology

Replication initiatives in psychology continue to gather considerable attention from far outside the field, as well as controversy from within. Some accomplishments of these initiatives are noted, but this article focuses on why they do not provide a general solution for what ails psychology. There are inherent limitations to mass replications ever being conducted in many areas of psychology, both in terms of their practicality and their prospects for improving the science. Unnecessary compromises were built into the ground rules for design and publication of the Open Science Collaboration: Psychology that undermine its effectiveness. Some ground rules could actually be flipped into guidance for how not to conduct replications. Greater adherence to best publication practices, transparency in the design and publishing of research, strengthening of independent post-publication peer review and firmer enforcement of rules about data sharing and declarations of conflict of interest would make many replications unnecessary. Yet, it has been difficult to move beyond simple endorsement of these measures to consistent implementation. Given the strong institutional support for questionable publication practices, progress will depend on effective individual and collective use of social media to expose lapses and demand reform. Some recent incidents highlight the necessity of this.

Effects of Art on Surgical Patients: A Systematic Review and Meta-analysis

OBJECTIVES

The aim of the study was to assess the effect of art including ambient features such as music, interior design including visual art, and architectural features on health outcomes in surgical patients.

BACKGROUND

Healing environments can have a positive influence on many patients, but data focusing on art in surgical patients remain scarce.

METHODS

We conducted a systematic search following the PRISMA guidelines from January 2000 to October 2014 on art in surgical patients. For music interventions, we pooled controlled studies measuring health outcomes (eg, pain, anxiety, blood pressure, and heart rate) in a meta-analysis. For other art forms (ambient and architectural features and interior design), we did a narrative review, also including nonsurgical patients, and looked for examples covering 3 countries.

RESULTS

Our search identified 1101 hits with 48 studies focusing on art in surgical patients: 47 studies on musical intervention and 1 on sunlight. The meta-analysis of these studies disclosed significant effects for music on pain after surgery, anxiety, systolic blood pressure, and heart rate, when compared with control groups without music. Effects of music were larger with self-selected music, and lower in surgical interventions performed under general anesthesia. Interior design features such as nature images and more spacious rooms, and architectural features providing more sunlight had positive effects on anxiety and postoperative pain.

CONCLUSIONS

Self-selected music for surgical patients is an effective and low-cost intervention to enhance well being and possibly faster recovery. Although potentially very important, the impact of environmental features and spacious architecture with wide access to sunlight remains poorly explored in surgery. Further experimental research is needed to better assess the magnitude of the impact and cost effectiveness.

A 'reluctant' critical review: 'Manual for evidence-based clinical practice (2015)'

BACKGROUND

The Users' Guides to the Medical Literature Manual has been a major influence on the teaching and practice of health care globally.

METHODS

The 3rd edition of the multi-authored Manual was reviewed using the principles outlined in Evidence-based Medicine (EBM) texts. One 'clinical scenario' was selected for critical appraisal, as were several chapters; objectivity was enhanced by citing references to support opinions. RESULTS (SUMMARY OF THE APPRAISAL): (1) Strengths: Clinical pearls, too numerous to list.

EXAMPLES

(i) evidence is never enough to drive clinical decision making; (ii) do not rush to adopt new interventions; and (iii) question efficacy data based only on surrogate markers. (2) Weaknesses: The Manual shares shortcomings of textbooks discussed by Straus et al.: (i) references may not be current, important ones may be excluded and citations may be selective; (ii) often, opinion-based; and (iii) delays between revisions. (3) Notable omissions: Little or no discussion of: (i) important segments of the population: those <18 years of age, >65 years of age and those with multimorbidity; (ii) surgical disciplines; (iii) Greenhalgh et al.'s essay on EBM; (iv) alternate views on the hierarchy of evidence; and (vi) critical thinking. (4) Additional issues: (i) Omission of important references on dabigatran (clinical scenario: chapter 13.1); (ii) authors' advice (Chapter 13.3) to 'bypass the discussion section of published research'; and (iii) the advocacy of pre-appraised sources of evidence and network meta-analysis without warnings about limitations, are critiqued.

CONCLUSION

The Manual has several clinical pearls but readers should also be aware of shortcomings.

What's in a Surname? Physique, Aptitude, and Sports Type Comparisons between Tailors and Smiths

Combined heredity of surnames and physique, coupled with past marriage patterns and trade-specific physical aptitude and selection factors, may have led to differential assortment of bodily characteristics among present-day men with specific trade-reflecting surnames (Tailor vs. Smith). Two studies reported here were partially consistent with this genetic-social hypothesis, first proposed by Bäumler (1980). Study 1 (N = 224) indicated significantly higher self-rated physical aptitude for prototypically strength-related activities (professions, sports, hobbies) in a random sample of Smiths. The counterpart effect (higher aptitude for dexterity-related activities among Tailors) was directionally correct, but not significant, and Tailor-Smith differences in basic physique variables were nil. Study 2 examined two large total-population-of-interest datasets (Austria/Germany combined, and UK: N = 7001 and 20532) of men’s national high-score lists for track-and-field events requiring different physiques. In both datasets, proportions of Smiths significantly increased from light-stature over medium-stature to heavy-stature sports categories. The predicted counterpart effect (decreasing prevalences of Tailors along these categories) was not supported. Related prior findings, the viability of possible alternative interpretations of the evidence (differential positive selection for trades and occupations, differential endogamy and assortative mating patterns, implicit egotism effects), and directions for further inquiry are discussed in conclusion.

Replication, falsification, and the crisis of confidence in social psychology

The (latest) crisis in confidence in social psychology has generated much heated discussion about the importance of replication, including how it should be carried out as well as interpreted by scholars in the field. For example, what does it mean if a replication attempt “fails”—does it mean that the original results, or the theory that predicted them, have been falsified? And how should “failed” replications affect our belief in the validity of the original research? In this paper, we consider the replication debate from a historical and philosophical perspective, and provide a conceptual analysis of both replication and falsification as they pertain to this important discussion. Along the way, we highlight the importance of auxiliary assumptions (for both testing theories and attempting replications), and introduce a Bayesian framework for assessing “failed” replications in terms of how they should affect our confidence in original findings.

A generalized view of self-citation: direct, co-author, collaborative, and coercive induced self-citation

The phenomenon of self-citation can present in many different forms, including direct, co-author, collaborative, and coercive induced self-citation. It can also pertain to the citation of single scientists, groups of scientists, journals, and institutions. This article presents some case studies of extreme self-citation practices. It also discusses the implications of different types of self-citation. Self-citation is not necessarily inappropriate by default. In fact, usually it is fully appropriate but often it is even necessary. Conversely, inappropriate self-citation practices may be highly misleading and may distort the scientific literature. Coercive induced self-citation is the most difficult to discover. Coercive Induced self-citation may happen directly from reviewers of articles, but also indirectly from reviewers of grants, scientific advisors who steer a research agenda, and leaders of funding agencies who may espouse spending disproportionately large funds in research domains that perpetuate their own self-legacy. Inappropriate self-citation can be only a surrogate marker of what might be much greater distortions of the scientific corpus towards conformity to specific opinions and biases. Inappropriate self-citations eventually affect also impact metrics. Different impact metrics vary in the extent to which they can be gamed through self-citation practices. Citation indices that are more gaming-proof are available and should be more widely used. We need more empirical studies to dissect the impact of different types of inappropriate self-citation and to examine the effectiveness of interventions to limit them.

Evidence-informed person-centred health care (part II): are 'cognitive biases plus' underlying the EBM paradigm responsible for undermining the quality of evidence?

INTRODUCTION

Recently, some leaders of the evidence-based medicine (EBM) movement drew attention to the "unintended" negative consequences associated with EBM. The term 'cognitive biases plus' was introduced in part I to encompass cognitive biases, conflicts of interests, fallacies and certain behaviours.

HYPOTHESIS

'Cognitive biases plus' in those closely involved in creating and promoting the EBM paradigm are responsible for their (1) inability to anticipate and then recognize flaws in the tenets of EBM; (2) discounting alternative views; and (3) delaying reform.

METHODS

A narrative review style was used, with methods as in part I.

APPRAISAL OF LITERATURE

Over the past two decades there has been mounting qualitative and quantitative methodological evidence to suggest that the faith placed in (1) the EBM hierarchy with randomized controlled trials and systematic reviews at the summit; (2) the reliability of biostatistical methods to quantitate data; and (3) the primacy of sources of pre-appraised evidence, is seriously misplaced. Consequently, the evidence that informs person-centred care is compromised.

DISCUSSION

Arguments focusing on 'cognitive biases plus' are offered to support our hypothesis. To the best of our knowledge, EBM proponents have not provided an explanation.

CONCLUSIONS

Reform is urgently needed to minimize continuing risks to patients. If our hypothesis is correct, then in addition to the suggestions made in part I, deficiencies in the paradigm must be corrected. Meaningful solutions are only possible if the biases of scientific inbreeding and groupthink are minimized by collaboration between EBM leaders and those who have been sounding warning bells.

Evidence-informed person-centered healthcare part I: do 'cognitive biases plus' at organizational levels influence quality of evidence?

INTRODUCTION

There is increasing concern about the unreliability of much of health care evidence, especially in its application to individuals.

HYPOTHESIS

Cognitive biases, financial and non-financial conflicts of interest, and ethical violations (which, together with fallacies, we collectively refer to as 'cognitive biases plus') at the levels of individuals and organizations involved in health care undermine the evidence that informs person-centred care.

METHODS

This study used qualitative review of the pertinent literature from basic, medical and social sciences, ethics, philosophy, law etc.

RESULTS

Financial conflicts of interest (primarily industry related) have become systemic in several organizations that influence health care evidence. There is also plausible evidence for non-financial conflicts of interest, especially in academic organizations. Financial and non-financial conflicts of interest frequently result in self-serving bias. Self-serving bias can lead to self-deception and rationalization of actions that entrench self-serving behaviour, both potentially resulting in unethical acts. Individuals and organizations are also susceptible to other cognitive biases. Qualitative evidence suggests that 'cognitive biases plus' can erode the quality of evidence.

CONCLUSIONS

'Cognitive biases plus' are hard wired, primarily at the unconscious level, and the resulting behaviours are not easily corrected. Social behavioural researchers advocate multi-pronged measures in similar situations: (i) abolish incentives that spawn self-serving bias; (ii) enforce severe deterrents for breaches of conduct; (iii) value integrity; (iv) strengthen self-awareness; and (v) design curricula especially at the trainee level to promote awareness of consequences to society. Virtuous professionals and organizations are essential to fulfil the vision for high-quality individualized health care globally.

HELOW: a program for testing extreme homogeneity in meta-analysis

Meta-analysis aims to synthesize results from different studies. Although, in a meta-analysis the presence of large between-study heterogeneity is routinely evaluated, in some instances is also important to probe whether there is extreme between-study homogeneity (i.e. extreme low between-study heterogeneity). HELOW (HEterogeneity LOW) is a program for testing extreme homogeneity in a meta-analysis of risk ratios when binary outcome and Mantel-Haenszel fixed effects summary risk ratio estimate are employed. The significance of extreme homogeneity is assessed using a Monte Carlo test. Extreme homogeneity may yield insights for the statistical and clinical interpretation of the data.

External validation of multivariable prediction models: a systematic review of methodological conduct and reporting

BACKGROUND

Before considering whether to use a multivariable (diagnostic or prognostic) prediction model, it is essential that its performance be evaluated in data that were not used to develop the model (referred to as external validation). We critically appraised the methodological conduct and reporting of external validation studies of multivariable prediction models.

METHODS

We conducted a systematic review of articles describing some form of external validation of one or more multivariable prediction models indexed in PubMed core clinical journals published in 2010. Study data were extracted in duplicate on design, sample size, handling of missing data, reference to the original study developing the prediction models and predictive performance measures.

RESULTS

11,826 articles were identified and 78 were included for full review, which described the evaluation of 120 prediction models. in participant data that were not used to develop the model. Thirty-three articles described both the development of a prediction model and an evaluation of its performance on a separate dataset, and 45 articles described only the evaluation of an existing published prediction model on another dataset. Fifty-seven percent of the prediction models were presented and evaluated as simplified scoring systems. Sixteen percent of articles failed to report the number of outcome events in the validation datasets. Fifty-four percent of studies made no explicit mention of missing data. Sixty-seven percent did not report evaluating model calibration whilst most studies evaluated model discrimination. It was often unclear whether the reported performance measures were for the full regression model or for the simplified models.

CONCLUSIONS

The vast majority of studies describing some form of external validation of a multivariable prediction model were poorly reported with key details frequently not presented. The validation studies were characterised by poor design, inappropriate handling and acknowledgement of missing data and one of the most key performance measures of prediction models i.e. calibration often omitted from the publication. It may therefore not be surprising that an overwhelming majority of developed prediction models are not used in practice, when there is a dearth of well-conducted and clearly reported (external validation) studies describing their performance on independent participant data.

Modelling the effects of subjective and objective decision making in scientific peer review

The objective of science is to advance knowledge, primarily in two interlinked ways: circulating ideas, and defending or criticizing the ideas of others. Peer review acts as the gatekeeper to these mechanisms. Given the increasing concern surrounding the reproducibility of much published research, it is critical to understand whether peer review is intrinsically susceptible to failure, or whether other extrinsic factors are responsible that distort scientists' decisions. Here we show that even when scientists are motivated to promote the truth, their behaviour may be influenced, and even dominated, by information gleaned from their peers' behaviour, rather than by their personal dispositions. This phenomenon, known as herding, subjects the scientific community to an inherent risk of converging on an incorrect answer and raises the possibility that, under certain conditions, science may not be self-correcting. We further demonstrate that exercising some subjectivity in reviewer decisions, which serves to curb the herding process, can be beneficial for the scientific community in processing available information to estimate truth more accurately. By examining the impact of different models of reviewer decisions on the dynamic process of publication, and thereby on eventual aggregation of knowledge, we provide a new perspective on the ongoing discussion of how the peer-review process may be improved.

Predictive value of social inhibition and negative affectivity for cardiovascular events and mortality in patients with coronary artery disease: the type D personality construct

OBJECTIVE

Methodological considerations and selected null findings indicate the need to reexamine the Type D construct. We investigated whether associations with cardiac events in patients with coronary artery disease (CAD) involve the specific combination of negative affectivity (NA) and social inhibition (SI), or rather the main effect of either trait alone.

METHODS

In this 5-year follow-up of 541 patients with CAD, the Type D construct (14-item Type D Scale) was tested by examining a) the interaction of continuous NA and SI z scores and b) a four-group classification defined by low/high trait scores. End points were major adverse cardiac events (MACEs; death, myocardial infarction [MI], coronary revascularization) and cardiac death/MI.

RESULTS

At follow-up, 113 patients had a MACE, including 47 patients with cardiac death/MI. After adjustment for disease severity, age, sex, and main trait effects, the interaction of NA and SI z scores was associated with MACE (odds ratio [OR] = 1.36, 95% confidence interval [CI] = 1.11-1.67). This continuous measure of Type D was also associated with cardiac death/MI (OR = 1.48, 95% CI = 1.11-1.96) and remained an independent predictor of events after adjustment for depressive symptoms. Using a cutoff of 10 on both NA and SI scales, Type D was associated with an adjusted OR of 1.74 (95% CI = 1.11-2.73) for MACE and an OR of 2.35 (95% CI = 1.26-4.38) for death/MI but was unrelated to noncardiac death. Patients with high NA or SI alone were not at increased risk.

CONCLUSIONS

Continuous (NA × SI interaction) and dichotomized measures of Type D were associated with cardiovascular events in patients with CAD. Research is needed to explore moderating factors that may alter this association.

Triple P-Positive Parenting programs: the folly of basing social policy on underpowered flawed studies

Wilson et al. provided a valuable systematic and meta-analytic review of the Triple P-Positive Parenting program in which they identified substantial problems in the quality of available evidence. Their review largely escaped unscathed after Sanders et al.'s critical commentary. However, both of these sources overlook the most serious problem with the Triple P literature, namely, the over-reliance on positive but substantially underpowered trials. Such trials are particularly susceptible to risks of bias and investigator manipulation of apparent results. We offer a justification for the criterion of no fewer than 35 participants in either the intervention or control group. Applying this criterion, 19 of the 23 trials identified by Wilson et al. were eliminated. A number of these trials were so small that it would be statistically improbable that they would detect an effect even if it were present. We argued that clinicians and policymakers implementing Triple P programs incorporate evaluations to ensure that goals are being met and resources are not being squandered.Please see related articles http://www.biomedcentral.com/1741-7015/10/130 and http://www.biomedcentral.com/1741-7015/10/145.