Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty

This study explores how researchers' analytical choices affect the reliability of scientific findings. Most discussions of reliability problems in science focus on systematic biases. We broaden the lens to emphasize the idiosyncrasy of conscious and unconscious decisions that researchers make during data analysis. We coordinated 161 researchers in 73 research teams and observed their research decisions as they used the same data to independently test the same prominent social science hypothesis: that greater immigration reduces support for social policies among the public. In this typical case of social science research, research teams reported both widely diverging numerical findings and substantive conclusions despite identical start conditions. Researchers' expertise, prior beliefs, and expectations barely predict the wide variation in research outcomes. More than 95% of the total variance in numerical results remains unexplained even after qualitative coding of all identifiable decisions in each team's workflow. This reveals a universe of uncertainty that remains hidden when considering a single study in isolation. The idiosyncratic nature of how researchers' results and conclusions varied is a previously underappreciated explanation for why many scientific hypotheses remain contested. These results call for greater epistemic humility and clarity in reporting scientific findings.

Examining the Behaviour subscale of the Hypoglycaemia Fear Survey: an international study

AIMS

The Hypoglycemia Fear Survey (HFS)-II Behaviour and Worry subscales were developed to measure behaviours and anxiety related to hypoglycaemia in diabetes. However, previous studies found lower reliability in the HFS Behaviour subscale and inconsistent relationships with glucose control. The purpose of this study was to conduct extensive analyses of the internal structure of the HFS Behaviour subscale's internal structure and its relationships with diabetes outcomes, including HbA1c and episodes of severe hypoglycaemia.

METHODS

HFS-II survey data from 1460 adults with Type 1 diabetes were collected from five countries. This aggregated sample underwent exploratory factor analysis and item analysis to determine the internal structure of the survey and subscales.

RESULTS

A three-factor solution showed the best fit for the HFS, with two subscales emerging from the HFS Behaviour representing tendencies towards (1) maintenance of high blood glucose and (2) avoidance of hypoglycaemic risks by other behaviours, and a third single HFS Worry subscale. Subscale item analysis showed excellent fit, separation and good point-measure correlations. All subscales demonstrated acceptable (0.75) to excellent (0.94) internal reliability. HbA(1c) correlated with Maintain High Blood Glucose subscale scores, r = 0.14, P < 0.001, and severe hypoglycaemia frequency correlated with all subscales.

CONCLUSIONS

The HFS Worry subscale measures one construct of anxiety about various aspects of hypoglycaemia. In contrast, the HFS Behaviour subscale appears to measure two distinct aspects of behavioural avoidance to prevent hypoglycaemia, actions which maintain high blood glucose and other behaviours to avoid hypoglycaemic risk. These results demonstrate the clinical importance of the HFS Behaviour subscales and their differential relationships with measures of diabetes outcome such as HbA1c .

Extraction of process zones and low-dimensional attractive subspaces in stochastic fracture mechanics

We propose to identify process zones in heterogeneous materials by tailored statistical tools. The process zone is redefined as the part of the structure where the random process cannot be correctly approximated in a low-dimensional deterministic space. Such a low-dimensional space is obtained by a spectral analysis performed on pre-computed solution samples. A greedy algorithm is proposed to identify both process zone and low-dimensional representative subspace for the solution in the complementary region. In addition to the novelty of the tools proposed in this paper for the analysis of localised phenomena, we show that the reduced space generated by the method is a valid basis for the construction of a reduced order model.

Design of case-controls studies with unscreened controls

Traditionally in genetic case-control studies controls have been screened to exclude subjects with a personal history of illness. This control group has the advantage of optimal power to detect loci involved in illness, but requires more work and may incur substantial cost in recruitment. An alternative approach to screening is to use unscreened controls sampled from the general population. Such controls are generally plentiful and inexpensive, but in general there is a risk that some may have the same disease as the cases, which will reduce power to detect associations. We have quantified the extent of this power loss, and produced mathematical formulae for the number of unscreened controls necessary to achieve the same power as a fixed sample of screened controls. The effect of using unscreened controls will also depend on the ratio of the number of screened controls to cases specified in the original study design, and this is also investigated. We have also investigated the cost-benefits of the screened and unscreened approaches, according to variation in the relative costs of sampling screened and unscreened controls, together with genotyping costs. We have, thus, identified the range of situations in which using unscreened controls is a cost-effective alternative to the screened control method and could be considered when designing a study. In many of the typical, real-world situations in complex genetics, the use of unscreened controls is potentially cost-effective and can, in general, be considered for disorders with population prevalence Kp < 0.2. With the steady reduction in genotyping costs and the availability of common sets of "population controls" this design is likely to become increasingly cost effective.

A burning issue. Phantom limb pain and psychological preparation of the patient for amputation

Fantasies concerning an amputated limb can contribute to the occurrence of persistent phantom limb pain. We report a case in which burning pain perceived as located in the amputated lower extremities was related to the patient's feelings about incineration of the removed limbs against her wishes. Hypnotherapy involving elucidation of the fantasy and suggestion was successfully employed in this case and may be a helpful approach in other such cases. Importantly, adequate preparation of the patient for amputation, including an awareness of concerns about the disposition of the limb, may help prevent pathological limb sensations.