Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty

This study explores how researchers' analytical choices affect the reliability of scientific findings. Most discussions of reliability problems in science focus on systematic biases. We broaden the lens to emphasize the idiosyncrasy of conscious and unconscious decisions that researchers make during data analysis. We coordinated 161 researchers in 73 research teams and observed their research decisions as they used the same data to independently test the same prominent social science hypothesis: that greater immigration reduces support for social policies among the public. In this typical case of social science research, research teams reported both widely diverging numerical findings and substantive conclusions despite identical start conditions. Researchers' expertise, prior beliefs, and expectations barely predict the wide variation in research outcomes. More than 95% of the total variance in numerical results remains unexplained even after qualitative coding of all identifiable decisions in each team's workflow. This reveals a universe of uncertainty that remains hidden when considering a single study in isolation. The idiosyncratic nature of how researchers' results and conclusions varied is a previously underappreciated explanation for why many scientific hypotheses remain contested. These results call for greater epistemic humility and clarity in reporting scientific findings.

Does suffering suffice? An experimental assessment of desert retributivism

Michael S. Moore is among the most prominent normative theorists to argue that retributive justice, understood as the deserved suffering of offenders, justifies punishment. Moore claims that the principle of retributive justice is pervasively supported by our judgments of justice and sufficient to ground punishment. We offer an experimental assessment of these two claims, (1) the pervasiveness claim, according to which people are widely prone to endorse retributive judgments, and (2) the sufficiency claim, according to which no non-retributive principle is necessary for justifying punishment. We test these two claims in a survey and a related survey experiment in which we present participants (N = ~900) with the stylized description of a criminal case. Our results seem to invalidate claim (1) and provide mixed results concerning claim (2). We conclude that retributive justice theories which advance either of these two claims need to reassess their evidential support.

Trust and cooperative behavior: Evidence from the realm of data-sharing

Trust is praised by many social scientists as the foundation of functioning social systems owing to its assumed connection to cooperative behavior. The existence of such a link is still subject to debate. In the present study, we first highlight important conceptual issues within this debate. Second, we examine previous evidence, highlighting several issues. Third, we present findings from an original experiment, in which we tried to identify a “real” situation that allowed us to measure both trust and cooperation. People’s expectations and behavior when they decide to share (or not) their data represents such a situation, and we make use of corresponding data. We found that there is no relationship between trust and cooperation. This non-relationship may be rationalized in different ways which, in turn, provides important lessons for the study of the trust—behavior nexus beyond the particular situation we study empirically.

Serum KL-6 as a novel disease marker in adolescent and adult cystic fibrosis

BACKGROUND

Cystic fibrosis (CF) is a chronic progressive disease leading to obstructive pulmonary impairment, fibrosis and shortened life expectancy. Serum levels of KL-6, high molecular weight human MUC1 mucin, are increased in the majority of patients with various interstitial lung disorders. Whether they are also elevated in CF has not been investigated before.

OBJECTIVE

To evaluate whether serum KL-6 levels are elevated and correlate with pulmonary function variables in CF.

DESIGN

Serum KL-6, lactate dehydrogenase (LDH) and C-reactive protein (CRP) levels were measured in 72 consecutive CF and 80 age- and sex-matched healthy control subjects. The relationship between serum KL-6 levels and pulmonary function variables was analyzed.

RESULTS

Serum KL-6 levels in CF patients were significantly increased compared to healthy subjects. Receiver operating characteristic curve analysis revealed that the diagnostic accuracy of KL-6 was better than that of LDH and CRP. Serum KL-6 levels showed an inverse relationship with vital capacity (VC) % predicted and forced expiratory volume in one second (FEV1) % predicted.

CONCLUSIONS

Serum KL-6 levels are elevated and appear to be correlated with pulmonary function variables in CF. These results suggest that KL-6 may be a useful noninvasive marker to monitor disease severity.

[Comparability of various measurement methods in nasal provocation with allergens]

Defined criteria for nasal provocation exist only for the anterior Rhinomanometrie (aRM). During the last years there are other techniques for nasal provocation in use. In this study the new instruments were compared with the established aRM. We examined 14 patients with perennial rhinitis. All showed positive skin test and RAST against D. pteronyssinus. The controls were 13 atopic patients sensitized against pollen but not against D. pteronyssinus and 20 healthy volunteers. All subjects underwent nasal provocation with D. pteronyssinus. On the first day the provocation was unilateral. The instruments were the aRM, the Oscilloresistometry (ORM), the Peak Nasal Inspiratory Flowmeter (PNIF) and the Acoustic Rhinometry (Rhinoklack). On the second day the provocation was bilateral with ORM and PNIF. First we determined the limits for separation between a positive and negative nasal provocation test for the different instruments. Then we calculated the sensitivity and specificity for the instruments. By unilateral provocation there was a good sensitivity for all devices. The specificity was very good with the aRM, ORM and PNIF. In conclusion the PNIF is as good as the other instruments, but the Rhinoklack was less specific than the others.

Biomarkers of genetic damage and inflammation in blood and bronchoalveolar lavage fluid among former German uranium miners: a pilot study

Former East German uranium miners who are known to have been exposed to radon are estimated to be at high risk for lung carcinogenesis. Among these miners over 200 occupationally caused lung cancer cases are expected to occur each year, resulting in a total of 7,000-24,000 excess lung cancer cases in the coming years. It is still unknown whether there is a correlation between biomarkers and the exposure of the uranium miners to ionizing radiation that might enable us to trace those miners with high lung cancer risk. The primary aim of this pilot study was to test the possibility of performing a biomarker study in this unique cohort of former uranium miners in spite of several limitations that had to be taken into consideration when comparing them with healthy controls, such as old age, age-dependent diseases and potential confounding artefacts from dissimilar smoking patterns. The second aim was to test a range of biomarkers for DNA damage and inflammation in leukocytes and bronchoalveolar fluid for their ability to detect biological effects. In this cohort of miners we found an increased frequency of chromosomal aberrations in blood lymphocytes and an increased prevalence of both fibronectin and tumour necrosis factor alpha in the bronchoalveolar fluid.

Elevated levels of soluble TNF receptors in bronchoalveolar lavage fluid in extrinsic allergic alveolitis

BACKGROUND

Soluble TNF receptors (sTNFR1 and sTNFR2) are inhibitors of TNF and can block TNF bioactivity. TNF plays an important role in the development of extrinsic allergic alveolitis (EAA).

OBJECTIVE

To evaluate whether sTNFR1 and sTNFR2 are locally increased in EAA.

METHODS

We measured sTNFR1 and sTNFR2 in bronchoalveolar lavage fluid (BALF) and serum from nine EAA patients and 11 control subjects using an ELISA method.

RESULTS

BALF sTNFR1 and sTNFR2 levels were 0.24+/- 0.04 ng/mL and 0.59+/-0.16 ng/mL in EAA patients, and thus significantly elevated in comparison with the controls (0.13+/-0.02 ng/mL and 0.08+/-0.04 ng/mL, both P<0.05). Serum sTNFR levels were not significantly different between the two groups. Both sTNFR1 and sTNFR2 concentrations in BALF correlated significantly with the lymphocyte percentage of BALF (r = 0.57 and 0.81, respectively).

CONCLUSION

The two alveolar sTNFRs, particularly sTNFR2, may be involved in the pathogenesis of EAA as counter-regulators of TNF.

[Diffuse panbronchiolitis. A rare differential diagnosis of chronic obstructive lung disease]

HISTORY AND ADMISSION FINDINGS

A 36-year-old Korean had since childhood suffered from a chronic progressive lung disease, marked by dyspnoea, productive cough and recurrent sinusitis. He was in a reduced general condition on admission. There were hyperresonant sounds on palpation over both lungs, a low diaphragm and slight vesicular breath sound as well as occasional rales and sibilant rhonchi.

INVESTIGATIONS

Laboratory tests showed increased blood cell sedimentation of 52 mm at 1 h, leukocytosis of 24.3/nl, and a raised cold-agglutinin titre. Haemophilus influenza and Pseudomonas aerogenes were cultured from the bronchial secretion. Lung function tests revealed severe obstruction with hyperinflation and partial respiratory failure with a much reduced 1-sec forced expiratory volume (FEV1) of 0.71, 19% of normal; residual volume (RV) 6.61, 550% of normal; arterial oxygen partial pressure 61 mm Hg; arterial CO2 partial pressure 36 mm Hg. Chest X-ray showed emphysematous changes with diffuse small nodular shadows. Bronchoalveolar lavage revealed 96% neutrophils, indicating marked granulocytic inflammation. A video-assisted thoracoscopic biopsy was performed which showed diffuse panbronchiolitis.

TREATMENT AND COURSE

The symptoms, radiological changes and lung functions clearly improved under long-term treatment with 500 mg erythromycin daily.

CONCLUSION

Diffuse bronchiolitis is rarely seen outside of Asia, but it should be included in the differential diagnosis because, in contrast to other forms of chronic obstructive lung disease, it responds to long-term erythromycin medication.

[Bilateral lung volume reduction in patients with severe pulmonary emphysema]

For selected patients with severe pulmonary obstructive disease (COPD) and dyspnea despite conservative treatment, lung volume reduction (LVR) may to be of significant value as a surgical option. From May 1994 to March 1996 43 patients with COPD underwent through different procedures a LVR, 25 of them bilateral through median sternotomy or thoracotomies. With regard to chest computed tomography and perfusion/ventilation scan of the lungs the most affected portions are excised with the use of a linear stapling device. In all patients staplers was fitted with strips of bovine pericardium attached on both sides to eliminate postoperative leakage. There has been no early postoperative mortality (30 days). Follow-up ranges from 1 to 21 months. Preoperative and postoperative assessment of lung function tests, efficiency of respiratory pump, grading of dyspnea and life quality was prospective investigated. The mean improvement of FEV1 was 79%, PO2 at rest 8 mmHg. TLC has been reduced by 19%, RV by 31%. These changes have been associated with marked relief of dyspnea (grading of dyspnea reduced from 3.4 to 1.6) and improvement in exercise tolerance (6-min walk-test from 137 +/- 65 m to 265 +/- 175 m). Also significant changes the maximal inspiratory pressure, the mouth occlusion pressure and the transdiaphragmatic pressure as parameter of respiratory muscle function. Although the follow-up period is short, these results suggest that in selected patients with COPD, LVR improve lung function, gas exchange, efficiency of respiratory pump and quality of life and reduce the grading of dyspnea.

Feather wreath lung: chasing a dead bird

We report the case of a 50 year old clerk with chronic extrinsic allergic alveolitis due to a wreath of feathers from his dead budgerigar hanging in his living-room. Diagnosis was confirmed by serum precipitins, bronchoalveolar lavage and open lung biopsy. It is important to recognize unusual sources of exposure in this disease.

Soluble CD14 is increased in bronchoalveolar lavage of active sarcoidosis and correlates with alveolar macrophage membrane-bound CD14

CD14 is a myeloid differentiation antigen which exists in a membrane-bound (55 kD) and a soluble (48 kD) form. This antigen is a receptor for lipopolysaccharide (LPS) structures and triggers the production of various cytokines. The aim of this study was to evaluate whether in active sarcoidosis, a disease with increased proportions of alveolar macrophages (AM) with CD14 expression in BAL fluid, the soluble form of CD14 (sCD14) is also increased. The sCD14 levels were measured in BAL fluid with an ELISA, and membrane-bound CD14 was determined by an immunoperoxidase assay, in active sarcoidosis (n = 13), inactive sarcoidosis (n = 9), idiopathic pulmonary fibrosis (IPF) (n = 6), and control subjects (n = 8). Higher concentrations of sCD14 were present in BAL fluid of patients with active sarcoidosis (58 +/- 34 ng/ml) than in those with inactive disease (13 +/- 10 ng/ml), patients with IPF (5 +/- 5 ng/ml), or control subjects (10 +/- 8% ng/ml) (p < 0.01). Similarly, the proportions of AM expressing membrane-bound CD14 were increased in active sarcoidosis (91 +/- 6%) compared with inactive sarcoidosis (82 +/- 6%), patients with IPF (76 +/- 13%), and control subjects (79 +/- 9%) (p < .05). In sarcoidosis, a significant correlation was found between the sCD14 concentration in BAL fluid and AM membrane expression of CD14 (r = 0.57, p < 0.01). We conclude that sCD14 is increased in BAL of active sarcoidosis suggesting a potential role for this substance as marker of activity and in the pathogenesis of pulmonary sarcoidosis.

[Detection of mast cells in bronchoalveolar lavage with various stains]

The study analyzed the value of the May-Grünwald-Giemsa-stain to detect mast cells in bronchoalveolar lavage fluid. 32 consecutive BAL samples were stained with May-Grünwald-Giemsa and with Toluidine blue, a special stain for mast cells. Both stains were equally suitable to detect mast cells in BAL. The Diffquick-stain, however, was not able to stain mast cells in BAL.

Selective loss of expression of a tumor-associated antigen on a human leukemia cell line induced by treatment with monoclonal antibody and complement

The sensitivity of a common acute lymphoblastic leukemia-associated antigen (cALLa)-positive, human leukemia pre-B-cell line to killing by antibody and complement was studied. A stable subpopulation was selected by its ability to survive four sequential treatments with excess monoclonal antibody (MoAb) directed against an Mr 24,000 glycoprotein associated with human leukemia cells and excess rabbit complement. Analysis of the antigen expression by individual cells within the parental and the selected cell populations was achieved by flow cytometry and demonstrated a marked decrease of the leukemia-associated antigen expression on individual cells within the selected subpopulation. These low-antigen-density cells were stable in subculture, and the immunoglobulin heavy-chain gene rearrangement of the parent population and the low-antigen-density subpopulation were identical, indicating that they were derived from a single cell source. The selection of this subpopulation was specific in that the expression of a second antigen recognized by a cALLa-specific MoAb was not affected. The presence of subpopulations of tumor cells with low levels of surface antigen expression that are resistant to killing upon addition of excess antibody and complement will prove to be an obstacle to the use of this approach to eliminate tumor cells from bone marrow that is to be used for autologous transplantation.