Relevance relations for the concept of reproducibility

The Tangled Knots of Neuroscientific Experimentation

The experimental method has promoted the popularity of neuroscientific research on the human mind. In this interdisciplinary enterprise, the experimental method, with its roots in natural science and experimental psychology, is often uncritically accepted as the royal road to investigate the human mind not only by neuroscientists, but by many philosophers as well, especially those inclined to some form of naturalism. It is rarely disputed that experiments reveal actual states of nature (here: of mind and/or brain). Experimental results are used to picture the human person or subject as an illusionary construct resulting from neuronal interactions. The present paper sketches some of the limitations of neuroscientific experiments in order to demonstrate that cognitive neuroscience is far from relying on firm methodological grounds. Numerous issues still have to be solved, some of which date back to the early days of modern science. At least, to make experiments work, many theoretical presuppositions have to be accepted and decisions of relevance have to be made in the scientific process. This implies that all scientific endeavor is constituted by persons making free decisions for good reasons, despite all reductionist claims to the contrary. The fact that we as scientists have to distinguish relevant from irrelevant aspects of experimental procedures is also crucial for dealing with the current replicability crisis in the life sciences including neuroscience.

Link and System-Level NOMA Simulator: The Reproducibility of Research

This study focuses on the design of a MATLAB platform for non-orthogonal multiple access (NOMA) based systems with link-level and system-level analyses. Among the different potential candidates for 5G, NOMA is gaining considerable attention owing to the many-fold increase in spectral efficiency as compared to orthogonal multiple access (OMA). In this study, a NOMA simulator is presented for two and more than two users in a single cell for link-level analysis; whereas, for system-level analysis, seven cells and 19 cells scenarios were considered. Long-term evolution (LTE) was used as the baseline for the NOMA simulator, while bit error rate (BER), throughput and spectral efficiency are used as performance metrics to analyze the simulator performance. Moreover, we demonstrated the application of the NOMA simulator for different simulation scenarios through examples. In addition, the performance of multi-carrier NOMA (MC-NOMA) was evaluated in the presence of AWGN, impulse noise, and intercell interference. To circumvent channel impairments, channel coding with linear precoding is suggested to improve the BER performance of the system.

Interlaboratory comparison of humic substances compositional space as measured by Fourier transform ion cyclotron resonance mass spectrometry (IUPAC Technical Report)

Interlaboratory comparison on the determination of the molecular composition of humic substances (HS) was undertaken in the framework of IUPAC project 2016-015-2-600. The analysis was conducted using high resolution mass spectrometry, nominally, Fourier transform ion cyclotron resonance mass spectrometry (FTICR MS) with electrospray ionization. Six samples of HS from freshwater, soil, and leonardite were used for this study, including one sample of humic acids (HA) from coal (leonardite), two samples of soil HA (the sod-podzolic soil and chernozem), two samples of soil fulvic acids (FA) (the sod-podzolic soil and chernozem), and one sample of freshwater humic acids (the Suwannee River). The samples were analyzed on five different FTICR MS instruments using the routine conditions applied in each participating laboratory. The results were collected as mass lists, which were further assigned formulae for the determination of molecular composition. The similarity of the obtained data was evaluated using appropriate statistical metrics. The results have shown that direct comparison of discrete stoichiometries assigned to the mass lists obtained by the different laboratories yielded poor results with low values of the Jaccard similarity score – not exceeding 0.56 (not more than 56 % of the similar peaks). The least similarity was observed for the aromatics-rich HA samples from leonardite (coal) and the chernozem soil, which might be connected to difficulties in their ionization. The reliable similarity among the data obtained in this intercomparison study was achieved only by transforming a singular point (stoichiometry) in van Krevelen diagram into a sizeable pixel (a number of closely located stoichiometries), which can be calculated from the population density distribution. The conclusion was made that, so far, these are descriptors of occupation density distribution, which provide the metrics compliant with the data quality requirements, such as the reproducibility of the data measurements on different instruments.

Methodological evaluation of reviews that support recommendations from three consensus workshops in periodontology

OBJECTIVE

This study aimed to evaluate comprehensiveness and reproducibility of reviews that support consensus guidelines in periodontology.

METHODS

We included the reviews that support consensus guidelines from three workshops in periodontology, which were overseen by likely the two most important organisations in the field: the European Federation of Periodontology and the American Academy of Periodontology. We independently evaluated the comprehensiveness of literature searches by determining whether authors had searched reference lists, journals, registries and grey literature and whether the searches were limited to only one or a few languages. We evaluated whether review authors reported the eligibility criteria, the search strategies, and the list of included/excluded articles. We tested whether the search and selection of articles in one major database was reproducible.

RESULTS

Twenty-nine reviews were evaluated. Two (7%) reviews reported grey literature searches, and more than two-thirds of the reviews did not report hand-searching. Almost half of the reviews did not report whether there was language restriction for the literature searches. Two-thirds of the reviews reported the use of keywords only (without Boolean operators). One-fourth of the reviews reported the presence of a list of excluded articles after the full-text assessment. None of the reviews reported a detailed list of excluded articles after screening of titles/abstracts. None of the reviews reported enough information to allow reproduction of the findings of the PubMed search.

CONCLUSIONS

There is room to improve the reporting of the methodologies that are used in reviews that support periodontology consensus guidelines, although heterogeneity in reporting was found across all the reviews.

Independent discussion sections for improving inferential reproducibility in published research

There is a reproducibility crisis in science. There are many potential contributors to replication failure in research across the translational continuum. In this perspective piece, we focus on the narrow topic of inferential reproducibility. Although replication of methods and results is necessary to demonstrate reproducibility, it is not sufficient. Also fundamental is consistent interpretation in the Discussion section. Current deficiencies in the Discussion sections of manuscripts might limit the inferential reproducibility of scientific research. Lack of contextualisation using systematic reviews, overinterpretation and misinterpretation of results, and insufficient acknowledgement of limitations are common problems in Discussion sections; these deficiencies can harm the translational process. Proposed solutions include eliminating or not reading Discussions, writing accompanying editorials, and post-publication review and comments; however, none of these solutions works very well. A second Discussion written by an independent author with appropriate expertise in research methodology is a new testable solution that could help probe inferential reproducibility, and address some deficiencies in primary Discussion sections.

Three Dimensions of Reproducibility in Natural Language Processing

Despite considerable recent attention to problems with reproducibility of scientific research, there is a striking lack of agreement about the definition of the term. That is a problem, because the lack of a consensus definition makes it difficult to compare studies of reproducibility, and thus to have even a broad overview of the state of the issue in natural language processing. This paper proposes an ontology of reproducibility in that field. Its goal is to enhance both future research and communication about the topic, and retrospective meta-analyses. We show that three dimensions of reproducibility, corresponding to three kinds of claims in natural language processing papers, can account for a variety of types of research reports. These dimensions are reproducibility of a conclusion, of a finding, and of a value. Three biomedical natural language processing papers by the authors of this paper are analyzed with respect to these dimensions.

Contextual emergence of mental states

The concept of contextual emergence has been proposed as a non-reductive, yet well-defined relation between different levels of description of physical and other systems. It yields a formally sound and empirically applicable procedure to translate between descriptive levels in an overall consistent fashion. This will be discussed for the contextual emergence of mental states from a neural level of description.

Image-based three-dimensional analysis to characterize the texture of porous scaffolds

The aim of the present study is to characterize the microstructure of composite scaffolds for bone tissue regeneration containing different ratios of chitosan/gelatin blend and bioactive glasses. Starting from realistic 3D models of the scaffolds reconstructed from micro-CT images, the level of heterogeneity of scaffold architecture is evaluated performing a lacunarity analysis. The results demonstrate that the presence of the bioactive glass component affects not only macroscopic features such as porosity, but mainly scaffold microarchitecture giving rise to structural heterogeneity, which could have an impact on the local cell-scaffold interaction and scaffold performances. The adopted approach allows to investigate the scale-dependent pore distribution within the scaffold and the related structural heterogeneity features, providing a comprehensive characterization of the scaffold texture.