A Review of the Scientific Rigor, Reproducibility, and Transparency Studies Conducted by the ABRF Research Groups

A scoping review on what constitutes a good research culture

Background

The crisis in research culture is well documented, covering issues such as a tendency for quantity over quality, unhealthy competitive environments, and assessment based on publications, journal prestige and funding. In response, research institutions need to assess their own practices to promote and advocate for change in the current research ecosystem. The purpose of the scoping review was to explore ' What does the evidence say about the 'problem' with 'poor' research culture, what are the benefits of 'good' research culture, and what does 'good' look like?'

Aims

To examine the peer-reviewed and grey literature to explore the interplay between research culture, open research, career paths, recognition and rewards, and equality, diversity, and inclusion, as part of a larger programme of activity for a research institution.

Methods

A scoping review was undertaken. Six databases were searched along with grey literature. Eligible literature had relevance to academic research institutions, addressed research culture, and were published between January 2017 to May 2022. Evidence was mapped and themed to specific categories. The search strategy, screening and analysis took place between April-May 2022.

Results

1666 titles and abstracts, and 924 full text articles were assessed for eligibility. Of these, 253 articles met the eligibility criteria for inclusion. A purposive sampling of relevant websites was drawn from to complement the review, resulting in 102 records included in the review. Key areas for consideration were identified across the four themes of job security, wellbeing and equality of opportunity, teamwork and interdisciplinary, and research quality and accountability.

Conclusions

There are opportunities for research institutions to improve their own practice, however institutional solutions cannot act in isolation. Research institutions and research funders need to work together to build a more sustainable and inclusive research culture that is diverse in nature and supports individuals' well-being, career progression and performance.

Tissue-Like 3D Standard and Protocols for Microscope Quality Management

This article outlines a global study conducted by the Association of Biomedical Resource Facilities (ABRF) Light Microscopy Research Group (LMRG). The results present a novel 3D tissue-like biologically relevant standard sample that is affordable and straightforward to prepare. Detailed sample preparation, instrument-specific image acquisition protocols and image analysis methods are presented and made available to the community. The standard consists of sub-resolution and large well characterized relative intensity fluorescence microspheres embedded in a 120 µm thick 3D gel with a refractive index of 1.365. The standard allows the evaluation of several properties as a function of depth. These include the following: 1) microscope resolution with automated analysis of the point-spread function (PSF), 2) automated signal-to-noise ratio analysis, 3) calibration and correction of fluorescence intensity loss, and 4) quantitative relative intensity. Results demonstrate expected refractive index mismatch dependent losses in intensity and resolution with depth, but the relative intensities of different objects at similar depths are maintained. This is a robust standard showing reproducible results across laboratories, microscope manufacturers and objective lens types (e.g., magnification, immersion medium). Thus, these tools will be valuable for the global community to benchmark fluorescence microscopes and will contribute to improved scientific rigor and reproducibility.

Multi-laboratory experiment PME11 for the standardization of phosphoproteome analysis

Global analysis of protein phosphorylation by mass spectrometry proteomic techniques has emerged in the last decades as a powerful tool in biological and biomedical research. However, there are several factors that make the global study of the phosphoproteome more challenging than measuring non-modified proteins. The low stoichiometry of the phosphorylated species and the need to retrieve residue specific information require particular attention on sample preparation, data acquisition and processing to ensure reproducibility, qualitative and quantitative robustness and ample phosphoproteome coverage in phosphoproteomic workflows. Aiming to investigate the effect of different variables in the performance of proteome wide phosphoprotein analysis protocols, ProteoRed-ISCIII and EuPA launched the Proteomics Multicentric Experiment 11 (PME11). A reference sample consisting of a yeast protein extract spiked in with different amounts of a phosphomix standard (Sigma/Merck) was distributed to 31 laboratories around the globe. Thirty-six datasets from 23 laboratories were analyzed. Our results indicate the suitability of the PME11 reference sample to benchmark and optimize phosphoproteomics strategies, weighing the influence of different factors, as well as to rank intra and inter laboratory performance.

The development of QERM scoring system for comprehensive assessment of the Quality of Empirical Research in Medicine - Part 1

Purpose

Whereas a large number of features are mentioned to connote the quality of medical research, no tool is available to comprehensively measure it objectively across different types of studies. Also, all the available tools are for reporting, and none includes quality of the inputs and the process of research. The present paper is aimed to initiate a discussion on the need to develop such a comprehensive scoring system (in the first place), to show that it is feasible, and to describe the process of developing a credible system.

Method

An expert group comprising researchers, reviewers, and editors of medical journals extensively reviewed the literature on the quality of medical research and held detailed discussions to parse quality at all stages of medical research into specific domains and items that can be assigned scores on the pattern of quality-of-life score.

Results

Besides identifying the domains of the quality of medical research, a comprehensive tool for scoring emerged that can be possibly used to objectively measure the quality of empirical research comprising surveys, trials, and observational studies. Thus, this can be used as a tool to assess Quality of Empirical Research in Medicine (QERM). The expert group confirmed its face and content validity. The tool can be used by the researchers for self-assessment and improvement before submission of a paper for publication, and the reviewers and editors can use this for assessing the submissions. Published papers can also be rated such as those included in a meta-analysis.

Conclusion

It is feasible to devise a comprehensive scoring system comprising domains and items for assessing the quality of medical research end-to-end from choosing a problem to publication. The proposed scoring system needs to be reviewed by the researchers and needs to be validated.

Simple Electric Device to Isolate Nucleic Acids from Whole Blood Optimized for Point of Care Testing of Brain Damage

BACKGROUND

Detection or monitoring of brain damage is a clinically crucial issue. Nucleic acids in the whole blood can be used as biomarkers for brain injury. Polymerase chain reaction (PCR) which is one of the most commonly used molecular diagnostic assays requires isolated nucleic acids to initiate amplification. Currently used nucleic acid isolation procedures are complicated and require laboratory equipments.

OBJECTIVE

In this study, we tried to develop a simple and convenient method to isolate nucleic acids from the whole blood sample using a tiny battery-powered electric device. The quality of the isolated nucleic acids should be suitable for PCR assay without extra preparation.

METHODS

A plastic device with separation chamber was designed and printed with a 3D printer. Two platinum electrodes were placed on both sides and a battery was used to supply the electricity. To choose the optimal nucleic acid isolation condition, diverse lysis buffers and separation buffers were evaluated, and the duration and voltage of the electricity were tested. Western blot analysis and PCR assay were used to determine the quality of the separated nucleic acids.

RESULTS

2ul of whole blood was applied to the cathode side of the separation chamber containing 78 ul of normal saline. When the electricity at 5 V was applied for 5 min, nucleic acids were separated from segment 1 to 3 of the separation chamber. The concentration of nucleic acids peaked around 7~8 mm from cathode side. PCR assay using the separation buffer as the template was performed successfully both in conventional and realtime PCR methods. The hemoglobin in the whole blood did not show the inhibitory effect in our separation system and it may be due to structural modification of hemoglobin during electric separation.

CONCLUSION

Our simple electric device can separate nucleic acids from the whole blood sample by applying electricity at 5 V for 5 min. The separation buffer solution taken from the device can be used for PCR assay successfully.

Best practices and tools for reporting reproducible fluorescence microscopy methods

Although fluorescence microscopy is ubiquitous in biomedical research, microscopy methods reporting is inconsistent and perhaps undervalued. We emphasize the importance of appropriate microscopy methods reporting and seek to educate researchers about how microscopy metadata impact data interpretation. We provide comprehensive guidelines and resources to enable accurate reporting for the most common fluorescence light microscopy modalities. We aim to improve microscopy reporting, thus improving the quality, rigor and reproducibility of image-based science.

Establishing a national strategy for shared research resources in biomedical sciences

Contemporary science has become increasingly multi-disciplinary and team-based, resulting in unprecedented growth in biomedical innovation and technology over the last several decades. Collaborative research efforts have enabled investigators to respond to the demands of an increasingly complex 21st century landscape, including pressing scientific challenges such as the COVID-19 pandemic. A major contributing factor to the success of team science is the mobilization of core facilities and shared research resources (SRRs), the scientific instrumentation and expertise that exist within research organizations that enable widespread access to advanced technologies for trainees, faculty, and staff. For over 40 years, SRRs have played a key role in accelerating biomedical research discoveries, yet a national strategy that addresses how to leverage these resources to enhance team science and achieve shared scientific goals is noticeably absent. We believe a national strategy for biomedical SRRs-led by the National Institutes of Health-is crucial to advance key national initiatives, enable long-term research efficiency, and provide a solid foundation for the next generation of scientists.