Why moral psychology needs personality psychology

People vary in how they perceive, think about, and respond to moral issues. Clearly, we cannot fully understand the psychology of morality without accounting for individual differences in moral functioning. But decades of neglect of and explicit skepticism toward such individual differences has resulted in a lack of integration between moral psychology and personality psychology-the study of psychological differences between people. In recent years, these barriers to progress have started to break down. This special issue aims to celebrate and further increase the visibility of the personality psychology of morality. Here, we introduce the articles in this special issue by highlighting some important contributions a personality-based perspective has to offer moral psychology-particularly in comparison to the currently prominent social psychological approach. We show that personality psychology is well-placed to (a) contribute toward a rigorous empirical foundation for moral psychology, (b) tackle the conceptualization and assessment of stable moral tendencies, (c) assess the predictive validity of moral traits in relation to consequential outcomes, (d) uncover the mechanisms underlying individual differences in moral judgments and behavior, and (e) provide insights into moral development. For these reasons, we believe that moral psychology needs personality psychology to reach its full scholarly potential.

Sex as a biological variable for cardiovascular physiology

There have been ongoing efforts by federal agencies and scientific communities since the early 1990s to incorporate sex and/or gender in all aspects of cardiovascular research. Scientific journals provide a critical function as change agents to influence transformation by encouraging submissions for topic areas, and by setting standards and expectations for articles submitted to the journal. As part of ongoing efforts to advance sex and gender in cardiovascular physiology research, the American Journal of Physiology-Heart and Circulatory Physiology recently launched a call for papers on Considering Sex as a Biological Variable. This call was an overwhelming success, resulting in 78 articles published in this collection. This review summarizes the major themes of the collection, including Sex as a Biological Variable Within: Endothelial Cell and Vascular Physiology, Cardiovascular Immunity and Inflammation, Metabolism and Mitochondrial Energy, Extracellular Matrix Turnover and Fibrosis, Neurohormonal Signaling, and Cardiovascular Clinical and Epidemiology Assessments. Several articles also focused on establishing rigor and reproducibility of key physiological measurements involved in cardiovascular health and disease, as well as recommendations and considerations for study design. Combined, these articles summarize our current understanding of sex and gender influences on cardiovascular physiology and pathophysiology and provide insight into future directions needed to further expand our knowledge.

Minimal information for studies of extracellular vesicles (MISEV2023): From basic to advanced approaches

Extracellular vesicles (EVs), through their complex cargo, can reflect the state of their cell of origin and change the functions and phenotypes of other cells. These features indicate strong biomarker and therapeutic potential and have generated broad interest, as evidenced by the steady year-on-year increase in the numbers of scientific publications about EVs. Important advances have been made in EV metrology and in understanding and applying EV biology. However, hurdles remain to realising the potential of EVs in domains ranging from basic biology to clinical applications due to challenges in EV nomenclature, separation from non-vesicular extracellular particles, characterisation and functional studies. To address the challenges and opportunities in this rapidly evolving field, the International Society for Extracellular Vesicles (ISEV) updates its 'Minimal Information for Studies of Extracellular Vesicles', which was first published in 2014 and then in 2018 as MISEV2014 and MISEV2018, respectively. The goal of the current document, MISEV2023, is to provide researchers with an updated snapshot of available approaches and their advantages and limitations for production, separation and characterisation of EVs from multiple sources, including cell culture, body fluids and solid tissues. In addition to presenting the latest state of the art in basic principles of EV research, this document also covers advanced techniques and approaches that are currently expanding the boundaries of the field. MISEV2023 also includes new sections on EV release and uptake and a brief discussion of in vivo approaches to study EVs. Compiling feedback from ISEV expert task forces and more than 1000 researchers, this document conveys the current state of EV research to facilitate robust scientific discoveries and move the field forward even more rapidly.

Psychometric properties of the TACT framework-Determining rigor in qualitative research

Introduction

The credibility of qualitative research has long been debated, with critics emphasizing the lack of rigor and the challenges of demonstrating it. In qualitative research, rigor encompasses explicit, detailed descriptions of various research stages, including problem framing, study design, data collection, analysis, and reporting. The diversity inherent in qualitative research, originating from various beliefs and paradigms, challenges establishing universal guidelines for determining its rigor. Additionally, researchers' often unrecorded thought processes in qualitative studies further complicate the assessment of research quality.

Methods

To address these concerns, this article builds on the TACT framework, which was developed to teach postgraduate students and those new to qualitative research to identify and apply rigorous principles and indicators in qualitative research. The research reported in this article focuses on creating a scale designed to evaluate the psychometric properties of the TACT framework. This involves analyzing the stability of its dimensions and understanding its effectiveness as a tool for teaching and research.

Results

The study's findings indicate that the TACT framework, when assessed through the newly developed scale, exhibits stable dimensions consistent with rigorous qualitative research principles. The framework effectively guides postgraduate students and new researchers in assessing the rigor of qualitative research processes and outcomes.

Discussion

The application of the TACT framework and its evaluation scale reveals several insights. Firstly, it demonstrates the framework's utility in bridging the gap in pedagogical tools for teaching rigor in qualitative research methods. Secondly, it highlights the framework's potential in providing a structured approach to undertaking qualitative research, which is essential given this field's diverse methodologies and paradigms. However, the TACT framework remains a guide to enhancing rigor in qualitative research throughout all the various phases but by no means a measure of rigor.

Conclusion

In conclusion, the TACT framework and its accompanying evaluative scale represent significant steps toward standardizing and enhancing the rigor of qualitative research, particularly for postgraduate students and early career researchers. While it does not solve all challenges associated with obtaining and demonstrating rigor in qualitative research, it provides a valuable tool for assessing and ensuring research quality, thereby addressing some of the longstanding criticisms of the quality of research obtained through qualitative methods.

Seeing beyond the blot: A critical look at assumptions and raw data interpretation in Western blotting

Rapid advancements in technology refine our understanding of intricate biological processes, but a crucial emphasis remains on understanding the assumptions and sources of uncertainty underlying biological measurements. This is particularly critical in cell signaling research, where a quantitative understanding of the fundamental mechanisms governing these transient events is essential for drug development, given their importance in both homeostatic and pathogenic processes. Western blotting, a technique developed decades ago, remains an indispensable tool for investigating cell signaling, protein expression, and protein-protein interactions. While improvements in statistical analysis and methodology reporting have undoubtedly enhanced data quality, understanding the underlying assumptions and limitations of visual inspection in Western blotting can provide valuable additional information for evaluating experimental conclusions. Using the example of agonist-induced receptor post-translational modification, we highlight the theoretical and experimental assumptions associated with Western blotting and demonstrate how raw blot data can offer clues to experimental variability that may not be fully captured by statistical analyses and reported methodologies. This article is not intended as a comprehensive technical review of Western blotting. Instead, we leverage an illustrative example to demonstrate how assumptions about experimental design and data normalization can be revealed within raw data and subsequently influence data interpretation.

Tenth Aquatic Models of Human Disease Conference 2022 Workshop Report: Aquatics Nutrition and Reference Diet Development

Progress in biomedical research requires rigorous studies and reproducible outcomes. However, despite recent achievements, standard reference diets (SRDs) for aquatic model organisms, vital for supporting scientific rigor and reproducibility, are yet to be adopted. At this workshop, we presented findings from a 7-month diet test study, tightly coordinated and conducted across three aquatic research facilities: Zebrafish International Resource Center (ZIRC), Kent and Sharpton laboratories (Oregon State University), and Xiphophorus Genetic Stock Center (XGSC, Texas State University). We compared the impact of two commercial diets and a suggested zebrafish SRD on general fish husbandry, microbiome composition, and health in three fish species (zebrafish, Xiphophorus, and Medaka), and three zebrafish wild-type strains. We reported outcomes, gathered community feedback, and addressed the aquatic research community's need for SRD development. Discussions underscored the influence of diet on aquatic research variability, emphasizing the need for SRDs to control cross-experiment and cross-laboratory reproducibility. Species-specific reference diets are essential for model organism health and consistent research outcomes.

Incorrect analysis in "Effects of the application of a food processing-based classification system in obese women: A randomized controlled pilot study" has resulted in incorrect conclusions of demonstrated effects where no such effects have been demonstrated

In their 2023 Nutrition and Health paper "Effects of the application of a food processing-based classification system in obese women: A randomized controlled pilot study", Giacomello et al. investigated the effects of an educational intervention based on the Dietary Guidelines for the Brazilian Population among obese women. The authors concluded that the intervention significantly improved weight loss, quality of life, components of metabolic syndrome, and pain. However, we believe the statistical analysis employed in the study was flawed. The authors used within-group changes to draw conclusions, which is known as a difference in nominal significance error. This error has the potential to inflate Type I error rates substantially. To address this issue, we re-analyzed the data obtained from the authors. We focused on body mass and hip circumference and replicated the incorrectly chosen within-group analyses, which remained significant. However, to properly evaluate the intervention's effectiveness, it is essential to compare the differences between the groups directly. Therefore, we calculated change scores for each participant and used independent samples t-tests and linear mixed models to compare between-group differences. Both methods yielded similar non-significant p-values, indicating that there is no significant effect of treatment on body mass or hip circumference. The original paper's conclusions regarding the effectiveness of the intervention are not supported by the proper statistical analysis. The data should be re-analyzed using appropriate between-group comparisons, and the corrected results should be published, or the incorrect results and original paper should be retracted.

Transparency in Infectious Disease Research: Meta-research Survey of Specialty Journals

BACKGROUND

Infectious diseases carry large global burdens and have implications for society at large. Therefore, reproducible, transparent research is extremely important.

METHODS

We evaluated transparency indicators (code and data sharing, registration, and conflict and funding disclosures) in the 5340 PubMed Central Open Access articles published in 2019 or 2021 in the 9 most cited specialty journals in infectious diseases using the text-mining R package, rtransparent.

RESULTS

A total of 5340 articles were evaluated (1860 published in 2019 and 3480 in 2021 [of which 1828 were on coronavirus disease 2019, or COVID-19]). Text mining identified code sharing in 98 (2%) articles, data sharing in 498 (9%), registration in 446 (8%), conflict of interest disclosures in 4209 (79%), and funding disclosures in 4866 (91%). There were substantial differences across the 9 journals: 1%-9% for code sharing, 5%-25% for data sharing, 1%-31% for registration, 7%-100% for conflicts of interest, and 65%-100% for funding disclosures. Validation-corrected imputed estimates were 3%, 11%, 8%, 79%, and 92%, respectively. There were no major differences between articles published in 2019 and non-COVID-19 articles in 2021. In 2021, non-COVID-19 articles had more data sharing (12%) than COVID-19 articles (4%).

CONCLUSIONS

Data sharing, code sharing, and registration are very uncommon in infectious disease specialty journals. Increased transparency is required.

Open Science in eating disorders: Using current evidence to inspire a plan for increasing the transparency of our research

OBJECTIVE

There is increasing consensus that open science practices improve the transparency and quality of clinical science. However, several barriers impede the implementation of these practices at the individual, institutional, and field levels; understanding and addressing these barriers is critical to promoting targeted efforts in increasing effective uptake of open science.

METHODS

Within this research forum, we drew from publicly available online information sources to identify initial characterizations of researchers engaged in several types of open science practices in the field of eating disorders. We use these observations to discuss potential barriers and recommendations for next steps in the promotion of these practices.

RESULTS

Data from online open science repositories suggest that individuals using these publishing approaches with pre-prints and articles with eating-disorder-relevant content are predominantly non-male gender identifying, early to mid-career stage, and are more likely to be European-, United States-, or Canada-based.

DISCUSSION

We outline recommendations for tangible ways that the eating disorder field can support broad, increased uptake of open science practices, including supporting initiatives to increase knowledge and correct misconceptions; and prioritizing the development and accessibility of open science resources.

PUBLIC SIGNIFICANCE STATEMENT

The use of open science practices has the potential to increase the transparency and quality of clinical science. This Forum uses publicly sourced online data to characterize researchers engaged in open science practices in the field of eating disorders. These observations provide an important framework from which to discuss potential barriers to open science and recommendations for next steps in the promotion of these practices.

Rigor, Reproducibility, and Transparency in Shared Research Resources: Follow-Up Survey and Recommendations for Improvements

Rigor, reproducibility, and transparency (RR&T) are essential components of all scientific pursuits. Shared research resources, also known as core facilities, are on the frontlines of ensuring robust RR&T practices. The Association of Biomolecular Resource Facilities Committee on Core Rigor and Reproducibility conducted a follow-up survey 4 years after the initial 2017 survey to determine if core facilities have seen a positive impact of new RR&T initiatives (including guidance from the National Institutes of Health, new scientific journal requirements on transparency and data provenance, and educational tools from professional organizations). While there were fewer participants in the most recent survey, the respondents' opinions on the role of core facilities and level of best practices adoption remained the same. Overall, the respondents agreed that procedures should be implemented by core facilities to ensure scientific RR&T. They also indicated that there is a strong correlation between institutions that emphasize RR&T and core customers using this expertise in grant applications and publications. The survey also assessed the impact of the COVID-19 pandemic on core operations and RR&T. The answers to these pandemic-related questions revealed that many of the strategies aimed at increasing efficiencies are also best practices related to RR&T, including the development of standard operating procedures, supply chain management, and cross training. Given the consistent and compelling awareness of the importance of RR&T expressed by core directors in 2017 and 2021 contrasted with the lack of apparent improvements over this time period, the authors recommend an adoption of RR&T statements by all core laboratories. Adhering to the RR&T guidelines will result in more efficient training, better compliance, and improved experimental approaches empowering cores to become "rigor champions."

Casting New Light on Statistical Power: An Illuminating Analogy and Strategies to Avoid Underpowered Trials

Current standards for methodological rigor and trial reporting underscore the critical issue of statistical power. Still, the chance of detecting most effects reported in randomized controlled trials in medicine and other disciplines is currently lower than winning a toss of a fair coin. Here we propose that investigators who retain a practical understanding of how statistical power works can proactively avoid the potentially devastating consequences of underpowered trials. We first offer a vivid, carefully constructed analogy that illuminates the underlying relationships among 3 of the 5 essential parameters-namely, statistical power, effect size, and sample size-while holding the remaining 2 parameters constant (type of statistical test and significance level). Second, we extend the analogy to a set of critical scenarios in which investigators commonly miss detecting intervention effects due to insufficient statistical power. Third, we highlight effective pragmatic strategies for the design and conduct of sufficiently powered trials, without increasing sample size.

Moving Towards FAIR Data Practices in Pharmacy Education

Pharmacy schools are generating significant amounts of data across the training continuum, including data about student selection, performance, and job placement. However, current data practices limit the Academy's ability to effectively leverage the vast amounts of data available within and across pharmacy institutions. To improve data practices and promote the quality and reusability of data, a set of guiding principles for data management and stewardship were developed and published in 2016. The FAIR principles state that digital objects should be findable (ie, data have a unique identifier and are registered in a searchable resource), accessible (ie, data are retrievable by their identifier using an open, free, standardized protocol), interoperable (ie, data use a formal, accessible, shared, and broadly applicable language, and include qualified references to other data), and reusable (ie, data are described with accurate and relevant attributes, released with a data usage license, and meet domain-relevant community standards). This commentary advocates for improved data practices and provides recommendations for advancing FAIR data principles in pharmacy education.

Good clinical practice improves rigor and transparency: Lessons from the ACTIVE trial

Clinical trials are governed by principles of good clinical practice (GCP), which can strengthen the achievement of rigor, reproducibility, and transparency in scientific research. Rigor, reproducibility, and transparency are key for producing findings with greater certainty. Clinical trials are closely supervised, often by a clinical trial coordinating center, data safety and monitoring board, and a funding agency, with policies that are a manifestation of GCP and support rigor, reproducibility, and transparency. The multisite Advanced Cognitive Training for Independent and Vital Elderly (ACTIVE) study is an example clinical trial of relevance to a psychology and aging audience that utilized many protocols that can be applied to single-laboratory designs, including a manualized protocol with accompanying scientific rationale, predefined analysis plans, standardization of procedures across field sites, assurance of competence of study staff in study procedures, transparent coding/entry/transmittal of data, regular quality assurance, and open publication of data. Despite substantial resource discrepancies between the two, single-laboratory studies can model the GCP principles utilized in large clinical trials to provide an excellent foundation for rigor, reproducibility, and transparency. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Updating MISEV: Evolving the minimal requirements for studies of extracellular vesicles

The minimal information for studies of extracellular vesicles (EVs, MISEV) is a field-consensus rigour initiative of the International Society for Extracellular Vesicles (ISEV). The last update to MISEV, MISEV2018, was informed by input from more than 400 scientists and made recommendations in the six broad topics of EV nomenclature, sample collection and pre-processing, EV separation and concentration, characterization, functional studies, and reporting requirements/exceptions. To gather opinions on MISEV and ideas for new updates, the ISEV Board of Directors canvassed previous MISEV authors and society members. Here, we share conclusions that are relevant to the ongoing evolution of the MISEV initiative and other ISEV rigour and standardization efforts.

Optimization of capsaicin-induced dermal blood flow measurement by laser Doppler imaging in cynomolgus macaque

BACKGROUND

Capsaicin is used in several areas of non-human primate research including allodynia and dermal blood flow (DBF). The capsaicin-induced DBF increase was measured using laser Doppler imaging (LDI), but this response is known to diminish upon repeated topical applications. Refinement of the experimental procedures could improve the rigor and reproducibility of the DBF migraine model.

METHODS

Optimal anatomical site in cynomolgus was determined, and conditions and experimental settings for DBF measurement using LDI were established. Then, two study design trial structures were compared.

RESULTS

Medial thigh was the preferrable site, and an ethanol-Tween 20 formulation of capsaicin was desirable. A 1-week washout for contralateral side or 2-week washout for ipsilateral side was necessary to eradicate capsaicin desensitization.

CONCLUSIONS

With the established technicality in DBF measurements in cynomolgus macaques, the capsaicin-induced DBF model may be utilized in translational medical research in developing migraine therapeutics.

The science of science: Clinical Science launches a new translational meta-research collection

Clinical Science is proud to launch a new translational meta-research collection. Meta-research, or the science of science, applies the scientific method to study science itself. Meta-research is a powerful tool for identifying common problems in scientific papers, assessing their impact, and testing solutions to improve the transparency, rigor, trustworthiness, and usefulness of biomedical research. The collection welcomes science of science studies that link basic science to disease mechanisms, as well as meta-research articles highlighting opportunities to improve transparency, rigor, and reproducibility among the types of papers published in Clinical Science. Submissions might include science of science studies that explore factors linked to successful translation, or meta-research on experimental methods or study designs that are often used in translational research. We hope that this collection will encourage scientists to think critically about current practices and take advantage of opportunities to make their own research more transparent, rigorous, and reproducible.

A community-led initiative for training in reproducible research

Open and reproducible research practices increase the reusability and impact of scientific research. The reproducibility of research results is influenced by many factors, most of which can be addressed by improved education and training. Here we describe how workshops developed by the Reproducibility for Everyone (R4E) initiative can be customized to provide researchers at all career stages and across most disciplines with education and training in reproducible research practices. The R4E initiative, which is led by volunteers, has reached more than 3000 researchers worldwide to date, and all workshop materials, including accompanying resources, are available under a CC-BY 4.0 license at https://www.repro4everyone.org/.

How to Conduct Responsible Research: A Guide for Graduate Students

Researchers must conduct research responsibly for it to have an impact and to safeguard trust in science. Essential responsibilities of researchers include using rigorous, reproducible research methods, reporting findings in a trustworthy manner, and giving the researchers who contributed appropriate authorship credit. This "how-to" guide covers strategies and practices for doing reproducible research and being a responsible author. The article also covers how to utilize decision-making strategies when uncertain about the best way to proceed in a challenging situation. The advice focuses especially on graduate students, but is appropriate for undergraduates and experienced researchers. It begins with an overview of responsible conduct of research, research misconduct, and ethical behavior in the scientific workplace. The takeaway message is that responsible conduct of research requires a thoughtful approach to doing research in order to ensure trustworthy results and conclusions, and that researchers receive fair credit. © 2021 Wiley Periodicals LLC.

Survey of Nursing Faculty Preparation for Guiding DNP Quality Improvement Projects

BACKGROUND

Quality improvement (QI) projects comprise the majority of University of Maryland School of Nursing (UMSON) Doctor of Nursing Practice (DNP) projects.

METHODS

An online survey was completed by 51% (n = 38) of faculty, who teach or mentor DNP students, and was analyzed using quantitative and descriptive methods.

RESULTS

Faculty were somewhat or not familiar with developing a QI charter 68.4%, human error theory and error proofing 63.2%, driver diagrams 60.5%, characteristics of high-reliability organizations 60.5%, and Standards for Quality Improvement Reporting Excellence (SQUIRE) guidelines 55.3%. The faculty were most interested in learning more about (n = 97 responses) were human error theory and error proofing (28.9%), SQUIRE guidelines (26.3%), statistical process control (21.1%), and implementation strategies and tactics (21.1%). The most commonly identified challenges included identifying QI projects (24%), project time constraints (16%), keeping up-to-date on QI concepts, methods, and tools (12%), and balancing professional workload (10%).

CONCLUSIONS

Gaps in self-reported QI knowledge indicate there is a need for further development of DNP and PhD prepared faculty at the UMSON.

Exclusion of Expert Contributors From Authorship Limits the Quality of Scientific Articles

Veterinary pathologists are key contributors to multidisciplinary biomedical research. However, they are occasionally excluded from authorship in published articles despite their substantial intellectual and data contributions. To better understand the potential origins and implications of this practice, we identified and analyzed 29 scientific publications where the contributing pathologist was excluded as an author. The amount of pathologist-generated data contributions were similar to the calculated average contributions for authors, suggesting that the amount of data contributed by the pathologist was not a valid factor for their exclusion from authorship. We then studied publications with pathologist-generated contributions to compare the effects of inclusion or exclusion of the pathologist as an author. Exclusion of the pathologist from authorship was associated with significantly lower markers of rigor and reproducibility compared to articles in which the pathologist was included as author. Although this study did not find justification for the exclusion of pathologists from authorship, potential consequences of their exclusion on data quality were readily detectable.

The quality of systematic reviews and meta-analyses assessing the treatment and management of male infertility

BACKGROUND

Male factor infertility (MFI) is a common medical condition which requires high-quality research to guide clinical practice; however, systematic reviews (SRs) and meta-analyses (MAs) often vary in quality, raising concerns regarding the validity of their results. We sought to perform an objective analysis of SRs and MAs in MFI treatment and management and to report on the quality of published literature.

METHODS

A comprehensive search in PubMed/MEDLINE and Embase was used to identify relevant publications. Primary search terms were male infertility, male sterility, and male subfertility. Two authors independently performed searches, screened citations for eligibility, extracted data for analysis, and graded methodological quality using the validated AMSTAR (A Measurement Tool to Assess Systematic Reviews) instrument, a validated tool used in the critical appraisal of SRs/MAs.

RESULTS

Of 27 publications met inclusion criteria and were included in the analysis. Mean AMSTAR score (± SD) among all publications was 7.4 (1.9) out of 11, reflecting "fair to good" quality. Non-pharmacological medical treatment for MFI was the most commonly assessed intervention (n = 13, 48.1%). No publications met all AMSTAR criteria. While the number of SRs/MAs has increased over time (P = 0.037), the quality of publications has not significantly changed (P = 0.72). SRs/MAs of the Cochrane Library had higher AMSTAR score than non-Cochrane SRs/MAs (8.5 vs 6.3, P = 0.002).

CONCLUSIONS

The methodological quality of SRs/MAs should be assessed to ensure high-quality evidence for clinical practice guidelines in MFI treatment and management. This review highlights a need for increased effort to publish high-quality studies in MFI treatment and management.

Time to fiddle with your unpublished data

Most scientific investigators conduct well-designed and controlled preclinical experiments generating data that are difficult to explain, contrast with existing scientific dogma, or represent a perceived negative result. It is common for these findings to remain hidden away in a drawer from the greater scientific community. However, these unseen results can lead to publication bias, have the potential to significantly advance scientific disciplines if they are published, and can help investigators avoid repeating experiments that have already been done, thus saving money and time. Moreover, these unexpected data may actually have significance if re-interpreted leading to new hypotheses. This editorial commentary highlights a novel user-friendly tool developed by Bernard and colleagues (REF) to help investigators determine appropriate options for disseminating unpublished data in order to make them available to the broader scientific community. In addition, this commentary serves as an announcement for an upcoming special call for papers on meta-research to be published in Clinical Science. Meta-research is the evaluation and study of existing scientific literature and data. It is an evolving field dedicated to improving rigor and reproducibility in science, an endeavor to which Clinical Science and Portland Press are committed.

Recommendations to enhance rigor and reproducibility in biomedical research

Biomedical research depends increasingly on computational tools, but mechanisms ensuring open data, open software, and reproducibility are variably enforced by academic institutions, funders, and publishers. Publications may present software for which source code or documentation are or become unavailable; this compromises the role of peer review in evaluating technical strength and scientific contribution. Incomplete ancillary information for an academic software package may bias or limit subsequent work. We provide 8 recommendations to improve reproducibility, transparency, and rigor in computational biology-precisely the values that should be emphasized in life science curricula. Our recommendations for improving software availability, usability, and archival stability aim to foster a sustainable data science ecosystem in life science research.

Getting Rigor Right

A colleague of mine once shared a story he heard of an organic chemistry teacher who would tell people he had one of the most important, lifesaving jobs in the world. Why? Because he made sure academically unprepared premed students never made it to medical school. Teachers can have many possible motivations for maintaining high academic standards in their courses. A desire to ensure future health practitioners are adequately prepared to engage in the intellectual rigor of their profession may be just one of them.

Rigor in Qualitative Methods: An Evaluation of Strategies Among Underrepresented Rural Communities

Achieving rigor using selected criteria to determine trustworthiness using qualitative methods has been without critical evaluation. In this article, strategies such as prolonged engagement and thick, rich description; negative case analysis; peer review or briefing; clarifying researcher bias; member checking; and, investigator triangulation and intercoder reliability are evaluated for appropriateness among an African American Appalachian rural population. Achieving rigor using qualitative methods among participants living in rural communities is time intensive requiring attention to quality versus quantity of time spent in interviews, building trusting relationships, an awareness of interviewer bias and assumptions, and appropriately evaluated strategies that enhance validity. Strategies to achieve rigor in qualitative methods should not be used as a one size fits all approach as this practice might actually diminish rigor. Among underrepresented populations, strategies should be adapted or not used at all.

Reproducibility and Rigor in Animal-Based Research

Increasing focus on issues of research reproducibility affords us the opportunity to review some of the key issues related in vivo research. First, we set out some key definitions, to guide the reader through the rest of the paper. Next we consider issues of epistemology, of how animal experiments lead to changes in our understanding of biomedicine and, potentially, to the development of new therapeutics. Here we consider the meaning of statistical significance; the importance of understanding whether findings have general truth; and the advances in knowledge which can result from 'failed' replication. Then, we consider weaknesses in the design, conduct and reporting of experiments, and review evidence for this from systematic reviews and from experimental studies addressing these issues. We consider the impact that these weaknesses have on the development of new treatments for human disease, and reflect on the response to these issues from the biomedical research community. Finally, we consider strategies for improvement including increased use of brief, pre-registered study protocols; pre-registration, open publication and open data; and the central importance of education in improving research performance.

MiSet RFC Standards: Defining a Universal Minimum Set of Standards Required for Reproducibility and Rigor in Research Flow Cytometry Experiments

Poor adherence to best practices, insufficient training, and pressure to produce data quickly may lead to publications of suboptimal biomedical research flow cytometry data, which contributes to the body of irreproducible research findings. In addition, documentation of compliance with best flow cytometry practices for submission, visualization, and publication of flow cytometry data is currently endorsed by very few scientific journals, which is particularly concerning as numerous peer-reviewed flow cytometry publications emphasize instrumentation, experimental design, and data analysis as important sources of variability. Guidelines and resources for adequate reporting, annotation and deposition of flow cytometry experiments are provided by MIFlowCyt and the FlowRepository database, and comprehensive expert recommendations covering principles and techniques of field-specific flow cytometry applications have been published. To facilitate the integration of quality-defining parameters into manuscript and grant submission and publication requirements across biomedical fields that rely on the use of flow-cytometry-based techniques, a single comprehensive yet easily and universally applicable document is needed. To produce such a list of gold-standard parameters that assess whether a research flow cytometry experiment has been planned, conducted, interpreted, and reported at the highest standard, a new initiative defining the minimum set of standards a robust and rigorous research flow experiment must fulfill (MiSet RFC Standards) was proposed at CYTO 2019. MiSet RFC Standards will integrate and simplify existing resources to provide a universal benchmark a flow cytometry experiment can easily be measured against. The goal of MiSET RFC Standards is its integration into peer-review and publication procedures through partnership with stakeholders, journals and publishers in biomedical and translational research. This article introduces the aims and anticipated timeline and discusses strategies for interdisciplinary consensus and implementation. A single-resource broadly applicable guideline will harmonize standards across different fields of biomedical research and lead to publication of more robust research findings. © 2019 International Society for Advancement of Cytometry.

Survey on Scientific Shared Resource Rigor and Reproducibility

Shared scientific resources, also known as core facilities, support a significant portion of the research conducted at biomolecular research institutions. The Association of Biomolecular Resource Facilities (ABRF) established the Committee on Core Rigor and Reproducibility (CCoRRe) to further its mission of integrating advanced technologies, education, and communication in the operations of shared scientific resources in support of reproducible research. In order to first assess the needs of the scientific shared resource community, the CCoRRe solicited feedback from ABRF members via a survey. The purpose of the survey was to gain information on how U.S. National Institutes of Health (NIH) initiatives on advancing scientific rigor and reproducibility influenced current services and new technology development. In addition, the survey aimed to identify the challenges and opportunities related to implementation of new reporting requirements and to identify new practices and resources needed to ensure rigorous research. The results revealed a surprising unfamiliarity with the NIH guidelines. Many of the perceived challenges to the effective implementation of best practices (i.e., those designed to ensure rigor and reproducibility) were similarly noted as a challenge to effective provision of support services in a core setting. Further, most cores routinely use best practices and offer services that support rigor and reproducibility. These services include access to well-maintained instrumentation and training on experimental design and data analysis as well as data management. Feedback from this survey will enable the ABRF to build better educational resources and share critical best-practice guidelines. These resources will become important tools to the core community and the researchers they serve to impact rigor and transparency across the range of science and technology.

Concept Analysis of Rigor in Nursing Education

The term rigor appears in nursing education literature, but the definitions are inconsistent. Likewise, there is variation in the implementation of rigor in nursing curricula and courses. Nurse educators need a clear and shared understanding of rigor to prepare graduates to use critical thinking. The purpose of this concept analysis is to gain understanding of the meaning of rigor in nursing education so that educators can establish a standardized definition, which will contribute to consistency of its application in learning environments with uniform criteria for curricula evaluation.

Rigor and Reproducibility: A Systematic Review of Salivary Cortisol Sampling and Reporting Parameters Used in Cancer Survivorship Research

Salivary cortisol is a commonly used biomarker in cancer survivorship research; however, variations in sampling protocols and parameter reporting limit comparisons across studies. Standardized practices to provide rigor and reproducibility of diurnal salivary cortisol sampling and reporting are not well established. Previous systematic reviews examining relationships between diurnal salivary cortisol and clinical outcomes have resulted in mixed findings. It remains unclear which sampling protocols and reporting parameters offer the greatest utility for clinical research. This review examines diurnal salivary cortisol sampling protocols and reporting parameters to evaluate whether a standardized approach is recommended. A comprehensive search of intervention studies among adult cancer survivors including diurnal salivary cortisol resulted in 30 articles for review. Sampling protocols ranged from 1 to 4 days with the majority of studies sampling cortisol for 2 days. Sampling instances ranged from 2 to 7 times per day, with the majority collecting at 4 time points per day. Diurnal cortisol slope and cortisol awakening response (CAR) were the most commonly reported parameters associated with clinical outcomes. Flattened cortisol slopes, blunted CARs, and elevated evening cortisol concentrations were associated with poorer psychosocial and physiological outcomes. Based on our review, we propose that a rigorous, standardized diurnal salivary cortisol sampling protocol should include sampling at key diurnal times across ≥3 consecutive days to report diurnal cortisol parameters (i.e., CAR and slope) and objective measures of participant protocol adherence. Diminishing budgetary resources and efforts to minimize participant burden dictate the importance of standardized cortisol sampling protocols and reporting parameters.

Incidences of problematic cell lines are lower in papers that use RRIDs to identify cell lines

The use of misidentified and contaminated cell lines continues to be a problem in biomedical research. Research Resource Identifiers (RRIDs) should reduce the prevalence of misidentified and contaminated cell lines in the literature by alerting researchers to cell lines that are on the list of problematic cell lines, which is maintained by the International Cell Line Authentication Committee (ICLAC) and the Cellosaurus database. To test this assertion, we text-mined the methods sections of about two million papers in PubMed Central, identifying 305,161 unique cell-line names in 150,459 articles. We estimate that 8.6% of these cell lines were on the list of problematic cell lines, whereas only 3.3% of the cell lines in the 634 papers that included RRIDs were on the problematic list. This suggests that the use of RRIDs is associated with a lower reported use of problematic cell lines.

Minimal information for studies of extracellular vesicles 2018 (MISEV2018): a position statement of the International Society for Extracellular Vesicles and update of the MISEV2014 guidelines

The last decade has seen a sharp increase in the number of scientific publications describing physiological and pathological functions of extracellular vesicles (EVs), a collective term covering various subtypes of cell-released, membranous structures, called exosomes, microvesicles, microparticles, ectosomes, oncosomes, apoptotic bodies, and many other names. However, specific issues arise when working with these entities, whose size and amount often make them difficult to obtain as relatively pure preparations, and to characterize properly. The International Society for Extracellular Vesicles (ISEV) proposed Minimal Information for Studies of Extracellular Vesicles ("MISEV") guidelines for the field in 2014. We now update these "MISEV2014" guidelines based on evolution of the collective knowledge in the last four years. An important point to consider is that ascribing a specific function to EVs in general, or to subtypes of EVs, requires reporting of specific information beyond mere description of function in a crude, potentially contaminated, and heterogeneous preparation. For example, claims that exosomes are endowed with exquisite and specific activities remain difficult to support experimentally, given our still limited knowledge of their specific molecular machineries of biogenesis and release, as compared with other biophysically similar EVs. The MISEV2018 guidelines include tables and outlines of suggested protocols and steps to follow to document specific EV-associated functional activities. Finally, a checklist is provided with summaries of key points.

Why Is It so Hard to Do Good Science?

"Good science" means answering important questions convincingly, a challenging endeavor under the best of circumstances. Our inability to replicate many biomedical studies has been the subject of numerous commentaries both in the scientific and lay press. In response, statistics has re-emerged as a necessary tool to improve the objectivity of study conclusions. However, psychological aspects of decision making introduce preconceived preferences into scientific judgment that cannot be eliminated by any statistical method. The psychology of decision making, expounded by Kahneman, Tversky, and Thaler, is well known in the field of economics, but the underlying concepts of cognitive psychology are also relevant to scientific judgments. I repeated experiments carried out on undergraduates by Kahneman and colleagues four to five decades ago, but with scientists, and obtained essentially the same results. The experiments were in the form of written reactions to scenarios, and participants were scientists at all career stages. The findings reinforce the roles that two inherent intuitions play in scientific decision making: our drive to create a coherent narrative from new data regardless of its quality or relevance and our inclination to seek patterns in data whether they exist or not. Moreover, we do not always consider how likely a result is regardless of its p value. Low statistical power and inattention to principles underpinning Bayesian statistics reduce experimental rigor, but mitigating skills can be learned. Overcoming our natural human tendency to make quick decisions and jump to conclusions is a deeper obstacle to doing good science; this too can be learned.

Multi-Center Pre-clinical Consortia to Enhance Translation of Therapies and Biomarkers for Traumatic Brain Injury: Operation Brain Trauma Therapy and Beyond

Current approaches have failed to yield success in the translation of neuroprotective therapies from the pre-clinical to the clinical arena for traumatic brain injury (TBI). Numerous explanations have been put forth in both the pre-clinical and clinical arenas. Operation Brain Trauma Therapy (OBTT), a pre-clinical therapy and biomarker screening consortium has, to date, evaluated 10 therapies and assessed three serum biomarkers in nearly 1,500 animals across three rat models and a micro pig model of TBI. OBTT provides a unique platform to exploit heterogeneity of TBI and execute the research needed to identify effective injury specific therapies toward precision medicine. It also represents one of the first multi-center pre-clinical consortia for TBI, and through its work has yielded insight into the challenges and opportunities of this approach. In this review, important concepts related to consortium infrastructure, modeling, therapy selection, dosing and target engagement, outcomes, analytical approaches, reproducibility, and standardization will be discussed, with a focus on strategies to embellish and improve the chances for future success. We also address issues spanning the continuum of care. Linking the findings of optimized pre-clinical consortia to novel clinical trial designs has great potential to help address the barriers in translation and produce successes in both therapy and biomarker development across the field of TBI and beyond.

Rigor and reproducibility via laboratory studies of eating behavior: A focused update and conceptual review

OBJECTIVE

The eating behavior of individuals with eating disorders has been examined in laboratory settings over the last 30 years. In this focused review, we build on prior research and highlight several feeding laboratory paradigms that have successfully demonstrated quantifiable and observable behavioral disturbances, and thereby add rigor and reproducibility to the examination of disturbances of eating behavior. This review describes the measures commonly obtained via these laboratory techniques. Supporting Information Appendices with detailed information about implementation are provided to allow for the reproducible execution of these techniques across labs.

METHODS/RESULTS

Literature documenting the existence of objective abnormalities in eating behavior among individuals with eating disorders or in comparison to healthy controls (n > 40) is briefly summarized. These protocols, conducted across at least 17 independent labs, are sensitive and reproducible, can be used to assess subjective and physiological parameters associated with eating, and elucidate the impact of treatment. Laboratory studies from patients with eating disorders compared with healthy controls reproducibly demonstrate both that patients with Anorexia Nervosa ingest fewer calories and that individuals with Bulimia Nervosa and Binge-Eating Disorder ingest more calories when asked to binge-eat.

DISCUSSION

Feeding laboratory studies have the potential for quantifying the characteristic behavioral psychopathology of patients with eating disorders, and may provide a useful tool to explore the potential utility of new treatments for individuals with Anorexia Nervosa, Bulimia Nervosa, and Binge-Eating Disorder.

Issues with data and analyses: Errors, underlying themes, and potential solutions

Some aspects of science, taken at the broadest level, are universal in empirical research. These include collecting, analyzing, and reporting data. In each of these aspects, errors can and do occur. In this work, we first discuss the importance of focusing on statistical and data errors to continually improve the practice of science. We then describe underlying themes of the types of errors and postulate contributing factors. To do so, we describe a case series of relatively severe data and statistical errors coupled with surveys of some types of errors to better characterize the magnitude, frequency, and trends. Having examined these errors, we then discuss the consequences of specific errors or classes of errors. Finally, given the extracted themes, we discuss methodological, cultural, and system-level approaches to reducing the frequency of commonly observed errors. These approaches will plausibly contribute to the self-critical, self-correcting, ever-evolving practice of science, and ultimately to furthering knowledge.

Elegant Science

Elegance is a prized quality in science that is associated with simplicity and explanatory power. This essay considers the qualities that make a scientific model, experiment, method, or theory "elegant," with a focus on the life sciences. We propose a definition of elegance that includes clarity, cleverness, correctness, explanatory power, parsimony, and beauty. The pursuit of elegance can improve the quality of science, but elegance must be pursued with caution, as the truth is sometimes inelegant.

From Reproducibility to Translation in Neurodegenerative Disease

Despite tremendous investment and preclinical success in neurodegenerative disease, effective disease-altering treatments for patients have remained elusive. One highly cited reason for this discrepancy is flawed animal study design and reporting. If this can be broadly remedied, reproducibility of preclinical studies will improve. However, without concurrent efforts to improve generalizability, these improvements may not translate effectively from animal experiments to more complex human neurodegenerative diseases. Mechanistic and phenotypic variability of neurodegenerative disease is such that most models are only able to interrogate individual aspects of complex phenomena. One approach is to consider animals as models of individual targets rather than as models of individual diseases and to migrate the concept of predictive validity from the individual model to the body of experiments that demonstrate translatability of a target. Both exploratory and therapeutic preclinical studies are dependent upon study design methods that promote rigor and reproducibility. However, the body of evidence that is needed to demonstrate efficacy in therapeutic studies is substantially broader than that needed for exploratory studies. In addition to requiring rigor within individual experiments, convincing evidence for therapeutic potential must assess the relationships between model choice, intended goal of the intervention, pharmacologic criteria, and integration of biomarker data with outcome measures that are clinically relevant to humans. It is conceivable that proof-of-concept studies will migrate to cell-based systems and that animal systems will be increasingly reserved for more distal translational purposes. If this occurs, it is likely to prompt reexamination of what the term "translational" truly means.