|
1
|
Matiasz NJ, Wood J, Silva AJ. Quantifying convergence and consistency. Eur J Neurosci 2024; 60:6391-6394. [PMID: 39403790 DOI: 10.1111/ejn.16561] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/01/2024] [Revised: 08/21/2024] [Accepted: 09/19/2024] [Indexed: 11/16/2024]
Abstract
The reproducibility crisis highlights several unresolved issues in science, including the need to develop measures that gauge both the consistency and convergence of data sets. While existing meta-analytic methods quantify the consistency of evidence, they do not quantify its convergence: the extent to which different types of empirical methods have provided evidence to support a hypothesis. To address this gap in meta-analysis, we and colleagues developed a summary metric-the cumulative evidence index (CEI)-which uses Bayesian statistics to quantify the degree of both consistency and convergence of evidence regarding causal hypotheses between two phenomena. Here, we outline the CEI's underlying model, which quantifies the extent to which studies of four types-positive intervention, negative intervention, positive non-intervention and negative non-intervention-lend credence to any of three types of causal relations: excitatory, inhibitory or no-connection. Along with p-values and other measures, the CEI can provide a more holistic perspective on a set of evidence by quantitatively expressing epistemic principles that scientists regularly employ qualitatively. The CEI can thus address the reproducibility crisis by formally demonstrating how convergent evidence across multiple study types can yield progress toward scientific consensus, even when an individual type of study fails to yield reproducible results.
Collapse
Affiliation(s)
- Nicholas J Matiasz
- Department of Neurobiology, David Geffen School of Medicine at UCLA, University of California Los Angeles, Los Angeles, California, USA
- Medical Imaging Informatics, Department of Radiological Sciences, University of California Los Angeles, Los Angeles, California, USA
- Department of Bioengineering, University of California Los Angeles, Los Angeles, California, USA
| | - Justin Wood
- Department of Neurobiology, David Geffen School of Medicine at UCLA, University of California Los Angeles, Los Angeles, California, USA
- Medical Imaging Informatics, Department of Radiological Sciences, University of California Los Angeles, Los Angeles, California, USA
- Department of Computer Science, University of California Los Angeles, Los Angeles, California, USA
| | - Alcino J Silva
- Department of Neurobiology, David Geffen School of Medicine at UCLA, University of California Los Angeles, Los Angeles, California, USA
- Department of Psychiatry & Biobehavioral Sciences, University of California Los Angeles, Los Angeles, California, USA
- Department of Psychology, University of California Los Angeles, Los Angeles, California, USA
- Integrative Center for Learning and Memory, University of California Los Angeles, Los Angeles, California, USA
- Brain Research Institute, University of California Los Angeles, Los Angeles, California, USA
| |
Collapse
|
|
2
|
London AJ. Artificial intelligence in medicine: Overcoming or recapitulating structural challenges to improving patient care? Cell Rep Med 2022; 3:100622. [PMID: 35584620 PMCID: PMC9133460 DOI: 10.1016/j.xcrm.2022.100622] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2021] [Revised: 02/10/2022] [Accepted: 04/06/2022] [Indexed: 01/09/2023]
Abstract
There is considerable enthusiasm about the prospect that artificial intelligence (AI) will help to improve the safety and efficacy of health services and the efficiency of health systems. To realize this potential, however, AI systems will have to overcome structural problems in the culture and practice of medicine and the organization of health systems that impact the data from which AI models are built, the environments into which they will be deployed, and the practices and incentives that structure their development. This perspective elaborates on some of these structural challenges and provides recommendations to address potential shortcomings.
Collapse
Affiliation(s)
- Alex John London
- Department of Philosophy and Center for Ethics and Policy, Carnegie Mellon University, Pittsburgh, PA 15228, USA.
| |
Collapse
|