1
|
Gu Z, Jamison KW, Khosla M, Allen EJ, Wu Y, Naselaris T, Kay K, Sabuncu MR, Kuceyeski A. NeuroGen: Activation optimized image synthesis for discovery neuroscience. Neuroimage 2021; 247:118812. [PMID: 34936922 PMCID: PMC8845078 DOI: 10.1016/j.neuroimage.2021.118812] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2021] [Revised: 10/11/2021] [Accepted: 12/12/2021] [Indexed: 11/24/2022] Open
Abstract
Functional MRI (fMRI) is a powerful technique that has allowed us to characterize visual cortex responses to stimuli, yet such experiments are by nature constructed based on a priori hypotheses, limited to the set of images presented to the individual while they are in the scanner, are subject to noise in the observed brain responses, and may vary widely across individuals. In this work, we propose a novel computational strategy, which we call NeuroGen, to overcome these limitations and develop a powerful tool for human vision neuroscience discovery. NeuroGen combines an fMRI-trained neural encoding model of human vision with a deep generative network to synthesize images predicted to achieve a target pattern of macro-scale brain activation. We demonstrate that the reduction of noise that the encoding model provides, coupled with the generative network’s ability to produce images of high fidelity, results in a robust discovery architecture for visual neuroscience. By using only a small number of synthetic images created by NeuroGen, we demonstrate that we can detect and amplify differences in regional and individual human brain response patterns to visual stimuli. We then verify that these discoveries are reflected in the several thousand observed image responses measured with fMRI. We further demonstrate that NeuroGen can create synthetic images predicted to achieve regional response patterns not achievable by the best-matching natural images. The NeuroGen framework extends the utility of brain encoding models and opens up a new avenue for exploring, and possibly precisely controlling, the human visual system.
Collapse
Affiliation(s)
- Zijin Gu
- School of Electrical and Computer Engineering, Cornell University, Ithaca, New York, USA
| | | | - Meenakshi Khosla
- School of Electrical and Computer Engineering, Cornell University, Ithaca, New York, USA
| | - Emily J Allen
- Center for Magnetic Resonance Research(CMRR), Department of Radiology, University of Minnesota, Minneapolis, Minnesota, USA; Department of Neuroscience, University of Minnesota, Minneapolis, Minnesota, USA
| | - Yihan Wu
- Center for Magnetic Resonance Research(CMRR), Department of Radiology, University of Minnesota, Minneapolis, Minnesota, USA
| | - Thomas Naselaris
- Department of Psychology, University of Minnesota, Minneapolis, Minnesota, USA
| | - Kendrick Kay
- Center for Magnetic Resonance Research(CMRR), Department of Radiology, University of Minnesota, Minneapolis, Minnesota, USA
| | - Mert R Sabuncu
- School of Electrical and Computer Engineering, Cornell University, Ithaca, New York, USA
| | - Amy Kuceyeski
- Department of Radiology, Weill Cornell Medicine, New York, New York, USA.
| |
Collapse
|
2
|
Zaitsev M, Akin B, LeVan P, Knowles BR. Prospective motion correction in functional MRI. Neuroimage 2016; 154:33-42. [PMID: 27845256 DOI: 10.1016/j.neuroimage.2016.11.014] [Citation(s) in RCA: 72] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2016] [Revised: 11/04/2016] [Accepted: 11/05/2016] [Indexed: 10/20/2022] Open
Abstract
Due to the intrinsic low sensitivity of BOLD-fMRI long scanning is required. Subject motion during fMRI scans reduces statistical significance of the activation maps and increases the prevalence of false activations. Motion correction is therefore an essential tool for a successful fMRI data analysis. Retrospective motion correction techniques are now commonplace and are incorporated into a wide range of fMRI analysis toolboxes. These techniques are advantageous due to robustness, sequence independence and have minimal impact on the fMRI study setup. Retrospective techniques however, do not provide an accurate intra-volume correction, nor can these techniques correct for the spin-history effects. The application of prospective motion correction in fMRI appears to be effective in reducing false positives and increasing sensitivity when compared to retrospective techniques, particularly in the cases of substantial motion. Especially advantageous in this regard is the combination of prospective motion correction with dynamic distortion correction. Nevertheless, none of the recent methods are able to recover activations in presence of motion that are comparable to no-motion conditions, which motivates further research in the area of adaptive dynamic imaging.
Collapse
Affiliation(s)
- Maxim Zaitsev
- Department of Radiology - Medical Physics, University of Freiburg, Faculty of Medicine, University of Freiburg - Medical Centre, Freiburg, Germany.
| | - Burak Akin
- Department of Radiology - Medical Physics, University of Freiburg, Faculty of Medicine, University of Freiburg - Medical Centre, Freiburg, Germany
| | - Pierre LeVan
- Department of Radiology - Medical Physics, University of Freiburg, Faculty of Medicine, University of Freiburg - Medical Centre, Freiburg, Germany
| | - Benjamin R Knowles
- Department of Radiology - Medical Physics, University of Freiburg, Faculty of Medicine, University of Freiburg - Medical Centre, Freiburg, Germany
| |
Collapse
|