1
|
Chen Z, Blair GJ, Cao C, Zhou J, Aharoni D, Golshani P, Blair HT, Cong J. FPGA-Based In-Vivo Calcium Image Decoding for Closed-Loop Feedback Applications. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2023; 17:169-179. [PMID: 37071510 PMCID: PMC10414190 DOI: 10.1109/tbcas.2023.3268130] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/11/2023]
Abstract
Miniaturized calcium imaging is an emerging neural recording technique that has been widely used for monitoring neural activity on a large scale at a specific brain region of rats or mice. Most existing calcium-image analysis pipelines operate offline. This results in long processing latency, making it difficult to realize closed-loop feedback stimulation for brain research. In recent work, we have proposed an FPGA-based real-time calcium image processing pipeline for closed-loop feedback applications. It can perform real-time calcium image motion correction, enhancement, fast trace extraction, and real-time decoding from extracted traces. Here, we extend this work by proposing a variety of neural network based methods for real-time decoding and evaluate the tradeoff among these decoding methods and accelerator designs. We introduce the implementation of the neural network based decoders on the FPGA, and show their speedup against the implementation on the ARM processor. Our FPGA implementation enables the real-time calcium image decoding with sub-ms processing latency for closed-loop feedback applications.
Collapse
|
2
|
Chen Z, Blair GJ, Guo C, Zhou J, Romero-Sosa JL, Izquierdo A, Golshani P, Cong J, Aharoni D, Blair HT. A hardware system for real-time decoding of in vivo calcium imaging data. eLife 2023; 12:78344. [PMID: 36692269 PMCID: PMC9908073 DOI: 10.7554/elife.78344] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2022] [Accepted: 01/23/2023] [Indexed: 01/25/2023] Open
Abstract
Epifluorescence miniature microscopes ('miniscopes') are widely used for in vivo calcium imaging of neural population activity. Imaging data are typically collected during a behavioral task and stored for later offline analysis, but emerging techniques for online imaging can support novel closed-loop experiments in which neural population activity is decoded in real time to trigger neurostimulation or sensory feedback. To achieve short feedback latencies, online imaging systems must be optimally designed to maximize computational speed and efficiency while minimizing errors in population decoding. Here we introduce DeCalciOn, an open-source device for real-time imaging and population decoding of in vivo calcium signals that is hardware compatible with all miniscopes that use the UCLA Data Acquisition (DAQ) interface. DeCalciOn performs online motion stabilization, neural enhancement, calcium trace extraction, and decoding of up to 1024 traces per frame at latencies of <50 ms after fluorescence photons arrive at the miniscope image sensor. We show that DeCalciOn can accurately decode the position of rats (n = 12) running on a linear track from calcium fluorescence in the hippocampal CA1 layer, and can categorically classify behaviors performed by rats (n = 2) during an instrumental task from calcium fluorescence in orbitofrontal cortex. DeCalciOn achieves high decoding accuracy at short latencies using innovations such as field-programmable gate array hardware for real-time image processing and contour-free methods to efficiently extract calcium traces from sensor images. In summary, our system offers an affordable plug-and-play solution for real-time calcium imaging experiments in behaving animals.
Collapse
Affiliation(s)
- Zhe Chen
- Department of Electrical and Computer Engineering, University of California, Los AngelesLos AngelesUnited States
- Department of Psychology, University of California, Los AngelesLos AngelesUnited States
| | - Garrett J Blair
- Department of Psychology, University of California, Los AngelesLos AngelesUnited States
| | - Changliang Guo
- David Geffen School of Medicine, University of California, Los AngelesLos AngelesUnited States
- Department of Neurology, David Geffen School of Medicine, University of California, Los AngelesLos AngelesUnited States
| | - Jim Zhou
- Department of Electrical and Computer Engineering, University of California, Los AngelesLos AngelesUnited States
| | - Juan-Luis Romero-Sosa
- Department of Psychology, University of California, Los AngelesLos AngelesUnited States
| | - Alicia Izquierdo
- Department of Psychology, University of California, Los AngelesLos AngelesUnited States
- Integrative Center for Learning and Memory, University of California, Los AngelesLos AngelesUnited States
| | - Peyman Golshani
- David Geffen School of Medicine, University of California, Los AngelesLos AngelesUnited States
- Department of Neurology, David Geffen School of Medicine, University of California, Los AngelesLos AngelesUnited States
- Integrative Center for Learning and Memory, University of California, Los AngelesLos AngelesUnited States
| | - Jason Cong
- Department of Electrical and Computer Engineering, University of California, Los AngelesLos AngelesUnited States
| | - Daniel Aharoni
- David Geffen School of Medicine, University of California, Los AngelesLos AngelesUnited States
- Department of Neurology, David Geffen School of Medicine, University of California, Los AngelesLos AngelesUnited States
- Integrative Center for Learning and Memory, University of California, Los AngelesLos AngelesUnited States
| | - Hugh T Blair
- Department of Psychology, University of California, Los AngelesLos AngelesUnited States
- Integrative Center for Learning and Memory, University of California, Los AngelesLos AngelesUnited States
| |
Collapse
|