1
|
Wang C, Li X, Wan R, Chen J, Ye J, Li K, Li A, Tai R, Sepe A. Accelerating imaging research at large-scale scientific facilities through scientific computing. JOURNAL OF SYNCHROTRON RADIATION 2024; 31:1317-1326. [PMID: 39190504 PMCID: PMC11371030 DOI: 10.1107/s1600577524007239] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/29/2024] [Accepted: 07/21/2024] [Indexed: 08/29/2024]
Abstract
To date, computed tomography experiments, carried-out at synchrotron radiation facilities worldwide, pose a tremendous challenge in terms of the breadth and complexity of the experimental datasets produced. Furthermore, near real-time three-dimensional reconstruction capabilities are becoming a crucial requirement in order to perform high-quality and result-informed synchrotron imaging experiments, where a large amount of data is collected and processed within a short time window. To address these challenges, we have developed and deployed a synchrotron computed tomography framework designed to automatically process online the experimental data from the synchrotron imaging beamlines, while leveraging the high-performance computing cluster capabilities to accelerate the real-time feedback to the users on their experimental results. We have, further, integrated it within a modern unified national authentication and data management framework, which we have developed and deployed, spanning the entire data lifecycle of a large-scale scientific facility. In this study, the overall architecture, functional modules and workflow design of our synchrotron computed tomography framework are presented in detail. Moreover, the successful integration of the imaging beamlines at the Shanghai Synchrotron Radiation Facility into our scientific computing framework is also detailed, which, ultimately, resulted in accelerating and fully automating their entire data processing pipelines. In fact, when compared with the original three-dimensional tomography reconstruction approaches, the implementation of our synchrotron computed tomography framework led to an acceleration in the experimental data processing capabilities, while maintaining a high level of integration with all the beamline processing software and systems.
Collapse
Affiliation(s)
- Chunpeng Wang
- Big Data Science CenterShanghai Synchrotron Radiation Facility, Shanghai Advanced Research Institute, Chinese Academy of SciencesNo. 239 Zhangheng RoadShanghai201210People’s Republic of China
| | - Xiaoyun Li
- Big Data Science CenterShanghai Synchrotron Radiation Facility, Shanghai Advanced Research Institute, Chinese Academy of SciencesNo. 239 Zhangheng RoadShanghai201210People’s Republic of China
| | - Rongzheng Wan
- Big Data Science CenterShanghai Synchrotron Radiation Facility, Shanghai Advanced Research Institute, Chinese Academy of SciencesNo. 239 Zhangheng RoadShanghai201210People’s Republic of China
| | - Jige Chen
- Big Data Science CenterShanghai Synchrotron Radiation Facility, Shanghai Advanced Research Institute, Chinese Academy of SciencesNo. 239 Zhangheng RoadShanghai201210People’s Republic of China
| | - Jing Ye
- Big Data Science CenterShanghai Synchrotron Radiation Facility, Shanghai Advanced Research Institute, Chinese Academy of SciencesNo. 239 Zhangheng RoadShanghai201210People’s Republic of China
| | - Ke Li
- Shanghai Synchrotron Radiation Facility, Shanghai Advanced Research Institute, Chinese Academy of SciencesNo. 239 Zhangheng RoadShanghai201210People’s Republic of China
| | - Aiguo Li
- Shanghai Synchrotron Radiation Facility, Shanghai Advanced Research Institute, Chinese Academy of SciencesNo. 239 Zhangheng RoadShanghai201210People’s Republic of China
| | - Renzhong Tai
- Shanghai Synchrotron Radiation Facility, Shanghai Advanced Research Institute, Chinese Academy of SciencesNo. 239 Zhangheng RoadShanghai201210People’s Republic of China
| | - Alessandro Sepe
- Big Data Science CenterShanghai Synchrotron Radiation Facility, Shanghai Advanced Research Institute, Chinese Academy of SciencesNo. 239 Zhangheng RoadShanghai201210People’s Republic of China
| |
Collapse
|
2
|
Nikitin V. TomocuPy - efficient GPU-based tomographic reconstruction with asynchronous data processing. JOURNAL OF SYNCHROTRON RADIATION 2023; 30:179-191. [PMID: 36601936 PMCID: PMC9814072 DOI: 10.1107/s1600577522010311] [Citation(s) in RCA: 12] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/10/2022] [Accepted: 10/25/2022] [Indexed: 06/17/2023]
Abstract
Fast 3D data analysis and steering of a tomographic experiment by changing environmental conditions or acquisition parameters require fast, close to real-time, 3D reconstruction of large data volumes. Here a performance-optimized TomocuPy package is presented as a GPU alternative to the commonly used central processing unit (CPU) based TomoPy package for tomographic reconstruction. TomocuPy utilizes modern hardware capabilities to organize a 3D asynchronous reconstruction involving parallel read/write operations with storage drives, CPU-GPU data transfers, and GPU computations. In the asynchronous reconstruction, all the operations are timely overlapped to almost fully hide all data management time. Since most cameras work with less than 16-bit digital output, the memory usage and processing speed are furthermore optimized by using 16-bit floating-point arithmetic. As a result, 3D reconstruction with TomocuPy became 20-30 times faster than its multi-threaded CPU equivalent. Full reconstruction (including read/write operations and methods initialization) of a 20483 tomographic volume takes less than 7 s on a single Nvidia Tesla A100 and PCIe 4.0 NVMe SSD, and scales almost linearly increasing the data size. To simplify operation at synchrotron beamlines, TomocuPy provides an easy-to-use command-line interface. Efficacy of the package was demonstrated during a tomographic experiment on gas-hydrate formation in porous samples, where a steering option was implemented as a lens-changing mechanism for zooming to regions of interest.
Collapse
Affiliation(s)
- Viktor Nikitin
- Advanced Photon Source, Argonne National Laboratory, Lemont, IL 60439, USA
| |
Collapse
|
3
|
Trinkle S, Foxley S, Kasthuri N, Rivière PL. Synchrotron X-ray micro-CT as a validation dataset for diffusion MRI in whole mouse brain. Magn Reson Med 2021; 86:1067-1076. [PMID: 33768633 PMCID: PMC8076078 DOI: 10.1002/mrm.28776] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2020] [Revised: 01/26/2021] [Accepted: 02/28/2021] [Indexed: 12/13/2022]
Abstract
PURPOSE To introduce synchrotron X-ray microcomputed tomography (microCT) and demonstrate its use as a natively isotropic, nondestructive, 3D validation modality for diffusion MRI in whole, fixed mouse brain. METHODS Postmortem diffusion MRI and microCT data were acquired of the same whole mouse brain. Diffusion data were processed using constrained spherical deconvolution. Synchrotron data were acquired at an isotropic voxel size of 1.17 μm. Structure tensor analysis was used to calculate fiber orientation distribution functions from the microCT data. A pipeline was developed to spatially register the 2 datasets in order to perform qualitative comparisons of fiber geometries represented by fiber orientation distribution functions. Fiber orientations from both modalities were used to perform whole-brain deterministic tractography to demonstrate validation of long-range white matter pathways. RESULTS Fiber orientation distribution functions were able to be extracted throughout the entire microCT dataset, with spatial registration to diffusion MRI simplified due to the whole brain extent of the microCT data. Fiber orientations and tract pathways showed good agreement between modalities. CONCLUSION Synchrotron microCT is a potentially valuable new tool for future multi-scale diffusion MRI validation studies, providing comparable value to optical histology validation methods while addressing some key limitations in data acquisition and ease of processing.
Collapse
Affiliation(s)
- Scott Trinkle
- Department of Radiology, University of Chicago, Chicago, IL, USA
| | - Sean Foxley
- Department of Radiology, University of Chicago, Chicago, IL, USA
| | | | | |
Collapse
|
4
|
Du M, Vescovi R, Fezzaa K, Jacobsen C, Gürsoy D. X-ray tomography of extended objects: a comparison of data acquisition approaches. JOURNAL OF THE OPTICAL SOCIETY OF AMERICA. A, OPTICS, IMAGE SCIENCE, AND VISION 2018; 35:1871-1879. [PMID: 30461846 PMCID: PMC6994859 DOI: 10.1364/josaa.35.001871] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/11/2018] [Accepted: 08/03/2018] [Indexed: 06/01/2023]
Abstract
The penetration power of x rays allows one to image large objects, while their short wavelength allows for high spatial resolution. As a result, with synchrotron sources, one has the potential to obtain tomographic images of centimeter-sized specimens with sub-micrometer pixel sizes. However, limitations on beam and detector size make it difficult to acquire such data of this sort in a single take, necessitating strategies for combining data from multiple regions. One strategy is to acquire a tiled set of local tomograms by rotating the specimen around each of the local tomogram center positions. Another strategy, sinogram oriented acquisition, involves the collection of projections at multiple offset positions from the rotation axis followed by data merging and reconstruction. We have carried out a simulation study to compare these two approaches in terms of radiation dose applied to the specimen, and reconstructed image quality. Local tomography acquisition involves an easier data alignment problem, and immediate viewing of subregions before the entire dataset has been acquired. Sinogram oriented acquisition involves a more difficult data assembly and alignment procedure, and it is more sensitive to accumulative registration error. However, sinogram oriented acquisition is more dose efficient, involves fewer translation motions of the object, and avoids certain artifacts of local tomography.
Collapse
|
5
|
Vescovi R, Du M, de Andrade V, Scullin W, Gürsoy D, Jacobsen C. Tomosaic: efficient acquisition and reconstruction of teravoxel tomography data using limited-size synchrotron X-ray beams. JOURNAL OF SYNCHROTRON RADIATION 2018; 25:1478-1489. [PMID: 30179188 PMCID: PMC6140399 DOI: 10.1107/s1600577518010093] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/06/2018] [Accepted: 06/24/2018] [Indexed: 05/13/2023]
Abstract
X-rays offer high penetration with the potential for tomography of centimetre-sized specimens, but synchrotron beamlines often provide illumination that is only millimetres wide. Here an approach is demonstrated termed Tomosaic for tomographic imaging of large samples that extend beyond the illumination field of view of an X-ray imaging system. This includes software modules for image stitching and calibration, while making use of existing modules available in other packages for alignment and reconstruction. The approach is compatible with conventional beamline hardware, while providing a dose-efficient method of data acquisition. By using parallelization on a distributed computing system, it provides a solution for handling teravoxel-sized or larger datasets that cannot be processed on a single workstation in a reasonable time. Using experimental data, the package is shown to provide good quality three-dimensional reconstruction for centimetre-sized samples with sub-micrometre pixel size.
Collapse
Affiliation(s)
- Rafael Vescovi
- Advanced Photon Source, Argonne National Laboratory, Argonne, IL 60439, USA
| | - Ming Du
- Department of Materials Science, Northwestern University, Evanston, IL 60208, USA
| | - Vincent de Andrade
- Advanced Photon Source, Argonne National Laboratory, Argonne, IL 60439, USA
| | - William Scullin
- Argonne Leadership Computing Facility, Argonne National Laboratory, Argonne, IL 60439, USA
| | - Doǧa Gürsoy
- Advanced Photon Source, Argonne National Laboratory, Argonne, IL 60439, USA
- Department of Electrical Engineering and Computer Science, Northwestern University, Evanston, IL 60208, USA
| | - Chris Jacobsen
- Advanced Photon Source, Argonne National Laboratory, Argonne, IL 60439, USA
- Department of Physics and Astronomy, Northwestern University, Evanston, IL 60208, USA
- Chemistry of Life Processes Institute, Northwestern University, Evanston, IL 60208, USA
| |
Collapse
|
6
|
Fonseca MDC, Araujo BHS, Dias CSB, Archilha NL, Neto DPA, Cavalheiro E, Westfahl H, da Silva AJR, Franchini KG. High-resolution synchrotron-based X-ray microtomography as a tool to unveil the three-dimensional neuronal architecture of the brain. Sci Rep 2018; 8:12074. [PMID: 30104676 PMCID: PMC6089932 DOI: 10.1038/s41598-018-30501-x] [Citation(s) in RCA: 31] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2018] [Accepted: 08/01/2018] [Indexed: 11/09/2022] Open
Abstract
The assessment of neuronal number, spatial organization and connectivity is fundamental for a complete understanding of brain function. However, the evaluation of the three-dimensional (3D) brain cytoarchitecture at cellular resolution persists as a great challenge in the field of neuroscience. In this context, X-ray microtomography has shown to be a valuable non-destructive tool for imaging a broad range of samples, from dense materials to soft biological specimens, arisen as a new method for deciphering the cytoarchitecture and connectivity of the brain. In this work we present a method for imaging whole neurons in the brain, combining synchrotron-based X-ray microtomography with the Golgi-Cox mercury-based impregnation protocol. In contrast to optical 3D techniques, the approach shown here does neither require tissue slicing or clearing, and allows the investigation of several cells within a 3D region of the brain.
Collapse
Affiliation(s)
- Matheus de Castro Fonseca
- Brazilian Biosciences National Laboratory (LNBio), Brazilian Center for Research in Energy and Materials (CNPEM), Zip Code 13083-970, Campinas, Sao Paulo, Brazil.
| | - Bruno Henrique Silva Araujo
- Brazilian Biosciences National Laboratory (LNBio), Brazilian Center for Research in Energy and Materials (CNPEM), Zip Code 13083-970, Campinas, Sao Paulo, Brazil
| | - Carlos Sato Baraldi Dias
- Brazilian Synchrotron Light National Laboratory (LNLS), Brazilian Center for Research in Energy and Materials (CNPEM), Zip Code 13083-970, Campinas, Sao Paulo, Brazil
| | - Nathaly Lopes Archilha
- Brazilian Synchrotron Light National Laboratory (LNLS), Brazilian Center for Research in Energy and Materials (CNPEM), Zip Code 13083-970, Campinas, Sao Paulo, Brazil
| | - Dionísio Pedro Amorim Neto
- Brazilian Biosciences National Laboratory (LNBio), Brazilian Center for Research in Energy and Materials (CNPEM), Zip Code 13083-970, Campinas, Sao Paulo, Brazil
| | - Esper Cavalheiro
- Department of Neurology and Neurosurgery, Federal University of São Paulo (UNIFESP/EPM), Zip Code 04021-001, São Paulo, São Paulo, Brazil
| | - Harry Westfahl
- Brazilian Synchrotron Light National Laboratory (LNLS), Brazilian Center for Research in Energy and Materials (CNPEM), Zip Code 13083-970, Campinas, Sao Paulo, Brazil
| | - Antônio José Roque da Silva
- Brazilian Synchrotron Light National Laboratory (LNLS), Brazilian Center for Research in Energy and Materials (CNPEM), Zip Code 13083-970, Campinas, Sao Paulo, Brazil
| | - Kleber Gomes Franchini
- Brazilian Biosciences National Laboratory (LNBio), Brazilian Center for Research in Energy and Materials (CNPEM), Zip Code 13083-970, Campinas, Sao Paulo, Brazil
| |
Collapse
|