1
|
Obrecht M, Zurbruegg S, Accart N, Lambert C, Doelemeyer A, Ledermann B, Beckmann N. Magnetic resonance imaging and ultrasound elastography in the context of preclinical pharmacological research: significance for the 3R principles. Front Pharmacol 2023; 14:1177421. [PMID: 37448960 PMCID: PMC10337591 DOI: 10.3389/fphar.2023.1177421] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2023] [Accepted: 06/16/2023] [Indexed: 07/18/2023] Open
Abstract
The 3Rs principles-reduction, refinement, replacement-are at the core of preclinical research within drug discovery, which still relies to a great extent on the availability of models of disease in animals. Minimizing their distress, reducing their number as well as searching for means to replace them in experimental studies are constant objectives in this area. Due to its non-invasive character in vivo imaging supports these efforts by enabling repeated longitudinal assessments in each animal which serves as its own control, thereby enabling to reduce considerably the animal utilization in the experiments. The repetitive monitoring of pathology progression and the effects of therapy becomes feasible by assessment of quantitative biomarkers. Moreover, imaging has translational prospects by facilitating the comparison of studies performed in small rodents and humans. Also, learnings from the clinic may be potentially back-translated to preclinical settings and therefore contribute to refining animal investigations. By concentrating on activities around the application of magnetic resonance imaging (MRI) and ultrasound elastography to small rodent models of disease, we aim to illustrate how in vivo imaging contributes primarily to reduction and refinement in the context of pharmacological research.
Collapse
Affiliation(s)
- Michael Obrecht
- Diseases of Aging and Regenerative Medicines, Novartis Institutes for BioMedical Research, Basel, Switzerland
| | - Stefan Zurbruegg
- Neurosciences Department, Novartis Institutes for BioMedical Research, Basel, Switzerland
| | - Nathalie Accart
- Diseases of Aging and Regenerative Medicines, Novartis Institutes for BioMedical Research, Basel, Switzerland
| | - Christian Lambert
- Diseases of Aging and Regenerative Medicines, Novartis Institutes for BioMedical Research, Basel, Switzerland
| | - Arno Doelemeyer
- Diseases of Aging and Regenerative Medicines, Novartis Institutes for BioMedical Research, Basel, Switzerland
| | - Birgit Ledermann
- 3Rs Leader, Novartis Institutes for BioMedical Research, Basel, Switzerland
| | - Nicolau Beckmann
- Diseases of Aging and Regenerative Medicines, Novartis Institutes for BioMedical Research, Basel, Switzerland
| |
Collapse
|
2
|
Carmichael O. The Role of fMRI in Drug Development: An Update. ADVANCES IN NEUROBIOLOGY 2023; 30:299-333. [PMID: 36928856 DOI: 10.1007/978-3-031-21054-9_13] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/18/2023]
Abstract
Functional magnetic resonance imaging (fMRI) of the brain is a technology that holds great potential for increasing the efficiency of drug development for the central nervous system (CNS). In preclinical studies and both early- and late-phase human trials, fMRI has the potential to improve cross-species translation of drug effects, help to de-risk compounds early in development, and contribute to the portfolio of evidence for a compound's efficacy and mechanism of action. However, to date, the utilization of fMRI in the CNS drug development process has been limited. The purpose of this chapter is to explore this mismatch between potential and utilization. This chapter provides introductory material related to fMRI and drug development, describes what is required of fMRI measurements for them to be useful in a drug development setting, lists current capabilities of fMRI in this setting and challenges faced in its utilization, and ends with directions for future development of capabilities in this arena. This chapter is the 5-year update of material from a previously published workshop summary (Carmichael et al., Drug DiscovToday 23(2):333-348, 2018).
Collapse
Affiliation(s)
- Owen Carmichael
- Pennington Biomedical Research Center, Baton Rouge, LA, USA.
| |
Collapse
|
3
|
Comparison of test–retest reliability of BOLD and pCASL fMRI in a two-center study. BMC Med Imaging 2022; 22:62. [PMID: 35366813 PMCID: PMC8977011 DOI: 10.1186/s12880-022-00791-9] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2021] [Accepted: 03/21/2022] [Indexed: 11/17/2022] Open
Abstract
Background The establishment of test–retest reliability and reproducibility (TRR) is an important part of validating any research tool, including functional magnetic resonance imaging (fMRI). The primary objective of this study is to investigate the reliability of pseudo-Continuous Arterial Spin Labeling (pCASL) and Blood Oxygen Level Dependent (BOLD) fMRI data acquired across two different scanners in a sample of healthy adults. While single site/single scanner studies have shown acceptable repeatability, TRR of both in a practical multisite study occurring in two facilities spread out across the country with weeks to months between scans is critically needed. Methods Ten subjects were imaged with similar 3 T MRI scanners at the University of Pittsburgh and Massachusetts General Hospital. Finger-tapping and Resting-state data were acquired for both techniques. Analysis of the resting state data for functional connectivity was performed with the Functional Connectivity Toolbox, while analysis of the finger tapping data was accomplished with FSL. pCASL Blood flow data was generated using AST Toolbox. Activated areas and networks were identified via pre-defined atlases and dual-regression techniques. Analysis for TRR was conducted by comparing pCASL and BOLD images in terms of Intraclass correlation coefficients, Dice Similarity Coefficients, and repeated measures ANOVA. Results Both BOLD and pCASL scans showed strong activation and correlation between the two locations for the finger tapping tasks. Functional connectivity analyses identified elements of the default mode network in all resting scans at both locations. Multivariate repeated measures ANOVA showed significant variability between subjects, but no significant variability for location. Global CBF was very similar between the two scanning locations, and repeated measures ANOVA showed no significant differences between the two scanning locations. Conclusions The results of this study show that when similar scanner hardware and software is coupled with identical data analysis protocols, consistent and reproducible functional brain images can be acquired across sites. The variability seen in the activation maps is greater for pCASL versus BOLD images, as expected, however groups maps are remarkably similar despite the low number of subjects. This demonstrates that multi-site fMRI studies of task-based and resting state brain activity is feasible.
Collapse
|
4
|
Ekhtiari H, Zare-Bidoky M, Sangchooli A, Janes AC, Kaufman MJ, Oliver JA, Prisciandaro JJ, Wüstenberg T, Anton RF, Bach P, Baldacchino A, Beck A, Bjork JM, Brewer J, Childress AR, Claus ED, Courtney KE, Ebrahimi M, Filbey FM, Ghahremani DG, Azbari PG, Goldstein RZ, Goudriaan AE, Grodin EN, Hamilton JP, Hanlon CA, Hassani-Abharian P, Heinz A, Joseph JE, Kiefer F, Zonoozi AK, Kober H, Kuplicki R, Li Q, London ED, McClernon J, Noori HR, Owens MM, Paulus MP, Perini I, Potenza M, Potvin S, Ray L, Schacht JP, Seo D, Sinha R, Smolka MN, Spanagel R, Steele VR, Stein EA, Steins-Loeber S, Tapert SF, Verdejo-Garcia A, Vollstädt-Klein S, Wetherill RR, Wilson SJ, Witkiewitz K, Yuan K, Zhang X, Zilverstand A. A methodological checklist for fMRI drug cue reactivity studies: development and expert consensus. Nat Protoc 2022; 17:567-595. [PMID: 35121856 PMCID: PMC9063851 DOI: 10.1038/s41596-021-00649-4] [Citation(s) in RCA: 29] [Impact Index Per Article: 14.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2020] [Accepted: 10/21/2021] [Indexed: 12/23/2022]
Abstract
Cue reactivity is one of the most frequently used paradigms in functional magnetic resonance imaging (fMRI) studies of substance use disorders (SUDs). Although there have been promising results elucidating the neurocognitive mechanisms of SUDs and SUD treatments, the interpretability and reproducibility of these studies is limited by incomplete reporting of participants' characteristics, task design, craving assessment, scanning preparation and analysis decisions in fMRI drug cue reactivity (FDCR) experiments. This hampers clinical translation, not least because systematic review and meta-analysis of published work are difficult. This consensus paper and Delphi study aims to outline the important methodological aspects of FDCR research, present structured recommendations for more comprehensive methods reporting and review the FDCR literature to assess the reporting of items that are deemed important. Forty-five FDCR scientists from around the world participated in this study. First, an initial checklist of items deemed important in FDCR studies was developed by several members of the Enhanced NeuroImaging Genetics through Meta-Analyses (ENIGMA) Addiction working group on the basis of a systematic review. Using a modified Delphi consensus method, all experts were asked to comment on, revise or add items to the initial checklist, and then to rate the importance of each item in subsequent rounds. The reporting status of the items in the final checklist was investigated in 108 recently published FDCR studies identified through a systematic review. By the final round, 38 items reached the consensus threshold and were classified under seven major categories: 'Participants' Characteristics', 'General fMRI Information', 'General Task Information', 'Cue Information', 'Craving Assessment Inside Scanner', 'Craving Assessment Outside Scanner' and 'Pre- and Post-Scanning Considerations'. The review of the 108 FDCR papers revealed significant gaps in the reporting of the items considered important by the experts. For instance, whereas items in the 'General fMRI Information' category were reported in 90.5% of the reviewed papers, items in the 'Pre- and Post-Scanning Considerations' category were reported by only 44.7% of reviewed FDCR studies. Considering the notable and sometimes unexpected gaps in the reporting of items deemed to be important by experts in any FDCR study, the protocols could benefit from the adoption of reporting standards. This checklist, a living document to be updated as the field and its methods advance, can help improve experimental design, reporting and the widespread understanding of the FDCR protocols. This checklist can also provide a sample for developing consensus statements for protocols in other areas of task-based fMRI.
Collapse
Affiliation(s)
- Hamed Ekhtiari
- Laureate Institute for Brain Research, Tulsa, OK, USA.
- Department of Psychiatry & Behavioral Sciences, University of Minnesota, Minneapolis, MN, USA.
| | - Mehran Zare-Bidoky
- Iranian National Center for Addiction Studies (INCAS), Tehran University of Medical Sciences, Tehran, Iran
- Shahid-Sadoughi University of Medical Sciences, Yazd, Iran
| | - Arshiya Sangchooli
- Iranian National Center for Addiction Studies (INCAS), Tehran University of Medical Sciences, Tehran, Iran
| | - Amy C Janes
- Department of Psychiatry, McLean Hospital, Harvard Medical School, Belmont, MA, USA
| | - Marc J Kaufman
- Department of Psychiatry, McLean Hospital, Harvard Medical School, Belmont, MA, USA
| | - Jason A Oliver
- Department of Psychiatry and Behavioral Sciences, Duke University, Durham, NC, USA
- TSET Health Promotion Research Center, Stephenson Cancer Center, Oklahoma City, OK, USA
- Department of Psychiatry & Behavioral Sciences, Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
| | - James J Prisciandaro
- Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina, Charleston, SC, USA
| | - Torsten Wüstenberg
- Department of Psychiatry and Neurosciences, Charité Campus Mitte, Charité-Universitätsmedizin Berlin, Berlin, Germany
| | - Raymond F Anton
- Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina, Charleston, SC, USA
| | - Patrick Bach
- Department of Addictive Behaviour and Addiction Medicine, Central Institute of Mental Health (CIMH), Heidelberg University, Mannheim, Germany
| | - Alex Baldacchino
- Division of Population Studies and Behavioural Sciences, St Andrews University Medical School, University of St Andrews, Scotland, UK
| | - Anne Beck
- Department of Psychiatry and Neurosciences, Charité Campus Mitte, Charité-Universitätsmedizin Berlin, Berlin, Germany
- Faculty of Health, Health and Medical University, Campus Potsdam, Potsdam, Germany
| | - James M Bjork
- Department of Psychiatry, Virginia Commonwealth University, Richmond, VA, USA
| | - Judson Brewer
- Department of Behavioral and Social Sciences, Brown University School of Public Health, Providence, RI, USA
| | - Anna Rose Childress
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Eric D Claus
- Department of Biobehavioral Health, The Pennsylvania State University, University Park, PA, USA
| | - Kelly E Courtney
- Department of Psychiatry, University of California, San Diego, La Jolla, CA, USA
| | - Mohsen Ebrahimi
- Iranian National Center for Addiction Studies (INCAS), Tehran University of Medical Sciences, Tehran, Iran
| | - Francesca M Filbey
- Center for BrainHealth, School of Behavioral and Brain Sciences, University of Texas at Dallas, Dallas, TX, USA
| | - Dara G Ghahremani
- Department of Psychiatry and Biobehavioral Sciences, University of California, Los Angeles, Los Angeles, CA, USA
| | - Peyman Ghobadi Azbari
- Iranian National Center for Addiction Studies (INCAS), Tehran University of Medical Sciences, Tehran, Iran
- Department of Biomedical Engineering, Shahed University, Tehran, Iran
| | - Rita Z Goldstein
- Departments of Psychiatry & Neuroscience, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Anna E Goudriaan
- Department of Psychiatry, Amsterdam University Medical Center, University of Amsterdam and Amsterdam Neuroscience, Amsterdam, The Netherlands
| | - Erica N Grodin
- Department of Psychiatry and Biobehavioral Sciences, University of California, Los Angeles, Los Angeles, CA, USA
| | - J Paul Hamilton
- Center for Social and Affective Neuroscience, Department of Biomedical and Clinical Sciences, Linköping University, Linköping, Sweden
- Center for Medical Image Science and Visualization, Department of Biomedical and Clinical Sciences, Linköping University, Linköping, Sweden
| | - Colleen A Hanlon
- Department of Cancer Biology, Wake Forest School of Medicine, Winston-Salem, NC, USA
| | | | - Andreas Heinz
- Department of Psychiatry and Neurosciences, Charité Campus Mitte, Charité-Universitätsmedizin Berlin, Berlin, Germany
| | - Jane E Joseph
- Department of Neuroscience, Medical University of South Carolina, Charleston, SC, USA
| | - Falk Kiefer
- Department of Addictive Behaviour and Addiction Medicine, Central Institute of Mental Health (CIMH), Heidelberg University, Mannheim, Germany
| | - Arash Khojasteh Zonoozi
- Iranian National Center for Addiction Studies (INCAS), Tehran University of Medical Sciences, Tehran, Iran
- Mashhad University of Medical Sciences, Mashhad, Iran
| | - Hedy Kober
- Department of Psychiatry, Yale School of Medicine, New Haven, CT, USA
| | | | - Qiang Li
- Department of Radiology, Tangdu Hospital, Fourth Military Medical University, Xi'an, China
| | - Edythe D London
- Department of Psychiatry and Biobehavioral Sciences, University of California, Los Angeles, Los Angeles, CA, USA
| | - Joseph McClernon
- Department of Psychiatry and Behavioral Sciences, Duke University, Durham, NC, USA
| | - Hamid R Noori
- International Center for Primate Brain Research, Center for Excellence in Brain Science and Intelligence Technology (CEBSIT)/Institute of Neuroscience (ION), Chinese Academy of Sciences, Shanghai, China
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Max M Owens
- Department of Psychiatry, University of Vermont, Burlington, VT, USA
| | | | - Irene Perini
- Center for Social and Affective Neuroscience, Department of Biomedical and Clinical Sciences, Linköping University, Linköping, Sweden
- Center for Medical Image Science and Visualization, Department of Biomedical and Clinical Sciences, Linköping University, Linköping, Sweden
| | - Marc Potenza
- Department of Psychiatry, Yale School of Medicine, New Haven, CT, USA
- Connecticut Mental Health Center, New Haven, CT, USA
- Connecticut Council on Problem Gambling, Wethersfield, CT, USA
- Department of Neuroscience, Child Study Center and Wu Tsai Institute, Yale School of Medicine, New Haven, CT, USA
| | - Stéphane Potvin
- Centre de recherche de l'Institut Universitaire en Santé Mentale de Montréal, University of Montreal, Montreal, Canada
| | - Lara Ray
- Department of Psychiatry and Biobehavioral Sciences, University of California, Los Angeles, Los Angeles, CA, USA
| | | | - Dongju Seo
- Department of Psychiatry, Yale School of Medicine, New Haven, CT, USA
| | - Rajita Sinha
- Department of Psychiatry, Yale School of Medicine, New Haven, CT, USA
| | - Michael N Smolka
- Department of Psychiatry, Technische Universität Dresden, Dresden, Germany
| | - Rainer Spanagel
- Institute of Psychopharmacology, Central Institute of Mental Health, Mannheim, Germany
| | - Vaughn R Steele
- Department of Psychiatry, Yale School of Medicine, New Haven, CT, USA
| | - Elliot A Stein
- Intramural Research Program, National Institute on Drug Abuse, Baltimore, MD, USA
| | - Sabine Steins-Loeber
- Department of Clinical Psychology and Psychotherapy, Otto-Friedrich-University of Bamberg, Bamberg, Germany
| | - Susan F Tapert
- Department of Psychiatry, University of California, San Diego, La Jolla, CA, USA
| | | | - Sabine Vollstädt-Klein
- Department of Addictive Behaviour and Addiction Medicine, Central Institute of Mental Health (CIMH), Heidelberg University, Mannheim, Germany
| | - Reagan R Wetherill
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Stephen J Wilson
- Department of Psychology, The Pennsylvania State University, University Park, PA, USA
| | - Katie Witkiewitz
- Department of Psychology, University of New Mexico, Albuquerque, NM, USA
| | - Kai Yuan
- School of Life Science and Technology, Xidian University, Xi'an, China
| | - Xiaochu Zhang
- Department of Psychology, School of Humanities and Social Science, University of Science and Technology of China, Anhui, China
- Department of Radiology, First Affiliated Hospital of USTC, Hefei National Laboratory for Physical Science at the Microscale and School of Life Science, Division of Life Science and Medicine, University of Science and Technology of China, Anhui, China
| | - Anna Zilverstand
- Department of Psychiatry & Behavioral Sciences, University of Minnesota, Minneapolis, MN, USA
| |
Collapse
|
5
|
Sadraee A, Paulus M, Ekhtiari H. fMRI as an outcome measure in clinical trials: A systematic review in clinicaltrials.gov. Brain Behav 2021; 11:e02089. [PMID: 33662169 PMCID: PMC8119793 DOI: 10.1002/brb3.2089] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/17/2019] [Revised: 12/29/2020] [Accepted: 01/02/2021] [Indexed: 12/13/2022] Open
Abstract
INTRODUCTION More than one-thousand trials with functional magnetic resonance imaging (fMRI) as an outcome measure were registered in clinicaltrials.gov at the time of writing this article. However, 93% of these registered trials are still not completed with published results and there is no picture available about methodological dimensions of these ongoing trials with fMRI as an outcome measure. METHODS We collected trials that use fMRI as an outcome measure in the ClinicalTrials.gov registry on 13 October 2018 and reviewed each trial's record entry. Eligible trials' characteristics were extracted and summarized. RESULTS In total, 1,386 clinical trials were identified that reported fMRI in their outcome measures with fMRI as the only primary outcome in 33% of them. 82% of fMRI trials were started after 2011. The most frequent intervention was drug (pharmacological intervention) (29%). 57% of trials had parallel assignment design and 20% were designed for cross-over assignment. For task-based fMRI, cognitive systems (46%) based on Research Domain Criteria (RDoC) was the most frequent domain of tasks. Less than one-third of trials (28%) registered at least one region of interest for their analysis. Food cue reactivity task, pain perception task, n-back task, and monetary incentive delay task were recruited in more than 25 registered trials. CONCLUSION The number of fMRI trials (fMRI as an outcome measure) with both task and rest protocols is growing rapidly. Our study suggests a growing need for harmonization and standardized checklists on both methods and analysis for preregistration of fMRI-based outcomes in clinical trials.
Collapse
Affiliation(s)
- Alaleh Sadraee
- Institute for Cognitive Science StudiesTehranIran
- Iranian National Center for Addiction StudiesTehran University of Medical SciencesTehranIran
| | | | | |
Collapse
|
6
|
Carmichael O, Schwarz AJ, Chatham CH, Scott D, Turner JA, Upadhyay J, Coimbra A, Goodman JA, Baumgartner R, English BA, Apolzan JW, Shankapal P, Hawkins KR. The role of fMRI in drug development. Drug Discov Today 2018; 23:333-348. [PMID: 29154758 PMCID: PMC5931333 DOI: 10.1016/j.drudis.2017.11.012] [Citation(s) in RCA: 38] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2017] [Revised: 10/19/2017] [Accepted: 11/13/2017] [Indexed: 12/17/2022]
Abstract
Functional magnetic resonance imaging (fMRI) has been known for over a decade to have the potential to greatly enhance the process of developing novel therapeutic drugs for prevalent health conditions. However, the use of fMRI in drug development continues to be relatively limited because of a variety of technical, biological, and strategic barriers that continue to limit progress. Here, we briefly review the roles that fMRI can have in the drug development process and the requirements it must meet to be useful in this setting. We then provide an update on our current understanding of the strengths and limitations of fMRI as a tool for drug developers and recommend activities to enhance its utility.
Collapse
Affiliation(s)
- Owen Carmichael
- Pennington Biomedical Research Center, Baton Rouge, LA, USA.
| | | | - Christopher H Chatham
- Translational Medicine Neuroscience and Biomarkers, Roche Innovation Center, Basel, Switzerland
| | | | - Jessica A Turner
- Psychology Department & Neuroscience Institute, Georgia State University, Atlanta, GA, USA
| | | | | | | | - Richard Baumgartner
- Biostatistics and Research Decision Sciences (BARDS), Merck & Co., Inc., Kenilworth, NJ, USA
| | | | - John W Apolzan
- Pennington Biomedical Research Center, Baton Rouge, LA, USA
| | | | | |
Collapse
|
7
|
Wylie KP, Smucny J, Legget KT, Tregellas JR. Targeting Functional Biomarkers in Schizophrenia with Neuroimaging. Curr Pharm Des 2017; 22:2117-23. [PMID: 26818860 DOI: 10.2174/1381612822666160127113912] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2015] [Accepted: 01/26/2016] [Indexed: 01/09/2023]
Abstract
Many of the most debilitating symptoms for psychiatric disorders such as schizophrenia remain poorly treated. As such, the development of novel treatments is urgently needed. Unfortunately, the costs associated with high failure rates for investigational compounds as they enter clinical trials has led to pharmaceutical companies downsizing or eliminating research programs needed to develop these drugs. One way of increasing the probability of success for investigational compounds is to incorporate alternative methods of identifying biological targets in order to more effectively screen new drugs. A promising method of accomplishing this goal for psychiatric drugs is to use functional magnetic resonance imaging (fMRI). fMRI investigates neural circuits, shedding light on the biology that generates symptoms such as hallucinations. Once identified, relevant neural circuits can be targeted with pharmacologic interventions and the response to these drugs measured with fMRI. This review describes the early use of fMRI in this context, and discusses the alpha7 nicotinic receptor agonist 3-(2,4-dimethoxybenzylidene) anabaseine (DMXB-A), as an example of the potential value of fMRI for psychiatric drug development.
Collapse
Affiliation(s)
- Korey P Wylie
- Department of Psychiatry, Anschutz Medical Campus, Bldg. 500, Mail Stop F546, 13001 East 17th Place, Aurora, CO, 80045, USA.
| | | | | | | |
Collapse
|
8
|
Khalili-Mahani N, Rombouts SARB, van Osch MJP, Duff EP, Carbonell F, Nickerson LD, Becerra L, Dahan A, Evans AC, Soucy JP, Wise R, Zijdenbos AP, van Gerven JM. Biomarkers, designs, and interpretations of resting-state fMRI in translational pharmacological research: A review of state-of-the-Art, challenges, and opportunities for studying brain chemistry. Hum Brain Mapp 2017; 38:2276-2325. [PMID: 28145075 DOI: 10.1002/hbm.23516] [Citation(s) in RCA: 44] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2016] [Revised: 11/21/2016] [Accepted: 01/04/2017] [Indexed: 12/11/2022] Open
Abstract
A decade of research and development in resting-state functional MRI (RSfMRI) has opened new translational and clinical research frontiers. This review aims to bridge between technical and clinical researchers who seek reliable neuroimaging biomarkers for studying drug interactions with the brain. About 85 pharma-RSfMRI studies using BOLD signal (75% of all) or arterial spin labeling (ASL) were surveyed to investigate the acute effects of psychoactive drugs. Experimental designs and objectives include drug fingerprinting dose-response evaluation, biomarker validation and calibration, and translational studies. Common biomarkers in these studies include functional connectivity, graph metrics, cerebral blood flow and the amplitude and spectrum of BOLD fluctuations. Overall, RSfMRI-derived biomarkers seem to be sensitive to spatiotemporal dynamics of drug interactions with the brain. However, drugs cause both central and peripheral effects, thus exacerbate difficulties related to biological confounds, structured noise from motion and physiological confounds, as well as modeling and inference testing. Currently, these issues are not well explored, and heterogeneities in experimental design, data acquisition and preprocessing make comparative or meta-analysis of existing reports impossible. A unifying collaborative framework for data-sharing and data-mining is thus necessary for investigating the commonalities and differences in biomarker sensitivity and specificity, and establishing guidelines. Multimodal datasets including sham-placebo or active control sessions and repeated measurements of various psychometric, physiological, metabolic and neuroimaging phenotypes are essential for pharmacokinetic/pharmacodynamic modeling and interpretation of the findings. We provide a list of basic minimum and advanced options that can be considered in design and analyses of future pharma-RSfMRI studies. Hum Brain Mapp 38:2276-2325, 2017. © 2017 Wiley Periodicals, Inc.
Collapse
Affiliation(s)
- Najmeh Khalili-Mahani
- McGill Centre for Integrative Neuroscience, Montreal Neurological Institute, McGill University, Montreal, Canada.,PERFORM Centre, Concordia University, Montreal, Canada
| | - Serge A R B Rombouts
- Department of Radiology, Leiden University Medical Centre, Leiden, The Netherlands.,Institute of Psychology and Leiden Institute for Brain and Cognition, Leiden University, Leiden, The Netherlands
| | | | - Eugene P Duff
- Institute of Psychology and Leiden Institute for Brain and Cognition, Leiden University, Leiden, The Netherlands.,Oxford Centre for Functional MRI of the Brain, Oxford University, Oxford, United Kingdom
| | | | - Lisa D Nickerson
- McLean Hospital, Belmont, Massachusetts.,Harvard Medical School, Boston, Massachusetts
| | - Lino Becerra
- Center for Pain and the Brain, Harvard Medical School & Boston Children's Hospital, Boston, Massachusetts
| | - Albert Dahan
- Department of Anesthesiology, Leiden University Medical Centre, Leiden, The Netherlands
| | - Alan C Evans
- McGill Centre for Integrative Neuroscience, Montreal Neurological Institute, McGill University, Montreal, Canada.,McConnell Brain Imaging Centre, Montreal Neurological Institute, McGill University, Montreal, Canada
| | - Jean-Paul Soucy
- PERFORM Centre, Concordia University, Montreal, Canada.,McConnell Brain Imaging Centre, Montreal Neurological Institute, McGill University, Montreal, Canada
| | - Richard Wise
- Cardiff University Brain Research Imaging Centre (CUBRIC), School of Psychology, Cardiff University, Cardiff, United Kingdom
| | - Alex P Zijdenbos
- McGill Centre for Integrative Neuroscience, Montreal Neurological Institute, McGill University, Montreal, Canada.,Biospective Inc, Montreal, Quebec, Canada
| | - Joop M van Gerven
- Centre for Human Drug Research, Leiden University Medical Centre, Leiden, The Netherlands
| |
Collapse
|
9
|
Doyle OM, Mehta MA, Brammer MJ. The role of machine learning in neuroimaging for drug discovery and development. Psychopharmacology (Berl) 2015; 232:4179-89. [PMID: 26014110 DOI: 10.1007/s00213-015-3968-0] [Citation(s) in RCA: 34] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/25/2015] [Accepted: 05/11/2015] [Indexed: 12/30/2022]
Abstract
Neuroimaging has been identified as a potentially powerful probe for the in vivo study of drug effects on the brain with utility across several phases of drug development spanning preclinical and clinical investigations. Specifically, neuroimaging can provide insight into drug penetration and distribution, target engagement, pharmacodynamics, mechanistic action and potential indicators of clinical efficacy. In this review, we focus on machine learning approaches for neuroimaging which enable us to make predictions at the individual level based on the distributed effects across the whole brain. Crucially, these approaches can be trained on data from one study and applied to an independent study and, unlike group-level statistics, can be readily use to assess the generalisability to unseen data. In this review, we present examples and suggestions for how machine learning could help answer fundamental questions spanning the drug discovery pipeline: (1) Who should I recruit for this study? (2) What should I measure and when should I measure it? (3) How does the pharmacological agent behave using an experimental medicine model?, and (4) How does a compound differ from and/or resemble existing compounds? Specifically, we present studies from the literature and we suggest areas for the focus of future development. Further refinement and tailoring of machine learning techniques may help realise their tremendous potential for drug discovery and drug validation.
Collapse
Affiliation(s)
- Orla M Doyle
- Department of Neuroimaging, Institute of Psychiatry, Psychology and Neuroscience, King's College London, De Crespigny Park, London, SE5 8AF, UK.
| | - Mitul A Mehta
- Department of Neuroimaging, Institute of Psychiatry, Psychology and Neuroscience, King's College London, De Crespigny Park, London, SE5 8AF, UK
| | - Michael J Brammer
- Department of Neuroimaging, Institute of Psychiatry, Psychology and Neuroscience, King's College London, De Crespigny Park, London, SE5 8AF, UK
| |
Collapse
|
10
|
Test–retest reliability of evoked heat stimulation BOLD fMRI. J Neurosci Methods 2015; 253:38-46. [DOI: 10.1016/j.jneumeth.2015.06.001] [Citation(s) in RCA: 23] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2015] [Revised: 06/01/2015] [Accepted: 06/03/2015] [Indexed: 11/19/2022]
|
11
|
Hargreaves RJ, Hoppin J, Sevigny J, Patel S, Chiao P, Klimas M, Verma A. Optimizing Central Nervous System Drug Development Using Molecular Imaging. Clin Pharmacol Ther 2015; 98:47-60. [PMID: 25869938 DOI: 10.1002/cpt.132] [Citation(s) in RCA: 24] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2015] [Accepted: 04/07/2015] [Indexed: 12/12/2022]
Abstract
Advances in multimodality fusion imaging technologies promise to accelerate the understanding of the systems biology of disease and help in the development of new therapeutics. The use of molecular imaging biomarkers has been proven to shorten cycle times for central nervous system (CNS) drug development and thereby increase the efficiency and return on investment from research. Imaging biomarkers can be used to help select the molecules, doses, and patients most likely to test therapeutic hypotheses by stopping those that have little chance of success and accelerating those with potential to achieve beneficial clinical outcomes. CNS imaging biomarkers have the potential to drive new medical care practices for patients in the latent phases of progressive neurodegenerative disorders by enabling the detection, preventative treatment, and tracking of disease in a paradigm shift from today's approaches that have to see the overt symptoms of disease before treating it.
Collapse
Affiliation(s)
| | - J Hoppin
- inviCRO, LLC, Boston, Massachusetts, USA
| | - J Sevigny
- Biogen, Cambridge, Massachusetts, USA
| | - S Patel
- Biogen, Cambridge, Massachusetts, USA
| | - P Chiao
- Biogen, Cambridge, Massachusetts, USA
| | - M Klimas
- Merck Research Laboratories, West Point, Pennsylvania, USA
| | - A Verma
- Biogen, Cambridge, Massachusetts, USA
| |
Collapse
|
12
|
Millan MJ, Goodwin GM, Meyer-Lindenberg A, Ögren SO, Ögren SO. 60 years of advances in neuropsychopharmacology for improving brain health, renewed hope for progress. Eur Neuropsychopharmacol 2015; 25:591-8. [PMID: 25799919 DOI: 10.1016/j.euroneuro.2015.01.015] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/09/2015] [Accepted: 01/28/2015] [Indexed: 02/01/2023]
Abstract
Pharmacotherapy is effective in helping many patients suffering from psychiatric and neurological disorders, and both psychotherapeutic and stimulation-based techniques likewise have important roles to play in their treatment. However, therapeutic progress has recently been slow. Future success for improving the control and prevention of brain disorders will depend upon deeper insights into their causes and pathophysiological substrates. It will also necessitate new and more rigorous methods for identifying, validating, developing and clinically deploying new treatments. A field of Research and Development (R and D) that remains critical to this endeavour is Neuropsychopharmacology which transformed the lives of patients by introducing pharmacological treatments for psychiatric disorder some 60 years ago. For about half of this time, the European College of Neuropsychopharmacology (ECNP) has fostered efforts to enhance our understanding of the brain, and to improve the management of psychiatric disorders. Further, together with partners in academia and industry, and in discussions with regulators and patients, the ECNP is implicated in new initiatives to achieve this goal. This is then an opportune moment to survey the field, to analyse what we have learned from the achievements and failures of the past, and to identify major challenges for the future. It is also important to highlight strategies that are being put in place in the quest for more effective treatment of brain disorders: from experimental research and drug discovery to clinical development and collaborative ventures for reinforcing "R and D". The present article sets the scene, then introduces and interlinks the eight articles that comprise this Special Volume of European Neuropsychopharmacology. A broad-based suite of themes is covered embracing: the past, present and future of "R and D" for psychiatric disorders; complementary contributions of genetics and epigenetics; efforts to improve the treatment of depression, neurodevelopmental and neurodegenerative disorders; and advances in the analysis and neuroimaging of cellular and cerebral circuits.
Collapse
Affiliation(s)
- Mark J Millan
- Pole for Innovation in Neurosciences, IDR Servier, 125 chemin de ronde, 78290 Croissy sur Seine, France.
| | - Guy M Goodwin
- University Department of Psychiatry, Oxford University, Warneford Hospital, Oxford OX3 7JX, England
| | - Andreas Meyer-Lindenberg
- Central Institute of Mental Health, University of Heidelberg/Medical Faculty Mannheim, J5, D-68159 Mannheim, Germany
| | - Sven Ove Ögren
- Department of Neuroscience, Karolinska Institutet, Retzius väg 8, S-17177 Stockholm, Sweden
| | | |
Collapse
|
13
|
Millan MJ, Goodwin GM, Meyer-Lindenberg A, Ove Ögren S. Learning from the past and looking to the future: Emerging perspectives for improving the treatment of psychiatric disorders. Eur Neuropsychopharmacol 2015; 25:599-656. [PMID: 25836356 DOI: 10.1016/j.euroneuro.2015.01.016] [Citation(s) in RCA: 101] [Impact Index Per Article: 11.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/09/2015] [Accepted: 01/28/2015] [Indexed: 02/06/2023]
Abstract
Modern neuropsychopharmacology commenced in the 1950s with the serendipitous discovery of first-generation antipsychotics and antidepressants which were therapeutically effective yet had marked adverse effects. Today, a broader palette of safer and better-tolerated agents is available for helping people that suffer from schizophrenia, depression and other psychiatric disorders, while complementary approaches like psychotherapy also have important roles to play in their treatment, both alone and in association with medication. Nonetheless, despite considerable efforts, current management is still only partially effective, and highly-prevalent psychiatric disorders of the brain continue to represent a huge personal and socio-economic burden. The lack of success in discovering more effective pharmacotherapy has contributed, together with many other factors, to a relative disengagement by pharmaceutical firms from neuropsychiatry. Nonetheless, interest remains high, and partnerships are proliferating with academic centres which are increasingly integrating drug discovery and translational research into their traditional activities. This is, then, a time of transition and an opportune moment to thoroughly survey the field. Accordingly, the present paper, first, chronicles the discovery and development of psychotropic agents, focusing in particular on their mechanisms of action and therapeutic utility, and how problems faced were eventually overcome. Second, it discusses the lessons learned from past successes and failures, and how they are being applied to promote future progress. Third, it comprehensively surveys emerging strategies that are (1), improving our understanding of the diagnosis and classification of psychiatric disorders; (2), deepening knowledge of their underlying risk factors and pathophysiological substrates; (3), refining cellular and animal models for discovery and validation of novel therapeutic agents; (4), improving the design and outcome of clinical trials; (5), moving towards reliable biomarkers of patient subpopulations and medication efficacy and (6), promoting collaborative approaches to innovation by uniting key partners from the regulators, industry and academia to patients. Notwithstanding the challenges ahead, the many changes and ideas articulated herein provide new hope and something of a framework for progress towards the improved prevention and relief of psychiatric and other CNS disorders, an urgent mission for our Century.
Collapse
Affiliation(s)
- Mark J Millan
- Pole for Innovation in Neurosciences, IDR Servier, 125 chemin de ronde, 78290 Croissy sur Seine, France.
| | - Guy M Goodwin
- University Department of Psychiatry, Oxford University, Warneford Hospital, Oxford OX3 7JX, England, UK
| | - Andreas Meyer-Lindenberg
- Central Institute of Mental Health, University of Heidelberg/Medical Faculty Mannheim, J5, D-68159 Mannheim, Germany
| | - Sven Ove Ögren
- Department of Neuroscience, Karolinska Institutet, Retzius väg 8, S-17177 Stockholm, Sweden
| |
Collapse
|
14
|
Borsook D, Hargreaves R, Bountra C, Porreca F. Lost but making progress--Where will new analgesic drugs come from? Sci Transl Med 2015; 6:249sr3. [PMID: 25122640 DOI: 10.1126/scitranslmed.3008320] [Citation(s) in RCA: 75] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
There is a critical need for effective new pharmacotherapies for pain. The paucity of new drugs successfully reaching the clinic calls for a reassessment of current analgesic drug discovery approaches. Many points early in the discovery process present significant hurdles, making it critical to exploit advances in pain neurobiology to increase the probability of success. In this review, we highlight approaches that are being pursued vigorously by the pain community for drug discovery, including innovative preclinical pain models, insights from genetics, mechanistic phenotyping of pain patients, development of biomarkers, and emerging insights into chronic pain as a disorder of both the periphery and the brain. Collaborative efforts between pharmaceutical, academic, and public entities to advance research in these areas promise to de-risk potential targets, stimulate investment, and speed evaluation and development of better pain therapies.
Collapse
Affiliation(s)
- David Borsook
- Center for Pain and the Brain, Boston Children's Hospital, Harvard Medical School, Boston, MA 02115, USA
| | - Richard Hargreaves
- Center for Pain and the Brain, Boston Children's Hospital, Harvard Medical School, Boston, MA 02115, USA
| | - Chas Bountra
- Department of Clinical Medicine, University of Oxford, Oxford OX1 2JD, UK
| | - Frank Porreca
- Center for Pain and the Brain and Department of Pharmacology, University of Arizona, Tucson, AZ 85724, USA.
| |
Collapse
|
15
|
Modo M, Kolosnjaj-Tabi J, Nicholls F, Ling W, Wilhelm C, Debarge O, Gazeau F, Clement O. Considerations for the clinical use of contrast agents for cellular MRI in regenerative medicine. CONTRAST MEDIA & MOLECULAR IMAGING 2014; 8:439-55. [PMID: 24375900 DOI: 10.1002/cmmi.1547] [Citation(s) in RCA: 32] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/28/2013] [Revised: 04/21/2013] [Accepted: 05/09/2013] [Indexed: 12/24/2022]
Abstract
Advances in regenerative medicine are rapidly transforming healthcare. A cornerstone of regenerative medicine is the introduction of cells that were grown or manipulated in vitro. Key questions that arise after these cells are re-introduced are: whether these cells are localized in the appropriate site; whether cells survive; and whether these cells migrate. These questions predominantly relate to the safety of the therapeutic approach (i.e. tumorigenesis), but certain aspects can also influence the efficacy of the therapeutic approach (e.g. site of injection). The European Medicines Agency has indicated that suitable methods for stem cell tracking should be applied where these methods are available. We here discuss the European regulatory framework, as well as the scientific evidence, that should be considered to facilitate the potential clinical implementation of magnetic resonance imaging contrast media to track implanted/injected cells in human studies.
Collapse
Affiliation(s)
- Michel Modo
- University of Pittsburgh, Department of Radiology, McGowan Institute for Regenerative Medicine, Pittsburgh, PA, 15203, USA
| | | | | | | | | | | | | | | |
Collapse
|
16
|
English BA, Thomas K, Johnstone J, Bazih A, Gertsik L, Ereshefsky L. Use of translational pharmacodynamic biomarkers in early-phase clinical studies for schizophrenia. Biomark Med 2014; 8:29-49. [PMID: 24325223 DOI: 10.2217/bmm.13.135] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022] Open
Abstract
Schizophrenia is a severe mental disorder characterized by cognitive deficits, and positive and negative symptoms. The development of effective pharmacological compounds for the treatment of schizophrenia has proven challenging and costly, with many compounds failing during clinical trials. Many failures occur due to disease heterogeneity and lack of predictive preclinical models and biomarkers that readily translate to humans during early characterization of novel antipsychotic compounds. Traditional early-phase trials consist of single- or multiple-dose designs aimed at determining the safety and tolerability of an investigational compound in healthy volunteers. However, by incorporating a translational approach employing methodologies derived from preclinical studies, such as EEG measures and imaging, into the traditional Phase I program, critical information regarding a compound's dose-response effects on pharmacodynamic biomarkers can be acquired. Furthermore, combined with the use of patients with stable schizophrenia in early-phase clinical trials, significant 'de-risking' and more confident 'go/no-go' decisions are possible.
Collapse
|
17
|
Use of functional imaging across clinical phases in CNS drug development. Transl Psychiatry 2013; 3:e282. [PMID: 23860483 PMCID: PMC3731782 DOI: 10.1038/tp.2013.43] [Citation(s) in RCA: 42] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/20/2013] [Accepted: 03/15/2013] [Indexed: 12/20/2022] Open
Abstract
The use of novel brain biomarkers using nuclear magnetic resonance imaging holds potential of making central nervous system (CNS) drug development more efficient. By evaluating changes in brain function in the disease state or drug effects on brain function, the technology opens up the possibility of obtaining objective data on drug effects in the living awake brain. By providing objective data, imaging may improve the probability of success of identifying useful drugs to treat CNS diseases across all clinical phases (I-IV) of drug development. The evolution of functional imaging and the promise it holds to contribute to drug development will require the development of standards (including good imaging practice), but, if well integrated into drug development, functional imaging can define markers of CNS penetration, drug dosing and target engagement (even for drugs that are not amenable to positron emission tomography imaging) in phase I; differentiate objective measures of efficacy and side effects and responders vs non-responders in phase II, evaluate differences between placebo and drug in phase III trials and provide insights into disease modification in phase IV trials.
Collapse
|
18
|
Plichta MM, Schwarz AJ, Grimm O, Morgen K, Mier D, Haddad L, Gerdes ABM, Sauer C, Tost H, Esslinger C, Colman P, Wilson F, Kirsch P, Meyer-Lindenberg A. Test-retest reliability of evoked BOLD signals from a cognitive-emotive fMRI test battery. Neuroimage 2012; 60:1746-58. [PMID: 22330316 DOI: 10.1016/j.neuroimage.2012.01.129] [Citation(s) in RCA: 222] [Impact Index Per Article: 18.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2011] [Revised: 01/26/2012] [Accepted: 01/28/2012] [Indexed: 11/26/2022] Open
Abstract
Even more than in cognitive research applications, moving fMRI to the clinic and the drug development process requires the generation of stable and reliable signal changes. The performance characteristics of the fMRI paradigm constrain experimental power and may require different study designs (e.g., crossover vs. parallel groups), yet fMRI reliability characteristics can be strongly dependent on the nature of the fMRI task. The present study investigated both within-subject and group-level reliability of a combined three-task fMRI battery targeting three systems of wide applicability in clinical and cognitive neuroscience: an emotional (face matching), a motivational (monetary reward anticipation) and a cognitive (n-back working memory) task. A group of 25 young, healthy volunteers were scanned twice on a 3T MRI scanner with a mean test-retest interval of 14.6 days. FMRI reliability was quantified using the intraclass correlation coefficient (ICC) applied at three different levels ranging from a global to a localized and fine spatial scale: (1) reliability of group-level activation maps over the whole brain and within targeted regions of interest (ROIs); (2) within-subject reliability of ROI-mean amplitudes and (3) within-subject reliability of individual voxels in the target ROIs. Results showed robust evoked activation of all three tasks in their respective target regions (emotional task=amygdala; motivational task=ventral striatum; cognitive task=right dorsolateral prefrontal cortex and parietal cortices) with high effect sizes (ES) of ROI-mean summary values (ES=1.11-1.44 for the faces task, 0.96-1.43 for the reward task, 0.83-2.58 for the n-back task). Reliability of group level activation was excellent for all three tasks with ICCs of 0.89-0.98 at the whole brain level and 0.66-0.97 within target ROIs. Within-subject reliability of ROI-mean amplitudes across sessions was fair to good for the reward task (ICCs=0.56-0.62) and, dependent on the particular ROI, also fair-to-good for the n-back task (ICCs=0.44-0.57) but lower for the faces task (ICC=-0.02-0.16). In conclusion, all three tasks are well suited to between-subject designs, including imaging genetics. When specific recommendations are followed, the n-back and reward task are also suited for within-subject designs, including pharmaco-fMRI. The present study provides task-specific fMRI reliability performance measures that will inform the optimal use, powering and design of fMRI studies using comparable tasks.
Collapse
Affiliation(s)
- Michael M Plichta
- Central Institute of Mental Health, Department of Psychiatry and Psychotherapy, University of Heidelberg/Medical Faculty Mannheim, Mannheim, Germany.
| | | | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|