1
|
Tesink V, Douglas T, Forsberg L, Ligthart S, Meynen G. Right to mental integrity and neurotechnologies: implications of the extended mind thesis. JOURNAL OF MEDICAL ETHICS 2024; 50:656-663. [PMID: 38408854 DOI: 10.1136/jme-2023-109645] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/09/2023] [Accepted: 01/14/2024] [Indexed: 02/28/2024]
Abstract
The possibility of neurotechnological interference with our brain and mind raises questions about the moral rights that would protect against the (mis)use of these technologies. One such moral right that has received recent attention is the right to mental integrity. Though the metaphysical boundaries of the mind are a matter of live debate, most defences of this moral right seem to assume an internalist (brain-based) view of the mind. In this article, we will examine what an extended account of the mind might imply for the right to mental integrity and the protection it provides against neurotechnologies. We argue that, on an extended account of the mind, the scope of the right to mental integrity would expand significantly, implying that neurotechnologies would no longer pose a uniquely serious threat to the right. In addition, some neurotechnologies may even be protected by the right to mental integrity, as the technologies would become part of the mind. We conclude that adopting an extended account of the mind has significant implications for the right to mental integrity in terms of its protective scope and capacity to protect against neurotechnologies, demonstrating that metaphysical assumptions about the mind play an important role in determining the moral protection provided by the right.
Collapse
Affiliation(s)
- Vera Tesink
- Department of Philosophy, Faculty of Humanities, Vrije Universiteit Amsterdam, Amsterdam, Netherlands
| | - Thomas Douglas
- Oxford Uehiro Centre for Practical Ethics, Faculty of Philosophy, University of Oxford, Oxford, UK
- Jesus College, University of Oxford, Oxford, UK
| | - Lisa Forsberg
- Oxford Uehiro Centre for Practical Ethics, Faculty of Philosophy, University of Oxford, Oxford, UK
| | - Sjors Ligthart
- Department of Criminal Law, Tilburg University, Tilburg, Netherlands
- Willem Pompe Institute for Criminal Law and Criminology and UCALL, Utrecht University, Utrecht, Netherlands
| | - Gerben Meynen
- Department of Philosophy, Faculty of Humanities, Vrije Universiteit Amsterdam, Amsterdam, Netherlands
- Willem Pompe Institute for Criminal Law and Criminology and UCALL, Utrecht University, Utrecht, Netherlands
| |
Collapse
|
2
|
Holiel HA, Fawzi SA, Al-Atabany W. Pre-processing visual scenes for retinal prosthesis systems: A comprehensive review. Artif Organs 2024. [PMID: 39023279 DOI: 10.1111/aor.14824] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2023] [Revised: 05/13/2024] [Accepted: 06/21/2024] [Indexed: 07/20/2024]
Abstract
BACKGROUND Retinal prostheses offer hope for individuals with degenerative retinal diseases by stimulating the remaining retinal cells to partially restore their vision. This review delves into the current advancements in retinal prosthesis technology, with a special emphasis on the pivotal role that image processing and machine learning techniques play in this evolution. METHODS We provide a comprehensive analysis of the existing implantable devices and optogenetic strategies, delineating their advantages, limitations, and challenges in addressing complex visual tasks. The review extends to various image processing algorithms and deep learning architectures that have been implemented to enhance the functionality of retinal prosthetic devices. We also illustrate the testing results by demonstrating the clinical trials or using Simulated Prosthetic Vision (SPV) through phosphene simulations, which is a critical aspect of simulating visual perception for retinal prosthesis users. RESULTS Our review highlights the significant progress in retinal prosthesis technology, particularly its capacity to augment visual perception among the visually impaired. It discusses the integration between image processing and deep learning, illustrating their impact on individual interactions and navigations within the environment through applying clinical trials and also illustrating the limitations of some techniques to be used with current devices, as some approaches only use simulation even on sighted-normal individuals or rely on qualitative analysis, where some consider realistic perception models and others do not. CONCLUSION This interdisciplinary field holds promise for the future of retinal prostheses, with the potential to significantly enhance the quality of life for individuals with retinal prostheses. Future research directions should pivot towards optimizing phosphene simulations for SPV approaches, considering the distorted and confusing nature of phosphene perception, thereby enriching the visual perception provided by these prosthetic devices. This endeavor will not only improve navigational independence but also facilitate a more immersive interaction with the environment.
Collapse
Affiliation(s)
- Heidi Ahmed Holiel
- Medical Imaging and Image Processing Research Group, Center for Informatics Science, Nile University, Sheikh Zayed City, Egypt
| | - Sahar Ali Fawzi
- Medical Imaging and Image Processing Research Group, Center for Informatics Science, Nile University, Sheikh Zayed City, Egypt
- Systems and Biomedical Engineering Department, Cairo University, Giza, Egypt
| | - Walid Al-Atabany
- Medical Imaging and Image Processing Research Group, Center for Informatics Science, Nile University, Sheikh Zayed City, Egypt
- Biomedical Engineering Department, Helwan University, Helwan, Egypt
| |
Collapse
|
3
|
Manero A, Rivera V, Fu Q, Schwartzman JD, Prock-Gibbs H, Shah N, Gandhi D, White E, Crawford KE, Coathup MJ. Emerging Medical Technologies and Their Use in Bionic Repair and Human Augmentation. Bioengineering (Basel) 2024; 11:695. [PMID: 39061777 PMCID: PMC11274085 DOI: 10.3390/bioengineering11070695] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2024] [Revised: 07/04/2024] [Accepted: 07/07/2024] [Indexed: 07/28/2024] Open
Abstract
As both the proportion of older people and the length of life increases globally, a rise in age-related degenerative diseases, disability, and prolonged dependency is projected. However, more sophisticated biomedical materials, as well as an improved understanding of human disease, is forecast to revolutionize the diagnosis and treatment of conditions ranging from osteoarthritis to Alzheimer's disease as well as impact disease prevention. Another, albeit quieter, revolution is also taking place within society: human augmentation. In this context, humans seek to improve themselves, metamorphosing through self-discipline or more recently, through use of emerging medical technologies, with the goal of transcending aging and mortality. In this review, and in the pursuit of improved medical care following aging, disease, disability, or injury, we first highlight cutting-edge and emerging materials-based neuroprosthetic technologies designed to restore limb or organ function. We highlight the potential for these technologies to be utilized to augment human performance beyond the range of natural performance. We discuss and explore the growing social movement of human augmentation and the idea that it is possible and desirable to use emerging technologies to push the boundaries of what it means to be a healthy human into the realm of superhuman performance and intelligence. This potential future capability is contrasted with limitations in the right-to-repair legislation, which may create challenges for patients. Now is the time for continued discussion of the ethical strategies for research, implementation, and long-term device sustainability or repair.
Collapse
Affiliation(s)
- Albert Manero
- Limbitless Solutions, University of Central Florida, 12703 Research Parkway, Suite 100, Orlando, FL 32826, USA (V.R.)
- Biionix Cluster, University of Central Florida, Orlando, FL 32827, USA; (Q.F.); (K.E.C.)
| | - Viviana Rivera
- Limbitless Solutions, University of Central Florida, 12703 Research Parkway, Suite 100, Orlando, FL 32826, USA (V.R.)
| | - Qiushi Fu
- Biionix Cluster, University of Central Florida, Orlando, FL 32827, USA; (Q.F.); (K.E.C.)
- Department of Mechanical and Aerospace Engineering, University of Central Florida, Orlando, FL 32816, USA
| | - Jonathan D. Schwartzman
- College of Medicine, University of Central Florida, Orlando, FL 32827, USA; (J.D.S.); (H.P.-G.); (N.S.); (D.G.); (E.W.)
| | - Hannah Prock-Gibbs
- College of Medicine, University of Central Florida, Orlando, FL 32827, USA; (J.D.S.); (H.P.-G.); (N.S.); (D.G.); (E.W.)
| | - Neel Shah
- College of Medicine, University of Central Florida, Orlando, FL 32827, USA; (J.D.S.); (H.P.-G.); (N.S.); (D.G.); (E.W.)
| | - Deep Gandhi
- College of Medicine, University of Central Florida, Orlando, FL 32827, USA; (J.D.S.); (H.P.-G.); (N.S.); (D.G.); (E.W.)
| | - Evan White
- College of Medicine, University of Central Florida, Orlando, FL 32827, USA; (J.D.S.); (H.P.-G.); (N.S.); (D.G.); (E.W.)
| | - Kaitlyn E. Crawford
- Biionix Cluster, University of Central Florida, Orlando, FL 32827, USA; (Q.F.); (K.E.C.)
- Department of Materials Science and Engineering, University of Central Florida, Orlando, FL 32816, USA
| | - Melanie J. Coathup
- Biionix Cluster, University of Central Florida, Orlando, FL 32827, USA; (Q.F.); (K.E.C.)
- College of Medicine, University of Central Florida, Orlando, FL 32827, USA; (J.D.S.); (H.P.-G.); (N.S.); (D.G.); (E.W.)
| |
Collapse
|
4
|
Zhang B, Zhang R, Zhao J, Yang J, Xu S. The mechanism of human color vision and potential implanted devices for artificial color vision. Front Neurosci 2024; 18:1408087. [PMID: 38962178 PMCID: PMC11221215 DOI: 10.3389/fnins.2024.1408087] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2024] [Accepted: 05/31/2024] [Indexed: 07/05/2024] Open
Abstract
Vision plays a major role in perceiving external stimuli and information in our daily lives. The neural mechanism of color vision is complicated, involving the co-ordinated functions of a variety of cells, such as retinal cells and lateral geniculate nucleus cells, as well as multiple levels of the visual cortex. In this work, we reviewed the history of experimental and theoretical studies on this issue, from the fundamental functions of the individual cells of the visual system to the coding in the transmission of neural signals and sophisticated brain processes at different levels. We discuss various hypotheses, models, and theories related to the color vision mechanism and present some suggestions for developing novel implanted devices that may help restore color vision in visually impaired people or introduce artificial color vision to those who need it.
Collapse
Affiliation(s)
- Bingao Zhang
- Key Laboratory for the Physics and Chemistry of Nanodevices, Institute of Physical Electronics, Department of Electronics, Peking University, Beijing, China
| | - Rong Zhang
- Key Laboratory for the Physics and Chemistry of Nanodevices, Institute of Physical Electronics, Department of Electronics, Peking University, Beijing, China
| | - Jingjin Zhao
- Key Laboratory for the Physics and Chemistry of Nanodevices, Institute of Physical Electronics, Department of Electronics, Peking University, Beijing, China
| | - Jiarui Yang
- Beijing Key Laboratory of Restoration of Damaged Ocular Nerve, Department of Ophthalmology, Peking University Third Hospital, Beijing, China
| | - Shengyong Xu
- Key Laboratory for the Physics and Chemistry of Nanodevices, Institute of Physical Electronics, Department of Electronics, Peking University, Beijing, China
| |
Collapse
|
5
|
Nadolskis LG, Turkstra LM, Larnyo E, Beyeler M. Great expectations: Aligning visual prosthetic development with implantee needs. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2024:2024.03.12.24304186. [PMID: 38559196 PMCID: PMC10980134 DOI: 10.1101/2024.03.12.24304186] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 04/04/2024]
Abstract
Purpose Visual prosthetics have emerged as a promising assistive technology for individuals with vision loss, yet research often overlooks the human aspects of this technology. While previous studies have concentrated on the perceptual experiences of implant recipients (implantees) or the attitudes of potential implantees towards near-future implants, a systematic account of how current implants are being used in everyday life is still lacking. Methods We interviewed six recipients of the most widely used visual implants (Argus II and Orion) and six leading researchers in the field. Through thematic and statistical analyses, we explored the daily usage of these implants by implantees and compared their responses to the expectations of researchers. We also sought implantees' input on desired features for future versions, aiming to inform the development of the next generation of implants. Results Although implants are designed to facilitate various daily activities, we found that implantees use them less frequently than researchers expected. This discrepancy primarily stems from issues with usability and reliability, with implantees finding alternative methods to accomplish tasks, reducing the need to rely on the implant. For future implants, implantees emphasized the desire for improved vision, smart integration, and increased independence. Conclusions Our study reveals a significant gap between researcher expectations and implantee experiences with visual prostheses, underscoring the importance of focusing future research on usability and real-world application. Translational relevance This work advocates for a better alignment between technology development and implantee needs to enhance clinical relevance and practical utility of visual prosthetics.
Collapse
Affiliation(s)
- Lucas Gil Nadolskis
- Interdepartmental Graduate Program in Dynamical Neuroscience, University of California, Santa Barbara
| | - Lily Marie Turkstra
- Department of Psychological & Brain Sciences, University of California, Santa Barbara
| | - Ebenezer Larnyo
- Center for Black Studies Research, University of California, Santa Barbara
| | - Michael Beyeler
- Department of Psychological & Brain Sciences, University of California, Santa Barbara
- Department of Computer Science, University of California, Santa Barbara
| |
Collapse
|
6
|
van der Grinten M, de Ruyter van Steveninck J, Lozano A, Pijnacker L, Rueckauer B, Roelfsema P, van Gerven M, van Wezel R, Güçlü U, Güçlütürk Y. Towards biologically plausible phosphene simulation for the differentiable optimization of visual cortical prostheses. eLife 2024; 13:e85812. [PMID: 38386406 PMCID: PMC10883675 DOI: 10.7554/elife.85812] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2022] [Accepted: 01/21/2024] [Indexed: 02/23/2024] Open
Abstract
Blindness affects millions of people around the world. A promising solution to restoring a form of vision for some individuals are cortical visual prostheses, which bypass part of the impaired visual pathway by converting camera input to electrical stimulation of the visual system. The artificially induced visual percept (a pattern of localized light flashes, or 'phosphenes') has limited resolution, and a great portion of the field's research is devoted to optimizing the efficacy, efficiency, and practical usefulness of the encoding of visual information. A commonly exploited method is non-invasive functional evaluation in sighted subjects or with computational models by using simulated prosthetic vision (SPV) pipelines. An important challenge in this approach is to balance enhanced perceptual realism, biologically plausibility, and real-time performance in the simulation of cortical prosthetic vision. We present a biologically plausible, PyTorch-based phosphene simulator that can run in real-time and uses differentiable operations to allow for gradient-based computational optimization of phosphene encoding models. The simulator integrates a wide range of clinical results with neurophysiological evidence in humans and non-human primates. The pipeline includes a model of the retinotopic organization and cortical magnification of the visual cortex. Moreover, the quantitative effects of stimulation parameters and temporal dynamics on phosphene characteristics are incorporated. Our results demonstrate the simulator's suitability for both computational applications such as end-to-end deep learning-based prosthetic vision optimization as well as behavioral experiments. The modular and open-source software provides a flexible simulation framework for computational, clinical, and behavioral neuroscientists working on visual neuroprosthetics.
Collapse
Affiliation(s)
| | | | - Antonio Lozano
- Netherlands Institute for Neuroscience, Vrije Universiteit, Amsterdam, Netherlands
| | - Laura Pijnacker
- Donders Institute for Brain Cognition and Behaviour, Radboud University Nijmegen, Nijmegen, Netherlands
| | - Bodo Rueckauer
- Donders Institute for Brain Cognition and Behaviour, Radboud University Nijmegen, Nijmegen, Netherlands
| | - Pieter Roelfsema
- Netherlands Institute for Neuroscience, Vrije Universiteit, Amsterdam, Netherlands
| | - Marcel van Gerven
- Donders Institute for Brain Cognition and Behaviour, Radboud University Nijmegen, Nijmegen, Netherlands
| | - Richard van Wezel
- Donders Institute for Brain Cognition and Behaviour, Radboud University Nijmegen, Nijmegen, Netherlands
- Biomedical Signals and Systems Group, University of Twente, Enschede, Netherlands
| | - Umut Güçlü
- Donders Institute for Brain Cognition and Behaviour, Radboud University Nijmegen, Nijmegen, Netherlands
| | - Yağmur Güçlütürk
- Donders Institute for Brain Cognition and Behaviour, Radboud University Nijmegen, Nijmegen, Netherlands
| |
Collapse
|
7
|
Donati E, Valle G. Neuromorphic hardware for somatosensory neuroprostheses. Nat Commun 2024; 15:556. [PMID: 38228580 PMCID: PMC10791662 DOI: 10.1038/s41467-024-44723-3] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2022] [Accepted: 01/03/2024] [Indexed: 01/18/2024] Open
Abstract
In individuals with sensory-motor impairments, missing limb functions can be restored using neuroprosthetic devices that directly interface with the nervous system. However, restoring the natural tactile experience through electrical neural stimulation requires complex encoding strategies. Indeed, they are presently limited in effectively conveying or restoring tactile sensations by bandwidth constraints. Neuromorphic technology, which mimics the natural behavior of neurons and synapses, holds promise for replicating the encoding of natural touch, potentially informing neurostimulation design. In this perspective, we propose that incorporating neuromorphic technologies into neuroprostheses could be an effective approach for developing more natural human-machine interfaces, potentially leading to advancements in device performance, acceptability, and embeddability. We also highlight ongoing challenges and the required actions to facilitate the future integration of these advanced technologies.
Collapse
Affiliation(s)
- Elisa Donati
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, Switzerland.
| | - Giacomo Valle
- Department of Organismal Biology and Anatomy, University of Chicago, Chicago, IL, USA.
| |
Collapse
|
8
|
Granley J, Fauvel T, Chalk M, Beyeler M. Human-in-the-Loop Optimization for Deep Stimulus Encoding in Visual Prostheses. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 2023; 36:79376-79398. [PMID: 38984104 PMCID: PMC11232484] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Subscribe] [Scholar Register] [Indexed: 07/11/2024]
Abstract
Neuroprostheses show potential in restoring lost sensory function and enhancing human capabilities, but the sensations produced by current devices often seem unnatural or distorted. Exact placement of implants and differences in individual perception lead to significant variations in stimulus response, making personalized stimulus optimization a key challenge. Bayesian optimization could be used to optimize patient-specific stimulation parameters with limited noisy observations, but is not feasible for high-dimensional stimuli. Alternatively, deep learning models can optimize stimulus encoding strategies, but typically assume perfect knowledge of patient-specific variations. Here we propose a novel, practically feasible approach that overcomes both of these fundamental limitations. First, a deep encoder network is trained to produce optimal stimuli for any individual patient by inverting a forward model mapping electrical stimuli to visual percepts. Second, a preferential Bayesian optimization strategy utilizes this encoder to optimize patient-specific parameters for a new patient, using a minimal number of pairwise comparisons between candidate stimuli. We demonstrate the viability of this approach on a novel, state-of-the-art visual prosthesis model. We show that our approach quickly learns a personalized stimulus encoder, leads to dramatic improvements in the quality of restored vision, and is robust to noisy patient feedback and misspecifications in the underlying forward model. Overall, our results suggest that combining the strengths of deep learning and Bayesian optimization could significantly improve the perceptual experience of patients fitted with visual prostheses and may prove a viable solution for a range of neuroprosthetic technologies.
Collapse
Affiliation(s)
- Jacob Granley
- Department of Computer Science, University of California, Santa Barbara
| | - Tristan Fauvel
- Institut de la Vision, Sorbonne Université, 17 rue Moreau, F-75012 Paris, France, Now with Quinten Health
| | - Matthew Chalk
- Institut de la Vision, Sorbonne Université, 17 rue Moreau, F-75012 Paris, France
| | - Michael Beyeler
- Department of Computer Science, Department of Psychological & Brain Sciences, University of California, Santa Barbara
| |
Collapse
|
9
|
Hernández-Sebastián N, Carpio-Verdín VM, Ambriz-Vargas F, Morales-Morales F, Benítez-Lara A, Buenrostro-Jáuregui MH, Bojorges-Valdez E, Barrientos-García B. Fabrication and Characterization of a Flexible Thin-Film-Based Array of Microelectrodes for Corneal Electrical Stimulation. MICROMACHINES 2023; 14:1999. [PMID: 38004856 PMCID: PMC10673352 DOI: 10.3390/mi14111999] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/01/2023] [Revised: 10/21/2023] [Accepted: 10/22/2023] [Indexed: 11/26/2023]
Abstract
The electric stimulation (ES) of the cornea is a novel therapeutic approach to the treatment of degenerative visual diseases. Currently, ES is delivered by placing a mono-element electrode on the surface of the cornea that uniformly stimulates the eye along the electrode site. It has been reported that a certain degree of correlation exists between the location of the stimulated retinal area and the position of the electrode. Therefore, in this study, we present the development of a sectioned surface electrode for selective electric stimulation of the human cornea. The proposed device consists of 16 independent microelectrodes, a reference electrode, and 18 contact pads. The microelectrodes have a size of 200 µm × 200 µm, are arranged in a 4 × 4 matrix, and cover a total stimulation area of 16 mm2. The proposed fabrication process, based on surface micromachining technology and flexible electronics, uses only three materials: polyimide, aluminum, and titanium, which allow us to obtain a simplified, ergonomic, and reproducible fabrication process. The fabricated prototype was validated to laboratory level by electrical and electrochemical tests, showing a relatively high electrical conductivity and average impedance from 712 kΩ to 1.4 MΩ at the clinically relevant frequency range (from 11 Hz to 30 Hz). Additionally, the biocompatibility of the electrode prototype was demonstrated by performing in vivo tests and by analyzing the polyimide films using Fourier transform infrared spectroscopy (FTIR). The resulting electrode prototype is robust, mechanically flexible, and biocompatible, with a high potential to be used for selective ES of the cornea.
Collapse
Affiliation(s)
- Natiely Hernández-Sebastián
- Centro de Investigaciones en Óptica, A.C. Loma del Bosque 115, León 37150, Mexico; (V.M.C.-V.); (F.A.-V.); (F.M.-M.); (B.B.-G.)
| | - Víctor Manuel Carpio-Verdín
- Centro de Investigaciones en Óptica, A.C. Loma del Bosque 115, León 37150, Mexico; (V.M.C.-V.); (F.A.-V.); (F.M.-M.); (B.B.-G.)
| | - Fabián Ambriz-Vargas
- Centro de Investigaciones en Óptica, A.C. Loma del Bosque 115, León 37150, Mexico; (V.M.C.-V.); (F.A.-V.); (F.M.-M.); (B.B.-G.)
| | - Francisco Morales-Morales
- Centro de Investigaciones en Óptica, A.C. Loma del Bosque 115, León 37150, Mexico; (V.M.C.-V.); (F.A.-V.); (F.M.-M.); (B.B.-G.)
| | - Alfredo Benítez-Lara
- CONAHCYT, Centro de Investigaciones en Óptica, A.C. Loma del Bosque 115, León 37150, Mexico;
| | | | - Erik Bojorges-Valdez
- Departamento de Estudios en Ingeniería para la Innovación, Universidad Iberoamericana Ciudad de México, Ciudad de México 01219, Mexico;
| | - Bernardino Barrientos-García
- Centro de Investigaciones en Óptica, A.C. Loma del Bosque 115, León 37150, Mexico; (V.M.C.-V.); (F.A.-V.); (F.M.-M.); (B.B.-G.)
| |
Collapse
|
10
|
Karadima V, Pezaris EA, Pezaris JS. Attitudes of potential recipients toward emerging visual prosthesis technologies. Sci Rep 2023; 13:10963. [PMID: 37414798 PMCID: PMC10325978 DOI: 10.1038/s41598-023-36913-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2023] [Accepted: 06/12/2023] [Indexed: 07/08/2023] Open
Abstract
With the advent of multiple visual prosthesis devices to treat blindness, the question of how potential patients view such interventions becomes important in order to understand the levels of expectation and acceptance, and the perceived risk-reward balance across the different device approaches. Building on previous work on single device approaches done with blind individuals in Chicago and Detroit, USA, Melbourne, Australia, and Bejing, China, we investigated attitudes in blind individuals in Athens, Greece with coverage expanded to three of the contemporary approaches, Retinal, Thalamic, and Cortical. We presented an informational lecture on the approaches, had potential participants fill out a preliminary Questionnaire 1, then organized selected subjects into focus groups for guided discussion on visual prostheses, and finally had these subjects fill out a more detailed Questionnaire 2. We report here the first quantitative data that compares multiple prosthesis approaches. Our primary findings are that for these potential patients, perceived risk continues to outweigh perceived benefits, with the Retinal approach having the least negative overall impression and the Cortical approach the most negative. Concerns about the quality of restored vision were primary. Factors that drove the choice of hypothetical participation in a clinical trial were age and years of blindness. Secondary factors focused on positive clinical outcomes. The focus groups served to swing the impressions of each approach from neutrality toward the extremes of a Likert scale, and shifted the overall willingness to participate in a clinical trial from neutral to negative. These results, coupled with informal assessment of audience questions after the informational lecture, suggest that a substantial improvement in performance over currently available devices will be necessary before visual prostheses gain wide acceptance.
Collapse
Affiliation(s)
- Vicky Karadima
- Multisensory and Temporal Processing Lab (MultiTimeLab), Department of Psychology, Panteion University of Social and Political Sciences, Athens, Greece
| | | | - John S Pezaris
- Department of Neurosurgery, Massachusetts General Hospital, Boston, MA, USA.
- Department of Neurosurgery, Harvard Medical School, Boston, MA, USA.
| |
Collapse
|
11
|
Kasowski J, Johnson BA, Neydavood R, Akkaraju A, Beyeler M. A systematic review of extended reality (XR) for understanding and augmenting vision loss. J Vis 2023; 23:5. [PMID: 37140911 PMCID: PMC10166121 DOI: 10.1167/jov.23.5.5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2022] [Accepted: 04/04/2023] [Indexed: 05/05/2023] Open
Abstract
Over the past decade, extended reality (XR) has emerged as an assistive technology not only to augment residual vision of people losing their sight but also to study the rudimentary vision restored to blind people by a visual neuroprosthesis. A defining quality of these XR technologies is their ability to update the stimulus based on the user's eye, head, or body movements. To make the best use of these emerging technologies, it is valuable and timely to understand the state of this research and identify any shortcomings that are present. Here we present a systematic literature review of 227 publications from 106 different venues assessing the potential of XR technology to further visual accessibility. In contrast to other reviews, we sample studies from multiple scientific disciplines, focus on technology that augments a person's residual vision, and require studies to feature a quantitative evaluation with appropriate end users. We summarize prominent findings from different XR research areas, show how the landscape has changed over the past decade, and identify scientific gaps in the literature. Specifically, we highlight the need for real-world validation, the broadening of end-user participation, and a more nuanced understanding of the usability of different XR-based accessibility aids.
Collapse
Affiliation(s)
- Justin Kasowski
- Graduate Program in Dynamical Neuroscience, University of California, Santa Barbara, CA, USA
| | - Byron A Johnson
- Department of Psychological & Brain Sciences, University of California, Santa Barbara, CA, USA
| | - Ryan Neydavood
- Department of Psychological & Brain Sciences, University of California, Santa Barbara, CA, USA
| | - Anvitha Akkaraju
- Department of Psychological & Brain Sciences, University of California, Santa Barbara, CA, USA
| | - Michael Beyeler
- Department of Psychological & Brain Sciences, University of California, Santa Barbara, CA, USA
- Department of Computer Science, University of California, Santa Barbara, CA, USA
| |
Collapse
|
12
|
Paré S, Bleau M, Dricot L, Ptito M, Kupers R. Brain structural changes in blindness: a systematic review and an anatomical likelihood estimation (ALE) meta-analysis. Neurosci Biobehav Rev 2023; 150:105165. [PMID: 37054803 DOI: 10.1016/j.neubiorev.2023.105165] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2022] [Revised: 03/23/2023] [Accepted: 04/09/2023] [Indexed: 04/15/2023]
Abstract
In recent decades, numerous structural brain imaging studies investigated purported morphometric changes in early (EB) and late onset blindness (LB). The results of these studies have not yielded very consistent results, neither with respect to the type, nor to the anatomical locations of the brain morphometric alterations. To better characterize the effects of blindness on brain morphometry, we performed a systematic review and an Anatomical-Likelihood-Estimation (ALE) coordinate-based-meta-analysis of 65 eligible studies on brain structural changes in EB and LB, including 890 EB, 466 LB and 1257 sighted controls. Results revealed atrophic changes throughout the whole extent of the retino-geniculo-striate system in both EB and LB, whereas changes in areas beyond the occipital lobe occurred in EB only. We discuss the nature of some of the contradictory findings with respect to the used brain imaging methodologies and characteristics of the blind populations such as the onset, duration and cause of blindness. Future studies should aim for much larger sample sizes, eventually by merging data from different brain imaging centers using the same imaging sequences, opt for multimodal structural brain imaging, and go beyond a purely structural approach by combining functional with structural connectivity network analyses.
Collapse
Affiliation(s)
- Samuel Paré
- School of Optometry, University of Montreal, Montreal, Qc, Canada
| | - Maxime Bleau
- School of Optometry, University of Montreal, Montreal, Qc, Canada
| | - Laurence Dricot
- Institute of NeuroScience (IoNS), Université catholique de Louvain (UCLouvain), Bruxelles, Belgium
| | - Maurice Ptito
- School of Optometry, University of Montreal, Montreal, Qc, Canada; Department of Neurology and Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, Qc, Canada; Department of Neuroscience, University of Copenhagen, Copenhagen, Denmark
| | - Ron Kupers
- School of Optometry, University of Montreal, Montreal, Qc, Canada; Institute of NeuroScience (IoNS), Université catholique de Louvain (UCLouvain), Bruxelles, Belgium; Department of Neuroscience, University of Copenhagen, Copenhagen, Denmark.
| |
Collapse
|
13
|
Elnabawy RH, Abdennadher S, Hellwich O, Eldawlatly S. Object recognition and localization enhancement in visual prostheses: a real-time mixed reality simulation. Biomed Eng Online 2022; 21:91. [PMID: 36566183 DOI: 10.1186/s12938-022-01059-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2022] [Accepted: 12/12/2022] [Indexed: 12/25/2022] Open
Abstract
Blindness is a main threat that affects the daily life activities of any human. Visual prostheses have been introduced to provide artificial vision to the blind with the aim of allowing them to restore confidence and independence. In this article, we propose an approach that involves four image enhancement techniques to facilitate object recognition and localization for visual prostheses users. These techniques are clip art representation of the objects, edge sharpening, corner enhancement and electrode dropout handling. The proposed techniques are tested in a real-time mixed reality simulation environment that mimics vision perceived by visual prostheses users. Twelve experiments were conducted to measure the performance of the participants in object recognition and localization. The experiments involved single objects, multiple objects and navigation. To evaluate the performance of the participants in objects recognition, we measure their recognition time, recognition accuracy and confidence level. For object localization, two metrics were used to measure the performance of the participants which are the grasping attempt time and the grasping accuracy. The results demonstrate that using all enhancement techniques simultaneously gives higher accuracy, higher confidence level and less time for recognizing and grasping objects in comparison to not applying the enhancement techniques or applying pair-wise combinations of them. Visual prostheses could benefit from the proposed approach to provide users with an enhanced perception.
Collapse
Affiliation(s)
- Reham H Elnabawy
- Digital Media Engineering and Technology Department, Faculty of Media Engineering and Technology, German University in Cairo, Cairo, Egypt
| | - Slim Abdennadher
- Computer Science and Engineering Department, Faculty of Media Engineering and Technology, German University in Cairo, Cairo, Egypt.,Computer Science Department, Faculty of Informatics and Computer Science, German International University, New Administrative Capital, Egypt
| | - Olaf Hellwich
- Chair of Computer Vision and Remote Sensing, Technische Universität Berlin, Berlin, Germany
| | - Seif Eldawlatly
- Computer and Systems Engineering Department, Faculty of Engineering, Ain Shams University, 1 El-Sarayat St., Abbassia, Cairo, Egypt. .,Computer Science and Engineering Department, The American University in Cairo, Cairo, Egypt.
| |
Collapse
|
14
|
Grani F, Soto-Sánchez C, Fimia A, Fernández E. Toward a personalized closed-loop stimulation of the visual cortex: Advances and challenges. Front Cell Neurosci 2022; 16:1034270. [PMID: 36582211 PMCID: PMC9792612 DOI: 10.3389/fncel.2022.1034270] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2022] [Accepted: 11/24/2022] [Indexed: 12/15/2022] Open
Abstract
Current cortical visual prosthesis approaches are primarily unidirectional and do not consider the feed-back circuits that exist in just about every part of the nervous system. Herein, we provide a brief overview of some recent developments for better controlling brain stimulation and present preliminary human data indicating that closed-loop strategies could considerably enhance the effectiveness, safety, and long-term stability of visual cortex stimulation. We propose that the development of improved closed-loop strategies may help to enhance our capacity to communicate with the brain.
Collapse
Affiliation(s)
- Fabrizio Grani
- Institute of Bioengineering, Universidad Miguel Hernández de Elche, Elche, Spain
| | - Cristina Soto-Sánchez
- Institute of Bioengineering, Universidad Miguel Hernández de Elche, Elche, Spain,Biomedical Research Networking Center in Bioengineering, Biomaterials and Nanomedicine (CIBER-BBN), Madrid, Spain
| | - Antonio Fimia
- Departamento de Ciencia de Materiales, Óptica y Tecnología Electrónica, Universidad Miguel Hernández de Elche, Elche, Spain
| | - Eduardo Fernández
- Institute of Bioengineering, Universidad Miguel Hernández de Elche, Elche, Spain,Biomedical Research Networking Center in Bioengineering, Biomaterials and Nanomedicine (CIBER-BBN), Madrid, Spain,*Correspondence: Eduardo Fernández,
| |
Collapse
|
15
|
Beyeler M, Sanchez-Garcia M. Towards a Smart Bionic Eye: AI-powered artificial vision for the treatment of incurable blindness. J Neural Eng 2022; 19:10.1088/1741-2552/aca69d. [PMID: 36541463 PMCID: PMC10507809 DOI: 10.1088/1741-2552/aca69d] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2022] [Accepted: 11/28/2022] [Indexed: 11/30/2022]
Abstract
Objective.How can we return a functional form of sight to people who are living with incurable blindness? Despite recent advances in the development of visual neuroprostheses, the quality of current prosthetic vision is still rudimentary and does not differ much across different device technologies.Approach.Rather than aiming to represent the visual scene as naturally as possible, aSmart Bionic Eyecould provide visual augmentations through the means of artificial intelligence-based scene understanding, tailored to specific real-world tasks that are known to affect the quality of life of people who are blind, such as face recognition, outdoor navigation, and self-care.Main results.Complementary to existing research aiming to restore natural vision, we propose a patient-centered approach to incorporate deep learning-based visual augmentations into the next generation of devices.Significance.The ability of a visual prosthesis to support everyday tasks might make the difference between abandoned technology and a widely adopted next-generation neuroprosthetic device.
Collapse
Affiliation(s)
- Michael Beyeler
- Department of Computer Science,University of California,Santa Barbara, CA, United States of America
- Department of Psychological & Brain Sciences, University of California, Santa Barbara, CA, United States of America
| | - Melani Sanchez-Garcia
- Department of Computer Science,University of California,Santa Barbara, CA, United States of America
| |
Collapse
|
16
|
Granley J, Relic L, Beyeler M. Hybrid Neural Autoencoders for Stimulus Encoding in Visual and Other Sensory Neuroprostheses. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 2022; 35:22671-22685. [PMID: 37719469 PMCID: PMC10504858] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Subscribe] [Scholar Register] [Indexed: 09/19/2023]
Abstract
Sensory neuroprostheses are emerging as a promising technology to restore lost sensory function or augment human capabilities. However, sensations elicited by current devices often appear artificial and distorted. Although current models can predict the neural or perceptual response to an electrical stimulus, an optimal stimulation strategy solves the inverse problem: what is the required stimulus to produce a desired response? Here, we frame this as an end-to-end optimization problem, where a deep neural network stimulus encoder is trained to invert a known and fixed forward model that approximates the underlying biological system. As a proof of concept, we demonstrate the effectiveness of this hybrid neural autoencoder (HNA) in visual neuroprostheses. We find that HNA produces high-fidelity patient-specific stimuli representing handwritten digits and segmented images of everyday objects, and significantly outperforms conventional encoding strategies across all simulated patients. Overall this is an important step towards the long-standing challenge of restoring high-quality vision to people living with incurable blindness and may prove a promising solution for a variety of neuroprosthetic technologies.
Collapse
Affiliation(s)
- Jacob Granley
- Department of Computer Science, University of California, Santa Barbara
| | - Lucas Relic
- Department of Computer Science, University of California, Santa Barbara
| | - Michael Beyeler
- Department of Computer Science, University of California, Santa Barbara; Department of Psychological & Brain Sciences, University of California, Santa Barbara
| |
Collapse
|
17
|
Dong M, Coleman HA, Tonta MA, Xiong Z, Li D, Thomas S, Liu M, Fallon JB, Parkington HC, Forsythe JS. Rapid electrophoretic deposition of biocompatible graphene coatings for high-performance recording neural electrodes. NANOSCALE 2022; 14:15845-15858. [PMID: 36259692 DOI: 10.1039/d2nr04421h] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
The electrical and biological interfacial properties of invasive electrodes have a significant impact on the performance and longevity of neural recordings in the brain. In this study, we demonstrated rapid electrophoretic deposition and electrochemical reduction of graphene oxide (GO) on metal-based neural electrodes. Scanning electron microscopy (SEM), X-ray photoelectron spectroscopy (XPS) and other characterizations confirmed the existence of a uniform and effectively reduced graphene oxide coating. Electrochemically reduced graphene oxide (ErGO) coated Pt/Ir neural electrodes exhibited 15.2-fold increase in charge storage capacity (CSC) and 90% decrease in impedance with only 3.8% increase in electrode diameter. Patch clamp electrophysiology and calcium imaging of primary rat hippocampus neurons cultured on ErGO demonstrated that there was no adverse impact on the functional development of neurons. Immunostaining showed a balanced growth of excitatory and inhibitory neurons, and astrocytes. Acute recordings from the auditory cortex and chronic recordings (19 days) from the somatosensory cortex found ErGO coating improved the performance of neural electrodes in signal-to-noise ratio (SNR) and amplitude of signals. The proposed approach not only provides an in-depth evaluation of the effect of ErGO coating on neural electrodes but also widens the coating methods of commercial neural electrodes.
Collapse
Affiliation(s)
- Miheng Dong
- Department of Materials Science and Engineering, Monash Institute of Medical Engineering, Monash University, Clayton, VIC 3800, Australia.
- Monash Suzhou Research Institute, Monash University, Suzhou SIP 250000, China
| | - Harold A Coleman
- Department of Physiology, Biomedicine Discovery Institute, Monash University, Clayton, VIC 3800, Australia
| | - Mary A Tonta
- Department of Physiology, Biomedicine Discovery Institute, Monash University, Clayton, VIC 3800, Australia
| | - Zhiyuan Xiong
- Department of Chemical Engineering, University of Melbourne, Parkville, Victoria, Australia
| | - Dan Li
- Department of Chemical Engineering, University of Melbourne, Parkville, Victoria, Australia
| | - Sebastian Thomas
- Department of Materials Science and Engineering, Monash Institute of Medical Engineering, Monash University, Clayton, VIC 3800, Australia.
| | - Minsu Liu
- Department of Materials Science and Engineering, Monash Institute of Medical Engineering, Monash University, Clayton, VIC 3800, Australia.
- Monash Suzhou Research Institute, Monash University, Suzhou SIP 250000, China
- Foshan (Southern China) Institute for New Materials, Foshan 528200, China
| | - James B Fallon
- The Bionics Institute, East Melbourne, Victoria 3002, Australia
- Medical Bionics Department, University of Melbourne, Parkville, Victoria 3010, Australia
| | - Helena C Parkington
- Department of Physiology, Biomedicine Discovery Institute, Monash University, Clayton, VIC 3800, Australia
| | - John S Forsythe
- Department of Materials Science and Engineering, Monash Institute of Medical Engineering, Monash University, Clayton, VIC 3800, Australia.
| |
Collapse
|
18
|
Pavlov VA, Tracey KJ. Bioelectronic medicine: Preclinical insights and clinical advances. Neuron 2022; 110:3627-3644. [PMID: 36174571 PMCID: PMC10155266 DOI: 10.1016/j.neuron.2022.09.003] [Citation(s) in RCA: 24] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/19/2022] [Revised: 07/28/2022] [Accepted: 09/02/2022] [Indexed: 11/17/2022]
Abstract
The nervous system maintains homeostasis and health. Homeostatic disruptions underlying the pathobiology of many diseases can be controlled by bioelectronic devices targeting CNS and peripheral neural circuits. New insights into the regulatory functions of the nervous system and technological developments in bioelectronics drive progress in the emerging field of bioelectronic medicine. Here, we provide an overview of key aspects of preclinical research, translation, and clinical advances in bioelectronic medicine.
Collapse
Affiliation(s)
- Valentin A Pavlov
- Institute of Bioelectronic Medicine, the Feinstein Institutes for Medical Research, Northwell Health, Manhasset, NY, USA; Elmezzi Graduate School of Molecular Medicine, Northwell Health, Manhasset, NY, USA; Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY, USA.
| | - Kevin J Tracey
- Institute of Bioelectronic Medicine, the Feinstein Institutes for Medical Research, Northwell Health, Manhasset, NY, USA; Elmezzi Graduate School of Molecular Medicine, Northwell Health, Manhasset, NY, USA; Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY, USA.
| |
Collapse
|
19
|
Wang J, Zhao R, Li P, Fang Z, Li Q, Han Y, Zhou R, Zhang Y. Clinical Progress and Optimization of Information Processing in Artificial Visual Prostheses. SENSORS (BASEL, SWITZERLAND) 2022; 22:6544. [PMID: 36081002 PMCID: PMC9460383 DOI: 10.3390/s22176544] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/26/2022] [Revised: 08/22/2022] [Accepted: 08/26/2022] [Indexed: 06/15/2023]
Abstract
Visual prostheses, used to assist in restoring functional vision to the visually impaired, convert captured external images into corresponding electrical stimulation patterns that are stimulated by implanted microelectrodes to induce phosphenes and eventually visual perception. Detecting and providing useful visual information to the prosthesis wearer under limited artificial vision has been an important concern in the field of visual prosthesis. Along with the development of prosthetic device design and stimulus encoding methods, researchers have explored the possibility of the application of computer vision by simulating visual perception under prosthetic vision. Effective image processing in computer vision is performed to optimize artificial visual information and improve the ability to restore various important visual functions in implant recipients, allowing them to better achieve their daily demands. This paper first reviews the recent clinical implantation of different types of visual prostheses, summarizes the artificial visual perception of implant recipients, and especially focuses on its irregularities, such as dropout and distorted phosphenes. Then, the important aspects of computer vision in the optimization of visual information processing are reviewed, and the possibilities and shortcomings of these solutions are discussed. Ultimately, the development direction and emphasis issues for improving the performance of visual prosthesis devices are summarized.
Collapse
Affiliation(s)
- Jing Wang
- School of Information, Shanghai Ocean University, Shanghai 201306, China
- Key Laboratory of Fishery Information, Ministry of Agriculture, Shanghai 200335, China
| | - Rongfeng Zhao
- School of Information, Shanghai Ocean University, Shanghai 201306, China
| | - Peitong Li
- School of Information, Shanghai Ocean University, Shanghai 201306, China
| | - Zhiqiang Fang
- School of Information, Shanghai Ocean University, Shanghai 201306, China
| | - Qianqian Li
- School of Information, Shanghai Ocean University, Shanghai 201306, China
| | - Yanling Han
- School of Information, Shanghai Ocean University, Shanghai 201306, China
| | - Ruyan Zhou
- School of Information, Shanghai Ocean University, Shanghai 201306, China
| | - Yun Zhang
- School of Information, Shanghai Ocean University, Shanghai 201306, China
| |
Collapse
|
20
|
Badadhe JD, Roh H, Lee BC, Kim JH, Im M. Ultrasound stimulation for non-invasive visual prostheses. Front Cell Neurosci 2022; 16:971148. [PMID: 35990889 PMCID: PMC9382087 DOI: 10.3389/fncel.2022.971148] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2022] [Accepted: 07/18/2022] [Indexed: 12/04/2022] Open
Abstract
Globally, it is estimated there are more than 2.2 billion visually impaired people. Visual diseases such as retinitis pigmentosa, age-related macular degeneration, glaucoma, and optic neuritis can cause irreversible profound vision loss. Many groups have investigated different approaches such as microelectronic prostheses, optogenetics, stem cell therapy, and gene therapy to restore vision. However, these methods have some limitations such as invasive implantation surgery and unknown long-term risk of genetic manipulation. In addition to the safety of ultrasound as a medical imaging modality, ultrasound stimulation can be a viable non-invasive alternative approach for the sight restoration because of its ability to non-invasively control neuronal activities. Indeed, recent studies have demonstrated ultrasound stimulation can successfully modulate retinal/brain neuronal activities without causing any damage to the nerve cells. Superior penetration depth and high spatial resolution of focused ultrasound can open a new avenue in neuromodulation researches. This review summarizes the latest research results about neural responses to ultrasound stimulation. Also, this work provides an overview of technical viewpoints in the future design of a miniaturized ultrasound transducer for a non-invasive acoustic visual prosthesis for non-surgical and painless restoration of vision.
Collapse
Affiliation(s)
- Jaya Dilip Badadhe
- Brain Science Institute, Korea Institute of Science and Technology (KIST), Seoul, South Korea
- Division of Bio-Medical Science & Technology, KIST School, University of Science and Technology (UST), Seoul, South Korea
| | - Hyeonhee Roh
- Brain Science Institute, Korea Institute of Science and Technology (KIST), Seoul, South Korea
- School of Electrical Engineering, College of Engineering, Korea University, Seoul, South Korea
| | - Byung Chul Lee
- Brain Science Institute, Korea Institute of Science and Technology (KIST), Seoul, South Korea
- Division of Bio-Medical Science & Technology, KIST School, University of Science and Technology (UST), Seoul, South Korea
- KHU-KIST Department of Converging Science and Technology, Kyung Hee University, Seoul, South Korea
| | - Jae Hun Kim
- Sensor System Research Center, Korea Institute of Science and Technology (KIST), Seoul, South Korea
| | - Maesoon Im
- Brain Science Institute, Korea Institute of Science and Technology (KIST), Seoul, South Korea
- Division of Bio-Medical Science & Technology, KIST School, University of Science and Technology (UST), Seoul, South Korea
| |
Collapse
|
21
|
Hoogsteen KM, Szpiro S, Kreiman G, Peli E. Beyond the Cane: Describing Urban Scenes to Blind People for Mobility Tasks. ACM TRANSACTIONS ON ACCESSIBLE COMPUTING 2022; 15. [DOI: 10.1145/3522757] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Abstract
Blind people face difficulties with independent mobility, impacting employment prospects, social inclusion, and quality of life. Given the advancements in computer vision, with more efficient and effective automated information extraction from visual scenes, it is important to determine what information is worth conveying to blind travelers, especially since people have a limited capacity to receive and process sensory information. We aimed to investigate which objects in a street scene are useful to describe and how those objects should be described. Thirteen cane-using participants, five of whom were early blind, took part in two urban walking experiments. In the first experiment, participants were asked to voice their information needs in the form of questions to the experimenter. In the second experiment, participants were asked to score scene descriptions and navigation instructions, provided by the experimenter, in terms of their usefulness. The descriptions included a variety of objects with various annotations per object. Additionally, we asked participants to rank order the objects and the different descriptions per object in terms of priority and explain why the provided information is or is not useful to them. The results reveal differences between early and late blind participants. Late blind participants requested information more frequently and prioritized information about objects’ locations. Our results illustrate how different factors, such as the level of detail, relative position, and what type of information is provided when describing an object, affected the usefulness of scene descriptions. Participants explained how they (indirectly) used information, but they were frequently unable to explain their ratings. The results distinguish between various types of travel information, underscore the importance of featuring these types at multiple levels of abstraction, and highlight gaps in current understanding of travel information needs. Elucidating the information needs of blind travelers is critical for the development of more useful assistive technologies.
Collapse
Affiliation(s)
- Karst M.P. Hoogsteen
- Schepens Eye Research Institute, Mass Eye and Ear, Department of Ophthalmology, Harvard Medical School, Boston, Massachusetts, United States of America
| | - Sarit Szpiro
- Department of Special Education, University of Haifa, Haifa, Israel
| | - Gabriel Kreiman
- Boston Children's Hospital, Harvard Medical School, Boston, Massachusetts, United States of America
- Center for Brains, Minds, and Machines, Cambridge, Massachusetts, United States of America
| | - Eli Peli
- Schepens Eye Research Institute, Mass Eye and Ear, Department of Ophthalmology, Harvard Medical School, Boston, Massachusetts, United States of America
| |
Collapse
|
22
|
Kasowski J, Beyeler M. Immersive Virtual Reality Simulations of Bionic Vision. AUGMENTED HUMANS 2022 2022; 2022:82-93. [PMID: 35856703 PMCID: PMC9289996 DOI: 10.1145/3519391.3522752] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/21/2023]
Abstract
Bionic vision uses neuroprostheses to restore useful vision to people living with incurable blindness. However, a major outstanding challenge is predicting what people "see" when they use their devices. The limited field of view of current devices necessitates head movements to scan the scene, which is difficult to simulate on a computer screen. In addition, many computational models of bionic vision lack biological realism. To address these challenges, we present VR-SPV, an open-source virtual reality toolbox for simulated prosthetic vision that uses a psychophysically validated computational model to allow sighted participants to "see through the eyes" of a bionic eye user. To demonstrate its utility, we systematically evaluated how clinically reported visual distortions affect performance in a letter recognition and an immersive obstacle avoidance task. Our results highlight the importance of using an appropriate phosphene model when predicting visual outcomes for bionic vision.
Collapse
|
23
|
de Ruyter van Steveninck J, Güçlü U, van Wezel R, van Gerven M. End-to-end optimization of prosthetic vision. J Vis 2022; 22:20. [PMID: 35703408 PMCID: PMC8899855 DOI: 10.1167/jov.22.2.20] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
Neural prosthetics may provide a promising solution to restore visual perception in some forms of blindness. The restored prosthetic percept is rudimentary compared to normal vision and can be optimized with a variety of image preprocessing techniques to maximize relevant information transfer. Extracting the most useful features from a visual scene is a nontrivial task and optimal preprocessing choices strongly depend on the context. Despite rapid advancements in deep learning, research currently faces a difficult challenge in finding a general and automated preprocessing strategy that can be tailored to specific tasks or user requirements. In this paper, we present a novel deep learning approach that explicitly addresses this issue by optimizing the entire process of phosphene generation in an end-to-end fashion. The proposed model is based on a deep auto-encoder architecture and includes a highly adjustable simulation module of prosthetic vision. In computational validation experiments, we show that such an approach is able to automatically find a task-specific stimulation protocol. The results of these proof-of-principle experiments illustrate the potential of end-to-end optimization for prosthetic vision. The presented approach is highly modular and our approach could be extended to automated dynamic optimization of prosthetic vision for everyday tasks, given any specific constraints, accommodating individual requirements of the end-user.
Collapse
Affiliation(s)
- Jaap de Ruyter van Steveninck
- Department of Artificial Intelligence, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands
- Department of Biophysics, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands
| | - Umut Güçlü
- Department of Artificial Intelligence, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands
| | - Richard van Wezel
- Department of Biophysics, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands
- Biomedical Signal and Systems, MIRA Institute for Biomedical Technology and Technical Medicine, University of Twente, Enschede, The Netherlands
| | - Marcel van Gerven
- Department of Artificial Intelligence, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands
| |
Collapse
|
24
|
Zibetti C. Deciphering the Retinal Epigenome during Development, Disease and Reprogramming: Advancements, Challenges and Perspectives. Cells 2022; 11:cells11050806. [PMID: 35269428 PMCID: PMC8908986 DOI: 10.3390/cells11050806] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/14/2022] [Revised: 02/15/2022] [Accepted: 02/18/2022] [Indexed: 02/01/2023] Open
Abstract
Retinal neurogenesis is driven by concerted actions of transcription factors, some of which are expressed in a continuum and across several cell subtypes throughout development. While seemingly redundant, many factors diversify their regulatory outcome on gene expression, by coordinating variations in chromatin landscapes to drive divergent retinal specification programs. Recent studies have furthered the understanding of the epigenetic contribution to the progression of age-related macular degeneration, a leading cause of blindness in the elderly. The knowledge of the epigenomic mechanisms that control the acquisition and stabilization of retinal cell fates and are evoked upon damage, holds the potential for the treatment of retinal degeneration. Herein, this review presents the state-of-the-art approaches to investigate the retinal epigenome during development, disease, and reprogramming. A pipeline is then reviewed to functionally interrogate the epigenetic and transcriptional networks underlying cell fate specification, relying on a truly unbiased screening of open chromatin states. The related work proposes an inferential model to identify gene regulatory networks, features the first footprinting analysis and the first tentative, systematic query of candidate pioneer factors in the retina ever conducted in any model organism, leading to the identification of previously uncharacterized master regulators of retinal cell identity, such as the nuclear factor I, NFI. This pipeline is virtually applicable to the study of genetic programs and candidate pioneer factors in any developmental context. Finally, challenges and limitations intrinsic to the current next-generation sequencing techniques are discussed, as well as recent advances in super-resolution imaging, enabling spatio-temporal resolution of the genome.
Collapse
Affiliation(s)
- Cristina Zibetti
- Department of Ophthalmology, Institute of Clinical Medicine, University of Oslo, Kirkeveien 166, Building 36, 0455 Oslo, Norway
| |
Collapse
|
25
|
Ptito M, Bleau M, Bouskila J. The Retina: A Window into the Brain. Cells 2021; 10:cells10123269. [PMID: 34943777 PMCID: PMC8699497 DOI: 10.3390/cells10123269] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2021] [Accepted: 11/18/2021] [Indexed: 12/18/2022] Open
Affiliation(s)
- Maurice Ptito
- School of Optometry, University of Montreal, Montreal, QC H3T 1P1, Canada; (M.B.); (J.B.)
- Department of Neuroscience, Copenhagen University, 2200 Copenhagen, Denmark
- Department of Neurology and Neurosurgery, Montreal Neurological Institute, McGill University, Montreal, QC H3A 2B4, Canada
- Correspondence:
| | - Maxime Bleau
- School of Optometry, University of Montreal, Montreal, QC H3T 1P1, Canada; (M.B.); (J.B.)
| | - Joseph Bouskila
- School of Optometry, University of Montreal, Montreal, QC H3T 1P1, Canada; (M.B.); (J.B.)
| |
Collapse
|
26
|
Papadopoulos N, Melanitis N, Lozano A, Soto-Sanchez C, Fernandez E, Nikita KS. Machine Learning Method for Functional Assessment of Retinal Models. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:4293-4296. [PMID: 34892171 DOI: 10.1109/embc46164.2021.9629599] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Challenges in the field of retinal prostheses motivate the development of retinal models to accurately simulate Retinal Ganglion Cells (RGCs) responses. The goal of retinal prostheses is to enable blind individuals to solve complex, reallife visual tasks. In this paper, we introduce the functional assessment (FA) of retinal models, which describes the concept of evaluating the performance of retinal models on visual understanding tasks. We present a machine learning method for FA: we feed traditional machine learning classifiers with RGC responses generated by retinal models, to solve object and digit recognition tasks (CIFAR-10, MNIST, Fashion MNIST, Imagenette). We examined critical FA aspects, including how the performance of FA depends on the task, how to optimally feed RGC responses to the classifiers and how the number of output neurons correlates with the model's accuracy. To increase the number of output neurons, we manipulated input images - by splitting and then feeding them to the retinal model and we found that image splitting does not significantly improve the model's accuracy. We also show that differences in the structure of datasets result in largely divergent performance of the retinal model (MNIST and Fashion MNIST exceeded 80% accuracy, while CIFAR-10 and Imagenette achieved ∼40%). Furthermore, retinal models which perform better in standard evaluation, i.e. more accurately predict RGC response, perform better in FA as well. However, unlike standard evaluation, FA results can be straightforwardly interpreted in the context of comparing the quality of visual perception.
Collapse
|
27
|
Melanitis N, Nakopoulos G, Lozano A, Soto-Sanchez C, Fernandez E, Nikita KS. Using Biologically-inspired Image Features to Model Retinal Response: Evidence from Biological Datasets. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:3378-3381. [PMID: 34891964 DOI: 10.1109/embc46164.2021.9629869] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Retinal models are needed to simulate the translation of visual percepts to Retinal Ganglion Cells (RGCs) neural spike trains, through which visual information is transmitted to the brain. Restoring vision through neural prostheses motivates the development of accurate retinal models. We integrate biologically-inspired image features to RGC models. We trained Linear-Nonlinear models using response data from biological retinae. We show that augmenting raw image input with retina-inspired image features leads to performance improvements: in a smaller (30sec. of retina recordings) set integration of features leads to improved models in approximately $\frac{2}{3}$ of the modeled RGCS; in a larger (4min. recording) we show that utilizing Spike Triggered Average analysis to localize RGCs in input images and extract features in a cell-based manner leads to improved models in all (except two) of the modeled RGCs.
Collapse
|
28
|
Ptito M, Paré S, Dricot L, Cavaliere C, Tomaiuolo F, Kupers R. A quantitative analysis of the retinofugal projections in congenital and late-onset blindness. NEUROIMAGE-CLINICAL 2021; 32:102809. [PMID: 34509923 PMCID: PMC8435915 DOI: 10.1016/j.nicl.2021.102809] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/23/2021] [Revised: 07/14/2021] [Accepted: 08/24/2021] [Indexed: 01/22/2023]
Abstract
Congenital (CB) and late blind (LB) affects the integrity brain visual structures. We measured the integrity of the retino-fugal system using structural MRI images. Optic nerve, optic tract, optic chiasm and LGN were reduced by 50 to 60% in CB and LB. There were no differences between CB and LB. In LB, optic nerve volume correlated negatively with blindness duration.
Vision loss early in life has dramatic consequences on the organization of the visual system and hence on structural plasticity of its remnant components. Most of the studies on the anatomical changes in the brain following visual deprivation have focused on the re-organization of the visual cortex and its afferent and efferent projections. In this study, we performed a quantitative analysis of the volume and size of the optic chiasm, optic nerve, optic tract and the lateral geniculate nucleus (LGN), the retino recipient thalamic nucleus. Analysis was carried out on structural T1-weighted MRIs from 22 congenitally blind (CB), 14 late blind (LB) and 29 age -and sex-matched sighted control (SC) subjects. We manually segmented the optic nerve, optic chiasm and optic tract, while LGN volumes were extracted using in-house software. We also measured voxel intensity of optic nerve, optic chiasm and optic tract. Mean volumes of the optic nerve, optic tract and optic chiasm were reduced by 50 to 60% in both CB and LB participants. No significant differences were found between the congenitally and late-onset blind participants for any of the measures. Our data further revealed reduced white matter voxel intensities in optic nerve, optic chiasm and optic tract in blind compared to sighted participants, suggesting decreased myelin content in the atrophied white matter. The LGN was reduced by 50% and 44% in CB and LB, respectively. In LB, optic nerve volume correlated negatively with the blindness duration index; no such correlation was found for optic chiasm, optic tract and LGN. The observation that despite the absence of visual input about half of the subcortical retinofugal projections are structurally preserved raises the question of their functional role. One possibility is that the surviving fibers play a role in the maintenance of circadian rhythms in the blind through the intrinsically photosensitive melanopsin-containing retinal ganglion cells.
Collapse
Affiliation(s)
- Maurice Ptito
- School of Optometry, University of Montreal, Montreal, QC, Canada; BRAINlab, University of Copenhagen, Copenhagen, Denmark; Danish Research Center for Magnetic Resonance (DRCMR), Copenhagen University Hospital, Hvidovre, Denmark
| | - Samuel Paré
- School of Optometry, University of Montreal, Montreal, QC, Canada
| | - Laurence Dricot
- Institute of NeuroScience (IoNS), Université catholique de Louvain (UCLouvain), Belgium
| | - Carlo Cavaliere
- IRCCS SDN, Naples, Italy; Coma Science Group, Cyclotron Research Center and Neurology Department, University and University Hospital of Liège, Liège, Belgium
| | - Francesco Tomaiuolo
- Univesità degli Studi di Messina, Dipartimento di Medicina Clinica e Sperimentale
| | - Ron Kupers
- School of Optometry, University of Montreal, Montreal, QC, Canada; BRAINlab, University of Copenhagen, Copenhagen, Denmark; Institute of NeuroScience (IoNS), Université catholique de Louvain (UCLouvain), Belgium.
| |
Collapse
|
29
|
Park Y, Park SY, Eom K. Current Review of Optical Neural Interfaces for Clinical Applications. MICROMACHINES 2021; 12:925. [PMID: 34442547 PMCID: PMC8400671 DOI: 10.3390/mi12080925] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/15/2021] [Revised: 07/20/2021] [Accepted: 07/29/2021] [Indexed: 11/16/2022]
Abstract
Neural interfaces, which enable the recording and stimulation of living neurons, have emerged as valuable tools in understanding the brain in health and disease, as well as serving as neural prostheses. While neural interfaces are typically based on electrical transduction, alternative energy modalities have been explored to create safe and effective approaches. Among these approaches, optical methods of linking neurons to the outside world have gained attention because light offers high spatial selectivity and decreased invasiveness. Here, we review the current state-of-art of optical neural interfaces and their clinical applications. Optical neural interfaces can be categorized into optical control and optical readout, each of which can be divided into intrinsic and extrinsic approaches. We discuss the advantages and disadvantages of each of these methods and offer a comparison of relative performance. Future directions, including their clinical opportunities, are discussed with regard to the optical properties of biological tissue.
Collapse
Affiliation(s)
| | - Sung-Yun Park
- Department of Electronics Engineering, College of Engineering, Pusan National University, Busan 46241, Korea;
| | - Kyungsik Eom
- Department of Electronics Engineering, College of Engineering, Pusan National University, Busan 46241, Korea;
| |
Collapse
|
30
|
Zhang Z, Tian G, Duan X, Chen HL, Kim Richie DH. Nanostructured PEDOT Coatings for Electrode-Neuron Integration. ACS APPLIED BIO MATERIALS 2021; 4:5556-5565. [PMID: 35006733 DOI: 10.1021/acsabm.1c00375] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
Neural electrodes have been developed for the diagnosis and treatment of stroke, sensory deficits, and neurological disorders based on the electrical stimulation of nerve tissue and recording of neural electrical activity. A low interface impedance and large active surface area for charge transfer and intimate contact between neurons and the electrode are critical to obtain high-quality neural signal and effective stimulation without causing damage to both tissue and electrode. In this study, a nanostructured poly(3,4-ethylenedioxythiophene) (PEDOT) coating with lots of long protrusions was created via a one-step electrochemical polymerization from a dichloromethane solution without any rigid or soft templates. The nanostructures on the PEDOT coating were basically formed by intertwined PEDOT nanofibers, which further enhanced the active surface area. The fuzzy PEDOT-modified microelectrodes exhibited an impedance as low as 1 kΩ at 1 kHz, which is much lower than those produced from aqueous 3,4-ethylenedioxythiophene (EDOT) solution, and it was comparable with PEDOT films or composites created from/with template materials. Also, more than 150 times larger charge storage capacity density was obtained compared to the unmodified microelectrode. An in vitro biocompatibility test performed on PC12 cells and primary cells suggested that the PEDOT coatings support cell adhesion, growth, and neurite extension. These results suggest the great potential of the nanostructured PEDOT coating as an electroactive and biosafe intimate contact between the implanted neural electrode and neurons.
Collapse
Affiliation(s)
- Ziyi Zhang
- School of Animal Science and Technology, Guangxi University, Nanning 530004, Guangxi, People's Republic of China
| | - Guangzhao Tian
- School of Animal Science and Technology, Guangxi University, Nanning 530004, Guangxi, People's Republic of China
| | - Xiaoge Duan
- School of Animal Science and Technology, Guangxi University, Nanning 530004, Guangxi, People's Republic of China
| | - Hai-Lan Chen
- School of Animal Science and Technology, Guangxi University, Nanning 530004, Guangxi, People's Republic of China
| | - Dong-Hwan Kim Richie
- School of Chemical Engineering, Sungkyunkwan University (SKKU), Suwon 16419, Republic of Korea.,Biomedical Institute for Convergence at SKKU (BICS), Sungkyunkwan University (SKKU), Suwon 16419, Republic of Korea
| |
Collapse
|
31
|
Gebrehiwot AN, Kato T, Nakazawa K. Inducing lateralized phosphenes over the occipital lobe using transcranial magnetic stimulation to navigate a virtual environment. PLoS One 2021; 16:e0249996. [PMID: 33852643 PMCID: PMC8046218 DOI: 10.1371/journal.pone.0249996] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2020] [Accepted: 03/29/2021] [Indexed: 11/19/2022] Open
Abstract
Electrical stimulation involving visual areas of the brain produces artificial light percepts called phosphenes. These visual percepts have been extensively investigated in previous studies involving intracortical microsimulation (ICMS) and serve as the basis for developing a visual prosthesis for the blind. Although advances have been achieved, many challenges still remain with implementing a functional ICMS for visual rehabilitation purposes. Transcranial magnetic stimulation (TMS) over the primary occipital lobe offers an alternative method to produce phosphenes non-invasively. A main challenge facing blind individuals involves navigation. Within the scientific community, methods to evaluate the ability of a visual prosthesis to facilitate in navigation has been neglected. In this study, we investigate the effectiveness of evoking lateralized phosphenes to navigate a computer simulated virtual environment. More importantly, we demonstrate how virtual environments along with the development of a visual prosthesis share a mutual relationship benefiting both patients and researchers. Using two TMS devices, a pair of 40mm figure-of-eight coils were placed over each occipital hemisphere resulting in lateralized phosphene perception. Participants were tasked with making a series of left and right turns using peripheral devices depending on the visual hemifield in which a phosphene is present. If a participant was able to accurately perceive all ten phosphenes, the simulated target is able to advance and fully exit the virtual environment. Our findings demonstrate that participants can interpret lateralized phosphenes while highlighting the integration of computer based virtual environments to evaluate the capability of a visual prosthesis during navigation.
Collapse
Affiliation(s)
| | - Tatsuya Kato
- Department of Life Sciences, The University of Tokyo, Tokyo, Japan
- Japan Society for the Promotion of Science, Tokyo, Japan
| | - Kimitaka Nakazawa
- Department of Life Sciences, The University of Tokyo, Tokyo, Japan
- * E-mail:
| |
Collapse
|
32
|
Ptito M, Bleau M, Djerourou I, Paré S, Schneider FC, Chebat DR. Brain-Machine Interfaces to Assist the Blind. Front Hum Neurosci 2021; 15:638887. [PMID: 33633557 PMCID: PMC7901898 DOI: 10.3389/fnhum.2021.638887] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2020] [Accepted: 01/19/2021] [Indexed: 12/31/2022] Open
Abstract
The loss or absence of vision is probably one of the most incapacitating events that can befall a human being. The importance of vision for humans is also reflected in brain anatomy as approximately one third of the human brain is devoted to vision. It is therefore unsurprising that throughout history many attempts have been undertaken to develop devices aiming at substituting for a missing visual capacity. In this review, we present two concepts that have been prevalent over the last two decades. The first concept is sensory substitution, which refers to the use of another sensory modality to perform a task that is normally primarily sub-served by the lost sense. The second concept is cross-modal plasticity, which occurs when loss of input in one sensory modality leads to reorganization in brain representation of other sensory modalities. Both phenomena are training-dependent. We also briefly describe the history of blindness from ancient times to modernity, and then proceed to address the means that have been used to help blind individuals, with an emphasis on modern technologies, invasive (various type of surgical implants) and non-invasive devices. With the advent of brain imaging, it has become possible to peer into the neural substrates of sensory substitution and highlight the magnitude of the plastic processes that lead to a rewired brain. Finally, we will address the important question of the value and practicality of the available technologies and future directions.
Collapse
Affiliation(s)
- Maurice Ptito
- École d’Optométrie, Université de Montréal, Montréal, QC, Canada
- Department of Nuclear Medicine, University of Southern Denmark, Odense, Denmark
- Department of Neuroscience, University of Copenhagen, Copenhagen, Denmark
| | - Maxime Bleau
- École d’Optométrie, Université de Montréal, Montréal, QC, Canada
| | - Ismaël Djerourou
- École d’Optométrie, Université de Montréal, Montréal, QC, Canada
| | - Samuel Paré
- École d’Optométrie, Université de Montréal, Montréal, QC, Canada
| | - Fabien C. Schneider
- TAPE EA7423 University of Lyon-Saint Etienne, Saint Etienne, France
- Neuroradiology Unit, University Hospital of Saint-Etienne, Saint-Etienne, France
| | - Daniel-Robert Chebat
- Visual and Cognitive Neuroscience Laboratory (VCN Lab), Department of Psychology, Faculty of Social Sciences and Humanities, Ariel University, Ariel, Israël
- Navigation and Accessibility Research Center of Ariel University (NARCA), Ariel, Israël
| |
Collapse
|
33
|
Fernández E, Alfaro A, González-López P. Toward Long-Term Communication With the Brain in the Blind by Intracortical Stimulation: Challenges and Future Prospects. Front Neurosci 2020; 14:681. [PMID: 32848535 PMCID: PMC7431631 DOI: 10.3389/fnins.2020.00681] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2019] [Accepted: 06/03/2020] [Indexed: 11/15/2022] Open
Abstract
The restoration of a useful visual sense in a profoundly blind person by direct electrical stimulation of the visual cortex has been a subject of study for many years. However, the field of cortically based sight restoration has made few advances in the last few decades, and many problems remain. In this context, the scientific and technological problems associated with safe and effective communication with the brain are very complex, and there are still many unresolved issues delaying its development. In this work, we review some of the biological and technical issues that still remain to be solved, including long-term biotolerability, the number of electrodes required to provide useful vision, and the delivery of information to the implants. Furthermore, we emphasize the possible role of the neuroplastic changes that follow vision loss in the success of this approach. We propose that increased collaborations among clinicians, basic researchers, and neural engineers will enhance our ability to send meaningful information to the brain and restore a limited but useful sense of vision to many blind individuals.
Collapse
Affiliation(s)
- Eduardo Fernández
- Institute of Bioengineering, Universidad Miguel Hernández, Elche, Spain
- Center for Biomedical Research in the Network in Bioengineering, Biomaterials and Nanomedicine (CIBER-BBN), Madrid, Spain
- John A. Moran Eye Center, University of Utah, Salt Lake City, UT, United States
| | - Arantxa Alfaro
- Center for Biomedical Research in the Network in Bioengineering, Biomaterials and Nanomedicine (CIBER-BBN), Madrid, Spain
- Hospital Vega Baja, Orihuela, Spain
| | - Pablo González-López
- Institute of Bioengineering, Universidad Miguel Hernández, Elche, Spain
- Hospital General Universitario de Alicante, Alicante, Spain
| |
Collapse
|
34
|
Lozano A, Suárez JS, Soto-Sánchez C, Garrigós J, Martínez-Alvarez JJ, Ferrández JM, Fernández E. Neurolight: A Deep Learning Neural Interface for Cortical Visual Prostheses. Int J Neural Syst 2020; 30:2050045. [DOI: 10.1142/s0129065720500458] [Citation(s) in RCA: 23] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Visual neuroprosthesis, that provide electrical stimulation along several sites of the human visual system, constitute a potential tool for vision restoration for the blind. Scientific and technological progress in the fields of neural engineering and artificial vision comes with new theories and tools that, along with the dawn of modern artificial intelligence, constitute a promising framework for the further development of neurotechnology. In the framework of the development of a Cortical Visual Neuroprosthesis for the blind (CORTIVIS), we are now facing the challenge of developing not only computationally powerful tools and flexible approaches that will allow us to provide some degree of functional vision to individuals who are profoundly blind. In this work, we propose a general neuroprosthesis framework composed of several task-oriented and visual encoding modules. We address the development and implementation of computational models of the firing rates of retinal ganglion cells and design a tool — Neurolight — that allows these models to be interfaced with intracortical microelectrodes in order to create electrical stimulation patterns that can evoke useful perceptions. In addition, the developed framework allows the deployment of a diverse array of state-of-the-art deep-learning techniques for task-oriented and general image pre-processing, such as semantic segmentation and object detection in our system’s pipeline. To the best of our knowledge, this constitutes the first deep-learning-based system designed to directly interface with the visual brain through an intracortical microelectrode array. We implement the complete pipeline, from obtaining a video stream to developing and deploying task-oriented deep-learning models and predictive models of retinal ganglion cells’ encoding of visual inputs under the control of a neurostimulation device able to send electrical train pulses to a microelectrode array implanted at the visual cortex.
Collapse
Affiliation(s)
- Antonio Lozano
- Departamento de Electrónica, Tecnología de Computadoras y Proyectos, Universidad Politécnica de Cartagena, 30202 Cartagena, Spain
| | - Juan Sebastián Suárez
- Instituto de Bioingeniería, Universidad Miguel Hernández, 03202 Alicante, Spain
- CIBER-BBN, 28029 Madrid, Spain
| | - Cristina Soto-Sánchez
- Instituto de Bioingeniería, Universidad Miguel Hernández, 03202 Alicante, Spain
- CIBER-BBN, 28029 Madrid, Spain
| | - Javier Garrigós
- Departamento de Electrónica, Tecnología de Computadoras y Proyectos, Universidad Politécnica de Cartagena, 30202 Cartagena, Spain
| | - J. Javier Martínez-Alvarez
- Departamento de Electrónica, Tecnología de Computadoras y Proyectos, Universidad Politécnica de Cartagena, 30202 Cartagena, Spain
| | - J. Manuel Ferrández
- Departamento de Electrónica, Tecnología de Computadoras y Proyectos, Universidad Politécnica de Cartagena, 30202 Cartagena, Spain
| | - Eduardo Fernández
- Instituto de Bioingeniería, Universidad Miguel Hernández, 03202 Alicante, Spain
| |
Collapse
|
35
|
Mikhaylov A, Pimashkin A, Pigareva Y, Gerasimova S, Gryaznov E, Shchanikov S, Zuev A, Talanov M, Lavrov I, Demin V, Erokhin V, Lobov S, Mukhina I, Kazantsev V, Wu H, Spagnolo B. Neurohybrid Memristive CMOS-Integrated Systems for Biosensors and Neuroprosthetics. Front Neurosci 2020; 14:358. [PMID: 32410943 PMCID: PMC7199501 DOI: 10.3389/fnins.2020.00358] [Citation(s) in RCA: 110] [Impact Index Per Article: 27.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2019] [Accepted: 03/24/2020] [Indexed: 11/18/2022] Open
Abstract
Here we provide a perspective concept of neurohybrid memristive chip based on the combination of living neural networks cultivated in microfluidic/microelectrode system, metal-oxide memristive devices or arrays integrated with mixed-signal CMOS layer to control the analog memristive circuits, process the decoded information, and arrange a feedback stimulation of biological culture as parts of a bidirectional neurointerface. Our main focus is on the state-of-the-art approaches for cultivation and spatial ordering of the network of dissociated hippocampal neuron cells, fabrication of a large-scale cross-bar array of memristive devices tailored using device engineering, resistive state programming, or non-linear dynamics, as well as hardware implementation of spiking neural networks (SNNs) based on the arrays of memristive devices and integrated CMOS electronics. The concept represents an example of a brain-on-chip system belonging to a more general class of memristive neurohybrid systems for a new-generation robotics, artificial intelligence, and personalized medicine, discussed in the framework of the proposed roadmap for the next decade period.
Collapse
Affiliation(s)
- Alexey Mikhaylov
- Lobachevsky State University of Nizhny Novgorod, Nizhny Novgorod, Russia
| | - Alexey Pimashkin
- Lobachevsky State University of Nizhny Novgorod, Nizhny Novgorod, Russia
| | - Yana Pigareva
- Lobachevsky State University of Nizhny Novgorod, Nizhny Novgorod, Russia
| | | | - Evgeny Gryaznov
- Lobachevsky State University of Nizhny Novgorod, Nizhny Novgorod, Russia
| | - Sergey Shchanikov
- Department of Information Technologies, Vladimir State University, Murom, Russia
| | - Anton Zuev
- Department of Information Technologies, Vladimir State University, Murom, Russia
| | - Max Talanov
- Neuroscience Laboratory, Kazan Federal University, Kazan, Russia
| | - Igor Lavrov
- Department of Neurologic Surgery, Mayo Clinic, Rochester, MN, United States
- Laboratory of Motor Neurorehabilitation, Kazan Federal University, Kazan, Russia
| | | | - Victor Erokhin
- Neuroscience Laboratory, Kazan Federal University, Kazan, Russia
- Kurchatov Institute, Moscow, Russia
- CNR-Institute of Materials for Electronics and Magnetism, Italian National Research Council, Parma, Italy
| | - Sergey Lobov
- Lobachevsky State University of Nizhny Novgorod, Nizhny Novgorod, Russia
- Center for Technologies in Robotics and Mechatronics Components, Innopolis University, Innopolis, Russia
| | - Irina Mukhina
- Lobachevsky State University of Nizhny Novgorod, Nizhny Novgorod, Russia
- Cell Technology Group, Privolzhsky Research Medical University, Nizhny Novgorod, Russia
| | - Victor Kazantsev
- Lobachevsky State University of Nizhny Novgorod, Nizhny Novgorod, Russia
- Center for Technologies in Robotics and Mechatronics Components, Innopolis University, Innopolis, Russia
| | - Huaqiang Wu
- Institute of Microelectronics, Tsinghua University, Beijing, China
| | - Bernardo Spagnolo
- Lobachevsky State University of Nizhny Novgorod, Nizhny Novgorod, Russia
- Dipartimento di Fisica e Chimica-Emilio Segrè, Group of Interdisciplinary Theoretical Physics, Università di Palermo and CNISM, Unità di Palermo, Palermo, Italy
- Istituto Nazionale di Fisica Nucleare, Sezione di Catania, Catania, Italy
| |
Collapse
|
36
|
Yang JW, Yu ZY, Cheng SJ, Chung JHY, Liu X, Wu CY, Lin SF, Chen GY. Graphene Oxide-Based Nanomaterials: An Insight into Retinal Prosthesis. Int J Mol Sci 2020; 21:E2957. [PMID: 32331417 PMCID: PMC7216005 DOI: 10.3390/ijms21082957] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2020] [Revised: 04/17/2020] [Accepted: 04/20/2020] [Indexed: 12/27/2022] Open
Abstract
Retinal prosthesis has recently emerged as a treatment strategy for retinopathies, providing excellent assistance in the treatment of age-related macular degeneration (AMD) and retinitis pigmentosa. The potential application of graphene oxide (GO), a highly biocompatible nanomaterial with superior physicochemical properties, in the fabrication of electrodes for retinal prosthesis, is reviewed in this article. This review integrates insights from biological medicine and nanotechnology, with electronic and electrical engineering technological breakthroughs, and aims to highlight innovative objectives in developing biomedical applications of retinal prosthesis.
Collapse
Affiliation(s)
- Jia-Wei Yang
- Department of Electrical and Computer Engineering, College of Electrical and Computer Engineering, National Chiao Tung University, Hsinchu 300, Taiwan; (J.-W.Y.); (S.-J.C.); (S.-F.L.)
- Institute of Biomedical Engineering, College of Electrical and Computer Engineering, National Chiao Tung University, Hsinchu 300, Taiwan;
| | - Zih-Yu Yu
- Institute of Biomedical Engineering, College of Electrical and Computer Engineering, National Chiao Tung University, Hsinchu 300, Taiwan;
| | - Sheng-Jen Cheng
- Department of Electrical and Computer Engineering, College of Electrical and Computer Engineering, National Chiao Tung University, Hsinchu 300, Taiwan; (J.-W.Y.); (S.-J.C.); (S.-F.L.)
- Institute of Biomedical Engineering, College of Electrical and Computer Engineering, National Chiao Tung University, Hsinchu 300, Taiwan;
| | - Johnson H. Y. Chung
- ARC Centre of Excellence for Electromaterials Science, Intelligent Polymer Research Institute, University of Wollongong, Wollongong, NSW 2500, Australia; (J.H.Y.C.); (X.L.)
| | - Xiao Liu
- ARC Centre of Excellence for Electromaterials Science, Intelligent Polymer Research Institute, University of Wollongong, Wollongong, NSW 2500, Australia; (J.H.Y.C.); (X.L.)
| | - Chung-Yu Wu
- Department of Electrical Engineering, National Chiao Tung University, Hsinchu, 300, Taiwan;
| | - Shien-Fong Lin
- Department of Electrical and Computer Engineering, College of Electrical and Computer Engineering, National Chiao Tung University, Hsinchu 300, Taiwan; (J.-W.Y.); (S.-J.C.); (S.-F.L.)
- Institute of Biomedical Engineering, College of Electrical and Computer Engineering, National Chiao Tung University, Hsinchu 300, Taiwan;
| | - Guan-Yu Chen
- Department of Electrical and Computer Engineering, College of Electrical and Computer Engineering, National Chiao Tung University, Hsinchu 300, Taiwan; (J.-W.Y.); (S.-J.C.); (S.-F.L.)
- Institute of Biomedical Engineering, College of Electrical and Computer Engineering, National Chiao Tung University, Hsinchu 300, Taiwan;
- Department of Biological Science and Technology, National Chiao Tung University, Hsinchu 300, Taiwan
| |
Collapse
|
37
|
Pavlov VA, Chavan SS, Tracey KJ. Bioelectronic Medicine: From Preclinical Studies on the Inflammatory Reflex to New Approaches in Disease Diagnosis and Treatment. Cold Spring Harb Perspect Med 2020; 10:a034140. [PMID: 31138538 PMCID: PMC7050582 DOI: 10.1101/cshperspect.a034140] [Citation(s) in RCA: 43] [Impact Index Per Article: 10.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/11/2022]
Abstract
Bioelectronic medicine is an evolving field in which new insights into the regulatory role of the nervous system and new developments in bioelectronic technology result in novel approaches in disease diagnosis and treatment. Studies on the immunoregulatory function of the vagus nerve and the inflammatory reflex have a specific place in bioelectronic medicine. These studies recently led to clinical trials with bioelectronic vagus nerve stimulation in inflammatory diseases and other conditions. Here, we outline key findings from this preclinical and clinical research. We also point to other aspects and pillars of interdisciplinary research and technological developments in bioelectronic medicine.
Collapse
Affiliation(s)
- Valentin A Pavlov
- Center for Biomedical Science and Bioelectronic Medicine, The Feinstein Institute for Medical Research, Northwell Health, Manhasset, New York 11030
- Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York 11550
| | - Sangeeta S Chavan
- Center for Biomedical Science and Bioelectronic Medicine, The Feinstein Institute for Medical Research, Northwell Health, Manhasset, New York 11030
- Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York 11550
| | - Kevin J Tracey
- Center for Biomedical Science and Bioelectronic Medicine, The Feinstein Institute for Medical Research, Northwell Health, Manhasset, New York 11030
- Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York 11550
| |
Collapse
|
38
|
Crespo-Cano R, Cuenca-Asensi S, Fernández E, Martínez-Álvarez A. Metaheuristic Optimisation Algorithms for Tuning a Bioinspired Retinal Model. SENSORS (BASEL, SWITZERLAND) 2019; 19:E4834. [PMID: 31698827 PMCID: PMC6891458 DOI: 10.3390/s19224834] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/27/2019] [Revised: 10/31/2019] [Accepted: 11/03/2019] [Indexed: 11/22/2022]
Abstract
A significant challenge in neuroscience is understanding how visual information is encoded in the retina. Such knowledge is extremely important for the purpose of designing bioinspired sensors and artificial retinal systems that will, in so far as may be possible, be capable of mimicking vertebrate retinal behaviour. In this study, we report the tuning of a reliable computational bioinspired retinal model with various algorithms to improve the mimicry of the model. Its main contribution is two-fold. First, given the multi-objective nature of the problem, an automatic multi-objective optimisation strategy is proposed through the use of four biological-based metrics, which are used to adjust the retinal model for accurate prediction of retinal ganglion cell responses. Second, a subset of population-based search heuristics-genetic algorithms (SPEA2, NSGA-II and NSGA-III), particle swarm optimisation (PSO) and differential evolution (DE)-are explored to identify the best algorithm for fine-tuning the retinal model, by comparing performance across a hypervolume metric. Nonparametric statistical tests are used to perform a rigorous comparison between all the metaheuristics. The best results were achieved with the PSO algorithm on the basis of the largest hypervolume that was achieved, well-distributed elements and high numbers on the Pareto front.
Collapse
Affiliation(s)
- Rubén Crespo-Cano
- Department of Computer Technology, University of Alicante, 03690 Alicante, Spain; (R.C.-C.); (S.C.-A.)
| | - Sergio Cuenca-Asensi
- Department of Computer Technology, University of Alicante, 03690 Alicante, Spain; (R.C.-C.); (S.C.-A.)
| | - Eduardo Fernández
- Institute of Bioengineering, University Miguel Hernández and CIBER BBN, 03202 Elche (Alicante), Spain;
| | - Antonio Martínez-Álvarez
- Department of Computer Technology, University of Alicante, 03690 Alicante, Spain; (R.C.-C.); (S.C.-A.)
| |
Collapse
|
39
|
Pavlov VA, Tracey KJ. Bioelectronic medicine: updates, challenges and paths forward. Bioelectron Med 2019; 5:1. [PMID: 32232092 PMCID: PMC7098260 DOI: 10.1186/s42234-019-0018-y] [Citation(s) in RCA: 31] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2019] [Accepted: 01/11/2019] [Indexed: 12/30/2022] Open
Affiliation(s)
- Valentin A Pavlov
- 1Center for Biomedical Science and Center for Bioelectronic Medicine, The Feinstein Institute for Medical Research, Northwell Health System, Manhasset, NY USA
- Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY USA
| | - Kevin J Tracey
- 1Center for Biomedical Science and Center for Bioelectronic Medicine, The Feinstein Institute for Medical Research, Northwell Health System, Manhasset, NY USA
- Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY USA
| |
Collapse
|