A Survey of Brain-Inspired Intelligent Robots: Integration of Vision, Decision, Motion Control, and Musculoskeletal Systems

Utilising redundancy in musculoskeletal systems for adaptive stiffness and muscle failure compensation: a model-free inverse statics approach

Vertebrates possess a biomechanical structure with redundant muscles, enabling adaptability in uncertain and complex environments. Harnessing this inspiration, musculoskeletal systems offer advantages like variable stiffness and resilience to actuator failure and fatigue. Despite their potential, the complex structure presents modelling challenges that are difficult to explicitly formulate and control. This difficulty arises from the need for comprehensive knowledge of the musculoskeletal system, including details such as muscle arrangement, and fully accessible muscle and joint states. Whilst existing model-free methods do not need explicit formulations, they also underutilise the benefits of muscle redundancy. Consequently, they necessitate retraining in the event of muscle failure and require manual tuning of parameters to control joint stiffness limiting their applications under unknown payloads. Presented here is a model-free local inverse statics controller for musculoskeletal systems, employing a feedforward neural network trained on motor babbling data. Experiments with a musculoskeletal leg model showcase the controller's adaptability to complex structures, including mono and bi-articulate muscles. The controller can compensate for changes such as weight variations, muscle failures, and environmental interactions, retaining reasonable accuracy without the need for any additional retraining.

The Framework and Memristive Circuit Design for Multisensory Mutual Associative Memory Networks

In this work, we propose a multisensory mutual associative memory networks framework and memristive circuit to mimic the ability of the biological brain to make associations of information received simultaneously. The circuit inspired by neural mechanisms of associative memory cells mainly consists of three modules: 1) the storage neurons module, which encodes external multimodal information into the firing rate of spikes; 2) the synapse module, which uses the nonvolatility memristor to achieve weight adjustment and associative learning; and 3) the retrieval neuron module, which feeds the retrieval signal output from each sensory pathway to other sensory pathways, so that achieve mutual association and retrieval between multiple modalities. Different from other one-to-one or many-to-one unidirectional associative memory work, this circuit achieves bidirectional association from multiple modalities to multiple modalities. In addition, we simulate the acquisition, extinction, recovery, transmission, and consolidation properties of associative memory. The circuit is applied to cross-modal association of image and audio recognition results, and episodic memory is simulated, where multiple images in a scene are intramodal associated. With power and area analysis, the circuit is validated as hardware-friendly. Further research to extend this work into large-scale associative memory networks, combined with visual-auditory-tactile-gustatory sensory sensors, is promising for application in intelligent robotic platforms to facilitate the development of neuromorphic systems and brain-like intelligence.

Bridging Neuroscience and Robotics: Spiking Neural Networks in Action

Robots are becoming increasingly sophisticated in the execution of complex tasks. However, an area that requires development is the ability to act in dynamically changing environments. To advance this, developments have turned towards understanding the human brain and applying this to improve robotics. The present study used electroencephalogram (EEG) data recorded from 54 human participants whilst they performed a two-choice task. A build-up of motor activity starting around 400 ms before response onset, also known as the lateralized readiness potential (LRP), was observed. This indicates that actions are not simply binary processes but rather, response-preparation is gradual and occurs in a temporal window that can interact with the environment. In parallel, a robot arm executing a pick-and-place task was developed. The understanding from the EEG data and the robot arm were integrated into the final system, which included cell assemblies (CAs)-a simulated spiking neural network-to inform the robot to place the object left or right. Results showed that the neural data from the robot simulation were largely consistent with the human data. This neurorobotics study provides an example of how to integrate human brain recordings with simulated neural networks in order to drive a robot.

A New Compound-Limbs Paradigm: Integrating Upper-Limb Swing Improves Lower-Limb Stepping Intention Decoding From EEG

Brain-computer interface (BCI) systems based on spontaneous electroencephalography (EEG) hold the promise to implement human voluntary control of lower-extremity powered exoskeletons. However, current EEG-BCI paradigms do not consider the cooperation of upper and lower limbs during walking, which is inconsistent with natural human stepping patterns. To deal with this problem, this study proposed a stepping-matched human EEG-BCI paradigm that involved actions of both unilateral lower and contralateral upper limbs (also referred to as compound-limbs movement). Experiments were conducted in motor execution (ME) and motor imagery (MI) conditions to validate the feasibility. Common spatial pattern (CSP) proposed subject-specific CSP (SSCSP), and filter-bank CSP (FBCSP) methods were applied for feature extraction, respectively. The best average classification results based on SSCSP indicated that the accuracies of compound-limbs paradigms in ME and MI conditions achieved 89.02% ± 12.84% and 73.70% ± 12.47%, respectively. Although they were 2.03% and 5.68% lower than those of the single-upper-limb mode that does not match human stepping patterns, they were 24.30% and 11.02% higher than those of the single-lower-limb mode. These findings indicated that the proposed compound-limbs EEG-BCI paradigm is feasible for decoding human stepping intention and thus provides a potential way for natural human control of walking assistance devices.

Arc fault detection using artificial intelligence: Challenges and benefits

This systematic review aims to investigate recent developments in the area of arc fault detection. The rising demand for electricity and concomitant expansion of energy systems has resulted in a heightened risk of arc faults and the likelihood of related fires, presenting a matter of considerable concern. To address this challenge, this review focuses on the role of artificial intelligence (AI) in arc fault detection, with the objective of illuminating its advantages and identifying current limitations. Through a meticulous literature selection process, a total of 63 articles were included in the final analysis. The findings of this review suggest that AI plays a significant role in enhancing the accuracy and speed of detection and allowing for customization to specific types of faults in arc fault detection. Simultaneously, three major challenges were also identified, including missed and false detections, the restricted application of neural networks and the paucity of relevant data. In conclusion, AI has exhibited tremendous potential for transforming the field of arc fault detection and holds substantial promise for enhancing electrical safety.

A systematic review of image-guided, surgical robot-assisted percutaneous puncture: Challenges and benefits

Percutaneous puncture is a common medical procedure that involves accessing an internal organ or tissue through the skin. Image guidance and surgical robots have been increasingly used to assist with percutaneous procedures, but the challenges and benefits of these technologies have not been thoroughly explored. The aims of this systematic review are to furnish an overview of the challenges and benefits of image-guided, surgical robot-assisted percutaneous puncture and to provide evidence on this approach. We searched several electronic databases for studies on image-guided, surgical robot-assisted percutaneous punctures published between January 2018 and December 2022. The final analysis refers to 53 studies in total. The results of this review suggest that image guidance and surgical robots can improve the accuracy and precision of percutaneous procedures, decrease radiation exposure to patients and medical personnel and lower the risk of complications. However, there are many challenges related to the use of these technologies, such as the integration of the robot and operating room, immature robotic perception, and deviation of needle insertion. In conclusion, image-guided, surgical robot-assisted percutaneous puncture offers many potential benefits, but further research is needed to fully understand the challenges and optimize the utilization of these technologies in clinical practice.

Design of motor cable artificial muscle (MC-AM) with tendon sheath–pulley system (TSPS) for musculoskeletal robot

In an unstructured environment, the arm can perform complicated tasks with rapidity, flexibility, and robustness. It is difficult to configure multiple artificial muscles similar to an arm in the compact space of a robotic arm. When muscle tension is transferred, mechanisms like tendon-sheath/tendon-pulley may be installed in a compact space to develop musculoskeletal robots that are closer to the arm. However, handling variable frictional nonlinearity and elastic cable deformation is necessary for transmission stability. In this study, the modular artificial muscle system (MAMS), including motor cable artificial muscle and tendon sheath–pulley system (TSPS), that can be installed remotely and transmit muscle tension in narrow paths, is designed. The feed-forward multi-layer neural network (FF-MNN) approach is utilized to discuss the relationship between the measurable input tension of TSPS and the unmeasurable output tension and cable elongation. Subsequently, the lightweight musculoskeletal arm (LM-Arm) is built to verify the validity of MAMS. Through trials, the experiments of MAMS after friction compensating and the LM-Arm’s end-point 3D trajectory tracking are investigated. The results show that average errors of the active and passive muscles tension are 3.87 N and 3.51 N, respectively, under conditions of larger load and higher contraction velocity. The average muscle length error of trajectory tracking is 0.00078 m (0.72%). The suggested MAMS may successfully build a musculoskeletal robot that has similar flexibility and morphology to the arm. It can also be utilized to power various pieces of machinery, such as rescue robot, invasive surgical robots, dexterous hands, and wearable exoskeletons.

Recent advancements in digital health management using multi-modal signal monitoring

Healthcare is the method of keeping or enhancing physical and mental well-being with its aid of illness and injury prevention, diagnosis, and treatment. The majority of conventional healthcare practices involve manual management and upkeep of client demographic information, case histories, diagnoses, medications, invoicing, and drug stock upkeep, which can result in human errors that have an impact on clients. By linking all the essential parameter monitoring equipment through a network with a decision-support system, digital health management based on Internet of Things (IoT) eliminates human errors and aids the doctor in making more accurate and timely diagnoses. The term "Internet of Medical Things" (IoMT) refers to medical devices that have the ability to communicate data over a network without requiring human-to-human or human-to-computer interaction. Meanwhile, more effective monitoring gadgets have been made due to the technology advancements, and these devices can typically record a few physiological signals simultaneously, including the electrocardiogram (ECG) signal, the electroglottography (EGG) signal, the electroencephalogram (EEG) signal, and the electrooculogram (EOG) signal. Yet, there has not been much research on the connection between digital health management and multi-modal signal monitoring. To bridge the gap, this article reviews the latest advancements in digital health management using multi-modal signal monitoring. Specifically, three digital health processes, namely, lower-limb data collection, statistical analysis of lower-limb data, and lower-limb rehabilitation via digital health management, are covered in this article, with the aim to fully review the current application of digital health technology in lower-limb symptom recovery.

Actuation delay compensation of robots in semi-physical test

In general, the traditional spacecraft semi-physical docking tests include the evaluation of docking and separation performance. However, these tests often rely on "specific" equipment, such as specially designed actuators and fast-response hydraulic systems, to meet the stringent dynamic response requirements of semi-physical testing. In this paper, a novel docking test platform is designed based on a general-purpose industrial manipulator using 3-D force and 3-D torque sensors. Different from the traditional solution, this novel platform is well-assembled and cost-effective. Furthermore, an actuation delay compensation method is introduced to improve the performance. Finally, the proposed method is evaluated using simulations. The results show that the novel method is with promising performance in terms of actuation delay compensation.

A novel "five-in-one" comprehensive medical care framework for rehabilitation and nursing

With the evolution of society, the world has entered a moderate stage of aging. Not surprisingly, the aging problem in the world is getting more intense, resulting in the increasing demand for higher-quality and well-organized medical and elderly care services. To cope with that, many researchers have dedicated themselves to advancing the medical care system based on data or platforms. However, they have ignored the life cycle, health service and management and the inevitable shift of living scenarios for the elderly. Therefore, the study aims to improve health conditions and enhance senior citizens' life quality and happiness index. In this paper, we build a unified body for people in their old age, bridging the disconnection between medical care and elderly care and constructing the "five-in-one" comprehensive medical care framework. It should be mentioned that the system takes the human life cycle as its axis, relies on the supply side and supply chain management, integrates medicine, industry, literature and science as methods, and takes health service management as a requirement. Furthermore, a case study on upper limb rehabilitation is elaborated along the "five-in-one" comprehensive medical care framework to confirm the effectiveness of the novel system.

Immersive virtual reality application for intelligent manufacturing: Applications and art design

Intelligent manufacturing (IM), sometimes referred to as smart manufacturing (SM), is the use of real-time data analysis, machine learning, and artificial intelligence (AI) in the production process to achieve the aforementioned efficiencies. Human-machine interaction technology has recently been a hot issue in smart manufacturing. The unique interactivity of virtual reality (VR) innovations makes it possible to create a virtual world and allow users to communicate with that environment, providing users with an interface to be immersed in the digital world of the smart factory. And virtual reality technology aims to stimulate the imagination and creativity of creators to the maximum extent possible for reconstructing the natural world in a virtual environment, generating new emotions, and transcending time and space in the familiar and unfamiliar virtual world. Recent years have seen a great leap in the development of intelligent manufacturing and virtual reality technologies, yet little research has been done to combine the two popular trends. To fill this gap, this paper specifically employs Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) guidelines to conduct a systematic review of the applications of virtual reality in smart manufacturing. Moreover, the practical challenges and the possible future direction will also be covered.

Robust tactile object recognition in open-set scenarios using Gaussian prototype learning

Tactile object recognition is crucial for effective grasping and manipulation. Recently, it has started to attract increasing attention in robotic applications. While there are many works on tactile object recognition and they also achieved promising performances in some applications, most of them are usually limited to closed world scenarios, where the object instances to be recognition in deployment are known and the same as that of during training. Since robots usually operate in realistic open-set scenarios, they inevitably encounter unknown objects. If automation systems falsely recognize unknown objects as one of the known classes based on the pre-trained model, it can lead to potentially catastrophic consequences. It motivates us to break the closed world assumption and to study tactile object recognition in realistic open-set conditions. Although several open-set recognition methods have been proposed, they focused on visual tasks and may not be suitable for tactile recognition. It is mainly due to that these methods do not take into account the special characteristic of tactile data in their models. To this end, we develop a novel Gaussian Prototype Learning method for robust tactile object recognition. Particularly, the proposed method converts feature distributions to probabilistic representations, and exploit uncertainty for tactile recognition in open-set scenarios. Experiments on the two tactile recognition benchmarks demonstrate the effectiveness of the proposed method on open-set tasks.

Asynchronous Intermittent Regulation of Human Arm Movement with Markovian Jumping Parameters

In this paper, the regulation stability problem of the human arm continuous movement is investigated based on Markovian jumping parameters. In particular, the intermittent control mechanism is adopted in the arm movement regulation procedure to model the human intermittent motor control strategy. Furthermore, by taking into account the Markovian jumping parameters with different modes, the asynchronous regulation issue is proposed to model mode mismatch between the motor control and arm movement. On the basis of model transformation, sufficient stability conditions are established during the arm movements, and the desired regulation gain can be obtained by the convex optimization method. In the end, an illustrative example is presented to show the applicability and effectiveness of our developed model and optimized regulation approach.

A Survey of Multifingered Robotic Manipulation: Biological Results, Structural Evolvements, and Learning Methods

Multifingered robotic hands (usually referred to as dexterous hands) are designed to achieve human-level or human-like manipulations for robots or as prostheses for the disabled. The research dates back 30 years ago, yet, there remain great challenges to effectively design and control them due to their high dimensionality of configuration, frequently switched interaction modes, and various task generalization requirements. This article aims to give a brief overview of multifingered robotic manipulation from three aspects: a) the biological results, b) the structural evolvements, and c) the learning methods, and discuss potential future directions. First, we investigate the structure and principle of hand-centered visual sensing, tactile sensing, and motor control and related behavioral results. Then, we review several typical multifingered dexterous hands from task scenarios, actuation mechanisms, and in-hand sensors points. Third, we report the recent progress of various learning-based multifingered manipulation methods, including but not limited to reinforcement learning, imitation learning, and other sub-class methods. The article concludes with open issues and our thoughts on future directions.

Pneumatic Soft Robots: Challenges and Benefits

In the field of robotics, soft robots have been showing great potential in the areas of medical care, education, service, rescue, exploration, detection, and wearable devices due to their inherently high flexibility, good compliance, excellent adaptability, and natural and safe interactivity. Pneumatic soft robots occupy an essential position among soft robots because of their features such as lightweight, high efficiency, non-pollution, and environmental adaptability. Thanks to its mentioned benefits, increasing research interests have been attracted to the development of novel types of pneumatic soft robots in the last decades. This article aims to investigate the solutions to develop and research the pneumatic soft robot. This paper reviews the status and the main progress of the recent research on pneumatic soft robots. Furthermore, a discussion about the challenges and benefits of the recent advancement of the pneumatic soft robot is provided.