Engagnition: A multi-dimensional dataset for engagement recognition of children with autism spectrum disorder

Engagement plays a key role in improving the cognitive and motor development of children with autism spectrum disorder (ASD). Sensing and recognizing their engagement is crucial before sustaining and improving the engagement. Engaging technologies involving interactive and multi-sensory stimuli have improved engagement and alleviated hyperactive and stereotyped behaviors. However, due to the scarcity of data on engagement recognition for children with ASD, limited access to and small pools of participants, and the prohibitive application requirements such as robots, high cost, and expertise, implementation in real world is challenging. However, serious games have the potential to overcome those drawbacks and are suitable for practical use in the field. This study proposes Engagnition, a dataset for engagement recognition of children with ASD (N = 57) using a serious game, "Defeat the Monster," based on enhancing recognition and classification skills. The dataset consists of physiological and behavioral responses, annotated by experts. For technical validation, we report the distributions of engagement and intervention, and the signal-to-noise ratio of physiological signals.

Anomaly Detection Methods in Autonomous Robotic Missions

Since 2015, there has been an increase in articles on anomaly detection in robotic systems, reflecting its growing importance in improving the robustness and reliability of the increasingly utilized autonomous robots. This review paper investigates the literature on the detection of anomalies in Autonomous Robotic Missions (ARMs). It reveals different perspectives on anomaly and juxtaposition to fault detection. To reach a consensus, we infer a unified understanding of anomalies that encapsulate their various characteristics observed in ARMs and propose a classification of anomalies in terms of spatial, temporal, and spatiotemporal elements based on their fundamental features. Further, the paper discusses the implications of the proposed unified understanding and classification in ARMs and provides future directions. We envisage a study surrounding the specific use of the term anomaly, and methods for their detection could contribute to and accelerate the research and development of a universal anomaly detection system for ARMs.

Adversarial Data Augmentation for HMM-Based Anomaly Detection

In this work, we concentrate on the detection of anomalous behaviors in systems operating in the physical world and for which it is usually not possible to have a complete set of all possible anomalies in advance. We present a data augmentation and retraining approach based on adversarial learning for improving anomaly detection. In particular, we first define a method for generating adversarial examples for anomaly detectors based on Hidden Markov Models (HMMs). Then, we present a data augmentation and retraining technique that uses these adversarial examples to improve anomaly detection performance. Finally, we evaluate our adversarial data augmentation and retraining approach on four datasets showing that it achieves a statistically significant performance improvement and enhances the robustness to adversarial attacks. Key differences from the state-of-the-art on adversarial data augmentation are the focus on multivariate time series (as opposed to images), the context of one-class classification (in contrast to standard multi-class classification), and the use of HMMs (in contrast to neural networks).

A Systematic Literature Review of Decision-Making and Control Systems for Autonomous and Social Robots

In the last years, considerable research has been carried out to develop robots that can improve our quality of life during tedious and challenging tasks. In these contexts, robots operating without human supervision open many possibilities to assist people in their daily activities. When autonomous robots collaborate with humans, social skills are necessary for adequate communication and cooperation. Considering these facts, endowing autonomous and social robots with decision-making and control models is critical for appropriately fulfiling their initial goals. This manuscript presents a systematic review of the evolution of decision-making systems and control architectures for autonomous and social robots in the last three decades. These architectures have been incorporating new methods based on biologically inspired models and Machine Learning to enhance these systems’ possibilities to developed societies. The review explores the most novel advances in each application area, comparing their most essential features. Additionally, we describe the current challenges of software architecture devoted to action selection, an analysis not provided in similar reviews of behavioural models for autonomous and social robots. Finally, we present the future directions that these systems can take in the future.

Machine Learning-Assisted Approaches in Modernized Plant Breeding Programs

In the face of a growing global population, plant breeding is being used as a sustainable tool for increasing food security. A wide range of high-throughput omics technologies have been developed and used in plant breeding to accelerate crop improvement and develop new varieties with higher yield performance and greater resilience to climate changes, pests, and diseases. With the use of these new advanced technologies, large amounts of data have been generated on the genetic architecture of plants, which can be exploited for manipulating the key characteristics of plants that are important for crop improvement. Therefore, plant breeders have relied on high-performance computing, bioinformatics tools, and artificial intelligence (AI), such as machine-learning (ML) methods, to efficiently analyze this vast amount of complex data. The use of bigdata coupled with ML in plant breeding has the potential to revolutionize the field and increase food security. In this review, some of the challenges of this method along with some of the opportunities it can create will be discussed. In particular, we provide information about the basis of bigdata, AI, ML, and their related sub-groups. In addition, the bases and functions of some learning algorithms that are commonly used in plant breeding, three common data integration strategies for the better integration of different breeding datasets using appropriate learning algorithms, and future prospects for the application of novel algorithms in plant breeding will be discussed. The use of ML algorithms in plant breeding will equip breeders with efficient and effective tools to accelerate the development of new plant varieties and improve the efficiency of the breeding process, which are important for tackling some of the challenges facing agriculture in the era of climate change.

Adaptive Control Algorithm for Trajectory Tracking of Underactuated Unmanned Surface Vehicle (UUSV)

The ability to track the trajectory or path on the sea surface remains a key measurement in the control system of an unmanned surface vehicle (USV). In this research, the designed algorithm defines the path and minimizes the disturbances to zero. Underactuated USVs are ships or boats, which operate on the water surface without a crew and the underactuated system has low actuators than its degree of freedom (DOF). The adaptive control strategy is applied to the unbounded system as the ocean due to its high performance, while the robust control system attains high performance in the bounded system. To consider the trajectory tracking of UUSV and provide an optimal control strategy for a ship in the presence of external disturbances, the study presents a controller-based on model reference adaptive control with an integrator (MRACI), which guarantees the stability of a closed-loop system. The vehicle experiences variations in the system response due to external disturbance. This abovementioned scheme reduces the variations to nearly equal to zero making the vehicle stable. First, use computer-based simulation to verify the proposed controller under two different scenarios. Then, simulation results show that the designed scheme lowers the errors and performs well.

Integrating Social Assistive Robots, IoT, Virtual Communities and Smart Objects to Assist at-Home Independently Living Elders: the MoveCare Project

The integration of Ambient Assisted Living (AAL) frameworks with Socially Assistive Robots (SARs) has proven useful for monitoring and assisting older adults in their own home. However, the difficulties associated with long-term deployments in real-world complex environments are still highly under-explored. In this work, we first present the MoveCare system, an unobtrusive platform that, through the integration of a SAR into an AAL framework, aimed to monitor, assist and provide social, cognitive, and physical stimulation in the own houses of elders living alone and at risk of falling into frailty. We then focus on the evaluation and analysis of a long-term pilot campaign of more than 300 weeks of usages. We evaluated the system's acceptability and feasibility through various questionnaires and empirically assessed the impact of the presence of an assistive robot by deploying the system with and without it. Our results provide strong empirical evidence that Socially Assistive Robots integrated with monitoring and stimulation platforms can be successfully used for long-term support to older adults. We describe how the robot's presence significantly incentivised the use of the system, but slightly lowered the system's overall acceptability. Finally, we emphasise that real-world long-term deployment of SARs introduces a significant technical, organisational, and logistical overhead that should not be neglected nor underestimated in the pursuit of long-term robust systems. We hope that the findings and lessons learned from our work can bring value towards future long-term real-world and widespread use of SARs.

AGDS: adaptive goal-directed strategy for swarm drones flying through unknown environments

This paper aims to address a challenging problem of a drone swarm for a specific mission by reaching a desired region, through an unknown environment. A bio-inspired flocking algorithm with adaptive goal-directed strategy (AGDS) is proposed and developed for the drones swarmed across unknown environments. Each drone employs a biological visual mechanism to sense obstacles in within local perceptible scopes. Task information of the destination is only given to a few specified drones (named as informed agents), rather than to all other individual drones (uninformed agents). With the proposed flocking swarm, the informed agents operate collectively with the remaining uninformed agents to achieve a common and overall mission. By virtue of numerical simulation, the AGDS and non-adaptive goal-directed strategy (non-AGDS) are both presented and evaluated. Experiments by flying six DJI Tello quadrotors indoor are conducted to validate the developed flocking algorithm. Additional validations within canyon-like complicated scenarios have also been carried out. Both simulation and experimental results demonstrate the efficiency of the proposed swarm flocking algorithm with AGDS.

Self-reconfigurable robot vision pipeline for safer adaptation to varying pavements width and surface conditions

This work presents the vision pipeline for our in-house developed autonomous reconfigurable pavement sweeping robot named Panthera. As the goal of Panthera is to be an autonomous self-reconfigurable robot, it has to understand the type of pavement it is moving in so that it can adapt smoothly to changing pavement width and perform cleaning operations more efficiently and safely. deep learning (DL) based vision pipeline is proposed for the Panthera robot to recognize pavement features, including pavement type identification, pavement surface condition prediction, and pavement width estimation. The DeepLabv3+ semantic segmentation algorithm was customized to identify the pavement type classification, an eight-layer CNN was proposed for pavement surface condition prediction. Furthermore, pavement width estimation was computed by fusing the segmented pavement region on the depth map. In the end, the fuzzy inference system was implemented by taking input as the pavement width and its conditions detected and output as the safe operational speed. The vision pipeline was trained using the DL provided with the custom pavement images dataset. The performance was evaluated using offline test and real-time field trial images captured through the reconfigurable robot Panthera stereo vision sensor. In the experimental analysis, the DL-based vision pipeline components scored 88.02% and 93.22% accuracy for pavement segmentation and pavement surface condition assessment, respectively, and took approximately 10 ms computation time to process the single image frame from the vision sensor using the onboard computer.

Motion Planning of an Inchworm Robot Based on Improved Adaptive PSO

Focusing on the motion energy consumption of a self-developed inchworm robot’s peristaltic gait, based on the “error tracking” of cubic polynomial programming in Cartesian space and seventh polynomial programming in joint space, we propose an optimal motion planning method of energy consumption considering both kinematic and dynamic constraints. Firstly, we offer a mathematical description of the energy consumption and space curve similarity operator. Secondly, we describe the mathematical models of the robot trajectory and path that were established in terms of their dynamics and kinematics. Then, we propose a motion planning method based on improved adaptive particle swarm optimization (PSO) to accelerate the convergence speed of the algorithm and ensure the accuracy of the model calculation. Finally, we outline the simulation test carried out to measure the inchworm-like robot’s creeping gait. The results show that the motion path obtained by using the planning method proposed in this paper is the one with the least energy consumption by the robot among all the comparison paths. Moreover, compared with other algorithms, it was found that the result obtained by using the algorithm proposed in this paper is the one with the shortest solution time and the lowest energy consumption under the same iteration times. The calculation results verify the feasibility and effectiveness of the planning method.

Role of machine and organizational structure in science

The progress of science increasingly relies on machine learning (ML) and machines work alongside humans in various domains of science. This study investigates the team structure of ML-related projects and analyzes the contribution of ML to scientific knowledge production under different team structure, drawing on bibliometric analyses of 25,000 scientific publications in various disciplines. Our regression analyses suggest that (1) interdisciplinary collaboration between domain scientists and computer scientists as well as the engagement of interdisciplinary individuals who have expertise in both domain and computer sciences are common in ML-related projects; (2) the engagement of interdisciplinary individuals seem more important in achieving high impact and novel discoveries, especially when a project employs computational and domain approaches interdependently; and (3) the contribution of ML and its implication to team structure depend on the depth of ML.

Toward Benchmarking of Long-Term Spatio-Temporal Maps of Pedestrian Flows for Human-Aware Navigation

Despite the advances in mobile robotics, the introduction of autonomous robots in human-populated environments is rather slow. One of the fundamental reasons is the acceptance of robots by people directly affected by a robot’s presence. Understanding human behavior and dynamics is essential for planning when and how robots should traverse busy environments without disrupting people’s natural motion and causing irritation. Research has exploited various techniques to build spatio-temporal representations of people’s presence and flows and compared their applicability to plan optimal paths in the future. Many comparisons of how dynamic map-building techniques show how one method compares on a dataset versus another, but without consistent datasets and high-quality comparison metrics, it is difficult to assess how these various methods compare as a whole and in specific tasks. This article proposes a methodology for creating high-quality criteria with interpretable results for comparing long-term spatio-temporal representations for human-aware path planning and human-aware navigation scheduling. Two criteria derived from the methodology are then applied to compare the representations built by the techniques found in the literature. The approaches are compared on a real-world, long-term dataset, and the conception is validated in a field experiment on a robotic platform deployed in a human-populated environment. Our results indicate that continuous spatio-temporal methods independently modeling spatial and temporal phenomena outperformed other modeling approaches. Our results provide a baseline for future work to compare a wide range of methods employed for long-term navigation and provide researchers with an understanding of how these various methods compare in various scenarios.

Self-Supervised Robust Feature Matching Pipeline for Teach and Repeat Navigation

The performance of deep neural networks and the low costs of computational hardware has made computer vision a popular choice in many robotic systems. An attractive feature of deep-learned methods is their ability to cope with appearance changes caused by day-night cycles and seasonal variations. However, deep learning of neural networks typically relies on large numbers of hand-annotated images, which requires significant effort for data collection and annotation. We present a method that allows autonomous, self-supervised training of a neural network in visual teach-and-repeat (VT&R) tasks, where a mobile robot has to traverse a previously taught path repeatedly. Our method is based on a fusion of two image registration schemes: one based on a Siamese neural network and another on point-feature matching. As the robot traverses the taught paths, it uses the results of feature-based matching to train the neural network, which, in turn, provides coarse registration estimates to the feature matcher. We show that as the neural network gets trained, the accuracy and robustness of the navigation increases, making the robot capable of dealing with significant changes in the environment. This method can significantly reduce the data annotation efforts when designing new robotic systems or introducing robots into new environments. Moreover, the method provides annotated datasets that can be deployed in other navigation systems. To promote the reproducibility of the research presented herein, we provide our datasets, codes and trained models online.

Exploring Effects of Information Filtering With a VR Interface for Multi-Robot Supervision

Supervising and controlling remote robot systems currently requires many specialised operators to have knowledge of the internal state of the system in addition to the environment. For applications such as remote maintenance of future nuclear fusion reactors, the number of robots (and hence supervisors) required to maintain or decommission a facility is too large to be financially feasible. To address this issue, this work explores the idea of intelligently filtering information so that a single user can supervise multiple robots safely. We gathered feedback from participants using five methods for teleoperating a semi-autonomous multi-robot system via Virtual Reality (VR). We present a novel 3D interaction method to filter the displayed information to allow the user to read information from the environment without being overwhelmed. The novelty of the interface design is the link between Semantic and Spatial filtering and the hierarchical information contained within the multi robot system. We conducted a user study including a cohort of expert robot teleoperators comparing these methods; highlighting the significant effects of 3D interface design on the performance and perceived workload of a user teleoperating many robot agents in complex environments. The results from this experiment and subjective user feedback will inform future investigations that build upon this initial work.

Mapping Industry 4.0 Enabling Technologies into United Nations Sustainability Development Goals

The emerging of the fourth industrial revolution, also known as Industry 4.0 (I4.0), from the advancement in several technologies is viewed not only to promote economic growth, but also to enable a greener future. The 2030 Agenda of the United Nations for sustainable development sets out clear goals for the industry to foster the economy, while preserving social well-being and ecological validity. However, the influence of I4.0 technologies on the achievement of the Sustainable Development Goals (SDG) has not been conclusively or systematically investigated. By understanding the link between the I4.0 technologies and the SDGs, researchers can better support policymakers to consider the technological advancement in updating and harmonizing policies and strategies in different sectors (i.e., education, industry, and governmental) with the SDGs. To address this gap, academic experts in this paper have investigated the influence of I4.0 technologies on the sustainability targets identified by the UN. Key I4.0 element technologies have been classified to enable a quantitative mapping with the 17 SDGs. The results indicate that the majority of the I4.0 technologies can contribute positively to achieving the UN agenda. It was also found that the effects of the technologies on individual goals varies between direct and strong, and indirect and weak influences. The main insights and lessons learned from the mapping are provided to support future policy.

On the 5G and Beyond

This article provides an overview of the fifth generation of cellular communications (5G) and beyond. It presents the transmission techniques of current 5G communications and those expected of future developments, namely a brief study of non-orthogonal multiple access (NOMA) using the single carrier with frequency domain equalization (SC-FDE) block transmission technique, evidencing its added value in terms of spectral efficiency. An introduction to the sixth generation of cellular communications (6G) is also provided. The insertion of 5G and 6G within the Fourth Industrial Revolution framework (also known as Industry 4.0) is also dealt with. Consisting of a change in paradigm, when compared to previous generations, 5G supports a myriad of new services based on the Internet of things (IoT) and on vehicle-to-vehicle (V2V) communications, supporting technologies such as autonomous driving, smart cities, and remote surgery. The new services provided by 5G are supported by new techniques, such as millimeter waves (mm-wave), in addition to traditional microwave communication, and by massive multiple-input multiple-output (m-MIMO) technology. These techniques were not employed in the fourth generation of cellular communications (4G). While 5G plays an important role in the initial implementation of the Fourth Industrial Revolution, 6G will address a number of new services such as virtual reality (VR), augmented reality (AR), holographic services, the advanced Internet of things (IoT), AI-infused applications, wireless brain–computer interaction (BCI), and mobility at higher speeds. The current research on systems beyond 5G indicates that these applications shall be supported by new MIMO techniques and make use of terahertz (THz) bands.

Trajectory Planning and the Target Search by the Mobile Robot in an Environment Using a Behavior-Based Neural Network Approach

Navigation and path analysis in a cluttered environment is a challenging task over the last few decades. In this paper, a behavior-based neural network (BNN) and reactive control architecture have been presented for navigation of the mobile robot. Two different reactive behaviors have been taken as inputs function. Obstacle position is the first reactive behavior given by u(o), whereas obstacle angle u(n) according to the target position is the second reactive behavior. The angular velocity and steering angle are the output of the controller. The backpropagation architecture reduces the errors of weight function and records the best weight data that match the BNN controller. Using the BNN algorithm, the robot reacts quickly as compared to other developed techniques. To validate the performance of the controller, simulation and experimental results have been compared in the common platforms. The deviation in results for both the scenarios is found to be within 10%. The results of the BNN algorithm have also been compared with other existing techniques. Effectiveness of the proposed technique is measured in terms of smoothness of the realistic path, collision point detection, path length, and performance time.