Evaluation of Biomechanical and Mental Workload During Human-Robot Collaborative Pollination Task

OBJECTIVE

The purpose of this study is to identify the potential biomechanical and cognitive workload effects induced by human robot collaborative pollination task, how additional cues and reliability of the robot influence these effects and whether interacting with the robot influences the participant's anxiety and attitude towards robots.

BACKGROUND

Human-Robot Collaboration (HRC) could be used to alleviate pollinator shortages and robot performance issues. However, the effects of HRC for this setting have not been investigated.

METHODS

Sixteen participants were recruited. Four HRC modes, no cue, with cue, unreliable, and manual control were included. Three categories of dependent variables were measured: (1) spine kinematics (L5/S1, L1/T12, and T1/C7), (2) pupillary activation data, and (3) subjective measures such as perceived workload, robot-related anxiety, and negative attitudes towards robotics.

RESULTS

HRC reduced anxiety towards the cobot, decreased joint angles and angular velocity for the L5/S1 and L1/T12 joints, and reduced pupil dilation, with the "with cue" mode producing the lowest values. However, unreliability was detrimental to these gains. In addition, HRC resulted in a higher flexion angle for the neck (i.e., T1/C7).

CONCLUSION

HRC reduced the physical and mental workload during the simulated pollination task. Benefits of the additional cue were minimal compared to no cues. The increased joint angle in the neck and unreliability affecting lower and mid back joint angles and workload requires further investigation.

APPLICATION

These findings could be used to inform design decisions for HRC frameworks for agricultural applications that are cognizant of the different effects induced by HRC.

Structural neural correlates of mental fatigue and reward-induced improvement in performance

Neuroimaging studies investigating the association between mental fatigue (henceforth fatigue) and brain physiology have identified many brain regions that may underly the cognitive changes induced by fatigue. These studies focused on the functional changes and functional connectivity of the brain relating to fatigue. The structural correlates of fatigue, however, have received little attention. To fill this gap, this study explored the associations of fatigue with cortical thickness of frontal and parietal regions. In addition, we aimed to explore the associations between reward-induced improvement in performance and neuroanatomical markers in fatigued individuals. Thirty-nine healthy volunteers performed the psychomotor vigilance task for 15 min (i.e., 3 time-on-task blocks of 5 min) out of scanner; followed by an additional rewarded block of the task lasting 5 min. Baseline high-resolution T1-weigthed MR images were obtained. Reaction time increased with time-on-task but got faster again in the rewarded block. Participants' subjective fatigue increased during task performance. In addition, we found that higher increase in subjective mental fatigue was associated with the cortical thickness of the following areas: bilateral precuneus, right precentral gyrus; right pars triangularis and left superior frontal gyrus. Our results suggest that individual differences in subjective mental fatigue may be explained by differences in the degree of cortical thickness of areas that are associated with motor processes, executive functions, intrinsic alertness and are parts of the default mode network.

Trust in Shared-Space Collaborative Robots: Shedding Light on the Human Brain

BACKGROUND

Industry 4.0 is currently underway allowing for improved manufacturing processes that leverage the collective advantages of human and robot agents. Consideration of trust can improve the quality and safety in such shared-space human-robot collaboration environments.

OBJECTIVE

The use of physiological response to monitor and understand trust is currently limited due to a lack of knowledge on physiological indicators of trust. This study examines neural responses to trust within a shared-workcell human-robot collaboration task as well as discusses the use of granular and multimodal perspectives to study trust.

METHODS

Sixteen sex-balanced participants completed a surface finishing task in collaboration with a UR10 collaborative robot. All participants underwent robot reliability conditions and robot assistance level conditions. Brain activation and connectivity using functional near infrared spectroscopy, subjective responses, and performance were measured throughout the study.

RESULTS

Significantly, increased neural activation was observed in response to faulty robot behavior within the medial and right dorsolateral prefrontal cortex (PFC). A similar trend was observed for the anterior PFC, primary motor cortex, and primary visual cortex. Faulty robot behavior also resulted in reduced functional connectivity strengths throughout the brain.

DISCUSSION

These findings implicate regions in the prefrontal cortex along with specific connectivity patterns as signifiers of distrusting conditions. The neural response may be indicative of how trust is influenced, measured, and manifested for human-robot collaboration that requires active teaming.

APPLICATION

Neuroergonomic response metrics can reveal new perspectives on trust in automation that subjective responses alone are not able to provide.

The effect of sleepiness in situation awareness: A scoping review

BACKGROUND

Situational awareness is the acquisition of information from elements present in the work environment, the perception of the meaning of this information, and the prediction of future working conditions. Sleepiness and fatigue can influence an individual's ability to reach situation awareness, decision-making, and performance on a task.

OBJECTIVE

This scoping review examines methods used to assess situational awareness, fatigue, sleepiness, and their interrelationships.

METHODS

A systematic search of online databases was conducted to identify experimental, peer-reviewed articles published in English between 2017 and 2022. A total of 29 publications were selected for analysis.

RESULTS

The selected studies originated from various countries, primarily in the northern hemisphere. Health and automotive engineering were the academic categories with the highest publications. The studies employed objective and subjective methods to assess situational awareness, fatigue, and sleepiness.

CONCLUSIONS

Most studies reported a decline in situational awareness during fatigue and sleepiness conditions, although one study did not find this association. Future research should focus on employing objective methods to analyze cognitive factors, increasing sample sizes, and conducting testing in real-world situations.

From brain to worksite: the role of fNIRS in cognitive studies and worker safety

Effective hazard recognition and decision-making are crucial factors in ensuring workplace safety in the construction industry. Workers' cognition closely relates to that hazard-handling behavior. Functional near-infrared spectroscopy (fNIRS) is a neurotechique tool that can evaluate the concentration vibration of oxygenated hemoglobin [ H b O 2 ] and deoxygenated hemoglobin [H b R ] to reflect the cognition process. It is essential to monitor workers' brain activity by fNIRS to analyze their cognitive status and reveal the mechanism in hazard recognition and decision-making process, providing guidance for capability evaluation and management enhancement. This review offers a systematic assessment of fNIRS, encompassing the basic theory, experiment analysis, data analysis, and discussion. A literature search and content analysis are conducted to identify the application of fNIRS in construction safety research, the limitations of selected studies, and the prospects of fNIRS in future research. This article serves as a guide for researchers keen on harnessing fNIRS to bolster construction safety standards and forwards insightful recommendations for subsequent studies.

Work performance in industry: The impact of mental fatigue and a passive back exoskeleton on work efficiency

Mental fatigue (MF) is likely to occur in the industrial working population. However, the link between MF and industrial work performance has not been investigated, nor how this interacts with a passive lower back exoskeleton used during industrial work. Therefore, to elucidate its potential effect(s), this study investigated the accuracy of work performance and movement duration through a dual task paradigm and compared results between mentally fatigued volunteers and controls, with and without the exoskeleton. No main effects of MF and the exoskeleton were found. However, when mentally fatigued and wearing the exoskeleton, movement duration significantly increased compared to the baseline condition (β_MF:Exo = 0.17, p = .02, ω² = .03), suggesting an important interaction between the exoskeleton and one's psychobiological state. Importantly, presented data indicate a negative effect on production efficiency through increased performance time. Further research into the cognitive aspects of industrial work performance and human-exoskeleton interaction is therefore warranted.

Research Perspectives in Collaborative Assembly: A Review

In recent years, the emergence of Industry 4.0 technologies has introduced manufacturing disruptions that necessitate the development of accompanying socio-technical solutions. There is growing interest for manufacturing enterprises to embrace the drivers of the Smart Industry paradigm. Among these drivers, human–robot physical co-manipulation of objects has gained significant interest in the literature on assembly operations. Motivated by the requirement for human dyads between the human and the robot counterpart, this study investigates recent literature on the implementation methods of human–robot collaborative assembly scenarios. Using a combination of strings, the researchers performed a systematic review search, sourcing 451 publications from various databases (Science Direct (253), IEEE Xplore (49), Emerald (32), PudMed (21) and SpringerLink (96)). A coding assignment in Eppi-Reviewer helped screen the literature based on ‘exclude’ and ‘include’ criteria. The final number of full-text publications considered in this literature review is 118 peer-reviewed research articles published up until September 2022. The findings anticipate that research publications in the fields of human–robot collaborative assembly will continue to grow. Understanding and modeling the human interaction and behavior in robot co-assembly is crucial to the development of future sustainable smart factories. Machine vision and digital twins modeling begin to emerge as promising interfaces for the evaluation of tasks distribution strategies for mitigating the actual human ergonomic and safety risks in collaborative assembly solutions design.

Ergonomic human-robot collaboration in industry: A review

In the current industrial context, the importance of assessing and improving workers' health conditions is widely recognised. Both physical and psycho-social factors contribute to jeopardising the underlying comfort and well-being, boosting the occurrence of diseases and injuries, and affecting their quality of life. Human-robot interaction and collaboration frameworks stand out among the possible solutions to prevent and mitigate workplace risk factors. The increasingly advanced control strategies and planning schemes featured by collaborative robots have the potential to foster fruitful and efficient coordination during the execution of hybrid tasks, by meeting their human counterparts' needs and limits. To this end, a thorough and comprehensive evaluation of an individual's ergonomics, i.e. direct effect of workload on the human psycho-physical state, must be taken into account. In this review article, we provide an overview of the existing ergonomics assessment tools as well as the available monitoring technologies to drive and adapt a collaborative robot's behaviour. Preliminary attempts of ergonomic human-robot collaboration frameworks are presented next, discussing state-of-the-art limitations and challenges. Future trends and promising themes are finally highlighted, aiming to promote safety, health, and equality in worldwide workplaces.

Physiological and perceptual consequences of trust in collaborative robots: An empirical investigation of human and robot factors

Measuring trust is an important element of effective human-robot collaborations (HRCs). It has largely relied on subjective responses and thus cannot be readily used for adapting robots in shared operations, particularly in shared-space manufacturing applications. Additionally, whether trust in such HRCs differ under altered operator cognitive states or with sex remains unknown. This study examined the impacts of operator cognitive fatigue, robot reliability, and operator sex on trust symptoms in collaborative robots through both objective measures (i.e., performance, heart rate variability) and subjective measures (i.e., surveys). Male and female participants were recruited to perform a metal surface polishing task in partnership with a collaborative robot (UR10), in which they underwent reliability conditions (reliable, unreliable) and cognitive fatigue conditions (fatigued, not fatigued). As compared to the reliable conditions, unreliable robot manipulations resulted in perceived trust, an increase in both sympathetic and parasympathetic activity, and operator-induced reduction in task efficiency and accuracy but not precision. Cognitive fatigue was shown to correlate with higher fatigue scores and reduced task efficiency, more severely impacting females. The results highlight key interplays between operator states of fatigue, sex, and robot reliability on both subjective and objective responses of trust. These findings provide a strong foundation for future investigations on better understanding the relationship between human factors and trust in HRC as well as aid in developing more diagnostic and deployable measures of trust.

Generalisable machine learning models trained on heart rate variability data to predict mental fatigue

A prolonged period of cognitive performance often leads to mental fatigue, a psychobiological state that increases the risk of injury and accidents. Previous studies have trained machine learning algorithms on Heart Rate Variability (HRV) data to detect fatigue in order to prevent its consequences. However, the results of these studies cannot be generalised because of various methodological issues including the use of only one type of cognitive task to induce fatigue which makes any predictions task-specific. In this study, we combined the datasets of three experiments each of which applied different cognitive tasks for fatigue induction and trained algorithms that detect fatigue and predict its severity. We also tested different time window lengths and compared algorithms trained on resting and task related data. We found that classification performance was best when the support vector classifier was trained on task related HRV calculated for a 5-min time window (AUC = 0.843, accuracy = 0.761). For the prediction of fatigue severity, CatBoost regression showed the best performance when trained on 3-min HRV data and self-reported measures (R² = 0.248, RMSE = 17.058). These results indicate that both the detection and prediction of fatigue based on HRV are effective when machine learning models are trained on heterogeneous, multi-task datasets.

Development of Modular and Adaptive Laboratory Set-Up for Neuroergonomic and Human-Robot Interaction Research

The industry increasingly insists on academic cooperation to solve the identified problems such as workers' performance, wellbeing, job satisfaction, and injuries. It causes an unsafe and unpleasant working environment that directly impacts the quality of the product, workers' productivity, and effectiveness. This study aimed to give a specialized solution for tests and explore possible solutions to the given problem in neuroergonomics and human–robot interaction. The designed modular and adaptive laboratory model of the industrial assembly workstation represents the laboratory infrastructure for conducting advanced research in the field of ergonomics, neuroergonomics, and human–robot interaction. It meets the operator's anatomical, anthropometric, physiological, and biomechanical characteristics. Comparing standard, ergonomic, guided, and collaborative work will be possible based on workstation construction and integrated elements. These possibilities allow the industry to try, analyze, and get answers for an identified problem, the condition, habits, and behavior of operators in the workplace. The set-up includes a workstation with an industry work chair, a Poka–Yoke system, adequate lighting, an audio 5.0 system, containers with parts and tools, EEG devices (a cap and smartfones), an EMG device, touchscreen PC screen, and collaborative robot. The first phase of the neuroergonomic study was performed according to the most common industry tasks defined as manual, monotonous, and repetitive activities. Participants have a task to assemble the developed prototype model of an industrial product using prepared parts and elements, and instructed by the installed touchscreen PC. In the beginning, the participant gets all the necessary information about the experiment and gets 15 min of practice. After the introductory part, the EEG device is mounted and prepared for recording. The experiment starts with relaxing music for 5 min. The whole experiment lasts two sessions per 60 min each, with a 15 min break between the sessions. Based on the first experiments, it is possible to develop, construct, and conduct complex experiments for industrial purposes to improve the physical, cognitive, and organizational aspects and increase workers' productivity, efficiency, and effectiveness. It has highlighted the possibility of applying modular and adaptive ergonomic research laboratory experimental set-up to transform standard workplaces into the workplaces of the future.

Human Factors Considerations and Metrics in Shared Space Human-Robot Collaboration: A Systematic Review

The degree of successful human-robot collaboration is dependent on the joint consideration of robot factors (RF) and human factors (HF). Depending on the state of the operator, a change in a robot factor, such as the behavior or level of autonomy, can be perceived differently and affect how the operator chooses to interact with and utilize the robot. This interaction can affect system performance and safety in dynamic ways. The theory of human factors in human-automation interaction has long been studied; however, the formal investigation of these HFs in shared space human-robot collaboration (HRC) and the potential interactive effects between covariate HFs (HF-HF) and HF-RF in shared space collaborative robotics requires additional investigation. Furthermore, methodological applications to measure or manipulate these factors can provide insights into contextual effects and potential for improved measurement techniques. As such, a systematic literature review was performed to evaluate the most frequently addressed operator HF states in shared space HRC, the methods used to quantify these states, and the implications of the states on HRC. The three most frequently measured states are: trust, cognitive workload, and anxiety, with subjective questionnaires universally the most common method to quantify operator states, excluding fatigue where electromyography is more common. Furthermore, the majority of included studies evaluate the effect of manipulating RFs on HFs, but few explain the effect of the HFs on system attributes or performance. For those that provided this information, HFs have been shown to impact system efficiency and response time, collaborative performance and quality of work, and operator utilization strategy.

Ergonomics and Human Factors as a Requirement to Implement Safer Collaborative Robotic Workstations: A Literature Review

There is a worldwide interest in implementing collaborative robots (Cobots) to reduce work-related Musculoskeletal Disorders (WMSD) risk. While prior work in this field has recognized the importance of considering Ergonomics & Human Factors (E&HF) in the design phase, most works tend to highlight workstations’ improvements due to Human-Robot Collaboration (HRC). Based on a literature review, the current study summarises studies where E&HF was considered a requirement rather than an output. In this article, the authors are interested in understanding the existing studies focused on Cobots’ implementation with ergonomic requirements, and the methods applied to design safer collaborative workstations. This review was performed in four prominent publications databases: Scopus, Web of Science, Pubmed, and Google Scholar, searching for the keywords ‘Collaborative robots’ or ‘Cobots’ or ‘HRC’ and ‘Ergonomics’ or ‘Human factors’. Based on the inclusion criterion, 20 articles were reviewed, and the main conclusions of each are provided. Additionally, the focus was given to the segmentation between studies considering E&HF during the design phase of HRC systems and studies applying E&HF in real-time on HRC systems. The results demonstrate the novelty of this topic, especially of the real-time applications of ergonomics as a requirement. Globally, the results of the reviewed studies showed the potential of E&HF requirements integrated into HRC systems as a relevant input for reducing WMSD risk.