A Neural Networks Approach to Determine Factors Associated With Self-Reported Discomfort in Picking Tasks

OBJECTIVE

A neural networks approach has been proposed to handle various inputs such as postural, anthropometric and environmental variables in order to estimate self-reported discomfort in picking tasks. An input reduction method has been proposed, reducing the input variables to the minimum data required to estimate self-reported discomfort with similar accuracy as the neural network fed with all variables.

BACKGROUND

Previous works have attempted to explore the relationship between several factors and self-reported discomfort using observational methods. The results showed that this relationship was not a simple linear relationship. Another study used neural networks to model the function returning reported discomfort according to static posture, age, and anthropometrics variables. The results demonstrated the model's ability to predict reported discomfort. But all the available variables were used to design the neural network.

METHOD

Eleven subjects carried-out picking tasks with various masses (0, 1, 3 kg) and imposed duration (5, 10, or 15 s). Continuous REBA score, anthropometric and environmental data were computed, and subjects' discomfort were collected. The data set of this work consisted in the computed continuous REBA score, anthropometric, environmental data and collected subjects' discomfort.

RESULTS

The results showed that the correlation between the estimated and experimental tested data was equal to 0.775 when using all the 14 available variables. After data reduction, only 6 variables were left, with a very close performance when predicting discomfort.

CONCLUSION

A neural network approach has been proposed to estimate self-reported discomfort according to a minimum set of postural, anthropometric and environmental variables in picking tasks.

APPLICATION

This method has the potential to support ergonomists in workstation designing processes, by adding discomfort prediction to virtual manikins' behaviors in simulation tools.

Validation of an ergonomic assessment method using Kinect data in real workplace conditions

Evaluating potential musculoskeletal disorders risks in real workstations is challenging as the environment is cluttered, which makes it difficult to accurately assess workers' postures. Being marker-free and calibration-free, Microsoft Kinect is a promising device although it may be sensitive to occlusions. We propose and evaluate a RULA ergonomic assessment in real work conditions using recently published occlusion-resistant Kinect skeleton data correction. First, we compared postures estimated with this method to ground-truth data, in standardized laboratory conditions. Second, we compared RULA scores to those provided by two professional experts, in a non-laboratory cluttered workplace condition. The results show that the corrected Kinect data can provide more accurate RULA grand scores, even under sub-optimal conditions induced by the workplace environment. This study opens new perspectives in musculoskeletal risk assessment as it provides the ergonomists with 30 Hz continuous information that could be analyzed offline and in a real-time framework.

Filtered pose graph for efficient kinect pose reconstruction

Being marker-free and calibration free, Microsoft Kinect is nowadays widely used in many motion-based applications, such as user training for complex industrial tasks and ergonomics pose evaluation. The major problem of Kinect is the placement requirement to obtain accurate poses, as well as its weakness against occlusions. To improve the robustness of Kinect in interactive motion-based applications, real-time data-driven pose reconstruction has been proposed. The idea is to utilize a database of accurately captured human poses as a prior to optimize the Kinect recognized ones, in order to estimate the true poses performed by the user. The key research problem is to identify the most relevant poses in the database for accurate and efficient reconstruction. In this paper, we propose a new pose reconstruction method based on modelling the pose database with a structure called Filtered Pose Graph, which indicates the intrinsic correspondence between poses. Such a graph not only speeds up the database poses selection process, but also improves the relevance of the selected poses for higher quality reconstruction. We apply the proposed method in a challenging environment of industrial context that involves sub-optimal Kinect placement and a large amount of occlusion. Experimental results show that our real-time system reconstructs Kinect poses more accurately than existing methods.

Visual Perspective and Feedback Guidance for VR Free-Throw Training

Accurate distance perception and natural interactions are mandatory conditions when training precision aiming tasks in VR. However, many factors specific to virtual environments (VEs) lead to differences in the way users execute a motor task in VR versus the real world. To investigate these differences, the authors performed a study on basketball beginners' free-throw performance in VEs under different visual conditions. Although the success rate is not statistically different, some adaptations occurred in the way the users performed the task, depending on the visual conditions. In the third-person perspective visual condition, the release parameters indicate that the users more accurately estimated distance to target. Adding visual guidance information (gradual depth information showing the ideal ball trajectory) also led to more natural motor behavior. The final aim of this study was to develop a reliable basketball free-throw training system in VEs, so the authors compared beginners' performances in VR with experts' models of performance. Their results show that most of the performance variables tended to evolve closer to the experts' performance during the training in the VE.

New lower-limb gait asymmetry indices based on a depth camera

Background: Various asymmetry indices have been proposed to compare the spatiotemporal, kinematic and kinetic parameters of lower limbs during the gait cycle. However, these indices rely on gait measurement systems that are costly and generally require manual examination, calibration procedures and the precise placement of sensors/markers on the body of the patient. Methods: To overcome these issues, this paper proposes a new asymmetry index, which uses an inexpensive, easy-to-use and markerless depth camera (Microsoft Kinect™) output. This asymmetry index directly uses depth images provided by the Kinect™ without requiring joint localization. It is based on the longitudinal spatial difference between lower-limb movements during the gait cycle. To evaluate the relevance of this index, fifteen healthy subjects were tested on a treadmill walking normally and then via an artificially-induced gait asymmetry with a thick sole placed under one shoe. The gait movement was simultaneously recorded using a Kinect™ placed in front of the subject and a motion capture system. Results: The proposed longitudinal index distinguished asymmetrical gait (p < 0.001), while other symmetry indices based on spatiotemporal gait parameters failed using such Kinect™ skeleton measurements. Moreover, the correlation coefficient between this index measured by Kinect™ and the ground truth of this index measured by motion capture is 0.968. Conclusion: This gait asymmetry index measured with a Kinect™ is low cost, easy to use and is a promising development for clinical gait analysis.

Detection of gait cycles in treadmill walking using a Kinect

Treadmill walking is commonly used to analyze several gait cycles in a limited space. Depth cameras, such as the low-cost and easy-to-use Kinect sensor, look promising for gait analysis on a treadmill for routine outpatient clinics. However, gait analysis is based on accurately detecting gait events (such as heel-strike) by tracking the feet which may be incorrectly recognized with Kinect. Indeed depth images could lead to confusion between the ground and the feet around the contact phase. To tackle this problem we assume that heel-strike events could be indirectly estimated by searching for extreme values of the distance between knee joints along the walking longitudinal axis. To evaluate this assumption, the motion of 11 healthy subjects walking on a treadmill was recorded using both an optoelectronic system and Kinect. The measures were compared to reference heel-strike events obtained with vertical foot velocity. When using the optoelectronic system to assess knee joints, heel-strike estimation errors were very small (29±18ms) leading to small cycle durations errors (0±15ms). To locate knees in depth map (Kinect), we used anthropometrical data to select the body point located at a constant height where the knee should be based on a reference posture. This Kinect approach gave heel-strike errors of 17±24ms (mean cycle duration error: 0±12ms). Using this same anthropometric methodology with optoelectronic data, the heel-strike error was 12±12ms (mean cycle duration error: 0±11ms). Compared to previous studies using Kinect, heel-strike and gait cycles were more accurately estimated, which could improve clinical gait analysis with such sensor.

Pose estimation with a Kinect for ergonomic studies: evaluation of the accuracy using a virtual mannequin

Analyzing human poses with a Kinect is a promising method to evaluate potentials risks of musculoskeletal disorders at workstations. In ecological situations, complex 3D poses and constraints imposed by the environment make it difficult to obtain reliable kinematic information. Thus, being able to predict the potential accuracy of the measurement for such complex 3D poses and sensor placements is challenging in classical experimental setups. To tackle this problem, we propose a new evaluation method based on a virtual mannequin. In this study, we apply this method to the evaluation of joint positions (shoulder, elbow, and wrist), joint angles (shoulder and elbow), and the corresponding RULA (a popular ergonomics assessment grid) upper-limb score for a large set of poses and sensor placements. Thanks to this evaluation method, more than 500,000 configurations have been automatically tested, which would be almost impossible to evaluate with classical protocols. The results show that the kinematic information obtained by the Kinect software is generally accurate enough to fill in ergonomic assessment grids. However inaccuracy strongly increases for some specific poses and sensor positions. Using this evaluation method enabled us to report configurations that could lead to these high inaccuracies. As a supplementary material, we provide a software tool to help designers to evaluate the expected accuracy of this sensor for a set of upper-limb configurations. Results obtained with the virtual mannequin are in accordance with those obtained from a real subject for a limited set of poses and sensor placements.

Toward "pseudo-haptic avatars": modifying the visual animation of self-avatar can simulate the perception of weight lifting

In this paper we study how the visual animation of a self-avatar can be artificially modified in real-time in order to generate different haptic perceptions. In our experimental setup, participants could watch their self-avatar in a virtual environment in mirror mode while performing a weight lifting task. Users could map their gestures on the self-animated avatar in real-time using a Kinect. We introduce three kinds of modification of the visual animation of the self-avatar according to the effort delivered by the virtual avatar: 1) changes on the spatial mapping between the user&#146;s gestures and the avatar, 2) different motion profiles of the animation, and 3) changes in the posture of the avatar (upper-body inclination). The experimental task consisted of a weight lifting task in which participants had to order four virtual dumbbells according to their virtual weight. The user had to lift each virtual dumbbells by means of a tangible stick, the animation of the avatar was modulated according to the virtual weight of the dumbbell. The results showed that the altering the spatial mapping delivered the best performance. Nevertheless, participants globally appreciated all the different visual effects. Our results pave the way to the exploitation of such novel techniques in various VR applications such as sport training, exercise games, or industrial training scenarios in single or collaborative mode.

Personified and multistate camera motions for first-person navigation in desktop virtual reality

In this paper we introduce novel 'Camera Motions' (CMs) to improve the sensations related to locomotion in virtual environments (VE). Traditional Camera Motions are artificial oscillating motions applied to the subjective viewpoint when walking in the VE, and they are meant to evoke and reproduce the visual flow generated during a human walk. Our novel camera motions are: (1) multistate, (2) personified, and (3) they can take into account the topography of the virtual terrain. Being multistate, our CMs can account for different states of locomotion in VE namely: walking, but also running and sprinting. Being personified, our CMs can be adapted to avatar's physiology such as to its size, weight or training status. They can then take into account avatar's fatigue and recuperation for updating visual CMs accordingly. Last, our approach is adapted to the topography of the VE. Running over a strong positive slope would rapidly decrease the advance speed of the avatar, increase its energy loss, and eventually change the locomotion mode, influencing the visual feedback of the camera motions. Our new approach relies on a locomotion simulator partially inspired by human physiology and implemented for a real-time use in Desktop VR. We have conducted a series of experiments to evaluate the perception of our new CMs by naive participants. Results notably show that participants could discriminate and perceive transitions between the different locomotion modes, by relying exclusively on our CMs. They could also perceive some properties of the avatar being used and, overall, very well appreciated the new CMs techniques. Taken together, our results suggest that our new CMs could be introduced in Desktop VR applications involving first-person navigation, in order to enhance sensations of walking, running, and sprinting, with potentially different avatars and over uneven terrains, such as for: training, virtual visits or video games.

Lower limb movement asymmetry measurement with a depth camera

The gait movement seems simple at first glance, but in reality it is a very complex neural and biomechanical process. In particular, if a person is affected by a disease or an injury, the gait may be modified. The left-right asymmetry of this movement can be related to neurological diseases, segment length differences or joint deficiencies. This paper proposes a novel method to analyze the asymmetry of lower limb movement which aims to be usable in daily clinical practice. This is done by recording the subject walking on a treadmill with a depth camera and then assessing left-right depth differences for the lower limbs during the gait cycle using horizontal flipping and registration of the depth images half a gait cycle apart. Validation on 20 subjects for normal gait and simulated pathologies (with a 5 cm sole), showed that this system is able to distinguish the asymmetry introduced. The major interest of this method is the low cost of the material needed and its easy setup in a clinical environment.

Contactless abnormal gait detection

We present a new method to detect abnormal gait based on the symmetry verification of the two-leg movement. Unlike other methods requiring special motion captors, the proposed method uses image processing techniques to correctly track leg movement. Our method first divides each leg into upper and lower parts using anatomical knowledge. Then each part is characterised by two straight lines approximating its two borders. Finally, leg movement is represented by the angle evolution of these lines. In this process, we propose a new line approximation algorithm which is robust to the outliers caused by incorrect separation of leg into upper / lower parts. In our experiment, the proposed method got very encouraging results. With 281 normal / abnormal gait videos of 9 people, this method achieved a classification accuracy of 91%.

Gait analysis with multiple depth cameras

The gait movement seems simple at first glance, but in reality it is a very complex neural and biomechanical process. In particular, if a person is affected by a disease or an injury, the gait may be modified. To help detecting such change, we propose a new method based on multiple depth cameras. The aim of this paper is to show the possibility to reconstruct the body 3D volume in real time during gait in order to detect a pathological problem related to this movement and eventually improve diagnosis. Preliminary results showed that the system is sensitive to gait change produced by a heel prosthesis (heel cup) inserted in one shoe of subjects walking on a treadmill. The system detected a difference between maximal forward and backward positions of lower limbs for this pathological walk, a difference that was negligible for normal walk. These promising results were obtained with only 3 low cost depth cameras; we therefore believe that such methodology opens a new and affordable way for 3D volumetric gait analysis.

Fall detection with multiple cameras: an occlusion-resistant method based on 3-D silhouette vertical distribution

According to the demographic evolution in industrialized countries, more and more elderly people will experience falls at home and will require emergency services. The main problem comes from fall-prone elderly living alone at home. To resolve this lack of safety, we propose a new method to detect falls at home, based on a multiple-cameras network for reconstructing the 3-D shape of people. Fall events are detected by analyzing the volume distribution along the vertical axis, and an alarm is triggered when the major part of this distribution is abnormally near the floor during a predefined period of time, which implies that a person has fallen on the floor. This method was validated with videos of a healthy subject who performed 24 realistic scenarios showing 22 fall events and 24 cofounding events (11 crouching position, 9 sitting position, and 4 lying on a sofa position) under several camera configurations, and achieved 99.7% sensitivity and specificity or better with four cameras or more. A real-time implementation using a graphic processing unit (GPU) reached 10 frames per second (fps) with 8 cameras, and 16 fps with 3 cameras.

Using virtual reality to analyze sports performance

Improving performance in sports can be difficult because many biomechanical, physiological, and psychological factors come into play during competition. A better understanding of the perception-action loop employed by athletes is necessary. This requires isolating contributing factors to determine their role in player performance. Because of its inherent limitations, video playback doesn't permit such in-depth analysis. Interactive, immersive virtual reality (VR) can overcome these limitations and foster a better understanding of sports performance from a behavioral-neuroscience perspective. Two case studies using VR technology and a sophisticated animation engine demonstrate how to use information from visual displays to inform a player's future course of action.

Does the Level of Graphical Detail of a Virtual Handball Thrower Influence a Goalkeeper's Motor Response?

The authors investigated how different levels of detail (LODs) of a virtual throwing action can influence a handball goalkeeper's motor response. Goalkeepers attempted to stop a virtual ball emanating from five different graphical LODs of the same virtual throwing action. The five levels of detail were: a textured reference level (L0), a non-textured level (L1), a wire-frame level (L2), a point-light-display (PLD) representation (L3) and a PLD level with reduced ball size (L4). For each motor response made by the goalkeeper we measured and analyzed the time to respond (TTR), the percentage of successful motor responses, the distance between the ball and the closest limb (when the stopping motion was incorrect) and the kinematics of the motion. Results showed that TTR, percentage of successful motor responses and distance with the closest limb were not significantly different for any of the five different graphical LODs. However the kinematics of the motion revealed that the trajectory of the stopping limb was significantly different when comparing the L1 and L3 levels, and when comparing the L1 and L4 levels. These differences in the control of the goalkeeper's actions suggests that the different level of information available in the PLD representations (L3 and L4) are causing the goalkeeper to adopt different motor strategies to control the approach of their limb to stop the ball. Key pointsVirtual reality technology can be used to analyze sport performance because it enables standardization and reproduction of sport situations.Defining a minimal graphical level of detail of a virtual action could decrease the real time calculation of a virtual reality system.A Point Light Display graphical representation of a virtual throwing motion seems to influence the regulation of action of real handball goalkeepers.

From bone to plausible bipedal locomotion using inverse kinematics

The purpose of this study is to validate a method based on anatomical data and biomechanical locomotor hypotheses that could be applied in palaeontology to simulate locomotion in fossil hominids. The main problem is to ensure that purely mathematical simulation, based on anatomical descriptions, is enough to test hypotheses on human motion control. A 3D geometric model of the lower limb was therefore processed from anatomical descriptions. From this 3D model, we developed a method to retrieve natural lower-limb motion depending on chosen constraints. We assumed that the role of lower-limb motion is to make the feet move from one footprint to the next by following a trajectory that resembles that of living humans (primary task). This method based on inverse kinematics also allows biomechanical laws of bipedal locomotion to be taken into account (secondary tasks). The laws tested in this study relate to preserving joint limits, minimizing energy and minimizing the distance to a rest posture proposed by anthropologists and viewed as input to our system. A weighted sum of the resulting derivable cost functions enabled us to select a specific solution in the null space of the primary task. In order to validate this approach, we compared simulated and captured motion from ten subjects for whom anthropometrical data were recorded. We concluded that this "anatomically based bipedalism simulation" seems promising as a means of investigating natural locomotion behaviour and might also be used to retrieve natural locomotion in fossil hominids where only little knowledge is available.

Using virtual reality to analyze links between handball thrower kinematics and goalkeeper's reactions

This work investigates the design of a new method to evaluate the importance of visual elements taken into account by a handball goalkeeper facing a thrower. Virtual reality was used to design and reproduce standardised situations in a controlled environment. Under such conditions, it was possible to isolate for investigation one visual element in the thrower's gestures. The goalkeeper's movements were recorded in order to compare his reactions to two separate throws where only one visual element was modified. Our systems allowed us to measure and record the effects of small changes in the thrower's movements. With the numerical values we obtained from our results we were able to come up with a scale of significance for each isolated element. These preliminary results look promising for neuroscience, allowing us to better understand the strategies used in duel situations.

Do handball throws always exhibit a proximal-to-distal segmental sequence?

Previous studies on overarm throwing have described a proximal-to-distal segmental sequence. The proximal segments reached their maximal linear velocities before the distal ones. In handball, no study has demonstrated this sequence from the upper torso to the wrist, although a recent study did present a different organization. The aim of this study was to analyse the throwing arm segmental organization during handball throwing. We found that the maximal linear velocity of the shoulder occurred after the maximal linear velocity of the elbow. Moreover, the maximal angular velocity of the upper torso occurred later than that of the elbow. Hence, contrary to other disciplines, the rotation of the upper torso was not suddenly stopped just after the forward arm motion was initiated. These results may apply to handball in general or be specific to the population of handball players studied. It may be advisable in future studies to include international players.

Does training have consequences for the walk-run transition speed?

A number of authors when studying the walk-run transition phenomenon focused either on the mechanical or energy expenditure whilst only a few used both parameters concurrently. Moreover the literature demonstrates that the contribution of these variables changes along with the level and method of training. Consequently the purpose of this study is to find, by analyzing concurrently these two variables, if the walk-run transition speed is linked to the type of training. To this end we calculated two theoretical transition speeds: one based on the metabolic energy expenditure St(1) and the second one based on the internal work St(2). Subjects were divided into three groups (untrained, sprint and endurance-trained men) who were required to walk and run on a treadmill at increasing speeds. Firstly we show that the relationship between St(1) and St(2) differs depending on the groups. Sprinters have a significantly lower St(2) than St(1) whereas the opposite is found for untrained subjects. We also show that the transition speed is linked to the subject's type of training. To conclude it seems that acquiring running techniques through specific training has consequences for the walk-run transition phenomenon.