Measuring elemental time and duty cycle using automated video processing

Comparison of the observer, single-frame video and computer vision hand activity levels

Observer, manual single-frame video, and automated computer vision measures of the Hand Activity Level (HAL) were compared. HAL can be measured three ways: (1) observer rating (HALO), (2) calculated from single-frame multimedia video task analysis for measuring frequency (F) and duty cycle (D) (HALF), or (3) from automated computer vision (HALC). This study analysed videos collected from three prospective cohort studies to ascertain HALO, HALF, and HALC for 419 industrial videos. Although the differences for the three methods were relatively small on average (<1), they were statistically significant (p < .001). A difference between the HALC and HALF ratings within ±1 point on the HAL scale was the most consistent, where more than two thirds (68%) of all the cases were within that range and had a linear regression through the mean coefficient of 1.03 (R2 = 0.89). The results suggest that the computer vision methodology yields comparable results as single-frame video analysis.Practitioner summary: The ACGIH Hand Activity Level (HAL) was obtained for 419 industrial tasks using three methods: observation, calculated using single-frame video analysis and computer vision. The computer vision methodology produced results that were comparable to single-frame video analysis.

Angle measurement stability and cycle counting accuracy of hours-long duration IMU based arm motion tracking with application to normal shoulder ADLs

BACKGROUND

Inertial measurement units are increasing used for monitoring joint motion, but there is a need to demonstrate their suitability during hours-long continuous use, as well as a need for validated methods to count arm cycles and provide descriptions of typical cycles.

RESEARCH QUESTION

Do IMU sensors and rainflow counting have sufficient accuracy for tracking and cycle counting of hours-long continuous arm motion? If so, what are the cycle rates of normal arm ADL and is there a representative cycle that can serve as a 'gait cycle' for the arm?

METHODS

IMU sensors continuously tracked a robot performing 8 h of simulated cyclic arm motion. Error in the angle measurements was regressed against time to determine the rate of error and the total accumulated error. Additionally, the cycle count accuracy of rainflow, peak/valley, and Fourier transform counting methods was evaluated.

RESULTS

Over 8 h the IMU measurements accumulated a maximum 0.473° of error and the rainflow method counted cycles with less than 1% error. Applying rainflow counting to normal shoulder ADL resulted in an average rate of 533 elevation cycles per day.Tabulating the ADL cycles by mean and range values into a matrix and calculating the centroid, the single best values representing arm elevation cycles were a mean of 22.4° and a range of 21.6°.

SIGNIFICANCE

IMU sensors can track arm motion for 8 h with little increase in error, though during longer durations error may reach unacceptable levels. For normal arm ADL, the rainflow determined count of arm elevation full-cycles differed from previous estimates based on peak/valley counting. From the rainflow counting, a single cycle representation of cycle mean and range was determined that can be used as a 'gait cycle' for the shoulder.

The role of machine learning in the primary prevention of work-related musculoskeletal disorders: A scoping review

To determine the applications of machine learning (ML) techniques used for the primary prevention of work-related musculoskeletal disorders (WMSDs), a scoping review was conducted using seven literature databases. Of the 4,639 initial results, 130 primary research studies were deemed relevant for inclusion. Studies were reviewed and classified as a contribution to one of six steps within the primary WMSD prevention research framework by van der Beek et al. (2017). ML techniques provided the greatest contributions to the development of interventions (48 studies), followed by risk factor identification (33 studies), underlying mechanisms (29 studies), incidence of WMSDs (14 studies), evaluation of interventions (6 studies), and implementation of effective interventions (0 studies). Nearly a quarter (23.8%) of all included studies were published in 2020. These findings provide insight into the breadth of ML techniques used for primary WMSD prevention and can help identify areas for future research and development.

Assessment of work-related hand and elbow workloads using measurement-based TLV for HAL

Direct-measurement-based methods for assessing workloads of the hand or elbow in the field are rare. Aim of the study was to develop such a method based on the Threshold Limit Value for Hand Activity Level (TLV for HAL). Hence, HAL was quantified using kinematic data (mean power frequencies, angular velocities and micro-pauses) and combined with electromyographic data (root-mean-square values) in order to generate a measurement-based TLV for HAL (mTLV for HAL). The multi-sensor system CUELA including inertial sensors, potentiometers and a 4-channel surface electromyography module was used. For wrist and elbow regions, associations between mTLV for HAL and disorders/complaints (quantified by odds ratios (OR [95%-confidence interval])) were tested exploratively within a cross-sectional field study with 500 participants. Higher workloads were frequently significantly associated with arthrosis of distal joints (9.23 [3.29-25.87]), wrist complaints (2.89 [1.63-5.11]) or elbow complaints (1.99 [1.08-3.67]). The new method could extend previous application possibilities.

Using Surgeon Hand Motions to Predict Surgical Maneuvers

OBJECTIVE

This study explores how common machine learning techniques can predict surgical maneuvers from a continuous video record of surgical benchtop simulations.

BACKGROUND

Automatic computer vision recognition of surgical maneuvers (suturing, tying, and transition) could expedite video review and objective assessment of surgeries.

METHOD

We recorded hand movements of 37 clinicians performing simple and running subcuticular suturing benchtop simulations, and applied three machine learning techniques (decision trees, random forests, and hidden Markov models) to classify surgical maneuvers every 2 s (60 frames) of video.

RESULTS

Random forest predictions of surgical video correctly classified 74% of all video segments into suturing, tying, and transition states for a randomly selected test set. Hidden Markov model adjustments improved the random forest predictions to 79% for simple interrupted suturing on a subset of randomly selected participants.

CONCLUSION

Random forest predictions aided by hidden Markov modeling provided the best prediction of surgical maneuvers. Training of models across all users improved prediction accuracy by 10% compared with a random selection of participants.

APPLICATION

Marker-less video hand tracking can predict surgical maneuvers from a continuous video record with similar accuracy as robot-assisted surgical platforms, and may enable more efficient video review of surgical procedures for training and coaching.

Predicting Sagittal Plane Lifting Postures From Image Bounding Box Dimensions

OBJECTIVE

A method for automatically classifying lifting postures from simple features in video recordings was developed and tested. We explored if an "elastic" rectangular bounding box, drawn tightly around the subject, can be used for classifying standing, stooping, and squatting at the lift origin and destination.

BACKGROUND

Current marker-less video tracking methods depend on a priori skeletal human models, which are prone to error from poor illumination, obstructions, and difficulty placing cameras in the field. Robust computer vision algorithms based on spatiotemporal features were previously applied for evaluating repetitive motion tasks, exertion frequency, and duty cycle.

METHODS

Mannequin poses were systematically generated using the Michigan 3DSSPP software for a wide range of hand locations and lifting postures. The stature-normalized height and width of a bounding box were measured in the sagittal plane and when rotated horizontally by 30°. After randomly ordering the data, a classification and regression tree algorithm was trained to classify the lifting postures.

RESULTS

The resulting tree had four levels and four splits, misclassifying 0.36% training-set cases. The algorithm was tested using 30 video clips of industrial lifting tasks, misclassifying 3.33% test-set cases. The sensitivity and specificity, respectively, were 100.0% and 100.0% for squatting, 90.0% and 100.0% for stooping, and 100.0% and 95.0% for standing.

CONCLUSIONS

The tree classification algorithm is capable of classifying lifting postures based only on dimensions of bounding boxes.

APPLICATIONS

It is anticipated that this practical algorithm can be implemented on handheld devices such as a smartphone, making it readily accessible to practitioners.