The accuracy of conventional 2D video for quantifying upper limb kinematics in repetitive motion occupational tasks

Comparison of the observer, single-frame video and computer vision hand activity levels

Observer, manual single-frame video, and automated computer vision measures of the Hand Activity Level (HAL) were compared. HAL can be measured three ways: (1) observer rating (HALO), (2) calculated from single-frame multimedia video task analysis for measuring frequency (F) and duty cycle (D) (HALF), or (3) from automated computer vision (HALC). This study analysed videos collected from three prospective cohort studies to ascertain HALO, HALF, and HALC for 419 industrial videos. Although the differences for the three methods were relatively small on average (<1), they were statistically significant (p < .001). A difference between the HALC and HALF ratings within ±1 point on the HAL scale was the most consistent, where more than two thirds (68%) of all the cases were within that range and had a linear regression through the mean coefficient of 1.03 (R2 = 0.89). The results suggest that the computer vision methodology yields comparable results as single-frame video analysis.Practitioner summary: The ACGIH Hand Activity Level (HAL) was obtained for 419 industrial tasks using three methods: observation, calculated using single-frame video analysis and computer vision. The computer vision methodology produced results that were comparable to single-frame video analysis.

AMRTA-X: Grasp Kinematic Analysis during Myoelectric Prehension Orthosis and Body Powered Prehension Orthosis's Usage on Brachial Plexus Injury Patients

Brachial Plexus Injury (BPI) results in decreased motor function in upper extremity and leads to reduced hand grasping movement. Orthotic prehension is designed to create artificial grasp movements in paralyzed hand. This study was to compare grasp kinematic improvement between body powered and myoelectric prehension orthosis usage in patients with BPI. This study was a single group without control and post test with experimental study. The subjects of the study (n = 11) were brachial plexus injury patients with non-functional hand strength. Joint motion and angular velocity of metacarpophalangeal (MCP) and proximal interphalangeal (PIP) joint of index finger were evaluated. There was an improvement in joint motion and angular velocity after both orthosis usage. Joint motion in MCP and PIP, Angular velocity in MCP were not significantly different between myoelectric and body powered and myoelectric prehension orthosis usage. PIP angular velocity improvement were better after body powered prehension orthosis usage (p= 0.03).In conclusion, body powered and myolectric prehension orthosis usage improved kinematic parameter of index finger’s MCP and PIP joint. PIP angular velocity was better after body powered prehension orthosis usage.

Using Surgeon Hand Motions to Predict Surgical Maneuvers

OBJECTIVE

This study explores how common machine learning techniques can predict surgical maneuvers from a continuous video record of surgical benchtop simulations.

BACKGROUND

Automatic computer vision recognition of surgical maneuvers (suturing, tying, and transition) could expedite video review and objective assessment of surgeries.

METHOD

We recorded hand movements of 37 clinicians performing simple and running subcuticular suturing benchtop simulations, and applied three machine learning techniques (decision trees, random forests, and hidden Markov models) to classify surgical maneuvers every 2 s (60 frames) of video.

RESULTS

Random forest predictions of surgical video correctly classified 74% of all video segments into suturing, tying, and transition states for a randomly selected test set. Hidden Markov model adjustments improved the random forest predictions to 79% for simple interrupted suturing on a subset of randomly selected participants.

CONCLUSION

Random forest predictions aided by hidden Markov modeling provided the best prediction of surgical maneuvers. Training of models across all users improved prediction accuracy by 10% compared with a random selection of participants.

APPLICATION

Marker-less video hand tracking can predict surgical maneuvers from a continuous video record with similar accuracy as robot-assisted surgical platforms, and may enable more efficient video review of surgical procedures for training and coaching.

Modeling Surgical Technical Skill Using Expert Assessment for Automated Computer Rating

OBJECTIVE

Computer vision was used to predict expert performance ratings from surgeon hand motions for tying and suturing tasks.

SUMMARY BACKGROUND DATA

Existing methods, including the objective structured assessment of technical skills (OSATS), have proven reliable, but do not readily discriminate at the task level. Computer vision may be used for evaluating distinct task performance throughout an operation.

METHODS

Open surgeries was videoed and surgeon hands were tracked without using sensors or markers. An expert panel of 3 attending surgeons rated tying and suturing video clips on continuous scales from 0 to 10 along 3 task measures adapted from the broader OSATS: motion economy, fluidity of motion, and tissue handling. Empirical models were developed to predict the expert consensus ratings based on the hand kinematic data records.

RESULTS

The predicted versus panel ratings for suturing had slopes from 0.73 to 1, and intercepts from 0.36 to 1.54 (Average R2 = 0.81). Predicted versus panel ratings for tying had slopes from 0.39 to 0.88, and intercepts from 0.79 to 4.36 (Average R2 = 0.57). The mean square error among predicted and expert ratings was consistently less than the mean squared difference among individual expert ratings and the eventual consensus ratings.

CONCLUSIONS

The computer algorithm consistently predicted the panel ratings of individual tasks, and were more objective and reliable than individual assessment by surgical experts.

Pilot testing of the spring operated wearable enhancer for arm rehabilitation (SpringWear)

Background

Robotic devices for neurorehabilitation of movement impairments in persons with stroke have been studied extensively. However, the vast majority of these devices only allow practice of stereotyped components of simulated functional tasks in the clinic. Previously we developed SpringWear, a wearable, spring operated, upper extremity exoskeleton capable of assisting movements during real-life functional activities, potentially in the home. SpringWear assists shoulder flexion, elbow extension and forearm supination/pronation. The assistance profiles were designed to approximate the torque required to move the joint passively through its range. These three assisted DOF are combined with two passive shoulder DOF, allowing complex multi-joint movement patterns.

Methods

We performed a cross-sectional study to assess changes in movement patterns when assisted by SpringWear. Thirteen persons with chronic stroke performed range of motion (ROM) and functional tasks, including pick and place tasks with various objects. Sensors on the device measured rotation at all 5 DOF and a kinematic model calculated position of the wrist relative to the shoulder. Within subject t-tests were used to determine changes with assistance from SpringWear.

Results

Maximum shoulder flexion, elbow extension and forearm pronation/supination angles increased significantly during both ROM and functional tasks (p < 0.002). Elbow flexion/extension ROM also increased significantly (p < 0.001). When the subjects volitionally held up the arm against gravity, extension at the index finger proximal interphalangeal joint increased significantly (p = 0.033) when assisted by SpringWear. The forward reach workspace increased 19% (p = 0.002). Nine subjects could not complete the functional tasks unassisted and only one showed improvement on task completion with SpringWear.

Conclusions

SpringWear increased the usable workspace during reaching movements, but there was no consistent improvement in the ability to complete functional tasks. Assistance levels at the shoulder were increased only until the shoulder could be voluntarily held at 90 degrees of flexion. A higher level of assistance may have yielded better results. Also combining SpringWear with HandSOME, an exoskeleton for assisting hand opening, may yield the most dramatic improvements in functional task performance. These low-cost devices can potentially reduce effort and improve performance during task practice, increasing adherence to home training programs for rehabilitation.

Musculoskeletal disorders as a fatigue failure process: evidence, implications and research needs

Mounting evidence suggests that musculoskeletal disorders (MSDs) may be the result of a fatigue failure process in musculoskeletal tissues. Evaluations of MSD risk in epidemiological studies and current MSD risk assessment tools, however, have not yet incorporated important principles of fatigue failure analysis in their appraisals of MSD risk. This article examines the evidence suggesting that fatigue failure may play an important role in the aetiology of MSDs, assesses important implications with respect to MSD risk assessment and discusses research needs that may be required to advance the scientific community's ability to more effectively prevent the development of MSDs. Practitioner Summary: Evidence suggests that musculoskeletal disorders (MSDs) may result from a fatigue failure process. This article proposes a unifying framework that aims to explain why exposure to physical risk factors contributes to the development of work-related MSDs. Implications of that framework are discussed.

Measuring elemental time and duty cycle using automated video processing

A marker-less 2D video algorithm measured hand kinematics (location, velocity and acceleration) in a paced repetitive laboratory task for varying hand activity levels (HAL). The decision tree (DT) algorithm identified the trajectory of the hand using spatiotemporal relationships during the exertion and rest states. The feature vector training (FVT) method utilised the k-nearest neighbourhood classifier, trained using a set of samples or the first cycle. The average duty cycle (DC) error using the DT algorithm was 2.7%. The FVT algorithm had an average 3.3% error when trained using the first cycle sample of each repetitive task, and had a 2.8% average error when trained using several representative repetitive cycles. Error for HAL was 0.1 for both algorithms, which was considered negligible. Elemental time, stratified by task and subject, were not statistically different from ground truth (p < 0.05). Both algorithms performed well for automatically measuring elapsed time, DC and HAL. Practitioner Summary: A completely automated approach for measuring elapsed time and DC was developed using marker-less video tracking and the tracked kinematic record. Such an approach is automatic, repeatable, objective and unobtrusive, and is suitable for evaluating repetitive exertions, muscle fatigue and manual tasks.

Evaluation of Simulated Clinical Breast Exam Motion Patterns Using Marker-Less Video Tracking

OBJECTIVE

This study investigates using marker-less video tracking to evaluate hands-on clinical skills during simulated clinical breast examinations (CBEs).

BACKGROUND

There are currently no standardized and widely accepted CBE screening techniques.

METHODS

Experienced physicians attending a national conference conducted simulated CBEs presenting different pathologies with distinct tumorous lesions. Single hand exam motion was recorded and analyzed using marker-less video tracking. Four kinematic measures were developed to describe temporal (time pressing and time searching) and spatial (area covered and distance explored) patterns.

RESULTS

Mean differences between time pressing, area covered, and distance explored varied across the simulated lesions. Exams were objectively categorized as either sporadic, localized, thorough, or efficient for both temporal and spatial categories based on spatiotemporal characteristics. The majority of trials were temporally or spatially thorough (78% and 91%), exhibiting proportionally greater time pressing and time searching (temporally thorough) and greater area probed with greater distance explored (spatially thorough). More efficient exams exhibited proportionally more time pressing with less time searching (temporally efficient) and greater area probed with less distance explored (spatially efficient). Just two (5.9 %) of the trials exhibited both high temporal and spatial efficiency.

CONCLUSIONS

Marker-less video tracking was used to discriminate different examination techniques and measure when an exam changes from general searching to specific probing. The majority of participants exhibited more thorough than efficient patterns.

APPLICATION

Marker-less video kinematic tracking may be useful for quantifying clinical skills for training and assessment.