1
|
Radwin RG, Hu YH, Akkas O, Bao S, Harris-Adamson C, Lin JH, Meyers AR, Rempel D. Comparison of the observer, single-frame video and computer vision hand activity levels. ERGONOMICS 2023; 66:1132-1141. [PMID: 36227226 PMCID: PMC10130228 DOI: 10.1080/00140139.2022.2136407] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/14/2022] [Accepted: 10/10/2022] [Indexed: 05/11/2023]
Abstract
Observer, manual single-frame video, and automated computer vision measures of the Hand Activity Level (HAL) were compared. HAL can be measured three ways: (1) observer rating (HALO), (2) calculated from single-frame multimedia video task analysis for measuring frequency (F) and duty cycle (D) (HALF), or (3) from automated computer vision (HALC). This study analysed videos collected from three prospective cohort studies to ascertain HALO, HALF, and HALC for 419 industrial videos. Although the differences for the three methods were relatively small on average (<1), they were statistically significant (p < .001). A difference between the HALC and HALF ratings within ±1 point on the HAL scale was the most consistent, where more than two thirds (68%) of all the cases were within that range and had a linear regression through the mean coefficient of 1.03 (R2 = 0.89). The results suggest that the computer vision methodology yields comparable results as single-frame video analysis.Practitioner summary: The ACGIH Hand Activity Level (HAL) was obtained for 419 industrial tasks using three methods: observation, calculated using single-frame video analysis and computer vision. The computer vision methodology produced results that were comparable to single-frame video analysis.
Collapse
Affiliation(s)
| | - Yu Hen Hu
- University of Wisconsin, Madison, WI, USA
| | - Oguz Akkas
- University of Wisconsin, Madison, WI, USA
| | - Stephen Bao
- SHARP Program, Washington State Department of Labor and Industries, Olympia, WA, USA
| | | | - Jia-Hua Lin
- SHARP Program, Washington State Department of Labor and Industries, Olympia, WA, USA
| | - Alysha R. Meyers
- Division of Field Studies and Engineering, National Institute for Occupational Safety and Health, Cincinnati, OH, USA
| | - David Rempel
- University of California-San Francisco, San Francisco, CA, USA
| |
Collapse
|
2
|
Fundhi KAP. AMRTA-X: Grasp Kinematic Analysis during Myoelectric Prehension Orthosis and Body Powered Prehension Orthosis's Usage on Brachial Plexus Injury Patients. FOLIA MEDICA INDONESIANA 2021. [DOI: 10.20473/fmi.v57i1.9035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Brachial Plexus Injury (BPI) results in decreased motor function in upper extremity and leads to reduced hand grasping movement. Orthotic prehension is designed to create artificial grasp movements in paralyzed hand. This study was to compare grasp kinematic improvement between body powered and myoelectric prehension orthosis usage in patients with BPI. This study was a single group without control and post test with experimental study. The subjects of the study (n = 11) were brachial plexus injury patients with non-functional hand strength. Joint motion and angular velocity of metacarpophalangeal (MCP) and proximal interphalangeal (PIP) joint of index finger were evaluated. There was an improvement in joint motion and angular velocity after both orthosis usage. Joint motion in MCP and PIP, Angular velocity in MCP were not significantly different between myoelectric and body powered and myoelectric prehension orthosis usage. PIP angular velocity improvement were better after body powered prehension orthosis usage (p= 0.03).In conclusion, body powered and myolectric prehension orthosis usage improved kinematic parameter of index finger’s MCP and PIP joint. PIP angular velocity was better after body powered prehension orthosis usage.
Collapse
|
3
|
Azari DP, Hu YH, Miller BL, Le BV, Radwin RG. Using Surgeon Hand Motions to Predict Surgical Maneuvers. HUMAN FACTORS 2019; 61:1326-1339. [PMID: 31013463 DOI: 10.1177/0018720819838901] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
OBJECTIVE This study explores how common machine learning techniques can predict surgical maneuvers from a continuous video record of surgical benchtop simulations. BACKGROUND Automatic computer vision recognition of surgical maneuvers (suturing, tying, and transition) could expedite video review and objective assessment of surgeries. METHOD We recorded hand movements of 37 clinicians performing simple and running subcuticular suturing benchtop simulations, and applied three machine learning techniques (decision trees, random forests, and hidden Markov models) to classify surgical maneuvers every 2 s (60 frames) of video. RESULTS Random forest predictions of surgical video correctly classified 74% of all video segments into suturing, tying, and transition states for a randomly selected test set. Hidden Markov model adjustments improved the random forest predictions to 79% for simple interrupted suturing on a subset of randomly selected participants. CONCLUSION Random forest predictions aided by hidden Markov modeling provided the best prediction of surgical maneuvers. Training of models across all users improved prediction accuracy by 10% compared with a random selection of participants. APPLICATION Marker-less video hand tracking can predict surgical maneuvers from a continuous video record with similar accuracy as robot-assisted surgical platforms, and may enable more efficient video review of surgical procedures for training and coaching.
Collapse
Affiliation(s)
| | - Yu Hen Hu
- University of Wisconsin-Madison, USA
| | | | | | | |
Collapse
|
4
|
Azari DP, Frasier LL, Quamme SRP, Greenberg CC, Pugh C, Greenberg JA, Radwin RG. Modeling Surgical Technical Skill Using Expert Assessment for Automated Computer Rating. Ann Surg 2019; 269:574-581. [PMID: 28885509 PMCID: PMC7412996 DOI: 10.1097/sla.0000000000002478] [Citation(s) in RCA: 42] [Impact Index Per Article: 8.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/01/2023]
Abstract
OBJECTIVE Computer vision was used to predict expert performance ratings from surgeon hand motions for tying and suturing tasks. SUMMARY BACKGROUND DATA Existing methods, including the objective structured assessment of technical skills (OSATS), have proven reliable, but do not readily discriminate at the task level. Computer vision may be used for evaluating distinct task performance throughout an operation. METHODS Open surgeries was videoed and surgeon hands were tracked without using sensors or markers. An expert panel of 3 attending surgeons rated tying and suturing video clips on continuous scales from 0 to 10 along 3 task measures adapted from the broader OSATS: motion economy, fluidity of motion, and tissue handling. Empirical models were developed to predict the expert consensus ratings based on the hand kinematic data records. RESULTS The predicted versus panel ratings for suturing had slopes from 0.73 to 1, and intercepts from 0.36 to 1.54 (Average R2 = 0.81). Predicted versus panel ratings for tying had slopes from 0.39 to 0.88, and intercepts from 0.79 to 4.36 (Average R2 = 0.57). The mean square error among predicted and expert ratings was consistently less than the mean squared difference among individual expert ratings and the eventual consensus ratings. CONCLUSIONS The computer algorithm consistently predicted the panel ratings of individual tasks, and were more objective and reliable than individual assessment by surgical experts.
Collapse
Affiliation(s)
- David P. Azari
- Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, WI
| | - Lane L. Frasier
- Wisconsin Surgical Outcomes Research (WiSOR) Program, Department of Surgery, University of Wisconsin-Madison, Madison, WI
| | | | - Caprice C. Greenberg
- Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, WI
- Wisconsin Surgical Outcomes Research (WiSOR) Program, Department of Surgery, University of Wisconsin-Madison, Madison, WI
| | - Carla Pugh
- Wisconsin Surgical Outcomes Research (WiSOR) Program, Department of Surgery, University of Wisconsin-Madison, Madison, WI
| | - Jacob A. Greenberg
- Wisconsin Surgical Outcomes Research (WiSOR) Program, Department of Surgery, University of Wisconsin-Madison, Madison, WI
| | - Robert G. Radwin
- Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, WI
- Department of Biomedical Engineering, University of Wisconsin-Madison, Madison, WI
| |
Collapse
|
5
|
Chen J, Lum PS. Pilot testing of the spring operated wearable enhancer for arm rehabilitation (SpringWear). J Neuroeng Rehabil 2018; 15:13. [PMID: 29499712 PMCID: PMC5833152 DOI: 10.1186/s12984-018-0352-4] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2017] [Accepted: 02/07/2018] [Indexed: 12/15/2022] Open
Abstract
Background Robotic devices for neurorehabilitation of movement impairments in persons with stroke have been studied extensively. However, the vast majority of these devices only allow practice of stereotyped components of simulated functional tasks in the clinic. Previously we developed SpringWear, a wearable, spring operated, upper extremity exoskeleton capable of assisting movements during real-life functional activities, potentially in the home. SpringWear assists shoulder flexion, elbow extension and forearm supination/pronation. The assistance profiles were designed to approximate the torque required to move the joint passively through its range. These three assisted DOF are combined with two passive shoulder DOF, allowing complex multi-joint movement patterns. Methods We performed a cross-sectional study to assess changes in movement patterns when assisted by SpringWear. Thirteen persons with chronic stroke performed range of motion (ROM) and functional tasks, including pick and place tasks with various objects. Sensors on the device measured rotation at all 5 DOF and a kinematic model calculated position of the wrist relative to the shoulder. Within subject t-tests were used to determine changes with assistance from SpringWear. Results Maximum shoulder flexion, elbow extension and forearm pronation/supination angles increased significantly during both ROM and functional tasks (p < 0.002). Elbow flexion/extension ROM also increased significantly (p < 0.001). When the subjects volitionally held up the arm against gravity, extension at the index finger proximal interphalangeal joint increased significantly (p = 0.033) when assisted by SpringWear. The forward reach workspace increased 19% (p = 0.002). Nine subjects could not complete the functional tasks unassisted and only one showed improvement on task completion with SpringWear. Conclusions SpringWear increased the usable workspace during reaching movements, but there was no consistent improvement in the ability to complete functional tasks. Assistance levels at the shoulder were increased only until the shoulder could be voluntarily held at 90 degrees of flexion. A higher level of assistance may have yielded better results. Also combining SpringWear with HandSOME, an exoskeleton for assisting hand opening, may yield the most dramatic improvements in functional task performance. These low-cost devices can potentially reduce effort and improve performance during task practice, increasing adherence to home training programs for rehabilitation.
Collapse
Affiliation(s)
- Ji Chen
- Department of Biomedical Engineering, Catholic University of America, Washington, DC, 20064, USA. .,Center for Applied Biomechanics and Rehabilitation Research MedStar National Rehabilitation Hospital, Washington, DC, 20010, USA. .,The Functional & Applied Biomechanics Section Rehabilitation Medicine Department Clinical Center, National Institutes of Health, Bethesda, MD, 20892, USA.
| | - Peter S Lum
- Department of Biomedical Engineering, Catholic University of America, Washington, DC, 20064, USA.,Center for Applied Biomechanics and Rehabilitation Research MedStar National Rehabilitation Hospital, Washington, DC, 20010, USA
| |
Collapse
|
6
|
Gallagher S, Schall MC. Musculoskeletal disorders as a fatigue failure process: evidence, implications and research needs. ERGONOMICS 2017; 60:255-269. [PMID: 27376409 DOI: 10.1080/00140139.2016.1208848] [Citation(s) in RCA: 70] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
Mounting evidence suggests that musculoskeletal disorders (MSDs) may be the result of a fatigue failure process in musculoskeletal tissues. Evaluations of MSD risk in epidemiological studies and current MSD risk assessment tools, however, have not yet incorporated important principles of fatigue failure analysis in their appraisals of MSD risk. This article examines the evidence suggesting that fatigue failure may play an important role in the aetiology of MSDs, assesses important implications with respect to MSD risk assessment and discusses research needs that may be required to advance the scientific community's ability to more effectively prevent the development of MSDs. Practitioner Summary: Evidence suggests that musculoskeletal disorders (MSDs) may result from a fatigue failure process. This article proposes a unifying framework that aims to explain why exposure to physical risk factors contributes to the development of work-related MSDs. Implications of that framework are discussed.
Collapse
Affiliation(s)
- Sean Gallagher
- a Industrial and Systems Engineering , Auburn University , Auburn , AL , USA
| | - Mark C Schall
- a Industrial and Systems Engineering , Auburn University , Auburn , AL , USA
| |
Collapse
|
7
|
Akkas O, Lee CH, Hu YH, Yen TY, Radwin RG. Measuring elemental time and duty cycle using automated video processing. ERGONOMICS 2016; 59:1514-1525. [PMID: 26848051 PMCID: PMC5226076 DOI: 10.1080/00140139.2016.1146347] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/11/2023]
Abstract
A marker-less 2D video algorithm measured hand kinematics (location, velocity and acceleration) in a paced repetitive laboratory task for varying hand activity levels (HAL). The decision tree (DT) algorithm identified the trajectory of the hand using spatiotemporal relationships during the exertion and rest states. The feature vector training (FVT) method utilised the k-nearest neighbourhood classifier, trained using a set of samples or the first cycle. The average duty cycle (DC) error using the DT algorithm was 2.7%. The FVT algorithm had an average 3.3% error when trained using the first cycle sample of each repetitive task, and had a 2.8% average error when trained using several representative repetitive cycles. Error for HAL was 0.1 for both algorithms, which was considered negligible. Elemental time, stratified by task and subject, were not statistically different from ground truth (p < 0.05). Both algorithms performed well for automatically measuring elapsed time, DC and HAL. Practitioner Summary: A completely automated approach for measuring elapsed time and DC was developed using marker-less video tracking and the tracked kinematic record. Such an approach is automatic, repeatable, objective and unobtrusive, and is suitable for evaluating repetitive exertions, muscle fatigue and manual tasks.
Collapse
Affiliation(s)
- Oguz Akkas
- Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, WI 53706, USA
| | - Cheng-Hsien Lee
- Department of Electrical and Computer Engineering, University of Wisconsin-Madison, Madison, WI 53706, USA
| | - Yu Hen Hu
- Department of Electrical and Computer Engineering, University of Wisconsin-Madison, Madison, WI 53706, USA
| | - Thomas Y. Yen
- Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, WI 53706, USA
| | - Robert G. Radwin
- Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, WI 53706, USA
- Corresponding author. Robert G. Radwin, PhD, Department of Industrial and Systems Engineering, 1550 Engineering Drive, Madison, WI 53706,
| |
Collapse
|
8
|
Azari DP, Pugh CM, Laufer S, Kwan C, Chen CH, Yen TY, Hu YH, Radwin RG. Evaluation of Simulated Clinical Breast Exam Motion Patterns Using Marker-Less Video Tracking. HUMAN FACTORS 2016; 58:427-440. [PMID: 26546381 PMCID: PMC4924820 DOI: 10.1177/0018720815613919] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/11/2015] [Accepted: 09/30/2015] [Indexed: 06/05/2023]
Abstract
OBJECTIVE This study investigates using marker-less video tracking to evaluate hands-on clinical skills during simulated clinical breast examinations (CBEs). BACKGROUND There are currently no standardized and widely accepted CBE screening techniques. METHODS Experienced physicians attending a national conference conducted simulated CBEs presenting different pathologies with distinct tumorous lesions. Single hand exam motion was recorded and analyzed using marker-less video tracking. Four kinematic measures were developed to describe temporal (time pressing and time searching) and spatial (area covered and distance explored) patterns. RESULTS Mean differences between time pressing, area covered, and distance explored varied across the simulated lesions. Exams were objectively categorized as either sporadic, localized, thorough, or efficient for both temporal and spatial categories based on spatiotemporal characteristics. The majority of trials were temporally or spatially thorough (78% and 91%), exhibiting proportionally greater time pressing and time searching (temporally thorough) and greater area probed with greater distance explored (spatially thorough). More efficient exams exhibited proportionally more time pressing with less time searching (temporally efficient) and greater area probed with less distance explored (spatially efficient). Just two (5.9 %) of the trials exhibited both high temporal and spatial efficiency. CONCLUSIONS Marker-less video tracking was used to discriminate different examination techniques and measure when an exam changes from general searching to specific probing. The majority of participants exhibited more thorough than efficient patterns. APPLICATION Marker-less video kinematic tracking may be useful for quantifying clinical skills for training and assessment.
Collapse
Affiliation(s)
| | | | | | - Calvin Kwan
- University of Wisconsin-Madison, Madison, WI
| | | | | | - Yu Hen Hu
- University of Wisconsin-Madison, Madison, WI
| | | |
Collapse
|