Assessing physical activity intensity by video analysis

Design of public open space: Site features, playing, and physical activity

Not enough studies have examined how specific design features of public open space, such as movable site features, are associated with people's physical activity level or playfulness. To fill this gap, this study uses deep learning-based methods to extract visitors' movement trajectories (n = 18,592) from a time-lapse video of a promenade in Hong Kong. The trajectories are classified into different groups based on a set of movement indicators. Multinomial logistic regression is used to examine the relationship between trajectory types and the level of interaction with different site features. A one-way analysis of variance (ANOVA) is also used to compare the average amount of physical activity among different trajectory types. The results show that interaction with semi-fixed or movable site features is associated with higher odds of people having "playful" trajectories than other types of trajectories. People with "sporty" trajectories and "playful" trajectories on average have the highest amount of physical activity.

Automated High-Frequency Observations of Physical Activity Using Computer Vision

PURPOSE

To test the validity of the Ecological Video Identification of Physical Activity (EVIP) computer vision algorithms for automated video-based ecological assessment of physical activity in settings such as parks and schoolyards.

METHODS

Twenty-seven hours of video were collected from stationary overhead video cameras across 22 visits in nine sites capturing organized activities. Each person in the setting wore an accelerometer, and each second was classified as moderate-to-vigorous physical activity or sedentary/light activity. Data with 57,987 s were used to train and test computer vision algorithms for estimating the total number of people in the video and number of people active (in moderate-to-vigorous physical activity) each second. In the testing data set (38,658 s), video-based System for Observing Play and Recreation in Communities (SOPARC) observations were conducted every 5 min (130 observations). Concordance correlation coefficients (CCC) and mean absolute errors (MAE) assessed agreement between (1) EVIP and ground truth (people counts+accelerometry) and (2) SOPARC observation and ground truth. Site and scene-level correlates of error were investigated.

RESULTS

Agreement between EVIP and ground truth was high for number of people in the scene (CCC = 0.88; MAE = 2.70) and moderate for number of people active (CCC = 0.55; MAE = 2.57). The EVIP error was uncorrelated with camera placement, presence of obstructions or shadows, and setting type. For both number in scene and number active, EVIP outperformed SOPARC observations in estimating ground truth values (CCC were larger by 0.11-0.12 and MAE smaller by 41%-48%).

CONCLUSIONS

Computer vision algorithms are promising for automated assessment of setting-based physical activity. Such tools would require less manpower than human observation, produce more and potentially more accurate data, and allow for ongoing monitoring and feedback to inform interventions.

Complementing subjective with objective data in analysing expertise: A machine-learning approach applied to badminton

This study aimed to assess which combination of subjective and empirical data might help to identify the expertise level. A group of 10 expert coaches classified 40 participants in 5 different expertise groups based on the video footage of the rallies. The expertise levels were determined using a typology based on a continuum of 5 conative stages: (1) structural, (2) functional, (3) technical, (4) contextual, and (5) expertise. The video allowed empirical measurement of the duration of the rallies, and tri-axial accelerometers measured the intensity of the player's involvement. A principal component analysis showed that two dimensions explained 54.9% of the total variance in the data and that conative stage and empirical parameters during rallies (duration, intensity of the game) were correlated with axis 1, whereas duration and acceleration data between rallies were correlated with axis 2. A random forest algorithm showed that among the parameters considered, acceleration, duration of the rallies, and time between rallies could predict conative stages with a prediction accuracy above possibility. This study suggests that performance analysis benefits from the confrontation of subjective and objective data in order to design training plans according to the expertise level of the participants.

Quantifying Physical Activity in Young Children Using a Three-Dimensional Camera

The purpose of this study was to determine the feasibility and validity of using three-dimensional (3D) video data and computer vision to estimate physical activity intensities in young children. Families with children (2–5-years-old) were invited to participate in semi-structured 20-minute play sessions that included a range of indoor play activities. During the play session, children’s physical activity (PA) was recorded using a 3D camera. PA video data were analyzed via direct observation, and 3D PA video data were processed and converted into triaxial PA accelerations using computer vision. PA video data from children (n = 10) were analyzed using direct observation as the ground truth, and the Receiver Operating Characteristic Area Under the Curve (AUC) was calculated in order to determine the classification accuracy of a Classification and Regression Tree (CART) algorithm for estimating PA intensity from video data. A CART algorithm accurately estimated the proportion of time that children spent sedentary (AUC = 0.89) in light PA (AUC = 0.87) and moderate-vigorous PA (AUC = 0.92) during the play session, and there were no significant differences (p > 0.05) between the directly observed and CART-determined proportions of time spent in each activity intensity. A computer vision algorithm and 3D camera can be used to estimate the proportion of time that children spend in all activity intensities indoors.

Assessing Physical Activities Occurring on Sidewalks and Streets: Protocol for a Cross-Sectional Study

Background

A considerable proportion of outdoor physical activity (PA) is done on sidewalks and streets, necessitating the development of a reliable measure of PA performed in these settings. The Block Walk Method (BWM) is one of the more common approaches for this purpose. Although it utilizes reliable observation techniques and displays criterion validity, it remains relatively unchanged since its introduction in 2006. It is a nontechnical, labor-intensive, first generation method. Advancing the BWM would contribute significantly to our understanding of PA behavior.

Objective

This study will develop and test a new BWM that utilizes a wearable video device (WVD) and computer video analysis to assess PAs performed on sidewalks and streets. The specific aims are to improve the BWM by incorporating a WVD (eyeglasses with a high-definition video camera in the frame) into the methodology and advance this WVD-enhanced BWM by applying machine learning and recognition software to automatically extract information on PAs occurring on the sidewalks and streets from the videos.

Methods

Trained observers (1 wearing and 1 not wearing the WVD) will walk together at a set pace along predetermined 1000 ft sidewalk and street observation routes representing low, medium, and high walkable areas. During the walks, the non-WVD observer will use the traditional BWM to record the numbers of individuals standing, sitting, walking, biking, and running in observation fields along the routes. The WVD observer will continuously video the observation fields. Later, 2 investigators will view the videos to determine the number of individuals performing PAs in the observation fields. The video data will then be analyzed automatically using multiple deep convolutional neural networks (CNNs) to determine the number of humans in the observation fields and the type of PAs performed. Bland Altman methods and intraclass correlation coefficients (ICCs) will be used to assess agreement. Potential sources of error such as occlusions (eg, trees) will be assessed using moderator analyses.

Results

Outcomes from this study are pending; however, preliminary studies supporting the research protocol indicate that the BWM is reliable for determining the PA mode (Cramer V=.89; P<.001), the address where the PA occurred (Cohen kappa=.85; P<.001), and the number engaged in an observed PA (ICC=.85; P<.001). The number of individuals seen walking along routes was correlated with several environmental characteristics such as sidewalk quality (r=.39; P=.02) and neighborhood aesthetics (r=.49; P<.001). Furthermore, we have used CNNs to detect cars, bikes, and pedestrians as well as individuals using park facilities.

Conclusions

We expect the new approach will enhance measurement accuracy while reducing the burden of data collection. In the future, the capabilities of the WVD-CNN system will be expanded to allow for the determination of other characteristics captured in videos such as caloric expenditure and environmental conditions.

International Registered Report Identifier (IRRID)

PRR1-10.2196/12976

Validation of the Block Walk Method for Assessing Physical Activity occurring on Sidewalks/Streets

The block walk method (BWM) is one of the more common approaches for assessing physical activity (PA) performed on sidewalks/streets; however, it is non-technical, labor-intensive, and lacks validation. This study aimed to validate the BWM and examine the potential for using a wearable video device (WVD) to assess PA occurring on sidewalks/streets. Trained observers (one wearing and one not wearing the WVD) walked together and performed the BWM according to a previously developed protocol along routes in low, medium, and high walkable areas. Two experts then reviewed the videos. A total of 1150 (traditional) and 1087 (video review) individuals were observed during 900 min of observation. When larger numbers of individuals were observed, the traditional method overestimated the overall number of people as well as those walking and sitting/standing, while underestimating the number of runners. Valid estimates of PA occurring on sidewalks/streets can be obtained by the traditional BWM in low and medium walkability areas and/or with non-common activities (cycling); however, its validity is questionable when sidewalks/streets use volume is high. The use of WVDs in PA assessment has the potential to establish new levels of accuracy, reduce resource requirements, and open up the possibility for retrospective analysis.

Quantitative Contact-Less Estimation of Energy Expenditure from Video and 3D Imagery

Measurement of energy expenditure is an important tool in sport science and medicine, especially when trying to estimate the extent and intensity of physical activity. However, most approaches still rely on sensors or markers, placed directly on the body. In this paper, we present a novel approach using a fully contact-less, fully automatic method, that relies on computer vision algorithms and widely available and inexpensive imaging sensors. We rely on the estimation of the optical and scene flow to calculate Histograms of Oriented Optical Flow (HOOF) descriptors, which we subsequently augment with the Histograms of Absolute Flow Amplitude (HAFA). Descriptors are fed into regression model, which allows us to estimate energy consumption, and to a lesser extent, the heart rate. Our method has been tested both in lab environment and in realistic conditions of a sport match. Results confirm that these energy expenditures could be derived from purely contact-less observations. The proposed method can be used with different modalities, including near infrared imagery, which extends its future potential.

Automated Ecological Assessment of Physical Activity: Advancing Direct Observation

Technological advances provide opportunities for automating direct observations of physical activity, which allow for continuous monitoring and feedback. This pilot study evaluated the initial validity of computer vision algorithms for ecological assessment of physical activity. The sample comprised 6630 seconds per camera (three cameras in total) of video capturing up to nine participants engaged in sitting, standing, walking, and jogging in an open outdoor space while wearing accelerometers. Computer vision algorithms were developed to assess the number and proportion of people in sedentary, light, moderate, and vigorous activity, and group-based metabolic equivalents of tasks (MET)-minutes. Means and standard deviations (SD) of bias/difference values, and intraclass correlation coefficients (ICC) assessed the criterion validity compared to accelerometry separately for each camera. The number and proportion of participants sedentary and in moderate-to-vigorous physical activity (MVPA) had small biases (within 20% of the criterion mean) and the ICCs were excellent (0.82–0.98). Total MET-minutes were slightly underestimated by 9.3–17.1% and the ICCs were good (0.68–0.79). The standard deviations of the bias estimates were moderate-to-large relative to the means. The computer vision algorithms appeared to have acceptable sample-level validity (i.e., across a sample of time intervals) and are promising for automated ecological assessment of activity in open outdoor settings, but further development and testing is needed before such tools can be used in a diverse range of settings.