Predicting the effect of street environment on residents' mood states in large urban areas using machine learning and street view images

Perceptible landscape patterns reveal invisible socioeconomic profiles of cities

Urban landscape is directly perceived by residents and is a significant symbol of urbanization development. A comprehensive assessment of urban landscapes is crucial for guiding the development of inclusive, resilient, and sustainable cities and human settlements. Previous studies have primarily analyzed two-dimensional landscape indicators derived from satellite remote sensing, potentially overlooking the valuable insights provided by the three-dimensional configuration of landscapes. This limitation arises from the high cost of acquiring large-area three-dimensional data and the lack of effective assessment indicators. Here, we propose four urban landscapes indicators in three dimensions (UL3D): greenness, grayness, openness, and crowding. We construct the UL3D using 4.03 million street view images from 303 major cities in China, employing a deep learning approach. We combine urban background and two-dimensional urban landscape indicators with UL3D to predict the socioeconomic profiles of cities. The results show that UL3D indicators differs from two-dimensional landscape indicators, with a low average correlation coefficient of 0.31 between them. Urban landscapes had a changing point in 2018-2019 due to new urbanization initiatives, with grayness and crowding rates slowing, while openness increased. The incorporation of UL3D indicators significantly enhances the explanatory power of the regression model for predicting socioeconomic profiles. Specifically, GDP per capita, urban population rate, built-up area per capita, and hospital count correspond to improvements of 25.0%, 19.8%, 35.5%, and 19.2%, respectively. These findings indicate that UL3D indicators have the potential to reflect the socioeconomic profiles of cities.

Assessing the Impact of Urban Environments on Mental Health and Perception Using Deep Learning: A Review and Text Mining Analysis

Understanding how outdoor environments affect mental health outcomes is vital in today's fast-paced and urbanized society. Recently, advancements in data-gathering technologies and deep learning have facilitated the study of the relationship between the outdoor environment and human perception. In a systematic review, we investigate how deep learning techniques can shed light on a better understanding of the influence of outdoor environments on human perceptions and emotions, with an emphasis on mental health outcomes. We have systematically reviewed 40 articles published in SCOPUS and the Web of Science databases which were the published papers between 2016 and 2023. The study presents and utilizes a novel topic modeling method to identify coherent keywords. By extracting the top words of each research topic, and identifying the current topics, we indicate that current studies are classified into three areas. The first topic was "Urban Perception and Environmental Factors" where the studies aimed to evaluate perceptions and mental health outcomes. Within this topic, the studies were divided based on human emotions, mood, stress, and urban features impacts. The second topic was titled "Data Analysis and Urban Imagery in Modeling" which focused on refining deep learning techniques, data collection methods, and participants' variability to understand human perceptions more accurately. The last topic was named "Greenery and visual exposure in urban spaces" which focused on the impact of the amount and the exposure of green features on mental health and perceptions. Upon reviewing the papers, this study provides a guide for subsequent research to enhance the view of using deep learning techniques to understand how urban environments influence mental health. It also provides various suggestions that should be taken into account when planning outdoor spaces.

Prediction of driving stress on high-altitude expressway using driving environment features: A naturalistic driving study in Tibet

OBJECTIVE

Owing to the harsh environment in high-altitude areas, drivers experience significant driving stress. Compared with urban roads or expressways in low-altitude areas, the driving environment in high-altitude areas has distinct features, including mountainous environments and a higher proportion of trucks and buses. This study aims to investigate the feasibility of predicting stress levels through elements in the driving environment.

METHODS

Naturalistic driving tests were conducted on an expressway in Tibet. Driving stress was assessed using heart rate variability (HRV)-based indicators and classified using K-means clustering. A DeepLabv3 model was built to conduct semantic segmentation and extract environment elements from the driving scenarios recorded through a camera next to the driver's eyes. A decision tree and 4 other ensemble learning models based on decision trees were built to predict driving stress levels using the environment elements.

RESULTS

Fifty-six indicators were extracted from the driving environment. Results of the prediction models demonstrate that extreme gradient boosting has the best overall performance with the F1 score (harmonic mean of the precision and recall) and G-mean (geometric mean of sensitivity and specificity) reaching 0.855 and 0.890, respectively. Indicators based on the variation rate of trucks and buses have high feature importance and exhibit positive effects on driving stress. Indicators reflecting the proportion of mountain, road, and sky features negatively affect the expected levels of driving stress. Additionally, the mountain feature demonstrates multidimensional effects, because driving stress is positively affected by indicators of the variation rate for mountain elements.

CONCLUSIONS

This study validates the prediction of driving stress using environment elements in the driver's field of view and extends its application to high-altitude expressways with distinct environmental characteristics. This method provides a real-time, less intrusive, and safer method of driving stress assessment and prediction and also enhances the understanding of the environmental determinants of driving stress. The results hold promising applications, including the development of a driving state assessment and warning module as well as the identification of high-risk road sections and implementation of control measures.

Integrating street view images and deep learning to explore the association between human perceptions of the built environment and cardiovascular disease in older adults

Understanding how built environment attributes affect health remains important. While many studies have explored the objective characteristics of built environments that affect health outcomes, few have examined the role of human perceptions of built environments on physical health. Baidu Street View images and computer vision technological advances have helped researchers overcome the constraints of traditional methods of measuring human perceptions (e.g., these methods are laborious, time-consuming, and costly), allowing for large-scale measurements of human perceptions. This study estimated human perceptions of the built environment (e.g., beauty, boredom, depression, safety, vitality, and wealth) by adopting Baidu Street View images and deep learning algorithms. Negative binomial regression models were employed to analyze the relationship between human perceptions and cardiovascular disease in older adults (e.g., ischemic heart disease and cerebrovascular disease). The results indicated that wealth perception is negatively related to the risk of cardiovascular disease. However, depression and vitality perceptions are positively associated with the risk of cardiovascular disease. Furthermore, we found no relationship between beauty, boredom, safety perceptions, and the risk of cardiovascular disease. Our findings highlight the importance of human perceptions in the development of healthy city planning and facilitate a comprehensive understanding of the relationship between built environment characteristics and health outcomes in older adults. They also demonstrate that street view images have the potential to provide insights into this complicated issue, assisting in the formulation of refined interventions and health policies.

Reproduction, cultural symbolism, and online relationship: Constructing city spatial imagery on TikTok

The city on social media has become a hot topic in the study of city communication and city image. From the perspective of spatial theory and the communication characteristics of social media, this paper divides the spatial imagery of TikTok into three spaces: material space-cognitive attention, mental space-mental association, and relational space-emotional involvement. Based on the content analysis of 40 videos, we analyze the process of social media using cognition, association, and emotion as the starting points to increase the material space, expand the mental space, and expand the relational space. We find that spatial imagery can be co-constructed from the material space, mental space, and relational space. Lastly, the model is changed, and the value of using spatial theory to understand how city images are made is talked about.

The effect of the street environment on two types of essential physical activity in industrial neighborhoods from the perspective of public health: a study from the Harbin low-income population health survey, China

A large number of low-income residents in industrial neighborhoods rarely engage in recreational and physical activities in green spaces in extremely cold weather. This study mainly explores the relationship between the street environment and physical activities under special industrial properties and extreme cold weather conditions. In addition, we further divide essential physical activity into two categories, life-type and traffic-type physical activity, to explore and refine the related studies.We use principal component analysis to classify the street environment indicators and use multiple linear regression to analyze the impact of each indicator on different physical activities. The conclusions are as follows. For low-income people, the street environment in industrial neighborhoods has a much greater impact on life-type physical activity than traffic-type physical activity, and there is a conflict between the two. In addition, a high greening density is not conducive to either type of physical activity in the street environment. It reduces the paved area of streets and create sports conflicts between people undertaking different physical activities. The findings contribute to the development and optimization of public health research on environmental interventions in industrial neighborhood streets and enable effective recommendations for increasing outside physical activity among low-income people in severe weather conditions. In future studies, we will use the physical environment as a mediator to explore the relationship between the street environment and high-frequency chronic diseases in old industrial neighborhoods.

Measuring and modelling perceptions of the built environment for epidemiological research using crowd-sourcing and image-based deep learning models

BACKGROUND

Perceptions of the built environment, such as nature quality, beauty, relaxation, and safety, may be key factors linking the built environment to human health. However, few studies have examined these types of perceptions due to the difficulty in quantifying them objectively in large populations.

OBJECTIVE

To measure and predict perceptions of the built environment from street-view images using crowd-sourced methods and deep learning models for application in epidemiologic studies.

METHODS

We used the Amazon Mechanical-Turk crowdsourcing platform where participants compared two street-view images and quantified perceptions of nature quality, beauty, relaxation, and safety. We optimized street-view image sampling methods to improve the quality and resulting perception data specific to participants enrolled in the Washington State Twin Registry (WSTR) health study. We used a transfer learning approach to train deep learning models by leveraging existing image perception data from the PlacePulse 2.0 dataset, which includes 1.1 million image comparisons, and refining based on new WSTR perception data. Resulting models were applied to WSTR addresses to estimate exposures and evaluate associations with traditional built environment measures.

RESULTS

We collected over 36,000 image comparisons and calculated perception measures for each image. Our final deep learning models explained 77.6% of nature quality, 68.1% of beauty, 72.0% of relaxation, and 64.7% of safety in pairwise image comparisons. Applying transfer learning with the new perception labels specific to the WSTR yielded an average improvement of 3.8% for model performance. Perception measures were weakly to moderately correlated with traditional built environment exposures for WSTR participant addresses; for example, nature quality and NDVI (r = 0.55), neighborhood area deprivation (r = -0.16), and walkability (r = -0.20), respectively.

SIGNIFICANCE

We were able to measure and model perceptions of the built environment optimized for a specific health study. Future applications will examine associations between these exposure measures and mental health in the WSTR.

IMPACT STATEMENT

Built environments influence health through complex pathways. Perceptions of nature quality, beauty, relaxation and safety may be particularly import for understanding these linkages, but few studies to-date have examined these perceptions objectively for large populations. For quantitative research, an exposure measure must be reproducible, accurate, and precise--here we work to develop such measures for perceptions of the urban environment. We created crowd-sourced and image-based deep learning methods that were able to measure and model these perceptions. Future applications will apply these models to examine associations with mental health in the Washington State Twin Registry.

Continuous Neurophysiologic Data Accurately Predict Mood and Energy in the Elderly

The elderly have an elevated risk of clinical depression because of isolation from family and friends and a reticence to report their emotional states. The present study explored whether data from a commercial neuroscience platform could predict low mood and low energy in members of a retirement community. Neurophysiologic data were collected continuously for three weeks at 1Hz and averaged into hourly and daily measures, while mood and energy were captured with self-reports. Two neurophysiologic measures averaged over a day predicted low mood and low energy with 68% and 75% accuracy. Principal components analysis showed that neurologic variables were statistically associated with mood and energy two days in advance. Applying machine learning to hourly data classified low mood and low energy with 99% and 98% accuracy. Two-day lagged hourly neurophysiologic data predicted low mood and low energy with 98% and 96% accuracy. This study demonstrates that continuous measurement of neurophysiologic variables may be an effective way to reduce the incidence of mood disorders in vulnerable people by identifying when interventions are needed.