Street View Imagery (SVI) in the Built Environment: A Theoretical and Systematic Review

Open-source data pipeline for street-view images: A case study on community mobility during COVID-19 pandemic

Street View Images (SVI) are a common source of valuable data for researchers. Researchers have used SVI data for estimating pedestrian volumes, demographic surveillance, and to better understand built and natural environments in cityscapes. However, the most common source of publicly available SVI data is Google Street View. Google Street View images are collected infrequently, making temporal analysis challenging, especially in low population density areas. Our main contribution is the development of an open-source data pipeline for processing 360-degree video recorded from a car-mounted camera. The video data is used to generate SVIs, which then can be used as an input for longitudinal analysis. We demonstrate the use of the pipeline by collecting an SVI dataset over a 38-month longitudinal survey of Seattle, WA, USA during the COVID-19 pandemic. The output of our pipeline is validated through statistical analyses of pedestrian traffic in the images. We confirm known results in the literature and provide new insights into outdoor pedestrian traffic patterns. This study demonstrates the feasibility and value of collecting and using SVI for research purposes beyond what is possible with currently available SVI data. Our methods and dataset represent a first of its kind longitudinal collection and application of SVI data for research purposes. Limitations and future improvements to the data pipeline and case study are also discussed.

Multi-Task Classification for Improved Health Outcome Prediction Based on Environmental Indicators

This paper aims to address the challenges associated with evaluating the impact of neighborhood environments on health outcomes. Google street view (GSV) images provide a valuable tool for assessing neighborhood environments on a large scale. By annotating the GSV images with labels indicating the presence or absence of specific neighborhood features, we can develop classifiers capable of automatically analyzing and evaluating the environment. However, the process of labeling GSV images to analyze and evaluate the environment is a time-consuming and labor-intensive task. To overcome these challenges, we propose using a multi-task classifier to enhance the training of classifiers with limited supervised GSV data. Our multi-task classifier utilizes readily available, inexpensive online images collected from Flickr as a related classification task. The hypothesis is that a classifier trained on multiple related tasks is less likely to overfit to small amounts of training data and generalizes better to unseen data. We leverage the power of multiple related tasks to improve the classifier's overall performance and generalization capability. Here we show that, with the proposed learning paradigm, predicted labels for GSV test images are more accurate. Across different environment indicators, the accuracy, F1 score and balanced accuracy increase up to 6 % in the multi-task learning framework compared to its single-task learning counterpart. The enhanced accuracy of the predicted labels obtained through the multi-task classifier contributes to a more reliable and precise regression analysis determining the correlation between predicted built environment indicators and health outcomes. The R2 values calculated for different health outcomes improve by up to 4 % using multi-task learning detected indicators.

Computer Vision Technology for Monitoring of Indoor and Outdoor Environments and HVAC Equipment: A Review

Artificial intelligence technologies such as computer vision (CV), machine learning, Internet of Things (IoT), and robotics have advanced rapidly in recent years. The new technologies provide non-contact measurements in three areas: indoor environmental monitoring, outdoor environ-mental monitoring, and equipment monitoring. This paper summarizes the specific applications of non-contact measurement based on infrared images and visible images in the areas of personnel skin temperature, position posture, the urban physical environment, building construction safety, and equipment operation status. At the same time, the challenges and opportunities associated with the application of CV technology are anticipated.

An AI-based framework for studying visual diversity of urban neighborhoods and its relationship with socio-demographic variables

This study presents a framework to study quantitatively geographical visual diversities of urban neighborhood from a large collection of street-view images using an Artificial Intelligence (AI)-based image segmentation technique. A variety of diversity indices are computed from the extracted visual semantics. They are utilized to discover the relationships between urban visual appearance and socio-demographic variables. This study also validates the reliability of the method with human evaluators. The methodology and results obtained from this study can potentially be used to study urban features, locate houses, establish services, and better operate municipalities.

Using Convolutional Neural Networks to Derive Neighborhood Built Environments from Google Street View Images and Examine Their Associations with Health Outcomes

Built environment neighborhood characteristics are difficult to measure and assess on a large scale. Consequently, there is a lack of sufficient data that can help us investigate neighborhood characteristics as structural determinants of health on a national level. The objective of this study is to utilize publicly available Google Street View images as a data source for characterizing built environments and to examine the influence of built environments on chronic diseases and health behaviors in the United States. Data were collected by processing 164 million Google Street View images from November 2019 across the United States. Convolutional Neural Networks, a class of multi-layer deep neural networks, were used to extract features of the built environment. Validation analyses found accuracies of 82% or higher across neighborhood characteristics. In regression analyses controlling for census tract sociodemographics, we find that single-lane roads (an indicator of lower urban development) were linked with chronic conditions and worse mental health. Walkability and urbanicity indicators such as crosswalks, sidewalks, and two or more cars were associated with better health, including reduction in depression, obesity, high blood pressure, and high cholesterol. Street signs and streetlights were also found to be associated with decreased chronic conditions. Chain link fence (physical disorder indicator) was generally associated with poorer mental health. Living in neighborhoods with a built environment that supports social interaction and physical activity can lead to positive health outcomes. Computer vision models using manually annotated Google Street View images as a training dataset were able to accurately identify neighborhood built environment characteristics. These methods increases the feasibility, scale, and efficiency of neighborhood studies on health.