Recognition of Activities of Daily Living and Environments Using Acoustic Sensors Embedded on Mobile Devices

Intelligent ADL Recognition via IoT-Based Multimodal Deep Learning Framework

Smart home monitoring systems via internet of things (IoT) are required for taking care of elders at home. They provide the flexibility of monitoring elders remotely for their families and caregivers. Activities of daily living are an efficient way to effectively monitor elderly people at home and patients at caregiving facilities. The monitoring of such actions depends largely on IoT-based devices, either wireless or installed at different places. This paper proposes an effective and robust layered architecture using multisensory devices to recognize the activities of daily living from anywhere. Multimodality refers to the sensory devices of multiple types working together to achieve the objective of remote monitoring. Therefore, the proposed multimodal-based approach includes IoT devices, such as wearable inertial sensors and videos recorded during daily routines, fused together. The data from these multi-sensors have to be processed through a pre-processing layer through different stages, such as data filtration, segmentation, landmark detection, and 2D stick model. In next layer called the features processing, we have extracted, fused, and optimized different features from multimodal sensors. The final layer, called classification, has been utilized to recognize the activities of daily living via a deep learning technique known as convolutional neural network. It is observed from the proposed IoT-based multimodal layered system's results that an acceptable mean accuracy rate of 84.14% has been achieved.

Interdisciplinary perspectives on privacy awareness in lifelogging technology development

Population aging resulting from demographic changes requires some challenging decisions and necessary steps to be taken by different stakeholders to manage current and future demand for assistance and support. The consequences of population aging can be mitigated to some extent by assisting technologies that can support the autonomous living of older individuals and persons in need of care in their private environments as long as possible. A variety of technical solutions are already available on the market, but privacy protection is a serious, often neglected, issue when using such (assisting) technology. Thus, privacy needs to be thoroughly taken under consideration in this context. In a three-year project PAAL ('Privacy-Aware and Acceptable Lifelogging Services for Older and Frail People'), researchers from different disciplines, such as law, rehabilitation, human-computer interaction, and computer science, investigated the phenomenon of privacy when using assistive lifelogging technologies. In concrete terms, the concept of Privacy by Design was realized using two exemplary lifelogging applications in private and professional environments. A user-centered empirical approach was applied to the lifelogging technologies, investigating the perceptions and attitudes of (older) users with different health-related and biographical profiles. The knowledge gained through the interdisciplinary collaboration can improve the implementation and optimization of assistive applications. In this paper, partners of the PAAL project present insights gained from their cross-national, interdisciplinary work regarding privacy-aware and acceptable lifelogging technologies.

Daily motionless activities: A dataset with accelerometer, magnetometer, gyroscope, environment, and GPS data

The dataset presented in this paper presents a dataset related to three motionless activities, including driving, watching TV, and sleeping. During these activities, the mobile device may be positioned in different locations, including the pants pockets, in a wristband, over the bedside table, on a table, inside the car, or on other furniture, for the acquisition of accelerometer, magnetometer, gyroscope, GPS, and microphone data. The data was collected by 25 individuals (15 men and 10 women) in different environments in Covilhã and Fundão municipalities (Portugal). The dataset includes the sensors' captures related to a minimum of 2000 captures for each motionless activity, which corresponds to 2.8 h (approximately) for each one. This dataset includes 8.4 h (approximately) of captures for further analysis with data processing techniques, and machine learning methods. It will be useful for the complementary creation of a robust method for the identification of these type of activities.

Comparison of Pre-Trained CNNs for Audio Classification Using Transfer Learning

The paper investigates retraining options and the performance of pre-trained Convolutional Neural Networks (CNNs) for sound classification. CNNs were initially designed for image classification and recognition, and, at a second phase, they extended towards sound classification. Transfer learning is a promising paradigm, retraining already trained networks upon different datasets. We selected three ‘Image’- and two ‘Sound’-trained CNNs, namely, GoogLeNet, SqueezeNet, ShuffleNet, VGGish, and YAMNet, and applied transfer learning. We explored the influence of key retraining parameters, including the optimizer, the mini-batch size, the learning rate, and the number of epochs, on the classification accuracy and the processing time needed in terms of sound preprocessing for the preparation of the scalograms and spectrograms as well as CNN training. The UrbanSound8K, ESC-10, and Air Compressor open sound datasets were employed. Using a two-fold criterion based on classification accuracy and time needed, we selected the ‘champion’ transfer-learning parameter combinations, discussed the consistency of the classification results, and explored possible benefits from fusing the classification estimations. The Sound CNNs achieved better classification accuracy, reaching an average of 96.4% for UrbanSound8K, 91.25% for ESC-10, and 100% for the Air Compressor dataset.

Wearable energy efficient fitness tracker for sports person health monitoring application

Continuous evaluation of biological and physiological metrics of sports personalities, evaluating general health status, and alerting for life-saving treatments, is supposed to enhance efficiency and healthy performance. Wearable devices with acceptable form factors compact, flexibility, minimal power consumption, etc., are needed for continuous monitoring to avoid affecting everyday operations, thereby retaining functional effectiveness and consumer satisfaction. This research focuses on the acceleration tracker for particularizing the work. Acceleration data is typically collected on battery-powered sensors for activity detection, referring to an exchange between high-precision detection and energy-efficient processing. From a feature selection perspective, the paper explores this trade-off. It suggests an Energy-Efficient Behavior Recognition System with a comprehensive energy utilization model and the Multi-objective Algorithm of Particle Swarm Optimization (EEBRS-MPSO). Therefore, using Random Forest (RF) classifiers, the model and algorithm are tested to measure the precision of identification and obtain the task’s best performance with the lowest energy consumption, among other biologically-inspired algorithms. The findings indicate that energy consumption for data storage and data processing is minimized with magnitude relative to the raw data method by choosing suitable groups of attributes. Thus, the platform allows a scalable range of feature clusters that require the authors to provide an adequate power adjustment for given target use.

Improving Human Activity Monitoring by Imputation of Missing Sensory Data: Experimental Study

The automatic recognition of human activities with sensors available in off-the-shelf mobile devices has been the subject of different research studies in recent years. It may be useful for the monitoring of elderly people to present warning situations, monitoring the activity of sports people, and other possibilities. However, the acquisition of the data from different sensors may fail for different reasons, and the human activities are recognized with better accuracy if the different datasets are fulfilled. This paper focused on two stages of a system for the recognition of human activities: data imputation and data classification. Regarding the data imputation, a methodology for extrapolating the missing samples of a dataset to better recognize the human activities was proposed. The K-Nearest Neighbors (KNN) imputation technique was used to extrapolate the missing samples in dataset captures. Regarding the data classification, the accuracy of the previously implemented method, i.e., Deep Neural Networks (DNN) with normalized and non-normalized data, was improved in relation to the previous results without data imputation.

Analysis of the Results of Heel-Rise Test with Sensors: A Systematic Review

Strokes are a constant concern for people and pose a major health concern. Tests that allow detection and the rehabilitation of patients have started to become more important and essential. There are several tests used by physiotherapists to speed up the recovery process of patients. This article presents a systematic review of existing studies using the Heel-Rise Test and sensors (i.e., accelerometers, gyroscopes, pressure and tilt sensors) to estimate the different levels and health statuses of individuals. It was found that the most measured parameter was related to the number of repetitions, and the maximum number of repetitions for a healthy adult is 25 repetitions. As for future work, the implementation of these methods with a simple mobile device will facilitate the different measurements on this subject.

Measurement of Results of Functional Reach Test with Sensors: A Systematic Review

The test of physical conditions is important to treat and presents several diseases related to the movement. These diseases are mainly related to the physiotherapy and orthopedy, but it can be applied in a wide range of medical specialties. The Functional Reach Test is one of the most common physical tests used to measure the limit of stability that is highly important for older adults because their stability is reduced with aging. Thus, older adults are part of the population more exposed to stroke. This test may help in the measurement of the conditions related to post-stroke and stroke treatment. The movements related to this test may be recorded and recognized with the inertial sensors available in off-the-shelf mobile devices. This systematic review aims to determine how to determine the conditions related to this test, which can be detected, and which of the sensors are used for this purpose. The main contribution of this paper is to present the research on the state-of-the-art use of sensors available on off-the-shelf mobile devices to measure Functional Reach Test results. This research shows that the sensors that are used in the literature studies are inertial sensors and force sensors. The features extracted from the different studies are categorized as dynamic balance, quantitative, and raw statistics. These features are mainly used to recognize the different parameters of the test, and several accidents, including falling. The execution of this test may allow the early detection of different diseases.

Machine Learning Techniques with ECG and EEG Data: An Exploratory Study

Electrocardiography (ECG) and electroencephalography (EEG) are powerful tools in medicine for the analysis of various diseases. The emergence of affordable ECG and EEG sensors and ubiquitous mobile devices provides an opportunity to make such analysis accessible to everyone. In this paper, we propose the implementation of a neural network-based method for the automatic identification of the relationship between the previously known conditions of older adults and the different features calculated from the various signals. The data were collected using a smartphone and low-cost ECG and EEG sensors during the performance of the timed-up and go test. Different patterns related to the features extracted, such as heart rate, heart rate variability, average QRS amplitude, average R-R interval, and average R-S interval from ECG data, and the frequency and variability from the EEG data were identified. A combination of these parameters allowed us to identify the presence of certain diseases accurately. The analysis revealed that the different institutions and ages were mainly identified. Still, the various diseases and groups of diseases were difficult to recognize, because the frequency of the different diseases was rare in the considered population. Therefore, the test should be performed with more people to achieve better results.

Mobile Computing Technologies for Health and Mobility Assessment: Research Design and Results of the Timed Up and Go Test in Older Adults

Due to the increasing age of the European population, there is a growing interest in performing research that will aid in the timely and unobtrusive detection of emerging diseases. For such tasks, mobile devices have several sensors, facilitating the acquisition of diverse data. This study focuses on the analysis of the data collected from the mobile devices sensors and a pressure sensor connected to a Bitalino device for the measurement of the Timed-Up and Go test. The data acquisition was performed within different environments from multiple individuals with distinct types of diseases. Then this data was analyzed to estimate the various parameters of the Timed-Up and Go test. Firstly, the pressure sensor is used to extract the reaction and total test time. Secondly, the magnetometer sensors are used to identify the total test time and different parameters related to turning around. Finally, the accelerometer sensor is used to extract the reaction time, total test time, duration of turning around, going time, return time, and many other derived metrics. Our experiments showed that these parameters could be automatically and reliably detected with a mobile device. Moreover, we identified that the time to perform the Timed-Up and Go test increases with age and the presence of diseases related to locomotion.

Smart Environments and Social Robots for Age-Friendly Integrated Care Services

The world is facing major societal challenges because of an aging population that is putting increasing pressure on the sustainability of care. While demand for care and social services is steadily increasing, the supply is constrained by the decreasing workforce. The development of smart, physical, social and age-friendly environments is identified by World Health Organization (WHO) as a key intervention point for enabling older adults, enabling them to remain as much possible in their residences, delay institutionalization, and ultimately, improve quality of life. In this study, we survey smart environments, machine learning and robot assistive technologies that can offer support for the independent living of older adults and provide age-friendly care services. We describe two examples of integrated care services that are using assistive technologies in innovative ways to assess and deliver of timely interventions for polypharmacy management and for social and cognitive activity support in older adults. We describe the architectural views of these services, focusing on details about technology usage, end-user interaction flows and data models that are developed or enhanced to achieve the envisioned objective of healthier, safer, more independent and socially connected older people.

Promotion of Healthy Nutrition and Physical Activity Lifestyles for Teenagers: A Systematic Literature Review of The Current Methodologies

Amid obesity problems in the young population and apparent trends of spending a significant amount of time in a stationary position, promoting healthy nutrition and physical activities to teenagers is becoming increasingly important. It can rely on different methodologies, including a paper diary and mobile applications. However, the widespread use of mobile applications by teenagers suggests that they could be a more suitable tool for this purpose. This paper reviews the methodologies for promoting physical activities to healthy teenagers explored in different studies, excluding the analysis of different diseases. We found only nine studies working with teenagers and mobile applications to promote active lifestyles, including the focus on nutrition and physical activity. Studies report using different techniques to captivate the teenagers, including questionnaires and gamification techniques. We identified the common features used in different studies, which are: paper diary, diet diary, exercise diary, notifications, diet plan, physical activity registration, gamification, smoking cessation, pictures, game, and SMS, among others.

Identification of Daily Activites and Environments Based on the AdaBoost Method Using Mobile Device Data: A Systematic Review

Using the AdaBoost method may increase the accuracy and reliability of a framework for daily activities and environment recognition. Mobile devices have several types of sensors, including motion, magnetic, and location sensors, that allow accurate identification of daily activities and environment. This paper focuses on the review of the studies that use the AdaBoost method with the sensors available in mobile devices. This research identified the research works written in English about the recognition of daily activities and environment recognition using the AdaBoost method with the data obtained from the sensors available in mobile devices that were published between 2012 and 2018. Thus, 13 studies were selected and analysed from 151 identified records in the searched databases. The results proved the reliability of the method for daily activities and environment recognition, highlighting the use of several features, including the mean, standard deviation, pitch, roll, azimuth, and median absolute deviation of the signal of motion sensors, and the mean of the signal of magnetic sensors. When reported, the analysed studies presented an accuracy higher than 80% in recognition of daily activities and environments with the Adaboost method.

Activities of Daily Living and Environment Recognition Using Mobile Devices: A Comparative Study

The recognition of Activities of Daily Living (ADL) using the sensors available in off-the-shelf mobile devices with high accuracy is significant for the development of their framework. Previously, a framework that comprehends data acquisition, data processing, data cleaning, feature extraction, data fusion, and data classification was proposed. However, the results may be improved with the implementation of other methods. Similar to the initial proposal of the framework, this paper proposes the recognition of eight ADL, e.g., walking, running, standing, going upstairs, going downstairs, driving, sleeping, and watching television, and nine environments, e.g., bar, hall, kitchen, library, street, bedroom, living room, gym, and classroom, but using the Instance Based k-nearest neighbour (IBk) and AdaBoost methods as well. The primary purpose of this paper is to find the best machine learning method for ADL and environment recognition. The results obtained show that IBk and AdaBoost reported better results, with complex data than the deep neural network methods.