|
1
|
Konak O, van de Water R, Döring V, Fiedler T, Liebe L, Masopust L, Postnov K, Sauerwald F, Treykorn F, Wischmann A, Gjoreski H, Luštrek M, Arnrich B. HARE: Unifying the Human Activity Recognition Engineering Workflow. SENSORS (BASEL, SWITZERLAND) 2023; 23:9571. [PMID: 38067946 PMCID: PMC10708727 DOI: 10.3390/s23239571] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/13/2023] [Revised: 11/21/2023] [Accepted: 11/29/2023] [Indexed: 12/18/2023]
Abstract
Sensor-based human activity recognition is becoming ever more prevalent. The increasing importance of distinguishing human movements, particularly in healthcare, coincides with the advent of increasingly compact sensors. A complex sequence of individual steps currently characterizes the activity recognition pipeline. It involves separate data collection, preparation, and processing steps, resulting in a heterogeneous and fragmented process. To address these challenges, we present a comprehensive framework, HARE, which seamlessly integrates all necessary steps. HARE offers synchronized data collection and labeling, integrated pose estimation for data anonymization, a multimodal classification approach, and a novel method for determining optimal sensor placement to enhance classification results. Additionally, our framework incorporates real-time activity recognition with on-device model adaptation capabilities. To validate the effectiveness of our framework, we conducted extensive evaluations using diverse datasets, including our own collected dataset focusing on nursing activities. Our results show that HARE's multimodal and on-device trained model outperforms conventional single-modal and offline variants. Furthermore, our vision-based approach for optimal sensor placement yields comparable results to the trained model. Our work advances the field of sensor-based human activity recognition by introducing a comprehensive framework that streamlines data collection and classification while offering a novel method for determining optimal sensor placement.
Collapse
Affiliation(s)
- Orhan Konak
- Hasso Plattner Institute, University of Potsdam, 14482 Potsdam, Germany; (R.v.d.W.); (V.D.); (T.F.); (L.L.); (L.M.); (K.P.); (F.S.); (F.T.); (A.W.); (B.A.)
| | - Robin van de Water
- Hasso Plattner Institute, University of Potsdam, 14482 Potsdam, Germany; (R.v.d.W.); (V.D.); (T.F.); (L.L.); (L.M.); (K.P.); (F.S.); (F.T.); (A.W.); (B.A.)
| | - Valentin Döring
- Hasso Plattner Institute, University of Potsdam, 14482 Potsdam, Germany; (R.v.d.W.); (V.D.); (T.F.); (L.L.); (L.M.); (K.P.); (F.S.); (F.T.); (A.W.); (B.A.)
| | - Tobias Fiedler
- Hasso Plattner Institute, University of Potsdam, 14482 Potsdam, Germany; (R.v.d.W.); (V.D.); (T.F.); (L.L.); (L.M.); (K.P.); (F.S.); (F.T.); (A.W.); (B.A.)
| | - Lucas Liebe
- Hasso Plattner Institute, University of Potsdam, 14482 Potsdam, Germany; (R.v.d.W.); (V.D.); (T.F.); (L.L.); (L.M.); (K.P.); (F.S.); (F.T.); (A.W.); (B.A.)
| | - Leander Masopust
- Hasso Plattner Institute, University of Potsdam, 14482 Potsdam, Germany; (R.v.d.W.); (V.D.); (T.F.); (L.L.); (L.M.); (K.P.); (F.S.); (F.T.); (A.W.); (B.A.)
| | - Kirill Postnov
- Hasso Plattner Institute, University of Potsdam, 14482 Potsdam, Germany; (R.v.d.W.); (V.D.); (T.F.); (L.L.); (L.M.); (K.P.); (F.S.); (F.T.); (A.W.); (B.A.)
| | - Franz Sauerwald
- Hasso Plattner Institute, University of Potsdam, 14482 Potsdam, Germany; (R.v.d.W.); (V.D.); (T.F.); (L.L.); (L.M.); (K.P.); (F.S.); (F.T.); (A.W.); (B.A.)
| | - Felix Treykorn
- Hasso Plattner Institute, University of Potsdam, 14482 Potsdam, Germany; (R.v.d.W.); (V.D.); (T.F.); (L.L.); (L.M.); (K.P.); (F.S.); (F.T.); (A.W.); (B.A.)
| | - Alexander Wischmann
- Hasso Plattner Institute, University of Potsdam, 14482 Potsdam, Germany; (R.v.d.W.); (V.D.); (T.F.); (L.L.); (L.M.); (K.P.); (F.S.); (F.T.); (A.W.); (B.A.)
| | - Hristijan Gjoreski
- Faculty of Electrical Engineering and Information Technologies, Ss. Cyril and Methodius University in Skopje, 1000 Skopje, North Macedonia;
| | - Mitja Luštrek
- Department of Intelligent Systems, Jožef Stefan Institute, 1000 Ljubljana, Slovenia;
| | - Bert Arnrich
- Hasso Plattner Institute, University of Potsdam, 14482 Potsdam, Germany; (R.v.d.W.); (V.D.); (T.F.); (L.L.); (L.M.); (K.P.); (F.S.); (F.T.); (A.W.); (B.A.)
| |
Collapse
|