Household robot utilizing location information for human activity and habit understanding
Journal
Robotics and Autonomous Systems
Journal Volume
199
Start Page
105369
ISSN
0921-8890
Date Issued
2026-01-27
Author(s)
Abstract
This paper proposes an integrated system combining location estimation, human activity recognition (HAR), and plan recognition modules. In order to improve the HAR performance, we propose a location estimation system that fuses ResNet50-Places365 (Zhou et al., 2018) and our created estimator that leverages information on the distances between human and nearby objects. The location information from the location estimation system and the human skeleton information will be fed into HAR module governed by our developed activity-location graph convolutional neural network (AL-GCN). To explore more usage of the recognized activities, we propose a plan recognition system that updates the human's plan knowledge base while taking into account human's habits from time to time so as to make three important predictions, namely, next activity, objective, and plan. In our experiment, we evaluate our system on both dataset and real-world scenarios. In dataset evaluation, our location estimation system performs best with 92.83% accuracy, our AL-GCN model outperforms the state-of-the-art (SOTA) models with 94.33% accuracy on cross-subject evaluation, and our proposed plan recognition improves when habits are considered and knowledge base is updated. In the real-world experiments, the location estimation achieves 98% accuracy when in the living room, and our AL-GCN model improves the accuracy from 10% to 20% by including location information. Finally, our plan recognition shows that, by updating knowledge base, the predictions accuracy increases significantly.
Subjects
Household robot
Human activity recognition
Human habit
Location estimation
Plan recognition
Robot response
Publisher
Elsevier BV
Type
journal article