Personal knowledge base construction from multimodal data
Journal
ICMR 2021 - Proceedings of the 2021 International Conference on Multimedia Retrieval
Pages
496-500
Date Issued
2021
Author(s)
Abstract
With the passage of time, people often have misty memories of their past experiences. Information recall support for people by collecting personal lifelogs is emerging. Recently, people tend to record their daily life via filming Video Weblog (VLog), which contains visual and audio data. These large scale multimodal data can be used to support information recall service that enables users to query their past experiences. The challenging issue is the semantic gap between the visual concept and the textual query. In this paper, we aim to extract personal life events from vlogs shared on YouTube and construct a personal knowledge base (PKB) for individuals. A multitask learning model is proposed to extract the components of personal life events, such as subjects, predicates and objects. The evaluation is performed on a video collection from three YouTubers who are English native speakers. Experimental results show our model achieves promising performance. ? 2021 ACM.
Subjects
Life event extraction
Personal knowledge base construction
Social media
Knowledge based systems
Semantics
Information recalls
Knowledge base
Knowledge-base construction
Multi-modal data
Personal lives
Textual query
Video collections
Visual concept
Learning systems
SDGs
Type
conference paper