https://scholars.lib.ntu.edu.tw/handle/123456789/636152
標題: | Language Models are Causal Knowledge Extractors for Zero-shot Video Question Answering | 作者: | Su, Hung Ting Niu, Yulei Lin, Xudong WINSTON HSU Chang, Shih Fu |
公開日期: | 1-一月-2023 | 卷: | 2023-June | 來源出版物: | IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops | 摘要: | Causal Video Question Answering (CVidQA) queries not only association or temporal relations but also causal relations in a video. Existing question synthesis methods pretrained question generation (QG) systems on reading comprehension datasets with text descriptions as inputs. However, QG models only learn to ask association questions (e.g., "what is someone doing...") and result in inferior performance due to the poor transfer of association knowledge to CVidQA, which focuses on causal questions like "why is someone doing...". Observing this, we proposed to exploit causal knowledge to generate question-answer pairs, and proposed a novel framework, Causal Knowledge Extraction from Language Models (CaKE-LM), leveraging causal commonsense knowledge from language models to tackle CVidQA. To extract knowledge from LMs, CaKE-LM generates causal questions containing two events with one triggering another (e.g., "score a goal"triggers "soccer player kicking ball") by prompting LM with the action (soccer player kicking ball) to retrieve the intention (to score a goal). CaKE-LM significantly outperforms conventional methods by 4% to 6% of zero-shot CVidQA accuracy on NExT-QA and Causal-VidQA datasets. We also conduct comprehensive analyses and provide key findings for future research. |
URI: | https://scholars.lib.ntu.edu.tw/handle/123456789/636152 | ISBN: | 9798350302493 | ISSN: | 21607508 | DOI: | 10.1109/CVPRW59228.2023.00523 |
顯示於: | 資訊工程學系 |
在 IR 系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。