https://scholars.lib.ntu.edu.tw/handle/123456789/630026
標題: | Multi-modal Chatbot in Intelligent Manufacturing | 作者: | Chen, Tzu Yu Chiu, Yu Ching NANYI BI Tsai, Richard Tzong Han |
關鍵字: | Artificial intelligence | chatbot | Chatbot | human-robot interaction | Manufacturing | multi-modal intent classification | Robotic assembly | Robots | Task analysis | Visualization | 公開日期: | 25-五月-2021 | 出版社: | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC | 卷: | 9 | 起(迄)頁: | 82118-82129 | 來源出版物: | IEEE Access | 摘要: | Artificial intelligence (AI) has been widely used in various industries. In this work, we concentrate on what AI is capable of doing in manufacturing, in the form of a chatbot. We designed a chatbot that helps users complete an assembly task that simulates those in manufacturing settings. In order to recreate this setting, we have users assemble a Meccanoid robot through multiple stages, with the help of an interactive dialogue system. Based on classifying users’ intent, the chatbot is able to provide answers or instructions to the user when the user encounters problems during the assembly process. Our goal is to improve our system so that it can capture users’ needs by detecting their intent and therefore provide relevant and helpful information to the user. However, in a multiple-step task, we cannot rely on intent classification with user question utterance as the only input, as user questions raised from different steps may share the same intent but require different responses. In this paper, we proposed two methods to address this problem. One is that we capture not only textual features but also visual features through the YOLO-based Masker with CNN (YMC) model. Another is the usage of an Autoencoder to encode multi-modal features for user intent classification. By incorporating visual information, we have significantly improved the chatbot’s performance from the experiments conducted on different dataset. |
URI: | https://scholars.lib.ntu.edu.tw/handle/123456789/630026 | ISSN: | 2169-3536 | DOI: | 10.1109/ACCESS.2021.3083518 |
顯示於: | 資訊管理學系 |
在 IR 系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。