https://scholars.lib.ntu.edu.tw/handle/123456789/634351
標題: | On the Use of Unrealistic Predictions in Hundreds of Papers Evaluating Graph Representations | 作者: | Lin, Li Chung Liu, Cheng Hung Chen, Chih Ming Hsu, Kai Chin Wu, I. Feng Tsai, Ming Feng CHIH-JEN LIN |
公開日期: | 30-六月-2022 | 卷: | 36 | 來源出版物: | Proceedings of the 36th AAAI Conference on Artificial Intelligence, AAAI 2022 | 摘要: | Prediction using the ground truth sounds like an oxymoron in machine learning. However, such an unrealistic setting was used in hundreds, if not thousands of papers in the area of finding graph representations. To evaluate the multi-label problem of node classification by using the obtained representations, many works assume that the number of labels of each test instance is known in the prediction stage. In practice such ground truth information is rarely available, but we point out that such an inappropriate setting is now ubiquitous in this research area. We detailedly investigate why the situation occurs. Our analysis indicates that with unrealistic information, the performance is likely over-estimated. To see why suitable predictions were not used, we identify difficulties in applying some multi-label techniques. For the use in future studies, we propose simple and effective settings without using practically unknown information. Finally, we take this chance to compare major graph representation learning methods on multi-label node classification. |
URI: | https://scholars.lib.ntu.edu.tw/handle/123456789/634351 | ISBN: | 1577358767 |
顯示於: | 資訊工程學系 |
在 IR 系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。