Polly want a cracker: Analyzing performance of parroting on paraphrase generation datasets
Journal
EMNLP-IJCNLP 2019 - 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, Proceedings of the Conference
Pages
5960-5968
Date Issued
2020
Author(s)
Mao, H.-R.
Abstract
Paraphrase generation is an interesting and challenging NLP task which has numerous practical applications. In this paper, we analyze datasets commonly used for paraphrase generation research, and show that simply parroting input sentences surpasses state-of-the-art models in the literature when evaluated on standard metrics. Our findings illustrate that a model could be seemingly adept at generating paraphrases, despite only making trivial changes to the input sentence or even none at all. © 2019 Association for Computational Linguistics
Other Subjects
Standard metrics; State of the art; Natural language processing systems
Type
conference paper