Netizen-Style Commenting on Fashion Photos: Dataset and Diversity Measures
Journal
The Web Conference 2018 - Companion of the World Wide Web Conference, WWW 2018
Pages
395-402
Date Issued
2018
Author(s)
Abstract
Recently, deep neural network models have achieved promising results in image captioning task. Yet, "vanilla'' sentences, only describing shallow appearances (e.g., types, colors), generated by current works are not satisfied netizen style resulting in lacking engagements, contexts, and user intentions. To tackle this problem, we propose Netizen Style Commenting (NSC), to automatically generate characteristic comments to a user-contributed fashion photo. We are devoted to modulating the comments in a vivid "netizen'' style which reflects the culture in a designated social community and hopes to facilitate more engagement with users. In this work, we design a novel framework that consists of three major components: (1) We construct a large-scale clothing dataset named NetiLook, which contains 300K posts (photos) with 5M comments to discover netizen-style comments. (2) We propose three unique measures to estimate the diversity of comments. (3) We bring diversity by marrying topic models with neural networks to make up the insufficiency of conventional image captioning works. Experimenting over Flickr30k and our NetiLook datasets, we demonstrate our proposed approaches benefit fashion photo commenting and improve image captioning tasks both in accuracy and diversity. ? 2018 IW3C2 (International World Wide Web Conference Committee), published under Creative Commons CC BY 4.0 License.
Subjects
Deep neural networks; Large dataset; World Wide Web; Diversity measure; Image captioning; Netizen; Neural network model; Social communities; Topic model; User intention; Image enhancement
SDGs
Type
conference paper