Photo Filter Recommendation by Image Aesthetic Learning

Nowadays, social media have become popular platforms for the public to share photos. To apply effects on a photo or improve its quality, most social media provide filters by which users can change the appearance of their photos without domain knowledge. However, due to the growing number of filter types, it becomes a major issue for users to choose the best filter type instantly. For this purpose, learning image aesthetics takes an important role in image quality ranking problems. In these years, several research has proved that Convolutional Neural Networks (CNNs) outperform traditional methods in image aesthetic categorization, which classifies images into high or low quality. In this paper, we introduce a new method for image quality learning and a dataset of filtered images with comparison. Instead of binarizing image quality, we use different CNN architectures and a pairwise comparison loss function to learn the aesthetic response for an image. By utilizing pairwise image comparison, the models embed aesthetic responses in the hidden layers. Moreover, to improve the aesthetic ranking, the image category is integreated into the aesthetic-oriented models. To train our models and evaluate our method, we introduce a new dataset called Filter Aesthetic Comparison Dataset (FACD). The dataset contains more than 30,000 filtered images based on the AVA dataset and more than 40,000 image pairs with quality comparison annotations using Amazon Mechanical Turk. To our best knowledge, it is the first dataset containing filtered images and the user preference labels. The experimental results show that our method which learns aesthetic ranking by pairwise comparison outperforms the traditional aesthetic classification methods.