Frontalization and adaptive exponential ensemble rule for deep-learning-based facial expression recognition system
Journal
Signal Processing: Image Communication
Journal Volume
96
Date Issued
2021
Author(s)
Abstract
Automatic facial expression recognition (FER) is an important technique in human–computer interfaces and surveillance systems. It classifies the input facial image into one of the basic expressions (anger, sadness, surprise, happiness, disgust, fear, and neutral). There are two types of FER algorithms: feature-based and convolutional neural network (CNN)-based algorithms. The CNN is a powerful classifier, however, without proper auxiliary techniques, its performance may be limited. In this study, we improve the CNN-based FER system by utilizing face frontalization and the hierarchical architecture. The frontalization algorithm aligns the face by in-plane or out-of-plane, rotation, landmark point matching, and removing background noise. The proposed adaptive exponentially weighted average ensemble rule can determine the optimal weight according to the accuracy of classifiers to improve robustness. Experiments on several popular databases are performed and the results show that the proposed system has a very high accuracy and outperforms state-of-the-art FER systems. ? 2021 Elsevier B.V.
Subjects
Convolutional neural networks; Face recognition; Automatic facial expression recognition; Auxiliary techniques; Background noise; Facial expression recognition; Hierarchical architectures; State of the art; Surveillance systems; Weighted averages; Deep learning
Type
journal article