Compound Facial Emotion Recognition based on Facial Action Coding System and SHAP Values

Abstract

Human facial emotion recognition is a difficult task in computer-human interaction. Facial emotion recognition is required in many applications like medical, security, video games, e-physiotherapy, and counselling. Literature has
many studies that have focused only on 6 basic emotions but advanced studies
suggest human emotions are not limited to these 6 basic emotions. A human
face can exhibit many other emotions, which are generated by combining the
two basic emotions, these derived emotions are known as compound emotions.
Recognition of compound emotions is also a very important task; hence this
study proposes the use of the Facial Action Coding System (FACS) to identify
12 compound emotions. The authors identified and derived the intensities of
17 AUs with Openface library. Finally, two machine learning classifiers SVM
(Support Vector Machine) and KNN (K-nearest neighbour) were implemented
to identify 12 compound emotions, and results were compared. The experimental results show that the SVM classifier outperformed with an emotion recognition rate of 98.31% while the recognition rate of K-NN was 93.66%. The
authors also implemented SHAP values to observe the AUs association with
each compound emotion.