Intelligent facial expression recognition with adaptive feature extraction for a humanoid robot

Kamlesh Mistry, Li Zhang, John Barnden

Research output: Chapter in Book/Report/Conference proceedingChapterpeer-review

1 Citation (Scopus)


Automatic facial expression recognition plays an important role in agent-based interface development and datadriven animation. This paper presents an intelligent facial action and emotion recognition system for a humanoid robot. Motivated by the Facial Action Coding System, this research focuses on the recognition of seven basic emotions and 18 Action Units (AU). Since effective facial representations of original face images are vital for automatic facial emotion recognition, this research implements a novel shape and appearance feature extraction method, which integrates an Independent Active Appearance Model (AAM) with a rotation-invariant feature point detector, BRISK (Binary Robust Invariant Scalable Keypoints). In comparison to AAM with a traditional inverse compositional fitting, our model with BRISK fitting is with less computational cost and is capable of dealing with feature extraction from images of faces with rotations and scaling differences without prior training required. Subsequently shape and appearancebased neural network AU analyzers are used to respectively detect 18 AUs. Emotions are then decoded from the derived AUs using a neural network emotion recognizer. The system is integrated with a modern humanoid robot platform. Evaluation results indicate its high accuracy for AU and emotion recognition. It is also among the top performers on the extended Cohn-Kanade (CK+) database in comparison to other existing state-of-the-art applications.
Original languageEnglish
Title of host publicationProceedings of the 2015 International Joint Conference on Neural Networks (IJCNN)
Place of PublicationPiscataway, NJ
ISBN (Print)9781479919604
Publication statusPublished - 2015
EventInternational Joint Conference on Neural Networks (IJCNN 2015) - Killarney
Duration: 1 Jan 2015 → …


ConferenceInternational Joint Conference on Neural Networks (IJCNN 2015)
Period1/01/15 → …


Dive into the research topics of 'Intelligent facial expression recognition with adaptive feature extraction for a humanoid robot'. Together they form a unique fingerprint.

Cite this