Regional Attention Network (RAN) for Head Pose and Fine-Grained Gesture Recognition

Ardhendu Behera, Zachary Wharton, Yonghuai Liu, Morteza Ghahremani, Swagat Kumar, Nik Bessis

Research output: Contribution to journalArticlepeer-review

5 Citations (SciVal)
107 Downloads (Pure)

Abstract

Affect is often expressed via non-verbal body language such as actions/gestures, which are vital indicators for human behaviors. Recent studies on recognition of fine-grained actions/gestures in monocular images have mainly focused on modeling spatial configuration of body parts representing body pose, human-objects interactions and variations in local appearance. The results show that this is a brittle approach since it relies on accurate body parts/objects detection. In this work, we argue that there exist local discriminative semantic regions, whose 'informativeness' can be evaluated by the attention mechanism for inferring fine-grained gestures/actions. To this end, we propose a novel end-to-end regional attention network (RAN), which is a fully convolutional neural network (CNN) to combine multiple contextual regions through attention mechanism, focusing on parts of the images that are most relevant to a given task. Our regions consist of one or more consecutive cells and are adapted from the strategies used in computing HOG (Histogram of Oriented Gradient) descriptor. The model is extensively evaluated on ten datasets belonging to 3 different scenarios: 1) head pose recognition, 2) drivers state recognition, and 3) human action and facial expression recognition. The proposed approach outperforms the state-of-the-art by a considerable margin in different metrics.

Original languageEnglish
Pages (from-to)549-562
Number of pages14
JournalIEEE Transactions on Affective Computing
Volume14
Issue number1
Early online date16 Oct 2020
DOIs
Publication statusPublished - 01 Jan 2023

Keywords

  • Annotations
  • Attention Mechanism
  • Computer Vision
  • Convolutional Neural Network
  • Face recognition
  • Facial Expressions Recognition
  • Fine-grained Gesture Recognition
  • Gesture Recognition
  • Head
  • Head Pose Recognition
  • Human-Object Interaction
  • Image recognition
  • Magnetic heads
  • Pose estimation
  • Regional Attention Network
  • Task analysis
  • head pose recognition
  • regional attention network
  • human-object interaction
  • attention mechanism
  • convolutional neural network
  • Gesture recognition
  • facial expressions recognition
  • computer vision
  • fine-grained gesture recognition

Fingerprint

Dive into the research topics of 'Regional Attention Network (RAN) for Head Pose and Fine-Grained Gesture Recognition'. Together they form a unique fingerprint.

Cite this