Embedding Motion and Structure Features for Action Recognition

Xiantong Zhen, Ling Shao, Dacheng Tao, Xuelong Li

Research output: Contribution to journalArticlepeer-review

60 Citations (Scopus)


We propose a novel method to model human actions by explicitly coding motion and structure features that are separately extracted from video sequences. Firstly, the motion template (one feature map) is applied to encode the motion information and image planes (five feature maps) are extracted from the volume of differences of frames to capture the structure information. The Gaussian pyramid and center-surround operations are performed on each of the six obtained feature maps, decomposing each feature map into a set of subband maps. Biologically inspired features are then extracted by successively applying Gabor filtering and max pooling on each subband map. To make a compact representation, discriminative locality alignment is employed to embed the high-dimensional features into a low-dimensional manifold space. In contrast to sparse representations based on detected interest points, which suffer from the loss of structure information, the proposed model takes into account the motion and structure information simultaneously and integrates them in a unified framework; it therefore provides an informative and compact representation of human actions. The proposed method is evaluated on the KTH, the multiview IXMAS, and the challenging UCF sports datasets and outperforms state-of-the-art techniques on action recognition.
Original languageEnglish
Pages (from-to)1182-1190
JournalIEEE Transactions on Circuits and Systems for Video Technology
Issue number7
Publication statusPublished - Jul 2013


Dive into the research topics of 'Embedding Motion and Structure Features for Action Recognition'. Together they form a unique fingerprint.

Cite this