The document proposes a novel coarse-to-fine sparse representation approach for efficient human action recognition. It reduces the computational complexity of testing sparse representation-based classification (SRC) by constructing sub-sampled dictionaries at multiple levels of granularity. Specifically, it first builds a coarse-grained dictionary by randomly projecting and sub-sampling atoms from the training data. Then it selects a small number of candidate actions using the coarse dictionary before classifying the action using a pruned fine-grained dictionary constructed from the candidate classes only. Experimental results on a benchmark dataset show the proposed method achieves efficient recognition with little loss in accuracy compared to the conventional SRC approach.

1. I. INTRODUCTION
 Sparse representation-based classification (SRC) has
recently attracted substantial research attention
 However, the computational complexity of testing makes it
challenging to deploy SRC in practice
 We propose a novel method for human action recognition,
leveraging coarse-to-fine sparse representations that have
been obtained through dictionary sub-sampling
 The proposed method reduces the time complexity of
testing at no substantial loss in recognition accuracy
JongHo Leea, Hyun-seok Mina, Jeong-jik Seoa, Wesley De Nevea,b, and Yong Man Roa
aImage and Video Systems Lab, KAIST, Republic of Korea
bMultimedia Lab, Ghent University-iMinds, Belgium
website: http://ivylab.kaist.ac.kr
IEEE International Conference on Multimedia & Expo (ICME), July 2014, Chengdu, China
SUB-SAMPLED DICTIONARIES FOR COARSE-TO-FINE
SPARSE REPRESENTATION-BASED HUMAN ACTION RECOGNITION
e-mail: ymro@ee.kaist.ac.kr
II. PROPOSED APPROACH
1. Training
Fig. 2. Time complexity of different human action recognition approaches.
Fig. 1. Accuracy of different human action recognition approaches.
0
10
20
30
40
50
60
70
150 300 450 600 750 900 1050 1200 1350 1500
Timecomplexity(s)
Number of atoms(ls)
Proposed method with ds =48
Proposed method with ds =72
Proposed method with ds =144
Conventional method
0.76
0.78
0.8
0.82
0.84
0.86
0.88
0.9
150 300 450 600 750 900 1050 1200 1350 1500
Recognitionaccuracy
Number of atoms(ls)
Proposed method with ds =48
Proposed method with ds =72
Proposed method with ds =144
Conventional method
III. EXPERIMENTS
1. Experimental setup
 Dataset: UCF-50
 Feature: Cuboid detector + HOG descriptor
 Homotopy-based 𝑙1-norm minimization
2. Experimental results
 Conventional method: classification only uses the FGD
IV. CONCLUSIONS
 We proposed a novel method for human action recognition
using coarse-to-fine sparse representations
 The proposed method achieves efficient human action
recognition at no substantial loss in recognition accuracy
2. Testing
Y
Y𝑠
Random projection
Feature Extraction
Test video clip
…
Class 1 Class 2
Φ 𝑠,1 Φ 𝑠,2 Φ 𝑠,3 Φ 𝑠,𝐾
Sparse
Coefficients
Y𝑠
Ranking 1 𝐻+1 𝐻+4 𝑯
Candidate
Actions
Candidate Action Selection
Coarse-Grained
Dictionary (CGD)
O X X O
We select 𝐻
candidate
actions
Feature Extraction
…
Action 1 Action 2 Action 3 Action 𝐾
Training Dataset
Action 1 Action 2 Action 3 Action 𝐾
… … … …
…
Action 1Action 2Action 3 Action 𝐾
Fine-Grained
Dictionary
(FGD)
Coarse-Grained
Dictionary (CGD)Φ 𝑠,1 Φ 𝑠,2 Φ 𝑠,3 Φ 𝑠,𝐾
Φ 𝑜,1 Φ 𝑜,2 Φ 𝑜,3 Φ 𝑜,𝐾
Random projection (for reducing the dimension of the atoms)
Random sampling (for reducing the number of atoms)
Dictionary Construction
Action 1 Action 2 Action 3 Action 𝐾
…
…
Pruned FGD
Φ 𝑜,1 Φ 𝑜,2 Φ 𝑜,3 Φ 𝑜,𝐾
Action 1 Action 2 Action 3 Action 𝐾
…
Candidate
Actions
O X X O
Φ 𝑝𝑟,1
Action 1
Φ 𝑝𝑟,𝐻
Action 𝐾
… Pruned FGD𝐃 𝑝𝑟
 Classification
 We can find the sparse representation 𝐗 𝑝𝑟 of 𝐘 with 𝐃 𝑝𝑟
𝐘 = 𝐲1, 𝐲2, … , 𝐲 𝑚 , 𝐗 𝑝𝑟 = [𝐱 𝑝𝑟,1, 𝐱 𝑝𝑟,2, … , 𝐱 𝑝𝑟,𝑚]
 We label 𝐕 with the action 𝑘 that comes with the smallest
residual error 𝒓 𝑘 𝐲
𝒓 𝑘 𝐘 =
1
𝑚 𝑖=1
𝑚
𝐲𝑖 − 𝐃 𝑝𝑟 𝜹 𝑘 𝐱 𝑝𝑟,𝑖 𝟏
 𝜹 𝒌 𝐱 𝑝𝑟,𝑖 is a new vector whose only nonzero entries are the
entries in 𝐱 𝑝𝑟,𝑖 associated with the action 𝑘
Φ 𝑜,1 Φ 𝑜,2 Φ 𝑜,3 Φ 𝑜,𝐾