XCS- LBP Descriptor for Background Subtraction

1. An eXtended Center-Symmetric Local Binary Pattern for Background
Modeling and Subtraction in Videos
Caroline Silva, Thierry Bouwmans, Carl Fr´elicot
March 14, 2015 - Berlin

2. Summary
2/17
1 Introduction: Background Subtraction
2 Brief overview of LBP and its variants
3 The XCS-LBP Descriptor
4 Experimental Results
5 Conclusion and Future Research

3. Introduction
3/17
Background Subtraction
Figure: Block diagram of the background subtraction process.
Challenging situations
llumination changes, dynamic backgrounds, camera jitter, noise and shadows.
Common features
Color features, Edge features, Stereo features, Motion features, Texture features
A variety of local texture descriptors recently have attracted great attention for background modeling,
especially the Local Binary Pattern (LBP) because it’s simplicity and speed to computation.

4. Summary
4/17
1 Introduction: Background Subtraction
2 Brief overview of LBP and its variants
3 The XCS-LBP Descriptor
4 Experimental Results
5 Conclusion and Future Research

5. Brief overview of LBP and its variants
5/17
LBP descriptor
Figure: An example of LBP computation.
The LBP works in a 3 ×3 pixel block of an image. The
pixels in this block are thresholded by its center pixel
value, multiplied by powers of two and then summed
to obtain a label for the center pixel. The value LBP
code of a pixel (xc ,yc ) is given by:
LBPP ,R =
P−1
∑
p=0
s (gp −gc )2p
, (1)
where gc and gp denote the gray value of the central
pixel and its neighbors, R is the radius of the
neighborhood and P is the radius of the neighborhood.
The function s(x) is defined as follows:
s(x) =
1 x ≥ 0
0 otherwise.
(2)

6. Brief overview of LBP and its variants
6/17
LBP’s variants
Figure: Comparison of LBP and variants.

7. Brief overview of LBP and its variants
7/17
CS-LBP descriptor (Heikkil¨a et al., 2009)
Figure: The CS-LBP descriptor.
The underlying idea of CS-LBP is to compare the gray
levels of pairs of pixels in centered symmetric
directions instead of comparing the central pixel to its
neighbors. The CS-LBP operator is given by:
CS −LBPP,R (c) =
(P/2)−1
∑
i=0
s(gi −gi+(P/2))2i
(3)
where gi and gi+(P/2) are the gray values of
center-symmetric pairs of pixels, and s is the
thresholding function defined as:
s(x) =
1 if x > T
0 otherwise
(4)
where T is a user-defined threshold.

8. Summary
8/17
1 Introduction: Background Subtraction
2 Brief overview of LBP and its variants
3 The XCS-LBP Descriptor
4 Experimental Results
5 Conclusion and Future Research

9. Proposed XCS-LBP
9/17
Figure: The XCS-LBP descriptor.
The XCS-LBP (eXtended CS-LBP), expresses as:
XCS −LBPP,R (c) =
(P/2)−1
∑
i=0
s (g1(i,c)+g2(i,c))2i
(5)
The function s(x1 +x2) is defined as follows:
s(x1 +x2) =
1 if (x1 +x2) ≥ 0
0 otherwise.
(6)
and where g1(i,c) and g2(i,c) are defined by:
g1(i,c) = (gi −gi+(P/2))+gc
g2(i,c) = (gi −gc ) (gi+(P/2) −gc )
(7)
The main advantage
Produces a small histogram as CS-LBP,
but it extracts more image details.

10. Summary
10/17
1 Introduction: Background Subtraction
2 Brief overview of LBP and its variants
3 The XCS-LBP Descriptor
4 Experimental Results
5 Conclusion and Future Research

11. Experimental Results
11/17
We’ve compared XCS-LBP with three other texture descriptors among the reviewed ones,
namely :
Original LBP Ojala et al. (2002),
CS-LBP Heikkil¨a et al. (2009) and
CS-LDP Xue et al. (2011).
and we evaluate the performance with two popular background subtraction methods:
Adaptive Background Learning (ABL) (also know as Running Average) and
Gaussian Mixture Models (GMM).
The BMC (Background Models Challenge) data set of Vacavant et al. (2012) was chosen, and it
contains several synthetic and real world videos of outdoor situations (urban scenes).

12. Experimental Results
12/17
Figure: Background subtraction results using the GMM method on real world videos of the BMC – (a) original
frame, (b) ground truth, (c) LBP, (d) CS-LBP, (e) CS-LDP and (f) proposed XCS-LBP.
Figure: Performance of the different descriptors on real world videos of the BMC using the GMM method.

13. Experimental Results
13/17
Speed Comparison
MacBook Pro (OS X 10.9.4, 2.2 GHz Intel Core i7 and 8 GB - 1333 MHz DDR3) with
MATLAB R2013a.
Elapsed CPU times needed to segment the foreground masks by ABL and GMM methods,
averaged over nine real videos of BMC data set.
The reference is the fastest descriptor (original LBP), and the times are divided by LBP
ones.
XCS-LBP shows slightly better time performance than both CS-LBP and CS-LDP.
Figure: Elapsed CPU times over LBP times

14. Summary
14/17
1 Introduction: Background Subtraction
2 Brief overview of LBP and its variants
3 The XCS-LBP Descriptor
4 Experimental Results
5 Conclusion and Future Research

15. Conclusion and Future Research
15/17
The experimental results show that the XCS-LBP outperforms qualitatively and
quantitatively its direct competitors, making it a serious candidate for the background
subtraction task in computer vision applications.
The XCS-LBP produces a shorter histogram and it is more tolerant to illumination
changes and robust to noise.
Future works will explore how to extend the proposed descriptor to include temporal
relationships between neighboring pixels.
Source code and related libraries:
XCS- LBP Descriptor: http://lolynepacheco.wix.com/carolinesilva
BGSLibrary (Sobral, 2013): http://github.com/andrewssobral/bgslibrary
LRSLibrary (Sobral et al., 2014): http://github.com/andrewssobral/lrslibrary

16. 16/17
THANK YOU!!!

17. References
17/17
Heikkil¨a, M., Pietik¨ainen, M., and Schmid, C. (2009). Description of interest regions with local binary patterns. Pattern Recognition, 42:425–436.
Ojala, T., Pietik¨ainen, M., and M¨aenp¨a¨a, T. (2002). Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. on
Pattern Analysis and Machine Intelligence, pages 971–987.
Sobral, A. (2013). BGSLibrary: An opencv c++ background subtraction library. In IX Workshop de Vis ˜A£o Computacional (WVC’2013), Rio de Janeiro, Brazil.
Sobral, A., Baker, C. G., Bouwmans, T., and Zahzah, E. (2014). Incremental and multi-feature tensor subspace learning applied for background modeling and
subtraction. In International Conference on Image Analysis and Recognition (ICIAR’14). Lecture Notes in Computer Science (Springer LNCS).
Xue, G., Song, L., Sun, J., and Wu, M. (2011). Hybrid center-symmetric local pattern for dynamic background subtraction. In IEEE Int. Conf. on Multimedia and
Expo, pages 1–6.