This document presents a methodology for wavelet-based reflection symmetry detection using textural and color histograms. It extracts multiscale edge segments using Log-Gabor filters and measures symmetry based on edge orientations, local texture histograms, and color histograms. Evaluation on public datasets shows it outperforms previous methods in detecting single and multiple symmetries, with quantitative and qualitative results presented. Future work could improve the detection using continuous maximal-seeking.

1. Introduction
Related Work
Methodology
Results and Discussion
Wavelet-based Reflection Symmetry Detection via
Textural and Color Histograms
M. Elawady1
, C. Ducottet1
, O. Alata1
, C. Barat1
, P. Colantoni2
1
Universit´e de Lyon, CNRS, UMR 5516, Laboratoire Hubert Curien,
Universit´e de Saint-´Etienne, Jean-Monnet, F-42000 Saint-´Etienne, France
2
Universit´e Jean Monnet, CIEREC EA n0
3068, Saint-´Etienne, France
ICCV 2017 Workshop: Detecting Symmetry in the Wild
UMR • CNRS • 5516 • SAINT-ETIENNE
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 1 / 27

2. Introduction
Related Work
Methodology
Results and Discussion
Table of Contents
1 Introduction
Background
Problem Definition
2 Related Work
Intensity-based Methods
Edge-based Methods
3 Methodology
Motivation
Algorithm Details
4 Results and Discussion
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 2 / 27

3. Introduction
Related Work
Methodology
Results and Discussion
Background
Problem Definition
Table of Contents
1 Introduction
Background
Problem Definition
2 Related Work
Intensity-based Methods
Edge-based Methods
3 Methodology
Motivation
Algorithm Details
4 Results and Discussion
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 3 / 27

4. Introduction
Related Work
Methodology
Results and Discussion
Background
Problem Definition
Bilateral Symmetry
1Image from book: The Photographer’s Eye by Michael Freeman
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 4 / 27

5. Introduction
Related Work
Methodology
Results and Discussion
Background
Problem Definition
Detection of Global Symmetries
Axis Legend: Strong, Weak
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 5 / 27

6. Introduction
Related Work
Methodology
Results and Discussion
Intensity-based Methods
Edge-based Methods
Table of Contents
1 Introduction
Background
Problem Definition
2 Related Work
Intensity-based Methods
Edge-based Methods
3 Methodology
Motivation
Algorithm Details
4 Results and Discussion
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 6 / 27

7. Introduction
Related Work
Methodology
Results and Discussion
Intensity-based Methods
Edge-based Methods
Baseline (Loy 2006) and its Successor (Mo 2011)
The general scheme (Loy and Eklundh 2006 [1]) consists of:
Disadvantages:
Depending mainly on the properties of hand-crafted features (i.e. SIFT).
For example: (smooth objects with noisy background)
little feature points =⇒ lost symmetry.
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 7 / 27

8. Introduction
Related Work
Methodology
Results and Discussion
Intensity-based Methods
Edge-based Methods
First Work (Cic 2014)
Instead of SIFT, the general idea (Cicconet et al. 2014 [2]) is extracting a
regular set of wavelet segments with local edge amplitude and orientation.
Disadvantages:
Lacking neighborhood’s information inside the feature representation.
Depending on the scale parameter of the edge detector.
For example: (high texture objects with noisy background)
inferior symmetrical info =⇒ incorrect symmetry.
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 8 / 27

9. Introduction
Related Work
Methodology
Results and Discussion
Intensity-based Methods
Edge-based Methods
State-of-Art (Ela 2016)
(Single Symmetry) Investigating Cicconet’s edge features [2] within Loy’s
scheme [1] by adding neighboring-pixel information.
(1) Mul�scale Edge
Segment Extrac�on
(2) Triangula�on based on
Local Symmetry Weights:
• Geometry Edge Orienta�ons (Cic)
• Local Texture Histogram (Loy)
(3) Vo�ng Space for Peak Detec�on with Handling
Orienta�on Discon�nuity.
θ
ρ
0
π
Legend: Groundtruth, Our2016, Loy2006, Mo2011, Cic2014
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 9 / 27

10. Introduction
Related Work
Methodology
Results and Discussion
Motivation
Algorithm Details
Table of Contents
1 Introduction
Background
Problem Definition
2 Related Work
Intensity-based Methods
Edge-based Methods
3 Methodology
Motivation
Algorithm Details
4 Results and Discussion
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 10 / 27

11. Introduction
Related Work
Methodology
Results and Discussion
Motivation
Algorithm Details
Proposed Idea
Contribution:
Feature extraction using Log-Gabor filters.
Similarity measure based on textural and color image information.
(a) Multiscale Edge
Segment Extraction
(b) Triangulation based
on Symmetry Weights
θ
ρ
0
π
(c) Voting Space for Peak Detection
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 11 / 27

12. Introduction
Related Work
Methodology
Results and Discussion
Motivation
Algorithm Details
Feature Extraction I: Log-Gabor filter
Logarithmic transformation of a Gabor filter in the Fourier domain:
ˆG(η, α; s, o) = exp(−
(log( η
ηs
))2
2(log(ση))2
) exp(−
(α − αo)2
2σ2
α
) (1)
Advantages:
Better resolution in orientations
Avoid over-representation of low frequencies
Uniform cover of the Fourier domain
Fourier Real Imaginary
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 12 / 27

13. Introduction
Related Work
Methodology
Results and Discussion
Motivation
Algorithm Details
Feature Extraction II: Log-Gabor filter
The modulus of complex wavelet coefficients Is,o(x, y) are computed on
an image I (width W and height H) over multiple scales s ∈ {1, . . . , S}
and orientations o ∈ {zπ
O , z = 0, . . . , O − 1} as follows:
I →
GS
IGS
FT
→ ˆIGS (2)
Is,o(x, y) = |FT−1
(ˆIGS × ˆG)| (3)
I J(x, y) = max
s,o
Is,o(x, y) φ(x, y) = argo max
s,o
Is,o(x, y)
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 13 / 27

14. Introduction
Related Work
Methodology
Results and Discussion
Motivation
Algorithm Details
Similarity Measure I
Di : Neighboring cell of pi
Ji : Maximum wavelet response inside Di
φi : Orientation of Ji
ψi : HSV Color of Ji
hi
: Neighboring textural histogram [3]
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 14 / 27

15. Introduction
Related Work
Methodology
Results and Discussion
Motivation
Algorithm Details
Similarity Measure II: Color
Local HSV histogram gi
of size C with sub-sampling rate
(Chu
: Csa
: Cva
)
gi
(c) =
r∈D∗
i
1Ψc
(ψr ), (4)
c = (chu
, csa
, cva
),
chu
∈ {0, . . . , Chu
− 1},
csa
∈ {0, . . . , Csa
− 1},
cva
∈ {0, . . . , Cva
− 1},
Ψc = [2chu
π
Chu , 2(chu
+1)π
Chu [ × [ csa
Csa , csa
+1
Csa [ × [ cva
Cva , cva
+1
Cva [
where D∗
i is the neighborhood window around feature point pi
, ψc is a
sub-sampled set of HSV space, in terms of three components: hue (hu),
saturation (sa) and value (va). l1 normalization is applied to gi
(.) for
bin-wise histogram comparison.
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 15 / 27

16. Introduction
Related Work
Methodology
Results and Discussion
Motivation
Algorithm Details
Symmetry Triangulation I
A set of feature pairs (pi
, pj
) of size P(P−1)
2 are elected such that i = j,
and P is the number of feature points. The candidate axis is
parametrized by angle θi,j , and displacement ρi,j and has a symmetry
weight ωi,j defined as follows:
ωi,j = ω(pi
, pj
) = m(i, j) t(i, j) q(i, j) (5)
m(i, j) = |τi
R(T⊥
ij )τj
| (6)
t(i, j) =
N
n=1
min(hi
(n), ˜hj
(n)) (7)
q(i, j) =
C
c=1
min(gi
(c), gj
(c)) (8)
where τi
= [cos(φi ), sin(φi )]T
, R(T⊥
ij ) is the reflection matrix with
respect to the perpendicular of the line connecting (pi
, pj
) [2, 3], and ˜hj
is the mirror version of hj
histogram.
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 16 / 27

17. Introduction
Related Work
Methodology
Results and Discussion
Motivation
Algorithm Details
Symmetry Triangulation II
A symmetry histogram H(ρ, θ) is defined as the sum of the symmetry
weights of all pairs of feature points such as:
H(ρ, θ) =
pi
,pj
i=j
ωi,j δρ−ρi,j
δθ−θi,j
(9)
where δ is the Kronecker delta. The voting histogram H(ρ, θ) is
smoothed using a Gaussian kernel to output a proper density
representation. A1
A2
B3
B4
B1 B2
Input+GT
A1
A2
A2
B1 B2
B3 B4
B3 B4
Voting Space
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 17 / 27

18. Introduction
Related Work
Methodology
Results and Discussion
Table of Contents
1 Introduction
Background
Problem Definition
2 Related Work
Intensity-based Methods
Edge-based Methods
3 Methodology
Motivation
Algorithm Details
4 Results and Discussion
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 18 / 27

19. Introduction
Related Work
Methodology
Results and Discussion
Datasets and Evaluation
Datasets PSU AVA NY ICCV
Year 2010-2013 2016 2017 2017
#Images (Single) 157 253 176 100
#Images/#Symmetries (Multiple) 142/479 -/- 63/188 100/384
Evaluation γ (angle) ζ (distance)
CVPR2011 [4] 10◦
20% × len(GT)
CVPR2013 [5] 10◦
20% × min{len(MT), len(GT)}
ICCV2017 3◦
2.5% × min{W , H}
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 19 / 27

20. Introduction
Related Work
Methodology
Results and Discussion
Quantitative Results I - PR Curves
PSU (2010-2013) ICCV (2017)
X-axis: Recall, Y-axis: Precision
Evaluation Metrics: CVPR2013 [5]
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 20 / 27

21. Introduction
Related Work
Methodology
Results and Discussion
Quantitative Results II - TP
Datasets Loy[1] Cic[2] Ela[3] Lg LgC
PSU(157) 81 90 97 109 116
AVA(253) 174 124 170 191 197
NY(176) 98 92 109 125 132
ICCV17(100) 52 53 52 70 69
PSUm(142) 69 68 67 74 79
NYm(63) 32 36 36 39 40
ICCV17m(100) 54 39 53 53 56
Evaluation Metrics: CVPR2013 [5]
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 21 / 27

22. Introduction
Related Work
Methodology
Results and Discussion
Qualitative Results I - Single
Columns: (1) GT, (2) Loy2006 [1], (3) Ela2016 [3] (4) Our(Lg), and (5) Our(LgC)
Top 5 detections: red, yellow, green, blue, and magenta.
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 22 / 27

23. Introduction
Related Work
Methodology
Results and Discussion
Qualitative Results II - Multiple
Columns: (1) GT, (2) Loy2006 [1], (3) Ela2016 [3] (4) Our(Lg), and (5) Our(LgC)
Top 5 detections: red, yellow, green, blue, and magenta.
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 23 / 27

24. Introduction
Related Work
Methodology
Results and Discussion
Conclusion
Summary:
1 A reliable detection framework for global symmetries using Log-Gabor wavelet
response.
2 Introduction of symmetric metrics using textural and color information around
wavelet features.
3 Superior performance over single and multiple symmetries over all public
datasets.
Future work:
1 The proposed detection can be improved using a continuous maximal-seeking
technique to avoid over-extended axes.
2 Possibility of integration within retrieval systems for artistic photographs and
paintings in museums.
Source Code: http://github.com/mawady/ColorSymDetect
AVA Symmetry Dataset: http://github.com/mawady/AvaSym
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 24 / 27

25. Introduction
Related Work
Methodology
Results and Discussion
Comments about ICCV Reflection Symmetry
Typo Error for Loy algorithm in symmetry competition results
(ECCV 2006 not ECCV 2004)
Test image set is not published public as in competition phase 2
Evaluation metric is not complete (just TPR) and not correct (wrong
calculation in angle difference and removing candidate duplicates)
The two baseline codes are not published yet on ICCV2017 for
validation:
Loy2006: using PSU2013 version with same parameters?
Atadjanov2016: Not available! Not unified! Two different setups for
candidates thresholding for single (¡= 5) and multiple (¡= 10) as
stated in ECCV 2016 paper
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 25 / 27

26. Introduction
Related Work
Methodology
Results and Discussion
References I
[1] G. Loy and J.-O. Eklundh, “Detecting symmetry and symmetric constellations of
features,” in Computer Vision–ECCV 2006, pp. 508–521, Springer, 2006.
[2] M. Cicconet, D. Geiger, K. C. Gunsalus, and M. Werman, “Mirror symmetry
histograms for capturing geometric properties in images,” in Computer Vision and
Pattern Recognition (CVPR), 2014 IEEE Conference on, pp. 2981–2986, IEEE,
2014.
[3] M. Elawady, C. Barat, C. Ducottet, and P. Colantoni, “Global bilateral symmetry
detection using multiscale mirror histograms,” in International Conference on
Advanced Concepts for Intelligent Vision Systems, pp. 14–24, Springer, 2016.
[4] I. Rauschert, K. Brocklehurst, S. Kashyap, J. Liu, and Y. Liu, “First symmetry
detection competition: Summary and results,” tech. rep., Technical Report
CSE11-012, Department of Computer Science and Engineering, The Pennsylvania
State University, 2011.
[5] J. Liu, G. Slota, G. Zheng, Z. Wu, M. Park, S. Lee, I. Rauschert, and Y. Liu,
“Symmetry detection from realworld images competition 2013: Summary and
results,” in Computer Vision and Pattern Recognition Workshops (CVPRW), 2013
IEEE Conference on, pp. 200–205, IEEE, 2013.
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 26 / 27

27. Introduction
Related Work
Methodology
Results and Discussion
Questions?
M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 27 / 27