Injustice - Developers Among Us (SciFiDevCon 2024)
igarss.pdf
1. Introduc)on
BPT
construc)on
Pruning
strategy
Conclusions
Improved
BINARY
PARTITION
TREE
construc8on
for
hyperspectral
images:
Applica8on
to
object
detec8on
1,2
2
1
3
S.Valero
,
P.
Salembier
,
J.
Chanussot,
C.M.Cuadras
1
GIPSA-‐Lab,
Signal
&
Image
Dept,
Grenoble-‐INP,
Grenoble,
France
2
Signal
Theory
and
Comunic.
Dept,
Technical
University
of
Catalonia,
Spain
3
University
of
Barcelona,
Spain
2. Introduc)on
BPT
construc)on
Pruning
strategy
Conclusions
Outline
1 Introduc)on
v
Hyperspectral
imagery
v
BPT
2 Binary
Par))on
Tree
Construc)on
v Region
Model
v
Merging
Criterion
3 Pruning
Strategy
v
Road
detec)on
v
Building
detec)on
4 Conclusions
3. Introduc8on
BPT
construc8on
Hyperspectral
imagery
Pruning
strategy
BPT
Conclusions
Outline
1 Introduc)on
v
Hyperspectral
imagery
v
BPT
2 Binary
Par))on
Tree
Construc)on
v Region
Model
v
Merging
Criterion
3 Pruning
Strategy
v
Road
detec)on
v
Building
detec)on
4 Conclusions
4. Introduc)on
BPT
construc)on
Hyperspectral
imagery
Pruning
strategy
BPT
Conclusions
Hyperspectral
Imagery
v Different
analysis
techniques
have
been
proposed
in
the
literature.
Most
of
them
process
the
pixels
individually,
as
an
array
of
spectral
data
without
any
spa)al
structure
v Each
pixel
is
a
discrete
spectrum
containing
the
reflected
solar
radiance
of
the
X Pxy
N
spa)al
region
that
it
1
represents
Pxy
N Radiance
2
1
Wavelength Pixels
are
studied
y
as
isolated
discrete
spectra
5. Introduc8on
BPT
construc8on
Hyperspectral
imagery
Pruning
strategy
BPT
Conclusions
Hyperspectral
Imagery
v The
ini)al
pixel-‐based
representa)on
is
a
very
low
level
and
unstructured
representa)on
v Instead
of
working
with
a
pixel-‐based
representa)on,
Binary
Par88on
Trees
1
are
an
example
of
a
poweful
structured
region-‐based
image
representa)on.
2
v The
construc)on
of
Binary
Par88on
Trees
for
hyperspectral
images
has
been
recently
proposed
1 P.
Salembier
and
L.
Garrido.
Binary
par**on
tree
as
an
efficient
representa*on
for
image
processing,
segmenta*on,
and
informa*on
retrieval.
IEEE
Transac)ons
on
Image
Processing,
vol.9(4),
pp.561-‐576,
2000.
2 S.Valero,
P.Salembier
and
J.Chanussot.
New
hyperspectral
data
representa)on
using
Binary
Par))on
Tree.
IEEE
Proceedings
of
IGARSS,
2010
6. Introduc)on
BPT
construc)on
Hyperspectral
imagery
Pruning
strategy
BPT
Conclusions
BPT
v BPTs
can
be
interpreted
as
a
structured
image
representa)on
containing
a
set
of
hierarchical
regions
stored
in
a
tree
structure
v Each
node
represen)ng
a
region
in
the
image,
BPTs
allow
us
to
extract
many
par))ons
at
different
levels
of
resolu)on
7. Introduc8on
BPT
construc8on
Hyperspectral
imagery
Pruning
strategy
BPT
Conclusions
BPT
v BPTs
can
be
interpreted
as
a
structured
image
representa)on
containing
a
set
of
hierarchical
regions
stored
in
a
tree
structure
v Each
node
represen)ng
a
region
in
the
image,
BPTs
allow
us
to
extract
many
par))ons
at
different
levels
of
resolu)on
8. Introduc8on
BPT
construc8on
Hyperspectral
imagery
Pruning
strategy
BPT
Conclusions
BPT
v BPTs
can
be
interpreted
as
a
structured
image
representa)on
containing
a
set
of
hierarchical
regions
stored
in
a
tree
structure
v Each
node
represen)ng
a
region
in
the
image,
BPTs
allow
us
to
extract
many
par))ons
at
different
levels
of
resolu)on
?
How
can
BPT
be
extended
to
the
case
of
hyperspectral
data
?
9. Introduc8on
BPT
construc8on
Hyperspectral
imagery
Pruning
strategy
BPT
Conclusions
Aim:
BPT
for
HS
image
analysis
Hyperspectral
Classifica)on
image
CONSTRUCTION
PRUNING
Object
detec)on
Segmenta)on
v We
propose
to
construct
a
BPT
in
order
to
represent
an
HS
image
with
a
new
region-‐based
hierarchical
representa)on
Hyperspectral
BPT
image
representa)on
10. Introduc8on
BPT
construc8on
Hyperspectral
imagery
Pruning
strategy
BPT
Conclusions
Aim:
BPT
for
HS
image
analysis
Hyperspectral
Classifica)on
image
CONSTRUCTION
PRUNING
Object
detec)on
Segmenta)on
v In
this
paper:
Pruning
strategy
aiming
at
object
detec)on
is
proposed
BPT
Hyperspectral
Search
image
The
pruning
look
for
regions
characterized
by
some
features
11. Introduc8on
BPT
construc8on
Region
Model
Pruning
strategy
Merging
Criterion
Conclusions
Outline
1 Introduc)on
v
Hyperspectral
imagery
v
BPT
2 Binary
Par))on
Tree
Construc)on
v Region
Model
v
Merging
Criterion
3 Pruning
Strategy
v
Road
detec)on
v
Building
detec)on
4 Conclusions
12. Introduc8on
BPT
construc8on
Pruning
strategy
Conclusions
Hyperspectral
Imagery
v The
BPT
is
a
hierarchical
tree
G
structure
represen)ng
an
image
G
v The
tree
leaves
correspond
to
individual
pixels,
whereas
the
root
E
represents
the
en)re
image
F
F
v The
remaining
nodes
represent
regions
formed
by
the
merging
of
E
two
children
E
C
D
C
D
v The
tree
construc)on
is
performed
by
an
itera)ve
region
merging
algorithm
A
B
B
A
B
C
D
13. Introduc8on
BPT
construc8on
Pruning
strategy
Conclusions
Hyperspectral
Imagery
v The
BPT
is
a
hierarchical
tree
structure
represen)ng
an
image
v The
tree
leaves
correspond
to
individual
pixels,
whereas
the
root
represents
the
en)re
image
v The
remaining
nodes
represent
regions
formed
by
the
merging
of
two
children
C
D
v The
tree
construc)on
is
performed
by
an
itera)ve
region
merging
algorithm
A
B
B
A
B
C
D
14. Introduc8on
BPT
construc8on
Pruning
strategy
Conclusions
Hyperspectral
Imagery
v The
BPT
is
a
hierarchical
tree
structure
represen)ng
an
image
v The
tree
leaves
correspond
to
individual
pixels,
whereas
the
root
represents
the
en)re
image
v The
remaining
nodes
represent
regions
formed
by
the
merging
of
E
two
children
E
C
D
C
D
v The
tree
construc)on
is
performed
by
an
itera)ve
region
merging
algorithm
A
B
B
A
B
C
D
15. Introduc8on
BPT
construc8on
Pruning
strategy
Conclusions
Hyperspectral
Imagery
v The
BPT
is
a
hierarchical
tree
structure
represen)ng
an
image
v The
tree
leaves
correspond
to
individual
pixels,
whereas
the
root
E
represents
the
en)re
image
F
F
v The
remaining
nodes
represent
regions
formed
by
the
merging
of
E
two
children
E
C
D
C
D
v The
tree
construc)on
is
performed
by
an
itera)ve
region
merging
algorithm
A
B
B
A
B
C
D
16. Introduc8on
BPT
construc8on
Pruning
strategy
Conclusions
Hyperspectral
Imagery
v The
BPT
is
a
hierarchical
tree
G
structure
represen)ng
an
image
G
v The
tree
leaves
correspond
to
individual
pixels,
whereas
the
root
E
represents
the
en)re
image
F
F
v The
remaining
nodes
represent
regions
formed
by
the
merging
of
E
two
children
E
C
D
C
D
v The
tree
construc)on
is
performed
by
an
itera)ve
region
merging
algorithm
A
B
B
A
B
C
D
17. Introduc8on
BPT
construc8on
Region
Model
Pruning
strategy
Merging
Criterion
Conclusions
Hyperspectral
Imagery
The
crea)on
of
BPT
implies
two
important
no)ons
Rij
v Region
model
MRi
It
specifies
how
an
hyperspectral
region
O(Ri,Rj)
is
represented
and
how
to
model
the
O(Ri,Rj)
union
of
two
regions.
Rj
Ri
v Merging
criterion
O(Ri,Rj)
The
similarity
between
neighboring
regions
determining
the
merging
order
18. Introduc8on
BPT
construc8on
Region
Model
Pruning
strategy
Merging
Criterion
Conclusions
Aim:
BPT
for
HS
image
analysis
Hyperspectral
Classifica)on
image
CONSTRUCTION
PRUNING
Object
detec)on
Segmenta)on
v How
to
represent
hyperspectral
image
regions?
v Which
similarity
measure
defines
a
good
merging
order?
19. Introduc8on
BPT
construc8on
Region
Model
Pruning
strategy
Merging
Criterion
Conclusions
Region
Model
We
propose
a
non-‐parametric
sta)s)cal
region
model
2
Radiance
Pixel
1
consis)ng
in
a
set
of
N
probability
density
Pixel
2
func)ons
Pixel
3
A
B
where
each
Pi
represents
the
probabili)y
that
the
spectra
data
Wavelength
λ
set
has
a
specific
radiance
value
λi
in
the
wavelength
λi
Hλi
2
F.
Calderero
and
F.
Marques.Region-‐Merging
techniques
using
informa*on
theory
sta*s*cal
measures.
IEEE
Transac)ons
on
Image
Processing,
vol.19,
pp.1567-‐1586,
2010.
B
A
Nbins
20. Introduc8on
BPT
construc8on
Region
Model
Pruning
strategy
Merging
Criterion
Conclusions
Aim:
BPT
for
HS
image
analysis
Hyperspectral
Classifica)on
image
CONSTRUCTION
PRUNING
Object
detec)on
Segmenta)on
v How
to
represent
hyperspectral
image
regions?
v Which
similarity
measure
defines
a
good
merging
order?
21. Introduc8on
BPT
construc8on
Region
Model
Pruning
strategy
Merging
Criterion
Conclusions
Merging
Criterion
Step
1
Step
2
Principal
Coordinates
Mul8dimensional
Scaling
Associa8on
Measure
Mul8dimensional
Scaling
Principal
Coordinates
22. Introduc8on
BPT
construc8on
Region
Model
Pruning
strategy
Merging
Criterion
Conclusions
Merging
Criterion
Step
1
Mul8dimensional
Principal
Coordinates
Scaling
Analyze
the
inter-‐waveband
similarity
rela)onships
for
each
data
via
metric
scaling
to
obtain
the
principal
coordinates
Mul8dimensional
Scaling
Principal
Coordinates
23. Introduc8on
BPT
construc8on
Region
Model
Pruning
strategy
Merging
Criterion
Conclusions
Merging
Criterion:
Step
2
Step
2
Principal
Coordinates
PC1
An
associa)on
measure
Mul8dimensional
is
defined
by
considering
Scaling
that
PC1
and
PC2
form
a
Associa8on
mul)variate
regression
Measure
model
Mul8dimensional
Scaling
Principal
Coordinates
PC2
PC1=
PC2
β
+e
Are
Regression
Coefficients
equal
to
0
???
24. Introduc8on
BPT
construc8on
Pruning
strategy
Conclusions
Rosis
Hyperspectral
data
Data
Set
:
Rosis
Pavia
Center
103
bands
RGB
Composi)on
103
bands
BPT
is
constructed
by
using
the
proposed
merging
order
Nregions=22
Nregions=32
Nregions=42
25. Introduc8on
BPT
construc8on
Pruning
strategy
Conclusions
Rosis
Hyperspectral
data
Ground
truth
manually
created
A
symmetric
distance
for
object
evalua)on:
It
is
defined
as
the
he
minimum
number
of
pixels
whose
labels
should
be
changed
to
achieve
perfect
matching,
normalized
by
the
total
number
of
pixels
of
the
image
minus
one
NRegions=22
NRegions=22
Symmetric
distance
Symmetric
distance
to
ground
truth
to
ground
truth
equal
to
0.48
equal
to
0.227
[Tilton,
2005]
RHSEG
soiware
Hierarchical
BPT
level
26. Introduc8on
BPT
construc8on
Road
detec)on
Pruning
strategy
Building
detec)on
Conclusions
Outline
1 Introduc)on
v
Hyperspectral
imagery
v
BPT
2 Binary
Par))on
Tree
Construc)on
v Region
Model
v
Merging
Criterion
3 Pruning
Strategy
v
Road
detec)on
v
Building
detec)on
4 Conclusions
27. Introduc8on
BPT
construc8on
Hyperspectral
imagery
Pruning
strategy
BPT
Conclusions
Aim:
BPT
for
HS
image
analysis
Hyperspectral
Classifica)on
image
CONSTRUCTION
PRUNING
Object
detec)on
Segmenta)on
v Pruning
strategy
aiming
at
object
detec)on
is
proposed
BPT
Hyperspectral
Search
image
The
pruning
look
for
regions
characterized
by
some
features
28. Introduc8on
BPT
construc8on
Road
detec)on
Pruning
strategy
Building
detec)on
Conclusions
Object
Detec8on
Strategy
v Hyperspectral
object
detec)on
has
been
mainly
developed
in
the
context
of
pixel-‐wise
spectral
classifica)on.
v Objets
are
not
only
characterized
by
their
spectral
signature.
v Spa)al
features
such
as
shape,
area,
orienta)on
can
also
contribute
significantly
to
the
detec)on.
v Roads
appear
as
elongated
structures
having
fairly
homogeneous
radiance
values
usually
corresponding
to
asphalt.
Besides
the
informa8on
provided
by
asphalt
spectrum,
a
road
contains
important
spa8al
features
29. scheme. The strategy is to analyze the BPT using a set of descrip- with the classical RHSEG [11]. In the case of R
scheme. The strategy is to analyze the BPT using a set of descrip- ity criterion used is SAM with spectral clustering w
Introduc8on
forfor each node. The work presented here proposes the
tors computed
tors computed
each node. The work presented here proposes the ity criterion used is SAM with spectral clustering
evaluate the resulting partitions, the symmetric di
analysis of three different descriptors for each node: evaluate the resulting partitions, the symmetric
BPT
construc8on
different descriptors for each node:
analysis of three Road
detec)on
is used as a partition quality evaluation. Having a
is used as a partition quality evaluation. Having
Pruning
strategy
D = {Dshape , Dspectral , Darea } ground truth GT , the symmetric distance corresp
Building
detec)on
D = {Dshape , Dspectral , Darea }
(7)
(7)
ground truth GT , the symmetric distance corre
mum number of pixels whose labels should be cha
Conclusions
mum number of pixels whose labels should be
The proposed shape, spectral and area descriptors are related P to achieve a perfect matching with GT , norma
The proposed shape, spectral and area descriptors are related P to achieve a perfect matching with GT , nor
to the specficic object of interest. Studying D from the leaves to number of pixels in the image. The manually crea
to the specficic object of interest. Studying D from the leaves to number of pixels in the image. The manually c
Object
Detec8on
Strategy
the root, the approach consists in removing all nodes that signifi- in Fig. 4(b).
the root, the approach consists in removing all nodes that signifi- in Fig. 4(b).
cantly differ from the characterization proposed by a reference Dref . Fig. 4(c)(d) show the segmentation results obtain
cantly differ from the characterization proposed by a reference Dref . Fig. 4(c)(d) show the segmentation results obta
Hence, given this reference, the idea consists in considering that the RHSEG, respectively. In both cases, the resulting
Hence, given this reference, the idea consists in considering that the RHSEG, respectively. In both cases, the resultin
searched object instances are defined by the closest nodes to the root 63 regions. Comparing both results, the quantiti
searched object instances are defined by the closest nodes to the root 63 regions. Comparing both results, the quan
node that have descriptors close to the Dref . model. In order to visual evaluation corroborates that the partition ex
node that have descriptors close to the Dref . model. In order to visual evaluation corroborates that the partition
illustrate the generality of the approach, we describe two detection BPT are much closer to the ground truth than the on
illustrate the generality of the approach, we describe two detection BPT are much closer to the ground truth than the
examples: roads and building in urban scenes. RHSEG. This experiment has been done with sev
examples: roads and building in urban scenes. RHSEG. This experiment has been done with s
acquired by different hyperspectral sensors, but, d
acquired by different hyperspectral sensors, but
tations, we can only present one data set. Howeve
3.1. Detection of roads
3.1. Detection of roads We
analyse
each
tations, we can only present one data set. Howe
were the same on the remaining dataset.
were the same on the remaining dataset.
Roads appear as elongated structures having fairly homogeneous ra- A second set of experiments are conducted no
Roads appear as elongated structures having fairly homogeneous ra-
diance values usually corresponding to asphalt. Given their charac- BPT
to
look
for
the
A second set of experiments are conducted
tection and recognition. We compare the classica
v Firstly,
the
spa)al
and
the
spectral
diance values usually corresponding to asphalt. Given their charac-
teristic shape, Dshape is the elongation of the region. In order to
tection and recognition. We compare the class
sification against the strategy proposed in section 3
compute it, we first define the smallest rectangular bounding box
descriptors
are
computed
for
all
compute it, we first define the smallest rectangular bounding box nodes
having
teristic shape, Dshape is the elongation of the region. In order to sification against the strategy proposed in sectio
road detection. The pixel-wise classification con
road detection. The pixel-wise classification c
specific
spa8al
and
BPT
nodes
spectral
descriptors
v Secondly,
following
a
bojom-‐up
strategy,
the
pruning
select
nodes
closer
to
the
root
which
have
a
low
elonga)on,
a
high
correla)on
between
asphalt
and
an
area
higher
than
a
threshold
Star8ng
from
the
leaves
30. Introduc8on
BPT
construc8on
Road
detec8on
Pruning
strategy
Building
detec)on
Conclusions
Object
Detec8on:
Example
of
Roads
v Roads
appear
as
The
ra)o
between
the
height
and
the
width
of
the
minimum
elongated
structures
bounding
box
containing
the
region
Oriented
Bounding
Box
containing
the
region
height
width
31. Introduc8on
BPT
construc8on
Road
detec8on
Pruning
strategy
Building
detec)on
Conclusions
Object
Detec8on:
Example
of
Roads
v Roads
have
fairly
Correla)on
coefficient
between
the
mean
spectra
of
the
homogeneous
radiance
region
and
the
asphalt
reference
spectrum
values
usually
corresponding
to
asphalt
Mean
Correla)on
Spectrum
Coefficient
Pixel
1
Asphalt
reference
spectrum
Radiance
Pixel
2
Pixel
3
Wavelength
λ
32. Introduc8on
BPT
construc8on
Road
detec)on
Pruning
strategy
Building
detec)on
Conclusions
Object
Detec8on:
Example
of
Roads
Par88ons
contained
in
BPT
Hydice
Hyperspectral
image
27
regions
37
regions
57
regions
Pixel-‐wise
Asphalt
detec)on
BPT
pruning
strategy
(Spectra
whose
Correla)on
oriented
to
object
with
asphalt
is
higher
than
0.9)
detec)on
33. Introduc8on
BPT
construc8on
Road
detec)on
Pruning
strategy
Building
detec)on
Conclusions
Object
Detec8on:
Example
of
Building
Par88ons
contained
in
BPT
Hydice
Hyperspectral
image
27
regions
37
regions
57
regions
Pixel-‐wise
Building
detec)on
BPT
pruning
strategy
(Spectra
whose
Correla)on
with
oriented
to
object
reference
is
higher
than
0.9)
detec)on
34. Introduc)on
BPT
construc)on
Pruning
strategy
Conclusions
Conclusions
v A
new
region
merging
algorithm
has
been
presented
here
to
construct
BPTs
as
a
hyperspectral
region-‐based
and
hierarchical
representa)on.
v Being
a
generic
representa)on,
many
tree
processing
techniques
can
be
formulated
as
pruning
strategies
for
many
applica)ons.
Here,
as
an
example
of
BPT
processing,
a
pruning
strategy
has
been
proposed
for
object
detec)on.
v A
new
similarity
measure
for
merging
hyperspectral
regions
have
been
proposed
taking
into
account
spa)al
and
spectral
correla)ons.
It
introduces
a
local
dimension
reduc)on
which
is
different
from
the
classical
point
of
view
where
the
reduc)on
is
applied
globally
on
the
en)re
image.
v The
processing
example
has
shown
the
advantage
of
using
region-‐based
representa)ons.