Object counting in high resolution remote sensing images with OTB
1. Motivation Original Simplified Conclusion
Object counting in high resolution remote
sensing images with OTB
E. Christophe1 , J. Inglada2
1 C ENTRE FOR R EMOTE I MAGING , S ENSING AND P ROCESSING ,
N ATIONAL U NIVERSITY OF S INGAPORE
2 C ENTRE N ATIONAL D ’É TUDES S PATIALES , TOULOUSE , F RANCE
IGARSS 2009, Cape Town
2. Motivation Original Simplified Conclusion
Outline
Motivation
Original solution
Workflow
Pan Sharpening
Classification
Segmentation: Mean shift
Vector data
Simplified versions
Trade-off
Description
Results
Conclusion
IGARSS 2009, Cape Town
3. Motivation Original Simplified Conclusion
Motivation
Object counting
Correspond to a wide range of problems for remote
sensing data users
Often have to be performed on large area
Time consuming
Examples
Houses in a city: particularly for country where urban
planning data is not available
Tents in refugee camp
Tree stands in a field
IGARSS 2009, Cape Town
4. Motivation Original Simplified Conclusion
PRRS 2008 algorithm performance contest
Time constraints: can’t spend months refining the
algorithm
Deliver a result: whole processing chain required
Each step can be improved
Goal: illustrate the on the shelf approach
Contest
Count the building from a Quickbird scene over Legaspi,
Philippines
XS and Pan images were provided
Aksoy et al., ”Performance evaluation of building detection and digital surface model extraction algorithms:
Outcomes of the PRRS 2008 algorithm performance contest,” in 5th IAPR Workshop on Pattern Recognition
in Remote Sensing, Tampa, Florida, Dec. 2008.
IGARSS 2009, Cape Town
5. Motivation Original Simplified Conclusion Workflow PanSharp Classif. Segment. Vector data
Outline
Motivation
Original solution
Workflow
Pan Sharpening
Classification
Segmentation: Mean shift
Vector data
Simplified versions
Trade-off
Description
Results
Conclusion
IGARSS 2009, Cape Town
6. Motivation Original Simplified Conclusion Workflow PanSharp Classif. Segment. Vector data
Algorithm description
Pan
Mul
IGARSS 2009, Cape Town
7. Motivation Original Simplified Conclusion Workflow PanSharp Classif. Segment. Vector data
Algorithm description
Pan
Pan sharpening
Mul
IGARSS 2009, Cape Town
8. Motivation Original Simplified Conclusion Workflow PanSharp Classif. Segment. Vector data
Algorithm description
Classification
Pan
Pan sharpening
Mul
IGARSS 2009, Cape Town
9. Motivation Original Simplified Conclusion Workflow PanSharp Classif. Segment. Vector data
Algorithm description
Classification
Pan
Pan sharpening Segmentation
Mul
IGARSS 2009, Cape Town
10. Motivation Original Simplified Conclusion Workflow PanSharp Classif. Segment. Vector data
Algorithm description
Classification
Pan
Pan sharpening Segmentation Vectorization
Mul
IGARSS 2009, Cape Town
11. Motivation Original Simplified Conclusion Workflow PanSharp Classif. Segment. Vector data
Algorithm description
Classification
Pan
Pan sharpening Segmentation Vectorization
Mul
Edge dectect.
IGARSS 2009, Cape Town
12. Motivation Original Simplified Conclusion Workflow PanSharp Classif. Segment. Vector data
Algorithm description
Classification
Pan
Pan sharpening Segmentation Vectorization Refinement
Mul
Edge dectect.
IGARSS 2009, Cape Town
13. Motivation Original Simplified Conclusion Workflow PanSharp Classif. Segment. Vector data
Algorithm description
Classification
Pan
Pan sharpening Segmentation Vectorization Refinement Obj
Mul
Edge dectect.
IGARSS 2009, Cape Town
14. Motivation Original Simplified Conclusion Workflow PanSharp Classif. Segment. Vector data
Preprocessing
Pansharpening
The pansharpening is the first step to perform to take
advantage of the high resolution of the Panchromatic band
(61 cm) with the four spectral bands of the multispectral.
Pan Mul Pan Shapening
IGARSS 2009, Cape Town
15. Motivation Original Simplified Conclusion Workflow PanSharp Classif. Segment. Vector data
Classification
Obvious sources of errors
There is some obvious sources of error: boat in the middle of the water which
look like houses, cars in the middle of the street
Classification (used as a mask) can help remove these sources of error
Classification
Here we used a simple classification by SVM
Non classification specialists just provided a few samples per class (water,
vegetation, road, shadows, 4 colors of buildings)
Only a pixel classification: no use of texture here (that was before all the textures
were introduces in OTB)
IGARSS 2009, Cape Town
16. Motivation Original Simplified Conclusion Workflow PanSharp Classif. Segment. Vector data
QB scene Land cover classification
IGARSS 2009, Cape Town
17. Motivation Original Simplified Conclusion Workflow PanSharp Classif. Segment. Vector data
Segmentation: Mean shift
Too much details
Higher resolution is better but. . . sometimes, you would like
less details (roof superstructures, cars)
What details to remove?
Mean shift algorithm
D. Commaniciu,“Mean shift: A robust approach toward feature space analysis,”IEEE Transactions on Pattern
Analysis and Machine Intelligence, vol. 24, no. 5, pp. 603–619, May 2002.
IGARSS 2009, Cape Town
18. Motivation Original Simplified Conclusion Workflow PanSharp Classif. Segment. Vector data
Pan Sharpened Mean shift clustering
IGARSS 2009, Cape Town
19. Motivation Original Simplified Conclusion Workflow PanSharp Classif. Segment. Vector data
Refining the boundaries
Simplification
Easier to handle vector data than raster: vectorization
The vectorization led to too many contour point:
simplification of the points which are roughly aligned
Fine adjustment
Using an image contour as an input, an energy is
computed along the polygon contour
Introducing a random perturbation in the position of each
point, the energy is maximized
Only a very basic optimization used here (ground for
improvement)
IGARSS 2009, Cape Town
20. Motivation Original Simplified Conclusion Workflow PanSharp Classif. Segment. Vector data
Filtering
Filtering on compacity
Buildings are usually compact
A
C = 4π L2
IGARSS 2009, Cape Town
21. Motivation Original Simplified Conclusion Workflow PanSharp Classif. Segment. Vector data
Performances on the contest
Results
Two results were submitted with a difference mainly in the classification
One result was very close with 3600 building detected for 3065 in the ground
truth
Interesting to see that most other algorithms tend to underdetection while the
proposed algorithm tends to overdetect.
Evaluation criteria
{Correct, over, under, missed} detection and false alarm rates based on
Overlapping Area Matrix
Maximum-weight bipartite graph matching
Normalized Hamming distance
Clustering indices (Fowlkes-Mallows and Jaccard index)
IGARSS 2009, Cape Town
22. Motivation Original Simplified Conclusion Workflow PanSharp Classif. Segment. Vector data
Performances on the contest
Conclusion on these results
Particularly hard to conclude given the wide variety of
criteria: organizer of the contest have been careful not to
declare an overall winner
However, the proposed methods provided good
performances (particularly on the clusterings indices
criteria) with a bias towards over segmentation.
IGARSS 2009, Cape Town
23. Motivation Original Simplified Conclusion Trade-off Description Results
Outline
Motivation
Original solution
Workflow
Pan Sharpening
Classification
Segmentation: Mean shift
Vector data
Simplified versions
Trade-off
Description
Results
Conclusion
IGARSS 2009, Cape Town
24. Motivation Original Simplified Conclusion Trade-off Description Results
Trade-off of the previous method
Drawbacks
The previous method relies on the classification of the image
require a good understanding of the algorithm that follow
influence significantly the output
Simplified version
different trade-offs on complexity-performance
remove the classification step
just require the operator to click on several (2 to 5)
examples of objects
IGARSS 2009, Cape Town
25. Motivation Original Simplified Conclusion Trade-off Description Results
Simplified version
Algorithm
Produce a likelihood map of the region containing the
objects of interest
Followed by the same step as the previous algorithm:
segmentation, vectorization,. . .
Likelihood map: 2 choices
Spectral angle
One class SVM
IGARSS 2009, Cape Town
26. Motivation Original Simplified Conclusion Trade-off Description Results
Algorithm description
One class SVM
Spectral angle
Pan
Pan sharpening Segmentation Vectorization Refinement Obj
Mul
Edge dectect.
IGARSS 2009, Cape Town
27. Motivation Original Simplified Conclusion Trade-off Description Results
Results of the simplified version
Poorer preformances
Not as good as the original one (expected) ⇒ required to
understand the algorithm
Advice
Spectral angle: spectral characteristics of the objects are
stable
SVM: better when object have radiometric differences but
more samples are required
IGARSS 2009, Cape Town
28. Motivation Original Simplified Conclusion
Outline
Motivation
Original solution
Workflow
Pan Sharpening
Classification
Segmentation: Mean shift
Vector data
Simplified versions
Trade-off
Description
Results
Conclusion
IGARSS 2009, Cape Town
29. Motivation Original Simplified Conclusion
Conclusion
Modular processing chain
An application with GUI is
available
It can be used for processing
remote sensing images (no
constraint on the size)
Can be easily modified and
improved as the steps are
modular and follow the pipeline
philosophy.
IGARSS 2009, Cape Town