This document presents an approach called Pairwise Geometric Matching (PGM) for improving object retrieval in large image datasets. PGM uses the pairwise relations between matched local image features to encode global geometric transformations between images. It consists of three main steps: handling redundant matches, finding global geometric relations, and enforcing local pairwise relations based on the global information. Experiments on standard datasets show PGM outperforms state-of-the-art methods for relevant-irrelevant classification and retrieval accuracy while maintaining efficient computation times for large-scale retrieval.

1. Pairwise Geometric Matching
for Large-scale Object
Retrieval (CVPR ’15)
Xinchao Li, Martha Larson, Alan Hanjalic
Delft Multimedia Information Retrieval Lab, Netherlands

2. Pairwise Geometric Matching for Large-scale Object Retrieval
Introduction
• Object-based Image Retrieval
Query Image
Image Corpus
Relevent Images

3. Pairwise Geometric Matching for Large-scale Object Retrieval
Introduction
• General System
• Initial ranking:
compact image representation
BOF, HE, VLAD
• Spatial verification:
geometric details of the matching points
• Model-based methods with RANSAC
• Model-free methods

4. Pairwise Geometric Matching for Large-scale Object Retrieval
Introduction
• General System
• Initial ranking:
compact image representation
BOF, HE, VLAD
• Spatial verification:
geometric details of the matching points
• Model-based methods with RANSAC
• Model-free methods

5. Pairwise Geometric Matching for Large-scale Object Retrieval
Key Idea
Query Image Relevent Image

6. Pairwise Geometric Matching for Large-scale Object Retrieval
Key Idea
Query Image Relevent Image

7. Pairwise Geometric Matching for Large-scale Object Retrieval
Key Idea
• The pairwise relations between correspondences reflect global
geometric relations between the two images.

8. Pairwise Geometric Matching for Large-scale Object Retrieval
Approach
We realize our idea with a Pairwise Geometric Matching (PGM)
approach, which consists of three main steps:
• 1vs1, handle the redundancy of one-to-many correspondences.
• HV, find the global geometric relations between images.
• PG, use the global geometric relations to enforce the local
pairwise relations of pairs of correspondences.

9. Pairwise Geometric Matching for Large-scale Object Retrieval
Approach

10. Pairwise Geometric Matching for Large-scale Object Retrieval
Experiments
• Dataset
• Three classic datasets: Oxford, Holidays, and Barcelona
• Distractors: 10 million geo-tagged photos from Flickr
which are distributed all around the world, except for
Oxford and Barcelona regions.

11. Pairwise Geometric Matching for Large-scale Object Retrieval
Experiments
• Evaluation procedure
• Relevent-irrelevant classification: precision-recall
• Retrieval: mAP
• Computational efficiency: run time
• Implementation
• Object Retrieval Framework: BOF, HE
• MapReduce-based distributed fashion with Hadoop
cluster from SURFsara[1] : 1500 cores.
[1] The Dutch national e-infrastructure with the support of SURF Foundation.

12. Pairwise Geometric Matching for Large-scale Object Retrieval
• Relevant-irrelevant classification: precision-recall
[1] HPM: Y. Avrithis and G. Tolias. Hough pyramid matching: Speeded-up geometry re-
ranking for large scale image retrieval. IJCV, 107(1):1–19, 2014.

13. Pairwise Geometric Matching for Large-scale Object Retrieval
• Retrieval: mAP of BOF-based and HE-based systems against
different sizes of image database with fixed reranking range.

14. Pairwise Geometric Matching for Large-scale Object Retrieval
• Retrieval: mAP of BOF-based system against 1M image
database with different reranking ranges.

15. Pairwise Geometric Matching for Large-scale Object Retrieval
• Computational efficiency: run time
• Distribution of the percentage of selected matches
• Run time efficiency

16. Pairwise Geometric Matching for Large-scale Object Retrieval
Conclusion
The results indicate the suitability of PGM as a
solution for large-scale object retrieval at an
acceptable computational cost.
The superiority of PGM compared to the state-
of-the-art solutions is achieved by using global
scale and rotation relations to enforce the local
consistency of geometric relations derived from
the locations of pairwise correspondences.

17. Pairwise Geometric Matching for Large-scale Object Retrieval
Thank you!
X.Li-3@tudelft.nl