1
Location Recognition Given: A query image A database of images with known locations Two types of approaches: Direct matching: directly match image features to 3D points (high memory requirement) Retrieval based: retrieve a short list of most similar images and perform image matching We base our approach on retrieval based framework, but using a different representation

4
Locations in an image graph Question: How can we learn from the image graph? One approach is to formulate as a similarity measure learning problem

5
Global similarity learning Idea: similar to learning for image matching, learn a global similarity measure using matching/non-matching image pairs Problem: not specific enough for recognize location within a large dataset

11
Approach At training time: 1.Obtain image matching graph 2.Compute a covering of the graph with a set of subgraphsLearn and calibrate an SVM-based similarity measure for each subgraph At testing time: 1.Use the models in Step 3 to compute a similarity between the query image to each database image, and generate a ranked shortlist of possible image matchesPerform geometric verification with the top database images in the shortlist (image matching)

12
Approach At training time: 1.Obtain image matching graph 2.Compute a covering of the graph with a set of subgraphsLearn and calibrate an SVM-based similarity measure for each subgraph At testing time: 1.Use the models in Step 3 to compute a similarity between the query image to each database image, and generate a ranked shortlist of possible image matchesPerform geometric verification with the top database images in the shortlist (image matching)

13
Computing Subgraphs What makes a good subgraph? contain images that are largely similarhas as many positive examples as possible Minimum dominating set (NP-hard) We use a greedy algorithm: at each iteration select the image that covers the most “uncovered” images “Covered”: is selected or has at least one neighbor that is selected Selected images (center images) define the neighborhoods One image could belong to multiple neighborhoods

15
Approach At training time: 1.Obtain image matching graph 2.Compute a covering of the graph with a set of subgraphsLearn and calibrate an SVM-based similarity measure for each subgraph At testing time: 1.Use the models in Step 3 to compute a similarity between the query image to each database image, and generate a ranked shortlist of possible image matchesPerform geometric verification with the top database images in the shortlist (image matching)

16
Similarity Measure Learning Each image is represented as BoW vector For each subgraph, define Positives: subgraph membersNegatives: subsample the rest Learn an SVM classifier and calibrate to output a probability value using Platt’s method

17
Approach At training time: 1.Obtain image matching graph 2.Compute a covering of the graph with a set of subgraphsLearn and calibrate an SVM-based similarity measure for each subgraph At testing time: 1.Use the models in Step 3 to compute a similarity between the query image to each database image, and generate a ranked shortlist of possible image matchesPerform geometric verification with the top database images in the shortlist (image matching)

18
Generating a Short List Given a query image, every subgraph has a probability score However, a short list of most similar images from all database images is needed A simple approach: concatenate subgraph members according to scores rank within each subgraph using BoW similarity for image appearing in multiple subgraphs, only count the highest scoring subgraph

19
Generating a Short List A simple approach: concatenating subgraph members according to scores

20
Problem: If the first subgraph/neighborhood is wrong, the result is worse than BoW ranking, which has more diversity Idea: to encourage diversity, rank using conditional probabilities conditioned on first failure to obtain the 2nd candidate, and so on Generating a Short List update factor

21
Generating a Short List Example ranking result with encouraged diversity v.s. without Regularization using BoW ranking also helps increase diversity

22
Approach At training time: 1.Obtain image matching graph 2.Compute a covering of the graph with a set of subgraphsLearn and calibrate an SVM-based similarity measure for each subgraph At testing time: 1.Use the models in Step 3 to compute a similarity between the query image to each database image, and generate a ranked shortlist of possible image matchesPerform geometric verification with the top database images in the shortlist (image matching)

23
Experiments Key bottleneck: quality of the shortlist The higher the first match appears, the less computation is needed different from retrieval, where recall is also important Evaluate using accuracies at top k (k = 1, 2, 5, 10) Baselines 1.BoW ranking 2.Co-ocset (BoW ranking considering co-occuring stat.) 3.GPS based approach 4.Two alternative learning formulations: a.single global similarity using image pairs b.instance similarity using each image as center