Robust Object Recognition with Cortex-like Mechanisms

1. Robust Object Recognition with Cortex-like Mechanisms (PAMI, 06) Presented by Ala Stolpnik T. Serre, L. Wolf, S. Bileschi, M. Riesenhuber, and T. Poggio

3. Scene Understanding Watch Out! Probably Hanging Out

4. The StreetScenes Database 3,547 Images, all taken with the same camera , of the same type of scene , and hand labeled with the same objects , using the same labeling rules . Database Performance Measures Approach sky road tree building bicycle pedestrian car Object 2562 3400 4932 5067 209 1449 5799 # Labeled Examples

5. More StreetScenes Examples Database Performance Measures Approach

6. Even More Street Scenes Examples Database Performance Measures Approach

7. Challenges: In-class variability Partial, or weak labeling Includes Rigid, Articulated and Amorphous objects Database Performance Measures Approach

8. Challenges: In-class variability Partial, or weak labeling Includes Rigid , Articulated and Amorphous objects Database Performance Measures Approach

9. Texture Sample Locations Building, Tree, Road and Sky Hand-drawn Labels Training Sample Locations Database Performance Measures Approach

10. Input image Segmented image Texture classification Windowing Crop classification Output Texture-based objects pathway (e.g., trees, road..) Shape-based objects pathway (e.g., pedestrians, cars..) car car ped Approach Two Slightly Different Pathways

11. Texture-based Object Detection Input image Classification Smoothing Over Segmentation Tree / Not-Tree Standard Model Feature Extraction Classification Database Performance Measures Approach Feature Vector Decision Feature Vector Decision

12. Shape-based Object Detection Windowing Crop classification Output car car ped Car / Not-Car Standard Model Feature Extraction Statistical learning Classification Database Performance Measures Approach Feature Vector Decision

13. Standard Model Features from a neuroscience view. Retina Complexity Approach

17. Gabor filter - rotation Input sample Thetha = 0 Thetha = 90 Approach We use 4 different orientations: 0, 45, 90, 135

18. Gabor filter - scaling Lambda = 3.5 Lambda = 22.8 Lambda = 10.3 Approach We use 16 different scales from Lambda=3.5 to 22.8

22. Input Image S1 C1 S1 C1 S2 C2 Local maximization takes place in each orientation channel separately, and also over nearby scales. Approach

24. S1 -> C1 Approach S1 C1 S2 C2

27. C2 C2 is simply the global maximum of the S2 response image. S1 C1 S2 C2 Each Prototype gives rise to one C2 value. C2 = max ( ) Size of patch, sampling rate, etc. are Parameters of the system. Approach

32. StreetScenes Database. Subjective Results Results

33. StreetScenes Database. Subjective Results Results

34. C2 vs. Sift – number of features Results

35. C2 vs. Sift – number of training examples Results

36. Object specific vs. universal features Results

38. Thanks!