This document summarizes object tracking methods, including representations of objects, features for tracking, detection approaches, tracking algorithms, and future directions. It discusses representing objects as points, patches, or contours, using features like color, edges, texture, and optical flow for detection and tracking. Detection can be done through point detection, background subtraction, segmentation, and supervised learning. Tracking algorithms include point tracking, kernel tracking, and silhouette tracking. The document outlines challenges like occlusion, camera motion, and non-rigid objects that remain for future work in object tracking.

1. object tracking:a survey Nhat ‘Rich’ Nguyen Vision Seminar February 2010 Based on a paper by Yilmaz et al

2. Definition 2 Tracking is the problem of estimatingthe trajectory of an object in the image plane as it moves around a scene.

3. Applications 3 Motion Recognition Automated Surveillance Video Indexing Human Computer Interaction Traffic Monitoring Vehicle Navigation

4. Problems Projection Noises Complex shape Complex motion Non-rigid Occlusions Lighting Real-time 4

5. Questions Which object representation is suitable? Which image features should be used? How should motion, appearance of the object be modeled? 5 Help you to design an object tracking system

6. Overview Object Representations Features Selection Object Detection Object Tracking Future Direction 6

7. 1. Object Representation 7 [How to represent an object for tracking]

8. 8 Shape - Points Centroid Multiple Points Control Points

9. 9 Shape - Patches Rectangular Patch Elliptical Patch Multiple Patches

10. 10 Shape - Contour Complete Contour Skeletal Model Silhouette

11. 11 Appearance – Prob. Densities Gaussian Histogram Mixture of Gaussians

12. 12 Appearance – Models Geometric Template Active Contour Multi-view Appearance

13. 2. Feature Selection 13 [Which feature can be easily distinguished?]

14. 14 Color HSV RGB LAB

15. 15 Edges Canny Edge Detector

16. 16 Optical Flow Dense field of displacement vectors which defines the translation of each pixel

17. 17 Texture Gray-level Co-occurence Matrix

18. 18 Texture – Law’s measures 1-D: kernel for Level, Edge, Spot, Wave, and Ripple 2-D: convoluting a vertical and a horizontal 1-D kernel

19. 3. Object Detection 19 [To track, we first detect.]

20. 20 Approaches Point Detector Harris SIFT Background Subtraction Segmentation Mean shift Graph cuts Active Contours Supervised Learning Adaptive Boosting Support Vector Machines

21. 21 Point Detectors Harris SIFT

22. 22 Background Subtraction

23. 23 Segmentation

24. 24 Segmentation - Mean shift

25. 25 Segmentation - Mean shift

26. 26 Segmentation – Graph-cuts

27. 27 Segmentation – Active Contour

28. 28 Supervised Learning Learning Examples Features Supervised Learners Input Classification

29. 29 Adaptive Boosting

30. 30 Support Vector Machine

31. 4. Object Tracking 31 [State-of-the-art methods.]

32. 32 Approaches Point Tracking [Multi-point Correspondence] Kernel Tracking [Parametric Transformation] Silhouette Tracking [Contour Evolution]

33. 33 Taxonomy

34. 34 Deterministic All possible Associations Unique Associations Multi-frame Correspondence Optimal Assignment Methods: Hungarian vs. Greedy

35. 35 Motion Constraints Proximity Small change in velocity Maximum Velocity Common Motion Rigidity

36. 36 Examples Rotating dish Flying birds

37. 37 State Estimation

38. Estimate the state of a linear system. The state is Gaussian distributed. Filters 38 Kalman The state is NOT Gaussian distributed. Particle Instead of nearest neighbor, offer a probabilistic approach for data association No entering or exiting objects Joint Probability Data Association Multiple Hypothesis Exhaustively enumerate all possible associations.

39. 39 Evaluation

40. 40 Template Matching Brute force Similarity measure: cross correlation - specifies candidate template position - object template in previous frame

41. 41 Mean Shift Tracker

42. 42 KLT Feature Tracker Compute the translation of a rectangular region centered on an interest point. Evaluate the quality by computing the affine transformation between corresponding patches.

43. 43 Eigen Tracker Subspace-based approach for multi-view appearance. Uses eigenspace for similarity instead of SSD, or correlation. Allows distortion in the template.

44. 44 SVM Tracker Positive samples consist of images of the object to be tracked. Negative samples consist of images of background object. Maximizes the SVM classification score over image region to estimate the object position. Knowledge about background object is explicitly incorporated in the tracker.

45. 45 Evaluation

46. 46 Shape Matching Similar to Template Matching Use Hausdorff distance measure to identify most mismatch edges. Emphasize parts of model that are not drastically affected by object motion. Examples of a person walking : head and torso vs. arms and legs.

47. 47 State Space Model State is term of shape and motion parameters of the contour Control points of the contour moves on the spring stiffness parameters Measurements consist of the image edges computed in the normal direction of the contour

50. 50 Evaluation

51. 5. Future Direction 51 [What’s left for us?]

52. Depth information Occlusion Resolution Moving cameras Non-overlapping view Multiple Camera Tracking 52

53. Broadcast news or home videos. Noisy, compressed, unstructured, multiple views. Severe occlusion, object partially visible. Employ audio in addition to video. Unconstrained Videos 53

54. Ability to learn object model online. Unsupervised learning of object models for multiple non-rigid moving object from a single camera. Efficient Online Estimation 54

55. Require detection at some point. State-of-the-art tracking methods. Point correspondence Geometric models Contour evolution Dependency on context of use. Give valuable insight and encourage new research. Concluding Remarks 55