SlideShare une entreprise Scribd logo
1  sur  39
Télécharger pour lire hors ligne
REAL-TIME OBJECT DETECTION
AND TRACKING
                                  By:
                   Vanya V. Valindria
                    Hammad Naeem
                             Rui Hua
Outline
 • Introduction
 • Hardware in RT Object Detetion & Tracking
 • Methods
      Traditional Methods:    Modern Methods:
      Absolute Differences
         Census Method            KLT
     Feature Based Method       Meanshift


 • Result and Conclusion
Introduction
Definition:
Object detection
 detect a particular
 object in an image

Object tracking
 to track an object
(or multiple objects)
over a sequence of
images
Application: Traffic Information




http://www.youtube.com/watch?v=vA35sXbn7zs
Application: Surveillance




http://www.youtube.com/watch?v=o25fClk9cdg
Application: Mobile Robot




http://www.youtube.com/watch?v=Q4zycRGJFFs
Problems??
• Temporal variation/dynamic environment

• Abrupt object or camera motion

• Multi-camera? Multi-objects?

• Computational expensive
Hardware in Real-time Tracking

• MEMORY
Important  Tracking system encountering limited memory
  problems.


• FRAME RATE
 ~30 FPS

• PROCESSORS - DSP
• Allow saturated arithmetic operation
• Powerful operation ability
• Can do several memory accesses in a single instruction
Real Time Object Tracking
Object Detection and Tracking
• In a video sequence an object is said to be in
  motion, if it is changing its location with
  respect to its background

• The motion tracking is actually the process of
  keeping tracks of that moving object in video
  sequence i.e. position of moving object at
  certain time etc.
Flow Chart
                Idle


               Image
             acquisition


              Object
             Detection


               Image
             acquisition


               Object
              tracking



               Object
                           No
               Lost?



                  Y es
Method 1: Absolute Differences
= Image subtraction

        D(t)=I(ti) – I(tj)

Gives an image frame with changed and
 unchanged regions


Ideal Case for no motion: I(ti) = I(tj), D(t)=0
Moving
objects
are
detected
Results:




     Frame1                   Frame10




           Difference of Two Frames
Absolute Difference
Methods for Motion Detection

     Frame Differencing
     Background Subtraction

Draw Backs:

involves a lot of computations

 Not feasible for DSP implementation
Method 2: Census Transforms
124   74    32                                     1    1       0

124   64    18   If (Center pixel < Neighbor
                                                   1    x       0
                            pixel)
                      Neighbor pixel = 1
157   116   84                                     1    1       1


                              Signature Vector Generation

                                               1   1        0

                                               1   x        0
                     Signature Vector
      11011101                                 1   1        1
Image
128 26       125 243 87                                         Signature Vectors
96    76     43     236 125
                                                                10110101
                                                                00101011
128 129 235 229 209            Signatur vector generation for
                                                                    .
                                          all pixels
228 251 229 221 234
                                                                    .
                                                                    .
227 221 35          58   98                                     10111010

                     List                10110101
                  Generation                                       List
                                         00101011               population
                                         .
                                         .
                                         .
                    Generated List       10111010


                                                  Signature vector
                                                      matching
Census Transform:
 Advantages:

  Compare only two values 0 or 1.
  Similar Illumination Variation for pixel and
  neighbouring pixels

 Draw Backs:

  As we only deal with only 0`s and 1`s, this method is
  sensitive to noise.
  Calculate, store and match process  computationally
  Expensive
Method 3: Morphology Based
 Object Tracking


Background       Frame         Object
 estimation   differencing   Registration
Morphology Based Object Tracking
           • Image Differencing
Background • Thresholding
Estimation



               • Contours are registered
  Object       • Width, height and histogram are recorded for each contour
Registration



               • Each object represented by a feature vector (the length,
  Feature        width, area and histogram of the object)
   Vector
Segmented object




  Tracked object
Morphology Based Techniques
Advantages:

Can Track Multiple objects  Objects are registered
 based on their anatomy
 Helpful for Object Merging

Draw Backs:

Object registration  complex and slow process
 For multiple object registration per frame  more
 complex
Method 4: Lucas-Kanade Technique
• Visual motion pattern of objects and surface in a
  scene  by Optical Flow




              Frame 1          Frame 2
Method 5: Mean shift
• An algorithm that iteratively shifts a data point to the
  average of data points in its neighborhood

    Choose a search window         Compute the MEAN
   size in the initial location      location in the
                                     search window




          Repeat until            Center the search
          convergence                 window
                                    at the mean
Intuitive Description
                                                Region of
                                                 interest

                                                Center of
                                                  mass




                                                Mean Shift
                                                 vector
          Objective : Find the densest region
             Distribution of identical balls
Intuitive Description                           Region of
                                                 interest

                                                Center of
                                                  mass




                                                Mean Shift
                                                 vector
          Objective : Find the densest region
             Distribution of identical balls
Intuitive Description
                                                Region of
                                                 interest

                                                Center of
                                                  mass




                                                Mean Shift
                                                 vector
          Objective : Find the densest region
             Distribution of identical balls
Intuitive Description
                                                Region of
                                                 interest

                                                Center of
                                                  mass




                                                Mean Shift
                                                 vector
          Objective : Find the densest region
             Distribution of identical balls
Intuitive Description
                                                Region of
                                                 interest

                                                Center of
                                                  mass




                                                Mean Shift
                                                 vector
          Objective : Find the densest region
             Distribution of identical balls
Intuitive Description
                                                Region of
                                                 interest

                                                Center of
                                                  mass




                                                Mean Shift
                                                 vector
          Objective : Find the densest region
             Distribution of identical balls
Intuitive Description
                                                Region of
                                                 interest

                                                Center of
                                                  mass




          Objective : Find the densest region
             Distribution of identical balls
Process
CAMSHIFT
--Continously Adaptive Meanshift
Modified to adapt dynamically to the colour
 probability distributions
More real time

        For each frame-> MEAN-SHIFT is
         applied with several iteration



          Store the location of the mean and
          calculate new window size for next
                         frame
New development
• Combine with different features. SIFT features,
  colour feature & texture information

• Camshift algorithm combined with the Kalman
  filter.
Result
                         Arithmetic and   Time taken
            Algorithm         Logic            by
                           operations      Algorithm
          Absolute
                            4230100           16
         Differencing
           Census
                            2416000           5. 4
          Transform
         Morphological
                            352210            14.2
           Tracking
         Kanade Lucas       500825           0.486
Comparison
                                               Computationally
                        Easy to implement
                                                  expensive
 Absolute Differences   Allows continuous        Slow and low
                             tracking                accuracy

                                               Computationally
                                                 expensive
   Census Transform     Immune to noise and
                        Illumination changes    Complex if 
                                               Multiple objects
                                                 per frame


                        Can track multiple          Slow
    Feature Based
                           objects well
                                                Large Memory
                                                 consumption
Comparison

                            High accuracy
         KLT
                         Less execution time   Large memory

                         Robust to noise and
                           dynamic scene



                                                Ineffective if
                        Computationally less   there is heavy
 MeanShift & CAMShift       expensive             occlusion
Conclusion
• KLT algorithm has the best performance with
  higher accuracy and less computation time

• It requires combination of methods to achieve
  the appropriate object detection and tracking
  according to the proposed scenario
References
 •   S. Shah, T. Khattak, M. Farooq, Y. Khawaja, A. Bais, A. Anees, and M. Khan, “Real Time Object
     Tracking in a Video Sequence Using a Fixed Point DSP,” Advances in Visual Computing, pp. 879–
     888.

 •   K. Huang, L. Wang, T. Tan, and S. Maybank, “A real-time object detecting and tracking system
     for outdoor night surveillance,” Pattern Recognition, vol. 41, no. 1, pp. 432–444, 2008.

 •   J. Li, F. Li, and M. Zhang, “A Real-time Detecting and Tracking Method for Moving Objects Based
     on Color Video,” in 2009 Sixth International Conference on Computer Graphics, Imaging and
     Visualization. IEEE, 2009, pp. 317–322.

 •   W. Junqiu and Y. Yagi, “Integrating color and shapetexture features for adaptive real-time
     object tracking,” IEEE Trans on Image Processing, vol. 17, no. 2, pp. 235–240, 2008.

 •   Q. Wang and Z. Gao, “Study on a Real-Time Image Object Tracking System,” in Computer
     Science and Computational Technology, 2008. ISCSCT’08. International Symposium on, vol. 2,
     2008.

 •   Y. Meng, “Agent-based reconfigurable architecture for real-time object tracking,” Journal of
     Real-Time Image Processing, vol. 4, no. 4, pp. 339–351, 2009.

 •   [Y. Yao, C. Chen, A. Koschan, and M. Abidi, “Adaptive online camera coordination for multi-
     camera multi-target surveillance,” Computer Vision and Image Understanding, 2010.

Contenu connexe

Tendances

Introduction to object detection
Introduction to object detectionIntroduction to object detection
Introduction to object detectionBrodmann17
 
Object tracking a survey
Object tracking a surveyObject tracking a survey
Object tracking a surveyHaseeb Hassan
 
Object detection
Object detectionObject detection
Object detectionSomesh Vyas
 
Object detection presentation
Object detection presentationObject detection presentation
Object detection presentationAshwinBicholiya
 
Multiple object detection
Multiple object detectionMultiple object detection
Multiple object detectionSAURABH KUMAR
 
Visual Object Tracking: review
Visual Object Tracking: reviewVisual Object Tracking: review
Visual Object Tracking: reviewDmytro Mishkin
 
Deep learning based object detection basics
Deep learning based object detection basicsDeep learning based object detection basics
Deep learning based object detection basicsBrodmann17
 
Object Detection using Deep Neural Networks
Object Detection using Deep Neural NetworksObject Detection using Deep Neural Networks
Object Detection using Deep Neural NetworksUsman Qayyum
 
Multiple Object Tracking
Multiple Object TrackingMultiple Object Tracking
Multiple Object TrackingRainakSharma
 
Object Detection and Recognition
Object Detection and Recognition Object Detection and Recognition
Object Detection and Recognition Intel Nervana
 
Object detection with deep learning
Object detection with deep learningObject detection with deep learning
Object detection with deep learningSushant Shrivastava
 
Introduction to multiple object tracking
Introduction to multiple object trackingIntroduction to multiple object tracking
Introduction to multiple object trackingFan Yang
 
Real Time Object Dectection using machine learning
Real Time Object Dectection using machine learningReal Time Object Dectection using machine learning
Real Time Object Dectection using machine learningpratik pratyay
 
Deep sort and sort paper introduce presentation
Deep sort and sort paper introduce presentationDeep sort and sort paper introduce presentation
Deep sort and sort paper introduce presentation경훈 김
 
Deep learning for object detection
Deep learning for object detectionDeep learning for object detection
Deep learning for object detectionWenjing Chen
 
Moving Object Detection And Tracking Using CNN
Moving Object Detection And Tracking Using CNNMoving Object Detection And Tracking Using CNN
Moving Object Detection And Tracking Using CNNNITISHKUMAR1401
 

Tendances (20)

Object detection
Object detectionObject detection
Object detection
 
Introduction to object detection
Introduction to object detectionIntroduction to object detection
Introduction to object detection
 
Object tracking a survey
Object tracking a surveyObject tracking a survey
Object tracking a survey
 
Object detection
Object detectionObject detection
Object detection
 
Object detection presentation
Object detection presentationObject detection presentation
Object detection presentation
 
Object tracking final
Object tracking finalObject tracking final
Object tracking final
 
Moving object detection
Moving object detectionMoving object detection
Moving object detection
 
Multiple object detection
Multiple object detectionMultiple object detection
Multiple object detection
 
Visual Object Tracking: review
Visual Object Tracking: reviewVisual Object Tracking: review
Visual Object Tracking: review
 
Deep learning based object detection basics
Deep learning based object detection basicsDeep learning based object detection basics
Deep learning based object detection basics
 
Object Detection using Deep Neural Networks
Object Detection using Deep Neural NetworksObject Detection using Deep Neural Networks
Object Detection using Deep Neural Networks
 
Multiple Object Tracking
Multiple Object TrackingMultiple Object Tracking
Multiple Object Tracking
 
Object Detection and Recognition
Object Detection and Recognition Object Detection and Recognition
Object Detection and Recognition
 
Object detection with deep learning
Object detection with deep learningObject detection with deep learning
Object detection with deep learning
 
Introduction to multiple object tracking
Introduction to multiple object trackingIntroduction to multiple object tracking
Introduction to multiple object tracking
 
Real Time Object Dectection using machine learning
Real Time Object Dectection using machine learningReal Time Object Dectection using machine learning
Real Time Object Dectection using machine learning
 
Presentation of Visual Tracking
Presentation of Visual TrackingPresentation of Visual Tracking
Presentation of Visual Tracking
 
Deep sort and sort paper introduce presentation
Deep sort and sort paper introduce presentationDeep sort and sort paper introduce presentation
Deep sort and sort paper introduce presentation
 
Deep learning for object detection
Deep learning for object detectionDeep learning for object detection
Deep learning for object detection
 
Moving Object Detection And Tracking Using CNN
Moving Object Detection And Tracking Using CNNMoving Object Detection And Tracking Using CNN
Moving Object Detection And Tracking Using CNN
 

Plus de Vanya Valindria

Heart attack diagnosis from DE-MRI images
Heart attack diagnosis from DE-MRI imagesHeart attack diagnosis from DE-MRI images
Heart attack diagnosis from DE-MRI imagesVanya Valindria
 
Myocardial Infraction diagnosis using CT-Scan & MRI
Myocardial Infraction diagnosis using CT-Scan & MRIMyocardial Infraction diagnosis using CT-Scan & MRI
Myocardial Infraction diagnosis using CT-Scan & MRIVanya Valindria
 
Post-Processing of Prostate Perfusion MRI
Post-Processing of Prostate Perfusion MRIPost-Processing of Prostate Perfusion MRI
Post-Processing of Prostate Perfusion MRIVanya Valindria
 
Radon Transform - image analysis
Radon Transform - image analysisRadon Transform - image analysis
Radon Transform - image analysisVanya Valindria
 
Feature Hierarchies for Object Classification
Feature Hierarchies for Object Classification Feature Hierarchies for Object Classification
Feature Hierarchies for Object Classification Vanya Valindria
 
System-on-Chip Programmable Retina
System-on-Chip Programmable RetinaSystem-on-Chip Programmable Retina
System-on-Chip Programmable RetinaVanya Valindria
 
Shape from Distortion - 3D Digitization
Shape from Distortion - 3D DigitizationShape from Distortion - 3D Digitization
Shape from Distortion - 3D DigitizationVanya Valindria
 
Interlaced and progressive Camera
Interlaced and progressive CameraInterlaced and progressive Camera
Interlaced and progressive CameraVanya Valindria
 
Bagpipe as the Scotland National Instrument
Bagpipe as the Scotland National InstrumentBagpipe as the Scotland National Instrument
Bagpipe as the Scotland National InstrumentVanya Valindria
 
Indonesia - A Culture Overview
Indonesia - A Culture OverviewIndonesia - A Culture Overview
Indonesia - A Culture OverviewVanya Valindria
 

Plus de Vanya Valindria (13)

Bottle-in-Hangers
Bottle-in-HangersBottle-in-Hangers
Bottle-in-Hangers
 
Supermarket obesrvation
Supermarket obesrvationSupermarket obesrvation
Supermarket obesrvation
 
Heart attack diagnosis from DE-MRI images
Heart attack diagnosis from DE-MRI imagesHeart attack diagnosis from DE-MRI images
Heart attack diagnosis from DE-MRI images
 
Myocardial Infraction diagnosis using CT-Scan & MRI
Myocardial Infraction diagnosis using CT-Scan & MRIMyocardial Infraction diagnosis using CT-Scan & MRI
Myocardial Infraction diagnosis using CT-Scan & MRI
 
Post-Processing of Prostate Perfusion MRI
Post-Processing of Prostate Perfusion MRIPost-Processing of Prostate Perfusion MRI
Post-Processing of Prostate Perfusion MRI
 
Radon Transform - image analysis
Radon Transform - image analysisRadon Transform - image analysis
Radon Transform - image analysis
 
Feature Hierarchies for Object Classification
Feature Hierarchies for Object Classification Feature Hierarchies for Object Classification
Feature Hierarchies for Object Classification
 
System-on-Chip Programmable Retina
System-on-Chip Programmable RetinaSystem-on-Chip Programmable Retina
System-on-Chip Programmable Retina
 
Shape from Distortion - 3D Digitization
Shape from Distortion - 3D DigitizationShape from Distortion - 3D Digitization
Shape from Distortion - 3D Digitization
 
Shape context
Shape context Shape context
Shape context
 
Interlaced and progressive Camera
Interlaced and progressive CameraInterlaced and progressive Camera
Interlaced and progressive Camera
 
Bagpipe as the Scotland National Instrument
Bagpipe as the Scotland National InstrumentBagpipe as the Scotland National Instrument
Bagpipe as the Scotland National Instrument
 
Indonesia - A Culture Overview
Indonesia - A Culture OverviewIndonesia - A Culture Overview
Indonesia - A Culture Overview
 

Dernier

Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 

Dernier (20)

Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 

Real Time Object Tracking

  • 1. REAL-TIME OBJECT DETECTION AND TRACKING By: Vanya V. Valindria Hammad Naeem Rui Hua
  • 2. Outline • Introduction • Hardware in RT Object Detetion & Tracking • Methods Traditional Methods: Modern Methods: Absolute Differences Census Method KLT Feature Based Method Meanshift • Result and Conclusion
  • 3. Introduction Definition: Object detection  detect a particular object in an image Object tracking  to track an object (or multiple objects) over a sequence of images
  • 7. Problems?? • Temporal variation/dynamic environment • Abrupt object or camera motion • Multi-camera? Multi-objects? • Computational expensive
  • 8. Hardware in Real-time Tracking • MEMORY Important  Tracking system encountering limited memory problems. • FRAME RATE ~30 FPS • PROCESSORS - DSP • Allow saturated arithmetic operation • Powerful operation ability • Can do several memory accesses in a single instruction
  • 10. Object Detection and Tracking • In a video sequence an object is said to be in motion, if it is changing its location with respect to its background • The motion tracking is actually the process of keeping tracks of that moving object in video sequence i.e. position of moving object at certain time etc.
  • 11. Flow Chart Idle Image acquisition Object Detection Image acquisition Object tracking Object No Lost? Y es
  • 12. Method 1: Absolute Differences = Image subtraction D(t)=I(ti) – I(tj) Gives an image frame with changed and unchanged regions Ideal Case for no motion: I(ti) = I(tj), D(t)=0
  • 14. Results: Frame1 Frame10 Difference of Two Frames
  • 15. Absolute Difference Methods for Motion Detection  Frame Differencing  Background Subtraction Draw Backs: involves a lot of computations  Not feasible for DSP implementation
  • 16. Method 2: Census Transforms 124 74 32 1 1 0 124 64 18 If (Center pixel < Neighbor 1 x 0 pixel) Neighbor pixel = 1 157 116 84 1 1 1 Signature Vector Generation 1 1 0 1 x 0 Signature Vector 11011101 1 1 1
  • 17. Image 128 26 125 243 87 Signature Vectors 96 76 43 236 125 10110101 00101011 128 129 235 229 209 Signatur vector generation for . all pixels 228 251 229 221 234 . . 227 221 35 58 98 10111010 List 10110101 Generation List 00101011 population . . . Generated List 10111010 Signature vector matching
  • 18. Census Transform: Advantages:  Compare only two values 0 or 1.  Similar Illumination Variation for pixel and neighbouring pixels Draw Backs:  As we only deal with only 0`s and 1`s, this method is sensitive to noise.  Calculate, store and match process  computationally Expensive
  • 19. Method 3: Morphology Based Object Tracking Background Frame Object estimation differencing Registration
  • 20. Morphology Based Object Tracking • Image Differencing Background • Thresholding Estimation • Contours are registered Object • Width, height and histogram are recorded for each contour Registration • Each object represented by a feature vector (the length, Feature width, area and histogram of the object) Vector
  • 21. Segmented object Tracked object
  • 22. Morphology Based Techniques Advantages: Can Track Multiple objects  Objects are registered based on their anatomy  Helpful for Object Merging Draw Backs: Object registration  complex and slow process  For multiple object registration per frame  more complex
  • 23. Method 4: Lucas-Kanade Technique • Visual motion pattern of objects and surface in a scene  by Optical Flow Frame 1 Frame 2
  • 24. Method 5: Mean shift • An algorithm that iteratively shifts a data point to the average of data points in its neighborhood Choose a search window Compute the MEAN size in the initial location location in the search window Repeat until Center the search convergence window at the mean
  • 25. Intuitive Description Region of interest Center of mass Mean Shift vector Objective : Find the densest region Distribution of identical balls
  • 26. Intuitive Description Region of interest Center of mass Mean Shift vector Objective : Find the densest region Distribution of identical balls
  • 27. Intuitive Description Region of interest Center of mass Mean Shift vector Objective : Find the densest region Distribution of identical balls
  • 28. Intuitive Description Region of interest Center of mass Mean Shift vector Objective : Find the densest region Distribution of identical balls
  • 29. Intuitive Description Region of interest Center of mass Mean Shift vector Objective : Find the densest region Distribution of identical balls
  • 30. Intuitive Description Region of interest Center of mass Mean Shift vector Objective : Find the densest region Distribution of identical balls
  • 31. Intuitive Description Region of interest Center of mass Objective : Find the densest region Distribution of identical balls
  • 33. CAMSHIFT --Continously Adaptive Meanshift Modified to adapt dynamically to the colour probability distributions More real time For each frame-> MEAN-SHIFT is applied with several iteration Store the location of the mean and calculate new window size for next frame
  • 34. New development • Combine with different features. SIFT features, colour feature & texture information • Camshift algorithm combined with the Kalman filter.
  • 35. Result Arithmetic and Time taken Algorithm Logic by operations Algorithm Absolute 4230100 16 Differencing Census 2416000 5. 4 Transform Morphological 352210 14.2 Tracking Kanade Lucas 500825 0.486
  • 36. Comparison Computationally Easy to implement expensive Absolute Differences Allows continuous Slow and low tracking accuracy Computationally expensive Census Transform Immune to noise and Illumination changes Complex if  Multiple objects per frame Can track multiple Slow Feature Based objects well Large Memory consumption
  • 37. Comparison High accuracy KLT Less execution time Large memory Robust to noise and dynamic scene Ineffective if Computationally less there is heavy MeanShift & CAMShift expensive occlusion
  • 38. Conclusion • KLT algorithm has the best performance with higher accuracy and less computation time • It requires combination of methods to achieve the appropriate object detection and tracking according to the proposed scenario
  • 39. References • S. Shah, T. Khattak, M. Farooq, Y. Khawaja, A. Bais, A. Anees, and M. Khan, “Real Time Object Tracking in a Video Sequence Using a Fixed Point DSP,” Advances in Visual Computing, pp. 879– 888. • K. Huang, L. Wang, T. Tan, and S. Maybank, “A real-time object detecting and tracking system for outdoor night surveillance,” Pattern Recognition, vol. 41, no. 1, pp. 432–444, 2008. • J. Li, F. Li, and M. Zhang, “A Real-time Detecting and Tracking Method for Moving Objects Based on Color Video,” in 2009 Sixth International Conference on Computer Graphics, Imaging and Visualization. IEEE, 2009, pp. 317–322. • W. Junqiu and Y. Yagi, “Integrating color and shapetexture features for adaptive real-time object tracking,” IEEE Trans on Image Processing, vol. 17, no. 2, pp. 235–240, 2008. • Q. Wang and Z. Gao, “Study on a Real-Time Image Object Tracking System,” in Computer Science and Computational Technology, 2008. ISCSCT’08. International Symposium on, vol. 2, 2008. • Y. Meng, “Agent-based reconfigurable architecture for real-time object tracking,” Journal of Real-Time Image Processing, vol. 4, no. 4, pp. 339–351, 2009. • [Y. Yao, C. Chen, A. Koschan, and M. Abidi, “Adaptive online camera coordination for multi- camera multi-target surveillance,” Computer Vision and Image Understanding, 2010.

Notes de l'éditeur

  1. http://plaza.ufl.edu/lvtaoran/object%20tracking.htm
  2. Image/appearance based trackingThere are wide application in real time object detection and tracking.
  3. Multiobject (people) tracking within a video of a pedestrian passageway. The dynamic motion vectors attached to each individual represents direction of movement and speed.
  4. Here, we can see how a mobile robot can detect and track this red ball. It moves accordingly to the red ball movement.
  5. As the objects move over time, ther are different illumination and motions of small objects; due to perspective, occlusion, interaction between objects and appearance or disappearance of objects.2. to track targets and keep on detectingnew ones on a moving camera platform at the same time,the traditional motion detector based on the backgroundsubtraction can not be applied here.3. How to...In the same time..4. Because of comprehensive search ,it is hard to meet strict time constraint in real-time
  6. To solve these problems...The choice of hardware can increase the performance of object detection and traking in real time which has hard time constraintsMemory is....There are several processor, such as FPGA depends on the choice of soft processorASICulfill the speed criteria of real-time, but it is complicatedGPU  increase the speed up the computation at the bottom level method (optical flow)
  7. http://www.princeton.edu/~xm/vision05/Assignment3/Tracking/tracking.htm
  8. The first term is proportional to the density estimate at x computed with the kernel G. the second term is the mean shift. This part is the mean of the window. We calculate it by using a kernel function, which gives different weights to all points inside the window. the mean shift vector thus always points toward the direction of maximum increase in the density.
  9. http://www.codeproject.com/KB/GDI-plus/MeanshiftTracking.aspxChoose a search windowsize.2. Choose the initial locationof the search window.3. Compute the mean location(centroid of the data) in thesearch window.4. Center the search windowat the mean locationcomputed in Step 3.5. Repeat Steps 3 and 4 untilconvergence
  10. Camshift is based on mean shift, but the window size is changed in in video sequence, when the object moves, the size and locations of color distributions will change over time. Mean shift. mean shift algorithm uses fixed window size. So it might fail. However, camshift can deal with this problem by adjusting the window size according to the distribution. Real time
  11. 3. Firstly, initialize the Gaussian mixture model to get the background image, and then using the background differential with the current frame to detect the moving objects. Kalman filter is used to predict the centre of the searching window in the next frame. Then, camshift will find the optimum position of the target in order to modify the prediction. This can improve the speed of camshift algorithm and solve the occlusion problem.
  12. Theses results are from the paper by Shah, in which he has demonstrated the results of the all the above mentioned methods for the same task of tracking a moving ball with DSP hardware.As we can see that the Lukas kanade is the fastest compared with the traditional methods.
  13. The absolute difference and census transform are easy to implement but computationally expensive and slow. Feature based method can track multiple objects, but it is also slow.
  14. KLT algorithm is can detect objects fast and accurately and it is robust to noise and dynamic scene. but it requires large memory, when the search window size is large.Mean shift has low computation cost. But it might fail in case of heavy occlusion and it can only detect single object. This can be solved by combining different algorithm, for example, SIFT feature descriptor and Kalman filter.