This document discusses techniques for segmenting independently moving image regions using motion detection. It covers the following approaches:
1. Motion-based segmentation using optical flow to detect pixel-level motion between frames. This approach has limitations due to the aperture and occlusion problems.
2. Color-based and texture-based segmentation which learn background models (e.g. histograms or Gaussian distributions) for each pixel and detect foreground objects that differ significantly from the background models.
3. Dominant motion segmentation fits a single motion model to partition a frame into regions of global and local motion. Multiple motion segmentation estimates multiple motion models competing at each pixel.
2. Motion Segmentation
Foreground (object) and Background (noise)
Result could be a
Binary image, containing foreground only
Probability image, containing the likelihood of each pixel being
foreground
Approaches
Motion-based (optical flow)
Color-based
Texture-based
2
3. Motion (Change) Detection
Static camera : changed and unchanged regions.
Moving camera : global and local motion regions.
Limitations associated with motion estimation
Aperature problem : pixels in a flat image region may
appear stationary even if they are moving as a result of
an aperture problem (hence the need for hierarchical
methods)
Occlusion Problem : erroneous labels may be
assigned to pixels in covered or uncovered image
regions as a result of an occlusion problem.
7. 7
Threshold
Decide, when a pixel is supposed to be considered
as a background pixel, or when it is to be
considered as a foreground pixel:
Pixel is foreground pixel, if
Pixel is background pixel, if
Problem: What TH?!?
),( yx
THyxF ),(
),( yx
THyxF ),(
1. Save image in last frame
2. Capture camera image
3. Subtract image
4. Threshold
5. Delete noise
9. 125 4 3678
Frame
no. 1 2 3 4 5 6 7 8 9
x 154 160 160 160 157 130 112 79 0
y 13 19 19 19 16 19 19 19 0
w s s w w w w w
10. Dominant Motion : no optical flow
Spatio temporal intensity gradient.
Dominant motion segmentation
fitting a single parametric motion model
partition the frame in two pixel groups
Repeat step 1 only to well represented pixels group.
12. Multiple Motion Segmentation
Multiple motion models compete against each other at
each decision site. They consist estimating:
Motion within each region (motion model)
spatial support of each region
number of regions.
The problem : associate each pixel to the right motion
model, while simultaneously estimating motions and
supports.
Clustering (K-means, hough transform),
Maximum Likelihood (ML) (pixel based) and
Maximum A Posteriori probability (MAP).
Region based label assignment
15. 15
Motion Model
Predicted position at time t:
Brownian Motion: According to a Gaussian model
0’th order:
1’th order:
Similar for y
2’th order
Similar for y
),( tt yx
),(),( 11 tttt yxyx
1at timeinvelocity:1
11
t-xx
xtxx
t
ttt
1at timeinonaccelerati:
2
1
1
11
2
1
t-xx
xtxtxx
t
tttt
16. 16
Color Based segmentation : Background
Subtraction
Use Neighborhood relation!!
Compare pixel with its neighbors!!
Weight them!!
Learn the background and its variations!!
E.g. Gaussian models (mean,var) for each pixel!!!
E.g. a Histogram for each Pixel
The more images you train on the better!!
Algorithm:
Consider each pixel (x,y) in the input image and check, how much it
varies with respect to the mean and variance of the learned Gaussian
models?
1. Calculate mean and variance for each pixel
2. Capture camera image
3. Subtract image (= motion)
4. Weight the distances (new)
5. Threshold according to variance
6. Delete noise