At the end of this lesson, you should be able to;
define segmentation.
Describe edge based in segmentation.
describe thresholding and its properties.
apply edge detection and thresholding as segmentation techniques.
1. COMPUTER GRAPHICS & IMAGE PROCESSING
COM2304
Segmentation Techniques -I
K.A.S.H.Kulathilake
B.Sc. (Hons) IT (SLIIT), MCS (UCSC)
Rajarata University of Sri Lanka
Faculty of Applied Sciences
Department of Physical Sciences
2. Learning Outcomes
COM 2304 - Computer Graphics & Image
Processing
• At the end of this lecture, you should be
able to;
– define segmentation.
– Describe edge based in segmentation.
– describe thresholding and its properties.
– apply edge detection and thresholding as
segmentation techniques.
2
3. What is Segmentation?
• Segmentation subdivides an image into its
constituent regions or objects.
• The level of detail to which the subdivision is
carried depends on the problem being solved.
• In other terms, segmentation should stop
when the objects or region of interest (ROI) in
an application have been detected.
COM 2304 - Computer Graphics & Image
Processing
3
5. What is Segmentation? (Cont…)
• Segmentation accuracy determines the
eventual success or failure of computerized
analysis procedures.
• For this reason, considerable care should be
taken to improve the probability of accurate
segmentation.
COM 2304 - Computer Graphics & Image
Processing
5
6. What is Segmentation? (Cont…)
• Let R represent the entire spatial region
occupied by an image.
• we may view image segmentation as a process
that partitions R into n sub regions,
R1,R2,R3,……………..Rn, such that;
– Segmentation must be complete; i.e. every pixel
must be in a region.
COM 2304 - Computer Graphics & Image
Processing
6
7. What is Segmentation? (Cont…)
– Ri is a connected set, i=1,2,…………….,n.
– Ri∩Rj = φ for all i and j, i≠j.
• Regions must be disjoint
– Q(Ri) = TRUE for i=1,2,…….,n.
• Properties that must be satisfied by the pixels in a
segmented regions – ex. Q(Ri) = TRUE if all pixels in Ri
have the same intensity level.
– Q(RiURj) = FALSE for any adjacent regions Ri and
Rj.
• Indicates that two adjacent regions Ri and Rj must be
different in the sense of predicate Q.
COM 2304 - Computer Graphics & Image
Processing
7
8. Points, Lines and Edge Detection
• The focus of this topic is on segmentation methods that are
based on detecting sharp, local changes in intensity.
• Three types of images features in which we are interested
are;
– Edges: edge pixels are pixels at which the intensity of an image
function changes abruptly, and edges are sets of connected
edge pixels.
– Lines: line may be viewed as a edge segment in which the
intensity of the background on either side of the line is either
much higher or much lower than the intensity of the line pixels.
– Isolated point: It can be viewed as a line whose length and
width are equal to one pixel.
COM 2304 - Computer Graphics & Image
Processing
8
9. Background of Points, Lines and Edge
Detection
• Local changes in intensity can be detected
using derivatives.
• Derivatives of a digital function are defined in
terms of differences.
• First order derivatives:
– Must be zero in the areas of constant intensity.
– Must be nonzero at the onset of an intensity step or ramp.
– Must be nonzero along the ramp.
COM 2304 - Computer Graphics & Image
Processing
9
11. Background of Points, Lines and Edge
Detection (Cont…)
• Second order derivatives:
– Must be zero in the areas of constant intensity.
– Must be nonzero at the onset and end of an intensity
step or ramp.
– Must be zero along the ramp of constant slope.
• First order derivatives provides thick edges and second
order derivatives provides much finer ones (including
noise).
COM 2304 - Computer Graphics & Image
Processing
11
13. Background of Points, Lines and Edge
Detection (Cont…)
• Double Edge Effect in Second Order Derivatives:
– In both the ramp and step edges that the second
derivatives has opposite signs ( -ve to +ve or vise-
versa) as it transitions into and out of an edge.
– This is known as double edge effect and it is important
to locate edges.
• The sign of the second derivative is used also to
determine whether an edge is a transition from
light to dark (-ve +ve second derivative) or
from dark to light (+ve -ve second derivative),
where the sign is observed as we move into the
edge.
COM 2304 - Computer Graphics & Image
Processing
13
14. Edge Models
• Edge detection is the approach used most
frequently for segmenting images based on
abrupt(local) changes in intensity.
• Edge models are classified according to their
intensity profiles as follows:
– Step edges
– Ramp edges
– Roof edges
COM 2304 - Computer Graphics & Image
Processing
14
15. Edge Models (Cont…)
• Step Edges:
– A step edge involves a transition between two
intensity levels occurring ideally over the distance
of 1 pixel.
– Canny edge detection algorithm was derived using
a step edge model.
COM 2304 - Computer Graphics & Image
Processing
15
16. Edge Models (Cont…)
• Ramp Edges:
– In practice, digital images have edges that are blurred and
noisy.
– Blurring determined by limitations in the focusing
mechanism and the noise level determined principally by
the electronic components of the imaging system.
– In such situations, edges are more closely modeled as
having an intensity ramp profile.
– The slope of the ramp is inversely proportional to the
degree of blurring in the edge.
– It does not contains a thin path.
COM 2304 - Computer Graphics & Image
Processing
16
17. Edge Models (Cont…)
• Roof Edges:
– Roof edge being
determined by the
thickness and
sharpness of the line.
COM 2304 - Computer Graphics & Image
Processing
17
18. Edge Models (Cont…)
• 1st derivative tells us
where an edge is
• 2nd derivative can
be used to show
edge direction
• Zero crossings can be
used for locating the
canters of thick
edges.
COM 2304 - Computer Graphics & Image
Processing
18
Zero Crossing
19. Edge Models (Cont…)
• Derivative based edge detectors are extremely sensitive to noise ( 2nd
derivatives are more sensitive).
COM 2304 - Computer Graphics & Image
Processing
19
Images are with Gaussian
Noise
20. Detection of Isolated Points
• Point detection is based on second
derivatives.
• Point detection can be achieved simply using
the Laplacian mask given below:
• Points are detected at those pixels in the
subsequent filtered image that are above a set
threshold
COM 2304 - Computer Graphics & Image
Processing
20
21. Detection of Isolated Points (Cont…)
COM 2304 - Computer Graphics & Image
Processing
21
X-ray image of a
turbine blade
Result of point
detection
Result of
thresholding
22. Line Detection
• We can use Laplacian mask for line detection.
• It is assumed that the lines are thin with respect to
the size of the detector.
• The masks below will extract lines that are one pixel
thick and running in a particular direction.
COM 2304 - Computer Graphics & Image
Processing
22
23. Line Detection (Cont…)
COM 2304 - Computer Graphics & Image
Processing
23
Binary image of a wire bond mask
Horizontal +45° Verticle -45°
24. Edge Detection with Image Gradient
• Given a 3*3 region of an image the following edge detection
filters can be used under the 1st order derivatives.
COM 2304 - Computer Graphics & Image
Processing
24
25. Thresholding
• Objective is to partitioning images directly into regions based
on intensity values and/or properties of these values.
• Single value thresholding can be given mathematically as
follows:
COM 2304 - Computer Graphics & Image
Processing
25
Tyxfif
Tyxfif
yxg
),(0
),(1
),(
T
Histogram
Modes
27. Thresholding (Cont…)
• If you get the threshold wrong the results can
be disastrous
Threshold Too Low Threshold Too High
28. The Role of Noise in Image
Thresholding
COM 2304 - Computer Graphics & Image
Processing
28
Original image without
noise
Gaussian noise of 0 mean
and standard deviation of
10 intensity levels
Gaussian noise of 0 mean
and standard deviation of
50 intensity levels
29. The Role of Illumination &
Reflectance in Image Thresholding
COM 2304 - Computer Graphics & Image
Processing
29
Gaussian noise of 0 mean
and standard deviation of
10 intensity levels
Multiply the noise image with this variable intensity
function to obtain the non-uniformly illuminated
image.
30. The Role of Illumination &
Reflectance in Image Thresholding
(Cont…)
• Similar results would be obtained when
illumination was uniform and reflectance was not
in situations like natural reflectivity variations in
the surface of object and/ or background.
• The key idea of these experiments is that the
reflectance and the illumination play central role
in the success of image segmentation using
thresholding or other segmentation technique.
• Therefore, controlling these factors should be the
first step considered in the solution of
segmentation problem.
COM 2304 - Computer Graphics & Image
Processing
30
31. Basic Global Thresholding
• It is possible when the intensity distribution of
objects and background pixels are sufficiently
distinct.
• The basic global threshold, T, is calculated as
follows:
1. Select an initial estimate for T (typically the
average grey level in the image)
2. Segment the image using T to produce two groups
of pixels: G1 consisting of pixels with grey levels >T
and G2 consisting pixels with grey levels ≤ T
3. Compute the average grey levels of pixels in G1 to
give μ1 and G2 to give μ2
COM 2304 - Computer Graphics & Image
Processing
31
32. Basic Global Thresholding (Cont…)
4. Compute a new threshold value:
5. Repeat steps 2 – 4 until the difference in T in
successive iterations is less than a predefined
limit T∞
COM 2304 - Computer Graphics & Image
Processing
32
2
21
T
T∞ is used to control the number of iterations in
situations where speed is an important issue. In
general, the larger T∞ is, the fewer iterations the
algorithm will perform.
33. Basic Global Thresholding (Cont…)
COM 2304 - Computer Graphics & Image
Processing
33
Single value
thresholding only
works for bimodal
histograms
Images with other
kinds of histograms
need more than a
single threshold
This algorithm works very
well for finding thresholds
when the histogram has a
clear valley between the
two modes
34. Otsu Thresholding
• Otsu's thresholding method involves iterating
through all the possible threshold values and
calculating a measure of spread for the pixel
levels each side of the threshold, i.e. the pixels
that either fall in foreground or background.
• The aim is to find the threshold value where
the sum of foreground and background
spreads is at its minimum.
COM 2304 - Computer Graphics & Image
Processing
34
35. Otsu Thresholding (Cont…)
• The algorithm will be demonstrated using the simple
6x6 image shown above.
• The calculations for finding the foreground and
background variances (the measure of spread) for a
single threshold are explained next.
• In this case the threshold value is 3.
COM 2304 - Computer Graphics & Image
Processing
35
37. Otsu Thresholding (Cont…)
• The next step is to calculate the 'Within-Class
Variance'.
• This is simply the sum of the two variances
multiplied by their associated weights.
• This final value is the 'sum of weighted variances'
for the threshold value 3.
• This same calculation needs to be performed for
all the possible threshold values 0 to 5.
COM 2304 - Computer Graphics & Image
Processing
37
39. Otsu Thresholding (Cont…)
• It can be seen that for the threshold equal to 3, as well as being used for
the example, also has the lowest sum of weighted variances.
• Therefore, this is the final selected threshold.
• All pixels with a level less than 3 are background, all those with a level
equal to or greater than 3 are foreground.
• This approach for calculating Otsu's threshold is useful for explaining the
theory, but it is computationally intensive, especially if you have a full 8-bit
greyscale.
COM 2304 - Computer Graphics & Image
Processing
39
40. Otsu Thresholding (Cont…)
• Making Otsu faster using Between
class variance:
– A faster method of performing the calculations
which is much more appropriate for
implementations.
– So it can also be used for finding the best
threshold and therefore due to being simpler is a
much better approach to use.
COM 2304 - Computer Graphics & Image
Processing
40
41. Otsu Thresholding (Cont…)
– The table below shows the different variances for
each threshold value.
COM 2304 - Computer Graphics & Image
Processing
41
42. Otsu Thresholding (Cont…)
COM 2304 - Computer Graphics & Image
Processing
42
Global
Thresholding
T=169
Otsu
Thresholding
T= 181
43. Effect of Image Smoothing for
Thresholding
COM 2304 - Computer Graphics & Image
Processing
43
Noise image Histogram Otsu Thresholding
Smoothed Image
(5 by 5, Mean)
Histogram Otsu Thresholding
Boundary Distortion occurred
44. Effect of Image Smoothing for
Thresholding (Cont…)
COM 2304 - Computer Graphics & Image
Processing
44
Noise image Histogram Otsu Thresholding
Smoothed Image
(5 by 5, Mean)
Histogram Otsu Thresholding
Segmentation failed
45. Effect of Image Smoothing for
Thresholding (Cont…)
• Reason for above matter:
– Region is so small that its contribution to the
histogram is insignificant compared to the
intensity spread caused by noise.
– To overcome this problem from global
thresholding we can use edges.
COM 2304 - Computer Graphics & Image
Processing
45
46. Using Edges for Improving Global
Thresholding
• We can use pixels that lie on or near the edges
between objects and the background for
improving the shape of the histogram.
• Using pixels that satisfy some simple measures
based on Gradient and Laplacian operators has a
tendency to deepen the valley between
histogram peaks.
• In this approach, we assume that the edges
between the objects and background are known.
COM 2304 - Computer Graphics & Image
Processing
46
47. Using Edges for Improving Global
Thresholding (Cont…)
• Algorithm: f(x, y) is the input image
1. Compute the edge image ( use either gradient
magnitude or Laplacian method).
2. Specify a threshold value, T.
3. Threshold the image from step 1 using the threshold
from step 2 to produce a binary image, gT(x, y). Use this
image as a mask image to select pixels from f(x, y)
corresponding to “strong” edge pixels.
4. Compute a histogram using only the pixels in f(x, y) that
correspond to the locations of the 1-valued pixels in
gT(x,y).
5. Use the histogram from step 4 to segment f(x, y) globally
using, for example Otsu’s method.
COM 2304 - Computer Graphics & Image
Processing
47
48. Using Edges for Improving Global
Thresholding (Cont…)
COM 2304 - Computer Graphics & Image
Processing
48
Image of Yeast cells (only white patches ) Histogram Otsu Thresholding
Smoothed Image using Laplacian. T=115 Histogram Otsu Thresholding . Segmentation is
successful
49. Multiple Thresholds
• Objective of the multiple thresholding is to
separate the original image into multiple
classes.
COM 2304 - Computer Graphics & Image
Processing
49
Dark background
Illuminated area
Shadow
50. Variable Thresholding
• Image Partitioning
– One of the simplest approach to variable
thresholding is to subdivide an image into non-
overlapping rectangles.
– This approach is used to compensate for non-
uniformities in illumination and/or reflectance.
– The rectangles are chosen small enough to that
the illumination of each is approximately uniform.
COM 2304 - Computer Graphics & Image
Processing
50
51. Variable Thresholding (Cont…)
COM 2304 - Computer Graphics & Image
Processing
51
Noisy shaded image Histogram Otsu Thresholding
Image sub divide into six sub images Result of applying Otsu method to each sub
image individually
Errors in
segmentation
52. Variable Thresholding (Cont…)
– Six histograms of six sub images have produced
bimodel histograms as the result of this thresholding
approach.
– Hence, it leads to effective global thresholding.
– Image subdivision generally works well when the
object of interest and the background occupy regions
of reasonably comparable size.
– When this is not the case, the method typically fails
because of the likelihood of subdivisions containing
only object or background pixels.
•
COM 2304 - Computer Graphics & Image
Processing
52
53. Variable Thresholding (Cont…)
• Adaptive Thresholding:
– It is a modified threshold technique in which the
threshold level is itself variable.
– It is useful when there are strong illumination or
reflectance gradients that you need to threshold
relative to the general intensity gradients.
– Adaptive thresholding set on a pixel by pixel basis
by computing a weighted average of the b-by-b
region around each pixel location minus a
constant where b is given by block size.
COM 2304 - Computer Graphics & Image
Processing
53
56. Learning Outcomes Revisit
• Now, you should be able to;
– define segmentation.
– Describe edge based in segmentation.
– describe thresholding and its properties.
– apply edge detection and thresholding as
segmentation techniques.
COM 2304 - Computer Graphics & Image
Processing
56
57. QUESTIONS ?
Next Lecture – Segmentation Techniques - II
COM 2304 - Computer Graphics & Image
Processing
57