Image segmentation techniques
More information on this research can be found in:
Hussein, Rania, Frederic D. McKenzie. “Identifying Ambiguous Prostate Gland Contours from Histology Using Capsule Shape Information and Least Squares Curve Fitting.” The International Journal of Computer Assisted Radiology and Surgery ( IJCARS), Volume 2 Numbers 3-4, pp. 143-150, December 2007.
2. 2
Outline
Introduction
Edge-based segmentation
Region-based segmentation
– Region Growing
– Split-and-merge
Active contour models (snakes)
Application in medical imaging
Conclusion
Assignment 2
3. 3
Introduction
What is Image segmentation ?
– The different partitioning of an image into non-
overlapping, constituent regions which are
homogeneous with respect to some characteristic
such as intensity or texture.
– Each of such homogeneous regions may
represent an object.
4. 4
Introduction
The shape of an object can be described in terms of:
Its boundary – requires image edge detection
The region it occupies – requires image
segmentation in homogeneous regions, Image
regions generally have homogeneous
characteristics (e.g. intensity, texture)
Segmentation methods are then classified into
– Edge-based
– Region-based
5. 5
Edge-based Segmentation
Emphasis:
– Determine the boundaries that separate regions
Common approaches
– Find edge points in the image
Gradient based methods
Second order methods
– Linking these points in some way to produce
description of edges in terms of lines, curves etc
6. 6
Detecting Edge points-Gradient based
methods
An edge point is a point
where a discontinuity in
gradient occurs across
some line
Different types of
discontinuity are shown
in the figure
7. 7
Gradient based methods (cont.)
The gradient is a vector whose components
measure how rapidly pixel values are changing
with distance, in the x and y directions
8. 8
Gradient based methods (cont.)
Considering dx=dy=1 (pixel spacing) and considering
a point (i,j)
Δx = f(i+1,j) – f(i,j)
Δy = f(i,j+1) – f(i,j)
Δx and Δy can be calculated by convolving the image
with convolution masks
9. 9
Gradient based methods (cont.)
To approximate the gradient along directions at 45o and 135o
to the axes respectively,
known as Roberts edge operator. The corresponding
convolution masks are
Other 3x3 edge operators can be used such as Sobel and
Canny
10. 10
Example
Original image Image produced by
the horizontal
gradient calculation
Image produced by
the vertical gradient
calculation
11. 11
Example
Gradient image formed by combining horizontal
and vertical gradient detection
Gradient image is produced using
the magnitude M form
M = | Δx | + | Δy |
The gradient direction θ is equal
to
Θ = tan-1 (Δy / Δx )
12. 12
Implementation using Matlab
Read image and display it.
I = imread('coins.jpg'); imshow(I)
Apply the Sobel and Canny
edge detectors to the image
and display them
BW1 = edge(I,'sobel');
BW2 = edge(I,'canny');
imshow(BW1) figure,
imshow(BW2)
Original image “coins.jpg”
14. 14
Edge Linking
Edge detectors yield pixels that lie on edges
The objective is to replace many points on
edges with real edges.
Edge linking can be performed by:
– Local edge linkers – where edge points are
grouped according to their relationships with the
neighboring edge points.
– Global Edge Linkers – Hough transform
15. 15
Hough Transform
Allows recognition of global patterns in an
image
Finds curves like straight lines, circles, etc
Suppose that we are looking for straight lines
in an image
– If we take a point (x',y') in the image, all lines
which pass through that pixel have the form
y’ = mx’ +c
16. 16
Hough Transform
This equation can be
written as
c = -x’m + y’
where x’,y’ are constants
and m,c varies
Each different line
through the point (x',y')
corresponds to one of
the points on the line in
(m,c) space
Lines through a
point
17. 17
Hough Transform
All pixels which lie on the
same line in (x,y) space are
represented by lines which
all pass through a single
point in (m,c) space.
The single point through
which they all pass gives
the values of m and c in the
equation of the line y=mx+c.
18. 18
Hough Transform
The y=mx+c form for representing a straight line
breaks down for vertical lines, when m becomes
infinite.
To avoid this problem, it is better to describe straight
lines in the form of
x cos θ + y sin θ = r
i.e. a point in (x,y) space is now represented by a curve
in (r,θ) space rather than a straight line
19. 19
Hough Transform
To detect straight lines in an image
1. Quantize (m,c) space into a two-dimensional array A for
appropriate steps of m and c.
2. Initialize all elements of A(m,c) to zero.
3. For each pixel (x',y') which lies on some edge in the image,
add 1 to all elements of A(m,c) whose indices m and c
satisfy y'=mx'+c.
4. Search for elements of A(m,c) which have large values --
Each one found corresponds to a line in the original image.
20. 20
Hough Transform
To find circles, with equation
(x – a)2 + (y – b)2 = r2
– Every point in (x,y) space corresponds to a surface in (a,b,r) space
(as we can vary any two of a, b and r, but the third is determined
by the equation of the circle).
– The basic method is, thus, modified to use a three-dimensional
array A(a,b,r),
– All points in it which satisfy the equation for a circle are
incremented.
The technique takes rapidly increasing amounts of time for
more complicated curves as the number of variables (and
hence the number of dimensions of A) increases
22. 22
Region Growing
A simple approach to image segmentation is
to start from some pixels (seeds)
representing distinct image regions and to
grow them, until they cover the entire image
Before assigning a pixel x to a region Ri(k),
check if the region is homogeneous: i.e.
H(Ri(k) U {x}) = TRUE
23. 23
Region Growing
The arithmetic mean M and standard
deviation sd can be used to decide if merging
two regions R1,R2 is allowed
if |M1 – M2| < (k)*sd(i) , i = 1, 2 , merge the
two regions
where k is a certain threshold
24. 24
Split-and-Merge
The opposite approach to region growing is region
splitting.
The approach starts with the assumption that the
entire image is homogeneous
If the entire image is not homogeneous, the image is
split into four sub images
This splitting procedure is repeated recursively until
the image is split into homogeneous regions
25. 25
Split-and-Merge
Since the procedure is recursive, it produces
an image representation that can be
described by a tree whose nodes have four
children each
Such a tree is called a Quadtree.
28. 28
Split-and-Merge
Splitting techniques create regions that may
be adjacent and homogeneous, but not
merged.
Split and Merge method is an iterative
algorithm that includes both splitting and
merging at each iteration. It produces more
compact regions than the splitting algorithms
29. 29
Split-and-Merge Algorithm
If a region R is inhomogeneous
(H(R)= False) then split into four sub regions
If two adjacent regions Ri,Rj are
homogeneous (H(Ri U Rj) = TRUE), merge
them
Stop when no further splitting or merging is
possible
30. 30
Outline
Introduction
Edge-based segmentation
Region-based segmentation
– Region Growing
– Split-and-merge
Active contour models (snakes)
Application in medical imaging
Conclusion
31. 31
Active Contour Models (Snakes)
First introduced in 1987 by Kass et al, and gained
popularity since then.
Represents an object boundary as a parametric
curve.
An energy function E is associated with the curve.
The problem of finding object boundary is an energy
minimization problem.
32. 32
Framework for snakes
A higher level process or a user
initializes any curve close to the
object boundary.
The snake then starts
deforming and moving towards
the desired object boundary.
In the end it completely “wraps”
around the object.
(Digram courtesy “Snakes, shapes, gradient vector flow”, Xu, Prince)
33. 33
Snakes
Contour possesses an energy (Esnake) which is defined as the
sum of the three energy terms.
where Einternal represents the internal energy of the spline due to
bending, Eexternal denotes image forces, and Econstraint denotes
external constraint forces.
The energy terms are defined such that the final position of
the contour will have a minimum energy (Emin)
Therefore the problem of detecting objects reduces to an
energy minimization problem.
int intsnake ernal external constraE E E E
34. 34
Outline
Introduction
Edge-based segmentation
Region-based segmentation
– Region Growing
– Split-and-merge
Active contour models (snakes)
Application in medical imaging
Conclusion
35. 35
Application in medical imaging
In-vivo segmentation: automating or facilitating the
delineation of anatomical structures and other
regions of interest
Segmentation methods vary widely depending on
the specific application and imaging modality.
There is currently no single segmentation method
that yields acceptable results for every medical
image.
Selecting an appropriate approach to a
segmentation problem can be a difficult dilemma.
36. 36
Example
In reconstructing a 3D
model of a prostate, the
capsule contour needs to
be extracted from the
slices’ images
Capsule contour
37. 37
Example
The capsule consists of
collagen fibers tissues that
appear under the
microscope as wavy lines
(see figure)
However, the capsule line is sometimes unrecognizable because of the
naturally occurring intrusion of muscle into the prostate gland which
makes the segmentation problem more challenging.
38. 38
Summary
Introduction
Edge-based segmentation
Region-based segmentation
– Region Growing
– Split-and-merge
Active contour models (snakes)
Application in medical imaging
Conclusion
39. 39
Conclusion
Edge detection and region growing algorithms are
very popular in most commercial image analysis
tools
The reason is that they are simple to understand
and to implement, and they are very generic, as
they do not assume specific knowledge about the
objects to be analyzed.
These methods are often the starting point for more
sophisticated model-based methods
40. 40
Conclusion
For complex image data, such as medical images,
their usefulness is quite limited.
Effective image analysis methods must incorporate
a priori knowledge of the considered structures
such as photometric properties (object intensity,
contrast, texture); geometric properties (position,
shape, motion, deformation); and the context, such
as the relative position with respect to the other
objects in the neighborhood