SEGMENTATION BASICS, Evaluations of RGB Color space ,HSV VS RGB ,minimum cut ,Unknown clusters and centers

1. SEGMENTATION OF
FOREGROUND – BACKGROUND
FROM NATURAL IMAGES
B Y
AJAL.A.J
ASSISTANT PROFESSOR
UNIVERSAL ENGINEERING COLLEGE

2. OUTLINE
 Introduction
 Types of segmentation algorithms
 Evaluations of RGB Color space
 SEGMENTATION
 EXPERIMENTAL RESULTS
 Summary
 Appendix

3. ABSTRACT
 This paper presents a part of a more challenging research
project aimed at developing a computer vision system for a
robot capable of identifying all objects from known natural
backgrounds such as forest, sky, ocean, under-water scenes
and etc.
 Segmentation is an import issue in the field of machine vision
for detection and recognition of objects.
 The success of segmentation is solely depends on the
separation of foreground objects from background objects.
 We present a simple framework to extract the foreground
objects from the known natural backgrounds in still and moving
images using pixel based color segmentation in RGB space.

4. What is an Image?
 2D array of pixels
 Binary image (bitmap)
 Pixels are bits
 Grayscale image
 Pixels are scalars
 Typically 8 bits (0..255)
 Color images
 Pixels are vectors
 Order can vary: RGB,
BGR
 Sometimes includes
Alpha

5. What is an Image?
 2D array of pixels
 Binary image (bitmap)
 Pixels are bits
 Grayscale image
 Pixels are scalars
 Typically 8 bits (0..255)
 Color images
 Pixels are vectors
 Order can vary: RGB,
BGR
 Sometimes includes
Alpha

6. What is an Image?
 2D array of pixels
 Binary image (bitmap)
 Pixels are bits
 Grayscale image
 Pixels are scalars
 Typically 8 bits (0..255)
 Color images
 Pixels are vectors
 Order can vary: RGB,
BGR
 Sometimes includes
Alpha

7. What is an Image?
 2D array of pixels
 Binary image (bitmap)
 Pixels are bits
 Grayscale image
 Pixels are scalars
 Typically 8 bits (0..255)
 Color images
 Pixels are vectors
 Order can vary: RGB,
BGR
 Sometimes includes
Alpha

8. What is an Image?
 2D array of pixels
 Binary image (bitmap)
 Pixels are bits
 Grayscale image
 Pixels are scalars
 Typically 8 bits (0..255)
 Color images
 Pixels are vectors
 Order can vary: RGB,
BGR
 Sometimes includes
Alpha

9. HSV VS RGB.
 In day to day practice, we'll most likely use
two models:
HSV and RGB.
HSV stands for
Hue,
Saturation, and
Value,
and it uses these three concepts to describe a color.
RGB the three colors that make up an image on a monitor.

10. RGB Color cube

11. Color segmentation
 In the problem of segmentation, the goal is to separate spatial regions of
an image on the basis of similarity within each region and distinction
between different regions.
 Approaches to color-based segmentation range from empirical evaluation
of various color spaces, to clustering in feature space , to physics-based
modeling
 The essential difference between color segmentation and color recognition
is that the former uses color to separate objects without a priori
knowledge about specific surfaces; the latter attempts to recognize colors
of known color characteristics

13. Segmentation: Elephant and
Blind Men Syndrome

14. SEGMENTATION
Segmented image – giving us the outline of
her face, hand etc
Colour Image having
a bimodal histogram

15. Results on color segmentation

17. SEGMENTATION
 Partitioning images into meaningful
pieces, e.g. delineating regions of
anatomical interest.
 Edge based – find boundaries between
regions
 Pixel Classification – metrics classify regions
 Region based – similarity of pixels within a
segment

18. minimum cut
“allegiance” = cost of assigning two nodes to different
layers (foreground versus background)
foreground
node
background
node
pixel nodes
allegiance to
foreground
allegiance to
background
pixel-to-pixel
allegiance

19. minimum cut
“allegiance” = cost of assigning two nodes to different
layers (foreground versus background)
foreground
node
background
node
pixel nodes
allegiance to
foreground
allegiance to
background
pixel-to-pixel
allegiance

20. Normalized Cuts
• Graph partitioning technique
• Bi-partitions an edge-weighted graph in an optimal sense
• Normalized cut (Ncut) is the optimizing criterion
i j
wij
Edge weight => Similarity between i and j
A B
Minimize Ncut(A,B)
Nodes
• Image segmentation
• Each pixel is a node
• Edge weight is similarity between pixels
• Similarity based on color, texture and contour cues

21. 21
Unknown clusters and centers
Maximization step:
Find the center (mean)
of each class
Start with random
model parameters
Expectation step:
Classify each vector
to the closest center

22. 22
Finding the centers from known
clustering

23. Segmentation fault
 A segmentation fault (often shortened to
segfault) or access violation is a particular
error condition that can occur during the
operation of computer software.
 A segmentation fault occurs when a program attempts to access
a memory location that it is not allowed to access, or attempts to
access a memory location in a way that is not allowed (for
example, attempting to write to a read-only location, or to
overwrite part of the operating system).

24. Segmentation Methods
 Thresholding approaches
 Region Growing approaches
 Classifiers
 Clustering approaches
 Markov random fields (MRF) models
 Artificial neural networks
 Deformable models
 Atlas-guided approaches
24

25. Thresholding
 Suppose that an image, f(x,y), is composed of
light objects on a dark background, and the
following figure is the histogram of the image.
 Then, the objects can be extracted by
comparing pixel values with a threshold T.
25

26. Region Growing
1. Define seed point
2. Add n-neighbors to list L
3. Get and remove top of L
4. Test n-neighbors p
if p not treated
if P(p,R)=True then p→L
and add p to region
else p marked boundary
5. Go to 2 until L is empty
 Two Regions R and ¬ R
SeedpointsSeedpoints ElementinElementinL
BorderelementBorderelementRegionelementRegionelement

27. Our approach: The Algorithm
 The left and right images areThe left and right images are
segmented and each areasegmented and each area
identifies a node of a graphidentifies a node of a graph
 A bipartite graph matchingA bipartite graph matching
between the two graphs isbetween the two graphs is
computed in order to match eachcomputed in order to match each
area of the left image with onlyarea of the left image with only
one area of the right imageone area of the right image
 This process yields a list ofThis process yields a list of
reliably matched areas and a listreliably matched areas and a list
of so-called don’t care areas.of so-called don’t care areas.
 The Outputs of the algorithmThe Outputs of the algorithm
are the disparity map and theare the disparity map and the
performance mapperformance map

28. GPCA
Generalized Principal Component Analysis (GPCA)
method for.
 modeling and segmenting mixed data using a
collection of subspaces
 done by introducing certain algebraic models into
data clustering.
 Unique property (applied to images) is that it
decomposes images into regions with
fundamentally different characteristics and
derives an optimal PCA-based transformation for
each region.

29. Computing a principal component
analysis
To compute a principal
component analysis in SPSS,
select the Data Reduction |
Factor… command from the
Analyze menu.

30. Segmentation Example

31. Intelligent Scissors
 Fully automatic segmentation is an unsolved
problem due to wide variety of images.
 Intelligent Scissors is a semi-automatic
general purpose segmentation tool.
 The efficient and accurate boundary
extraction, which requires minimal user input
with a mouse, is obtained.
 The underlying mechanism for the Intelligent
Scissors is the “live-wire” path selection tool.

32. More Complex Segmentation
Methods - snakes

33. One More Thing

34. VLSI IMPLEMENTATION

35. Floor plan of the
prototype chip
Layout of the
encoder module

36. Pros & Cons
 Very useful for rapid prototyping
 Strongly growing community and code base
 Problems:
 Very complex
 Overhead -> higher run-times
 Still under development

37. Summary / Closing
Thoughts
 Segmentation is the essential but critical problem in
the field of machine vision. At a stretch, robotics can
not be done with a complete knowledge about
foreground and background objects.
 We have proposed pixel based color segmentation
approach to segment the known backgrounds such
as forest, sky, ocean, underwater scenes and etc.
which will be of unique color generally and the results
obtained were satisfactory.
 This color segmentation process will overcome the
main problems with change of pose and occlusion
and overcomes the limitation occurs in the motion
analysis and background subtraction methods.

38. Conclusions
 Translation (visual to semantic) model for object recognition
 Identify and evaluate low-level vision processes for recognition
 Feature evaluation
 Color and texture are the most important in that order
 Shape needs better segmentation methods
 Segmentation evaluation
 Performance depends on # regions for annotation
 Mean Shift and modified NCuts do better than original NCuts for # regions < 6
 Color constancy evaluation
 Training with illumination helps
 Color constancy processing helps (scale-by-max better than gray-world)

39. Reference Reading
 Digital Image Processing
Gonzalez & Woods,
Addison-Wesley 2002
 Computer Vision
Shapiro & Stockman,
Prentice-Hall 2001
 Computer Vision: A Modern Approach
Forsyth & Ponce,
Prentice-Hall 2002
 Introductory Techniques for 3D Computer Vision
Trucco & Verri,
Prentice-Hall 1998

40. REFERENCES :
 S. Belongie, C. Carson, H. Greenspan, and J. Malik, "Color and
texture-based image segmentation using EM and its application to
content-based image retrieval," 6th International Conference on
Computer Vision, pp.675–682, 1998.
 E. Saber, A.M. Tekalp, R. Eschbach, and K. Knox, "Automatic image
annotation using adaptive color classification," Graph. Models Image
Process., vol.58, no.2, pp.115–126, 1996.
 S.C. Pei and C.M. Cheng, "Extracting color features and dynamic
matching for image data-base retrieval," IEEE Trans. Circuits Syst.
Video Technol., vol.9, no.3, pp.501–512, April 1999.
 T. Pavlidis and Y.-T. Liow, "Integrating region growing and edge
detection," IEEE Trans. Pattern Anal. Mach. Intell., vol.12, no.3,
pp.225–233, March 1990.
 C.-C. Chu and J.K. Aggarwal, "The integration of image
segmentation maps using region and edge information," IEEE Trans.
Pattern Anal. Mach. Intell., vol.15, no.12, pp.1241–1252, Dec. 1993.
 J. Fan, D.K.Y. Yau, A.K. Elmagarmid, and W.G. Aref, "Automatic
image segmentation by integrating color-edge extraction and seeded
region growing," IEEE Trans. Image Process., vol.10, no.10,
pp.1454–1466, Oct. 2001.

41. QUERRIES ?
Thank you.

42. AJAL.A.J
ASSISTANT PROFESSOR
UNIVERSAL ENGINEERING
COLLEGE
THRISSURMAIL : ec2reach@gmail.com MOB : 0890 730