Red Eye Removal

1/26

Red Eye Removal
y

Tony Meccio

Red eye removal 17/04/2008

Overview 2/26

• I t d ti
Introduction t the Red Eye problem
to th R d E bl
• Red Eye prevention
• Red Eye detection
• Semi-automatic methods
• Automatic methods
•R dE
Red Eye correction
ti
• Desaturation
• Inpainting techniques
• False positives and unnatural corrections
a po a du au a o o
• Red Eye removal examples


The Red Eye problem 3/26

The Red E
Th R d Eye phenomenon is a well-
h i ll
known problem which happens
when taking flash-lighted p
g g pictures
of people.

The pupils in the picture appear red
instead of black.

This h
Thi happens more often when
ft h
using compact, consumer-oriented
cameras.
a a


Why the Red Eye? 4/26

Red Eye Cone
Eye

α β

• The red eye cone shines from the flashed eye
back at the flash with an angle α;
g ;
• Its red color is caused by the reflection of the
flash off the blood vessels o the retina;
a o b ood of a;
• The camera will record this red hue if the angle β
be ee
between the flash and the camera is not greater
e as a d e ca e a s o g ea e
than α.


Red Eye prevention 5/26

Eye
Red Eye Cone

α
β

• If the equipment allows it, the flash can be spaced
further away from the sensor in order to increase
y
β (not possible on compact devices);
• One o more additional flashes before picture
O or o add o a a b o p u
acquisition make the iris tighten and decrease α;
• This methods reduce the probability of the Red
s e ods educe e p obab y o e ed
Eye phenomenon but don’t remove it entirely.


Red Eye detection 6/26

Most f th ti
M t of the times, red eyes must be removed
d tb d
during post-processing.
For d
F red eyes to be successfully removed, they must
t b f ll d th t
be first detected then corrected.

Methods are classified according to the detection
phase:
h
• Semi-automatic methods ask the user to
manually l
ll localize th red eyes;
li the d
• Automatic methods detect the red eyes
themselves.
th l


Semi-automatic methods 7/26

The eyes are manually selected
using a visual interface (
g (Adobe
Photoshop ®, Corel Paint Shop
Pro ®, ACD See ®, etc.)
Pros:
• Eyes are easy to localize for
y y
men.
Cons:
• It may be difficult to have
such an interface on a mobile
device;
• Automatic methods are easier
uo a od a a
to use and more appealing.


Automatic methods 8/26

Automatic methods attempt to find red eyes on
A t ti th d tt t t fi d d
their own. The task is harder than it may seem:
No “
N “perfect” R d E
f t” Red Eye d t ti
detection method has
th d h
been developed yet.
Automatic methods extract features from images in
A t ti th d t tf t f i i
order to identify red eyes. Different methods work
on different features:

• Face detection • Skin detection
• Eye detection • Flash-noFlash
comparison


Face-based methods 9/26

•F
Faces are looked for using a multiple feature
l k df i lti l f t
object based approach;
•O
Once th f
the face features have been found then the
f t h b f d th th
research is restricted to red pixels.

Face
Extraction

Extracted Multiple Masks
Area Red-Eye
Red Eye Research


Eye-based methods 10/26

Similar t face detection, but more complex
Si il to f d t ti b t l
because the features are less evident:
•EEyes are seeked, matching fi d t
k d t hi fixed templates at
l t t
different resolutions with regular eyes present
into the images, or looking for red pixels using
g , g p g
computed color LUTs.

Initial Single Eye
g y Pairingg
Candidate
Verification Verification
Detection


Skin-based methods 11/26

• Ski i d t t d first by pixel colors;
Skin is detected fi t b i l l
• Red circular patches near the skin are then
looked for.
l k df
This approach is simpler and does not take into
account th presence of more complex features.
t the f l f t

Skin
Detection Red Circular Pairingg
Patches Verification

Research

Optional


Flash-noFlash methods 12/26

• Two different pictures,
one with flash and one
ith fl h d
without, are acquired
one after the other;;
• Red eyes are detected
as patches whose color is
p
red in the “flash”
image and black in the
“nonflash” i
“ fl h” image.
This approach has several drawbacks:
• The dimension of the buffer must double;
• The two images may be mis-aligned;
e o ages ay s a g ed;
• The subject(s) may move between acquisitions.


An algorithm in detail 13/26

Red Color
Input Skin Morphological Detection in
Detection Operators Skin Area
Image

Morphological
Operators

Corrected Red-Eye Pairing Single Eye
Verification Verification
Image
g Correction

The Algorithm is Skin Feature Extraction based.
• First the skin is extracted and morphologically
modified to blob the enhanced areas;
• Then a successive red color detection is performed to find
red eyes in the skin areas;
• The red regions are then dilated, eroded and analyzed
to identify the Red-Eyes pairs.


Example 14/26

Input Image Skin Detection Morphological Red Color Detection
Operators
in Skin Area

Morphological Pairing Corrected
Operators Verification
Image


Input/Output comparison 15/26


Algorithm drawbacks 16/26

• U bl t perform single red eye detection;
Unable to f i l d d t ti

• The skin detection is performed over the whole
image (slow);

• Big (
g (slow) morphological operators p
) p g p permit to g
get
good results only on small images (less than 1
Mpixels): it would require even bigger (and slower)
operators t operate on larger images.
t to t l i


Red eye correction 17/26

Once red eyes have been detected, they must be
O d h b d t t d th tb
corrected.

Red eye correction is quite simpler than detection,
but there are more diffi lt cases than others.
b t th difficult th th

Correction may vary from simple desaturation to
complete reconstruction of iris and pupil.


Desaturation 18/26

Desaturation means lowering/zeroing the chrominance
components while mantaining the luminance component
component.
It is the best way to correct “easy” red eyes.


Washed-out irises 19/26

Washed-out irises Wrong correction

Sometimes irises are totally washed out by reflected light.
In these cases a simple desaturation or color correction is not
enough.

It is necessary to use a more complex method to reconstruct a
realistic image of the eye.


Inpainting techniques 20/26

Some tools completely reconstruct the irises and the pupils
to
t replace the red eye (Jasc Paint Shop Pro ®).
l th d (J P i t Sh P ®)
The results, however, are often unrealistic and look like glass
eyes.


False positives 21/26

One of the biggest issue in red eye removal are false
p
positives in the detection phase.
p

Unwanted
corrections are
much less
desirable than
missing ones
ones.


Unnatural corrections 22/26

• Unnatural corrections are another
important issue.
• The most common ones are:
• Partial correction: only a
Partial correction
portion of the red pupil has
f
been corrected;
• Noisy correction: the presence
of heavy noise or jpeg
compression can introduce false
red pixels around the pupil and Noisy correction
thus a strange correction is
made over the iris;
• Wrong luminance correction:
in this case the disk has been
correctly found but the
correction is unnatural due to Wrong luminance
wrong luminance distribution. correction


Red Eye removal examples 23/26

StopRedeyes® AutoRemover®


References 26/26

• J.Y. Hardeberg, “Red Eye Removal using Digital Color Image Processing”, Conexant
Systems, Inc. , Redmond, Washington, USA
• M. Gaubatz, R. Ulichney, “Automatic Red-Eye Detection and Correction”, HP Lab, Cornell
University,
University ICIP 2002
• S. Ioffe, “Red Eye Detection With Machine Learning”, Fujifilm Software, California, ICIP
2003.
• F Gasparini, R. Schettini “Automatic Redeye Removal for Smart Enhancement of Photos of
F. Gasparini R Schettini, Automatic
Unknown Origin”, DISCO University of Milano-Bicocca, VIS 2005.
• H. Luo, J. Yen, D. Tretter, “An Efficient Redeye Detection and Correction Algorithm”, HP
labs, Palo Alto, California, ICPR 2004.
• A. Patti, K. Konstantinides, D. Tretter, Q. Lin, “Automatic Digital Redeye Reduction”, HP
Labs, Palo Alto California, ICIP 1998.
• L. Zhang, Y. Sun, M. Li, H. Zhang, “Automated Red-Eye Detection and Correction in Digital
Photographs”,
Photographs” Microsoft Research Asia China ICIP 2004
Asia, China,
• B. Smolka, K. Czubin, J.Y. Hardeberg, K.N. Palataniotis, M. Szczepanski, K. Wojciechowski,
“Towars Automatic Redeye Effect Removal”, Pattern Recognition Letter 2003.
• J S Schildkraut R T Gray “A Fully Automatic Redeye Detection and Correction Algorithm”,
J.S. Schildkraut, R. T. Gray, A Algorithm
Kodak Company, NY, ICIP 2002.
• R. Schettini, F. Gasparini, F. Chazli, ”A Modular Procedure for Automatic Redeye Correction
in Digital Photos”, DISCO University of Milano-Bicocca, SPIE 2004
• PETSCHNIGG, G., AGRAWALA, M., HOPPE, H., SZELISKI, R., COHEN, M., and TOYAMA, K.
2004. Digital photography with flash and no-flash image pairs. ACM Transactions on Graphics
23, 3 (Aug.), 664–672.


Red Eye Removal

Recommandé

Recommandé

Contenu connexe

Dernier

Dernier (20)

En vedette

En vedette (20)

Red Eye Removal