The document describes an automatic road sign recognition system developed for high resolution roadside video. It uses a color space method to detect signs by recognizing color patterns. Specifically, it detects signs by converting RGB images to HSI color space and matching hue values. The system was tested on various sign types under different lighting conditions with good detection results. In conclusion, the approach provides a robust and fast way to detect most common road signs in New Zealand videos and identify the sign type and location.

1. Automatic Road Sign
Recognition from Video
Dr Wei Liu
Senior Engineer

2. Presentation Outline
• Introduction
• Methodology
• Results
• Conclusions

3. Introduction
• Road signs provides important information for
guiding, warning, or regulating the drivers’
behaviour in order to make driving safer and
easier
• The Road Sign Recognition (RSR) is a field of
applied computer vision research concerned
with the automatic detection and classification of
traffic signs in traffic scene images acquired
from a moving car.

4. Introduction
• Pavement Management Services have
developed the first (and currently only) truly
spatially registered video system in Australia.
• The digital video system offers continuous, high
resolution video capture of five different views
along the roadway.

5. Introduction
• A road sign recognition system (RS2) has been
developed for the high resolution roadside video
recorded by PMS video system.
• The recognition process of RS2 is divided into three
distinct parts:
• Detection and Location
• Recognition and Classification
• Display and record for information of road signs

6. Introduction
• The PMS video system consists of five industrial quality
digital cameras mounted in any directional configuration
on a host vehicle.
• The cameras works well in varying and low light
conditions, at all times maintaining high shutter speeds
to eliminate motion blur.
• The cameras have individual image resolutions of:
• 768x576 (broadcast quality of road asset views)
• 1024x768 (high resolution image for pavement view)

7. Introduction
• Image capture and survey position are
determined by precision odometer and GPS
location equipment.
• The image capture trigger is accurate enough at
synchronising the image captured to make
panoramic views from collection of cameras at
high test speed (100km/hr).

8. Introduction
• Typically, high resolution images are collected
for every one meter of the road surface and
every ten meters of the roadside assets.
• The spatial reference is achieved within the
video itself by creating a ‘data-cloud’ of DGPS
points for each frame of the video, which gives it
the ability to locate and therefore ascribe a
DGPS coordinates to any fixed item within the
view of each of the five cameras.

9. Introduction
• PMSVideo is a computer software tool used to
enable the playback and examination of video
collected using Pavement Management
Services digital video system.
• The PMSVideo software allows the user to
find road sections according to the road
owners road referencing scheme and even
recording notes and other useful information
for use in other road management systems.

10. Introduction
• To ensure the creation of accurate location of
road assets in the video, a grid calibration
procedure for each camera is applied prior to the
commencement of the survey.
• After calibration, the PMS video system is able
to provide a three dimensional plot from a two
dimensional plot by mapping the world
coordinate to the views presented by each
camera with the same accuracy of DGPS data
cloud.

11. Introduction
• The difficulty in recognizing road signs is largely due to
the following reasons:
• The colors of road signs, particularly red, may fade
after long exposure to the sun.
• Air pollution and weather conditions may decrease
the visibility of road signs.
• Outdoor lighting conditions varying from day to night
may affect the colors of road signs.
• Obstacles, such as vehicles, pedestrians, and other
road signs, may partially occulde road signs.
• Video images of road signs will have motion blur if the
camcorder is mounted on a moving vehicle due to
vehicle vibration as well as motion.

12. Methodology
• While lots of attempts at automated sign
recognition were based on the detection of
shape patterns, the proposed method for PMS
Video detects road signs by recognising their
patterns in color space.

13. Methodology
• How can we quantitatively describe a color?
• we usually treat colors as RGB triples. The
three components define the amount of red,
green, and blue, respectively, whose
combination results in the desired color on a
computer screen. Typically, each channel
uses discrete values from 0 to 255.
• The color space formed by all possible RGB
values is also called the RGB space.

14. Methodology
• The RGB color space is easy to use and
represents color in the same way as the monitor
requires it for its display. However, for computer
vision applications such as the recognition of
objects, other color spaces are more useful.
• We will introduce the HSI color model, standing
for hue, saturation, and intensity.
• These dimensions characterize important object
properties more naturally as compared to the
RGB components.

15. Methodology
• HSI Color Space
• Hue is determined by the dominant wavelength in the
spectral distribution of light wavelengths.
• Saturation is the magnitude of the hue relative to
other wavelengths.
• It is defined as the amount of light at the dominant
wavelength divided by the amount of light at all
wavelengths.
• Intensity is a measure of the overall amount of light
within the visible spectrum.
• It is a scale factor that is applied across the entire
spectrum.

16. Methodology
• HSI Color Space
•Hue
•Saturation
•Brightness

17. Methodology
• Conversion from RGB to HSI
HSI
RGB

18. Methodology
• Conversion from RGB to HSI
)
)(
(
)
(
2
2
arccos
2
B
G
B
R
G
R
B
G
R
H
−
−
+
−
−
−
=
)
,
,
min(
3
1 B
G
R
B
G
R
S
+
+
−
=
3
B
G
R
I
+
+
=

19. Methodology
• Advantages of using HSI color space for Sign
Detection
• It allows a better tolerance to changes in
lighting conditions compared to other color
models
• A specific color can be recognized by
matching a small range of hue value.
• Ability to detect signs with different shape and
detect composite signs

20. Methodology
Morphological
operations
Region
elimination
Generate HSI
matrix
Sign Detection
Load
frames

21. Results
• Warning Signs-Normal Condition

22. Results
• Warning Signs-Under shade

23. Results
• Warning Signs-Under shade

24. Results
• Warning Signs-Multiple

25. Results
• Warning Signs-uneven lamination

26. Results
• Information Signs

27. Results
• Information Signs

28. Results
• Information Signs

29. Results
• Information Signs

30. Results
• Regulatory Signs

31. Results
• Regulatory Signs

32. Results
• Regulatory Signs

33. Results
• Regulatory Signs

34. Results
• Street Signs

35. Results
• Street Signs

36. Results
• Street Signs

37. Results
• Street Signs

38. Results
• Sign Recognition Results on Google Earth

39. Conclusions
• An automatic road sign recognition module from road
video collected by PMS video system was developed.
• The proposed approach is robust and fast for detection
of most of road signs commonly found in New Zealand,
including warning signs, information signs, regulatory
signs, and street signs.
• The sign recognition results include the exact location ,
type of road sign occurred in the video frame, and the
image containing the road signs detected, which can be
used for road sign condition evaluation.