1. Jake Larrimore, Andrew Stewart,
Esteban Bernal
Intelligent Image
Processing

2. What will be discussed?
1. What is Image Processing?
2. History
3. Push- Consumer driven
4. Pull- Industry (internally driven)
5. How does Image Processing work?
6. Advantages and Disadvantages
7. Current Applications
8. State of the art
9. Future direction

3. What is Intelligent Image Processing
• Well, can anyone define Intelligence?
• A rat can process visual data and interpret it in
order to solve problems, but we would not
consider a rat intelligent in comparison to a
human. Yet this simple task is extraordinarily
difficult to program a computer to do, and we
have come nowhere close to enabling computers
to match the image processing perceptibility of
rats.

4. Image Processing Defined
• Computer vision broadly refers to the discipline
where extraction of useful 2D and/or 3D
information from one or more images is of
interest [Chellepe et. al, 2005]
• Many computers and hand held devices have
cameras embedded in them, but they do not
process that information to perform a task,
therefore we cannot say that such a device has
vision

5. History
• Artificial Intelligence
• We live in a three-dimensional and dynamic
world. Therefore, in order for a robot or other
A.I. artifact to interact with its surroundings, it
must be able to obtain and process information
through some sort of sensing ability.

6. Defining the field
• What information should be extracted from the
outputs of visual sensors
• How is this information extracted
• How should this information be represented
• How must this information be used to allow a
robot system to perform its task
[Faugeras, 1949]

7. New Fields
• Neuromorphic Engineering - recreate the way
the eye and other neurobiological sensing
systems work and applying it to silicon chips.
• Imaging device must contend with shadows and
sunlight; conventional sensors, such as those in
digital cameras, can't capture pictures well
under these conditions

8. Push
• We now have regular access to computers with
dual core processors, and some with multi-level
processors that can manage multiple GHz.
• high speed networking

9. Pushing, continued
• artificial neural networks
▫ mathematical models derived from biological neural
networks
• After the development of the “back-propagation
learning algorithm for neural networks, [it was for
the] first time...feasible to train a non-linear neural
network equipped with layers of the so-called
hidden nodes [Egmont-Petersena et. al, 2001].”
▫ preprocessing images, image reconstruction, image
restoration, image enhancement, data reduction and
feature extraction, and image compression

10. Pull
• Robotic Vision
▫ Developed to give autonomous robots the ability
to interact freely with their environment
▫ Some scientists say that “autonomous navigation
has become a mandatory function of mobile
intelligent robots [Kim et. al, 2008].”

11. Pull- Defense and Security
• The ability to detect threats to the public without
human interaction would be vital to reducing the
cost, time, and efficiency of such security
▫ TSA and Airports
▫ Crowd Control/Riot Control
▫ Tracking of Fugatives

12. Pull- Safety on the Roads
• Traffic and automobiles
 implemented in automobiles, traffic lights,
and city streets
• National Highway Traffic and Safety
Administration reports that there were a
reported 6.4 million car accidents on the streets
of the U.S. causing over 230 billion dollars in
damage. These accidents killed almost 30
thousand people and injury about 2.9 million
people

13. Pull-Medical Fields
• (CT) scans are generally used to make the
diagnostic and to plan the surgery for liver
cancer
▫ radiologist must trace the contour of the liver
manually as well as the tumor and the main
vessels (which show up very similarly on scans)
• If we had intelligent processes to investigate
these scans to more accurately determine the
condition of patients and to enhance the scans to
produce better and more vivid images

14. Still Pulling- Manufacturing Automation
• computer–based machine vision system applied
in computer-aided inspection- Chips, coffee
beans, etc
• Safer work areas to ensure that workers are not
injured by automated devices

15. Pull- ENTERTAINMENT!
• 20 percent of households with more than $77,000 a
year in pretax income, more money is spent on
entertainment - $4,516 a year - than on health care,
utilities, clothing or food eaten at home [Darlin,
2005]
• Billions of dollars driving the market toward more
user friendly computer interaction
• We should be able to communicate in a more
intuitive manner, directly with a context-aware
environment, thus enabling them to achieve their
goals more easily and freeing their minds to think
even further ahead of their current tasks and
problems [Meyer et al, 2003]

16. More Entertainment
• QB1-They are using multiple cameras to achieve
depth perception in computers, which enables
them to have an interface based the user directly
touching and manipulating virtual components
positioned around his body
• Applications in gaming

17. Basic Idea
• Intelligent / non-intelligent
• Humanistic Intelligence: Recognizing that the
human brain is perhaps the greatest neural
network of its kind
▫ WearComp
▫ Eyetap

18. Basic Idea - WearComp
• “Always ready" device
• Six informational paths of interaction
▫ Unmonopolizing of user’s attention
▫ Attentive to the environment
▫ Communicative to others
▫ Unrestrictive
▫ Observable
▫ Controllable

19. Basic Idea - Eyetap
• Lightspace analyzer
• Lightspace modifier
• Lightspace synthesizer

20. Computer and Machine Vision
• CV: "the science and technology of machines
that see, where see in this case means that the
machine is able to extract information from an
image that is necessary to solve some task” -
Wikipedia
• CV: focus on the complex real-world situations
• MV: focus on machines that can see

21. Image Processing Chain (IPC)
• Describe the steps and operations involved to
successfully extract data from an image
• General operations utilized across different
image processing systems:

22. IPC – Pre-processing
• Suppress unwilling distortions
• Enhance Important features
• Divided in three operations
▫ Reconstruction
▫ Restoration
▫ Enhacement

23. IPC - Segmentation
• Partitioning into correlated and
not overlapped fragments
• Statistical pattern recognition
• Neural networks

24. IPC – Object Recognition
• Requires knowledge
• Knowledge representation:
▫ grammars and languages
▫ predicate logic
▫ production rules
▫ fuzzy logic
▫ semantic nets
▫ frames and scripts

25. IPC – Object Recognition

26. IPC – Image understanding
• Find a relation between the input images and
previously established models of the real world
[Sonka et al, 2008]
• eTRIMS project (University of Bonn)

27. Performance: Advantages
• Technology improvements
 storage
 processing power
 bandwidth and wireless access
 image resolution
 Supercomputing processing
• Facilitate human's life (Google goggles)
• Improve human's life (Medical usage, traffic safety)
• Improve economic (Manufacturing)

28. Performance Disadvantages
• Some technologies are expensive
 To develop
 To maintain
• Reduce the need of human work?
• Technical difficulties
 Lost of information
 Interpretation
 Noise
 Too much data

29. Applications
• Wherever you can image - Just a few examples:
 Automotive industry pedestrian detection
 Potato chips image processing system to control
quality
 Medical applications - diseases detection
 Traffic control
 Autonomous driven cars
• Limitations?
 Human's ability of understanding the brain

30. The state of the art
• Image processing
formerly the domain of
large institutions
• Very specific
applications
• Large projects
• Imaging technology is
now widely available
• Consumer products

31. Google Goggles
• First came text-based text searching
• Then came text-based image searching
• Now image-based image searching

32. Microsoft Photosynth
• Stitches together a three-dimensional scene
from several images of the same subject
• Creates a navigable scene

33. Microsoft Photosynth

34. Autonomous Driving
• An example of computer vision
• Norman Bel Geddes’ Futurama (1939)
• Still a ways off…

35. Lane Departure Warning System

36. Lane Departure Warning System
• Canny edge detection algorithm
• Line extraction by Hough transformation

37. Pedestrian tracking system
• Shape-based voting algorithm
• Similar Gaussian and Hough methods
• Other applications
▫ Automatic doors
▫ Light usage (efficiency)
▫ Cash register (security)

38. Hand & gesture tracking
• Control of entertainment systems
▫ XBox Kinnect
• Sign language

39. Face detection
• Many applications
▫ Bankcard identification
▫ Access control
▫ Security monitoring
▫ Biometrics systems
• Advancement based on
▫ Large image databases
▫ Advances in algorithms
▫ Methods for evaluating performance
• More difficult than simple line detection

40. Face detection
• Traditional methods
▫ PCA
▫ Neural networks
▫ Sparse graph matching
▫ HMMs
▫ Template matching
• Newer methods
▫ Improved template
matching w/ 3D
models
▫ Line Edge Map (LEM)
▫ SVMs

41. Face recognition

42. Face recognition

43. Medical imaging
• Computerized tomography
• Magnetic resonance imaging
• Ultrasound
• Nuclear medicine imaging
• Computerized hematological cell analysis

44. Medical imaging
• Knowledge based systems
▫ Rule based expert systems
▫ Structural-functional correlation
▫ Artifact reduction
• Trending towards convergence of artificial
intelligence and image analysis

45. Looking ahead…

46. Hyperspectral imaging
• Beyond the visible spectrum

47. Hyperspectral imaging

48. Hyperspectral imaging

49. Hyperspectral imaging

50. LIDAR
• Light Detection and Ranging
• Remote optical sensing technology
• Three-dimensional contoured imaging

51. LIDAR

52. LIDAR

53. Biologically motivated processing

54. Nonlinear methods
• Linear methods OUT
▫ Human visual system too complicated for linear
models
▫ Have difficulty removing unwanted noise
• Nonlinear methods IN
▫ Generally superior in edge smoothing,
enhancement, filtering, feature extraction, etc
▫ Computationally expensive
▫ Reduced cost makes these practical and effective