3. Augmented Reality Definition
Defining Characteristics [Azuma 97]
Combines Real and Virtual Images
- Both can be seen at the same time
Interactive in real-time
- Virtual content can be interacted with
Registered in 3D
- Virtual objects appear fixed in space
4. What is not Augmented Reality?
Location-based services
Barcode detection (QR-codes)
Augmenting still images
Special effects in movies
…
… but they can be combined with AR!
17. AR Displays
AR
Visual Displays
Primarily Indoor Primarily Outdoor
Environments (Daylight) Environments
Not Head-Mounted Head-Mounted Head-Mounted Not Head Mounted
Display (HMD) Display (HMD) (e.g. vehicle mounted)
Virtual Images Projection CRT Display Liquid Crystal Cathode Ray Tube (CRT) Projection Display
or Virtual Retinal Display (VRD) Navigational Aids in Cars
seen off windows using beamsplitter Displays LCDs Many Military Applications Military Airborne Applications
& Assistive Technologies
e.g. window e.g. Reach-In e.g. Shared Space e.g. WLVA e.g. Head-Up
reflections Magic Book and IVRD Display (HUD)
19. Head Mounted Displays (HMD)
- Display and Optics mounted on Head
- May or may not fully occlude real world
- Provide full-color images
- Considerations
• Cumbersome to wear
• Brightness
• Low power consumption
• Resolution limited
• Cost is high?
20. Key Properties of HMD
Field of View
Human eye 95 degrees horizontal, 60/70 degrees vertical
Resolution
> 320x240 pixel
Refresh Rate
Focus
Fixed/manual
Power
Size
21. Types of Head Mounted Displays
Occluded
The image cannot be
displayed. Your
computer may not have
enough memory to open
the image, or the image
may have been
See-thru
corrupted. Restart your
computer, and then open
the file again. If the red x
still appears, you may
have to delete the image
and then insert it again.
The
ima
ge
can
not
be
dis
play
ed.
You
r
co
mp
uter
Multiplexed
22. Immersive VR Architecture
Virtual
World
head position/orientation
Head! Non see-thru!
Tracker Image source
& optics
Host! Data Base! Rendering!
Frame!
Processor Model Engine
Buffer
virtual
to network object
Display!
Driver
23. See-thru AR Architecture
head position/orientation
Head! see-thru!
Tracker combiner
real world
Host! Data Base! Rendering!
Frame!
Processor Model Engine
Buffer
to network Virtual Image
Display! superimposed!
Driver over real world
object
Image source
30. The Virtual Retinal Display
Image scanned onto retina
Commercialized through Microvision
Nomad System - www.mvis.com
31. Strengths of optical AR
Simpler (cheaper)
Direct view of real world
Full resolution, no time delay (for real world)
Safety
Lower distortion
No eye displacement
but COASTAR video see-through avoids this
32. Video AR Architecture
Head-mounted
camera aligned to
head position/orientation display optics
Video image
Head!
Tracker of real world
Video!
Processor
Host! Graphics! Digital!
Frame!
Processor renderer Mixer
Buffer
to network
Display!
Driver
Virtual image
inset into
Non see-thru!
video of real
Image source world
& optics
38. Vuzix Display
www.vuzix.com
Wrap 920
$350 USD
Twin 640 x 480 LCD displays
31 degree diagonal field of view
Weighs less than three ounces
39. Strengths of Video AR
True occlusion
Kiyokawa optical display that supports occlusion
Digitized image of real world
Flexibility in composition
Matchable time delays
More registration, calibration strategies
Wide FOV is easier to support
40. Optical vs. Video AR Summary
Both have proponents
Video is more popular today?
Likely because lack of available optical products
Depends on application?
Manufacturing: optical is cheaper
Medical: video for calibration strategies
41. Eye multiplexed AR Architecture
head position/orientation
Head!
Tracker real world
Host! Data Base! Rendering!
Frame!
Processor Model Engine
Buffer
to network
Display! Virtual Image
Driver inset into!
real world scene
Opaque!
Image source
47. Projector-based AR
User (possibly
head-tracked)
Projector
Real objects Examples:
with retroreflective Raskar, MIT Media Lab
covering Inami, Tachi Lab, U. Tokyo
59. Virtual Showcase
Mirrors on a projection table
Head tracked stereo
Up to 4 users
Merges graphic and real objects
Exhibit/museum applications
Fraunhofer Institute (2001)
Bimber, Frohlich
72. The Registration Problem
Virtual and Real must stay properly aligned
If not:
Breaks the illusion that the two coexist
Prevents acceptance of many serious applications
73. Sources of registration errors
Static errors
Optical distortions
Mechanical misalignments
Tracker errors
Incorrect viewing parameters
Dynamic errors
System delays (largest source of error)
- 1 ms delay = 1/3 mm registration error
74. Reducing static errors
Distortion compensation
Manual adjustments
View-based or direct measurements
Camera calibration (video)
76. Dynamic errors
Application Loop
x,y,z
Tracking r,p,y
Calculate Render Draw to
Viewpoint Scene Display
Simulation
20 Hz = 50ms 500 Hz = 2ms 30 Hz = 33ms 60 Hz = 17ms
Total Delay = 50 + 2 + 33 + 17 = 102 ms
1 ms delay = 1/3 mm = 33mm error
77. Reducing dynamic errors (1)
Reduce system lag
Faster components/system modules
Reduce apparent lag
Image deflection
Image warping
78. Reducing System Lag
Application Loop
x,y,z
Tracking r,p,y
Calculate Render Draw to
Viewpoint Scene Display
Simulation
Faster Tracker Faster CPU Faster GPU Faster Display
79. Reducing Apparent Lag
Virtual Display Virtual Display
x,y,z
Physical r,p,y
Physical
Display Display
(640x480) (640x480)
Tracking
1280 x 960 Update 1280 x 960
Last known position Latest position
Application Loop
x,y,z
Tracking r,p,y
Calculate Render Draw to
Viewpoint Scene Display
Simulation
80. Reducing dynamic errors (2)
Match input streams (video)
Delay video of real world to match system lag
Predictive Tracking
Inertial sensors helpful
Azuma / Bishop 1994