SlideShare une entreprise Scribd logo
1  sur  126
Télécharger pour lire hors ligne
INTRODUCTION TO XR
COMP 4010 Lecture One
Mark Billinghurst
July 28th 2022
mark.billinghurst@unisa.edu.au
Who am I ..
• Mark Billinghurst
• Director of the Empathic Computing Lab
• Univ. South Australia, Univ. of Auckland
• Conducting research in:
• Collaborative AR, AR/VR Interface Design, HCI
• Empathic Computing
• Previous worked at:
• Google, Amazon, Nokia, British Telecom
• MIT, Univ. of Washington, NAIST, Univ. of Canterbury
• PhD from University of Washington (2002)
UniSA IVE - https://unisa.edu.au/research/IVE/
Class Logistics
• Weekly lecture (2 hrs)
• Thursday 10am – 12am
• Room: Online
• Weekly Practical (1 hr)
• Thursday 9am – 10am
• Assessment
• 3 projects @ 20%, 30%, 40%
• Class participation @ 10%
• What you will need
• iOS or Android phone/tablet
• Access to laptop/PC for development
Equipment Available
• VR Laboratory
• 6 VR capable PCs
• High end graphics cards
• 3 Oculus Rift HMDs
• Wide Field of View display
• 1 HTC Vive HMD
• Room scale tracking
• IVE Equipment
• 6 HoloLens2 AR displays
• See through AR display
• 10 Oculus Quest HMD
• Self contained tracking
Snap Lens Studio
Available from https://lensstudio.snapchat.com/
Unity3D
Available from www.unity3d.com
Lecture Schedule – 13 lectures
Assessment
• Assignment 1: Mobile AR (20%)
• Develop a Snap Lens
• AR Tracking and Interaction
• Assignment 2: Interactive VR (30%)
• Create an interactive VR scene
• Oculus Quest, HTC Vive, Rift
• Assignment 3: HMD Experience (40%)
• Develop either VR HMD experience
• Oculus Quest
• Class Participation (10%)
• Watch at least one short YouTube AR/VR video each week
• Be prepared to present video each week
WHAT IS AR/VR/MR/XR?
1967 – IBM 1401 – half of the computers in the world, $10,000/month to run
2013 Google Glass
The Incredible Disappearing Computer
1960-70’s
Room
1970-80’s
Desk
1980-90’s
Lap
1990-2000’s
Hand
2010 -
Head
Graphical User Interfaces
• Separation between real and digital worlds
• WIMP (Windows, Icons, Menus, Pointer) metaphor
Rekimoto, J. and Nagao, K. 1995. The world through the computer: computer augmented interaction with real world environments.
Making Interfaces Invisible
(c) Internet of Things
Internet of Things (IoT)..
• Embed computing and sensing in real world
• Smart objects, sensors, etc..
(c) Internet of Things
Virtual Reality (VR)
• Users immersed in Computer Generated environment
• HMD, gloves, 3D graphics, body tracking
The First VR Experience …
https://www.youtube.com/watch?v=pAC5SeNH8jw
Virtual Reality Definition
•Defining Characteristics
• Sense of Immersion
• User feels immersed in computer generated space
• Interactive in real-time
• The virtual content can be interacted with
• Independence
• User can have independent view and reaction to environment
David Zeltzer’s AIP Cube
Autonomy – User can to react to events
and stimuli.
Interaction – User can interact with
objects and environment.
Presence – User feels immersed through
sensory input and output channels
Interaction
Autonomy
Presence
VR
Zeltzer, D. (1992). Autonomy, interaction, and presence. Presence: Teleoperators & Virtual Environments, 1(1),127-132.
VR Demo
Types of VR
2
2
Augmented Reality (AR)
• Virtual Images blended with the real world
• See-through HMD, handheld display, viewpoint tracking, etc..
Augmented Reality Definition
•Defining Characteristics [Azuma 97]
• Combines Real and Virtual Images
• Both can be seen at the same time
• Interactive in real-time
• The virtual content can be interacted with
• Registered in 3D
• Virtual objects appear fixed in space
Azuma, R. T. (1997). A survey of augmented reality. Presence, 6(4), 355-385.
Augmented Reality
Star Wars - 1977
2008 - CNN
https://www.youtube.com/watch?v=thOxW19vsTg
https://www.youtube.com/watch?v=cCL3bFuC4IA
• Weak AR
• Imprecise tracking
• No knowledge of environment
• Limited interactivity
• Handheld AR
• Strong AR
• Very accurate tracking
• Seamless integration into real world
• Natural interaction
• Head mounted AR
Strong vs. Weak AR
Augmented RealityApplications
AR vs VR
From Reality to Virtual Reality
Internet of Things Augmented Reality Virtual Reality
Real World Virtual World
Milgram’s Mixed Reality (MR) Continuum
Augmented Reality Virtual Reality
Real World Virtual World
Mixed Reality
"...anywhere between the extrema of the virtuality continuum."
P. Milgram and A. F. Kishino, (1994) A Taxonomy of Mixed Reality Visual Displays
Internet of Things
Milgram’s Reality-Virtuality continuum
Mixed Reality
Reality - Virtuality (RV) Continuum
Real
Environment
Augmented
Reality (AR)
Augmented
Virtuality (AV)
Virtual
Environment
"...anywhere between the extrema of the virtuality continuum."
P. Milgram and A. F. Kishino, Taxonomy of Mixed Reality Visual Displays
IEICE Transactions on Information and Systems, E77-D(12), pp. 1321-1329, 1994.
AugmentedVirtuality
•VR with view of the real world
Extended Reality (XR)
Augmented Reality Virtual Reality
Real World Virtual World
Mixed Reality
Extended Reality
Internet of Things
Ubiquitous Computing Continuum
• Weiser Continuum
Milgram – Weiser
Continuum
Mixed Reality Revisited (2019)
• Interviewed experts + read papers
• ”.. no universally agreed on, one-size-fits-
all definition of MR”
• Six notions of MR
• Continuum, Synonym, Collaboration
• Combination, Alignment, Strong AR
Speicher, M., Hall, B. D., & Nebeling, M. (2019,
May). What is mixed reality?. In Proceedings of the
2019 CHI conference on human factors in
computing systems (pp. 1-15).
• adva
The Metaverse
• Neal Stephenson’s “SnowCrash” (1992)
”.. When you live in a shithole, there's always the
Metaverse, and in the Metaverse, Hiro
Protagonist is a warrior prince.”
Defining the Metaverse
• Real-time, 3D, Interactive, Social, Persistent
AWE 2022 John Riccitiello
CEO, Unity Technologies
Metaverse Definition (2007)
• Metaverse Roadmap
• http://metaverseroadmap.org/
• The Metaverse is the convergence of:
• 1) virtually enhanced physical reality
• 2) physically persistent virtual space
Metaverse Dimensions
• Augmentation technologies that layer information onto our
perception of the physical environment.
• Simulation refers to technologies that model reality
• Intimate technologies are focused inwardly, on the identity
and actions of the individual or object;
• External technologies are focused outwardly, towards the
world at large;
Metaverse Components
• Four Key Components
• Virtual Worlds
• Augmented Reality
• Mirror Worlds
• Lifelogging
Mirror Worlds
• Simulations of external space/content
• Capturing and sharing surroundings
• Photorealistic content
• Digital twins
Matterport Deep Mirror Google Street View
Soul Machines
Lifelogging
• Measuring user’s internal state
• Capturing physiological cues
• Recording everyday life
• Augmenting humans
Apple Fitbit Shimmer
OpenBCI
Sensing
Immersing
Augmenting
Capturing
HISTORY OF AR/VR
History Timeline
https://immersivelifeblog.files.wordpress.com/2015/04/vr_history.jpg
When anything new comes along, everyone, like a child
discovering the world thinks that they’ve invented it, but you
scratch a little and you find a caveman scratching on a wall
is creating virtual reality in a sense.
Morton Helig (Hammit 1993)
Early History (30,000 BC - )
The history of VR is rooted in human’s first
attempts to reproduce the world around them
1800’s – Capturing Reality
• Panoramas (1790s)
• Immersive paintings
• Photography (1820-30s)
• Oldest surviving photo (Niépce, 1826)
• Stereo imagery (1830s)
• Wheatstone (1832)
• Brewster (1851)
• Movies (1870s)
• Muybridge (1878)
• Roundhay Garden Scene (1888)
Stereo Viewers
Wheatstone (1832)
Brewster (1860)
Viewmaster (1939)
3D Cinema Golden Era (1950-60s)
• Polarized 3D projection or anaglyph (red/blue)
Pepper’s Ghost (1862)
• Projecting onto glass to make ghost image appear on stage
• Dates back to Giambattista della Porta (1584)
The Master Key (1901) – AR Glass
"It consists of this pair of spectacles. While you
wear them every one you meet will be marked
upon the forehead with a letter indicating his or
her character. The good will bear the letter 'G,'
the evil the letter 'E.' … Thus you may
determine by a single look the true natures of
all those you encounter.”
L. Frank Baum
AR display showing if people are good or evil
1900s – Interactive Experiences
• Early Simulators (<1960s)
• Flight simulation
• Sensorama (1955)
• Early HMDs (1960s)
• Philco, Ivan Sutherland
• Military + University Research (1970-80s)
• US Airforce, NASA, MIT, UNC
• First Commercial Wave (1980-90s)
• VPL, Virtual i-O, Division, Virtuality
• VR Arcades, Virtual Boy
Link Trainer (1929 – 1950s)
• Flight Simulator Training
• Full six degree of freedom rotation
• Force feedback and motion control
• Simulated instruments
• Modeling common flight conditions
• Over 500,000 pilots trained
Link Trainer Video (1966)
• https://www.youtube.com/watch?v=MEKkVg9NqGM
Early HMD Patents
Early HUD (1958)
F16 – Head Up Display
Showing flight
information over the
real world
Sensorama (1955)
• Created by Morton Heilig
• Experience Theater
• Multi-sensory
• Visuals
• Sound
• Wind
• Vibration
• Smell
• No financial support
• Commercial failure
First HMD? (1961)
• Philco Headsight – Remote Camera Viewing
• Showing live camera view in monocular HMD
Ivan Sutherland (1960s)
6
6
Ivan Sutherland’s Head-Mounted Display (1968)
Sutherland Display
Super Cockpit (1965-80’s)
• US Airforce Research Program
• Wright Patterson Air Force Base
• Tom Furness III
• Multisensory
• Visual, auditory, tactile
• Head, eye, speech, and hand input
• Addressing pilot information overload
• Flight controls and tasks too complicated
• Research only
• big system, not safe for ejecting
LEEP Optics (1979)
• Large Expanse, Extra Perspective optics
• Developed by Eric Howlett
• Lens design for extremely wide field of view
• High resolution in centre, lower resolution in periphery
• 90o direct FOV, 140o corneal FOV
• Used as basis for most VR HMDs
LEEP Optics Design
The Data Glove (1981-82)
• Precursor, Sayre Glove
• Univ. of Illinois, 1977
• Thomas Zimmerman (1982)
• Fiber optic bend sensors
• Detecting finger bending
• Commercialized by VPL
• Mattel PowerGlove (1989)
VPL DataGlove Demo
• https://www.youtube.com/watch?v=fs3AhNr5o6o
NASA VIEW/VIVED (1981-86)
• Early HMD (McGreevy Humphries)
• LCD “Watchman” displays
• VIEW (Scott Fisher)
• Polhemus tracker
• LEEP-based HMD
• 3D audio (Convolvotron)
• DataGlove gesture input
• Simple graphics
VPL Research (1985 – 1999)
• First Commercial VR Company
• Jaron Lanier, Jean-Jacques Grimaud
• Provided complete systems
• Displays, software, gloves, etc
• DataGlove, EyePhone, AudioSphere
The University of North Carolina
at Chapel Hill (1980s-1990s)
7
6
Head-Mounted Displays
Tracking, Haptics, Applications
University of Washington (1989 - )
• Human Interface Technology Laboratory (HIT Lab)
• Founded by Tom Furness III
• Many AR/VR Innovations
• Virtual Retinal Display
• ARToolKit AR Tracking library
• GreenSpace shared VR experience
• VR and pain care
• VR and Education
First Industrial Use of AR (1990’s)
• 1992: Tom Caudell at Boeing coined the term “AR.”
• Wire harness assembly application begun
• Lead by Tom Caudell, and David Mizell
CAVE (1992)
• Projection VR system
• 3-6 wall stereo projection, viewpoint tracking
• Developed at EVL, University of Illinois Chicago
• Commercialized by Mechdyne Corporation(1996)
C. Cruz-Neira, D. J. Sandin, T. A. DeFanti, R. V. Kenyon and J. C. Hart. "The CAVE: Audio Visual
Experience Automatic Virtual Environment", Communications of the ACM, vol. 35(6), 1992, pp. 64–72.
CAVE Demo Video
• https://www.youtube.com/watch?v=aKL0urEdtPU
Desktop VR - 1995
• Expensive - $150,000+
• 2 million polys/sec
• VGA HMD – 30 Hz
• Magnetic tracking
Mobile/Wearable Systems (1995)
• 1995 Navicam (Rekimoto)
• Handheld AR
• 1997 Touring Machine (Feiner)
• Backpack AR, GPS, see-through display
• 1998 Tinmith (Thomas, UniSA)
• Outdoor gaming, CAD
Virtual Reality was HOT! .. In 1995..
Rise of Commercial VR Companies
• W Industries/Virtuality (1985 - 97)
• Location based entertainment
• Virtuality VR Arcades
• Division (1989 – 1998)
• Turn key VR systems
• Visual programming tools
• Virtual i-O (1993 -1997)
• Inexpensive gamer HMDs
• Sense8 (1990 - 1998)
• WorldToolKit, WorldUp
• VR authoring tools
Dactyl Nightmare
• https://www.youtube.com/watch?v=L60wgPuuDpE
Overview of VR in the 1990’s
Development of AR Tools
• 1996 CyberCode (Rekimoto)
• First matrix code tracking
• 1999 ARToolKit (Kato & Billinghurst)
• Open source tracking library
Tracking Demos
https://www.youtube.com/watch?v=TqGAqAFlGg0
ARToolKit
Cybercode
• April 2007 Computer World
• VRVoted 7th on list of 21 biggest technology flops
• MS Bob #1
Lessons Learned
• Don’t believe the hype
• Not everything is better inVR
• Many factors determine technology acceptance
• Human Centered Design/Design for users
• Need to move from Demo to Production
• Profitable niche markets first
• Follow the money
Mobile Phone AR (2005)
• Mobile Phones
• camera
• processor
• display
• AR on Mobile Phones
• Simple graphics
• Optimized computer vision
• Collaborative Interaction
AR Advertising (HIT Lab NZ 2007)
• Txt message to download AR application (200K)
• See virtual content popping out of real paper advert
• Tested May 2007 by Saatchi and Saatchi
Wellington Zoo Demo
https://www.youtube.com/watch?v=edTjuXcce_c
Eye of judgement (2007)
• Sony Playstation3 game
• First AR console game
• Over 300,000 copies sold
• Used Eye camera + tracking cards
2007 - AR Reaches Mainstream
• MIT Technology Review
• March 2007
• One of the 10 most exciting
technologies
• Economist
• Dec 6th 2007
• Reality, only better
Google Searches for AR
• Cross over in 2009, with more interest in AR than VR
2008 - Browser BasedAR
• Flash + camera + 3D graphics
• ARToolKit ported to Flash
• High impact
• High marketing value
• Large potential install base
• 1.6 Billion web users
• Ease of development
• Lots of developers, mature tools
• Low cost of entry
• Browser, web camera
Demo: GE Smart Grid
• https://www.youtube.com/watch?v=vJO_AZkCL9U
2008: Location Aware Phones
Nokia Navigator
Motorola Droid
Outdoor Information Overlay
• Mobile phone based
• Tag real world locations
• GPS + Compass input
• Overlay graphics on live video
• Applications
• Travel guide, Advertising, etc.
• Wikitude, Metaio, Layar, etc..
• iOS/Android, Public API released
Layar Demo (2008)
• https://www.youtube.com/watch?v=b64_16K2e08
AR in Magazines (2009- )
• Esquire Magazine
• Dec 2009 issue
• 12 pages AR content
• Many Others
• Wired
• Colors
• Red Bull
• etc.
Esquire Demo
Google Glass (2011 - )
Google Glass Demo
VR Second Wave (2010 - )
• Palmer Luckey
• HMD hacker
• Mixed Reality Lab (MxR) intern
• Oculus Rift (2011 - )
• 2012 - $2.4 million kickstarter
• 2014 - $2B acquisition FaceBook
• $350 USD, 110o FOV
The Oculus Kickstarter Video
• https://www.youtube.com/watch?v=aNSYscbxFAw
Desktop VR in 2016
• Graphics Desktop
• $1,500 USD
• >4 Billion poly/sec
• $600 HMD
• 1080x1200, 90Hz
• Optical tracking
• Room scale
Oculus Rift
Sony Morpheus
HTC/Valve Vive
2016 - Rise of Consumer HMDs
HTC Vive
• Room scale tracking
• Gesture input devices
Example Vive App – Tilt Brush
• https://www.youtube.com/watch?v=ijukZmYFX-0
VR2GO (2013)
• MxR Lab
• 3D print VR viewer for mobiles
• Open source hardware + software
• http://projects.ict.usc.edu/mxr/diy/vr2go/
• Early Mobile VR viewer
Google Cardboard
• Released 2014 (Google 20% project)
• >5 million shipped/given away
• Easy to use developer tools
+ =
Multiple Mobile VR Viewers Available
Epson Moverio BT-300
▪ Stereo see-through display ($700)
▪ 1280 RGB x 720 pixels, 23 degree FOV, 30Hz, 69g
▪ Android Powered, separate controller
▪ VGA camera, GPS, gyro, accelerometer
Smart Glasses Available
Social Mobile Camera AR Apps (2015 - )
• SnapChat - Lenses, World Lenses
• Cinco de Mayo lens > 225 million views
• Facebook - Camera Effects
• Google – Word Lens/Translate
Hololens (2016)
• Integrated system – Windows
• Stereo see-through display
• Depth sensing tracking
• Voice and gesture interaction
• Note: Hololens2 coming September 2019
ARKit/ARcore (2017)
• Visual Inertial Odometry (VIO) systems
• Mobile phone pose tracked by
• Camera (Visual), Accelerometer & Gyroscope (Intertial)
• Features
• Plane detection, lighting detection, hardware optimisation
• Links
• https://developer.apple.com/arkit/
• https://developers.google.com/ar/
• https://www.youtube.com/watch?v=SgAnnwl2VB8
ARKIT3 Demo
MagicLeap ML-1 (2018)
• Bi-Focal Display – two focus planes
• Horizontal FoV of 40o, vertical FoV of 30o, diagonal value of 50o
• 1280×960 resolution, Eye-tracking
• Separate display and computer
• Nvidia "Parker" Tegra X2 CPU, 8GB RAM, 128 GB storeage
• 6 DOF handheld controller, magnetic tracking
History Summary
• 1960’s – 80’s: Early Experimentation
• 1980’s – 90’s: Basic Research
• Tracking, displays
• 1995 – 2005: Tools/Applications
• Interaction, usability, theory
• 2005 - : Commercial Applications
• Mobile, Games, Medical, Industry
QUESTIONS ?
Resources
• Azuma, R. T. (1997). A survey of augmented reality. Presence: teleoperators & virtual
environments, 6(4), 355-385.
• Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., & MacIntyre, B. (2001). Recent
advances in augmented reality. IEEE computer graphics and applications, 21(6), 34-47.
• Milgram, P., & Kishino, F. (1994). A taxonomy of mixed reality visual displays. IEICE
TRANSACTIONS on Information and Systems, 77(12), 1321-1329.
• Billinghurst, M., Clark, A., & Lee, G. (2015). A Survey of Augmented Reality. Foundations and
Trends in Human–Computer Interaction, 8(2-3), 73-272.
• Speicher, M., Hall, B. D., & Nebeling, M. (2019, May). What is mixed reality?. In Proceedings
of the 2019 CHI conference on human factors in computing systems (pp. 1-15).
www.empathiccomputing.org
@marknb00
mark.billinghurst@auckland.ac.nz

Contenu connexe

Tendances

Tendances (20)

2022 COMP4010 Lecture5: AR Prototyping
2022 COMP4010 Lecture5: AR Prototyping2022 COMP4010 Lecture5: AR Prototyping
2022 COMP4010 Lecture5: AR Prototyping
 
2022 COMP4010 Lecture4: AR Interaction
2022 COMP4010 Lecture4: AR Interaction2022 COMP4010 Lecture4: AR Interaction
2022 COMP4010 Lecture4: AR Interaction
 
2022 COMP4010 Lecture 6: Designing AR Systems
2022 COMP4010 Lecture 6: Designing AR Systems2022 COMP4010 Lecture 6: Designing AR Systems
2022 COMP4010 Lecture 6: Designing AR Systems
 
Research Directions in Transitional Interfaces
Research Directions in Transitional InterfacesResearch Directions in Transitional Interfaces
Research Directions in Transitional Interfaces
 
Comp4010 Lecture10 VR Interface Design
Comp4010 Lecture10 VR Interface DesignComp4010 Lecture10 VR Interface Design
Comp4010 Lecture10 VR Interface Design
 
Talk to Me: Using Virtual Avatars to Improve Remote Collaboration
Talk to Me: Using Virtual Avatars to Improve Remote CollaborationTalk to Me: Using Virtual Avatars to Improve Remote Collaboration
Talk to Me: Using Virtual Avatars to Improve Remote Collaboration
 
Comp4010 Lecture8 Introduction to VR
Comp4010 Lecture8 Introduction to VRComp4010 Lecture8 Introduction to VR
Comp4010 Lecture8 Introduction to VR
 
Comp4010 lecture3-AR Technology
Comp4010 lecture3-AR TechnologyComp4010 lecture3-AR Technology
Comp4010 lecture3-AR Technology
 
Comp 4010 2021 Lecture1-Introduction to XR
Comp 4010 2021 Lecture1-Introduction to XRComp 4010 2021 Lecture1-Introduction to XR
Comp 4010 2021 Lecture1-Introduction to XR
 
2022 COMP4010 Lecture3: AR Technology
2022 COMP4010 Lecture3: AR Technology2022 COMP4010 Lecture3: AR Technology
2022 COMP4010 Lecture3: AR Technology
 
Comp4010 Lecture9 VR Input and Systems
Comp4010 Lecture9 VR Input and SystemsComp4010 Lecture9 VR Input and Systems
Comp4010 Lecture9 VR Input and Systems
 
Advanced Methods for User Evaluation in AR/VR Studies
Advanced Methods for User Evaluation in AR/VR StudiesAdvanced Methods for User Evaluation in AR/VR Studies
Advanced Methods for User Evaluation in AR/VR Studies
 
2022 COMP 4010 Lecture 7: Introduction to VR
2022 COMP 4010 Lecture 7: Introduction to VR2022 COMP 4010 Lecture 7: Introduction to VR
2022 COMP 4010 Lecture 7: Introduction to VR
 
Grand Challenges for Mixed Reality
Grand Challenges for Mixed Reality Grand Challenges for Mixed Reality
Grand Challenges for Mixed Reality
 
Novel Interfaces for AR Systems
Novel Interfaces for AR SystemsNovel Interfaces for AR Systems
Novel Interfaces for AR Systems
 
ISS2022 Keynote
ISS2022 KeynoteISS2022 Keynote
ISS2022 Keynote
 
Lecture 2 Presence and Perception
Lecture 2 Presence and PerceptionLecture 2 Presence and Perception
Lecture 2 Presence and Perception
 
Comp4010 lecture11 VR Applications
Comp4010 lecture11 VR ApplicationsComp4010 lecture11 VR Applications
Comp4010 lecture11 VR Applications
 
Empathic Computing: Developing for the Whole Metaverse
Empathic Computing: Developing for the Whole MetaverseEmpathic Computing: Developing for the Whole Metaverse
Empathic Computing: Developing for the Whole Metaverse
 
Comp 4010 2021 Snap Tutorial 2
Comp 4010 2021 Snap Tutorial 2Comp 4010 2021 Snap Tutorial 2
Comp 4010 2021 Snap Tutorial 2
 

Similaire à 2022 COMP4010 Lecture1: Introduction to XR

Similaire à 2022 COMP4010 Lecture1: Introduction to XR (20)

COMP 4010 - Lecture1 Introduction to Virtual Reality
COMP 4010 - Lecture1 Introduction to Virtual RealityCOMP 4010 - Lecture1 Introduction to Virtual Reality
COMP 4010 - Lecture1 Introduction to Virtual Reality
 
COMP Lecture1 - Introduction to Virtual Reality
COMP Lecture1 - Introduction to Virtual RealityCOMP Lecture1 - Introduction to Virtual Reality
COMP Lecture1 - Introduction to Virtual Reality
 
Lecture1 introduction to VR
Lecture1 introduction to VRLecture1 introduction to VR
Lecture1 introduction to VR
 
COMP 4010 - Lecture 1: Introduction to Virtual Reality
COMP 4010 - Lecture 1: Introduction to Virtual RealityCOMP 4010 - Lecture 1: Introduction to Virtual Reality
COMP 4010 - Lecture 1: Introduction to Virtual Reality
 
Virtual Reality: Sensing the Possibilities
Virtual Reality: Sensing the PossibilitiesVirtual Reality: Sensing the Possibilities
Virtual Reality: Sensing the Possibilities
 
Introduction to Augmented Reality
Introduction to Augmented RealityIntroduction to Augmented Reality
Introduction to Augmented Reality
 
2016 AR Summer School - Lecture1
2016 AR Summer School - Lecture12016 AR Summer School - Lecture1
2016 AR Summer School - Lecture1
 
Beyond Reality (2027): The Future of Virtual and Augmented Reality
Beyond Reality (2027): The Future of Virtual and Augmented RealityBeyond Reality (2027): The Future of Virtual and Augmented Reality
Beyond Reality (2027): The Future of Virtual and Augmented Reality
 
Fifty Shades of Augmented Reality: Creating Connection Using AR
Fifty Shades of Augmented Reality: Creating Connection Using ARFifty Shades of Augmented Reality: Creating Connection Using AR
Fifty Shades of Augmented Reality: Creating Connection Using AR
 
Virtual reality
Virtual realityVirtual reality
Virtual reality
 
Mixed Reality in the Workspace
Mixed Reality in the WorkspaceMixed Reality in the Workspace
Mixed Reality in the Workspace
 
May the Force be with You
May the Force be with YouMay the Force be with You
May the Force be with You
 
Virtual Reality
Virtual RealityVirtual Reality
Virtual Reality
 
Building AR and VR Experiences
Building AR and VR ExperiencesBuilding AR and VR Experiences
Building AR and VR Experiences
 
Virtual reality
Virtual realityVirtual reality
Virtual reality
 
Lecture#01.pptx
Lecture#01.pptxLecture#01.pptx
Lecture#01.pptx
 
Latest trends in information technology
Latest trends in information technologyLatest trends in information technology
Latest trends in information technology
 
VR
VR VR
VR
 
모바일 AR 개요 및 Scan Search 사례 2부
모바일 AR 개요 및 Scan Search 사례 2부모바일 AR 개요 및 Scan Search 사례 2부
모바일 AR 개요 및 Scan Search 사례 2부
 
Virtual Reality
Virtual RealityVirtual Reality
Virtual Reality
 

Plus de Mark Billinghurst

Plus de Mark Billinghurst (11)

The Metaverse: Are We There Yet?
The  Metaverse:    Are   We  There  Yet?The  Metaverse:    Are   We  There  Yet?
The Metaverse: Are We There Yet?
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
IVE Industry Focused Event - Defence Sector 2024
IVE Industry Focused Event - Defence Sector 2024IVE Industry Focused Event - Defence Sector 2024
IVE Industry Focused Event - Defence Sector 2024
 
Future Research Directions for Augmented Reality
Future Research Directions for Augmented RealityFuture Research Directions for Augmented Reality
Future Research Directions for Augmented Reality
 
Evaluation Methods for Social XR Experiences
Evaluation Methods for Social XR ExperiencesEvaluation Methods for Social XR Experiences
Evaluation Methods for Social XR Experiences
 
Empathic Computing: Delivering the Potential of the Metaverse
Empathic Computing: Delivering  the Potential of the MetaverseEmpathic Computing: Delivering  the Potential of the Metaverse
Empathic Computing: Delivering the Potential of the Metaverse
 
Empathic Computing: Capturing the Potential of the Metaverse
Empathic Computing: Capturing the Potential of the MetaverseEmpathic Computing: Capturing the Potential of the Metaverse
Empathic Computing: Capturing the Potential of the Metaverse
 
Empathic Computing: Designing for the Broader Metaverse
Empathic Computing: Designing for the Broader MetaverseEmpathic Computing: Designing for the Broader Metaverse
Empathic Computing: Designing for the Broader Metaverse
 
Empathic Computing and Collaborative Immersive Analytics
Empathic Computing and Collaborative Immersive AnalyticsEmpathic Computing and Collaborative Immersive Analytics
Empathic Computing and Collaborative Immersive Analytics
 
Metaverse Learning
Metaverse LearningMetaverse Learning
Metaverse Learning
 
Advanced Methods for User Evaluation in Enterprise AR
Advanced Methods for User Evaluation in Enterprise ARAdvanced Methods for User Evaluation in Enterprise AR
Advanced Methods for User Evaluation in Enterprise AR
 

Dernier

Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
panagenda
 
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdfBreaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
UK Journal
 

Dernier (20)

Overview of Hyperledger Foundation
Overview of Hyperledger FoundationOverview of Hyperledger Foundation
Overview of Hyperledger Foundation
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
 
Working together SRE & Platform Engineering
Working together SRE & Platform EngineeringWorking together SRE & Platform Engineering
Working together SRE & Platform Engineering
 
State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!
 
Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024
 
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
 
Using IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & IrelandUsing IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & Ireland
 
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsContinuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
 
AI mind or machine power point presentation
AI mind or machine power point presentationAI mind or machine power point presentation
AI mind or machine power point presentation
 
Oauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftOauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoft
 
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdfBreaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
 
How we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdfHow we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdf
 
What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024
 
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
 
Easier, Faster, and More Powerful – Notes Document Properties Reimagined
Easier, Faster, and More Powerful – Notes Document Properties ReimaginedEasier, Faster, and More Powerful – Notes Document Properties Reimagined
Easier, Faster, and More Powerful – Notes Document Properties Reimagined
 
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfThe Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
 
TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024
 
ECS 2024 Teams Premium - Pretty Secure
ECS 2024   Teams Premium - Pretty SecureECS 2024   Teams Premium - Pretty Secure
ECS 2024 Teams Premium - Pretty Secure
 
Intro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджераIntro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджера
 
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdfIntroduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
 

2022 COMP4010 Lecture1: Introduction to XR

  • 1. INTRODUCTION TO XR COMP 4010 Lecture One Mark Billinghurst July 28th 2022 mark.billinghurst@unisa.edu.au
  • 2. Who am I .. • Mark Billinghurst • Director of the Empathic Computing Lab • Univ. South Australia, Univ. of Auckland • Conducting research in: • Collaborative AR, AR/VR Interface Design, HCI • Empathic Computing • Previous worked at: • Google, Amazon, Nokia, British Telecom • MIT, Univ. of Washington, NAIST, Univ. of Canterbury • PhD from University of Washington (2002)
  • 3. UniSA IVE - https://unisa.edu.au/research/IVE/
  • 4. Class Logistics • Weekly lecture (2 hrs) • Thursday 10am – 12am • Room: Online • Weekly Practical (1 hr) • Thursday 9am – 10am • Assessment • 3 projects @ 20%, 30%, 40% • Class participation @ 10% • What you will need • iOS or Android phone/tablet • Access to laptop/PC for development
  • 5. Equipment Available • VR Laboratory • 6 VR capable PCs • High end graphics cards • 3 Oculus Rift HMDs • Wide Field of View display • 1 HTC Vive HMD • Room scale tracking • IVE Equipment • 6 HoloLens2 AR displays • See through AR display • 10 Oculus Quest HMD • Self contained tracking
  • 6. Snap Lens Studio Available from https://lensstudio.snapchat.com/
  • 8. Lecture Schedule – 13 lectures
  • 9. Assessment • Assignment 1: Mobile AR (20%) • Develop a Snap Lens • AR Tracking and Interaction • Assignment 2: Interactive VR (30%) • Create an interactive VR scene • Oculus Quest, HTC Vive, Rift • Assignment 3: HMD Experience (40%) • Develop either VR HMD experience • Oculus Quest • Class Participation (10%) • Watch at least one short YouTube AR/VR video each week • Be prepared to present video each week
  • 11. 1967 – IBM 1401 – half of the computers in the world, $10,000/month to run
  • 13. The Incredible Disappearing Computer 1960-70’s Room 1970-80’s Desk 1980-90’s Lap 1990-2000’s Hand 2010 - Head
  • 14. Graphical User Interfaces • Separation between real and digital worlds • WIMP (Windows, Icons, Menus, Pointer) metaphor
  • 15. Rekimoto, J. and Nagao, K. 1995. The world through the computer: computer augmented interaction with real world environments. Making Interfaces Invisible (c) Internet of Things
  • 16. Internet of Things (IoT).. • Embed computing and sensing in real world • Smart objects, sensors, etc.. (c) Internet of Things
  • 17. Virtual Reality (VR) • Users immersed in Computer Generated environment • HMD, gloves, 3D graphics, body tracking
  • 18. The First VR Experience … https://www.youtube.com/watch?v=pAC5SeNH8jw
  • 19. Virtual Reality Definition •Defining Characteristics • Sense of Immersion • User feels immersed in computer generated space • Interactive in real-time • The virtual content can be interacted with • Independence • User can have independent view and reaction to environment
  • 20. David Zeltzer’s AIP Cube Autonomy – User can to react to events and stimuli. Interaction – User can interact with objects and environment. Presence – User feels immersed through sensory input and output channels Interaction Autonomy Presence VR Zeltzer, D. (1992). Autonomy, interaction, and presence. Presence: Teleoperators & Virtual Environments, 1(1),127-132.
  • 23. Augmented Reality (AR) • Virtual Images blended with the real world • See-through HMD, handheld display, viewpoint tracking, etc..
  • 24. Augmented Reality Definition •Defining Characteristics [Azuma 97] • Combines Real and Virtual Images • Both can be seen at the same time • Interactive in real-time • The virtual content can be interacted with • Registered in 3D • Virtual objects appear fixed in space Azuma, R. T. (1997). A survey of augmented reality. Presence, 6(4), 355-385.
  • 27.
  • 29. • Weak AR • Imprecise tracking • No knowledge of environment • Limited interactivity • Handheld AR • Strong AR • Very accurate tracking • Seamless integration into real world • Natural interaction • Head mounted AR Strong vs. Weak AR
  • 32. From Reality to Virtual Reality Internet of Things Augmented Reality Virtual Reality Real World Virtual World
  • 33. Milgram’s Mixed Reality (MR) Continuum Augmented Reality Virtual Reality Real World Virtual World Mixed Reality "...anywhere between the extrema of the virtuality continuum." P. Milgram and A. F. Kishino, (1994) A Taxonomy of Mixed Reality Visual Displays Internet of Things
  • 34. Milgram’s Reality-Virtuality continuum Mixed Reality Reality - Virtuality (RV) Continuum Real Environment Augmented Reality (AR) Augmented Virtuality (AV) Virtual Environment "...anywhere between the extrema of the virtuality continuum." P. Milgram and A. F. Kishino, Taxonomy of Mixed Reality Visual Displays IEICE Transactions on Information and Systems, E77-D(12), pp. 1321-1329, 1994.
  • 36. Extended Reality (XR) Augmented Reality Virtual Reality Real World Virtual World Mixed Reality Extended Reality Internet of Things
  • 39. Mixed Reality Revisited (2019) • Interviewed experts + read papers • ”.. no universally agreed on, one-size-fits- all definition of MR” • Six notions of MR • Continuum, Synonym, Collaboration • Combination, Alignment, Strong AR Speicher, M., Hall, B. D., & Nebeling, M. (2019, May). What is mixed reality?. In Proceedings of the 2019 CHI conference on human factors in computing systems (pp. 1-15).
  • 41. The Metaverse • Neal Stephenson’s “SnowCrash” (1992) ”.. When you live in a shithole, there's always the Metaverse, and in the Metaverse, Hiro Protagonist is a warrior prince.”
  • 42. Defining the Metaverse • Real-time, 3D, Interactive, Social, Persistent AWE 2022 John Riccitiello CEO, Unity Technologies
  • 43. Metaverse Definition (2007) • Metaverse Roadmap • http://metaverseroadmap.org/ • The Metaverse is the convergence of: • 1) virtually enhanced physical reality • 2) physically persistent virtual space
  • 44. Metaverse Dimensions • Augmentation technologies that layer information onto our perception of the physical environment. • Simulation refers to technologies that model reality • Intimate technologies are focused inwardly, on the identity and actions of the individual or object; • External technologies are focused outwardly, towards the world at large;
  • 45. Metaverse Components • Four Key Components • Virtual Worlds • Augmented Reality • Mirror Worlds • Lifelogging
  • 46. Mirror Worlds • Simulations of external space/content • Capturing and sharing surroundings • Photorealistic content • Digital twins Matterport Deep Mirror Google Street View Soul Machines
  • 47. Lifelogging • Measuring user’s internal state • Capturing physiological cues • Recording everyday life • Augmenting humans Apple Fitbit Shimmer OpenBCI
  • 51. When anything new comes along, everyone, like a child discovering the world thinks that they’ve invented it, but you scratch a little and you find a caveman scratching on a wall is creating virtual reality in a sense. Morton Helig (Hammit 1993)
  • 52. Early History (30,000 BC - ) The history of VR is rooted in human’s first attempts to reproduce the world around them
  • 53. 1800’s – Capturing Reality • Panoramas (1790s) • Immersive paintings • Photography (1820-30s) • Oldest surviving photo (Niépce, 1826) • Stereo imagery (1830s) • Wheatstone (1832) • Brewster (1851) • Movies (1870s) • Muybridge (1878) • Roundhay Garden Scene (1888)
  • 56. 3D Cinema Golden Era (1950-60s) • Polarized 3D projection or anaglyph (red/blue)
  • 57. Pepper’s Ghost (1862) • Projecting onto glass to make ghost image appear on stage • Dates back to Giambattista della Porta (1584)
  • 58. The Master Key (1901) – AR Glass "It consists of this pair of spectacles. While you wear them every one you meet will be marked upon the forehead with a letter indicating his or her character. The good will bear the letter 'G,' the evil the letter 'E.' … Thus you may determine by a single look the true natures of all those you encounter.” L. Frank Baum AR display showing if people are good or evil
  • 59. 1900s – Interactive Experiences • Early Simulators (<1960s) • Flight simulation • Sensorama (1955) • Early HMDs (1960s) • Philco, Ivan Sutherland • Military + University Research (1970-80s) • US Airforce, NASA, MIT, UNC • First Commercial Wave (1980-90s) • VPL, Virtual i-O, Division, Virtuality • VR Arcades, Virtual Boy
  • 60. Link Trainer (1929 – 1950s) • Flight Simulator Training • Full six degree of freedom rotation • Force feedback and motion control • Simulated instruments • Modeling common flight conditions • Over 500,000 pilots trained
  • 61. Link Trainer Video (1966) • https://www.youtube.com/watch?v=MEKkVg9NqGM
  • 63. Early HUD (1958) F16 – Head Up Display Showing flight information over the real world
  • 64. Sensorama (1955) • Created by Morton Heilig • Experience Theater • Multi-sensory • Visuals • Sound • Wind • Vibration • Smell • No financial support • Commercial failure
  • 65. First HMD? (1961) • Philco Headsight – Remote Camera Viewing • Showing live camera view in monocular HMD
  • 66. Ivan Sutherland (1960s) 6 6 Ivan Sutherland’s Head-Mounted Display (1968)
  • 68. Super Cockpit (1965-80’s) • US Airforce Research Program • Wright Patterson Air Force Base • Tom Furness III • Multisensory • Visual, auditory, tactile • Head, eye, speech, and hand input • Addressing pilot information overload • Flight controls and tasks too complicated • Research only • big system, not safe for ejecting
  • 69.
  • 70. LEEP Optics (1979) • Large Expanse, Extra Perspective optics • Developed by Eric Howlett • Lens design for extremely wide field of view • High resolution in centre, lower resolution in periphery • 90o direct FOV, 140o corneal FOV • Used as basis for most VR HMDs
  • 72. The Data Glove (1981-82) • Precursor, Sayre Glove • Univ. of Illinois, 1977 • Thomas Zimmerman (1982) • Fiber optic bend sensors • Detecting finger bending • Commercialized by VPL • Mattel PowerGlove (1989)
  • 73. VPL DataGlove Demo • https://www.youtube.com/watch?v=fs3AhNr5o6o
  • 74. NASA VIEW/VIVED (1981-86) • Early HMD (McGreevy Humphries) • LCD “Watchman” displays • VIEW (Scott Fisher) • Polhemus tracker • LEEP-based HMD • 3D audio (Convolvotron) • DataGlove gesture input • Simple graphics
  • 75. VPL Research (1985 – 1999) • First Commercial VR Company • Jaron Lanier, Jean-Jacques Grimaud • Provided complete systems • Displays, software, gloves, etc • DataGlove, EyePhone, AudioSphere
  • 76. The University of North Carolina at Chapel Hill (1980s-1990s) 7 6 Head-Mounted Displays Tracking, Haptics, Applications
  • 77. University of Washington (1989 - ) • Human Interface Technology Laboratory (HIT Lab) • Founded by Tom Furness III • Many AR/VR Innovations • Virtual Retinal Display • ARToolKit AR Tracking library • GreenSpace shared VR experience • VR and pain care • VR and Education
  • 78. First Industrial Use of AR (1990’s) • 1992: Tom Caudell at Boeing coined the term “AR.” • Wire harness assembly application begun • Lead by Tom Caudell, and David Mizell
  • 79. CAVE (1992) • Projection VR system • 3-6 wall stereo projection, viewpoint tracking • Developed at EVL, University of Illinois Chicago • Commercialized by Mechdyne Corporation(1996) C. Cruz-Neira, D. J. Sandin, T. A. DeFanti, R. V. Kenyon and J. C. Hart. "The CAVE: Audio Visual Experience Automatic Virtual Environment", Communications of the ACM, vol. 35(6), 1992, pp. 64–72.
  • 80. CAVE Demo Video • https://www.youtube.com/watch?v=aKL0urEdtPU
  • 81. Desktop VR - 1995 • Expensive - $150,000+ • 2 million polys/sec • VGA HMD – 30 Hz • Magnetic tracking
  • 82. Mobile/Wearable Systems (1995) • 1995 Navicam (Rekimoto) • Handheld AR • 1997 Touring Machine (Feiner) • Backpack AR, GPS, see-through display • 1998 Tinmith (Thomas, UniSA) • Outdoor gaming, CAD
  • 83. Virtual Reality was HOT! .. In 1995..
  • 84. Rise of Commercial VR Companies • W Industries/Virtuality (1985 - 97) • Location based entertainment • Virtuality VR Arcades • Division (1989 – 1998) • Turn key VR systems • Visual programming tools • Virtual i-O (1993 -1997) • Inexpensive gamer HMDs • Sense8 (1990 - 1998) • WorldToolKit, WorldUp • VR authoring tools
  • 86. Overview of VR in the 1990’s
  • 87. Development of AR Tools • 1996 CyberCode (Rekimoto) • First matrix code tracking • 1999 ARToolKit (Kato & Billinghurst) • Open source tracking library
  • 89.
  • 90. • April 2007 Computer World • VRVoted 7th on list of 21 biggest technology flops • MS Bob #1
  • 91. Lessons Learned • Don’t believe the hype • Not everything is better inVR • Many factors determine technology acceptance • Human Centered Design/Design for users • Need to move from Demo to Production • Profitable niche markets first • Follow the money
  • 92. Mobile Phone AR (2005) • Mobile Phones • camera • processor • display • AR on Mobile Phones • Simple graphics • Optimized computer vision • Collaborative Interaction
  • 93. AR Advertising (HIT Lab NZ 2007) • Txt message to download AR application (200K) • See virtual content popping out of real paper advert • Tested May 2007 by Saatchi and Saatchi
  • 95. Eye of judgement (2007) • Sony Playstation3 game • First AR console game • Over 300,000 copies sold • Used Eye camera + tracking cards
  • 96. 2007 - AR Reaches Mainstream • MIT Technology Review • March 2007 • One of the 10 most exciting technologies • Economist • Dec 6th 2007 • Reality, only better
  • 97. Google Searches for AR • Cross over in 2009, with more interest in AR than VR
  • 98. 2008 - Browser BasedAR • Flash + camera + 3D graphics • ARToolKit ported to Flash • High impact • High marketing value • Large potential install base • 1.6 Billion web users • Ease of development • Lots of developers, mature tools • Low cost of entry • Browser, web camera
  • 99. Demo: GE Smart Grid • https://www.youtube.com/watch?v=vJO_AZkCL9U
  • 100. 2008: Location Aware Phones Nokia Navigator Motorola Droid
  • 101. Outdoor Information Overlay • Mobile phone based • Tag real world locations • GPS + Compass input • Overlay graphics on live video • Applications • Travel guide, Advertising, etc. • Wikitude, Metaio, Layar, etc.. • iOS/Android, Public API released
  • 102. Layar Demo (2008) • https://www.youtube.com/watch?v=b64_16K2e08
  • 103. AR in Magazines (2009- ) • Esquire Magazine • Dec 2009 issue • 12 pages AR content • Many Others • Wired • Colors • Red Bull • etc.
  • 107. VR Second Wave (2010 - ) • Palmer Luckey • HMD hacker • Mixed Reality Lab (MxR) intern • Oculus Rift (2011 - ) • 2012 - $2.4 million kickstarter • 2014 - $2B acquisition FaceBook • $350 USD, 110o FOV
  • 108. The Oculus Kickstarter Video • https://www.youtube.com/watch?v=aNSYscbxFAw
  • 109. Desktop VR in 2016 • Graphics Desktop • $1,500 USD • >4 Billion poly/sec • $600 HMD • 1080x1200, 90Hz • Optical tracking • Room scale
  • 110. Oculus Rift Sony Morpheus HTC/Valve Vive 2016 - Rise of Consumer HMDs
  • 111. HTC Vive • Room scale tracking • Gesture input devices
  • 112. Example Vive App – Tilt Brush • https://www.youtube.com/watch?v=ijukZmYFX-0
  • 113. VR2GO (2013) • MxR Lab • 3D print VR viewer for mobiles • Open source hardware + software • http://projects.ict.usc.edu/mxr/diy/vr2go/ • Early Mobile VR viewer
  • 114. Google Cardboard • Released 2014 (Google 20% project) • >5 million shipped/given away • Easy to use developer tools + =
  • 115. Multiple Mobile VR Viewers Available
  • 116. Epson Moverio BT-300 ▪ Stereo see-through display ($700) ▪ 1280 RGB x 720 pixels, 23 degree FOV, 30Hz, 69g ▪ Android Powered, separate controller ▪ VGA camera, GPS, gyro, accelerometer
  • 118. Social Mobile Camera AR Apps (2015 - ) • SnapChat - Lenses, World Lenses • Cinco de Mayo lens > 225 million views • Facebook - Camera Effects • Google – Word Lens/Translate
  • 119. Hololens (2016) • Integrated system – Windows • Stereo see-through display • Depth sensing tracking • Voice and gesture interaction • Note: Hololens2 coming September 2019
  • 120. ARKit/ARcore (2017) • Visual Inertial Odometry (VIO) systems • Mobile phone pose tracked by • Camera (Visual), Accelerometer & Gyroscope (Intertial) • Features • Plane detection, lighting detection, hardware optimisation • Links • https://developer.apple.com/arkit/ • https://developers.google.com/ar/
  • 122. MagicLeap ML-1 (2018) • Bi-Focal Display – two focus planes • Horizontal FoV of 40o, vertical FoV of 30o, diagonal value of 50o • 1280×960 resolution, Eye-tracking • Separate display and computer • Nvidia "Parker" Tegra X2 CPU, 8GB RAM, 128 GB storeage • 6 DOF handheld controller, magnetic tracking
  • 123. History Summary • 1960’s – 80’s: Early Experimentation • 1980’s – 90’s: Basic Research • Tracking, displays • 1995 – 2005: Tools/Applications • Interaction, usability, theory • 2005 - : Commercial Applications • Mobile, Games, Medical, Industry
  • 125. Resources • Azuma, R. T. (1997). A survey of augmented reality. Presence: teleoperators & virtual environments, 6(4), 355-385. • Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., & MacIntyre, B. (2001). Recent advances in augmented reality. IEEE computer graphics and applications, 21(6), 34-47. • Milgram, P., & Kishino, F. (1994). A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems, 77(12), 1321-1329. • Billinghurst, M., Clark, A., & Lee, G. (2015). A Survey of Augmented Reality. Foundations and Trends in Human–Computer Interaction, 8(2-3), 73-272. • Speicher, M., Hall, B. D., & Nebeling, M. (2019, May). What is mixed reality?. In Proceedings of the 2019 CHI conference on human factors in computing systems (pp. 1-15).