Lecture 8 Introduction to Augmented Reality

1. LECTURE 8: INTRODUCTION TO AUGMENTED REALITY COMP 4010 – Virtual Reality Semester 5 – 2019 Bruce Thomas, Mark Billinghurst University of South Australia September 17th 2019

2. Augmented Reality 1977 – Star Wars

3. Augmented Reality Definition •Defining Characteristics [Azuma 97] • Combines Real and Virtual Images • Both can be seen at the same time • Interactive in real-time • The virtual content can be interacted with • Registered in 3D • Virtual objects appear fixed in space Azuma, R. T. (1997). A survey of augmented reality. Presence, 6(4), 355-385.

4. CNN 2008 Election Demo https://www.youtube.com/watch?v=v7fQ_EsMJMs

5. • Put AR pictures here Augmented Reality Examples

6. https://www.youtube.com/watch?v=aUPMDwypBkA QuiverVision – AR Marketing

7. HISTORY OF AR

9. Pepper’s Ghost (1862) • Projecting onto glass to make ghost image appear on stage • Dates back to Giambattista della Porta (1584)

10. The Master Key (1901) – AR Glass • "It consists of this pair of spectacles. While you wear them every one you meet will be marked upon the forehead with a letter indicating his or her character. The good will bear the letter 'G,' the evil the letter 'E.' … Thus you may determine by a single look the true natures of all those you encounter.” L. Frank Baum AR display showing if people are good or evil

11. Early HUD (1958) F16 – Head Up Display Showing flight information over the real world

12. First HMD (Maybe?) • Philco Headsight (1961) – Remote Camera Viewing • Showing live camera view in monocular HMD

13. Sutherland HMD • 1968: Sutherland / Sproull’s first HMD system • see-through, with ultrasonic/mechanical tracking

14. US Airforce HMDs 1960 - 70’s: US Air Force HMDs (T. Furness III)

15. SuperCockpit Program 1970 - 80’s: US Air Force Super Cockpit (T. Furness III)

16. Modern Airforce HMDs • Honeywell Integrated Helmet and Display Sighting System (IHADSS) on AH-64 Apache attack helicopter in 1985

17. F35- HMD Demo (2014) • https://www.youtube.com/watch?v=Ay6g66FbkmQ

18. First Industrial Use of AR • Early 1990’s: Boeing coined the term “AR.” • Wire harness assembly application begun • Lead by Tom Caudell, and David Mizell

19. Academic Research Beginning • University research begun at UNC and others • 1994: Motion stabilized display [Azuma] • 1995: Fiducial tracking in video [Bajura / Neumann] • 1996: UNC hybrid magnetic-vision tracker

20. Development of the Field • 1996: MIT Wearable Computing efforts • 1998: Dedicated conferences begin (ISMAR) • Late 90’s: Collaboration, outdoor, interaction • Late 90’s: Augmented sports broadcasts

21. Development of Tools •1996 CyberCode (Rekimoto) • First matrix code tracking •1999 ARToolKit (Kato & Billinghurst) • Open source tracking library

22. Tracking Demos https://www.youtube.com/watch?v=TqGAqAFlGg0 ARToolKit Cybercode https://www.youtube.com/watch?v=cNFwEMkLK4A

23. Mobile/Wearable Systems • 1995 Navicam (Rekimoto) • Handheld AR • 1997 Touring Machine (Feiner) • Backpack AR, GPS, see-through display • 1998 Tinmith (Thomas, UniSA) • Outdoor gaming, CAD

24. History Summary •1960’s – 80’s: Early Experimentation •1980’s – 90’s: Basic Research • Tracking, displays •1995 – 2005: Tools/Applications • Interaction, usability, theory •2005 - : Commercial Applications • Games, Medical, Industry

25. 2007 - AR Reaches Mainstream • MIT Technology Review • March 2007 • One of the 10 most exciting technologies • Economist • Dec 6th 2007 • Reality, only better

26. 2009 - AR in Magazines • Esquire Magazine • Dec 2009 issue • 12 pages AR content • Many Others • Wired • Colors • Red Bull • Etc

27. Esquire Demo https://www.youtube.com/watch?v=LGwHQwgBzSI&t=1s

28. Google Searches for AR • Cross over in 2009, with more interest in AR than VR

29. 2008 - Browser Based AR • Flash + camera + 3D graphics • ARToolKit ported to Flash • High impact • High marketing value • Large potential install base • 1.6 Billion web users • Ease of development • Lots of developers, mature tools • Low cost of entry • Browser, web camera

30. Demo: GE Smart Grid • https://www.youtube.com/watch?v=vJO_AZkCL9U

31. 2005 - Mobile Phone AR •Mobile Phones • camera • processor • display •AR on Mobile Phones • Simple graphics • Optimized computer vision • Collaborative Interaction

32. AR Advertising (HIT Lab NZ 2007) • Txt message to download AR application (200K) • See virtual content popping out of real paper advert • Tested May 2007 by Saatchi and Saatchi

33. Wellington Zoo Demo https://www.youtube.com/watch?v=edTjuXcce_c

34. 2008: Location Aware Phones Nokia NavigatorMotorola Droid

35. 2009 - Outdoor Information Overlay • Mobile phone based • Tag real world locations • GPS + Compass input • Overlay graphics on live video • Applications • Travel guide, Advertising, etc. • Wikitude, Layar, etc.. • iOS/Android, Public API released

36. Layar Demo (2007) • https://www.youtube.com/watch?v=b64_16K2e08

37. Google Glass (2011 - )

38. Google Glass Demo https://www.youtube.com/watch?v=fNATuCkRWFE

39. Epson Moverio BT-300 ▪ Stereo see-through display ($700) ▪ 1280 RGB x 720 pixels, 23 degree FOV, 30Hz, 69g ▪ Android Powered, separate controller ▪ VGA camera, GPS, gyro, accelerometer

40. Smart Glasses Available

41. Hololens (2016) - $3,000 USD • Integrated system – Windows • Stereo see-through display • Depth sensing tracking • Voice and gesture interaction • Note: Hololens2 coming September 2019

42. View Through Hololens https://www.youtube.com/watch?v=2c9eiVyO6gM

43. Sensors • 4 Environment camera sensors (two on each side) • A depth camera (Time of Flight, 120o FOV), a 2.4MP photo/HD video camera • An ambient light sensor, 3 DOF IMU

44. Computing • Custom mother board: 64 GB eMMC SSD and 8MB SRAM, 1GB of LPDDR3 RAM, Windows 10 • HPU: sensor processing and fusion, 8MB SRAM, 1GB of LPDDR3 RAM

45. Display • See-through display, 30°×17.5°field of view - 720p or 1268x720 per eye • two HD 16:9 light engines - liquid crystal on silicon displays • total internal reflection waveguide optics

46. MagicLeap ML-1 (2018) - $2,300 USD • Bi-Focal Display – two focus planes • Horizontal FoV of 40o, vertical FoV of 30o, diagonal value of 50o • 1280×960 resolution, Eye-tracking • Separate display and computer • Nvidia "Parker" Tegra X2 CPU, 8GB RAM, 128 GB storeage • 6 DOF handheld controller, magnetic tracking

47. https://www.youtube.com/watch?v=K5246156rcQ

48. Mobile Camera AR Apps (2015 - ) • SnapChat - Lenses, World Lenses • Cinco de Mayo lens > 225 million views • Facebook - Camera Effects • Google – Word Lens/Translate

49. ARKit/ARcore (2017) • Visual Inertial Odometry (VIO) systems • Mobile phone pose tracked by • Camera (Visual), Accelerometer & Gyroscope (Intertial) • Features • Plane detection, lighting detection, hardware optimisation • Links • https://developer.apple.com/arkit/ https://developers.google.com/ar/

50. ARKIT3 Features • The latest ARKIT features • People Occlusion • Motion Capture • Multiple Face Tracking • Support for Collaborative Sessions • Access to Simultaneous Front and Back Camera

51. • https://www.youtube.com/watch?v=SgAnnwl2VB8 ARKIT3 Demo

52. • Weak AR • Imprecise tracking • No knowledge of environment • Limited interactivity • Handheld AR • Strong AR • Very accurate tracking • Seamless integration into real world • Natural interaction • Head mounted AR Strong vs. Weak AR

53. Augmented Reality Today • Key Technologies Available • Robust tracking (Computer Vision, GPS/sensors) • Display (Handheld, HMDs) • Input Devices (Kinect, etc) • Developer tools (Vuforia, ARKIT, ARCore) • Commercial Business Growing • Gaming, GPS/Mobile, Online Advertisement • >$3Billion USD in 2017

54. AR Business Today • Marketing • Web-based, mobile • Gaming • Mobile, Physical input • Mobile AR • Geo-located information and service • Driving demand for high end phones • Upcoming areas • Manufacturing, Medical, Military

55. AR Commercial Landscape

57. • zxZD

58. Crossing the Chasm - 5-10 years

59. Gartner Hype Cycle 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 Augmented Reality

60. 2019 Hype Cycle • No AR • AR mature

61. Forecast Number of Users

62. Example: Pokemon GO Killer Combo: brand + social + mobile + geo-location + AR

63. Pokemon GO Effect • Fastest App to reach $500 million in Revenue • Only 63 days after launch, > $1 Billion in 6 months • Over 500 million downloads, > 25 million DAU • Nintendo stock price up by 50% (gain of $9 Billion USD)

64. Summary • Augmented Reality has a long history going back to the 1960’s • Interest in AR has exploded over the last few years and is being commercialized quickly • AR is growing in a number of areas • Mobile AR • Web based AR • Marketing experiences

65. SAMPLE AR APPLICATIONS

66. • Web based AR • Flash, HTML 5 based AR • Marketing, education • Outdoor Mobile AR • GPS, compass tracking • Viewing Points of Interest in real world • Eg: Layar, Wikitude • Handheld AR • Vision based tracking • Marketing, gaming • Location Based Experiences • HMD, fixed screens • Museums, point of sale, advertising Typical AR Experiences

67. AR Books – Markerless Tracking

68. Example: Haunted Book/AR Book • Camera hidden in lamp object • AR content seamlessly integrated into real book • Natural page turning/manipulation interaction Scherrer, C., Pilet, J., Fua, P., & Lepetit, V. (2008, September). The haunted book. In Proceedings of the 7th IEEE/ACM international Symposium on Mixed and Augmented Reality (pp. 163-164). IEEE Computer Society.

69. AR Book Demo https://www.youtube.com/watch?v=XbsfamN78Tc

70. Gaming: Rock-em Sock-em • Shared AR Demo • Markerless tracking

71. Rockem Sockem Demo https://www.youtube.com/watch?v=hXtq1qBMLIw

72. Sales and Marketing • Connect with brands and branded objects • Location Based Experiences • Lynx Angels • Web based • Rayban glasses • Mobile • Ford Ka campaign • Print based • Red Bull Magazine

73. Pepsi AR Experience (2014) • Video see-through AR in bus shelter • Bus shelter appears under attack

74. Pepsi Demo https://www.youtube.com/watch?v=Go9rf9GmYpM&t=7s

76. Maintenance Systems • Ngrain • http://www.ngrain.com/ • Training authoring tool • Model based AR tracking • ScopeAR • http://www.scopear.com/ • Remote assistance • Image based tracking

77. Ngrain Example https://www.youtube.com/watch?v=xthEaobGzs4

78. Medical AR Trials ú Sauer et al. 2000 at Siemens Corporate Research, NJ ú Stereo video see through F. Sauer, Ali Khamene, S. Vogt: An Augmented Reality Navigation System with a Single-Camera Tracker: System Design and Needle Biopsy Phantom Trial, MICCAI 2002

79. Medical Demo https://www.youtube.com/watch?v=kFQulgeYh7w

80. Interactive Museum Experiences § BlackMagic § Virtual America’s Cup § 410,000 people in six months § MagicPlanet § TeManawa science museum § Virtual Astronomy § Collaborative AR experience § AR Volcano § Interactive AR kiosk § Scienceworks museum, Melbourne

81. Digital Binocular Station http://www.DigitalBinocularStation.com/

82. CityViewARApplication • Visualize Christchurch before the earthquakes

83. CityViewARApplication • https://www.youtube.com/watch?v=fdgrXxJx4SE

84. Example: Holoportation • Augmented Reality + 3D capture + high bandwidth • http://research.microsoft.com/en- us/projects/holoportation/

85. HoloPortation Video https://www.youtube.com/watch?v=7d59O6cfaM0

86. UsingAR in theWorkplace • Blending digital/physical worlds.. • Productivity • Marketing • Training • Collaboration

87. Example: Productivity • AR for hands busy tasks • Warehouse picking (DHL, SAP) • Smart glasses show pick location • Simple user interface • Hands-free interaction • Improved performance RW VR

88. https://www.youtube.com/watch?v=ZWsBHISOqjA

89. Improved PerformanceTime

90. https://www.youtube.com/watch?v=qTblKJjTadQ

91. Example: Marketing • Add magic to companies products • Snapchat lenses • Face tracking technology • Simple interaction • Animated content • Significant ROI • $750K for 224 million views • c.f. $5 million for SuperBowl ad RW VR

92. https://www.youtube.com/watch?v=wwmmgAw8ZAE

93. Example: Collaboration • Use MR technology for remote collaboration • See through another’s eyes, virtual avatars • Microsoft Remote Assist • HoloLens see-through display • Remote video sharing • AR drawing/annotation • Document sharing • Group calling

94. https://www.youtube.com/watch?v=NSfzsw4eCXI

95. https://www.youtube.com/watch?v=t-2R1aHgzdA

96. Summary • AR technology can be used to develop a wide range of applications • Promising application areas include • Games • Education • Engineering • Medicine • Museums • Etc..

97. www.empathiccomputing.org @marknb00 mark.billinghurst@unisa.edu.au

Lecture 8 Introduction to Augmented Reality

Lecture 8 Introduction to Augmented Reality

Recommandé

Contenu connexe

Tendances

Tendances(20)

Similaire à Lecture 8 Introduction to Augmented Reality

Similaire à Lecture 8 Introduction to Augmented Reality(20)

Plus de Mark Billinghurst

Plus de Mark Billinghurst(20)

Dernier

Dernier(20)

Lecture 8 Introduction to Augmented Reality