This document discusses designing interactive augmented reality experiences for the web. It covers using smartphone AR capabilities like motion tracking and understanding the environment. It also discusses tools like WebARonARCore and WebARonARKit that allow bringing AR to the web. Prototypes were built to explore use cases, design principles, interfaces, and developer workflows. Code examples demonstrate techniques like loading assets, 3D rendering, model compression, and polyfills to enable AR capabilities on desktop. Guidelines and specifications and resources for getting started with web-based AR are also referenced.

1. Designing Interactive Web Based
AR Experiences
Reza Ali | @rezaali | WebXR | Daydream | Google | FITC Toronto 2018

4. Why smartphone AR?

5. Accurate 6-DOF motion tracking
Compositing the virtual and real
Basic world understanding:
• Feature points & marker detection*
• Horizontal & vertical plane detection*
• Basic illumination determination

7. The future of augmented reality
sounds a lot like the web

8. WebARonARCore WebARonARKit

9. WebARonARCore
github.com/google-ar/WebARonARCore
WebARonARKit
github.com/google-ar/WebARonARKit

10. three.ar.js
github.com/google-ar/
three.ar.js

12. We’ve been building prototypes to help explore:
• Impactful use cases (understanding, vibes)
• design principles (how, why, what, when, where)
• diegetic interfaces (XR UX)
• interaction design (raycast everything)
• rendering techniques (sdf font rendering)
• developer workflows (desktop prototyping)

13. WebAR-Article

17. 🤔
How long did it take?
What went into it?
How was this crafted?
How was this design?
Where is the code?

18. ES6

19. Minimizing the use of new
Scratch matrices, vectors, quaternions
ES6 imports / exports to avoid pollution
Used [raw, image, glslify, json] loaders
Serving & reload via webpack-dev-server
eslint to catch mistakes and garbage

21. Low Poly [Obj & Mtl]
Performant ⚡
Public & Remixable
CC-BY License 🙂
Beautiful & Playful
Baked AO 😋
poly.google.com poly.google.com

22. Converted .obj to .drc
Astronaut From Poly
185k > 9.5k ~20x 🤓
Astronaut From NASA
3.1M > 121k ~26x 😎
https://github.com/
google/draco github.com/google/draco

23. Responsive Layout via Bootstrap
CSS Styles via Bootswatch (Paper Theme)
three.js (shadows, lights, shaders, math)
Model sourced from poly.google.com
Model compressed ~20x by Draco
Desktop AR Prototyping Environment 💡

24. Polyfills AR api
🔥 RAY MARCHES 🔥
virtual scene (SDFs)
GPU Hit Computation
GPU > JS Encode float
Simple 1 line include
Rapidly iterate UX & UI
Desktop AR Prototyping Environment

25. Show off
Searching Reticle
State
Show off
Tracked Reticle
State
Reticle Searching State
Reticle Tracking State

26. Reticle Fade Out
AR Flow

27. Show off AR
Controls
Production
AR Controls (Debug View) AR Controls

28. AR Transition (0.25x Speed)
Entering & Exiting AR
1. Remove model
2. Remove SDF grid
3. Remove HUD
4. Fade out sphere
5. Fade in HUD
6. Enter AR flow

29. Entering & Exiting AR
1. Zero window scroll
2. Pop off canvas
3. Remove all content
4. Add canvas back
5. Add fullscreen CSS
6. Reverse for exiting

30. WebAR-Article
github.com/google-ar/
webar-article

31. Annotations
Animations
Interactions

32. ARControls

33. Poly API

34. Animation Test

35. Performance Test

36. SDF Font Rendering

37. Proximity Based Annotations

38. WebAR-Place

42. Microsite Design

46. Conclusions

47. Design Guidelines
designguidelines.withgo
ogle.com/ar-design

48. WebXR Spec
https://github.com/
immersive-web/webxr

49. Getting Started
developers.google.com/
ar/develop/web/getting-
started

50. Thanks!
@rezaali

51. Off the record

52. Exponential growth

53. Are we there yet?

54. Novel hardware & software

55. Connecting the dots…