Introduction, overview of course, details of all software and hardware used for this course

1. Build Your Own VR Display
An Introduction to VR Display Systems for Hobbyists and Educators
Gordon Wetzstein
Robert Konrad
Nitish Padmanaban
Hayato Ikoma

2. vir·tu·al re·al·i·ty
vərCH(əw)əl rē‘alədē
the computer-generated simulation of a three-dimensional image or
environment that can be interacted with in a seemingly real or
physical way by a person using special electronic equipment, such
as a helmet with a screen inside or gloves fitted with sensors.

3. VPL Research

5. Other Applications of VR
• architecture walkthroughs
• remote operation of vehicles (e.g., drone racing!)
• education
• communication
• virtual travel to expensive or risky environments
• training surgeons, pilots,...
• create empathy
• … VR is a new medium ...

6. a
trip
down
the
rabbit
hole

7. National Academy of Engineering
“Enhance Virtual Reality” is 1 of 14 NAE grand challenges
for engineering in the 21st century
image from NAE

8. Exciting Engineering Aspects of VR/AR
• CPU, GPU
• IPU, DPU?
• sensors & imaging
• computer vision
• scene understanding
• human perception
• displays: visual, auditory,
vestibular, haptic, …• VR cameras
• cloud computing
• shared experiences
• HCI
• compression,
streaming
images from facebook, Microsoft

9. Personal Computer
e.g. Commodore PET 1983
Laptop
e.g. Apple MacBook
Smartphone
e.g. Google Pixel
AR/VR
e.g. Microsoft Hololens
???

10. A Brief History of Virtual Reality
1838 1968 2012-2017
Stereoscopes
Wheatstone, Brewster, …
VR & AR
Ivan Sutherland
VR explosion
Oculus, Sony, HTC, MS, …
Nintendo
Virtual Boy
1995
???

12. Ivan Sutherland’s HMD
• optical see-through AR, including:
• displays (2x 1” CRTs)
• rendering
• head tracking
• interaction
• model generation
I. Sutherland “A head-mounted three-dimensional display”, Fall Joint Computer Conference 1968

13. Nintendo Virtual Boy
• 770,000 units sold, commercial failure – judge for yourself
Game: Red Alarm

14. =
?

15. 1968
1980s
2000s
electronic /
digital
HCI /
haptics
low cost,
high-res,
low-latency!

16. • Samsung 5.7” AMOLED:
1920x1080px, 75Hz
• 2 sets of lenses (for
different prescriptions)
• InvenSense 6-axis IMU
• ARM Cortex-M3 MCU
• …
https://www.ifixit.com/Teardown/Oculus+Rift+Development+Kit+2+Teardown/27613
OculusDK2Teardown
key factors:
low latency & wide fov!

17. Field of View!

18. Simple Magnifier-type VR Displays

19. See-through
AR Displays

20. Google Glass

21. See-through AR Displays - Pepper’s Ghost 1862

22. See-through AR Displays: Waveguides
Microsoft Hololens

23. Gordon Wetzstein Robert Konrad Nitish Padmanaban Hayato Ikoma
Who We Are
Assistant Professor of EE/CS Research Assistants and TAs of Stanford VR Course

24. Some Projects We’ve Worked On
“Layered 3D” SIGGRAPH 2011, “Polarization Fields” SIGGRAPH Asia 2011, “Tensor Displays” SIGGRAPH 2012, “Focus
3D” ToG 2013, “Eyeglasses-free Displays” SIGGRAPH 2014, “Light Field Projection” SIGGRAPH 2015, …

25. Some Projects We’ve Worked On
F. Huang, K. Chen, G. Wetzstein “The Light Field Stereoscope”, SIGGRAPH 2015
R. Konrad, E. Cooper, G. Wetzstein “Focus-tunable and Monovision Near-eye Displays”, SIGCHI 2016

26. Some Projects We’ve Worked On
R. Konrad, N. Padmanaban, K. Molner, E. Cooper, G. Wetzstein “Accommodation-
invariant Computational Near-eye Displays”, SIGGRAPH 2017
 SIGGRAPH 2017 Paper, Wed 9-10:30am, Room 150/151

27. Some Projects We’ve Worked On
N. Padmanaban, R. Konrad, T. Stramer, E. Cooper, G. Wetzstein “Optimizing
VR for All Users Through Gaze-contingent Focus Displays”, PNAS 2017
 SIGGRAPH 2017 Talk, Thu 3:45-5:15pm, Room 152

28. Stanford EE 267

29. About EE 267
• experimental hands-on class at Stanford, taught twice
• lectures + assignments = one big project – build your
own VR HMD
• all hardware provided, but must return at the end
• enrollment limited, because it’s a lab-based class and we
only have limited hardware kits

30. About EE 267
• learning goals:
• fundamental concepts of VR and Computer Graphics
• implement software + hardware of a head mounted display
• prerequisites: programming (C/C++, JavaScript), linear
algebra
• students: 80 undergrad + grad, teams of 2 get a
hardware kit

31. About EE 267
Lecture slides, course notes, assignments, detailed grading
breakdown, open source hardware, … available on
http://stanford.edu/class/ee267/

32. SIGGRAPH Course Overview
• Introduction and Overview, Gordon Wetzstein, 5 min
• Overview of DIY Hardware and Software, Gordon Wetzstein, 10 min
• Graphics Pipeline, Stereo Rendering, Lens Distortion, Robert Konrad, 60 min
• IMUs, Orientation & Position Tracking, Sensor Fusion, Gordon Wetzstein, 60 min
• Spatial Sound, Nitish Padmanaban, 30 min
• Cinematic VR – Build your own VR camera, Robert Konrad, 25 min
• Discussion and Q&A, All presenters, 5 min
• Live Demos throughout the course: Hayato Ikoma
More details & course notes: http://stanford.edu/class/ee267/

33. Build Your Own VR Display
Overview of DIY Hardware and Software
Gordon Wetzstein
Stanford University
stanford.edu/class/ee267/

34. HMD Housing
& Lenses
6” LCD &
HDMI Driver
Board
VRduino
Flex Sensors
Vibration
Motors
IMU &
Teensy
HDMI Cable
2x USB Cable

35. HMD Housing and Lenses
• View-Master VR Starter Kit ($15-
20) or Deluxe VR Viewer ($23)
• implements Google
Cardboard 1.0/2.0
• very durable – protect flimsy
LCDs
• may need to drill additional
holes

36. Display
• Topfoison LCDs:
• 1080p - $90
• 1440p (2K) - $100
• HDMI driver boards included
• no audio jack! doesn’t fit in housing
• super easy to use as external
monitor on desktop or laptop

37. Display
• same LCD you’ll find in many
smartphones
• reasonably robust (only broke a few)
• ideally use OLEDs, but haven’t found
any low-cost ones similar to these
LCDs

38. Strobing the Display Backlight
• for low-persistence applications, can
strobe LED backlight at 500Hz
• custom PCB with Arduino – email us
if you’re interested

39. VRduino
• Arduino-based open source
platform for:
• orientation tracking
• positional tracking
• interfacing with
other IO devices
• custom-design for EE 267
by Keenan Molner
• all HW-related files on
course website

40. VRduino
• Teensy 3.2 microcontroller
(48 MHz, $20) for all
processing & IO
• InvenSense 9250 IMU (9-
DOF, $6) for orientation
tracking
• Triad photodiodes &
precondition circuit ($1) for
position tracking with HTC
Lighthouse

41. VRduino

42. VRduino
IMU
Teensy 3.2

43. VRduino
Lighthouse
Select

44. VRduino
Photodiode 0 Photodiode 1
Photodiode 2
Photodiode 3

45. About EE 267 – VRduino
IMU
Teensy 3.2
4 photodiodes
other
GPIO pins

46. VRduino x=42mm, y=25mm
x
y
x=42mm, y=-25mm
x=-42mm, y=25mm
x=-42mm, y=-25mm

47. VRduino
pins 20,21
pins 22,23
pins 9,10
pins 5,6

48. VRduino
SCL: pin 19
SDA: pin 18

49. VRduino
Pin
Breakout

50. Student-built Input Devices
• data gloves with flex sensors
• different types of controllers with
tactile feedback via vibration motors
• all connected to VRduino GPIO pins
images from Adafruit.com

51. Live Demo