A short overview on VR, developing it in Unity and various tips for interaction and performance. This is presented in ProCodeCG's codeMeetUp() on 15 August 2016

1. DEVELOPING VR IN UNITY
ADITYO PRATOMO - LABTEK INDIE

2. DEVELOPING VR IN UNITY
ABOUT ME
▸ Chief Technology Officer at
Labtek Indie
▸ Chief Academic Officer at
Froyo Framework
▸ Certified Unity Developer
#20167UCD724
▸ Loves to tinker with tech

3. DEVELOPING VR IN UNITY
VR IN LABTEK INDIE
It’s Strange (2015)
Oculus Rift & Leap Motion, developed in Unity
https://irunsiregar.itch.io/its-strange
https://www.youtube.com/watch?v=oTii7SSfyLM

4. DEVELOPING VR IN UNITY
VR IN LABTEK INDIE
Combat League (2016)
Oculus Rift, developed in Unreal Engine
https://www.youtube.com/watch?v=1sHrSOe8JM8

5. DEVELOPING VR IN UNITY
VR IN LABTEK INDIE
99 Names (2015)
Web VR, developed in JS with ThreeJS
99names.adityo.net

6. DEVELOPING VR IN UNITY
VR IN LABTEK INDIE
Mixed Reality Experiment
Google Cardboard devloped in Unity with Vuforia SDK
http://www.adityo.net/?p=1146

7. DEVELOPING VR IN UNITY
WHAT I’LL TALK
▸ VR Tech Overview
▸ Desktop VR vs Mobile VR
▸ VR in Unity
▸ Interaction in VR
▸ Towards Better VR Experience
▸ Future Prediction

8. NOW IS THE TIME TO START MAKING SOMETHING IN VR, EVEN IF
YOU DON'T PLAN ON LAUNCHING IT. IT IS A REALLY GOOD IDEA
TO START EXPERIMENTING AND SEE WHAT YOU CAN LEARN.
EVEN IF YOU'RE NOT GOING TO LAUNCH SOMETHING, IT IS GOOD
TO HAVE EXPERIENCE IN VIRTUAL REALITY SO THAT IF IT MAKES
SENSE FOR YOU TO GET INTO IT, YOU'RE ABLE TO
Palmer Luckey (Co-founder of Oculus)
DEVELOPING VR IN UNITY

9. DEVELOPING VR IN UNITY
VR TECH OVERVIEW
▸ VR is a tech that creates new perception of reality in the
mind of its users
▸ Achieved through the use of HMD (Head Mounted
Display) that tricks its user into thinking he/she is in a
different place
▸ Closely related to AR, but not similar nor has hierarchical
relation

10. DEVELOPING VR IN UNITY
VR TECH OVERVIEW

11. DEVELOPING VR IN UNITY
VR TECH OVERVIEW
1. Lens
‣ Emulating depth via focal point adjustment
2. Display
‣ Stereo rendering
‣ FOV emulates real life view
3. Tracking Technology
‣ Tracking head movement
‣ Uses gyro and accelerometer in smartphone
4. 3D Audio
‣ Immersive 3D audio

12. DEVELOPING VR IN UNITY
CURRENT VR PLATFORMS
PC
ANDROID
CONSOLE
OCULUS RIFT HTC VIVE
GOOGLE VR
(CARDBOARD & DAYDREAM)
SAMSUNG
GEAR VR
PS VR
OSVR

13. DEVELOPING VR IN UNITY
DESKTOP VR VS MOBILE VR: FROM THE TECH SIDE
▸ Desktop VR requires more horse power, but has potential
for many kind of interaction and contents
▸ Mobile VR experience is hampered by the device’s spec
▸ Desktop VR can do positional tracking
▸ Mobile VR can only do head positional tracking
▸ Batteries vs Cable
▸ Both do VR content rendering in roughly the same way

14. DEVELOPING VR IN UNITY
DESKTOP VR VS MOBILE VR: FROM THE BUSINESS SIDE
▸ Mobile VR has lower entry barrier, mid-high end Android
phone is sufficient
▸ Desktop VR requires high end gaming rig
▸ Mobile VR has way larger potential market than Desktop
▸ Desktop VR will interests hardcore gamer
▸ Mobile VR will interests budget-oriented gadget
enthusiasts

15. DEVELOPING VR IN UNITY
DESKTOP VR VS MOBILE VR
▸ Mobile VR will help promoting VR to the masses
▸ Desktop VR will help pushing the boundaries, tech wise
▸ Both is ready for mass development, NOW!

16. DEVELOPING VR IN UNITY
WHY DEVELOP VR NOW?
▸ Relatively new tech
▸ Still no best practice sets in stone
▸ Experiments are highly appreciated
▸ Big or small developers, everybody are still finding its feet
▸ YOU can make the difference

17. DEVELOPING VR IN UNITY
VR IN UNITY PC
OCULUS RIFT HTC VIVE OSVR
OCULUS RIFT DK2 / CV1
WIN 7+
UNITY 5.X

18. DEVELOPING VR IN UNITY
VR IN UNITY PC
OCULUS RIFT HTC VIVE OSVR
HTC VIVE
WIN 7+
UNITY 5.X

19. DEVELOPING VR IN UNITY
VR IN UNITY PC
OCULUS RIFT HTC VIVE OSVR
OSVR HMD
WIN 7+
UNITY 5.X

20. DEVELOPING VR IN UNITY
VR IN UNITY ANDROID
GOOGLE VR
(CARDBOARD & DAYDREAM)
SAMSUNG
GEAR VR
ANDROID VR HMD
ANDROID 5.1 +
UNITY 5.X
GOOGLE CARDBOARD SDK
NEXUS 6 (DAYDREAM)

21. DEVELOPING VR IN UNITY
VR IN UNITY ANDROID
GOOGLE VR
(CARDBOARD & DAYDREAM)
SAMSUNG
GEAR VR
ANDROID VR HMD
ANDROID 5.1 +
UNITY 5.X
VUFORIA SDK

22. DEVELOPING VR IN UNITY
VR IN UNITY ANDROID
GOOGLE VR
(CARDBOARD & DAYDREAM)
SAMSUNG
GEAR VR
ANDROID 5.1 +
UNITY 5.X
SAMSUNG GALAXY S7
SAMSUNG GEAR VR

23. DEVELOPING VR IN UNITY
VR IN UNITY: GENERAL ARCHITECTURE
▸ Default camera is replaced
by vendor-specific VR camera
(usually comes as prefab)
▸ VR Camera will render the
game scene to be VR-ready
▸ Other game objects are used
as usual
GAME SCENE
VR CAMERA
OTHER GAME OBJECTS

24. DEVELOPING VR IN UNITY
VR IN UNITY: TIPS FOR THE FIRST TIMER
▸ Most of the time, you’ll wrestle with setting up specific
environment
▸ Oculus: Get the latest SDK and fulfil the requirements,
use Rift DK2 at least, don’t use laptop
▸ Android: setup Android SDK properly, Google and
Vuforia SDKs are generally easy to set up

25. DEVELOPING VR IN UNITY
VR IN UNITY
▸ After replacing the Main Camera object with the vendor-
specific, AR Camera prefab, you’re done.
▸ EASY!
▸ Demo…

26. WHAT ELSE TO
DO?
if getting VR to up and run is easy, then

27. LET’S TALK ABOUT
INTERACTION

28. DEVELOPING VR IN UNITY
INTERACTION IN VR
▸ With headset strapped on, it’s hard to look around
▸ Mouse + keyboard, won’t be the best way to interact

29. DEVELOPING VR IN UNITY
INTERACTION IN VR
LOOK
AROUND
GAMEPAD
INTERACTION
GAZE
INTERACTION
NATURAL INTERACTION (AVAILABLE NOW)
OCULUS
TOUCH
VIVE
CONTROLLER
GOOGLE
DAYDREAM
LEAP
MOTION
EXPERIMENTAL INTERACTION (DIY)
VOICE
CONTROLLER
CUSTOM ELECTRONIC
(ARDUINO / RPI / ETC)
?
easy to develop,
minimal interaction
harder to develop,
rich interaction

30. DEVELOPING VR IN UNITY
INTERACTION IN VR: LOOK AROUND
▸ The most basic thing you can do inside VR
▸ Replacing traditional FPS camera, controlled by mouse,
with your head
▸ Results in a natural, immersive experience
▸ Used in many VR videos
▸ Minimum interaction, but can still produces excellent
experience

31. DEVELOPING VR IN UNITY
360 PHOTOS / VIDEOS IN UNITY
▸ Generally speaking, here are the steps:
▸ Obtain 360 photo/video
▸ Make a skybox using sphere/cube
▸ Use the photo/video as the texture for the skybox

32. DEVELOPING VR IN UNITY
360 PHOTOS / VIDEOS IN UNITY

33. DEVELOPING VR IN UNITY
360 PHOTOS / VIDEOS IN UNITY
▸ Plugin:
▸ Easy Movie Texture (https://www.assetstore.unity3d.com/en/#!/
content/10032)
▸ Discussions:
▸ http://bernieroehl.com/360stereoinunity/
▸ http://forum.unity3d.com/threads/projecting-a-360-degree-vr-
video-on-a-sphere.376522/
▸ http://talesfromtherift.com/use-a-360-degree-panorama-as-a-
skybox/

34. DEVELOPING VR IN UNITY
GAZE INTERACTION
▸ Using eye as an input
▸ Activating an object by looking at it for a certain period
▸ (Optional) use Gaze Reticle to help user find it’s view’s
crosshair

35. DEVELOPING VR IN UNITY
GAZE INTERACTION

36. DEVELOPING VR IN UNITY
GAZE INTERACTION
▸ Google Cardboard SDK:
▸ Use GvrGaze script
▸ Drag the script from Assets/GoogleVR/Scripts/UI/
GvrGaze onto to the camera
▸ Implement IGvrGazePointer and
IGvrGazeResponder to the interactive object
▸ Vuforia SDK uses Google’s Gaze

37. DEVELOPING VR IN UNITY
NATURAL INTERACTION WITH LEAP MOTION
▸ Leap Motion offers the
ability to track hand
gestures
▸ It can be sticked in front of
Oculus/Vive to enable user
to see his/her hands
▸ Thus, enables a natural
interaction using hands,
while inside VR

38. DEVELOPING VR IN UNITY
NATURAL INTERACTION WITH LEAP MOTION

39. DEVELOPING VR WITH UNITY
NATURAL INTERACTION WITH LEAP MOTION
▸ Download and Import the Leap Motion Core Assets
Unity Package
▸ Use the LeapOVRPlayerController Prefab from the
Leap Motion + OVR directory as the main VR Camera
▸ The hand will be added to the screen and acts as a 3D
objects, just like the others (with colliders too!)

40. DEVELOPING VR IN UNITY
EXPERIMENTAL INTERACTION: VOICE CONTROL
▸ Use voice to give input to the VR world
▸ Closely tied to the deployment platform
▸ iOS, Android, Windows, Mac, has its own voice
recognition system
▸ Unity 5.4 can natively do voice recognition in Windows
(https://developer.microsoft.com/en-us/windows/
holographic/voice_input_in_unity)

41. DEVELOPING VR IN UNITY
EXPERIMENTAL INTERACTION: VOICE CONTROL
▸ Unity-only solution:
▸ Ispikit: https://github.com/ispikit/ispikit-unity
▸ Word Detection: https://www.assetstore.unity3d.com/en/#!/
content/4518
▸ Unity + other:
▸ Read Serial in Unity
▸ Do voice recognition in other using Python/NodeJS/etc and
send serial from there

42. DEVELOPING VR IN UNITY
EXPERIMENTAL INTERACTION: CUSTOM ELECTRONIC
▸ If you have something different, why not make it yourself
▸ Use Arduino / Raspberry Pi / etc. to read from certain
sensor and send its data to Unity
▸ Vital to create haptic feedback and experimenting with
different kinds of interaction
▸ Make one of a kind immersive experience

43. DEVELOPING VR IN UNITY
EXPERIMENTAL INTERACTION: CUSTOM ELECTRONIC

44. DEVELOPING VR IN UNITY
EXPERIMENTAL INTERACTION: CUSTOM ELECTRONIC
▸ Use Unity to receive Serial data
▸ Android & Microcontroller Bluetooth: https://
www.assetstore.unity3d.com/en/#!/content/16467
▸ SimpleSerialPort: https://www.assetstore.unity3d.com/en/#!/
content/53594
▸ Attach sensors to Arduino/Raspberry Pi/other
▸ Send data via Serial/Bluetooth
▸ Watch out for latency!

45. DEVELOPING VR IN UNITY
TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS
▸ Simulation sickness is VR’s biggest enemy
▸ Cause: brain is tricked to feel movement, while static
▸ When that trick is broken, nausea attacks, users opt out
from the experience (and probably won’t come back)

46. DEVELOPING VR IN UNITY
TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS
▸ Technical causes:
▸ Latency (maximum should be 20ms)
▸ Many research has gone into how to solve this issue
▸ Many solutions, but none has quite hits the mark

47. DEVELOPING VR IN UNITY
TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS
▸ Hardware design solution:
▸ More comfortable lens
▸ More comfortable HMD when strapped on

48. DEVELOPING VR IN UNITY
TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS
▸ Content design:
▸ Adding nose and feet
▸ Shows a fixed reference point in a frame

49. DEVELOPING VR IN UNITY
TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS
▸ Game design - movement:
▸ In-game movement have more latency potential than
standing still
▸ objects are continuously and dynamically rendered in real-
time
▸ more processing power
▸ To move around, use teleport
▸ Teleport can be done with gaze interaction, no controller is
required

50. DEVELOPING VR IN UNITY
TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS
Keep Talking and Nobody Explodes
www.keeptalkinggame.com

51. DEVELOPING VR IN UNITY
TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS
▸ Game design - movement:
▸ To move around:
▸ use teleport
▸ use pre-determined route
▸ Teleport can be done with gaze interaction, no controller is
required
▸ Pre-determined route will ease us in optimizing assets
▸ Don’t add unnecessary vertical movement

52. DEVELOPING VR IN UNITY
TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS
Teleport:
https://developer.oculus.com/blog/introducing-the-
oculus-sample-framework-for-unity-5/
INMIND VR:
https://developer.oculus.com/blog/introducing-the-
oculus-sample-framework-for-unity-5/

53. DEVELOPING VR IN UNITY
TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS
▸ Game design - camera view:
▸ It’s natural to have first person view, but it’s easier to
result in simulation sickness
▸ To reduce that, use third person view, it will still have
good experience

54. DEVELOPING VR IN UNITY
TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS
Chronos:
www.chronosvr.com
Colosse:
colossevr.tumblr.com

55. DEVELOPING VR IN UNITY
TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS
▸ Tech tips:
▸ Use low poly objects, with high-res textures to
compensate
▸ Toon-shading is okay, photorealistic can probably
causes latency

56. DEVELOPING VR IN UNITY
TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS
Job Simulator:
www.jobsimulatorgame.com
Land’s End:
www.landsendgame.com

57. DEVELOPING VR IN UNITY
TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS
▸ Tech tips:
▸ Use as many static object as possible (uses least RAM)
▸ Use LOD, culling and batching to prevent rendering too
many object at the same time
▸ Use as many baked lighting as possible
▸ https://unity3d.com/learn/tutorials/topics/virtual-reality/
optimisation-vr-unity

58. DEVELOPING VR IN UNITY
TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS

59. DEVELOPING VR IN UNITY
TOWARDS BETTER VR EXPERIENCE
▸ Experience design:
▸ Design experience from the ground up, don’t just adapt
already available 3D game to VR
▸ VR is now best experienced in short but meaningful time
(5-20 minutes)
▸ Design focal point, because the user can look into any
direction

60. DEVELOPING VR IN UNITY
TOWARDS BETTER VR EXPERIENCE
▸ UI design:
▸ Make sure the user can see the UI at any time
▸ Go for “no-UI” style, embed the required information in
the objects, just like in normal life
▸ Enable the player to look around, even when in pause
mode

61. DEVELOPING VR IN UNITY
TOWARDS BETTER VR EXPERIENCE

62. DEVELOPING VR IN UNITY
FUTURE PREDICTION
▸ Lower cost of desktop VR
▸ WebVR
▸ VR beyond entertainment

63. AT THE SAME TIME, I'D SAY DEVELOPERS SHOULD BE REALISTIC. DON'T
BELIEVE THE MOST OPTIMISTIC THINGS THE ANALYSTS SAY. ESPECIALLY
WHEN THOSE ANALYSTS DON'T ACTUALLY KNOW ANYTHING ABOUT VR.
IT'S GOING TO TAKE TIME FOR VIRTUAL REALITY TO BECOME TRULY
MASS MARKET AND SUCCESSFUL, SO PEOPLE SHOULD SCALE THEIR
EFFORTS APPROPRIATELY.
Palmer Luckey (Co-founder of Oculus)
DEVELOPING VR IN UNITY

64. THANK YOUdidit@labtekindie.com
@kotakmakan