This document discusses developing augmented and virtual reality applications. It defines mixed reality and explains why it is an emerging platform. It outlines tools like Unity and Unreal for building AR/VR experiences. It demonstrates VR interactions, locomotion, and adding voice commands. It provides tips for performance and targeting multiple headsets. It also covers hand tracking and developing for HoloLens. The key takeaway is to design for immersion, test on target devices, and leverage cloud services.

1. Video source: HoloLens 2 channel https://www.youtube.com/watch?v=eqFqtAJMtYE

2. Getting started with Unity and
AR/VR for the .NET developer
Davide Zordan
B: https://www.davide.dev
E: mail@davide.dev
T: @DavideZordan

4. Davide Zordan
Senior Software Engineer
MCSD, MCSA, MS, MCP
Microsoft MVP (2009-2014)

5. What is Mixed Reality?

6. The state of things as they “actually
exist” through our human senses
without any technology.
Artificially created sensory experiences of
people, environments and objects, which can
include sight, touch, hearing, and smell.
PHYSICAL REALITY VIRTUAL REALITY

7. MIXED REALITY
VIRTUAL REALITYPHYSICAL REALITY

8. PHYSICAL
REALITY
DIGITAL
REALITY
MIXED REALITY (MR)
WINDOWS MIXED REALITY HOLOGRAPHIC DEVICES (SEE-THROUGH) / MAGIC
LEAP ONE / Nreal Light / RealWare HMT-1 / LYNX-R1
OCULUS RIFT S - QUEST / HTC VIVE - COSMOS / VALVE INDEX /
WINDOWS MIXED REALITY IMMERSIVE DEVICES (OPAQUE) / Varjo XR-1
AR
TODAY
VR
TODAY
Physical - Digital Reality Spectrum
*The term mixed reality was originally introduced in a 1994 paper by Paul Milgram and Fumio Kishino, "A Taxonomy of Mixed Reality Visual Displays."

9. Why Mixed Reality?

10. VR/AR as next computing platform
Estimated $80bn revenue by 2025
MR is revolutionizing many processes

11. Gaming
Video entertainment
Healthcare
Real Estate
Retail
Education
Engineering
Military

12. How can we develop for AR/VR?

13. Designing great VR experiences
Place illusion
Embodiment illusion
Plausibility illusion
Source: https://www.coursera.org/lecture/introduction-virtual-reality/introduction-to-plausibility-illusion-psi-K5PGj

15. Development tools

16. Game Engines (Unity, Unreal Engine 4)
OpenGL, Microsoft DirectX, Vulkan
WebVR / WebXR
C#/.NET
C++
JavaScript

17. A new way to interact

18. VR interactions
Pointing
Selecting
Grabbing

19. VR Locomotion
Eye gazing
Teleportation
Dash

20. DEMO
VR locomotion and custom interactions

21. 1 – Using the SteamVR controllers bindings

22. 2 – Rotate the player

24. Adding vocal commands

25. 1 – Using the KeywordRecognizer

26. 2 – Shows UIs depending on voice commands

27. Targeting multiple headsets

28. 1 – Creating a VR headset manager

29. 2 – Initialising device-specific settings

30. DEMO
Speech recognition and multiple headsets
support

32. Performance tips

33. 1 - Mark not moving objects as static

34. 2 – Use baked lights

35. 3 – Avoid not performant code at runtime

36. Implementing Hand tracking

37. Image source: www.oculus.com/quest

38. Mixed Reality Toolkit for Unity
• https://aka.ms/MRTK
MRTK extensions for Oculus Quest
• https://github.com/HoloLabInc/MRTKExtensionForOculu
sQuest
• https://github.com/provencher/MRTK-Quest
https://aka.ms/MRTK
https://github.com/provencher/MRTK-Quest

39. DEMO
Hand tracking using Oculus Quest and
Mixed Reality Toolkit

41. Developing for AR

42. HoloLens 2
Source: https://news.microsoft.com/microsoft-at-mwc19/

49. Key takeaways:
Check performance early and often to avoid motion sickness
Test frequently using the target device
Design properly the experience for maximising the user immersion
Use cloud services to enhance the user experience (LUIS, speech, vision)

50. Summary
What is Mixed Reality
Why Mixed Reality
How to develop for AR/VR
VR Interactions
VR Locomotion
Adding vocal commands
Targeting multiple headsets
Performance tips
Hand-tracking
Developing for AR

51. “Any sufficiently advanced technology is
indistinguishable from magic”
Arthur C. Clark, Profiles of the future, 1961

52. Thanks
Davide Zordan
B: https://davide.dev
E: mail@davide.dev
T: @DavideZordan

53. References
AR / VR / Mixed Reality samples
https://github.com/davidezordan
SteamVR Unity plugin
https://github.com/ValveSoftware/steamvr_unity_plugin
Mixed Reality Toolkit
https://aka.ms/MRTK
Microsoft Ignite
https://8gportalvhdsf9v440s15hrt.blob.core.windows.net/ignite2017/session-presentations/BRK2404.PPTX
Goldman Sachs VR/AR report
https://www.goldmansachs.com/insights/pages/virtual-and-augmented-reality-report.html

54. References
Coursera “Introduction to Virtual Reality”
https://www.coursera.org/lecture/introduction-virtual-reality/introduction-to-plausibility-illusion-psi-K5PGj
Cognitive Services
https://azure.microsoft.com/en-us/services/cognitive-services
Oculus VR performance optimization guide
https://developer.oculus.com/documentation/pcsdk/latest/concepts/dg-performance-opt-guide
Mixed Reality Academy and Azure Services tutorial
https://docs.microsoft.com/en-us/windows/mixed-reality/tutorials
Unity: the most common mistakes to avoid
https://unity3d.com/how-to/unity-common-mistakes-to-avoid