The document discusses the capabilities and features of the HoloLens augmented reality headset. It describes how HoloLens blends interactive holographic content with the physical environment using features like gaze input, gesture input, voice commands, spatial mapping, and spatial sound. It provides examples of how these features can be used to build immersive experiences and applications for storytelling, exploration, collaboration, and more. Resources for developing for HoloLens using tools like the HoloToolkit and Windows Holographic platform are also mentioned.

1. You Had Me at HoloWorld
Dona Sarkar
Principal Product Person

2. 1994-2004 tech

3. 2005-2015 tech

4. 2016-???

6. the future is screenless

7. we are analog creatures

8. Human Environment
Machine
Tech

9. HoloLens: Windows 10 computer

10. design. explore. communicate.

11. HoloLens blends interactive holographic content
and your physical environment.

13. “What are holograms good for?”

14. experiences
and
information
embedded in
your
environment

15. Transportive Storytelling
transportive
storytelling

16. exploration
and
collaboration

17. Making
and
seeing
3D in 3D

18. Complement
existing
devices and
workflows

19. “What are holograms not good for?”

20. Challenges
in
holographic
design INDOORFOCUS BATTERYLIFE NEARCLIPPLANE ARMFATIGUE


21. Universal Windows
Platform
Tools
Ecosystem
Windows
Store
Windows
Holographic
Unity
DirectX
Visual Studio
Emulator

22. Build with Windows
Holographic Voice Commands
Spatial Mapping
Gesture Input
Spatial Sound World Coordinates
Gaze Input

23. • used for targeting and intent
• uses position and orientation of your user's
head, not their eyes, to determine their gaze
vector
• laser pointer from between the eyes
• as the user looks around app intersects this
ray, both with its own holograms and with
the spatial mapping mesh to determine
what virtual or real-world object your user
may be looking at.
• implement a cursor in HoloLens that
represents the user's gazeGaze Input

24. • allows interaction with hologram with hands or clicker
• gestures recognized by tracking position of either or
both hands that are visible to the device
• hands: ready state + pressed state
• clicker
• within the gesture frame
• 2 main gestures:
• Air-tap – activate a hologram
• Bloom – return to Shell
• app can recognize more by combining presses and
releases with movement of your hand or using a
clicker
Gesture Input

25. • users can gaze at a hologram and speak your command
• voice input is powered by the same engine that supports
speech in all other Universal Windows Apps.
• even without specifically adding voice support to your
app, your users can activate your holograms simply by
saying "select“
• “Hey Cortana” will enable you to do a multitude of things:
check weather, check battery, take a picture, etc.
• Manipulation commands:
• Face me
• Bigger | Enhance
• Smaller
Voice Commands

26. • holograms appear visually where the user's gaze
is pointing but sound can come from all directions;
above, below, behind, to the side, etc.
• use this feature to draw attention to an object that
might not currently be in the user's view
• user can perceive sounds to be emanating from a
source in the mixed-reality world.
• Walk toward it and away
• Behind the user
• sounds should come from realistic places. Eg: bird
chirp from the sky not the groundSpatial Sound

27. • detailed representation of real-world surfaces in
the environment
• apps align with user expectations by providing
familiar real-world behaviors and interactions.
• Placement
• Occlusion
• Physics
Spatial Mapping

28. - xyz coordinate system
- relative to the real world: 1 meter in the
real world is 1 meter in your app
- used to stabilize your hologram in a
realistic way
World Coordinates

30. Did you know…
- You can port any Windows 10 PC or Mobile app to HoloLens as a 2D app?
- You can see & record what the HoloLens is seeing by using the Windows Device Portal on a PC? Great
way to do demos for a large group.
- You can take a screenshot or a video
- “Cortana, take a picture”
- “Cortana, start recording”
…and immediately share on social
- The sharing experience (multiple people seeing the same hologram) can be used out of the box from the
HoloToolkit
- People feel like holograms are real
- If you place a holographic elephant in the room, people will walk around it, not through it
- If you are in a shared experience and others are shooting holographic projectiles at you, you’ll duck

31. Gaze
Input
Gesture
Input
Voice
Input
Spatial
Anchors
Spatial
Mapping
Spatial
Sound
Cursor
prefabs
Sharing
prefab
Spatial
prefabs
Utilities Shaders Tests
Holograms Materials Models Scripts Textures Shaders
App Experience – you build
HoloToolkit – you get open source for this
Windows Holographic – we made this

32. • Documentation
• Forums
• Academy
• Open Source
dev.windows.com/holographic

33. Dona Sarkar
twitter.com/donasarkar
insider.windows.com