Introduction to daydream for AnDevCon DC - 2017
•Télécharger en tant que PPTX, PDF•
1 j'aime•436 vues
This document provides an introduction and overview of virtual reality (VR) and Google's Daydream VR platform. It begins with an explanation of the differences between augmented reality and VR. It then discusses VR use cases and hardware requirements, including displays, tracking, and controllers. The document outlines Google's Daydream VR platform, including new features like WorldSense inside-out tracking. It also covers developing for Daydream using tools like Unity and publishing apps to meet quality standards. In closing, it discusses challenges and the future of Daydream VR.
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à Introduction to daydream for AnDevCon DC - 2017
Similaire à Introduction to daydream for AnDevCon DC - 2017 (20)
Dernier
Dernier (20)
Introduction to daydream for AnDevCon DC - 2017
- 1. Introduction to Virtual Reality The Road to Virtual Immersion • Jared Sheehan • Twitter: @jayroo5245 • http://meetup.com/DCAndroid • http://slideshare.net/jayroo5245
- 2. Android Summit – August 24th and 25th https://www.meetup.com/DCAndroid
- 3. Android Summit – August 24th and 25th https://tinyurl.com/AndroidSummit
- 4. Agenda • Augmented Reality vs Virtual Reality • VR Use cases • What is Virtual Reality • Daydream hardware • How it works • Developing • Questions?
- 5. Augmented Reality vs Virtual Reality • Augmented Reality • computer-generated visual layers atop an existing reality • Improve an existing visual scene • Interact with visual layers and touchpoints • Virtual Reality • Completely new computer-generated environment • Full immersion in a scene • Interact with the full environment
- 6. Augmented Reality vs Virtual Reality • Augmented reality and Virtual reality are inverse reflections of one in another with what each technology seeks to accomplish and deliver for the user. • Virtual reality offers a digital recreation of a real life setting, while augmented reality delivers virtual elements as an overlay to the real world. • Reality–virtuality_continuum
- 7. VR Use Cases – Google’s Version of Milgram’s Continuum
- 8. VR Use Cases – Interactive environment
- 9. VR Use Cases – Interactive environment
- 10. What is Virtual Reality – Hardware – Display
- 11. What is Virtual Reality – Hardware – Display – Stereoscopic 3D View What’s next? Anything else?
- 12. VR Use Cases – 360 Video
- 13. What is Virtual Reality – Hardware
- 14. VR Use Cases – 360 video vs VR explained
- 15. VR Use Cases – 360 video
- 16. VR Use Cases – 360 video
- 17. VR Use Cases – 360 video
- 18. VR Use Cases – 360 video
- 19. What is Virtual Reality – Hardware
- 20. What is Virtual Reality – Hardware – Optics
- 21. What is Virtual Reality – Hardware – Optics • Optical lenses make it possible to focus on a screen that is very close to your face. • Try holding your phone really close to your face. Can you see anything? • It focuses the light from a near eye display onto your eye in a way that makes it clear. • It greatly magnifies the image from your phone. • What are the side effects of magnifying images that close to your eye? • Reduces the total resolution of the user’s display. • Depending on the display the experience may seem pixelated.
- 22. What is Virtual Reality – Hardware – Display • Right behind the optical lenses sit one or more VR Displays: • Usually: High resolution, OlED displays that support low persistence features • Sometimes the Displays are built into the Headset: • HTC Vive and Oculus Rift • Displays can be external devices: • Daydream View, Cardboard, Samsung Gear • Low Persistence and motion blur: • Instead of showing a full image, VR will show you parts of an image. • Reduces motion blur and keeps the image clear as you navigate about. • Basically, hides the pixels as they change so the user doesn’t see/perceive blur.
- 23. What is Virtual Reality – Hardware – Tracking • Vital to VR technology: • The computer must know where the user is and is looking in space. • Rotational Tracking • All VR systems rely on an Inertial Measurement Unit (IMU) • Enables high speed rotational tracking • IMU’s cannot tell where an object is in space, only how it is rotated • Positional Tracking techniques: • Cameras • Lasers • Magnetic Fields • This is an active field of discussion and research. IE no accepted best practice yet
- 24. What is Virtual Reality – Degrees of Freedom (DOF) • 3 – DOF Tracking: • Head Rotation Tracking are accurate • Pitch / Yaw / Roll • X, Y and Z axis • No depth in the room • 6 – DOF Tracking: • Head Rotation Tracking are accurate • X, Y and Z axis • Pitch / Yaw / Roll • And Positions or body movement around a room • X, Y and Z axis • Surge / Heave / Moving Left or right
- 25. What is Virtual Reality – 3 DOF – Inertial Measurement Unit (IMU) • Inertial Measurement Units • Really good at quickly and accurately measuring movement • IMU Hardware - • Accelerometer • Gyroscope • Magnetometer • Constantly infer where the headset is pointed • All major 3 - DOF VR systems use some form of IMU For more information on using Android Device sensors: • Contextual Awareness - Android Summit 2016 • Slides • Github Project - makingsense The Daydream platform makes use of internal Android Device sensors to determine movement
- 26. What is Virtual Reality – 6 DOF – Major Players • Oculus Rift: • Tracking System: Constellation • Infrared LED system in the Headset • Camera captures the LED light patterns • Transmits to a computer and transposes the • images to the headset • This Model along with an internal IMU • Calculates the users head position • HTC Vive: • Tracking System: Lighthouse • Infrared Laser system in base stations • around the room • IMU Hardware: • Accelerometer • Gyroscope • Magnetometer • Constantly infer where the headset is pointed
- 27. Daydream!
- 28. Daydream – Google I/O 2017 • 8 Daydream ready phones • Standalone Daydream Headsets • no cables, phone or PC • 150+ Daydream apps • 3x improvement from Early 2017 • Daydream 2.0 Euphrates • Cast support – Share with friends on your TV • Coming later 2017
- 29. Daydream – What’s New in Daydream – 2017 - WorldSense • “WorldSense" • “Inside-out” Tracking • Does not need external sensors • Version of Tango for VR/AR • 6 Degrees of Freedom (6DOF) • Rotational and Positional Tracking • Dramatically improves the experience • VPS – navigate indoors using visual reference points as a sort of indoor GPS • Previous Daydream devices only had 3DOF • Only Rotational Tracking
- 30. Daydream – What’s New in Daydream – 2017 - WorldSense
- 31. Daydream – Google I/O 2017 • What are the new devices going to look like?
- 32. Cardboard versus Daydream • Cardboard – Inexpensive VR that works • No device hardware requirements • phone + headset-only • Input: • Uses the device’s magnetometer to register the movement of the included magnet as a click • 20% Google Project - $15 • Daydream – Virtual Reality for Grownups • Strict device hardware requirements • phone (optional) + headset + controller • Input: • Controller IMU sensor data • Click events • Relatively inexpensive - $79 • Standalone Headset - $TBD
- 34. Daydream Hardware – Device Hardware Requirements • Device Hardware Requirements: • Android 7.0 Compatibility Definitions Document • Hardware Requirements Highlights: • Bluetooth 4.2 LE • Display between 4.7 and 6 inches • Resolution at least 1080p @ 60Hz display with • 3ms or less latency and 5ms or less persistence. • (Quad HD or higher recommended.) • OpenGL ES 3.2 and Vulkan • Able to decode 2 instances of 60fps video simultaneously • Consistent 60fps rendering • Temperature sensors capable of reading device surface temperature
- 35. Daydream Hardware – Headset Requirements • Headset Requirements: • Lenses • Location to strap in a device that is 4.7 to 6.0 inches • Strap of some sort to hold the device place • Near Field Communication Chip • For automatically launching the VR experience • Daydream View: • Has no electronic components, so feel free to wash it
- 36. Daydream Hardware – Controller Requirements • Controller Requirements: • Sensor Hardware: • 3-DOF - Motion-sensing capabilities • Must include its own IMU • Accelerometer • Gyroscope • Orientation • Touchpad (Clickable) • Will send click events to device • Single click and Long press • Buttons: • Volume Buttons • App Button – Open to developer • Home Button • System functions • long-press to re-center the headset • 3D Pointer: Touchpad App Button Home Button Volume Buttons
- 37. Daydream Hardware – Controller Emulator • Controller Emulator App: • Most Mobile Phones have the correct hardware to act as a Daydream Controller • Download the Daydream Controller App • Pair your headset phone to your controller phone via Bluetooth • Configure the Controller Emulator device through Developer Options • Documentation • Off you go! Touchpad App Button Home Button Volume Buttons Double tap to emulate a click
- 38. Daydream Hardware – Sticker overlay - Controller Emulator
- 39. Daydream: How it works
- 40. Developing – VR development types and docs • Daydream Android VR SDK and VR NDK: • Google - Android VR support docs • Daydream iOS VR SDK: • Google - iOS VR support docs • Unity: • Unity - VR Support docs • Google - Unity support docs • Unreal Engine: • Unreal - VR Support docs • Google - Unreal support docs
- 41. What is Virtual Reality – Google VR for Android
- 42. Developing – GoogleVR Repo • https://github.com/googlevr
- 43. Developing – Platforms! • Scene Creation / Manipulation • Unity • UnReal • Asset / Model creation • Blender.org • Open source • Autodesk Maya • 3D Modeling and rendering tool
- 44. Virtual Reality – Unity Engine
- 45. Developing – Unity Platform Hierarchy Assets Scene Inspector
- 46. Daydream – Application Discovery – Daydream Home in VR Mode Discovery Windows Google Play
- 47. Developing – Publishing Daydream apps • Daydream App Quality Requirements: • “To ensure a great user experience, apps for Daydream must follow specific requirements for performance and usability”. • https://developers.google.com/vr/distribute/daydream/app-quality 1. Design Requirements: • UX-D1 – Users can focus on objects and read necessary text. • UX-C1 – Controller must be used as a laser pointer when clicking on UI targets. 2. Functionality requirements: • FN-M1 – App manifest does not request the NFC permission. • FN-S2 – App uses Daydream API calls to transition between activities. 3. Performance and stability requirements: • PS-P3 – App does not display a thermal warning during 30 minutes of usage. 4. Publishing requirements: • PB-P2 – App has a VR Icon.
- 48. Developing – Challenges • Simulator Sickness (Also called Cybersickness): • Can make people uncomfortable • Sick, headaches or nauseous • Anything that causes a mismatch sense of motion can do this • "Screen door" effect: • When users look at the display in their VR headset, they will often see a regular grid of lines. These are the spaces between the pixels, which you can only see because the screen is magnified and several inches away from your face. • Requires a Headset, Display (and perhaps a controller) of some sort: • Barrier to adoption
- 49. Developing – Daydream’s Future • Google Assistant integration: • Very natural Fit • Use Voice integration when users cannot physically touch their device • Multi Controller: • Why not? • A users offhand is just sitting there doing nothing • iPhone support: • Would require an SDK of some sort • Not sure if possible in Daydream’s current form • iOS provides much less Bluetooth control/customization • Hack it together: • https://www.techworm.net/2016/12/somebody-just-hacked-google- daydream-vr-make-work-iphone.html
- 50. Questions?
- 51. Virtual Reality – References • https://developers.google.com/vr/daydream/overview • http://www.theverge.com/2016/11/10/13578012/google-daydream-view-vr- review-mobile-headset-pixel • http://features.shopomo.com/electronics/google-daydream-new-mobile-vr- standard-compares-google-cardboard/ • http://www.popsci.com/google-daydream-vr-review • http://www.phonearena.com/news/Google-Daydream-VR-vs.-old-mobile-VR- Whats-the-difference_id86140 • http://www.androidcentral.com/google-reveals-hardware-requirements-daydream- vr • Unite 2016 - Making Daydream Real: Building for Google's New VR Platform • https://www.udacity.com/
Notes de l'éditeur
- Hello everyone, I am Jared Sheehan, I am a Lead engineer at Capital One. This talk will be an Introduction to VR with a focus on the Daydream platform. A little about me, I have been building Android applications and devices since 2010 and this is my first presentation at AnDevCon. This really is an introduction to VR and Daydream. As such, unlike most of my presentations, I will not show much, if any, code, though I may perhaps in other VR subsequent presentations. My thought process around this is really simple. I didn’t really know what VR was when it came out. It is very different then your standard android application that uses standard Views, Activities, fragments, intents, datastores etc. Its almost all about the visual experience and the user’s immersion and interaction with that environment. I am a cofounder and organizer of the DCAndroid meetup group, we have regular monthly meetings to nerd out on Androidy topics. You will see our new logo We have a brand new logo, which you can see right there. If you want a sticker come see me or tweet me afterwards. This presentation will be on slideshare after the session. Also – Please leave feedback. You can do this directly in the mobile app or there is a QR code link I can share afterwards
- We have officially passed 500 members woohoo!
- Be there or be square Very amount to get in, $75 for early birds. All proceeds go directly to Women Who Code Hot off the news press… for a limited time, the first official discount link for Android Summit. - Take pictures, email your friends, send a letter to your grandmother (cause you know she doesn’t do Twitter)
- Augmented Reality vs Virtual Reality – What’s the difference between the two VR Use cases – Why does VR matter and what makes it interesting What is Virtual Reality – What exactly is it? What is a display, what are the optics, what does 3DOF and 6DOF mean? VR Options – Is Daydream the only VR option out there? Hint: No Daydream hardware – What makes Daydream, daydream and not say… cardboard How it works – A bit on how it works Developing – If you wanted to build daydream applications, what do you need to do? Is it hard? Seems hard… Questions?
- AR - is a technology that layers computer-generated enhancements atop an existing reality in order to make it more meaningful through the ability to interact with it. AR is developed into apps and used on mobile devices to blend digital components into the real world in such a way that they enhance one another, but can also be told apart easily. (Now describe the scene) VR – an artificial, computer-generated simulation or recreation of a real life environment or situation. It immerses the user by making them feel like they are experiencing the simulated reality firsthand, primarily by stimulating their vision and hearing. (Now describe the scene)
- The virtuality continuum is a continuous scale ranging between the completely virtual, a virtuality, and the completely real, reality. AR use cases – It is used to display score overlays on telecasted sports games and pop out 3D emails, photos or text messages on mobile devices. Leaders of the tech industry are also using AR to do amazing and revolutionary things with holograms and motion activated commands. VR use cases - To create and enhance an imaginary reality for gaming, entertainment, and play (Such as video and computer games, or 3D movies, head mounted display). To enhance training for real life environments by creating a simulation of reality where people can practice beforehand (Such as flight simulators for pilots).
- Now onto VR
- Next I will discuss some use cases for VR, how the images are shown in the display and how your brain represents that in your head
- Binocular vision describes the way in which we see two views of the world simultaneously—the view from each eye is slightly different and our brain combines them into a single three-dimensional stereoscopic image, an experience known as stereopsis
- Notice the phone displaying a two views called binocular imaging which ends up being interpreted by the brain into a stereoscopic image, is a technique used to enable a three-dimensional effect, adding an illusion of depth to a flat image. Stereopsis, commonly (if imprecisely) known as depth perception, is the visual perception of differential distances among objects in one's line of sight.
- Stereoscopic 3D can add another level of immersion by adding depth data between the foreground and background. Your favorite 3D blockbuster films are typically shot with 2 lenses side by side, to give you a feeling of a different vantage point per eye. Like any production, this can look strange if poorly implemented, or absolutely amazing if done right. Stereoscopic 3D in VR, that depth information has to be overlaid and mapped to sphere. Because of parallax between cameras, this can be especially challenging. Any minor flaws or “stitch seams” in the footage are magnified in 3D, and sometimes anomalies occur in different places per eye - which makes it uncomfortable to watch.
- Now onto VR Hardware
- Google launched Jump, a platform for VR video, back at Google I/O 2016 -- along with a slightly bonkers camera rig called Odyssey co-designed with GoPro. Today, prospective content creators can put their name down to get early access to the hardware (pictured after the break). Specifically aimed at professional partners. We also get to know a little bit more about the rig, including its cost: an eye-watering (at least for us amateurs) $15,000. What do you get for your money? Well, 16 GoPros for starters (that accounts for $8,000 of the Odyssey's cost at the camera's $500 retail price). The rest of the package includes connectivity mounts ("bacpacs") for each camera, cables, memory cards, a pelican case and (of course) the cylindrical Odyssey rig itself. Once combined, the rig will shoot 2.7K video in 4:3 aspect ratio. It is, of course, all about the 360-degree/3D experience, and Google with GoPro hopes that Odyssey can raise the bar in terms of immersive video quality.
- A standard 360 video is a flat equirectangular spherical video similar to viewing the world map on a globe. If viewed using VR Headsets, it feels as if you are inside the globe and looking at the inner surface.
- Fox Sports VR App - produce a live VR stream of the game Have a basement/man (or woman) cave environment Choose what you want to view High profile college and nfl games - loaded with new features: the ability to rewind the game in 30-second increments (for iOS users only), the ability to control replays from different camera angles, live-stats integration, and a suite of highlights allowing easy access to on-demand content. Users will also have the ability to select their own camera angles, picking where they “sit” throughout the game.
- Fox Sports VR App - produce a live VR stream of the game loaded with new features: the ability to rewind the game in 30-second increments (for iOS users only), the ability to control replays from different camera angles, live-stats integration, and a suite of highlights allowing easy access to on-demand content. Users will also have the ability to select their own camera angles, picking where they “sit” throughout the game.
- Fox Sports VR App - produce a live VR stream of the game loaded with new features: the ability to rewind the game in 30-second increments (for iOS users only), the ability to control replays from different camera angles, live-stats integration, and a suite of highlights allowing easy access to on-demand content. Users will also have the ability to select their own camera angles, picking where they “sit” throughout the game.
- Fox Sports VR App – - Play that won the super bowl 51
- Now onto VR Hardware
- Now onto VR
- In virtual reality, "the brain is expecting everything to be in sync, but things are not always in sync," he said; the virtual world is "incomplete."
- It is the act of reducing/erasing motion blur, allowing the player to move their head and keep eyes fixed on one point, as humans do in reality We have talked about Optics, weve talked about Displays, what’s next? Tracking…
- 3-DOF detects rotational movement around the X, Y, and Z axis — the orientation. For head movements, that means being able to yaw, pitch, and roll your head (figure above), while keeping the rest of your body in the same location. 3-DOF in VR allows you to look around the virtual world from fixed points —think of a camera on a tripod. For many 360° spherical videos, 3-DOF will provide very immersive content, such as viewing sporting events from a particular seat or nature from a particular lookout point. 6-DOF detects rotational movement and translational movement — the orientation and position. This means that your body can now move from fixed viewpoints in the virtual world in the X, Y, and Z direction. 6-DOF in VR is very beneficial for experiences like gaming, where you can move freely in the virtual world and look around corners. However, even simple things, like looking at objects on a desk or shifting your head side-to-side can be compelling with 6-DOF. 6-DOF is more immersive since it captures our real movement and removes the sensory conflict between our vision and vestibular system (ear - motion, equilibrium, and spatial orientation).
- 3 DOF explained In the case of a phone being your display, it is also your IMU. This is why Google hopes Daydream will become the Ubiquitous VR provider. Most people already have all the hardware except the viewer
- 6 DOF players Oculus - Constellation - The system gets its name from a slew of infrared lights placed at strategic locations on both the Oculus Rift headset and the Oculus Touch controllers. These markers — laid out almost like a constellation — are picked up by the Oculus Sensors, which are designed to detect the light of the markers frame by frame. These frames are then processed by Oculus software on your computer to determine where in space you’re supposed to be. HTC vive Lighthouse -
- Enjoy high quality VR anywhere you want with no cables, phone or PC. Coming soon.
- WorldSense is a positional tracking system from Google that’s ‘inside-out’; that means it doesn’t need any external sensors or beacons to track the movement of your head through 3D space. The new fully self-contained ‘standalone’ VR headsets for the Daydream platform will use the new tech to allow for positional tracking which Google says “dramatically” improves the experience compared to prior Daydream devices (and we agree, good positional tracking is a huge benefit to immersion and comfort in VR). Those prior Daydream devices, which relied on typical smartphones, can only track the rotation of your head. Positional tracking allows the system to detect the movement of your head through space, like forward, backward, up and down motions. Rotational and positional tracking together are also often called ‘6 Degrees of Freedom’ or ‘6DOF’.
- Reference: https://gfycat.com/ZigzagBoilingCormorant How it works - Two wide angle cameras that detect the features of the room. Detects a table on the floor, items on the table. Fuse these images that detect objects with Sensor Data from the phone and you get the WorldSense VR experience. All this information is given to the Application within 5 milliseconds. All the while building a 3D model of the scene, re-recognizing items it has seen before to correct for things like drift. This is called ReLocalization (Refiguring out where you are in the room based on recognizing items already seen) This is called SLAM – Simultaneous Localization and Mapping.
- What are the new devices going to look like? Vive Daydream Standalone Headset Silhouette - It is significantly cheaper and easier to use an extra device to emulate the controller.
- Uses NFC to auto launch into ”VR Mode” Has Physical Alignment Dots to determine where your phone is, even if it’s a bit off it will automatic calibrations to get the display aligned. https://www.youtube.com/watch?v=ufFQXnIZpJY
- The controller is where a lot of the magic of Daydream comes in Controllers should be Accessible, Expressive and Portable. Volume buttons – You don’t have to awkwardly try to press the volume buttons on your device.. Clickable touchpad – All sorts of interesting things App Button – Developer can do whatever they like. Home button – Some system functions like long-press to re-center the headset and (shockingly) go home
- It is significantly cheaper and easier to use an extra device to emulate the contoller.
- App talks to SDK, which talks to Google VR Services witch handles all the BLE stuff
- Unity and Unreal Engine are multiplatform game development platforms
- Blender is the free and open source 3D creation suite. It supports the entirety of the 3D pipeline—modeling, rigging, animation, simulation, rendering, compositing and motion tracking, even video editing and game creation. Advanced users employ Blender’s API for Python scripting to customize the application and write specialized tools; often these are included in Blender’s future releases. Blender is well suited to individuals and small studios who benefit from its unified pipeline and responsive development process.
- Unity – - Unity is a cross-platform game engine developed by Unity Technologies and used to develop video games for PC, consoles, mobile devices and websites - You can create any 2D or 3D game with Unity. You can make it with ease, you can make it highly-optimized and beautiful, and you can deploy it with a click to more platforms than you have fingers and toes. What’s more, you can use Unity’s integrated services to speed up your development process, optimize your game, connect with an audience, and achieve success.
- - Enables Daydream and Cardboard app development in Unity. - Google partnered with Unity to ensure that Daydream was natively supported on Unity (from Day One) starting with Unity 5.6 - Google VR for Unity SDK – Native support for 360 video and input utils - The Google VR SDK for Unity provides additional features like spatialized audio, Daydream controller support, utilities and samples.
- Daydream Home is launched as soon as a user puts on the goggles Discovery Window – Curated content that a user can select from Google Play – The full power of Google Play Payments Discoverability 190+ countries Consistency Distribution Daydream – In app purchasing on launch Analytics experience
- Virtual reality sickness occurs when exposure to a virtual environment causes symptoms that are similar to motion sickness symptoms.[1] The most common symptoms are general discomfort, headache, stomach awareness, nausea, vomiting, pallor, sweating, fatigue, drowsiness, disorientation, and apathy
- https://www.techworm.net/2016/12/somebody-just-hacked-google-daydream-vr-make-work-iphone.html