Open Standards for Cross-Platform Gaming, Virtual & Augmented Reality | Neil Trevett

© Copyright Khronos Group 2018 - Page 1
Open Standards for Cross-Platform
Gaming, Virtual & Augmented Reality
Casual Connect 2018
Neil Trevett | Khronos President
NVIDIA | VP Developer Ecosystem
ntrevett@nvidia.com | @neilt3d
www.khronos.org

Khronos Mission
Software
Silicon
Khronos is an International Industry Consortium of over 100 companies creating
royalty-free, open standard APIs to enable software to access hardware
acceleration for 3D graphics, Virtual and Augmented Reality, Parallel Computing,
Neural Networks and Vision Processing

Khronos Open APIs for Gaming and VR/AR
Vision and sensor
processing -
including neural
networks
Generate Low Latency 3D
Content and Simulations
Interact with sensor, haptic
and VR/AR display devices
Download 3D
augmentation object
and scene data
Khronos 3D APIs are on almost
every Workstation, Desktop,
Phone, Devices and Web Browser
in the world = Billions of seats

The Need for Cross-Platform 3D Standards
Vulkan Provides
Clean, modern architecture | Low overhead, explicit GPU access
Portable across desktop and mobile | Multi-thread / multi-core friendly
Primary 3D API on Android Platform
Efficient, low-latency, predictable performance
Access to multiple platforms

Khronos APIs Powering VR/AR
3D on millions of mobile VR devices
OpenGL ES 3.2 released August 2015
Rich Desktop VR Functionality
OpenGL 4.6 released August 2017
High-performance, Low-latency
VR/AR RENDERING
Vulkan 1.0 released February 2016
Available on diverse platforms
Powering WebVR in browsers
WebGL 2.0 Released February 2017
+
Cross-Platform, Portable VR/AR
DEVICE MANAGEMENT
Device discovery
Movement tracking
Input and haptics
Flexible graphics configuration

XR = AR + VR
V1.0 - focused on VR
V
A
After 1.0 – equal focus on AR
Cross-Platform, Portable VR/AR DEVICE MANAGEMENT
Device discovery | Movement tracking | Input and haptics
Flexible graphics configuration

OpenXR – Solving AR/VR Fragmentation
Proprietary
Engine
VR
App
1
VR
App
2
VR
App
4
VR
App
3
VR
Device
1
VR
Device
2
VR
Device
3
VR
Device
5
VR
Device
4
After OpenXR
Wide Interoperability of
VR Apps and Devices
VR
App
1
VR
App
2
VR
App
4
VR
App
3
Proprietary
Engine
VR
Device
1
VR
Device
3
VR
Device
5
VR
Device
2
VR
Device
4
Before OpenXR
VR Market
Fragmentation
Device Layer
Application Interface
Device Discovery
Device Events
Sensor tracking
Haptics
HMD Parameters
Installable
Drivers

OpenXR Working Group Members
Design work started in December 2016
Typically 12-18 months to develop a V1.0 specification

OpenXR and VR Run-times – a Win-Win
VR Run-times
Application Interface
Proprietary
APIs
Extensions
Device Layer
Proprietary
Driver
Interfaces
Extensions
Access to any
OpenXR Application
Access to any
OpenXR Device
Any successful standard encourages and
enables healthy industry competition
OpenXR will not replace VR run-times – or
‘outlaw’ existing interfaces
OpenXR will simply provide cross-vendor
APIs that can be exposed by a runtime to
access more apps and devices
OpenXR for portable
AR AND VR apps and devices
with initial focus on VR

Vulkan Explicit GPU Control
GPU
High-level Driver
Abstraction
Layered GPU Control
Context management
Memory allocation
Full GLSL compiler
Error detection
Application
Single thread per context
GPU
Thin Driver
Explicit GPU Control
Application
Memory allocation
Thread management
Synchronization
Multi-threaded generation
of command buffers
Multiple Front-end
Compilers
GLSL, HLSL etc.
Loadable debug and
validation layers
Vulkan 1.0 provides access to
OpenGL ES 3.1 / OpenGL 4.X-class GPU functionality
but with increased performance and flexibility
SPIR-V
pre-compiled shaders
Complex drivers
cause overhead
and inconsistent
behavior across
vendors
Always active
error handling
Full GLSL
preprocessor and
compiler in
driver
OpenGL vs.
OpenGL ES
Resource management
offloaded to app:
low-overhead, low-latency
driver
Consistent behavior:
no ‘fighting with driver
heuristics’
Validation and debug layers
loaded only when needed
SPIR-V intermediate
language: shading language
flexibility
Multi-threaded command
creation. Multiple graphics,
command and DMA queues
Unified API across all
platforms with feature set
flexibility
Vulkan = high performance and
low latency 3D and GPU Compute.
Ideal for VR/AR applications

Pervasive Vulkan
All Major GPU Companies shipping Vulkan Drivers for Desktop and Mobile Platforms
Embedded LinuxAndroid TVAndroid 7.0+ Nintendo Switch
http://vulkan.gpuinfo.org/
Oculus RiftSteamVR
VR Platforms
Desktop, Mobile, Embedded and Console Platforms Supporting Vulkan
Including phones and tablets from Google, Huawei, Samsung, Sony, Xiaomi - both premium and mid-range devices
GearVR Google Daydream
Game Engines
Desktop

Increasing Vulkan Developer Momentum
http://www.game-debate.com/news/23312/up-for-debate-vulkan-vs-directx-12-which-is-the-better-graphics-api
http://www.game-debate.com/news/22525/star-citizen-directx-11-and-12-support-axed-in-favour-of-vulkan-api
Publicly announced games
as of November 2017
#Vulkan Games = 35
#DX12 Games = 25
https://en.wikipedia.org/wiki/List_of_games_with_Vulkan_support
24th August 2017
Many more mobile titles coming…http://www.intrinsic-engine.com/
http://www.pcgamer.com/ashes-of-the-singularity-escalation-v24-update-adds-vulkan-support/
Vulkan is Powering Mobile Gaming…

Vulkan Evolution
Feb16
Vulkan 1.0
Vulkan Extensions
Maintenance updates plus additional functionality
Explicit Building Blocks for VR
Explicit Building Blocks for Homogeneous Multi-GPU
Enhanced Windows System Integration
Increased Shader Language Flexibility – inc. HLSL
Enhanced Cross-Process and Cross-API Sharing
Vulkan Roadmap
Regular Vulkan Core Releases
Integrates proven KHR extensions
Roadmap Discussions
Pushes forward the envelope of GPU Acceleration:
Enhanced Compute and Language Flexibility
Optimized Vision and Inferencing Acceleration
Ray Tracing
Widened Platform Support
Through Vulkan Universal Portability
Including Vulkan on macOS and iOS
Strengthened Ecosystem and SDK
Enhanced developer and debugging tools
Regression testing for SDK stability
Enhanced Conformance Testing (API now has 198K test cases
- up from 107K last year)
Compiler robustness - including HLSL support
Explicit Access to GPU
Acceleration

Market Demand for Universal 3D Portability
Games
Engines
Native 3D
Apps
Browser
Engines Games
Engines
Native 3D
Apps
Browser
Engines
Vulkan Universally
Portable Subset
JavaScript and
WebAssembly Native bindings
for ‘nexgen WebGL’
Community Outreach at GDC 2017
Create a hybrid Portability API?
Feedback - AVOID CREATING A FOURTH API!!!
Would need new specification, CTS, Documentation.
Additional developer learning curve.
A whole new specification to name, brand, promote.
Would INCREASE industry fragmentation
Tools Layers
API Libraries
Shader Translators
MAP Vulkan to Metal
and DX12

Content
JavaScript, HTML, CSS, ...
WebGL Stack
JavaScript Middleware
CSS
JavaScript HTML5
Browser provides WebGL
3D engine alongside other HTML5
technologies - no plug-in required
OS Provided Drivers
WebGL uses OpenGL ES 2.0 or
Angle for OpenGL ES 2.0 over DX9
Content downloaded
from the Web
Middleware provides accessibility
for non-expert programmers
E.g. three.js library
Low-level WebGL API provide a
powerful foundation for a rich
JavaScript middleware
ecosystem
Reliable WebGL
relies on work by
both GPU and
Browser Vendors
->
Khronos has the
right membership
to enable that
cooperation
WebGL - the Worlds MOST Portable 3D Platform
Run 3D apps – unchanged – in any browser on any platform

2007
OpenGL ES 2.0
2012
OpenGL ES 3.0
32-bit integers and floats
NPOT, 3D/depth textures
Texture arrays
Multiple Render Targets
Programmable
Vertex and
fragment shaders
2016
General Purpose Compute
Multi-threaded Rendering
Low-latency renderring
2011
WebGL 1.0
March 2017
WebGL 2.0
Conformance Testing is vital for
Cross-Platform Reliability
WebGL 2.0 conformance tests are very thorough
10x more tests than WebGL 1.0 tests
‘WebGL
Next’?
Pervasive OpenGL ES 2.0
OpenGL and OpenGL ES ship on
every desktop and mobile OS
WebGL Evolution
Vulkan
Portability
Apple
GPUWeb W3C
Community Group?
??
2014
OpenGL ES 3.1
Compute Shaders
Apple not shipping ES 3.1
Compute shader extension?

WebGL Momentum – WebGL 2.0 is Here!
http://caniuse.com/#feat=webgl
WebGL 2.0 brings Desktop-class
graphics to the Web
The time to create a new class of
Web-based 3D Apps is now!

WebGL - Central to Web 3D/VR Ecosystem
Application
Middleware
Browser
Native
3D Web Apps VR Web AppsNative AR Apps

glTF – Cross-Platform 3D Asset Transmission
OpenGL Transmission Format
Efficient transmission of 3D
scenes and assets
glTF 2.0 has PBR!
Cool, portable materials
Rendering API independence
Released June 2017
glTF 1.0
Aimed at loading assets into WebGL apps
Uses GLSL for materials
Released December 2015
All glTF spec development
on open GitHub:
https://github.com/KhronosGroup/glTF

glTF 2.0 PBR – Consistency Across Engines
glTF Model on Laugh Engine over Vulkan
https://github.com/jian-ru/laugh_engine
glTF on WebGL Reference Implementation
http://github.khronos.org/glTF-WebGL-PBR/
glTF Model running on Metal PBR Renderer
https://twitter.com/warrenm/status/891558755657175040
glTF on View 3D over DX12
https://www.microsoft.com/en-us/store/p/windows-view-3d/9nblggh42ths

Collada2gltf Translator
COLLADA FOR AUTHORING INTERCHANGE
Retains extensive data to enable editable
assets to be passed between authoring tools
glTF FOR RUN-TIME TRANSMISSION
Compact file size and
efficient processing/import
Apps and Engines Based
on any 3D API
Many More…
glTF can be generated from intermediate
formats such as COLLADA, or exported directly
from authoring tools to fit individual use cases
More tools …
COLLADA is intended to be used for
intermediate interchange, glTF is
designed for run-time delivery
COLLADA and glTF 3D Asset Formats

Publicly Stated Support for glTF
Strong glTF Industry Support
Please let us know if your
company logo is not here and
you would like it added!

Khronos Cooperative Framework
Royalty-Free
Specifications
Adopters
Programs
Implementations, Tools
Documentation, Samples
Adopters
Build conformant
implementation and products
Developers
Develop applications
using the APIs
Educators / Certifiers
Create Courses
Training and Certification
Educator Guidelines
Courseware Materials
API Working Groups
(Industry, Academic and
Associate members)
$
$
Khronos Board
Strategy, budget and
oversight$$
One vote per industry member
Conformance
Tests
Open and
royalty-free.
Often open sourced
Advisory Panels
By invitation, no fee.
Provide requirements
and draft spec feedback
Industry and
Community
Feedback and
contributions on
released materials
Membership Fees
Academic $1,000
Small companies $175/employee
(min $3,500)
Non-profits $7,500
Larger companies - $18,000

© Copyright Khronos Group 2018 - Page 28© Copyright Khronos Group 2017 - Page 28
Invitation to Get Involved!
• Khronos is creating cutting-edge, royalty-free, open API acceleration standards
- Relevant and enabling for gaming, VR and AR
• Join Khronos - ensure these standards meet your company’s needs
- And the needs of the gaming industry
• Use Khronos standards to bring cutting edge capabilities to your products
- While reducing costs and accelerating time to market
• Khronos encourages your participation
- As a Member, Advisor or Adopter
• More Information
- ntrevett@nvidia.com | @neilt3d | www.khronos.org
- These slides will be posted on the Khronos website

Open Standards for Cross-Platform Gaming, Virtual & Augmented Reality | Neil Trevett

Recommandé

Recommandé

Contenu connexe

Similaire à Open Standards for Cross-Platform Gaming, Virtual & Augmented Reality | Neil Trevett

Similaire à Open Standards for Cross-Platform Gaming, Virtual & Augmented Reality | Neil Trevett (20)

Plus de Jessica Tams

Plus de Jessica Tams (20)

Dernier

Dernier (20)

Open Standards for Cross-Platform Gaming, Virtual & Augmented Reality | Neil Trevett