Presentation MM-4095, Innovating with AMD Multimedia Technologies, by Amit Mookerjee and Carl Wakeland at the AMD Developer Summit (APU13) November 11-13, 2013.
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
MM-4095, Innovating with AMD Multimedia Technologies, by Amit Mookerjee and Carl Wakeland
1. INNOVATING WITH AMD
MULTIMEDIA TECHNOLOGIES
AMIT MOOKERJEE
SENIOR MANAGER, MEDIA SOFTWARE SOLUTIONS, AMD
CARL WAKELAND
FELLOW DESIGN ENGINEER, AMD
2. AMD Media SDK
Enabling Developers to Provide the
Best Multimedia Experiences on the
AMD platform
3. Multimedia Experiences on AMD
Video Editing/Transcode
Cloud Gaming
Wireless Display
Video Conferencing
Remote Desktop
Video Playback
3 | INNOVATING WITH AMD MULTIMEDIA TECHNOLOGIES | NOVEMBER 13, 2013
4. AMD MULTIMEDIA SOFTWARE STACK
Applications
ISV
ISV
MFT
MFT
ISV
MFT
Media SDK
AMD Media Framework (AMF)
DirectX
OpenCL
Driver
AMD HW
4 | INNOVATING WITH AMD MULTIMEDIA TECHNOLOGIES | NOVEMBER 13, 2013
OpenGL
5. MEDIA SDK1.0 SUMMARY
MFTs for leveraging Media HW accelerator blocks for video encoding/decoding
Video Quality MFT for video pre/post processing
AMF-DEM library for use in low latency applications
Support for Windows 7/8 Desktop as well as Windows 8 Store Application*
* AMF-DEM library only supports classic desktop applications
5 | INNOVATING WITH AMD MULTIMEDIA TECHNOLOGIES | NOVEMBER 13, 2013
6. MEDIA SDK1.0 VALUE FOR DEVELOPERS
Leverage AMD’s industry leading multimedia IP
‒ Leverage well known video processing features like AMD Steady VideoTM
‒ Leverage the unique VCE-DEM feature on AMD APUs/dGPUs in low latency applications
like wireless display and remote desktop
Optimal use of Heterogeneous Compute in Multimedia Apps
‒ eg. Buffer sharing on GPU, DX to OpenCL inter-op
Reduce Time to Market for Multimedia ISVs
‒ Optimized sample code for targeted application use cases
‒ Developer collateral and support via forum
6 | INNOVATING WITH AMD MULTIMEDIA TECHNOLOGIES | NOVEMBER 13, 2013
11. BENEFITS OF AMD TRUEAUDIO ACCELERATION FOR GAMERS
Raising the bar for the audio gaming experience on
PC
‒ More audio objects and higher-resolution sound
environments, with minimal workload added to the CPU
‒ Eliminate the workload uncertainty factor sound designers
have when relying completely on CPU-based offload
‒ TrueAudio provides a baseline of bonus audio processing
throughput that sound designers can add to their typical
10% CPU utilization budget
Enhanced headset experience
‒ Implement enhanced environmental 3D object positioning
using position information from the game engine
‒ TrueAudio works inside the game application itself. OS
audio stack is not involved
** All features above are dependent on 3rd Party ISVs such as game developers, audio middleware, etc..
11 | INNOVATING WITH AMD MULTIMEDIA TECHNOLOGIES | NOVEMBER 13, 2013
12. OTHER POTENTIAL TRUEAUDIO USES
Content management
‒Audio search pre and post-processing
‒ Quickly navigate through hundreds of sounds to
find just the one you want
‒ Pick distinct songs out of your library just by
pointing the mouse and listening
Professional Audio Processing
‒Client digital audio workstation
applications
‒Use TrueAudio to accelerate :
‒ Music creation with complete control over
every aspect of the sound and add custom
effects
‒ Add dozens of layers of sound, music and
dialogue to your movie masterpiece
** All features above are dependent on 3rd Party ISVs, audio middleware, etc..
12 | INNOVATING WITH AMD MULTIMEDIA TECHNOLOGIES | NOVEMBER 13, 2013
13. AMD TRUEAUDIO HARDWARE ARCHITECTURE
Tensilica HiFi EP Subsystem
Bus Routing and Bridging
DSP0
Address Translation
Interrupt Routing
GPU Interrupt
Clock/Reset/Pwr Control
I-Cache
32KB
D-Cache
32KB
8 KB
Scratch
RAM
AMD TRUEAUDIO Registers
Semaphore I/F
Sync
Memory
Client/Target MMIO
MMIO
Debug
JTAG
Bus Bridging
Tensilica HiFi EP Subsystem
Address Translation
DSP1
I-Cache
32KB
D-Cache
32KB
8 KB
Scratch
RAM
Routing and Arbitration
Streaming DMA Engine
Address Translation
Memory Bus Interface with scattergather Translation Apertures
Tensilica HiFi EP Subsystem
DSP2
I-Cache
32KB
D-Cache
32KB
8 KB
Scratch
RAM
13 | INNOVATING WITH AMD MULTIMEDIA TECHNOLOGIES | NOVEMBER 13, 2013
384 K Shared Instruction/Data RAM
Memory
14. AMD TRUEAUDIO DSP FEATURES
Tensilica HiFi EP Subsystem
Tensilica HiFi EP instruction set
DSP0
I-Cache
32KB
D-Cache
32KB
8 KB
Scratch
RAM
Tensilica HiFi EP Subsystem
D-Cache
32KB
Tensilica Xtensa SP float support, DP assistance
(also supports some transcendentals)
32 KB instruction and data caches
8 KB of scratch RAM used for fast local operations
DSP1
I-Cache
32KB
Three Audio Optimized DSP cores
8 KB
Scratch
RAM
Tensilica HiFi EP Subsystem
External power/clock management for clock
freq/voltage
(up to 900 MHz)
DSP2
I-Cache
32KB
D-Cache
32KB
8 KB
Scratch
RAM
14 | INNOVATING WITH AMD MULTIMEDIA TECHNOLOGIES | NOVEMBER 13, 2013
15. AMD TRUEAUDIO SPECIFIC HARDWARE FEATURES
Streaming DMA engine
‒ Multichannel with
programmable descriptors, IOC
‒ Scatter gather architecture
Internal Shared memory
‒ 384 KB of shared internal
memory, 8 KB banks
‒ Independently arbitrated for
con concurrent access of DMA
and each DSP
Bus Routing and Bridging
‒ Up to 64 MB addressable space
in frame buffer
GPU Interrupt
Address Translation
Clock/Reset/Pwr Control
AMD TRUEAUDIO Registers
Semaphore I/F
Sync
Memory
Client/Target MMIO
MMIO
Debug
JTAG
Bus Bridging
Address Translation
Bus interface to frame
buffer/system memory
‒ High performance, low latency
access to frame buffer memory
Interrupt Routing
Routing and Arbitration
Streaming DMA Engine
Address Translation
Memory Bus Interface with scattergather Translation Apertures
‒ Windowed into internal AMD
TRUEAUDIO system memory
space
15 | INNOVATING WITH AMD MULTIMEDIA TECHNOLOGIES | NOVEMBER 13, 2013
384 K Shared Instruction/Data RAM
Memory
16. AUDIO ALGORITHMS COMPLETED AND IN DEVELOPMENT
AMD is engaged with tools and plugin developers to accelerate the following
with TrueAudio:
‒AudioKinetic
‒ Convolution reverb
‒ GenAudio AstoundSound RTI positional audio
‒FMOD
‒ FMOD Reverb
‒ ML1 Limiter from McDSP
‒ Others to be announced
16 | INNOVATING WITH AMD MULTIMEDIA TECHNOLOGIES | NOVEMBER 13, 2013
17. ACCESSING AMD TRUEAUDIO FUNCTIONALITY FROM APPLICATIONS
Apps & Game Engines
w/ Middleware
Support
Game Engines w/
Integrated Sound
Engine
Middleware API
Audio Middleware
Higher-Level API
Optional API Abstraction
Layer(s)
AMD Audio
API
TrueAudio User-mode Services Library
17 | INNOVATING WITH AMD MULTIMEDIA TECHNOLOGIES | NOVEMBER 13, 2013
AMD Audio
API
Audio Middleware
‒ Audio Middleware is responsible
for audio engine framework
‒ Audio algorithms typically ship
with middleware - these can be
accelerated with TrueAudio
accessible through AMD Audio
API
Game Engines
‒ Game engines can integrate
sound engine, but most work
with audio middleware vendors
such as FMOD, AudioKinetic
18. TRUEAUDIO ENABLEMENT FOR DEVELOPERS
3rd Party Middleware
Integration
‒ Less integration
work required
‒ Use existing high level
Middleware API to
accelerate plugins w/
TrueAudio
Choose from native
middleware plugins or 3rd party
‒ Middleware
licensing
required
18 | INNOVATING WITH AMD MULTIMEDIA TECHNOLOGIES | NOVEMBER 13, 2013
Game Engine
Middleware API
Middleware
Plugins
(ie. FMOD,
AudioKinetic)
3rd Party Plugins
(ie. McDSP)
Middleware
AMD Audio API
AMD TrueAudio
19. AUDIO DATA FLOW & OUTPUT SUPPORT
Application / Game
Audio Middleware
Effects & Algorithms
AMD
TRUEAUDIO
19 | INNOVATING WITH AMD MULTIMEDIA TECHNOLOGIES | NOVEMBER 13, 2013
Sound
Card
Driver
OS Driver Stack
Audio
Assets
3.5mm Audio Jack
USB
Audio
Driver
USB
HDMI/D
P Audio
Driver
HDMI/DP
20. SUMMARY
Programmable audio pipeline grants artistic
freedom to game audio engineers for sound
processing
Easy to access through popular audio libraries
used by top game developers
Fundamentally redefines the nature of a
modern PC graphics card
Spatialization, reverb, mastering limiters and
simultaneous voice decode are only the
beginning
20 | INNOVATING WITH AMD MULTIMEDIA TECHNOLOGIES | NOVEMBER 13, 2013