1. Sound Processing
in Consumer Electronics
and in PulseAudio
Alexander Patrakov

2. About Myself
● Author of some sound processing code
– Dcaenc: a Free DTS-compatible encoder
– Wine resampler
● Dayjob:

7. Hardware
LG 47LM640T
Onkyo TX-NR626

8. Common effects
● Equalizer presets
– Sound Mode, Listening Modes
● Virtual Surround
– Theater-Dimensional Mode
● Dynamic Range Compression
– Late-Night Mode

9. ● User EQ
● ClearVoice II
● Bass, Tremble
● Speaker Distance
Compensation
● Bass Management
● Digital room
Correction

10. 5.1 Speaker Setup
Delay is
needed!

12. Instead of the equalizer

13. It Just Works!

14. PulseAudio

15. DSP-related functionality
● Equalizer
● Resampler
● Channel Remixer
● Virtual Surround
● Bass Management
● Encoders & Decoders
● LADSPA Effects
● Echo Cancellation

16. In this talk
● Equalizer
● Resampler
● Channel Remixer
● Virtual Surround
● Bass Management
● Encoders & Decoders
● LADSPA Effects
● Echo Cancellation

17. module-equalizer-sink
Author: Jason Newton

18. Spectrum * Filter

19. Spectrum * Filter
?

20. Convolution
t

21. FIR filter design
● Given the desired frequency response
– Take IFFT
– Shift the signal to positive time values
– Truncate the signal to a suitable length
– Apply a windowing function
http://www.dspguide.com/ch17/1.htm

22. Divide and Conquer
FFT-based magic

23. Circular convolution
t

24. Convolution
t

25. Tricks Against Circular Convolution
● Overlap-Save
● Overlap-Add

26. Overlap-Save
Corrupted fragment
The input signal is split into overlapping segments
Corrupted fragments are thrown away
(the previous segment contains a non-corrupted version)

27. Overlap-Add
Space for growth
Pad the input signal fragments with zeroes
Perform circular convolution
Result: overlapped segments of output signal
Resolve the overlap by summation

28. module-equalizer-sink
● Where is this?
– Take IFFT
– Shift the signal to positive time values
– Truncate the signal to a suitable length
– Apply a windowing function

29. module-equalizer-sink
?
Latency = ?

30. module-equalizer-sink
/* FIXME: Please clean this up. I see more commented code lines
* than uncommented code lines. I am sorry, but I am too dumb to
* understand this. */
fftwf_malloc(); …; free();
Buffer Overflow

31. module-equalizer-sink
EPIC FAIL

32. module-ladspa-sink + mbeq +
veromix

33. module-virtual-surround-sink
5.1 recording Stereo Headphones
Author: Niels Ole Salscheider

34. Amplitude difference
Woof!
Woof!

35. Delay
Woof..!...Woof!

36. Spectral features

37. module-virtual-surround-sink
● Convolution with HRIR (Head Related Impulse
Response)
– HRIR has to be downloaded separately

38. Virtual Surround on Speakers
● Ambiophonics
– Speaker crosstalk cancellation
– Same technology as in soundbars
– Not in PulseAudio yet

39. HRIR (left ear)

40. HRIR (left ear)
Cutoff at 64 samples
(to save CPU time)

41. HRTF (left ear)

42. HRTF (left ear)
This
sounds
cheap

43. Conclusions

44. Open Tasks
● See GSOC thread: http://goo.gl/7fVwTk

45. Other Relevant Projects
● Speex: contains a denoiser
● Vlevel: dynamic range compressor
● BruteFIR: a fast convolution engine
● DRC-FIR: digital room correction