The microprocessor OMAP 3430 contains a dual-core architecture consisting on an ARM Cortex-A8 (General-Purpose Processor) and a DSP C64x (as an Image-Video-Audio Accelerator). The dspbridge is a kernel driver for the OMAP3 architecture. It provides a interface to control and communicate with the DSP, enabling parallel processing in embedded devices. This talk will be about how to build a system with the dspbridge driver and how to use it for multimedia applications.

1. The DSP/BIOS Bridge
Víctor Manuel Jáquez Leal
Igalia
06 February 2010
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 1 / 41

2. The TI OMAP3 processor
I The TI OMAP3 processor
P The DSP/BIOS Bridge
Q The ingredients
R Future
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 2 / 41

3. The TI OMAP3 processor
Parallel computing
Serial computing is now dead.
€—r—llel ™omputing @whi™h st—rted more th—n RH ye—rs —goA is —
revolution th—t is now upon us
Programming for serial computing is already dicult
€rogr—mming for p—r—llel ™omputing will only ex—™er˜—te this di0™ulty
por p—r—llelism to su™™eed it must produ™e ˜etter perform—n™eD
e0™ien™y —nd reli—˜ility
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 2 / 41

4. The TI OMAP3 processor
OMAP 3530/20
720 MHz ARM Cortex A8
520 MHz TMS320C64x+ DSP
POWERVR SGX Graphics Accelerator
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 3 / 41

5. The TI OMAP3 processor
Devices and boards using OMAP3
There are a lot!
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 4 / 41

6. The TI OMAP3 processor
The C64x+ DSP
Digital Signal Processor
ƒpe™i—lized mi™ropro™essor
por f—st exe™ution of digit—l sign—l pro™essing
vow power ™onsumption
Digital Signal Processing
we—surement —nd (ltering of ™ontinuous re—lEworld —n—log sign—ls
eudioD videoD spee™hD —re ex—mples of those sign—ls
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 5 / 41

7. The DSP/BIOS Bridge
DSP-GPP parallel computing
Features to control the DSP
Mechanisms to communicate with DSP
Enabling parallel processing for multimedia acceleration
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 6 / 41

8. The DSP/BIOS Bridge
Available drivers
dsp-gateway
heveloped ˜y xoki— for the w—emo snternet „—˜lets
st works on ywe€I —nd ywe€P
st9s produ™tion re—dy
st9s used on the xoki— xVHH —nd xVIH
st follows vinux st—nd—rds —nd it9s ™lose to upstre—m —™™ept—n™e
„here9s ™ode for ywe€Q ˜ut it h—sn9t ˜een thoroughly tested
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 7 / 41

9. The DSP/BIOS Bridge
Available drivers
dsp-bridge
yrigin—lly developed ˜y „s
st still doesn9t meet vinux st—nd—rds —lthough there h—s ˜een — lot of
progress
ynly the e‚w side is —v—il—˜le —s open sour™eY the hƒ€ side is
™ompletely ™losed
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 8 / 41

10. The DSP/BIOS Bridge
Available drivers
dsp-link
e slimmer version of the dspE˜ridge
elso developed ˜y „s
st supports — wide v—riety of devi™es @h—†in™iD ywe€PD ywe€QD et™A
„he kernel driver doesn9t meet the vinux kernel ™oding ™onventions
„he sour™es h—ven9t ˜een su˜mitted for reviewD —nd it is not ™urrently
pl—nned to ˜e merged into upstre—m kernels
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 9 / 41

11. The DSP/BIOS Bridge
DSP/BIOS Bridge
General Architecture
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 10 / 41

12. The DSP/BIOS Bridge
Architecture
It is designed for one GPP and one or more attached DSPs
„he q€€ is ™onsidered the m—ster or host pro™essor
„he —tt—™hed hƒ€s —re pro™essing resour™es th—t ™—n ˜e used ˜y
—ppli™—tions running on the q€€
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 11 / 41

13. The DSP/BIOS Bridge
Architecture
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 12 / 41

14. The DSP/BIOS Bridge
Architecture
The Bridge supplies a link between a GPP program and a DSP task
The communication link is partitioned into two types:
wess—ging @shortD (xedElength p—™ketsAX por p—ssing ™ontrol —nd st—tus
inform—tion
h—t— stre—ming @multipleD l—rge ˜u'ersAX for stre—ming re—lEtime d—t—
Each sub-link operates independently
A GPP client can specify what inputs and outputs a DSP task uses
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 13 / 41

15. The DSP/BIOS Bridge
GPP Sopftware Architecture
The GPP OS see the DSP just as another peripheral device
root@beagleboard:~# ls -la /dev/DspBridge
crw-rw---- 1 root root 251, 0 Jan 1 2000 /dev/DspBridge
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 14 / 41

16. The DSP/BIOS Bridge
DSP Software Architecture
From the DSP/BIOS perspective, the bridge provides
e devi™eEindependent stre—ming sGy @ƒ„‚wA interf—™e
e mess—ging interf—™e @xyhiA
e ‚esour™e w—n—ger @‚wA ƒerver
The task environment is established by the RM Server
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 15 / 41

17. The DSP/BIOS Bridge
Components
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 16 / 41

18. The DSP/BIOS Bridge
Components (GPP)
Resource Manager
hyn—mi™—lly inst—nti—ting hƒ€ resour™es
wonitoring hƒ€ resour™es
hyn—mi™—lly lo—ding hƒ€ ™ode —s needed
smplementing poli™ies for m—n—ging hƒ€ resour™es
Platform Manager
ƒt—ti™—lly lo—ding — ˜—se ™ode im—ge for the hƒ€
ƒt—rting —nd stopping the hƒ€
smplementing d—t— stre—ming
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 17 / 41

19. The DSP/BIOS Bridge
Components (GPP)
OS adaptation layer
DSP link driver for low level communication
A dynamic conguration database (DCD) stores conguration
information
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 18 / 41

20. The DSP/BIOS Bridge
Components (DSP)
DSP/BIOS communicates with the GPP via the link driver
On top of the DSP/BIOS sits the Resource Manager (RM) Server
hyn—mi™—lly ™re—teD exe™ute —nd destroy hƒ€ pro™essing nodes under
‚esour™e w—n—ger ™ontrol
‚outing mess—ges ˜etween the q€€ —nd individu—l nodes
eltering t—sk priorities
‚esponding to ‚esour™e w—n—ger ™on(gur—tion ™omm—nds
ƒt—tus queries
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 19 / 41

21. The DSP/BIOS Bridge
Components (DSP)
DSP task nodes
„hey —re sep—r—te exe™ution thre—ds running on the hƒ€
„hey implement ™ontrol or sign—l pro™essing —lgorithms
„hey ™ommuni™—te with one —notherD —nd with the q€€
via short xed length messages and/or device-independent stream I/O.
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 20 / 41

22. The DSP/BIOS Bridge
GPP Side interface
Manager
…sed to o˜t—in hƒ€ pro™essor —nd m—nipul—te node ™on(gur—tion
inform—tion
Processor
…sed to m—nipul—te hƒ€ pro™essor o˜je™tsD whi™h represent p—rti™ul—r
hƒ€ su˜systems linked to the q€€
€ro™essor o˜je™ts —re used to ™re—teD exe™ute —nd delete nodes on —
p—rti™ul—r hƒ€ su˜system
es hƒ€Gfsyƒ fridge ™lients m—ke pro™essor e€s ™—llsD the
™orresponding hƒ€ pro™essor will tr—nsition ˜etween — set of
preEde(ned st—tesF
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 21 / 41

23. The DSP/BIOS Bridge
GPP Side interface
Node
…sed to m—nipul—te node o˜je™tsD whi™h represent ™ontrol —nd sign—l
pro™essing elements running on — p—rti™ul—r hƒ€
Stream
…sed to m—nipul—te stre—m o˜je™tsD whi™h represent logi™—l ™h—nnels for
stre—ming d—t— ˜etween the q€€ —nd nodes on — p—rti™ul—r hƒ€
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 22 / 41

24. The DSP/BIOS Bridge
Sequence for nodes controlling
Load the device driver
root@beagleboard:~# lsmod
Module Size Used by
dspbridge 729 0
bridgedriver 187569 1
Load a base image to the DSP
~# cat /etc/modprobe.d/bridgedriver.conf
options bridgedriver base_img=/lib/dsp/baseimage.dof
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 23 / 41

25. The DSP/BIOS Bridge
Sequence for nodes controlling
Open a handle to the DSP/BIOS Bridge device
dsp •h— ndle a dsp•open @ A Y
i f @ dsp •h—n dle ` H A {
p r • e r r @ 4 f — i l e d t o open hƒ€4 A Y
r e t u r n − IY
}
Reserve GPP-side resources for controlling a particular DSP
i f @ 3 d s p • — t t — ™ h @ dsp•h—ndle D H D x…vv D 8p r o ™ A A {
p r • e r r @ 4 dsp — t t — ™ h f — i l e d 4 A Y
r e t a − IY
goto l e — v e Y
}
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 24 / 41

26. The DSP/BIOS Bridge
Sequence for nodes controlling
Allocate DSP node for the selected processor
const dsp•uuid•t u u i d a
{ H xIP—Q™Q™I D H xdHIS D H xIIdR D H x W f D H xTW D
{ H xHH D H x™H D H x R f D H xQ— D H xSW D H x — e } } Y
i f @ 3 d s p • n o d e • — l l o ™ — t e @ dsp•h—ndle D p r o™ D 8u u i d D
x…vv D x…vv D 8node A A {
p r • e r r @ 4 dsp node — l l o ™ — t e f — i l e d 4 A Y
r e t u r n x…vv Y
}
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 25 / 41

27. The DSP/BIOS Bridge
Sequence for nodes controlling
Create the node on the DSP
i f @ 3 dsp•node•™re—te @ dsp•h—ndle D node A A {
p r • e r r @ 4 dsp node ™ r e — t e f — i l e d 4 A Y
r e t u r n x…vv Y
}
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 26 / 41

28. The DSP/BIOS Bridge
Sequence for nodes controlling
Launch the task node into their execute phase
i f @ 3 dsp•node•run @ dsp•h—ndle D node A A {
p r • e r r @ 4 dsp node r u n f — i l e d 4 A Y
return f — l s e Y
}
yn™e the t—sk is runningD the q€€ ™lient ™—n stre—m d—t— ˜u'ers
toGfrom the t—sk —s well —s ex™h—nge short mess—ges with the t—sk
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 27 / 41

29. The DSP/BIOS Bridge
Sequence for nodes controlling
Stream data to/from DSP tasks
„he q€€ ™lient then —llo™—tes d—t— ˜u'ers for the stre—m
If the buer are already pre-allocated, the GPP client can prepare the
buers for the stream
yn™e —llo™—ted —nd prep—red
They can be used to submit buers to a stream
Submitting a data buer to a stream will not block GPP thread
execution.
They can request a buer back from a stream
Requesting a buer back from the stream may cause the GPP thread
to block
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 28 / 41

30. The DSP/BIOS Bridge
Sequence for nodes controlling
Exchange messages with DSP nodes
i f @ 3 dsp•send•mess—ge @ dsp•h—ndle D node D I D H D H A A {
p r • e r r @ 4 dsp node p u t mess—ge f — i l e d 4 A Y
continue Y
}
i f @ dsp•node•get•mess—ge @ dsp•h—ndle D node D8msg D H A A
p r i n t f @ 4 € i n g X s d 7d wsg 7d wem 7d’n4 D
msg F ™md D msg F —rg•I D msg F —rg•P A Y
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 29 / 41

31. The DSP/BIOS Bridge
Sequence for nodes controlling
Terminate DSP nodes
i f @ 3 dsp•node•termin—te @ dsp•h—ndle D node D
8e x i t • s t — t u s A A {
p r • e r r @ 4 dsp node t e r m i n — t e f — i l e d X 7l x 4 D
exit•st—tus AY
return f — l s e Y
}
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 30 / 41

32. The DSP/BIOS Bridge
Sequence for nodes controlling
Delete DSP nodes
i f @ 3 dsp•node•free @ dsp•h—ndle D node A A {
p r • e r r @ 4 dsp node f r e e f — i l e d 4 A Y
return f — l s e Y
}
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 31 / 41

33. The ingredients
The kernel
DSP/BIOS Bridge driver is not in Linus' branch yet
Neither in linux-omap's main branch
httpXGGgitFkernelForgGcpalinuxGkernelGgitGtmlindGlinuxEom—pE
PFTFgitY—ashortlogYharefsGhe—dsGdsp˜ridge
httpXGGdevFom—pzoomForgGcpatidsp˜ridgeGkernelE
dsp˜ridgeFgitY—ashortlogYharefsGhe—dsGdsp˜ridge
httpXGGgitoriousForgG£
felipe™GlinuxEom—pGfelipe™
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 32 / 41

34. The ingredients
The GPP libraries
TI dbapi
httpXGGdevFom—pzoomForgGcpatidsp˜ridgeGusersp—™eE
dsp˜ridgeFgitY—asumm—ry
dsp_bridge
httpXGGgithu˜F™omGfelipe™GgstEdsp
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 33 / 41

35. The ingredients
Applications
Samples
httpXGGgithu˜F™omGfelipe™GdspEdummy
httpXGGgitoriousForgGvj—quezE˜e—gle˜o—rdGdspEs—mples
Applications
httpXGGm—emoFgitoriousForgGm—emoEmultimedi—GdspEtools
httpXGGgithu˜F™omGfelipe™GgstEdsp
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 34 / 41

36. The ingredients
Socket Nodes
Samples
httpXGGdevFom—pzoomForgGcpatidsp˜ridgeGusersp—™eE
dsp˜ridgeFgitY—asumm—ry
Multimedia
€—rt of ypenweˆ
httpsXGGgforgeFtiF™omGgfGproje™tGopenm—xGfrsG
httpXGG™odeFentropyw—veF™omGgitcpaleonor—FgitY—atree
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 35 / 41

37. The ingredients
C64x+ toolchain
Free as beer
Compiler
httpsXGGwwwE
—FtiF™omGdownlo—dsGsds•supportG„sgodegener—tion„oolsGdownlo—dFhtm
DSP/BIOS (libraries)
httpXGGsoftw—reE
dlFtiF™omGdspsGdsps•registered•swGsdo•s˜Gt—rget™ontentG˜iosGindexFhtm
dobuild tools
€—rt of the usersp—™eEdsp˜ridge p—™k—ge
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 36 / 41

38. The ingredients
All together
Marmita
yi re™ipes overl—y
st is — work in progress
ynly tested in the fe—gle˜o—rd @rev fTA
winim—l im—ge @IHw˜A
httpXGGgitoriousForgGvj—quezE˜e—gle˜o—rdGm—rmit—
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 37 / 41

39. The ingredients
Marmita
It's based on Angstrom distribution
Provides recipes for
felipe™9s kernel PFTFQP
DSS2
dspbridge
gstEdsp
dspEtools
dspEs—mples
li˜˜ridge @dsp˜ridge e€sA
li˜omxEti
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 38 / 41

40. Future
OMAP4
DSP/BIOS Bridge will be deprecated :(
syslink is the new thing
ARM M3 1GHz dual core (Ducati)
DSP TMS320C64x (Tesla)
ARM A9 1GHz dual core
http://dev.omapzoom.org/?p=tisyslink/kernel-syslink.git;a=summary
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 39 / 41

41. Future
The trend
More cores
More processing units
More heterogeneity
MORE COMPLEXITY
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 40 / 41

42. Future
Thank you
Questions?
Víctor Manuel Jáquez Leal (Igalia) The DSP/BIOS Bridge 06 February 2010 41 / 41