Building systems with Kamaelia

Kamaelia:
SnapTogether Software

Michael Sparks
Senior Research Engineer
BBC Research & Innovation
Kamaelia Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/

BBC R&I ...

            PAL, Teletext, NICAM,
  DVB,  DAB, Freeview, TV Anytime
Majority of BBC Open Source Projects
      inc Dirac, Kamaelia


Key Research
goal?

To make concurrency

easy to work with


How???
Single Reader/Single Writer
Communicating Sequential Things
Original context: scalable servers

Component Composition
Written in Python
Ideas are language agnostic
Proof of concept in C++


How???

No, I don't believe in “one

language to rule them all”
*cough*Erlang*cough*


(demo)


ie this:


What I'm aiming to
get through

P2P Radio
What?
System

Walkthrough Q&A


What?


What if... ?


What has it been
used for?


Who's used it?

Radio & Music
Interactive


Who's used it?
Matt Biddulph


Who's used it?
Matt Biddulph
Open Days “this is what I need”


Who's used it?
Matt Biddulph
Record All Radio PVR


Who's used it?
Matt Biddulph
Podcasts


Who's used it?
Matt Biddulph
Podcasts
Proof of concept


Who's used it?
Matt Biddulph
Podcasts
Proof of concept
Prototyping for experimentation
for new services


Who's used it?
Prototyping for
experimentation for new
services

... without any involvement
from R&D


Who's used it?
Prototyping for
experimentation for new
services

... without any involvement
from R&D
due to being open source


Who's used it?

BBC Macro


Blog frontend

Michael Sparks, michaels@rd.bbc.co.uk, http://kamaelia.sf.net/
& Meta Store
Programme
bbc one transcoder
Kamaelia Backend

bbc two transcoder
DEMUXER

bbc three transcoder
TUNER

bbc four transcoder
cbbc transcoder
cbeebies transcoder

Kamaelia
news 24 transcoder
parliament transcoder

Who's used it?

Rapid Prototyping
of Collaborative
Community Radio


Who's used it? DJs in the
Channel Controller
Community

Control

Streaming Networ
Server k Mixer

Playout Web Front End

Linked by
Music Store
videoconf


Google's Summer of
Code Student Work
2006, 2007


Bit Torrent 2006
Programmes...
WEBSITE
Ingest
BitTorrent Metadata
Split
Store
Torrent Maker
Web Client
Torrent Peer

Swarm Store
Torrent Peer Access
Programmes...


Open GL 2006


2006
Trusted Communications
eg secure phone Decrypt
ie trusted by the user Audio Decode
Encrypt

Decrypt
BUFFER
Encrypt

Decrypt
AudioPlayer
Encrypt


2007
Filelike Interface
> (Much!) simpler usage outside
Kamaelia systems

IRC/AIM Components
> causes creation of components
> allows remote control of systems

Visual Component Generation
> Sub component model
> aim:easier creation of components
visually (rather than systems)

Other Systems
Reframing for mobile, remuxing DVBT
interactive aps for multicast streams,
Whiteboarding, VoIP testing, etc.


Walkthrough


Simple Streamer

Pipeline(
ReadFileAdaptor( filename = 'video.drc',
bitrate = 400000 ),
SingleServer( ),
).activate()

Pipeline(
TCPClient( host = quot;127.0.0.1quot;, port = 1601 ),
DiracDecoder( ),
MessageRateLimit( messages_per_second = 15,
buffer = 15 ),
VideoOverlay( ),
).run()


Simple Presentation Tool

Graphline(
NEXT = Button(caption=quot;Nextquot;, msg=quot;NEXTquot;, position=(72,8)),
PREV = Button(caption=quot;Previousquot;, msg=quot;PREVquot;, position=(8,8)),
FIRST = Button(caption=quot;Firstquot;, msg=quot;FIRSTquot;, position=(256,8)),
LAST = Button(caption=quot;Lastquot;, msg=quot;LASTquot;, position=(320,8)),

CHOOSER = Chooser(items = files),

IMAGE = Image(size=(800,600), position=(8,48)),
linkages = {
(quot;NEXTquot;,quot;outboxquot;) : (quot;CHOOSERquot;,quot;inboxquot;),
(quot;PREVquot;,quot;outboxquot;) : (quot;CHOOSERquot;,quot;inboxquot;),
(quot;FIRSTquot;,quot;outboxquot;) : (quot;CHOOSERquot;,quot;inboxquot;),
(quot;LASTquot;,quot;outboxquot;) : (quot;CHOOSERquot;,quot;inboxquot;),
(quot;CHOOSERquot;,quot;outboxquot;) : (quot;IMAGEquot;,quot;inboxquot;),
}
).run()


Diving Inside?
from Axon.ThreadedComponent import
threadedcomponent

class ConsoleReader(threadedcomponent):
def main(self):
eol = quot;nquot;
while 1:
# this blocks so we use a thread
line = raw_input(quot;>>> quot;)
line = line + eol
self.send(line, quot;outboxquot;)


Diving Inside?
from Axon.Component import component

class ConsoleEchoer(component):
def main(self):
while 1:
while self.dataReady(quot;inboxquot;):
data = self.recv(quot;inboxquot;)
_sys.stdout.write(str(data))
_sys.stdout.flush()

yield 1


Used thus...
Pipeline( ConsoleReader(),
ConsoleEchoer(),
).run()


Better Streamer

def UserProtocol():
return ReadFileAdaptor( filename = 'video.drc',
bitrate = 400000 ),

SimpleServer( protocol=UserProtocol,
port=1601 ).activate()

Pipeline(
DiracDecoder( ),
MessageRateLimit( messages_per_second = 15,
buffer = 15 ),
VideoOverlay( ),
).run()


Minimal Radio P2P Core

Backplane(“RADIO”).activate()

Pipeline(
PublishTo( “RADIO” ),
).activate()

Pipeline(
SubscribeTo(“RADIO”),
UnixProcess(“mplayer - “),
).activate()

def UserProtocol():
return SubscribeTo( “RADIO” ),

SimpleServer( protocol=UserProtocol,
port=1601 ).run()


Swarming P2P idea


Swarming P2P setup

class SimpleSwarm(Axon.Component.component):
clients = []
rr = 0
maxclients = 4
def __init__(self, port):
super(SimpleSwarm, self).__init__()
self.__class__.port = port


Swarming P2P server

def main(self):
while 1:
while self.dataReady(quot;inboxquot;):
data = self.recv(quot;inboxquot;)
req = data.split(quot; quot;)
if req[0] == quot;REQCONNECTquot;:
if len(self.clients) < self.maxclients:
reqfrom = req[1]
self.clients.append(reqfrom)
self.send( quot;CONNECT quot; + str(self.port) ,
quot;outboxquot; )
else:
reqconn = self.clients[self.__class__.rr]
self.__class__.rr = (self.__class__.rr+1) %
self.maxclients
self.send( quot;REQCONNECT quot; + reqconn )
if not self.anyReady():
self.pause()
yield 1


Swarming P2P client

@staticmethod
def clientRequest(rootip, rootport, myip, myport):
servip, servport = rootip, rootport
port = 0
while port == 0:
l = LikeFile(TCPClient(servip, servport))
l.activate()
l.send(quot;REQCONNECT %s:%dquot; % (myip, myport))
resp_raw = l.recv()
resp = resp_raw.split(quot; quot;)
if resp[0] == quot;CONNECTquot;:
port = int(resp[1])
ip = servip
elif resp[0] == quot;REQCONNECTquot;:
servip, servport = resp[1].split(quot;:quot;)
servport = int(servport)
return servip, port


P2P Radio – Client (was)

Pipeline(
PublishTo( “RADIO” ),
).activate()


Swarming Radio – Client

Graphline(CONFIGURE= ConnectToSwarm(rootip, rootcontrolport,
myip, mycontrolport),
CLIENT = Carousel(mkTCPClient),
PUBLISHTO = PublishTo(quot;RADIOquot;),
linkages = {
(quot;CONFIGUREquot;,quot;outboxquot;):(quot;CLIENTquot;, quot;nextquot;),
(quot;CLIENTquot;, quot;outboxquot;) : (quot;PUBLISHTOquot;, quot;inboxquot;),
}
).activate()


Swarming Radio – Helpers

class ConnectToSwarm(Axon.Component.component):
def __init__(self, rootip, rootport, myip, myport):
super(ConnectToSwarm, self).__init__()
self.rootip = rootip
self.rootport = rootport
self.myip = myip
self.myport = myport

def main(self):
servip, servport = SimpleSwarm.clientRequest(
self.rootip, self.rootport,
self.myip, self.myport)
yield 1
self.send( ( servip, servport ), quot;outboxquot;)
self.pause()
yield 1

def mkTCPClient(args): return TCPClient(*args)


Swarming Radio – Source

freq = 682.166670 # Sutton Coldfields
feparams = {
quot;inversionquot; : dvb3.frontend.INVERSION_AUTO,
quot;constellationquot; : dvb3.frontend.QAM_16,
quot;coderate_HPquot; : dvb3.frontend.FEC_3_4,
quot;coderate_LPquot; : dvb3.frontend.FEC_3_4,
}

Pipeline(
DVB_Multiplex(freq, [6210], feparams), # RADIO ONE
PublishTo(quot;RADIOquot;),
).activate()


Kamaelia is already a highly
capable platform that has
delivered some interesting
tools

Currently version 0.5.0
Version reflects where we want to be


However it also shows that
you can make
concurrency easier to work
with, even in a normal
language


You can build your own
version if you follow our
“mini axon” tutorial using
your language of choice


Suggestions:
Take it, use it
Build cool stuff with it
Break it, Fix it
Steal the ideas
Clone in your own
language

Thanks for listening :)

http://kamaelia.sf.net/
michael.sparks@bbc.co.uk


Building systems with Kamaelia

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (7)

Recently uploaded

Recently uploaded (20)

Building systems with Kamaelia