TeleoR is a major extension of Nilsson’s Teleo-Reactive (TR) rule based robotic agent programming language. Programs comprise sequences of guarded action rules grouped into parameterised procedures. The guards are deductive queries to a set of rapidly changing percept and other dynamic facts in the agent’s Belief Store. The actions are either tuples of primitive actions for external robotic resources, to be executed in parallel, or a single call to a TeleoR procedure, which can be a recursive call. The guards form a sub-goal tree routed at the guard of the first rule. When partially instantiated by the arguments of some call, this guard is the goal of the call. TeleoR extends TR in being typed and higher order, with extra forms of rules that allow finer control over sub-goal achieving task behaviour. Its Belief Store inference language is a higher order logic+function rule language, QuLog. QuLog also has action rules and primitive actions for updating the Belief Store and sending messages. The action of a TeleoR rule may be a combination of the action of a TR rule and a sequence of QuLog actions. TeleoR’s most important extension of TR is the concept of task atomic procedures, some arguments of which belong to a special but application specific resource type. This allows the high level programming of multitasking agents using multiple robotic resources. When two or more tasks need to use overlapping resources their use is alternated between task atomic calls in each task, in such a way that there is no interference, deadlock or task starvation. This multi-task programming is illustrated by giving the essentials of a program for an agent controlling two robotic arms in multiple block tower assembly tasks. It has been used to control both a Python interactive graphical simulation and a Baxter robot building real block towers, in each case with help or hindrance from a human. The arms move in parallel whenever it can be done without risk of clashing.

1. Background slides for
invited challenge demo at Rule-ML 2017
A software agent controlling 2 robot arms in
co-operating concurrent tasks
Robotic agent programming in QuLog and TeleoR
Keith Clark
Imperial College & University of Queensland
Joint work with
Peter Robinson
University of Queensland
1

2. QuLog - LP+FP+AR Language
• Flexibly typed, multi-threaded, higher order
• Relation and function defining rules
• Function calls and set expression arguments in relation calls
• Function rule guards can query relations
• Relation rules more declarative than Prolog, closer to predicate
logic
• Relations are typed and moded
• Dynamic relations defined only by facts
• Used for the agent’s Belief Store
• Top layer of action rules
• Update dynamic relations, fork threads, do I/O, communicate with
other Qulog, C, Python or Java processes
• Inter-process comms uses our Pedro pub/sub and addressed
message router
• Threads execute actions calling funs and rels as needed
• Rel and fun rules cannot call actions
2

3. TeleoR
• Development of Nilsson’s T-R robotic agent language of
guard ~> action
rules sequenced in parameterized procedures.
• guard is a QuLog deductive query to the BS
• action is a tuple of robotic actions executed in parallel or
a T-R procedure call, maybe a recursive call
• TeleoR has forms of rules and actions not in T-R
– Extra rules for more fine grained behaviour control
– Can combine robotic action with QuLog action sequence:
action ++ qact1 ;...; qact1
– This allows communication and updates of BS as parallel non-robotic actions
• Compile time guarantee:
– all rule actions are correctly typed and fully instantiated when a rule is fired
• The concept of task atomic procedures:
– Used for multi-tasking sharing one or more robotic resources
– No interference, no starvation, deadlock free
– Compiler generates code that uses BS for co-ordination 3

4. Deductive Belief Store
Dynamic facts
Relation and function rules
?
Percepts
Handler
Message
Handler
ag@host
Percept fact
Interpretations of
Sensor data
Mid-level
Action control
messages
Incoming Messages
and replies
Control
Thread
?
Atomic re-evaluation
of rule guards after
each atomic update
Atomic
Updates
Multi-threaded Agent Architecture
Only outgoing Messages
4
pedro Other Agents
Pub/Sub and Addressed Message Router
TeleoR +
QuLog
QuLog

5. One arm block tower building
5

6. Task Ontology
6
def block::= 1..9
durative pickup(block), put_on_block(block), put_on_table()
percept on(block,block), on_table(block), holding(block)
rel sub_tower(list(block)), tower(list(block))
sub_tower([B]) <= on_table(B)
sub_tower([B1,B2,..Blocks]) <=
on(B1,B2) &
sub_tower([B2,..Blocks])
tower([B,..Blocks]) <=
not on(_,B) &
sub_tower([B,..Blocks])

7. Nilsson’s universal conditional top-
level plan for block tower building
7
tel makeTower(list(block))
makeTower(Blocks){
tower(Blocks) ~> () % Goal is achieved
sub_tower(Blocks) & Blocks=[B,..] ~> make_clear(B)
Blocks=[B] ~> move_to_table(B)
Blocks=[B1,B2,..Bs] & tower([B2,..Bs]) ~> move_to_block(B1,B2)
Blocks=[B,..Bs] ~> makeTower(Bs)
}
Action achieves

8. Mutually recursive procedures
8
tel make_clear(block)
make_clear(Block){
not on(_,Block) ~> () % Goal is achieved
on(OnBlock,Block) ~> move_to_table(OnBlock)
}
tel move_to_table(block)
move_to_table(OnBlock){
on_table(OnBlock) ~> () % Goal is achieved
holding(OnBlock) ~> put_on_table()
not on(_, OnBlock) & not holding(_) ~> pickup(OnBlock)
not holding(_) ~> make_clear(OnBlock)
holding(_) ~> put_on_table()
}

9. Interleaving the building of several
towers
9
Just need to add:
task_start makeTower
task_atomic move_to_table, move_to_block

10. Baxter concurrent tower building
10

11. TR
procedures
Sensor
data
Control
actions
for
different
robotic
resources
BeliefStore
Dynamic
Facts
Percepts
Handler
TR
Evaluatio
n ThreadsAll
incoming
messages
Message
Handler Task1
Using
R1,R2
Task2
Using
R3
Outgoing
messages
Multi-tasking architecture
Task3
Waiting for
R1,R3
Task4
Waiting for
R1,R5Fixed
Facts &
Rules
?

12. Elements of the two arm tower
builder program
12
def table ::= table1 | shared | table2
def block ::= ...
def arm ::= arm1 | arm2
def resource::= arm || table % resource a reserved type name
durative pickup(arm, block, table), put_on_block(arm, block, table),
put_on_table(arm, table)
tel makeTower(arm, list(block), table)
makeTower(Arm,Blks,Tbl){ % Tbl is where tower must be built,
tower(Blks, Tbl) ~> {}
sub_tower(Blks, Tbl) & Blocks=[B,..] ~>
makeClear(Arm, B, Tbl)
Blks=[B,.. Bs] & tower(Bs, Tbl) ~>
moveAcrossToBlock(Arm, B, firstOf(Bs), Tbl)
Blks=[B] ~> moveAcrossToTable(Arm, B, Tbl)
Blks=[B,.. Bs] ~> makeTower(Arm, Bs, Tbl)
}
Entire control program 40 TeleoR rules clustered into 6 procedures.

13. Auxiliary procedures
13
tel moveAcrossToTable(arm, block, table)
% Will fetch a block from wherever it is and put it onto the table
% May need two calls to proc below using different arms to first
% move the block to shared then to destination table
task_atomic oneArmMoveToTable(arm, block, table, table)
% Has 3 resource args. an arm and two tables which might be the same
oneArmMoveToTable(Arm, Blk, FromTab, ToTab){
on(Blk, ToTab) ~> ()
clear(Blk) ~> clearOneArmMoveToTable(Arm, Blk, FromTab, ToTab)
not holding(Arm,_) ~> makeClear(Arm, Blk, FromTab)
holding(Arm,_) ~> put_on_table(Arm, FromTab)
}

14. TeleoR semantics and
implementation
14
• Formal State Transition Semantics
• Optimized Reference Implementation
• Runtime system that implements state transition
semantics
• Compile time analysis ensures rule guards are not
re-evaluated on BS update if no relevant change made
• Currently compiled to multi-threaded Qu-Prolog
• Will soon be compiled to specialized Abstract Machine Code
similar to but simpler than Warren’s Prolog AMC

15. Sources and software
15
Clark & Robinson, Robotic Agent Programming in TeleoR, ICRA 2015, IEEE
Clark & Robinson, Multi-tasking Robotic Agent Programming in TeleoR,
Research Report, on www.doc.ic.ac.uk/~klc
Clark et al, A Framework for Integrating Symbolic and Sub-symbolic
Representations, IJCAI 2016, AAAI Press
Programming Communicating Multi-tasking Robotic Agents:
A Teleo-Reactive Rule Based Approach, Springer, Early 2018
first 5 chapters at teleoreactiveprograms.net
including a formal operational semantics of Nilsson’s TR lang.
Clark & Robinson, Engineering Agent Applications in QuLog, Springer, 2017/8,
TeleoR and QuLog Software for Unix, Linux and OS X at
http://staff.itee.uq.edu.au/pjr/HomePages/QulogHome.html
Collaboration and users welcomed