SlideShare une entreprise Scribd logo
1  sur  14
PhD Report
                            Relatório de Doutoramento


                                          AFRANCI: An Architecture
                                               for Learning Agents
                                                  Francisco Antonio Fernandes Reinaldo
                                                                  Aluno de doutorameto

                                                                 Rui Camacho; Luis. P. Reis
                                                                             Supervisores

                                                                  {reifeup, rcamacho, lpreis}@fe.up.pt




                             Laboratório de Inteligência Artificial e Ciência de Computadores – LIACC.
                            Departamento de Engenharia Electrotécnica e de Computadores – DEEC.
                                        Faculdade de Engenharia da Universidade do Porto – FEUP.
                                              Rua Dr. Roberto Frias s/n, 4200-465. Porto – Portugal.
                                                     Tel: + 351 22 508 1400 - Fax: +351 22 508 1443.

Reinaldo, F. A. F.; Camacho, R.; Reis, L. P. AFRANCI: An Architecture for Learning Agents. Artificial
                                                   Intelligence. PhD Report. FEUP: Porto, 2005. 14 p.

                                                       e-doc: http://www.fe.up.pt/~reifeup/phdr.pdf

                                                              FEUP e Porto, 25 de Setembro de 2005.
Presentation

This report summarises the work done is the first year of Francisco Reinaldo´s PhD. It is
also included in the report the cognitive architecture proposal that will be developed and
evaluated in the remaining years of the PhD. The proposed architecture is adequate for
Cognitive Agents acting in dynamic and unpredictable environments. The framework
includes Machine Learning techniques to make adaptable agents.


Reinaldo attended the 14th International Conference on Learning and Extraction Behaviour
in FEUP, last year. He is a collaborative member of the research project “FC Portugal:
Coordination of Heterogeneous Teams in Search and Rescue Scenarios”, with grant n.
POSI/EIA/63240/2004. This year, he has collaborated in the realization of II SIEBRAP –
Seminar of Brazilian Researchers in Portugal, Porto. He has presented a poster and a talk in
the NN2005 - Summer School on Neural Networks, ISEP, Porto. He has delivered a paper in
RoboCup Rescue 2005 in Osaka, Japan, where he also was a judge in the Robots with 4 legs
championship “Sony AIBO” in RoboCup Rescue 2005. He reviewed papers in MASTA'05 -
3rd Workshop on Multi-Agent Systems: Theory and Applications, Porto. He has an accepted
paper in EPIA 2005 IROBOT Workshop to the title “Applying Biological Paradigms to Emerge
Behaviour in RoboCup Rescue Team, Porto. In addition, other paper has an acceptance in
ESM´2005 to the title “A Tool for Fast Development of Modular and Hierarchic Neural
Network-based Systems. Besides that, he was invited lecturer in FEUP, where he taught
Algorithms and Data Structures in classroom (AED) in 2004/2005 school year. Reinaldo
helped Luís P. Reis with students in undergraduate final work at FEUP.
3


Abstract

In this report, we describe the work done in the first year of the PhD. We present a proposal
for general purpose architecture AFRANCI with hybrid learning capabilities to development
of heterogeneous agents. We argue that the use of Machine Learning techniques is of crucial
importance for agents to act alone, in community, or/and in dynamic environments. The
AFRANCI architecture addresses hierarchic behaviour levels, but will be empirically
evaluated in the RoboCup Rescue International. The challenge of RoboCup Rescue is to
promote competitive international research by providing cooperation opportunities, so that
researchers can evaluate their research results. We claim that agents with proper learning
capabilities have good performances in the simulated league.



Keywords

Cognitive Agent Architectures, RoboCup Rescue League, Machine Learning.
Table of Contents



I – Introduction....................................................................................................................5
   1.1 Objectives and Motivations.........................................................................................5
II – First Year Study.............................................................................................................6
   2.1 Agent Architectures ...................................................................................................6
   2.2 RoboCup Rescue Teams ............................................................................................6
   2.3 Learning Techniques .................................................................................................6
     2.3.1 Artificial Neural Networks ....................................................................................6
     2.3.2 ILP ......................................................................................................................7
III – The Project...................................................................................................................8
   3.1 Introduction...............................................................................................................8
   3.2 Development of the Architecture.................................................................................8
   3.3 A Framework and Tool for developing Autonomous Agents........................................10
IV – Work Plan for Next Years............................................................................................12
   4.1 RoboCup Rescue Project ..........................................................................................12
V - References...................................................................................................................14
5


I – Introduction


This PhD report presents a proposal Cognitive Agent architecture together with a framework
that allows the development of Cognitive Agents. The proposed architecture will be evaluated
through agent’s operations in RoboCup Rescue Simulation System. The rescue league is
composed of agents that have general functionalities but some particular characteristics
(personalities) that define their heterogeneity.

The AFRANCI architecture offers fast reasoning and social capabilities to autonomous
agents. The architecture is inspired by the main features of global organization of the
mammalian brain to simulate the biologic behaviour level of the Central Nervous System
(CNS). The CNS [1] is formed by subsequent clusters of cognitive processes that are
arranged in a hierarchical way to overlap actions. A cognitive process is a decision-making
route of beliefs or desires that triggers actions for emerging behaviours. Like a cortex
(neocortex) [2], this hierarchical organization allows increasingly complex actions/decisions.
Our architecture has special features that enable agents to reach a goal, establish control in
asynchronous activities, and solve existential conflicts between reactivity, deliberation and
social activities.

Moreover, a framework will be developed to implement the theoretic ideas. This framework
will offer some base-classes to develop an Artificial Central Nervous System (ACNS). An
ACNS is a computational model of a CNS. The ACNS is responsible for emerging a general
agent. In addition, a Tool will be developed over the framework. This tool will design an
agent’s structure and particular learning modules to assemble heterogeneous agents.

This research focuses three main global areas: a) Cognitive System, composed by Cognitive
Processes, Behaviour-Based Architecture and Emerging Social Behaviours in a Multi-agent
System (MAS); b) Software Engineering where a framework behaviour-based will be built; c)
Simulation System that works with RoboCup Rescue as a test bed system.


1.1 Objectives and Motivations

The work has two objectives: first we identify relevant features of cognitive Agents and
propose an architecture to accommodate them; second we develop a framework to enable
the customization of Agents and to train the necessary modules.

Our motivation is to offer flexibility in the development of agents. As the general architecture
is concerned, we may customize a Cognitive Architecture and train it to a particular
application. We will use the RoboCup Rescue League in our proposal.

The main motivations of this work are:
   • Development of an agent’s architecture that will be used as pattern for designing
      agents;
   • Contribution in the improvement of straightforward agent leaning techniques that
      are based on similarity of behavioural human being architecture;
   • Providing of opportunities for students to perform significant test bed in classroom
      through which they can learn many of the train techniques of multi-agent system
      development;
   • Comprising the following domains: social, technical and cognitive. Social
      environment is significant because it focuses on human search and rescue mission
      planning. Technical research is important because it solves problems of rescue
      strategies and tactics for a fast and optimized saving of people, and it also preserves
      infrastructures, as well. Cognitive science is considered because it studies models of
      information and representation, capacity of human memory and biologic behaviour
      to be applied in a Multi-agent system.
6


II – First Year Study


During the first year of the PhD we identified and analysed the main agent architectures
[3-17], RoboCup Rescue teams [18-22] and Learning Techniques to propose our
architecture.


2.1 Agent Architectures

The surveyed agent behaviour architectures are the following. Brooks [3] has developed a
reactive architecture with levels of subsumption. Guessoum and Briot [4] have developed a
modular and generic agent architecture in Smalltalk named DIMA platform (Development
and Implementation of Multi-agent System). Hoff and Bekey [5] have used connectionist
architecture for learning coordination of different behaviours in a situated agent. Ito [6] has
used Reinforcement Learning (RL) architecture to realize complex tasks in agents. Shinoda
[7] has proposed a Framework for designing behaviours. Mitchell [8] has proposed a Plan-
and-Compile architecture named Theo - this architecture uses a plastic knowledge base to
learn facts and meta-facts. Carbonell, Knoblock and Minton [9] have proposed a Modular
architecture that stores symbolic knowledge in a way of First-Order Predicate Logic (FOPL)
which is called the Prodigy Description Language (PDL). Roisenberg [10] has developed a
PyramidNet Architecture that uses a modular and hierarchic approach which are based on
heterogeneous neural nets to create complex and intelligent behaviours in robotic systems.
Duffy [11] has used a hierarchic architecture to design deliberative and reactive levels
without the use of the connectionist paradigm. Bowling [12], Duffy [13] and Haikonen [14]
have offered a general architecture to design a cognitive machine architecture for the agent.
Taheri and Sadati [15] have proposed a new modular category for agent navigation, using
Fuzzy Logic. Steenekamp and Ross [16] have used the symbolic paradigm to deliberate the
level of actions for agents. The higher level and reactive level are used in some “routine”
decisions that are some encoded rules in their rule base. Patra and Mohanty [17] have
proposed an architecture to build software agents. The agent is modelled on belief-desire-
intention architecture and is composed of perceptions, actions and goal-directed
behaviours.


2.2 RoboCup Rescue Teams

The well most known rescue teams use agents with advanced learning and coordination
capabilities. DAMAS Rescue team [18] concentrates its work on improving the agent’s ability
to extinguish buildings on fire. The ability consists in a classification of the best fire to
extinguish by the perception learning method. ResQ Freiburg [19] deals with a sequence of
planning methods. The objective is to build hierarchic commands that imply hierarchic
behaviours. Caspian [20] uses a three phase model approach to develop their agents: World
Model, Dynamic Path Finding and Decision Making. SOS team [21] develops a general-
purpose rescue agent for implementing high-level strategies and learning algorithms. The
SOS agent has a state-based architecture, an explicit state-selection and state switching
policy. It uses two path-finding modules that are D* and Dijkstra’s algorithm and two neural
networks to make decisions. FC Portugal Rescue Team [22] uses high-level coordination
methodologies, Configurable Flexible Team Strategy and Situation Based Strategic
Positioning to coordinate agents.


2.3 Learning Techniques

2.3.1 Artificial Neural Networks

An Artificial Neural Network (ANN) is a data processing paradigm inspired by biological
neural networks. An ANN has the ability to learn from examples, which gives it a
7


considerable power and flexibility. ANN is composed of several interconnected neurons that
perform excitatory and inhibitory connections to adjust their synaptic weights between
neurons for solving a problem and to produce the correct output. The massive parallelism of
neurons in an ANN structure can offer data compression by storing the information in the
weights on the connections. The weights are supposed to adjust during the training stage
where examples are showed to the network. Depending on the network type and its
topology, an ANN has the capacity to obtain meaning from complex or imprecise inputs and
adjust to any situations or even to model a complex decision system.

The biologic nervous system may be structured with sufficient precision to obtain
behaviours emerging in a computational model. ANNs do not require a formalised algorithm
to achieve a goal. A suboptimal result is most necessary to solve huge problems instead of
optimal results because a final result is unclear in some cases. Depending on the ANN
topology we can process strings of symbols as a formal system that is interpreted by
humans as concepts but with supports for noisy or imprecise data. Similar to the brain,
ANN are robust in the presence of noise: small changes in an input signal will not
drastically affect a node's output. Nets are robust in the presence of failure: a change in a
weight may only affect the output for few of the possible input patterns. The net can deal
with “unseen” patterns and generalise from the training set. Nets are good at “perceptual”
tasks and associative recall. These are just the tasks that the symbolic approach has
difficulties in dealing with.

2.3.2 ILP

Inductive Logic Programming (ILP) has achieved considerable success in a wide range of
domains. These domains include the Life Sciences, Molecular Biology and Engineering. We
believe that ILP can be successfully applied to the synthesis of some conscious behaviour.

A typical ILP system constructs models from data by making a search through an ordered
hypothesis space (the models). An ILP system accepts examples as input, together with
relevant information (the background knowledge) and induces a model that “explains” the
examples given the background knowledge. The great advantage of ILP is the fact that it
uses a very powerful representation language: First Order Logic. This feature enables ILP to
induce very complex models. Another relevant feature of an ILP system concerns the ability
to combine very easily symbolic computations with numerical ones.
8


III – The Project


This proposal presents a modular and hierarchic behaviour-based architecture to be applied
in a multi-agent system. This architecture is composed by heterogeneous learning modules
arranged in behaviour levels. AFRANCI is an acronym of “Architecture For Rescue AgeNts
with Cognitive Insights”.


3.1 Introduction

We are using both the symbolic and the connectionist paradigms to develop AFRANCI
architecture. The idea of joining both research areas appeared in McCulloch´s work, which
had a strong biologic inspiration and was made in conformity with Pitts´s work using
mathematical concepts [23]. In the same direction, in order to construct hybrid systems, we
are capable of combining the characteristics of adaptability, robustness and uniformity that
are offered by neural networks with representation, inference and universality, native
characteristics of symbolic Artificial Intelligence [24].


3.2 Development of the Architecture

This general architecture with extended characteristics was inspired in the main parts of the
human being´s nervous structure. These parts are necessary to construct a general
computational agent. They are responsible for emerging behaviours. The biologic inspiration
of this work had origin in analysis of structure and functions of the brain and other nervous
areas of the body.

AFRANCI architecture emerged from the user’s difficulty to design agents using the same
agent body. The difficulty consists in using built-in characteristics to perform particular
tasks in a multi-agent system. The use of different approaches, presented in topic 2.1 and
2.2, gives us examples of how difficult it is to decide for a construction of an agent.

In order to develop an agent architecture, the first step is to restrict a behaviour module
with its class of problems. In our work, AFRANCI architecture organises the modules as a
collection of autonomous computational elements. In horizontal line, there are
homogeneous modules arranged in the same topology. In vertical line, there are
heterogeneous modules arranged in different kinds of complexity from simple to complex.

The AFRANCI structure supports a great variety of behaviours without decreasing of
performance. Using a top-down topology, it is possible to divide levels of complexity into
incremental functionality. The main idea focus on a quantity of cases where behaviour
modules are being used to run processes in different periods or a same process will be
performed for many modules in the “same time” in distinct and functional areas of the brain
[25]. Levels that are more abstract provide increasing complexity and subsume lower level
functionalities, Figure 1.
9




                     Figure 1. The analogy between a natural and computational system.

Figure 1 presents an analogy between a human and a computational agent. On the left of
the figure are represented the main abstract parts of the nervous system of a human being,
mapped on the right.

Continuing this analogy, the horizontal behaviour modules respect a hierarchy. The vertical
communication between heterogeneous levels happens from internal connections as well.
Each level is linked with classes of problems, which are capable of solving them. AFRANCI
architecture is arranged from lower level to higher level as stereotyped (reflexive and taxies),
reactive, instinctive and deliberative behaviour levels. Moreover, we have localized a social
module in the top level of the deliberative behavioural structure. The social module is
proposed to share learned experiences among agents. The differential of this architecture is
to support the management of different actions to achieve fast and optimized collaborative
outcomes.

Inside the architecture, each vertical level has actions that make connections with certain
stimulus or decisions, such as generating autonomy, reactivity, reasoning and social ability.
Local behaviours emerge in local modules that are arranged in the first behaviour level.
General behaviours emerge through other parts in the architecture. Social behaviour is
10


inside the top behaviour level which is sharing the reasoning space. The whole architecture
is involved in the learning and behaviour process.

The main advantages can be seen in both high and low parts of the architecture and the
topology. The low level architecture offers resources in order to facilitate a fast response.
This level has behaviour implemented by reflexive or/and reactive modules that are similar
to the human being´s biologic connection. With this approach, any adaptation state is
achieved in a dynamic environment. Otherwise, the use of advanced perception techniques
brings high performance in task development as well. In high level of architecture, agents
can explore the world and construct a mental map of environment by using mapping
techniques. In this case, every agent will contribute to feed the shared database. Using a
shared database, time exploration is optimized and a great reduction in size of the world
space is achieved. Problems such as solving an unknown but similar problem are solved as
well, because the architecture uses previously collected information in the database to feed
the making decision algorithms. Moreover, this topology solves existential conflicts between
reactivity, deliberation and social activity. In addition, a modular architecture has had good
acceptance because it produces good performance through separated but linked functional
components within the horizontal design of a system. Thus, autonomy and robustness
achieved to deal with large quantities of learning techniques with high performance because
of the use of a hierarchic and modular architecture.

The learning process is simple and can occur in two steps. Firstly, agents’ sensors collect
data from the environment and set them in a database. This database will feed the learning
algorithms arranged in low level. If the task can be solved in low level, then the action will
be performed; otherwise the information learned in low level is sent to high level. Secondly,
the agents´ decision is made through the use of Reinforcement Learning or Inductive Logic
Programming in high level to make decisions based on information sent by low level. For
Instance, these decisions can be planning or social behaviour. After finding a solution, the
behaviour is subsumed from high level to low level until it reaches the agent’s actuators.

This architecture enables users to develop any agent because they use a general
architecture composed by all behaviour levels. Using a framework to fashion the whole
architecture, the user can handle behaviour modules as blocks.


3.3 A Framework and Tool for developing Autonomous Agents

In order to deploy the architecture, we are converting it into a framework. A Framework is
an extensible set of base-classes (skeleton) to develop tools. These base-classes are concrete
or abstract classes that provide a collaborative pattern between instances earlier defined by
the framework. A Tool is an artefact that was extended by configuration or linking of
concrete classes and derivation of new concrete classes from abstract classes of the
framework.

AFRANCI Framework is the implemental part of AFRANCI Architecture. The framework
comes to reduce classic learning algorithms, to optimize performance and to extend current
interpretations of agent systems, behaviour-based architecture and agent-based techniques.

The framework skeleton will be built to extend application tools for the development of
agents. In order to produce rescue agents with specific functionalities, a tool will be
extended under this framework, Figure 2.
11




                                    Figure 2. Framework and Tool

The whole figure above is a complex structure that determines the tool. It is possible to see
two parts in Figure 2. The splotchy area shows a framework structure with its classes and
connections ready to be extended or shared. In the other part, below the splotchy area,
there are complements of classes that characterise the tool when it is added to the structure
of the framework. The use of the framework is necessary to produce a particular tool
because it reduces the quantity of line-code to be developed, tested and cleaned.

AFRANCI Tool will be robust to design agents’ structure. It will be able to design an Artificial
Central Nervous System (ACNS), explained previously in Section I. More specifically, users
will design specific agents’ structure by using general modules added to particular modules
form which “the personality” of the agent will emerge. The tool will make use of drag-and-
drop graphic blocks, as a puzzle toy, and automated generation opened-code to apply in
agents for performing tasks. The tasks can be actions for mitigation disasters, such as:
search and rescue of citizens who were getting buried under the debris; set free the blocked
roads and railways to allow rescue teams traffic; wounded refuge buildings; control and
extinguish fires; communication and synchronized coordination between platoon agents and
centre agents. We believe that using simple graphic interfaces and computational elements
is the best way to create hybrid levels and to gain complex behaviours. After finding each
module in the architecture as a simple puzzle toy, an automated opened-code is generated.
This process of production of agent’s code is very optimized and clean.
12


IV – Work Plan for Next Years

We have addressed methodologies to focus the development of a general agent. Next year we
will implement a general agent based on our architecture. Maybe adjusts will be necessary.
After this, we will construct rescue agents as our testbed.



4.1 RoboCup Rescue Project

The RoboCup Rescue is an international project that was started in 1999 to solve relevant
social problems by integration of disaster information, prediction, planning and training for
rescue actions. Built upon the success of RoboCup Soccer project, it offers a comprehensive
urban disaster simulator, forums of technical discussions and competitive evaluations for
researchers and practitioners.

The RoboCup Rescue Simulation League is an agent team formation that focuses on:
rescuing injured citizens and decreasing the damage of the collapsed town. In order to
perform these tasks, teammates use complex high-level plans, adjust of heterogeneous
behaviours and cooperation among them. Every year a competition is organized where
researchers meet, compare approaches and exchange ideas.

The RoboCup Rescue Simulator is divided by modules that can run in several computers.
These modules are independent to run particular characteristics of the world, as can be
seen in Figure 3. The modules also represent simulation domains as earthquake, buildings
on fire or collapsing, spreading of fire to neighbouring buildings, blocked ways and health
state of citizens.




                             Figure 3. RoboCup Rescue Simulation System.

In accordance with Figure 3, the main components that comprise the simulator are [26]:
  • Geographic Information System (GIS): the GIS module holds the state of the
      simulated world;
  • Kernel: this module is connected to all the other modules. The kernel updates the
      objects in the GIS and sends the world updated to every connected module.
  • Sub-simulators: Fire-simulator, Collapse-simulator, Traffic-simulator etc. are
      modules connected to the kernel, each one simulating a particular disaster feature –
      fire, collapses, traffic etc.
  • Agents: agent modules are connected to the kernel and represent intelligent entities in
      the real world, such as civilians, polices agents, fire agents etc.
  • Viewers: their task is to get the state of the world, communicating with the kernel
      module, and graphically displaying it, allowing the user to easily follow the simulation
      process.
13


The simulator presents a world model with heterogeneous agents, Figure 4.




                                    Figure 4. Rescue Scenario

The RoboCup Rescue scenario, Figure 4, includes approximately 40 agents in six different
types to preserve the town and citizens´ life: Fire Brigade agents, Police Force agents,
Ambulance agents and three different Control Centres. Centre agents coordinate platoons
with collaborative tasks that are based on information from the external world received by
mobile agents. These centre agents send/receive several messages of the world state to/from
its mobile agents. The updated messages are sent to the rescue agents. Platoons have
autonomy to learn and to create the best decision about unexpected states. Police agents
have the responsibility of clearing priority blocked ways for extreme traffic and to report to
base the situation of a citizen crying for help. The Police Force has 10 to 15 agents.
Ambulance agents search wounded buried citizens to transport them to refuges. Ambulance
Centre has 5 to 8 agents. Fire Brigade agents extinguish fires in buildings. The number of
brigade is between 10 and 15 agents. Each centre is responsible for its team. All tasks
performed by mobile agents are centred to save people’s life.
14


V - References


1.    Reinaldo, F.A.F., Projecting a framework and programming a system for development of modular and
      heterogeneous artificial neural networks., in Departament of Computer Science. 2003, Federal University of Santa
      Catarina: Florianópolis. p. 86.
2.    Kolb, B. and I.Q. Whishaw, An Introduction to Brain and Behavior. 2nd ed. 2005, New York: Worth Publishers
      Inc. 601.
3.    Brooks, R.A., A Robust Layered Control System For a Mobile Robot. Technical Report, in AI memo 864. 1985,
      Massachusetts Institute of Technology: Massachusetts, CA. p. 25.
4.    Guessoum, Z. and J.-P. Briot, From active objects to autonomous agents. Concurrency, IEEE [see also IEEE
      Parallel & Distributed Technology], 1999. 7(3): p. 68-76.
5.    Hoff, J. and G. Bekey. Architecture for behavior coordination learning. in Proceedings of the 1995 IEEE
      International Conference on Neural Networks. 1995: IEEE.
6.    Ito, K., et al. A study of reinforcement learning with knowledge sharing for distributed autonomous system. in
      Computational Intelligence in Robotics and Automation, 2003. Proceedings. 2003 IEEE International Symposium
      on. 2003.
7.    Shinoda, K., I. Noda, and M. Ohta. Behavior design of civilian agent for rescue simulation. in Proc. of Challenges
      in Open Agent Systems. AAMAS02. 2002, 7.
8.    Mitchell, T.M., et al., Theo: A framekwork for self-improving systems, in Architectures for Intelligence, i.K.
      VanLehn, Editor. 1991, Lawrence Erlbaum Associates: Hillsdale, New Jersey. p. 323-355.
9.    Carbonell, J., et al., PRODIGY: an integrated architecture for planning and learning. SIGART Bull., 1991. 2(4): p.
      51-55.
10.   Roisenberg, M., Emergency of the intelligence in autonomous agents through inspired models in the nature, in
      Department of Electric Engineering. Group of Researches in Biomedical Engineering. 1998, Federal University of
      Santa Catarina: Florianópolis. p. 206.
11.   Duffy, B.R., The Social Robot, in Department of Computer Science. 2000, University College Dublin (UCD):
      Dublin, Ireland.
12.   Bowling, M., Robocup rescue: Agent development kit. 2000, Avaliable at official RoboCup Rescue Web Site.
13.   Duffy, B.R., et al. Agent chameleons: agent minds and bodies. in Computer Animation and Social Agents, 2003.
      16th International Conference on. 2003.
14.   Haikonen, P.O.A. A modular neural system for machine cognition. in Neural Networks, 2000. IJCNN 2000,
      Proceedings of the IEEE-INNS-ENNS International Joint Conference on. 2000.
15.   Taheri, J. and N. Sadati. A fully modular online controller for robot navigation in static and dynamic environments.
      in Computational Intelligence in Robotics and Automation, 2003. Proceedings. 2003 IEEE International
      Symposium on. 2003.
16.   Steenekamp, P. and J. Roos. Implementation of distributed systems management policies: a framework for the
      application of intelligent agent technology. in Systems Management, 1996., Proceedings of IEEE Second
      International Workshop on. 1996.
17.   Patra, M.R. and H. Mohanty. A formal framework to build software agents. in Software Engineering Conference,
      2001. APSEC 2001. Eighth Asia-Pacific. 2001.
18.   Paquet, S., N. Bernier, and B. Chaib-draa, DAMAS-Rescue Description Paper. 2004. p. 12.
19.   Kleiner, A. and M. Brenner, ResQ Freiburg: Deliberative Limitation of Damage. 2004.
20.   Hamraz, S.H., S.S. Feyzabadi, and A.K. Motlagh, Caspian RoboCup Rescue Simulation Agent: Team Description.
      2005.
21.   Amraii, S.A., et al., S.O.S. 2004: an attempt towards a multi-agent rescue team. 2004, Amirkabir University of
      Tecnology: Tehran, Iran. p. 12.
22.   Lau, N., L.P. Reis, and F. Reinaldo, FC Portugal 2005 Rescue Team Description: Adapting Simulated Soccer
      Coordination Methodologies to the Search and Rescue Domain. 2005.
23.   McCulloch, W.S. and W. Pitts, A locical calculus of the ideas immanent in nervous activity. Bulletin Of
      Mathematical Biophysics. Vol. 5. 1943. 115-133.
24.   Waltz. Neural nets and AI: Time for a synthesis. in International Conference on Neural Networks. 1997. San
      Diego, CA.
25.   Eccles, J.C., The Cerebral Neocortex: A Theory of its Operation., in Cerebral Cortex, E.G.J.A. Peters, Editor.
      1984, Plenum Press: New York. p. 1-36.
26.   d'Agostino, F., et al. Monitoring and information fusion for search and rescue operations in large-scale disasters.
      in Information Fusion, 2002. Proceedings of the Fifth International Conference on. 2002.

Contenu connexe

Similaire à ph-report.doc

Workshop World-Class CI Function
Workshop  World-Class CI FunctionWorkshop  World-Class CI Function
Workshop World-Class CI Functionminiera
 
Industrial neuroscience in aviation evaluation of mental states in aviation ...
Industrial neuroscience in aviation  evaluation of mental states in aviation ...Industrial neuroscience in aviation  evaluation of mental states in aviation ...
Industrial neuroscience in aviation evaluation of mental states in aviation ...BatDeegii
 
Advanced Robotics Projects For Undergraduate Students
Advanced Robotics Projects For Undergraduate StudentsAdvanced Robotics Projects For Undergraduate Students
Advanced Robotics Projects For Undergraduate StudentsEmily Smith
 
Herramientas y técnicas para la Gestión del Conocimiento Nuclear
Herramientas y técnicas para la Gestión del Conocimiento NuclearHerramientas y técnicas para la Gestión del Conocimiento Nuclear
Herramientas y técnicas para la Gestión del Conocimiento NuclearEduardo Medina Gironzini
 
Organisational Interoperability in Practice at Universidad Politécnica de Madrid
Organisational Interoperability in Practice at Universidad Politécnica de MadridOrganisational Interoperability in Practice at Universidad Politécnica de Madrid
Organisational Interoperability in Practice at Universidad Politécnica de MadridOscar Corcho
 
Building repositories and increasing usage.
Building repositories  and increasing usage.Building repositories  and increasing usage.
Building repositories and increasing usage.Iryna Kuchma
 
Before you start: Strategic Planning and Business Cases
Before you start: Strategic Planning and Business CasesBefore you start: Strategic Planning and Business Cases
Before you start: Strategic Planning and Business CasesIryna Kuchma
 
Building repositories
Building repositories Building repositories
Building repositories Iryna Kuchma
 
Overview of the Maker Movement in the EU
Overview of the Maker Movement in the EU Overview of the Maker Movement in the EU
Overview of the Maker Movement in the EU Antonio Arcadu
 
Keepit Course 5: Concluding the course
Keepit Course 5: Concluding the courseKeepit Course 5: Concluding the course
Keepit Course 5: Concluding the courseJISC KeepIt project
 
Ontological Infrastructure for Interoperable Research Information Systems: HE...
Ontological Infrastructure for Interoperable Research Information Systems: HE...Ontological Infrastructure for Interoperable Research Information Systems: HE...
Ontological Infrastructure for Interoperable Research Information Systems: HE...Diego López-de-Ipiña González-de-Artaza
 
OpenACC Monthly Highlights: January 2024
OpenACC Monthly Highlights: January 2024OpenACC Monthly Highlights: January 2024
OpenACC Monthly Highlights: January 2024OpenACC
 
FoCAS Newsletter Issue 4: August 2014
 FoCAS Newsletter Issue 4: August 2014 FoCAS Newsletter Issue 4: August 2014
FoCAS Newsletter Issue 4: August 2014FoCAS Initiative
 
Articulo cientifico ijaerv13n12_05
Articulo cientifico ijaerv13n12_05Articulo cientifico ijaerv13n12_05
Articulo cientifico ijaerv13n12_05Nombre Apellidos
 
The LEGO Maturity & Capability Model Approach
The LEGO Maturity & Capability Model ApproachThe LEGO Maturity & Capability Model Approach
The LEGO Maturity & Capability Model ApproachLuigi Buglione
 
AbstractsBook_GCACSE2016 (3) (1)
AbstractsBook_GCACSE2016 (3) (1)AbstractsBook_GCACSE2016 (3) (1)
AbstractsBook_GCACSE2016 (3) (1)Nilay Salt
 

Similaire à ph-report.doc (20)

Workshop World-Class CI Function
Workshop  World-Class CI FunctionWorkshop  World-Class CI Function
Workshop World-Class CI Function
 
Industrial neuroscience in aviation evaluation of mental states in aviation ...
Industrial neuroscience in aviation  evaluation of mental states in aviation ...Industrial neuroscience in aviation  evaluation of mental states in aviation ...
Industrial neuroscience in aviation evaluation of mental states in aviation ...
 
Advanced Robotics Projects For Undergraduate Students
Advanced Robotics Projects For Undergraduate StudentsAdvanced Robotics Projects For Undergraduate Students
Advanced Robotics Projects For Undergraduate Students
 
Euro Cat Cscl Eu Workshop LSBU
Euro Cat Cscl Eu Workshop LSBUEuro Cat Cscl Eu Workshop LSBU
Euro Cat Cscl Eu Workshop LSBU
 
Herramientas y técnicas para la Gestión del Conocimiento Nuclear
Herramientas y técnicas para la Gestión del Conocimiento NuclearHerramientas y técnicas para la Gestión del Conocimiento Nuclear
Herramientas y técnicas para la Gestión del Conocimiento Nuclear
 
Organisational Interoperability in Practice at Universidad Politécnica de Madrid
Organisational Interoperability in Practice at Universidad Politécnica de MadridOrganisational Interoperability in Practice at Universidad Politécnica de Madrid
Organisational Interoperability in Practice at Universidad Politécnica de Madrid
 
Building repositories and increasing usage.
Building repositories  and increasing usage.Building repositories  and increasing usage.
Building repositories and increasing usage.
 
Before you start: Strategic Planning and Business Cases
Before you start: Strategic Planning and Business CasesBefore you start: Strategic Planning and Business Cases
Before you start: Strategic Planning and Business Cases
 
Building repositories
Building repositories Building repositories
Building repositories
 
Overview of the Maker Movement in the EU
Overview of the Maker Movement in the EU Overview of the Maker Movement in the EU
Overview of the Maker Movement in the EU
 
Mahdieh_Shojaei_Baghini_Resume
Mahdieh_Shojaei_Baghini_ResumeMahdieh_Shojaei_Baghini_Resume
Mahdieh_Shojaei_Baghini_Resume
 
Keepit Course 5: Concluding the course
Keepit Course 5: Concluding the courseKeepit Course 5: Concluding the course
Keepit Course 5: Concluding the course
 
Ontological Infrastructure for Interoperable Research Information Systems: HE...
Ontological Infrastructure for Interoperable Research Information Systems: HE...Ontological Infrastructure for Interoperable Research Information Systems: HE...
Ontological Infrastructure for Interoperable Research Information Systems: HE...
 
OpenACC Monthly Highlights: January 2024
OpenACC Monthly Highlights: January 2024OpenACC Monthly Highlights: January 2024
OpenACC Monthly Highlights: January 2024
 
FoCAS Newsletter Issue 4: August 2014
 FoCAS Newsletter Issue 4: August 2014 FoCAS Newsletter Issue 4: August 2014
FoCAS Newsletter Issue 4: August 2014
 
Articulo cientifico ijaerv13n12_05
Articulo cientifico ijaerv13n12_05Articulo cientifico ijaerv13n12_05
Articulo cientifico ijaerv13n12_05
 
The LEGO Maturity & Capability Model Approach
The LEGO Maturity & Capability Model ApproachThe LEGO Maturity & Capability Model Approach
The LEGO Maturity & Capability Model Approach
 
Annual Report
Annual ReportAnnual Report
Annual Report
 
Prbb Punk Lab
Prbb Punk LabPrbb Punk Lab
Prbb Punk Lab
 
AbstractsBook_GCACSE2016 (3) (1)
AbstractsBook_GCACSE2016 (3) (1)AbstractsBook_GCACSE2016 (3) (1)
AbstractsBook_GCACSE2016 (3) (1)
 

Plus de butest

EL MODELO DE NEGOCIO DE YOUTUBE
EL MODELO DE NEGOCIO DE YOUTUBEEL MODELO DE NEGOCIO DE YOUTUBE
EL MODELO DE NEGOCIO DE YOUTUBEbutest
 
1. MPEG I.B.P frame之不同
1. MPEG I.B.P frame之不同1. MPEG I.B.P frame之不同
1. MPEG I.B.P frame之不同butest
 
LESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIALLESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIALbutest
 
Timeline: The Life of Michael Jackson
Timeline: The Life of Michael JacksonTimeline: The Life of Michael Jackson
Timeline: The Life of Michael Jacksonbutest
 
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...butest
 
LESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIALLESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIALbutest
 
Com 380, Summer II
Com 380, Summer IICom 380, Summer II
Com 380, Summer IIbutest
 
The MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazz
The MYnstrel Free Press Volume 2: Economic Struggles, Meet JazzThe MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazz
The MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazzbutest
 
MICHAEL JACKSON.doc
MICHAEL JACKSON.docMICHAEL JACKSON.doc
MICHAEL JACKSON.docbutest
 
Social Networks: Twitter Facebook SL - Slide 1
Social Networks: Twitter Facebook SL - Slide 1Social Networks: Twitter Facebook SL - Slide 1
Social Networks: Twitter Facebook SL - Slide 1butest
 
Facebook
Facebook Facebook
Facebook butest
 
Executive Summary Hare Chevrolet is a General Motors dealership ...
Executive Summary Hare Chevrolet is a General Motors dealership ...Executive Summary Hare Chevrolet is a General Motors dealership ...
Executive Summary Hare Chevrolet is a General Motors dealership ...butest
 
Welcome to the Dougherty County Public Library's Facebook and ...
Welcome to the Dougherty County Public Library's Facebook and ...Welcome to the Dougherty County Public Library's Facebook and ...
Welcome to the Dougherty County Public Library's Facebook and ...butest
 
NEWS ANNOUNCEMENT
NEWS ANNOUNCEMENTNEWS ANNOUNCEMENT
NEWS ANNOUNCEMENTbutest
 
C-2100 Ultra Zoom.doc
C-2100 Ultra Zoom.docC-2100 Ultra Zoom.doc
C-2100 Ultra Zoom.docbutest
 
MAC Printing on ITS Printers.doc.doc
MAC Printing on ITS Printers.doc.docMAC Printing on ITS Printers.doc.doc
MAC Printing on ITS Printers.doc.docbutest
 
Mac OS X Guide.doc
Mac OS X Guide.docMac OS X Guide.doc
Mac OS X Guide.docbutest
 
WEB DESIGN!
WEB DESIGN!WEB DESIGN!
WEB DESIGN!butest
 

Plus de butest (20)

EL MODELO DE NEGOCIO DE YOUTUBE
EL MODELO DE NEGOCIO DE YOUTUBEEL MODELO DE NEGOCIO DE YOUTUBE
EL MODELO DE NEGOCIO DE YOUTUBE
 
1. MPEG I.B.P frame之不同
1. MPEG I.B.P frame之不同1. MPEG I.B.P frame之不同
1. MPEG I.B.P frame之不同
 
LESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIALLESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIAL
 
Timeline: The Life of Michael Jackson
Timeline: The Life of Michael JacksonTimeline: The Life of Michael Jackson
Timeline: The Life of Michael Jackson
 
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...
Popular Reading Last Updated April 1, 2010 Adams, Lorraine The ...
 
LESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIALLESSONS FROM THE MICHAEL JACKSON TRIAL
LESSONS FROM THE MICHAEL JACKSON TRIAL
 
Com 380, Summer II
Com 380, Summer IICom 380, Summer II
Com 380, Summer II
 
PPT
PPTPPT
PPT
 
The MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazz
The MYnstrel Free Press Volume 2: Economic Struggles, Meet JazzThe MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazz
The MYnstrel Free Press Volume 2: Economic Struggles, Meet Jazz
 
MICHAEL JACKSON.doc
MICHAEL JACKSON.docMICHAEL JACKSON.doc
MICHAEL JACKSON.doc
 
Social Networks: Twitter Facebook SL - Slide 1
Social Networks: Twitter Facebook SL - Slide 1Social Networks: Twitter Facebook SL - Slide 1
Social Networks: Twitter Facebook SL - Slide 1
 
Facebook
Facebook Facebook
Facebook
 
Executive Summary Hare Chevrolet is a General Motors dealership ...
Executive Summary Hare Chevrolet is a General Motors dealership ...Executive Summary Hare Chevrolet is a General Motors dealership ...
Executive Summary Hare Chevrolet is a General Motors dealership ...
 
Welcome to the Dougherty County Public Library's Facebook and ...
Welcome to the Dougherty County Public Library's Facebook and ...Welcome to the Dougherty County Public Library's Facebook and ...
Welcome to the Dougherty County Public Library's Facebook and ...
 
NEWS ANNOUNCEMENT
NEWS ANNOUNCEMENTNEWS ANNOUNCEMENT
NEWS ANNOUNCEMENT
 
C-2100 Ultra Zoom.doc
C-2100 Ultra Zoom.docC-2100 Ultra Zoom.doc
C-2100 Ultra Zoom.doc
 
MAC Printing on ITS Printers.doc.doc
MAC Printing on ITS Printers.doc.docMAC Printing on ITS Printers.doc.doc
MAC Printing on ITS Printers.doc.doc
 
Mac OS X Guide.doc
Mac OS X Guide.docMac OS X Guide.doc
Mac OS X Guide.doc
 
hier
hierhier
hier
 
WEB DESIGN!
WEB DESIGN!WEB DESIGN!
WEB DESIGN!
 

ph-report.doc

  • 1. PhD Report Relatório de Doutoramento AFRANCI: An Architecture for Learning Agents Francisco Antonio Fernandes Reinaldo Aluno de doutorameto Rui Camacho; Luis. P. Reis Supervisores {reifeup, rcamacho, lpreis}@fe.up.pt Laboratório de Inteligência Artificial e Ciência de Computadores – LIACC. Departamento de Engenharia Electrotécnica e de Computadores – DEEC. Faculdade de Engenharia da Universidade do Porto – FEUP. Rua Dr. Roberto Frias s/n, 4200-465. Porto – Portugal. Tel: + 351 22 508 1400 - Fax: +351 22 508 1443. Reinaldo, F. A. F.; Camacho, R.; Reis, L. P. AFRANCI: An Architecture for Learning Agents. Artificial Intelligence. PhD Report. FEUP: Porto, 2005. 14 p. e-doc: http://www.fe.up.pt/~reifeup/phdr.pdf FEUP e Porto, 25 de Setembro de 2005.
  • 2. Presentation This report summarises the work done is the first year of Francisco Reinaldo´s PhD. It is also included in the report the cognitive architecture proposal that will be developed and evaluated in the remaining years of the PhD. The proposed architecture is adequate for Cognitive Agents acting in dynamic and unpredictable environments. The framework includes Machine Learning techniques to make adaptable agents. Reinaldo attended the 14th International Conference on Learning and Extraction Behaviour in FEUP, last year. He is a collaborative member of the research project “FC Portugal: Coordination of Heterogeneous Teams in Search and Rescue Scenarios”, with grant n. POSI/EIA/63240/2004. This year, he has collaborated in the realization of II SIEBRAP – Seminar of Brazilian Researchers in Portugal, Porto. He has presented a poster and a talk in the NN2005 - Summer School on Neural Networks, ISEP, Porto. He has delivered a paper in RoboCup Rescue 2005 in Osaka, Japan, where he also was a judge in the Robots with 4 legs championship “Sony AIBO” in RoboCup Rescue 2005. He reviewed papers in MASTA'05 - 3rd Workshop on Multi-Agent Systems: Theory and Applications, Porto. He has an accepted paper in EPIA 2005 IROBOT Workshop to the title “Applying Biological Paradigms to Emerge Behaviour in RoboCup Rescue Team, Porto. In addition, other paper has an acceptance in ESM´2005 to the title “A Tool for Fast Development of Modular and Hierarchic Neural Network-based Systems. Besides that, he was invited lecturer in FEUP, where he taught Algorithms and Data Structures in classroom (AED) in 2004/2005 school year. Reinaldo helped Luís P. Reis with students in undergraduate final work at FEUP.
  • 3. 3 Abstract In this report, we describe the work done in the first year of the PhD. We present a proposal for general purpose architecture AFRANCI with hybrid learning capabilities to development of heterogeneous agents. We argue that the use of Machine Learning techniques is of crucial importance for agents to act alone, in community, or/and in dynamic environments. The AFRANCI architecture addresses hierarchic behaviour levels, but will be empirically evaluated in the RoboCup Rescue International. The challenge of RoboCup Rescue is to promote competitive international research by providing cooperation opportunities, so that researchers can evaluate their research results. We claim that agents with proper learning capabilities have good performances in the simulated league. Keywords Cognitive Agent Architectures, RoboCup Rescue League, Machine Learning.
  • 4. Table of Contents I – Introduction....................................................................................................................5 1.1 Objectives and Motivations.........................................................................................5 II – First Year Study.............................................................................................................6 2.1 Agent Architectures ...................................................................................................6 2.2 RoboCup Rescue Teams ............................................................................................6 2.3 Learning Techniques .................................................................................................6 2.3.1 Artificial Neural Networks ....................................................................................6 2.3.2 ILP ......................................................................................................................7 III – The Project...................................................................................................................8 3.1 Introduction...............................................................................................................8 3.2 Development of the Architecture.................................................................................8 3.3 A Framework and Tool for developing Autonomous Agents........................................10 IV – Work Plan for Next Years............................................................................................12 4.1 RoboCup Rescue Project ..........................................................................................12 V - References...................................................................................................................14
  • 5. 5 I – Introduction This PhD report presents a proposal Cognitive Agent architecture together with a framework that allows the development of Cognitive Agents. The proposed architecture will be evaluated through agent’s operations in RoboCup Rescue Simulation System. The rescue league is composed of agents that have general functionalities but some particular characteristics (personalities) that define their heterogeneity. The AFRANCI architecture offers fast reasoning and social capabilities to autonomous agents. The architecture is inspired by the main features of global organization of the mammalian brain to simulate the biologic behaviour level of the Central Nervous System (CNS). The CNS [1] is formed by subsequent clusters of cognitive processes that are arranged in a hierarchical way to overlap actions. A cognitive process is a decision-making route of beliefs or desires that triggers actions for emerging behaviours. Like a cortex (neocortex) [2], this hierarchical organization allows increasingly complex actions/decisions. Our architecture has special features that enable agents to reach a goal, establish control in asynchronous activities, and solve existential conflicts between reactivity, deliberation and social activities. Moreover, a framework will be developed to implement the theoretic ideas. This framework will offer some base-classes to develop an Artificial Central Nervous System (ACNS). An ACNS is a computational model of a CNS. The ACNS is responsible for emerging a general agent. In addition, a Tool will be developed over the framework. This tool will design an agent’s structure and particular learning modules to assemble heterogeneous agents. This research focuses three main global areas: a) Cognitive System, composed by Cognitive Processes, Behaviour-Based Architecture and Emerging Social Behaviours in a Multi-agent System (MAS); b) Software Engineering where a framework behaviour-based will be built; c) Simulation System that works with RoboCup Rescue as a test bed system. 1.1 Objectives and Motivations The work has two objectives: first we identify relevant features of cognitive Agents and propose an architecture to accommodate them; second we develop a framework to enable the customization of Agents and to train the necessary modules. Our motivation is to offer flexibility in the development of agents. As the general architecture is concerned, we may customize a Cognitive Architecture and train it to a particular application. We will use the RoboCup Rescue League in our proposal. The main motivations of this work are: • Development of an agent’s architecture that will be used as pattern for designing agents; • Contribution in the improvement of straightforward agent leaning techniques that are based on similarity of behavioural human being architecture; • Providing of opportunities for students to perform significant test bed in classroom through which they can learn many of the train techniques of multi-agent system development; • Comprising the following domains: social, technical and cognitive. Social environment is significant because it focuses on human search and rescue mission planning. Technical research is important because it solves problems of rescue strategies and tactics for a fast and optimized saving of people, and it also preserves infrastructures, as well. Cognitive science is considered because it studies models of information and representation, capacity of human memory and biologic behaviour to be applied in a Multi-agent system.
  • 6. 6 II – First Year Study During the first year of the PhD we identified and analysed the main agent architectures [3-17], RoboCup Rescue teams [18-22] and Learning Techniques to propose our architecture. 2.1 Agent Architectures The surveyed agent behaviour architectures are the following. Brooks [3] has developed a reactive architecture with levels of subsumption. Guessoum and Briot [4] have developed a modular and generic agent architecture in Smalltalk named DIMA platform (Development and Implementation of Multi-agent System). Hoff and Bekey [5] have used connectionist architecture for learning coordination of different behaviours in a situated agent. Ito [6] has used Reinforcement Learning (RL) architecture to realize complex tasks in agents. Shinoda [7] has proposed a Framework for designing behaviours. Mitchell [8] has proposed a Plan- and-Compile architecture named Theo - this architecture uses a plastic knowledge base to learn facts and meta-facts. Carbonell, Knoblock and Minton [9] have proposed a Modular architecture that stores symbolic knowledge in a way of First-Order Predicate Logic (FOPL) which is called the Prodigy Description Language (PDL). Roisenberg [10] has developed a PyramidNet Architecture that uses a modular and hierarchic approach which are based on heterogeneous neural nets to create complex and intelligent behaviours in robotic systems. Duffy [11] has used a hierarchic architecture to design deliberative and reactive levels without the use of the connectionist paradigm. Bowling [12], Duffy [13] and Haikonen [14] have offered a general architecture to design a cognitive machine architecture for the agent. Taheri and Sadati [15] have proposed a new modular category for agent navigation, using Fuzzy Logic. Steenekamp and Ross [16] have used the symbolic paradigm to deliberate the level of actions for agents. The higher level and reactive level are used in some “routine” decisions that are some encoded rules in their rule base. Patra and Mohanty [17] have proposed an architecture to build software agents. The agent is modelled on belief-desire- intention architecture and is composed of perceptions, actions and goal-directed behaviours. 2.2 RoboCup Rescue Teams The well most known rescue teams use agents with advanced learning and coordination capabilities. DAMAS Rescue team [18] concentrates its work on improving the agent’s ability to extinguish buildings on fire. The ability consists in a classification of the best fire to extinguish by the perception learning method. ResQ Freiburg [19] deals with a sequence of planning methods. The objective is to build hierarchic commands that imply hierarchic behaviours. Caspian [20] uses a three phase model approach to develop their agents: World Model, Dynamic Path Finding and Decision Making. SOS team [21] develops a general- purpose rescue agent for implementing high-level strategies and learning algorithms. The SOS agent has a state-based architecture, an explicit state-selection and state switching policy. It uses two path-finding modules that are D* and Dijkstra’s algorithm and two neural networks to make decisions. FC Portugal Rescue Team [22] uses high-level coordination methodologies, Configurable Flexible Team Strategy and Situation Based Strategic Positioning to coordinate agents. 2.3 Learning Techniques 2.3.1 Artificial Neural Networks An Artificial Neural Network (ANN) is a data processing paradigm inspired by biological neural networks. An ANN has the ability to learn from examples, which gives it a
  • 7. 7 considerable power and flexibility. ANN is composed of several interconnected neurons that perform excitatory and inhibitory connections to adjust their synaptic weights between neurons for solving a problem and to produce the correct output. The massive parallelism of neurons in an ANN structure can offer data compression by storing the information in the weights on the connections. The weights are supposed to adjust during the training stage where examples are showed to the network. Depending on the network type and its topology, an ANN has the capacity to obtain meaning from complex or imprecise inputs and adjust to any situations or even to model a complex decision system. The biologic nervous system may be structured with sufficient precision to obtain behaviours emerging in a computational model. ANNs do not require a formalised algorithm to achieve a goal. A suboptimal result is most necessary to solve huge problems instead of optimal results because a final result is unclear in some cases. Depending on the ANN topology we can process strings of symbols as a formal system that is interpreted by humans as concepts but with supports for noisy or imprecise data. Similar to the brain, ANN are robust in the presence of noise: small changes in an input signal will not drastically affect a node's output. Nets are robust in the presence of failure: a change in a weight may only affect the output for few of the possible input patterns. The net can deal with “unseen” patterns and generalise from the training set. Nets are good at “perceptual” tasks and associative recall. These are just the tasks that the symbolic approach has difficulties in dealing with. 2.3.2 ILP Inductive Logic Programming (ILP) has achieved considerable success in a wide range of domains. These domains include the Life Sciences, Molecular Biology and Engineering. We believe that ILP can be successfully applied to the synthesis of some conscious behaviour. A typical ILP system constructs models from data by making a search through an ordered hypothesis space (the models). An ILP system accepts examples as input, together with relevant information (the background knowledge) and induces a model that “explains” the examples given the background knowledge. The great advantage of ILP is the fact that it uses a very powerful representation language: First Order Logic. This feature enables ILP to induce very complex models. Another relevant feature of an ILP system concerns the ability to combine very easily symbolic computations with numerical ones.
  • 8. 8 III – The Project This proposal presents a modular and hierarchic behaviour-based architecture to be applied in a multi-agent system. This architecture is composed by heterogeneous learning modules arranged in behaviour levels. AFRANCI is an acronym of “Architecture For Rescue AgeNts with Cognitive Insights”. 3.1 Introduction We are using both the symbolic and the connectionist paradigms to develop AFRANCI architecture. The idea of joining both research areas appeared in McCulloch´s work, which had a strong biologic inspiration and was made in conformity with Pitts´s work using mathematical concepts [23]. In the same direction, in order to construct hybrid systems, we are capable of combining the characteristics of adaptability, robustness and uniformity that are offered by neural networks with representation, inference and universality, native characteristics of symbolic Artificial Intelligence [24]. 3.2 Development of the Architecture This general architecture with extended characteristics was inspired in the main parts of the human being´s nervous structure. These parts are necessary to construct a general computational agent. They are responsible for emerging behaviours. The biologic inspiration of this work had origin in analysis of structure and functions of the brain and other nervous areas of the body. AFRANCI architecture emerged from the user’s difficulty to design agents using the same agent body. The difficulty consists in using built-in characteristics to perform particular tasks in a multi-agent system. The use of different approaches, presented in topic 2.1 and 2.2, gives us examples of how difficult it is to decide for a construction of an agent. In order to develop an agent architecture, the first step is to restrict a behaviour module with its class of problems. In our work, AFRANCI architecture organises the modules as a collection of autonomous computational elements. In horizontal line, there are homogeneous modules arranged in the same topology. In vertical line, there are heterogeneous modules arranged in different kinds of complexity from simple to complex. The AFRANCI structure supports a great variety of behaviours without decreasing of performance. Using a top-down topology, it is possible to divide levels of complexity into incremental functionality. The main idea focus on a quantity of cases where behaviour modules are being used to run processes in different periods or a same process will be performed for many modules in the “same time” in distinct and functional areas of the brain [25]. Levels that are more abstract provide increasing complexity and subsume lower level functionalities, Figure 1.
  • 9. 9 Figure 1. The analogy between a natural and computational system. Figure 1 presents an analogy between a human and a computational agent. On the left of the figure are represented the main abstract parts of the nervous system of a human being, mapped on the right. Continuing this analogy, the horizontal behaviour modules respect a hierarchy. The vertical communication between heterogeneous levels happens from internal connections as well. Each level is linked with classes of problems, which are capable of solving them. AFRANCI architecture is arranged from lower level to higher level as stereotyped (reflexive and taxies), reactive, instinctive and deliberative behaviour levels. Moreover, we have localized a social module in the top level of the deliberative behavioural structure. The social module is proposed to share learned experiences among agents. The differential of this architecture is to support the management of different actions to achieve fast and optimized collaborative outcomes. Inside the architecture, each vertical level has actions that make connections with certain stimulus or decisions, such as generating autonomy, reactivity, reasoning and social ability. Local behaviours emerge in local modules that are arranged in the first behaviour level. General behaviours emerge through other parts in the architecture. Social behaviour is
  • 10. 10 inside the top behaviour level which is sharing the reasoning space. The whole architecture is involved in the learning and behaviour process. The main advantages can be seen in both high and low parts of the architecture and the topology. The low level architecture offers resources in order to facilitate a fast response. This level has behaviour implemented by reflexive or/and reactive modules that are similar to the human being´s biologic connection. With this approach, any adaptation state is achieved in a dynamic environment. Otherwise, the use of advanced perception techniques brings high performance in task development as well. In high level of architecture, agents can explore the world and construct a mental map of environment by using mapping techniques. In this case, every agent will contribute to feed the shared database. Using a shared database, time exploration is optimized and a great reduction in size of the world space is achieved. Problems such as solving an unknown but similar problem are solved as well, because the architecture uses previously collected information in the database to feed the making decision algorithms. Moreover, this topology solves existential conflicts between reactivity, deliberation and social activity. In addition, a modular architecture has had good acceptance because it produces good performance through separated but linked functional components within the horizontal design of a system. Thus, autonomy and robustness achieved to deal with large quantities of learning techniques with high performance because of the use of a hierarchic and modular architecture. The learning process is simple and can occur in two steps. Firstly, agents’ sensors collect data from the environment and set them in a database. This database will feed the learning algorithms arranged in low level. If the task can be solved in low level, then the action will be performed; otherwise the information learned in low level is sent to high level. Secondly, the agents´ decision is made through the use of Reinforcement Learning or Inductive Logic Programming in high level to make decisions based on information sent by low level. For Instance, these decisions can be planning or social behaviour. After finding a solution, the behaviour is subsumed from high level to low level until it reaches the agent’s actuators. This architecture enables users to develop any agent because they use a general architecture composed by all behaviour levels. Using a framework to fashion the whole architecture, the user can handle behaviour modules as blocks. 3.3 A Framework and Tool for developing Autonomous Agents In order to deploy the architecture, we are converting it into a framework. A Framework is an extensible set of base-classes (skeleton) to develop tools. These base-classes are concrete or abstract classes that provide a collaborative pattern between instances earlier defined by the framework. A Tool is an artefact that was extended by configuration or linking of concrete classes and derivation of new concrete classes from abstract classes of the framework. AFRANCI Framework is the implemental part of AFRANCI Architecture. The framework comes to reduce classic learning algorithms, to optimize performance and to extend current interpretations of agent systems, behaviour-based architecture and agent-based techniques. The framework skeleton will be built to extend application tools for the development of agents. In order to produce rescue agents with specific functionalities, a tool will be extended under this framework, Figure 2.
  • 11. 11 Figure 2. Framework and Tool The whole figure above is a complex structure that determines the tool. It is possible to see two parts in Figure 2. The splotchy area shows a framework structure with its classes and connections ready to be extended or shared. In the other part, below the splotchy area, there are complements of classes that characterise the tool when it is added to the structure of the framework. The use of the framework is necessary to produce a particular tool because it reduces the quantity of line-code to be developed, tested and cleaned. AFRANCI Tool will be robust to design agents’ structure. It will be able to design an Artificial Central Nervous System (ACNS), explained previously in Section I. More specifically, users will design specific agents’ structure by using general modules added to particular modules form which “the personality” of the agent will emerge. The tool will make use of drag-and- drop graphic blocks, as a puzzle toy, and automated generation opened-code to apply in agents for performing tasks. The tasks can be actions for mitigation disasters, such as: search and rescue of citizens who were getting buried under the debris; set free the blocked roads and railways to allow rescue teams traffic; wounded refuge buildings; control and extinguish fires; communication and synchronized coordination between platoon agents and centre agents. We believe that using simple graphic interfaces and computational elements is the best way to create hybrid levels and to gain complex behaviours. After finding each module in the architecture as a simple puzzle toy, an automated opened-code is generated. This process of production of agent’s code is very optimized and clean.
  • 12. 12 IV – Work Plan for Next Years We have addressed methodologies to focus the development of a general agent. Next year we will implement a general agent based on our architecture. Maybe adjusts will be necessary. After this, we will construct rescue agents as our testbed. 4.1 RoboCup Rescue Project The RoboCup Rescue is an international project that was started in 1999 to solve relevant social problems by integration of disaster information, prediction, planning and training for rescue actions. Built upon the success of RoboCup Soccer project, it offers a comprehensive urban disaster simulator, forums of technical discussions and competitive evaluations for researchers and practitioners. The RoboCup Rescue Simulation League is an agent team formation that focuses on: rescuing injured citizens and decreasing the damage of the collapsed town. In order to perform these tasks, teammates use complex high-level plans, adjust of heterogeneous behaviours and cooperation among them. Every year a competition is organized where researchers meet, compare approaches and exchange ideas. The RoboCup Rescue Simulator is divided by modules that can run in several computers. These modules are independent to run particular characteristics of the world, as can be seen in Figure 3. The modules also represent simulation domains as earthquake, buildings on fire or collapsing, spreading of fire to neighbouring buildings, blocked ways and health state of citizens. Figure 3. RoboCup Rescue Simulation System. In accordance with Figure 3, the main components that comprise the simulator are [26]: • Geographic Information System (GIS): the GIS module holds the state of the simulated world; • Kernel: this module is connected to all the other modules. The kernel updates the objects in the GIS and sends the world updated to every connected module. • Sub-simulators: Fire-simulator, Collapse-simulator, Traffic-simulator etc. are modules connected to the kernel, each one simulating a particular disaster feature – fire, collapses, traffic etc. • Agents: agent modules are connected to the kernel and represent intelligent entities in the real world, such as civilians, polices agents, fire agents etc. • Viewers: their task is to get the state of the world, communicating with the kernel module, and graphically displaying it, allowing the user to easily follow the simulation process.
  • 13. 13 The simulator presents a world model with heterogeneous agents, Figure 4. Figure 4. Rescue Scenario The RoboCup Rescue scenario, Figure 4, includes approximately 40 agents in six different types to preserve the town and citizens´ life: Fire Brigade agents, Police Force agents, Ambulance agents and three different Control Centres. Centre agents coordinate platoons with collaborative tasks that are based on information from the external world received by mobile agents. These centre agents send/receive several messages of the world state to/from its mobile agents. The updated messages are sent to the rescue agents. Platoons have autonomy to learn and to create the best decision about unexpected states. Police agents have the responsibility of clearing priority blocked ways for extreme traffic and to report to base the situation of a citizen crying for help. The Police Force has 10 to 15 agents. Ambulance agents search wounded buried citizens to transport them to refuges. Ambulance Centre has 5 to 8 agents. Fire Brigade agents extinguish fires in buildings. The number of brigade is between 10 and 15 agents. Each centre is responsible for its team. All tasks performed by mobile agents are centred to save people’s life.
  • 14. 14 V - References 1. Reinaldo, F.A.F., Projecting a framework and programming a system for development of modular and heterogeneous artificial neural networks., in Departament of Computer Science. 2003, Federal University of Santa Catarina: Florianópolis. p. 86. 2. Kolb, B. and I.Q. Whishaw, An Introduction to Brain and Behavior. 2nd ed. 2005, New York: Worth Publishers Inc. 601. 3. Brooks, R.A., A Robust Layered Control System For a Mobile Robot. Technical Report, in AI memo 864. 1985, Massachusetts Institute of Technology: Massachusetts, CA. p. 25. 4. Guessoum, Z. and J.-P. Briot, From active objects to autonomous agents. Concurrency, IEEE [see also IEEE Parallel & Distributed Technology], 1999. 7(3): p. 68-76. 5. Hoff, J. and G. Bekey. Architecture for behavior coordination learning. in Proceedings of the 1995 IEEE International Conference on Neural Networks. 1995: IEEE. 6. Ito, K., et al. A study of reinforcement learning with knowledge sharing for distributed autonomous system. in Computational Intelligence in Robotics and Automation, 2003. Proceedings. 2003 IEEE International Symposium on. 2003. 7. Shinoda, K., I. Noda, and M. Ohta. Behavior design of civilian agent for rescue simulation. in Proc. of Challenges in Open Agent Systems. AAMAS02. 2002, 7. 8. Mitchell, T.M., et al., Theo: A framekwork for self-improving systems, in Architectures for Intelligence, i.K. VanLehn, Editor. 1991, Lawrence Erlbaum Associates: Hillsdale, New Jersey. p. 323-355. 9. Carbonell, J., et al., PRODIGY: an integrated architecture for planning and learning. SIGART Bull., 1991. 2(4): p. 51-55. 10. Roisenberg, M., Emergency of the intelligence in autonomous agents through inspired models in the nature, in Department of Electric Engineering. Group of Researches in Biomedical Engineering. 1998, Federal University of Santa Catarina: Florianópolis. p. 206. 11. Duffy, B.R., The Social Robot, in Department of Computer Science. 2000, University College Dublin (UCD): Dublin, Ireland. 12. Bowling, M., Robocup rescue: Agent development kit. 2000, Avaliable at official RoboCup Rescue Web Site. 13. Duffy, B.R., et al. Agent chameleons: agent minds and bodies. in Computer Animation and Social Agents, 2003. 16th International Conference on. 2003. 14. Haikonen, P.O.A. A modular neural system for machine cognition. in Neural Networks, 2000. IJCNN 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on. 2000. 15. Taheri, J. and N. Sadati. A fully modular online controller for robot navigation in static and dynamic environments. in Computational Intelligence in Robotics and Automation, 2003. Proceedings. 2003 IEEE International Symposium on. 2003. 16. Steenekamp, P. and J. Roos. Implementation of distributed systems management policies: a framework for the application of intelligent agent technology. in Systems Management, 1996., Proceedings of IEEE Second International Workshop on. 1996. 17. Patra, M.R. and H. Mohanty. A formal framework to build software agents. in Software Engineering Conference, 2001. APSEC 2001. Eighth Asia-Pacific. 2001. 18. Paquet, S., N. Bernier, and B. Chaib-draa, DAMAS-Rescue Description Paper. 2004. p. 12. 19. Kleiner, A. and M. Brenner, ResQ Freiburg: Deliberative Limitation of Damage. 2004. 20. Hamraz, S.H., S.S. Feyzabadi, and A.K. Motlagh, Caspian RoboCup Rescue Simulation Agent: Team Description. 2005. 21. Amraii, S.A., et al., S.O.S. 2004: an attempt towards a multi-agent rescue team. 2004, Amirkabir University of Tecnology: Tehran, Iran. p. 12. 22. Lau, N., L.P. Reis, and F. Reinaldo, FC Portugal 2005 Rescue Team Description: Adapting Simulated Soccer Coordination Methodologies to the Search and Rescue Domain. 2005. 23. McCulloch, W.S. and W. Pitts, A locical calculus of the ideas immanent in nervous activity. Bulletin Of Mathematical Biophysics. Vol. 5. 1943. 115-133. 24. Waltz. Neural nets and AI: Time for a synthesis. in International Conference on Neural Networks. 1997. San Diego, CA. 25. Eccles, J.C., The Cerebral Neocortex: A Theory of its Operation., in Cerebral Cortex, E.G.J.A. Peters, Editor. 1984, Plenum Press: New York. p. 1-36. 26. d'Agostino, F., et al. Monitoring and information fusion for search and rescue operations in large-scale disasters. in Information Fusion, 2002. Proceedings of the Fifth International Conference on. 2002.