In: Anais do I Workshop Tidia, São Paulo, novembro, 2004 Em inglês , 9 páginas. Autores: Heloisa Vieira da Rocha, Joice Lee Otsuka, Vitor Seixas Bernardes

1. A multiagent system for formative assessment support in Learning
Management Systems
Joice Lee Otsuka, Vítor Sexto Bernardes, Heloísa Vieira da Rocha
Institute of Computing – State University of Campinas
Nucleus of Informatics Applied to Education – State University of Campinas
P.O. Box 6176 13083-970 - Campinas, SP – Brazil
Phone +55 19 3788-5842
{joice,vitor.bernardes,heloisa}@ic.unicamp.br
Abstract
Formative Assessment has been adopted as an alternative to the traditional model based on cumulative
exams and has revealed itself as a crucial importance to distance education, by helping the learners’
behavior perception and the problems identification and making possible more effective advice in the time
of learning process, even at distance. Nevertheless, the educators need to take extra steps to monitor
learners’ participation and to regulate the learning processes in distance courses, resulting in the educators’
overload and, consequently, the formative assessment high costs.
The possibility of saving all interactions in computer-based courses for later analysis can be explored to
provide formative support. This paper presents the main results of a doctoral research that proposes a
multiagent system (MAS) to support formative assessment in Learning Management Systems. The
proposed MAS architecture definition process and some considerations about the implementation of a
prototype that has been integrated into TelEduc LMS are discussed.
Keywords: multiagent systems, distance education, formative assessment.
online assessment presents some intrinsic
1 Introduction difficulties, such as the absence of the face-
The formative assessment has been used as to-face interactions feedback, the lack of
an alternative to the traditional assessment instructor control over the assessment and
based on cumulative exams, and allows the the authentication problem (who is
continuous monitoring and orientation of the performing the assessment?). The
learning process. This assessment approach informative and regulatory characteristics of
presents informative and regulatory roles, this assessment approach have revealed
providing information to the two main actors themselves as a crucial importance to
of the teaching and learning process: the distance education, by helping the learners’
educator, that will be informed about the behavior perception and the problems
real effects of his/her pedagogical actions, identification, making possible more
and will be able to regulate them; and the effective advice in the time of learning
learner, that will have the opportunity to be process, even at distance.
conscious of his/her difficulties and,
Nevertheless, nowadays most Learning
possibly, to recognize and to review his/her
Management Systems (LMSs) do not
own mistakes [Hadji 2001].
present features specially designed to help
In distance education, the formative this assessment approach. Thus, the
assessment is even more important since the educators need to take extra steps to
TIDIA First Workshop – E-Learning Project November 2004

2. monitor, to analyze and to advise the researches on education [Hadji 2001]
learning process. This scenario has been the [Perrenoud 1998], the research group’s
main problem of formative assessment, face- practical experience in computer-mediated
to-face as well as at distance. distance courses [Otsuka & Rocha 2002]
[Ferreira, Otsuka & Rocha 2003] and the
This paper presents the main results of
analysis of other researches and experiences
a doctoral research that proposes a
on learning assessment in distance education
multiagent system (MAS) to support
[Thorpe 1998] [Hopper 1998] [Nelson 1998]
formative assessment in LMSs, aiming to
[Cerny 2001] [Gomez 2001].
reduce the instruct overload and
consequently, the formative assessment The requirements of two main actors in
costs. formative assessment process, educators and
learners, were identified. Thus, the
The paper is structured in five sections.
requirements to support the educators’
Section 2 shows the main phases of the
needs are:
definition process that resulted in the
proposed MAS architecture. Section 3 To receive, continuously, relevant and
presents the agents knowledge base reliable information about learner’s
definition and section 4 presents some participation at each proposed activity,
considerations about the implementation of throughout the course;
a prototype of the proposed architecture that
To receive the consolidation of learner’s
has been implemented and integrated into
participation at all activities proposed,
TelEduc1 [Rocha 2002, Rocha et al. 2002], a
making possible a global monitoring of
LMS that has been developed at State
learners’ participation and the
University of Campinas since 1997, and
identification of the learners’
which is available as free software under the
participation and learning styles;
open-source GNU Public License. At last,
section 5 presents some final considerations To have mechanisms to help the
about this work. analysis and interpretation of the above
information, just in time, with coherence
2 The Multiagent System and impartiality;
Architecture definition To receive help on the assessment of
The definition process of a MAS activities, by means of the preanalysis of
architecture to support formative assessment learners’ participation based on criteria
in learning management systems was defined in the activities plan, and the
composed by the following main processes: elaboration of a detailed and consistent
the requirements specification and the feedback about these assessments;
process of analysis and design based on To have flexibility to define the
Gaia methodology [Wooldridge, Jennings & assessment criteria of each activity.
Kinny 2000]. The next subsections present
the main results obtained in these phases and The requirements to support learners’
the resulted proposed architecture. needs are:
To receive, continuously, monitoring
2.1 Requirements Specification
reports and their learning profiles, that
The first step in the definition of the provide a feedback of their participation
proposed MAS architecture was the main in each proposed activities and in the
requirements specification. These main whole course, helping them to be
requirements have been obtained through conscious of their difficulties and
progresses;
1
http://teleduc.nied.unicamp.br
TIDIA First Workshop – E-Learning Project November 2004

3. To consult other learners’ learning
profiles, making possible the knowledge Course Monitoring Agent 1.. n
of the group learning profile, what might
lead to improvements in the
collaborative learning.
collect and inform consolidate
The next step was the specification of update participation global
activities indices analysis
agents “roles”, that is, the behaviors that
need to be implemented by the MAS to
support the requirements specified earlier.
The roles specification was based on Gaia * 1..m
methodology, and it is presented in the next Activities Integrator Agent
subsection. Monitoring Agent
2.2 Analysis and design based on Gaia
methodology notify new
update analyze events with request for
The Gaia methodology [Wooldridge, activities indices activities monitoring
Jennings & Kinny 2000] was adopted
because it is simple and efficient, and
Figure 1 – The Agent Model of the proposed MAS: the
provides effective support to the agent types are represented by rectangle and the agent roles
specification of agents behavior (Roles are represented by rounded rectangles. The agents cardinality
is represented by: * (0 or more), 1..n (1 to n)
Model), the inter-role protocols definitions
(Interaction Model), the agent types
definition (Agent Model), as well as the
Course Monitoring Agent: this agent
identification of communication links
type is responsible for ongoing
between agent types (Acquaintance Model).
monitoring of learners’ participation
At analysis phase the Roles Model and throughout a course. It acts accessing
the Interaction Model were defined. The the course database and collecting
Roles Model specifies a set of agent indices from learners’ participation in
behaviors needed to support the each activity proposed (for example,
requirements presented at subsection 2.1. synchronous and asynchronous
The Interaction Model identifies the discussion, projects, reports and so on).
communication protocols that will be The collected indices are sent to the
specified to make possible the inter-role appropriate Activity Monitoring Agent
interactions. responsible for the analysis of activities.
At design phase the Agent Model and This agent type is also responsible for a
the Acquaintance Model were defined. The global analysis of learners’ participation
Agent Model defines the agent types, that is in one course, creating the learning
best thought of as a set of agent roles. To profiles. The MAS can have 1 to n
improve the system performance, the roles agents of this type, where n is the
were grouped trying to obtain a minimal number of current courses in the LMS;
number of agent types. This process resulted Activities Monitoring Agent: each
in an Agent Model composed by 3 agent agent of this type is responsible for the
types (Figure 1): analysis of learners’ participation
indices in every activity developed in
one specific tool of the LMS (such as
Discussion Forums, Chat, Portfolio,
Mail, Bulletin Board and so on). The
MAS can have 1 to m agents of this
TIDIA First Workshop – E-Learning Project November 2004

4. type, where m is the number of addition of a new Activity Monitoring
monitored tools in the LMS; Agent, responsible for the analysis of
learners’ participation in the context of the
Integrator Agent: this agent type
new tool. The Course Monitoring Agent’s
provides the MAS interface with the
roles may be slightly changed to be able to
educator: it receives events from LMS
deal with the new tool monitoring.
and informs them to the appropriated
Nevertheless, they are precise changes, that
agents in the MAS. A new Integrator
will not interfere in the previously defined
Agent should be created when one
roles (see discussion on section 4.3).
educator logs in the LMS and should
stay “alive” until the end of the session.
The Acquaintance Model was obtained
Multiagent System Architecture
directly from the Agent Model (Figure 2), LMS
and identifies the communication links Monitoring criteries <session x>
Monitoring request <session 2>
among the three different agent types with
<session 1>
the purpose of identify any potential Integrator Agent
communication bottlenecks, that could cause Assessment
Agent Communication Chanel
problems at run-time [Wooldridge, Jennings Discussion Forum Activities
& Kinny 2000]. Discussions Forum Monitoring Agent
Chat Chat Activities Monitoring Agent
Course Monitoring 1..n
* Agent
Tool m Tool m Activities Monitoring Agent
Integrator
Agent Course 1
Monitoring Agent
Course1
Activities 1..m Course 2
Monitoring Agent Monitoring Agent
Course2
Figure 2 – The Acquaintance Model of the proposed MAS Course n
Monitoring Agent
Course N
2.3 The multiagent system architecture n
The MAS architecture has been obtained Figure 3 – The proposed MAS integrated with a LMS
from the two process presented in the
previous subsections. Figure 3 shows the
proposed architecture integrated into a LMS 3 The knowledge base definition
that contains n courses and m tools to be
monitored. Thus, to support this LMS the The roles of Course Monitoring Agents and
proposed MAS architecture could include n Activities Monitoring Agents involve
Course Monitoring Agents and m Activity information analysis and new knowledge
Monitoring Agents. At each time an creation. Thus, the definition of these
educator logs in the LMS, one Integrator agents’ knowledge bases was needed. A
Agent is created to provide the interface knowledge base has been defined for each
between the MAS and the educator in the Activity Monitoring Agent to make possible
current session. the analysis of learners’ participation in
activities developed in the monitored tool.
The proposed architecture seems to be
easily extended to support the monitoring of The knowledge representation has been
new tools incorporated into the LMS. This is done by means of frames and production
achieved by the implementation and rules. This process has been supported by
TIDIA First Workshop – E-Learning Project November 2004

5. Protégé-20002 and Algernon3 tools that are implementation, and on the integration of
presented at section 4.1.2. An initial set of the proposed MAS into TelEduc.
rules has been defined with the participation
of researchers of Nucleus of Informatics 4.1 Technology investigation
Applied to Education (Nied) at State Some tools have been analyzed in order to
University of Campinas, based on group’s make easier the implementation of the
experiences with formative assessment in proposed MAS, namely, tools to support the
distance education. development of multiagent systems, tools to
At the time of an activity creation, the support the construction of the knowledge
educator selects some criteria that are going base and inference systems. In the
to reflect the rules to be considered on this implementation of the prototype, the
activity monitoring. The educator also can following tools have been adopted: the Jade
define some reference indices that will be MAS development framework4, the Protégé-
used as parameters of rules at the execution 2000 knowledge base editor5 and the
time, providing some flexibility to these Algernon rule-based inference system6, all
rules. of them are briefly described in the
following subsections.
Thus, when new monitoring indices are
collected through a Discussion Forums’ 4.1.1 Jade
Activity and instantiated into the Discussion
Forums’ Activities Monitoring Agent’ JADE is free software and is distributed by
knowledge base, these rules are executed TILAB, as open source software under the
and new knowledge can be produced. terms of the LGPL (Lesser General Public
License Version 2). It is implemented in
The knowledge base definition is at Java7 and aims to make easier the
initial phase and may be refined with users’ development of multiagent systems by
suggestions. In order to make this providing a comprehensive set of services
refinement possible, these knowledge bases and an API to support the development of
modeling have been focused on an easily agents in compliance with the FIPA
extended and not source-code dependent specifications [Gluz & Viccari 2003].
solution. In order to make easier the new
rules creation, the design of an interface that The JADE framework provides an API
makes possible the creation of new rules with several classes that represent common
directly by final users is proposed as future entities in MAS, supporting the definition of
work. agents, agents' behaviors, FIPA-ACL-
compliant messages, interaction protocols,
and so on [Bellifemine et al. 2003]. In the
4 Implementation Considerations
time of vocabulary and semantics definition
A prototype of the proposed MAS has been of the agents’ communication content, a
implemented and it is going to be integrated solution based on Java objects usage as the
into the TelEduc LMS, composing the content of the messages exchanged among
solution for formative assessment support the agents has been adopted, as proposed in
proposed for this system in [Otsuka et al. [Vaucher & Ncho 2004]. The JADE
2003]. The following subsections present utilization has been essential to the agents
some considerations on the technological implementation in the proposed MAS,
support that has been explored in the
4
http://sharon.cselt.it/projects/jade/
5
http://protege.stanford.edu/
2 6
http://protege.stanford.edu/ http://smi.stanford.edu/people/hewett/research/ai/
3
http://smi.stanford.edu/people/hewett/research/ai/ algernon/
7
algernon/ http://java.sun.com/
TIDIA First Workshop – E-Learning Project November 2004

6. simplifying and significantly speeding up completely independent of the agents Java
the process. code, thus making the maintenance of
knowledge bases easier.
4.1.2 Protégé–2000 and Algernon
4.2 Technology integration
Protégé-2000 is an extensible, platform-
independent ontology and knowledge-base The three different technologies discussed
editor, developed by Stanford Medical are Java-based and offer APIs that make
Informatics at the Stanford University easier the integration among Jade, Protégé-
School of Medicine. Protégé is available as 2000 and Algernon. As seen before, building
free software under the open-source Mozilla an agent with the Jade framework is realized
Public License [Gennari et al. 2002]. by extending classes already defined in the
framework. The manipulation of one agent’s
Protégé provides a simple and powerful knowledge base is possible by importing
graphical interface, integrating the ontology classes from Algernon's APIs.
definition and creation tasks and the
insertion of instances into knowledge base The server runs by starting a Jade
task. Another very interesting feature is the platform, which is the environment where
Protégé-2000's extensibility, since it is a the agents will live in, and then creating
platform that can be extended with graphical Activity and Course Monitoring agents.
widgets such as tables, diagrams and These agents' knowledge bases edited with
animation components. Also, due to its the Protégé-2000 GUI are stored in files that
plugin architecture, it is possible to can be accessed using Algernon's API in the
customize and extend Protégé's features agents code. This API allows creating an
according to the project specific needs instance of the Algernon inference machine,
[Musen et al. 2000]. adding a knowledge base to the list of KBs
this Algernon instance accesses, and
In the prototype implementation, the querying and inserting data into them.
Algernon Tab plugin8 has been used, which
is responsible for the integration of Protégé Thus, after the definition of frames,
and Algernon, an inference system that slots and relations between frames, all using
provides backward and forward chaining Protégé-2000's graphical interface, the
rules to infer data in knowledge bases. This insertion of instances into an agent's
plugin makes possible to execute Algernon knowledge base is done using Algernon's
commands directly in the Protégé-2000 API methods. When these data are inserted
GUI, and also operates directly on the into the knowledge base, the previously
latter’s knowledge bases, requiring no defined rules (using the Algernon Tab
mapping between different memory areas. plugin) are automatically triggered in the
This direct manipulation of Protégé-2000 Algernon inference machine to process the
data is very interesting, because it provides new entries and to try to recognize any
some performance gain [Hewett 2003]. pattern. Whenever the inference machine
Algernon is available as free software under discovers something, a new instance is
the Mozilla Public License as well. created reporting this inferred situation,
which can be queried later by the
The adoption of Protégé-2000 and community agents.
Algernon in the prototype implementation
has made possible using Protégé-2000's 4.3 Integration of the proposed MAS
graphical interface directly to create into TelEduc
ontologies and knowledge-base rules. This is
The prototype integration into the TelEduc
LMS has started by focusing on the learners'
8
http://algernon-j.sourceforge.net/doc/algernon- participation monitoring in two of the
protege.html
TIDIA First Workshop – E-Learning Project November 2004

7. communication tools of this LMS, the possible to easily extend the system to
Discussion Forums and the Portfolio. This monitor new tools, since the core of the
decision is explained by the relevance of system will already be implemented and
these two tools in the context of formative most of the existing classes will possibly be
assessment. These tools, since they are used and/or extended to implement new
asynchronous, allow deeper reflection by the agents. Thus, the extension of the system to
learner at the moment of his/her monitor a new tool is expected to be done in
participation, resulting in more relevant and 3 main steps:
consistent contribution. The asynchronous
Creation of new methods for the Course
characteristic also helps the educator’s
Monitoring Agent (to collect
action in the monitoring and regulating
participation indices related to the new
learners' participation throughout the
tool);
activities development, although this process
Creation of new methods for the
is generally too expensive due to the huge
Integrator Agent (to receive requests for
volume of information to be analyzed.
reports about the new tool monitoring
The Discussion Forums tool interface and to present the analysis results);
has been adapted to make possible select the Implementation and instantiation of a
degree of relevance (not relevant, low new Activity Monitoring Agent and
relevance, medium relevance, relevant, high creation of the knowledge base of this
relevance) of a message. Thus, it will be new agent to analyze the learners’
possible to record the participation as well participation indices in activities
as to analyze the records of the learners’ developed with the new tool.
participation assessments.
The integration of the proposed MAS 5 Final Considerations
into TelEduc has been done through the
Integrator agent that runs at an user's This paper presented the definition process
browser's applet and is responsible for the of a MAS architecture to support formative
communication with server-side agents assessment in learning management
notifying relevant events at LMS. The systems. The main goals of this process was
Integrator agent is activated when necessary to propose one solution that: (1) tries to
by invoking one of its methods, which is minimize the number of agents, avoiding
done by a Javascript function call in the one fast explosion of this number when
LMS code, that passes on to the applet, and many courses and tools must be monitored;
thus to the Integrator agent in it, whatever (2) is extensible, making possible to easily
information is needed. The Integrator agent, extend the system to monitor new tools; (3)
once receives these data, must pass it to the is generic, making possible to easily adapt it
server-side agents (Activity and Course to other LMSs.
Monitoring agents), by sending an ACL- The knowledge base definition was
message through HTTP using Jade's HTTP concerned with: (1) the flexibility, making
MTP (Message Transport Protocol). Once possible the rules adaptation according to
the server-side agents receive the data, they the educator’s needs; (2) the easy rule set
process it according to predefined protocols, extension, by disconnecting the rules’
and the communication between the edition and system implementation.
Integrator agent and the server-side agents is
done for that given piece of information that The Gaia methodology employment
server-side agents needed to know. was fundamental in the analysis and design
phases of the MAS architecture, helping
After the implementation of the MAS mainly in the roles and agent types
for the monitoring these two tools, it will be definition. In the implementation phase, the
TIDIA First Workshop – E-Learning Project November 2004

8. utilization of Jade, Protégé-2000 and (multiple choice questions, true or false
Algernon technologies was fundamental and subjective questions) to supply
because they significantly made easier and TelEduc users’ needs [Freitas 2002].
sped up this process. The next steps of this The proposed MAS architecture should
project involve: (1) to finish the agents’ be appropriately extended to support
behaviors implementation; (2) to refine the these tools monitoring.
knowledge bases of Discussion Forums and
This model explores the ease of
Portfolio Activities Monitoring Agents; (3)
recording anything that happens in a
to design the MAS user interface, that
computer-mediated distance course, in order
challenges to present a huge volume of
to provide formative assessment support, by
information and to maintain the consistence
means of agent technology.
with the current TelEduc user interface.
The proposed MAS is part of the 6 References
formative assessment support model Bellifemine, F. ; Caire, G.; Trucco, T.; Rimassa, G
proposed to TelEduc in [Otsuka 2002b, (2003) . JADE Programmer’s GUIDE.
Otsuka et al. 2003]. The other researches Available at:
that compose this model are briefly http://sharon.cselt.it/projects/jade/
described below:
Cerny, R.Z. (2001). Uma reflexão sobre a
One proposes and implements the avaliação formativa na educação a
Assessments Tool, aiming to make distância. UFSC, 2001. Available at:
easier the assessment management (by http://www.anped.org.br/24/tp1.htm
storing and loading observations and Ferreira, T. B. (2003). Gerenciador de Avaliações:
grades assigned by educator during the Uma Ferramenta de Auxílio à Avaliação
proposed activities development) Formativa para o Ambiente de Educação a
[Ferreira 2003]. These research results Distância TelEduc. Master Thesis, Institute of
have been used in the proposed MAS Computing/State University of Campinas,
architecture, providing important December, 2003.
learners’ participation qualitative indices Ferreira, T. B.; Otsuka, J.; Rocha, H. (2003).
to be analyzed by the agents of the Interface para Auxílio à Avaliação Formativa
proposed architecture; no Ambiente TelEduc. In: Proceedings of XIV
Simpósio Brasileiro de Informática na
Another that explores interface agents to Educação (SBIE 2003). Rio de Janeiro,
do an initial selection of the relevant November, 2003.
messages of one chat session, according
Freitas, C. E. F. (2002). Desenvolvimento de uma
to educator’s interests [Lachi, Otsuka e
Ferramenta de Avaliação Objetiva para
Rocha 2002; Lachi 2003], in order to Ambientes Computacionais de EaD. Master
reduce the educator overload during the Qualify, Institute of Computing/ State
analysis of a chat session log. In order to University of Campinas, December, 2002.
integrate this new chat tool into the
Gennari, J. ; Musen, M. A.; Fergerson, R. W. ;
proposed architecture, its interface
Grosso, W. E.; Crubézy, M.; Eriksson, H. ;
should be adapted to make possible Noy, N. F. ; Tu, S. W. (2002) The Evolution
indicate the relevance degree of each of Protégé: An Environment for Knowledge-
selected message (not relevant, low Based Systems Development, 2002.
relevance, medium relevance, relevant, Available at: http://protege.stanford.edu/
high relevance). Besides, the proposed
Gluz, J. C.; Viccari, R. M. (2003) Linguagens de
architecture should be extended to make Comunicação entre Agentes: Fundamentos
possible monitor this new chat tool; Padrões e Perspectivas. In Vieira, R., Osório,
The last one aims the development of a F., and Rezende, S., eds., Proceedings of III
set of exercises management tools Jornada de Mini-Cursos de Inteligência
TIDIA First Workshop – E-Learning Project November 2004

9. Artificial, vol. 8. Campinas: SBC. 53-102,
July, 2003.
Otsuka, J. L.; Rocha, H. V. (2002) Avaliação
Gomez, M.V. Avaliação Formativa e Continuada Formativa em Ambientes de EaD. In:
da Educação Baseada na Internet. Proceedings Proceedings of XIII Simpósio Brasileiro de
of VI Congresso Internacional de Educação a Informática na Educação (SBIE 2002). São
Distância. ABED, 1999. Leopoldo, 12-14 de novembro, 2002
Hadji, C. (2001). Avaliação Desmistificada. Porto Otsuka, J. L., Ferreira, T. B., Lachi, R. L., Rocha,
Alegre: Artmed, 2001. H. V.(2003) Um Modelo de Suporte à
Avaliação Formativa no Ambiente TelEduc.
Hewett M.S. (2003). Algernon in Java
Revista Brasileira de Informática na
Documentation – Available at: http://algernon-
Educação, SBC, 2003.
j.sourceforge.net/doc
Perrenoud, P.(1999). Avaliação: da excelência à
Hopper, M. (1998) Assessment in WWW-Based
regulação das aprendizagens entre duas
Learning Systems: Opportunities and
lógicas. Porto Alegre: Artes Médicas, 1999.
Challenges. Journal of Universal Computer
Science 4, no. 4 (1998): 329-347. Available at: Rocha et al.(2002). Projeto TelEduc: Pesquisa e
http://www.jucs.org/jucs_4_4/assessment_in_ Desenvolvimento de Tecnologia para
www_based/paper.html Educação a Distância. Prêmio de Excelência
ABED/Embratel 2002. IX Congresso
Lachi, R. L. Chapa: Um Agente de Interface para
Internacional de Educação a Distância da
Ferramentas de Bate-papo em Ambientes de
ABED, September, 2002.
Ensino à Distância na Web. Master Thesis,
Institute of Computing/ State University of Rocha, H. (2002). O ambiente TelEduc para
Campinas, 2003. Educação à Distância baseada na Web:
Princípios, Funcionalidades e Perspectivas de
Lachi, R; Otsuka, J.; Rocha, H. (2002). Uso de
desenvolvimento. In: Moraes, M.C. (Org).
Agentes de Interface no Suporte à Análise de
Educação à Distância: Fundamentos e
Sessões de Bate-Papo. In: Proceedings of 5th
Práticas. Campinas, SP:Unicamp/Nied, 2002,
Workshop on Human Factors in Computer
pp. 197-212.
Systems (IHC 2002). Fortaleza, October,
2002. Thorpe, M. (1998) Assessment and 'Third
Generation' Distance Education. Distance
Musen, M. A.; Fergerson, R. W. ; Grosso, W. E.;
Education 19, n. 2, 1998: 265-286. (EJ 582
Noy, N. F.; Crubezy M., & Gennari, J.
131).
H..(2000). Component-Based Support for
Building Knowledge-Acquisition Systems. In: Vaucher, J. Ncho, A. (2004). Jade Primer.
Proceedings of Conference on Intelligent Available at:
Information Processing (IIP 2000) of the http://www.iro.umontreal.ca/%7Evaucher/Age
International Federation for Information nts/Jade/JadePrimer.html.
Processing World Computer Congress (WCC
Wooldridge, M.; Jennings, N.R.; Kinny, D. (2000)
2000), Beijing, 2000.
The Gaia Methodology for Agent-Oriented
Nelson, G. E. (1998). On-Line Evaluation: Analysis and Design. Journal of Autonomous
Multiple Choice, Discussion Questions, Essay, Agents and Multi-Agent Systems Vol. 3. No.
and Authentic Projects. In: Third Teaching in 3 285-312.
the Community Colleges Online Conference,
Kapiolani Community College, Hawaii, April
7-9, 1998. (ED 430 695).
Otsuka, J. L. (2002). Um Modelo Baseado em
Agentes de Interface para o Suporte à
Avaliação Formativa em Ambientes de
Educação a Distância. Campinas: Instituto de
Computação da UNICAMP. Doctoral Qualify.
Institute of Computing / State University of
TIDIA First Workshop – E-Learning Project November 2004