Of the two papers, I decided to focus more on the Intelligent Systems paper
because of their focus on ACL comparisons. The author makes a clear
distinction between "CORBA" and ACL as

"ACLs handle propositions, rules and actions instead of simple objects
with no semantics associated with them."

"An ACL message describes a desired state in a declarative language,
rather than a procedure or method."

I write these out explicitly because of their unique language and general
importance to the paper's main goal.
Having worked with CORBA and agent architectures, I am somewhat confused by
the author's definition of ACL. The CORBA COSS language does provide
certain capabilities to interpert rules and actions with a certain logic
state (or "propositions" as referred to by the author). An example of one
such implementation with CORBA as a so called rule handler is provided in
the "Session component" and "Entity component" on this link
http://www.orocos.org/corba.html
Therefore, the Intelligent Systems paper has some misrepresentations in
distinctions between CORBA and ACLs. After further research of the ACLs such
as KQML and FIPA, it does not seem that these new communication languages
lived past year 2000. I'm interested to know if there is a current standard
Agent communication method among researchers.
Marco Huerta

Comments about Agent Communication Languages:
First I want to point that I was amazed the moment I realized how
important language has been in the area of interactions between the
members of a community and even more when I read about how
complicated and demanding it has been coming to agreements on
communications among agents. Through this article I had a handle on
how much effort has been conducted in standardization of ACLs and the
role that many different public and private organizations have played
in these enterprises. However it is sad that with no surprise, the
detonator of such efforts is the Military Industry.
This article together with the research I have conducted brought me
to the conclusion that everyday FIPA ACL will keep on gaining more
preference in front of KQML
Concerning the first layer of the common-language problem, through
this article I realized the importance of the OMG not only in the
"syntactic translation between languages in the same family" as
stated in the article but also its consequences of the integration of
legacy systems and another standardization effort, the UML (Unified
Modeling Language). And regarding the second layer of the problem I
clearly understood the need of ontologies and how important efforts
like those conducted at Stanford are important to reach a common
agreement in ontologies' definitions.
It is clear that ACLs surpass by far Remote Procedure Calling and
Remote Method Invocation by the capabilities of ACLs to describe
desired states in declarative language rather than a procedure or
method and the consequences of this in elaborating conversations or
the exchange of shared experiences and long-term strategies to alter
interlocutors' BDI states
Comments on Team Formation
This article was by far more complicated than any other I have read
so far for this course
The first part is very clear when elaborating about joint action from
a team perspective, and its difference with purely coordinated
actions.
As Brooks stated, the theories need more elaboration when they are
being applied to real world, and as stated by the authors of this
article, it is the exposition to uncertainties of the real world what
on the one hand complicates joint activities but on the other, makes
their study more interesting.
In the second part, Joint Actions, they write "our notion of
commitment in turn, was specified as a goal that persists over time"
from my view point this does not make sense in their previous
definition of individual intentions as "internal commitments to
perform an action while in a certain mental state"
The four major challenges to any joint actions' theory that they pose
before starting their definition are very interesting.
The definition of goal is very unclear to me because I do not
understand when they say that "goals are the propositions that are
true in all these worlds "(the most desirable worlds, a subset of the
belief-accessible ones). Could we elaborate more on this in class as
well as what "action expressions" are?
The formal definitions are not difficult to understand when I read
them separately as well as the theorems cites, they completely make
sense; however, when I tried to follow the authors' discussion on
them it turned to be very difficult to understand all their points
I would like to know how to program agents' goals following their
definitions

Jimmmy Doyle

Starting with the paper by Cohen, Levesque, and Smith, I concluded
that the following key points were of the utmost importance:

Joint intentions can best be described as joint commitments to
perform a collective action while in a certain shared mental state.

The essence of team behavior is cooperation; if a team has a goal
and a plan to achieve the goal, team members are individually
committed to the successes of their portion of the plan, the overall
team goal, and other team members achieving their portions of the
overall joint plan.

Other terms and concepts that are fundamental to the understanding
of the paper include the ideas of joint commitment, weak achievement
goals (WAG), sincerity, and persistent goals.

An example of joint intentions not discussed in the journal paper is
that of the plan of a jam-based musical act. Each member of the act
(which functions as a team) may be committed to performing a piece in
a spontaneous, yet still coherent and enjoyable manner. The
individual members (agents) must be able to read each other's actions
prior to their successive executions, else the act will have no "band
dynamic" and will sound entirely incoherent. A well-established,
experienced act will know, based upon their own internal language and
typical behaviors, what to expect from each member and will perform
their own parts to the piece (even in spontaneous instances) in a
manner that sounds almost rehearsed and entirely pre-planned.
My response to this paper was that it did not grab my attention very
well. I felt as though it wasn't very well-written, and though it was
easy to understand, it was extremely verbose. I felt that the author
took otherwise simple concepts and elaborated on them to the point of
irrelevance. I cannot say that I agree with the author's interest in
this topic. I didn't believe that the paper provided anything
profound or unique to the subject.
Aside from questions regarding the paper's relevance, one question I
had was:

What are 'illocutionary effects?' And 'perlocutionary?'

I felt the article by Labrou, Finin, and Peng was much
better-presented.
The key points I came across were:

An Agent Communication Language (ACL) provides agents with a means of
exchanging information and knowledge.

The common-language problem is that of syntactic translation between
languages in the same family or between families of languages.

KQML is a high-level, message-oriented communication language and
protocol for information exchange independent of content syntax and
applicable ontology.

FIPA ACL is superficially similar to KQML except for different names
for some reserved primitives.

Some questions that occurred to me while reading the article, though,
include:

What are 'remote procedure call' and 'remote method innovation?'

What is CORBA? If if couldn't do the tasks on pg 46 that ACLs can,
then what DID it do?

What is 'Common Lisp?'

Finally, this may seem like a conceptual debate question more than
anything, but it seems odd to me that we consider these languages
being 'spoken' by the agents. Doesn't it make more sense to think of
it as another mode by which the humans who designed them are
communicating? Is this a reasonable question, or am I thinking too
much? For example, if one considers a scheduling agent that
communicates with other agents to create a hospital schedule, is it
really doing the work of scheduling, or does that goal belong to the
human who designed it? This may be a fruitless argument, but I can't
get it out of the back of my head as I read articles like this.

Thomas Nelson
KQML et al seem to be overkill for our soccer agents. In general, it seems
to me very useful to divide MAS into two broad categories:

A single programming team writes all the agent AI.

Multiple agents are written in different times by different people.

for category (1), I think we can build stronger agents by anticipating
conflict before runntime, and coming up with a rigid decision making
process. For example, only the closest agent goes to the ball, which
prevents conflict. Similarly, changing formations (from 4-4-2 to 4-3-3) is
triggered by a global change everyone can see. This removes ambiguity.
For (2), you need a more robust communication system, something like the
types described.
On the symphony as a MAS: if agent communication was going on during last
night's performance, I couldn't see it. The conductor was waving his arms
around, but I left with the impression that if he had suddenly dropped his
baton or sneezed or something, the orchestra would have continued along just
fine. Perhaps it's because I don't know enough about music. Now, what
would be really interesting is an improv symphony! The conductor starts a
rhythm and points at different sections. The section leaders choose a
baseline melody in time with the conductor, and the rest of the section
joins in. That would be a real MAS system, and I bet it would sound very
interesting.
Luis Guimbarda

The model for "establishing and discharging joint commitment"
presented in "On Team Building" was very interesting, and I enjoyed it
very much. Every premise seemed reasonable, and every point made
sense. Despite its abstractness, the description of this model was
complete enough that one could implement such a system for
team-building, given a sufficiently oriented BDI architecture.
Granted, all that was presented was a semantic specification to form
and break teams, without regard to the conditions or actions before,
during, and after the forming of those teams. However, it seems to me
that those are the domain of agents' designers.
Having said that, I do have some issues with this paper. First, of
the authors assumptions, including agent sincerity, I felt that of
perfect memory (Section 3, used in the definitions of belief, goal,
and mutual belief) was the hardest to justify, and was made with the
most ease. Of course, this is obviously an issue with humans, but I
feel that this is just as much an issue with artificial agents. Be
they robotic or software agents, they are susceptible to memory
alteration and degradation. Consider their domains.
One motivation behind building robotic agents is that they can
operate in environments unsafe and unnatural for humans. Factors in
these extreme environments--temperature, radiation, gravity,
etc.--could all make perfect memory impossible. Many software agents
are made to operate on the Internet. Whether it's referred to as the
colloquial "information superhighway" or my "information public
swimming pool", the Internet poses well known threats to the integrity
of any system accessing it--agent or otherwise.
Second, insistence on mutual belief among all parties in the team
seems unscalable. Even allowing broadcasts, each time an agent
attempts to join the team, it must receive an assertion message from
every current team member. This could be easily alleviated, however,
by organizing the team into a hierarchy (partially ordered, such that
'representatives' are responsible for their constituents, or totally
ordered, like the convoy analogy), or even partitioning the team
(possibly following the organization of experts from the Sycara
reading) such that MB within the partition is as defined, and then as
a partition held with respect to other partitions.
Thirdly, the specification doesn't allow for maintenance goals. This
is an even smaller issue than that of scalability, and may be simply
my ignorance of the formal language used. Which brings me to my last
issue with the paper: the language. The definitions made me appreciate
all the parentheses used in Lisp. While several readings did little
for my immediate understanding, I felt I could intuit the meaning
behind the more complex communicatives, like accede and assert.
However, I think that all the KQML derivatives alluded to in the ACL
article come from this tendency to intuit rather than comprehend. A
more comprehensible formal language is necessary for this
specification.
Finally, I did have 2 questions. What is a non-institutional
illocutionary act? It is 'defined' in section 5, paragraph 1, somewhat
circularly. And what is the distinction between illocutionary and
perlocutionary acts? The authors assume the usual distinction between
them, but I don't see the difference between wanting and intending an
effect, at least not how it's described in the paper.

The first article that seemed more interesting to me is the one on team
evolving.

I do agree that at some level, teams only play the strategies proposed by
the programmers just as our assignments; however I think that by citing an
article and stating that the intend of their entire development is to
demonstrate the incongruence of it, seems to me very audacious.

I think that their idea of having a reward that accounts for events other
than goals is pretty straightforward and it seems very similar to the idea
of my team to evolve skills or non basic behaviors (called ADF) and
positions (formations). And I regard the same about the fact of having the
evolving teams playing against each other and leaving out the bad
performers. What it seems not that straightforward is the idea of having
three types of tests, the empty field, against kick posts and against
Humboldt University Team.

The article is easily understandable. Even I am not familiar with the use of
libsclient the way they explain their implementation is helpful in case you
want to use their approach

What did not entirely make sense to me is:

Their scoring policy in the sense of why they are using weights that
go from most to least score while the actual scoring elements go from least
to most (the opposite way)?

Why when recombination occurs they do not want the extremes of teams
to combine. By doing these they eliminate the possibility to combine the
best player that could be in one team with the best center forward that
could be from another team. Even after they elaborate more about this
considering the role of ADFs I think that is not yet a good reason.

The article that I liked the most was the one of Action Selection where very
clearly the authors expose their implementations of:

Option-evaluation

Leading passes

Algorithms for computing interception times

Force fields

All the implementations are explained clearly even when they contain soft
Boolean expressions that are difficult to understand at the beginning but
that completely make sense.

Finally the way they present the results of their experimentations is easily
understandable and the figure of the components of the force-fields is very
explanative.

Doubt: What is a hill climbing approach to program development?

Jummy Doyle

For this week's readings, I selected to read Reactive Deliberation: An
Architecture for Real-Time Intelligent Control in Dynamic Environments
by Micheal K. Sahota and Multi-Robot Decision Making Using
Coordination Graphs by Jelle R. Kok, Matthijs T. J. Spaan, and Nikos
Vlassis. I selected these two readings because I thought they would
be best suited for a project area that will closesly resemble the
goals of my graduate research.

For my project, I am tentatively planning to focus on response time
and decision-making in intelligent control. I feel that an
investigation into this topic for the RoboCup tournament will help me
in my graduate research, which centers around real-time response for
multi-agent systems. By developing algorithms that allow for the
players in the game to respond faster and with better decisions, I
hope to be able to find methods that can be applied to the problem
facing my area of study.

Upon completing the readings, I came up with the following summaries and
questions.

Starting with the paper by Sahota, I concluded that the following key points
were of the utmost importance:

Reactive deliberation is a robot architecture that has been designed to
overcome some of the problems posed by dynamic robot environments, such as
the
goal of operating in dynamic domains while keeping pace with changes in the
environment.

Reactive deliberation integrates reactive and goal-directed activity by
partitioning the robot controller into a deliberator and an executor.

The executor is composed of a collection of action schemas - robot
programs
that interact with the environment in real-time to accomplish specific
actions.

Computations in action schemas are resricted to those that can keep pace
with
the environment, so lengthy computations are performed in the deliberator.

The deliberator is composed of concurrently active modules called
behaviours -
robot programs that compute an action that may, if executed, bring about a
specific goal.

My response to this paper was that it was very well-written and very easy to
understand. I also agreed with much of the author's arguments, especially
those referring to the need of robot architectures to be able to ask the
questions "what to do" and "how to do it" all in real-time. Also, I
appreciated this paper's relation to past readings, and even references to
them, i.e. Brooks' Subsumption architecture.

Some questions I had, though, include:

What is an "arbitrary length plan?"

Why is the lack of negotiation between agents a problem? I thought this
was
desirable because it enhanced speed?

Also, an important question that I asked myself after reading the
paper was: Will this paper help with either of my projects? The
answer is probably 'no' to both, unfortunately. I thought when
reading the abstract that it might help me to gain a better
understanding of how to create agents with quick response times that
are still focused on achieving a goal. Unfortunately, it looks as
though the paper was more concerned with robotics than I wanted it to
be.

I felt the article by Kok, Spaan, and Vlassis was much better-suited
towards my projects, but was more difficult to understand.

The key points I came across were:

An approach to the problem of enhancing coordination in a multi-agent
system
involves the use of coordination graphs.

In coordination graphs, each node represents an agent, and an edge
indicates
that the corresponding agents have to coordinate their actions.

In this architecture, the state is discretized such that roles are
assigned to
the agents and the CG-based method is applied to the derived set of roles.

Each agent is eliminated from the graph by maximizing its "local payoff."

This paper was particularly difficult for me to understand, especially
given its mathematical terminology and mapping functions. One
question that I had was:

Is there a better explanation for a "Nash equilibrium" than the one
presented in the paper?

Upon reading the paper, I decided that it might be helpful to me on
either of my projects. Because it focuses more on inter-agent
communication and computational algorithms, it definitely has more
potential than the other paper. However, that doesn't necessarily
guarantee that I will be able to apply this knowledge either.

Thomas Nelson

The two articles I read are "Evolving Team Darwin United" by Andre and
Teller, and "Effective methods for reinforcement learning in large
multi-agent domains" by Reidmiller and Withopf. I thought both
articles were interesting and thought provoking. In the "Evolving
Team Darwin United" paper, the authors make a claim about when a
system can be effectively "evolved from scratch". It's an interesting
question: when does human knowledge help solve a problem, and when
might it get in the way? It seems that one of the big advantages to
ML in this domain is that it won't fall for any false parallels
between real soccer and simulated soccer. But this method definately
has its limitations; in particular, I noticed the Darwin United team
did not try to evolve world modeling behavior. I don't know what
aspects of world modeling make it unsuitable to evolved behavior;
maybe it's just that for humans the algorithm seems clear or
managable, and so they see no point in asking a computer to evolve it?

In "Effective methods for reinforcement learning in large multi-agent
domains", I was struck by the rigorous mathematics used. They proved
that their SDQ algorithm solves a SDMDP in a finite number of steps.
I don't know enough about markov decision processes to know if that's
an impressive result or not, but it seems good to me. I wonder if
they came up with SDQ first, and then tried to prove a result about
it, or started by asking "what algorithm can gives a guaranteed
optimal behavior in a finite number of steps?" and derived the
algorithm from there. It's also worth noting that they don't attempt
to evolve their attack strategy "from scratch" as did darwin united;
they begin with a rule that the player closest to the ball must move
towards it.

Starting with the paper by Parunak, I concluded that the
following key points were of the utmost importance:

General principles stemming from behaviors of naturally occurring systems of
simple agents can be used in aritificial multi-agent systems to support overall
system behavior significantly more complex than the behavior of individual
agents.

Important aspects of a natural system to consider when attempting to model it
include: system behavior, individual agent responsibilities, and integration of
individual behaviors into the system behavior.

Some examples of natural systems analyzed in this paper include: Path Planning
and Brood Sorting by Ants, Nest Building by Termites, Task Differentiation by
Wasps, Flocking of Birds and Fish, and Prey Surrounding by Wolves.

Some important ideas for building multi-agent systems that model natural ones
include: keeping agents small (in mass, time, and sensing/action scopes),
keeping the system decentralized, providing agent diversity, and enabling
information sharing.

An example of a system that could benefit from a simpler,
less-centralized system of smaller agents might be that of a floor
cleaning system of robots. With more smaller robots, the job could
get done a lot faster. However, a system of lots of simple robots
could be more expensive and produce work of lower quality than just
one that meticulously goes over the whole floor, takes longer, but
ensures a quality performance.

My response to this paper was that it was actually very intriguing. I
thought it to be my favorite so far this year. This may be because it
was easy to understand and focused on fun issues like discussing
fascinating animal/insect behavior rather than just bombarding the
reader with technical jargon. However, I will add that I had
difficulty (and often a lack of patience) with some of the equations
designed to model the systems.

I felt the article by Svennebring and Koenig titled "Trail-Laying Robots for
Robust Terrain Coverage" was also a very interesting read.

The key points I came across were:

The notion of trail-laying robots as an alternative to agents that are
constantly aware of their location on a field may have many benefits.

Trail-laying robots need only simple sensors.

The simplest real-time search method is likely to be node counting - a
behavior that focuses on counting the number of times an area has been visited
and increasing it by one upon departure.

Trail-laying robots need not communicate with each other.

The robot "Pebbles" was designed to execute trail-laying activities by sensing
its own black pen trails and avoiding obstacles and previous trails.

A question that occurred to me while reading the article was:
- What real-life mechanisms provided for "node counting?" Was this method,
since it is consider the best real-time search method, ever employed by
Pebbles? I get the impression that the sensors Pebbles has simply detect
trails, not how heavily trafficked each cell on a grid may be. Was this the
only method investigated?

Luis Guimbarda

Following are my thoughts on each of this week's assigned articles. As
always, I permit the posting of my response. First, I respond to "Go To The
Ant".

I found an informative article discussing and comparing the AI and
philosophy Frame Problem: http://plato.stanford.edu/entries/frame-problem/

Referring to 'discrete event' agents and environments: "This model...
leads to an (unrealistic) identity between an agent's actions and the
resulting change in the environment, which in turn contributes to classical
AI conundrums such as the Frame Problem." (sec 2.4.1, par 1) I disagree that
such an identity is unrealistic. Granted, the issue of imperfect (fallible)
action must be and is taken into account in the design of most of the agents
I've seen. While most interesting environments do not allow perfect action,
it is reasonable to assume certain changes in the environment follow from
certain actions; I have little reason to doubt that the light will turn on
if I flip on the wall switch.

This paper brought up many interesting points, but the one I felt was
most important was the need for diversity. Diversity is the basis for the
fault tolerance and self-organizing (emergent) behaviour of a swarm MAS.

This diversity doesn't necessarily come from design, however. "The
important observation is that the advantage of the larger population lies
not merely in numbers, but in the diversity that results from physical
exclusion laws." (sec 4.5.1, par 2) Being in different locations provide
even identical agents with unique perspectives, and thus increases
diversity.

It's good to be prepared; that's how diversity improves fault tolerance.
"One price of being able to thrive in a constantly changing environment is a
willingness to support a diverse set of resources that go beyond the needs
of the immediate environment." (sec 4.5.1, par 6)

I appreciate and understand the notion of entropy presented in this
paper. "The system can reduce entropy at the macro level by generating more
than enough entropy at the micro level to pay its second-law [of
thermodynamics] debt." (sec 4.6, par 3) However, I'm not confident in the
validity of this metaphor. It feels forced. I'm reminded of the
misapplication of the theory of natural selection to coin 'Social
Darwinism'; natural selection, as the name suggests, applies to the natural
mechanisms of survivability, and was not meant for artificial social
structures.

Trying hard to tie "Trail Laying Robots" to the Robocup problem, I did
come up with one thing. The concept of node counting can be implemented in a
Robocup team to increase the ball distribution. The ball takes the role of
the trail laying agent, with each player as a node on an undirected graph.
In this case, the node with the ball would determine which node the ball
goes to next, and each node keeps track of it's node count, incrementing the
count when it kicks the ball. But, since agents can display no persistent
indication of their node count, the only way for agents to detect the count
of their neighboring teammates is for everyone to say their count
repeatedly. "Disconnecting" parts of the graph can keep the ball in a
particular part of the field. While this method may ensure a more even
distribution of the ball, it's actual benefit is unclear without testing. (A
thought: perhaps players only announce their node counts when they are ready
to receive the ball: when they don't have the ball already, or when they're
"open".)

The self updating probability in the agents described in "Self-Organized
Task Allocation" seems similar to the transfer of "force" in the wasp task
differentiation example given in "Go To The Ant". I imagine that,
considering the need for an "entropy leak" as stated in "Ant", substituting
the "Variable Delta Rule" (sec 3 par 2) with a force transfer system will
improve performance. If nothing else, one could test the importance of
entropy leaks so.

Thomas Nelson

I really enjoyed the Parunak paper. It seems like designing swarm
algorithms requires a similar type of thinking to designing recursive
algorithms. In both, the task isn't designing an algorithm that will
solve the problem, but designing an algorithm that will get us a few
steps closer to solving the problem, and is guaranteed to fit well
with other iterations of itself. In my introductory CS classes, we
spent a lot of time studying object-oriented design. If Parunak is
right, perhaps the next generation of programmers will study
agent-oriented design, although for many algorithms it's probably
overkill.

One part of the paper that puzzled me was principle (5), that agents
should leak entropy. It isn't clear what this means. This idea of
dissipating disorder seems nice, but it doesn't match very well with
Shannon or Boltzmann entropy as I understand them. "Insect colonies
leak entropy by depositing pheromones whose molecules, evaporating and
spreading through the environment under Brownian motion, generate
entropy." The fact that the molecules dissapate has only a minor
effect on their primary function, which is to serve as information
transmitions to other agents. This doesn't really generate entropy;
if anything, it counters it, since under shannon's model entropy is
directly related to unpredictability, and the messages increase the
predictablity of food source locations. What am I missing? These
ideas, and the relationship between evolution, entropy, and chaotic or
dynamical systems, seems fundamental to the problem of multiagent
systems. But at present I think we don't really have the vocabulary
or framework to properly understand the concepts. Perhaps a paradigm
shift is in order.

Mickey Ristroph

Reading critically is causing me to vacillate quite a bit in my
opinions on approaches to building multiagent systems. My respond to
many of the previous readings sounded similar to this:

"This {concept|algorithm|model|architecture} is a clever solution to
the particular problem described. However, I think in the larger
picture, the approach is flawed. A simple and elegant implementation
of a truly artificially intelligent system will solve this problem and
many others. Some system modeled after what happens in nature will be
a cure-all for issues in multiagent systems."

However, after reading "Go to the Ant", which makes some claims
similar to mine, I actually saw more value in all the previous
readings I had somewhat discredited. It could be argued that one of
the greatest advantages of computers is ability to program with
exactness and provability, unlike an ant colony. I am certainly still
in favor of attempts to develop a massively parallel multiagent system
with tiny, individually "stupid" agents that amalgamate to something
with real intelligence. However, I am definitely going to have to
think about this more before I decide which basket to put all my eggs
in.

Marco Huerta

The "Go to the Ant" article is extremely enjoyable, interesting and
clear from the beginning; moreover, incredibly cunning, its author
took the name from a biblical statement!

Starting with a brief history of programming it expresses very clearly
the differences of agents against other previous programming
paradigms. Basically containing code, state and control and lately
organization capabilities is how an agent is characterized; however,
for more explicitness, more elaborated definitions of MASs, agents,
environment and their composing elements as well as their relations
are presented.

Homodynamic and Heterodynamic Systems, models of coupling between
agent and environment present very different problems to be addressed
as they are described in the article.

The next part of the article presents very interesting examples of
natural MASs related to ants, termites, wasps, birds, fish and wolves,
all of them sharing principles of self-organization. The author
clearly specifies behavior, responsibilities and integration for each
of them

In section four, views from different authors are presented and mostly
important: seven principles to design MASs. Some of the most important
being: correspondence to things, smallness, decentralization,
diversity, dissipative, and concurrency.

Of particular importance for me are the industrial applications of
agents and the insight to the AARIA project mentioned.

Among other things I won't ever forget of this article is "We used to
think that bugs were the problem in manufacturing software, Now we
suspect the may be the solution!"

This is the second paper by Parunak I have read, the other related to
the AARIA project, "Manufacturing over the Internet and into Your
Living Room", is also very clear and interesting and I feel like
reading again these two articles to follow the enjoyable pattern in
which they are written.

The second article I chose was the one on Self-Organised Task
Allocation in a Group of Robots based where the goal is to find the
optimimum number of robots such that they do not interfere each other
while increasing the retrieval time to bring objects to a target
location. This is called "prey retrieval" usign task allocation.

First they describe the Lego robots' hardware and behavior that they
used in a circular arena then they explain the experiments they ran to
prove that individual adaptation during a life time leads to
self-organized task-alocation in a colony and better fit indiviualds,
in this case mechanically, are more likely to be selected for the
tasks.

In "BDI Agents", the authors claim to argue the necessity, but
not the sufficiency, of belief, desire and intention. What other
attributes, if any, can be used to supplement BDI? Further, did they
actually argue necessity? It seems to me that they asserted it by
including belief, desire and intention into their architecture.

"As the agent has no direct control over it's beliefs and
desires, there is no way that it can adopt or effectively realize a
commitment strategy over these attitudes." (BDI Logics, par 9) It
would be interesting to see where removing this constraint would
lead. The idea is not without precedent; in psychology it's called
cognitive restructuring. A simple example of this is a student who is
bad at math because he hates it. It's often suggested that the student
can improve his performance by adopting a more positive attitude
towards math.

"The conditions under which a plan can be chosen as an option
are specified by an invocation condition and a precondition..."
(Abstract Architecture, par 8) Why distinguish between those two
conditions? If a plan can't be invoked without a precondition holding,
why isn't that precondition entailed by the invocation condition, so
as to merit a separate precondition?

Initially, my impression of the reservation system as
presented in "Traffic" is that if at any time a vehicle fails to make
a reservation, then the probability of it getting a reservation in the
future is decreased because it must decelerate in anticipation of
stopping. Was this phenomenon observed? I'd imagine that consciously
attempting to reduce the decrease in probability (should it even be
the case) would significantly improve the performance of the
reservation system

Thomas Nelson

I was fortunate enough to see Kurt deliver a talk to Surge (Science
undergraduate research group), where we could see the cool simulations
run. It's an interesting idea, and I think about it alot when I'm
stuck in traffic. >From an MAS perspective, it's an excellent example
of the great things communication can accomplish. I think maybe he
gave the talk after this was published, because there are a few things
he mentioned I didn't see in the paper. One was setting the
controller so that no agent can get a reservation while a closer agent
is still waiting. Another was having a priority system for
ambulances, etc. This could be modified so people could pay for
higher light priority, like paying for first class shipping on mail.
Or it could be built so that more passengers have higher priority, to
encourage carpooling. There's still a long way to go before we can
implement his intersections, though. I think people would be more
willing to trust computer drivers on highways first, where there
aren't pedestrians and things like that.

The other article is also interesting, although I wish it had gone
into a little more detail about methods used, like we saw for the
Cohen "On team formation" article. they say their method can
implement any of 12 other methods, but I guess to know what those
methods are, I'd have to go read the cited papers.

Marco Huerta

First I discuss the BDI Agents article:

3rd paragraph, 1st column, page 2: I do not see why they say
that the system -referring to the control- is nondeterministic, if
using the decision trees and considering the values obtained from
them, the system will "determine" the actions to take; then the system
is deterministic!

6th paragraph, 1st column, page 2: When they claim: "the actions
or procedures that achieve the objectives are dependent on the state
of the environment and are independent of the internal state of the
system". I also disagree, because the system is influencing the
environment, thus indirectly, affecting what the actions need to be.

2nd paragraph, 2nd column, page 2: I think a brief
introduction to branching trees before assigning this reading would
have been very useful - I am not sure about this because probably
students in CS are very familiar with them- however, in my case, I
spent some time reading in the some of the basics of how these trees
work. First I downloaded the article by Emerson, which I found works
or worked in the CS Depatment at UT (which surprised me being the now
famous BDI based on his research). But his chapter was very long so I
quitted soon. Then I end up reading some slides from Andrew ... from
CMU. I still do not know what he means in 2nd paragraph, 2nd column,
page 3 "a payoff function that maps terminal nodes to real numbers".

4th paragraph, 2nd column, page 3: I think a figure of the branch tree
before and after the transformation that they apply would have been
very illustrative in this case

5th paragraph, 2nd column, page 3: What
is an accessibility relation

4th paragraph, 1st column, page 4: Why you would like to change a real
number probability and payoff to dichotomous?

5th paragraph, 1st column, page 4: What is closed under implication?

2nd paragraph, 2nd column, page 5: Why not to use objects instead of
structures to represent beliefs, desires, and intentions?

5th paragraph, 1st column, page 6: When they say "one way of tailoring
and thus improving the process of option generation is to insert an
additional procedure" why you need to generate options if you already
had plans or `deliberated options'?

While looking additional information on BDI I found the same article
assigned for reading in a completely different format. I wonder if
you can send the exact same article to be published in different
places.

This article left me very interested in Decision Theory and
in the work of Bratman. And though I did not understand many things
the topic definitively passionates me and I would continue researching
it.

The Traffic Reservation article: The other article was very easy and
good. Reading it took me 1/6 the time of the first. It was interesting
and I found the animations on the web very enjoyable and making easy
to understand the research and its results. However, the article
seemed to be written in a complete rush. Just like trying to submit
your homework before 10pm on Monday

I chose to read Chapter Six from the textbook for this week's reading
assignment. The chapter was very easy to understand and the content
was actually very easy to get through. I found that any questions I
had were answered as I went.

One example of a situation that can be modeled as a game matrix is
that of two people potentially becoming romantically involved. If one
person asks out the other (i.e. cooperating), the other has the option
of either accepting their offer (cooperating) or rejecting it
(defecting). The same goes for if the other person asks first.
However, both people still have the option defecting, and hence
remaining friends. The game matrix would be as follows:

i cooperates i defects
5 2
j cooperates 5 0
2 3
j defects 0 3

In this matrix, the points are assigned to the following outcomes:

If both i and j cooperate, the feelings are mutual, and a
satisfying relationship may ensue. This benefits both parties
mutually and is the best outcome for both.

However, if the feelings are not mutual, the outcome of one person
attempting to cooperate will result in the other defecting. This
person will end up feeling somewhat flattered, hence some points, but
not as satisfied as if the person had never asked, as this tends to
complicate otherwise healthy friendships. The rejected person, on the
other hand, gets no points from this outcome, as they will likely feel
embarrassed and foolish (and hence wishing they had continued to
defect).

If neither person pursues the other, this results in both people
choosing to defect. A mutual defection results in a healthy
friendship (assuming both persons still wish to maintain this
relationship) and hence, 3 points to both. This is not quite as
satisfying as the romantic relationship, but still better than a
complicated, unhealthy friendship.

Hence is the game of love.

The Nash equilibria and how a person should play the game depends
on how interested i believes j is. If i believes j is uninterested,
then i would be better off remaining friends with j. However, if i
believes the feelings are mutual, i should attempt to cooperate
romantically with j, therefore achieving the highest amount of points
for both. The problem arises when i (or j) mistakenly believes that j
(or i) is interested, when in fact, this wasn't the case. Here, one
person ends up rejected and embarrassed, and the other is deprived of
a friend. This is the worst case scenario.

The Nash equilibria depend on the believes of the people involved. If
i cooperates, j can do no better than to cooperate (assuming the
feelings are mutual - i.e., j believes i to be a suitable match). If
this isn't the case, j can actually do no better than to defect. But,
then, if j were to defect, i would have done no better than to defect.
Hence, there are two true equilibria.

Marco Huerta

Based on the author's explanations it seems clear how the Nash
Equilibria arise. However, I would like to understand more thoroughly
why for the table of symmetric scenarios presented some times we have
only one Nash Equilibrium point and why for others we have two.

There is a sentence I did not quite understand: page 166: "For the
result seems to imply that cooperation can only arise as a result of
irrational behavior, and that cooperative behavior can be exploited by
those who behave rationally"

The reading in general was very interesting and especially, the
examples and some remarks made by the author make it very enjoyable.
Besides I think he is very clear in the use and explanation of the
notation. Nevertheless, I found some errors like in the table
presented for the Stag Hunt where the lower-left corner is wrong.

This was a very philosophical enjoyable reading

Luis Guimbarda

In this response, I chose to identify an application not mentioned
in the reading that can be modeled as a matrix game. As always, I
permit the posting of this response.

I had trouble coming up with more abstract situations that can be
modeled as a matrix game, but I do remember a game show called "Friend
or Foe" that is similar to the prisoners' dilemma. Teams of two would
compete with each other for cash, and each round a team would be
eliminated, finally leaving one. Then, the members of the final team
would indicate whether they were friend or foe by making a gesture of
an open palm or closed fist, respectively. This gesture was made in
secret, and simultaneously revealed. If both indicate friend, they'd
split the cash winnings evenly. If one was a friend and the other a
foe, then the foe received all the cash. And if both were foes,
neither would get any cash. So, let the C represent the cash winnings
for the team, with players 1 and 2. Then the game matrix would look as
follows:

If a player chooses to be a friend, he can gain half the cash or
none of it, and if he chooses to be a foe, then he can gain all the
cash or none of it. Since every outcome of one strategy is not
preferable to every outcome of the other strategy, there is no
dominant strategy.

However, since the total gain is C if at least one player is a
friend and 0 otherwise, it would be rational for either player to
choose to be a friend. Knowing this, a player could become a foe and
take the whole cash pot. But, once a player has chosen to be a foe,
the other can't improve his position by changing his choice; he will
get no money regardless. Therefore, there are three Nash equilibriums:
when either player is a foe and the other a friend, and when both are
foes.

I always wonder why, more often than not, both players chose to be
foes. It make sense after this analysis, and further exemplifies the
rule that "the house always wins".

Mickey Ristroph

I had heard about many of the Game Theory models and example problems
discussed in McCain's book, but I had never read anything consolidated
explanation using consistent language. So, I found the reading very
interesting. However, I am having a little bit of a problem
understanding how this is going to tie in with the other aspects of
multiagent systems we have discussed. The generic question "How can I
maximize my benefit?", asked often in the text, sounds like maximizing
the fitness function in genetic programming. However, the approaches
are entirely different. Game Theory describes ways to optimize certain
classes of fitness functions, whereas genetic programming can be used
given any well-defined fitness function. It is the same difference
between optimizations in mathematics. One way to find extrema of a
function is to find the points at which the derivative is
zero. However, we don't know how to symbolically find derivatives for
all functions. There are numeric approaches that are more generic but
perhaps less effective, such as the simplex algorithm. Game Theory, in
this regard, seems to be a layer of abstraction on discreet calculus.

After completing the readings for this week, I came up with the
following ideas for experiments for my team's final project:

For each of the experiments, I decided to focus on the aspect of
player positioning in our project. Since our project is centered
around the coaching program, the use of statistics is crucial. The
program makes use of different log files to identify patterns in game
play. For my experiments, I propose comparing the live game play to
log files where a pattern has been implimented and attempting to find
a correlation.

The null hypothesis in each case is that there will be a correlation.
If this proves to be incorrect, then the program should note that the
pattern has not been implimented, and will therefore not make a
declaration.

T-Test: For the t-test, I propose using the idea of grid spaces and a
matrix that will be incremented by one each time the observed player
enters a particular space. (Our group is actually using this method,
with the grid spaces referred to as "bins.") Then, histograms will be
made of both the X and Y grid spaces to see where the player happens
to position himself most frequently. Since there is no telling how
often a player will move (or if he will at all), the t-test must be
used here. Otherwise, comparing the live gameplay log file and the
pattern log file will prove almost impossible because movement cannot
be paired. Once the data is taken, the t-test can be used to see if
there is, in fact, a correlation between the player's positioning
during the live game and that of the pattern log file.

Paired T-Test: For the paired t-test, I propose using the idea of
recording the player's position at each cycle. This way, assuming
that both the live game file and the pattern file have the same amount
of cycles, the different files can be paired such that they may be
compared in this manner. Through this test, the average positions as
well as the standard deviations may be analyzed to identify a
correlation. If one in fact exists, then the program should make the
proper declaration.

Chi-Squared Test: For the chi-squared test, I propose first analyzing
the data from the pattern files and allowing them to be the "expected"
results. Then, the program should analyze the positioning of the
player (with either the grid spaces method or the cycle method) in the
live gameplay and treat this data as the "observed" data. To
calculate the chi squared value, I would suggest using the "expected"
standard deviation, but the "observed" x values and mean. This should
allow for a valid way of analyzing the chi squared value. If there is
a correlation between the two log files, then the chi squared value
should approach the number of cycles or the number of recorded
repositionings (depending on which approach was used) and the program
should make the proper declaration. In other words, the difference
between each position and the mean in the "observed" data should be
close to the standard deviation of the "expected" data. If no
correlation exists, the chi squared value will be "much" bigger than
the number of cycles or recorded repositionings and no declaration
should be made.

Marco Huerta

First experiment: A t-test could be used to prove that after using our
genetic algorithm, our team has effectively improved its playing
capabilities.

The null hypothesis is: Is the mean score of the unevolved team equal
to that of the evolved?

And the alternative hypothesis is: Is the mean score of the evolved
team higher?

The experiment would be conducted like this:

Initialize my unevolved team with random formation
values and have it play against the three opponent teams. Save the
average score against the three teams. Repeat this step 50 times

After evolving the team, I would have the team playing
against the three opponent teams fifty times too and save the average
score against the opponent teams. Since the formation is the
characteristic that has been evolved for my team, it is not necessary
to assign it random values.

Calculate overall average score an variance for each
team and determine the confidence interval for an alpha=0.05

Conclude that the overall average means are different
with 95% confidence if they fall outside the null hypothesis
interval.

Second experiment: An analysis of variance ANOVA to determine against
which of the opponent teams, the listos team is better fitted.

The null hypothesis would be that the listos team is equally fitted
to play against any of the opponent teams

The experiment would be run like this:

I would have the listos team playing against each of the
three opponent teams fifty times and collect the scores.

Obtain statistics from the competitions against each opponent

Then I would use JMP software to obtain the ANOVA table
feeding the values from step 2.

Finally I will analyze the results with an alpha=0.05 level
of significance

Thomas Nelson

For our experiment, we plan to develop good positions for our team. We can do
a T-test where we play many games with the old positions against a particular
team or suite of teams, and then play the same games with our new positions,
and compare the average score differential between the two teams. We would
test to see if the results seem statistically better. For our experiments, a
paired t test doesn't really make sense. However, we could do chi-squared
tests, where each we compare our new and old teams against a set of opposition
teams, something like:

opp1 opp2 opp3
old x y z
new u v w

Then we could learn if our team seems to have improved particularly against
one opponent.

Upon completing the reading for this week, I came up with the following
summaries and questions.
I concluded that the following key points were of the utmost importance:

The reading focussed on multiagent negotiation in situations where
agents may have different goals, and each agent is trying to maximize
its own good without concern for the global good.

The main question is what social outcomes follow a given protocol
which guarantees that each agent's desired local strategy is best for
that agent.

A solution x is Pareto efficient - i.e. Pareto optimal - if there
is no other solution x' such that at least one agent is better off in
x' than in x and no agent is worse off in x' than in x.

In Nash equilibrium, each agent chooses a strategy that is a best
response to the other agents' strategies.

The reading discussed different interaction protocols for various
mechanisms of interaction, including voting, auctions, bargaining,
markets, contracting, and coalition formation.

An auction consists of an auctioneer and potential bidders.

Some common auction settings include private value, common value,
and correlated value. Different protocols include the English,
first-price sealed bid, Dutch, and Vickrey auctions.

The computational cost of full lookahead may be prohibitively great.

The market can have two types of agents: consumers and producers.

The operational motivation behind market mechanisms is that the
agents can find an efficient joint solution while never centralizing
all the information or control.

The capability of reallocationg tasks among agents is a key
feature in automated negotiation systems.

Coalition formation in characteristic function games (CFGs)
includes three activities: coalition structure generation, solving the
optimization problem of each coalition, and dividing the value of the
generated solution among agents.

My response to this paper was that it did not grab my attention very
well. I was also very frustrated by reading it because I thought it
was far too loaded with confusing equations and terminology and not
nearly enough real-world examples. I felt as though this paper was
extremely difficult to understand, even to the point where I could
barely even conjure intelligent questions about it. But the best I
could do would include the following:

What are NII, EDI, etc.?

In voting, how did the first and 1rst and 3rd desideratum get
relaxed? What did they even mean? (pg 15)

What does Theorem 5.2 mean? And 5.3?

In regards to pg. 18, if we're taxing people who's vote changed the outcome,
then wouldn't the people who voted 'no' to the pool win again?

What exactly is counterspeculation?

I don't understand coalition stability.

In the contracting section, why are we suddenly talking about
'oracles?' And by 'oracles' does the author mean what I think he
means?

As you can see, this paper left me thoroughly confused. Sorry,
but I think this is my least favorite reading this semester.

Marco Huerta

This is a very interesting chapter of a researcher that just came to UT
this semester. It presents a nice summary of the state-of-the-art in
multiagent negotiation for self-interested agents.

The first section after a brief introduction presents different
criteria to evaluate negotiation protocols, one of the most
interesting being: stability. From my particular interests in
efficiency I find the "Computational Efficiency" criterion
particularly appealing, since as presented by the author provides an
incredible interesting approach when analysis the trade off between
the cost of the process and the solution quality. Finally, from this
section where the author states that some games do not present Nash
Equilibrium, I would like to ask for one example.

The next section deals with different interaction protocols. The first
of them, voting, leaves me with a doubt: How irrelevant alternatives
are defined such as that in the mentioned desiderata for ideal social
choice rule they do not alter consumer rankings while under the
relaxation of the third desiderata they can split the majority?
Moreover is there a typo? Because I consider that is not the relaxation
of asymmetry and transitivity but of independence of irrelevant
alternatives: the fifth criterion instead of the third. In the same
section the Borda count protocol and its paradox are quite interesting.

Insincere voters are a special case and I would like that Dr. Stone
elaborates more on:
"A protocol is said to implement a particular social choice function if
the protocol has an equilibrium whose outcome is the same as the
outcome of the social choice function would be if the agents revealed
their types truthfully".

From 5.3 I do not understand how if each agent's type has its
preference outcomes' order, it can happen that some agent gets his most
preferred outcomes chosen no matter what type the others reveal. How
can this happen if the social choice function is truthfully
implementable in dominant strategy?

I consider the next part: Auctions easier to read and though the author
considers interesting problems that rise in this protocols including
bidder collusion I would like to pose these 2 cases: when all the
bidders on a contract agree who is going to win and they agree to round
robin different contracts and when one bidder in a contract buys
(bribes) the auctioneer as to make it appear as the winner even when he
is not.

The last part to be read: bargaining is also appealing. In particular I
would like to explore more in class the formula of theorem 5.8 for
Axiomatic Bargaining Theory. The Strategic counterpart is easier to
understand especially after two weeks of game theory.

Thomas Nelson

The authors lay out 6 criteria for a good voting system. Of these, only the
first 4 seem interesting and useful to me. The last 2 don't seem necessary.
Yes, in human terms we prefer not to have dictators, but if the system is
constructed in a way that provides equal opportunity dictatorship, this seems
ok to me. Also that the scheme should be independent of irrelevant
alternatives seems dubious to me.

Also, I think it's interesting that the authors don't mention bidding as a
type of voting, i.e. each person pays for the weight of their vote; since they
discuss bidding, it seems like a rational extension, and probably the closest
to what happens in real life.