Evolutionary Learning of Goal-Oriented Communication Strategies in Multi-Agent Systems

Journal of Automation, Mobile Robotics & Intelligent Systems VOLUME 9, N° 3 2015 Evolutionary Learning of Goal-Oriented Communication Strategies in Multi-Agent Systems Submitted: 8 April 2015; Accepted 28 May Alhanoof Althnian, Arvin Agah DOI: 10.14313/JAMRIS_3-2015/24 Abstract: Previous studies in multi-agent systems have observed that varying the type of information that agents communicate, such as goals and beliefs, has a significant impact on the performance of the system with respect to different, usually conflicting, performance metrics, such as speed of solution, communication efficiency, and travel distance/cost. Therefore, when designing a communication strategy for a multi-agent system, it is unlikely that one strategy can perform well with respect to all of performance metrics. Yet, it is not clear in advance, which strategy will be the best with respect to each metric. With multi-agent systems being a common paradigm for building distributed systems in different domains, performance goals can vary from one application to the other according to the domain’s specifications and requirements. To address this issue, this work proposes a genetic algorithm-based approach for learning a goaloriented communication strategy. The approach enables learning an effective communication strategy with respect to flexible, user-defined measurable performance goals. The learned strategy will determine what, when, and to whom information should be communicated during the course of task execution in order to improve the performance of the system with respect to the stated goal. Our preliminary evaluation shows that the proposed approach has promising results and the learned strategies have significant usefulness in improving the performance of the system with respect to the goals. Keywords: multi-agent system, communication strategy, evolutionary communication, and genetic algorithms 1. Introduction 52 A number of research efforts investigated the importance of communication and its impact on the performance of multi-agent systems. Studies are usually conducted by varying communication conditions and testing the performance of the system. We consider the work in [1] and [19], where experiments were carried out to study the effect of communicating different types of information when agents are assigned different tasks. As shown in Figure 1.a, the process starts by manually determining the type of information that agents are allowed to communicate - i.e. none, only goals, only beliefs, or both. Then, the average performance of the system over multiple runs is measured with respect to different metrics such as time to complete, interference, communication efficiency, and duplication of efforts. In their work, whenever agents are allowed to communicate information, they broadcast every value update once obtained. Authors concluded that varying the type of information that agents communicate can significantly affect the performance of the multi-agent system with respect to different metrics, especially if no implicit communication present. Moreover, their results showed that more communication does not always guarantee better performance. The latter conclusion has crucial implication. It indicates that even in applications where communication is free, the system designer should not allow full communication and assume that the system is performing at its best level. The progress that the work presented above made in understanding the impact of communication on MAS performance, together with the fact that system designers must choose a performance goal [1], motivated us to propose a genetic algorithm-based approach for learning a goal-oriented communication strategy. Therefore, rather than manually creating different communication conditions (as in Figure 1.a), the system designer can start from selecting his performance goal and feed it to the learning system. The system, then, learns a centralized goal-effective strategy that determines what information instances should be communicated during task execution, when, and to whom in order to achieve the best performance of the system with respect to the selected goal (see Figure 1.b). During task execution, agents execute the evolved strategy in a decentralized manner. At each time-step, each agent consults the strategy to decide whether it needs to communicate or not. Therefore, in this work, agents’ collective behavior, and hence performance, improves as a result of executing a goaloriented communication strategy, evolved offline by the GA. The goal of this work is to propose and preliminary test an evolutionary approach that automatically generates an effective communication strategy with respect to a user-defined performance goal in multi-agent systems. Our ultimate aim is to allow system designers to easily vary the goal and automatically obtain the corresponding communication strategy. Therefore, the system designer does not need to know or analyze the properties of each information instance and its effect on the performance goal of the system. This can eliminate a significant design task in developing a multi-agent system. Moreover, the proposed approach can assist system designers to find Journal of Automation, Mobile Robotics & Intelligent Systems out the potentially best performance that the system can achieve with respect to a specific goal, such as the minimum time or energy that a task takes to be completed. Therefore, a system designer will be able to choose among the performance of the system with multiple communication strategies of varying goals and select the one that has the best fit to the system’s needs. A multi-agent version of the Wumpus World [16, 22] is used in this work as a testing domain for our approach, where a team of carriers and fighters cooperate to kill wumpuses and collect gold. a) b) Fig. 1. Reversal of the (a) communication-to-performance investigation process to obtain (b) performanceto-communication learning process The remainder of this paper is organized as follows. Section 2 briefly overviews related work. In section 3, we analyze the problem of designing a goaloriented communication strategy for a multi-agent system, and explain why it is a challenge to design one manually. The Wumpus World, our test domain, is explained in section 4. In section 5, we provide detailed information on how we utilize a genetic algorithm to design a learning system that automatically generates an effective goal-oriented communication strategy. We show, in section 6, promising results in preliminary experiments with two performance goals using the Wumpus World domain, and we finally conclude in section 7 and discuss future work. VOLUME 9, N° 3 2015 2. Related Work Given the impact of using an effective communication on MAS performance, it is not surprising that approaches for learning all aspects of communication, including language, protocols, and strategies, have been proposed in the literature (...truncated)