In the multi-agent approach to software design, the functionality of a complex system is expressed through interaction between autonomous processing entities called agents. The approach is particularly suited for problems which are either inherently distributed, where autonomous subjects emerge directly from the specification, or extremely complex, so that it is very beneficial to express the solution through decoupled autonomous specialized subsystems. Communication in a multi-agent system is usually modelled after known interaction patterns between humans or animals, so that agents oftenfeature anthropomorphic qualities such asbeliefs, desiresorintentions. Analysisofpractical aspects of multi-agent architectures in the field of computer vision implies experiments with large scale software designs. That explains, in part, a rather weak attention in scientific publications and a relatively slow progress towards industrial applications. This work considers a multi-agent system in the domain of distributed computer vision, in which the agents are organized within a multilevel hierarchical structure. The objective of the system is to track multiple moving objects in real time using cooperation of several controllable cameras. Each camera is assigned a separate network connected computer which is controlled by an observer agent, i.e. a program which processes acquired images and controls the viewing direction of the camera. Observers adapt their viewing directions in order to achieve optimal tracking results according to their responsibilities, which are related either to individual objects or to particular parts of the scene. The cooperation of observers is managed by the second kind of agents in the system, which are called coordinators. The tasks of a coordinator consist of integrating individual views into the comprehensive representation of the scene, and formulating global objectives of the distributed system into responsibilities of individual observers. Corrections of observer responsibilities often invalidate their tracking contexts which is reflected by a decreasing quality of provided data and an increasing uncertainty of the scene representation. The frequency of corrections therefore must be carefully regulated, which suggests that a robust autonomous adaptation of the observers to the changes in the scene is extremely beneficial for the correct functioning of the system. In order to achieve a greater flexibility, an extension of the basic architecture is proposed, which deals with observers being capable of autonomous movement. Following the need for their occasional absolute localization, it is suitable to introduce an additional requirement, that observers should be able to detect their moving peers as well. Consequently, it can be stated that, in such a system, moving observers have a dual role: as an observer and as a tracked object. Finally, a multilevel extension of the basic architecture is also introduced, in which each inferior coordinator acts as an observer to the assigned superior coordinator. In such an arrangement, inferior coordinators fulfill their observer responsibilities by delegating jobs to assigned agents. The system for distributed visual tracking is considered in the context of localizing a group of simple moving robots by a global vision approach. The approach implies building a common infrastructure capable of providing positions of all robots in the area, as a network service available to general public. The infrastructure could be used by independent robot controllers, which would then be able to make autonomous plans of robot actions. It is shown that such network interface can be fitted into proposed architecture, within a specialized coordinator on the top of the hierarchy.