The thesis addresses the problem of the visual localization of mobile agents in challenging scenarios. As challenging scenarios, we consider the environments populated by humans, i.e., the environments in which humans and robots coexist and cooperate. A good example of such an environment is an automated warehouse where humans and robots cooperate to increase the efficiency of the warehouse operation. The safety requirements for such coexistence are divided into several levels, one of which concerns the redirection of robots to avoid human-robot encounters. The locations of the robots are already known to the warehouse management system and the remaining element, localization of humans, requires a special approach since the conditions in the warehouse do not satisfy the static environment assumption of all common localization approaches. As minimal changes in the warehouse appearance are allowed, the localization method provides a real-time location estimate using only wearable sensors and a processing unit. The doctoral thesis presents the method for robust localization of autonomous agents in challenging scenarios. The scientific contribution consists of the following components: 1) A computationally efficient method for disparity estimation using stereo visual odometry based disparity search space reduction. 2) A method for stereo visual localization of autonomous agents in robotized warehouses supported by other wearable sensors. 3) A procedure for robustness improvement of stereo visual localization of autonomous agents in challenging scenarios.