Agent-based image analysis of remote sensing data (ABIA)

Since the beginning of the millennium two major new technologies have emerged within the remote sensing community: very high resolution (VHR) remote sensing data and object-based image analysis (OBIA). Both of which have led to a paradigm change in analyzing remote sensing data: from pixel-based to object-based methods. The latter allows the analysis of remote sensing data beyond spectral statistical parameters, using further object properties such as shape and spatial context. Although spatial and thematic accuracy of the results were improved through OBIA, the creation of robust, object-based solutions for automated image analysis of a set of images or even large image archives still remains hardly feasible. The major reasons are given by the high complexity of image contents - especially of VHR data - and the hardly predictable variability of the objects` qualities in diverging image data. In order to obtain acceptable results, the developed OBIA rule sets still need to be adapted manually according to changing image and object properties. Respectively, the degree of automation for typical remote sensing applications, such as updating geo data bases, change detection and environmental monitoring is limited. As a potential approach of solving this problem we investigate the coupling, extension and integration of OBIA with the agent-based paradigm: (software-)agents can act autonomously in complex environments and react flexible to unpredictable changes. Additionally, if embedded in a Multi Agent System (MAS), they can interact and cooperate with each other in order to achieve common and individual goals. These fundamental abilities allow agent-based systems to deal with complex and unpredictable situations, as well as with incomplete information in a much more flexible and robust manner compared to conventional systems (e.g. object-based). Therefore, agents and agent based systems are meanwhile used in a variety of applications which require a high degree of flexibility and robustness. In GIScience, agents are mainly used for simulating complex space-temporal processes. Nevertheless, in this research project we are focusing on the process controlling aspects of agent-based systems, namely: how we can use principles of the agent-based paradigm for flexible and adaptive control of processes for image analysis of remote sensing data. We hypothesize that we can increase the robustness and transferability of image analysis methods with this approach. Necessary adaptations to changing image data and varying object properties will be performed autonomously. In particular we will investigate a) how we can transfer agent-based principles to object-based image analysis of remote sensing data in general (conceptual framework), b) how and to what degree we need to extend on recent OBIA methods, and c) how we can realize agent-based principles in already existing OBIA environments.

Modern remote sensing is only of value for our society if we can extract the information hidden in it in a meaningful and reliable way. However, automatically gathering this information is still challenging. Key technologies such as Object Based Image Analysis (OBIA) allow accurate and meaningful results since they incorporate expert knowledge from the application domain and the image processing domain. Despite various promises from industry, a fully automated and reliable image analysis is not possible today at least in the remote sensing domain. Nevertheless, the ever increasing number of remote sensing data and the growth of remote sensing archives require more automation. Some recent methods are automated. Nevertheless, the vast majority necessitates manual adjustments according to the images variations. Unfortunately, these variations are hardly predictable in their kind and amount due to reasons, such as atmospheric conditions, different sun angles, seasonal effects or sensor specific conditions. Thus, methods and solutions successfully applied for one scene might fail when applied to another scene. Technologies capable to react flexibly and robust on these unpredictable image variations by design could increase the automation of remote sensing image analysis significantly. Against this background the aim of ABIA is to investigate whether it is possible to increase the degree of automation of (object-based) remote sensing image analysis by incorporating software agents. Software agents are already applied in various fields wherever autonomous, goal driven, flexible and robust behaviour of hard- and/or software is necessary, such as robotics, autonomous driving, web crawlers or bot nets. Multiple software agents can be organized in so-called Multi Agent Systems (MAS) with agents of different capabilities but common goals - comparable to ant colonies. MAS are also used to simulate societal or biological processes such as settlement simulations, migration movements or prey-and-predator models in so-called Agent Based Models (ABM). Within this project we created a conceptual framework for ABIA which integrates concepts from agent-based programming and OBIA. The framework has been implemented in differ- ent ways and was tested using comprehensive scenario. As we found out, it is possible to let soft- ware agents autonomously adapt existing rule sets and methods and thereby improve existing classification results. We could also demonstrate that image regions can behave like software agents and autonomously try to improve themselves in order to represent objects of interest in the image as good as possible. Although our results are not perfect, they confirm our ambitious long-term re- search goals and open up new research questions such as: how to deal with multiple correct results? Could learning mechanisms improve the agents behaviour and if so, how? Could the ABIA technology also be used for simulating complex systems? Can we extend the ABIA technology for Big Data analysis in the context of remote sensing and beyond?