Trainable Feature Aggregation

In many applications of image processing a number of objects - usually represented by feature vectors - is extracted from an image and needs to be classified as a whole set of objects. This is usually done by using an aggregation function that combines all the feature vectors and a classifier that operates on these aggregated features. Such combined aggregation/classification functions should fulfill a number of properties: (1) They have to handle a varying number of objects (feature vectors) in the set, since the number of objects found in an image may vary from image to image. (2) The aggregation should be able to represent more complex operations than just sum, average or simple logical combinations of the classification results for each object. (3) The result of the classification has to be invariant to the order in which the single feature vectors are aggregated. (4) The combined aggregation and classification should be adaptive and thus able to learn from data. Currently, there exist no mathematical structures that fulfill all of these properties to a high degree. In particular, the problem of trainable aggregation functions has not been investigated in great detail. Within this project we are thus aiming at the development of a trainable aggregation/classification method that fulfills all of the properties (1)- (4). We will use recurrent neural networks as trainable aggregation/classification functions. The recursive structure of the network allows a varying number of feature vectors to be presented to the network and provides sufficient flexibility to learn even complicated aggregation functions. The focus of the research work will therefore be on the relationship between the complexity of the aggregation and the structure of the network, on how to achieve order invariance of the network, and on appropriate training methods. For each of these topics we will investigate a number of different approaches, which will finally lead to a new type of neural network incorporating the desired properties.

In many applications of image processing a number of objects - usually represented by feature vectors - is extracted from an image and needs to be classified as a whole set of objects. This is usually done by using an aggregation function that combines all the feature vectors and a classifier that operates on these aggregated features. Such combined aggregation/classification functions should fulfill a number of properties: 1. They have to handle a varying number of objects (feature vectors) in the set, since the number of objects found in an image may vary from image to image. 2. The aggregation should be able to represent more complex operations than just sum, average or simple logical combinations of the classification results for each object. 3. The result of the classification has to be invariant to the order in which the single feature vectors are aggregated. 4. The combined aggregation and classification should be adaptive and thus able to learn from data. Currently, there exist no mathematical structures that fulfill all of these properties to a high degree. In particular, the problem of trainable aggregation functions has not been investigated in great detail. Within this project we are thus aiming at the development of a trainable aggregation/classification method that fulfills all of the properties (1)- (4). We will use recurrent neural networks as trainable aggregation/classification functions. The recursive structure of the network allows a varying number of feature vectors to be presented to the network and provides sufficient flexibility to learn even complicated aggregation functions. The focus of the research work will therefore be on the relationship between the complexity of the aggregation and the structure of the network, on how to achieve order invariance of the network, and on appropriate training methods. For each of these topics we will investigate a number of different approaches, which will finally lead to a new type of neural network incorporating the desired properties.