Efficient Segmentation for Multimedia Semantic Extraction

The aim of the project is to develop efficient image segmentation algorithms for use in the automated extraction of semantics from multimedia data. Due to the immense amount of multimedia data currently accessible on the world wide web, it is incredibly difficult to locate exactly the image or video that one is looking for. Enriching this data with additional layers of automatically generated semantic metadata as well as with artificial intelligence to reason about the (meta)data, is the only conceivable way to easily search through their complex content. However, the automatic extraction of semantically rich metadata from the computationally accessible ("low-level") features poses tremendous scientific and technological challenges. The European Union has recognised these challenges by declaring Semantic-based knowledge systems to be one of the strategic objectives in the Information Society Technologies (IST) thematic area of the European 6th Framework Programme. The PRIP group of the Vienna University of Technology is a member of the MUSCLE (Multimedia Understanding through Semantics, Computation and LEarning) Network of Excellence (NoE) within this programme, which, through exchanges with the MUSCLE partners, will ensure that research carried out in this FWF project will have a Europe-wide impact. The PRIP group intends to concentrate on obtaining the best features possible for use in the extraction of semantics from images and video sequences. For example, a segmentation algorithm which takes all the available physics- based information in a scene into account to segment the scene into actual physical objects (e.g., regions made of the same material) instead of arbitrary regions influenced by lighting and shadow would simplify further stages of semantic extraction immensely. Our innovation will be in the combination of spatial segmentation techniques (e.g., the watershed algorithm) with physics-based segmentation techniques through the use of hierarchies of image partitions. In the process of reaching this goal, we also intend to make significant contributions to the fields of mathematical morphology for colour images and colour texture analysis.

The aim of the project is to develop efficient image segmentation algorithms for use in the automated extraction of semantics from multimedia data. Due to the immense amount of multimedia data currently accessible on the world wide web, it is incredibly difficult to locate exactly the image or video that one is looking for. Enriching this data with additional layers of automatically generated semantic metadata as well as with artificial intelligence to reason about the (meta)data, is the only conceivable way to easily search through their complex content. However, the automatic extraction of semantically rich metadata from the computationally accessible ("low-level") features poses tremendous scientific and technological challenges. The European Union has recognised these challenges by declaring Semantic-based knowledge systems to be one of the strategic objectives in the Information Society Technologies (IST) thematic area of the European 6th Framework Programme. The PRIP group of the Vienna University of Technology is a member of the MUSCLE (Multimedia Understanding through Semantics, Computation and LEarning) Network of Excellence (NoE) within this programme, which, through exchanges with the MUSCLE partners, will ensure that research carried out in this FWF project will have a Europe-wide impact. The PRIP group intends to concentrate on obtaining the best features possible for use in the extraction of semantics from images and video sequences. For example, a segmentation algorithm which takes all the available physics- based information in a scene into account to segment the scene into actual physical objects (e.g., regions made of the same material) instead of arbitrary regions influenced by lighting and shadow would simplify further stages of semantic extraction immensely. Our innovation will be in the combination of spatial segmentation techniques (e.g., the watershed algorithm) with physics-based segmentation techniques through the use of hierarchies of image partitions. In the process of reaching this goal, we also intend to make significant contributions to the fields of mathematical morphology for colour images and colour texture analysis.