Describing Quality Adaptation and Content of AV-media

The purpose of our project is the definition of a common framework for modeling content information and quality adaptation capabilities of audio-visual data and the implementation of this framework in a query, indexing, and retrieval system. Content-related information consist of low-level descriptions (e.g. some detected regions of interest; color histograms of these regions) and high-level descriptions of the data content (e.g. name and profession of a person appearing in the media). Our system shall be able to represent hybrid features and thus support both low- and high- level indexing, with a smooth interfacing between them. Quality adaptation related information comprises the description of adaptation algorithms and parameters as well as their effects on actual video streams. In this context, we intend to model perceptual quality of adapted streams so that the effect of adaptation can be judged more accurately; for instance a color-to-monochrome transformation of a video is only meaningful, if the adapted video has still sufficient perceptual quality. Descriptions which are stored in the meta-database, will be encoded as an MPEG-7 description. Such a model enables us to provide active components on the delivery path to the client (network routers and proxy caches) with valuable information to govern or enhance their media scaling, buffering, and caching policies. Thus, the definition and modeling of the adaptation options will be performed jointly with the project proposed by Hellwagner/Böszörmenyi. The benefits of our common modeling framework are therefore twofold: (1) the user of the multimedia system can specify more precisely her/his requirements, both content-specific and Quality-of-Service (QoS)-specific; (2) the meta-database is moved away from its maverick role of being an outsider in the coverage of the media delivery to become an active player in the distributed multimedia architecture. The realization of a query and retrieval system, adopted for the pre-requisites of the user, requires the development of a query language with related graphical interfaces. Moreover, we shall design a query optimizer which translates the queries to the retrieval system. The optimizer is the heart of our query system and allows us to manage even complex queries (e.g. give me all video clips where a player scored a goal by head from a distance of max. 10 meters after a pass over thirty meters). The cost-based query optimizer is intended to be realized on the basis of a transformation- rule system, allowing a maximum of flexibility. The introduction of an innovative multimedia algebra enables query reordering and access path selection. An important point of concern is interactivity. We therefore introduce interactive interfaces and enhance the query optimization and processing to reflect a wide range of options to manage alternative delivery services and content types. Examples of beneficiaries are mobile end users, who are not able to view high-quality videos. If a submitted query would return only such kind of media, the query processor will check for alternative media, e.g key-frames of the videos, which meet the user`s requirements. A final and important point is the integration of our prototype with the active network components of the project proposed by Hellwagner/Böszörmenyi and the demonstration of our mechanisms in a larger multimedia scenario.

A huge amount of multimedia content is created through ever new digital recorders and is then transmitted over optical, wireless and wired networks. Hence, there is an increasing need of addressing the problems in multimedia search, storage and delivery. Such a need arose as a result of the inherently ]arge data sizes and contextual factors such as device capabilities, user preferences, real-time streaming requirements, and network bandwidth constraints. In this context, the CODAC project offers a set of solutions for the problem of efficient storage, semantic search and efficient transmission of multimedia data and associated metadata in a distributed system. We started by designing a new conceptual framework for multimedia metadata in the form of a lifecycle, which includes modeling of multimedia content and context. We thereby relied an the MPEG-7 standard for multimedia content descriptions and an the MPEG-21 standard for describing the context information, such es the device capabilities and user preferences. In joint work with industrial partners, we proposed our results to MPEG, and our generic Bitstream Description (gBSD) and the MPEG-7 Index Systems became part of the standards MPEG-7/B, resp. MPEG-21. Based an the conceptual framework, we realized the MPEG-7 Multimedia Database (MPEG-7 MMDB). It is the core of CODAC, since it has to manage and to deliver all generated metadata. The MPEG-7 MMDB is an extension of an Oracle database and is able to store, index and query multimedia metadata based an the MPEG-7 standard. It consists of three innovative parts: the Multimedia Database Schema based an MPEG-7; the Multimedia Indexing Framework (MIF) supporting query optimization; and a set of internal and external libraries for programming and using the MMDB. The MPEG-7 MMDB is an active component in a distributed system. This component creates and manages metadata that describe both the source and the variations of multimedia data, being useful for content adaptation. This helps a distributed system to identify the most appropriate variation that meets the required Quality of Service (QoS). For the purpose of variation creation, variation products were defined. The widely known reduction methods such as temporal, spatial, and colour were implemented. Two new variation methods, viz., object-based and segment-based variations were introduced. A unifying framework of the variation methods, called the Variation Factory, was implemented.