Tracking with Structure in Computer Vision (TWIST-CV)

The task of tracking objects in image sequences is very important in computer vision. Tracking is, for example, indispensable for automatically following people in scenes filmed by surveillance cameras, or for following the position of a head and hands in a human-computer interaction application. An interesting extension is to use 3D information obtained from two or more cameras to assist in the tracking. There exist many approaches to solve the problem of object tracking. Although these approaches are successful, it is often the case that they are not robust enough, or that different approaches need to be used for different applications. Recent work by one of the project partners (PRIP) has shown that the use of matching of graph pyramids and of combinatorial map pyramids is a powerful means to solve problems in computer vision. Promising initial results have been obtained for applying this methodology to tracking. The main goal of the proposed project is to develop a general framework that enables solutions to practical problems of computer vision, in particular Tracking, using approaches that strongly use image structure. The project will make use of structural techniques such as graph and combinatorial map image representations, graph and combinatorial map pyramids and matching to attempt to provide a solution to the following tasks within a single framework: 1. Finding object correspondences in image sequences (Tracking). 2. Finding object correspondences in images taken from different viewpoints (Stereo matching). 3. Finding object correspondences in image sequences taken from different viewpoints (a combination of the above two techniques). The use of this single framework would simplify the solutions of many practical problems. In order to properly evaluate the developed algorithms and framework, we intend to rigorously compare them to existing algorithms. To do this, we will make use of existing benchmarking databases and of data arising from real applications in surveillance and man-machine interfaces that are investigated by the second project partner (ACV) in industrial research projects.

The task of tracking objects in image sequences is very important in computer vision. Tracking is, for example, indispensable for automatically following people in scenes filmed by surveillance cameras, or for following the position of a head and hands in a human-computer interaction application. An interesting extension is to use 3D information obtained from two or more cameras to assist in the tracking. There exist many approaches to solve the problem of object tracking. Although these approaches are successful, it is often the case that they are not robust enough, or that different approaches need to be used for different applications. Recent work by one of the project partners (PRIP) has shown that the use of matching of graph pyramids and of combinatorial map pyramids is a powerful means to solve problems in computer vision. Promising initial results have been obtained for applying this methodology to tracking. The main goal of the proposed project is to develop a general framework that enables solutions to practical problems of computer vision, in particular Tracking, using approaches that strongly use image structure. The project will make use of structural techniques such as graph and combinatorial map image representations, graph and combinatorial map pyramids and matching to attempt to provide a solution to the following tasks within a single framework: 1. Finding object correspondences in image sequences (Tracking). 2. Finding object correspondences in images taken from different viewpoints (Stereo matching). 3. Finding object correspondences in image sequences taken from different viewpoints (a combination of the above two techniques). The use of this single framework would simplify the solutions of many practical problems. In order to properly evaluate the developed algorithms and framework, we intend to rigorously compare them to existing algorithms. To do this, we will make use of existing benchmarking databases and of data arising from real applications in surveillance and man-machine interfaces that are investigated by the second project partner (ACV) in industrial research projects.