Advanced Learning for Tracking and Detection

The goal of this project is to significantly advance the state of the art in detection and tracking. This is possible by joining the complementary expertise of three leading computer vision labs in Europe. Medical work-flow analysis, recorded by multiple cameras, will serve as a complex test-bed to (i) pose new challenges for visual detection and tracking and (ii) to benchmark our novel algorithms on a complex real world scenario. Abstract knowledge about actions which are being performed is valuable in the operating room and can be used for many applications such as task planning. Our objective is to analyze the (large scale) events happening in the medical operating room, equipped with multiple cameras, in order to prepare the necessary input for a fully automatic work-flow analysis. From an algorithmic point of view this project will focus on the development of novel tracking methods for textured (and more importantly) un-textured objects (equipment in the operation room); person detection using a multi-camera set-up. The robustness and adaptivity of these tasks will be significantly enhanced by novel unsupervised/semi-supervised on-line learning methods which are a further focus of research within this project. Learning will be an integral part of tracking and detection. By tightly integrating these modules we expect increased learning performance because of the increased label quality and significantly increased tracking and detection performance by scene specific adapted models. The detection and tracking results will be used in analyzing behavioral patterns in the operation room. All developed components will be used in the medical workflow analysis task.

In this project part of the DACH project we investigated statistical learning algorithms to advance object detection and tracking by considering a challenging real-world scenario. In particular, we focused on medical workflow analysis, as operation rooms can serve as a complex test-bed to benchmark our algorithms and to pose new challenges for visual detection and tracking. Our objective was to analyze the events and actions happening in medical operation rooms, recorded from multiple cameras, to prepare the necessary inputs for a fully automatic workflow analysis. To this end, we did research on novel tracking and detection methods as these are the major preprocessing steps for any automated analysis system. Our research resulted in several improvements to object detection frameworks. On the one hand, we were able to improve the detection performance of the popular Hough Forest framework by incorporating an efficient re-weighting step. This approach achieved excellent results, especially in crowded situations, i.e. whenever persons stand close to each other. We increased the flexibility of such classifiers by adapting it to the online learning case. Furthermore, by analyzing the internals of these detection frameworks, we were able to significantly speed up multi-object detection, which is a crucial step towards real-time performance. On the other hand, we introduced a novel training scheme for the Random Forest framework which enables the use of well-defined loss functions, while still being able to parallelize the computations. These improved detection results also supported our research on tracking-by-detection algorithms. In particular, we introduced novel tracking approaches to cope with difficult situations, such as crowded scenes or similar visual appearance of persons. To this end, we investigated both geometric and appearance cues to reliably and robustly link object detections into target trajectories. Our research resulted in novel cost functions for data association schemes and new appearance models which significantly improved the tracking performance. In conclusion, we can say that we clearly achieved and even outreached our initial project goals. The most important results have already been published at relevant conferences and journals.