Next-Generation Video Browsing

Video browsing is known as the interactive process of quickly navigating through the content of a video in an explorative fashion. In difference to video retrieval the purpose of video browsing is manifold: (1) get a quick overview of the content of a video and its structure, (2) find out where potentially interesting segments are located, and (3) interactively search for a specific part of content that cannot be found through video retrieval applications. Although many video browsing approaches have been proposed over the past years, there is great potential to significantly improve existing tools by exploiting features of modern mobile devices. In the Next-Generation Video Browsing project we want to translate our results from previous research on video browsing and 3D search interfaces to this interesting and challenging new domain and answer several new research questions. We focus on video browsing with modern tablet computers that provide high computing power, new interaction features, and communication functions that enable completely new use scenarios for video browsing.

Assume you have a large collection of videos without textual annotations and you want to find a specific video segment in it very quickly (e.g., in a personal video archive with recordings of the last 10 years). Which software tool would you use for that task? We argue that most users would simply employ a file browser to go from one video to another and use a simple video player software to inspect each one, because they have no better tool at hand for such situations. A simple video player, however, only allows for linear search in video content, which is cumbersome and time-consuming. In this project we investigated how to design better tools for content-based search in videos, in particularly when used on tablet devices, and how these search tools could benefit from the powerful features provided by tablets (e.g., multi-touch interaction and high computing power). We focused on improving video content navigation and visualization of the video structure through 3D models and color-sorted arrangement of images. To this end, we performed several initial user studies in order to find out how users search with common video player software and used the results for the design of novel search features. In order to evaluate these novel search features we developed software tools with new interaction models and search features and performed comparative user studies with them. The results of these studies show that the newly proposed features and interfaces provide significantly better search performance and are easier to use for the vast majority of tested users. We also showed that tablets and smartphones are capable of providing performance intensive 3D interfaces that present content from image and video collections in a very intuitive way for example an interactive color-sorted globe or an interactive three-dimensional film strip and that such representations allow to find specific content more quickly than similar 2D representations. Moreover, it was investigated how several multimedia devices could be used together for collaborative search in video, in order to jointly find desired content faster than a single person would do. Such search scenarios are important when a large amount of video data, which was recorded just recently, needs to be inspected in short time (e.g., in situations like the terrible Boston Marathon Bombings in 2013, where investigators needed to inspect all recordings of surveillance cameras for suspicious activities as fast as possible). The findings of this research project can be used for the design and development of video interaction software in all domains where content-based search in video is needed.