Deep Pictures:Creating visual and Haptic Vector Images

Due to its analytic nature, vector graphics allow many convenient and more semantic image editing tasks compared to common pixel-based graphics. However, pixel graphics are still much more common today, with few exceptions in representing text, technical CAD drawings or geographic maps. We attribute this fact to two major problems that are described in turn. We present numerous ideas for solving these problems, which we will investigate in our project. The first problem is that no efficient tools are available for creating vector graphics from natural, pixel-based images. Existing fully automatic approaches perform very poorly. As the general image vectorization problem is to some extent of artistic nature, we propose semi-automatic approaches that combine user input with powerful automatic image processing methods. As a particularly interesting sub-problem, we will merge raster images with widely varying zoom levels to form a single vector graphics representation. Another interesting sub problem is the efficient conversion of texture images with repetitive content. A second problem is the limited expressiveness associated with traditional vector graphics. Based on our recent results, we will visually improve vector representations so that natural images appear more realistic. Here we especially consider image details, as present in textured objects, without adding excessive amounts of geometry beyond what can be handled by a user. Aside from purely visual results, our research will be driven by a second, particularly challenging application of vector graphics: the creation of relief interpretations from paintings, used as learning tools (touch reliefs) for blind and visually impaired people. Exploiting the analytic nature of vector graphics as geometric support structure has the potential to significantly simplify and to speed up this tedious process. By using the result of our image vectorization as outlined above, we aim to develop techniques for shape creation of individual scene elements, including free form modeling tools and making use of available 3D objects, where a particular challenge is the modeling of undercuts in high reliefs. We will also investigate how the modeled objects can be efficiently composed to a scene. This includes tools to optimally utilize a given depth budget, and 2D scene layout so that important objects can be enlarged, as is needed for touch reliefs. The resulting system will enable artistically trained people to quickly create a visually and haptically convincing, high-quality relief interpretation. A selection of the modeled touch reliefs will be physically produced, and we will evaluate their quality and expressiveness with blind and visually impaired people, experts of fine arts and art historians. This way, our results will directly be beneficial for all parties involved, enabling more institutions to open their collection to all people without barriers.

A vector graphics is composed of a set of geometric primitives such as lines, curves or shapes in contrast to the pixel grid of common raster images. In this project we took ad-vantage of the analytic nature of vector graphics to research novel, semantically meaningful image creation and editing metaphors, which we identified as being one main reason why vector graphics is still less used compared to pixel images, with few exceptions in representing text, technical drawings and geographic maps. In one line of research, we developed novel ways to semi-automatically populate existing vector graphics with new objects, in order to create very complex images. As a key component, we invented new ways to describe how an object relates to things in its direct neighborhood. This allows us to place new objects driven by simple exemplar placements provided by a user. Starting from a general setup where only the surrounding vector graphics geometry is considered, we pushed this idea further to geometries with labels attached, such as indoor scenes with furniture, walls, windows and doors. The resulting concept can be used to efficiently furnish rooms, create entire buildings or perform complex landscaping in a meaningful way while requiring only limited, very intuitive user interaction. In a second line of research, we enhanced the expressiveness of vector graphics by researching new mathematical models that we used to model complex color gradients and shading effects. The developed models are equally expressive as the best models introduced recently, but they are a lot less computational demanding and behave more intuitive during edits and in animations. Aside from purely visual results, our research was driven by a specific second application: the creation of relief interpretations from paintings, used as touch reliefs for blind and visually impaired people. Exploiting the analytic nature of vector graphics as geometric support structure showed potential to significantly simplify and to speed up this tedious process. In particular, the novel tools developed in this project were used for touch relief modeling and composition which helps artistically trained people to quickly create a visually and haptically convincing, high-quality relief interpretation, to ease their production and to enrich them with a gesture-controlled audio guide. A selection of the modeled touch reliefs have been physically produced, and their quality and expressiveness was discussed and evaluated with blind and visually impaired people, experts of fine arts and art historians. The results of our research are directly beneficial for visually impaired people several examples emerged during the project already. In addition, we envision cultural and economic implications caused by a more widespread use of vector graphics but also more general 3D graphics, which was made possible with our research.