Modern Functional Analysis in Computer Graphics

Creating photorealistic images has always been an essential goal in computer graphics. The image generation process builds on a complex mathematical construct, the so-called rendering equation. This equation defines how light interacts with surfaces in a virtual scene, and involves complex surface description models that describe important effects like reflections, glossy surface interactions, and indirect illumination. Solving this equation can be achieved by investing a large amount of time and computational resources, but intelligent methods have been found that greatly speed up the calculations up to an interactive or even real-time frame rates. Both interactive as well as any non-interactive applications such as computer games, visual effects, architectural lighting simulation, urban and automotive design, disaster simulation and many other applications that depend on an accurate light simulation, profit from efficient ways to calculate light transport. These methods can be categorized by being a part of the mathematical field of functional analysis where a large body of research exists because it forms the basis for scientific fields such as quantum mechanics, chaos and ergodic theory, vision and signal processing besides countless specialized applications in areas like structural mechanics, simulation and other engineering problems. Applications of Fourier or Laplace transformations, Spherical Harmonics or Wavelets, just to name a few important approaches, are ubiquitous. However, despite the considerable amount of research work devoted to finding methods to calculate and analyze the complex light transport in a virtual scene, they remain challenging issues and many inherent properties of light transport are largely unknown. Over the course of the last 10 years, a more general form of wavelets, named anisotropic wavelets that introduce directionality to the basis definitions have been proposed. In particular, curvelets and contourlets have already proven to be powerful tools in astrophysics, seismology, fluid dynamics and vision due to their unique properties optimized for natural signals. Yet, anisotropic wavelets have not been considered for light transport, though they have several advantages over standard wavelets such as a higher sparsity or near-optimal representation. Therefore, the main goal of this project is to develop methods based on anisotropic wavelets that calculate all aspects of light transport more efficiently, delivering a higher image quality with fewer resources, including an adaptation to all principal domains used in computer graphics. Due to their properties, anisotropic wavelets also form an excellent foundation to perform a fundamental multi-scale and multi-directional analysis of light transport which leads to a better understanding and deeper into the process of light transport in virtual scenes.

Creating photorealistic images has always been an essential goal in computer graphics. This goal can nowadays be achieved with brute-force methods that use a large amount of time and resources. This is unfortunate, as fast and even real-time solutions to the global illumination problem would have many practical applications: computer games, architectural lighting simulation, urban design reviews, automotive applications, disaster simulation and many other applications depend on an accurate simulation of light transport. The main goal of this project was to use the recent advances in applied functional analysis to develop novel and more efficient approaches to the light transport problem. Beyond this goal, we also studied how humans experience the rendered images, which led to important insights that are likely to change how different light transport effects (e.g., soft shadows, anti-aliasing) are modeled in computer graphics research.When running most off-line photorealistic rendering techniques, after a short time, an initial noisy image appears, and as more ray samples are computed, the noise gradually vanishes after minutes or even hours. We have experimented with filtering techniques that are able to clean up some of these noisy artifacts, thereby speeding up the rendering process significantly by making it possible to already use the early rendering results.In the area of interactive rendering, we have created a set of techniques that are able to model subsurface light transport in real time, opening up the possibility of efficiently rendering translucent materials, such as skin, marble, wax and plant leaves. This work has attracted significant media attention and is expected to show up in many of the video games of the future. We have also developed a method to approximate global illumination light transport effects interactively. Using this technique, a complete scene can be light-mapped in the order of seconds to minutes, allowing fast and consistent previews and first-person navigation complex scenes.In summary, during this project, we have endeavored to push the limits of off-line and real-time rendering, while taking into consideration important insights regarding how humans perceive the generated images. Many of these techniques were published and presented at high-ranking peer-reviewed venues, and we expect many of them to be appealing choices for industrial use. Our efforts also resulted in a fair number of awards, prizes, interviews and media articles both inside and outside of Austria.