A Test Suite for Photorealistic Rendering and Filtering

Rendering photorealistic images has been a long-standing problem in computer graphics. Photorealistic image synthesis is still a vibrant and active research field with many open problems. Most offline methods use path-sampling techniques to evaluate the rendering equation to account for sophisticated light-transport effects. However, this procedure takes up to hours, where the inaccuracy of the initial estimation shows up as noise in the resulting images. In order to alleviate this, sophisticated light-transport algorithms and a number of noise-filtering techniques have been developed. Despite the fact that a large body of research exists in both directions, there are no standardized datasets that enable us to adequately assess their strengths and weaknesses. It would be of utmost importance to be able to compare existing light-transport and noise-filtering algorithms in a scientifically sound way. In this project, we will therefore create such a dataset, and provide the following contributions: (1) a set of scene descriptions that can be used to test individual features of these systems, e.g., dealing with a variety of material models, high-resolution geometry, textured inputs, and a variety of lighting effects. (2) a large number of rendered images of these scenes with different noisiness, auxiliary buffers to maximize compatibility with the state-of-the-art noise filtering algorithms, and fully converged reference images for easy comparisons against the denoised outputs, and (3) a method to ensure parameter coverage, so that the dataset does not become prohibitively large, but still covers salient rendering configurations, that reveal the most interesting cases. In summary, we propose to create a fertile ground for assessing the quality of different photorealistic rendering techniques. We believe that this would lead to significantly higher quality scientific works in the field.

The generation of photorealistic images is an important aspect in many areas, including movie production and architectural visualisation. In order to mix images of computer-generated objects with real-world film footage, those artificial objects need to look as realistic as possible. Some movies are even completely computer generated, and the imagery needs to look believable in order to immerse the viewer into the story. In the architectural context, it is often important to be able to predict how a room or building will look before it is built. Such pre-visualization images need to resemble reality as closely as possible. All this can be achieved by performing computationally expensive simulations of light transport on the computer. Researchers are continuously developing more advanced and efficient algorithms for this purpose. However, it is not trivial to assess how well a newly invented algorithm performs. One reason is that all algorithms have different strengths and weaknesses. For example, an algorithm might be efficient for a certain scene but inefficient in a different one. Thus, it is important to test algorithms on a wide variety of different scenes to reliably evaluate them. Furthermore, these algorithms rely upon computations that involve randomness. This randomness manifests itself as noise in the images, which makes the comparison of different images and algorithms challenging because it can skew the results of the evaluation. Unfortunately, commonly used methodologies for comparing different rendering algorithms do not take this noise into account. In this project, those challenges were addressed by creating a new open data set that includes many different and challenging test scenes to help researchers test and evaluate new algorithms. The data set follows a modular approach in order to support easy recombination of assets to build new scenes. For easy distribution, the data set is made available via a newly developed web repository, which supports easy download and sharing of the data sets and additional user-created test scenes. Furthermore, new statistically motivated methodologies and tools for comparing different rendering methods were researched that take the inherent noise in the images into account, allowing for a more reliable evaluation of algorithms. The results of this project - data sets as well as algorithms - are openly available to support future research into more advanced rendering methods for the generation of photorealistic images for all kinds of practical applications.