Fully Programmable GPU Pipelines

A modern computer system has a graphics processing unit (GPU) with enormous computational power. However, the overall hardware architecture of the GPU has not changed for almost 15 years. A fixed- function pipeline with a static sequence of stages supports a certain type of graphics application, primarily designed to deliver computer games. With new GPU programming languages such as CUDA, we can use the GPU power for computing simulations, but we cannot easily build new graphics pipelines, because certain parts of the GPU are not available using CUDA. In this project, we propose to overcome this restriction with a new software framework, which supplements the missing parts of a graphics pipeline as a framework for a GPU programming language. This is challenging, because the framework has to be very efficient, or it will not be able to compete with a standard graphics pipeline. However, we can make use of the high flexibility afforded by a software implementation to produce a competitive framework. More important than pure efficiency is that, with this software framework for graphics pipelines, we can then overcome all the restrictions of the conventional pipeline. For example, we can replace a rectangular dense framebuffer with a much more efficient irregular pixel structure. We can also replace uniform sampling patters for pixels with non-uniform ones, feeding into new Virtual Reality devices such as the Oculus Rift.

Real-time rendering is not only important in the entertainment industry, but for all types of visual applications, such as medical data visualization, simulations, education, or architecture. The real-time rendering pipeline is typically tied to efficient hardware, i.e., the graphics processing unit (GPU). However, while tight coupling to a hardware architecture yields high performance and power efficiency, flexibility is sacrificed. Thus, it is not surprising that the real-time rendering pipelines hardly changed since the introduction of programmable shading. Although programmability added flexibility within certain stages of the pipeline, the pipeline itself remains rigid. This inflexible design restricts research and development of new rendering architectures. Any novel rendering approach, which does not follow the predefined pipeline, is doomed to fail, as it can never compete in performance with approaches that fit the hardware pipeline. In this project we showed that rendering pipelines can run in a complete software approach. To reach this result, efficient dynamic scheduling of the pipeline stages is essential. Scheduling must consider multiple trade-offs, like generating parallelism and keeping data compact, allowing for the execution on arbitrary processing cores and keeping data local for faster access, or streamlining the processing and supporting dynamic decision making. Considering these trade-offs and deriving general solutions to multiple scheduling problems, we derived a complete streaming rendering pipeline, whose performance is reasonably close to the hardware pipeline, while offering unprecedented flexibility. With this approach, we showed that novel rendering algorithms can easily be derived and that small changes to the pipeline can have a lasting impact on rendering quality and rendering speed. Our pipeline cannot only be used to alter the rendering pipeline, but to also experiment with alternative hardware designs and completely new rendering pipelines. For example, we presented the novel rendering pipeline for vector graphics, typically used for font rendering and on websites. Our hierarchical rasterizer for vector graphics rendering not only outperforms the state-of-the-art hardware-supported rendering methods, but also achieves significantly better quality. Another example for the advantages of our approach is the application to virtual reality rendering on head-mounted displays. For this example, we could significantly reduce the perceived latency and thus tackle one of major issues with head-mounted displays: motion sickness. Finally, we applied the scheduling solutions devised for rendering pipelines to other domains, including sparse linear algebra operations---typically found in material simulation, dynamic graph processing---such as large social networks, and mesh processing---creation and manipulation of three dimensional models. In all domains, we outperformed the previous state-of-the-art, including handcrafted solutions from both research and industry. These surprising results show that advanced, adaptive scheduling strategies have the potential to transform multiple domains relying on efficient computation on manycore processors.