Automatic Portable Performance for Heterogeneous Multi-cores

The efficient mapping of program parallelism to multi-core, heterogeneous processors is extremely challenging and highly dependent on the underlying architecture. The overall objective of this project is to investigate a novel parallel compiler approach that can automatically learn how to best map program parallelism to multi-core, heterogeneous platforms. Rather than hard-coding a compiler strategy for each parallel platform, we aim to explore an innovative, portable, parallel compiler approach that can automatically self-adapt to any heterogeneous hardware and can improve its performance over time. This is achieved by employing machine learning approaches which first learn the optimisation space off-line and then automatically derive a strategy that attempts to generate the "best" mapping for any user program. This predictive modelling approach can be further extended to on-line adaptation to manage contention for resources. This project is aimed at exploring multi-core system software that is "future scalable" and if successful, will have a wide range of applications from high performance computing, to desktops, to embedded mobile devices. It will allow portable performance of parallel programs across platforms. Given the increasing prevalence of heterogeneous multi-cores such an approach, if successful, will have international academic impact and be of significant benefit to EU industry. Multi-core based processors are common-place and are widely seen as the most viable means of delivering performance with increasing transistor densities. Concurrent with this rise in multi-cores has been the adoption of specialised or customised cores such as general-purpose graphics processing units (GP-GPU). Unfortunately, there is no clear way for application software to harness this performance. Current compiler technology, whose role is to map software to the underlying hardware is simply incapable of doing this, requiring significant manual intervention from the programmer, tuning for each system. As the number and diversity of cores increases, this software gap grows larger and will be the fundamental critical issue for computing systems in 5 years time.

The efficient parallelization and optimization of computer programs on modern heterogeneous parallel computers is extremely challenging and highly dependent on the underlying architecture. The overall objective of this project was to investigate a novel parallel compiler approach that can automatically learn how to best map program parallelism to modern parallel computers with GPUs (graphics processing units) that are widely available in consumer, industry and scientific computers. Rather than hard-coding a compiler strategy for each parallel computer, we aimed to explore an innovative, portable, parallel compiler approach that can automatically self-adapt to any heterogeneous hardware and can improve its performance over time. This has been achieved by employing machine learning approaches which first learn the optimization space off-line and then automatically derive a strategy that attempts to generate the best mapping for any user program. For our project we focused on OpenCL which is the first open standard for cross-platform parallel computing. Writing programs for heterogeneous systems is a challenging task due to the difference in processing capabilities, memory availability, and communication latencies of different computational resources. Our research conducted as part of this project, included load balancing of OpenCL tasks over heterogeneous compute nodes. It is important to understand, that the best performing task partitioning commonly is application, input data and machine specific. We developed a new solution based on machine learning for which we demonstrated a performance improvement of approximately 25 % over previous approaches based on a large variety of test codes. A crucial component of this solution was a prediction model which has been created based on artificial neural networks and principal component analysis that tries to estimate an efficient task partitioning and thereby guides our compiler-based optimization. This approach has been implemented as part of the Insieme source to source compiler as developed and maintained by the University of Innsbruck. Furthermore, we - for the first time - extended OpenCL to work also for clusters of GPUs and achieved an efficiency of 64 % for a variety of codes. The main development goals were to achieve performance improvement and productivity by simplifying program development. This project aimed at exploring multi-core system software that is future scalable and has a wide range of applications from high performance computing, to desktops, to embedded mobile devices. It allows portable performance of parallel programs across platforms. Given the increasing prevalence of heterogeneous multi-cores on a large variety of computers, our work and results are a good starting point to effectively use heterogeneous computers with reduced execution times and increased productivity for parallel programs.