DYNAMIC RUNTIME SYSTEM FOR FUTURE PARALLEL ARCHITECTURES

Over the last decade the architecture of computing systems has undergone significant changes. Multi- core processors are now ubiquitous, and only software that is properly parallelized will run faster on new processor generations. The need for more energy efficient systems is now resulting in another disruptive technology shift towards heterogeneous multi-core systems, which combine different types of execution units with often dynamically varying performance characteristics. Heterogeneous multicore architectures add another level of complexity to the development of efficient parallel software only a diminishing number of expert programmers is capable of coping with. Given the rapid innovation cycles in processor architectures as well as the complexity and wide variety of heterogeneous parallel systems, the current approach of manually adapting and optimizing applications for different parallel architectures leads to very high costs for software and thus becomes unsustainable. Thus, a new approach to the development of parallel applications is needed where programmers only specify the potential parallelism in their programs but not how parallel execution is implemented on a specific system. Key to such a new approach is a dynamic runtime system that is capable of adapting programs at runtime to a specific architecture. In this project our goal is to develop new methods for dynamic runtime systems to facilitate the development of efficient applications for current and future heterogeneous parallel architectures and to improve performance portability across different types of heterogeneous systems. Our work will be based on the Open Community Runtime (OCR), an open specification of a dynamic runtime system, developed by leading US research institutions. We will develop new runtime techniques that support dynamic partitioning and mapping of both computations and data and that provide the ability to dynamically re-balance parallel applications in response to changing machine characteristics and application workloads. A special focus will be on techniques that improve data locality, since the performance of applications is limited by the cost of data movement. The research into dynamic runtime systems and heterogeneous parallel systems will be relevant across the whole spectrum of computing systems, from mobile and embedded systems all the way to high-end systems and supercomputers. Our developments will help to accelerate the development of much-needed higher-level programming models and domain-specific languages and to unlock the tremendous performance potential of future parallel systems for new application areas.

Recent developments of computer architectures have been characterized by a continued increase in parallelism, specialization, and heterogeneity, resulting in extremely complex architectures only a diminishing number of expert programmers are capable of coping with. Given the rapid innovation cycles as well as the complexity and wide variety of heterogeneous parallel systems, the still prevail ing approach of manually adapting and optimizing applications for different parallel architectures leads to very high costs for software and thus has become unsustainable. Addressing these issues, the overarching goal of this project was to investigate, design and implement new programming methods and tools to simplify the development of efficient yet portable applications for current and future parallel systems. Our approach is based on separating the specification of parallelism from its concrete implementation on a specific target architecture. Key to this approach is a task-based dynamic runtime system that is capable of adapting programs at runtime to a specific architecture. Major research objectives addressed in the course of the project included runt ime support for heterogeneous architectures, new locality-aware scheduling strategies for architectures with complex memory hierarchies, development of platform descriptors to facilitate program adaptation for different target architectures, mechanisms for adapting task granularity, and high-level programming abstractions. Given the increasing importance of data analytics applications and their coupling with traditional HPC simulations, we also explored new strategies for optimizing collocated applications through adaptive scheduling strategies. We believe that task-based runtime systems provide the required flexibility for efficiently realizing future application scenarios such as, for example, in-situ data analytics and coupled simulation/machine learning. Our research contributed to the advancement of the state-of-the-art in programming support for current and future parallel architectures and HPC systems. Although our research was mainly done in the context of the Open Community Runtime (OCR) and our open-source implementation OCR-Vx, our work is applicable to similar systems developed in Europe and the US. Given the continuously increasing complexity of emerging processor architectures, the insatiable demand for computational power, and the convergence of high performance computing with data analytics and machine learning, new methods for efficient and flexible software as offered by task -based runtimes have become increasingly important. We believe that our project has made significant contributions to these developments with many opportunities for future work along these lines. Our research results provide evidence to the hypothesis that asynchronous, task -based runtimes and associated programming systems are a promising way forward to the development of new programming models and application software that can seamlessly take full advantage of future processor architectures.