CELERITY: advanCed modELing for scalabIE distRIbuted runTime SYstems

High-Performance Computing (HPC) plays a fundamental role in enabling scientific progress, as improvements in many areas of science critically depend on advances in computational modeling and processing power. The next major milestone for the HPC community is the transition from peta to exascale, which imposes difficult research challenges such as programming models for scientific productivity, scalable system software design, and energy efficiency. We propose the CELERITY environment to support the effective development of energy- and performance-efficient, predictably scalable and easy-to-program parallel applications targeting large-scale homogenous and heterogeneous HPC clusters. The CELERITY environment, thanks to its high-level programming model, assures high productivity by relieving the programmer of labor-intensive low-level concerns such as task partitioning and distribution -- which are particularly demanding when targeting heterogeneous distributed architectures. To provide high performance, CELERITY will combine novel static kernel analyses integrated with dynamic information provided by the runtime system, enabling modeling and predictions of parallel scalability as well as energy consumption for a given input application. The modeling will be based on advanced machine learning methodologies such as structural learning, leveraging the inner representation of the data to deliver accurate predictions. We plan to assess our system first on a small- scale heterogeneous cluster instrumented with very accurate energy measurement devices, and successively on several large-scale clusters with the available instrumentation provided by the computing infrastructures, using a broad collection of application benchmarks.

The Celerity project aims to provide an efficient, high-level platform for developing and running high-performance computing (HPC) applications on parallel computers with accelerator hardware. Existing programming approaches are limited to single accelerators, vendor-controlled APIs or low-level abstractions that combine multiple existing APIs. While this approach can achieve good performance, it requires developer expertise in both distributed memory parallel programming and accelerator computing, greatly increases development and maintenance efforts, and often suffers from limited performance portability. Celerity proposes a minimal set of extensions to SYCL which allows programs to target clusters of accelerators with only very minor modifications to a single-node SYCL program. Celerity uses SYCL which is a single-source, modern C++ for targeting heterogeneous parallel architectures. We have demonstrated that the Celerity API achieves a reduction by more than 50% in terms of developing a code for accelerator-based parallel computers compared to conventional approaches based on coupled APIs. While the SYCL-based Celerity API leads to significantly less complex code than a traditional approach that couple multiple APIs, it still requires developers to be familiar with the general concepts of accelerator development and all associated programming complexity. As an alternative at an even higher level of abstraction we designed and developed a high-level API which transparently maps from a functional stream processing specification to the Celerity API. The Celerity runtime system reduces the runtime of Celerity programs by leveraging data structures that allow the runtime to make work division and distribution decisions while keeping track of data location and dependencies. As part of Celerity we collaborate with a partner group from the University of Salerno which leveraged machine learning to predict the performance of various work distribution strategies to guide the optimization of the runtime of Celerity programs. Overall Celerity advanced the state-of-the-art in programming and runtime system techniques for performance-oriented programs that can benefit from parallel computers with a large number of accelerator hardware. Although existing approaches can achieve very good performance but at the cost of more complex development effort compared to Celerity.