Offline and Online Autotuning of Parallel Applications

Many scientific applications, such as weather forecast or earthquake simulations, need to be execu- ted on large, parallel machines to speed up the computation. These parallel machines are comprised of hundreds or thousands of compute nodes, where each compute node is similar to a common desktop machine. These parallel applications are most often built on top of the Message Passing Interface (MPI), which is a standard for data communication. As a result, the run-time of these applications depends on the efficiency of the underlying MPI implementation. It is therefore of utmost importance to provide the best possible MPI implementation for a given system. Much research has been done to develop scalable, efficient implementations of specific MPI functions. For this reason, MPI libraries offer a large set of algorithms and provide many run-time parameters for the purpose of adapting (tuning) themselves to a given parallel machine. In our project, we will tackle the problem of optimizing the run-time parameters of MPI libraries in an automated fashion. The problem is that current MPI libraries provide several hundreds of tunable parameters, which results in a tremendously large search space. Therefore, a brute-force approach of testing every combination of parameters would take far too long and is thus impractical. Statistical methods can help us to successively reduce the number of parameters that need to be considered. In order to select the best possible algorithm for specific use cases, we apply modern machine learning techniques. Overall, we will devise and develop a software prototype that can automatically tune MPI libraries to a given parallel machine.