Algebraic Statistics and Symbolic Computation

Algebraic Statistics is a research area which has gained a lot of popularity in the past few decades. As the name suggests, it combines two main branches of mathematics: statistics and algebra. The purpose is to apply results and methods from algebra to solve problems arising in statistics. The algebraic methods employed here are related to symbolic computation (computer algebra) and deal with symbolic summation and integration problems. By design these methods can only operate on inputs from the class of holonomic functions. However, this class is rather large as it contains many elementary functions and numerous special functions. Very recently, in 2011, two new algorithms have been proposed which deal with holonomic functions: the holonomic gradient method (HGM) for their numerical evaluation, and the holonomic gradient descent (HGD) for finding local minima. HGM and HGD are hybrid numeric-symbolic algorithms, which can be applied in statistics to evaluate important quantities of interest, such as normalizing constants for families of probability distributions, or maximum likelihood estimates. They are therefore an interesting alternative to the Monte-Carlo simulation, at least for some classes of problems. In several applications the HGM and HGD have already proven to be very powerful and robust methods. In this project we want to apply the HGM and the HGD to important problems from statistics: inter alia, to evaluating the distribution of the eigenvalues of Wishart matrices, to the conditional maximum likelihood estimation of the generalized hypergeometric distribution for contingency tables, and to the performance analysis of MIMO wireless communication systems. These applications pose many challenging questions to symbolic computation, some of which we intend to answer in the frame of the project. Another central goal of the project is the development of new algorithms for the treatment of holonomic functions, and their implementation. The character of the project is both international and interdisciplinary, as it involves researchers from Austria and Japan, statisticians and computer algebraists. From the computer algebra point of view, the interesting aspect of the project is the stimulation of scientific exchange between two symbolic computation communities which have worked on similar problems, but using different methods. From the statistics point of view, we expect substantial progress in the theory of multivariate distributions, and the promotion of a novel and very useful methodology.

Algebraic Statistics is a relatively new research area and has gained a lot of attention in the past two decades. As the name suggests, it combines two main branches of mathematics: statistics and algebra. Its purpose is to apply results and methods from algebra to solve problems arising in statistics. The algebraic methods employed here originate in symbolic computation (computer algebra) and deal with symbolic summation and integration problems, by means of the so-called holonomic systems approach. By design these methods can only operate on inputs from the class of holonomic functions, however, this class is rather large as it contains many elementary functions and numerous special functions. In 2011, two new algorithms have been proposed which deal with holonomic functions: the holonomic gradient method (HGM) for their numerical evaluation, and the holonomic gradient descent (HGD) for finding local minima. HGM and HGD are hybrid numeric-symbolic algorithms, which can be applied in statistics to evaluate important quantities of interest, such as normalizing constants for families of probability distributions, or maximum likelihood estimates. They are therefore an interesting alternative to the widely used Monte-Carlo simulation, at least for some classes of problems. In this project we have applied the HGM to several statistical problems, for example, to estimate the distribution of the largest eigenvalues of real Wishart matrices, which is crucial in the performance analysis of wireless communication systems. These applications posed many computational challenges, so that this research at the same time promoted the field of symbolic computation considerably. Two software packages for the calculation of zonal polynomials, which are of interest to statisticians, have been produced. Another central goal was the creation of new algorithms for the treatment of holonomic functions. In the frame of this project, we have developed desingularization techniques for D-finite systems and for q-difference equations, we solved a long-standing conjecture by Wilf and Zeilberger on the equivalence between holonomicity and properness for hypergeometric terms, and designed algorithms for computing exact solutions of nonlinear algebraic differential equations. These results have been implemented in mathematical software that is freely available for other researchers to use. Finally, several results in the fields of statistical physics and combinatorics were achieved. To name just a few of them: (1) Diagonals of rational functions are ubiquitious in mathematical physics, and we demonstrated how to express these objects in terms of well-known special functions (hypergeometric and Heun functions). (2) We devised a new technique for computing the number of assembly modes of rigid frameworks. (3) We answered an open question on the minimal number of monochromatic Schur triples.