Infinite neural networks for physics-informed learning

Machine learning is a driving force behind recent societal change, technological breakthroughs and an accompanying wave of automation and digitization. From facial recognition, exploration of new materials and pharmaceuticals, to recommendations on Google and YouTube, even autonomous driving cars are promised. These ideas have now also found their way into physics, in the hope of finding new approaches to old unsolved problems, also in the engineering sciences. Since a pioneering paper by Raissi, Perdikaris & Karniadakis in 2018/19 on physics-informed neural networks, there have been previously unseen tectonic shifts in computational physics and scientific computing. During the learning process, a learning machine, in this case an artificial neural network in particular, is provided with external information about physical laws in the form of error signals. This approach is simple and powerful, but nevertheless suffers from inherent problems that are already known from computer science, such as the fact that these machines can only be successfully trained with very large amounts of data. However, the reality of the natural scientist is completely different: In most cases, only a small amount of data is available compared to the complexity of the learning machine and the associated amount of data required. The typical scenario in the natural sciences therefore revolves around "smart data" rather than "big data". This project aims to pursue a radically new approach with which physical laws can be integrated directly into the internal structure of learning machines. The approach stems from a new perspective on physics-based machine learning through the lens of probability theory. This new approach leverages the observation, that neural networks, the workhorses of modern AI research, can be described as stochastic processes. If the networks become infinitely large, or in practice simply large enough, then they suddenly behave like a Gaussian stochastic process. Special Gaussian processes in turn satisfy special physical laws in the form of differential equations. This mathematical bridge can be used to consistently translate physical laws into a neural network architecture. A machine constructed in this way on the basis of physical laws differs significantly in its complexity from machines that are merely informed by physics from the outside, and promises precision and computing time that exceed the current state of research many times over. First, however, we must use mathematical methods to understand how these infinite networks interact with differential equations.