Generalized information theoretic approaches for history-dependent processes

While a newborn can become a composer, politician, physicist, actor, or anything else, the chances for a 65 year old physics professor to become a concert pianist are practically zero. This illustrates a common property of many complex systems: Their future evolution depends on the history of how they have evolved up to now. Literally hundreds of examples of these kind of systems exist: Financial markets, ecosystems, political regimes or biological organisms. In general these systems evolve in time. In evolving systems, innovations and novelties are built from previous stages of the system. This leads to an entangled structure of these systems, many features of which can only be explained by looking at its history. Therefore, we refer to them as systems with history dependence. In spite the ubiquity of history-dependent systems in the real world, it is fair to say that they have so far been almost inaccessible to quantitative and predictive science, due to a number of mathematical and statistical difficulties. Obviously the lack of a quantitative understanding of history- dependent processes makes them difficult to manage, and often they are beyond any control. These difficulties arise from the impossibility to understand them as sequences of independent stages, which invalidates the classical approach taken by physics when dealing with systems with a large number of elements. In addition, the statistical patterns emerging from these systems show usually what physicists called scale-invariance, an intriguing property by which the statistics remains invariant no matter the scale we perform the observations. A series of recent mathematical developments based on information theory open the door to explore history-dependent systems through a solid and quantitative mathematical perspective. Based on these recent achievements, in this project we want to develop a unified framework to deal with history-dependent systems in a way which is analogue to the study of physical systems. The theory we want to develop would include the current standard approaches (those to systems with no history-dependence) as the particular case where history-dependence vanishes. Our aim is to provide a theoretical tool that could help us to understand the properties of these systems, predict its behaviour in simple (but informative) cases, and eventually better understand the emergence of scale- invariance in complex systems. We will apply our new mathematical tools to explain several representative systems to try to explain the emergence of the scale-invariant patterns observed in them: The behaviour of individuals in a virtual society, urban movement and information flows through internet. The achievement of our objectives would represent a significant step towards the understanding of the key properties of history-dependent systems, some of them of crucial importance for our everyday life.

The public vision of randomness is a dice, thrown independently many times. This idea underlies modern information theory. Although this metaphor has been extremely useful to understand countless natural and artificial systems, it lacks a crucial ingredient for the understanding of complex systems with randomness, namely, path dependence. Path dependence does not exclude randomness, but tells us that decision events of the past constrain - or eventually, render possible - the potential events of the future. It defines, therefore, a much more sophisticated notion of randomness which is closer to our intuition about real systems. How do information theoretic concepts look under this view? How is the statistics changed in systems that are path dependent? Both questions are crucial for the understanding of many stochastic complex systems. We discovered that the concept of information in such systems is much more involved than expected. Entropy, or information gain, takes a new mathematical form that differs from the standard view, providing very different predictions than classical statistical inference with standard entropy. This result is important and challenges the use of inference methods based on standard information theory. The new concept shows the necessity to incorporate additional basic assumptions on the behaviour of the system that are usually neglected. At the level of prediction of statistical patterns, we showed that the predictions using the modified entropies differ completely from those using the standard form. Amasingly, within our new framework, we can capture and understand the origin of fat-tailed distributions, that dominate the statistics of complex systems. This ubiquity can practically not be understood within the standard framework. Our results could explain also the ubiquity of fat-tailed distributions in "evolving systems". Moreover, the results can be used in a practical context. They could become important for the management of traffic, production systems or internet navigation.