Interpretable and Reliable Evolving Fuzzy Systems

Predictive models are nowadays routinely used in information, control, and decision support systems, finding applications in mobile devices, technical systems, and industrial automation. Since the complexity of these systems is permanently increasing, the data-driven construction of such models using machine learning methods is becoming increasingly popular. Thus, instead of letting a human expert specify a model by hand, the idea is to induce the model from observed data in a fully or at least partially automated way. A data-driven approach to model design becomes even more appealing in dynamic environments where the input- output behaviour of a system may permanently change in the course of time. Under these conditions, a model has to be adapted in a continuous way, a task that cannot be performed by hand. In fact, from a learning point of view, this scenario is challenging and involves a number of requirements. Most notably, learning algorithms must be incremental, since storing the whole data is not possible and, moreover, re-building a model repeatedly from scratch too time-consuming. Sometimes, such algorithms must even meet hard time constraints and update models in real-time. In the recent literature, the term "evolving system" has been coined to characterize systems of the above kind, and the problem of learning and adapting models in a continuous, data-driven way has received quite some attention. In particular, the learning of evolving fuzzy systems has been studied intensively in recent years, largely motivated by the flexibility of fuzzy (rule-based) systems and the advantages they offer from a modelling point of view. By definition, the changes that an evolving fuzzy system may undergo over time are not restricted to the values of individual parameters. Instead, also the structure of the system may change. Until now, research in the emerging field of evolving fuzzy systems has mainly focused on learning models with a high predictive accuracy. Even though this aspect is of course very important, it is not sufficient from a practical point of view, since a user such as, for example, a human operator of an industrial process, is likely to refuse a learning system producing overly complex models that he cannot understand. In fact, such models will hardly be judged any better than typical "black-box" approaches such as neural networks. Without any doubt, fuzzy systems do have the potential to offer both, accuracy and transparency, and the goal of this project is to exploit this high potential. This way, we hope to lift the conception of evolving fuzzy systems from its current state as a new research direction in data-driven model design to an emerging technology ready to be used in real-world applications. More concretely, this project will produce concepts, methods, and algorithms for making evolving fuzzy systems more user-friendly. First of all, this will be achieved by developing (accuracy-preserving) methods for reducing the complexity of fuzzy models, thereby making them more transparent and possibly amenable to interpretable linguistic representations. Another important aspect and user requirement is reliability. In this regard, different types of uncertainty concerning the model itself and its predictions have to be captured and represented. Roughly speaking, the goal is to make a model "self-aware" in the sense of being able to judge its own reliability. Finally, novel visualization techniques and methods shall be developed that allow a human user to interact with the learning system in a convenient way. Jointly, these contributions will greatly increase the practical usefulness, acceptability, attraction and motivation for users to interact with this type of model, and thus provide the basis for a successful application of this novel approach in practice.

The major intention of the project was to provide concepts and algorithms for assuring a better interpretability and reliability of evolving fuzzy systems (EFS). EFS can be seen as an important part of the evolving intelligent systems community, equipped with an own journal at Springer (Heidelberg) since 2010 (Evolving Systems) and organizing yearly an international conference (EAIS) sponsored by IEEE. At the time when the proposal was written (during 2008) and finally accepted at the beginning of 2010 by both parties, FWF and DFG, there have been several approaches in the field of EFS available, which, however, were solely focused on precise modeling issues. That is, the ultimate goal of all these approaches was to achieve as much as quality as possible on the classification resp. prediction of new query points, without giving a damn about any aspects towards model interpretation, knowledge gaining and understanding. In this project, we could provide new avenues and develop new concepts in the sense to make evolving fuzzy systems more transparent and interpretable during adaptive, on-line learning processes. This may have an essential benefit for experts in knowledge gaining and understanding of the system, as well as in human-machine interaction scenarios, where the human user and the machine are supposed to communicate on a higher level (active learning and teaching). This hybrid learning concept is currently motivated and discussed in the (scientific) evolving intelligent systems community under the term Human-Inspired Evolving Machines (HIEM) and respected as one future generation of evolving intelligent systems, see http://en.wikipedia.org/wiki/Evolving_intelligent_system. Interpretability of models trained by the machine is an absolute necessity to stimulate such a communication and to lay the foundation stone of HIEM [N1]. Reliability of evolving fuzzy systems comes with two facets: 1.) to ensure stability and robustness of the models during on-line learning and 2.) to provide an enhanced interpretation of model predictions. In particular, to address interpretability in EFS, several developments were conducted, ranging from on-line complexity reduction via the assurance of several linguistic interpretability criteria (distinguishability, simplicity, completeness, rule base consistency, feature and rule importance) to visual interpretability. Reliability was handled 1.) in terms of developing algorithms which can handle drift and shift occurrences in on-line streams by introducing more flexibility in the model updates; and 2.) in terms of novel uncertainty concepts (conflict and ignorance), which could be also integrated into active learning schemes for incremental classification scenarios in order to reduce operators' feedback efforts. This may have a significant impact in the industry to reduce costs for sample annotation, supervision and monitoring systems (will be investigated in a follow-up project). The integration of the Austrian sub-project has been achieved by putting emphasis on the development of complexity reduction, linguistic interpretability and reliability concepts; visual interpretability and user interaction issues have been mainly conducted by the bilateral German partner with support of the Austrian partner. Joint publications could be achieved in the fields of complexity reduction, drift handling (for more stability) and hybrid modeling aspects as part of user interaction (successfully applied in textile industry).