Discriminative Learing of Bayesian Network Classifiers

Over the last decade, Bayesian networks have become the method of choice for representation of uncertainty in machine learning. Bayesian networks are used in many research areas such as bioinformatics, computer vision, speech recognition, error-correcting coding theory, and artificial intelligence. Currently, the research is focused on two main issues. First, much work is devoted to finding more efficient approximate inference algorithms. Second, there has been much interest in learning the parameters and the structure of Bayesian networks from data. Basically, there are two main paradigms for learning in the machine learning community: generative and discriminative learning. There is a strong belief in the scientific community that discriminative classifiers have to be preferred in reasoning tasks. The aim of the proposed research is to work on discriminative structure and parameter learning methods for Bayesian networks and to propose conditions for discriminative structures to be sufficient even trained only with maximum likelihood parameter training. Additionally, we want to perform an extensive experimental comparison between the developed discriminative approaches and well known generative methods. For the experiments, we want to use data sets from the UCI repository and from a surface inspection task available at our institute.

Humans produce large quantities of data. Image and audio data, e.g. diagnostic imaging methods or audio recordings, contain usually uncertain information. Solely logic-based methods are often inadequate for evaluation of this kind of data. Therefore, Bayesian networks can be applied to find a probabilistic interpretation of the data. Over the last decade, Bayesian networks have become the method of choice for representation of uncertainty in machine learning. We performed research on discriminative structure and parameter learning methods for Bayesian networks. In particular, we introduced a simple computationally efficient order-based greedy heuristic for learning discriminative structure within generative Bayesian network classifiers. Furthermore, we developed discriminative parameter learning algorithms resulting in improved classification performance. All developed algorithms have been applied to speech and handwritten digit classification problems.