Learning of Bayesian Network Classifiers and Sum-Product Networks

Discriminative learning of Bayesian networks (BNs) for classification tasks is often beneficial compared to generative learning. This is particularly true in case of model mismatch, i.e. when the BN cannot represent the true data distribution. In the past, we developed maximum margin parameter learning for Bayesian network classifiers and Gaussian Mixture models. Furthermore, we used the margin objective for approximate and exact structure learning. This research is extended within this proposal. The focus is three-fold: (i) Extension of margin-based parameter learning to a hybrid paradigm merging the advantages of generative and discriminative learning. We aim at extending our learning framework to semi- supervised, missing features, and latent variable scenarios. This requires efficient inference during iterative parameter optimization. Additionally, both the discriminative and hybrid learning approach are introduced to potentially deep sum-product networks (SPNs). They explicitly represent the inference process, i.e. structures (including latent variables) exhibiting computational benefits for inference can be exploited. (ii) Discriminative search-and-score structure learning in BNs is time-consuming. We are interested in approximating the non-decomposable discriminative score by a decomposable surrogate to ease the computational costs for score evaluation in BNs. Furthermore, we aim at developing structure learning algorithms for SPNs introducing a global scoring function with an inference cost penalty. (iii) To consolidate SPNs with respect to empirical performance we will compare all developed models to popular generative and discriminative models from the deep community, i.e. restricted Boltzmann machine, auto-encoders, deep belief networks, multi-layer perceptron. Additionally, one particularly interesting recent deep model generative stochastic networks is considered.

Discriminative learning of probabilistic graphical models (PGM) for classification tasks is often beneficial compared to generative learning. This is particularly true in case of model mismatch, i.e. when the PGM cannot represent the true data distribution. The focus of this research project is three-fold: (i) Extension of discriminative parameter learning to a hybrid paradigm merging the advantages of generative and discriminative learning. Furthermore, we extended our learning framework to semi- supervised and missing features scenarios (ii) We developed structure learning algorithms for sum-product networks (SPNs) a particular type of PGM - to better model data distributions. (iii) We applied the developed models to benchmark data and to single channel source separation (SCSS), and bandwidth extension. Furthermore, we performed an extensive comparison to state-of- the-art deep models.