Copulas in Machine Learning Workshop 2011
TechTalks from event: Copulas in Machine Learning Workshop 2011
We are still uploading slides and videos for this event. Please excuse any discrepancy.
High-dimensional Copula Constructions in Machine Learning: An OverviewWith the ``discovery'' of copulas by machine learning researchers, several works have emerged that focus on the high-dimensional scenario. This talk will provide a brief overview of these works and cover tree-averaged distributions (Kirshner), the nonparanormal (Liu, Lafferty and Wasserman), copula processes (Wilson and Ghahramani), kernel-based copula processes (Jaimungal and Ng), and copula networks (Elidan). Special emphasis will be given to the high level similarities and differences between these works.
Invited Talk: Exploiting Copula Parameterizations in Graphical Model Construction and LearningGraphical models and copulas are two sets of tools for multivariate analysis. Both are in some sense pathways to the construction of multivariate distributions using modular representations. The former focuses on languages to express conditional independence constraints, factorizations and efficient inference algorithms. The latter allows for the encoding of some marginal features of the joint distribution (univariate marginals, in particular) directly, without resorting to an inference algorithm. In this talk we exploit copula parameterizations in two graphical modeling tasks: parameterizing decomposable models and building proposal distributions for inference with Markov chain Monte Carlo; parameterizing directed mixed graph models and providing simple estimation algorithms based on composite likelihood methods.
Copula Mixture Model for Dependency-seeking ClusteringWe introduce a Dirichlet prior mixture of meta-Gaussian distributions to perform dependency-seeking clustering when co-occurring samples from different data sources are available. The model extends Bayesian mixtures of Canonical Correlation Analysis clustering methods to multivariate data distributed with arbitrary continuous margins. Using meta-Gaussian distributions gives the freedom to specify each margin separately and thereby also enables clustering in the joint space when the data are differently distributed in the different views. The Bayesian mixture formulation retains the advantages of using a Dirichlet prior. We do not need to specify the number of clusters and the model is less prone to overfitting than non-Bayesian alternatives. Inference is carried out using a Markov chain sampling method for Dirichlet process mixture models with non-conjugate prior adapted to the copula mixture model. Results on different simulated data sets show significant improvement compared to a Dirichlet prior Gaussian mixture and a mixture of CCA model.
Expectation Propagation for the Estimation of Conditional Bivariate CopulasWe present a semi-parametric method for the estimation of the copula of two random variables X and Y when conditioning to an additional covariate Z. The conditional bivariate copula is described using a parametric model fully specified in terms of Kendall's tau. The dependence of the conditional copula on Z is captured by expressing tau as a function of Z. In particular, tau is obtained by filtering a non-linear latent function, which is evaluated on Z, through a sigmoid-like function. A Gaussian process prior is assumed for the latent function and approximate Bayesian inference is performed using expectation propagation. A series of experiments with simulated and real-world data illustrate the advantages of the proposed approach.
Robust Nonparametric Copula Based Dependence EstimatorsA fundamental problem in statistics is the estimation of dependence between random variables. While information theory provides standard measures of dependence (e.g. Shannon-, Renyi-, Tsallis-mutual information), it is still unknown how to estimate these quantities from i.i.d. samples in the most efficient way. In this presentation we review some of our recent results on copula based nonparametric dependence estimators and demonstrate their robustness to outliers both theoretically in terms of finite-sample breakdown points and by numerical experiments in independent subspace analysis and image registration.