Bonjour à tous,

Vous êtes cordialement invité.e.s à l'évaluation du Predoc III de Divyat Mahajan, le 29 août à 10h (mode hybride).

Jury

Abstract

Despite significant advances in representation learning, success in out-of-distribution (OOD) generalization and reasoning remains elusive. There is growing interest in incorporating causal principles into representation learning as a principled approach to tackle these challenges. Causal factorization helps with efficient OOD generalization as it enables us to model distribution shifts as sparse mechanism shifts, hence we can adapt faster to them. However, a major challenge is that the causal factors responsible for the data generation process are not directly observable. Therefore, it is essential to learn representations that disentangle these latent (causal) factors of variation from high-dimensional observations. This thesis proposal aims to explore the various frameworks that identify latent factors with theoretical guarantees and develop practical approaches that leverage them for constructing reliable machine learning models.

First, we build on existing work that utilizes auxiliary information for disentangled representation learning, focusing on a multi-task supervised learning framework. We demonstrate that it is possible to identify latent variables even when labels do not render the latent variables conditionally independent, challenging this common assumption from previous research.

We then shift our focus to unsupervised disentanglement, introducing a class of decoders inspired by object-centric learning methods, which we term additive decoders. Our findings show that additive decoders can disentangle latent variables under minimal assumptions on their distribution, without the need for any weak supervision. Unlike prior works, we establish a formal connection between disentangled factors and their compositional ability when using additive decoders. Finally, we propose future directions that extend additive decoders to more flexible additive energy models. Preliminary results suggest that additive energy models can achieve compositional generalization with discrete factors, offering potential benefits for the task of group-robust prediction.