Abstract :

Adversarial learning has emerged as the learning paradigm of choice across a varied range of tasks in computer vision related applications such as image synthesis and domain adaptation. A recent but well-known direction in the area is representation learning via bi-directional graphical models that leverage adversarial training. Adversarially Learned Inference(ALI) extends Generative Adversarial Networks(GANs) to infer meaningful representations given the data through an inference mechanism. The representations are useful both for semantic analysis and as features for downstream tasks such as semi-supervised learning. In this work we study these bi-directional graphical models in detail, and propose hierarchical extensions to alleviate their common criticism of unfaithful reconstruction of the target distribution. We also showcase the usefulness of the hierarchical semantic representations both qualitatively and quantitatively. Relying on the generative capability and efficient inference pathway bestowed by the bi-directional models, we further propose a novel approach to model 3D geometry of a scene from a single RGB image as a step towards modelling the properties of real world. This has far reaching applications in Robotics and Simulations. Our approach employs an ALI based mechanism for learning true representations of the 3D scenes from image pixels. This in-turn allows us to synthesize novel scene configurations. Motivated from the positive yields of this approach, as future work we seek to model the properties of real world in a more challenging setting considering the aspects of realistic lighting (eg., area lights, diffuse reflections, shadows, etc.) and materials into account along with motion based properties.