We are happy to invite you to Edward Hu's Predoc III defense on Tuesday, August, 22nd, at 10:00 am (hybrid).

Vous êtes cordialement invité.e.s à la présentation du sujet de recherche de  Edward Hu, le mardi 22 août, à 10h00 (hybride)

Title: Building a Reasoning Machine

Date: August 22nd, 2023 - 10:00am-12:00pm (Montreal time)

Location: Auditorium 1 - 6650 rue Saint Urbain

Link:  https://meet.google.com/jqi-bdys-hzc 

Modern deep learning excels at directly modeling high-dimensional data on a large scale. Human intelligence, however, involves modeling unobserved low-dimensional reasoning steps. We cast the problem of reasoning as the learning of latent variable models with a low-dimensional latent space that exhibits inductive biases such as sparsity and modularity. The difficulty of learning such models comes from the need to perform inference on the intractable posterior over compositional latent objects.

We adopt generative flow networks (GFlowNets), algorithms that train amortized samplers from energy functions, for learning latent variable models in a novel algorithm called GFlowNet-EM.

More generally, we advocate for the separation of the knowledge representation, which benefits from better generalization through a shorter description length, and the machinery that performs inference over it, which benefits from the representation power of large neural networks. Two concrete follow-up directions are proposed. In the first direction, we separate the knowledge in a pre-trained language model from the machinery that performs inference over that knowledge, using tools from GFlowNets. In the second direction, we explore the discovery of meaningful building blocks of knowledge under a Bayesian framework enabled by GFlowNet-EM.