Bonjour à tous et à toutes,

Hello everyone,

Vous êtes tous et toutes cordialement invité.es à assister à la soutenance de thèse de Junyi Li (RALI).

You’re cordially invited to the PhD defence of Junyi Li (RALI).

Title: Robust, Efficient, and Knowledge-Augmented Text Generation with Pre-trained Language Models.

Date: jeudi 4 septembre 10h00 (4st septembre 10AM (ET))

La soutenance aura lieu en ligne :

https://umontreal.zoom.us/j/82310045459?pwd=xb9frpHu9ZqVJDJ0I6LGaWJWGgIlKU.1

Jury

Abstract:

Pre-trained Language Models (PLMs) have significantly advanced the field of text generation. However, their practical application is often hindered by challenges related to systematic capability evaluation, high computational costs for training and inference, and limitations imposed by static and outdated internal knowledge. This thesis addresses these critical challenges to make PLM-based text generation more robust, efficient, and reliable. First, we develop ElitePLM, a comprehensive evaluation framework that systematically assesses the general language abilities of various PLMs. Second, we propose PTG (Prompt Transfer for Text Generation), which leverages prompt-based transfer learning to effectively transfer knowledge from source tasks to new generation tasks with minimal parameter updates. Third, to tackle inference inefficiency, we introduce ELMER, a non autoregressive model, which integrates an early exit strategy with a novel Layer Permutation Language Modeling (LPLM) pre-training objective, significantly speeding up generation while maintaining competitive performance. Fourth, to overcome the limitations of PLMs’ internal knowledge, we present UniWeb that augments PLMs with dynamic and comprehensive knowledge retrieved from the online Web. Collectively, the methodologies and frameworks developed in this thesis contribute to a more thorough evaluation of PLMs and offer novel solutions for their efficient training, rapid inference, and enhanced factual grounding.