Bonjour à tous,

Vous êtes cordialement invité.e.s à l'évaluation du Predoc III de Jianan Zhao, le 27 août à 8 am (À distance)

Jury

Abstract

Foundation Models (FMs) have revolutionized the fields of computer vision (CV) and natural language processing (NLP) through their ability to generalize across a variety of downstream tasks with minimal fine-tuning. However, these models primarily excel in handling one-dimensional short sequential data, leaving a gap in their ability to process more complex structured data that is prevalent in real-world scenarios. Structured data, such as those found in molecular graphs or long sequences, presents challenges that go beyond the linear processing capabilities of traditional FMs. This report introduces the concept of Structure Foundation Models, designed specifically to address the challenges posed by structured data. We focus on two primary forms of structured data: graph structures and long sequence structures and the interactions of graphs and sequences.

Specifically, we first study how to apply the existing success of sequence foundation models, e.g. LLMs, to graphs. We propose GLEM, a learning paradigm for fusing graph neural networks and language models with a variational Expectation-Maximization (EM) framework; and GraphText, a language for graphs that converts graph tasks to text reasoning problems ready for LLM. Then, we study the problem of a graph foundation model for node classification, where we propose GraphAny, a foundation model capable of performing inference on previously unseen graphs. Additionally, we also envision and discuss future work for long-sequence foundation models that are both efficient and effective.

By proposing novel solutions and suggesting avenues for future research, this report aims to contribute to the emerging field of structure foundation models.