Towards Semantic Versioning of Foundation Models

Par

Bram Adams

Université de Queen

Jeudi 26 juin 2025, 14:00-16:00 EST, Salle 3195

Pavillon André-Aisenstadt, Université de Montréal, 2920 Chemin de la Tour

Abstract: Developers of FMware, i.e., software products involving foundation models (FMs), are experiencing an explosion of model variants and versions to work and cope with, both closed and open source, because of competing model architectures, optimization of model size to available hardware resources, and alignment to downstream tasks. Organizations eager to integrate these FMs into innovative FMware need to select the most reliable model for their specific use case or determine any backwards compatibility issues with the currently used model version merely based on the models' names, marketing info and (if available) online documentation and performance data. While software engineers developed and adopted semantic versioning practices to deal with such challenges, these practices hardly exist for the different components of FMware (models, prompts, model chains and agents), forcing FMware organizations to resort to trial-and-error and (educated) guesswork of the impact of new FMware component versions. This talk presents empirical insights obtained by mining thousands of open-source FMs, prompts, leaderboards and Model Context Protocol (MCP) servers, trying to understand the challenges faced by FMware stakeholders, as well as identifying opportunities for more sustainable FM model versioning and evolution practices.

Bio: Bram Adams is a full professor at Queen's University (until June 2020: Polytechnique Montreal). He obtained his PhD in 2008 at Ghent University's GH-SEL lab (Belgium). His work on software release engineering and software analytics received the 2021 Mining Software Repositories Foundational Contribution Award, and has been published at premier software engineering venues such as ICSE, FSE, MSR, ICSME, EMSE and TSE. In addition to co-organizing the RELENG International Workshop on Release Engineering from 2013 to 2015 (and the 1st/2nd IEEE Software Special Issue on Release Engineering), he co-organized, amongst others, SEMLA 2018/19, FM+SE Vision 2030 and AIware 2024. He has been PC co-chair of SCAM 2013, SANER 2015, ICSME 2016 and MSR 2019; ICSE 2023 software analytics area co-chair, and general chair of MSR 2025. He is a Senior IEEE Member.