Neurosymbolic Reinforcement Learning

Par

Abhinav Verma

University of Texas, Austin.

Résumé:

Recent advances in Artificial Intelligence (AI) have been driven by deep neural networks. However, neural networks have certain well-known flaws: they are difficult to interpret and verify, have high variability, and lack domain awareness. These issues create a deficiency of trust and are hence a significant impediment to the deployment of AI in safety-critical applications. In this talk, I will present work that addresses these drawbacks via neurosymbolic learning in the reinforcement learning paradigm. Neurosymbolic agents combine experience based neural learning with partial symbolic knowledge expressed via programs in a Domain Specific Language (DSL). Using a DSL provides a principled mechanism to leverage high-level abstractions for machine learning models, and establishes a synergistic relationship between machine learning and program synthesis.

To overcome the challenges of policy search in non-differentiable program space we introduce a meta-algorithm that is based on mirror descent, program synthesis, and imitation learning. This approach interleaves the use of synthesized symbolic programs to regularize neural learning with the imitation of gradient-based learning to improve the quality of synthesized programs. This perspective allows us to prove robust expected regret bounds and finite-sample guarantees for this algorithm. The theoretical results guaranteeing more reliable learning are accompanied by promising empirical results on complex tasks such as learning autonomous driving agents and generating interpretable programs for behavior annotation.

Biographie :

Abhinav Verma is a PhD Candidate at the University of Texas at Austin, where he is advised by Swarat Chaudhuri. His research lies at the intersection of machine learning and formal methods, with a focus on building intelligent systems that are reliable, transparent, and secure. His work builds connections between the symbolic reasoning and inductive learning paradigms of artificial intelligence. He is currently supported by a JP Morgan AI Research PhD Fellowship.