Two Case Studies of Hybrid Approaches to NLP: Collective Social Text Mining and Enhanced Attention Mechanisms for Neural Machine Translation

Par

Xiaohua Liu

Résumé:

NLP tasks that employ statistical or deep learning heavily depend on annotated data, which is not often available. Properly integrating heuristic knowledge into NLP represents a promising way to attack the lack of annotated data. This talk introduces two groups of pilot studies that leverage heuristic knowledge to boost the performance of NLP with a limited amount of annotated data: 1) A framework of collective social text mining motivated by heuristic knowledge on a social network. It aggregates similar Tweets into a macro context and runs KNNs+CRFs or SVMs+Graph Random Walks to consider the current Tweet and the macro context; and 2) Coverage, context gate and reconstruction for neural machine translation motivated by several well adopted practices in statistical machine translation. They are implemented as auxiliary attention/reconstruction sub neural networks and can be seamlessly integrated into the encoder-decoder-attention framework.

Biographie :

Xiaohua Liu has 14 years of research experiences on NLP and its applications as a principal researcher of Huawei and project lead researcher of Microsoft. He has published 50+ papers on social text mining, machine translation and language learning. He has made contributions to several widely used productions including Bing Dictionary and Xiaodu Zaijia (Baidu Smart Speakers). He received his PhD from Harbin Institute of Technologies in 2011.