COLING Tutorial

The 27th International Conference on Computational Linguistics (COLING 2018) will take place on August 20-25 in Santa Fe, New-Mexico, USA.

Title: Deep Bayesian Learning and Understanding

Time: 9:00-12:00 on Aug 20

Place: Peralta, Santa Fe Community Convention Center

Description: This tutorial will introduce the advances in deep Bayesian learning with many applications for natural language understanding ranging from speech recognition to document summarization, text classification, image caption generation, sentence generation, dialogue control, sentiment classification, recommendation system, question answering and machine translation, to name a few. Traditionally, “deep learning” is taken to be a learning process where the inference or optimization is based on the real-valued deterministic model. The “semantic structure” in words, sentences, entities, actions and documents drawn from a large vocabulary may not be well expressed or correctly optimized in mathematical logic or computer programs. The “distribution function” in discrete or continuous latent variable model for natural language may not be properly decomposed or estimated in model inference. This tutorial will address the fundamentals of statistical models and neural networks, and focus on a series of advanced deep Bayesian models including hierarchical Dirichlet process, Chinese restaurant process, hierarchical Pitman-Yor process, Indian buffet process, recurrent neural network (RNN), long short-term memory, sequence-to-sequence model, variational auto-encoder (VAE), generative adversarial network (GAN), attention mechanism, memory-augmented neural network, stochastic neural network, predictive neural network, policy gradient and reinforcement learning. We will present how these models are connected and why they work for a variety of applications on symbolic and complex patterns in natural language. The variational inference and sampling method will be formulated to tackle the optimization for complicated models. The word and sentence embeddings, clustering and co-clustering will be merged with linguistic and semantic constraints. A series of case studies will be presented to tackle different issues in deep Bayesian learning and understanding. At last, we will point out a number of directions and outlooks for future studies.

Organization: The presentation of this tutorial is arranged into five parts. First of all, we will share the current status of researches on natural language understanding, statistical modeling and deep neural network and explain the key issues in deep Bayesian learning for discrete-valued observation data and latent semantics. A new paradigm called the symbolic neural learning is introduced to extend how data analysis is performed from language processing to semantic learning and memory networking. Secondly, we address a number of Bayesian models ranging from latent variable model to variational Bayesian inference, sampling method and Bayesian nonparametric learning for hierarchical, thematic and sparse topics from natural language. In the third part, a series of deep models including deep unfolding, RNN, sequence-to-sequence learning, convolutional neural network, GAN and VAE will be introduced. The coffee break will be arranged within this part. Next, the fourth part will focus on a variety of advanced studies which illustrate how deep Bayesian learning is developed to infer the sophisticated recurrent models for natural language understanding. In particular, the memory network, sequence GAN, neural variational learning, neural discrete representation, and reinforcement learning are introduced in various deep models which open a window to more practical tasks, e.g. reading comprehension, sentence generation and dialogue system. In the final part, we will spotlight on some future directions for deep language understanding which can handle the challenges of big data, heterogeneous condition and dynamic system. In particular, deep learning, structural learning, temporal modeling, long history representation and stochastic learning will be emphasized. (Slides)