KDD18 London: Deep Learning and Natural Language Processing with Apache MXNet Gluon

Time: Tuesday, August 21, 2018
Location: ICC Capital Suite Room 14+15+16

Presenter: Alex Smola, Leonard Lausen, Haibin Lin



While deep learning has rapidly emerged as the dominant approach to training predictive models for large-scale machine learning problems, these algorithms push the limits of available hardware, requiring specialized frameworks optimized for GPUs and distributed cloud-based training. Moreover, especially in natural language processing (NLP), models contain a variety of moving parts: character-based encoders, pre-trained word embeddings, long-short term memory (LSTM) cells, and beam search for decoding sequential outputs, among others.

This tutorial introduces GluonNLP (GitHub), a powerful new toolkit that combines MXNet’s speed, the user-friendly Gluon frontend, and an extensive new library automating the most painful aspects of deep learning for NLP. In this full-day tutorial, we will start off with a crash course on deep learning with Gluon, covering data, autodiff, and deep (convolutional and recurrent) neural networks. Then we’ll dive into GluonNLP, demonstrating how to work with word embeddings (both pre-trained and from scratch), language models, and the popular Transformer model for machine translation.

AWS Icon   AmazonAI Icon   Apache Incubator Icon   MXNet Icon   Gluon Icon


While most of the notebooks we prepared can run from the comfort of your laptop, some more interesting real-life problems benefit from more computation power. At the tutorial, we will provide each participant with a $50 AWS credit code for you to try out these problems yourself on the Amazon EC2 machines.

In preparation for the hands-on tutorial, please make sure that you have an AWS account with at least one p2.8xlarge AND one p3.2xlarge instance in EU (Ireland) available for launch. You may register an AWS account and follow the instructions on this page to verify and request for p2.8xlarge AND one p3.2xlarge instance limits in EU (Ireland) region.

Materials for the tutorial can be found in KDD18-Gluon repository.

Send an email to join #gluon-nlp Slack channel and say Hi.

Update: you may also directly use this sign-up link to join our slack channel.


Time Title Slides Notebooks
8:30-9:15 Installation and Basics (NDArray, AutoGrad, Libraries) link link link
9:15-9:30 Neural Networks 101 (MLP, ConvNet, LSTM, Loss, SGD) - Part I link link
9:30-10:00 Coffee Break
10:00-10:30 Neural Networks 101 (MLP, ConvNet, LSTM, Loss, SGD) - Part II link
10:30-11:00 Computer Vision 101 (GluonCV) link link
11:00-11:30 Parallel and Distributed Training link link
11:30-12:00 Data I/O in NLP (and Iterators) link
12:00-13:30 Lunch Break
13:30-14:15 Embeddings link link
14:15-15:00 Language Models (LM) link link
15:00-15:30 Sequence Generation from LM link link
15:30-16:00 Coffee Break
16:00-16:15 Sentiment Analysis link
16:15-17:00 Transformer Models and Machine Translation link link
17:00-17:30 Questions

Have questions? Interested in joining AWS? Contact us at amazonai-kdd18@amazon.com