1. 2.3.

Ann Arbor Spark

A2D-NLP | Your Neural Network for NLP is Probably Wrong

Thursday, February 27 2020 at 6:30pm - 8:30pm

To efficiently train Deep Learning models processing multiple examples at once—aka batching—is paramount. The rub is that in NLP data often have differing sizes. We rectify this by adding empty data to smaller examples to bring them up to size: a technique we call padding. Padding comes with its own set of problems; if you aren’t careful, calculations will include your padding elements and produce incorrect results.

In this practitioner focused talk we cover how to batch, pad, and mask a range of neural network layers common in NLP including operations that obviously require masking like mean pooling, token level losses, and attention; complex operations like the CRF forward algorithm and Viterbi decoding; and subtle operations that you might not expect to require masking such as max pooling following a 1D convolution.

About the Speaker:

Brian Lester manufactures chatbots with Deep Learning as an Artificial Intelligence Engineer at Interactions. He is a core maintainer of mead-baseline which can help you avoid most of the pitfalls outlined in the talk. It also includes the fastest bLSTM-CRF this side of the Mississippi. He’s always itching to spend more of his free time doing NLP, so if you want to collaborate on a research project, feel free to contact him.

Ann Arbor SPARK Central Innovation Center
330 East Liberty Street
Ann Arbor, MI, USA