This course covers a wide range of tasks in Natural Language Processing from basic to advanced: sentiment analysis, summarization, dialogue state tracking, to name a few. Upon completing, you will be able to recognize NLP tasks in your day-to-day work, propose approaches, and judge what techniques are likely to work well. The final project is devoted to one of the most hot topics in today’s NLP. You will build your own conversational chat-bot that will assist with search on StackOverflow website. The project will be based on practical assignments of the course, that will give you hands-on experience with such tasks as text classification, named entities recognition, and duplicates detection.
Throughout the lectures, we will aim at finding a balance between traditional and deep learning techniques in NLP and cover them in parallel. For example, we will discuss word alignment models in machine translation and see how similar it is to attention mechanism in encoder-decoder neural networks. Core techniques are not treated as black boxes. On the contrary, you will get in-depth understanding of what’s happening inside. To succeed in that, we expect your familiarity with the basics of linear algebra and probability theory, machine learning setup, and deep neural networks. Some materials are based on one-month-old papers and introduce you to the very state-of-the-art in NLP research.
Do you have technical problems? Write to us: coursera@hse.ru

MV

Definitely best course in the Specialization! Lecturers, projects and forum - everything is super organized. Only StarSpace was pain in the ass, but I managed :)

TL

Jul 08, 2018

Filled StarFilled StarFilled StarFilled StarFilled Star

Anna is a great instructor. She can explain the concept and mathematical formulas in a clear way. The design of assignment is both interesting and practical.

レッスンから

Dialog systems

This week we will overview so-called task-oriented dialog systems like Apple Siri or Amazon Alexa. We will look in details at main building blocks of such systems namely Natural Language Understanding (NLU) and Dialog Manager (DM). We hope this week will encourage you to build your own dialog system as a final project!

講師

Anna Potapenko

Researcher

Alexey Zobnin

Accosiate professor

Anna Kozlova

Team Lead

Sergey Yudin

Analyst-developer

Andrei Zimovnov

Senior Lecturer

字幕

Hi. In this video, we'll talk about context utilization in our NLU. Let me remind you why we need context. We can have a dialect like this. User says, "Give me directions from LA," and we understand that we need, we have a missing slot so we ask, "Where do you want to go?" And then the user says, "San Francisco." And when we have the next utterance, it would be very nice if intent classifier and slot tagger could use the previous context, and it could understand that, that San Francisco is actually, @To slot that we are waiting, and the intent didn't change, and we had context for that. A proper way to do this is called memory networks. Let's see how it might work. We have a history of utterances, and let's call them x's, and that is our utterances. Then we passed them through a special RNN, that will encode them into memory vectors. And we take out with two utterances passed through these RNN, and we have some memory vectors. And these are dense vectors just like neural networks like. Okay. So we can encode all the utterances we had before into the memory. Let's see how we can use that memory. Then when a new utterance comes, and this is utterance C in the lower left corner, then we actually encoded into the vector of the same size as our memory, and we use a special RNN for that, called RNN for input. And when we have that, orange "u" vector, we actually, this is actually the representation of our current utterance, and what we need to do is we need to match this current utterance with all the utterances that we had before in that memory. And for that, we use a dark product with the representations of utterances we had before, and that actually gives us, after applying soft marks, we can actually have a knowledge attention distribution. So we know what knowledge, what previous knowledge is relevant to our current utterance and which is not. And we can actually take all the memory vectors, and we can take them with weights of this attention distribution, and we have a final vector which is a weighted sum. We can edit to our representation of an utterance, which is an orange vector, and we can pass it through some fully connected layers and get the final vector "o" which is the knowledge encoding of our current utterance and the knowledge that we had before. What do we do with that vector? That vector actually accumulates all the context of the dialect that we had before. And so, we can actually use it in our RNN for tagging, let's say. Now, let's say how we can implement that knowledge vector into tagging RNN. We can edit as input on every step of our RNN tagger, and that is a memory vector that doesn't change, and if we train it end to end, then we might have a better quality because we use context here. Okay. So this is an overview of the whole architecture. We have historical utterances, and we use a special RNN to turn them into memory vectors. Then we use attention mechanism when a new utterance comes, and we actually know which prior knowledge is relevant to us at the current stage and which is not. And we use that information in the RNN tagger that gives us slot tagging sequence. Let's see how it actually works. If we evaluate the slot tagger on multi-turn data set, when the dialect is along, and we actually measure F1, F1-measure here. And let's compare RNN tagger without context, and these memory networks architecture. We can see that this model performs better and not only on the first turn but also on the consecutive turns as well. And overall, it gives a significant improvement to the F1 score, like 47, comparing with 6 to 7. So, let me summarize. You can make your NLU context-aware with memory networks. In the previous weeks, in the previous videos, we actually overviewed how you can do that in a simple manner, but memory network seems to be the right approach to this. In the next video, we will take a look at lexicon utilization in our NLU. You can think of lexicon as, let's say, a list of all music artists. We already know that this is a knowledge base, and let's try to use that in our intent classifier and slot tagger.