Attention for Neural Networks, Clearly Explained!!!

I feel like I am watching a cartoon as a kid. :)

I was stunned when you start the video with a catch jingle man, cheers :D

Hello, Thank you for the video, but I am so confused that some terms introduced in original 'Attention is All You Need' paper were not mentioned in video, for example, keys, values, and queries. Furthermore, in the paper, authors don't talk about cosine similarity and LSTM application. Can you please clarify this case a little bit much better?

I can relate to Squatch so much😅. If he would have been a real person, he would have been a great friend of mine😁

Thanks

The level of explainability from this video is top-notch. I always watch your video first to grasp the concept then do the implementation on my own. Thank you so much for this work !

great

Thank you for the awesome video. I have a question. What does the similarity score entails in reality? I assume that the Ws and Bs are being optimized by backpropagation in order to give larger positive values to synonyms, close to 0 values to unrelated words and large negative values to antonyms. Is this a right assumption?

We are comparing the score then shoulnt we divide the denominator then?

Frankly, Josh , if take view of Transformer Self-attention, this video seems meaninglessful because Self-attention can do much more better than what you mentioned. If so, why we need to take this lesson?

Bad content, Focus on content, clear theory and not on sarcasm. It isn't helping

Thanks Professor Josh for such a great tutorial ! It was very informative !

I had a little confusion about the final fully connected layer. It takes in separate attention values for each input word. But doesn't this mean that the dimension of the input depends on how many input words there are (thus it would be difficult to generalize for arbitrarily long sentences)? Did I misunderstand something?

I am always amazed by your tutorials! Thanks. And when we can expect the transformer tutorial to be uploaded?

I have been waiting for this for a long time

Do you have any courses with start-to-finish projects for people who are only just getting interested in machine learning?
Your explanations on the mathematical concepts has been great and I'd be more than happy to pay for a course that implements some of these concepts into real world examples

Thank you for this explanation. But my question is how with backprogation are the weights and bias adjusted in such a model like this. if you could explain that i would deeply appreciate it.

it would not be possible to translate the other older videos you explain very well.❤

You have a talent for explaining these things in a straightforward way. Love your videos. You have no video about Transformers yet, right?

You're amazing Josh, thank you so much for all this content <3.

A video on Siamese Networks would be cool, esp. Siamese BERT-Networks

hey there josh @statquest, your videos are really awsome and super helpful, thus i was wondering when will your video for transformer model come out

Could you do a video about Bert? Architectures like these can be very helpful on NLP and I think a lot of folks will benefit from that :)

Will there be a video about transformers?

Great work, Josh! Listening to my deep learning lectures and reading papers become way easier after watching your videoes, because you explain the big picture and the context so well!! Eagerly waiting for the transformers video!

I'm excited for the video about transformers. Thank you Josh, your videos are extremely helpful

Great videos! So after watching technical videos I think complicating the math has no effect on removing bias from the model. In the future one can find a model with self-encoder-soft-attention-direct-decoder you name it, but it's still garbage in garbage out. Do you think there is a way to plug a fairness/bias filter to the layers so instead of trying to filter the output of the model you just don't produce unfair output? It's like preventing a disease instead of looking for a cure. Obviously I'm not an expert and just trying to get a direction for my personal ethics research out of this naive question. Thanks!

Since you asked for video suggestions in another video: A video about the EM and Mean Shift algorithm would be great!

You made me really excited for transformers 😅

One thing that eludes me, after watching the video once again is, on what basis can we compare hidden states of encoder and decoder. Why are they comparable at all? I understand we can compare word embeddings, but hidden states?

“Statquest is all you need” — I really needed this video for my NLP course but glad it’s out now. I got an A+ for the course, your precious videos helped a lot!

Hey! Great video, this is really helping me with neural networks at the university, do we have a date for when the transformer video comes out?

Thank you very much!

This channel is pure gold. I'm a machine learning and deep learning student.

can't wait for the next StatQuest

Really looking forward to your explanation of Transformers!!!

The best explanation of Attention that I have come across so far ...
Thanks a bunch❤

Y ahora en español? Casi no lo creo, este canal es increible😭 muchas gracias por tus videos !!!

Hello! Can we get a video on Gaussian Processes? many thanks!!!

waiting really hard for the transformer video.

omg! where's the transformers video? my test is tomorrow ahhaha

Oups 🙊 What is « Seq2Seq » I think I will have to check out the quest and then I will be happy to come back to learn with Josh 😅 I am impatient to learn Attention for Neural Networks Clearly Explained

Amazing video Josh! Waiting for the transformer video. Hopefully it'll come out soon. Thanks for everything!

Fantastic video, indeed! Is the attention described in the video the same as in the attention paper? I didn't see the mention of QKV in the video and would like to know whether it was omitted to simplify or by mistake.