PickVidsDo we need Attention? - Linear RNNs and State Space Models (SSMs) for NLP
Why it was recommended
I highly recommend Sasha Rush's video on it
Strong recommendationa_wild_dandan · Original
What it covers
Sasha Rush surveys linear RNNs and state space models as replacements for attention in transformer-style models, in a talk for MLSys. It covers the foundational. It is for someone who wants the derivation, proof, or implementation detail.
Video details
- Channel
- Sasha Rush
- Length
- 39 min 43 sec
- Year
- 2023
More in this topic
- The spelled-out intro to neural networks and backpropagation: building micrograd
- But what is a neural network? | Deep learning chapter 1
- AlphaGo - The Movie | Full award-winning documentary
- Transformers, the tech behind LLMs | Deep Learning Chapter 5
- USENIX Security '18-Q: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?
- Lecture 1 | Machine Learning (Stanford)