Dynamic Sparse Attention (DSA)

Visualizing the lightweight Indexer Network

Click "Next Step" to begin
Query: $q_{index}$
$ \times $
Key: $k_{index}$
$ \rightarrow $
Relevance Score
Current Query
Indexing (Evaluating)
Selected (Top-K)
Ignored