Major
Mamba: Linear-Time Sequence Modeling with Selective State Spaces
Researchers introduced Mamba, a new architecture combining efficient hardware-aware designs with selective state spaces. Mamba achieves linear-time complexity O(L) vs Transformer quadratic O(L²), matching or exceeding Transformer performance on language tasks while being 5x faster at inference. It handles extremely long sequences (millions of tokens) efficiently.
- Linear time complexity O(L)
- 5x faster inference than Transformers
- Selective state space mechanism
- Handles millions of tokens
- Challenges Transformer dominance