Variational Linear Attention documentation

Add your content using reStructuredText syntax. See the reStructuredText documentation for details.

Contents:

• Variational Linear Attention (VLA)
  • The Bottleneck in Sequence Modeling
  • The VLA Formulation
  • Documentation Architecture
• Mathematical Primitives
  • 1. Scaled Inner-Product Scoring
  • 2. Sherman–Morrison Rank-1 Inverse Tracking
  • 3. The Woodbury Matrix Generalization
  • 4. Analytical Optimal Coefficient Recovery (\(\alpha^*\))
  • 5. Recurrent State Modulations
• Penalty Builder Module
  • Overview
  • Parameterizations
  • Recurrence & Inverse Tracking
  • Inputs and Outputs
  • Computation Graph
• Variational Linear Attention Core
  • Key Modules
• Empirical Evaluation & Analysis
  • 1. Symbolic & Diagnostic Probes
  • 2. Theoretical Memory Thresholds
  • 3. The Long Range Arena (LRA)
  • 4. VLA v3: Architecture Scaling & Stability Frontiers
  • Conclusion
• Getting Started & Reproducibility
  • Environment Setup
  • 1. Verifying Core Primitives
  • 2. Running Synthetic Memory Tasks
  • 3. Running LRA Benchmarks
  • 4. Re-generating Paper Plots
  • 5. Re-generating VLA v3 Benchmarks
  • Utilizing VLA in Your Project