Quasar: Scaling Context to Millions of Tokens with Linear Attention
How continuous-time attention mechanisms break the quadratic bottleneck and enable 10M+ token context at 1/10th the cost of standard transformers.
SILX AI — Open Foundation Models for Long-Context Intelligence
Novel architectures. Millions of tokens of context. Trained on decentralized compute. All open-weight.
What We Work On
Novel architectures and training methods that move the field forward.
Linear Attention
Standard transformers scale quadratically. Our Quasar architecture uses continuous-time attention that scales linearly — handling millions of tokens at a fraction of the compute.
Decentralized Pretraining
We train on Bittensor's distributed compute network. Miners compete to produce the best model checkpoints, making frontier-class training accessible without centralized GPU clusters.
Open Weights
Every model we release ships with full weights under Apache 2.0. No gated access, no waitlists. Download from Hugging Face and deploy on your own infrastructure.
Quasar
Our first foundation model series. Long-context reasoning with linear attention.
Quasar-2M-Base
LiveFoundation model for long-context understanding and reasoning
Parameters
2.4B
Context
2M tokens
Architecture
Linear Attention
Trained on
2T tokens
Open WeightsApache 2.0Bittensor Subnet 24
Download on Hugging FaceLatest
Decentralized Model Training on Bittensor Subnet 24
Our subnet architecture: how miners compete to train long-context checkpoints, and how validators verify quality across sequence lengths from 8K to 2M tokens.
Quasar-2M-Base: Open Weights on Hugging Face
2.4 billion parameters, 2 million token context window, trained on 2 trillion tokens. Released under Apache 2.0 — download and deploy today.