ai-sage/GigaChat3.5-432B-A28B-Reasoning-bf16
Text Generation • 438B • Updated • 5
🔄 In a Training Loop
Amir Matiseev
Qwmir
AI & ML interests
None yet
Recent Activity
liked a model about 11 hours ago
ai-sage/GigaChat3.5-432B-A28B-Reasoning-bf16 liked a model about 17 hours ago
deepseek-ai/DeepSeek-V4.1-Flash updated a model 7 days ago
MasterX-AI-lab/JPT1Organizations
liked a
model about 11 hours ago
liked a
model about 17 hours ago
updated a
model 7 days ago
liked a
model 10 days ago
liked a
model 16 days ago
liked a
model 22 days ago
published a
model 23 days ago
liked a
model 29 days ago
liked a
model about 1 month ago
liked a
model about 2 months ago
Are you ok?
🧠 1
15
#1 opened 2 months ago
by 
AGIvan
liked 5
models 2 months ago
prostochel097/alphagpt-1
Text Generation • 12.5M • Updated • 135 • 3
prostochel097/alphagpt-2
Updated • 2
ai-sage/GigaChat3.5-432B-A28B-GGUF
Text Generation • 434B • Updated • 462 • 31
ai-sage/GigaChat3.5-432B-A28B
Text Generation • 434B • Updated • 633 • 73
ai-sage/GFusion-10B-A1.8B
Text Generation • 11B • Updated • 246 • 20
liked 2
models 6 months ago
reacted to OzTianlu's post with 🤗 7 months ago
Post
3467
O(1) inference is the foundational design of Spartacus-1B-Instruct 🛡️ !
NoesisLab/Spartacus-1B-Instruct
We have successfully replaced the KV-cache bottleneck inherent in Softmax Attention with Causal Monoid State Compression. By defining the causal history as a monoid recurrence, , the entire prefix is lossily compressed into a fixed-size state matrix per head.
The technical core of this architecture relies on the associativity of the monoid operator:
Training: parallel prefix scan using Triton-accelerated JIT kernels to compute all prefix states simultaneously.
Inference: True sequential updates. Memory and time complexity per token are decoupled from sequence length.
Explicit Causality: We discard RoPE and attention masks. Causality is a first-class citizen, explicitly modeled through learned, content-dependent decay gates.
Current zero-shot benchmarks demonstrate that Spartacus-1B-Instruct (1.3B) is already outperforming established sub-quadratic models like Mamba-1.4B and RWKV-6-1.6B on ARC-Challenge (0.3063). Recent integration of structured Chain-of-Thought (CoT) data has further pushed reasoning accuracy to 75%.
The "Spartacus" era is about scaling intelligence, not the memory wall ♾️.
NoesisLab/Spartacus-1B-Instruct
We have successfully replaced the KV-cache bottleneck inherent in Softmax Attention with Causal Monoid State Compression. By defining the causal history as a monoid recurrence, , the entire prefix is lossily compressed into a fixed-size state matrix per head.
The technical core of this architecture relies on the associativity of the monoid operator:
Training: parallel prefix scan using Triton-accelerated JIT kernels to compute all prefix states simultaneously.
Inference: True sequential updates. Memory and time complexity per token are decoupled from sequence length.
Explicit Causality: We discard RoPE and attention masks. Causality is a first-class citizen, explicitly modeled through learned, content-dependent decay gates.
Current zero-shot benchmarks demonstrate that Spartacus-1B-Instruct (1.3B) is already outperforming established sub-quadratic models like Mamba-1.4B and RWKV-6-1.6B on ARC-Challenge (0.3063). Recent integration of structured Chain-of-Thought (CoT) data has further pushed reasoning accuracy to 75%.
The "Spartacus" era is about scaling intelligence, not the memory wall ♾️.
upvoted a collection 7 months ago