Hugging Face's logo
ManniX's picture

ManniX PRO

ManniX-ITA
30 6 28
Dante2's profile picture as564335sa's profile picture chmielvu's profile picture
·
https://github.com/mann1x

AI & ML interests

None yet

Recent Activity

updated a model 1 day ago
ManniX-ITA/opencoti-llamafile
posted an update 2 days ago
🚀 New release: Qwen3.6-27B-A3B-Coder A code-specialized MoE carved out of Qwen3.6-35B-A3B by pure expert pruning — no fine-tuning, no distillation. I profiled all 256 experts on balanced corpora plus targeted code benchmarks (LiveCodeBench + MultiPL-E), built a competence map with the code classes up-weighted 1.5×, and dropped the 72 weakest experts per layer (256→184, ~35B→27B). Router, attention, norms, the MTP head and the vision tower are all preserved; active params stay at A3B and routing is baked to top-10 (revert to top-8 anytime). 📊 Benchmarks (Q6_K, temp 0.6): • MultiPL-E 0.840 • HumanEval 0.970 • LiveCodeBench 0.688 • GSM8K 0.970 · ARC-C 0.944 · AIME 0.733 • GPQA-Diamond 0.773 · MATH-500 0.620 · IFEval 0.730 • Average 0.808 27B footprint, A3B speed, coding that punches well above its size — and the preserved MTP head gives you speculative decoding out of the box (text + vision). 🔗 Model: https://huggingface.co/ManniX-ITA/Qwen3.6-27B-A3B-Coder 📦 GGUF (+MTP): https://huggingface.co/ManniX-ITA/Qwen3.6-27B-A3B-Coder-MTP-GGUF 🦙 Ollama: https://ollama.com/mannix/qwen3.6-27b-a3b-coder
repliedto their post 3 days ago
opencoti-llamafile — single-file inference engine for long-context agentic serving This is the bundled runtime of opencoti, a soft fork of opencode adding a multi-tier local inference engine. The main opencoti project is not yet publicly released on GitHub — its engine ships here first as a standalone, zero-dependency single file usable with any OpenAI-compatible client. One executable (Cosmopolitan APE) runs on Linux, Windows, macOS & BSD. Three artifacts: x86_64 with embedded CUDA (sm_75→sm_120, Turing→Blackwell), a bare Windows .exe, and (untested!) aarch64 with embedded sbsa CUDA for DGX Spark (GB10). Side-load DSOs under dso/. Base: llamafile 0.10.3 / llama.cpp + 80 additive patches (full series published in this repo). Runs every llama.cpp-supported GGUF; tuning targets Gemma-4 (26B-A4B MoE, dense 12B/31B, E-series) and Qwen 2.5/3/3.5/3.6 (dense, MoE, recurrent-hybrid). Highlights: - DCA (dual-chunk attention): context extension validated to 768k–1M - MTP speculative decoding: Qwen NextN self-spec (fused N-step) + Gemma-4 assistant drafter (dual-context) — up to ~1.9× decode - KV-quant ladder: TurboQuant 2/3-bit, TCQ trellis-coded 2/3-bit, asymmetric hi-K/cheap-V mixes, q6_0 - Rolling-KV window: streaming host-RAM spill of KV overflow — contexts beyond VRAM, auto T*-aware policy - Mixed position-axis KV: f16 recent window ⊕ quantized tail, auto-tiered at boot - Agentic multi-session serving: PolyKV shared-prefix KV pool, session-keyed KV reuse, lazy slot grow/shrink - Sparse-V + sparse attention (vertical-slash) long-ctx decode wins - Runtime introspection (/props, /slots), layer-repeat (RYS) self-stacking, and more Every feature is gated by KLD / logit-equivalence and RULER retrieval evals — quality numbers live in the docs alongside the binaries. Grab a binary + MANIFEST + USAGE.md and point your agent at it. Feedback very welcome! https://huggingface.co/ManniX-ITA/opencoti-llamafile
View all activity

Organizations

None yet