Quantized Models (GGUF, IQ, Imatrix)
Collection
Various GGUF quantizations of small models. Models with a "checkmark" are personal favorites. An "orange arrow" means it's being uploaded. • 97 items • Updated • 71
Hello there.
Here I share my personal GGUF-Imatrix quants of Violet_Magcap-12B.
sillytavern, presets, mistral, chatml,
roleplay, conversational, reasoning, smart
"It will help you solve problems. It will also make you question your existence."
"Use wisely—or don't."
Please check out the original model card as well for added context and model information.
Discussions
- General discussion and author feedback.
Feedback is always welcome for potential issues with quants and as a way to guide the author in the future iterations.
Your comments for them are appreciated!
SillyTavern
- [SillyTavern Presets]
Initially recommended master-import presets and an additional quick-replies set.
[Click Here] [Please Read] - Additional setup.
Reasoning Block + Prefix:
ChatML Format:
Mistral Format:
- Downloads last month
- 2,591
Hardware compatibility
Log In to add your hardware
3-bit
4-bit
5-bit
6-bit
8-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for Lewdiculous/Violet_Magcap-12B-GGUF-IQ-Imatrix
Base model
Nitral-AI/Captain-Eris_Violet-GRPO-v0.420 Finetuned
Nitral-AI/Violet_Magcap-12B