Hugging Face's logo

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Back to feed
DavidAU 
posted an update 30 days ago
Post
10967
Qwen3.6 27B - NEO-Code Imatrix Max GGUF Quants [exceeds Unsloth in key metrics]:

All quants benchmarked with 5 key metrics.
A DAVIDAU vs UNSLOTH Metrics showdown.
Quant quality exceeds Unsloth in key metrics.
IQ2_M to Q6 available.
Standout: IQ4XS at 94% of BF16 precision.
Full explainer for Quant metrics.

DavidAU/Qwen3.6-27B-NEO-CODE-Di-IMatrix-MAX-GGUF
Trickhat

No advantage and chance for a Q8?

·
DavidAU

I may make a Q6 high and/or a Q8 Hybrid and/or Q8 "HI".
Imatrix does not have any affect on Q8 or BF16 ; unless the other tensors in the model are set at Q6 or lower.

A Q8 "HI" is a special case; where one or more tensors/layers are set at BF16.
malv-c

can you be super nice to give a mcp version ???

·
DavidAU

As of this writing:

There are pipeline (issues as well as optimizations) issues still currently, and it is not widely supported in some AI Apps.

Specifically:
Ggufs:

  • Imatrix is not yet supported for MTP.
  • Not all AI apps have updated to support it -> result -> MTP ggufs do not work at all.
  • Misc issues with speed still being worked on.
In this post