SmolVLM2-2.2B-Instruct-GGUF (Q4_K_M)

Q4_K_M quantized GGUF conversion of SmolVLM2-2.2B-Instruct.

Files

File	Size	Description
SmolVLM2-2.2B-Instruct-Q4_K_M.gguf	1.0 GB	Main model (Q4_K_M quantized)
mmproj-SmolVLM2-2.2B-Instruct-f16.gguf	832 MB	Vision projector (F16)

Usage

With Ollama

FROM ./SmolVLM2-2.2B-Instruct-Q4_K_M.gguf
PROJECTOR ./mmproj-SmolVLM2-2.2B-Instruct-f16.gguf
PARAMETER num_ctx 4096
PARAMETER temperature 0.1

With llama.cpp

./llama-mtmd-cli -m SmolVLM2-2.2B-Instruct-Q4_K_M.gguf --mmproj mmproj-SmolVLM2-2.2B-Instruct-f16.gguf --image screenshot.png -p "Click the Submit button"

License

Apache 2.0