bf16 mmproj
[65319] srv process_chun: processing image...
[65319] encoding image slice...
[65319] D:\a\llama.cpp\llama.cpp\ggml\src\ggml-cuda\im2col.cu:84: GGML_ASSERT(dst->type == GGML_TYPE_F16 || dst->type == GGML_TYPE_F32) failed
srv operator(): http client error: Failed to read connection
←[0msrv log_server_r: request: POST /v1/chat/completions 127.0.0.1 500
srv operator(): instance name=Mistral-Small-3.2-24B-Instruct exited with status -1073740791
Seems like the bf16 mmproj has the same bug as glm 4.6v / 4.6v flash. Could you have a look please?
related:
https://github.com/ggml-org/llama.cpp/pull/15367
https://github.com/ggml-org/llama.cpp/issues/18122
https://github.com/ggml-org/llama.cpp/issues/15536
The bf16 mmproj is from June. It seems like lcpp has fixed this in August. A reconvert would fix it. Much appreciated.
oh lordy, that's annoying :') I'll add it to my todo list today (when my power comes back on)
Can confirm it's fixed! Thanks!