hf-internal-testing/transformers_ci_push
Updated • 372
AI & ML interests
None defined yet.
Recent Activity
View all activity
hf-transformers-bot
updated a
dataset about 3 hours ago
Post
50
🌐 I've just published Sentence Transformers v5.4 to make the project fully multimodal for embeddings and reranking. The release also includes a modular CrossEncoder, and automatic Flash Attention 2 input flattening. Details:
You can now use SentenceTransformer and CrossEncoder with text, images, audio, and video, with the same familiar API. That means you can compute embeddings for an image and a text query using model.encode(), compare them with model.similarity(), and it just works. Models like Qwen3-VL-Embedding-2B and jinaai/jina-reranker-m0 are supported out of the box.
Beyond multimodal, I also fully modularized the CrossEncoder class. It's now a torch.nn.Sequential of composable modules, just like SentenceTransformer has been. This unlocked support for generative rerankers (CausalLM-based models like mxbai-rerank-v2 and the Qwen3 rerankers) via a new LogitScore module, which wasn't possible before without custom code.
Also, Flash Attention 2 now automatically skips padding for text-only inputs. If your batch has a mix of short and long texts, this gives you a nice speedup and lower VRAM usage for free.
I wrote a blog post walking through the multimodal features with practical examples. Check it out if you want to get started, or just point your Agent to the URL: https://huggingface.co/blog/multimodal-sentence-transformers
This release has set up the groundwork for more easily introducing late-interaction models (both text-only and multimodal) into Sentence Transformers in the next major release. I'm looking forward to it!
You can now use SentenceTransformer and CrossEncoder with text, images, audio, and video, with the same familiar API. That means you can compute embeddings for an image and a text query using model.encode(), compare them with model.similarity(), and it just works. Models like Qwen3-VL-Embedding-2B and jinaai/jina-reranker-m0 are supported out of the box.
Beyond multimodal, I also fully modularized the CrossEncoder class. It's now a torch.nn.Sequential of composable modules, just like SentenceTransformer has been. This unlocked support for generative rerankers (CausalLM-based models like mxbai-rerank-v2 and the Qwen3 rerankers) via a new LogitScore module, which wasn't possible before without custom code.
Also, Flash Attention 2 now automatically skips padding for text-only inputs. If your batch has a mix of short and long texts, this gives you a nice speedup and lower VRAM usage for free.
I wrote a blog post walking through the multimodal features with practical examples. Check it out if you want to get started, or just point your Agent to the URL: https://huggingface.co/blog/multimodal-sentence-transformers
This release has set up the groundwork for more easily introducing late-interaction models (both text-only and multimodal) into Sentence Transformers in the next major release. I'm looking forward to it!
hf-transformers-bot
updated a
dataset about 9 hours ago
nielsr
submitted a
paper to Daily Papers about 10 hours ago
ArthurZ
updated a
dataset about 13 hours ago
hf-transformers-bot
updated a
dataset about 14 hours ago
hf-transformers-bot
updated a
dataset about 15 hours ago
ArthurZ
published a
dataset about 21 hours ago
ArthurZ
updated a
dataset about 21 hours ago
ArthurZ
published a
dataset about 22 hours ago
hf-transformers-bot
updated a
dataset 1 day ago
nielsr
submitted a
paper to Daily Papers 6 days ago
hf-transformers-bot
updated a
dataset 6 days ago
hf-transformers-bot
published a
dataset 6 days ago
ArthurZ
updated a
dataset 9 days ago
ArthurZ
published a
dataset 9 days ago
nielsr
submitted a
paper to Daily Papers 17 days ago
nielsr
submitted a
paper to Daily Papers 21 days ago
nielsr
submitted a
paper to Daily Papers 22 days ago