Explore virtual emotions.
♾️ Thinking
Wop
wop
AI & ML interests
AI research
AGI
Recent Activity
repliedto their post 2 minutes ago
# One Script, Every Benchmark
Every Bench Labs benchmark had its own eval files. Now there's one script, hosted in the leaderboard Space:
```bash
curl -sLO https://huggingface.co/spaces/bench-labs/BenchLabs-Leaderboard/resolve/main/script.py
pip install torch transformers
python script.py --model your/model
```
It runs the full suite with the official scoring — Effortless (exact-match), Easy (hybrid category-aware), Mid (loglikelihood: `acc`, `acc_norm`, `soft_score_norm`) — and reports every category and subcategory, not just one number.
Output includes `leaderboard.json`: a ready-to-paste `models.json` entry. Run the script, paste it, open a PR on the [leaderboard](https://huggingface.co/spaces/bench-labs/BenchLabs-Leaderboard). Done. https://huggingface.co/bench-labs reacted to theirpost with 🔥 15 minutes ago
# PixelModel v1
Last month we released PixelModel — a neural network whose weights are literally the pixels of a PNG. It was a toy: 202,752 parameters, welded to 32×32 output, trained on six solid-color swatches. It scored FID 566.84 on the Tiny-T2I-Leaderboard, mostly by producing the same yellow noise for every prompt.
Today we're releasing PixelModel v1. It is 8.5× smaller — 23,747 parameters — and it beats v0 on both benchmark metrics while being trained on 20,000 real MS-COCO caption/image pairs instead of six color swatches. The entire model now fits in a 160×149 PNG.
That image is not a visualization of the model. It is the model. All 23,747 weights, one per pixel.
## links
https://huggingface.co/bench-labs
Blog post [read it here ⇗](https://huggingface.co/spaces/bench-labs/blog?post=pixelmodel-v1.html)
See us on the [Leaderboard ⇗](https://huggingface.co/spaces/FlameF0X/Tiny-T2I-Leaderboard)
Model card [here](https://huggingface.co/bench-labs/pixelmodel-v1)
## The catch
A 23K-parameter model does not draw sandwiches. With ~1 parameter per training image, the loss-minimizing behavior is to output the average of all plausible images for a caption — caption-conditioned color, light, and layout statistics. Food prompts come out warm and brown; sky prompts come out cool and bright. That is the ceiling for this size class, and we'd rather show it than crop around it.
# cherry on top 🍒
The model generates 600 images (cpu) in 5 (five) seconds.
Thats 5000 images in 24 seconds on cpu.
The model trained on cpu for just 30 minutes. repliedto their post about 11 hours ago
# One Script, Every Benchmark
Every Bench Labs benchmark had its own eval files. Now there's one script, hosted in the leaderboard Space:
```bash
curl -sLO https://huggingface.co/spaces/bench-labs/BenchLabs-Leaderboard/resolve/main/script.py
pip install torch transformers
python script.py --model your/model
```
It runs the full suite with the official scoring — Effortless (exact-match), Easy (hybrid category-aware), Mid (loglikelihood: `acc`, `acc_norm`, `soft_score_norm`) — and reports every category and subcategory, not just one number.
Output includes `leaderboard.json`: a ready-to-paste `models.json` entry. Run the script, paste it, open a PR on the [leaderboard](https://huggingface.co/spaces/bench-labs/BenchLabs-Leaderboard). Done. https://huggingface.co/bench-labsOrganizations
