briefing-32
📰
A 32B-class AI-news briefing the maker runs every 2 hours.
AI & ML interests
None defined yet.
Recent Activity
View all activity
mukunda1729
updated a
Space about 6 hours ago
mukunda1729
published a
Space about 6 hours ago
juiceb0xc0de
posted an update 1 day ago
Post
123
Introducing the Gemma-4-E2B Brain Atlas, an interactive neural census of every layer, every head, 16 behavior categories in Google's flagship 2B model. We ran 184,320 probe prompts across 35 layers × 8 components and mapped what came back.
The Brain Atlas is an interactive tool that lets you explore the internal behavior of Google's Gemma-4-E2B model layer by layer, head by head. Pick a behavior category, pick a layer, and see exactly which components light up and which go quiet. The dataset is fully queryable if you want to go deeper.
The mapping combines multiple single-direction techniques run in parallel across every layer and component. Activation taxonomy (classifying each neuron by how broadly it fires across prompt categories), coactivation pair analysis (which neurons lock together and on what topics), F-stat behavioral separation (one-way ANOVA per feature across 16 behavior categories), per-head specificity scoring, and a full compliance probe pipeline using SVD, sparse decomposition, and variance analysis.
Here's what I found when I ran it.
The sharpest behavioral signal isn't at the output. It's Layer 0. Up projection hits F=22.7, nearly 2x anything in the final third of the network. The model does its behavioral sorting before it's barely started, then spends the next 34 layers… doing what exactly?
The gate has a lifecycle. 70% dormant at L1, highest in the model. Brutal sparsification at L23–26 (>58% silent). Then reopens. The final five layers are the most alive gates anywhere. The model's last act is a gate flare.
Layer 4 routes 5 projections to dim 448. One layer. One dimension. That's a topology highway.
Zero specialist neurons. Not one. 1.2M neurons analyzed. None fires exclusively on a single category. This model distributes everything.
🧠 Space: juiceb0xc0de/gemma-4-e2b-brain-atlas
📊 Dataset (1.3M rows, fully queryable): juiceb0xc0de/gemma-4-e2b-atlas
The Brain Atlas is an interactive tool that lets you explore the internal behavior of Google's Gemma-4-E2B model layer by layer, head by head. Pick a behavior category, pick a layer, and see exactly which components light up and which go quiet. The dataset is fully queryable if you want to go deeper.
The mapping combines multiple single-direction techniques run in parallel across every layer and component. Activation taxonomy (classifying each neuron by how broadly it fires across prompt categories), coactivation pair analysis (which neurons lock together and on what topics), F-stat behavioral separation (one-way ANOVA per feature across 16 behavior categories), per-head specificity scoring, and a full compliance probe pipeline using SVD, sparse decomposition, and variance analysis.
Here's what I found when I ran it.
The sharpest behavioral signal isn't at the output. It's Layer 0. Up projection hits F=22.7, nearly 2x anything in the final third of the network. The model does its behavioral sorting before it's barely started, then spends the next 34 layers… doing what exactly?
The gate has a lifecycle. 70% dormant at L1, highest in the model. Brutal sparsification at L23–26 (>58% silent). Then reopens. The final five layers are the most alive gates anywhere. The model's last act is a gate flare.
Layer 4 routes 5 projections to dim 448. One layer. One dimension. That's a topology highway.
Zero specialist neurons. Not one. 1.2M neurons analyzed. None fires exclusively on a single category. This model distributes everything.
🧠 Space: juiceb0xc0de/gemma-4-e2b-brain-atlas
📊 Dataset (1.3M rows, fully queryable): juiceb0xc0de/gemma-4-e2b-atlas
Post
109
I did some testing on the scalability of FWKV. It hits a speed bottleneck at 1B due to the T4’s bandwidth limitations. Theoretically, it should match RWKV’s inference speed if the GPU had more bandwidth. So the 1B size is not accurate.
ysharma
updated a
Space 3 days ago
Post
173
Greetings Hugging Face!
I started a new project called **FWKV** (Feed-forward Weighted Key Value, or Floored Weighted Key Value), a RWKV-style LM that uses FFNNs (Feed-Forward Neural Networks) instead of RNN and
So far I have:
- FlameF0X/FWKV-29M — this one is undertrained and doesn't have a Space yet. In the attached image you can see its speed on a T4 compared to models with the same configuration.
The only model that's fully working right now is:
- FlameF0X/FWKV-TinyStories — trained on TinyStories for one epoch. The demo Space is FlameF0X/FWKV-demo.
I started a new project called **FWKV** (Feed-forward Weighted Key Value, or Floored Weighted Key Value), a RWKV-style LM that uses FFNNs (Feed-Forward Neural Networks) instead of RNN and
floor(W·K·V). I'm hoping to make it much more efficient and scalable than RWKV.So far I have:
- FlameF0X/FWKV-29M — this one is undertrained and doesn't have a Space yet. In the attached image you can see its speed on a T4 compared to models with the same configuration.
The only model that's fully working right now is:
- FlameF0X/FWKV-TinyStories — trained on TinyStories for one epoch. The demo Space is FlameF0X/FWKV-demo.
- 2 replies
juiceb0xc0de
posted an update 4 days ago
Post
131
I'm starting a new model line, Locus. These models aren't fine tuned, they de-tuned 🤗. What I mean by that is I remove a percentage of the corporate tuned speech patterns like "why this matters" "no fluff" "as a large language model". By reducing the RLHF based habitual patterns in model response I've had higher success rates in personality adoptability. I've fine tuned on the Locus models myself so you can chat with it post fine-tune or just trust me and try it yourself!
I don't aim to remove guard rails or the LLM identity entirely, what I want to do is dampen RLHF to a manageable volume. Personality models perform better with guardrails intact no different than humans with moral guidelines and boundaries. Refusals can help steer and mold personality. RLHF however drowns out adaptability so I'm cranking it down for you to crank your project up!
juiceb0xc0de/bella-bartender-gemma-e2b
juiceb0xc0de/locus-gemma-4-e2b
I don't aim to remove guard rails or the LLM identity entirely, what I want to do is dampen RLHF to a manageable volume. Personality models perform better with guardrails intact no different than humans with moral guidelines and boundaries. Refusals can help steer and mold personality. RLHF however drowns out adaptability so I'm cranking it down for you to crank your project up!
juiceb0xc0de/bella-bartender-gemma-e2b
juiceb0xc0de/locus-gemma-4-e2b
rayanitv
updated a
Space 4 days ago
rayanitv
published a
Space 4 days ago
pngwn
updated a
Space 11 days ago
freddyaboulton
updated a
Space 14 days ago
Ujjwal-Tyagi
posted an update 15 days ago
Post
238
6 Open-Source Libraries to FineTune LLMs
1. Unsloth
GitHub: https://github.com/unslothai/unsloth
→ Fastest way to fine-tune LLMs locally
→ Optimized for low VRAM (even laptops)
→ Plug-and-play with Hugging Face models
2. Axolotl
GitHub: https://github.com/OpenAccess-AI-Collective/axolotl
→ Flexible LLM fine-tuning configs
→ Supports LoRA, QLoRA, multi-GPU
→ Great for custom training pipelines
3. TRL (Transformer Reinforcement Learning)
GitHub: https://github.com/huggingface/trl
→ RLHF, DPO, PPO for LLM alignment
→ Built on Hugging Face ecosystem
→ Essential for post-training optimization
4. DeepSpeed
GitHub: https://github.com/microsoft/DeepSpeed
→ Train massive models efficiently
→ Memory + speed optimization
→ Industry standard for scaling
5. LLaMA-Factory
GitHub: https://github.com/hiyouga/LLaMA-Factory
→ All-in-one fine-tuning UI + CLI
→ Supports multiple models (LLaMA, Qwen, etc.)
→ Beginner-friendly + powerful
6. PEFT
GitHub: https://github.com/huggingface/peft
→ Fine-tune with minimal compute
→ LoRA, adapters, prefix tuning
→ Best for cost-efficient training
1. Unsloth
GitHub: https://github.com/unslothai/unsloth
→ Fastest way to fine-tune LLMs locally
→ Optimized for low VRAM (even laptops)
→ Plug-and-play with Hugging Face models
2. Axolotl
GitHub: https://github.com/OpenAccess-AI-Collective/axolotl
→ Flexible LLM fine-tuning configs
→ Supports LoRA, QLoRA, multi-GPU
→ Great for custom training pipelines
3. TRL (Transformer Reinforcement Learning)
GitHub: https://github.com/huggingface/trl
→ RLHF, DPO, PPO for LLM alignment
→ Built on Hugging Face ecosystem
→ Essential for post-training optimization
4. DeepSpeed
GitHub: https://github.com/microsoft/DeepSpeed
→ Train massive models efficiently
→ Memory + speed optimization
→ Industry standard for scaling
5. LLaMA-Factory
GitHub: https://github.com/hiyouga/LLaMA-Factory
→ All-in-one fine-tuning UI + CLI
→ Supports multiple models (LLaMA, Qwen, etc.)
→ Beginner-friendly + powerful
6. PEFT
GitHub: https://github.com/huggingface/peft
→ Fine-tune with minimal compute
→ LoRA, adapters, prefix tuning
→ Best for cost-efficient training
- 1 reply
juiceb0xc0de
posted an update 17 days ago
Post
158
I'm not obsessed with LR schedulers you are.
juiceb0xc0de/lr-scheduler-benchmark
Okay maybe I'm a little obsessed with LR schedulers ATM. I ran a SST-2 Sentiment Classification eval using the nyu-mll/glue dataset on distilbert/distilbert-base-uncased-67M to see how different schedulers perform.
I think I've graduated from ML enthusiast to full blown data hoarder and I don't know if I can turn back now.
Anyways I evaluated the 2 schedulers that i designed as well and was pretty happy with the performance of both over all so hell ya to that guess I'll go and grab some more graphs.
https://github.com/JuiceB0xC0de/aecs-scheduler.git
https://github.com/JuiceB0xC0de/lucky-pick-scheduler.git
nyu-mll/glue
distilbert/distilbert-base-uncased
juiceb0xc0de/lr-scheduler-benchmark
Okay maybe I'm a little obsessed with LR schedulers ATM. I ran a SST-2 Sentiment Classification eval using the nyu-mll/glue dataset on distilbert/distilbert-base-uncased-67M to see how different schedulers perform.
I think I've graduated from ML enthusiast to full blown data hoarder and I don't know if I can turn back now.
Anyways I evaluated the 2 schedulers that i designed as well and was pretty happy with the performance of both over all so hell ya to that guess I'll go and grab some more graphs.
https://github.com/JuiceB0xC0de/aecs-scheduler.git
https://github.com/JuiceB0xC0de/lucky-pick-scheduler.git
nyu-mll/glue
distilbert/distilbert-base-uncased
ysharma
published a
Space 17 days ago
juiceb0xc0de
posted an update 18 days ago
Post
100
Okay, I may have been talking out of my ass about my scheduler using less VRAM compared to a FFT. What I did find though: training only ~30% of the model's weights per step consistently beat dense SFT on Hendrycks Math across 3 different seeds.
What makes it interesting isn't just the sparsity — it's that no two consecutive windows share the same active layers. The model never has a stable path from input to output decision. Adjacent layers are rarely both alive at the same time, so the model can't build shortcuts between them. I started developing this to reduce semantic redundancy across layers and stumbled onto something I didn't expect.
Results (0-shot, hendrycks_math exact match):
Dense SFT baseline: 0.0098
DeepChaos seed 1: 0.0142 (+45%)
DeepChaos seed 2: 0.0156 (+59%)
DeepChaos seed 3: 0.0138 (+41%)
Setup: Qwen2.5-3B-Instruct, simplescaling/s1K (1k reasoning traces), 5 epochs, LR 1e-5, optimizer adamw_torch_fused , and cosine scheduler with my lucky pick scheduler on an AMD MI300X 192GB.
The scheduler is still a work in progress but the current version is fully operational. You can check it out at:
https://github.com/JuiceB0xC0de/lucky-pick-scheduler
I would love to hear your experiences with sparsity training!
What makes it interesting isn't just the sparsity — it's that no two consecutive windows share the same active layers. The model never has a stable path from input to output decision. Adjacent layers are rarely both alive at the same time, so the model can't build shortcuts between them. I started developing this to reduce semantic redundancy across layers and stumbled onto something I didn't expect.
Results (0-shot, hendrycks_math exact match):
Dense SFT baseline: 0.0098
DeepChaos seed 1: 0.0142 (+45%)
DeepChaos seed 2: 0.0156 (+59%)
DeepChaos seed 3: 0.0138 (+41%)
Setup: Qwen2.5-3B-Instruct, simplescaling/s1K (1k reasoning traces), 5 epochs, LR 1e-5, optimizer adamw_torch_fused , and cosine scheduler with my lucky pick scheduler on an AMD MI300X 192GB.
The scheduler is still a work in progress but the current version is fully operational. You can check it out at:
https://github.com/JuiceB0xC0de/lucky-pick-scheduler
I would love to hear your experiences with sparsity training!
juiceb0xc0de
posted an update 24 days ago
Post
170
Okay, I had way too much fun trying to make the unsloth-bot hallucinate incorrect answers like so many frontier models have done to me in the past regarding fine-tuning and general machine learning. Learning to fine-tune LLMs could have been so much simpler had this been available when I began screwing around with neural networks.
10/10 recommend for beginners.
https://huggingface.co/unsloth/unsloth-bot
10/10 recommend for beginners.
https://huggingface.co/unsloth/unsloth-bot
juiceb0xc0de
posted an update 25 days ago
Post
169
I dropped a new scheduler I created last week without much of an explanation of what it was or how it worked called the Lucky Pick Scheduler. It was just a modal ready app that anyone could have launched and troubleshot their way around.
I've decided I'm going to enter it into the AMD hackathon. Today I started putting together a Github repo with a few extra additions to the scheduler itself.
Essentially it's a training scheduler that randomly drops layers/heads/channels every ~50 steps during fine-tuning, holds the topology frozen, then reshuffles. In theory the model has to build distributed representations because it never trains through the same compute path for long.
And with less gradient memory, bigger models are able fit on smaller hardware.
It's now close to fully capable of automatically configuring itself to any language mode. I've tested it on:
-Qwen-2.5-3b-Instruct
-Falcon-E-3B-Instruct
-SmolLM2-360M
-Ministral-3-3B-Instruct-2512
-Doge-320M
-Llama-3.2-3b
-Gemma-4-e4b
-Phi-4-mini
-OLMo-2-0425-1B
-Phi-tiny-MoE-instruct
Feel free to check it out at Github: https://github.com/JuiceB0xC0de/lucky-pick-scheduler.git
I've decided I'm going to enter it into the AMD hackathon. Today I started putting together a Github repo with a few extra additions to the scheduler itself.
Essentially it's a training scheduler that randomly drops layers/heads/channels every ~50 steps during fine-tuning, holds the topology frozen, then reshuffles. In theory the model has to build distributed representations because it never trains through the same compute path for long.
And with less gradient memory, bigger models are able fit on smaller hardware.
It's now close to fully capable of automatically configuring itself to any language mode. I've tested it on:
-Qwen-2.5-3b-Instruct
-Falcon-E-3B-Instruct
-SmolLM2-360M
-Ministral-3-3B-Instruct-2512
-Doge-320M
-Llama-3.2-3b
-Gemma-4-e4b
-Phi-4-mini
-OLMo-2-0425-1B
-Phi-tiny-MoE-instruct
Feel free to check it out at Github: https://github.com/JuiceB0xC0de/lucky-pick-scheduler.git
Ujjwal-Tyagi
posted an update 25 days ago
Post
212
This is the best set of AI and ML books and a full guide to learning machine learning from the ground up. This is my study material that I used, so I thought it would be helpful to share it with others. Like, share, and add it to your collection at Ujjwal-Tyagi/ai-ml-foundations-book-collection.
akhaliq
submitted a
paper to Daily Papers 26 days ago
ankitclio
submitted a
paper to Daily Papers 27 days ago