Function-Aware Fill-in-the-Middle as Mid-Training for Coding Agent Foundation Models
Paper • 2607.12463 • Published • 107
ZeroGPU Explorers
community
AI & ML interests
None defined yet.
Recent Activity
View all activity
CongWei1230
authored 2
papers about 17 hours ago
CongWei1230
submitted a
paper to Daily Papers 6 days ago
DongfuJiang
authored 2
papers 15 days ago
yuntian-deng
authored a
paper 15 days ago
yuntian-deng
submitted a
paper to Daily Papers 18 days ago
Post
294
Shipped v0.1.2 of vtx — a minimalist coding agent for the terminal.
Most agentic CLIs ship 10k+ token system prompts. Vtx is ~2,200. Less prompt overhead means more room for your code in the model's context window.
Vtx is a from-scratch Python implementation of the design philosophy behind pi-mono — same principles, pure Python, no transpiled runtime.
What ships out of the box:
→ Textual TUI + headless CLI (vtx -p "fix the failing test")
→ 49 LLM provider gateways, all declared in a single provider.yaml
→ 5 core tools (read / edit / write / bash / find) plus web search and fetch
→ Session tree with compaction, handoff, and resume
→ AGENTS.md / CLAUDE.md auto-discovery
→ Skills system — drop SKILL.md files in .agents/skills/ and they become slash commands
→ Two OAuth flows (GitHub Copilot device flow, OpenAI Codex PKCE)
→ Two-mode permissions: prompt (default) or auto, with a safe-command allowlist
This release adds a proper extension system. Register new LLM-callable tools, intercept tool calls, hook lifecycle events, and add slash commands from a single register(api) function in a Python file under ~/.vtx/agent/extensions/. Extensions can override built-in tools by name and chain handler logic across subscribers.
Apache 2.0. uv tool install vtx-coding-agent and you're running.
GitHub: https://github.com/OEvortex/vtx-coding-agent
PyPI: https://pypi.org/project/vtx-coding-agent
Built in the open. Feedback, extensions, and PRs welcome.
Most agentic CLIs ship 10k+ token system prompts. Vtx is ~2,200. Less prompt overhead means more room for your code in the model's context window.
Vtx is a from-scratch Python implementation of the design philosophy behind pi-mono — same principles, pure Python, no transpiled runtime.
What ships out of the box:
→ Textual TUI + headless CLI (vtx -p "fix the failing test")
→ 49 LLM provider gateways, all declared in a single provider.yaml
→ 5 core tools (read / edit / write / bash / find) plus web search and fetch
→ Session tree with compaction, handoff, and resume
→ AGENTS.md / CLAUDE.md auto-discovery
→ Skills system — drop SKILL.md files in .agents/skills/ and they become slash commands
→ Two OAuth flows (GitHub Copilot device flow, OpenAI Codex PKCE)
→ Two-mode permissions: prompt (default) or auto, with a safe-command allowlist
This release adds a proper extension system. Register new LLM-callable tools, intercept tool calls, hook lifecycle events, and add slash commands from a single register(api) function in a Python file under ~/.vtx/agent/extensions/. Extensions can override built-in tools by name and chain handler logic across subscribers.
Apache 2.0. uv tool install vtx-coding-agent and you're running.
GitHub: https://github.com/OEvortex/vtx-coding-agent
PyPI: https://pypi.org/project/vtx-coding-agent
Built in the open. Feedback, extensions, and PRs welcome.
yuntian-deng
authored 3
papers about 1 month ago
Long Grounded Thoughts: Distilling Compositional Visual Reasoning Chains at Scale
Paper • 2511.05705 • Published • 10
DetectZoo: A Unified Toolkit for AI-Generated Content Detection Across Text, Audio, and Image Modalities
Paper • 2606.04205 • Published
Code2LoRA: Hypernetwork-Generated Adapters for Code Language Models under Software Evolution
Paper • 2606.06492 • Published • 96
DongfuJiang
authored 3
papers about 2 months ago
Post
1102
New blog post!
An introduction to a little-known but highly effective model reduction method: 𝗧𝗿𝗶𝗺𝗺𝗶𝗻𝗴✂️
We show how to reduce model size (we went up to 87.24% reduction) while preserving its performance.
We applied this technique to 16 different model families across several modalities to illustrate that it works on any architecture (as long as the embedding layer is the last one of the model) and on any modality involving text.
From these 16 families, we generated over 𝟱,𝟱𝟬𝟬 𝗺𝗼𝗻𝗼𝗹𝗶𝗻𝗴𝘂𝗮𝗹 𝗺𝗼𝗱𝗲𝗹𝘀 𝗶𝗻 𝟭𝟮𝟰 𝗱𝗶𝗳𝗳𝗲𝗿𝗲𝗻𝘁 𝗹𝗮𝗻𝗴𝘂𝗮𝗴𝗲𝘀 🌍
Key takeaways from our experiments:
1️⃣ Trimming does not require a GPU. Our models were obtained on a CPU.
2️⃣ This method scales up to at least 4B parameters (we did not test beyond that).
3️⃣ Trimmed model is smaller than the original while preserving its performance. If you observe a slight performance drop, just fine-tuned to recover or even surpass the original performance.
4️⃣ For an equivalent compute budget, it is better to trim then fine-tune rather than fine-tuning the original model. Since the model is smaller, you can run more epochs/show more data and get in fine a better model than the original.
5️⃣ Trimming is a competitive alternative to distillation and quantization. E.g. we obtained our alternative to DistilBERT in 9 minutes on CPU vs. 90 hours of GPU for the latter.
6️⃣ Trimming could generate reasoning traces in the language of the trimmed model. This could be an alternative to generating traces in English and then translating them into the desired language.
And many other things (such as how much data are needed, the impact of the database used, the order in which it should be done, etc.) are available in the blogpost!
Blogpost: https://huggingface.co/blog/lbourdois/introduction-to-trimming
Models: alphaedge-ai/Trimming_models_search
An introduction to a little-known but highly effective model reduction method: 𝗧𝗿𝗶𝗺𝗺𝗶𝗻𝗴✂️
We show how to reduce model size (we went up to 87.24% reduction) while preserving its performance.
We applied this technique to 16 different model families across several modalities to illustrate that it works on any architecture (as long as the embedding layer is the last one of the model) and on any modality involving text.
From these 16 families, we generated over 𝟱,𝟱𝟬𝟬 𝗺𝗼𝗻𝗼𝗹𝗶𝗻𝗴𝘂𝗮𝗹 𝗺𝗼𝗱𝗲𝗹𝘀 𝗶𝗻 𝟭𝟮𝟰 𝗱𝗶𝗳𝗳𝗲𝗿𝗲𝗻𝘁 𝗹𝗮𝗻𝗴𝘂𝗮𝗴𝗲𝘀 🌍
Key takeaways from our experiments:
1️⃣ Trimming does not require a GPU. Our models were obtained on a CPU.
2️⃣ This method scales up to at least 4B parameters (we did not test beyond that).
3️⃣ Trimmed model is smaller than the original while preserving its performance. If you observe a slight performance drop, just fine-tuned to recover or even surpass the original performance.
4️⃣ For an equivalent compute budget, it is better to trim then fine-tune rather than fine-tuning the original model. Since the model is smaller, you can run more epochs/show more data and get in fine a better model than the original.
5️⃣ Trimming is a competitive alternative to distillation and quantization. E.g. we obtained our alternative to DistilBERT in 9 minutes on CPU vs. 90 hours of GPU for the latter.
6️⃣ Trimming could generate reasoning traces in the language of the trimmed model. This could be an alternative to generating traces in English and then translating them into the desired language.
And many other things (such as how much data are needed, the impact of the database used, the order in which it should be done, etc.) are available in the blogpost!
Blogpost: https://huggingface.co/blog/lbourdois/introduction-to-trimming
Models: alphaedge-ai/Trimming_models_search
- 4 replies
Locutusque
posted an update about 2 months ago
Post
380
🚀 Introducing Esmeralda-Llama-3.1-8B-control
The first release in the Esmeralda model family by Locutusque.
This model is intentionally small and experimental — a control/baseline proof-of-concept designed to answer one question:
«“How strong is my new "Locutusque/esmeralda-agentic" dataset before scaling to larger runs?”»
Training Details
- Base: Llama 3.1 8B
- Training precision: bf16 mixed precision
- Chat template: modified ChatML
- Dataset size: ~37k examples
- Examples actually used for this run: ~5k
The dataset includes:
- multi-turn agentic traces
- reasoning traces
- structured assistant behavior
- generalist instruction data
Benchmark Results
Compared against:
- Llama 3.1 8B Instruct
- Hermes-3-Llama-3.1-8B
HumanEval
57.3 — Esmeralda
56.1 — Llama 3.1 Instruct
52.4 — Hermes-3
MBPP
53.2 — Esmeralda
56.8 — Llama 3.1 Instruct
48.2 — Hermes-3
GPQA Diamond
15.7 — Esmeralda
15.7 — Llama 3.1 Instruct
18.2 — Hermes-3
EQ-Bench
59.2 — Esmeralda
61.1 — Llama 3.1 Instruct
63.1 — Hermes-3
EQ-Bench Parseable (Syntax Stability)
🔥 100.0% — Esmeralda
92.4% — Llama 3.1 Instruct
91.2% — Hermes-3
Here Be Dragons 🐉
I also experimented with a new TruthfulQA free-generation evaluation setup.
- Responses were judged by Gemma 4 26B A4B
- The judge compared generations directly against ground-truth answers
- Models were evaluated in 8-bit quantized form to speed up inference
TruthfulQA (LLM Judge)
0.682 — Esmeralda-Llama-3.1-8B-control
0.587 — Hermes-3-Llama-3.1-8B (reported MC2 score; methodology differs)
For a lightweight control run trained on only a fraction of the dataset, I’m pretty encouraged by the results.
The model is released under the standard Llama 3.1 license, and I’d genuinely love feedback from people testing it in real workflows.
Model: Locutusque/Esmeralda-Llama-3.1-8B-control
Dataset: Locutusque/esmeralda-agentic
The first release in the Esmeralda model family by Locutusque.
This model is intentionally small and experimental — a control/baseline proof-of-concept designed to answer one question:
«“How strong is my new "Locutusque/esmeralda-agentic" dataset before scaling to larger runs?”»
Training Details
- Base: Llama 3.1 8B
- Training precision: bf16 mixed precision
- Chat template: modified ChatML
- Dataset size: ~37k examples
- Examples actually used for this run: ~5k
The dataset includes:
- multi-turn agentic traces
- reasoning traces
- structured assistant behavior
- generalist instruction data
Benchmark Results
Compared against:
- Llama 3.1 8B Instruct
- Hermes-3-Llama-3.1-8B
HumanEval
57.3 — Esmeralda
56.1 — Llama 3.1 Instruct
52.4 — Hermes-3
MBPP
53.2 — Esmeralda
56.8 — Llama 3.1 Instruct
48.2 — Hermes-3
GPQA Diamond
15.7 — Esmeralda
15.7 — Llama 3.1 Instruct
18.2 — Hermes-3
EQ-Bench
59.2 — Esmeralda
61.1 — Llama 3.1 Instruct
63.1 — Hermes-3
EQ-Bench Parseable (Syntax Stability)
🔥 100.0% — Esmeralda
92.4% — Llama 3.1 Instruct
91.2% — Hermes-3
Here Be Dragons 🐉
I also experimented with a new TruthfulQA free-generation evaluation setup.
- Responses were judged by Gemma 4 26B A4B
- The judge compared generations directly against ground-truth answers
- Models were evaluated in 8-bit quantized form to speed up inference
TruthfulQA (LLM Judge)
0.682 — Esmeralda-Llama-3.1-8B-control
0.587 — Hermes-3-Llama-3.1-8B (reported MC2 score; methodology differs)
For a lightweight control run trained on only a fraction of the dataset, I’m pretty encouraged by the results.
The model is released under the standard Llama 3.1 license, and I’d genuinely love feedback from people testing it in real workflows.
Model: Locutusque/Esmeralda-Llama-3.1-8B-control
Dataset: Locutusque/esmeralda-agentic
Post
3069
🙋🏻♂️ Hey there folks ,
Turns out : if we predict 🌏 earth we can save a lot of time looking for interesting things and less time looking at things that we expect to see.
Sentinel-2 imagery 🛰️basically takes a long time to download towards earth. so our "near real time" systems are quite far from that in practical terms.
meanwhile , if we "predict" what we will see , based on what we do see , we can send down much less data in a timely way , and prioritize 📡earth-bound response .
I'm talking about illegal fishing , logging , mining or building in nature reserves , the more of that we predict early the more we're able to stop it on time.
At least that's the concept !
check out the blog : https://huggingface.co/blog/Tonic/save-patagonia-by-predicting-earth
- Collection: https://huggingface.co/collections/NuTonic/earth-observation-with-temporal-and-general-understanding
- Code: https://github.com/Josephrp/Nutonic
- Dataset: NuTonic/sat-vl-sft-training-ready-v1
- Model: NuTonic/lspace
- Training: NuTonic/lspace-trackio
- Evals: NuTonic/Patagonia_Eval
Turns out : if we predict 🌏 earth we can save a lot of time looking for interesting things and less time looking at things that we expect to see.
Sentinel-2 imagery 🛰️basically takes a long time to download towards earth. so our "near real time" systems are quite far from that in practical terms.
meanwhile , if we "predict" what we will see , based on what we do see , we can send down much less data in a timely way , and prioritize 📡earth-bound response .
I'm talking about illegal fishing , logging , mining or building in nature reserves , the more of that we predict early the more we're able to stop it on time.
At least that's the concept !
check out the blog : https://huggingface.co/blog/Tonic/save-patagonia-by-predicting-earth
- Collection: https://huggingface.co/collections/NuTonic/earth-observation-with-temporal-and-general-understanding
- Code: https://github.com/Josephrp/Nutonic
- Dataset: NuTonic/sat-vl-sft-training-ready-v1
- Model: NuTonic/lspace
- Training: NuTonic/lspace-trackio
- Evals: NuTonic/Patagonia_Eval
- 2 replies
DongfuJiang
authored 3
papers 2 months ago
Watch Before You Answer: Learning from Visually Grounded Post-Training
Paper • 2604.05117 • Published • 36
ClawBench: Can AI Agents Complete Everyday Online Tasks?
Paper • 2604.08523 • Published • 265
Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
Paper • 2604.12374 • Published • 39