| --- |
| license: apache-2.0 |
| pipeline_tag: text-generation |
| extra_gated_prompt: "You agree to not use this model (or future versions) to conduct experiments that cause harm to any person or group." |
| extra_gated_fields: |
| Company: text |
| Country: country |
| Specific date: date_picker |
| I want to use this model for: |
| type: select |
| options: |
| - Work |
| - Research |
| - Education |
| - Hobby |
| - label: Other |
| value: other |
| I agree to use this model in good faith ONLY: checkbox |
| --- |
| <style> |
| *, html, body, div { |
| color: gray; |
| background: black !important; |
| border: none; |
| } |
| |
| img { |
| filter: contrast(1.3); |
| user-select: none; |
| transition: all 0.2s ease; |
| border-radius: .5rem; |
| display: block !important; |
| margin: 1rem auto !important; |
| } |
| |
| img:hover { |
| transform: rotate(2deg); |
| filter: invert(100%); |
| } |
| |
| @import url('https://fonts.googleapis.com/css2?family=Vollkorn:ital,wght@0,400..900;1,400..900&display=swap'); |
| </style> |
| <body> |
| <div style="background-color: transparent; border-radius: .5rem; padding: 2rem; font-family: monospace; font-size: .85rem; text-align: justify;"> |
| |
| <p align="center"> |
| <img src="https://huggingface.co/appvoid/arco-3/resolve/main/super-cubby.png" alt="cubby"> |
| </p> |
| |
| In this repository, we propose the next iteration of `arco`, a new meta-learner small language model. Now with `qwen` as the base architecture for improvements. |
| |
| During previous research, we first noticed a dramatic underpeformance on fewshot prompting from previous `arco` series (regardless of benchmark improvements on arc) so we decided that the main concept to work on was making a more robust fewshot learning by focusing directly on tasks that improve that skill with a stronger baseline model like `qwen` family. |
| |
| After several merging iterations with some openly available models, we finally achieved a strong baseline for a meta-learner model which we called [`arco-3`](https://huggingface.co/appvoid/arco-3-gguf). This model will serve as the starting point for future fewshot finetunings and experiments. |
| |
| #### prompt |
| There is no prompt intentionally set. |
| |
| #### benchmarks |
| ##### **meta arena** |
| We tested around 65 models against each other with fewshot tasks and used `gemini-2.5-pro` to chose the best answers from each one. Currently, it ranks 13th in [meta-arena](https://huggingface.co/spaces/appvoid/meta-arena). |
| |
| <p align="center"> |
| <img src="https://huggingface.co/appvoid/arco-3/resolve/main/meta-arena.png" alt="meta arena"> |
| </p> |
| |
| ##### **variance** |
| We also tested the model against some popular small models on "power" distribution for our 5 typically chosen language modeling benchmarks. |
| <img src="https://huggingface.co/appvoid/arco-3/resolve/main/variance.png" alt="variance"> |
| |
| ##### **language modeling** |
| To our surprise, this model also improved some language modeling tasks over the base model on several well-known benchmarks. |
| |
| | Parameters | Model | MMLU | ARC-C | HellaSwag | PIQA | Winogrande | Average | |
| | -----------|--------------------------------|-------|-------|-----------|--------|------------|---------| |
| | 0.6b | qwen 3 |40.31| 34.47 | 47.38 | 67.46 | 56.04 | 49.13 | |
| | 0.6b | arco 3 | **43.34** | **36.01** | **49.56** | **68.17** | **58.09** | **51.03** | |
| |
| #### strengths |
| - Strong bias to format |
| - Excellent classifier |
| - State-of-the-art paraphrasing |
| - Vocabulary/Idiomatic understanding |
| |
| #### limitations |
| - Lack of creative outputs |
| - Extremely poor summarization skills |
| - Poor causality understanding |
| - Hallucinations |
| |
| We have a plan to tackle each one of these issues for them to be corrected in the future. |
| |
| |
| #### supporters |
| <a href="https://ko-fi.com/appvoid" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" style="height: 34px !important; margin-top: -4px;width: 128px !important; filter: contrast(2) grayscale(100%) brightness(100%);" ></a> |
| |
| |
| #### trivia |
| `arco` means "bow" in spanish, which is just another way to say that hits its target fast and accurately. |
| |
| |
| **Note**: the model has not been tested as a chat assistant and it might not work as intended, use with caution. |
| </div> |
| </body> |
