fungamer2/CustomThinking
Viewer
• Updated
• 4.96k • 63
Nehemiah
fungamer2
·
AI & ML interests
None yet
Recent Activity
updated
a dataset 4 days ago
fungamer2/CustomThinking published
a model 4 days ago
fungamer2/SmolLM2-Thinking-Test1.1 updated
a model 4 days ago
fungamer2/SmolLM2-Thinking-Test1.1 Organizations
updated a dataset 4 days ago
published a model 4 days ago
updated a model 4 days ago
published a model 13 days ago
updated a model 13 days ago
updated a Space 16 days ago
updated a model 16 days ago
updated a dataset 17 days ago
updated a model about 1 month ago
updated a dataset about 1 month ago
reacted to marksverdhei's post with 👍👍 about 1 month ago
Post
4576
Poll: Will 2026 be the year of subquadratic attention?
The transformer architecture is cursed by its computational complexity.
It is why you run out of tokens and have to compact. But some would argue that this is a feature not a bug and that this is also why these models are so good. We've been doing a lot of research on trying to make equally good models that are computationally cheaper, But so far, none of the approaches have stood the test of time. Or so it seems.
Please vote, don't be shy. Remember that the Dunning-Kruger effect is very real, so the person who knows less about transformers than you is going to vote. We want everyone's opinion, no matter confidence.
👍 if you think at least one frontier model* will have no O(n^2) attention by the end of 2026
🔥 If you disagree
* Frontier models - models that match / outperform the flagship claude, gemini or chatgpt at the time on multiple popular benchmarks
The transformer architecture is cursed by its computational complexity.
It is why you run out of tokens and have to compact. But some would argue that this is a feature not a bug and that this is also why these models are so good. We've been doing a lot of research on trying to make equally good models that are computationally cheaper, But so far, none of the approaches have stood the test of time. Or so it seems.
Please vote, don't be shy. Remember that the Dunning-Kruger effect is very real, so the person who knows less about transformers than you is going to vote. We want everyone's opinion, no matter confidence.
👍 if you think at least one frontier model* will have no O(n^2) attention by the end of 2026
🔥 If you disagree
* Frontier models - models that match / outperform the flagship claude, gemini or chatgpt at the time on multiple popular benchmarks
- 4 replies
updated a dataset about 1 month ago
Your training has failed ❌
#2 opened about 1 month ago
by
fungamer2
updated a dataset about 1 month ago
published a dataset about 1 month ago
liked a model about 1 month ago
updated a Space about 1 month ago