High schooler by day, LLM builder by night. Driven by a deep love for both Physics and AI. Currently spending my runtime building on Hugging Face, experimenting with transformer architectures, and training custom LLMs.
We're excited to release BananaMind 2.1 Pico Preview!
It includes the first preview of our BananaMind 2.1 architecture! This model gets near BananaMind 2 Micro performance at half the parameters and 37.5x less tokens! Thats insane!
The current architectures includes about 500K parameters of the total 1.5M parameters in n-gram embeddings and the layer 2 is run twice.
It also includes XSA and the XSA refresh gate.
We're still going to improve the architecture in the final release.
Hello! Tomorrow, OpenCerebral will be releasing Boris-1.7-D60M-n30M โ an experimental architecture. It will be testing a new data mixture, a new tokenizer, and testing Qwen4-like n-gram embeddings.
Following this will be Boris-1.8-D60M-n30M, which will test both the n-gram embeddings AND a new architecture.