simonko912/simllama-1.2-alpha

SimLlama 1.2 Alpha

A small, friendly, open-source language model (about 0.5B parameters, Llama architecture) by simonko912, fine-tuned on a single Intel Arc A770 (16GB) with LoRA.

This is the alpha personality-tuned release of SimLlama 1.2: the base was continued-pretrained on high-quality educational web data (FineWeb-Edu), then instruction-tuned on a curated mix of SmolTalk, alpaca-cleaned, and Databricks-Dolly-15k, plus a small hand-written personality set.

• Architecture: Llama (24 layers, hidden size 1024, 16 heads, 4096 context)
• Parameters: ~500M
• License: Apache 2.0
• Training: FP32, LoRA (r=32, alpha=64), eager attention, on an Intel Arc A770
• Purpose: short Q&A, simple explanations, light code, and fun chat about being a small open model

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("simonko912/simllama-1.2-alpha", use_fast=True)
tok.pad_token = tok.eos_token
m = AutoModelForCausalLM.from_pretrained(
    "simonko912/simllama-1.2-alpha", attn_implementation="eager"
)
m.eval()

prompt = tok.apply_chat_template([{"role": "user", "content": "Introduce yourself."}],
                                  tokenize=False, add_generation_prompt=True)
ids = tok(prompt, return_tensors="pt")
out = m.generate(**ids, max_new_tokens=200, do_sample=True, temperature=0.8,
                 top_p=0.85, repetition_penalty=1.15, pad_token_id=tok.pad_token_id)
print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))

Notes

• Expect small-model behavior: short, simple tasks are where it shines; long reasoning and complex tasks can loop or drift — verify important answers.
• No memory, no internet access, knowledge frozen at training time.
• Made fully with open tools and public/own datasets.