UMSR-Reasoner-7B (INT8)

8-bit bitsandbytes runtime quantization profile.

Model Tree

Source model: https://huggingface.co/NorthernTribe-Research/UMSR-Reasoner-7B
Variant repo: https://huggingface.co/NorthernTribe-Research/UMSR-Reasoner-7B-INT8
Dataset: https://huggingface.co/datasets/NorthernTribe-Research/UMSR-v1

Quantization Profile

Variant: int8
Scheme: bitsandbytes int8
Materialized weights in this repo: False

Usage

This variant repository is a profile card. Load weights from the source model id.

from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig

model_id = "NorthernTribe-Research/UMSR-Reasoner-7B"
bnb_config = BitsAndBytesConfig(load_in_8bit=True)

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    quantization_config=bnb_config,
)

Output Contract

<final_answer>...</final_answer>

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for NorthernTribe-Research/UMSR-Reasoner-7B-INT8

Base model

NorthernTribe-Research/UMSR-Reasoner-7B

Finetuned

(2)

this model