Instructions to use QCRI/Fanar-1-9B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use QCRI/Fanar-1-9B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="QCRI/Fanar-1-9B")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("QCRI/Fanar-1-9B") model = AutoModelForCausalLM.from_pretrained("QCRI/Fanar-1-9B", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use QCRI/Fanar-1-9B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "QCRI/Fanar-1-9B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "QCRI/Fanar-1-9B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/QCRI/Fanar-1-9B
- SGLang
How to use QCRI/Fanar-1-9B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "QCRI/Fanar-1-9B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "QCRI/Fanar-1-9B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "QCRI/Fanar-1-9B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "QCRI/Fanar-1-9B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use QCRI/Fanar-1-9B with Docker Model Runner:
docker model run hf.co/QCRI/Fanar-1-9B
Commit History
Update model_max_length in tokenizer config to 4096 fbcfee7 verified
Update README.md 604f6e3 verified
Upload fanar_logo.jpg 103f2d0 verified
Update README.md b8e78b0 verified
Update README.md 4a82767 verified
Corrected few typos. 6c4f3f7 verified
Update config.json 251b65e verified
Update README.md 5a1bc40 verified
Update README.md c819763 verified
Update README.md d08b6bb verified
Update README.md e57340f verified
Update README.md 18f8f6a verified
initial commit without readme e0d24d5
Mohammed Shahmeer (QCRI) commited on