ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools
Paper โข 2406.12793 โข Published โข 35
ChatGLM3-6B-Base
๐ป Github Repo โข ๐ฆ Twitter โข ๐ [GLM@ACL 22] [GitHub] โข ๐ [GLM-130B@ICLR 23] [GitHub]
๐ Join our Slack and WeChat
๐Experience the larger-scale ChatGLM model at chatglm.cn
ไป็ป (Introduction)
ChatGLM3-6B ๆฏ ChatGLM ็ณปๅๆๆฐไธไปฃ็ๅผๆบๆจกๅ๏ผๅจไฟ็ไบๅไธคไปฃๆจกๅๅฏน่ฏๆต็ ใ้จ็ฝฒ้จๆงไฝ็ญไผๅคไผ็ง็นๆง็ๅบ็กไธ๏ผChatGLM3-6B ๅผๅ ฅไบๅฆไธ็นๆง๏ผ
- ๆดๅผบๅคง็ๅบ็กๆจกๅ๏ผ ChatGLM3-6B ็ๅบ็กๆจกๅ ChatGLM3-6B-Base ้็จไบๆดๅคๆ ท็่ฎญ็ปๆฐๆฎใๆดๅ ๅ็่ฎญ็ปๆญฅๆฐๅๆดๅ็็่ฎญ็ป็ญ็ฅใๅจ่ฏญไนใๆฐๅญฆใๆจ็ใไปฃ็ ใ็ฅ่ฏ็ญไธๅ่งๅบฆ็ๆฐๆฎ้ไธๆต่ฏๆพ็คบ๏ผChatGLM3-6B-Base ๅ ทๆๅจ 10B ไปฅไธ็้ข่ฎญ็ปๆจกๅไธญๆๅผบ็ๆง่ฝใ
- ๆดๅฎๆด็ๅ่ฝๆฏๆ๏ผ ChatGLM3-6B ้็จไบๅ จๆฐ่ฎพ่ฎก็ Prompt ๆ ผๅผ๏ผ้คๆญฃๅธธ็ๅค่ฝฎๅฏน่ฏๅคใๅๆถๅ็ๆฏๆๅทฅๅ ท่ฐ็จ๏ผFunction Call๏ผใไปฃ็ ๆง่ก๏ผCode Interpreter๏ผๅ Agent ไปปๅก็ญๅคๆๅบๆฏใ
- ๆดๅ จ้ข็ๅผๆบๅบๅ๏ผ ้คไบๅฏน่ฏๆจกๅ ChatGLM3-6B ๅค๏ผ่ฟๅผๆบไบๅบ็กๆจกๅ ChatGLM-6B-Baseใ้ฟๆๆฌๅฏน่ฏๆจกๅ ChatGLM3-6B-32Kใไปฅไธๆๆๆ้ๅฏนๅญฆๆฏ็ ็ฉถๅฎๅ จๅผๆพ๏ผๅจๅกซๅ้ฎๅท่ฟ่ก็ป่ฎฐๅไบฆๅ ่ฎธๅ ่ดนๅไธไฝฟ็จใ
ๆฌไปๅบไธบ ChatGLM3-6B ็ๅบ็กๆจกๅ ChatGLM3-6B-Baseใ
ChatGLM3-6B is the latest open-source model in the ChatGLM series. While retaining many excellent features such as smooth dialogue and low deployment threshold from the previous two generations, ChatGLM3-6B introduces the following features:
- More Powerful Base Model: The base model of ChatGLM3-6B, ChatGLM3-6B-Base, employs a more diverse training dataset, more sufficient training steps, and a more reasonable training strategy. Evaluations on datasets such as semantics, mathematics, reasoning, code, knowledge, etc., show that ChatGLM3-6B-Base has the strongest performance among pre-trained models under 10B.
- More Comprehensive Function Support: ChatGLM3-6B adopts a newly designed Prompt format, in addition to the normal multi-turn dialogue. It also natively supports function call, code interpreter, and complex scenarios such as agent tasks.
- More Comprehensive Open-source Series: In addition to the dialogue model ChatGLM3-6B, the base model ChatGLM-6B-Base and the long-text dialogue model ChatGLM3-6B-32K are also open-sourced. All the weights are fully open for academic research, and after completing the questionnaire registration, they are also allowed for free commercial use.
This repo is ChatGLM3-6B-Base, the base model of ChatGLM3-6B.
่ฝฏไปถไพ่ต (Dependencies)
pip install protobuf transformers==4.30.2 cpm_kernels torch>=2.0 gradio mdtex2html sentencepiece accelerate
ไปฃ็ ่ฐ็จ (Code Usage)
ไฝไธบๆฒกๆ็ป่ฟไบบ็ฑปๆๅพๅฏน้ฝ็ๆจกๅ๏ผChatGLM3-6B-Base ไธ่ฝ็จไบๅค่ฝฎๅฏน่ฏใไฝๆฏๅฏไปฅ่ฟ่กๆๆฌ็ปญๅใ
As a model that has not been aligned with human intent, ChatGLM3-6B-Base cannot be used for multi-turn conversations. However, text completion is possible.
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm3-6b-base", trust_remote_code=True)
model = AutoModel.from_pretrained("THUDM/chatglm3-6b-base", trust_remote_code=True).half().cuda()
inputs = tokenizer(["ไปๅคฉๅคฉๆฐ็ไธ้"], return_tensors="pt").to('cuda')
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0].tolist()))
ๅ ณไบๆดๅค็ไฝฟ็จ่ฏดๆ๏ผๅ ๆฌๅฆไฝ่ฟ่กๅฝไปค่กๅ็ฝ้กต็ๆฌ็ DEMO๏ผไปฅๅไฝฟ็จๆจกๅ้ๅไปฅ่็ๆพๅญ๏ผ่ฏทๅ่ๆไปฌ็ Github Repoใ
For more instructions, including how to run CLI and web demos, and model quantization, please refer to our Github Repo.
ๅ่ฎฎ (License)
ๆฌไปๅบ็ไปฃ็ ไพ็ ง Apache-2.0 ๅ่ฎฎๅผๆบ๏ผChatGLM3-6B ๆจกๅ็ๆ้็ไฝฟ็จๅ้่ฆ้ตๅพช Model Licenseใ
The code in this repository is open-sourced under the Apache-2.0 license, while the use of the ChatGLM3-6B model weights needs to comply with the Model License.
ๅผ็จ (Citation)
ๅฆๆไฝ ่งๅพๆไปฌ็ๅทฅไฝๆๅธฎๅฉ็่ฏ๏ผ่ฏท่่ๅผ็จไธๅ่ฎบๆใ
If you find our work helpful, please consider citing the following papers.
@misc{glm2024chatglm,
title={ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools},
author={Team GLM and Aohan Zeng and Bin Xu and Bowen Wang and Chenhui Zhang and Da Yin and Diego Rojas and Guanyu Feng and Hanlin Zhao and Hanyu Lai and Hao Yu and Hongning Wang and Jiadai Sun and Jiajie Zhang and Jiale Cheng and Jiayi Gui and Jie Tang and Jing Zhang and Juanzi Li and Lei Zhao and Lindong Wu and Lucen Zhong and Mingdao Liu and Minlie Huang and Peng Zhang and Qinkai Zheng and Rui Lu and Shuaiqi Duan and Shudan Zhang and Shulin Cao and Shuxun Yang and Weng Lam Tam and Wenyi Zhao and Xiao Liu and Xiao Xia and Xiaohan Zhang and Xiaotao Gu and Xin Lv and Xinghan Liu and Xinyi Liu and Xinyue Yang and Xixuan Song and Xunkai Zhang and Yifan An and Yifan Xu and Yilin Niu and Yuantao Yang and Yueyan Li and Yushi Bai and Yuxiao Dong and Zehan Qi and Zhaoyu Wang and Zhen Yang and Zhengxiao Du and Zhenyu Hou and Zihan Wang},
year={2024},
eprint={2406.12793},
archivePrefix={arXiv},
primaryClass={id='cs.CL' full_name='Computation and Language' is_active=True alt_name='cmp-lg' in_archive='cs' is_general=False description='Covers natural language processing. Roughly includes material in ACM Subject Class I.2.7. Note that work on artificial languages (programming languages, logics, formal systems) that does not explicitly address natural-language issues broadly construed (natural-language processing, computational linguistics, speech, text retrieval, etc.) is not appropriate for this area.'}
}
- Downloads last month
- 4,345
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐ Ask for provider support