ChatGLM2安装

AI/ 框架 AI 模型本地

创建时间:2024-03-20 23:26

字数:1.2k 阅读:

ChatGLM2安装

ChatGLM2安装

代码下载

下载指定commit的代码，当前教程基于当前commit编写。

1
2
3

git clone https://github.com/THUDM/ChatGLM2-6B
cd ChatGLM2-6B
git checkout 921d7e9adc69020a19169d1ba4f76c2675a2dd29

模型下载

set GIT_LFS_SKIP_SMUDGE=1
git clone https://huggingface.co/THUDM/chatglm2-6b
cd chatglm2-6b
git lfs pull
git checkout 7fabe56db91e085c9c027f56f1c654d137bdba40

安装环境

1
2
3

cd ChatGLM-6B
pip install torch==2.2.1 torchvision==0.17.1 torchaudio==2.2.1 --index-url https://download.pytorch.org/whl/cu121
pip install -r requirements-smile.txt

requirements-smile.txt

accelerate==0.28.0
aiofiles==23.2.1
aiohttp==3.9.3
aiosignal==1.3.1
altair==5.2.0
annotated-types==0.6.0
anyio==4.3.0
async-timeout==4.0.3
attrs==23.2.0
blinker==1.7.0
cachetools==5.3.3
certifi==2024.2.2
charset-normalizer==3.3.2
click==8.1.7
colorama==0.4.6
contourpy==1.2.0
cpm-kernels==1.0.11
cycler==0.12.1
exceptiongroup==1.2.0
fastapi==0.110.0
ffmpy==0.3.2
filelock==3.13.1
fonttools==4.49.0
frozenlist==1.4.1
fsspec==2024.2.0
gitdb==4.0.11
GitPython==3.1.42
gradio==3.50.2
gradio_client==0.6.1
h11==0.14.0
httpcore==1.0.4
httpx==0.27.0
huggingface-hub==0.21.4
idna==3.6
importlib_resources==6.3.0
Jinja2==3.1.3
jsonschema==4.21.1
jsonschema-specifications==2023.12.1
kiwisolver==1.4.5
latex2mathml==3.77.0
linkify-it-py==2.0.3
Markdown==3.6
markdown-it-py==2.2.0
MarkupSafe==2.1.5
matplotlib==3.8.3
mdit-py-plugins==0.3.3
mdtex2html==1.3.0
mdurl==0.1.2
mpmath==1.3.0
multidict==6.0.5
networkx==3.2.1
numpy==1.26.4
orjson==3.9.15
packaging==23.2
pandas==2.2.1
pillow==10.2.0
protobuf==4.25.3
psutil==5.9.8
pyarrow==15.0.1
pydantic==2.6.4
pydantic_core==2.16.3
pydeck==0.8.1b0
pydub==0.25.1
Pygments==2.17.2
pyparsing==3.1.2
python-dateutil==2.9.0.post0
python-multipart==0.0.9
pytz==2024.1
PyYAML==6.0.1
referencing==0.33.0
regex==2023.12.25
requests==2.31.0
rich==13.7.1
rpds-py==0.18.0
ruff==0.3.2
safetensors==0.4.2
semantic-version==2.10.0
sentencepiece==0.2.0
shellingham==1.5.4
six==1.16.0
smmap==5.0.1
sniffio==1.3.1
sse-starlette==2.0.0
starlette==0.36.3
streamlit==1.32.2
sympy==1.12
tenacity==8.2.3
tokenizers==0.13.3
toml==0.10.2
tomlkit==0.12.0
toolz==0.12.1
# torch==2.2.1+cu121
# torchaudio==2.2.1+cu121
# torchvision==0.17.1+cu121
tornado==6.4
tqdm==4.66.2
transformers==4.30.2
typer==0.9.0
typing_extensions==4.10.0
tzdata==2024.1
uc-micro-py==1.0.3
urllib3==2.2.1
uvicorn==0.28.0
watchdog==4.0.0
websockets==11.0.3
yarl==1.9.4

模型路径修改

将代码中的模型路径改为自己的模型路径

1 2	tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm2-6b", trust_remote_code=True) model = AutoModel.from_pretrained("THUDM/chatglm2-6b", trust_remote_code=True).cuda()

改为

1 2	tokenizer = AutoTokenizer.from_pretrained("I:\AI\chatglm2-6b", trust_remote_code=True) model = AutoModel.from_pretrained("I:\AI\chatglm2-6b", trust_remote_code=True).cuda()

显存适配

根据自己的显存修改运行脚本

量化等级	编码 2048 长度的最小显存	生成 8192 长度的最小显存
FP16 / BF16	13.1 GB	12.8 GB
INT8	8.2 GB	8.1 GB
INT4	5.5 GB	5.1 GB

FP16 / BF16

1 2	tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm2-6b", trust_remote_code=True) model = AutoModel.from_pretrained("THUDM/chatglm2-6b", trust_remote_code=True).cuda()

INT8

1 2	tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm2-6b", trust_remote_code=True) model = AutoModel.from_pretrained("THUDM/chatglm2-6b", trust_remote_code=True).quantize(8).half().cuda()

INT4

1 2	tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm2-6b", trust_remote_code=True) model = AutoModel.from_pretrained("THUDM/chatglm2-6b", trust_remote_code=True).quantize(4).half().cuda()

使用

网页版

1	python web_demo.py

打开网址 http://127.0.0.1:7860
就可以进行聊天了

StreamLit网页

1	streamlit run web_demo2.py

API部署

1	python api.py

发送请求

1
2
3

curl -X POST "http://127.0.0.1:8000" \
     -H 'Content-Type: application/json' \
     -d '{"prompt": "你好", "history": []}'

回复结果

{"response":"你好👋！我是人工智能助手 ChatGLM2-6B，很高兴见到你，欢迎问我任何问题。","history":[["你好","你好👋！我是人工智能助手 ChatGLM2-6B，很高兴见到你，欢迎问我任何问题。"]],"status":200,"time":"2024-03-21 12:01:12"}

OpenAI API部署

1	python openai_api.py

发送请求

import openai
if __name__ == "__main__":
    openai.api_base = "http://localhost:8000/v1"
    openai.api_key = "none"
    for chunk in openai.ChatCompletion.create(
        model="chatglm2-6b",
        messages=[
            {"role": "user", "content": "你好"}
        ],
        stream=True
    ):
        if hasattr(chunk.choices[0].delta, "content"):
            print(chunk.choices[0].delta.content, end="", flush=True)

微信：宏沉一笑

公众号：漫步之行

签名：Smile every day
名字：宏沉一笑
邮箱：whghcyx@outlook.com
个人网站：https://whg555.github.io

转载请注明来源，欢迎对文章中的引用来源进行考证，欢迎指出任何有错误或不够清晰的表达。可以在下面评论区评论，也可以邮件至 whghcyx@outlook.com