7 Commits

Author SHA1 Message Date
zR
3e7c74ec7a 128k模型无法推理的问题修复 2024-04-14 19:00:46 +08:00
zR
8272667430 更新VLLM的书写方式 2024-04-14 14:23:11 +08:00
Y.W. Fang
75b144b04d fix typo in demo 2024-02-02 21:28:14 +08:00
ywfang
d39a6a10bb
Merge branch 'main' into feat/allow-set-torch-dtype 2024-02-02 17:00:22 +08:00
Su Yang
d64fe362fc
feat: allow user set torch dtype 2024-02-02 14:46:41 +08:00
Su Yang
6c4bbad9ed
feat: allow user change the demo host and port 2024-02-02 14:32:09 +08:00
Y.W. Fang
6cfeec2e54 update demo with vllm 2024-02-01 13:13:27 +08:00