From e65930db1c87c11937838654ca98916616d4d7ad Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E5=88=98=E4=B8=B9?= Date: Tue, 25 Jun 2024 11:50:53 +0800 Subject: [PATCH] =?UTF-8?q?=E5=A2=9E=E5=8A=A0=E4=BA=86readme=E4=B8=AD?= =?UTF-8?q?=E6=A8=A1=E5=9E=8B=E9=87=8F=E5=8C=96=E7=9A=84=E5=AF=BC=E8=88=AA?= =?UTF-8?q?=E4=BB=A5=E5=8F=8A=E7=A4=BA=E4=BE=8B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/README.md b/README.md index b47b471..6cf28b7 100644 --- a/README.md +++ b/README.md @@ -49,6 +49,7 @@ MiniCPM 是面壁智能与清华大学自然语言处理实验室共同开源的 - [更新日志](#0) - [模型下载](#1) - [快速上手](#2) +- [模型量化](#quantize) - [开源社区](#community) - [评测结果](#3) - [手机部署](#4) @@ -258,7 +259,9 @@ print(model.response("<用户>山东省最高的山是哪座山, 它比黄山高 ```shell python -m mlx_lm.generate --model mlx-community/MiniCPM-2B-sft-bf16-llama-format-mlx --prompt "hello, tell me a joke." --trust-remote-code ``` +

+## 模型量化 **gptq量化** 1. 首先git获取[minicpm_gptqd代码](https://github.com/LDLINGLINGLING/AutoGPTQ/tree/minicpm_gptq) 2. 进入minicpm_gptqd主目录./AutoGPTQ,命令行输入: