From 58ff788df667b493146af0ed88fecc63c02438e3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E5=88=98=E4=B8=B9?= Date: Thu, 27 Jun 2024 09:42:20 +0800 Subject: [PATCH] =?UTF-8?q?=E4=BF=AE=E6=94=B9=E4=BA=86awq=E7=9A=84?= =?UTF-8?q?=E9=87=8F=E5=8C=96readme?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- README.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 6b61a02..0c4a69d 100644 --- a/README.md +++ b/README.md @@ -312,11 +312,14 @@ print(model.response("<用户>山东省最高的山是哪座山, 它比黄山高 custom_data=[{'question':'过敏性鼻炎有什么症状？','answer':'过敏性鼻炎可能鼻塞，流鼻涕，头痛等症状反复发作，严重时建议及时就医。'}, {'question':'1+1等于多少？','answer':'等于2'}] ``` -4. 根据选择的数据集，修改quantize/awq_quantize.py 第三十八行： +4. 根据选择的数据集，修改quantize/awq_quantize.py 为以下三行代码其中一行： ```python - model.quantize(tokenizer, quant_config=quant_config, calib_data=load_wikitext(quant_data_path=quant_data_path))#使用wikitext进行量化 - model.quantize(tokenizer, quant_config=quant_config, calib_data=load_alpaca(quant_data_path=quant_data_path))#使用alpaca进行量化 - model.quantize(tokenizer, quant_config=quant_config, calib_data=load_cust_data(quant_data_path=quant_data_path))#使用自定义数据集进行量化 + #使用wikitext进行量化 + model.quantize(tokenizer, quant_config=quant_config, calib_data=load_wikitext(quant_data_path=quant_data_path)) + #使用alpaca进行量化 + model.quantize(tokenizer, quant_config=quant_config, calib_data=load_alpaca(quant_data_path=quant_data_path)) + #使用自定义数据集进行量化 + model.quantize(tokenizer, quant_config=quant_config, calib_data=load_cust_data(quant_data_path=quant_data_path)) ``` 5. 运行quantize/awq_quantize.py文件,在设置的quan_path目录下可得awq量化后的模型。