RYDE-WORK/ktransformers
1
0
Fork 0
mirror of https://github.com/RYDE-WORK/ktransformers.git synced 2026-01-19 21:03:18 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge pull request #51 from molamooo/fix-f16-dequantize-device

Browse Source 
[fix] f16 dequantize device ignored
This commit is contained in:
UnicornChan 2024-08-22 16:31:45 +08:00 committed by GitHub
commit 1f85db3d73
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
ktransformers/util/custom_gguf.py
View File

@ -681,7 +681,7 @@ def dequantize_f16_gpu(data, device):
res = torch.from_numpy(data)
res_gpu = torch.empty_like(res, device=device)
res_gpu.copy_(res)
return res
return res_gpu
GGML_DEQUANTIZE = {
"F32": dequantize_f32,