RYDE-WORK/ktransformers
1
0
Fork 0
mirror of https://github.com/RYDE-WORK/ktransformers.git synced 2026-01-19 12:43:16 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

Commit Graph

  • 7e1fe256c8 optimize GPU Atream 2025-02-21 05:06:57 +00:00
  • 25c5bddd08
         Merge pull request #506 from makllama/musa Azure 2025-02-20 22:50:31 +08:00
  • 1dd84b4a5b
         Merge pull request #550 from kvcache-ai/docker_dev ZiWei Yuan 2025-02-20 22:29:56 +08:00
  • ed74f91a25 ⚡ docker-iamge workflow liam 2025-02-20 22:27:51 +08:00
  • a6aee1c54a Merge branch 'main' of https://github.com/kvcache-ai/ktransformers into main liam 2025-02-20 22:00:55 +08:00
  • 64a00651b1
         Merge pull request #232 from HandSonic/patch-1 ZiWei Yuan 2025-02-20 22:00:16 +08:00
  • 97e1dc97f6 ⚡ fix .so bug liam 2025-02-20 21:24:46 +08:00
  • 574b13390b ⚡ fin docker-image liam 2025-02-20 12:23:11 +08:00
  • b05cc5685d fix: fix SSE formatting xv44586 2025-02-20 15:01:35 +08:00
  • 0cdc446a19 默认并行编译 miaooo0000OOOO 2025-02-20 11:09:16 +08:00
  • 2207f6cd14 feat: Support Moore Threads GPU Xiaodong Ye 2025-02-19 18:26:55 +08:00
  • b4fb633991
         Merge pull request #501 from Zhoneym/main Atream 2025-02-19 16:46:48 +08:00
  • c92a300bd8
         Necessary tips for Node.js related issues Zhoneym 2025-02-19 16:37:18 +08:00
  • c9eb1f76f6
         Merge pull request #384 from ubergarm/add-tutorial-notes Atream 2025-02-19 13:46:49 +08:00
  • 89f8218a2a
         Merge pull request #487 from kvcache-ai/clean_pr Atream 2025-02-19 12:46:48 +08:00
  • a529518346 clean PR code and disable flashinfer Atream 2025-02-19 04:42:47 +00:00
  • cf4da5fd47
         Merge pull request #382 from ceerRep/server-prefix-cache Atream 2025-02-19 12:33:36 +08:00
  • ea75849d03
         Merge pull request #481 from Zhoneym/main Azure 2025-02-19 11:19:11 +08:00
  • c72291832d
         Update install.md Zhoneym 2025-02-19 10:39:43 +08:00
  • 3015e9dfa7
         Modify and add any incorrect or missing content in the install.md file. Zhoneym 2025-02-19 10:01:16 +08:00
  • 584c7d5639 fix: object type for non-streaming response ceerrep 2025-02-18 23:49:51 +08:00
  • 6d45871de8 fix: workaround return dummy usage ceerrep 2025-02-18 22:39:49 +08:00
  • 09f5c5e77e
         Merge pull request #476 from kvcache-ai/V0.2.1.post1 ZiWei Yuan 2025-02-18 21:48:30 +08:00
  • 75d7d423b0 🔖 release v0.2.1.post1 liam 2025-02-18 20:45:48 +08:00
  • a435e1bbcd
         Merge pull request #406 from hoshinohikari/main Azure 2025-02-18 20:09:59 +08:00
  • db61375f1a Merge remote-tracking branch 'kvai/main' into server-prefix-cache ceerrep 2025-02-18 17:51:22 +08:00
  • 19fd24f461
         Merge pull request #412 from haojiubujian1985/delete-unused-link Azure 2025-02-18 14:58:10 +08:00
  • 1c1769a579
         Merge pull request #413 from kvcache-ai/fix_precision_MLA Atream 2025-02-18 14:44:50 +08:00
  • 592e13d453 ⚡ add mmlu_pro test liam 2025-02-18 14:43:38 +08:00
  • 1054ee3ae4
         Merge pull request #450 from kvcache-ai/update-WeChatGroup Atream 2025-02-18 14:34:07 +08:00
  • 261d5d393f
         Add files via upload Atream 2025-02-18 14:28:29 +08:00
  • 2671fc7f4b
         Merge pull request #445 from dedfaf/main Azure 2025-02-18 14:23:29 +08:00
  • ccf0f1910f
         Merge pull request #442 from Zhoneym/patch-1 UnicornChan 2025-02-18 13:58:44 +08:00
  • 07a0555016 ⚡ fix device and add test liam 2025-02-18 12:52:17 +08:00
  • 73d072f609 Merge branch 'fix_precision_MLA' of https://github.com/kvcache-ai/ktransformers into server-prefix-cache ceerrep 2025-02-18 11:44:28 +08:00
  • 2ffb43f974 fix: adapt prefix cache in forward_linux_flashinfer ceerrep 2025-02-18 11:40:15 +08:00
  • f029588b61 fix server warmup Xie Weiyu 2025-02-18 11:39:45 +08:00
  • 8b221f532a
         Fix `GLIBCXX_3.4.30' not found in Anaconda Zhoneym 2025-02-18 11:32:42 +08:00
  • c70b6f4d5b fix: use 'cuda:0' by default if torch_device is 'cuda' ceerrep 2025-02-18 11:15:17 +08:00
  • 3e5c5a2f4a Fix typo in DeepseekR1_V3_tutorial.md Line 174 dedfaf 2025-02-18 10:57:12 +08:00
  • 9f1da18630 update install.sh Xie Weiyu 2025-02-17 20:50:19 +08:00
  • c176e516b5 server mix mla Xie Weiyu 2025-02-17 20:40:28 +08:00
  • ee24eb8dc3 fix: fix server for triton kernel ceerrep 2025-02-17 18:08:45 +08:00
  • bb1cadfff3 Merge branch 'fix_precision_MLA' of https://github.com/kvcache-ai/ktransformers into server-prefix-cache ceerrep 2025-02-17 18:08:04 +08:00
  • 038bc30888 fix precision bug imported by position_ids in 0.2.0 Atream 2025-02-17 09:23:14 +00:00
  • 49c6e2fc04 Update README_ZH.md xubo 2025-02-17 16:56:44 +08:00
  • 6d9182795c
         Fix cmake error caused by lack of environment variables in Windows environment hoshinohikari 2025-02-17 15:56:56 +08:00
  • cd9f7f8f34 fix: server: drop <think> tag in chat template ceerrep 2025-02-17 14:14:10 +08:00
  • bb39eeb005 Add notes to DeepSeek-R1 tutorial documentation John W. Leimgruber III 2025-02-16 13:40:27 -05:00
  • ca2090d89b feat: use model name in openai endpoint ceerrep 2025-02-17 00:11:54 +08:00
  • 5ac266085e fix: use flash_attn for faster prefill ceerrep 2025-02-17 00:11:24 +08:00
  • bb0ccc7b1a feat: add prefix cache for server ceerrep 2025-02-17 00:10:55 +08:00
  • c515cc49a5
         Merge pull request #355 from kvcache-ai/qiyuxinlin-patch-1 wang jiahao 2025-02-16 15:05:43 +08:00
  • 24de607be4
         Merge pull request #357 from kvcache-ai/qiyuxinlin-patch-2 wang jiahao 2025-02-16 15:05:19 +08:00
  • b84524622e support bf16 read Atream 2025-02-16 06:43:27 +00:00
  • 76554dd674
         Update SUMMARY.md wang jiahao 2025-02-16 14:30:22 +08:00
  • 56d19d6115
         Update install.md wang jiahao 2025-02-16 14:23:56 +08:00
  • 9f9c3738f9
         Merge pull request #354 from Azure-Tang/fix-mockTritonMLA Azure 2025-02-16 14:06:54 +08:00
  • ff6b265e53 Mock triton mla due to precision issue Azure 2025-02-16 06:03:12 +00:00
  • c5f036e8a4
         Merge pull request #333 from kvcache-ai/feat_experts_gpu Atream 2025-02-15 23:30:24 +08:00
  • 8ed8eb2a9e
         Update FAQ.md Atream 2025-02-15 23:27:35 +08:00
  • c189d55bd1 toy support for experts on GPU, no CUDA Graph Atream 2025-02-15 15:16:00 +00:00
  • ae8da019c1
         Merge pull request #330 from hrz6976/fix-nonetype wang jiahao 2025-02-15 22:50:51 +08:00
  • 56382aa869
         Merge pull request #290 from ZhangShuaiyi/dev/check_gguf_path Azure 2025-02-15 22:48:57 +08:00
  • c8bf250104
         Merge pull request #324 from BiFangKNT/patch-2 Azure 2025-02-15 22:23:09 +08:00
  • 4516282ccc Fix NoneType object has no attribute zero_ 12f23eddde 2025-02-15 22:04:45 +08:00
  • 3c6035aa8a
         Merge pull request #329 from kvcache-ai/fix_doc ZiWei Yuan 2025-02-15 21:45:19 +08:00
  • 69b00753c2 📝 fix typo liam 2025-02-15 21:42:59 +08:00
  • bd693b698f
         Update DeepseekR1_V3_tutorial.md 彼方 2025-02-15 19:43:18 +08:00
  • 65d73ea3f8
         Merge pull request #317 from kvcache-ai/develop-0.2.1 UnicornChan 2025-02-15 16:19:18 +08:00
  • 718a71b3a9
         Merge pull request #316 from KMSorSMS/main ZiWei Yuan 2025-02-15 16:19:00 +08:00
  • 13382f88ab 📝 update V0.2.1 Doc liam 2025-02-15 16:06:56 +08:00
  • 0e4b7a3929 [feature] update docker image and entrypoint chenxl 2025-02-15 07:55:33 +00:00
  • f9f9f746c0
         Merge pull request #315 from kvcache-ai/Atream-add-adapted Atream 2025-02-15 15:45:34 +08:00
  • 92399283b6
         Update attention.py Atream 2025-02-15 15:43:35 +08:00
  • d90749d35d
         Update triton_attention.py Atream 2025-02-15 15:41:01 +08:00
  • 22280bf17f get dirname if gguf_path is a file Shuaiyi 2025-02-14 10:22:56 +00:00
  • 1548c99234
         Merge pull request #307 from Azure-Tang/main Azure 2025-02-15 10:06:56 +08:00
  • 227e81b0d3 update zh readme Azure 2025-02-15 01:54:01 +00:00
  • ef89b1520b * Reorganize documentation/README * Consolidate the installation section, as it's currently too cluttered * Move the Multi-GPU section to the top-level structure * Add a **detailed** tutorial on registering extra GPU memory with Marlin Azure 2025-02-14 19:57:09 +00:00
  • b0b90270d8
         Merge pull request #306 from kvcache-ai/revert-305-main Azure 2025-02-15 03:44:42 +08:00
  • 4f4ed36442
         Revert "[update] Reorganize documentation/README" Azure 2025-02-15 03:43:48 +08:00
  • 19d4a50b1c
         [update] Reorganize documentation/README Azure 2025-02-15 03:41:43 +08:00
  • 483182fc3a fix typo and detail Azure 2025-02-14 19:40:15 +00:00
  • 823b25eec9 Reorganize documentation/README Azure 2025-02-14 19:08:17 +00:00
  • cc8d627e32
         Merge pull request #301 from kvcache-ai/fix-cuda-graph-bug Atream 2025-02-14 23:54:43 +08:00
  • 1946493f2d warm_up before capture Atream 2025-02-14 15:52:21 +00:00
  • cadd55078f
         Merge pull request #295 from kvcache-ai/update-wechatgroup Atream 2025-02-14 19:52:05 +08:00
  • e153d78227
         Add files via upload Atream 2025-02-14 19:51:11 +08:00
  • 96e6dff7ac
         Delete WeChatGrouop.png Atream 2025-02-14 19:49:14 +08:00
  • 885a91e7db
         Merge pull request #294 from kvcache-ai/feat-fast-MLA Atream 2025-02-14 19:40:36 +08:00
  • 1084d4e4b4 linux support triton MLA kernel Atream 2025-02-14 11:38:55 +00:00
  • 6738908699
         Merge pull request #280 from Azure-Tang/main Azure 2025-02-14 17:12:14 +08:00
  • 1b1f417267 Fix incorrect image content in the document Azure 2025-02-14 09:04:22 +00:00
  • f4bb374eaf
         Merge pull request #254 from Azure-Tang/main Azure 2025-02-14 11:54:14 +08:00
  • 95c81eaf01
         Merge branch 'kvcache-ai:main' into main Azure 2025-02-14 11:53:52 +08:00
  • 0a9c59922a
         Merge pull request #255 from kvcache-ai/update-wechatgroup Atream 2025-02-14 11:08:59 +08:00
  • ce7210321a
         Add files via upload Atream 2025-02-14 11:06:56 +08:00
  • b7653b9c4f add V3/R1 8 gpu yaml example Azure 2025-02-14 02:56:13 +00:00
  • e612b14739
         Merge pull request #247 from liugddx/patch-1 Azure 2025-02-14 10:37:32 +08:00