Merge pull request #26 from kvcache-ai/windows

[fix] linux and windows can all find CPUInfer in current Directory

README.md

@@ -82,11 +82,12 @@ Some preparation:
 <h3>Installation</h3>
 
 1. Use a Docker image, see [documentation for Docker](./doc/en/docker.md) 
-2. You can install using Pypi:
+2. You can install using Pypi (for linux):
 
    ```
    pip install ktransformers --no-build-isolation
    ```
+   for windows we prepare a pre compiled whl package in [ktransformers-0.1.1+cu125torch24avx2-cp311-cp311-win_amd64.whl](https://github.com/kvcache-ai/ktransformers/releases/download/v0.1.1/ktransformers-0.1.1+cu125torch24avx2-cp311-cp311-win_amd64.whl), which require cuda-12.5, torch-2.4, python-3.11, more pre compiled package are being produced. 
 
 3. Or you can download source code and compile:
    - init source code 
@@ -97,11 +98,16 @@ Some preparation:
      git submodule update
      ```
    - [Optional] If you want to run with website, please [compile the website](./doc/en/api/server/website.md) before execute ```bash install.sh```
-   - Compile and install
+   - Compile and install (for Linux)
      ```
      bash install.sh
      ```
 
+   - Compile and install(for Windows)
+     ```
+     install.bat
+     ``` 
+
 <h3>Local Chat</h3>
 We provide a simple command-line local chat Python script that you can run for testing. 
 

install.bat

@@ -0,0 +1,16 @@
+@echo off
+
+REM clear build dirs
+rmdir /S /Q ktransformers\ktransformers_ext\build
+rmdir /S /Q ktransformers\ktransformers_ext\cuda\build
+rmdir /S /Q ktransformers\ktransformers_ext\cuda\dist
+rmdir /S /Q ktransformers\ktransformers_ext\out
+del /F /Q ktransformers\ktransformers_ext\cuda\*.egg-info
+
+echo Installing python dependencies from requirements.txt
+pip install -r requirements-local_chat.txt
+
+echo Installing ktransformers
+set KTRANSFORMERS_FORCE_BUILD=TRUE
+pip install . --no-build-isolation
+echo Installation completed successfully

install.sh

@@ -11,5 +11,5 @@ echo "Installing python dependencies from requirements.txt"
 pip install -r requirements-local_chat.txt
 
 echo "Installing ktransformers"
-pip install . --no-build-isolation
+KTRANSFORMERS_FORCE_BUILD=TRUE pip install . --no-build-isolation
 echo "Installation completed successfully"

ktransformers/ktransformers_ext/CMakeLists.txt

@@ -1,4 +1,4 @@
-cmake_minimum_required(VERSION 3.17)
+cmake_minimum_required(VERSION 3.16)
 project(cpuinfer_ext VERSION 0.1.0)
 
 set(CMAKE_CXX_STANDARD 17)

ktransformers/operators/experts.py

@@ -19,8 +19,9 @@ import torch
 import sys, os
 from ktransformers.operators.base_operator import BaseInjectedModule
 
-#sys.path.append(os.path.dirname(__file__) + "/../ktransformers_ext/build/")
-sys.path.append(os.path.dirname(__file__) + "\\..\\ktransformers_ext\\build\\Release")
+sys.path.append(os.path.join(os.path.dirname(__file__), "..", "ktransformers_ext", "build"))
+sys.path.append(os.path.join(os.path.dirname(__file__), "..", "ktransformers_ext", "build", "Release"))
+sys.path.append(os.path.join(os.path.dirname(__file__), "..", "ktransformers_ext", "build", "Debug"))
 import cpuinfer_ext
 from cpuinfer_ext.moe import MOEConfig, MOE
 import ctypes