RYDE-WORK/MiniCPM
1
0
Fork 0
mirror of https://github.com/RYDE-WORK/MiniCPM.git synced 2026-02-05 22:53:34 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

Update README-en.md

Browse Source
This commit is contained in:
William 2024-02-01 13:48:01 +08:00 committed by GitHub
parent 2e5b9d95c6
commit 77e57b4f52
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 38 additions and 42 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

42
README-en.md
View File

@ -144,34 +144,30 @@ The capital city of China is Beijing. Beijing is not only the political center o
## Deployment on mobile phones ## Deployment on mobile phones
<!-- 进行Int4量化后，MiniCPM只占2GB空间，具备在端侧手机进行模型部署的条件。 #### Tutorial
对此，我们针对Android和Harmony系统使用开源框架MLC-LLM进行模型适配，针对iPhone系统使用开源框架LLMFarm进行模型适配，并分别选取了部分端侧手机设备进行了测试。 -->
After INT4 quantization, MiniCPM only occupies 2GB of space, meeting the requirements of inference on edge devices.
We utilize the open-source framework [MLC-LLM](https://github.com/mlc-ai/mlc-llm) for deployment on Android and Harmony OS. For deployment on IOS, we adapt MiniCPM using [LLMFarm](https://github.com/guinmoon/LLMFarm). We select some mobile phones for testing respectively. * After INT4 quantization, MiniCPM only occupies 2GB of space, meeting the requirements of inference on end devices.
* We have made different adaptations for different operating systems.
### Tutorial * **Note: The current open-source framework is still improving its support for mobile phones, and not all chips and operating system versions can successfully run MLC-LLM or LLMFarm.**
* Android, Harmony OS
#### Android * Adapt based on [MLC-LLM](https://github.com/mlc-ai/mlc-llm).
<!-- android编译安装MiniCPM指南 [EN](https://github.com/OpenBMB/mlc-MiniCPM/blob/main/README.md) --> * Adapted for text model MiniCPM, and multimodel model MiniCPM-V.
[Compilation and installation on Android](https://github.com/OpenBMB/mlc-MiniCPM/blob/main/README.md) * Support MiniCPM-2B-SFT-INT4、MiniCPM-2B-DPO-INT4、MiniCPM-V.
* [Compile and Installation Guide](https://github.com/OpenBMB/mlc-MiniCPM/blob/main/README.md)
#### IOS * iOS
<!-- [ios编译安装MiniCPM指南](https://github.com/OpenBMB/LLMFarm) --> * Adapt based on [LLMFarm](https://github.com/guinmoon/LLMFarm).
[Compilation and installation on IOS](https://github.com/OpenBMB/LLMFarm) * Adapted for text model MiniCPM.
* Support MiniCPM-2B-SFT-INT4、MiniCPM-2B-DPO-INT4.
#### Multimodal * [Compile and Installation Guide](https://github.com/OpenBMB/LLMFarm)
### Performance ### Performance
<!-- 我们并为针对手机部署进行深度优化，仅验证MiniCPM使用手机芯片进行推理的可行性。 * We did not conduct in-depth optimization and system testing on the mobile inference model, only verifying the feasibility of MiniCPM using mobile phone chips for inference.
**我们也欢迎更多开发者进一步调优并更新下面的测试列表，不断提升端侧大模型在手机上的推理性能。** --> * There have been no previous attempts to deploy multimodal models on mobile phones. We have verified the feasibility of deploying MiniCPM-V on mobile phones based on MLC-LLM this time, and it can input and output normally. However, there also exist a problem of long image processing time, which needs further optimization :)
Instead of conducting in-depth optimization for deployment on mobile phones, we only verify the feasibility of MiniCPM using mobile chips for inference. * **We welcome more developers to continuously improve the inference performance of LLMs on mobile phones and update the test results below.**
**We welcome more developers to continuously improve the inference performance of LLMs on mobile phones and update the test results below.**
| Mobile Phones | OS | Processor | Memory（GB） | Inference Throughput（token/s） | | Mobile Phones | OS | Processor | Memory（GB） | Inference Throughput（token/s） |
|-|-|-|-|-| | ----------------- | ------------- | ------------------ | ------------ | ------------------------------- |
| OPPO Find N3 | Android 13 | snapdragon 8 Gen2 | 12 | 6.5 | | OPPO Find N3 | Android 13 | snapdragon 8 Gen2 | 12 | 6.5 |
| Samsung S23 Ultra | Android 14 | snapdragon 8 Gen2 | 12 | 6.4 | | Samsung S23 Ultra | Android 14 | snapdragon 8 Gen2 | 12 | 6.4 |
| Meizu M182Q | Android 11 | snapdragon 888Plus | 8 | 3.7 | | Meizu M182Q | Android 11 | snapdragon 888Plus | 8 | 3.7 |
@ -191,7 +187,7 @@ Instead of conducting in-depth optimization for deployment on mobile phones, we
| iPhone 12 | iOS 17.2.1 | A14 | 4 | 5.8 | | iPhone 12 | iOS 17.2.1 | A14 | 4 | 5.8 |
| iPhone 11 | iOS 16.6 | A13 | 4 | 4.6 | | iPhone 11 | iOS 16.6 | A13 | 4 | 4.6 |
![多模态样例](https://github.com/OpenBMB/OmniLMM/blob/main/assets/Snake_cn_Mushroom_en.gif) ![multimodel demo](https://github.com/OpenBMB/OmniLMM/blob/main/assets/Snake_cn_Mushroom_en.gif)
## Demo & API ## Demo & API