7 Commits

Author SHA1 Message Date
imClumsyPanda
fbaca1009e update requirements.txt, requirements_api.txt, test_different_splitter.py and chinese_recursive_text_splitter.py 2023-09-14 22:59:05 +08:00
zR
bfdbe69fa1
增加了自定义分词器适配 (#1462)
* 添加了自定义分词器适配和测试文件
---------

Co-authored-by: zR <zRzRzRzRzRzRzR>
2023-09-13 15:42:12 +08:00
imClumsyPanda
4aa14b859e
增加 ChineseRecursiveTextSplitter (#1447)
* add RapidOCRPDFLoader

* update mypdfloader.py and requirements.txt

* add myimgloader.py

* add test samples

* add TODO to mypdfloader

* add loaders to KnowledgeFile class

* add loaders to KnowledgeFile class

* add ChineseRecursiveTextSplitter

* add ChineseRecursiveTextSplitter
2023-09-12 17:38:52 +08:00
imClumsyPanda
8d463a31fd update import pkgs and format 2023-08-10 21:50:38 +08:00
imClumsyPanda
8a4d9168fa update import pkgs and format 2023-08-10 21:26:05 +08:00
imClumsyPanda
24a280ce8c re-add zh_title_enhance.py 2023-08-09 23:09:24 +08:00
imClumsyPanda
dcf49a59ef v0.2.0 first commit 2023-07-27 23:22:07 +08:00