57 Commits

Author SHA1 Message Date
imClumsyPanda
7863e0fea8 updata MyFAISS 2023-06-12 00:06:06 +08:00
imClumsyPanda
f1f742ce44 add self-defined class MyFAISS 2023-06-07 23:18:47 +08:00
imClumsyPanda
8b7c2e417c add support for csv file 2023-06-07 22:14:08 +08:00
shrimp
57b4f9306d
修复BUG (#570)
一、修复模型历史记录仅一条问题,修改了模型加载代码,已修复。 
二、修复模型历史记录仅一条问题,修改了webui有一个地方有问题,已修复。
三、知识库单条数据入库因知识库名称缓存问题导致的BUG,也已修复。
2023-06-07 18:12:51 +08:00
imClumsyPanda
b1ba2003be update webui.py 2023-05-31 20:55:45 +08:00
liunux4odoo
88fee48d8c
use lru_cache to optimize loading of local vector store (#496)
* Add files via upload

* Update README.md

* use lru_cache to optimize loading of local vector store

---------

Co-authored-by: imClumsyPanda <littlepanda0716@gmail.com>
Co-authored-by: liunux <liunux@office>
2023-05-30 23:05:39 +08:00
Ding Junyao
f7e120fe56
add tree func for reading files in dirs (#471) 2023-05-27 22:00:43 +08:00
glide-the
c4ee36b8ac 删除 AnswerResultStream 、generate_with_callback收集器 2023-05-25 21:07:40 +08:00
imClumsyPanda
bca32cae83 update local_doc_qa.py 2023-05-22 18:09:23 +08:00
imClumsyPanda
c79058f064
更新 local_doc_qa.py 2023-05-22 13:14:23 +08:00
yawudede
4634a578f4
local_doc_qa.py 2023-05-22 10:39:55 +08:00
yawudede
9140a5e70c
local_doc_qa.py 2023-05-22 10:36:41 +08:00
yawudede
037b5e4502
local_doc_qa.py
添加精排
2023-05-22 10:32:58 +08:00
imClumsyPanda
e352a04cbb merge master 2023-05-22 00:18:48 +08:00
imClumsyPanda
9c422cc6bc update bing_search.py 2023-05-21 22:08:38 +08:00
imClumsyPanda
2681728329 update local_doc_qa.py 2023-05-20 01:54:08 +08:00
imClumsyPanda
e8a37ff4c7 update loader.py 2023-05-20 01:24:35 +08:00
zhenkaivip
9f5d1afc31
fix bug : 1、历史最后一条丢失;2、上下两行连读造成歧义 (#415)
* fix _call bug

* fix

* fix bug : 上下两行连读造成歧义
2023-05-20 01:19:22 +08:00
glide-the
33bbb4779e llm_model_dict 处理了loader的一些预设行为,如加载位置,模型名称,模型处理器实例, 定义checkpoint名称和远程路径
loader.py: 模型重载
定义 generatorAnswer 增加 AnswerResultStream
   定义generate_with_callback收集器,在每次响应时将队列数据同步到AnswerResult
requirements.txt 变更项目依赖
2023-05-18 22:54:41 +08:00
imClumsyPanda
621a0fe686 update cli_demo.py 2023-05-13 11:45:57 +08:00
zhenkaivip
dd93837343
使用paddleocr实现 (#342)
* jpg and png ocr

* fix

* write docs to tmp file

* fix

* [BUGFIX] local_doc_qa.py line 172: logging have no end args. (#323)

* image loader

* fix

* fix

* update api.py

* update api.py

* update api.py

* update README.md

* update api.py

* add pdf_loader

* fix

---------

Co-authored-by: RainGather <3255329+RainGather@users.noreply.github.com>
Co-authored-by: imClumsyPanda <littlepanda0716@gmail.com>
2023-05-13 08:45:17 +08:00
Tyler Luan
64406cee45
Add MOSS (#317)
* Add MOSS

* Update local_doc_qa.py

---------

Co-authored-by: imClumsyPanda <littlepanda0716@gmail.com>
2023-05-11 18:42:19 +08:00
imClumsyPanda
5852b4c62e update pdf read strategy 2023-05-11 10:54:45 +08:00
imClumsyPanda
fbdc62d910 merge master 2023-05-10 17:59:28 +08:00
shrimp
55504fcd4d
新增加知识库测试能力 (#302) 2023-05-10 17:18:20 +08:00
JohnSaxon
466bfb7a3f
logger bug introduced by https://github.com/imClumsyPanda/langchain-ChatGLM/pull/268 , fixed #295 (#297) 2023-05-10 13:28:44 +08:00
imClumsyPanda
c613c41d0e update local_doc_qa.py 2023-05-08 23:49:57 +08:00
imClumsyPanda
e2d7452c7b update chinese_text_splitter.py 2023-05-06 23:26:49 +08:00
keenzhu
2c72a00954
Update local_doc_qa.py
增加命令行导入的进度条显示
增加文本提示
2023-05-06 18:39:58 +08:00
shrimp
14d998b8e6
可选择lora权重加载 (#231)
* Add files via upload

增加lora权重使用

* Update model_config.py

* Add files via upload

修复一个小错误,少写了模型加载

* 使用lora微调的权重

使用lora微调的权重

* Update model_config.py
2023-05-04 21:01:55 +08:00
Winter
47922d2ee3
fix: 扩展文档的代码逻辑 (#227)
* fix: 扩展文档的代码逻辑

* Update local_doc_qa.py

---------

Co-authored-by: imClumsyPanda <littlepanda0716@gmail.com>
2023-05-04 20:58:15 +08:00
imClumsyPanda
07ff81a119 update torch_gc 2023-05-04 20:48:36 +08:00
imClumsyPanda
2c1fd2bdd5 add api.py 2023-05-03 22:31:28 +08:00
imClumsyPanda
a4e67a67b4 update local_doc_qa.py 2023-05-02 01:11:05 +08:00
imClumsyPanda
4df9d76f8a add streaming option in configs/model_config.py 2023-05-01 23:52:28 +08:00
imClumsyPanda
0e8cc0d16c add streaming option in configs/model_config.py 2023-05-01 23:52:28 +08:00
imClumsyPanda
98a8281b29 update text_splitter 2023-04-28 00:02:42 +08:00
imClumsyPanda
966def8cfe add stream support to webui.py 2023-04-26 23:27:15 +08:00
imClumsyPanda
b4aefca555 add stream support to cli_demo.py 2023-04-26 23:27:15 +08:00
imClumsyPanda
88ab9a1d21 update webui.py and local_doc_qa.py 2023-04-26 23:27:15 +08:00
wangxinkai
059fe82887 feat: 添加mmr相似度搜索,支持返回相似度分数 2023-04-26 23:27:15 +08:00
imClumsyPanda
a1033698dd update local_doc_qa.py, webui.py and README.md 2023-04-26 23:27:15 +08:00
imClumsyPanda
0706a29057 update embedding define in local_doc_qa.py and langchain version in requirements.txt 2023-04-22 12:30:27 +08:00
imClumsyPanda
b9d8824fa5 Update local_doc_qa.py 2023-04-21 21:27:20 +08:00
imClumsyPanda
ec677beed0 Update local_doc_qa.py 2023-04-21 21:27:20 +08:00
soon
37ceeae6e2 提交更 2023-04-21 21:27:20 +08:00
imClumsyPanda
f147043253 fix bug in chatglm_llm.py 2023-04-20 12:50:51 +08:00
imClumsyPanda
7d4560e599 update local_doc_qa.py 2023-04-19 21:29:20 +08:00
imClumsyPanda
635aa2d2ac update webui.py 2023-04-18 22:31:55 +08:00
wangxinkai
5a0429b2af feat: add api for knowledge_based QA 2023-04-18 10:08:36 +08:00