681 Commits

Author SHA1 Message Date
H
61557a101a
fix botocore (#1414)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-07-08 16:20:19 +08:00
H
6144a109ab
Add Support for AWS Bedrock (#1408)
### What problem does this PR solve?

#308 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: KevinHuSh <kevinhu.sh@gmail.com>
2024-07-08 09:37:34 +08:00
KevinHuSh
258c9ea644
add keyword extraction in graph (#1373)
### What problem does this PR solve?
#918 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-07-04 15:57:25 +08:00
KevinHuSh
4122695a1a
refine templates of graph (#1368)
### What problem does this PR solve?

#918 
### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-07-04 10:33:49 +08:00
LiYongHui
a6765e9ca4
Integrates LLM Azure OpenAI (#1318)
### What problem does this PR solve?

feat: Integrates LLM Azure OpenAI #716 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### Other
It's just the back-end code, the front-end needs to provide the Azure
OpenAI model addition form.
   
#### Required parameters

- base_url
- api_key

---------

Co-authored-by: yonghui li <yonghui.li@bondex.com.cn>
2024-07-04 09:57:16 +08:00
KevinHuSh
7c9ea5cad9
add interpreter to graph (#1347)
### What problem does this PR solve?

#918 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-07-03 12:15:15 +08:00
KevinHuSh
92e9320657
upgrade laws parser of docx (#1332)
### What problem does this PR solve?


### Type of change

- [x] Refactoring
2024-07-01 15:50:24 +08:00
Zhedong Cen
fc7cc1d36c
Optimize docx handle method in laws parser (#1302)
### What problem does this PR solve?

Optimize docx handle method in laws parser

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-06-28 17:42:59 +08:00
KevinHuSh
0ce720a247
fix mem leak for local reranker (#1295)
### What problem does this PR solve?

#1288
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-06-27 14:57:24 +08:00
zhuhao
47926a95ae
Fix ragflow may encounter an OOM (Out Of Memory) when there are a lot of conversations (#1292)
### What problem does this PR solve?

Fix ragflow may encounter an OOM (Out Of Memory) when there are a lot of
conversations.
#1288

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: zhuhao <zhuhao@linklogis.com>
2024-06-27 14:48:49 +08:00
Zhedong Cen
a95c1d45f0
Support table for markdown file in general parser (#1278)
### What problem does this PR solve?

Support extracting table for markdown file in general parser

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-06-27 14:38:35 +08:00
Zhedong Cen
b75bb1d8d3
Support displaying tables in the chunks of pdf file when using QA parser (#1263)
### What problem does this PR solve?

Support displaying tables in the chunks of pdf file when using QA parser

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-06-24 19:02:18 +08:00
Zhedong Cen
38bd02f402
Support displaying images in the chunks of docx files when using general parser (#1253)
### What problem does this PR solve?

Support displaying images in chunks of docx files when using general
parser

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-06-24 16:29:36 +08:00
GYH
4fcd05ad23
fix Rerank Vector Similarity Score (#1249)
### What problem does this PR solve?

#1243 
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-06-24 12:25:50 +08:00
Zhedong Cen
f8fe4154e8
Place pdf's image at the correct position in QA parser (#1235)
### What problem does this PR solve?

Place pdf's image at the correct position in QA parser

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-06-24 10:41:03 +08:00
KevinHuSh
15bf9f8c25
refine code to prevent exception (#1231)
### What problem does this PR solve?


### Type of change

- [x] Refactoring
2024-06-21 14:06:46 +08:00
Wang Baoling
18f4a6b35c
feat: support json file (#1217)
### What problem does this PR solve?

feat: support json file.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: KevinHuSh <kevinhu.sh@gmail.com>
2024-06-21 10:42:29 +08:00
Zhedong Cen
3c1444ab19
Add docx support for manual parser (#1227)
### What problem does this PR solve?

Add docx support for manual parser

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-06-20 17:03:02 +08:00
Zhedong Cen
fb56a29478
Add docx support for QA parser (#1213)
### What problem does this PR solve?

Add docx support for QA parser

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-06-20 16:09:09 +08:00
KevinHuSh
e35f7610e7
fix too long query exception (#1195)
### What problem does this PR solve?

#1161 
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-06-18 09:50:59 +08:00
Zhedong Cen
7920a5c78d
Add markdown support for QA parser (#1180)
### What problem does this PR solve?

Add markdown support for QA parser

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-06-18 09:45:13 +08:00
yungongzi
c68767acdd
Fix VolcEngine BUG (#1165)
### What problem does this PR solve?
- Fix a bug for VolcEngine
- After testing, the current VolcEngine configuration also supports the
Doubao series
_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: 海贼宅 <stu_xyx@163.com>
2024-06-14 19:49:28 +08:00
Zhedong Cen
90975460af
Add pdf support for QA parser (#1155)
### What problem does this PR solve?

Support extracting questions and answers from PDF files

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-06-14 15:12:39 +08:00
KevinHuSh
7dc39cbfa6
add support for mistral (#1153)
### What problem does this PR solve?

#433 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-06-14 11:32:58 +08:00
KevinHuSh
2023fdc13e
fix file preview in file management (#1151)
### What problem does this PR solve?

fix file preview in file management

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-06-14 10:33:59 +08:00
KevinHuSh
e05395d2a7
fix multi-modual bug (#1127)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-06-12 14:21:35 +08:00
KevinHuSh
abcd3d2469
refactor (#1124)
### What problem does this PR solve?


### Type of change

- [x] Refactoring
2024-06-12 11:02:15 +08:00
Michael Feil
68a698655a
infinity: Update embedding_model.py (#1109)
### What problem does this PR solve?

I implemented infinity, a fast vector embeddings engine. 

### Type of change


- [x] Performance Improvement
- [X] Other (please describe):
2024-06-11 08:23:58 +08:00
KevinHuSh
923c3b8cac
fix bug in api (#1088)
### What problem does this PR solve?

#1075 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-06-07 09:56:14 +08:00
Wang Baoling
722c342d56
fix: bug similarity() in YoudaoRerank (#1084)
### What problem does this PR solve?

bix fix

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-06-07 09:04:53 +08:00
CKLogic
195498daaa
feat: Support Password Access for ElasticSearch (#1072)
### What problem does this PR solve?

Using password authentication to access ElasticSearch is essential,
especially in a production environment.

This PR will enable password access support.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-06-06 13:19:26 +08:00
KevinHuSh
4454ba7a1e
add self-rag (#1070)
### What problem does this PR solve?

#1069 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-06-06 11:13:39 +08:00
KevinHuSh
b8eedbdd86
refine rerank (#1056)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-06-04 17:27:00 +08:00
KevinHuSh
cc064040a2
refine API request data processing (#1031)
### What problem does this PR solve?

#1024 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-06-03 09:02:25 +08:00
Wang Baoling
c58a1c48eb
Fix: bug #991 (#1013)
### What problem does this PR solve?

issue #991

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: KevinHuSh <kevinhu.sh@gmail.com>
2024-05-31 18:03:47 +08:00
cecilia-uu
260c68f60c
Adding the Minimax model (#1009)
### What problem does this PR solve?

Added support for MiniMax LLM

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: cecilia-uu <konghui1996@163.com>
2024-05-31 16:38:53 +08:00
GYH
b85c15cc96
Add file rag/svr/discord_svr.py (#1008)
### What problem does this PR solve?

File rag/svr/discord_svr.py is for discord bot.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-31 13:47:15 +08:00
Jin Hai
9ed0e50f6b
Update info (#1005)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2024-05-31 09:53:04 +08:00
KevinHuSh
b9bb11879f
fix #994 (#1006)
### What problem does this PR solve?

#994 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-31 09:46:22 +08:00
KevinHuSh
dc7afe46fb
fix bug 994 ,991 (#1004)
### What problem does this PR solve?

#994 
#991 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-31 09:24:24 +08:00
KevinHuSh
117a173fff
fix tk_count undefine issue (#996)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-30 16:18:15 +08:00
KevinHuSh
77363a0875
fix bge rerank normalize issue (#988)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-30 12:55:17 +08:00
KevinHuSh
f077b57f8b
set ollama keep_alive (#985)
### What problem does this PR solve?

#980 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-30 11:27:58 +08:00
KevinHuSh
0171082cc5
fix create dialog bug (#982)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-30 09:25:05 +08:00
Zhedong Cen
8dd45459be
Add support for HTML file (#973)
### What problem does this PR solve?

Add support for HTML file

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-30 09:12:55 +08:00
CKLogic
dded365b8d
Fix:After being idle for a while, new tasks need to be cancel and redo (#958)
### What problem does this PR solve?

After being idle for a while (When Redis Queue exceeds the
SVR_QUEUE_RETENTION(60*60) expiration time), new tasks need to be cancel
and redo.

When use xgroup_create to create a consumer group, set the ID to "$",
meaning that only messages added to the stream after the group is
created will be visible to new consumers. If the application scenario
requires processing messages that already exist in the queue, you might
need to change this ID to "0", so that the new consumer group can read
all messages from the beginning.


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-30 09:03:11 +08:00
KevinHuSh
758eb03ccb
fix jina adding issure and term weight refinement (#974)
### What problem does this PR solve?

#724 #162

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2024-05-29 19:38:57 +08:00
KevinHuSh
614defec21
add rerank model (#969)
### What problem does this PR solve?

feat: add rerank models to the project #724 #162

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-29 16:50:02 +08:00
KevinHuSh
7eee193956
fix #917 #915 (#946)
### What problem does this PR solve?

#917 
#915

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-28 11:13:02 +08:00
yungongzi
9ffd7ae321
Added support for Baichuan LLM (#934)
### What problem does this PR solve?

- Added support for Baichuan LLM

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: 海贼宅 <stu_xyx@163.com>
2024-05-28 09:09:37 +08:00