185 Commits

Author SHA1 Message Date
H
6144a109ab
Add Support for AWS Bedrock (#1408)
### What problem does this PR solve?

#308 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: KevinHuSh <kevinhu.sh@gmail.com>
2024-07-08 09:37:34 +08:00
KevinHuSh
b3ebc66b13
be more specific for error message (#1409)
### What problem does this PR solve?

#918 

### Type of change

- [x] Refactoring
2024-07-08 09:32:44 +08:00
zhuhao
dcb3fb2073
fix: use user-defined rerank model's top_k parameter when knowledge Q&A conversation (#1396)
### What problem does this PR solve?

During knowledge Q&A conversations, the user-defined rerank model's
top_k parameter was not used

#1395 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-07-08 09:25:49 +08:00
cecilia-uu
306108fe0e
API: Download doc api (#1354)
### What problem does this PR solve?

Adds download_document api

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-07-04 16:33:55 +08:00
KevinHuSh
258c9ea644
add keyword extraction in graph (#1373)
### What problem does this PR solve?
#918 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-07-04 15:57:25 +08:00
KevinHuSh
4122695a1a
refine templates of graph (#1368)
### What problem does this PR solve?

#918 
### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-07-04 10:33:49 +08:00
LiYongHui
a6765e9ca4
Integrates LLM Azure OpenAI (#1318)
### What problem does this PR solve?

feat: Integrates LLM Azure OpenAI #716 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### Other
It's just the back-end code, the front-end needs to provide the Azure
OpenAI model addition form.
   
#### Required parameters

- base_url
- api_key

---------

Co-authored-by: yonghui li <yonghui.li@bondex.com.cn>
2024-07-04 09:57:16 +08:00
cecilia-uu
1defc83506
API: create update_doc method (#1341)
### What problem does this PR solve?

Adds the API method of updating documents.


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-07-03 15:14:34 +08:00
KevinHuSh
7c9ea5cad9
add interpreter to graph (#1347)
### What problem does this PR solve?

#918 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-07-03 12:15:15 +08:00
cecilia-uu
b5389f487c
API: created list_doc (#1327)
### What problem does this PR solve?

Adds the api of listing documentation.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-07-01 18:15:00 +08:00
KevinHuSh
92e9320657
upgrade laws parser of docx (#1332)
### What problem does this PR solve?


### Type of change

- [x] Refactoring
2024-07-01 15:50:24 +08:00
cecilia-uu
8c9b54db31
API: completed delete_doc api (#1290)
### What problem does this PR solve?

Adds the functionality of deleting documentation

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-06-28 14:27:57 +08:00
cecilia-uu
f6ae8fcb71
API: upload document api (#1264)
### What problem does this PR solve?

API: Adds the feature of uploading document.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-06-25 12:16:28 +08:00
Wang Baoling
e99e8b93fb
fix:failed to Create new Chunk in database #1159 (#1214)
### What problem does this PR solve?

fix bug: [1159](https://github.com/infiniflow/ragflow/issues/1159)
using embd which user configured at knowledgebase when create new chunk
in database

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-06-20 09:43:39 +08:00
cecilia-uu
5ec19b5f53
created get_dataset, update_dataset API and fixed: delete (#1201)
### What problem does this PR solve?

Added get_dataset and update_dataset API.
Fixed delete_dataset.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
2024-06-19 18:01:38 +08:00
Wang Baoling
0b90aab22c
fix: using embd which user configured at knowledgebase (#1163)
### What problem does this PR solve?
as title
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-06-19 14:44:25 +08:00
cecilia-uu
8d667d5abd
fixed: duplicate name (#1202)
### What problem does this PR solve?

Duplicate method name.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-06-18 16:28:46 +08:00
cecilia-uu
d9a9be4b4c
added documentation for api and fixed: duplicate get_dataset() (#1190)
### What problem does this PR solve?

Added the documentation for api and fixed duplicate get_dataset()
methods.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
2024-06-17 17:54:06 +08:00
cecilia-uu
6be3626372
delete_dataset method and tests created (#1186)
### What problem does this PR solve?

This PR have completed both HTTP API and Python SDK for
'delete_dataset". In addition, there are tests for it.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-06-17 15:10:05 +08:00
cecilia-uu
1eb4caf02a
create list_dataset api and tests (#1138)
### What problem does this PR solve?

This PR have completed both HTTP API and Python SDK for 'list_dataset".
In addition, there are tests for it.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-06-17 12:19:05 +08:00
KevinHuSh
7dc39cbfa6
add support for mistral (#1153)
### What problem does this PR solve?

#433 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-06-14 11:32:58 +08:00
KevinHuSh
a25d32496c
support graph (#1152)
### What problem does this PR solve?

#918 
### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-06-14 10:49:36 +08:00
KevinHuSh
2023fdc13e
fix file preview in file management (#1151)
### What problem does this PR solve?

fix file preview in file management

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-06-14 10:33:59 +08:00
KevinHuSh
abcd3d2469
refactor (#1124)
### What problem does this PR solve?


### Type of change

- [x] Refactoring
2024-06-12 11:02:15 +08:00
Jin Hai
cf2f6592dd
API: create dataset (#1106)
### What problem does this PR solve?

This PR have finished 'create dataset' of both HTTP API and Python SDK.
HTTP API:
```
curl --request POST --url http://<HOST_ADDRESS>/api/v1/dataset   --header 'Content-Type: application/json' --header 'Authorization: <ACCESS_KEY>' --data-binary '{
  "name": "<DATASET_NAME>"
}'
```

Python SDK:
```
from ragflow.ragflow import RAGFLow
ragflow = RAGFLow('<ACCESS_KEY>', 'http://127.0.0.1:9380')
ragflow.create_dataset("dataset1")

```

TODO: 
- ACCESS_KEY is the login_token when user login RAGFlow, currently.
RAGFlow should have the function that user can add/delete access_key.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2024-06-11 11:16:37 +08:00
Fakai Zhao
7eb69fe6d9
Supports obtaining PDF documents from web pages (#1107)
### What problem does this PR solve?

Knowledge base management supports crawling information from web pages
and generating PDF documents

### Type of change
- [x] New Feature (Support document from web pages)
2024-06-11 10:45:19 +08:00
Wang Baoling
d0951ee27b
fix: logger formater is not work (#1090)
### What problem does this PR solve?

as title

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-06-07 13:48:56 +08:00
KevinHuSh
923c3b8cac
fix bug in api (#1088)
### What problem does this PR solve?

#1075 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-06-07 09:56:14 +08:00
KevinHuSh
d9dc183a0e
rm wrongly uploaded py (#1073)
### What problem does this PR solve?


### Type of change


- [x] Refactoring
2024-06-06 13:49:48 +08:00
KevinHuSh
4454ba7a1e
add self-rag (#1070)
### What problem does this PR solve?

#1069 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-06-06 11:13:39 +08:00
balibabu
9517a27844
fix: fixed the problem that the api will be called directly after selecting the chat assistant picture (#1034)
### What problem does this PR solve?

fix: fixed the problem that the api will be called directly after
selecting the chat assistant picture #1033

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-06-03 13:42:56 +08:00
KevinHuSh
cc064040a2
refine API request data processing (#1031)
### What problem does this PR solve?

#1024 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-06-03 09:02:25 +08:00
KevinHuSh
1de31ca9f6
refine file select code (#1021)
### What problem does this PR solve?

#1015 

### Type of change

- [x] Refactoring
2024-05-31 19:44:33 +08:00
Zhedong Cen
4ec845c0a6
Add API for moving files (#1016)
### What problem does this PR solve?

Add backend API support for moving files into other directory

### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2024-05-31 18:11:25 +08:00
Wang Baoling
c58a1c48eb
Fix: bug #991 (#1013)
### What problem does this PR solve?

issue #991

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: KevinHuSh <kevinhu.sh@gmail.com>
2024-05-31 18:03:47 +08:00
cecilia-uu
260c68f60c
Adding the Minimax model (#1009)
### What problem does this PR solve?

Added support for MiniMax LLM

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: cecilia-uu <konghui1996@163.com>
2024-05-31 16:38:53 +08:00
KevinHuSh
5d2f7136dd
fix chunk modification bug (#1011)
### What problem does this PR solve?

As title.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-31 15:45:11 +08:00
KevinHuSh
83803a72ee
fix ollama bug (#999)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-30 18:03:36 +08:00
KevinHuSh
f077b57f8b
set ollama keep_alive (#985)
### What problem does this PR solve?

#980 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-30 11:27:58 +08:00
KevinHuSh
0171082cc5
fix create dialog bug (#982)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-30 09:25:05 +08:00
Zhedong Cen
8dd45459be
Add support for HTML file (#973)
### What problem does this PR solve?

Add support for HTML file

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-30 09:12:55 +08:00
KevinHuSh
758eb03ccb
fix jina adding issure and term weight refinement (#974)
### What problem does this PR solve?

#724 #162

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2024-05-29 19:38:57 +08:00
KevinHuSh
614defec21
add rerank model (#969)
### What problem does this PR solve?

feat: add rerank models to the project #724 #162

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-29 16:50:02 +08:00
KevinHuSh
daa4799385
limit the system context length of conversation messages. (#962)
### What problem does this PR solve?

#951 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-29 10:40:07 +08:00
yungongzi
9ffd7ae321
Added support for Baichuan LLM (#934)
### What problem does this PR solve?

- Added support for Baichuan LLM

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: 海贼宅 <stu_xyx@163.com>
2024-05-28 09:09:37 +08:00
GYH
7d8e03ec38
Update docnm_kwd to doc_name (#939)
### What problem does this PR solve?

Update docnm_kwd to doc_name 
#908 

### Type of change


- [x] Refactoring
2024-05-27 19:14:04 +08:00
KevinHuSh
46454362d7
fix raptor bugs (#928)
### What problem does this PR solve?

#922 
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-27 11:01:20 +08:00
dashi6174
20b57144b0
syntax error (#924)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-27 08:20:32 +08:00
KevinHuSh
6f99bbbb08
add raptor (#899)
### What problem does this PR solve?

#882 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-23 14:31:16 +08:00
yungongzi
eb51ad73d6
Add support for VolcEngine - the current version supports SDK2 (#885)
- The main idea is to assemble **ak**, **sk**, and **ep_id** into a
dictionary and store it in the database **api_key** field
- I don’t know much about the front-end, so I learned from Ollama, which
may be redundant.

### Configuration method

- model name

- Format requirements: {"VolcEngine model name":"endpoint_id"}
    - For example: {"Skylark-pro-32K":"ep-xxxxxxxxx"}
    
- Volcano ACCESS_KEY
- Format requirements: VOLC_ACCESSKEY of the volcano engine
corresponding to the model

- Volcano SECRET_KEY
- Format requirements: VOLC_SECRETKEY of the volcano engine
corresponding to the model
    
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-23 11:15:29 +08:00