### What problem does this PR solve?
The encryption should first perform base64 encoding and then encrypt, to
maintain consistency with the frontend
#1620
### Type of change
- [x] Refactoring
### What problem does this PR solve?
This PR solves the problem that when you delete selected chunks, the
chunks can be deleted but Chunk Number doesn't change. Now you delete
one chunk, the Chunk Number is reduced by one, delete two chunks, the
Chunk Number is reduced by two...
#900
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
Signed-off-by: seaver <zhudan187@qq.com>
### What problem does this PR solve?
#762
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
---------
Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
### What problem does this PR solve?
fix function create to slove the problem of creating a chunk and
increasing the chunk number by 2.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue) #900
---------
Signed-off-by: seaver <zhudan187@qq.com>
### What problem does this PR solve?
Aims to stop the process of parsing.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
#1353
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
---------
Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
### What problem does this PR solve?
#1467
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
---------
Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
#1432#1447
This PR adds support for the GROQ LLM (Large Language Model).
Groq is an AI solutions company delivering ultra-low latency inference
with the first-ever LPU™ Inference Engine. The Groq API enables
developers to integrate state-of-the-art LLMs, such as Llama-2 and
llama3-70b-8192, into low latency applications with the request limits
specified below. Learn more at [groq.com](https://groq.com/).
Supported Models
| ID | Requests per Minute | Requests per Day | Tokens per Minute |
|----------------------|---------------------|------------------|-------------------|
| gemma-7b-it | 30 | 14,400 | 15,000 |
| gemma2-9b-it | 30 | 14,400 | 15,000 |
| llama3-70b-8192 | 30 | 14,400 | 6,000 |
| llama3-8b-8192 | 30 | 14,400 | 30,000 |
| mixtral-8x7b-32768 | 30 | 14,400 | 5,000 |
---------
Co-authored-by: paresh0628 <paresh.tuvoc@gmail.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
### What problem does this PR solve?
support xinference rerank model
#1455
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
Make the document start parsing.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
#1036
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
### What problem does this PR solve?
#308
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
---------
Co-authored-by: KevinHuSh <kevinhu.sh@gmail.com>
### What problem does this PR solve?
During knowledge Q&A conversations, the user-defined rerank model's
top_k parameter was not used
#1395
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
### What problem does this PR solve?
feat: Integrates LLM Azure OpenAI #716
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### Other
It's just the back-end code, the front-end needs to provide the Azure
OpenAI model addition form.
#### Required parameters
- base_url
- api_key
---------
Co-authored-by: yonghui li <yonghui.li@bondex.com.cn>
### What problem does this PR solve?
Adds the API method of updating documents.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
Adds the api of listing documentation.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
Adds the functionality of deleting documentation
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
API: Adds the feature of uploading document.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
fix bug: [1159](https://github.com/infiniflow/ragflow/issues/1159)
using embd which user configured at knowledgebase when create new chunk
in database
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
### What problem does this PR solve?
Added get_dataset and update_dataset API.
Fixed delete_dataset.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
### What problem does this PR solve?
Added the documentation for api and fixed duplicate get_dataset()
methods.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
### What problem does this PR solve?
This PR have completed both HTTP API and Python SDK for
'delete_dataset". In addition, there are tests for it.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
This PR have completed both HTTP API and Python SDK for 'list_dataset".
In addition, there are tests for it.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
This PR have finished 'create dataset' of both HTTP API and Python SDK.
HTTP API:
```
curl --request POST --url http://<HOST_ADDRESS>/api/v1/dataset --header 'Content-Type: application/json' --header 'Authorization: <ACCESS_KEY>' --data-binary '{
"name": "<DATASET_NAME>"
}'
```
Python SDK:
```
from ragflow.ragflow import RAGFLow
ragflow = RAGFLow('<ACCESS_KEY>', 'http://127.0.0.1:9380')
ragflow.create_dataset("dataset1")
```
TODO:
- ACCESS_KEY is the login_token when user login RAGFlow, currently.
RAGFlow should have the function that user can add/delete access_key.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
---------
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
### What problem does this PR solve?
Knowledge base management supports crawling information from web pages
and generating PDF documents
### Type of change
- [x] New Feature (Support document from web pages)
