2894 Commits

Author SHA1 Message Date
Stephen Hu
65537b8200
Fix:Set CUDA_VISIBLE_DEVICES In DefaultEmbedding (#7465)
### What problem does this PR solve?
https://github.com/infiniflow/ragflow/issues/7420

### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2025-05-06 14:38:36 +08:00
Raffaele Mancuso
60787f8d5d
Fix Ollama instructions (#7478)
Fix instructions for Ollama

### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2025-05-06 13:57:39 +08:00
Raffaele Mancuso
c4b3d3af95
Fix instructions for Ollama (#7468)
1. Use `host.docker.internal` as base URL
2. Fix numbers in list
3. Make clear what is the console input and what is the output

### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2025-05-06 09:47:19 +08:00
Yongteng Lei
f29a5de9f5
Fix: filed_map was incorrectly persisted (#7443)
### What problem does this PR solve?

Fix `filed_map` was incorrectly persisted. #7412 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-05-06 09:44:38 +08:00
balibabu
cb37f00a8f
Feat: Modify the style of the dataset page #3221 (#7446)
### What problem does this PR solve?

Feat:  Modify the style of the dataset page #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-05-02 21:27:21 +08:00
liu an
fc379e90d1
Fix: change create dataset htto api delimiter default value to r'\n' (#7434)
### What problem does this PR solve?

change create dataset delimiter default value to r'\n'

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-04-30 17:43:42 +08:00
balibabu
fea9d970ec
Feat: Modify the dataset list page style #3221 (#7437)
### What problem does this PR solve?

Feat: Modify the dataset list page style #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-04-30 15:37:16 +08:00
Song Fuchang
6e7dd54a50
Feat: Support passing knowledge base id as variable in retrieval component (#7088)
### What problem does this PR solve?

Fix #6600

Hello, I have the same business requirement as #6600. My use case is: 

We have many departments (> 20 now and increasing), and each department
has its own knowledge base. Because the agent workflow is the same, so I
want to change the knowledge base on the fly, instead of creating agents
for every department.

It now looks like this:


![屏幕截图_20250416_212622](https://github.com/user-attachments/assets/5cb3dade-d4fb-4591-ade3-4b9c54387911)

Knowledge bases can be selected from the dropdown, and passed through
the variables in the table. All selected knowledge bases are used for
retrieval.

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2025-04-30 15:32:14 +08:00
writinwaters
f56b651acb
Built-in reranker models have been removed from official deliveries. (#7439)
### What problem does this PR solve?

### Type of change


- [x] Documentation Update
2025-04-30 15:28:03 +08:00
Stephen Hu
2dbcc0a1bf
Fix: Tried to fix the fid mis match under some cases (#7426)
### What problem does this PR solve?

https://github.com/infiniflow/ragflow/issues/7407

Based on this context, I think there should be some reasons that let
some LLMs have a mismatch (add the wrong "@xxx"),
So I think when use fid can not fetch llm then tried to just use name
should can fetch it.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-04-30 14:55:21 +08:00
liu an
1f82889001
Fix: create dataset remove unnecessary parameter constraints (#7432)
### What problem does this PR solve?

Remove unnecessary parameter restrictions in dataset creation API

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-04-30 14:50:23 +08:00
liu an
e6c824e606
Test: Update tests to use new fixture instead of deprecated one (#7431)
### What problem does this PR solve?

Deprecate get_dataset_id_and_document_id fixture, use add_document
instead

### Type of change

- [x] Update test cases
2025-04-30 14:49:26 +08:00
so95
e2b0bceb1b
Feat: filler list by user change input (#7389)
### What problem does this PR solve?

filler list by user change input

![Recording2025-04-28163440-ezgif
com-video-to-gif-converter](https://github.com/user-attachments/assets/6ff2cfea-dea9-4293-b9a6-b4c61ab9a549)

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-04-30 14:48:41 +08:00
writinwaters
713c055e04
DOC: Added a UI tip for document parsing (#7430)
### What problem does this PR solve?

### Type of change

- [x] Documentation Update
2025-04-30 13:10:13 +08:00
balibabu
1fc52033ba
Feat: Using IconFont as an additional icon library #3221 (#7427)
### What problem does this PR solve?
Feat: Using IconFont as an additional icon library #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-04-30 13:09:42 +08:00
alkscr
ab27609a64
Fix: whole knowledge graph lost after removing any document in the knowledge base (#7151)
### What problem does this PR solve?

When you removed any document in a knowledge base using knowledge graph,
the graph's `removed_kwd` is set to "Y".
However, in the function `graphrag.utils.get_gaph`, `rebuild_graph`
method is passed and directly return `None` while `removed_kwd=Y`,
making residual part of the graph abandoned (but old entity data still
exist in db).

Besides, infinity instance actually pass deleting graph components'
`source_id` when removing document. It may cause wrong graph after
rebuild.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-04-30 09:43:17 +08:00
balibabu
538a408608
Feat: Modify background color of Card #3221 (#7421)
### What problem does this PR solve?

Feat: Modify background color of Card #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-04-30 09:12:28 +08:00
Yongteng Lei
093d280528
Feat: add Qwen3 and OpenAI o series (#7415)
### What problem does this PR solve?

Qwen3 and more LLMs.

Close #7296

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-04-29 18:26:29 +08:00
balibabu
de166d0ff2
Feat: Add a language switch drop-down box to the top navigation bar #3221 (#7416)
### What problem does this PR solve?

Feat: Add a language switch drop-down box to the top navigation bar
#3221
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-04-29 18:20:46 +08:00
Stephen Hu
942b94fc3c
feat: dataset filter by parsing status (#7404)
### What problem does this PR solve?

https://github.com/infiniflow/ragflow/issues/5931

### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2025-04-29 17:29:58 +08:00
balibabu
77bb7750e9
Feat: Modify the segmented component style #3221 (#7409)
### What problem does this PR solve?

Feat: Modify the segmented component style #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-04-29 17:05:23 +08:00
liu an
78380fa181
Refa: http API create dataset and test cases (#7393)
### What problem does this PR solve?

This PR introduces Pydantic-based validation for the create dataset HTTP
API, improving code clarity and robustness. Key changes include:
1. Pydantic Validation
2. ​​Error Handling
3. Test Updates
4. Documentation

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Documentation Update
- [x] Refactoring
2025-04-29 16:53:57 +08:00
Stephen Hu
c88e4b3fc0
Fix: improve recover_pending_tasks timeout (#7408)
### What problem does this PR solve?

Fix the redis lock will always timeout (change the logic order release
lock first)

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-04-29 16:50:39 +08:00
balibabu
552475dd5c
Feat: Adjust the style of the home page #3221 (#7405)
### What problem does this PR solve?

Feat: Adjust the style of the home page #3321

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-04-29 15:32:50 +08:00
so95
c69fbca24f
fixed missing list input ref in query (#7375)
### What problem does this PR solve?

fixed missing list input ref in query

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-04-29 13:03:36 +08:00
balibabu
5bb1c383ac
Feat: Bind data to the agent module of the home page #3221 (#7385)
### What problem does this PR solve?

Feat: Bind data to the agent module of the home page #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-04-29 09:50:54 +08:00
Kevin Hu
c7310f7fb2
Refa: similarity calculations. (#7381)
### What problem does this PR solve?


### Type of change

- [x] Refactoring
2025-04-28 19:17:11 +08:00
Chaoxi Weng
3a43043c8a
Feat: Add support for OAuth2 and OpenID Connect (OIDC) authentication (#7379)
### What problem does this PR solve?

Add support for OAuth2 and OpenID Connect (OIDC) authentication,
allowing OAuth/OIDC authentication using the specified routes:
- `/login/<channel>`: Initiates the OAuth flow for the specified channel
- `/oauth/callback/<channel>`: Handles the OAuth callback after
successful authentication

The callback URL should be configured in your OAuth provider as:
```
https://your-app.com/oauth/callback/<channel>
```

For detailed instructions on configuring **service_conf.yaml.template**,
see: `./api/apps/auth/README.md#usage`.

- Related issues
#3495  

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
2025-04-28 16:15:52 +08:00
writinwaters
dbfa859ca3
Knowledge graph no longer exists as a chunking method (#7382)
### What problem does this PR solve?

### Type of change

- [x] Documentation Update
2025-04-28 15:58:20 +08:00
Qi
53c59c47a1
Fix:Update chat assistant with an empty dataset (#7354)
### What problem does this PR solve?

When updating a chat assistant using API,if the dataset attached by the
current chat assistant is not empty,setting dataset to
null("dataset_ids":[]) will cause update failure:'dataset_ids' can't be
empty

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-04-28 15:19:21 +08:00
balibabu
af393b0003
Feat: Add AsyncTreeSelect component #3221 (#7377)
### What problem does this PR solve?

Feat: Add AsyncTreeSelect component #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-04-28 14:58:33 +08:00
Stephen Hu
1a5608d0f8
Fix: Add title_tks for Pictures (#7365)
### What problem does this PR solve?
https://github.com/infiniflow/ragflow/issues/7362

append title_tks
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2025-04-28 13:35:34 +08:00
Neal Davis
23dcbc94ef
feat: replace models of novita (#7360)
### What problem does this PR solve?

Replace models of novita

### Type of change

- [x] Other (please describe): Replace models of novita
2025-04-28 13:35:09 +08:00
liuzhenghua
af770c5ced
perf: Optimize GraphRAG’s LOOP_PROMPT (#7356)
### What problem does this PR solve?

当前graphrag的LOOP_PROMPT,会导致模型输出Y之后,继续补充了实体和关系,比较浪费时间。参照[graph
rag](https://github.com/microsoft/graphrag/blob/main/graphrag/prompts/index/extract_graph.py)最新的代码,修改了LOOP_PROMPT,经过验证,修改后可以稳定的输出Y停止。

Currently, GraphRAG’s LOOP_PROMPT causes the model to keep appending
entities and relationships even after outputting “Y,” which wastes time.
Referring to the latest code in
[graphRAG](https://github.com/microsoft/graphrag/blob/main/graphrag/prompts/index/extract_graph.py),
I modified the LOOP_PROMPT, and after verification the updated prompt
reliably outputs “Y” and stops.

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [x] Performance Improvement
- [ ] Other (please describe):

Co-authored-by: liuzhenghua-jk <liuzhenghua-jk@360shuke.com>
2025-04-28 13:31:04 +08:00
balibabu
8ce5e69b2f
Feat: Preview the file #3221 (#7355)
### What problem does this PR solve?

Feat: Preview the file #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-04-27 18:50:24 +08:00
yiminghub2024
1aa97600df
Fix mcp server startup failure (#7329)
### What problem does this PR solve?
0.18.0 mcp server can not start with upgrade from 0.17.2 or new install
except rebuild all docker

Close #7321

mcp server can not start auto from docker :
2025-04-25 17:30:44,512 INFO 25 task_executor_2a9f3e2de99a_0 reported
heartbeat: {"name": "task_executor_2a9f3e2de99a_0", "now":
"2025-04-25T17:30:44.509+08:00", "boot_at":
"2025-04-25T16:43:33.038+08:00", "pending": 0, "lag": 0, "done": 0,
"failed": 0, "current": {}}
usage: server.py [-h] [--base_url BASE_URL] [--host HOST] [--port PORT]
                 [--mode MODE] [--api_key API_KEY]
server.py: error: unrecognized arguments:

problem:
server.py in docker start arguments not correct , so mcp server start
fail
reason:
```
1. docker-copose.yaml
     example  - --mcp-host-api-key="ragflow-12345678" is wrong.  do not add "" to key or it says:"api-key wrong"
2.docker file  entrypoint.sh  can not translate config to exec command , we need mapping file from host to docker
     - ./entrypoint.sh:/ragflow/entrypoint.sh
3.just add one code raw fix all probelm 
```
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Documentation Update
- [x] Performance Improvement

---------

Co-authored-by: Yongteng Lei <yongtengrey@outlook.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-04-27 18:47:12 +08:00
xiaosl-cell
969c596d4c
Fix: tenant_id spelling error. (#7331)
### What problem does this PR solve?

In the generate_confirmation_token method, a spelling error was found
with 'tenent_id'. The correct spelling should be 'tenant_id'.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

Co-authored-by: shengliang xiao <shengliangxiao2024@gmail.com>
2025-04-27 17:34:13 +08:00
Jason Li
67b087019c
Update Groq AI Model Config (#7335)
With current config will get error "Fail to access model(gemma-7b-it)
using this api key"
Since the model has been removed, according to Groq official document:
https://console.groq.com/docs/models

### Type of change

- [ x] Bug Fix (non-breaking change which fixes an issue)
2025-04-27 17:05:25 +08:00
balibabu
6a45d93005
Feat: Batch operations on documents in a dataset #3221 (#7352)
### What problem does this PR solve?

Feat: Batch operations on documents in a dataset #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-04-27 17:00:41 +08:00
writinwaters
43e507d554
Updated RAPTOR-specific UI (#7348)
### What problem does this PR solve?

### Type of change

- [x] Documentation Update
2025-04-27 16:49:14 +08:00
Yongteng Lei
a4be6c50cf
[BREAKING CHANGE] GET to POST: enhance document list capability (#7349)
### What problem does this PR solve?

Enhance capability of `list_docs`.

Breaking change: change method from `GET` to `POST`.

### Type of change

- [x] Refactoring
- [x] Enhancement with breaking change
2025-04-27 16:48:27 +08:00
balibabu
5043143bc5
Feat: Create empty document. #3221 (#7343)
### What problem does this PR solve?

Feat: Create empty document. #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-04-27 16:12:10 +08:00
balibabu
bdebd1b2e3
Feat: Filter document by running status and file type. #3221 (#7340)
### What problem does this PR solve?
Feat: Filter document by running status and file type. #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-04-27 14:39:05 +08:00
writinwaters
dadd8d9f94
DOC: Miscellaneous UI and editorial updates (#7324)
### What problem does this PR solve?



### Type of change


- [x] Documentation Update
2025-04-27 11:44:08 +08:00
Stephen Hu
3da8776a3c
Fix: Creating Knowledge Base Support Enter Key (#7258)
### What problem does this PR solve?


[https://github.com/infiniflow/ragflow/issues/7180](https://github.com/infiniflow/ragflow/issues/7180)
When creating a knowledge base, support the enter key
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-04-25 18:53:52 +08:00
balibabu
3052006ba8
Feat: Save document metadata #3221 (#7323)
### What problem does this PR solve?

Feat: Save document metadata #3221
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-04-25 18:38:15 +08:00
Stephen Hu
1662c7eda3
Feat: Markdown add image (#7124)
### What problem does this PR solve?

https://github.com/infiniflow/ragflow/issues/6984

1. Markdown parser supports get pictures
2. For Native, when handling Markdown, it will handle images
3. improve merge and 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-04-25 18:35:28 +08:00
balibabu
fef44a71c5
Feat: Save the configuration information of the knowledge base document #3221 (#7317)
### What problem does this PR solve?

Feat: Save the configuration information of the knowledge base document
#3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-04-25 17:31:28 +08:00
Kevin Hu
b271cc34b3
Fix: LLM generated tag issue. (#7301)
### What problem does this PR solve?
#7298

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-04-25 14:38:34 +08:00
alulala
eead838353
Fix pymysql interface error (#7295)
### What problem does this PR solve?

According to the
[[Rucongzhang](https://github.com/Rucongzhang)](https://github.com/infiniflow/ragflow/pull/7057#issuecomment-2827410047)
I added DB reconnection strategy in function `update_by_id`
2025-04-25 13:29:47 +08:00