321 Commits

Author SHA1 Message Date
Yongteng Lei
d7774cf049
Fix: fix document concurrent upload issue (#6095)
### What problem does this PR solve?

Resolve document concurrent upload issue. #6039 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-14 16:31:44 +08:00
Kevin Hu
b77e844fc3
Fix: none parse_config updating. (#6092)
### What problem does this PR solve?

#6081

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-14 16:06:16 +08:00
Kevin Hu
42eb99554f
Feat: add token comsumption & speed to little lamp. (#6077)
### What problem does this PR solve?

#6059

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-03-14 13:37:31 +08:00
Kevin Hu
7463241896
Fix: empty doc id validation. (#6064)
### What problem does this PR solve?

#6031

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-14 11:45:44 +08:00
utopia2077
2d4a60cae6
Fix: Reduce excessive IO operations by loading LLM factory configurations (#6047)
…ions

### What problem does this PR solve?

This PR fixes an issue where the application was repeatedly reading the
llm_factories.json file from disk in multiple places, which could lead
to "Too many open files" errors under high load conditions. The fix
centralizes the file reading operation in the settings.py module and
stores the data in a global variable that can be accessed by other
modules.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [x] Performance Improvement
- [ ] Other (please describe):
2025-03-14 09:54:38 +08:00
Xinghan Pan
47926f7d21
Improve API Documentation, Standardize Error Handling, and Enhance Comments (#5990)
### What problem does this PR solve?  
- The API documentation lacks detailed error code explanations. Added
error code tables to `python_api_reference.md` and
`http_api_reference.md` to clarify possible error codes and their
meanings.
- Error handling in the codebase is inconsistent. Standardized error
handling logic in `sdk/python/ragflow_sdk/modules/chunk.py`.
- Improved API comments by adding standardized docstrings to enhance
code readability and maintainability.

### Type of change  
- [x] Documentation Update  
- [x] Refactoring
2025-03-13 19:06:50 +08:00
so95
3c43a7aee8
For an Agent with an Input Begin value, on the first call the return … (#5957)
…session_id does not exist in the session

For an Agent with an Input Begin value, on the first call the return
session_id does not exist in the session

### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-12 17:01:44 +08:00
liwenju0
e3ea4b7ec2
Fix: Add Knowledge Base Document Parsing Status Check (#5966)
When creating and updating chats, add a check for the parsing status of
knowledge base documents. Ensure that all documents have been parsed
before allowing chat creation to improve user experience and system
stability.

**Main Changes:**

- Add document parsing status check logic in `chat.py`.
- Implement the `is_parsed_done` method in `knowledgebase_service.py`.
- Prevent chat creation when documents are being parsed or parsing has
failed.

### What problem does this PR solve?

fix this bug:https://github.com/infiniflow/ragflow/issues/5960

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: wenju.li <wenju.li@deepctr.cn>
2025-03-12 16:07:45 +08:00
Kevin Hu
caecaa7562
Feat: apply LLM to optimize citations. (#5935)
### What problem does this PR solve?

#5905

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-03-11 19:56:21 +08:00
Kevin Hu
90d18143ba
Refa: add prompt to empty retrieved answwer. (#5892)
### What problem does this PR solve?

#5883

### Type of change

- [x] Refactoring
2025-03-11 13:11:14 +08:00
Kevin Hu
8e965040ce
Fix: rm <think> for ES sql generation. (#5881)
### What problem does this PR solve?

#5850

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-11 10:41:19 +08:00
donblack01
b1a46d5adc Fix:when start with source code not in docker env report 'UnicodeDec… (#5802)
### What problem does this PR solve?

fix:when start with  source code not in docker env report
"UnicodeDecodeError: 'gbk' codec can't decode byte 0xad in position 5:
illegal multibyte sequence" in windows

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: tangyu <1@1.com>
2025-03-10 11:22:06 +08:00
hy89
66938e0b68
Feat(api): Add dsl parameters to control whether dsl fields are included (#5769)
1. **Issue**: When calling `list_agent_session` via the HTTP API, users
may only need to display conversation messages, and do not want to see
the associated dsl, which can be very large. Therefore, consider adding
a control option to determine whether the DSL should be returned, with
the default being to return it.

2. **Documentation Discrepancy**: In the HTTP API documentation, under
"List agent sessions," the "Response" section states that the "data"
field is a dictionary when "success" is returned. However, the actual
returned data is a list. This discrepancy has been corrected.
2025-03-07 16:58:00 +08:00
Zanyatta
0e3e129a83
Fix: Resolve inconsistency in APIToken dialog_id field definition (#5749)
The `dialog_id` field was inconsistently defined:
- In the `migrate_db()` function, it was set to `null=True`.
- In the model class, it was defined as `null=False`.

This inconsistency caused an issue during the initial deployment where
the database table did not allow `dialog_id` to be null. As a result,
calling `APITokenService.save(**obj)` in `system_app.py` raised the
following error:

```
peewee.IntegrityError: null value in column "dialog_id" violates not-null constraint
```

### What problem does this PR solve?

Error: peewee.IntegrityError: null value in column "dialog_id" violates
not-null constraint

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-07 13:26:08 +08:00
Kevin Hu
9fc7174612
Fix: too long context during KG issue. (#5723)
### What problem does this PR solve?

#5088

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-06 19:21:07 +08:00
yihong
4326873af6
refactor: no need to inherit in python3 clean the code (#5659)
### What problem does this PR solve?

As title

### Type of change


- [x] Refactoring

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
2025-03-05 18:03:53 +08:00
Kevin Hu
ec68ab1c8c
Fix: search citation issue. (#5657)
### What problem does this PR solve?
#5649
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-05 17:25:47 +08:00
Zhichang Yu
f65c3ae62b
Refactored DocumentService.update_progress (#5642)
### What problem does this PR solve?

Refactored DocumentService.update_progress

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-05 14:48:03 +08:00
Yongteng Lei
f6dd2cd1af
Fix: fix may lose part of information of last stream chunck (#5584)
### What problem does this PR solve?

 Fix may lose part of information of last stream chunck

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-04 11:58:10 +08:00
Zhichang Yu
c813c1ff4c
Made task_executor async to speedup parsing (#5530)
### What problem does this PR solve?

Made task_executor async to speedup parsing

### Type of change

- [x] Performance Improvement
2025-03-03 18:59:49 +08:00
Kevin Hu
7a81fa00e9
Optimize prompt. (#5541)
### What problem does this PR solve?

#5526

### Type of change

- [x] Performance Improvement
2025-03-03 13:12:38 +08:00
Kevin Hu
606ed0c8ab
Fix: in case running KG repeatly. (#5538)
### What problem does this PR solve?

#5512

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-03 12:22:36 +08:00
Kevin Hu
5fdfb8d465
Fix: rm think if stream is Flase. (#5458)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-28 10:05:18 +08:00
Kevin Hu
fa76974e24
Fix issue of ask API. (#5400)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-26 19:45:22 +08:00
liwenju0
a9e4695b74
Fix:validate knowledge base association before document upload (#5373)
### What problem does this PR solve?

fix this bug: https://github.com/infiniflow/ragflow/issues/5368

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: wenju.li <wenju.li@deepctr.cn>
2025-02-26 15:47:34 +08:00
Kevin Hu
4f40f685d9
Code refactor (#5371)
### What problem does this PR solve?

#5173

### Type of change

- [x] Refactoring
2025-02-26 15:40:52 +08:00
Kevin Hu
4e2afcd3b8
Fix FlagRerank max_length issue. (#5366)
### What problem does this PR solve?

#5352

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-26 11:01:13 +08:00
Kevin Hu
53b9e7b52f
Add tavily as web searh tool. (#5349)
### What problem does this PR solve?

#5198

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-02-26 10:21:04 +08:00
Kevin Hu
b3d579e2c1
Refine prompt of agentic search. (#5312)
### What problem does this PR solve?

#5173

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-02-25 09:21:52 +08:00
Kevin Hu
3444cb15e3
Refine search query. (#5235)
### What problem does this PR solve?

#5173
#5214

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-21 18:32:32 +08:00
Kevin Hu
f5d63bb7df
Support chat solo. (#5218)
### What problem does this PR solve?

#5216

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-02-21 12:24:02 +08:00
Kevin Hu
7b3d700d5f
Apply agentic searching. (#5196)
### What problem does this PR solve?

#5173

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-02-20 17:41:01 +08:00
liwenju0
3ced290eb5
Feat: Add support for document meta fields update through api (#5120)
### What problem does this PR solve?

add support for update document meta data through  api
### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: wenju.li <wenju.li@deepctr.cn>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-02-19 13:39:31 +08:00
zhxlp
00c7ddbc9b
Fix: The max tokens defined by the tenant are not used (#4297) (#2817) (#5066)
### What problem does this PR solve?

Fix: The max tokens defined by the tenant are not used (#4297) (#2817)


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-02-18 13:42:22 +08:00
Kevin Hu
84b4b38cbb
Remove <think> for exeSql component. (#5069)
### What problem does this PR solve?

#5061
#5067

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-18 13:39:37 +08:00
kuschzzp
88daa349f9
Optimize conversation when uploading attachments (#4964)
### What problem does this PR solve?

#4929

### Type of change

- [x] Performance Improvement
2025-02-17 12:03:04 +08:00
Kevin Hu
810f997276
Fix <think> in keywords or question auto-generations. (#5021)
### What problem does this PR solve?

**#4983**

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-17 11:20:57 +08:00
Kevin Hu
849d9eb463
Ignore tenant not found error while increasing token usage. (#4950)
### What problem does this PR solve?

#4940

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-14 11:10:49 +08:00
Peterson Alves
042f4c90c6
Fixes KeyError: 'content' when using stream=False (#4944)
### 🛠 Fixes `KeyError: 'content'` when using `stream=False`

#### 🔍 Problem  
When calling the chat API with `stream=False`, the code attempts to
access `msg[-1]["content"]` without verifying if the key exists. This
causes a `KeyError` when the message structure does not contain
`"content"`.

This issue was discussed in
[#4885](https://github.com/infiniflow/ragflow/issues/4885), where we
analyzed the root cause. The error does not occur with `stream=True`, as
the response is processed differently.

####  Solution  
- **Logging Fix:**  
  - Before accessing `msg[-1]["content"]`, we check if the key exists.  
- If it does not exist, a default value (`"[content not available]"`) is
used to prevent errors.

- **Structural Fix in `msg` Construction:**  
- Ensured that every message in `msg` contains the `"content"` key, even
if empty.
- This fixes the issue at its root and ensures consistent behavior
between `stream=True` and `stream=False`.

#### 🔄 Impact  
- Prevents the `KeyError` without affecting normal application flow.  
- Ensures the integrity of the `msg` structure, avoiding future
failures.



### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-14 10:27:01 +08:00
Kevin Hu
6fa34d5532
Fix KG circle. (#4823)
### What problem does this PR solve?

#4760

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-10 11:02:29 +08:00
davidche
588207d7c1
optimize TenantLLMService.increase_usage for "can't update token usag… (#4755)
…e error " message

### What problem does this PR solve?

optimize TenantLLMService.increase_usage Performance

### Type of change

- [x] Performance Improvement

Co-authored-by: che_shuai <che_shuai@massclouds.com>
2025-02-07 12:16:17 +08:00
Kevin Hu
e9ccba0395
Add timestamp to messages (#4624)
### What problem does this PR solve?


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-01-24 11:07:55 +08:00
Kevin Hu
e14d6ae441
Refactor. (#4612)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
2025-01-23 18:56:02 +08:00
Kevin Hu
86892959a0
Rebuild graph when it's out of time. (#4607)
### What problem does this PR solve?

#4543

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring
2025-01-23 17:26:20 +08:00
Zhichang Yu
240e7d7c22
Unified user_service.py (#4606)
### What problem does this PR solve?

Unified user_service.py

### Type of change

- [x] Refactoring
2025-01-23 15:49:21 +08:00
Kevin Hu
dd0ebbea35
Light GraphRAG (#4585)
### What problem does this PR solve?

#4543

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-01-22 19:43:14 +08:00
Kevin Hu
e478586a8e
Refactor. (#4487)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
2025-01-15 14:06:46 +08:00
Kevin Hu
f318342c8e
Recalling the file uploaded while chatting. (#4472)
### What problem does this PR solve?

#4445

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-01-14 12:05:20 +08:00
Kevin Hu
46c52d65b7
Add meta data while chatting. (#4455)
### What problem does this PR solve?

#3690

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-01-13 14:35:24 +08:00
Kevin Hu
7d909d4d1b
Add doc meta data. (#4442)
### What problem does this PR solve?

#3690

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-01-10 19:06:59 +08:00