686 Commits

Author SHA1 Message Date
Kevin Hu
90d18143ba
Refa: add prompt to empty retrieved answwer. (#5892)
### What problem does this PR solve?

#5883

### Type of change

- [x] Refactoring
2025-03-11 13:11:14 +08:00
Kevin Hu
7b96146d3f
Fix: check desc parameter value. (#5884)
### What problem does this PR solve?

#5851

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-11 11:43:21 +08:00
Kevin Hu
8e965040ce
Fix: rm <think> for ES sql generation. (#5881)
### What problem does this PR solve?

#5850

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-11 10:41:19 +08:00
Kevin Hu
780ee2b2be
Fix: empty dataset parser id. (#5878)
### What problem does this PR solve?

#5709

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-11 10:23:08 +08:00
Raghav Patidar
6f9cd96ec5
Fix: dataset_ids parameter (#5864)
### What problem does this PR solve?

Fixed  #5839
This PR fix  error code 102, stating dataset_ids is required.

curl --request POST \
     --url http://{address}/api/v1/chats \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer <YOUR_API_KEY>' \
     --data '{
         "name": "test_chat"
     }'
     
     this is not getting datasetids , fix for it. 

file location : sdk\python\ragflow_sdk\ragflow.py

added : "dataset_ids": dataset_list if dataset_list else [],



### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Raghav <2020csb1115@iitrpr.ac.in>
2025-03-11 09:44:06 +08:00
hy89
8ba1e6c183
Feat: add sync_dsl parameter to support synchronizing modifications to existing sessions (#5843)
When accessing the /api/v1/agents/{agent_id}/completions API, sessions
created before agent modifications retain the old DSL data. To use the
latest agent configuration (like new prompts) in historical sessions, I
added the sync_dsl parameter. It defaults to False to maintain existing
behavior and only synchronizes when set to True. If needed, a manual
synchronization API can be created to trigger the sync explicitly.
2025-03-10 17:46:08 +08:00
Zhichang Yu
6ec6ca6971
Refactor graphrag to remove redis lock (#5828)
### What problem does this PR solve?

Refactor graphrag to remove redis lock

### Type of change

- [x] Refactoring
2025-03-10 15:15:06 +08:00
Kevin Hu
82f5d901c8 Refa: add model. (#5820)
### What problem does this PR solve?

#5783

### Type of change

- [x] Refactoring
2025-03-10 11:22:06 +08:00
dek
dc4d4342cd Fix: broken /api/v1/chats endpoint (#5785)
### What problem does this PR solve?

The `/api/v1/chats` API endpoint was broken, any GET request got the
following response:
```
{"code":100,"data":null,"message":"TypeError(\"'int' object is not callable\")"}
```

With this log ragflow-server side:

```
2025-03-07 14:36:26,297 ERROR    20 'int' object is not callable
Traceback (most recent call last):
  File "/ragflow/.venv/lib/python3.10/site-packages/flask/app.py", line 880, in full_dispatch_request
    rv = self.dispatch_request()
  File "/ragflow/.venv/lib/python3.10/site-packages/flask/app.py", line 865, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
  File "/ragflow/api/utils/api_utils.py", line 303, in decorated_function
    return func(*args, **kwargs)
  File "/ragflow/api/apps/sdk/chat.py", line 323, in list_chat
    logging.WARN(f"Don't exist the kb {kb_id}")
TypeError: 'int' object is not callable
2025-03-07 14:36:26,298 INFO     20 172.18.0.6 - - [07/Mar/2025 14:36:26] "GET /api/v1/chats HTTP/1.1" 200 -
``` 
This was caused by the incorrect use of `logging.WARN` as a method (it's
a loglevel object), instead of the correct `logging.warning()` method.

This PR fixes that, and also rewrites the message to be grammaticaly
correct.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-10 11:22:06 +08:00
donblack01
b1a46d5adc Fix:when start with source code not in docker env report 'UnicodeDec… (#5802)
### What problem does this PR solve?

fix:when start with  source code not in docker env report
"UnicodeDecodeError: 'gbk' codec can't decode byte 0xad in position 5:
illegal multibyte sequence" in windows

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: tangyu <1@1.com>
2025-03-10 11:22:06 +08:00
kaiyuan Zhang
50c510d16b Fix: bugs mentioned by#5760 (#5778)
### What problem does this PR solve?

Fixed the issue of "stop deleting when encountering invalid dataset ID"

#5760

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-03-10 11:22:06 +08:00
hy89
66938e0b68
Feat(api): Add dsl parameters to control whether dsl fields are included (#5769)
1. **Issue**: When calling `list_agent_session` via the HTTP API, users
may only need to display conversation messages, and do not want to see
the associated dsl, which can be very large. Therefore, consider adding
a control option to determine whether the DSL should be returned, with
the default being to return it.

2. **Documentation Discrepancy**: In the HTTP API documentation, under
"List agent sessions," the "Response" section states that the "data"
field is a dictionary when "success" is returned. However, the actual
returned data is a list. This discrepancy has been corrected.
2025-03-07 16:58:00 +08:00
Kevin Hu
3418984848
Fix: meta fields updata issue, (#5764)
### What problem does this PR solve?

#4789

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-07 16:21:27 +08:00
Kevin Hu
da3f279495
Fix: add the validation for parser_config. (#5755)
### What problem does this PR solve?

#5719

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-07 15:34:34 +08:00
Zanyatta
0e3e129a83
Fix: Resolve inconsistency in APIToken dialog_id field definition (#5749)
The `dialog_id` field was inconsistently defined:
- In the `migrate_db()` function, it was set to `null=True`.
- In the model class, it was defined as `null=False`.

This inconsistency caused an issue during the initial deployment where
the database table did not allow `dialog_id` to be null. As a result,
calling `APITokenService.save(**obj)` in `system_app.py` raised the
following error:

```
peewee.IntegrityError: null value in column "dialog_id" violates not-null constraint
```

### What problem does this PR solve?

Error: peewee.IntegrityError: null value in column "dialog_id" violates
not-null constraint

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-07 13:26:08 +08:00
Kevin Hu
c87b58511e
Fix: API empty field input. (#5748)
### What problem does this PR solve?

#5709

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-07 13:11:07 +08:00
Kevin Hu
9fc7174612
Fix: too long context during KG issue. (#5723)
### What problem does this PR solve?

#5088

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-06 19:21:07 +08:00
Kevin Hu
ff35c140dc
Refa: remove dataset language and validate dataset name length. (#5707)
### What problem does this PR solve?

#5686
#5702

### Type of change

- [x] Refactoring
2025-03-06 17:08:28 +08:00
yihong
4326873af6
refactor: no need to inherit in python3 clean the code (#5659)
### What problem does this PR solve?

As title

### Type of change


- [x] Refactoring

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
2025-03-05 18:03:53 +08:00
Kevin Hu
ec68ab1c8c
Fix: search citation issue. (#5657)
### What problem does this PR solve?
#5649
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-05 17:25:47 +08:00
Omar Leonardo Sanchez Granados
11e3f5e8b2
Feat: Upload file UI/UX enhancements (#5359)
### What problem does this PR solve?

Modifies the UX for uploading process on the website.

- Adds option to parse on creation the files
- Adds progress bar to display progress of chunk
- Adds per file feedback on uploading operation

#### Screenshots:

- Show files uploading:

![image](https://github.com/user-attachments/assets/a5693f42-8232-4d5c-a240-20ed343634a5)

- Errors on specific files

![image](https://github.com/user-attachments/assets/986a7f54-ab32-4634-89ab-a098fe1954aa)


### Type of change

- [X] New Feature (non-breaking change which adds functionality)
2025-03-05 15:20:32 +08:00
Zhichang Yu
f65c3ae62b
Refactored DocumentService.update_progress (#5642)
### What problem does this PR solve?

Refactored DocumentService.update_progress

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-05 14:48:03 +08:00
kunyi-yang
555c70672e
Fix:Fix the bug of incorrectly gets the APIToken. (#5597)
### What problem does this PR solve?

Fix the issue where, when getting a user's APIToken, if the user is part
of another user's team, it incorrectly gets the Team owner's APIToken
instead.


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-04 19:35:42 +08:00
Kevin Hu
688cb8f19d
Fix: remove KB id restriction while creating chat. (#5588)
### What problem does this PR solve?

#5586

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-04 12:36:37 +08:00
Yongteng Lei
f6dd2cd1af
Fix: fix may lose part of information of last stream chunck (#5584)
### What problem does this PR solve?

 Fix may lose part of information of last stream chunck

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-04 11:58:10 +08:00
Zhichang Yu
c813c1ff4c
Made task_executor async to speedup parsing (#5530)
### What problem does this PR solve?

Made task_executor async to speedup parsing

### Type of change

- [x] Performance Improvement
2025-03-03 18:59:49 +08:00
Debug Doctor
76cb4cd174
Feat: add 'delete' for agent's sessions api and unify apis of agent sdk (#5525)
### What problem does this PR solve?

Add sessions deletion support for agent in http and python api

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [x] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-03-03 17:15:16 +08:00
Kevin Hu
7a81fa00e9
Optimize prompt. (#5541)
### What problem does this PR solve?

#5526

### Type of change

- [x] Performance Improvement
2025-03-03 13:12:38 +08:00
Kevin Hu
606ed0c8ab
Fix: in case running KG repeatly. (#5538)
### What problem does this PR solve?

#5512

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-03 12:22:36 +08:00
Felipe Hertzer
8b1a4365ed
Fix email validation regex (#5533)
### What problem does this PR solve?

This pull request aims to fix a bug that prevents certain email
addresses from signing up. The affected TLDs were returning 'invalid
email address' errors:

.museum
.software
.photography
.technology
.marketing
.education
.international
.community
.construction
.government
.consulting
....

### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)
2025-03-03 10:55:10 +08:00
yihong
8a2542157f
Fix: possible memory leaks close #5277 (#5500)
### What problem does this PR solve?

close #5277 by make sure the file close

### Type of change

- [x] Performance Improvement

---------

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
2025-03-03 10:26:45 +08:00
yihong
622b72db4b
Fix: add ctrl+c signal for better exit (#5469)
### What problem does this PR solve?

This patch add signal for ctrl + c that can exit the code friendly
cause code base use thread daemon can not exit friendly for being
started.

how to reproduce
1. docker-compose -f docker/docker-compose-base.yml up
2. other window `bash docker/launch_backend_service.sh`
3. stop 1 first
4. try to stop 2 then two thread can not exit which must use `kill pid`

This patch fix it 
and should fix most the related issues in the `issues`

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
2025-02-28 14:52:40 +08:00
Kevin Hu
5fdfb8d465
Fix: rm think if stream is Flase. (#5458)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-28 10:05:18 +08:00
hy89
651422127c
Feat: Accessing Alibaba Cloud OSS with Amazon S3 SDK (#5438)
Accessing Alibaba Cloud OSS with Amazon S3 SDK
2025-02-27 17:02:42 +08:00
Kevin Hu
4c9a3e918f
Fix: add image2text issue. (#5431)
### What problem does this PR solve?

#5356

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-27 14:06:49 +08:00
Kevin Hu
5beb022ee1
Fix: string format error. (#5422)
### What problem does this PR solve?

#5404

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-27 12:01:46 +08:00
Kevin Hu
afaa7144a5
Fix: issue of no id for /datasets/<dataset_id>/documents (#5420)
### What problem does this PR solve?

#5401

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-27 10:39:34 +08:00
Kevin Hu
fa76974e24
Fix issue of ask API. (#5400)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-26 19:45:22 +08:00
so95
fefea3a2a5
Fixed OpenAI compatibility stream [DONE] (#5389)
Fixed OpenAI compatibility stream [DONE]



- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-02-26 17:55:12 +08:00
Yongteng Lei
0e920a91dd
FIX: correct typo (#5387)
### What problem does this PR solve?

Correct typo in supported_models file

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-26 17:21:09 +08:00
Yongteng Lei
cdcaae17c6
Feat: add VLLM (#5380)
### What problem does this PR solve?

Read to add VLMM.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-02-26 16:04:53 +08:00
k
5cab6c4ccb
Fix:HTTP API -> Stop parsing documents(AttributeError: ‘list‘ object … (#5375)
…has no attribute ‘id‘)

### What problem does this PR solve?

No PR

![image](https://github.com/user-attachments/assets/988d31bc-6551-4bb8-846c-cbbc1883d804)


![image](https://github.com/user-attachments/assets/8b09681b-1239-4ed9-8bc3-11436c5e90bc)

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2025-02-26 15:57:50 +08:00
Yongteng Lei
b3b341173f
DOCS: add OpenAI-compatible http and python api reference (#5374)
### What problem does this PR solve?

Add OpenAI-compatible http and python api reference

### Type of change

- [x] Documentation Update

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
Co-authored-by: writinwaters <93570324+writinwaters@users.noreply.github.com>
2025-02-26 15:52:26 +08:00
liwenju0
a9e4695b74
Fix:validate knowledge base association before document upload (#5373)
### What problem does this PR solve?

fix this bug: https://github.com/infiniflow/ragflow/issues/5368

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: wenju.li <wenju.li@deepctr.cn>
2025-02-26 15:47:34 +08:00
Kevin Hu
4f40f685d9
Code refactor (#5371)
### What problem does this PR solve?

#5173

### Type of change

- [x] Refactoring
2025-02-26 15:40:52 +08:00
Yongteng Lei
5c6a7cb4b8
Added OpenAI-like completion api (#5351)
### What problem does this PR solve?

Added OpenAI-like completion api, related to #4672, #4705 

This function allows users to interact with a model to get responses
based on a series of messages.
If `stream` is set to True, the response will be streamed in chunks,
mimicking the OpenAI-style API.

#### Example usage:

```bash
curl -X POST https://ragflow_address.com/api/v1/chats_openai/<chat_id>/chat/completions \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer $RAGFLOW_API_KEY" \
    -d '{
        "model": "model",
        "messages": [{"role": "user", "content": "Say this is a test!"}],
        "stream": true
    }'
```

Alternatively, you can use Python's `OpenAI` client:

```python
from openai import OpenAI

model = "model"
client = OpenAI(api_key="ragflow-api-key", base_url=f"http://ragflow_address/api/v1/chats_openai/<chat_id>")

completion = client.chat.completions.create(
    model=model,
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Who you are?"},
        {"role": "assistant", "content": "I am an AI assistant named..."},
        {"role": "user", "content": "Can you tell me how to install neovim"},
    ],
    stream=True
)

stream = True
if stream:
    for chunk in completion:
        print(chunk)
else:
    print(completion.choices[0].message.content)
```
### Type of change
- [x] New Feature (non-breaking change which adds functionality)

### Related Issues
Related to #4672, #4705
2025-02-26 11:37:29 +08:00
Kevin Hu
4e2afcd3b8
Fix FlagRerank max_length issue. (#5366)
### What problem does this PR solve?

#5352

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-26 11:01:13 +08:00
Zhenglin Dong
11e6d84d46
Fix: 'Chunk not found!' error in team-sharing knowledge base. (#5361)
### What problem does this PR solve?

As issue #3268 mentioned, "Chun not found!" exception will occur,
especially during the teamwork of knowledge bases.

### The reason of this bug

"tenants" are the people on current_user's team, including the team
owner itself. The old one only checks the first "tenant", tenants[0],
which will cause error when anyone editing the chunk that is not in
tenants[0]'s knowledge base.

My modification won't introduce new errors while iterate all the tenant
then retrieve knowledge bases of each.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-26 10:24:35 +08:00
Kevin Hu
53b9e7b52f
Add tavily as web searh tool. (#5349)
### What problem does this PR solve?

#5198

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-02-26 10:21:04 +08:00
Kevin Hu
b3d579e2c1
Refine prompt of agentic search. (#5312)
### What problem does this PR solve?

#5173

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-02-25 09:21:52 +08:00