3009 Commits

Author SHA1 Message Date
balibabu
ac2643700b
Feat: Add return value widget to CodeForm #3221 (#7776)
### What problem does this PR solve?
Feat: Add return value widget  to CodeForm #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-05-21 19:35:27 +08:00
balibabu
558b252c5a
Feat: Switching the programming language of the code operator will switch the corresponding language template #3221 (#7770)
### What problem does this PR solve?

Feat: Switching the programming language of the code operator will
switch the corresponding language template #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-05-21 18:22:06 +08:00
balibabu
754a5e1cee
Feat: Fixed the issue where the page would refresh continuously when opening the sheet on the right side of the canvas #3221 (#7756)
### What problem does this PR solve?

Feat: Fixed the issue where the page would refresh continuously when
opening the sheet on the right side of the canvas #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-05-21 17:03:16 +08:00
Stephen Hu
e3e7c7ddaa
Feat: delete useless image blobs when task executor meet edge cases (#7727)
### What problem does this PR solve?

delete useless image blobs when the task executor meets edge cases

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-05-21 10:22:30 +08:00
writinwaters
76b278af8e
0519 pdfparser (#7747)
### What problem does this PR solve?


### Type of change


- [x] Documentation Update
2025-05-20 19:41:55 +08:00
balibabu
1c6320828c
Feat: Rename agent #3221 (#7740)
### What problem does this PR solve?

Feat: Rename agent #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-05-20 19:13:19 +08:00
balibabu
d72468426e
Feat: Render the agent list page by page #3221 (#7736)
### What problem does this PR solve?

Feat: Render the agent list page by page #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-05-20 16:03:55 +08:00
balibabu
796f4032b8
Feat: Migrate the code operator to the new agent. #3221 (#7731)
### What problem does this PR solve?

Feat: Migrate the code operator to the new agent. #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-05-20 15:30:56 +08:00
balibabu
1ae7b942d9
Feat: The image displayed in the reply message can also be clicked to display the location of the source document where the slice is located #7623 (#7723)
### What problem does this PR solve?

Feat: The image displayed in the reply message can also be clicked to
display the location of the source document where the slice is located
#7623

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-05-20 10:40:53 +08:00
liu an
fed1221302
Refa: HTTP API list datasets / test cases / docs (#7720)
### What problem does this PR solve?

This PR introduces Pydantic-based validation for the list datasets HTTP
API, improving code clarity and robustness. Key changes include:

Pydantic Validation
Error Handling
Test Updates
Documentation Updates

### Type of change

- [x] Documentation Update
- [x] Refactoring
2025-05-20 09:58:26 +08:00
Chaoxi Weng
6ed81d6774
Feat: Add OAuth state parameter for CSRF protection (#7709)
### What problem does this PR solve?

Add OAuth `state` parameter for CSRF protection:
- Updated `get_authorization_url()` to accept an optional state
parameter
- Generated a unique state value during OAuth login and stored in
session
- Verified state parameter in callback to ensure request legitimacy

This PR follows OAuth 2.0 security best practices by ensuring that the
authorization request originates from the same user who initiated the
flow.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-05-20 09:40:31 +08:00
donblack01
115850945e
Fix:When you create a new API module named xxxa_api, the access route will become xxx instead of xxxa. For example, when I create a new API module named 'data_api', the access route will become 'dat' instead of 'data (#7325)
### What problem does this PR solve?

Fix:When you create a new API module named xxxa_api, the access route
will become xxx instead of xxxa. For example, when I create a new API
module named 'data_api', the access route will become 'dat' instead of
'data'
Fix:Fixed the issue where the new knowledge base would not be renamed
when there was a knowledge base with the same name

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: tangyu <1@1.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-05-20 09:39:26 +08:00
balibabu
8e87436725
Feat: Modify the Python language template code of the code operator #4977 (#7714)
### What problem does this PR solve?

Feat: Modify the Python language template code of the code operator
#4977
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-05-19 19:34:43 +08:00
Yongteng Lei
e8e2a95165
Refa: more fallbacks for bad citation format (#7710)
### What problem does this PR solve?

More fallbacks for bad citation format

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring
2025-05-19 19:34:05 +08:00
Yongteng Lei
b908c33464
Fix: uncaptured image data with position information (#7683)
### What problem does this PR solve?

Fixed uncaptured figure data with position information. #7466, #7681

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-05-19 19:33:28 +08:00
Yongteng Lei
0ebf05440e
Feat: repair corrupted PDF files on upload automatically (#7693)
### What problem does this PR solve?

Try the best to repair corrupted PDF files on upload automatically.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-05-19 14:54:06 +08:00
liwenju0
7df1bd4b4a
When creating an assistant, no dataset is specified, a different default system promt is used (#7690)
### What problem does this PR solve?

- Updated the dialog settings function to add a default prompt
configuration for no dataset.
- The prompt configuration will be determined based on the presence of
`kb_ids` in the request.


### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (Non-breaking change, adding functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

---------

Co-authored-by: wenju.li <wenju.li@deepctr.cn>
2025-05-19 11:33:54 +08:00
S0b3Rr
5d21cc3660
fix: Fix the problem that concurrent execution limit in task executor fails and causes OOM (issue#7580) (#7700)
### What problem does this PR solve?

## Cause of the bug:
During the execution process, due to improper use of trio
CapacityLimiter, the configuration parameter MAX_CONCURRENT_TASKS is
invalid, causing the executor to take out a large number of tasks from
the Redis queue at one time.

This behavior will cause the task executor to occupy too much memory and
be killed by the OS when a large number of tasks exist at the same time.
As a result, all executing tasks are suspended.

## Fix:
Added the task_manager method to the entry of /rag/svr/task_executor.py
to make CapacityLimiter effective. Deleted the invalid async with
statement.

## Fix result:
After testing, the task executor execution meets expectations, that is:
concurrent execution of up to $MAX_CONCURRENT_TASKS tasks.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2025-05-19 10:25:56 +08:00
Song Fuchang
b0275b8483
Fix: value too long error for chat name (#7697)
### What problem does this PR solve?

Hello, when I input a very long line in the chat input box, it will fail
with following error:

```
2025-05-17 16:11:26,004 ERROR    182558 value too long for type character varying(255)
Traceback (most recent call last):
  File "/var/home/sfc/Projects/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3291, in execute_sql
    cursor.execute(sql, params or ())
psycopg2.errors.StringDataRightTruncation: value too long for type character varying(255)


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/home/sfc/Projects/ragflow/api/apps/conversation_app.py", line 68, in set_conversation
    ConversationService.save(**conv)
  File "/var/home/sfc/Projects/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3128, in inner
    return fn(*args, **kwargs)
  File "/var/home/sfc/Projects/ragflow/api/db/services/common_service.py", line 145, in save
    return cls.save_n(**kwargs)
  File "/var/home/sfc/Projects/ragflow/api/db/services/common_service.py", line 139, in save_n
    sample_obj = cls.model(**kwargs).save(force_insert=True)
  File "/var/home/sfc/Projects/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 6923, in save
    pk = self.insert(**field_dict).execute()
  File "/var/home/sfc/Projects/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 2011, in inner
    return method(self, database, *args, **kwargs)
  File "/var/home/sfc/Projects/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 2082, in execute
    return self._execute(database)
  File "/var/home/sfc/Projects/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 2887, in _execute
    return super(Insert, self)._execute(database)
  File "/var/home/sfc/Projects/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 2598, in _execute
    cursor = self.execute_returning(database)
  File "/var/home/sfc/Projects/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 2605, in execute_returning
    cursor = database.execute(self)
  File "/var/home/sfc/Projects/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3299, in execute
    return self.execute_sql(sql, params)
  File "/var/home/sfc/Projects/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3289, in execute_sql
    with __exception_wrapper__:
  File "/var/home/sfc/Projects/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3059, in __exit__
    reraise(new_type, new_type(exc_value, *exc_args), traceback)
  File "/var/home/sfc/Projects/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 192, in reraise
    raise value.with_traceback(tb)
  File "/var/home/sfc/Projects/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3291, in execute_sql
    cursor.execute(sql, params or ())
peewee.DataError: value too long for type character varying(255)
```

This PR fix it by truncate the `name` field in the `set_conversation`
method in the `conversation_app.py`.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2025-05-19 10:25:41 +08:00
writinwaters
86c6fee320
Docs: Added an FAQ (#7694)
### What problem does this PR solve?

### Type of change

- [x] Documentation Update
2025-05-19 09:58:10 +08:00
writinwaters
c0bee906d2
Docs: Added a guide on switching document engine (#7692)
### What problem does this PR solve?

### Type of change

- [x] Documentation Update
2025-05-16 19:02:36 +08:00
balibabu
bfaa469b9a
Feat: Rendering recall test page #3221 (#7689)
### What problem does this PR solve?

Feat: Rendering recall test page #3221
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-05-16 18:56:48 +08:00
balibabu
d73a08b9eb
Fix: Fixed the issue where message references could not be displayed (#7691)
### What problem does this PR solve?

Fix: Fixed the issue where message references could not be displayed

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-05-16 18:56:25 +08:00
Song Fuchang
a1f06a4fdc
Feat: Support tool calling in Generate component (#7572)
### What problem does this PR solve?

Hello, our use case requires LLM agent to invoke some tools, so I made a
simple implementation here.

This PR does two things:

1. A simple plugin mechanism based on `pluginlib`:

This mechanism lives in the `plugin` directory. It will only load
plugins from `plugin/embedded_plugins` for now.

A sample plugin `bad_calculator.py` is placed in
`plugin/embedded_plugins/llm_tools`, it accepts two numbers `a` and `b`,
then give a wrong result `a + b + 100`.

In the future, it can load plugins from external location with little
code change.

Plugins are divided into different types. The only plugin type supported
in this PR is `llm_tools`, which must implement the `LLMToolPlugin`
class in the `plugin/llm_tool_plugin.py`.
More plugin types can be added in the future.

2. A tool selector in the `Generate` component:

Added a tool selector to select one or more tools for LLM:


![image](https://github.com/user-attachments/assets/74a21fdf-9333-4175-991b-43df6524c5dc)

And with the `bad_calculator` tool, it results this with the `qwen-max`
model:


![image](https://github.com/user-attachments/assets/93aff9c4-8550-414a-90a2-1a15a5249d94)


### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2025-05-16 16:32:19 +08:00
writinwaters
cb26564d50
Docs: Added contribution guidelines and sandbox-related tips (#7685)
### What problem does this PR solve?

### Type of change

- [x] Documentation Update
2025-05-16 16:28:21 +08:00
liu an
59705a1c1d
Test: change variable for ZHIPU_AI_API_KEY (#7684)
### What problem does this PR solve?

change variable for ZHIPU_AI_API_KEY

### Type of change

- [x] Update test case
2025-05-16 15:58:54 +08:00
Chaoxi Weng
205974c359
Docs: Improve oauth configuration documentation and examples (#7675)
### What problem does this PR solve?

Improve oauth configuration documentation and examples.

- Related pull requests: 
  - #7379
  - #7553
  - #7587
- Related issues:
  -  #3495
### Type of change

- [x] Documentation Update
2025-05-16 14:17:39 +08:00
liu an
04edf9729f
Test: use environment variable for ZHIPU_AI_API_KEY (#7680)
### What problem does this PR solve?

use environment variable for ZHIPU_AI_API_KEY

### Type of change

- [x] Test update
2025-05-16 13:51:21 +08:00
balibabu
bb1268ef4b
Fix: Fixed the issue where the height of the chat page shared externally did not fill the window #7460 (#7682)
### What problem does this PR solve?
Fix: Fixed the issue where the height of the chat page shared externally
did not fill the window #7460
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-05-16 13:45:44 +08:00
Yongteng Lei
c5826d4720
Feat: launch sandbox from docker-compose (#7671)
### What problem does this PR solve?

Launch sandbox from docker-compose.
#4977
### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update

---------

Co-authored-by: writinwaters <93570324+writinwaters@users.noreply.github.com>
2025-05-16 11:14:57 +08:00
Stephen Hu
deb2faf7aa
Fix:Fail to get list_sessions (#7678)
### What problem does this PR solve?

Close #7655

Based on the codes atthe api_app, I think the reference is one-to-one
with the message
`
    def fillin_conv(ans):
        nonlocal conv, message_id
        if not conv.reference:
            conv.reference.append(ans["reference"])
        else:
            conv.reference[-1] = ans["reference"]
conv.message[-1] = {"role": "assistant", "content": ans["answer"], "id":
message_id}
        ans["id"] = message_id
`



### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2025-05-16 10:58:28 +08:00
Yongteng Lei
2777941b4e
Feat: add code agent component (#7672)
### What problem does this PR solve?

Add code agent component.
#4977
### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-05-16 10:27:47 +08:00
liu an
ae8b628f0a
Refa: HTTP API delete dataset / test cases / docs (#7657)
### What problem does this PR solve?

This PR introduces Pydantic-based validation for the delete dataset HTTP
API, improving code clarity and robustness. Key changes include:

1. Pydantic Validation
2. Error Handling
3. Test Updates
4. Documentation Updates

### Type of change

- [x] Documentation Update
- [x] Refactoring
2025-05-16 10:16:43 +08:00
balibabu
0e9ff8c1f7
Feat: Fixed the issue where the dataset configuration page kept refreshing #3221 (#7666)
### What problem does this PR solve?

Feat: Fixed the issue where the dataset configuration page kept
refreshing #3221
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-05-16 09:53:47 +08:00
balibabu
d373c46976
Fix: Use DOMPurify to filter out dangerous HTML #7668 (#7669)
### What problem does this PR solve?

Fix: Use DOMPurify to filter out dangerous HTML #7668

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-05-16 09:53:34 +08:00
balibabu
008e55a65e
Feat: Add the JS code (or other) executor component to Agent. #4977 (#7677)
### What problem does this PR solve?

Feat: Add the JS code (or other) executor component to Agent. #4977

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-05-16 09:53:00 +08:00
writinwaters
772992812a
Docs: Added a guide on AI search (#7674)
### What problem does this PR solve?

### Type of change

- [x] Documentation Update
2025-05-15 18:41:39 +08:00
Chaoxi Weng
a8542508b7
Refa: Deprecate /github_callback in favor of /oauth/callback/<channel> for GitHub OAuth integration (#7587)
### What problem does this PR solve?

Deprecate `/github_callback` route in favor of
`/oauth/callback/<channel>` for GitHub OAuth integration:

- Added GitHub OAuth support in the authentication module
- Introduced `GithubOAuthClient` with methods to fetch and normalize
user info
  - Updated `CLIENT_TYPES` to include GitHub OAuth client
- Deprecated `/github_callback` route and suggested using the generic
`/oauth/callback/<channel>` route

---
- Related pull requests: 
  - #7379
  - #7553 

### Usage

- [Create a GitHub OAuth
App](https://github.com/settings/applications/new) to obtain the
`client_id` and `client_secret`, configure the authorization callback
url: `https://your-app.com/v1/user/oauth/callback/github`
- Edit `service_conf.yaml.template`:
  ```yaml
  # ...
  oauth:
    github:
      type: "github"
      icon: "github"
      display_name: "Github"
      client_id: "your_client_id"
      client_secret: "your_client_secret"
      redirect_uri: "https://your-app.com/v1/user/oauth/callback/github"
  # ...
  ```

### Type of change

- [x] Documentation Update
- [x] Refactoring (non-breaking change)
2025-05-15 14:39:37 +08:00
Gifford Nowland
0b4d366514
Fix: Setuptools project.license as a TOML table deprecation (#7652)
TOML-table-based project.license is deprecated as per PEP 639, see:
https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license-and-license-files

### What problem does this PR solve?

The following error when building project (e.g. `uv build`)

```
SetuptoolsDeprecationWarning: `project.license` as a TOML table is deprecated
!!

        ********************************************************************************
        Please use a simple string containing a SPDX expression for `project.license`. You can also use `project.license-files`. (Both options available on setuptools>=77.0.0).

        By 2026-Feb-18, you need to update your project and remove deprecated calls
        or your builds will no longer be supported.

        See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
        ********************************************************************************

!!
```

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2025-05-15 12:23:29 +08:00
Gifford Nowland
e7a84bdac2
Fix: setuptools package definitions (#7654)
### What problem does this PR solve?
For `uv package`/`uv pip install ".[full]"`, bug introduced in #6370:

* Removes erroneous (non-package) directories (`helm`, `flask_session`)
* Adds `mcp.server` package
* Resolves "warning: package would be ignored" ambiguity by changing
`sdk` to `sdk.python.ragflow_sdk`
* Resolves "error: package directory 'intergrations' does not exist" by
including `intergrations.chatgpt-on-wechat.plugins` explicitly
* Also rearranges packages in alphabetical order, for DX.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2025-05-15 12:22:56 +08:00
Stephen Hu
d2b346cf9e
Feat: When Delete Chunk Will Also Delete Chunk Related Image (#7656)
### What problem does this PR solve?

When Delete Chunk Will Also Delete Chunk Related Image

### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2025-05-15 11:53:47 +08:00
writinwaters
1d0dcddf61
Docs: Miscellaneous UI updates (#7648)
### What problem does this PR solve?


### Type of change


- [x] Documentation Update
2025-05-15 11:35:52 +08:00
Kevin Hu
d49025b501
Trival. (#7653)
### What problem does this PR solve?

#7623

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-05-15 11:03:05 +08:00
Gifford Nowland
dd0fd13ea8
Fix: Anonymize profile input defaults (#7649)
Remove PII from webapp profile page input defaults

### Type of change

- [x] Other (please describe): Chore
2025-05-15 09:19:35 +08:00
Debug Doctor
36e32dde1a
Feat: update llm factories for SILICONFLOW (#7620)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Other (please describe): llm factories update
2025-05-14 19:46:27 +08:00
writinwaters
53a2c8e452
Docs: Chat assistant relative time expressions were enabled in 0.17.1. (#7647)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change


- [x] Documentation Update
2025-05-14 19:42:27 +08:00
balibabu
5218ff775c
Feat: Add data set configuration form #3221 (#7646)
### What problem does this PR solve?

Feat: Add data set configuration form #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-05-14 19:09:01 +08:00
balibabu
5d5dbb3bcb
Feat: Display inline (non-quoted) images in the chat and search modules #7623 (#7638)
### What problem does this PR solve?

Feat: Display inline (non-quoted) images in the chat and search modules
#7623

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-05-14 15:44:54 +08:00
Chenzy
5a0273e3ea
Docs: update 7 readme (#7639)
### What problem does this PR solve?

Update 7 readme

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2025-05-14 15:44:24 +08:00
Stephen Hu
ce81e470e3
Fix:Agent running message i10n (#7635)
### What problem does this PR solve?

Close #7612

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-05-14 14:27:12 +08:00