2799 Commits

Author SHA1 Message Date
Yongteng Lei
542cf16292
Feat: add project_id and project_name to Langfuse API (#6481)
### What problem does this PR solve?

Enhance Langfuse API: add project_id and project_name

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-03-25 10:36:34 +08:00
liu an
27989eb9a5
Test: Add list chunk checkpoint for the add chunk API (#6482)
### What problem does this PR solve?

Add list chunk checkpoint for the add chunk API

### Type of change

- [x] update test cases
2025-03-25 10:36:21 +08:00
Graf2242
05997e8215
Remove thinking block from keyword node's result (#6474)
### What problem does this PR solve?

For now, if you use thinking model (deepseek-r1:32b with ollama server
in my case) in "Keyword" node, result contains all <think> block and so
node return not only keywords

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2025-03-25 10:22:41 +08:00
Stephen Hu
5d9afce12d
Feat: improve the performance for '/upload' API (#6479)
### What problem does this PR solve?
improve the logic to fetch parent folder, remove the useless DB IO logic

### Type of change

- [x] Performance Improvement
2025-03-25 10:22:19 +08:00
Yongteng Lei
ee6a0bd9db
Refa: enhancement: enhance the prompt of related_question API (#6463)
### What problem does this PR solve?

Enhance the prompt of `related_question` API.

### Type of change

- [x] Enhancement
- [x] Documentation Update
2025-03-25 10:00:10 +08:00
liu an
b6f3242c6c
Test: Update test cases to reduce execution time (#6470)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] update test cases
2025-03-25 09:17:05 +08:00
utopia2077
390086c6ab
Fix: split process bug in graphrag extract (#6423)
### What problem does this PR solve?

1. miss completion delimiter.
2. miss bracket process.
3. doc_ids return by update_graph is a set, and insert operation in
extract_community need a list.


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-24 21:41:20 +08:00
writinwaters
a40c5aea83
Miscellaneous UI updates (#6471)
### What problem does this PR solve?



### Type of change


- [x] Documentation Update
2025-03-24 19:36:47 +08:00
Stephen Hu
f691b4ddd2
Feat: Improve "/convert" API's performance (#6465)
### What problem does this PR solve?

for batch requests based on get_by_ids to fetch all files first replace
the O(n) IO logic.

### Type of change


- [x] Performance Improvement
2025-03-24 19:08:22 +08:00
balibabu
3c57a9986c
Feat: Add LangfuseCard component. #6155 (#6468)
### What problem does this PR solve?

Feat: Add LangfuseCard component. #6155

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-03-24 19:07:55 +08:00
Yongteng Lei
5e0a77df2b
Feat: add Langfuse APIs (#6460)
### What problem does this PR solve?

Add Langfuse APIs

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-03-24 18:25:43 +08:00
Yongteng Lei
66e557b6c0
Fix: Langfuse update model has no fields attribute (#6453)
### What problem does this PR solve?

Langfuse update model has no fields attribute

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-24 15:37:14 +08:00
Roccat
200b6f55c6
Fix: NameError: free variable 'langfuse_generation' referenced before assignment in enclosing scope (#6451)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: lizheng@ssc-hn.com <lizheng@ssc-hn.com>
2025-03-24 15:14:36 +08:00
Kevin Hu
b77ce4e846
Feat: support api-key for Ollama. (#6448)
### What problem does this PR solve?

#6189

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-24 14:53:17 +08:00
Yongteng Lei
85eb367ede
Feat: add basic Langfuse support for LLM module (#6443)
### What problem does this PR solve?

#6155

Add basic Langfuse support for LLM module.

A trace example:

<img width="755" alt="image"
src="https://github.com/user-attachments/assets/25c1f852-5116-486c-a47f-6097187142ca"
/>


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-03-24 13:18:47 +08:00
liu an
0b63346a1a
Test: Update test case for #6081 (#6446)
### What problem does this PR solve?

Update test case for #6081

### Type of change

- [x] Update test case
2025-03-24 13:18:12 +08:00
Kevin Hu
85eb3775d6
Refa: update Anthropic models. (#6445)
### What problem does this PR solve?

#6421

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-24 12:34:57 +08:00
liu an
e4c8d703b5
Test: Update test cases for PR #6194 #6259 #6376 (#6444)
### What problem does this PR solve?

PR #6194 #6259 #6376

### Type of change

- [x] Update test cases
2025-03-24 12:01:33 +08:00
balibabu
60afb63d44
Feat: Add background-core-standard to tailwind.css #3221 (#6437)
### What problem does this PR solve?

Feat: Add background-core-standard to tailwind.css #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-03-24 10:51:46 +08:00
Kevin Hu
ee5aa51d43
Fix: point in tag issue. (#6436)
### What problem does this PR solve?

#6414

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-24 10:45:29 +08:00
zhou
a6aed0da46
Fix: rerank with YoudaoRerank issue. (#6396)
### What problem does this PR solve?

Fix rerank with YoudaoRerank issue,"'YoudaoRerank' object has no
attribute '_dynamic_batch_size'"


![17425412353825](https://github.com/user-attachments/assets/9ed304c7-317a-440e-acff-fe895fc20f07)


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-24 10:09:16 +08:00
Stephen Hu
d77380f024
Feat: support pic base bullet for PPT (#6406)
### What problem does this PR solve?

support pic base bullet for PPT

modify one mistake in document

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-03-24 09:31:31 +08:00
fansir
efc4796f01
Fix ratelimit errors during document parsing (#6413)
### What problem does this PR solve?

When using the online large model API knowledge base to extract
knowledge graphs, frequent Rate Limit Errors were triggered,
causing document parsing to fail. This commit fixes the issue by
optimizing API calls in the following way:
Added exponential backoff and jitter to the API call to reduce the
frequency of Rate Limit Errors.


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2025-03-22 23:07:03 +08:00
Richard
d869e4d43f
Fix: Preserve quotes while handling variable substitution withTemplate component. (#6410)
###Address Problem:
The original implementation used re.sub(r"(\\\"|\")", "", content) which
stripped all quotes from the processed content. While this worked for
simple Jinja2-rendered templates, it caused formatting issues when :
-Quotes were required in the final output (e.g., JSON, Python Code
strings)

###Solution:
    1. Selective JSON Serialization.
    2. Removed Global Quote Removal

### What problem does this PR solve?

This PR addresses an issue in template processing where all quotation
marks (" and \") were being removed from content, potentially corrupting
string formatting in rendered outputs. **In fact, extra quotes is
generated by json.dumps(v, ensure_ascii=False).**

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-21 19:44:03 +08:00
liu an
8eefc8b5fe
Test: Added test cases for Add Chunk HTTP API (#6408)
### What problem does this PR solve?

cover [add
chunk](https://ragflow.io/docs/v0.17.2/http_api_reference#add-chunk)
endpoints

### Type of change

- [x] Add test cases
2025-03-21 19:16:30 +08:00
fansir
4091af4560
Fix: multiple top-level packages error in Python project (#6370)
### What problem does this PR solve?

This PR resolves the issue of multiple top-level packages being detected
in the Python project, which caused errors when using uv pip install.
The problem occurred because the project had multiple directories files
at the root level, leading to a flat-layout error.
To fix this, the pyproject.toml file was updated to explicitly list the
packages using the [tool.setuptools] section. This ensures that the
correct packages are included during installation, avoiding the
flat-layout error.
Type of change

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-21 18:44:49 +08:00
Kevin Hu
394d1a86f6
Fix: add chunk, empty question issue. (#6405)
### What problem does this PR solve?

#6404

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-21 18:44:12 +08:00
balibabu
d88964f629
Feat: If the Transfer item is disabled, the item cannot be edited. #3221 (#6409)
### What problem does this PR solve?

Feat: If the Transfer item is disabled, the item cannot be edited. #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-03-21 18:42:52 +08:00
fansir
0e0ebaac5f
Feat: Adds hierarchical title path tracking for tables in DOCX documents to improve context association (#6374)
### What problem does this PR solve?

Adds hierarchical title path tracking for tables in DOCX documents to
improve context association. Previously, extracted tables lacked
positional context within document structure.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-03-21 18:42:36 +08:00
Kevin Hu
8b7e53e643
Fix: miss calculate of token number. (#6401)
### What problem does this PR solve?

#6308

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-21 17:30:38 +08:00
writinwaters
979cdc3626
UI updates. (#6398)
### What problem does this PR solve?

Updated UI descriptions for delimiters and recommended chunk size

### Type of change

- [x] Documentation Update
2025-03-21 16:50:20 +08:00
Kevin Hu
a2a4bfe3e3
Fix: change ollama default num_ctx. (#6395)
### What problem does this PR solve?

#6163

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-21 16:22:03 +08:00
zhou
85480f6292
Fix: the error of Ollama embeddings interface returning "500 Internal Server Error" (#6350)
### What problem does this PR solve?

Fix the error where the Ollama embeddings interface returns a “500
Internal Server Error” when using models such as xiaobu-embedding-v2 for
embedding.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-21 15:25:48 +08:00
andy
f537b6ca00
Fix: flow list translate to zh (#6371)
### What problem does this PR solve?

Add the Chinese translation of 'noMoreData' on the flow list page

### Type of change

- [x] Refactoring
2025-03-21 14:54:12 +08:00
Kevin Hu
b5471978b0
Fix: add chunk api, empty content issue (#6390)
### What problem does this PR solve?

#6387

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-21 14:05:59 +08:00
liwenju0
efdfb39a33
Feat: Add Duplicate ID Check and Update Deletion Logic (#6376)
- Introduce the `check_duplicate_ids` function in `dataset.py` and
`doc.py` to check for and handle duplicate IDs.
- Update the deletion operation to ensure that when deleting datasets
and documents, error messages regarding duplicate IDs can be returned.
- Implement the `check_duplicate_ids` function in `api_utils.py` to
return unique IDs and error messages for duplicate IDs.


### What problem does this PR solve?

Close https://github.com/infiniflow/ragflow/issues/6234

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: wenju.li <wenju.li@deepctr.cn>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-03-21 14:05:17 +08:00
Yingfeng
7cc5603a82
Fix broken discord invitation links (#6388)
### Type of change

- [x] Documentation Update
2025-03-21 13:38:34 +08:00
Kevin Hu
9ed004e90d
Refa: control the simi for entity resolution. (#6386)
### What problem does this PR solve?

#6352

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-21 13:16:34 +08:00
Kevin Hu
d83911b632
Fix: huggingface rerank model issue. (#6385)
### What problem does this PR solve?

#6348

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-21 12:43:32 +08:00
Kevin Hu
bc58ecbfd7
Remove feature_request.md (#6383)
### What problem does this PR solve?


### Type of change


- [x] Refactoring
2025-03-21 12:03:38 +08:00
Kevin Hu
221eae2c59
Refa: refine template. (#6382)
### What problem does this PR solve?

### Type of change


- [x] Refactoring
2025-03-21 11:58:10 +08:00
Kevin Hu
37303e38ec
Refa: refine template. (#6381)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
2025-03-21 11:55:01 +08:00
Kevin Hu
b754bd523a
Fix: let quot stay. (#6377)
### What problem does this PR solve?

#6337

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-21 11:47:42 +08:00
liwenju0
1bb990719e
Feat: Add user registration toggle feature (#6327)
### What problem does this PR solve?

Feat: Add user registration toggle feature. Added a user registration
toggle REGISTER_ENABLED in the settings and .env config file. The user
creation interface now checks the state of this toggle to control the
enabling and disabling of the user registration feature.

the front-end implementation is done, the registration button does not
appear if registration is not allowed. I did the actual tests on my
local server and it worked smoothly.
### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: wenju.li <wenju.li@deepctr.cn>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-03-21 09:38:15 +08:00
lgphone
7f80d7304d
Fix: Optimized the get_by_id method to resolve the issue of missing exceptions and improve query performance (#6320)
Fix: Optimized the get_by_id method to resolve the issue of missing
exceptions and improve query performance

### What problem does this PR solve?

Optimized the get_by_id method to resolve the issue of missing
exceptions and improve query performance.
Optimization details:
1. The original method used a custom query method that required
concatenating SQL, which impacted performance.
2. The query method returned a list, which needed to be accessed by
index, posing a risk of index out-of-bounds errors.
3. The original method used except Exception to catch all errors, which
is not a best practice in Python programming and may lead to missing
exceptions. The get_or_none method accurately catches DoesNotExist
errors while allowing other errors to be raised normally.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Performance Improvement
2025-03-20 23:23:48 +08:00
Zhichang Yu
ca9c3e59fa
Call register_scripts on connecting redis (#6361)
### What problem does this PR solve?

Call register_scripts on connecting redis

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-20 23:20:37 +08:00
Yongteng Lei
674f94228b
Chore: unify Ruff config and enable async checks (ASYNC, TRIO) (#6351)
### What problem does this PR solve?

Unify Ruff config and enable async checks (ASYNC, TRIO)

### Type of change

- [x] CI/CD or tooling improvement
2025-03-20 22:31:18 +08:00
liwenju0
ef7e96e486
Feat: Add the functionality to load environment variables from a .env file (#6331)
### Change Content

- A new function `load_env_file` has been added to load environment
variables from a .env file in the current script directory.
- If the .env file exists, the variables within it will be loaded; if it
does not exist, a warning message will be output.

I found this issue while testing this pr:
https://github.com/infiniflow/ragflow/pull/6327. The locally started
server did not read the REGISTER_ENABLED variables in the .env. The
result has always been the default True
### What problem does this PR solve?

Follow the tutorial in the README.md to start from source code. base's
container that is es、redis,etc will load .env. Therefore,
`launch_backend_service.sh` should also load .env to be consistent with
the configuration of the docker container when it was started

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2025-03-20 18:35:04 +08:00
Zhichang Yu
dba0caa00b
Fix update_progress (#6340)
### What problem does this PR solve?

Fix update_progress

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-20 17:01:28 +08:00
hy89
1d9ca172e3
Fix(api): correct document parsing progress check logic (#6318)
- Fix incorrect progress check condition that prevented re-parsing of
completed documents
- Allow parsing for documents with progress 0.0 (not started) or 1.0
(completed)
- Only block parsing for documents currently in progress (0.0 < progress
< 1.0)

Close #6312

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-03-20 16:00:17 +08:00