1544 Commits

Author SHA1 Message Date
balibabu
c86afff447
feat: Limit the maximum value of auto keywords to 30 #2687 (#2991)
### What problem does this PR solve?

feat: Limit the maximum value of auto keywords to 30 #2687

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-10-24 09:35:34 +08:00
david
b73fe0cc3c
Integrating RAGFlow API as a Plugin into ChatGPT-on-WeChat (#2988)
### What problem does this PR solve?

This PR introduces the `ragflow_chat` plugin for the ChatGPT-on-WeChat
project, extending its functionality by integrating Retrieval-Augmented
Generation (RAG) capabilities. It allows users to have more contextually
relevant conversations by retrieving information from external knowledge
sources (via the RAGFlow API) and incorporating it into their chat
interactions.

The primary goal of this PR is to enable seamless communication between
ChatGPT-on-WeChat and the RAGFlow server, improving response accuracy by
embedding knowledge-based data into conversational flows. This is
particularly useful for users who need more complex, data-driven
responses.

This PR adds a new plugin that enhances ChatGPT-on-WeChat with Ragflow
capabilities, allowing for a more enriched conversational experience
driven by external knowledge.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-10-24 09:35:11 +08:00
Jin Hai
2a614e0e23
Remove defaults to 'None' (#2996)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Documentation Update

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2024-10-23 22:58:37 +08:00
liuhua
50b425cf89
Test Cases (#2993)
### What problem does this PR solve?

Test Cases

### Type of change

- [x] Refactoring

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-10-23 22:58:27 +08:00
writinwaters
2174c350be
Updated HTTP API Reference (document, chat assistant, session, chat) (#2994)
### What problem does this PR solve?



### Type of change


- [x] Documentation Update
2024-10-23 20:07:47 +08:00
Kevin Hu
7f81fc8f9b
refactor auto keywords and auto question (#2990)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
2024-10-23 17:00:56 +08:00
Hui Peng
f090075cb2
allowing docker container to access service on host (#2895)
### What problem does this PR solve?

1. services running (e.g., ollama) running on the host could not be
accessed from docker containers

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
2024-10-23 16:43:21 +08:00
balibabu
ec6d942d83
feat: Added auto_keywords and auto_questions fields to the parsing configuration page #2687 (#2987)
### What problem does this PR solve?

feat: Added auto_keywords and auto_questions fields to the parsing
configuration page #2687

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-10-23 15:45:03 +08:00
liuhua
8714754afc
Fix some issues in API (#2982)
### What problem does this PR solve?

Fix some issues in API

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-10-23 12:02:18 +08:00
writinwaters
43b959fe58
minor (#2984)
### What problem does this PR solve?



### Type of change


- [x] Documentation Update
2024-10-23 11:00:35 +08:00
Kevin Hu
320e8f6553
fix generate string join issue (#2983)
### What problem does this PR solve?

#2921

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-10-23 10:54:04 +08:00
Kevin Hu
89d5b2414e
fix SILICONFLOW rerank error (#2980)
### What problem does this PR solve?

#2977

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-10-23 10:12:39 +08:00
writinwaters
91ea559f9e
DRAFT: Updated python and http api references (#2973)
### What problem does this PR solve?


### Type of change

- [x] Documentation Update
2024-10-22 17:10:23 +08:00
Yinquan WANG
445dce4363
[Bug]: unnecessary auto-increment calculations in the tokens statistics of the chat model (#2969)
### What problem does this PR solve?

the details is shown in
https://github.com/infiniflow/ragflow/issues/2968

### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-10-22 16:26:04 +08:00
Kevin Hu
1fce6caf80
make titles in markdown not be splited with following content (#2971)
### What problem does this PR solve?

#2970 
### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2024-10-22 15:25:23 +08:00
Kevin Hu
adb0a93d95
add component invoke (#2967)
### What problem does this PR solve?

#2908

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-10-22 14:16:44 +08:00
Kevin Hu
226bdd6e99
add auto keywords and auto-question (#2965)
### What problem does this PR solve?

#2687

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-10-22 13:12:49 +08:00
Yinquan WANG
5aa9d7787e
[Bug]: When use OpenAI chat model , raise ERROR: 'CompletionUsage' object has no attribute 'get' #2948 (#2949)
[Bug]: When use OpenAI chat model , raise ERROR: 'CompletionUsage'
object has no attribute 'get' #2948

### What problem does this PR solve?

the detail of this PR is shown at
https://github.com/infiniflow/ragflow/issues/2948

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-10-22 11:40:05 +08:00
Kevin Hu
b2524eec49
fix sequence2txt error and usage total token issue (#2961)
### What problem does this PR solve?

#1363

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-10-22 11:38:37 +08:00
wwwlll
6a4858a7ee
Fix thumbnail_img NoneType error (#2941)
### What problem does this PR solve?

fix thumbnail_img NoneType error

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-10-22 09:21:05 +08:00
writinwaters
1a623df849
DRAFT: Miscellaneous updates to HTTP API reference (#2923)
### What problem does this PR solve?


### Type of change


- [x] Documentation Update
2024-10-21 19:50:45 +08:00
Kevin Hu
bfc07fe4f9
bigger resolution for OCR (#2919)
### What problem does this PR solve?



### Type of change

- [x] Performance Improvement
2024-10-21 16:25:42 +08:00
Kevin Hu
3e702aa4ac
add package crawl4ai (#2918)
### What problem does this PR solve?



### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-10-21 15:07:45 +08:00
Kevin Hu
2ced25c676
fix thumbnail issue (#2917)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-10-21 14:33:26 +08:00
liuhua
1935c3be1a
Fix some issues in API (#2902)
### What problem does this PR solve?

Fix some issues in API

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-10-21 14:29:06 +08:00
balibabu
609cfa7b5f
feat: Replace crawler icon #2915 (#2916)
### What problem does this PR solve?

feat: Replace crawler icon #2915

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-10-21 14:28:30 +08:00
chongchuanbing
ac26d09a59
Feature/feat1017 (#2872)
### What problem does this PR solve?

1. fix: mid map show error in knowledge graph, juse because
```@antv/g6```version changed
2. feat: concurrent threads configuration support in graph extractor
3. fix: used tokens update failed for tenant
4. feat: timeout configuration support for llm
5. fix: regex error in graph extractor
6. feat: qwen rerank(```gte-rerank```) support
7. fix: timeout deal in knowledge graph index process. Now chat by
stream output, also, it is configuratable.
8. feat: ```qwen-long``` model configuration

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: chongchuanbing <chongchuanbing@gmail.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-10-21 12:11:08 +08:00
wwwlll
4bdf3fd48e
Add agent component for web crawler (#2878)
### What problem does this PR solve?

Add agent component for  web crawler

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-10-21 11:38:41 +08:00
Kevin Hu
c1d0473f49
add zhipu glm-4-9b (#2912)
### What problem does this PR solve?

#2910

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-10-21 10:30:35 +08:00
Ziyu Huang
e5f7733b31
Resolves #2905 openai compatible model provider add llama.cpp rerank support (#2906)
### What problem does this PR solve?
Resolve #2905 



due to the in-consistent of token size, I make it safe to limit 500 in
code, since there is no config param to control

my llama.cpp run set -ub to 1024:

${llama_path}/bin/llama-server --host 0.0.0.0 --port 9901 -ub 1024 -ngl
99 -m $gguf_file --reranking "$@"





### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Here is my test Ragflow use llama.cpp

```
lot update_slots: id  0 | task 458 | prompt done, n_past = 416, n_tokens = 416
slot      release: id  0 | task 458 | stop processing: n_past = 416, truncated = 0
slot launch_slot_: id  0 | task 459 | processing task
slot update_slots: id  0 | task 459 | tokenizing prompt, len = 2
slot update_slots: id  0 | task 459 | prompt tokenized, n_ctx_slot = 8192, n_keep = 0, n_prompt_tokens = 111
slot update_slots: id  0 | task 459 | kv cache rm [0, end)
slot update_slots: id  0 | task 459 | prompt processing progress, n_past = 111, n_tokens = 111, progress = 1.000000
slot update_slots: id  0 | task 459 | prompt done, n_past = 111, n_tokens = 111
slot      release: id  0 | task 459 | stop processing: n_past = 111, truncated = 0
srv  update_slots: all slots are idle
request: POST /rerank 172.23.0.4 200

```
2024-10-21 10:06:29 +08:00
writinwaters
5aec1e3e17
DRAFT: Miscellaneous updates to HTTP API. Tried to finish off Python API ref… (#2909)
…erence but failed.

### What problem does this PR solve?



### Type of change


- [x] Documentation Update
2024-10-21 09:47:59 +08:00
Kaleo
1d6bcf5aa2
add nginx path for sdk handlers (#2899) (#2900)
### What problem does this PR solve?

add the nginx path `/api` for sdk handlers 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-10-21 09:47:45 +08:00
writinwaters
1e6d44d6ef
DRAFT: Miscellaneous proofedits on Python APIs (#2903)
### What problem does this PR solve?



### Type of change


- [x] Documentation Update
2024-10-19 19:46:13 +08:00
writinwaters
cec208051f
DRAFT: Updated chunk APIs (#2901)
### What problem does this PR solve?



### Type of change


- [x] Documentation Update

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2024-10-18 20:56:33 +08:00
balibabu
526fcbbfde
fix: Fixed the issue of error reporting when uploading files in the chat box #2897 (#2898)
### What problem does this PR solve?

fix: Fixed the issue of error reporting when uploading files in the chat
box #2897

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-10-18 17:21:12 +08:00
Kevin Hu
c760f058df
add owner check for team work (#2892)
### What problem does this PR solve?

#2834

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-10-18 13:48:57 +08:00
balibabu
8fdfa0f669
feat: Use Badge.Ribbon to distinguish the teams to which the knowledge base belongs #2846 (#2891)
### What problem does this PR solve?

feat: Use Badge.Ribbon to distinguish the teams to which the knowledge
base belongs #2846

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-10-18 12:50:06 +08:00
liuhua
ceecac69e9
Delete useless files (#2889)
### What problem does this PR solve?

Delete useless files

### Type of change


- [x] Other (please describe):
Delete useless files

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-10-18 11:30:40 +08:00
Kevin Hu
e0c0bdeb0a
add team tag to kb (#2890)
### What problem does this PR solve?
#2834

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-10-18 11:30:19 +08:00
balibabu
cf3106040a
feat: Bind data to TenantTable #2846 (#2883)
### What problem does this PR solve?

feat: Bind data to TenantTable #2846
feat: Add TenantTable

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-10-18 09:21:01 +08:00
writinwaters
791afbba15
Miscellaneous minor updates (#2885)
### What problem does this PR solve?



### Type of change


- [x] Documentation Update
2024-10-17 19:52:35 +08:00
writinwaters
8358245f64
Draft: Updated file management-related APIs (#2882)
### What problem does this PR solve?

Updated file management-related APIs

### Type of change

- [x] Documentation Update
2024-10-17 18:19:17 +08:00
Zhichang Yu
396bb4b688
Fixed docker build (#2881)
### What problem does this PR solve?

Fixed docker build

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-10-17 16:34:21 +08:00
balibabu
167b4af52b
feat: Load markdown file with "asset/source" #1739 (#2880)
### What problem does this PR solve?

feat: Load markdown file with "asset/source" #17339

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-10-17 16:03:13 +08:00
balibabu
bedb05012d
feat: Configure the root directory alias #1739 (#2875)
### What problem does this PR solve?

feat: Configure the root directory alias #1739

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-10-17 11:36:01 +08:00
Kevin Hu
6a60e26020
update dashscope (#2871)
### What problem does this PR solve?
#2857
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-10-17 09:52:31 +08:00
writinwaters
6496055e23
Updated session APIs (#2868)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Documentation Update

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2024-10-16 20:38:19 +08:00
liuhua
dab92ac1e8
Refactor Chunk API (#2855)
### What problem does this PR solve?

Refactor Chunk API
#2846
### Type of change


- [x] Refactoring

---------

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-10-16 18:41:24 +08:00
Kevin Hu
b9fa00f341
add API for tenant function (#2866)
### What problem does this PR solve?

feat: API access key management
https://github.com/infiniflow/ragflow/issues/2846
feat: Render markdown file with remark-loader
https://github.com/infiniflow/ragflow/issues/2846

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-10-16 16:10:24 +08:00
balibabu
e5d3ab0332
feat: API access key management #2846 (#2865)
### What problem does this PR solve?

feat: API access key management #2846
feat: Render markdown file with remark-loader #2846

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-10-16 15:57:39 +08:00