1423 Commits

Author SHA1 Message Date
Kevin Hu
bfc07fe4f9
bigger resolution for OCR (#2919)
### What problem does this PR solve?



### Type of change

- [x] Performance Improvement
2024-10-21 16:25:42 +08:00
Kevin Hu
3e702aa4ac
add package crawl4ai (#2918)
### What problem does this PR solve?



### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-10-21 15:07:45 +08:00
Kevin Hu
2ced25c676
fix thumbnail issue (#2917)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-10-21 14:33:26 +08:00
liuhua
1935c3be1a
Fix some issues in API (#2902)
### What problem does this PR solve?

Fix some issues in API

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-10-21 14:29:06 +08:00
balibabu
609cfa7b5f
feat: Replace crawler icon #2915 (#2916)
### What problem does this PR solve?

feat: Replace crawler icon #2915

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-10-21 14:28:30 +08:00
chongchuanbing
ac26d09a59
Feature/feat1017 (#2872)
### What problem does this PR solve?

1. fix: mid map show error in knowledge graph, juse because
```@antv/g6```version changed
2. feat: concurrent threads configuration support in graph extractor
3. fix: used tokens update failed for tenant
4. feat: timeout configuration support for llm
5. fix: regex error in graph extractor
6. feat: qwen rerank(```gte-rerank```) support
7. fix: timeout deal in knowledge graph index process. Now chat by
stream output, also, it is configuratable.
8. feat: ```qwen-long``` model configuration

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: chongchuanbing <chongchuanbing@gmail.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-10-21 12:11:08 +08:00
wwwlll
4bdf3fd48e
Add agent component for web crawler (#2878)
### What problem does this PR solve?

Add agent component for  web crawler

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-10-21 11:38:41 +08:00
Kevin Hu
c1d0473f49
add zhipu glm-4-9b (#2912)
### What problem does this PR solve?

#2910

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-10-21 10:30:35 +08:00
Ziyu Huang
e5f7733b31
Resolves #2905 openai compatible model provider add llama.cpp rerank support (#2906)
### What problem does this PR solve?
Resolve #2905 



due to the in-consistent of token size, I make it safe to limit 500 in
code, since there is no config param to control

my llama.cpp run set -ub to 1024:

${llama_path}/bin/llama-server --host 0.0.0.0 --port 9901 -ub 1024 -ngl
99 -m $gguf_file --reranking "$@"





### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Here is my test Ragflow use llama.cpp

```
lot update_slots: id  0 | task 458 | prompt done, n_past = 416, n_tokens = 416
slot      release: id  0 | task 458 | stop processing: n_past = 416, truncated = 0
slot launch_slot_: id  0 | task 459 | processing task
slot update_slots: id  0 | task 459 | tokenizing prompt, len = 2
slot update_slots: id  0 | task 459 | prompt tokenized, n_ctx_slot = 8192, n_keep = 0, n_prompt_tokens = 111
slot update_slots: id  0 | task 459 | kv cache rm [0, end)
slot update_slots: id  0 | task 459 | prompt processing progress, n_past = 111, n_tokens = 111, progress = 1.000000
slot update_slots: id  0 | task 459 | prompt done, n_past = 111, n_tokens = 111
slot      release: id  0 | task 459 | stop processing: n_past = 111, truncated = 0
srv  update_slots: all slots are idle
request: POST /rerank 172.23.0.4 200

```
2024-10-21 10:06:29 +08:00
writinwaters
5aec1e3e17
DRAFT: Miscellaneous updates to HTTP API. Tried to finish off Python API ref… (#2909)
…erence but failed.

### What problem does this PR solve?



### Type of change


- [x] Documentation Update
2024-10-21 09:47:59 +08:00
Kaleo
1d6bcf5aa2
add nginx path for sdk handlers (#2899) (#2900)
### What problem does this PR solve?

add the nginx path `/api` for sdk handlers 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-10-21 09:47:45 +08:00
writinwaters
1e6d44d6ef
DRAFT: Miscellaneous proofedits on Python APIs (#2903)
### What problem does this PR solve?



### Type of change


- [x] Documentation Update
2024-10-19 19:46:13 +08:00
writinwaters
cec208051f
DRAFT: Updated chunk APIs (#2901)
### What problem does this PR solve?



### Type of change


- [x] Documentation Update

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2024-10-18 20:56:33 +08:00
balibabu
526fcbbfde
fix: Fixed the issue of error reporting when uploading files in the chat box #2897 (#2898)
### What problem does this PR solve?

fix: Fixed the issue of error reporting when uploading files in the chat
box #2897

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-10-18 17:21:12 +08:00
Kevin Hu
c760f058df
add owner check for team work (#2892)
### What problem does this PR solve?

#2834

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-10-18 13:48:57 +08:00
balibabu
8fdfa0f669
feat: Use Badge.Ribbon to distinguish the teams to which the knowledge base belongs #2846 (#2891)
### What problem does this PR solve?

feat: Use Badge.Ribbon to distinguish the teams to which the knowledge
base belongs #2846

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-10-18 12:50:06 +08:00
liuhua
ceecac69e9
Delete useless files (#2889)
### What problem does this PR solve?

Delete useless files

### Type of change


- [x] Other (please describe):
Delete useless files

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-10-18 11:30:40 +08:00
Kevin Hu
e0c0bdeb0a
add team tag to kb (#2890)
### What problem does this PR solve?
#2834

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-10-18 11:30:19 +08:00
balibabu
cf3106040a
feat: Bind data to TenantTable #2846 (#2883)
### What problem does this PR solve?

feat: Bind data to TenantTable #2846
feat: Add TenantTable

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-10-18 09:21:01 +08:00
writinwaters
791afbba15
Miscellaneous minor updates (#2885)
### What problem does this PR solve?



### Type of change


- [x] Documentation Update
2024-10-17 19:52:35 +08:00
writinwaters
8358245f64
Draft: Updated file management-related APIs (#2882)
### What problem does this PR solve?

Updated file management-related APIs

### Type of change

- [x] Documentation Update
2024-10-17 18:19:17 +08:00
Zhichang Yu
396bb4b688
Fixed docker build (#2881)
### What problem does this PR solve?

Fixed docker build

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-10-17 16:34:21 +08:00
balibabu
167b4af52b
feat: Load markdown file with "asset/source" #1739 (#2880)
### What problem does this PR solve?

feat: Load markdown file with "asset/source" #17339

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-10-17 16:03:13 +08:00
balibabu
bedb05012d
feat: Configure the root directory alias #1739 (#2875)
### What problem does this PR solve?

feat: Configure the root directory alias #1739

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-10-17 11:36:01 +08:00
Kevin Hu
6a60e26020
update dashscope (#2871)
### What problem does this PR solve?
#2857
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-10-17 09:52:31 +08:00
writinwaters
6496055e23
Updated session APIs (#2868)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Documentation Update

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2024-10-16 20:38:19 +08:00
liuhua
dab92ac1e8
Refactor Chunk API (#2855)
### What problem does this PR solve?

Refactor Chunk API
#2846
### Type of change


- [x] Refactoring

---------

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-10-16 18:41:24 +08:00
Kevin Hu
b9fa00f341
add API for tenant function (#2866)
### What problem does this PR solve?

feat: API access key management
https://github.com/infiniflow/ragflow/issues/2846
feat: Render markdown file with remark-loader
https://github.com/infiniflow/ragflow/issues/2846

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-10-16 16:10:24 +08:00
balibabu
e5d3ab0332
feat: API access key management #2846 (#2865)
### What problem does this PR solve?

feat: API access key management #2846
feat: Render markdown file with remark-loader #2846

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-10-16 15:57:39 +08:00
0000sir
4991107822
Fix keys of Xinference deployed models, especially has the same model name with public hosted models. (#2832)
### What problem does this PR solve?

Fix keys of Xinference deployed models, especially has the same model
name with public hosted models.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: 0000sir <0000sir@gmail.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-10-16 10:21:08 +08:00
Kevin Hu
51ecda0ff5
refactor (#2858)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
2024-10-16 10:17:05 +08:00
Dustin Kaiser
6850fd69c6
Enhance email validation: Allow top-level domains with 5 letters (#2856)
### What problem does this PR solve?

Currently singing up to ragflow using a mail-adress with associated
top-level domains that have more than 4 chars will fail due to a regex
validation that enforces just this.

In our use case, we'd like to use e-mail addresses with `.swiss`
top-level domains, which is a valid TLD associated with the country
switzerland in the IANA root database.

This change makes the validation accept 5-letter TLDs.


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Other (please describe): Making validation for lenient, accepting
more valid input.
2024-10-16 09:34:45 +08:00
HowardChan
e1e5711680
Feat:Compatible with Dify's External Knowledge API (#2848)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._
Fixes #2731 
### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-10-15 17:47:24 +08:00
Kevin Hu
4463128436
add rm token (#2850)
### What problem does this PR solve?

#2846

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-10-15 16:48:38 +08:00
Kevin Hu
c8783672d7
refactor api util (#2849)
### What problem does this PR solve?


### Type of change

- [x] Refactoring
2024-10-15 16:11:26 +08:00
Kevin Hu
ce495e4e3e
refine API token application (#2847)
### What problem does this PR solve?

#2846

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-10-15 15:47:40 +08:00
balibabu
fcabdf7745
feat: Generate operator names in an auto-incremental manner #1739 (#2844)
### What problem does this PR solve?

feat: Generate operator names in an auto-incremental manner #1739

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-10-15 15:36:09 +08:00
Kevin Hu
b540d41cdc
let presentation do raptor (#2838)
### What problem does this PR solve?

#2837

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-10-15 10:11:09 +08:00
writinwaters
260d694bbc
Updated chat APIs (#2831)
### What problem does this PR solve?



### Type of change

- [x] Documentation Update

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2024-10-14 20:48:23 +08:00
liuhua
6329427ad5
Refactor Document API (#2833)
### What problem does this PR solve?

Refactor Document API

### Type of change


- [x] Refactoring

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-10-14 20:03:33 +08:00
balibabu
df223eddf3
feat: Fix translation issue of message_history_window_size #1739 (#2828)
### What problem does this PR solve?

feat: Fix translation issue of message_history_window_size #1739

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-10-14 15:30:20 +08:00
balibabu
85b359556e
feat: Add message_history_window_size to CategorizeForm #1739 (#2826)
### What problem does this PR solve?

feat: Add message_history_window_size to CategorizeForm #1739
### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-10-14 15:18:45 +08:00
Kevin Hu
b164116277
refine token similarity (#2824)
### What problem does this PR solve?


### Type of change

- [x] Performance Improvement
2024-10-14 13:33:18 +08:00
writinwaters
8e5efcc47f
Updated dataset APIs (#2820)
### What problem does this PR solve?


### Type of change


- [x] Documentation Update
2024-10-12 20:07:21 +08:00
liuhua
6eed115723
Refactor API for document and session (#2819)
### What problem does this PR solve?

Refactor API for document and session.

### Type of change


- [x] Refactoring

---------

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-10-12 19:35:19 +08:00
Kevin Hu
7d80fc474c
refine components retrieval and rewrite (#2818)
### What problem does this PR solve?


### Type of change

- [x] Performance Improvement
2024-10-12 18:47:25 +08:00
liuhua
a20b82092f
Refactor Chat API (#2804)
### What problem does this PR solve?

Refactor Chat API

### Type of change

- [x] Refactoring

---------

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-10-12 13:48:43 +08:00
chongchuanbing
2a86472b88
fix chat and thumbnail bug (#2803)
### What problem does this PR solve?

1. fix white screen issue when chat response
2. thumbnail bug when document not support

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

---------

Co-authored-by: chongchuanbing <chongchuanbing@gmail.com>
2024-10-11 16:10:27 +08:00
Kevin Hu
190eea7097
trival (#2808)
### What problem does this PR solve?



### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-10-11 15:33:38 +08:00
Kevin Hu
2d1c83da59
fix LIGHTEN issue (#2806)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-10-11 15:01:27 +08:00