469 Commits

Author SHA1 Message Date
Kevin Hu
7f44cf543a
move import positions (#2753)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
2024-10-09 10:34:58 +08:00
JobSmithManipulation
16472eb3ea
solve knowledgegraph issue when calling gemini model (#2738)
### What problem does this PR solve?
#2720

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-10-08 18:27:04 +08:00
Jake Stride
d92acdcf1d
Update azure_sas_conn.py - fixing container_url typo (#2740)
### What problem does this PR solve?

Fixes #2739

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-10-08 18:26:30 +08:00
Kevin Hu
04ff9cda7c
expand rerank range (#2746)
### What problem does this PR solve?


### Type of change

- [x] Performance Improvement
2024-10-08 16:34:33 +08:00
JobSmithManipulation
a3ab5ba9ac
support sequence2txt and tts model in Xinference (#2696)
### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-10-08 10:43:18 +08:00
Omar Leonardo Sanchez Granados
34761fa4ca
Fix/bedrock issues (#2718)
### What problem does this PR solve?

Adding a Bedrock API key for Claude Sonnet was broken. I find the issue
came up when trying to test the LLM configuration, the system is a
required parameter in boto3.

As well, there were problems in Bedrock implementation for embeddings
when trying to encode queries.

### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)
2024-10-05 16:44:50 +08:00
Kevin Hu
0a7654c747
fix error in exception (#2694)
### What problem does this PR solve?
#2670

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-09-30 17:54:27 +08:00
yqkcn
570ad420a8
remove unused import (#2679)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
2024-09-30 16:59:39 +08:00
Kevin Hu
fc867cb959
rename get_txt to get_text (#2649)
### What problem does this PR solve?



### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-09-29 12:47:09 +08:00
yqkcn
aea553c3a8
Add get_txt function (#2639)
### What problem does this PR solve?

Add get_txt function to reduce duplicate code

### Type of change

- [x] Refactoring

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-09-29 10:29:56 +08:00
yqkcn
e82e8fde13
Fix logger error (#2643)
### What problem does this PR solve?

Fix logger error: AttributeError: 'Logger' object has no attribute
'basicConfig'

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-09-29 09:49:59 +08:00
yqkcn
a44ed9626a
handle nits in task_executor (#2637)
### What problem does this PR solve?

- fix typo
- fix string format
- format import

### Type of change

- [x] Refactoring
2024-09-29 09:49:45 +08:00
JobSmithManipulation
96f56a3c43
add huggingface model (#2624)
### What problem does this PR solve?

#2469

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-09-27 19:15:38 +08:00
Zhichang Yu
1b2f66fc11
Added doc on dev-slim (#2627)
Added doc on dev-slim

### Type of change

- [x] Documentation Update
- [x] Refactoring

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-09-27 19:15:27 +08:00
yqkcn
34abcf7704
style: fix typo and format code (#2618)
### What problem does this PR solve?

- Fix typo
- Remove unused import
- Format code

### Type of change

- [x] Other (please describe): typo and format
2024-09-27 13:17:25 +08:00
liuhua
e11a74eed5
Update Yichat base_url (#2620)
### What problem does this PR solve?

Update Yichat base_url

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-09-27 12:55:58 +08:00
liuhua
b68d349bd6
Fix: renrank_model and pdf_parser bugs | Update: session API (#2601)
### What problem does this PR solve?

Fix: renrank_model and pdf_parser bugs | Update: session API
#2575
#2559
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring

---------

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-09-26 16:05:25 +08:00
Kevin Hu
dda1367ab2
make it lighten (#2577)
### What problem does this PR solve?

#2295

### Type of change

- [x] Refactoring
2024-09-25 13:38:40 +08:00
lidp
08d5637770
Fix tokenizer bug (#2573)
### What problem does this PR solve?

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-09-25 10:30:49 +08:00
Kevin Hu
7bb28ca2bd
add lighten control (#2567)
### What problem does this PR solve?

#2295

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-09-24 19:22:01 +08:00
Kevin Hu
ecf441c830
refine using rerank model (#2553)
### What problem does this PR solve?

#2552

### Type of change

- [x] Performance Improvement
2024-09-24 12:38:18 +08:00
liuhua
d9c2a128a5
SparkTTS (#2535)
### What problem does this PR solve?

SparkTTS

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-09-24 12:15:12 +08:00
Kevin Hu
4a6a2a0f1b
refine xinference (#2521)
### What problem does this PR solve?

#1588

### Type of change

- [x] Refactoring
2024-09-20 18:37:01 +08:00
Kevin Hu
78856703c4
make excel parsing configurable (#2517)
### What problem does this PR solve?

#2516

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-09-20 15:33:38 +08:00
Kevin Hu
099c37ba95
rm key set in xinference (#2511)
### What problem does this PR solve?

#2492

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-09-20 10:55:52 +08:00
Kevin Hu
5968f148bc
refactor add LLM (#2508)
### What problem does this PR solve?

#2487

### Type of change

- [x] Refactoring
2024-09-20 10:20:35 +08:00
Fachuan Bai
ddb8be9219
Web: Display the icon of the currently used storage. (#2504)
https://github.com/infiniflow/ragflow/issues/2503


### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Before:

<img width="611" alt="image"
src="https://github.com/user-attachments/assets/02a3a1ee-7bfb-4fe0-9b15-11ced69cc8a3">

After:

<img width="796" alt="image"
src="https://github.com/user-attachments/assets/371136af-8d16-47aa-909b-26609d3ad60e">

<img width="557" alt="image"
src="https://github.com/user-attachments/assets/9268362f-2b6a-4ea1-9fe7-659f7292e5e1">
2024-09-20 09:49:16 +08:00
Fachuan Bai
422c229e52
Storage: Rename all the variables about get file to storage from minio. (#2497)
https://github.com/infiniflow/ragflow/issues/2496

### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [x] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-09-19 19:19:27 +08:00
Kevin Hu
b5d1d2fec4
refine TTS (#2500)
### What problem does this PR solve?

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-09-19 19:15:16 +08:00
liuhua
d545633a6c
OpenAITTS (#2493)
### What problem does this PR solve?

OpenAITTS

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-09-19 16:55:18 +08:00
Kevin Hu
01acc3fd5a
fix duplicated llm name betweeen different suppliers (#2477)
### What problem does this PR solve?

#2465

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-09-18 16:09:22 +08:00
Kevin Hu
3044cb85fd
fix batch size error for qianwen embedding (#2431)
### What problem does this PR solve?

#2402

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-09-14 14:40:57 +08:00
Kevin Hu
54342ae0a2
boost highlight performace (#2419)
### What problem does this PR solve?

#2415

### Type of change

- [x] Performance Improvement
2024-09-13 18:10:32 +08:00
Kevin Hu
9d4bb5767c
make highlight friendly to English (#2417)
### What problem does this PR solve?

#2415

### Type of change

- [x] Performance Improvement
2024-09-13 17:03:51 +08:00
lidp
cfae63d107
Add RAGFlow benchmark (#2387)
### What problem does this PR solve?

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-09-12 19:01:00 +08:00
Kevin Hu
4730145696
debug backend API for TAB 'search' (#2389)
### What problem does this PR solve?
#2247

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-09-12 17:51:20 +08:00
Kevin Hu
333608a1d4
add search TAB backend api (#2375)
### What problem does this PR solve?
 #2247

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-09-11 19:49:18 +08:00
黄腾
35b7d17d97
fix SILICONFLOW embedding error (#2363)
### What problem does this PR solve?

#2335  fix SILICONFLOW embedding error

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
2024-09-11 12:17:44 +08:00
lidp
7241c73c7a
Add benchmark ndcg@10 (#2326)
### What problem does this PR solve?

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-09-09 19:20:00 +08:00
黄腾
cb69c742b0
add support for TongyiQwen tts (#2311)
### What problem does this PR solve?

add support for TongyiQwen tts
#1853

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
2024-09-09 11:01:43 +08:00
Kevin Hu
2ac72899ef
reduce interval of task executor heart beat (#2308)
### What problem does this PR solve?


### Type of change

- [x] Performance Improvement
- [ ] Other (please describe):
2024-09-09 10:19:10 +08:00
Fachuan Bai
8dd3adc443
Storage: Support the s3, azure blob as the object storage of ragflow. (#2278)
### What problem does this PR solve?

issue: https://github.com/infiniflow/ragflow/issues/2277

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-09-09 09:41:14 +08:00
Kevin Hu
0fe19f3fbc
fix QWenSeq2txt bug (#2245)
### What problem does this PR solve?

#2243

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-09-04 18:25:43 +08:00
黄腾
2871455e4e
fix zhipuCV bug (#2215)
### What problem does this PR solve?

#2198  fix zhipuCV bug

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
2024-09-03 15:11:53 +08:00
Kevin Hu
5a2c542ce2
make term similarity robust (#2212)
### What problem does this PR solve?


### Type of change

- [x] Performance Improvement
2024-09-03 14:30:07 +08:00
Kevin Hu
6d232f1bdb
enable 3 char words to finegrind tokenize (#2210)
### What problem does this PR solve?


### Type of change


- [x] Performance Improvement
2024-09-03 13:37:32 +08:00
Kevin Hu
b10eb8d085
fix re sub bug (#2199)
### What problem does this PR solve?



### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-09-02 18:53:30 +08:00
黄腾
ad09d4bb24
fix tts interface error (#2197)
### What problem does this PR solve?

fix tts interface error

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-09-02 18:40:57 +08:00
黄腾
5decdde182
add support for Google Cloud (#2175)
### What problem does this PR solve?

#1853 add support for Google Cloud

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
2024-09-02 12:06:41 +08:00
Kevin Hu
fc6d8ee77f
ignore when save image fail (#2178)
### What problem does this PR solve?


### Type of change
- [x] Performance Improvement
2024-08-30 18:41:31 +08:00