1203 Commits

Author SHA1 Message Date
JobSmithManipulation
99a7c0fb97
update sdk document and chunk (#2421)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-09-14 13:24:21 +08:00
Vitaliy Groshev
7e75b9d778
fix parsing spaces in russian language PDFs (#1987) (#2427)
### What problem does this PR solve?

[#1987](https://github.com/infiniflow/ragflow/issues/1987)

When scanning PDF files character by character, the parser excluded
spaces if the string did not match regex. Text from [Russian
documents](https://github.com/user-attachments/files/16659706/dogovor_oferta.pdf)
needs spaces, but it does not match the regex because it uses different
alphabet. That's why PDFs were parsed incorrectly and were almost
unusable as source. Fixed that by adding Russian alphabet to regex.

There might be problems with other languages that use different
alphabets. I additionally tested [PDF in
Spanish](https://www.scusd.edu/sites/main/files/file-attachments/howtohelpyourchildsucceedinschoolspanish.pdf?1338307816)
and old [a-zA-Z...] regex parses it correctly with spaces.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-09-14 13:14:39 +08:00
writinwaters
a467f31238
minor (#2422)
### What problem does this PR solve?


### Type of change

- [x] Documentation Update
2024-09-14 09:34:35 +08:00
Kevin Hu
54342ae0a2
boost highlight performace (#2419)
### What problem does this PR solve?

#2415

### Type of change

- [x] Performance Improvement
2024-09-13 18:10:32 +08:00
writinwaters
bdcf195b20
Initial draft of Create a General-purpose chatbot (#2411)
### What problem does this PR solve?


### Type of change

- [x] Documentation Update
2024-09-13 17:21:03 +08:00
Kevin Hu
3f571a13c2
fix empty children in mindmap (#2418)
### What problem does this PR solve?

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-09-13 17:19:47 +08:00
Kevin Hu
9d4bb5767c
make highlight friendly to English (#2417)
### What problem does this PR solve?

#2415

### Type of change

- [x] Performance Improvement
2024-09-13 17:03:51 +08:00
Kevin Hu
5e7b93e802
add updates for README (#2413)
### What problem does this PR solve?



### Type of change

- [x] Documentation Update
2024-09-13 14:31:04 +08:00
balibabu
ec4def9a44
feat: When the mindmap data is empty, it will not be displayed on the search page #2247 (#2414)
### What problem does this PR solve?

feat: When the mindmap data is empty, it will not be displayed on the
search page #2247
### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2024-09-13 14:30:51 +08:00
balibabu
2bd71d722b
feat: Modify the style of the answer card on the search page #2247 (#2412)
### What problem does this PR solve?

feat: Modify the style of the answer card on the search page #2247

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-09-13 12:31:31 +08:00
balibabu
8f2c0176b4
feat: Use Spin to wrap the chunk list on the search page #2247 (#2409)
### What problem does this PR solve?

feat: Use Spin to wrap the chunk list on the search page #2247
### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-09-13 11:38:09 +08:00
Kevin Hu
b261b6aac0
fix pip install error (#2407)
### What problem does this PR solve?

#2356

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-09-13 10:06:54 +08:00
balibabu
cbdf54cf36
feat: Click on the chunk on the search page to locate the corresponding file location #2247 (#2399)
### What problem does this PR solve?

feat: Click on the chunk on the search page to locate the corresponding
file location #2247

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-09-13 08:54:26 +08:00
balibabu
db0606e064
feat: Wrap the searched chunk with a Popover #2247 (#2398)
### What problem does this PR solve?

feat: Wrap the searched chunk with a Popover #2247

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-09-12 19:15:44 +08:00
lidp
cfae63d107
Add RAGFlow benchmark (#2387)
### What problem does this PR solve?

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-09-12 19:01:00 +08:00
lidp
88f8c8ed86
Fix volcengine yfinance confliction (#2386)
### What problem does this PR solve?

#2379 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-09-12 19:00:35 +08:00
balibabu
4158697fe6
feat: Add component AkShare #1739 (#2390)
### What problem does this PR solve?

 feat: Add component AkShare #1739

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-09-12 17:58:05 +08:00
balibabu
5f9cb16a3c
feat: Add component WenCai #1739 (#2388)
### What problem does this PR solve?

feat: Add component WenCai #1739

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-09-12 17:51:43 +08:00
Kevin Hu
4730145696
debug backend API for TAB 'search' (#2389)
### What problem does this PR solve?
#2247

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-09-12 17:51:20 +08:00
balibabu
68d0210e92
feat: Use Tree to display the knowledge base list on the search page #2247 (#2385)
### What problem does this PR solve?

feat: Use Tree to display the knowledge base list on the search page
#2247
### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-09-12 17:23:32 +08:00
Fachuan Bai
f8e9a0590f
Common: Support postgreSQL database as the metadata db. (#2357)
https://github.com/infiniflow/ragflow/issues/2356

### What problem does this PR solve?

As title

### Type of change

- [X] New Feature (non-breaking change which adds functionality)
2024-09-12 15:12:39 +08:00
liuhua
ba834aee26
Add a default value for do_refer in Dialog (#2383)
### What problem does this PR solve?

Add a default value for do_refer in Dialog

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-09-12 15:11:57 +08:00
balibabu
983540614e
feat: Cover the entire search page with a background image #2247 (#2381)
### What problem does this PR solve?

feat: Cover the entire search page with a background image #2247

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-09-12 14:20:04 +08:00
JobSmithManipulation
6722b3d558
update sdk document (#2374)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-09-12 14:19:45 +08:00
balibabu
6000c3e304
feat: Catching errors with IndentedTree #2247 (#2380)
### What problem does this PR solve?

feat: Catching errors with IndentedTree #2247

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-09-12 13:34:33 +08:00
Kevin Hu
333608a1d4
add search TAB backend api (#2375)
### What problem does this PR solve?
 #2247

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-09-11 19:49:18 +08:00
balibabu
8052cbc70e
feat: Retrieval chunks by page #2247 (#2373)
### What problem does this PR solve?

feat: Retrieval chunks by page #2247

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-09-11 19:48:11 +08:00
Kevin Hu
b0e0e1fdd0
fix json error (#2372)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-09-11 19:10:49 +08:00
balibabu
8e3228d461
feat: Catch errors in getting mindmap #2247 (#2368)
### What problem does this PR solve?

feat: Catch errors in getting mindmap #2247

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-09-11 16:19:14 +08:00
balibabu
f789098e9f
feat: Proxy the api address to the local nginx address #2350 (#2366)
### What problem does this PR solve?

feat: Proxy the api address to the local nginx address #2350

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-09-11 15:17:52 +08:00
黄腾
d6e6c530d7
fix OpenRouter add bug and the way to add OpenRouter model (#2364)
### What problem does this PR solve?

#2359  fix OpenRouter add bug and the way to add OpenRouter model

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
2024-09-11 15:10:25 +08:00
lidp
22c5affacc
Update templates:Text2sql and DB Description (#2361)
### What problem does this PR solve?


### Type of change

- [x] Refactoring
2024-09-11 12:25:57 +08:00
黄腾
35b7d17d97
fix SILICONFLOW embedding error (#2363)
### What problem does this PR solve?

#2335  fix SILICONFLOW embedding error

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
2024-09-11 12:17:44 +08:00
liuhua
1fc14ff6d4
SDK for session (#2354)
### What problem does this PR solve?

Includes SDK for creating, updating sessions, getting sessions, listing
sessions, and dialogues
#1102 
### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-09-11 12:03:55 +08:00
Wang
7fad48f42c
fix: empty or contains only empty strings. (#2347)
### What problem does this PR solve?
the setting was kept empty for Empty_response. In expectation, this case
should get a response from the LLM if can't find the references from the
knowledgebase.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)


![image](https://github.com/user-attachments/assets/9c382b1d-40f6-43b0-848c-fa6863f9a253)

![image](https://github.com/user-attachments/assets/032d2001-97a2-4faa-91bf-c9c57caf2070)

Co-authored-by: Theta Wang (ncu) <chunshan.connect@gmail.com>
2024-09-11 09:32:12 +08:00
balibabu
77988fe3c2
fix: 123.60.95.134 redirect attack #2350 (#2352)
### What problem does this PR solve?
fix: 123.60.95.134 redirect attack #2350


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-09-10 17:25:15 +08:00
Kevin Hu
cb00f36f62
fix categorize error (#2348)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-09-10 16:03:30 +08:00
balibabu
7edb4ad7dc
fix: Make markdown support line breaks #2315 (#2343)
### What problem does this PR solve?

fix: Make markdown support line breaks #2315

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-09-10 12:15:02 +08:00
Kevin Hu
66c54e75f3
add default model types (#2342)
### What problem does this PR solve?


### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2024-09-10 11:39:44 +08:00
Kevin Hu
f60dfffb4b
add model types to factories API (#2341)
### What problem does this PR solve?

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2024-09-10 11:26:01 +08:00
balibabu
f1ad778250
feat: Dynamically change the background image on the search homepage every day #2247 (#2338)
### What problem does this PR solve?

feat: Dynamically change the background image on the search homepage
every day #2247

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-09-10 11:25:12 +08:00
writinwaters
7c8f159751
Fixed a docusaurus display issue (#2340)
### What problem does this PR solve?


### Type of change

- [x] Documentation Update
2024-09-10 11:24:59 +08:00
writinwaters
c57cc0769b
Added a brief introduction to Agentic RAG (#2331)
### What problem does this PR solve?



### Type of change


- [x] Documentation Update
2024-09-09 20:16:28 +08:00
writinwaters
869df1f704
minor (#2328)
### What problem does this PR solve?



### Type of change

- [x] Documentation Update
2024-09-09 19:30:17 +08:00
balibabu
42eeb38247
feat: Add RetrievalDocuments to SearchPage #2247 (#2327)
### What problem does this PR solve?
feat: Add RetrievalDocuments to SearchPage #2247
feat: Click on the link in the reference to display the pdf drawer #2247

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-09-09 19:20:16 +08:00
lidp
7241c73c7a
Add benchmark ndcg@10 (#2326)
### What problem does this PR solve?

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-09-09 19:20:00 +08:00
LiuHua
336a639164
SDK for session (#2312)
### What problem does this PR solve?

SDK for session
#1102 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Feiue <10215101452@stu.ecun.edu.cn>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-09-09 17:18:08 +08:00
balibabu
ceae4df889
feat: The search box is displayed globally when the page is loaded for the first time #2247 (#2325)
### What problem does this PR solve?

feat: The search box is displayed globally when the page is loaded for
the first time #2247

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-09-09 16:37:04 +08:00
Kevin Hu
884dcbcb7e
updates dead link (#2324)
### What problem does this PR solve?

### Type of change

- [x] Documentation Update
2024-09-09 16:36:52 +08:00
Kevin Hu
4b57177523
fix error for files from filemanager (#2323)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-09-09 14:27:22 +08:00