52 Commits

Author SHA1 Message Date
Kevin Hu
daddfc9e1b
Remove dup gb2312, solve currupt error. (#5326)
### What problem does this PR solve?

#5252 
#5325

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-25 12:22:37 +08:00
Kevin Hu
3444cb15e3
Refine search query. (#5235)
### What problem does this PR solve?

#5173
#5214

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-21 18:32:32 +08:00
Kevin Hu
7b3d700d5f
Apply agentic searching. (#5196)
### What problem does this PR solve?

#5173

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-02-20 17:41:01 +08:00
Kevin Hu
f374dd38b6
Fix divided by zero issue. (#4784)
### What problem does this PR solve?

#4779

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-08 10:36:26 +08:00
Luo Pan
68d46b2a1e
Fix bug in hierarchical_merge function (#4006)
### What problem does this PR solve?

Fix hierarchical_merge function. From idx vs. actual value to actual
value vs. actual value.
Related issue #4003 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: luopan <luopan@example.com>
2024-12-13 08:50:58 +08:00
Zhichang Yu
03f00c9e6f
Rename page_num_list, top_list, position_list (#3940)
### What problem does this PR solve?

Rename page_num_list, top_list, position_list to page_num_int, top_int,
position_int

### Type of change

- [x] Refactoring
2024-12-10 16:32:58 +08:00
Zhichang Yu
7a6bf4326e
Fixed log not displaying (#3946)
### What problem does this PR solve?

Fixed log not displaying

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-12-10 09:36:59 +08:00
Zhichang Yu
0d68a6cd1b
Fix errors detected by Ruff (#3918)
### What problem does this PR solve?

Fix errors detected by Ruff

### Type of change

- [x] Refactoring
2024-12-08 14:21:12 +08:00
Jin Hai
6657ca7cde
Change default error message to English (#3838)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2024-12-04 09:34:49 +08:00
Zhichang Yu
bc701d7b4c
Edit chunk shall update instead of insert it (#3709)
### What problem does this PR solve?

Edit chunk shall update instead of insert it. Close #3679 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-28 13:00:38 +08:00
liuhua
5c59651bda
Fix the bug causing garbled text (#3640)
### What problem does this PR solve?

Fix the bug causing garbled text #3613

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-11-26 12:06:56 +08:00
Zhichang Yu
30f6421760
Use consistent log file names, introduced initLogger (#3403)
### What problem does this PR solve?

Use consistent log file names, introduced initLogger

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [x] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-11-14 17:13:48 +08:00
Zhichang Yu
a2a5631da4
Rework logging (#3358)
Unified all log files into one.

### What problem does this PR solve?

Unified all log files into one.

### Type of change

- [x] Refactoring
2024-11-12 17:35:13 +08:00
Zhichang Yu
f4c52371ab
Integration with Infinity (#2894)
### What problem does this PR solve?

Integration with Infinity

- Replaced ELASTICSEARCH with dataStoreConn
- Renamed deleteByQuery with delete
- Renamed bulk to upsertBulk
- getHighlight, getAggregation
- Fix KGSearch.search
- Moved Dealer.sql_retrieval to es_conn.py


### Type of change

- [x] Refactoring
2024-11-12 14:59:41 +08:00
Kevin Hu
226bdd6e99
add auto keywords and auto-question (#2965)
### What problem does this PR solve?

#2687

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-10-22 13:12:49 +08:00
Kevin Hu
190eea7097
trival (#2808)
### What problem does this PR solve?



### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-10-11 15:33:38 +08:00
Kevin Hu
2d1c83da59
fix LIGHTEN issue (#2806)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-10-11 15:01:27 +08:00
Kevin Hu
9d4bb5767c
make highlight friendly to English (#2417)
### What problem does this PR solve?

#2415

### Type of change

- [x] Performance Improvement
2024-09-13 17:03:51 +08:00
Jin Hai
6b3a40be5c
Format file format from Windows/dos to Unix (#1949)
### What problem does this PR solve?

Related source file is in Windows/DOS format, they are format to Unix
format.

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2024-08-15 09:17:36 +08:00
Kevin Hu
2452c5624f
remove duplicated key in mind map (#1809)
### What problem does this PR solve?

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-08-05 15:57:33 +08:00
Kevin Hu
152072f900
Add graphrag (#1793)
### What problem does this PR solve?

#1594

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-08-02 18:51:14 +08:00
H
79c873344b
Fix docs parser (#1714)
### What problem does this PR solve?

#1711 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-07-26 10:52:56 +08:00
Kevin Hu
c92d334b29
fix bug of regx (#1703)
### What problem does this PR solve?

#1689

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-07-25 14:30:58 +08:00
KevinHuSh
7c9ea5cad9
add interpreter to graph (#1347)
### What problem does this PR solve?

#918 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-07-03 12:15:15 +08:00
KevinHuSh
92e9320657
upgrade laws parser of docx (#1332)
### What problem does this PR solve?


### Type of change

- [x] Refactoring
2024-07-01 15:50:24 +08:00
Zhedong Cen
fc7cc1d36c
Optimize docx handle method in laws parser (#1302)
### What problem does this PR solve?

Optimize docx handle method in laws parser

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-06-28 17:42:59 +08:00
Zhedong Cen
38bd02f402
Support displaying images in the chunks of docx files when using general parser (#1253)
### What problem does this PR solve?

Support displaying images in chunks of docx files when using general
parser

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-06-24 16:29:36 +08:00
Wang Baoling
18f4a6b35c
feat: support json file (#1217)
### What problem does this PR solve?

feat: support json file.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: KevinHuSh <kevinhu.sh@gmail.com>
2024-06-21 10:42:29 +08:00
Zhedong Cen
3c1444ab19
Add docx support for manual parser (#1227)
### What problem does this PR solve?

Add docx support for manual parser

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-06-20 17:03:02 +08:00
Zhedong Cen
90975460af
Add pdf support for QA parser (#1155)
### What problem does this PR solve?

Support extracting questions and answers from PDF files

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-06-14 15:12:39 +08:00
Jin Hai
9ed0e50f6b
Update info (#1005)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2024-05-31 09:53:04 +08:00
KevinHuSh
c6b6c748ae
fix file encoding detection bug (#653)
### What problem does this PR solve?

#651 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-07 10:01:24 +08:00
KevinHuSh
8c07992b6c
refine code (#595)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
2024-04-28 19:13:33 +08:00
KevinHuSh
ed6081845a
Fit a lot of encodings for text file. (#458)
### What problem does this PR solve?

#384

### Type of change

- [x] Performance Improvement
2024-04-19 18:02:53 +08:00
KevinHuSh
24c15daaed
fix es exception (#298)
### What problem does this PR solve?

Issue link:#295
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-10 16:00:48 +08:00
KevinHuSh
23b448cf96
fix docker compose issue (#238)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

Issue link:#[[Link the issue
here](https://github.com/infiniflow/ragflow/issues/226)]

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-07 09:04:32 +08:00
KevinHuSh
71fe314955
refine page ranges (#147) 2024-03-25 13:11:57 +08:00
KevinHuSh
f6aee7f230
add use layout or not option (#145)
* add use layout or not option

* trival
2024-03-22 19:21:09 +08:00
KevinHuSh
6999598101
refine for English corpus (#135) 2024-03-20 16:56:16 +08:00
KevinHuSh
9a843667b3
fix github account login issue (#132) 2024-03-19 15:31:47 +08:00
KevinHuSh
9da671b951
refine manul parser (#131) 2024-03-19 12:26:04 +08:00
KevinHuSh
b89ac3c4be
chage tas execution logic (#103) 2024-03-06 19:16:31 +08:00
KevinHuSh
d7c362f237
adjust hierarchical_merge strategy (#100) 2024-03-06 09:09:16 +08:00
KevinHuSh
8a57f2afd5
change callback strategy, add timezone to docker (#96) 2024-03-05 12:08:41 +08:00
KevinHuSh
7bfaf0df29
fix position extraction bug (#93)
* fix position extraction bug

* remove delimiter for naive parser
2024-03-04 17:08:35 +08:00
KevinHuSh
685b4d8a95
fix table desc bugs, add positions to chunks (#91) 2024-03-04 14:42:26 +08:00
KevinHuSh
8a726fb04b
solve task execution issues (#90) 2024-03-01 19:48:01 +08:00
KevinHuSh
4568a4b2cb
refine admin initialization (#75) 2024-02-27 14:57:34 +08:00
KevinHuSh
7fd1eca582
init README of deepdoc, add picture processer. (#71)
* init README of deepdoc, add picture processer.

* add resume parsing
2024-02-23 18:28:12 +08:00
KevinHuSh
96a1a44cb6 add paper & manual parser (#46) 2024-01-30 18:28:09 +08:00