72 Commits

Author SHA1 Message Date
Kevin Hu
39b96849a9
Fix window size issue of ES. (#5175)
### What problem does this PR solve?

#5152

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-20 12:54:29 +08:00
Kevin Hu
ef95f08c48
Remove redandent code. (#5121)
### What problem does this PR solve?

#5107

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-19 15:46:52 +08:00
Kevin Hu
84b4b38cbb
Remove <think> for exeSql component. (#5069)
### What problem does this PR solve?

#5061
#5067

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-18 13:39:37 +08:00
Kevin Hu
f46448d04c
Remove <think> for KG extraction. (#5027)
### What problem does this PR solve?

#4946

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-17 14:06:06 +08:00
Kevin Hu
7c90b87715
Fix window size of ES issue. (#5026)
### What problem does this PR solve?

#5015

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-17 12:48:56 +08:00
Kevin Hu
f29da49893
Fix keyerror issue while rebuilding graph. (#5022)
### What problem does this PR solve?

#4995

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-17 12:02:44 +08:00
Kevin Hu
1287558f24
Fix xinference chat role order issue. (#4898)
### What problem does this PR solve?

#4831

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-12 13:15:23 +08:00
Kevin Hu
0d3ed37b48
Make the update script shorter. (#4854)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-10 18:18:49 +08:00
Kevin Hu
448fa1c4d4
Robust for abnormal response from LLMs. (#4747)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-06 17:34:53 +08:00
Kevin Hu
2a07eb69a7
Fix too long context issue. (#4735)
### What problem does this PR solve?

#4728

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-06 11:37:23 +08:00
Kevin Hu
283d036cba
Fitin for infinity. (#4722)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
2025-02-05 16:47:05 +08:00
Kevin Hu
6f30397bb5
Infinity adapt to graphrag. (#4663)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-01-27 18:35:18 +08:00
Kevin Hu
42f7261509
Fix param error. (#4645)
### What problem does this PR solve?

#4633

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-01-26 10:40:02 +08:00
Kevin Hu
71c132f76d
Make infinity adapt (#4635)
### What problem does this PR solve?

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-01-24 17:45:04 +08:00
Kevin Hu
86892959a0
Rebuild graph when it's out of time. (#4607)
### What problem does this PR solve?

#4543

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring
2025-01-23 17:26:20 +08:00
Kevin Hu
dd0ebbea35
Light GraphRAG (#4585)
### What problem does this PR solve?

#4543

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-01-22 19:43:14 +08:00
Kevin Hu
c5da3cdd97
Tagging (#4426)
### What problem does this PR solve?

#4367

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-01-09 17:07:21 +08:00
Kevin Hu
8fb18f37f6
Code refactor. (#4291)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
2024-12-30 18:38:51 +08:00
Kevin Hu
b635002666
Fix duplicated communitiy (#4187)
### What problem does this PR solve?

#4180

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-12-23 17:07:12 +08:00
Kevin Hu
6379a934ff
Fix redis get error. (#4140)
### What problem does this PR solve?

#4126
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-12-20 10:39:50 +08:00
Kevin Hu
fddac1345d
Fix raptor resuable issue. (#4063)
### What problem does this PR solve?

#4045

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-12-17 15:28:35 +08:00
Kevin Hu
cb6e9ce164
Cache the result from llm for graphrag and raptor (#4051)
### What problem does this PR solve?

#4045

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-12-17 09:48:03 +08:00
Zhichang Yu
03f00c9e6f
Rename page_num_list, top_list, position_list (#3940)
### What problem does this PR solve?

Rename page_num_list, top_list, position_list to page_num_int, top_int,
position_int

### Type of change

- [x] Refactoring
2024-12-10 16:32:58 +08:00
Zhichang Yu
1254ecf445
Added static check at PR CI (#3921)
### What problem does this PR solve?

Added static check at PR CI

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring
2024-12-08 21:23:51 +08:00
Zhichang Yu
0d68a6cd1b
Fix errors detected by Ruff (#3918)
### What problem does this PR solve?

Fix errors detected by Ruff

### Type of change

- [x] Refactoring
2024-12-08 14:21:12 +08:00
Zhichang Yu
90ffcb4ddb
Fix graphrag + infinity bugs (#3681)
### What problem does this PR solve?

Fix graphrag + infinity bugs

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-27 12:45:43 +08:00
Kevin Hu
0891a393d7
Let ThreadPool exit gracefully. (#3653)
### What problem does this PR solve?

#3646

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-26 16:31:07 +08:00
Kevin Hu
58a2200b80
Fix: search issue for graphrag (#3546)
### What problem does this PR solve?

#3538

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-21 11:13:29 +08:00
Jin Hai
2044bb0039 Fix bugs (#3502)
### What problem does this PR solve?

1. Remove unused code
2. Fix type mismatch, in nlp search and infinity search interface
3. Fix chunk list, get all chunks of this user.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: jinhai <haijin.chn@gmail.com>
2024-11-20 20:52:23 +08:00
roc king
bcb6f7168f
[bug]import typing.cast for leiden.py (#3493)
### What problem does this PR solve?

leiden alg throws exception for lack func cast definition

### Type of change

- Bug Fix (non-breaking change which fixes an issue)
2024-11-19 18:42:30 +08:00
Zhichang Yu
dec9b3e540
Fix logs. Use dict.pop instead of del. Close #3473 (#3484)
### What problem does this PR solve?

Fix logs. Use dict.pop instead of del.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-19 14:15:25 +08:00
Zhichang Yu
4413683898
Introduced beartype (#3460)
### What problem does this PR solve?

Introduced [beartype](https://github.com/beartype/beartype) for runtime
type-checking.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-11-18 17:38:17 +08:00
Kevin Hu
a1d01a1b2f
enlarge the default token length of RAPTOR summarization (#3454)
### What problem does this PR solve?

#3426

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-11-18 10:15:26 +08:00
Jin Hai
1e90a1bf36
Move settings initialization after module init phase (#3438)
### What problem does this PR solve?

1. Module init won't connect database any more.
2. Config in settings need to be used with settings.CONFIG_NAME

### Type of change

- [x] Refactoring

Signed-off-by: jinhai <haijin.chn@gmail.com>
2024-11-15 17:30:56 +08:00
Zhichang Yu
30f6421760
Use consistent log file names, introduced initLogger (#3403)
### What problem does this PR solve?

Use consistent log file names, introduced initLogger

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [x] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-11-14 17:13:48 +08:00
Zhichang Yu
a2a5631da4
Rework logging (#3358)
Unified all log files into one.

### What problem does this PR solve?

Unified all log files into one.

### Type of change

- [x] Refactoring
2024-11-12 17:35:13 +08:00
Zhichang Yu
f4c52371ab
Integration with Infinity (#2894)
### What problem does this PR solve?

Integration with Infinity

- Replaced ELASTICSEARCH with dataStoreConn
- Renamed deleteByQuery with delete
- Renamed bulk to upsertBulk
- getHighlight, getAggregation
- Fix KGSearch.search
- Moved Dealer.sql_retrieval to es_conn.py


### Type of change

- [x] Refactoring
2024-11-12 14:59:41 +08:00
Kevin Hu
88072b1e90
fix: double brace issue (#3328)
### What problem does this PR solve?

#3299

### Type of change

- [x] Performance Improvement
2024-11-11 12:07:02 +08:00
Kevin Hu
ff2bbb487f
fix index of range (#3279)
### What problem does this PR solve?

#3273

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-08 09:58:11 +08:00
Kevin Hu
8d9238db14
fix es search parameter error (#3169)
### What problem does this PR solve?

#3151

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-04 09:53:41 +08:00
chongchuanbing
ac26d09a59
Feature/feat1017 (#2872)
### What problem does this PR solve?

1. fix: mid map show error in knowledge graph, juse because
```@antv/g6```version changed
2. feat: concurrent threads configuration support in graph extractor
3. fix: used tokens update failed for tenant
4. feat: timeout configuration support for llm
5. fix: regex error in graph extractor
6. feat: qwen rerank(```gte-rerank```) support
7. fix: timeout deal in knowledge graph index process. Now chat by
stream output, also, it is configuratable.
8. feat: ```qwen-long``` model configuration

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: chongchuanbing <chongchuanbing@gmail.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-10-21 12:11:08 +08:00
lidp
d6db805885
Refactoring entity_resolution (#2692)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
2024-09-30 17:18:09 +08:00
yqkcn
ae5a877ed4
Simplify the usage of dict (#2681)
### What problem does this PR solve?
Simplify the usage of dictionaries

### Type of change

- [x] Refactoring

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-09-30 16:54:25 +08:00
yqkcn
9945988e44
format mind_map_extractor code (#2686)
### What problem does this PR solve?

format mind_map_extractor code

### Type of change

- [x] Refactoring
2024-09-30 16:29:15 +08:00
yqkcn
604061c4a5
Fix mutable default argument (#2635)
### What problem does this PR solve?

The default value of Python function parameters cannot be mutable.
Modifying this parameter inside the function will permanently change the
default value

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-09-29 10:16:00 +08:00
yqkcn
34abcf7704
style: fix typo and format code (#2618)
### What problem does this PR solve?

- Fix typo
- Remove unused import
- Format code

### Type of change

- [x] Other (please describe): typo and format
2024-09-27 13:17:25 +08:00
yqkcn
4c0b79c4f6
remove repeat func (#2619)
### What problem does this PR solve?

- remove repeat func

### Type of change

- [x] Other (please describe): remove repeat func
2024-09-27 13:15:26 +08:00
Kevin Hu
3f571a13c2
fix empty children in mindmap (#2418)
### What problem does this PR solve?

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-09-13 17:19:47 +08:00
Kevin Hu
4730145696
debug backend API for TAB 'search' (#2389)
### What problem does this PR solve?
#2247

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-09-12 17:51:20 +08:00
Kevin Hu
333608a1d4
add search TAB backend api (#2375)
### What problem does this PR solve?
 #2247

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-09-11 19:49:18 +08:00