135 Commits

Author SHA1 Message Date
KevinHuSh
a2a6a35e94
fix doc number miss-match issue (#822)
### What problem does this PR solve?

#620 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-17 15:35:09 +08:00
KevinHuSh
9e5d501e83
fix data init error (#820)
### What problem does this PR solve?

#810 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-17 14:33:19 +08:00
KevinHuSh
e73ce39b66
Add 2 embeding models from OpenAI (#812)
### What problem does this PR solve?

#810 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-17 08:51:29 +08:00
KevinHuSh
95f809187e
add stream chat (#811)
### What problem does this PR solve?

#709 
### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-16 20:14:53 +08:00
KevinHuSh
d6772f5dd7
add version (#807)
### What problem does this PR solve?
#709 
### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-16 16:17:48 +08:00
KevinHuSh
1c4e92ed35
Knowledge base search is case sensitive (#797)
### What problem does this PR solve?
#793 
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-16 09:00:12 +08:00
KevinHuSh
d8c080ee52
fix bugs in searching file using keywords (#780)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-15 12:51:57 +08:00
KevinHuSh
aa1c915d6e
support gpt-4o (#773)
### What problem does this PR solve?
#771 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-15 11:16:08 +08:00
KevinHuSh
04a9e95161
let file in knowledgebases visible in file manager (#714)
### What problem does this PR solve?

Let file in knowledgebases visible in file manager.
#162 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-11 16:04:28 +08:00
KevinHuSh
7ddb2f19be
make sure to raise exception if redis is not there (#674)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
2024-05-08 15:20:45 +08:00
KevinHuSh
eb27a4309e
add support for deepseek (#668)
### What problem does this PR solve?

#666 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-08 10:30:02 +08:00
KevinHuSh
8d6d7f6887
fix task losting isssue (#665)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-07 20:46:45 +08:00
KevinHuSh
a5aed2412f
fix bugs (#662)
### What problem does this PR solve?

Fix import error for task_service.py

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-07 16:41:56 +08:00
Fakai Zhao
de839fc3f0
optimize srv broker and executor logic (#630)
### What problem does this PR solve?

Optimize task broker and executor for reduce memory usage and deployment
complexity.

### Type of change
- [x] Performance Improvement
- [x] Refactoring

### Change Log
- Enhance redis utils for message queue(use stream)
- Modify task broker logic via message queue (1.get parse event from
message queue 2.use ThreadPoolExecutor async executor )
- Modify the table column name of document and task (process_duation ->
process_duration maybe just a spelling mistake)
- Reformat some code style(just what i see)
- Add requirement_dev.txt for developer
- Add redis container on docker compose

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-05-07 11:43:33 +08:00
Moonlit
ca5acc151a
Refactor: Use TaskStatus enum for task status handling (#646)
### What problem does this PR solve?

This commit changes the status 'not started' from being hard-coded to
being maintained by the TaskStatus enum. This enhancement ensures
consistency across the codebase and improves maintainability.

### Type of change

- [x] Refactoring
2024-05-06 18:39:17 +08:00
KevinHuSh
5f03a4de11
remove redis (#629)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
2024-04-30 19:00:41 +08:00
KevinHuSh
8c07992b6c
refine code (#595)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
2024-04-28 19:13:33 +08:00
KevinHuSh
9d60a84958
refactor code (#583)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
2024-04-28 13:19:54 +08:00
KevinHuSh
038822f3bd
make cites in conversation API configurable (#576)
### What problem does this PR solve?

#566 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-28 11:56:17 +08:00
KevinHuSh
944776f207
fix bug about fetching file from minio (#574)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-28 09:57:40 +08:00
KevinHuSh
ab06f502d7
fix bug of file management (#565)
### What problem does this PR solve?

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-26 19:59:21 +08:00
KevinHuSh
f69ff39fa0
add file management feature (#560)
### What problem does this PR solve?

### Type of change

- [x] Documentation Update
2024-04-26 17:21:53 +08:00
KevinHuSh
b75d75e995
fix youdao bug (#551)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-26 09:58:22 +08:00
chrysanthemum-boy
26003b5076
Add upload file by knowledge base name API. (#539)
### What problem does this PR solve?
Add upload file by knowledge base name API.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update

---------

Co-authored-by: chrysanthemum-boy <fannc@qq.com>
2024-04-25 15:10:19 +08:00
KevinHuSh
66f8d35632
Refactor (#537)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
2024-04-25 14:14:28 +08:00
KevinHuSh
ed6081845a
Fit a lot of encodings for text file. (#458)
### What problem does this PR solve?

#384

### Type of change

- [x] Performance Improvement
2024-04-19 18:02:53 +08:00
KevinHuSh
453c29170f
make sure the models will not be load twice (#422)
### What problem does this PR solve?

#381 
### Type of change

- [x] Refactoring
2024-04-18 09:37:23 +08:00
KevinHuSh
03f8b01b3b
fix bug for fasetembed (#392)
### What problem does this PR solve?

Issue link:#325

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-16 19:12:12 +08:00
KevinHuSh
890561703b
Add bce-embedding and fastembed (#383)
### What problem does this PR solve?


Issue link:#326

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-04-16 16:42:19 +08:00
Anush
826ad6a33a
feat: FastEmbed embedding support (#291)
### Description

Following up on https://github.com/infiniflow/ragflow/pull/275, this PR
adds support for FastEmbed model configurations.

The options are not exhaustive. You can find the full list
[here](https://qdrant.github.io/fastembed/examples/Supported_Models/).

P.S. I ran into OOM issues when building the image.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: KevinHuSh <kevinhu.sh@gmail.com>
2024-04-15 15:58:06 +08:00
KevinHuSh
c39b751600
conversation API backend update (#360)
### What problem does this PR solve?


Issue link:#345

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-04-15 14:43:44 +08:00
KevinHuSh
4608cccd05
add new model gpt-3-turbo (#352)
### What problem does this PR solve?


Issue link:#351

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-04-15 08:58:42 +08:00
KevinHuSh
e20207101a
fix wrong log printting (#330)
### What problem does this PR solve?

Issue link:#325

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-12 09:08:08 +08:00
KevinHuSh
91068edf16
Support Xinference (#320)
### What problem does this PR solve?

Issue link:#299

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-04-11 18:22:25 +08:00
KevinHuSh
f6c7204002
refine log format (#312)
### What problem does this PR solve?

Issue link:#264
### Type of change


- [x] Documentation Update
- [x] Refactoring
2024-04-11 10:13:43 +08:00
Jin Hai
0b2808f990
Update default language to English (#309)
### What problem does this PR solve?

Update default language to English

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2024-04-11 08:17:15 +08:00
KevinHuSh
3708b97db9
Support Ollama (#261)
### What problem does this PR solve?

Issue link:#221

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-04-08 19:20:57 +08:00
KevinHuSh
591202721d
fix bug about field misspell (#243)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

Issue link:#[[Link the issue
here](https://github.com/infiniflow/ragflow/issues/242)]

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-07 14:18:01 +08:00
KevinHuSh
23b448cf96
fix docker compose issue (#238)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

Issue link:#[[Link the issue
here](https://github.com/infiniflow/ragflow/issues/226)]

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-07 09:04:32 +08:00
KevinHuSh
080cbd96e8
fix KB config reload bug (#213)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-03 11:55:43 +08:00
KevinHuSh
392e515c3f
fix bug about reload knowledgebase configuration reloading (#210)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

Issue link:#[[Link the issue
here](https://github.com/infiniflow/ragflow/issues/209)]

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-03 11:00:50 +08:00
KevinHuSh
8c1d31c6e5
fix set_api_key bug (#191)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

Issue link:#[Link the issue here]

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Breaking Change (fix or feature that could cause existing
functionality not to work as expected)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Test cases
- [ ] Python SDK impacted, Need to update PyPI
- [ ] Other (please describe):
2024-04-01 19:03:13 +08:00
KevinHuSh
0a9f589f9b
change defualt language to English (#185) 2024-04-01 10:54:01 +08:00
KevinHuSh
121c7a5681
refine error response, add set api-key MD (#178) 2024-03-31 19:09:42 +08:00
KevinHuSh
e876f58b4c
refine readme (#170) 2024-03-29 14:38:15 +08:00
KevinHuSh
05298d5626
refine README (#169) 2024-03-29 10:48:29 +08:00
KevinHuSh
38e5737067
add base url for OpenAI (#166) 2024-03-28 19:15:16 +08:00
KevinHuSh
fd7fcb5baf
apply pep8 formalize (#155) 2024-03-27 11:33:46 +08:00
KevinHuSh
71fe314955
refine page ranges (#147) 2024-03-25 13:11:57 +08:00
KevinHuSh
f6aee7f230
add use layout or not option (#145)
* add use layout or not option

* trival
2024-03-22 19:21:09 +08:00