dashi6174
9a01d1b876
The default max tokens of 215 is too small, answers are often cut off.I will modify it to 512 to address this issue. ( #845 )
...
### What problem does this PR solve?
### Type of change
- [x] Refactoring
2024-05-20 17:25:19 +08:00
KevinHuSh
2b36283712
fix english query bug ( #840 )
...
### What problem does this PR solve?
#834
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-20 12:23:51 +08:00
KevinHuSh
6683179d6a
fix bug about removing KB. ( #839 )
...
### What problem does this PR solve?
#838
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-20 09:23:57 +08:00
KevinHuSh
673a28e492
fix bug of chat without stream ( #830 )
...
### What problem does this PR solve?
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-17 20:03:00 +08:00
KevinHuSh
b3c923da6b
add doc ids in API: completion ( #827 )
...
### What problem does this PR solve?
#808
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2024-05-17 17:51:54 +08:00
KevinHuSh
a1586e0af9
correct mismatched kb doc number ( #826 )
...
### What problem does this PR solve?
#620
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-17 17:27:39 +08:00
KevinHuSh
f6a599461f
fix zhipuAI stream issue ( #825 )
...
### What problem does this PR solve?
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-17 17:07:33 +08:00
GYH
081f922ee6
0517 list chunks ( #821 )
...
### What problem does this PR solve?
#717
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2024-05-17 15:58:05 +08:00
KevinHuSh
a2a6a35e94
fix doc number miss-match issue ( #822 )
...
### What problem does this PR solve?
#620
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-17 15:35:09 +08:00
KevinHuSh
9e5d501e83
fix data init error ( #820 )
...
### What problem does this PR solve?
#810
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-17 14:33:19 +08:00
KevinHuSh
c3bc72dfd9
fix too large thumbnail issue ( #817 )
...
### What problem does this PR solve?
#709
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-17 14:04:21 +08:00
LiYongHui
2dd705fe68
feat: add feishu oauth ( #815 )
...
### What problem does this PR solve?
The back-end code adds Feishu oauth
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
Co-authored-by: yonghui li <yonghui.li@bondex.com.cn>
2024-05-17 13:47:05 +08:00
KevinHuSh
d1614107e2
fix stream chat for ollama ( #816 )
...
### What problem does this PR solve?
#709
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-17 12:07:00 +08:00
KevinHuSh
e73ce39b66
Add 2 embeding models from OpenAI ( #812 )
...
### What problem does this PR solve?
#810
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2024-05-17 08:51:29 +08:00
KevinHuSh
95f809187e
add stream chat ( #811 )
...
### What problem does this PR solve?
#709
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2024-05-16 20:14:53 +08:00
KevinHuSh
d6772f5dd7
add version ( #807 )
...
### What problem does this PR solve?
#709
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2024-05-16 16:17:48 +08:00
KevinHuSh
7b144cc086
fix: can't capitalize file or folder name ( #798 )
...
### What problem does this PR solve?
#792
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-16 09:10:29 +08:00
KevinHuSh
1c4e92ed35
Knowledge base search is case sensitive ( #797 )
...
### What problem does this PR solve?
#793
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-16 09:00:12 +08:00
dashi6174
6ff63ee2ba
Support for code files parse ( #789 )
...
### What problem does this PR solve?
_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2024-05-15 16:34:28 +08:00
KevinHuSh
d8c080ee52
fix bugs in searching file using keywords ( #780 )
...
### What problem does this PR solve?
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-15 12:51:57 +08:00
GYH
626ace8639
Updated document upload method ( #777 )
...
### What problem does this PR solve?
api_app.py
/document/upload
add two non mandatory parameters
parser_id:
[naive,qaresume,manual,table,paper,book,laws,presentation,picture,one]
run: 1
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2024-05-15 12:22:11 +08:00
KevinHuSh
aa1c915d6e
support gpt-4o ( #773 )
...
### What problem does this PR solve?
#771
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2024-05-15 11:16:08 +08:00
Jin Hai
d7744f5870
Refactor method name ( #760 )
...
### What problem does this PR solve?
#757
### Type of change
- [x] Refactoring
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2024-05-14 14:48:15 +08:00
KevinHuSh
aa03dfa453
fix bug of get file ( #746 )
...
### What problem does this PR solve?
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-13 14:02:38 +08:00
Sawyer
69b7c61498
fix: typo in user_app.py ( #740 )
...
### What problem does this PR solve?
_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._
### Type of change
- [x] Bug Fix (non-breaking change
- [x] Other (please describe): Fix typo
2024-05-13 09:25:45 +08:00
KevinHuSh
04a9e95161
let file in knowledgebases visible in file manager ( #714 )
...
### What problem does this PR solve?
Let file in knowledgebases visible in file manager.
#162
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2024-05-11 16:04:28 +08:00
KevinHuSh
7ddb2f19be
make sure to raise exception if redis is not there ( #674 )
...
### What problem does this PR solve?
### Type of change
- [x] Refactoring
2024-05-08 15:20:45 +08:00
KevinHuSh
eb27a4309e
add support for deepseek ( #668 )
...
### What problem does this PR solve?
#666
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2024-05-08 10:30:02 +08:00
KevinHuSh
8d6d7f6887
fix task losting isssue ( #665 )
...
### What problem does this PR solve?
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-07 20:46:45 +08:00
KevinHuSh
a5aed2412f
fix bugs ( #662 )
...
### What problem does this PR solve?
Fix import error for task_service.py
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-07 16:41:56 +08:00
Fakai Zhao
de839fc3f0
optimize srv broker and executor logic ( #630 )
...
### What problem does this PR solve?
Optimize task broker and executor for reduce memory usage and deployment
complexity.
### Type of change
- [x] Performance Improvement
- [x] Refactoring
### Change Log
- Enhance redis utils for message queue(use stream)
- Modify task broker logic via message queue (1.get parse event from
message queue 2.use ThreadPoolExecutor async executor )
- Modify the table column name of document and task (process_duation ->
process_duration maybe just a spelling mistake)
- Reformat some code style(just what i see)
- Add requirement_dev.txt for developer
- Add redis container on docker compose
---------
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-05-07 11:43:33 +08:00
KevinHuSh
c6b6c748ae
fix file encoding detection bug ( #653 )
...
### What problem does this PR solve?
#651
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-07 10:01:24 +08:00
Moonlit
ca5acc151a
Refactor: Use TaskStatus enum for task status handling ( #646 )
...
### What problem does this PR solve?
This commit changes the status 'not started' from being hard-coded to
being maintained by the TaskStatus enum. This enhancement ensures
consistency across the codebase and improves maintainability.
### Type of change
- [x] Refactoring
2024-05-06 18:39:17 +08:00
KevinHuSh
5f03a4de11
remove redis ( #629 )
...
### What problem does this PR solve?
### Type of change
- [x] Refactoring
2024-04-30 19:00:41 +08:00
KevinHuSh
cab274f560
remove PyMuPDF ( #618 )
...
### What problem does this PR solve?
#613
### Type of change
- [x] Other (please describe):
2024-04-30 12:38:09 +08:00
KevinHuSh
674b3aeafd
fix disable and enable llm setting in dialog ( #616 )
...
### What problem does this PR solve?
#614
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-30 11:04:14 +08:00
KevinHuSh
2af74cc494
refine docker layers ( #606 )
...
### What problem does this PR solve?
### Type of change
- [x] Performance Improvement
2024-04-29 17:57:40 +08:00
KevinHuSh
6874c6f3a7
refine document upload ( #602 )
...
### What problem does this PR solve?
#567
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-29 15:45:08 +08:00
KevinHuSh
8c07992b6c
refine code ( #595 )
...
### What problem does this PR solve?
### Type of change
- [x] Refactoring
2024-04-28 19:13:33 +08:00
KevinHuSh
9d60a84958
refactor code ( #583 )
...
### What problem does this PR solve?
### Type of change
- [x] Refactoring
2024-04-28 13:19:54 +08:00
KevinHuSh
038822f3bd
make cites in conversation API configurable ( #576 )
...
### What problem does this PR solve?
#566
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-28 11:56:17 +08:00
KevinHuSh
944776f207
fix bug about fetching file from minio ( #574 )
...
### What problem does this PR solve?
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-28 09:57:40 +08:00
KevinHuSh
ab06f502d7
fix bug of file management ( #565 )
...
### What problem does this PR solve?
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-26 19:59:21 +08:00
KevinHuSh
84b39c60f6
fix rename bug ( #562 )
...
### What problem does this PR solve?
fix rename file bugs
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-26 18:55:21 +08:00
KevinHuSh
f69ff39fa0
add file management feature ( #560 )
...
### What problem does this PR solve?
### Type of change
- [x] Documentation Update
2024-04-26 17:21:53 +08:00
KevinHuSh
b75d75e995
fix youdao bug ( #551 )
...
### What problem does this PR solve?
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-26 09:58:22 +08:00
chrysanthemum-boy
26003b5076
Add upload file by knowledge base name API. ( #539 )
...
### What problem does this PR solve?
Add upload file by knowledge base name API.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
---------
Co-authored-by: chrysanthemum-boy <fannc@qq.com>
2024-04-25 15:10:19 +08:00
KevinHuSh
66f8d35632
Refactor ( #537 )
...
### What problem does this PR solve?
### Type of change
- [x] Refactoring
2024-04-25 14:14:28 +08:00
chrysanthemum-boy
72384b191d
Add .doc file parser. ( #497 )
...
### What problem does this PR solve?
Add `.doc` file parser, using tika.
```
pip install tika
```
```
from tika import parser
from io import BytesIO
def extract_text_from_doc_bytes(doc_bytes):
file_like_object = BytesIO(doc_bytes)
parsed = parser.from_buffer(file_like_object)
return parsed["content"]
```
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
---------
Co-authored-by: chrysanthemum-boy <fannc@qq.com>
2024-04-23 15:31:43 +08:00
KevinHuSh
b8e58fe27a
add redis to accelerate access of minio ( #482 )
...
### What problem does this PR solve?
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2024-04-22 14:11:09 +08:00
