Fakai Zhao de839fc3f0

optimize srv broker and executor logic (#630) ...

### What problem does this PR solve?

Optimize task broker and executor for reduce memory usage and deployment
complexity.

### Type of change
- [x] Performance Improvement
- [x] Refactoring

### Change Log
- Enhance redis utils for message queue(use stream)
- Modify task broker logic via message queue (1.get parse event from
message queue 2.use ThreadPoolExecutor async executor )
- Modify the table column name of document and task (process_duation ->
process_duration maybe just a spelling mistake)
- Reformat some code style(just what i see)
- Add requirement_dev.txt for developer
- Add redis container on docker compose

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>

2024-05-07 11:43:33 +08:00

..

__init__.py

add redis to accelerate access of minio (#482)

2024-04-22 14:11:09 +08:00

api_app.py

refactor code (#583)

2024-04-28 13:19:54 +08:00

chunk_app.py

refine code (#595)

2024-04-28 19:13:33 +08:00

conversation_app.py

conversation API backend update (#360)

2024-04-15 14:43:44 +08:00

dialog_app.py

fix disable and enable llm setting in dialog (#616)

2024-04-30 11:04:14 +08:00

document_app.py

optimize srv broker and executor logic (#630)

2024-05-07 11:43:33 +08:00

file2document_app.py

refactor code (#583)

2024-04-28 13:19:54 +08:00

file_app.py

fix file encoding detection bug (#653)

2024-05-07 10:01:24 +08:00

kb_app.py

refactor code (#583)

2024-04-28 13:19:54 +08:00

llm_app.py

Refactor (#537)

2024-04-25 14:14:28 +08:00

user_app.py

fix bug of file management (#565)

2024-04-26 19:59:21 +08:00