31 Commits

Author SHA1 Message Date
-LAN-
559ab46ee1
fix: Removes redundant token calculations and updates dependencies
Eliminates unnecessary pre-calculation of token limits and recalculation of max tokens
across multiple app runners, simplifying the logic for prompt handling.

Updates tiktoken library from version 0.8.0 to 0.9.0 for improved tokenization performance.

Increases default token limit in TokenBufferMemory to accommodate larger prompt messages.

These changes streamline the token management process and leverage the latest
improvements in the tiktoken library.

Fixes potential token overflow issues and prepares the system for handling larger
inputs more efficiently.

Relates to internal optimization tasks.

Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-04-28 15:39:12 +08:00
yihong
56e15d09a9
feat: mypy for all type check (#10921) 2024-12-24 18:38:51 +08:00
非法操作
08ac36812b
feat: support LLM process document file (#10966)
Co-authored-by: -LAN- <laipz8200@outlook.com>
2024-11-22 19:32:44 +08:00
-LAN-
c5f7d650b5
feat: Allow using file variables directly in the LLM node and support more file types. (#10679)
Co-authored-by: Joel <iamjoel007@gmail.com>
2024-11-22 16:30:22 +08:00
-LAN-
25ca0278dd
refactor(core): Remove extra_config from File. (#10203) 2024-11-08 18:13:24 +08:00
非法操作
5f12616cb9
fix: file type document is not supported (#9618) 2024-10-22 16:33:50 +08:00
-LAN-
e61752bd3a
feat/enhance the multi-modal support (#8818) 2024-10-21 10:43:49 +08:00
kurokobo
857055b797
fix: remove the latest message from the user that does not have any answer yet (#9297) 2024-10-13 23:25:50 +08:00
Hash Brown
511ffa4698
fix: follow-up (suggested questions) does not refer to the most recen… (#9122) 2024-10-09 21:10:55 +08:00
Hash Brown
8c51d06222
feat: regenerate in Chat, agent and Chatflow app (#7661) 2024-09-22 03:15:11 +08:00
Bowen Liang
a1104ab97e
chore: refurish python code by applying Pylint linter rules (#8322) 2024-09-13 22:42:08 +08:00
Bowen Liang
40fb4d16ef
chore: refurbish Python code by applying refurb linter rules (#8296) 2024-09-12 15:50:49 +08:00
Bowen Liang
2cf1187b32
chore(api/core): apply ruff reformatting (#7624) 2024-09-10 17:00:20 +08:00
Even
c013086e64
fix: next suggest question logic problem (#6451)
Co-authored-by: evenyan <yikun.yan@ubtrobot.com>
2024-07-19 20:26:11 +08:00
takatost
d522308a29
chore: optimize memory fetch performance (#6039) 2024-07-07 08:54:24 +08:00
takatost
23dc6edb99
chore: optimize memory messages fetch count limit (#6021) 2024-07-06 03:25:38 +08:00
takatost
3929d289e0
feat: set default memory messages limit to infinite (#5002) 2024-06-06 17:39:44 +08:00
takatost
d1dbbc1e33
feat: backend model load balancing support (#4927) 2024-06-05 00:13:04 +08:00
takatost
4ad3f2cdc2
fix: image text when retrieve chat histories (#3220) 2024-04-09 15:20:45 +08:00
takatost
7753ba2d37
FEAT: NEW WORKFLOW ENGINE (#3160)
Co-authored-by: Joel <iamjoel007@gmail.com>
Co-authored-by: Yeuoly <admin@srmxy.cn>
Co-authored-by: JzoNg <jzongcode@gmail.com>
Co-authored-by: StyleZhang <jasonapring2015@outlook.com>
Co-authored-by: jyong <jyong@dify.ai>
Co-authored-by: nite-knite <nkCoding@gmail.com>
Co-authored-by: jyong <718720800@qq.com>
2024-04-08 18:51:46 +08:00
Vikey Chen
237bb8514e
replace message content type list to string when file_objs is empty .. (#2745) 2024-03-08 18:46:31 +08:00
Bowen Liang
843280f82b
enhancement: introduce Ruff for Python linter for reordering and removing unused imports with automated pre-commit and sytle check (#2366) 2024-02-06 13:21:13 +08:00
Bowen Liang
cc9e74123c
improve: introduce isort for linting Python imports (#1983) 2024-01-12 12:34:01 +08:00
takatost
d069c668f8
Model Runtime (#1858)
Co-authored-by: StyleZhang <jasonapring2015@outlook.com>
Co-authored-by: Garfield Dai <dai.hai@foxmail.com>
Co-authored-by: chenhe <guchenhe@gmail.com>
Co-authored-by: jyong <jyong@dify.ai>
Co-authored-by: Joel <iamjoel007@gmail.com>
Co-authored-by: Yeuoly <admin@srmxy.cn>
2024-01-02 23:42:00 +08:00
Charlie.Wei
6b57e4e0ff
Fix chitchat lost context (#1828)
Co-authored-by: luowei <glpat-EjySCyNjWiLqAED-YmwM>
Co-authored-by: crazywoola <427733928@qq.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
2023-12-23 10:05:17 +08:00
takatost
41d0a8b295
feat: [backend] vision support (#1510)
Co-authored-by: Garfield Dai <dai.hai@foxmail.com>
2023-11-13 22:05:46 +08:00
Garfield Dai
42a5b3ec17
feat: advanced prompt backend (#1301)
Co-authored-by: takatost <takatost@gmail.com>
2023-10-12 10:13:10 -05:00
takatost
5fa2161b05
feat: server multi models support (#799) 2023-08-12 00:57:00 +08:00
John Wang
7599f79a17
feat: claude api support (#572) 2023-07-17 00:14:19 +08:00
John Wang
3241e4015b
feat: upgrade langchain (#430)
Co-authored-by: jyong <718720800@qq.com>
2023-06-25 16:49:14 +08:00
John Wang
db896255d6 Initial commit 2023-05-15 08:51:32 +08:00