15 Commits

Author SHA1 Message Date
QuantumGhost
349c3cf7b8
feat(api): Add image multimodal support for LLMNode (#17372)
Enhance `LLMNode` with multimodal capability, introducing support for
image outputs.

This implementation extracts base64-encoded images from LLM responses,
saves them to the storage service, and records the file metadata in the
`ToolFile` table. In conversations, these images are rendered as
markdown-based inline images.
Additionally, the images are included in the LLMNode's output as
file variables, enabling subsequent nodes in the workflow to utilize them.

To integrate file outputs into workflows, adjustments to the frontend code
are necessary.

For multimodal output functionality, updates to related model configurations
are required. Currently, this capability has been applied exclusively to
Google's Gemini models.

Close #15814.

Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: -LAN- <laipz8200@outlook.com>
2025-04-30 17:28:02 +08:00
ZalterCitty
a1158cc946
fix: Update prompt message content types to use Literal and add union type for content (#17136)
Co-authored-by: 朱庆超 <zhuqingchao@xiaomi.com>
Co-authored-by: crazywoola <427733928@qq.com>
2025-04-22 16:17:55 +08:00
kenwoodjw
3306228840
fix: workflow file add related-id in iteration node (#16255)
Signed-off-by: kenwoodjw <blackxin55+@gmail.com>
2025-03-21 20:57:02 +08:00
yihong
56e15d09a9
feat: mypy for all type check (#10921) 2024-12-24 18:38:51 +08:00
-LAN-
996a9135f6
feat(llm_node): support order in text and files (#11837)
Signed-off-by: -LAN- <laipz8200@outlook.com>
2024-12-20 14:12:50 +08:00
非法操作
c9b4029ce7
chore: the consistency of MultiModalPromptMessageContent (#11721) 2024-12-17 15:01:38 +08:00
非法操作
74fdc16bd1
feat: enhance gemini models (#11497) 2024-12-17 12:05:13 +08:00
非法操作
9e23c3d625
chore: LOCAL_FILE also try to use remote_url as Prompt message (#11443) 2024-12-10 10:56:49 +08:00
-LAN-
c5f7d650b5
feat: Allow using file variables directly in the LLM node and support more file types. (#10679)
Co-authored-by: Joel <iamjoel007@gmail.com>
2024-11-22 16:30:22 +08:00
crazywoola
5656f81bde
Revert "fix the error of unable to retrieve url from file" (#10511) 2024-11-11 08:56:19 +08:00
Xiao Ley
5eb27afd63
fix the error of unable to retrieve url from file (#10498) 2024-11-11 08:47:47 +08:00
-LAN-
25ca0278dd
refactor(core): Remove extra_config from File. (#10203) 2024-11-08 18:13:24 +08:00
非法操作
033ab5490b
feat: support LLM understand video (#9828) 2024-11-08 13:22:52 +08:00
非法操作
ddb960ddfb
feat: support Vectorizer can be used in workflow (#9932) 2024-10-28 16:52:57 +08:00
-LAN-
e61752bd3a
feat/enhance the multi-modal support (#8818) 2024-10-21 10:43:49 +08:00