Add query parts to lamp (#2736)

### What problem does this PR solve?


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
This commit is contained in:
Kevin Hu 2024-10-08 12:53:04 +08:00 committed by GitHub
parent a3ab5ba9ac
commit b3b54680e7
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194

View File

@ -189,6 +189,7 @@ def chat(dialog, messages, stream=True, **kwargs):
used_token_count, msg = message_fit_in(msg, int(max_tokens * 0.97))
assert len(msg) >= 2, f"message_fit_in has bug: {msg}"
prompt = msg[0]["content"]
prompt += "\n\n### Query:\n%s" % " ".join(questions)
if "max_tokens" in gen_conf:
gen_conf["max_tokens"] = min(