Fix: limit the depth of DFS (#6101)

### What problem does this PR solve?

#6085

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
This commit is contained in:
Kevin Hu 2025-03-14 17:10:38 +08:00 committed by GitHub
parent b5ba8b783a
commit 485bc7d7d6
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194

View File

@ -118,6 +118,8 @@ class RagTokenizer:
def dfs_(self, chars, s, preTks, tkslist):
res = s
if len(tkslist) >= 2048:
return res
# if s > MAX_L or s>= len(chars):
if s >= len(chars):
tkslist.append(preTks)