1048 Commits

Author SHA1 Message Date
Kevin Hu
77f0fb03e3
fix parameter error (#1925)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-08-13 11:42:38 +08:00
Kevin Hu
da2d8b8267
boost paralal of graphrag (#1924)
### What problem does this PR solve?


### Type of change

- [x] Performance Improvement
2024-08-13 11:21:30 +08:00
balibabu
b75115264d
fix: Error on chat api,<BadRequestKeyError '400: Bad Request'> #1918 (#1923)
### What problem does this PR solve?

fix: Error on chat api,<BadRequestKeyError '400: Bad Request'> #1918

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-08-13 10:51:51 +08:00
Kevin Hu
8badf3f423
fix api argument error (#1920)
### What problem does this PR solve?

#1918 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-08-13 10:15:55 +08:00
balibabu
eb8feaf20a
feat: Added explanation on the parsing method of knowledge graph #1594 (#1916)
### What problem does this PR solve?

feat: Added explanation on the parsing method of knowledge graph #1594

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-08-12 18:56:01 +08:00
balibabu
936d8ab7dd
fix: Chunks cannot be deleted #1912 (#1913)
### What problem does this PR solve?

fix: Chunks cannot be deleted #1912

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-08-12 17:29:44 +08:00
黄腾
68d1315079
add support for novita.ai (#1910)
### What problem does this PR solve?

#1853  add support for novita.ai

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
2024-08-12 17:26:26 +08:00
balibabu
6baba54e9e
feat: Add delimiter field to naive parsing method #1909 (#1911)
### What problem does this PR solve?

feat: Add delimiter field to naive parsing method #1909
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-08-12 15:53:25 +08:00
balibabu
ad48e8d915
fix: When the component id is very long, the delete button of generate will be hidden #1906 (#1907)
### What problem does this PR solve?
fix: When the component id is very long, the delete button of generate
will be hidden #1906

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-08-12 15:42:00 +08:00
Kevin Hu
cafdee536f
add sql to naive parser (#1908)
### What problem does this PR solve?


### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2024-08-12 15:29:33 +08:00
H
cd861e3653
Fix agent non stream (#1904)
### What problem does this PR solve?

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-08-12 12:00:59 +08:00
黄腾
e9e39d57ce
add support for Upstage (#1902)
### What problem does this PR solve?

#1853  add support for Upstage

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
2024-08-12 11:06:25 +08:00
黄腾
94cb66ba80
add support for TogetherAI (#1890)
### What problem does this PR solve?

#1853 add support for TogetherAI

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-08-12 10:15:21 +08:00
黄腾
9a6dc89156
add support for PerfXCloud (#1883)
### What problem does this PR solve?

#1853  add support for PerfXCloud

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
2024-08-12 10:11:50 +08:00
H
fdd5b1b8cf
Fix token list , stats in api app.py (#1896)
### What problem does this PR solve?

#1842 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-08-09 19:03:01 +08:00
balibabu
827042f72b
feat: Expose the agent's chat window to third parties #1842 (#1897)
### What problem does this PR solve?

feat: Expose the agent's chat window to third parties #1842
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-08-09 18:59:16 +08:00
Kevin Hu
37be0ff3d3
remove qwen-v1-max (#1895)
### What problem does this PR solve?

#1748

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-08-09 18:58:58 +08:00
Kevin Hu
a313b77cdd
rm qwen-v1-max (#1894)
### What problem does this PR solve?

#1748

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-08-09 18:41:44 +08:00
Kevin Hu
4fecc2fae6
fix no modual error (#1892)
### What problem does this PR solve?

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-08-09 17:02:18 +08:00
H
ff75008801
Add agent api (#1888)
### What problem does this PR solve?

#1842 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-08-09 16:54:29 +08:00
Kevin Hu
e3cf14a3c9
add function: upload and parse (#1889)
### What problem does this PR solve?

#1880
### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-08-09 16:20:02 +08:00
黄腾
6529c764c9
fix: model type only support single value (#1884)
### What problem does this PR solve?

 model type only support single value
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
2024-08-09 15:24:41 +08:00
黄腾
44184d12a8
add using jina deploy local llm in deploy_local_llm.mdx (#1872)
### What problem does this PR solve?

add using jina deploy local llm in deploy_local_llm.mdx

### Type of change

- [x] Documentation Update

---------

Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
2024-08-09 15:24:09 +08:00
balibabu
8779aa1986
feat: Add component QWeather #1739 (#1881)
### What problem does this PR solve?

feat: Add component QWeather #1739

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-08-09 13:40:13 +08:00
黄腾
411c645134
Enhance the robustness of the code (#1879)
### What problem does this PR solve?

Enhance the robustness of the code

### Type of change

- [x] Refactoring

---------

Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
2024-08-09 10:18:08 +08:00
balibabu
afccbc88e8
feat: Replace the LocalAi icon #762 (#1875)
### What problem does this PR solve?

feat:  Replace the LocalAi icon #762
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-08-09 10:09:10 +08:00
Jin Hai
33e78cf638
Update version and format (#1878)
### What problem does this PR solve?

1. Update the change line to Unix style
2. Update version info.

### Type of change

- [x] Documentation Update

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2024-08-08 23:59:26 +08:00
H
193aa3ba88
Add component qweather (#1873)
### What problem does this PR solve?

#1739 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-08-08 17:57:46 +08:00
balibabu
ffb3fc4bf5
feat: Add component BaiduFanyi #1739 (#1874)
### What problem does this PR solve?

feat: Add component BaiduFanyi #1739

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-08-08 17:52:45 +08:00
balibabu
6ccfbca204
feat: Add component GitHub #1739 (#1871)
### What problem does this PR solve?

feat: Add component GitHub #1739

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-08-08 16:03:21 +08:00
balibabu
439da32234
feat: Add component DeepL #1739 (#1870)
### What problem does this PR solve?

feat: Add component DeepL #1739

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-08-08 15:08:31 +08:00
Kevin Hu
db8f83104f
less text, better extraction (#1869)
### What problem does this PR solve?

#1861

### Type of change

- [x] Refactoring
2024-08-08 13:56:30 +08:00
Tong Liu
f43db8bc51
fix code injection (#1868)
### What problem does this PR solve?

fix code injection in https://github.com/infiniflow/ragflow/issues/1860,
developers can have a check to see if the fix works as expected.

### Type of change

Vulnerability Fix
2024-08-08 13:44:55 +08:00
H
ce587cba56
Add GitHub, deepl, baidu-fanyi (#1857)
### What problem does this PR solve?

#1739 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-08-08 12:39:19 +08:00
Ding Jiatong
5164835681
add support for gpt-4o-mini (#1827)
### What problem does this PR solve?

add support for gpt-4o-mini

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-08-08 12:30:40 +08:00
Jason Lee
c981a57616
fix: Reference markers in the context may be carried over into the next answer (#1855)
The answer in the context carries reference markers and passes them to
the large model, which may include the markers in the new answer,
leading to abnormal reference points.
```
 {'role': 'assistant', 'content': '设置在地下或半地下空间 ##0$$。'}
```

![image](https://github.com/user-attachments/assets/bcfdb3fc-7b54-44cb-ab70-2f9b715d06b8)

### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- Bug Fix (non-breaking change which fixes an issue)
2024-08-08 12:25:57 +08:00
黄腾
c7d00c2272
remove jina pack in requirement file to fix package conflict (#1867)
### What problem does this PR solve?

#1824 #1822 remove jina pack in requirement file to fix package conflict

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
2024-08-08 12:10:14 +08:00
黄腾
aed1bbbcaa
add supprot for lepton (#1866)
### What problem does this PR solve?

add supprot for lepton
#1853

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
2024-08-08 12:09:50 +08:00
Kung Quang
19ded65c66
Fix a "TypeError: expected string or buffer bug" in docx files extracted using Knowledge Graph.#1859 (#1865)
### What problem does this PR solve?

Fix a "TypeError: expected string or buffer bug" in docx files extracted
using Knowledge Graph. #1859
```
Traceback (most recent call last):
  File "//Users/XXX/ragflow/rag/svr/task_executor.py", line 149, in build
    cks = chunker.chunk(row["name"], binary=binary, from_page=row["from_page"],
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/XXX/ragflow/rag/app/knowledge_graph.py", line 18, in chunk
    chunks = build_knowlege_graph_chunks(tenant_id, sections, callback,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/XXX/ragflow/graphrag/index.py", line 87, in build_knowlege_graph_chunks
    tkn_cnt = num_tokens_from_string(chunks[i])
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/XXX/github/ragflow/rag/utils/__init__.py", line 79, in num_tokens_from_string
    num_tokens = len(encoder.encode(string))
                     ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/XXX/tiktoken/core.py", line 116, in encode
    if match := _special_token_regex(disallowed_special).search(text):
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: expected string or buffer
```
This type is `Dict`
<img width="1689" alt="Pasted Graphic 3"
src="https://github.com/user-attachments/assets/e5ba5c45-df1d-4697-98c9-14365c839f20">
The correct type should be ` Str`
<img width="1725" alt="Pasted Graphic 2"
src="https://github.com/user-attachments/assets/e54d5e60-4ce4-4180-b394-24e485013534">

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-08-08 12:03:01 +08:00
balibabu
ad6def4178
fix: The size of operators created from agent templates is displayed incorrectly #1863 (#1864)
### What problem does this PR solve?
fix: The size of operators created from agent templates is displayed
incorrectly #1863
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-08-08 11:16:16 +08:00
balibabu
ed6a693820
feat: Do not display arrow icons on leaf node of folders #1826 (#1862)
### What problem does this PR solve?

feat: Do not display arrow icons on leaf node of folders #1826

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-08-08 10:50:41 +08:00
Kevin Hu
1d5a9b74ff
fix add slef base url openai error (#1854)
### What problem does this PR solve?

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-08-07 19:16:46 +08:00
黄腾
e34817c2a9
add support for cohere (#1849)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
2024-08-07 18:40:51 +08:00
黄腾
60428c4ad2
fix LocalAI add bug (#1851)
### What problem does this PR solve?

#1848 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
2024-08-07 18:10:42 +08:00
Kevin Hu
7bc9742674
refine mindmap (#1844)
### What problem does this PR solve?



### Type of change

- [x] Refactoring
2024-08-07 13:11:28 +08:00
Kevin Hu
a199572bf8
add callback to entity extraction (#1843)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
- [ ]
2024-08-07 12:17:02 +08:00
balibabu
06dfb83529
fix: Entity types are only displayed when the knowledge graph is selected #1594 (#1841)
### What problem does this PR solve?

fix: Entity types are only displayed when the knowledge graph is
selected #1594

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-08-07 10:43:10 +08:00
balibabu
3c19e3125b
fix: Embedding error in file parsing #1835 (#1839)
### What problem does this PR solve?

fix: Embedding error in file parsing #1835

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-08-07 10:22:40 +08:00
balibabu
4ae9de76d4
fix: Unable to create a new chat assistant after closing the edit modal #1833 (#1838)
### What problem does this PR solve?

fix: Unable to create a new chat assistant after closing the edit modal
#1833

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-08-07 10:19:14 +08:00
balibabu
c55e9d16da
feat: Move files in file manager #1826 (#1837)
### What problem does this PR solve?

feat: Move files in file manager #1826

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-08-07 10:12:11 +08:00