1578 Commits

Author SHA1 Message Date
Guido Schmutz
0c95a3382b
Dynamically create the service_conf.yaml file by replacing environment variables from .env (#3341)
### What problem does this PR solve?

This pull request implements the feature mentioned in #3322. 

Instead of manually having to edit the `service_conf.yaml` file when
changes have been made to `.env` and mapping it into the docker
container at runtime, a template file is used and the values replaced by
the environment variables from the `.env` file when the container is
started.
 

### Type of change

- [X] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
2024-11-12 22:56:53 +08:00
writinwaters
7274420ecd
Updated RAGFlow UI (#3362)
### What problem does this PR solve?



### Type of change


- [x] Documentation Update
2024-11-12 19:56:56 +08:00
Zhichang Yu
a2a5631da4
Rework logging (#3358)
Unified all log files into one.

### What problem does this PR solve?

Unified all log files into one.

### Type of change

- [x] Refactoring
2024-11-12 17:35:13 +08:00
liuhua
567a7563e7
Fix bugs in api and add examples (#3353)
### What problem does this PR solve?

Fix bugs in api.
Add simple examples for api.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-11-12 17:14:33 +08:00
Sanan bin Tahir
62a9afd382
Change launch backend script to handle errors gracefully (#3334)
### What problem does this PR solve?

The `launch_backend_service.sh` script enters infinite loops for both
the task executors and the backend server. When an error occurs in any
of these processes, the script continuously restarts them without
properly handling termination signals. This behavior causes the script
to even ignore interrupts, leading to persistent error messages and
making it difficult to exit the script gracefully.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Explanation of Modifications

1. **Signal Trapping with `trap`:** 
- The `trap cleanup SIGINT SIGTERM` line ensures that when a `SIGINT` or
`SIGTERM` signal is received, the cleanup function is invoked.
- The `cleanup` function sets the `STOP` flag to `true`, iterates
through all child process IDs stored in the `PIDS` array, and sends a
`kill` signal to each process to terminate them gracefully.
2. **Retry Limits:**
- Introduced a `MAX_RETRIES` variable to limit the number of restart
attempts for both `task_executor.py` and `ragflow_server.py`
- The loops now check if the retry count has reached the maximum limit.
If so, they invoke the `cleanup` function to terminate all processes and
exit the script.
3. **Process Tracking with `PIDS` Array:**
- After launching each background process (`task_exe` and `run_server`),
their Process IDs (PIDs) are stored in the `PIDS` array.
- This allows the `cleanup` function to terminate all child processes
effectively when needed.
4. **Graceful Shutdown:**
- When the `cleanup` function is called, it iterates over all child PIDs
and sends a termination signal (`kill`) to each, ensuring that all
subprocesses are stopped before the script exits.
5. **Logging Enhancements:**
- Added `echo` statements to provide clearer logs about the state of
each process, including attempts, successes, failures, and retries.
6. **Exit on Successful Completion:**
- If `ragflow_server.py` or a `task_executor.py` process exits with a
success code (0), the loop breaks, preventing unnecessary retries.

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-11-12 15:51:38 +08:00
balibabu
aa68d3b8db
fix: Cannot copy and paste text on agent page #3356 (#3357)
### What problem does this PR solve?

fix: Cannot copy and paste text on agent page #3356

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-12 15:49:45 +08:00
Kevin Hu
784ae896d1
add dependencies of chrome (#3352)
### What problem does this PR solve?



### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2024-11-12 15:49:33 +08:00
Zhichang Yu
f4c52371ab
Integration with Infinity (#2894)
### What problem does this PR solve?

Integration with Infinity

- Replaced ELASTICSEARCH with dataStoreConn
- Renamed deleteByQuery with delete
- Renamed bulk to upsertBulk
- getHighlight, getAggregation
- Fix KGSearch.search
- Moved Dealer.sql_retrieval to es_conn.py


### Type of change

- [x] Refactoring
2024-11-12 14:59:41 +08:00
balibabu
00b6000b76
feat: Disable automatic saving of agent during running agent #3349 (#3350)
### What problem does this PR solve?

feat: Disable automatic saving of agent during running agent #3349

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-11-12 12:47:36 +08:00
balibabu
db23d62827
feat: Add background colors such as inverse-strong #3221 (#3346)
### What problem does this PR solve?

feat: Add background colors such as inverse-strong #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-11-12 11:29:48 +08:00
Kevin Hu
70ea6661ed
add new models for zhipu-ai (#3348)
### What problem does this PR solve?

#3345

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-11-12 11:27:43 +08:00
haddad9
a01fceb328
feat: updates all readme to have a link to redirect to new locale (#3339)
### What problem does this PR solve?

Updates all readme to have a link to redirect to README_id.md for
documentation in Bahasa Indonesia

### Type of change

- [x] Documentation Update
2024-11-12 09:26:14 +08:00
haddad9
e9e98ea093
feat: documentation updates to support Bahasa Indonesia (#3315)
### What problem does this PR solve?

Add Readme docs in Indonesia's native language (Bahasa Indonesia) for
ragflow

### Type of change

- [x] Documentation Update
2024-11-11 19:33:23 +08:00
balibabu
528646a958
feat: Add custom background color #3221 (#3336)
### What problem does this PR solve?

feat: Add custom background color #3221

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-11-11 19:29:56 +08:00
writinwaters
8536335e63
Miscellaneous edits to RAGFlow's UI (#3337)
### What problem does this PR solve?



### Type of change

- [x] Documentation Update
2024-11-11 19:29:34 +08:00
Kevin Hu
88072b1e90
fix: double brace issue (#3328)
### What problem does this PR solve?

#3299

### Type of change

- [x] Performance Improvement
2024-11-11 12:07:02 +08:00
Kevin Hu
34d1daac67
fix: Anthropic param error (#3327)
### What problem does this PR solve?

#3263

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-11 11:54:14 +08:00
Kevin Hu
3faae0b2c2
trival (#3325)
### What problem does this PR solve?



### Type of change


- [x] Performance Improvement
2024-11-11 10:39:49 +08:00
Kevin Hu
5e5a35191e
fix benchmark issue (#3324)
### What problem does this PR solve?



### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-11 10:14:30 +08:00
Zhichang Yu
7c486ee3f9
Fix typo (#3319)
### What problem does this PR solve?

Fix typo

### Type of change

- [x] Refactoring
2024-11-11 09:36:39 +08:00
balibabu
20d686737a
feat: Switch the login page to the registration component by changing the routing parameters #3221 (#3307)
### What problem does this PR solve?
feat: Switch the login page to the registration component by changing
the routing parameters #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-11-08 19:47:22 +08:00
writinwaters
85047e7e36
Added configuration guideline (#3309)
### What problem does this PR solve?



### Type of change


- [x] Documentation Update
2024-11-08 19:46:18 +08:00
balibabu
ac64e35a45
feat: Translate autosaved #3301 (#3304)
### What problem does this PR solve?

[_Briefly describe what this PR aims to solve. Include background
context that will help reviewers understand the purpose of the
PR._](feat: Translate autosaved #3301)

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-11-08 18:27:42 +08:00
Kevin Hu
004487cca0
fix term weight issue (#3306)
### What problem does this PR solve?

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-08 18:25:23 +08:00
balibabu
74d1eeb4d3
feat: Automatically save agent page data #3301 (#3302)
### What problem does this PR solve?

feat: Automatically save agent page data #3301

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-11-08 17:28:11 +08:00
writinwaters
464a4d6ead
Added env. MACOS (#3297)
### What problem does this PR solve?



### Type of change


- [x] Documentation Update
2024-11-08 16:50:35 +08:00
writinwaters
3d3913419b
Updated .env and Docker README (#3295)
### What problem does this PR solve?


### Type of change


- [x] Documentation Update
2024-11-08 16:31:52 +08:00
balibabu
63f7d3bae2
feat: Support shortcut keys to copy nodes #3283 (#3293)
### What problem does this PR solve?

feat: Support shortcut keys to copy nodes #3283

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-11-08 15:50:01 +08:00
Kevin Hu
8b6e272197
fix: term weight issue (#3294)
### What problem does this PR solve?



### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-08 15:49:44 +08:00
balibabu
5205bdab24
feat: Modify the name of the answer operator #1739 (#3288)
### What problem does this PR solve?
feat: Modify the name of the answer operator #1739

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-11-08 13:56:43 +08:00
wjimenez123
37d4708880
Archivo ts con la traducción para el idioma español (#3274)
¿Qué problema resuelve este PR?
Archivo TS para la traducción al idioma español
Este archivo TS es un archivo de traducción que se utiliza para
proporcionar soporte multilingüe en las aplicaciones. Al agregar la
traducción al español, se facilita el cambio del idioma de la interfaz
de usuario para los usuarios hispanohablantes.

Tipo de cambio
- [x] Nueva funcionalidad (cambio no disruptivo que agrega una nueva
característica)

### What problem does this PR solve?

This TS file is a translation file used to provide multilingual support
in applications. By adding the Spanish translation, it facilitates
changing the user interface language for Spanish-speaking users._

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-11-08 13:44:03 +08:00
Kevin Hu
d88f0d43ea
make language judgement robuster (#3287)
### What problem does this PR solve?



### Type of change

- [x] Performance Improvement
2024-11-08 12:48:11 +08:00
balibabu
a2153d61ce
feat: Support shortcut keys to delete nodes #3283 (#3284)
### What problem does this PR solve?
feat: Support shortcut keys to delete nodes #3283

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-11-08 12:07:26 +08:00
Kevin Hu
f16ef57979
fix switch bug (#3280)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-08 10:08:57 +08:00
Kevin Hu
ff2bbb487f
fix index of range (#3279)
### What problem does this PR solve?

#3273

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-08 09:58:11 +08:00
writinwaters
416efbe7e8
Updated Docker README (#3272)
### What problem does this PR solve?


### Type of change

- [x] Documentation Update
2024-11-08 09:22:40 +08:00
kuschzzp
9c6cc20356
Fix:#3230 When parsing a docx file using the Book parsing method, to_page is always -1, resulting in a block count of 0 even if parsing is successful (#3249)
### What problem does this PR solve?

When parsing a docx file using the Book parsing method, to_page is
always -1, resulting in a block count of 0 even if parsing is successful

Fix:#3230

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-11-08 09:21:42 +08:00
writinwaters
7c0d28b62d
Refactored Docker README (#3269)
### What problem does this PR solve?



### Type of change

- [x] Documentation Update
2024-11-07 19:38:50 +08:00
liuhua
48ab6d7a45
Update authorization for team (#3262)
### What problem does this PR solve?

Update authorization for team.
#3253 #3233
### Type of change

- [x] Refactoring

---------

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-11-07 19:26:03 +08:00
balibabu
96b5d2b3a9
fix: The name of the copy operator is displayed the same as before ##3265 (#3266)
### What problem does this PR solve?

fix: The name of the copy operator is displayed the same as before
##3265

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-11-07 17:53:31 +08:00
haddad9
f45c29360c
New locale for Bahasa Indonesia support (#3255)
### What problem does this PR solve?
Add native translation in locales for Bahasa Indonesia to support new
local language

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-11-07 17:53:19 +08:00
writinwaters
cdcbe6c2b3
Fixed broken links (#3256)
### What problem does this PR solve?



### Type of change


- [x] Documentation Update
2024-11-07 14:12:32 +08:00
Mohammed Tawileh
5038552ed9
fix: improve embedding model validation logic for dataset operations (#3235)
What problem does this PR solve?
When creating or updating datasets with custom embedding models (e.g.,
Ollama), the validation logic was too restrictive and prevented valid
models from being used. The previous implementation would reject valid
custom models if they weren't in the predefined list, even when they
existed in TenantLLMService.

Changes:
- Simplify and improve the embedding model validation flow in
create/update endpoints
- Check TenantLLMService for custom models before rejecting
- Make validation logic more consistent between create and update
operations

### What problem does this PR solve?

This fix allows users to successfully create and update datasets with
custom embedding models while maintaining proper validation checks.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
Co-authored-by: liuhua <10215101452@stu.ecnu.edu.cn>
2024-11-07 10:36:28 +08:00
Kevin Hu
1b3e39dd12
fix ci issue (#3245)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-06 19:32:30 +08:00
Kevin Hu
fbcc0bb408
accelerate tokenize (#3244)
### What problem does this PR solve?


### Type of change

- [x] Performance Improvement
2024-11-06 18:54:41 +08:00
balibabu
d3bb5e9f3d
feat: Display input parameters on operator nodes #3240 (#3241)
### What problem does this PR solve?
feat: Display input parameters on operator nodes #3240


### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-11-06 18:48:05 +08:00
Kevin Hu
4097912d59
add inputs to display to every components (#3242)
### What problem does this PR solve?

#3240

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-11-06 18:47:53 +08:00
liuhua
f3aaa0d453
Add sdk for Agent API (#3220)
### What problem does this PR solve?

Add sdk for Agent API

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-11-06 18:03:45 +08:00
ksztone-huanggonghao
0dff64f6ad
fix: TypeError: only length-1 arrays can be converted to Python scalars (#3211)
### What problem does this PR solve?
fix "TypeError: only length-1 arrays can be converted to Python scalars"
while using cohere embedding model.

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)


![image](https://github.com/user-attachments/assets/2c21a69f-cd76-4d25-b320-058964812db8)
2024-11-06 11:15:00 +08:00
balibabu
601a128cd3
feat: Add next login page with shadcn/ui #3221 (#3231)
### What problem does this PR solve?

feat: Add next login page with shadcn/ui  #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-11-06 11:13:04 +08:00