Gergő Móricz
d3b821e827
Revert "Revert "temp: get acuc from main db""
...
This reverts commit 8566bff35c56ac042b54afd661a0cba7c4556e53.
2025-04-11 00:12:57 +02:00
Gergő Móricz
8566bff35c
Revert "temp: get acuc from main db"
...
This reverts commit f16f034463d4216de4a9ae12fcad18193d66bd6d.
2025-04-10 20:28:43 +02:00
Gergő Móricz
f16f034463
temp: get acuc from main db
2025-04-10 18:58:59 +02:00
Gergő Móricz
6a10f0689d
ACUC: Dynamic Limits (FIR-1641) ( #1434 )
...
* extend acuc definition
* kill plan
* stuff
* stupid tests
* feat: better acuc
* feat(acuc): mock ACUC when not using db auth
2025-04-10 18:49:23 +02:00
Gergő Móricz
f2865f6699
temp: disable acuc caching
2025-04-10 16:08:20 +02:00
Gergő Móricz
a461f72d17
temporarily disable some flaky tests
2025-04-10 13:07:09 +02:00
Gergő Móricz
415603acb0
fixes
2025-04-10 12:50:03 +02:00
Gergő Móricz
4294face78
feat(scrapeURL): reintroduce default timeout for simple queries ( #1440 )
...
* feat(scrapeURL): reintroduce default timeout
* fix
* adjust timeouts
2025-04-10 12:29:54 +02:00
Gergő Móricz
7bb5643028
feat(log_job): is_migrated: true
2025-04-10 08:48:37 +02:00
Gergő Móricz
d925bf2f68
feat(log_job): stop putting docs in the db ( #1438 )
...
* feat(log_job): stop putting jobs in the db
* fix parens
2025-04-09 21:29:56 +02:00
Ademílson Tonato
45efe3fd46
Merge pull request #1436 from mendableai/feat/increase-search-limit
...
feat(api/search): add search endpoint and update request limits
2025-04-09 19:04:31 +01:00
Gergő Móricz
78a920af61
fix(api/tests/scrape): bump timeout
2025-04-09 19:47:38 +02:00
Gergő Móricz
d3da790dc4
feat(extraction-service): teamId logging
2025-04-09 18:48:00 +02:00
Eric Ciarla
c81db8512a
Merge pull request #1418 from aparupganguly/Feature/llama4-extractor
...
Add examples/llama4-maverick-web-extractor
2025-04-09 12:24:34 -04:00
Ademílson F. Tonato
da2f17c757
feat(api/search): add search endpoint and update request limits
...
- Introduced a new POST endpoint for searching with a query and limit
- Updated the maximum limit for search results from 20 to 50 in the request schema
- Adjusted the default number of results in the Google search function from 7 to 5
2025-04-09 17:24:29 +01:00
Eric Ciarla
d506aece2d
Merge pull request #1416 from aparupganguly/Feature/llama4-mv-crawler
...
Add examples/ Llama 4 Maverick Crawler
2025-04-09 12:24:15 -04:00
Gergő Móricz
9fd735f3a1
feat(api/test/snips): disable flaky tests
2025-04-09 15:45:07 +02:00
Gergő Móricz
dc1a17d571
remove bad log
2025-04-09 13:04:00 +02:00
Gergő Móricz
3a8de846e3
read from GCS (again) ( #1433 )
...
* feat(crawl-status): retrieve job data from GCS
* feat(gcs-jobs/save): retrying saving metadata (might conflict)
* feat(gcs-jobs/save): retry save operation
* fix(gcs-jobs/save): respect metadata rules
* feat(crawl-status): log if gcs job is not found
* feat(ci/test/server): add gcs
2025-04-09 12:47:51 +02:00
Gergő Móricz
670e4a6bf1
Revert "feat(crawl-status): retrieve job data from GCS ( #1427 )"
...
This reverts commit 673bf6a2dea0f2340f964ba9c25cab8e92d929e4.
2025-04-09 12:28:46 +02:00
Gergő Móricz
673bf6a2de
feat(crawl-status): retrieve job data from GCS ( #1427 )
...
* feat(crawl-status): retrieve job data from GCS
* feat(gcs-jobs/save): retrying saving metadata (might conflict)
* feat(gcs-jobs/save): retry save operation
* fix(gcs-jobs/save): respect metadata rules
* feat(crawl-status): log if gcs job is not found
2025-04-09 12:27:23 +02:00
Gergő Móricz
ab6fb48e6e
bump ver
2025-04-08 21:11:42 +02:00
devin-ai-integration[bot]
8c801ed956
Rename 'compare' format and property to 'changeTracking' ( #1423 )
2025-04-08 21:09:31 +02:00
Gergő Móricz
62265c63c8
feat(log_job): use atob
2025-04-08 20:26:43 +02:00
Gergő Móricz
c69d156179
fix(log_job): use service account credentials
2025-04-08 20:18:48 +02:00
Gergő Móricz
37b13ba146
feat(log_job): allow use of api key if specified
2025-04-08 20:04:59 +02:00
Gergő Móricz
bd1c1b0012
feat(log_job): start saving jobs to GCS ( #1424 )
2025-04-08 19:28:21 +02:00
Aparup Ganguly
1321275102
Update Llama 4 Maverick extractor implementation
2025-04-08 20:52:06 +05:30
Ademílson F. Tonato
80bf732f50
feat: incorporate user preferences and notification categories
2025-04-07 15:59:29 +01:00
Aparup Ganguly
2f037fa1a7
Add examples/llama4-maverick-web-extractor
2025-04-07 19:06:17 +05:30
Aparup Ganguly
17ea3ff355
Add examples/ Llama 4 Maverick Crawler
2025-04-07 18:35:23 +05:30
Nicolas
66e65d9422
Merge branch 'main' of https://github.com/mendableai/firecrawl
2025-04-05 12:42:25 -04:00
Nicolas
f45fa12075
Update rate-limiter.ts
2025-04-05 12:42:24 -04:00
Gergő Móricz
f5e5bdb710
fix(llmExtract): arbitrary objects caused error to be thrown
2025-04-05 15:48:47 +02:00
Gergő Móricz
570809aa59
fix(unvisitedUrls): filter with crawler
...
Fixes #1410
2025-04-04 22:13:02 +02:00
Jingyu
6bed5eca50
fix(rust-sdk): remove rustfmt ( #1392 )
...
rustfmt is deprecated, it depends on a outdated extprim crate which cause test failed to run
2025-04-04 22:05:51 +02:00
Nicolas
41e094032f
Update email_notification.ts
2025-04-04 14:36:41 -04:00
Nicolas
e1e39f8836
Nick: send notifications for crawl+batch scrape
2025-04-04 14:34:48 -04:00
Gergő Móricz
7128f83a7a
fix(js-sdk): isows import issues (FIR-1586) (FIR-1536) ( #1411 )
...
* attempt
* improvements
* kill isows -- there's been native websocket support in node since 21
* clean up the diff
2025-04-04 17:54:37 +02:00
Ademílson Tonato
b57d5f2c4d
Merge pull request #1409 from mendableai/feat/crawl-scrape-limit-notification
...
feat(queue-jobs): add function to determine job type and update notification logic for concurrency limits
2025-04-03 18:29:00 +01:00
Ademílson F. Tonato
426151c9c9
feat(queue-jobs): add function to determine job type and update notification logic for concurrency limits
2025-04-03 17:02:51 +01:00
Gergő Móricz
8c1579df51
bump cc
2025-04-03 11:56:24 +02:00
Gergő Móricz
2e2c3d52ce
feat: add swoogo classes to force include main tags
2025-04-03 09:57:19 +02:00
Gergő Móricz
24f5199359
compare format (FIR-1560) ( #1405 )
2025-04-02 19:52:43 +02:00
Gergő Móricz
b3b63486f1
cc manual
2025-04-02 19:27:13 +02:00
Ademílson Tonato
3300c6c598
Merge pull request #1404 from mendableai/fix/add-notification-type
...
feat(notification): add notification message for concurrency limit reached
2025-04-02 17:39:59 +01:00
Ademílson F. Tonato
b900f34b5a
feat(notification): add notification message for concurrency limit reached
2025-04-02 17:36:11 +01:00
rafaelmmiller
7216799ca0
revert mog changes
2025-04-02 10:45:11 -03:00
Ademílson Tonato
73a297d6c8
Merge pull request #1398 from mendableai/refactor/email-concurrency-limit-reached
...
feat(queue-jobs): update notification logic for concurrency limits and add parameter (jsdocs) to batchScrapeUrls
2025-04-02 11:18:18 +01:00
Ademílson F. Tonato
7468464552
feat(queue-jobs): implement conditional notification for concurrency limits based on team subscription status
2025-04-01 19:50:26 +01:00
