Gergő Móricz
e6d8792a32
feat(crawl-status): log if gcs job is not found
2025-04-09 12:29:18 +02:00
Gergő Móricz
f64dd4a9a4
fix(gcs-jobs/save): respect metadata rules
2025-04-09 12:29:18 +02:00
Gergő Móricz
4709fbce6a
feat(gcs-jobs/save): retry save operation
2025-04-09 12:29:18 +02:00
Gergő Móricz
078273695a
feat(gcs-jobs/save): retrying saving metadata (might conflict)
2025-04-09 12:29:18 +02:00
Gergő Móricz
777914e968
feat(crawl-status): retrieve job data from GCS
2025-04-09 12:29:18 +02:00
Gergő Móricz
670e4a6bf1
Revert "feat(crawl-status): retrieve job data from GCS ( #1427 )"
...
This reverts commit 673bf6a2dea0f2340f964ba9c25cab8e92d929e4.
2025-04-09 12:28:46 +02:00
Gergő Móricz
673bf6a2de
feat(crawl-status): retrieve job data from GCS ( #1427 )
...
* feat(crawl-status): retrieve job data from GCS
* feat(gcs-jobs/save): retrying saving metadata (might conflict)
* feat(gcs-jobs/save): retry save operation
* fix(gcs-jobs/save): respect metadata rules
* feat(crawl-status): log if gcs job is not found
2025-04-09 12:27:23 +02:00
Gergő Móricz
ab6fb48e6e
bump ver
2025-04-08 21:11:42 +02:00
devin-ai-integration[bot]
8c801ed956
Rename 'compare' format and property to 'changeTracking' ( #1423 )
2025-04-08 21:09:31 +02:00
Gergő Móricz
62265c63c8
feat(log_job): use atob
2025-04-08 20:26:43 +02:00
Gergő Móricz
c69d156179
fix(log_job): use service account credentials
2025-04-08 20:18:48 +02:00
Gergő Móricz
37b13ba146
feat(log_job): allow use of api key if specified
2025-04-08 20:04:59 +02:00
Gergő Móricz
bd1c1b0012
feat(log_job): start saving jobs to GCS ( #1424 )
2025-04-08 19:28:21 +02:00
Nicolas
66e65d9422
Merge branch 'main' of https://github.com/mendableai/firecrawl
2025-04-05 12:42:25 -04:00
Nicolas
f45fa12075
Update rate-limiter.ts
2025-04-05 12:42:24 -04:00
Gergő Móricz
f5e5bdb710
fix(llmExtract): arbitrary objects caused error to be thrown
2025-04-05 15:48:47 +02:00
Gergő Móricz
570809aa59
fix(unvisitedUrls): filter with crawler
...
Fixes #1410
2025-04-04 22:13:02 +02:00
Jingyu
6bed5eca50
fix(rust-sdk): remove rustfmt ( #1392 )
...
rustfmt is deprecated, it depends on a outdated extprim crate which cause test failed to run
2025-04-04 22:05:51 +02:00
Nicolas
41e094032f
Update email_notification.ts
2025-04-04 14:36:41 -04:00
Nicolas
e1e39f8836
Nick: send notifications for crawl+batch scrape
2025-04-04 14:34:48 -04:00
Gergő Móricz
7128f83a7a
fix(js-sdk): isows import issues (FIR-1586) (FIR-1536) ( #1411 )
...
* attempt
* improvements
* kill isows -- there's been native websocket support in node since 21
* clean up the diff
2025-04-04 17:54:37 +02:00
Ademílson F. Tonato
426151c9c9
feat(queue-jobs): add function to determine job type and update notification logic for concurrency limits
2025-04-03 17:02:51 +01:00
Gergő Móricz
8c1579df51
bump cc
2025-04-03 11:56:24 +02:00
Gergő Móricz
2e2c3d52ce
feat: add swoogo classes to force include main tags
2025-04-03 09:57:19 +02:00
Gergő Móricz
24f5199359
compare format (FIR-1560) ( #1405 )
2025-04-02 19:52:43 +02:00
Gergő Móricz
b3b63486f1
cc manual
2025-04-02 19:27:13 +02:00
Ademílson F. Tonato
b900f34b5a
feat(notification): add notification message for concurrency limit reached
2025-04-02 17:36:11 +01:00
rafaelmmiller
7216799ca0
revert mog changes
2025-04-02 10:45:11 -03:00
Ademílson Tonato
73a297d6c8
Merge pull request #1398 from mendableai/refactor/email-concurrency-limit-reached
...
feat(queue-jobs): update notification logic for concurrency limits and add parameter (jsdocs) to batchScrapeUrls
2025-04-02 11:18:18 +01:00
Ademílson F. Tonato
7468464552
feat(queue-jobs): implement conditional notification for concurrency limits based on team subscription status
2025-04-01 19:50:26 +01:00
Nicolas
ee211132c8
Nick:
2025-04-01 21:06:27 +04:00
Nicolas
c4255f4fdd
Update auth.ts
2025-04-01 21:00:40 +04:00
Nicolas
b79b90fdd1
Update auth.ts
2025-04-01 20:53:43 +04:00
Ademílson F. Tonato
58e587d99e
feat(queue-jobs): update notification logic for concurrency limits and add parameter (jsdocs) to batchScrapeUrls
2025-03-31 13:27:36 +01:00
Gergő Móricz
e0a3c54967
new acuc
2025-03-30 17:32:24 +02:00
Gergő Móricz
b9dde3fc3d
temp: move more to main instance
2025-03-29 18:18:55 +01:00
Gergő Móricz
4f0510e71d
temp: switch over crawl fetches to main instance
2025-03-29 18:05:50 +01:00
Gergő Móricz
f0e0d3e2e3
fix(api): crawl origin tracking (FIR-1499)
2025-03-28 12:47:37 +01:00
Gergő Móricz
46048bc94d
feat(scrapeURL): return js returns from f-e (FIR-1535) ( #1385 )
...
* feat(scrapeURL): return js returns from f-e
* feat(js-sdk): handle new results
2025-03-28 12:42:25 +01:00
Gergő Móricz
e799cf206b
increase manual rl
2025-03-26 20:13:32 +01:00
Gergő Móricz
2b39788d73
manual rl
2025-03-26 20:00:25 +01:00
Gergő Móricz
42236ef0f0
feat(admin/check-fire-engine): better logging
2025-03-25 20:51:40 +01:00
Gergő Móricz
e8f27bef08
feat(rate-limiter): manual_etier2c
2025-03-25 16:24:01 +01:00
Nicolas
555dab50e7
Nick: bump
2025-03-24 20:17:22 +04:00
Nicolas
a50dc106ef
(feat/deep-research) Deep Research Alpha v1 - Structured Outputs + Customizability ( #1365 )
...
* Nick:
* Nick: structured output support
* Nick: support for zod and pydantic
2025-03-24 20:13:52 +04:00
Gergő Móricz
5e35782b5d
update acuc
2025-03-20 17:47:29 +01:00
Nicolas
867e545511
Update rate-limiter.ts
2025-03-19 23:25:05 -04:00
Nicolas
3e0d3db90e
Update rate-limiter.ts
2025-03-19 23:20:18 -04:00
Gergő Móricz
0bdaa97bfb
acuc fix
2025-03-19 21:10:52 +01:00
Gergő Móricz
6dc5b1cd64
feat(auth): acuc update
2025-03-19 17:39:59 +01:00
