2474 Commits

Author	SHA1	Message	Date
rafaelmmiller	3786ad7f5d	restart	2025-03-07 16:19:59 -03:00
Nicolas	1ced546bd0	Update rate-limiter.ts	2025-03-06 19:17:02 -03:00
Nicolas	f45b3c0155	Merge branch 'main' into fix/preview_token	2025-03-06 19:02:29 -03:00
Nicolas	b66e0ccb7d	Nick:	2025-03-06 18:36:01 -03:00
rafaelmmiller	55a4a3d768	preview-token	2025-03-06 18:16:07 -03:00
Gergő Móricz	39b1390289	fix: crawl	2025-03-06 21:57:42 +01:00
Gergő Móricz	a1e6c13b67	move crawl to read replica	2025-03-06 21:57:42 +01:00
Nicolas	982b3da27d	Update auth.ts	2025-03-06 17:53:41 -03:00
Nicolas	5a149a1c30	Revert "fix(auth): always use replica for acuc" ... This reverts commit 57b313608cb6e9f3c123e4754c653fef8dfacbdf.	2025-03-06 17:53:28 -03:00
Nicolas	783fad90dd	Nick: more read replicas	2025-03-06 17:45:51 -03:00
Nicolas	949fb68bdd	Merge branch 'main' of https://github.com/mendableai/firecrawl	2025-03-06 17:36:57 -03:00
Nicolas	4c4d51e60b	Revert "Update runWebScraper.ts" ... This reverts commit 40eacfacd4c5a782f722fd85e3935f4a1741b27a.	2025-03-06 17:36:08 -03:00
Gergő Móricz	57b313608c	fix(auth): always use replica for acuc	2025-03-06 21:35:30 +01:00
Gergő Móricz	7b05512054	fix(credit_billing): teams check	2025-03-06 21:30:57 +01:00
Nicolas	ae010a76c1	Update blocklist.ts	2025-03-06 17:11:00 -03:00
Gergő Móricz	e6c3f20944	fix(preview): temporarily disable	2025-03-06 21:08:12 +01:00
Nicolas	72d894c2a3	Update rate-limiter.ts	2025-03-06 17:06:42 -03:00
Nicolas	1de5a2c589	Update batch_billing.ts	2025-03-06 16:57:23 -03:00
Nicolas	eba28a4210	Merge branch 'main' of https://github.com/mendableai/firecrawl	2025-03-06 16:41:22 -03:00
Nicolas	40eacfacd4	Update runWebScraper.ts	2025-03-06 16:41:21 -03:00
Gergő Móricz	e1cfe1da48	feat(crawl): includes/excludes fixes (FIR-1300) (#1303) ... * feat(crawl): includes/excludes fixes pt. 1 * fix(snips): billing tests * drop tha logs * fix(ci): add replica url * feat(crawl): drop initial scrape if it's not included * feat(ci): more verbose logging * fix crawl path in test * fix(ci): wait for api * fix(snips/scrape/ad): test for more pixels * feat(js-sdk/crawl): add regexOnFullURL	2025-03-06 17:05:15 +01:00
Gergő Móricz	f8df18ed6a	feat(feng-check): run on chrome-cdp	2025-03-05 18:53:10 +01:00
Gergő Móricz	c22c87ab0a	fix(crawler): check for more strings	2025-03-05 18:48:05 +01:00
Nicolas	4902d0ac64	Nick: fixes	2025-03-04 23:22:58 -03:00
Nicolas	8cfc946cae	Nick: fix system prompt missing from extract params	2025-03-04 23:04:20 -03:00
Eric Ciarla	5a1886936c	Truncate llmstxt cache based on maxurls limit & improve maxurls handling (#1285) ... * init * Update generate-llmstxt-service.ts	2025-03-03 18:37:33 -03:00
Gergő Móricz	1beadf39f0	fix	2025-03-03 21:06:01 +01:00
Gergő Móricz	6dce8e9ddf	asd	2025-03-03 21:05:45 +01:00
Gergő Móricz	8620bf3d5e	fix(auth): split load evenly between two instances	2025-03-03 20:53:51 +01:00
Gergő Móricz	67ee2662de	feat(auth): force acuc on read replica	2025-03-03 20:36:54 +01:00
Nicolas	64af3ba728	Revert "Update auth.ts" ... This reverts commit fea249c568c4a8d63bcc2e560fd3655e73350be6.	2025-03-03 16:26:42 -03:00
Nicolas	9eb25786dc	Update deep-research-service.ts	2025-03-02 18:16:20 -03:00
Nicolas	aa54fd1658	Nick: only new activities/sources in the callback	2025-03-02 17:46:12 -03:00
Nicolas	33c59ee474	Nick: fixes	2025-03-02 17:41:19 -03:00
Nicolas	30c1e92a2e	Update deep-research-service.ts	2025-03-02 17:37:15 -03:00
Nicolas	22d4f0de38	(feat/deep-research) Alpha prep + Improvements (#1284) ... * Update index.ts * Nick: * Nick: topic -> query * Update deep-research.ts * Nick: bump * Nick: add onSource callback	2025-03-02 17:33:21 -03:00
Gergő Móricz	9ad947884d	feat(tests/snips): add billing tests + misc billing fixes (FIR-1280) (#1283) ... * feat(tests/snips): add billing tests + misc billing fixes * add testing key * asd	2025-03-02 16:51:42 -03:00
Gergő Móricz	4f25f12a12	fix(ai): handle if AI returns a JSON code block (#1280)	2025-03-02 15:25:24 -03:00
Gergő Móricz	e8c698d613	feat(crawler): handle cross-origin redirects differently than same-origin redirects (#1279)	2025-03-02 13:32:46 +01:00
Nicolas	fea249c568	Update auth.ts	2025-03-02 02:40:34 -03:00
Nicolas	99e61c967c	Update index.ts	2025-03-02 02:39:52 -03:00
Gergő Móricz	05c29e8299	js-sdk: bump	2025-02-28 12:40:39 +01:00
Gergő Móricz	904e69bfbc	feat(supabase): add read replica routing (#1274)	2025-02-28 09:52:26 +01:00
Nicolas	39b61132bf	Nick: fixed js sdk	2025-02-27 20:39:53 -03:00
Gergő Móricz	44bf59229a	fix(acuc): cache for 1 hour	2025-02-27 21:36:33 +01:00
Nicolas	b72e21a697	Nick: batch billing (#1264)	2025-02-27 20:18:03 +01:00
Nicolas	289e351c14	(feat/deep-research-alpha) Added Max Urls, Sources and Fixes (#1271) ... * Nick: fixes * Nick: * Update deep-research-status.ts	2025-02-27 13:24:40 -03:00
Gergő Móricz	1d3757b391	bump map to 30k	2025-02-27 12:44:23 +01:00
Grass Huang	7bf04d409a	fix(scraper): improve charset detection regex to accurately parse meta tags (#1265)	2025-02-26 17:31:06 +01:00
Nicolas	31df234127	Update log_job.ts	2025-02-25 21:01:05 -03:00