1085 Commits

Author	SHA1	Message	Date
Nicolas	b8b8522b33	Nick: fixed map exception	2024-09-19 12:49:33 -04:00
Nicolas	ef71940429	Merge pull request #674 from mendableai/nsc/map-pagination ... v1/map improvements + higher limits	2024-09-19 11:31:23 -04:00
Nicolas	00bb958b32	Update v1-openapi.json	2024-09-18 15:59:11 -04:00
Nicolas	03eb47d426	Merge branch 'main' of https://github.com/mendableai/firecrawl	2024-09-18 15:58:54 -04:00
Nicolas	3a4dd8fc7e	Nick: v1 openapi spec	2024-09-18 15:58:53 -04:00
Gergő Móricz	c28e1e2959	fix(v1/zod): formats: add error message if both sc and sc@fP	2024-09-18 19:35:38 +02:00
Nicolas	3c2bfe2da2	Update single_url.ts	2024-09-17 01:58:47 -04:00
Nicolas	18b024c238	Update single_url.ts	2024-09-17 01:41:46 -04:00
Nicolas	a4039bd008	Revert "Update single_url.ts" ... This reverts commit 0f8c0a570dca877d14d590e6002eaffd345a3927.	2024-09-16 23:36:38 -04:00
Nicolas	0f8c0a570d	Update single_url.ts	2024-09-16 21:44:56 -04:00
Nicolas	9da3432596	Update map.ts	2024-09-16 12:03:14 -04:00
Nicolas	af4804e13b	Merge branch 'main' into nsc/map-pagination	2024-09-16 11:04:44 -04:00
Nicolas	030ecab6ee	Update rate-limiter.ts	2024-09-13 18:09:59 -04:00
Gergo Moricz	000a316cc3	fix(fire-engine): poll more frequently	2024-09-13 16:41:27 +02:00
Gergo Moricz	f7c4cee404	fix(queue-worker): don't send LLM extract hallucination error to Sentry	2024-09-13 14:02:49 +02:00
Gergő Móricz	d497284b40	feat(api/queue): auto-remove completed jobs after 25 hours	2024-09-12 19:47:22 +02:00
Nicolas	620b02f9ca	Nick:	2024-09-12 12:51:14 -04:00
Nicolas	e1171ade1b	Merge branch 'main' into mog/fix-status-job-get	2024-09-12 11:35:36 -04:00
Nicolas	4cd1065ae2	Update rate-limiter.ts	2024-09-11 14:03:34 -04:00
Gergő Móricz	97ffabff3a	fix(v1): converting bad docs always gives null	2024-09-10 21:21:20 +02:00
Gergő Móricz	83a165db0f	fix(v0/scrape): ensure url is string	2024-09-10 21:18:53 +02:00
Gergő Móricz	ad1a6fbc74	fix(v1/map): handle invalid URLs gracefully	2024-09-10 19:41:01 +02:00
Gergő Móricz	4ebc35c9dd	fix(crawl-status): add success: true	2024-09-10 18:59:09 +02:00
Nicolas	a7406031fa	Merge branch 'main' of https://github.com/mendableai/firecrawl	2024-09-10 06:53:30 -03:00
Nicolas	a17e1cac92	Rate bump	2024-09-10 06:53:24 -03:00
Gergo Moricz	b4dbf75537	fix(v1): check if url is string in blocklistMiddleware ... Fixes FIRECRAWL-SCRAPER-JS-9Z	2024-09-10 10:25:14 +02:00
Gergo Moricz	26f2095de6	fix(v1): proper Invalid URL handling	2024-09-10 09:24:23 +02:00
Gergo Moricz	f8fbc71f91	fix(supabase-jobs): do not use RPCs ... RPCs are more failure-prone for this use case than regular queries are.	2024-09-10 09:20:18 +02:00
Gergo Moricz	a6bcf7b438	fix(v0/crawl-status): don't crash on big crawls when requesting jobs from supabase	2024-09-10 08:53:40 +02:00
Nicolas	17e419a7fb	Nick:	2024-09-09 21:06:23 -03:00
Nicolas	ca9a781eb7	Update index.ts	2024-09-09 12:27:55 -03:00
Nicolas	22a5e85899	Update index.ts	2024-09-09 12:26:55 -03:00
Nicolas	60a15d00eb	Update types.ts	2024-09-08 16:39:12 -03:00
Nicolas	fbdfa1256b	Update credit_billing.ts	2024-09-08 13:07:10 -03:00
Nicolas	48c665519e	Update credit_billing.ts	2024-09-07 13:42:45 -03:00
Nicolas	32097fa29d	Merge branch 'main' of https://github.com/mendableai/firecrawl	2024-09-07 13:12:52 -03:00
Nicolas	5758af3291	Update website_params.ts	2024-09-07 13:12:46 -03:00
Nicolas	79870e7305	Update excludeTags.ts	2024-09-06 20:15:26 -03:00
Nicolas	a0f9ab2be7	Update map.ts	2024-09-06 20:14:47 -03:00
Nicolas	f5b84e15e1	Update sitemap.ts	2024-09-05 17:52:27 -03:00
rafaelsideguide	82d6bf4ec8	feat(go-parser): singleton	2024-09-05 16:14:21 -03:00
Nicolas	cb630bfc34	Update fireEngine.ts	2024-09-05 14:24:10 -03:00
rafaelsideguide	8c1097e9e1	fix: pageOptions	2024-09-05 14:16:31 -03:00
rafaelsideguide	b301ffc922	added missing variables	2024-09-05 13:57:26 -03:00
Nicolas	c6f1d80992	Update crawl.ts	2024-09-05 13:03:43 -03:00
Nicolas	eb03a81152	Update crawl-status.ts	2024-09-05 12:55:04 -03:00
Nicolas	34adf43200	Merge branch 'main' into feat/go-html-to-md-parser	2024-09-05 12:13:14 -03:00
Nicolas	82cb80c817	Update map.ts	2024-09-04 23:46:18 -03:00
Nicolas	a0113dac37	Update credit_billing.ts	2024-09-04 16:54:20 -03:00
Nicolas	b7b99b53da	Merge branch 'main' into nsc/check-credits-optimization	2024-09-04 16:39:18 -03:00