1063 Commits

Author	SHA1	Message	Date
Nicolas	4cd1065ae2	Update rate-limiter.ts	2024-09-11 14:03:34 -04:00
Gergő Móricz	97ffabff3a	fix(v1): converting bad docs always gives null	2024-09-10 21:21:20 +02:00
Gergő Móricz	83a165db0f	fix(v0/scrape): ensure url is string	2024-09-10 21:18:53 +02:00
Gergő Móricz	ad1a6fbc74	fix(v1/map): handle invalid URLs gracefully	2024-09-10 19:41:01 +02:00
Gergő Móricz	4ebc35c9dd	fix(crawl-status): add success: true	2024-09-10 18:59:09 +02:00
Nicolas	a7406031fa	Merge branch 'main' of https://github.com/mendableai/firecrawl	2024-09-10 06:53:30 -03:00
Nicolas	a17e1cac92	Rate bump	2024-09-10 06:53:24 -03:00
Gergo Moricz	b4dbf75537	fix(v1): check if url is string in blocklistMiddleware ... Fixes FIRECRAWL-SCRAPER-JS-9Z	2024-09-10 10:25:14 +02:00
Gergo Moricz	26f2095de6	fix(v1): proper Invalid URL handling	2024-09-10 09:24:23 +02:00
Nicolas	17e419a7fb	Nick:	2024-09-09 21:06:23 -03:00
Nicolas	ca9a781eb7	Update index.ts	2024-09-09 12:27:55 -03:00
Nicolas	22a5e85899	Update index.ts	2024-09-09 12:26:55 -03:00
Nicolas	60a15d00eb	Update types.ts	2024-09-08 16:39:12 -03:00
Nicolas	fbdfa1256b	Update credit_billing.ts	2024-09-08 13:07:10 -03:00
Nicolas	48c665519e	Update credit_billing.ts	2024-09-07 13:42:45 -03:00
Nicolas	32097fa29d	Merge branch 'main' of https://github.com/mendableai/firecrawl	2024-09-07 13:12:52 -03:00
Nicolas	5758af3291	Update website_params.ts	2024-09-07 13:12:46 -03:00
Nicolas	79870e7305	Update excludeTags.ts	2024-09-06 20:15:26 -03:00
rafaelsideguide	82d6bf4ec8	feat(go-parser): singleton	2024-09-05 16:14:21 -03:00
Nicolas	cb630bfc34	Update fireEngine.ts	2024-09-05 14:24:10 -03:00
rafaelsideguide	8c1097e9e1	fix: pageOptions	2024-09-05 14:16:31 -03:00
rafaelsideguide	b301ffc922	added missing variables	2024-09-05 13:57:26 -03:00
Nicolas	c6f1d80992	Update crawl.ts	2024-09-05 13:03:43 -03:00
Nicolas	eb03a81152	Update crawl-status.ts	2024-09-05 12:55:04 -03:00
Nicolas	34adf43200	Merge branch 'main' into feat/go-html-to-md-parser	2024-09-05 12:13:14 -03:00
Nicolas	82cb80c817	Update map.ts	2024-09-04 23:46:18 -03:00
Nicolas	a0113dac37	Update credit_billing.ts	2024-09-04 16:54:20 -03:00
Nicolas	b7b99b53da	Merge branch 'main' into nsc/check-credits-optimization	2024-09-04 16:39:18 -03:00
rafaelsideguide	85b824e122	test: what about false false?	2024-09-04 16:35:32 -03:00
rafaelsideguide	78edf13ec6	test: usedbauth envs wth	2024-09-04 16:31:42 -03:00
rafaelsideguide	cb8571abad	fix: enforced dotenv config	2024-09-04 15:57:57 -03:00
Nicolas	5ecb243693	Nick:	2024-09-04 15:19:45 -03:00
rafaelsideguide	3f462eabe9	fix(cicd):	2024-09-04 12:27:46 -03:00
rafaelsideguide	364ba9f901	fix(cicd): mendable->firecrawl and waitfor	2024-09-04 12:05:02 -03:00
rafaelsideguide	ad950a6c9d	fixed controller res and tests	2024-09-04 11:29:32 -03:00
rafaelsideguide	74ac8915cd	details	2024-09-04 09:41:38 -03:00
rafaelsideguide	d836ba6782	added log to check response on cicd	2024-09-04 09:35:56 -03:00
Nicolas	049a11187d	Nick:	2024-09-03 21:09:32 -03:00
Nicolas	3072d4a333	Nick: fixed .sort coupons and sentry to withAuth	2024-09-03 21:02:41 -03:00
Nicolas	7561fd279f	Nick: debug the billing email system for free credits	2024-09-03 17:08:12 -03:00
rafaelsideguide	c5e1d77a82	added invalid html tests	2024-09-03 15:21:45 -03:00
rafaelsideguide	d60fa6e084	fixed dockerfile and function name. it's working	2024-09-03 14:08:07 -03:00
rafaelsideguide	ebf4035484	added log so we can check	2024-09-03 13:15:21 -03:00
rafaelsideguide	291d9e375b	now using compiled go/C lib with koffi	2024-09-03 10:56:07 -03:00
Nicolas	08a9cb8db4	Merge branch 'main' into pr/516	2024-09-02 23:32:23 -03:00
Nicolas	3244454826	Nick:	2024-09-02 19:36:36 -03:00
Nicolas	2444f7c450	Update scrape.ts	2024-09-02 18:51:18 -03:00
Nicolas	00dacaf935	Nick: scrape id when origin is website for report system	2024-09-02 18:43:52 -03:00
rafaelsideguide	48056ea1bd	feat: added go html to md parser	2024-09-02 14:15:56 -03:00
rafaelsideguide	036eb925a2	fix(v1): maxDepth	2024-09-02 09:55:35 -03:00