1937 Commits

Author	SHA1	Message	Date
Nicolas	cd08be7f37	Merge pull request #990 from RutamBhagat/python-sdk-conditionally-enforce-api-key ... feat(python-sdk): Make API key optional for self-hosted instances	2024-12-27 15:43:37 -03:00
Nicolas	c5b6495e48	Merge pull request #1015 from mendableai/nsc/improves-sitemap-fetching ... Improves sitemap fetching	2024-12-27 14:41:04 -03:00
Nicolas	e8f0a22ebe	Update v1-openapi.json	2024-12-27 13:59:43 -03:00
Nicolas	f7cfbba651	Merge branch 'main' into pr/1003	2024-12-27 13:59:24 -03:00
Gergő Móricz	4772951313	feat(scrapeURL/fire-engine): explicitly delete job after scrape	2024-12-27 16:44:41 +01:00
Gergő Móricz	c543f4f76c	feat(scrapeURL/pdf): update mock Blob implementation to pass TypeScript	2024-12-26 20:31:51 +01:00
Gergő Móricz	f15ef0e758	feat(scrapeURL/fire-engine/chrome-cdp): handle file downloads	2024-12-26 20:29:09 +01:00
Nicolas	4451c4f671	Nick:	2024-12-26 13:51:20 -03:00
Nicolas	bcc18e1c07	Merge branch 'main' into fix-sdk/next-in-when-502	2024-12-26 12:53:10 -03:00
Nicolas	4f65d350a3	Update package.json	2024-12-26 12:52:52 -03:00
Nicolas	c911aad228	Update package.json	2024-12-23 18:48:03 -03:00
Nicolas	b1a5625b22	Revert "Merge pull request #997 from mendableai/feat/sdk-without-ws" ... This reverts commit 53cda5f81c53d3de35925c610ce083923ca09fbe, reversing changes made to 51f79b55efadc53243a8c22d86bb2d08d878d524.	2024-12-23 18:45:51 -03:00
Nicolas	18ceaf10a5	Update .gitignore	2024-12-23 18:42:05 -03:00
Nicolas	0c1c4f2ede	Merge branch 'main' into feat/sdk-without-ws	2024-12-23 18:41:31 -03:00
Nicolas	67c643ad1c	Merge pull request #989 from RutamBhagat/js-sdk-conditionally-enforce-api-key ... feat(js-sdk): Make API key optional for self-hosted instances	2024-12-23 12:56:38 -03:00
RutamBhagat	ca2d3dc6d2	docs(credit-usage-api): add new endpoint documentation for credit usage	2024-12-21 06:24:53 -08:00
Thomas Kosmas	199bd2d1f4	Merge branch 'main' into feat/sdk-without-ws	2024-12-21 02:32:09 +02:00
Thomas Kosmas	a9d31c8e42	Merge branch 'main' into feat/sdk-without-ws	2024-12-21 02:30:40 +02:00
Thomas Kosmas	f043f5fd61	Enhance error handling in E2E tests and introduce CrawlWatcher tests ... - Updated error messages in E2E tests to provide clearer feedback for blocked URLs and invalid API keys. - Added new test suite for CrawlWatcher to ensure proper instantiation and error handling when WebSocket is unavailable. - Improved test conditions for URL scraping and crawling to reflect updated error responses.	2024-12-21 02:27:22 +02:00
Nicolas	d1f3e26f9e	Nick: blocklist string	2024-12-20 18:09:49 -03:00
Nicolas	ba95df96b1	Update rate-limiter.ts	2024-12-20 15:45:44 -03:00
Nicolas	6222152249	Nick: credit usage endpoint	2024-12-20 15:44:17 -03:00
rafaelmmiller	63bbeadbfc	Added try catch to message handler	2024-12-20 08:05:13 -03:00
RutamBhagat	2b488cac3d	chore: remove pytest dependency from pyproject.toml	2024-12-20 01:54:29 -08:00
Nicolas	3e60f175bb	Nick: prompt should be optional on /extract sdks	2024-12-19 16:14:49 -03:00
Nicolas	8063474c85	Update __init__.py	2024-12-19 16:14:17 -03:00
Nicolas	525a71d789	Update __init__.py	2024-12-19 16:10:42 -03:00
Nicolas	4fddc86e66	Update package.json	2024-12-19 16:09:08 -03:00
Nicolas	d67db99791	Merge pull request #994 from mendableai/feat/added-id-to-ws-sdks ... feat-SDK/added crawl id to ws	2024-12-19 16:08:08 -03:00
Nicolas	ed24853ca6	Merge pull request #996 from mendableai/fix/title-extra-info ... [BUG] fixed title extra info	2024-12-19 16:05:49 -03:00
Thomas Kosmas	c8cd0148dd	refactor: remove error logging for 'isows' module import in WebSocket initialization	2024-12-19 20:39:30 +02:00
Gergő Móricz	071b9a01c3	fix(scrapeURL/fire-engine): pass geolocation	2024-12-19 18:23:21 +01:00
Thomas Kosmas	6002bf3228	feat: dynamically import WebSocket module with error handling	2024-12-19 14:52:43 +02:00
rafaelmmiller	cf2ec77131	fixed title extra info	2024-12-19 08:32:10 -03:00
Nicolas	066071cd54	Update llmExtract.ts	2024-12-18 23:45:43 -03:00
Nicolas	05605112bb	Update extract.ts	2024-12-18 23:34:07 -03:00
Nicolas	2d37dca9dc	Nick: introduced system prompt to /extract	2024-12-18 22:10:41 -03:00
Nicolas	a759a7ab7a	Nick: small improvements	2024-12-18 21:45:06 -03:00
rafaelmmiller	19246f6289	feat-SDK/added crawl id to ws	2024-12-18 18:36:04 -03:00
Nicolas	e899ecbe44	Update llmExtract.ts	2024-12-18 16:52:05 -03:00
RutamBhagat	e776847c71	feat(js-sdk): improve API key handling for cloud vs self-hosted services in FirecrawlApp	2024-12-17 13:40:14 -08:00
RutamBhagat	29cea4c51d	feat(python-sdk): improve API key handling for cloud vs self-hosted services in FirecrawlApp	2024-12-17 13:31:35 -08:00
Móricz Gergő	bd36c441d3	feat(queue-worker): improve team-based logging	2024-12-17 22:06:36 +01:00
Móricz Gergő	780442d73b	feat: improve billing logging	2024-12-17 22:02:31 +01:00
Nicolas	ac187452c3	Nick: better filtering for urls that should be scraped	2024-12-17 17:34:55 -03:00
Nicolas	3b6edef9fa	chore: formatting	2024-12-17 16:58:57 -03:00
Nicolas	b9f621bed5	Nick: extract fixes	2024-12-17 16:58:35 -03:00
Nicolas	79e335636a	Nick: fixed extract issues	2024-12-17 16:40:45 -03:00
Nicolas	6d77879d68	Update extract.ts	2024-12-17 15:22:25 -03:00
Nicolas	e26a0a65a7	Merge branch 'main' of https://github.com/mendableai/firecrawl	2024-12-17 15:19:53 -03:00