1140 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
rafaelsideguide	b802ea02a1	small improvements ... - wait for getting results on crawl: sometimes crawl takes some a second to save the data on the db and this causes response.data to be empty - added timeout value to test script - increased http client timeout (llm extract was failing on e2e tests) - fixed env path on test script	2024-08-09 11:13:14 -03:00
KentHsu	b2e1b2ca68	chore: add go-sdk-tests job	2024-08-04 17:33:36 +08:00
KentHsu	1378ffc138	feat: add go-sdk	2024-08-04 17:33:33 +08:00
Nicolas	789c6cf5d7	Merge pull request #494 from mendableai/nsc/website-param-fixes ... Custom engine params fix	2024-08-02 19:29:03 -04:00
Nicolas	1742e4ceae	Nick:	2024-08-02 19:25:15 -04:00
Nicolas	311f812a96	Merge branch 'nsc/amazon'	2024-08-02 17:43:53 -04:00
Nicolas	39aecd974b	Update redis-health.ts	2024-08-02 17:43:45 -04:00
Nicolas	ddc0dac4e4	Merge pull request #492 from mendableai/nsc/amazon ... Fix AMZN | Removal of redis alerts	2024-08-02 14:52:09 -04:00
Nicolas	2e83a8a8b4	Delete check-redis.yml	2024-08-02 14:27:19 -04:00
Nicolas	b448e3c3ad	Update website_params.ts	2024-08-02 14:26:35 -04:00
Nicolas	893113a39e	Merge pull request #491 from mendableai/bugfix/issue-477 ... [Bug] Nested sitemaps	2024-08-02 13:03:28 -04:00
rafaelsideguide	4051630632	Update sitemap.ts	2024-08-02 11:32:48 -03:00
rafaelsideguide	8568b61015	bugfix for sitemaps	2024-08-02 11:03:01 -03:00
Nicolas	af68b7a785	Merge pull request #475 from mendableai/bugfix/issue-466 ... [Bug] pdfs and logging pdf events, also added trycatchs for docx	2024-08-01 22:05:26 -04:00
Nicolas	72eebb0aea	Merge pull request #485 from mendableai/bugfix/issue-435 ... [Bug] Issue with crawl going beyond Limit	2024-07-31 18:10:29 -04:00
rafaelsideguide	f48ff36b32	added .inc files and forced lower case comparison	2024-07-31 09:28:43 -03:00
Nicolas	4ab7fb007a	Merge pull request #484 from mendableai/nsc/fix-scaling-fix-email ... Fix parallelization issues with caching	2024-07-30 19:21:00 -04:00
Nicolas	ad6f6eff4b	Update fireEngine.ts	2024-07-30 19:15:54 -04:00
Nicolas	f9827b2151	Update credit_billing.ts	2024-07-30 19:13:17 -04:00
Nicolas	6d99dedd3c	Nick: fixed tests	2024-07-30 19:11:01 -04:00
Nicolas	a28ecc1f61	Nick: caching	2024-07-30 18:59:35 -04:00
Nicolas	86c61b5648	Merge pull request #483 from mendableai/feat/improv-self-host ... [Feat] Better self hosting guide	2024-07-30 17:08:41 -04:00
rafaelsideguide	c7a38a4ae2	Update SELF_HOST.md	2024-07-30 18:07:36 -03:00
rafaelsideguide	2d1ab43c27	Update SELF_HOST.md	2024-07-30 15:59:42 -03:00
Nicolas	065ede7ca6	Merge pull request #482 from mendableai/bugfix/switch-case-ratelimiter	2024-07-30 09:40:38 -04:00
rafaelsideguide	8f5174ffc7	Update auth.ts	2024-07-30 10:37:33 -03:00
rafaelsideguide	d25d7e7244	special case: developer.apple.com	2024-07-30 10:13:09 -03:00
Nicolas	473e0832a3	Merge branch 'nsc/fire-engine-first-layer'	2024-07-29 20:43:54 -04:00
Nicolas	5e8ffcf505	Update website_params.ts	2024-07-29 20:43:47 -04:00
Nicolas	aba95e5c17	Merge pull request #479 from mendableai/nsc/fire-engine-first-layer ... Fire-engine to be the default for our cloud-service	2024-07-29 20:36:10 -04:00
Nicolas	7b813883ef	Nick: first layer	2024-07-29 20:31:51 -04:00
Nicolas	8660df0eb0	Merge pull request #478 from mendableai/nsc/cacheable-lookup ... DNS Cacheable Lookup to avoid DNS blocking ops	2024-07-29 20:17:42 -04:00
Nicolas	e99c2568f4	Update auth.ts	2024-07-29 18:44:18 -04:00
Nicolas	968a2dc753	Nick:	2024-07-29 18:37:09 -04:00
Nicolas	04942bb9de	Nick:	2024-07-29 18:31:43 -04:00
Nicolas	267d4681bf	Merge branch 'main' of https://github.com/mendableai/firecrawl	2024-07-29 17:21:15 -04:00
Nicolas	b4833c1694	Nick: increasing default timeout to 45s	2024-07-29 17:21:11 -04:00
Nicolas	7fa08100bf	Merge pull request #414 from NiuBlibing/support_model_name ... support custom models	2024-07-29 13:21:29 -04:00
rafaelsideguide	49e3e64787	bugfix for pdfs and logging pdf events, also added trycatchs for docx	2024-07-29 14:13:46 -03:00
Nicolas	4c9d62f6d3	Nick: fixing sitemap fallback	2024-07-26 18:25:44 -04:00
Nicolas	091924a636	Nick: moving machines from mia to virginia	2024-07-26 17:37:46 -04:00
Nicolas	cb97871ff9	Merge branch 'main' of https://github.com/mendableai/firecrawl	2024-07-26 17:21:11 -04:00
Nicolas	ff4266f09e	Update pdfProcessor.ts	2024-07-26 17:21:09 -04:00
Nicolas	0c2e3a72cc	Merge pull request #460 from mendableai/nsc/admin-router ... Admin router + Improve redis notifications	2024-07-26 12:16:14 -04:00
rafaelsideguide	96cec2a673	fix checking scrape log success content length	2024-07-26 12:00:52 -03:00
Nicolas	542270f4c2	Merge pull request #461 from mendableai/nsc/small-handle-for-client-side-errors ... Client side error handling	2024-07-25 20:53:10 -04:00
Nicolas	dc6f825270	Update email_notification.ts	2024-07-25 20:43:50 -04:00
Nicolas	f82ca3be17	Nick:	2024-07-25 19:53:29 -04:00
Nicolas	01fab6e036	Update single_url.ts	2024-07-25 17:51:41 -04:00
Nicolas	56042d090c	Update single_url.ts	2024-07-25 17:48:44 -04:00