1170 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Quan Ming	a96ad4b0e2	Update redis url to use comment	2024-08-10 12:33:26 +08:00
Quan Ming	0221872a70	Update redis urls in example .env	2024-08-10 00:40:11 +08:00
Nicolas	bf1d325aa9	Merge pull request #512 from mendableai/nsc/digikey-tls ... Digikey support	2024-08-08 12:34:18 -04:00
Nicolas	f1f5605010	Update website_params.ts	2024-08-08 12:31:58 -04:00
Nicolas	b0abad07da	Merge pull request #496 from tak-s/improve-logging-level ... Improve logs	2024-08-07 22:01:12 -04:00
Nicolas	8a992b1596	Merge branch 'main' of https://github.com/mendableai/firecrawl	2024-08-07 10:40:06 -04:00
Nicolas	b12e1157cc	Nick: v35 bump	2024-08-07 10:40:00 -04:00
Gergo Moricz	fe9fdb578b	revert bad hotfixes	2024-08-07 16:34:25 +02:00
Gergo Moricz	b7c01dcb9b	fix(webScraperQueue): reduce retries to 2	2024-08-07 16:31:50 +02:00
Gergo Moricz	cdf7bad5b4	fix(runWebScraper): don't move to completed	2024-08-07 15:20:56 +02:00
Gergo Moricz	9df8719efa	fix(queue-worker): raise queue log level to info	2024-08-07 14:56:04 +02:00
Gergo Moricz	7bb922071c	fix(queue-worker): manually renew lock (testing)	2024-08-07 14:35:20 +02:00
Gergo Moricz	8216266d16	fix(scrape_log): display error properly	2024-08-07 14:19:20 +02:00
Gergo Moricz	2e2e80d679	fix(scrape-events): updateScrapeResult fix	2024-08-07 14:17:50 +02:00
Gergo Moricz	b5ec47fd96	fix(runWebScraper): don't fetch next job	2024-08-07 13:53:04 +02:00
Nicolas	44c9a22731	Merge pull request #508 from mendableai/mog/js-sdk-cjs ... fix(js-sdk): build both CJS and EJS versions	2024-08-06 19:46:20 -04:00
Gergo Moricz	020a5efdb7	Revert "Revert "Merge pull request #432 from mendableai/mog/js-sdk-cjs"" ... This reverts commit 5da4472842497dbe4d462609293917e0b2199fc8.	2024-08-07 01:27:26 +02:00
Gergő Móricz	7380d7799f	Merge branch 'main' into mog/js-sdk-cjs	2024-08-07 01:12:36 +02:00
Gergo Moricz	5f7724205f	fix(js-sdk): re-add types	2024-08-07 01:06:21 +02:00
Nicolas	f294d3922c	Nick: revert	2024-08-06 18:44:45 -04:00
Nicolas	5da4472842	Revert "Merge pull request #432 from mendableai/mog/js-sdk-cjs" ... This reverts commit bb90e03dea5dc2f475dbf8dc70425af9b3dfe246, reversing changes made to 3321ca9398b5a129ffc1689ec2a88b2468cf16e9.	2024-08-06 18:41:06 -04:00
Nicolas	a67a5c04c9	Revert "Merge pull request #432 from mendableai/mog/js-sdk-cjs" ... This reverts commit bb90e03dea5dc2f475dbf8dc70425af9b3dfe246, reversing changes made to 3321ca9398b5a129ffc1689ec2a88b2468cf16e9.	2024-08-06 18:02:56 -04:00
Nicolas	bb90e03dea	Merge pull request #432 from mendableai/mog/js-sdk-cjs ... fix(js-sdk): build both CommonJS and ESM versions	2024-08-06 17:38:57 -04:00
Nicolas	3321ca9398	Merge pull request #504 from mendableai/feat/fullpage-screenshot ... [Feat] Added fullpagescreenshot capabilities	2024-08-06 13:52:29 -04:00
rafaelsideguide	4d24a99d50	fix params	2024-08-06 09:34:43 -03:00
Nicolas	72f2c3616f	Merge pull request #503 from mendableai/bugfix/empty-excludes ... [Bug] Fixed the empty excludes.filter is undefined bug	2024-08-05 20:42:21 -04:00
rafaelsideguide	3edc3a3d15	added fullpagescreenshot capabilities, wip on fire-engine side	2024-08-05 18:17:37 -03:00
rafaelsideguide	f32e8de156	fixes the empty excludes.filter undefined bug	2024-08-05 18:13:31 -03:00
tak-s	af9bc5c8bb	Suppressed repetitive logs	2024-08-04 15:09:36 +09:00
tak-s	db926a4146	set LOGGING_LEVEL to environment	2024-08-04 15:05:53 +09:00
Nicolas	789c6cf5d7	Merge pull request #494 from mendableai/nsc/website-param-fixes ... Custom engine params fix	2024-08-02 19:29:03 -04:00
Nicolas	1742e4ceae	Nick:	2024-08-02 19:25:15 -04:00
Nicolas	311f812a96	Merge branch 'nsc/amazon'	2024-08-02 17:43:53 -04:00
Nicolas	39aecd974b	Update redis-health.ts	2024-08-02 17:43:45 -04:00
Nicolas	ddc0dac4e4	Merge pull request #492 from mendableai/nsc/amazon ... Fix AMZN | Removal of redis alerts	2024-08-02 14:52:09 -04:00
Nicolas	2e83a8a8b4	Delete check-redis.yml	2024-08-02 14:27:19 -04:00
Nicolas	b448e3c3ad	Update website_params.ts	2024-08-02 14:26:35 -04:00
Nicolas	893113a39e	Merge pull request #491 from mendableai/bugfix/issue-477 ... [Bug] Nested sitemaps	2024-08-02 13:03:28 -04:00
rafaelsideguide	4051630632	Update sitemap.ts	2024-08-02 11:32:48 -03:00
rafaelsideguide	8568b61015	bugfix for sitemaps	2024-08-02 11:03:01 -03:00
Nicolas	af68b7a785	Merge pull request #475 from mendableai/bugfix/issue-466 ... [Bug] pdfs and logging pdf events, also added trycatchs for docx	2024-08-01 22:05:26 -04:00
Nicolas	72eebb0aea	Merge pull request #485 from mendableai/bugfix/issue-435 ... [Bug] Issue with crawl going beyond Limit	2024-07-31 18:10:29 -04:00
rafaelsideguide	f48ff36b32	added .inc files and forced lower case comparison	2024-07-31 09:28:43 -03:00
Nicolas	4ab7fb007a	Merge pull request #484 from mendableai/nsc/fix-scaling-fix-email ... Fix parallelization issues with caching	2024-07-30 19:21:00 -04:00
Nicolas	ad6f6eff4b	Update fireEngine.ts	2024-07-30 19:15:54 -04:00
Nicolas	f9827b2151	Update credit_billing.ts	2024-07-30 19:13:17 -04:00
Nicolas	6d99dedd3c	Nick: fixed tests	2024-07-30 19:11:01 -04:00
Nicolas	a28ecc1f61	Nick: caching	2024-07-30 18:59:35 -04:00
Nicolas	86c61b5648	Merge pull request #483 from mendableai/feat/improv-self-host ... [Feat] Better self hosting guide	2024-07-30 17:08:41 -04:00
rafaelsideguide	c7a38a4ae2	Update SELF_HOST.md	2024-07-30 18:07:36 -03:00