2212 Commits

Author	SHA1	Message	Date
Nicolas	19cac2220f	Nick:	2024-10-23 16:31:01 -03:00
Nicolas	d8abd15716	Nick: from bulk to batch	2024-10-23 15:37:24 -03:00
Gergő Móricz	70c4e7c334	feat(bulk/scrape): check credits via url list length	2024-10-23 19:42:02 +02:00
Nicolas	66e505317e	Merge branch 'main' into mog/bulk-scrape	2024-10-23 14:36:26 -03:00
Nicolas	e0d3b761fc	Merge pull request #808 from mendableai/feat/skipTlsVerification ... feat: skipTlsVerification	2024-10-22 20:47:13 -03:00
Nicolas	d375bca167	Update acuc-cache-clear.ts	2024-10-22 20:42:59 -03:00
Nicolas	bbfdda8867	Nick: init	2024-10-22 19:47:23 -03:00
Thomas Kosmas	acde353e56	skipTlsVerification on robots.txt scraping	2024-10-23 01:07:03 +03:00
Thomas Kosmas	bd55464b52	skipTlsVerification	2024-10-22 22:28:02 +03:00
Gergő Móricz	6ed3104eb6	feat: clear ACUC cache endpoint based on team ID	2024-10-22 20:28:10 +02:00
Nicolas	d2344aa14b	Revert "Nick: improved map ranking algorithm" ... This reverts commit 7acd8d2edb6abc45a63fe1060377d2acb398ec36.	2024-10-21 16:11:32 -03:00
Nicolas	d31b85fa91	Merge pull request #793 from mendableai/fix/issue-665 ... [Bug] encoding error for special token	2024-10-21 12:24:46 -03:00
Nicolas	7acd8d2edb	Nick: improved map ranking algorithm	2024-10-19 13:27:47 -03:00
rafaelsideguide	aed11e72a6	fix encoding if error	2024-10-18 11:50:58 -03:00
Nicolas	79e65f31ef	Update v1.ts	2024-10-17 17:57:44 -03:00
Gergő Móricz	03b37998fd	feat: bulk scrape	2024-10-17 19:40:18 +02:00
Nicolas	081d7407b3	Merge pull request #788 from mendableai/nsc/log-extractpr-options ... Extractor options logging v1 fix	2024-10-16 23:51:22 -03:00
Nicolas	06b8d24a4c	Update scrape.ts	2024-10-16 23:50:21 -03:00
Nicolas	a73b06589c	Merge pull request #785 from mendableai/nsc/support-for-all-metadata ... Return all the website metadata	2024-10-16 23:37:26 -03:00
Nicolas	2ac50a16f5	Update metadata.ts	2024-10-16 23:37:07 -03:00
Nicolas	8974230db4	Nick: formatting + error handling	2024-10-16 23:35:03 -03:00
Nicolas	c0384ea381	Nick: added tests	2024-10-16 23:32:44 -03:00
Nicolas	417c7697c3	Update metadata.ts	2024-10-16 23:26:46 -03:00
Nicolas	ff906f7750	Update excludeTags.ts	2024-10-16 13:40:34 -03:00
Nicolas	2c1a98f019	Update excludeTags.ts	2024-10-16 13:37:40 -03:00
Nicolas	cf8fe93281	Update credit_billing.ts	2024-10-16 01:09:57 -03:00
Nicolas	e5a5ca2446	Update credit_billing.ts	2024-10-16 01:06:10 -03:00
Nicolas	027158fa44	Nick:	2024-10-15 21:47:27 -03:00
Nicolas	795e5a9228	Update metadata.ts	2024-10-15 21:36:13 -03:00
Nicolas	b4f6a0f919	Nick: geolocation	2024-10-15 21:12:33 -03:00
Nicolas	54a54b9f33	Nick: admin init	2024-10-15 17:28:28 -03:00
rafaelsideguide	4afcd16e02	performance improv for ws	2024-10-15 10:12:27 -03:00
rafaelsideguide	3afaab13d9	feat/improv-crawl-status-filters	2024-10-14 18:14:00 -03:00
rafaelsideguide	180801225b	fix/check files on crawl	2024-10-14 15:44:45 -03:00
Nicolas	e40036caf7	Merge branch 'main' of https://github.com/mendableai/firecrawl	2024-10-14 12:24:45 -03:00
Nicolas	c3a9630e33	Reapply "Merge pull request #773 from mendableai/nsc/retries-acuc-price-credits-fallback" ... This reverts commit a6888ce17b98e4a47fbfc9796911446a6513959c.	2024-10-14 12:24:34 -03:00
rafaelsideguide	2bf7b433e2	fixed file blocking process	2024-10-14 12:18:26 -03:00
rafaelsideguide	a6888ce17b	Revert "Merge pull request #773 from mendableai/nsc/retries-acuc-price-credits-fallback" ... This reverts commit ba9ad1ef7f02722ce88c1c29fa285befd3c3ec51, reversing changes made to 666082a7ddfa52242b4dd27f2c5f37f1f98a3302.	2024-10-14 10:32:09 -03:00
Nicolas	821c62c575	Update credit_billing.ts	2024-10-13 22:30:11 -03:00
Nicolas	78b6127d88	Nick: retries for acuc	2024-10-13 22:27:38 -03:00
Nicolas	03287821c2	Update index.ts	2024-10-12 19:49:37 -03:00
Nicolas	d3856371c9	Update index.ts	2024-10-12 19:36:49 -03:00
Nicolas	af06b42cb2	Update fireEngine.ts	2024-10-12 18:18:38 -03:00
Nicolas	35b15f1ee6	Update fireEngine.ts	2024-10-12 17:59:50 -03:00
Nicolas	961b1010cf	Nick: rm the cache for map for 24hrs	2024-10-12 17:48:37 -03:00
Nicolas	5f16688bd4	Merge pull request #766 from mendableai/doc/issue-764 ... [Doc] Better explained how includePaths and excludePaths work	2024-10-11 15:48:19 -03:00
rafaelsideguide	2d3d7c827a	fix/added unkwown status to job filter	2024-10-11 15:40:29 -03:00
Rafael Miller	ca51521625	Merge pull request #761 from mendableai/fix/filter-status-unknown-jobs ... [BUG] filters failed and unknown jobs now	2024-10-11 15:36:16 -03:00
Nicolas	0bff5b1a24	Update auth.ts	2024-10-11 15:29:25 -03:00
Nicolas	257a951132	Update auth.ts	2024-10-11 14:21:04 -03:00