AI/firecrawl
1
0
Fork 0
mirror of https://git.mirrors.martin98.com/https://github.com/mendableai/firecrawl synced 2025-08-18 16:55:59 +08:00
Code Issues Packages Projects Releases Wiki Activity
3,127 Commits 496 Branches 17 Tags
Commit Graph

2446 Commits

Author SHA1 Message Date
Nicolas
99c1af0a9f Update package.json 2024-09-11 14:59:36 -04:00
Gergő Móricz
5adfd74cc5 feat(js-sdk/test): add API_URL env var 2024-09-11 20:32:34 +02:00
Gergő Móricz
ad70c30be5 fix(js-sdk): check at bad if 2024-09-11 20:31:58 +02:00
Gergő Móricz
6e1cf2f40d feat(js-sdk): fixes, update tests 2024-09-11 20:15:43 +02:00
Nicolas
5791d3a0e5 Merge branch 'main' of https://github.com/mendableai/firecrawl 2024-09-11 14:03:36 -04:00
Nicolas
4cd1065ae2 Update rate-limiter.ts 2024-09-11 14:03:34 -04:00
Gergő Móricz
51da038405
Merge branch 'main' into fix-attw 2024-09-11 19:45:13 +02:00
Gergő Móricz
75f4bcd2ad
Merge pull request #614 from MonsterDeveloper/dependencies 
perf(js-sdk): move `dotenv` and `uuid` to `devDependencies`, fix `zod` import
 2024-09-11 19:34:50 +02:00
Andrei Bobkov
f6fc71b46a
fix(js-sdk): bring back cjs exports 2024-09-11 17:53:17 +03:00
Gergő Móricz
97ffabff3a fix(v1): converting bad docs always gives null 2024-09-10 21:21:20 +02:00
Gergő Móricz
83a165db0f fix(v0/scrape): ensure url is string 2024-09-10 21:18:53 +02:00
Gergő Móricz
ad1a6fbc74 fix(v1/map): handle invalid URLs gracefully 2024-09-10 19:41:01 +02:00
Gergő Móricz
e19f7a102e feat(js-sdk): paginate next on checkCrawlStatus + better types for CSR 2024-09-10 19:29:38 +02:00
Gergő Móricz
4ebc35c9dd fix(crawl-status): add success: true 2024-09-10 18:59:09 +02:00
rafaelsideguide
f855ad3436 bumping py-sdk version 2024-09-10 10:29:44 -03:00
rafaelsideguide
ee8a54213c fix(py-sdk): removed asyncio package 
tested websocket with example.py without asyncio and it works with no problem.
 2024-09-10 10:25:27 -03:00
rafaelsideguide
1074e976cd Merge branch 'main' of https://github.com/mendableai/firecrawl 2024-09-10 09:09:43 -03:00
rafaelsideguide
45237a29dd updated js-sdk examples 2024-09-10 09:09:39 -03:00
Nicolas
a7406031fa Merge branch 'main' of https://github.com/mendableai/firecrawl 2024-09-10 06:53:30 -03:00
Nicolas
a17e1cac92 Rate bump 2024-09-10 06:53:24 -03:00
Gergo Moricz
b4dbf75537 fix(v1): check if url is string in blocklistMiddleware 
Fixes FIRECRAWL-SCRAPER-JS-9Z
 2024-09-10 10:25:14 +02:00
Gergo Moricz
26f2095de6 fix(v1): proper Invalid URL handling 2024-09-10 09:24:23 +02:00
Gergo Moricz
f8fbc71f91 fix(supabase-jobs): do not use RPCs 
RPCs are more failure-prone for this use case than regular queries are.
 2024-09-10 09:20:18 +02:00
Gergo Moricz
a6bcf7b438 fix(v0/crawl-status): don't crash on big crawls when requesting jobs from supabase 2024-09-10 08:53:40 +02:00
Nicolas
17e419a7fb Nick: 2024-09-09 21:06:23 -03:00
Nicolas
ca9a781eb7 Update index.ts 2024-09-09 12:27:55 -03:00
Nicolas
22a5e85899 Update index.ts 2024-09-09 12:26:55 -03:00
Nicolas
60a15d00eb Update types.ts 2024-09-08 16:39:12 -03:00
Nicolas
fbdfa1256b Update credit_billing.ts 2024-09-08 13:07:10 -03:00
Nicolas
48c665519e Update credit_billing.ts 2024-09-07 13:42:45 -03:00
Nicolas
32097fa29d Merge branch 'main' of https://github.com/mendableai/firecrawl 2024-09-07 13:12:52 -03:00
Nicolas
5758af3291 Update website_params.ts 2024-09-07 13:12:46 -03:00
Nicolas
79870e7305 Update excludeTags.ts 2024-09-06 20:15:26 -03:00
Nicolas
a0f9ab2be7 Update map.ts 2024-09-06 20:14:47 -03:00
Nicolas
f5b84e15e1 Update sitemap.ts 2024-09-05 17:52:27 -03:00
rafaelsideguide
82d6bf4ec8 feat(go-parser): singleton 2024-09-05 16:14:21 -03:00
Nicolas
cb630bfc34 Update fireEngine.ts 2024-09-05 14:24:10 -03:00
rafaelsideguide
8c1097e9e1 fix: pageOptions 2024-09-05 14:16:31 -03:00
rafaelsideguide
b301ffc922 added missing variables 2024-09-05 13:57:26 -03:00
Nicolas
c6f1d80992 Update crawl.ts 2024-09-05 13:03:43 -03:00
Nicolas
eb03a81152 Update crawl-status.ts 2024-09-05 12:55:04 -03:00
Sebastjan Prachovskij
503e83e83e
Add SearchApi to search 
Add support for engines, improve status code error

Remove changes in package, add engine to env params

Improve description in env example

Remove unnecessary empty line

Improve text
 2024-09-05 18:36:59 +03:00
Nicolas
34adf43200 Merge branch 'main' into feat/go-html-to-md-parser 2024-09-05 12:13:14 -03:00
Nicolas
82cb80c817 Update map.ts 2024-09-04 23:46:18 -03:00
Nicolas
a0113dac37 Update credit_billing.ts 2024-09-04 16:54:20 -03:00
Nicolas
b7b99b53da Merge branch 'main' into nsc/check-credits-optimization 2024-09-04 16:39:18 -03:00
rafaelsideguide
85b824e122 test: what about false false? 2024-09-04 16:35:32 -03:00
rafaelsideguide
78edf13ec6 test: usedbauth envs wth 2024-09-04 16:31:42 -03:00
rafaelsideguide
cb8571abad fix: enforced dotenv config 2024-09-04 15:57:57 -03:00
Nicolas
5ecb243693 Nick: 2024-09-04 15:19:45 -03:00