AI/firecrawl
1
0
Fork 0
mirror of https://git.mirrors.martin98.com/https://github.com/mendableai/firecrawl synced 2025-08-19 23:29:07 +08:00
Code Issues Packages Projects Releases Wiki Activity
3,127 Commits 496 Branches 17 Tags
Commit Graph

1989 Commits

Author SHA1 Message Date
Nicolas
ef71940429
Merge pull request #674 from mendableai/nsc/map-pagination 
v1/map improvements + higher limits
 2024-09-19 11:31:23 -04:00
skeptrune
26825fc8fb bugfix: using onlyIncludeTags and removeTags together 2024-09-19 00:29:13 -07:00
Nicolas
00bb958b32 Update v1-openapi.json 2024-09-18 15:59:11 -04:00
Nicolas
03eb47d426 Merge branch 'main' of https://github.com/mendableai/firecrawl 2024-09-18 15:58:54 -04:00
Nicolas
3a4dd8fc7e Nick: v1 openapi spec 2024-09-18 15:58:53 -04:00
Gergő Móricz
712ca31615 minor fixes 2024-09-18 21:34:09 +02:00
Gergő Móricz
093c064bff feat(v1): add public actions api 2024-09-18 20:39:25 +02:00
Gergő Móricz
42d677fe3c feat(fire-engine): port waitFor and screenshot to use actions 2024-09-18 20:04:54 +02:00
Gergő Móricz
c28e1e2959 fix(v1/zod): formats: add error message if both sc and sc@fP 2024-09-18 19:35:38 +02:00
Nicolas
3c2bfe2da2
Update single_url.ts 2024-09-17 01:58:47 -04:00
Nicolas
18b024c238 Update single_url.ts 2024-09-17 01:41:46 -04:00
Nicolas
a4039bd008 Revert "Update single_url.ts" 
This reverts commit 0f8c0a570dca877d14d590e6002eaffd345a3927.
 2024-09-16 23:36:38 -04:00
Nicolas
0f8c0a570d
Update single_url.ts 2024-09-16 21:44:56 -04:00
Nicolas
9da3432596 Update map.ts 2024-09-16 12:03:14 -04:00
Nicolas
af4804e13b Merge branch 'main' into nsc/map-pagination 2024-09-16 11:04:44 -04:00
Nicolas
030ecab6ee Update rate-limiter.ts 2024-09-13 18:09:59 -04:00
Gergo Moricz
000a316cc3 fix(fire-engine): poll more frequently 2024-09-13 16:41:27 +02:00
Gergo Moricz
f7c4cee404 fix(queue-worker): don't send LLM extract hallucination error to Sentry 2024-09-13 14:02:49 +02:00
Gergő Móricz
d497284b40 feat(api/queue): auto-remove completed jobs after 25 hours 2024-09-12 19:47:22 +02:00
Nicolas
620b02f9ca Nick: 2024-09-12 12:51:14 -04:00
Nicolas
e1171ade1b Merge branch 'main' into mog/fix-status-job-get 2024-09-12 11:35:36 -04:00
Nicolas
4cd1065ae2 Update rate-limiter.ts 2024-09-11 14:03:34 -04:00
Gergő Móricz
97ffabff3a fix(v1): converting bad docs always gives null 2024-09-10 21:21:20 +02:00
Gergő Móricz
83a165db0f fix(v0/scrape): ensure url is string 2024-09-10 21:18:53 +02:00
Gergő Móricz
ad1a6fbc74 fix(v1/map): handle invalid URLs gracefully 2024-09-10 19:41:01 +02:00
Gergő Móricz
4ebc35c9dd fix(crawl-status): add success: true 2024-09-10 18:59:09 +02:00
Nicolas
a7406031fa Merge branch 'main' of https://github.com/mendableai/firecrawl 2024-09-10 06:53:30 -03:00
Nicolas
a17e1cac92 Rate bump 2024-09-10 06:53:24 -03:00
Gergo Moricz
b4dbf75537 fix(v1): check if url is string in blocklistMiddleware 
Fixes FIRECRAWL-SCRAPER-JS-9Z
 2024-09-10 10:25:14 +02:00
Gergo Moricz
26f2095de6 fix(v1): proper Invalid URL handling 2024-09-10 09:24:23 +02:00
Gergo Moricz
f8fbc71f91 fix(supabase-jobs): do not use RPCs 
RPCs are more failure-prone for this use case than regular queries are.
 2024-09-10 09:20:18 +02:00
Gergo Moricz
a6bcf7b438 fix(v0/crawl-status): don't crash on big crawls when requesting jobs from supabase 2024-09-10 08:53:40 +02:00
Nicolas
17e419a7fb Nick: 2024-09-09 21:06:23 -03:00
Nicolas
ca9a781eb7 Update index.ts 2024-09-09 12:27:55 -03:00
Nicolas
22a5e85899 Update index.ts 2024-09-09 12:26:55 -03:00
Nicolas
60a15d00eb Update types.ts 2024-09-08 16:39:12 -03:00
Nicolas
fbdfa1256b Update credit_billing.ts 2024-09-08 13:07:10 -03:00
Nicolas
48c665519e Update credit_billing.ts 2024-09-07 13:42:45 -03:00
Nicolas
32097fa29d Merge branch 'main' of https://github.com/mendableai/firecrawl 2024-09-07 13:12:52 -03:00
Nicolas
5758af3291 Update website_params.ts 2024-09-07 13:12:46 -03:00
Nicolas
79870e7305 Update excludeTags.ts 2024-09-06 20:15:26 -03:00
Nicolas
a0f9ab2be7 Update map.ts 2024-09-06 20:14:47 -03:00
Nicolas
f5b84e15e1 Update sitemap.ts 2024-09-05 17:52:27 -03:00
rafaelsideguide
82d6bf4ec8 feat(go-parser): singleton 2024-09-05 16:14:21 -03:00
Nicolas
cb630bfc34 Update fireEngine.ts 2024-09-05 14:24:10 -03:00
rafaelsideguide
8c1097e9e1 fix: pageOptions 2024-09-05 14:16:31 -03:00
rafaelsideguide
b301ffc922 added missing variables 2024-09-05 13:57:26 -03:00
Nicolas
c6f1d80992 Update crawl.ts 2024-09-05 13:03:43 -03:00
Nicolas
eb03a81152 Update crawl-status.ts 2024-09-05 12:55:04 -03:00
Sebastjan Prachovskij
503e83e83e
Add SearchApi to search 
Add support for engines, improve status code error

Remove changes in package, add engine to env params

Improve description in env example

Remove unnecessary empty line

Improve text
 2024-09-05 18:36:59 +03:00