AI/firecrawl
1
0
Fork 0
mirror of https://git.mirrors.martin98.com/https://github.com/mendableai/firecrawl synced 2025-06-22 01:20:40 +08:00
Code Issues Packages Projects Releases Wiki Activity
3,412 Commits 496 Branches 17 Tags
Commit Graph

3412 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Gergő Móricz
c22c87ab0a fix(crawler): check for more strings 2025-03-05 18:48:05 +01:00
Nicolas
4902d0ac64 Nick: fixes 2025-03-04 23:22:58 -03:00
Nicolas
8cfc946cae Nick: fix system prompt missing from extract params 2025-03-04 23:04:20 -03:00
Eric Ciarla
5a1886936c
Truncate llmstxt cache based on maxurls limit & improve maxurls handling (#1285) 
* init

* Update generate-llmstxt-service.ts
 2025-03-03 18:37:33 -03:00
Gergő Móricz
1beadf39f0 fix 2025-03-03 21:06:01 +01:00
Gergő Móricz
6dce8e9ddf asd 2025-03-03 21:05:45 +01:00
Gergő Móricz
8620bf3d5e fix(auth): split load evenly between two instances 2025-03-03 20:53:51 +01:00
Gergő Móricz
67ee2662de feat(auth): force acuc on read replica 2025-03-03 20:36:54 +01:00
Nicolas
64af3ba728 Revert "Update auth.ts" 
This reverts commit fea249c568c4a8d63bcc2e560fd3655e73350be6.
 2025-03-03 16:26:42 -03:00
Eric Ciarla
72eb360eba
Merge pull request #1291 from aparupganguly/feature/claude3.7-extractor 
examples/Add Claude 3.7 web extractor
 2025-03-03 13:41:47 -05:00
Aparup Ganguly
bced299e36 examples/Add Claude 3.7 web extractor 2025-03-03 17:48:52 +05:30
Nicolas
9eb25786dc Update deep-research-service.ts 2025-03-02 18:16:20 -03:00
Nicolas
aa54fd1658 Nick: only new activities/sources in the callback 2025-03-02 17:46:12 -03:00
Nicolas
33c59ee474 Nick: fixes 2025-03-02 17:41:19 -03:00
Nicolas
30c1e92a2e Update deep-research-service.ts 2025-03-02 17:37:15 -03:00
Nicolas
22d4f0de38
(feat/deep-research) Alpha prep + Improvements (#1284) 
* Update index.ts

* Nick:

* Nick: topic -> query

* Update deep-research.ts

* Nick: bump

* Nick: add onSource callback
 2025-03-02 17:33:21 -03:00
Gergő Móricz
9ad947884d
feat(tests/snips): add billing tests + misc billing fixes (FIR-1280) (#1283) 
* feat(tests/snips): add billing tests + misc billing fixes

* add testing key

* asd
 2025-03-02 16:51:42 -03:00
Gergő Móricz
4f25f12a12
fix(ai): handle if AI returns a JSON code block (#1280) 2025-03-02 15:25:24 -03:00
Eric Ciarla
522f2d2e3b
Merge pull request #1267 from ceewaigit/main 
Add groq_web_crawler example and dependencies
 2025-03-02 11:32:04 -05:00
Jean-Marie Porchet
42e92216b6
docs: remove undefined "required" field (#1282) 2025-03-02 17:27:52 +01:00
Gergő Móricz
e8c698d613
feat(crawler): handle cross-origin redirects differently than same-origin redirects (#1279) 2025-03-02 13:32:46 +01:00
Nicolas
fea249c568 Update auth.ts 2025-03-02 02:40:34 -03:00
Nicolas
99e61c967c Update index.ts 2025-03-02 02:39:52 -03:00
Eric Ciarla
ab8dcab6e2
Merge pull request #1276 from aparupganguly/feature/gpt4.5-crawler 2025-02-28 14:57:30 +01:00
Aparup Ganguly
06cdd988a4 examples/Add gpt 4.5 web crawler 2025-02-28 18:17:32 +05:30
Gergő Móricz
05c29e8299 js-sdk: bump 2025-02-28 12:40:39 +01:00
Gergő Móricz
856ec37d4c fix(ci/js-sdk): properly build SDK before publishing 2025-02-28 12:34:53 +01:00
Gergő Móricz
904e69bfbc
feat(supabase): add read replica routing (#1274) 2025-02-28 09:52:26 +01:00
Nicolas
39b61132bf Nick: fixed js sdk 2025-02-27 20:39:53 -03:00
Gergő Móricz
44bf59229a fix(acuc): cache for 1 hour 2025-02-27 21:36:33 +01:00
Nicolas
b72e21a697
Nick: batch billing (#1264) 2025-02-27 20:18:03 +01:00
Nicolas
289e351c14
(feat/deep-research-alpha) Added Max Urls, Sources and Fixes (#1271) 
* Nick: fixes

* Nick:

* Update deep-research-status.ts
 2025-02-27 13:24:40 -03:00
Gergő Móricz
1d3757b391 bump map to 30k 2025-02-27 12:44:23 +01:00
Benjy
78334e4ef4
feat(self-host/ai): pass in the ollama envs into docker compose (#1269) 2025-02-27 11:56:31 +01:00
Grass Huang
7bf04d409a
fix(scraper): improve charset detection regex to accurately parse meta tags (#1265) 2025-02-26 17:31:06 +01:00
ceewaigithub
75ac980fe4 Add groq_web_crawler example and dependencies 2025-02-26 22:24:38 +08:00
Nicolas
bf1a79588e Merge branch 'feat/fire-index' 2025-02-25 21:01:30 -03:00
Nicolas
31df234127 Update log_job.ts 2025-02-25 21:01:05 -03:00
Nicolas
ec90aaffd6 Update log_job.ts 2025-02-25 21:01:00 -03:00
Nicolas
0f0520391a
Update log_job.ts (#1263) 2025-02-25 19:35:57 -03:00
Nicolas
59d09f5c45 Update log_job.ts 2025-02-25 19:32:16 -03:00
Gergő Móricz
115b6b61c4 add initial codeowners 2025-02-25 14:28:09 +01:00
Gergő Móricz
51bc7757ef
fix(self-host/compose): pass SearXNG params 2025-02-25 11:19:27 +01:00
Eric Ciarla
2da6d7bdb4
Merge pull request #1257 from aparupganguly/feature/claude3.7-crawler 
Implemented claude 3.7 web crawler
 2025-02-24 15:46:52 -05:00
Gergő Móricz
8c42b08b7e
feat(v1/crawl-status-ws): update behavior to ignore errors like regular crawl-status (#1234) 2025-02-24 21:44:29 +01:00
Aparup Ganguly
6508afc624 Implemented claude 3.7 2025-02-25 01:54:25 +05:30
Gergő Móricz
15489be542
feat(self-host/ai): use any OpenAI-compatible API (#1245) 2025-02-23 09:07:32 +01:00
Gergő Móricz
b88b573909 docker: force host to 0.0.0.0 to fix env precedence issues 
Fixes #1225
 2025-02-23 08:49:31 +01:00
Nicolas
b24ac0f6b5
Nick: done (#1237) 2025-02-22 20:18:46 -03:00
Nicolas
5ab86b8b43
(fix/token-slicer) Fixes extract token limit issues (#1236) 
* Nick: fixes extract token limit errors

* Update llmExtract.ts

* Update llmExtract.ts
 2025-02-21 20:44:42 +01:00