AI/firecrawl
1
0
Fork 0
mirror of https://git.mirrors.martin98.com/https://github.com/mendableai/firecrawl synced 2025-08-14 14:45:59 +08:00
Code Issues Packages Projects Releases Wiki Activity

Update blocklist.ts

Browse Source
This commit is contained in:
Nicolas 2024-05-24 14:53:04 -07:00
parent 4ce28593b2
commit e5c8719554
1 changed files with 0 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
apps/api/src/scraper/WebScraper/utils/blocklist.ts
View File

@ -1,7 +1,6 @@
const socialMediaBlocklist = [ const socialMediaBlocklist = [
'facebook.com', 'facebook.com',
'twitter.com', 'twitter.com',
'x.com',
'instagram.com', 'instagram.com',
'linkedin.com', 'linkedin.com',
'pinterest.com', 'pinterest.com',