AI/firecrawl
1
0
Fork 0
mirror of https://git.mirrors.martin98.com/https://github.com/mendableai/firecrawl synced 2025-08-12 18:38:59 +08:00
Code Issues Packages Projects Releases Wiki Activity

added library.tiktok to allowedKeywords

Browse Source
This commit is contained in:
rafaelmmiller 2024-11-27 10:10:43 -03:00
parent ea1d3d7025
commit b69c6f9f95
1 changed files with 1 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
apps/api/src/scraper/WebScraper/utils/blocklist.ts
View File

@ -38,6 +38,7 @@ const allowedKeywords = [
'press', 'press',
'conditions', 'conditions',
'tos', 'tos',
'://library.tiktok.com',
'://ads.tiktok.com', '://ads.tiktok.com',
'://tiktok.com/business', '://tiktok.com/business',
'://developers.facebook.com' '://developers.facebook.com'