23 Commits

Author	SHA1	Message	Date
Nicolas	e5b797549e	Merge branch 'main' into feat/scrape-monitoring	2024-07-25 16:21:02 -04:00
rafaelsideguide	309728a482	updated logs	2024-07-25 09:48:06 -03:00
Gergo Moricz	7cd9bf92e3	feat: scrape event logging to DB	2024-07-24 14:31:25 +02:00
rafaelsideguide	4381109dd8	added default values and fixed pdf bug	2024-06-26 09:00:54 -03:00
Rafael Miller	3e2e76311c	Merge branch 'main' into feat/issue-205	2024-06-14 11:25:20 -03:00
rafaelsideguide	676d6e8ab5	Added pageOptions.removeTags	2024-06-13 10:51:05 -03:00
rafaelsideguide	e37d151404	added parsePDF option to pageOptions ... user can decide if they are going to let us take care of the parse or they are going to parse the pdf by themselves	2024-06-12 15:06:47 -03:00
rafaelsideguide	01c9f071fa	fixed	2024-06-12 11:27:06 -03:00
rafaelsideguide	ee282c3d55	Added allowBackwardCrawling option	2024-06-11 15:24:39 -03:00
rafaelsideguide	184e4678f1	bugfix on idempotency key check	2024-05-23 11:47:04 -03:00
rafaelsideguide	3f460af6c5	Added idempotency key to crawl route	2024-05-07 15:29:27 -03:00
Nicolas	bdbee963f7	Merge branch 'main' into nsc/cancel-job	2024-05-07 10:13:43 -07:00
rafaelsideguide	e1f52c538f	nested includeHtml inside pageOptions	2024-05-07 13:40:24 -03:00
Nicolas	6d5da358cc	Nick: cancel job	2024-05-06 17:16:43 -07:00
rafaelsideguide	509250c4ef	changed to includeHtml	2024-05-06 19:45:56 -03:00
rafaelsideguide	538355f1af	Added toMarkdown option	2024-05-06 11:36:44 -03:00
Nicolas	f3c190c21c	Nick:	2024-04-23 16:47:24 -07:00
rafaelsideguide	849c0b6ebf	[Feat] Added blocklist for social media urls	2024-04-23 18:50:35 -03:00
Nicolas	898d729a84	Nick: tests	2024-04-21 11:27:31 -07:00
Nicolas	5cdbf3a0ac	Nick: cleaner functions to handle authenticated requests that dont require ifs everywhere	2024-04-21 10:36:48 -07:00
Caleb Peffer	be75aaa195	Caleb: first version of supabase proxy to make db authentication optional	2024-04-21 09:31:22 -07:00
Nicolas	0db0874b00	Nick:	2024-04-20 19:37:45 -07:00
Nicolas	23b2190e5d	Nick:	2024-04-20 16:38:05 -07:00