3 Commits

Author SHA1 Message Date
Gergő Móricz
e74e4bcefc feat(runWebScraper): retry a scrape max 3 times in a crawl if the status code is failure 2024-12-14 00:54:05 +01:00
Gergő Móricz
85cbfbb5bb fix(crawl): disable smart wait
This increases the reliability/deterministic-ness of crawls.
2024-12-10 21:12:31 +01:00
Gergő Móricz
877f072e3c feat: crawl log parser (poc) 2024-12-09 23:40:47 +01:00