482 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Yanlong Wang	b2ba2f9459	fix: beware cf-browser-rendering capacity	2025-04-02 20:10:33 +08:00
Yanlong Wang	f9223d78eb	saas: use compression	2025-04-02 19:16:42 +08:00
Yanlong Wang	1bf0568489	fix: serp usage	2025-04-02 16:41:05 +08:00
Yanlong Wang	f4d0eade60	fix: serp index	2025-04-02 16:17:30 +08:00
Yanlong Wang	b58a2db69a	fix: add 0.0.0.0/32 to the non Public CIDRs (#1181) ... * fix: add to the non Public CIDRs (#1177) * fix: move 0.0.0.0 to bottom, add cidr suffix --------- Co-authored-by: Cr4zy Uru5 <crazyurus@vip.qq.com>	2025-04-02 15:26:43 +08:00
Yanlong Wang	12ba1bcfad	feat: serp endpoint (#1180) ... * wip * wip * fix * wip * fix: add jitter to user cache * cd * fix * fix * fix: user cache age comparison * fix: try to partition apiroll query * bump: deps * wip * cd * feat: fallback for serp * fix * cd * fix * fix * serp: stop hiding expense * serp: enable fallback by default	2025-04-02 14:58:13 +08:00
Aaron Ji	919a81d7c9	chore: check RTL language	2025-04-02 14:28:26 +08:00
Aaron Ji	a0ca9ce2df	chore: enable fallback by default for s.jina.ai	2025-04-02 13:26:48 +08:00
Aaron Ji	69e02cb6e9	feat: support fallback query (#1179) ... * feat: support fallback query * chore: update scaler value	2025-04-02 13:14:30 +08:00
yanlong.wang	1a3a5ad355	bump: deps	2025-04-01 19:39:17 +08:00
yanlong.wang	4498af7280	fix: try to partition apiroll query	2025-04-01 18:28:48 +08:00
yanlong.wang	0238d26984	fix: user cache age comparison	2025-04-01 16:56:21 +08:00
yanlong.wang	f3b82a4d50	fix: add jitter to user cache	2025-04-01 15:05:29 +08:00
yanlong.wang	988413dc6b	saas: async auth/rate-limit path for high-freq keys	2025-04-01 13:20:25 +08:00
Yanlong Wang	9fbd751b65	fix: expose status code/text from curl	2025-03-27 19:23:34 +08:00
yanlong.wang	3dc902e4a2	fix: puppeteer tricks	2025-03-26 19:40:32 +08:00
yanlong.wang	c758ec5b51	saas: minor adjustments for search	2025-03-26 15:21:26 +08:00
yanlong.wang	f472f02a50	fix: country code validation	2025-03-26 14:50:08 +08:00
yanlong.wang	2073c40a4f	fix: remove ip based query filling for serper	2025-03-26 14:37:18 +08:00
yanlong.wang	65ed1157c2	fix: charge amount	2025-03-26 14:15:55 +08:00
yanlong.wang	e953c01e58	fix: formatted usage entry	2025-03-26 14:07:36 +08:00
Yanlong Wang	e9d69e6201	searcher: image/news/bing search (#1176) ... * wip * refactor: more serper search features * fix: usage naming in crawl	2025-03-26 13:47:07 +08:00
yanlong.wang	51779dc2eb	saas: new rate limit policy	2025-03-26 11:52:37 +08:00
yanlong.wang	ab715271a2	deps: bump readability to latest	2025-03-24 17:29:09 +08:00
yanlong.wang	a30a865140	fix: curl failure should return 4xx if specified explicitly	2025-03-24 16:33:43 +08:00
yanlong.wang	bc8dea9139	fix: remove readerlm automatic retry	2025-03-24 16:21:10 +08:00
yanlong.wang	61a1252762	fix: removing core-js pollution	2025-03-24 16:20:51 +08:00
Aaron Ji	b304d5809d	feat: search return with date (#1173) ... * feat: search return with date * cleanup: keep changes to searcher --------- Co-authored-by: yanlong.wang <yanlong.wang@naiver.org>	2025-03-24 14:46:22 +08:00
yanlong.wang	f7f6a98839	fix: readerlm params	2025-03-24 13:57:51 +08:00
Yanlong Wang	f3654a786b	fix: beware readerlm capacity drain	2025-03-24 10:15:16 +08:00
Yanlong Wang	a471a6137c	fix: do img filtering in node instead of browser	2025-03-22 12:33:06 +08:00
Yanlong Wang	3b0e0207f7	cd: get font	2025-03-21 22:51:23 +08:00
Yanlong Wang	3bb731519b	fix: generated alt	2025-03-21 22:43:19 +08:00
Yanlong Wang	2a30fce1cc	fix: bring back content based respond timing	2025-03-19 19:03:51 +08:00
yanlong.wang	3a40db2590	fix: cache use and edge cases	2025-03-17 15:10:07 +08:00
Yanlong Wang	b4b99f0096	cd: eu	2025-03-14 16:26:22 +08:00
Aaron Ji	02a219a470	fix: restrict auto-fill to first page fetch	2025-03-14 13:07:15 +08:00
yanlong.wang	dd76cf3866	docs: mention about issues caused by upstream	2025-03-13 16:30:22 +08:00
yanlong.wang	b9ad4facf5	fix	2025-03-13 16:25:59 +08:00
yanlong.wang	ff7612414c	chore: use env variable to switch browser headless mode	2025-03-13 16:00:55 +08:00
yanlong.wang	adce02d708	cleanup	2025-03-13 15:57:36 +08:00
yanlong.wang	ff595c2b4c	improve: cache usage and detection of js-required pages	2025-03-13 15:55:12 +08:00
Aaron Ji	a5e3c2d281	fix: return 10 search results by default	2025-03-13 15:00:51 +08:00
Aaron Ji	f1aed656b7	fix: some response could miss meta field (#1165)	2025-03-13 11:51:24 +08:00
Yanlong Wang	9415c6a2be	fix: track if snapshot html modified by js	2025-03-12 22:26:10 +08:00
Yanlong Wang	6027963670	fix: tweak default timing to be more conservative	2025-03-12 18:32:08 +08:00
Yanlong Wang	8121d62324	fix: custom script may listen to mutationIdle	2025-03-12 18:15:06 +08:00
Yanlong Wang	f7dbadffb7	behavior change: ditch content based return timing, adopt mutationIdle as default timing	2025-03-12 18:04:22 +08:00
Yanlong Wang	5141814bc9	docs: mention about discarded	2025-03-12 10:47:01 +08:00
Yanlong Wang	cadb2fd94c	feat: provide option to discard links	2025-03-12 10:46:01 +08:00