256 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
yanlong.wang	918bbecd50	wip	2024-09-18 18:46:05 +08:00
yanlong.wang	438ae594c9	Merge branch 'main' into ast-to-ast	2024-09-18 12:18:59 +08:00
Yanlong Wang	39e49cac63	fix: 3xx not considered errors	2024-09-18 02:35:36 +08:00
Yanlong Wang	96ce7f5aac	fix: iframe should not actively report snapshot	2024-09-18 02:33:11 +08:00
Yanlong Wang	87a6578970	chore: deployment tweak	2024-09-18 00:12:29 +08:00
Yanlong Wang	c36aa730b4	fix: target selector	2024-09-17 17:47:01 +08:00
Yanlong Wang	88a9da33e4	wip	2024-09-17 17:21:34 +08:00
Zhaofeng Miao	e27bcaca77	feat: add adaptive crawler (#112)	2024-09-13 14:08:07 +08:00
Yanlong Wang	f8bc4877ef	fix	2024-09-12 19:50:46 +08:00
Yanlong Wang	1bd3ed7125	fix: description from jsdom	2024-09-12 19:09:19 +08:00
Yanlong Wang	6e05ea2243	feat: warn on non 200 response	2024-09-12 19:05:06 +08:00
yanlong.wang	b3647492a8	wip	2024-09-11 18:38:03 +08:00
Zhaofeng Miao	6147a28609	feat: return description	2024-09-11 15:21:39 +08:00
yanlong.wang	039b1363cd	Merge remote-tracking branch 'origin/main' into ast-to-ast	2024-09-11 11:14:41 +08:00
yanlong.wang	aad9096119	fix: bump deps	2024-09-10 17:13:13 +08:00
yanlong.wang	5b85fe450f	bump: deps	2024-09-10 16:41:54 +08:00
yanlong.wang	0da04847b5	bump: deps	2024-09-10 16:31:08 +08:00
yanlong.wang	145a9f8f88	bump: deps	2024-09-10 15:38:00 +08:00
yanlong.wang	9e4442fdf4	wip	2024-09-10 15:37:44 +08:00
yanlong.wang	e2aed6dd97	fix: attachment pdf	2024-09-10 12:02:22 +08:00
Yanlong Wang	c5abdf8570	tweak: try alleviate oom killed issue	2024-09-08 16:29:45 +08:00
Yanlong Wang	e324c4667f	feat: support explicit q passing in search	2024-09-08 10:38:35 +08:00
Yanlong Wang	607407f740	fix: pdf detection	2024-09-08 10:14:34 +08:00
Yanlong Wang	94170db060	fix: performance issue of jsdom	2024-09-08 00:50:15 +08:00
yanlong.wang	5171e5f94b	fix: req cap issues	2024-09-02 14:30:23 +08:00
yanlong.wang	405fe6372e	fix: bump deps	2024-09-02 13:59:40 +08:00
Yanlong Wang	42700d1a85	fix: cache with locale	2024-08-30 18:37:23 +08:00
Zhaofeng Miao	6900e0241c	feat: allow passing pdf without url param (#111)	2024-08-30 11:04:32 +08:00
Yanlong Wang	9a514cd473	fix: cap browser request freq to avoid block from google	2024-08-29 09:28:17 +08:00
Zhaofeng Miao	7e6c2fcf48	feat: add referer param	2024-08-22 16:48:47 +08:00
Zhaofeng Miao	080056e889	feat: allow passing base64 encoded pdf	2024-08-22 14:56:09 +08:00
Zhaofeng Miao	de50c93825	feat: expose X-Locale parameter	2024-08-20 16:14:48 +08:00
Yanlong Wang	fb5bd58ee4	feat: return usage tokens in json	2024-08-16 20:32:38 +08:00
Yanlong Wang	c7860e615c	fix: set-cookie	2024-08-16 19:37:13 +08:00
Yanlong Wang	df58fcb3fa	fix: alleviate search performance issue	2024-08-09 15:03:24 +08:00
Yanlong Wang	eb74e9c6f8	fix: remove select element from markdown to walk around turndown performance issue	2024-08-09 10:55:36 +08:00
Yanlong Wang	e4ef6cb0f9	chore: reduce fetch count in search	2024-08-09 10:29:50 +08:00
Yanlong Wang	e529369ba6	fix: search with failed pages	2024-08-08 15:49:23 +08:00
Yanlong Wang	0dd05b5dab	chore: tweak concurrency	2024-08-06 17:58:27 +08:00
Yanlong Wang	7af2bde01f	fix: html rebasing with <base> tag	2024-08-06 13:15:10 +08:00
Yanlong Wang	40e91853e2	fix	2024-08-02 20:10:17 +08:00
Yanlong Wang	cda0f371e1	feat: updated rate policy	2024-08-02 19:39:51 +08:00
Yanlong Wang	0a2c0932fd	fix	2024-08-02 17:13:50 +08:00
Yanlong Wang	ee632199df	fix	2024-08-02 17:12:10 +08:00
Yanlong Wang	0a33207f8f	fix: another approach to suspected DoS abuse	2024-08-02 17:04:13 +08:00
yanlong.wang	e658e8102c	fix	2024-08-01 20:07:39 +08:00
yanlong.wang	f4f189c8e6	fix	2024-08-01 19:51:53 +08:00
yanlong.wang	54fa5feb7f	fix	2024-08-01 19:49:40 +08:00
yanlong.wang	d0a922144d	fix	2024-08-01 19:48:00 +08:00
yanlong.wang	6fb5df97cc	fix: abuse of flooding elements	2024-08-01 19:34:39 +08:00