fix(weibo): detail 模式支持传入微博帖子 URL - #978
Open
ybwbqg9379 wants to merge 1 commit into
Open
Conversation
--specified_id 的帮助文案声明支持完整 URL 或 ID，但微博分支直接把原始输入透传给 config.WEIBO_SPECIFIED_ID_LIST，导致传 URL 时详情页请求和评论接口参数都变成畸形地址。 Web UI 前端虽能识别 ID，提交给后端的仍是原始 URL，所以走的是同一条失败路径。 参照贴吧已有的 _normalize_tieba_note_id，新增 _normalize_weibo_note_id 从 weibo.com/<uid>/<bid>、m.weibo.cn/detail/<mid>、m.weibo.cn/status/<bid> 三种常见 形态中提取帖子 ID，纯 ID 保持原样。补充离线单测覆盖归一化函数与完整 CLI 解析。 Fixes NanmiCoder#914
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
问题
#914 报告微博 detail 模式传入帖子 URL 时无法爬取。根因在
cmd_arg/arg.py：--specified_id帮助文案声明支持 URL 或 ID，但微博分支把原始输入直接赋给config.WEIBO_SPECIFIED_ID_LIST，于是get_note_info_by_id拼出的详情页地址和评论接口的id/mid参数都是整段 URL。Web UI 的表单提交给后端的也是原始 URL，最终经crawler_manager.py走同一条 CLI，所以两端表现一致。修复
参照贴吧已有的
_normalize_tieba_note_id，新增_normalize_weibo_note_id，覆盖三种常见帖子 URL：https://weibo.com/<uid>/<bid>?refer_flag=...https://m.weibo.cn/detail/<mid>https://m.weibo.cn/status/<bid>纯 ID 输入保持原样，其他平台分支不受影响。
测试
新增
tests/test_cmd_arg_weibo.py：6 个参数化用例覆盖归一化函数，1 个用例走完整parse_cmd验证config.WEIBO_SPECIFIED_ID_LIST结果，与现有test_cmd_arg_tieba.py结构一致，全部离线。uv run pytest tests：103 passed。pre-commit 全部通过。Fixes #914