iqUdZddlZddlZddlZddlmZmZmZddlmZddl m Z m Z ddl m Z mZddlmZddlmZdd lmZdd lmZdd lmZeeZgd Ze eed <dZdddddddZe eefed<dZ!dZ"dZ#dZ$ej%dZ&Gdde Z'Gdde Z(GddZ)dS) u{Zhihu Hunter — vault scanning, question discovery, and answer drafting. Workflow: 1. scan_and_hunt() → scans recent vault files, extracts keywords, searches Tavily for matching Zhihu questions. 2. draft_answer() → uses existing RAG + ZhihuGenerator to produce a Zhihu-style answer draft ready for review. N)datetimetimezone timedelta)Path)ListOptional) BaseModelField) setup_logger) GeminiLLM)ZhihuGenerator)ObsidianIndexer)ChromaVectorStore)notesArticleauto_report/deep-dive CLAW/deepfact Clippings WATCH_DIRSrrrrr)rarticle clippingsclawdeepfactz deep-dive DIR_SHORTCUTS zzhihu\.com/question/\d+ceZdZUdZeed<eed<dZeeed<ee Z e eed<edZ eed <dZ eeed <dZeeed <dZeeed <dS) ZhihuQuestionz2A Zhihu question candidate surfaced by the hunter.titleurlNsnippetdefault_factorykeywordscdtjtjSNtzrnowrutc isoformat5/root/projects/butler/slack_bot/zhihu/zhihu_hunter.pyzZhihuQuestion.E!  = = = G G I Ir0found_at description source_fileoutline)__name__ __module__ __qualname____doc__str__annotations__r#rr listr&rr4r5r6r7r/r0r1r r =s<< JJJ HHH!GXc]!!!%555Hd3i555EIIHc"&K#%%%!%K#%%%!GXc]!!!!!r0r ceZdZUdZeed<eed<edZeed<ee Z e eed<dZ e eed <dS) AnswerDraftz/A generated answer draft awaiting human review.questioncontentcdtjtjSr(r+r/r0r1r2zAnswerDraft.Rr3r0r$ generated_at rag_sourcesN vault_file)r8r9r:r;r r=r<r rDr>rErrFrr/r0r1r@r@Ls99 LLLIIL##U4888Kc888 $J $$$$$r0r@c $eZdZdZdedededdfdZ d)d ed e e e d e de e fd Z de de e fdZde de e fdZde de e fdZde dee e e ffdZde de fdZd*de de defdZ d+d ed e e e d e de e fdZ d,ded e e e de efdZ d-de ed ed!ede fd"Z d*d#e d e de e fd$Zd%e e de e fd&Zd'e de efd(ZdS). ZhihuHunterzDiscovers Zhihu questions from vault content and drafts answers. Example: hunter = ZhihuHunter(vault_path, vector_store, indexer) questions = hunter.scan_and_hunt(since_days=3) draft = hunter.draft_answer(questions[0]) vault_path vector_storeindexerreturnNcV||_||_||_t|_dS)aInitialize the hunter. Args: vault_path: Root path of the Obsidian vault. vector_store: ChromaDB vector store for semantic retrieval. indexer: ObsidianIndexer for keyword-based search and writing samples. N)rIrJrKr llm)selfrIrJrKs r1__init__zZhihuHunter.__init__cs)%( ;;r0 since_daysdirs topic_hintcd|pd|rd|dndz}td|d||||}|std gStd |||}td t |d |S) aScan vault files and surface matching Zhihu questions. Args: since_days: Age threshold for non-Article directories. dirs: Restrict scan to these vault dirs (relative paths). None = all WATCH_DIRS. topic_hint: Optional focus hint injected into keyword extraction prompt. Returns: List of ZhihuQuestion candidates (up to _MAX_QUESTIONS_TOTAL). zdirs=allz , topic=''rRzZhihuHunter: scanning vault ())rTrUz&No keywords extracted from vault fileszExtracted keywords: Found z Zhihu questions)loggerinfo_scan_vault_for_keywords_find_questionslen)rOrSrTrU scan_descr& questionss r1 scan_and_huntzZhihuHunter.scan_and_huntws ,DME++J/^/H:/H/H/H/H\^_  @I@@@AAA00$S]0^^  KK@ A A AI 5855666((22  =S^^===>>>r0rel_pathcJ|j|z }|std||dd}t jdd|tj}t d|d t|d | |d d }|st d gSt d|| |}|D] }||_ |S)aExtract keywords from one vault file and find matching Zhihu questions. The returned questions carry ``source_file`` set to ``rel_path`` so that ``draft_answer()`` will ground the generated answer in that article. Args: rel_path: Path relative to vault_path (e.g. "Article/8-xxx-Revised.md"). Returns: List of ZhihuQuestion candidates with source_file populated. Raises: FileNotFoundError: If the file does not exist under the vault root. u文件不存在：utf-8ignoreencodingerrors^---\n.*?\n---\nrRflagszscan_single_file:  ( chars)Nz&No keywords extracted from single filezKeywords from file: )rIexistsFileNotFoundError read_textresubDOTALLstripr[r\r__extract_keywords_with_llmr^r6)rOrc file_pathtextr&raqs r1scan_single_filezZhihuHunter.scan_single_files.Oh. !! F#$D$D$DEE E""GH"EEv)2t29EEEKKMM GGGSYYGGGHHH224;??  KK@ A A AI 5855666((22  % %A$AMMr0patternc t|j|dd}|std|d|D}td|dt |d|g}t |d krd nd }|D]} |d d }tj dd|tj  }| d|j d|d|w#t$r*}td|d|Yd}~d}~wwxYwd|} || dd} | stdgStd| || } t'|d|j} | D] } | | _ | S)aXFind vault files matching a glob pattern and hunt Zhihu questions. All matched files are combined for keyword extraction, so the questions reflect the full set of content. The most-recently-modified matched file is stored as ``source_file`` to ground the answer draft. Args: pattern: Glob pattern relative to vault_path (e.g. "Article/*openclaw*-Revised.md"). Returns: List of ZhihuQuestion candidates with source_file set. Raises: FileNotFoundError: If no files match the pattern. c4|jSN)statst_mtime)ps r1r2z-ZhihuHunter.scan_glob_files..s!&&((+r0T)keyreverseu没有文件匹配：cg|] }|j Sr/)name).0rs r1 z/ZhihuHunter.scan_glob_files..s)))A)))r0zscan_glob_files: 'u' → z files: i rerfrgrjrRrk[] NCould not read :  --- @z-No keywords extracted from glob-matched fileszKeywords from glob: r)sortedrIglobrqr[r\r_rrrsrtrurvappendrOSErrordebugjoinrwr^r< relative_tor6)rOr|matchednamespartsper_file_budgetfpryecombinedr&raprimaryrzs r1scan_glob_fileszZhihuHunter.scan_glob_filessM" O  ) )++     G#$EG$E$EFF F))))) UUUG UUeUUVVV"%g,,!"3"3$$ : :B :||WX|FFv12t29MMMSSUU EEET2B?2B-CEEFFFF : : : 8r88Q8899999999 :!%%e,,228ETE?CC  KKG H H HI 5855666((22 gaj,,T_==>> $ $A#AMMsA3D E D==Er"c tjdd|dddddd}||\}}t d|d |d t|pdd t||| gS) akCreate a question card directly from a Zhihu question URL. Used when the user pastes a URL they were invited to answer. Fetches the page title and description so the card displays readable info. Args: url: A zhihu.com/question/ URL. Returns: Single-element list containing the ZhihuQuestion. [>\'")\].,;]+$rR<?r#/zDirect URL hunt: u → title=z, desc=z chars)r!r"r5) rsrtlstripsplitrstrip_fetch_question_infor[r\r_r )rOr" clean_urlr!r5s r1hunt_direct_urlzZhihuHunter.hunt_direct_urlsF,b#66==cBBHHMMaPVVWZ[[\]^eefijj !66yAA{ n nnnnPST_TecePfPfnnnoooEykRRRSSr0cNttjddz dz } ddlm}ddlm}ddl}|5}|j d }| d }| rV| |d } | dg} | r|| |} || | |dd| dd} | d} | r&| } | sC| pd} | dd} t5jdd| } d}| d}|rv|}|dd}t5jdd|}|sh| d}|rQ|dpd}t5jdd|}|rd|vrd}|r |dd}nd}|| r| dkr| |fcdddSdddn #1swxYwYn7#t>$r*}t@!d|d |Yd}~nd}~wwxYw|"d!d!d"}d#|dfS)$aFetch the question title and description via Playwright. Falls back to a URL-derived label if Playwright is unavailable or fails. Args: url: Cleaned question URL. Returns: Tuple of (title, description). Description may be None. datazzhihu_state.jsonr)sync_playwright)StealthNT)headlesszoMozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36) user_agentre)rhcookiesdomcontentloadedi:) wait_untiltimeoutirRzh1.QuestionHeader-titleu - 知乎u^\(\d+ 条消息\)\s*zdiv.QuestionRichTextu​u…?\s*显示全部\s*$zmeta[property="og:description"]rBu…$u知乎，中文互联网u知乎z%Playwright question fetch failed for rru知乎问题 #)#r__file__resolveparentsplaywright.sync_apirplaywright_stealthrjsonchromiumlaunch new_contextrploadsrrget add_cookiesnew_pageapply_stealth_syncgotowait_for_timeoutquery_selector inner_textrvr!rrsrtreplace get_attributeclose Exceptionr[rr)rOr" state_filerr_jsonrbrowsercontextrrpager!title_elr5desc_elog_elrqids r1rz ZhihuHunter._fetch_question_infosK(^^++--5a86ADVV ? M ; ; ; ; ; ; 2 2 2 2 2 2  ""8 .a*++T+::!--9.$$&&5 ;;z';';W';'M'MNND"hhy"55G5++G444'')) ,,T222 #*">K"-"5"5h"C"C"I"I"K"KK"$&)CR"U"U"["["]"]K"O //0QRREO',':':9'E'E'K&R&R&T&T &(fWb+&F&F&L&L&N&N '#=#L#L"&K'"-ete"4KK"&K .Uh.. +-q8 .8 .8 .8 .8 .8 .8 .8 .8 .8 .8 .8 .8 .8 .8 .8 .8 .8 .8 .8 .8 .8 .8 .r M M M LLKKKKK L L L L L L L L Mjjoo##C((,%%%t++sBL;KL/ L;# L;/L33L;6L37L;; M/ M**M/queryctjd}|sdS ddlm}||}||dddd g }|d g}|sdSd |D}d |S#t$r(}t d|Yd}~dSd}~wwxYw)aSearch the web for recent articles/news about the query topic. Results are injected into the answer prompt so it reads like it comes from someone who's actively following the latest developments. Args: query: Topic or question title to search. Returns: Formatted string of recent snippets, or empty string on failure. TAVILY_API_KEYrRr TavilyClientapi_keybasicrZ zhihu.com)r search_depth max_resultsdaysexclude_domainsresultsc g|]N}d|dd|dddd|ddd d OS) z- r!rmr"rz): rBrRN)rr)rrs r1rz9ZhihuHunter._fetch_recent_web_context..nst[QwZZZ1U8>>##6#6q#9ZZaeeIr>R>RSWTWSW>XZZr0 z!Recent web context fetch failed: N) osenvironrtavilyrsearchrrr[r) rOrrrclientritemsrrs r1_fetch_recent_web_contextz%ZhihuHunter._fetch_recent_web_contextRs*..!122 2  + + + + + +!\'222Fmm$!, $GKK 2..E rE99U## #    LL@Q@@ A A A22222 sAB , B B?B::B?rAfeedbackc td|j|rd|dddndzg}|j|jd}|D].}|t |jj/|j |jd }|D]Y}| d d  d  }|r||Zd} |j r|j|j z } | dd} t!jdd| t j } | dd} |d t |j jtd|j dt)| dn<#t*$r/} td|j d| Yd} ~ nd} ~ wwxYwt/|j |j} |j}|jr |d|jz }|jr |d|jz }|jr7|d|jz }td|jdd| r|d| z }||j}|r3|d|z }td t)|d|r|d!|z }| |g"\}}t;||t=t> |#S)$uGenerate a Zhihu-style answer draft for a question. Delegates to the existing ZhihuGenerator (reuses prompt + style guide). The RAG context is enriched by both semantic (ChromaDB) and keyword-based (ObsidianIndexer) retrieval. Args: question: ZhihuQuestion to answer. feedback: Optional revision instructions from the user (e.g. "更口语化一些"). Returns: AnswerDraft with generated content and source list. zDrafting answer for: z [feedback: N(]rRr)top_k)limitrrz# rerfrgrjrkizLoaded primary article: rmrnzCould not read source file ru 问题详细描述：u | 背景：u 用户提供的回答思路（必须严格按照这个逻辑框架展开，从知识库中找到对应的内容来支撑每个论点，不要自由发挥框架结构）： zInjected user outline: <ur 主要参考文章（请基于此文的观点和论据来回答，可直接引用其中的分析和结论）： u 近期相关资讯（最近90天内的动态，用来补充最新工具/发布/讨论，让回答更有时效感，自然地融入即可，不要刻意罗列）： zInjected recent web context (u 用户修改意见：)history)rArBrE)!r[r\r!rJsearch_knowledgerr source_pathrrKrrrvr6rIrrrsrtruinsertr_rwarningr r5r#r7rchatr@r>dictfromkeys)rOrArrEsemantic_resultsrkeyword_resultsnote first_lineprimary_articlerrawr generator prompt_inputrecent_contextrB_s r1 draft_answerzZhihuHunter.draft_answerws  >@@J /"":...   Z8#77B ZllGHlEEf0"cKKKQQSS"%ete*""1d8+?&@&@&EFFF lx7KllsSbOcOclllmmmm Z Z ZXX=QXXUVXXYYYYYYYY Z#4<1BCC ~   O N8LNN NL   > =8+;== =L   M Hu}vFHH L KKK(2B3B32GKK L L L   TBQTT L 77GG  V q`nqq L KKTN8K8KTTT U U U  C BBB BL^^L"^== T]];7788    s!B5G H!%H  Hc&tjtjt |z }|||}|sgS|pt }|tgkrdnd}|||}| ||S)amCollect vault files and extract topic keywords via LLM. Args: since_days: Age threshold for non-Article directories. dirs: Dirs to scan; None = all WATCH_DIRS. topic_hint: Optional focus instruction for keyword extraction. Returns: Deduplicated list of topic keywords (up to _MAX_KEYWORDS). r))r)rTr)max_chars_per_file)rU) rr,rr-r_collect_recent_filesr _ARTICLE_DIR_read_files_summaryrw) rOrSrTrUcutofffiles scan_dirsrrs r1r]z$ZhihuHunter._scan_vault_for_keywordss ... 1K1K1KK**6*== I&J !*|n!.sAaD/?qqa!A$.Gr0)rc|d S)Nrr/rs r1r2z3ZhihuHunter._collect_recent_files..s qter0c3"K|] \}}}|V dSrr/)rr rs r1 z4ZhihuHunter._collect_recent_files..!s(EE71aQEEEEEEr0z Collected z files from vault (dirs=rY)rrIis_dirr[rrglobrrrr fromtimestamprr-rlowerendswithrrsort_MAX_FILES_ARTICLE_MAX_FILES_PER_DIRextendr_) rOrrTr collectedrel_dir watch_dir candidatesmd_filemtimefile_dt is_revised max_filess r1rz!ZhihuHunter._collect_recent_filess.!,DD* "   F FG'1I##%%  JyJJKKK:!>}!M!MJ%%uj'&BCCCCH,&&$G$GHHH. OO444.   EEj).DEEE E E E E V#i..VV)VVVWWWs)A C94AC99 DDrrr max_charsrcg}|}|D]}|dkrn |dd}tjdd|tj}|dt ||}|d |jd ||t|z}#t$r*} t d |d | Yd} ~ d} ~ wwxYwd |S)aRead and concatenate file contents up to a character budget. Each file is capped at max_chars_per_file so that no single large file exhausts the total budget and crowds out other directories. Args: files: List of markdown file paths. max_chars: Total character budget across all files. max_chars_per_file: Per-file cap to ensure directory diversity. Returns: Concatenated content string with file-name headers. rrerfrgrjrRrkNrrrrr) rrrsrtrurvminrrr_rr[rr) rOrr0rr remainingfrychunkrs r1rzZhihuHunter._read_files_summary&s%&  9 9AA~~ 9{{GH{EEv12t29MMMSSUU@c"4i@@@A 333E33444SZZ'  9 9 9 7q77A7788888888 9!!%(((sBB(( C2 CC combined_textcp|rd|dnd}dtd|d|ddd } |j|g\}}d |D}|dtS#t $r)}t d |gcYd}~Sd}~wwxYw) u,Use LLM to distill topic keywords from aggregated vault content. Args: combined_text: Concatenated recent vault content. topic_hint: Optional focus instruction (e.g. "agent开发"). Returns: List of keyword strings (up to _MAX_KEYWORDS). u - 重点关注与「u9」相关的话题，优先提取这个方向的关键词rRu你是一位知乎内容运营专家。请从以下笔记内容中提炼出适合在知乎搜索问题的关键词。 要求： - 输出 u@ 个以内的关键词，每行一个，无序号无符号 - 关键词要在知乎上有真实讨论热度，即普通中国用户会关心并提问的话题领域 - 避免过于偏门的专有名词（如具体项目代号、小众产品名），要用它所属的上一级概念 例：「OpenClaw」→「AI智能体安全」；「BadJs」→「微信监控机制」；「Moon Monitor」→「健康监测设备」 - 不要太泛（如"AI""技术""健康"），也不要太窄（如只有一个工具代码名称） - 优先选择最近讨论最多、最有深度的话题uA - 只输出关键词列表，不要解释 === 笔记内容 === Nrorcg|]Q}||d=|RS)r)rv startswith)rlines r1rz:ZhihuHunter._extract_keywords_with_llm..ms^::<<)- (?(?(D(D r0zKeyword extraction failed: ) _MAX_KEYWORDSrNgenerate_responserv splitlinesrr[error) rOr6rUtopic_instructionpromptresponser r&rs r1rwz&ZhihuHunter._extract_keywords_with_llmKs  " jZ j j j j!     :K  uu    (44VR@@KHa$NN,,7799H N]N+ +    LL:q:: ; ; ;IIIIII sAB B5 B0*B50B5r&c dddlm}tjd}|st d||}t }g}|D]} |||}|D]C} | dd} t | s4tj d| } | sLtj d d| d d d d dd } tj d | r| |vr|| |t#| d|| | ddddpd|gEc#t$$r+} t&d|d| Yd} ~ d} ~ wwxYwt&dt-|dt.d|dt.S)aSearch Tavily for Zhihu questions matching each keyword. Collects candidates from ALL keywords before truncating so that later keywords (which can cover different facets) are not silently dropped. Args: keywords: List of topic keywords to search. Returns: Deduplicated ZhihuQuestion list (up to _MAX_QUESTIONS_TOTAL). rrrzTAVILY_API_KEY not setrr"rRz https?://\S+rrrrz/answer/|/comment/r!rBN)r!r"r#r&z"Tavily search failed for keyword 'z': rZz Zhihu questions (capped at rY)rrrrr RuntimeErrorset_tavily_search_MIN_QUESTION_URL_RErrsrtgrouprraddrr rr[rr\r__MAX_QUESTIONS_TOTAL)rOr&rrr seen_urlsrakwritemr"_mrrs r1r^zZhihuHunter._find_questionsysw ('''''*..!122 9788 8g...!ee )+  P PB P--fb99#D((5"--C/66s;;! ?C88B! "'8"bhhqkk J J P PQT U UVW X ^ ^_b c cde f m mnq r rIy!6 BB!  I-- MM),,,$$%"&((7B"7"7 )$(HHY$;$;DSD$A$IT&(T . P P PNBNN1NNOOOOOOOO P  `S^^``I]```aaa...//sEF88 G- G((G-keywordcd|}td|||dddgd}|dgS) a Run a single Tavily search scoped to zhihu.com/question. Args: client: Authenticated TavilyClient instance. keyword: Search keyword. Returns: List of result dicts from Tavily (url, title, content, score). zsite:zhihu.com/question zTavily search: rrrim)rrrinclude_domainsrr)r[rrr)rOrrOrrAs r1rFzZhihuHunter._tavily_searchsk5744 .u..///== (M !  ||Ir***r0)rQNrR)rR)NrRr)rr)r8r9r:r;rrrrPintrrr<r rbr{rrtuplerrr@r r]rrrrwr^rrFr/r0r1rHrHZs@(!   ,$( tCy!  m  :""m1D""""H6s6tM/B6666pT3T4 +>TTTT"O,O,c8C=6H0IO,O,O,O,b#s#s####JT T ]T cT ;T T T T v%) PPPtCy!P P c PPPP>%)999tCy!9 d 9999|"% #)#)Dz#)#) #) #)#)#)#)P**** c ****\40S 40d=6I40404040l+c+d4j++++++r0rH)*r;rrsrequestsrrrpathlibrtypingrrpydanticr r health.utils.logging_configr slack_bot.llm.geminir slack_bot.obsidian.generatorsr slack_bot.obsidian.indexerrslack_bot.obsidian.vector_storerr8r[rr<r=rrrr%r$r;rJcompilerGr r@rHr/r0r1r^s 2222222222!!!!!!!!%%%%%%%%444444******888888666666====== h    DI !!) !! tCH~ !rz"<== " " " " "I " " " % % % % %) % % %h +h +h +h +h +h +h +h +h +h +r0