"""Phase 5 自檢:BM25 檢索排序、MCP search_wiki / read_page、 路徑跳脫防護(信任邊界)、token 上限截斷。 執行:.venv/Scripts/python tools/selfcheck_search.py """ import os import pathlib import shutil import sys import tempfile sys.path.insert(0, str(pathlib.Path(__file__).parent)) import kb import search CFG = """ollama: {base_url: "http://localhost:11434", timeout_seconds: 5} tasks: ingest: {model: "fake:test", temperature: 0.2, max_retries: 2} lint: {model: "fake:test", temperature: 0.1, max_retries: 2} fallback: {model: "fake:fb"} limits: {mcp_response_max_tokens: 100, ingest_chunk_max_chars: 12000} """ def page(root, rel, title, desc, tags, body): meta = {"type": {"summaries": "summary", "entities": "entity", "concepts": "concept"}[rel.split("/")[1]], "title": title, "description": desc, "tags": tags, "timestamp": "2026-07-14", "sources": ["raw/converted/x.md#abcd1234"], "status": "draft"} (root / rel).write_text(kb.dump_page(meta, body), encoding="utf-8") def main(): tmp = pathlib.Path(tempfile.mkdtemp(prefix="ppqa-search-")) try: root = tmp / "repo" for sub in ("config", "wiki/summaries", "wiki/entities", "wiki/concepts"): (root / sub).mkdir(parents=True) (root / "config/models.yaml").write_text(CFG, encoding="utf-8") page(root, "wiki/entities/payment-gateway.md", "支付閘道", "支付閘道模組", ["gateway"], "壓力測試時出現逾時缺陷,重試機制未生效。") page(root, "wiki/concepts/aml-testing.md", "AML 測試準則", "可疑交易監控測試綜合", ["aml", "compliance"], "大額交易與分散交易樣態的監控測試要求。") page(root, "wiki/summaries/long-doc.md", "長文件摘要", "截斷測試用", ["test"], "逾時。" * 500) # 與 payment-gateway 共用核心詞「支付閘道」→ df=2, N=4,正是 BM25Okapi # 原式 idf 歸零的配置;沒有這頁就測不出核心詞落空的 bug page(root, "wiki/summaries/gateway-report.md", "支付閘道版本沿革", "支付閘道改版紀錄", ["gateway"], "支付閘道於 2026 年改版,新增分期付款。") hits = search.search(root, "支付閘道 逾時") assert hits and hits[0]["path"] == "wiki/entities/payment-gateway.md", hits assert all(h["score"] > 0 for h in hits) hits2 = search.search(root, "可疑交易監控") assert hits2[0]["path"] == "wiki/concepts/aml-testing.md", hits2 assert search.search(root, "zzz-nonexistent-term") == [] print("PASS: BM25 檢索排序(繁中 bigram)") # 核心詞(出現在多頁)不得因 idf <= 0 被整批濾掉 core = search.search(root, "支付閘道") paths = {h["path"] for h in core} assert {"wiki/entities/payment-gateway.md", "wiki/summaries/gateway-report.md"} <= paths, core assert all(h["score"] > 0 for h in core), core print("PASS: 核心詞跨頁共用仍可檢索(idf 恆正)") os.environ["PP_QA_ROOT"] = str(root) import importlib sys.path.insert(0, str(pathlib.Path(__file__).parents[1] / "mcp")) server = importlib.import_module("server") res = server.search_wiki("支付閘道 逾時", top_k=99) # top_k 超限 → 收斂為 10 assert res[0]["path"] == "wiki/entities/payment-gateway.md" pg = server.read_page("wiki/entities/payment-gateway.md") assert pg["frontmatter"]["title"] == "支付閘道" and not pg["truncated"] assert pg["source_refs"] == ["raw/converted/x.md#abcd1234"] long = server.read_page("wiki/summaries/long-doc.md") assert long["truncated"] and len(long["body"]) <= 150 and long["source_refs"] print("PASS: MCP search_wiki + read_page(含 token 截斷、source_refs 保留)") for bad in ("../AGENTS.md", "wiki/../config/models.yaml", "wiki/entities/x.txt"): try: server.read_page(bad) raise AssertionError(f"應拒絕:{bad}") except (ValueError, FileNotFoundError) as e: assert isinstance(e, ValueError), f"{bad} 應為 ValueError" try: server.read_page("wiki/entities/no-such.md") raise AssertionError("應拋 FileNotFoundError") except FileNotFoundError: pass print("PASS: 路徑跳脫防護(信任邊界)") print("ALL PASS") finally: os.environ.pop("PP_QA_ROOT", None) shutil.rmtree(tmp, ignore_errors=True) if __name__ == "__main__": main()