diff --git a/skills/deeppapernote/SKILL.md b/skills/deeppapernote/SKILL.md index 3e1bb2a..1dd56e9 100644 --- a/skills/deeppapernote/SKILL.md +++ b/skills/deeppapernote/SKILL.md @@ -1,6 +1,6 @@ --- name: deeppapernote -description: Generate a high-quality deep-reading note for a single paper and write it into an Obsidian-style vault. Use when the user gives a paper title, DOI, URL, arXiv ID, Zotero item, or local PDF and wants a polished Markdown note with strong structure, evidence-based analysis, and figure placeholders. +description: Generate a high-quality deep-reading note for a single paper and write it into an Obsidian-style vault. Use when the user gives a paper title, DOI, URL, arXiv ID, Zotero item, or local PDF and wants a polished Markdown note with strong structure, evidence-based analysis, and figure placeholders. Output language follows configuration (zh-CN, en, or ja). Japanese trigger phrases 「この論文の精読ノートを作って」「この論文をObsidianノートにまとめて」「論文を読んで日本語のMarkdownノートにして」. --- # DeepPaperNote @@ -20,6 +20,11 @@ English trigger examples: - `Generate a deep-reading note for this paper` - `Turn this paper into an Obsidian research note` +Japanese trigger examples: +- `この論文の精読ノートを作って` +- `この論文をObsidianノートにまとめて` +- `この論文を読んで日本語のMarkdownノートにして` + ## User Configuration Before a normal paper run, read `references/user-configuration.md` for configuration admission, migration, repair, Run Overrides, and Preference Changes. @@ -28,7 +33,7 @@ Resolve Run Overrides from the explicit request, CLI, and current process enviro ## Language Integrity Contract -After Configuration Readiness, resolve one `output_language` (`zh-CN` or `en`) for the run. `source_manifest.language_hint` describes source text only and never selects the note profile. +After Configuration Readiness, resolve one `output_language` (`zh-CN`, `en`, or `ja`) for the run. `source_manifest.language_hint` describes source text only and never selects the note profile. Bind that exact value through Save Target Admission → Figure Plan → Figure/Table Decisions → Synthesis Bundle → Note Plan → Grounding Lint → Final Note Lint → Final Quality Review → Final Readability Review → Formal Save: @@ -136,7 +141,7 @@ Non-negotiable rules: - fail-closed: if a usable PDF or sufficient evidence cannot be obtained after supported acquisition paths, stop and ask for better source material rather than producing a finished degraded note - model-first: scripts structure evidence, but the model must decide emphasis, contribution, mechanism, limitations, and final prose in the configured language - required structure: include the localized canonical sections in the order declared by `writing_contract.must_include_sections` -- abstract fidelity: preserve the original abstract's meaning without adding later evidence or model judgments; translate it in `zh-CN` mode and render it faithfully in English in `en` mode +- abstract fidelity: preserve the original abstract's meaning without adding later evidence or model judgments; translate it in `zh-CN` and `ja` modes and render it faithfully in English in `en` mode - mechanism depth: method, framework, and system papers should include the localized mechanism-flow subsection under the localized method section, normally as a 3 to 4 step numbered flow with input, operation, and output destination - placeholder-first figures: plan major figure/table placeholders first; replace one only when identity match and visual usability are both strong; otherwise keep the placeholder @@ -193,7 +198,7 @@ Formal Save states: - After the synthesis bundle is built, complete the model-led Visual Review Gate and Figure/Table Decision Freeze before creating `note_plan`; no `review_pending` item may cross that boundary. - Pass the grounding and final-note figure gates before advancing; revise any failed decision coverage, insertion, structure, or status check. - An `insert` decision is complete only after Formal Save materializes the selected image into the paper-local `images/` directory and the write succeeds. -- The note must pass the style gate for its configured language: `zh-CN` rejects mixed Chinese-English prose artifacts, while `en` rejects Chinese prose outside citation metadata. +- The note must pass the style gate for its configured language: `zh-CN` rejects mixed Chinese-English prose artifacts, `en` rejects Chinese prose outside citation metadata, and `ja` rejects simplified-Chinese leftovers and mixed Japanese-English prose outside citation metadata. - The style gate also rejects mechanical term-replacement artifacts such as `KV缓存 of`, `批量ing`, `In相关 Researcher`, or `Single 序列 generation`; rewrite the sentence naturally instead of preserving a partially translated phrase. - Style gate enforcement: when `lint_note.py` output contains `passes_style_gate: false`, fix the reported issues and re-run lint. Keep fixing and re-running until lint passes — multiple rounds are normal and expected. Do not decide that any failure is an acceptable exception — proper nouns, math formulas, and citation metadata are not automatic exemptions. Only escalate to the user if the same failures appear unchanged across multiple rounds with no reduction, indicating the model is unable to make further progress independently. - If PDF or evidence quality is insufficient for a real deep note, fail closed: stop, report the blocked stage, and ask for the better PDF, OCR/source material, or other input needed to continue. diff --git a/skills/deeppapernote/references/domain_rules.ja.yaml b/skills/deeppapernote/references/domain_rules.ja.yaml new file mode 100644 index 0000000..86d93ab --- /dev/null +++ b/skills/deeppapernote/references/domain_rules.ja.yaml @@ -0,0 +1,209 @@ +# Japanese domain routing taxonomy, selected automatically when output_language is ja. +# Same aliases/keywords as domain_rules.yaml; only folder labels are translated. +# Copy to ~/.deeppapernote/domain_rules.yaml (or point DEEPPAPERNOTE_DOMAIN_RULES at a copy) to customize. +# Application domains are tried before fallback method domains. +domains: + - label: 医療・健康 + aliases: + - healthcare + - medical + - clinical medicine + specialized_folders: + - メンタルヘルス + keywords: + - clinical + - patient + - patients + - depression + - anxiety + - mental health + - psychiatric + - psychology + - therapy + - counseling + - symptom + - diagnosis + - screening + - hospital + - healthcare + - medical + methods: [] + + - label: 法律 + aliases: + - legal + - law + keywords: + - legal + - law + - court + - judge + - contract + - statute + - regulation + - litigation + - case law + methods: [] + + - label: 教育 + aliases: + - education + - educational + keywords: + - education + - student + - teacher + - classroom + - curriculum + - tutoring + - learning analytics + - pedagogy + methods: [] + + - label: 金融 + aliases: + - finance + - financial + keywords: + - finance + - financial + - stock + - market + - trading + - portfolio + - risk + - credit + - banking + - investment + methods: [] + + - label: ロボティクス + aliases: + - robotics + - robotic + keywords: + - robot + - robotics + - robotic + - manipulation + - navigation + - control policy + - locomotion + - autonomous driving + - embodied + methods: + - diffusion policy + + - label: ソフトウェア工学 + aliases: + - software engineering + keywords: + - software engineering + - code generation + - program repair + - bug + - repository + - developer + - code review + - test generation + - compiler + methods: [] + + - label: 生物医学 + aliases: + - biomedical + - bioinformatics + keywords: + - biomedical + - genomics + - protein + - drug discovery + - molecular + - cell + - gene + - bioinformatics + methods: [] + + - label: メンタルヘルス + route_to: 医療・健康 + aliases: + - mental health + - psychology + - psychiatry + keywords: + - depression + - anxiety + - mental health + - psychiatric + - psychology + - therapy + - counseling + - symptom + methods: [] + + - label: 推薦システム + aliases: + - recommender systems + - recommendation + keywords: + - recommendation + - recommender + - ctr prediction + - ranking system + - personalization + methods: [] + +fallback_domains: + - label: 大規模言語モデル + aliases: + - llm + - large language model + - language model + - foundation model + keywords: + - large language model + - llm + - foundation model + - gpt + - transformer + - instruction tuning + - pretrain + - pre-training + - language model + - agent + - multi-agent + - multi agent + - reasoning + - multimodal + - retrieval-augmented generation + - rag + - in-context learning + - long-context + - long context + - mixture-of-experts + - mixture of experts + - moe + - alignment + - rlhf + methods: [] + + - label: 機械学習 + aliases: + - machine learning + - ml + keywords: + - machine learning + - deep learning + - neural network + - representation learning + - reinforcement learning + - computer vision + - graph neural network + - speech recognition + methods: [] + + - label: 未分類 + aliases: + - unclassified + keywords: [] + methods: [] diff --git a/skills/deeppapernote/references/output-language.md b/skills/deeppapernote/references/output-language.md index 597b025..ddb2655 100644 --- a/skills/deeppapernote/references/output-language.md +++ b/skills/deeppapernote/references/output-language.md @@ -1,13 +1,14 @@ # Output Language -DeepPaperNote supports two output schemas: +DeepPaperNote supports three output schemas: | Setting | Language | |---|---| | `zh-CN` | Simplified Chinese | | `en` | English | +| `ja` | Japanese | -Resolve the profile through `user-configuration.md`. For a single command, use `--language en` with `run_pipeline.py`, `build_synthesis_bundle.py`, `lint_note.py`, or `write_obsidian_note.py` where applicable. A Run Override never changes User Configuration. +Resolve the profile through `user-configuration.md`. For a single command, use `--language en` (or `--language ja`) with `run_pipeline.py`, `build_synthesis_bundle.py`, `lint_note.py`, or `write_obsidian_note.py` where applicable. A Run Override never changes User Configuration. ## Simplified Chinese note schema @@ -66,6 +67,46 @@ For a materialized image, use the normal image embed followed immediately by one The English style gate checks headings, Core Information labels, figure callouts, inserted-image captions, and prose. Mark original non-English metadata with inline code inside `Core Information`, or with inline code or a Markdown link inside `References`; closed fenced code blocks and URLs keep their normal source text. In prose, a CJK identifier or stable proper noun must be an HTTP Markdown link or Obsidian wikilink. A source formula may retain only these CJK `\operatorname{...}` labels: `输入`, `输出`, `损失`, `状态`, `动作`, `奖励`, `标签`, `样本`, `预测`, and `目标`. The surrounding text remains subject to the English gate; inline code and free-form math text do not exempt Chinese prose. +## Japanese note schema + +Use these top-level sections in this order: + +1. `基本情報` +2. `要旨の翻訳` +3. `新規性` +4. `一言まとめ` +5. `研究課題` +6. `データとタスク定義` +7. `手法の骨子` +8. `主要な結果` +9. `深掘り分析` +10. `限界` +11. `私のメモ` +12. `参考文献` + +The allowed 基本情報 fields, in order, are: + +`タイトル`, `タイトル訳`, `著者`, `所属`, `発表時期`, `発表媒体`, `DOI`, `arXiv`, `論文リンク`, `コード / プロジェクト`, `データ / リソース`, `論文タイプ`. + +Use `### 機構フロー` for the mechanism-flow subsection. Each figure placeholder uses: + +```md +> [!figure] Figure 2 図の内容を表す短い見出し +> 推奨位置：手法の骨子 +> 配置理由：この図は実行経路を明確にする。 +> 現在の状態：プレースホルダを保持。切り出した画像は不完全。 +``` + +For a materialized image, use the normal image embed followed immediately by one italic caption beginning with `論文原図番号：`. + +`要旨の翻訳` is a faithful Japanese translation of the source abstract. Preserve its meaning and scope; contribution claims, result interpretation, and hindsight judgment belong later unless the source abstract itself contains them. + +Write natural Japanese prose (常体・敬体 consistently within a note). Keep stable proper nouns, model names, dataset names, and metric names in their original form; do not force-translate them. Mechanism-flow steps should name 入力 / 操作 / 出力 explicitly. + +The Japanese style gate checks headings, 基本情報 labels, figure callouts, inserted-image captions, and prose. Because kanji are shared with Chinese, the gate looks for simplified-Chinese-only characters (机, 图, 论, 议, 态, ...) that never appear in Japanese; any such character in a heading, label, or prose line is a leftover from the Chinese template and fails `passes_style_gate`. Original Chinese metadata is allowed inside `基本情報` and `参考文献` when marked with inline code or a Markdown link, and URLs and closed fenced code blocks keep their source text. Lines that mix Japanese with four or more English words including English function words are also rejected as mixed-language prose. + +Domain folders: the skill ships `references/domain_rules.yaml` with Chinese folder labels and `references/domain_rules.ja.yaml` with Japanese ones (医療・健康, 法律, 教育, 金融, ロボティクス, ソフトウェア工学, 生物医学, メンタルヘルス, 推薦システム, 大規模言語モデル, 機械学習, 未分類). When the resolved `output_language` is `ja`, `resolve_domain_subdir` selects the Japanese file automatically, so a Japanese Vault gets Japanese folder names without extra setup. To customize labels, keep a copy outside the skill, either at `~/.deeppapernote/domain_rules.yaml` (next to `config.json`) or at the path named by `DEEPPAPERNOTE_DOMAIN_RULES`; both take precedence over any shipped file, and fallback folder labels are resolved through the `healthcare`, `machine learning`, and `unclassified` aliases of that taxonomy. Existing domain folders already present under the papers root are still reused regardless of language. + ## Contract ownership `SKILL.md` owns the cross-stage Language Integrity Contract. This reference owns only the profile-specific schema and labels above; apply them under the resolved language carried by that contract. diff --git a/skills/deeppapernote/references/user-configuration.md b/skills/deeppapernote/references/user-configuration.md index 77541c8..b372014 100644 --- a/skills/deeppapernote/references/user-configuration.md +++ b/skills/deeppapernote/references/user-configuration.md @@ -4,7 +4,7 @@ DeepPaperNote can resolve a run entirely from Run Overrides. It also supports on For isolated validation only, `DEEPPAPERNOTE_CONFIG_PATH` may relocate this one file for the current process. It is not a preference, does not create a second configuration layer, and must not be persisted into the configuration itself. -- `output_language`: `zh-CN` or `en` +- `output_language`: `zh-CN`, `en`, or `ja` - `save_mode`: `workspace` or `obsidian` - `obsidian_vault`: existing absolute directory, required only in Obsidian mode - `papers_dir`: safe relative path inside the Vault, required only in Obsidian mode diff --git a/skills/deeppapernote/scripts/build_synthesis_bundle.py b/skills/deeppapernote/scripts/build_synthesis_bundle.py index e1af552..fe9fdfb 100644 --- a/skills/deeppapernote/scripts/build_synthesis_bundle.py +++ b/skills/deeppapernote/scripts/build_synthesis_bundle.py @@ -35,7 +35,7 @@ def parser() -> argparse.ArgumentParser: required=True, help="Figure/table decision JSON path or string.", ) - p.add_argument("--language", default="", help="Run Override for output language: en or zh-CN.") + p.add_argument("--language", default="", help="Run Override for output language: en, zh-CN, or ja.") p.add_argument("--output", default="", help="Output JSON path.") return p diff --git a/skills/deeppapernote/scripts/common.py b/skills/deeppapernote/scripts/common.py index 1cb6865..a485141 100644 --- a/skills/deeppapernote/scripts/common.py +++ b/skills/deeppapernote/scripts/common.py @@ -18,9 +18,11 @@ from typing import Any from user_configuration import ( + OUTPUT_LANGUAGES, inspect_configuration, resolve_preferences, resolve_run_overrides, + user_config_path, ) ARXIV_NS = { @@ -2569,6 +2571,21 @@ def resolve_note_asset_dir(note_path: str | Path, asset_subdir: str) -> Path: DOMAIN_RULES_PATH = Path(__file__).resolve().parents[1] / "references" / "domain_rules.yaml" +# Translations of the hardcoded Chinese fallback labels, applied only when no taxonomy +# file could be loaded (e.g. the shipped YAML is unreadable). Labels absent here are +# identical in both languages (法律, 教育, 金融, 生物医学). +DOMAIN_LABEL_TRANSLATIONS: dict[str, dict[str, str]] = { + "ja": { + "医疗健康": "医療・健康", + "机器人": "ロボティクス", + "软件工程": "ソフトウェア工学", + "心理健康": "メンタルヘルス", + "推荐系统": "推薦システム", + "大模型": "大規模言語モデル", + "机器学习": "機械学習", + "未分类": "未分類", + }, +} DOMAIN_LIST_KEYS = ( "aliases", "keywords", @@ -2780,12 +2797,29 @@ def resolve_note_asset_dir(note_path: str | Path, asset_subdir: str) -> Path: } -def _copy_default_domain_rules() -> dict[str, list[dict[str, Any]]]: +def _translate_domain_label(label: str, output_language: str | None) -> str: + return DOMAIN_LABEL_TRANSLATIONS.get(output_language or "", {}).get(label, label) + + +def _copy_default_domain_rules( + output_language: str | None = None, +) -> dict[str, list[dict[str, Any]]]: + def translate(value: Any) -> str: + return _translate_domain_label(str(value), output_language) + + def copy_rule(rule: dict[str, Any]) -> dict[str, Any]: + copied: dict[str, Any] = {} + for key, value in rule.items(): + if key in ("label", "route_to"): + copied[key] = translate(value) + elif key == "specialized_folders": + copied[key] = [translate(item) for item in value] + else: + copied[key] = list(value) if isinstance(value, list) else value + return copied + return { - section: [ - {key: list(value) if isinstance(value, list) else value for key, value in rule.items()} - for rule in DEFAULT_DOMAIN_RULES[section] - ] + section: [copy_rule(rule) for rule in DEFAULT_DOMAIN_RULES[section]] for section in DOMAIN_SECTIONS } @@ -2955,17 +2989,45 @@ def _normalize_domain_rules( return normalized -def load_domain_rules() -> dict[str, list[dict[str, Any]]]: +def shipped_domain_rules_path(output_language: str | None = None) -> Path: + """Return the shipped taxonomy: `domain_rules..yaml` when present, else the default.""" + language = (output_language or "").strip() + if language: + localized = DOMAIN_RULES_PATH.with_name(f"domain_rules.{language}.yaml") + if localized.is_file(): + return localized + return DOMAIN_RULES_PATH + + +def resolve_domain_rules_path(output_language: str | None = None) -> Path: + """Pick the domain taxonomy file: env override, then the user config directory, then the + skill default. + + The default file ships Chinese folder labels and `domain_rules.ja.yaml` ships Japanese + ones, chosen by `output_language`; a user who wants other labels keeps a copy outside + the skill so upstream updates never overwrite it. + """ + override = os.environ.get("DEEPPAPERNOTE_DOMAIN_RULES", "").strip() + if override: + return Path(override).expanduser() + user_rules = user_config_path().parent / "domain_rules.yaml" + if user_rules.is_file(): + return user_rules + return shipped_domain_rules_path(output_language) + + +def load_domain_rules(output_language: str | None = None) -> dict[str, list[dict[str, Any]]]: try: - if not DOMAIN_RULES_PATH.exists(): - return _copy_default_domain_rules() - parsed = _parse_domain_rules_yaml(DOMAIN_RULES_PATH.read_text(encoding="utf-8-sig")) + rules_path = resolve_domain_rules_path(output_language) + if not rules_path.exists(): + return _copy_default_domain_rules(output_language) + parsed = _parse_domain_rules_yaml(rules_path.read_text(encoding="utf-8-sig")) normalized = _normalize_domain_rules(parsed) if normalized is None: - return _copy_default_domain_rules() + return _copy_default_domain_rules(output_language) return normalized except Exception: - return _copy_default_domain_rules() + return _copy_default_domain_rules(output_language) def _normalized_domain_label(value: str) -> str: @@ -3033,9 +3095,11 @@ def _score_domain_for_inference(rule: dict[str, Any], text: str, *, fallback: bo ) -def infer_domain_label(title: str, abstract: str = "") -> str: +def infer_domain_label( + title: str, abstract: str = "", *, output_language: str | None = None +) -> str: lower = normalize_whitespace(f"{title} {abstract}").lower() - rules = load_domain_rules() + rules = load_domain_rules(output_language) scored: list[tuple[int, str]] = [] for rule in rules["domains"]: score = _score_domain_for_inference(rule, lower, fallback=False) @@ -3055,10 +3119,28 @@ def infer_domain_label(title: str, abstract: str = "") -> str: paper_type, _ = infer_paper_type(title, abstract) if paper_type == "clinical_or_psychology_empirical": - return "医疗健康" + return _domain_label_by_alias(rules, "healthcare", "医疗健康", output_language) if paper_type == "AI_method": - return "机器学习" - return "未分类" + return _domain_label_by_alias(rules, "machine learning", "机器学习", output_language) + return _domain_label_by_alias(rules, "unclassified", "未分类", output_language) + + +def _domain_label_by_alias( + rules: dict[str, list[dict[str, Any]]], + alias: str, + default: str, + output_language: str | None = None, +) -> str: + """Resolve a fallback folder label through the loaded taxonomy so translated rules stay in charge.""" + wanted = _normalized_domain_label(alias) + for section in DOMAIN_SECTIONS: + for rule in rules[section]: + aliases = { + _normalized_domain_label(item) for item in _as_string_list(rule.get("aliases")) + } + if wanted in aliases: + return _domain_route_label(rule) + return _translate_domain_label(default, output_language) def is_probable_paper_folder(path: Path) -> bool: @@ -3068,8 +3150,7 @@ def is_probable_paper_folder(path: Path) -> bool: (path / marker).exists() for marker in ( f"{path.name}.md", - f"{path.name}.zh-CN.md", - f"{path.name}.en.md", + *(f"{path.name}.{language}.md" for language in sorted(OUTPUT_LANGUAGES)), ".deeppapernote.json", ) ) @@ -3091,13 +3172,20 @@ def existing_domain_dirs(config: dict[str, Any]) -> list[str]: return names -def domain_name_score(domain_name: str, label: str, title: str, abstract: str) -> int: +def domain_name_score( + domain_name: str, + label: str, + title: str, + abstract: str, + *, + output_language: str | None = None, +) -> int: name = domain_name.strip().lower() score = 0 if name == label.lower(): score += 100 lower = normalize_whitespace(f"{title} {abstract}").lower() - rules = load_domain_rules() + rules = load_domain_rules(output_language) label_rules = _rules_for_label(rules, label) label_is_application = any(section == "domains" for section, _ in label_rules) known_fallback_name = any( @@ -3127,13 +3215,17 @@ def resolve_domain_subdir( ) -> str: if subdir.strip(): return subdir.strip() - label = infer_domain_label(title, abstract) + # Folder labels follow the note's output language (ja -> domain_rules.ja.yaml). + output_language = str(config.get("output_language", "")).strip() or None + label = infer_domain_label(title, abstract, output_language=output_language) existing = existing_domain_dirs(config) if existing: best_name = "" best_score = -1 for domain_name in existing: - score = domain_name_score(domain_name, label, title, abstract) + score = domain_name_score( + domain_name, label, title, abstract, output_language=output_language + ) if score > best_score: best_name = domain_name best_score = score diff --git a/skills/deeppapernote/scripts/contracts.py b/skills/deeppapernote/scripts/contracts.py index 0d37641..fc693a4 100644 --- a/skills/deeppapernote/scripts/contracts.py +++ b/skills/deeppapernote/scripts/contracts.py @@ -411,8 +411,153 @@ def required_field_value_error( } +PAPER_TYPE_CONTRACTS_JA: dict[str, dict[str, Any]] = { + "AI_method": { + "paper_type": "AI_method", + "reader_lens": "手法の機構を再現できる技術読者向け", + "section_focus": ["問題設定", "手法の機構", "学習/推論フロー", "主要な数式", "比較ベースライン", "アブレーションと失敗の境界"], + "required_checks": ["機構フロー、主要な数式、実験デザイン、アブレーションの含意、失敗の境界を説明する必要がある。"], + "formula_rules": ["手法の理解に必須の 1〜3 個の主要な数式のみを残し、その工学的含意を説明する。"], + "avoid_rules": ["AI_method でない論文を無理にモデルアーキテクチャとして書き換えない。"], + "boundary_questions": [ + "中心的な機構の利得は、主結果が示唆するだけでなく、どの実験やアブレーションで裏付けられているか？", + "どの比較が現在のデータ・ベースライン・計算資源・プロトコルの下でのみ有効で、汎用的な状況へ外挿できないか？", + "論文は失敗・劣化・不安定・コスト増の証拠を示しているか。示していない場合、結論の境界は何か？", + ], + "section_semantics": { + "データとタスク定義": "データセット、入出力、評価タスク、実験設定。", + "主要な結果": "主結果、強力なベースライン、アブレーション、重要な数値。", + "手法の骨子": "モデル・アルゴリズム・学習または推論の機構。", + "深掘り分析": "手法がなぜ有効か、どこが脆いか、再現・拡張のコスト。", + "研究課題": "手法が解決しようとする具体的な技術課題と、既存手法の弱点。", + }, + "recommended_subsections": { + "手法の骨子": ["機構フロー", "モデル構造", "学習目標", "推論・サンプリング経路", "主要な実装詳細"], + "主要な結果": ["主要結果と強力なベースライン", "アブレーションが示すもの", "失敗・不安定な設定"], + "深掘り分析": ["なぜ有効か", "計算量とスケーラビリティ", "再現時の注意点"], + }, + "mechanism_flow_contract": { + "apply_when_paper_type_in": ["AI_method"], + "required_step_count": "3_to_4", + "required_step_fields": ["input", "operation", "output_destination"], + }, + }, + "benchmark_or_dataset": { + "paper_type": "benchmark_or_dataset", + "reader_lens": "benchmark/dataset の有用性とバイアスの境界を判断したい研究者向け", + "section_focus": ["タスク分割", "データ出所と構築フロー", "アノテーションプロトコル", "評価指標", "カバレッジ範囲とバイアス", "サンプル統計とデータ公開制限"], + "required_checks": ["データ出所、構築/アノテーションのフロー、評価指標、ベースライン性能、サンプル統計、データ公開やプライバシー制限、適用範囲の境界を説明する必要がある。"], + "formula_rules": ["中心的な評価指標、サンプリング規則、分割定義のみを残す。"], + "avoid_rules": ["データ構築フローをモデルの pipeline として書かない。"], + "boundary_questions": [ + "この benchmark/dataset が実際に測定している構成概念は何で、どの能力は間接的な近似に過ぎないか？", + "タスク・ラベル・サンプリング・フィルタ・評価プロトコルは、どのようなカバレッジの欠落やバイアスを持ち込むか？", + "ベースライン結果は評価セットに識別力があることを示すのか、それとも特定タイプのモデルがこのプロトコルに適応しただけか？", + "サンプルの長さ、コーパス長、人口統計、クラス分布、データのアクセス可能性やプライバシー制限は、再現と外挿にどう影響するか？", + ], + "section_semantics": { + "データとタスク定義": "データ出所、タスク分割、ラベル/課題の定義、サンプル範囲。", + "主要な結果": "ベースライン性能、難易度分布、カバレッジ範囲、バイアス。", + "手法の骨子": "データ構築・選別・アノテーション・評価プロトコル。モデルの pipeline としては書かない。", + "深掘り分析": "実際に何を測れているか、そして何を代表できないか。", + "研究課題": "この benchmark/dataset が埋めようとする評価またはデータのギャップ。", + }, + "recommended_subsections": { + "データとタスク定義": ["データ出所", "タスク分割", "アノテーション/選別プロトコル"], + "手法の骨子": ["構築フロー", "評価プロトコル", "ベースライン設定"], + "主要な結果": ["ベースライン性能", "難易度分布", "カバレッジとバイアス"], + "深掘り分析": ["ベンチマークが実際に測るもの", "適用範囲の境界"], + }, + }, + "clinical_or_psychology_empirical": { + "paper_type": "clinical_or_psychology_empirical", + "reader_lens": "臨床/心理学のサンプル、変数関係、外挿の境界に注目する研究読者向け", + "section_focus": ["サンプル出所", "選択・除外基準", "変数または尺度", "分析パイプライン", "効果量と不確実性", "サンプル統計・倫理・データのアクセス可能性"], + "required_checks": ["相関・予測・群間差・因果的解釈を区別し、サンプル統計、倫理/プライバシー上の制約、外挿の境界を説明する必要がある。"], + "formula_rules": ["中心的な統計モデル、効果量、信頼区間、尺度の定義のみを残す。"], + "avoid_rules": ["相関、予測性能、群間差を、証明されていない因果結論として書かない。"], + "boundary_questions": [ + "サンプル出所、選択・除外基準、測定ツール、アノテーション手順は、外挿をどのように制限するか？", + "結果は相関・予測・群間差・因果的解釈のどれを支持するか。論文のデザインが証明できる範囲を越えないこと。", + "臨床または心理学的意義は、未観測の交絡、尺度の閾値、テキスト/音声の欠落、場面の制約に依存していないか？", + "サンプル構成、データ欠損、プライバシー制限、資料の非公開は、再現と再分析をどのように制限するか？", + ], + "section_semantics": { + "データとタスク定義": "サンプル出所、選択・除外基準、変数/尺度、測定方法。", + "主要な結果": "主効果、相関、群間差、不確実性や有意性。", + "手法の骨子": "研究デザイン、群分け、測定手順、統計解析の道筋。", + "深掘り分析": "結果の解釈、因果の境界、臨床/心理学的意義、外挿の限界。", + "研究課題": "臨床・心理学・行動科学における研究課題、仮説、変数間の関係。", + }, + "recommended_subsections": { + "データとタスク定義": ["サンプルと選択・除外基準", "変数と尺度", "測定手順"], + "手法の骨子": ["研究デザイン", "分析モデル", "主要な比較"], + "主要な結果": ["主要な効果", "不確実性と有意性", "臨床・心理学的解釈"], + "深掘り分析": ["因果解釈の境界", "外挿の限界"], + }, + }, + "humanities_or_social_science": { + "paper_type": "humanities_or_social_science", + "reader_lens": "理論的枠組み、資料の解釈、論証の構造に注目する研究読者向け", + "section_focus": ["研究対象", "資料の出所", "理論的枠組み", "論証の道筋", "概念的貢献", "解釈の境界"], + "required_checks": ["著者の論証、資料的証拠、規範的判断、実験的事実を区別する必要がある。"], + "formula_rules": ["通常は数式を無理に残さない。中心的な形式的定義やコーディング規則のみを残す。"], + "avoid_rules": ["規範的判断、テキスト解釈、事例分析を実験的事実として書かない。"], + "boundary_questions": [ + "著者の解釈は、どの資料・事例・理論的前提に依存しているか？", + "同じく資料を説明できる代替的解釈は存在するか。論文はそれをどのように排除したか、あるいは排除していないか？", + "どの結論が概念的貢献や規範的判断であり、経験的事実として直接扱えないものか？", + ], + "section_semantics": { + "データとタスク定義": "資料、事例、テキスト、インタビュー、アーカイブ、コーパスの範囲。ML task としては書かない。", + "主要な結果": "中心的な解釈的発見、概念的貢献、既存の見解への修正。", + "手法の骨子": "理論的枠組み、概念の区別、論証の道筋。", + "深掘り分析": "論証の強さ、資料の境界、解釈の代替可能性、転用可能性。", + "研究課題": "著者が説明しようとする社会・文化・歴史・制度・理論上の問い。", + }, + "recommended_subsections": { + "データとタスク定義": ["資料の範囲", "選択基準", "事例・コーパスの境界"], + "手法の骨子": ["理論的枠組み", "概念の区別", "論証の道筋"], + "主要な結果": ["中心的な解釈的発見", "概念的貢献"], + "深掘り分析": ["論証の強さ", "代替的解釈", "資料の境界"], + }, + }, + "survey_or_review": { + "paper_type": "survey_or_review", + "reader_lens": "サーベイの脈絡、分類体系、証拠の境界を整理する必要のある研究読者向け", + "section_focus": ["サーベイの範囲", "選定・除外基準", "テーマ分類", "手法の系譜", "合意と対立", "未解決問題"], + "required_checks": ["サーベイの範囲、文献の選定、分類体系、合意と対立、未解決問題を説明する必要がある。"], + "formula_rules": ["分類軸、選定・除外の基準、証拠の集約規則、meta-analysis の統計量のみを残す。"], + "avoid_rules": ["サーベイ中の代表的な結論を、著者自身が行った単一の実験結果として書かない。"], + "boundary_questions": [ + "検索範囲、選定・除外基準、分類軸は、どの研究の流れを見落とすか？", + "サーベイが示すのは分野の合意か、著者の分類か、それとも未解決の対立か？", + "どのトレンド結論がカバレッジ内の文献分布に由来し、技術成熟度の判断として直接扱えないか？", + ], + "section_semantics": { + "データとタスク定義": "対象とする文献範囲、検索/選別基準、レビュー対象。", + "主要な結果": "分野の合意、対立、トレンド、代表的な方向性、未解決問題。", + "手法の骨子": "分類体系、レビューの構成方法、証拠統合のロジック。単一論文の手法構成としては書かない。", + "深掘り分析": "サーベイがカバーしていない盲点、分類体系の説明力、今後の研究機会。", + "研究課題": "サーベイが整理しようとする分野の問い、論争、知識のギャップ。", + }, + "recommended_subsections": { + "データとタスク定義": ["サーベイの範囲", "選定・除外基準", "文献カバレッジ"], + "手法の骨子": ["分類体系", "手法の系譜", "証拠の整理方法"], + "主要な結果": ["代表的な方向性", "合意と対立", "未解決問題"], + "深掘り分析": ["分類体系の限界", "未カバー領域", "今後の研究機会"], + }, + }, +} + + def paper_type_contracts(language: str | None = None) -> dict[str, dict[str, Any]]: - return deepcopy(PAPER_TYPE_CONTRACTS_EN if normalize_output_language(language) == "en" else PAPER_TYPE_CONTRACTS) + resolved = normalize_output_language(language) + if resolved == "en": + return deepcopy(PAPER_TYPE_CONTRACTS_EN) + if resolved == "ja": + return deepcopy(PAPER_TYPE_CONTRACTS_JA) + return deepcopy(PAPER_TYPE_CONTRACTS) WRITING_CONTRACT_RULES: dict[str, Any] = { "required_sections": NOTE_REQUIRED_SECTIONS, diff --git a/skills/deeppapernote/scripts/lint_grounding.py b/skills/deeppapernote/scripts/lint_grounding.py index 9c6b257..561ca2a 100644 --- a/skills/deeppapernote/scripts/lint_grounding.py +++ b/skills/deeppapernote/scripts/lint_grounding.py @@ -191,7 +191,7 @@ def parser() -> argparse.ArgumentParser: p.add_argument( "--language", default="", - help="Run Override for output language: en or zh-CN.", + help="Run Override for output language: en, zh-CN, or ja.", ) return p diff --git a/skills/deeppapernote/scripts/lint_note.py b/skills/deeppapernote/scripts/lint_note.py index 619babd..10e88cd 100644 --- a/skills/deeppapernote/scripts/lint_note.py +++ b/skills/deeppapernote/scripts/lint_note.py @@ -63,6 +63,11 @@ def figure_prefix(key: str) -> str: "未放置", "未处理", "待补", + # ja + "残り", + "未配置", + "未処理", + "補充待ち", } FIGURE_BUCKET_VISUAL_TOKENS = { @@ -71,6 +76,11 @@ def figure_prefix(key: str) -> str: "图片", "图表", "占位", + # ja + "図", + "画像", + "図表", + "プレースホルダ", } ENGLISH_FIGURE_BUCKET_RESIDUE_TOKENS = { @@ -100,6 +110,10 @@ def figure_prefix(key: str) -> str: | (?:图表|图片|图|表)\s*占位\s*[:：]\s*\S+ | + \[\s*(?:図表|画像|図|表)\s*プレースホルダ\s*\|[^\]]+\] + | + (?:図表|画像|図|表)\s*プレースホルダ\s*[:：]\s*\S+ + | \[\s*(?:figure|fig|table)\s+placeholder\s*(?:\||:|\]|-|\s+(?:fig(?:ure)?|table)\.?\s*\d) | (?:figure|fig|table)\s+placeholder\s*(?:\||:|-|\s+(?:fig(?:ure)?|table)\.?\s*\d) @@ -114,6 +128,10 @@ def figure_prefix(key: str) -> str: | (?:替换|插入)\s*为\s*真实图片 | + (?:置換|挿入|コピー|複製|書き込み|物化)\s*(?:済み|した|完了) + | + 実(?:画像|図)(?:に|へ)\s*(?:置換|差し替え) + | \b(?:inserted|replaced|copied|materialized)\b ) """, @@ -137,6 +155,14 @@ def figure_prefix(key: str) -> str: | 高\s*置信(?:度)?[^。；，\n>]{0,12}候选 | + 候補[^。；，\n>]{0,24}(?]{0,12}(?]{0,12}候補 + | usable\s+candidate | readable\s+crop @@ -158,6 +184,8 @@ def figure_prefix(key: str) -> str: | 无法稳定|不可独立解释|质量门|reject_visual_quality | + 混入|汚染|隣接|切れ|途切れ|欠落|欠けて|不完全|単独では解釈できない|品質ゲート + | partial|subpanel|contaminat|truncat|incomplete|missing | caption\s*(?:missing|cut|truncated) @@ -169,13 +197,13 @@ def figure_prefix(key: str) -> str: USABLE_CANDIDATE_MATERIALIZATION_BLOCKED_RE = re.compile( r""" (?: - (?:materialize_figure_asset\.py|物化|复制|拷贝|写入|权限|permission|工具|copy) + (?:materialize_figure_asset\.py|物化|复制|拷贝|写入|权限|permission|工具|copy|コピー|書き込み|権限|ツール) [^。；\n]{0,40} - (?:失败|不足|拒绝|denied|blocked|error|报错) + (?:失败|不足|拒绝|denied|blocked|error|报错|失敗|拒否|エラー) | - (?:失败|不足|拒绝|denied|blocked|error|报错) + (?:失败|不足|拒绝|denied|blocked|error|报错|失敗|拒否|エラー) [^。；\n]{0,40} - (?:materialize|物化|复制|拷贝|写入|权限|permission|copy) + (?:materialize|物化|复制|拷贝|写入|权限|permission|copy|コピー|書き込み|権限) ) """, flags=re.IGNORECASE | re.VERBOSE, @@ -184,13 +212,13 @@ def figure_prefix(key: str) -> str: MISSING_ASSET_MATERIALIZATION_RE = re.compile( r""" (?: - (?:资产缺失|未找到|没有|缺少|asset_candidate_missing|candidate\s+missing) + (?:资产缺失|未找到|没有|缺少|asset_candidate_missing|candidate\s+missing|候補が(?:ない|見つからない)|アセット(?:が)?(?:欠落|なし)) [^。；\n]{0,50} - (?:materialize_figure_asset\.py|物化|复制|拷贝|写入|权限|permission|copy|blocked) + (?:materialize_figure_asset\.py|物化|复制|拷贝|写入|权限|permission|copy|blocked|コピー|書き込み|権限) | - (?:materialize_figure_asset\.py|物化|复制|拷贝|写入|权限|permission|copy|blocked) + (?:materialize_figure_asset\.py|物化|复制|拷贝|写入|权限|permission|copy|blocked|コピー|書き込み|権限) [^。；\n]{0,50} - (?:资产缺失|未找到|没有|缺少|asset_candidate_missing|candidate\s+missing) + (?:资产缺失|未找到|没有|缺少|asset_candidate_missing|candidate\s+missing|候補が(?:ない|見つからない)|アセット(?:が)?(?:欠落|なし)) ) """, flags=re.IGNORECASE | re.VERBOSE, @@ -211,6 +239,18 @@ def figure_prefix(key: str) -> str: r"\\operatorname\{(?:输入|输出|损失|状态|动作|奖励|标签|样本|预测|目标)\}" ) ENGLISH_MATH_SPAN_RE = re.compile(r"\$\$[^$\n]+\$\$|\$[^$\n]+\$") +# 漢字（zh-CN / ja 共通）と仮名（ja）。混在言語判定と文中改行判定で使う。 +TARGET_SCRIPT_RE = re.compile(r"[\u3040-\u30ff\u4e00-\u9fff]") +# \u65e5\u672c\u8a9e\u306e\u5e38\u7528\u6f22\u5b57\u30fb\u65b0\u5b57\u4f53\u306b\u306f\u73fe\u308c\u306a\u3044\u7c21\u4f53\u5b57\u3002ja \u30d7\u30ed\u30d5\u30a1\u30a4\u30eb\u3067 +# \u4e2d\u56fd\u8a9e\u30c6\u30f3\u30d7\u30ec\u30fc\u30c8\u306e\u898b\u51fa\u3057\u30fb\u30e9\u30d9\u30eb\uff08\u673a\u5236\u6d41\u7a0b / \u5efa\u8bae\u4f4d\u7f6e \u306a\u3069\uff09\u306e\u6b8b\u5b58\u3092\u691c\u51fa\u3059\u308b\u3002 +# \u65e5\u672c\u8a9e\u3067\u3082\u4f7f\u3046\u5b57\uff08\u6570\u30fb\u636e\u30fb\u540e\u30fb\u6ca1\u30fb\u91cc\u30fb\u4e0e\u30fb\u4f1a\u30fb\u533a\u30fb\u53cc\u30fb\u968f\u30fb\u51c6 \u306a\u3069\uff09\u306f\u610f\u56f3\u7684\u306b\u542b\u3081\u306a\u3044\u3002 +SIMPLIFIED_CHINESE_ONLY_CHARS = ( + "\u4eec\u8fd9\u4e2a\u4e3a\u4ece\u8bf4\u65f6\u673a\u5173\u952e\u7ed3\u8bba\u56fe\u5e94\u8be5\u8fdb\u663e\u5b9e\u73b0\u8ba4\u8bc6\u8ba9\u8fd8\u8fc7\u6837\u79cd\u4e49\u672f\u573a\u8bad\u7ec3\u8bd5\u9a8c\u8bc4\u6d4b\u6784\u9636" + "\u8f93\u6001\u52a8\u52a1\u5904\u5907\u7ea7\u7c7b\u7ec4\u7ec7\u7edc\u7f51\u7ebf\u7f16\u8bd1\u5bfc\u603b\u8fbe\u53d1\u957f\u95e8\u95ee\u9898\u4f18\u52bf\u9009\u62e9\u9519\u8bef\u786e\u590d\u6742\u7b80\u5355\u5c42\u91ca\u8ba1\u8bbe" + "\u6743\u9884\u68c0\u62a5\u8865\u635f\u6807\u9891\u89c6\u89c9\u8bed\u8bcd\u6c47\u5e93\u522b\u7f13\u5757\u89c4\u73af\u7edf\u4f20\u53d8\u6362\u8f6c\u8f7d\u8f6e\u8f66\u9875\u7efc\u4e60\u5bf9\u8bf7\u8c22\u4e48\u5417\u5462\u5427\u5f88" + "\u8bae\u7b5b\u7eb3" +) +SIMPLIFIED_CHINESE_ONLY_RE = re.compile(f"[{''.join(sorted(set(SIMPLIFIED_CHINESE_ONLY_CHARS)))}]") RUNTIME_ARTIFACT_REFERENCE_PATTERNS = [ re.compile( @@ -253,7 +293,7 @@ def parser() -> argparse.ArgumentParser: p.add_argument("--plan-file", default="", help="Optional note_plan JSON path. Defaults to sibling .plan.json.") p.add_argument("--output", default="", help="Output JSON path.") p.add_argument("--paper-id", default="", help="Canonical paper id.") - p.add_argument("--language", default="", help="Run Override for output language: en or zh-CN.") + p.add_argument("--language", default="", help="Run Override for output language: en, zh-CN, or ja.") return p @@ -369,8 +409,13 @@ def front_matter_order_warnings(text: str) -> list[str]: return warnings +STRICT_TOP_LEVEL_PROFILE_LANGUAGES = {"en", "ja"} +# 中国語プロファイル向けの機械翻訳痕跡パターンは日本語には適用しない +MECHANICAL_TRANSLATION_CHECK_LANGUAGES = {"zh-CN"} + + def english_top_level_section_warnings(text: str) -> list[str]: - if ACTIVE_LANGUAGE != "en": + if ACTIVE_LANGUAGE not in STRICT_TOP_LEVEL_PROFILE_LANGUAGES: return [] actual = re.findall(r"^##\s+(.+?)\s*$", text, flags=re.MULTILINE) return [] if actual == list(REQUIRED_SECTIONS) else ["top_level_section_profile_invalid"] @@ -421,6 +466,13 @@ def inspect_reference_hygiene(text: str) -> list[dict[str, object]]: "model", "system", "module", + # ja + "モデル", + "フレームワーク", + "システム", + "モジュール", + "エンコーダ", + "デコーダ", ] MECHANISM_IO_TOKENS = [ @@ -434,6 +486,11 @@ def inspect_reference_hygiene(text: str) -> list[dict[str, object]]: "output", "produces", "returns", + # ja + "入力", + "出力", + "渡す", + "得る", ] MECHANISM_ACTION_TOKENS = [ @@ -456,6 +513,16 @@ def inspect_reference_hygiene(text: str) -> list[dict[str, object]]: "decode", "update", "aggregate", + # ja + "整列", + "圧縮", + "投影", + "抽出", + "符号化", + "復号", + "連結", + "更新", + "集約", ] @@ -501,6 +568,13 @@ def inspect_reference_hygiene(text: str) -> list[dict[str, object]]: r"^参见原论文[。.!！]*$", r"^这里记录.*[。.!！]*$", r"^本节记录.*[。.!！]*$", + # ja + r"^未記入[。.!！]*$", + r"^後で追記[。.!！]*$", + r"^なし[。.!！]*$", + r"^省略[。.!！]*$", + r"^原論文を参照[。.!！]*$", + r"^ここに.*を記録.*[。.!！]*$", ] GENERIC_INNOVATION_PATTERNS = [ @@ -508,6 +582,10 @@ def inspect_reference_hygiene(text: str) -> list[dict[str, object]]: r"具有创新性", r"novel approach", r"首次实现", + # ja + r"新しい手法を提案(?:する|した)?", + r"新規性が(?:ある|高い)", + r"初めて実現(?:した)?", ] GENERIC_KEY_RESULT_PATTERNS = [ @@ -515,6 +593,11 @@ def inspect_reference_hygiene(text: str) -> list[dict[str, object]]: r"结果表明.*有效", r"取得(?:了)?较好效果", r"性能.*优越", + # ja + r"実験結果.*手法.*有効", + r"結果.*有効(?:である|だ)", + r"良好な(?:結果|効果|性能)", + r"性能.*優(?:れ|位|越)", ] GENERIC_LIMITATION_PATTERNS = [ @@ -523,11 +606,27 @@ def inspect_reference_hygiene(text: str) -> list[dict[str, object]]: r"future work can", r"more data", r"后续.*扩展", + # ja + r"今後の(?:課題|研究).*データ", + r"(?:さらなる|より多くの)データが必要", + r"今後.*拡張", ] -HONEST_MISSING_TOKENS = ("本文未给出", "论文未给出", "未报告", "没有报告", "未提供") -HONEST_MISSING_BASIS_TOKENS = ("依据", "正文", "附录", "表格", "coverage", "作者") -HONEST_MISSING_IMPACT_TOKENS = ("影响", "限制", "受限", "不能", "无法", "结论强度") +HONEST_MISSING_TOKENS = ( + "本文未给出", "论文未给出", "未报告", "没有报告", "未提供", + # ja + "報告されていない", "未報告", "未提供", "示されていない", "記載がない", +) +HONEST_MISSING_BASIS_TOKENS = ( + "依据", "正文", "附录", "表格", "coverage", "作者", + # ja + "根拠", "本文", "付録", "表", +) +HONEST_MISSING_IMPACT_TOKENS = ( + "影响", "限制", "受限", "不能", "无法", "结论强度", + # ja + "影響", "制限", "制約", "できない", "不可能", "結論の強さ", +) DOUBLE_ESCAPED_TEX_COMMANDS = { "alpha", @@ -604,6 +703,29 @@ def subsection_name_for_line(lines: list[str], line_index: int) -> str: return current_subsection +def simplified_chinese_leftover_issue( + lines: list[str], idx: int, stripped: str +) -> dict[str, object] | None: + """日本語ノートに簡体字中国語の見出し・ラベル・本文が残っていれば問題として返す。 + + 漢字は日中で共有されるため、日本語では使われない簡体字（機→机、図→图 など） + だけを手がかりにする。基本情報と参考文献の原語メタデータ、URL、インラインコード、 + リンクは対象外。 + """ + section_name = section_name_for_line(lines, idx - 1) + checked = stripped + if section_name in {section("core_information"), section("references")}: + for pattern in ENGLISH_METADATA_SOURCE_SPAN_PATTERNS: + checked = pattern.sub("", checked) + if stripped.startswith(f"*{FIGURE_LABELS['original_caption']}"): + checked = ENGLISH_METADATA_SOURCE_SPAN_PATTERNS[0].sub("", checked) + checked = ENGLISH_CJK_ENTITY_LINK_RE.sub("", checked) + checked = HTTP_URL_RE.sub("", checked) + if not SIMPLIFIED_CHINESE_ONLY_RE.search(checked): + return None + return {"line_number": idx, "line": stripped, "reason": "simplified_chinese_text_present"} + + def mixed_language_issues(text: str) -> list[dict[str, object]]: issues: list[dict[str, object]] = [] lines = text.splitlines() @@ -638,13 +760,18 @@ def mixed_language_issues(text: str) -> list[dict[str, object]]: if re.search(r"[\u4e00-\u9fff]", checked): issues.append({"line_number": idx, "line": stripped, "reason": "non_english_text_present"}) continue + if ACTIVE_LANGUAGE == "ja" and idx not in fenced_code_lines: + simplified_issue = simplified_chinese_leftover_issue(lines, idx, stripped) + if simplified_issue: + issues.append(simplified_issue) + continue if is_exempt_line(line): continue section_name = section_name_for_line(lines, idx - 1) subsection_name = subsection_name_for_line(lines, idx - 1) if section_name in {section("core_information"), section("references")}: continue - if not re.search(r"[\u4e00-\u9fff]", stripped): + if not TARGET_SCRIPT_RE.search(stripped): continue english_words = re.findall(r"\b[A-Za-z][A-Za-z0-9.-]*\b", stripped) if len(english_words) < 4: @@ -664,7 +791,7 @@ def mixed_language_issues(text: str) -> list[dict[str, object]]: def mechanical_translation_artifact_issues(text: str) -> list[dict[str, object]]: - if ACTIVE_LANGUAGE == "en": + if ACTIVE_LANGUAGE not in MECHANICAL_TRANSLATION_CHECK_LANGUAGES: return [] issues: list[dict[str, object]] = [] for idx, line in enumerate(text.splitlines(), start=1): @@ -1099,9 +1226,9 @@ def suspicious_mid_sentence_linebreaks(text: str) -> list[dict[str, object]]: if re.search(r"[。！？.!?：:]$", current): continue if not re.search(r"[，,；;、）)\]」』]$", current): - if not re.search(r"[A-Za-z0-9`\u4e00-\u9fff]$", current): + if not re.search(r"[A-Za-z0-9`\u3040-\u30ff\u4e00-\u9fff]$", current): continue - if not re.match(r"^[A-Za-z0-9`\u4e00-\u9fff(（“‘\"]", nxt): + if not re.match(r"^[A-Za-z0-9`\u3040-\u30ff\u4e00-\u9fff(（“‘\"]", nxt): continue issues.append( { @@ -1479,7 +1606,7 @@ def meaningful_units(body: str, generic_patterns: list[str] | None = None) -> li def has_number_token(text: str) -> bool: - return bool(re.search(r"\d+(?:\.\d+)?\s*(?:%|％|[A-Za-z\u4e00-\u9fff]{0,8})", text)) + return bool(re.search(r"\d+(?:\.\d+)?\s*(?:%|％|[A-Za-z\u3040-\u30ff\u4e00-\u9fff]{0,8})", text)) def is_honest_missing_declaration(text: str) -> bool: diff --git a/skills/deeppapernote/scripts/localization.py b/skills/deeppapernote/scripts/localization.py index a73b028..62208b6 100644 --- a/skills/deeppapernote/scripts/localization.py +++ b/skills/deeppapernote/scripts/localization.py @@ -8,8 +8,12 @@ from typing import Any DEFAULT_OUTPUT_LANGUAGE = "zh-CN" -SUPPORTED_OUTPUT_LANGUAGES = ("zh-CN", "en") -_ALIASES = {"zh": "zh-CN", "zh-cn": "zh-CN", "zh_cn": "zh-CN", "chinese": "zh-CN", "en": "en", "en-us": "en", "en_us": "en", "english": "en"} +SUPPORTED_OUTPUT_LANGUAGES = ("zh-CN", "en", "ja") +_ALIASES = { + "zh": "zh-CN", "zh-cn": "zh-CN", "zh_cn": "zh-CN", "chinese": "zh-CN", + "en": "en", "en-us": "en", "en_us": "en", "english": "en", + "ja": "ja", "ja-jp": "ja", "ja_jp": "ja", "japanese": "ja", "日本語": "ja", +} _SCHEMAS: dict[str, dict[str, Any]] = { "zh-CN": { "sections": {"core_information": "核心信息", "abstract": "原文摘要翻译", "contributions": "创新点", "one_sentence_summary": "一句话总结", "research_questions": "研究问题", "data_and_task": "数据与任务定义", "method": "方法主线", "key_results": "关键结果", "deep_analysis": "深度分析", "limitations": "局限", "my_notes": "我的笔记", "references": "引用"}, @@ -34,6 +38,22 @@ ], }, }, + "ja": { + "sections": {"core_information": "基本情報", "abstract": "要旨の翻訳", "contributions": "新規性", "one_sentence_summary": "一言まとめ", "research_questions": "研究課題", "data_and_task": "データとタスク定義", "method": "手法の骨子", "key_results": "主要な結果", "deep_analysis": "深掘り分析", "limitations": "限界", "my_notes": "私のメモ", "references": "参考文献"}, + "core_info_fields": ("タイトル", "タイトル訳", "著者", "所属", "発表時期", "発表媒体", "DOI", "arXiv", "論文リンク", "コード / プロジェクト", "データ / リソース", "論文タイプ"), + "core_info_aliases": {}, + "figure_labels": {"location": "推奨位置：", "reason": "配置理由：", "status": "現在の状態：", "original_caption": "論文原図番号："}, + "mechanism_flow": "機構フロー", + "abstract_contract": { + "source": "source_abstract", + "requirement": "faithful_rendering_in_output_language", + "forbidden_additions": [ + "later_contribution_claims", + "later_result_interpretation", + "hindsight_judgment", + ], + }, + }, } def normalize_output_language(value: str | None = None) -> str: diff --git a/skills/deeppapernote/scripts/plan_figure_table_decisions.py b/skills/deeppapernote/scripts/plan_figure_table_decisions.py index 9c25338..82fe238 100644 --- a/skills/deeppapernote/scripts/plan_figure_table_decisions.py +++ b/skills/deeppapernote/scripts/plan_figure_table_decisions.py @@ -44,7 +44,7 @@ def parser() -> argparse.ArgumentParser: p.add_argument( "--language", default="", - help="Run Override for output language: en or zh-CN.", + help="Run Override for output language: en, zh-CN, or ja.", ) return p diff --git a/skills/deeppapernote/scripts/plan_figures.py b/skills/deeppapernote/scripts/plan_figures.py index 039e3f9..2d768c3 100644 --- a/skills/deeppapernote/scripts/plan_figures.py +++ b/skills/deeppapernote/scripts/plan_figures.py @@ -17,7 +17,7 @@ def parser() -> argparse.ArgumentParser: p.add_argument("--assets", default="", help="PDF assets JSON path or string.") p.add_argument("--output", default="", help="Output JSON path.") p.add_argument("--paper-id", default="", help="Canonical paper id.") - p.add_argument("--language", default="", help="Output language: en or zh-CN.") + p.add_argument("--language", default="", help="Output language: en, zh-CN, or ja.") p.add_argument("--max-items", type=int, default=12, help="Maximum number of figure/table items to keep. 0 means keep all.") return p @@ -160,13 +160,29 @@ def _classify_caption_kind_zh(item_id: str, caption: str) -> tuple[str, str, str "supporting_figure": ("Deep Analysis", "This supporting visual helps explain the authors' argument in Deep Analysis."), } +JAPANESE_FIGURE_PLACEMENT: dict[str, tuple[str, str]] = { + "main_result": ("主要な結果", "この図表は主結果を担っており、主要な結果に置くのが適切である。"), + "data_or_task_overview": ("データとタスク定義", "この図はデータとタスクの出所・構築・選別・範囲を説明する。"), + "method_overview": ("機構フロー", "この図は手法またはシステムの流れを要約しており、対応が確実なら機構フローに置く。"), + "data_or_task": ("データとタスク定義", "この図はタスク設定・サンプル・データセットを明確にする。"), + "method_detail": ("手法の骨子", "この図は内部機構や実行状態を説明しており、手法の骨子に置くのが適切である。"), + "table_result": ("主要な結果", "この結果表は読者が中心的な数値的証拠を見つける助けになる。"), + "supporting_figure": ("深掘り分析", "この補助図は深掘り分析で著者の論点を説明する助けになる。"), +} + +LOCALIZED_FIGURE_PLACEMENT: dict[str, dict[str, tuple[str, str]]] = { + "en": ENGLISH_FIGURE_PLACEMENT, + "ja": JAPANESE_FIGURE_PLACEMENT, +} + def classify_caption_kind(item_id: str, caption: str, language: str | None = None) -> tuple[str, str, str]: result = _classify_caption_kind_zh(item_id, caption) - if normalize_output_language(language) != "en": + placement = LOCALIZED_FIGURE_PLACEMENT.get(normalize_output_language(language)) + if placement is None: return result kind = result[0] - section, reason = ENGLISH_FIGURE_PLACEMENT[kind] + section, reason = placement[kind] return kind, section, reason diff --git a/skills/deeppapernote/scripts/run_pipeline.py b/skills/deeppapernote/scripts/run_pipeline.py index cadc46b..542e615 100644 --- a/skills/deeppapernote/scripts/run_pipeline.py +++ b/skills/deeppapernote/scripts/run_pipeline.py @@ -34,7 +34,7 @@ def parser() -> argparse.ArgumentParser: p.add_argument( "--language", default="", - choices=("", "en", "zh-CN"), + choices=("", "en", "zh-CN", "ja"), help="Run Override for the output language contract.", ) p.add_argument("--save-mode", choices=("workspace", "obsidian"), default="") diff --git a/skills/deeppapernote/scripts/user_configuration.py b/skills/deeppapernote/scripts/user_configuration.py index 8e09dbf..7b02156 100644 --- a/skills/deeppapernote/scripts/user_configuration.py +++ b/skills/deeppapernote/scripts/user_configuration.py @@ -13,7 +13,7 @@ from pathlib import Path, PureWindowsPath from typing import Any, Mapping, Sequence -OUTPUT_LANGUAGES = {"zh-CN", "en"} +OUTPUT_LANGUAGES = {"zh-CN", "en", "ja"} SAVE_MODES = {"workspace", "obsidian"} KNOWN_FIELDS = ("output_language", "save_mode", "obsidian_vault", "papers_dir") ALWAYS_REQUIRED = ("output_language", "save_mode") @@ -136,7 +136,7 @@ def _validate(configuration: Mapping[str, Any]) -> tuple[list[dict[str, str]], l language = str(configuration.get("output_language", "")).strip() save_mode = str(configuration.get("save_mode", "")).strip() if language and language not in OUTPUT_LANGUAGES: - issues.append(_issue("output_language", "invalid_enum", "Expected zh-CN or en.")) + issues.append(_issue("output_language", "invalid_enum", "Expected zh-CN, en, or ja.")) if save_mode and save_mode not in SAVE_MODES: issues.append(_issue("save_mode", "invalid_enum", "Expected workspace or obsidian.")) if save_mode == "obsidian": diff --git a/skills/deeppapernote/scripts/write_obsidian_note.py b/skills/deeppapernote/scripts/write_obsidian_note.py index 0cb3790..369dab1 100644 --- a/skills/deeppapernote/scripts/write_obsidian_note.py +++ b/skills/deeppapernote/scripts/write_obsidian_note.py @@ -108,7 +108,7 @@ def parser() -> argparse.ArgumentParser: p.add_argument("--filename", default="", help="Explicit note filename.") p.add_argument("--asset-subdir", default="images", help="Asset folder name relative to the note directory.") p.add_argument("--paper-id", default="", help="Canonical paper id.") - p.add_argument("--language", default="", help="Run Override for output language: en or zh-CN.") + p.add_argument("--language", default="", help="Run Override for output language: en, zh-CN, or ja.") p.add_argument( "--preflight", action="store_true", diff --git a/tests/test_output_language_ja.py b/tests/test_output_language_ja.py new file mode 100644 index 0000000..fba57d5 --- /dev/null +++ b/tests/test_output_language_ja.py @@ -0,0 +1,620 @@ +"""日本語（ja）出力プロファイルの契約・lint・図表配置を検証する。""" + +from __future__ import annotations + +import json +import subprocess +import sys +from pathlib import Path + +import pytest +from build_synthesis_bundle import compact_writing_contract +from contracts import paper_type_contracts +from lint_grounding import validate_note_plan +from localization import ( + normalize_output_language, + require_artifact_output_language, + required_sections, +) +from plan_figures import build_figure_items +from user_configuration import inspect_configuration + +LINT_SCRIPT = Path(__file__).resolve().parents[1] / "skills/deeppapernote/scripts/lint_note.py" + +JAPANESE_SECTIONS = ( + "基本情報", + "要旨の翻訳", + "新規性", + "一言まとめ", + "研究課題", + "データとタスク定義", + "手法の骨子", + "主要な結果", + "深掘り分析", + "限界", + "私のメモ", + "参考文献", +) + + +def japanese_note() -> str: + return """--- +tags: + - papers/methods +aliases: + - "Auditable Tool Use" +date: 2024 +doi: 10.1234/example +--- + +# Auditable Tool Use + +## 基本情報 + +- タイトル: Auditable Tool Use +- タイトル訳: 監査可能なツール利用 +- 著者: Smith et al. +- 発表時期: 2024 +- 発表媒体: Example Journal +- DOI: 10.1234/example +- 論文タイプ: AI_method + +## 要旨の翻訳 + +本論文は多段階の質問応答のために監査可能な状態機械を構築し、明示的な失敗記録が回答の信頼性を改善するかを評価する。 + +## 新規性 + +- 証拠選択とツール状態の追跡を一つの実行記録に統合し、失敗した証拠が黙って信頼できる入力に変わることを防ぐ。 +- 証拠の欠落と推論の誤りを区別する明示的なロールバック状態を追加し、最終回答の追跡を可能にする。 + +## 一言まとめ + +明示的なツール状態の記録は、多段階質問応答における誤りの伝播を減らす。 + +## 研究課題 + +検索が不完全で、外部ツールが失敗し、中間結果が誤用されるとき、多段階質問応答システムはどのように追跡可能性を保てるか。 + +## データとタスク定義 + +入力は質問、候補証拠、利用可能なツールであり、出力は回答、状態トレース、そして完了が支持されない場合の失敗ラベルである。 + +## 手法の骨子 + +### 機構フロー + +1. **入力:** 質問と候補証拠。**操作:** 関連する証拠を抽出する。**出力:** 根拠づけられた初期状態。 +2. **入力:** 現在の状態とツール登録簿。**操作:** 要求を利用可能なツールに整列させる。**出力:** 計画された呼び出し。 +3. **入力:** ツール出力と確信度。**操作:** 状態を更新するかロールバックする。**出力:** 監査可能な実行記録。 +4. **入力:** 最終状態。**操作:** 回答または拒否を復号する。**出力:** 出所を伴う応答。 + +> [!figure] Figure 1 システム概要 +> 推奨位置：手法の骨子 +> 配置理由：この図は証拠とツール状態が実行チェーンをどう流れるかを示す。 +> 現在の状態：プレースホルダを保持。抽出した切り出しは不完全で、単独では解釈できない。 + +## 主要な結果 + +三つのデータセットにわたり、回答精度は 71.2% から 78.5% に向上し、追跡不能な誤りは 18% から 9% に減少した。 + +## 深掘り分析 + +重要な貢献はスコアの向上だけではない。失敗した呼び出しは隠れた中間状態ではなく検査可能な証拠となり、監査と的を絞った復旧を支える。 + +## 限界 + +評価は英語の質問応答データと狭いツール集合に限られており、マルチモーダルなツールや高遅延サービスに対する頑健性は確立していない。 + +## 私のメモ + +状態記録の設計は、ソース資料の欠落とモデルの解釈失敗を分離するため、証拠優先の論文ワークフローでも再利用できる。 + +## 参考文献 + +- Smith et al. (2024). Auditable Tool Use for Multi-hop Question Answering. DOI: 10.1234/example +""" + + +def plan_payload() -> dict: + return { + "output_language": "ja", + "paper_type": "AI_method", + "paper_type_rationale": "モデルの機構を提案し評価する論文である。", + "dominant_domain": "reasoning", + "must_cover": ["手法の骨子"], + "key_numbers": ["78.5%"], + "real_comparisons": ["71.2% 対 78.5%"], + "central_claims": [{ + "claim": "手法は追跡可能性を改善する。", + "supporting_evidence": [{"section_id": "sec:results"}], + "what_it_actually_proves": "報告されたプロトコルはツール状態を記録する。", + "what_it_does_not_prove": "本番環境での頑健性は証明していない。", + }], + "claim_boundaries": ["証拠は報告されたワークフローに限られる。"], + "negative_or_limiting_results": ["マルチモーダルなツールは試験されていない。"], + "mechanism_result_map": ["ロールバック状態が追跡不能な誤りの減少を説明する。"], + "comparative_positioning": ["回答のみのベースラインと比較。"], + "reuse_takeaways": ["失敗状態を明示的に追跡する。"], + "followup_questions": ["欠落・遅延したツール出力を試験する。"], + "section_plan": [{"section": "手法の骨子", "evidence_sources": [{"section_id": "sec:method"}]}], + } + + +def run_lint(tmp_path: Path, note: str, *extra_args: str) -> dict: + note_path = tmp_path / "paper.md" + plan_path = tmp_path / "paper.plan.json" + output_path = tmp_path / "lint.json" + note_path.write_text(note, encoding="utf-8") + plan_path.write_text(json.dumps(plan_payload(), ensure_ascii=False), encoding="utf-8") + subprocess.run( + [ + sys.executable, + str(LINT_SCRIPT), + *extra_args, + "--input", + str(note_path), + "--plan-file", + str(plan_path), + "--output", + str(output_path), + ], + check=True, + ) + return json.loads(output_path.read_text(encoding="utf-8")) + + +def test_japanese_language_aliases() -> None: + assert normalize_output_language("ja") == "ja" + assert normalize_output_language("Japanese") == "ja" + assert normalize_output_language("ja-JP") == "ja" + assert normalize_output_language("日本語") == "ja" + assert require_artifact_output_language({"output_language": "ja"}, "Note Plan", "ja") == "ja" + with pytest.raises(ValueError): + require_artifact_output_language({"output_language": "ja"}, "Note Plan", "en") + + +def test_japanese_user_configuration_accepts_ja(tmp_path: Path) -> None: + path = tmp_path / "config.json" + path.write_text( + json.dumps({"output_language": "ja", "save_mode": "workspace"}), + encoding="utf-8", + ) + result = inspect_configuration(config_path=path, environ={}) + assert result["state"] == "ready" + assert result["affected_fields"] == [] + assert result["configuration"]["output_language"] == "ja" + + +def test_japanese_user_configuration_reports_invalid_enum_with_ja_choice(tmp_path: Path) -> None: + path = tmp_path / "config.json" + path.write_text( + json.dumps({"output_language": "fr", "save_mode": "workspace"}), + encoding="utf-8", + ) + result = inspect_configuration(config_path=path, environ={}) + assert result["state"] != "ready" + messages = [issue["message"] for issue in result["issues"]] + assert any("ja" in message for message in messages) + + +def test_japanese_contract_exposes_localized_schema() -> None: + contract = compact_writing_contract("ja") + assert contract["language"] == "ja" + assert tuple(contract["must_include_sections"]) == JAPANESE_SECTIONS + assert tuple(required_sections("ja")) == JAPANESE_SECTIONS + assert contract["mechanism_flow_heading"] == "機構フロー" + assert contract["core_info_fields"] == [ + "タイトル", + "タイトル訳", + "著者", + "所属", + "発表時期", + "発表媒体", + "DOI", + "arXiv", + "論文リンク", + "コード / プロジェクト", + "データ / リソース", + "論文タイプ", + ] + assert contract["figure_labels"] == { + "location": "推奨位置：", + "reason": "配置理由：", + "status": "現在の状態：", + "original_caption": "論文原図番号：", + } + assert contract["abstract_contract"]["requirement"] == "faithful_rendering_in_output_language" + # 日本語プロファイルの契約本文に簡体字中国語のセクション名が残っていないこと + dumped = json.dumps(contract, ensure_ascii=False) + for leftover in ("核心信息", "方法主线", "关键结果", "深度分析", "我的笔记"): + assert leftover not in dumped + + +def test_japanese_paper_type_contracts_use_japanese_section_names() -> None: + contracts = paper_type_contracts("ja") + assert set(contracts) == { + "AI_method", + "benchmark_or_dataset", + "clinical_or_psychology_empirical", + "humanities_or_social_science", + "survey_or_review", + } + ai_method = contracts["AI_method"] + assert "手法の骨子" in ai_method["section_semantics"] + assert "機構フロー" in ai_method["recommended_subsections"]["手法の骨子"] + assert ai_method["mechanism_flow_contract"]["required_step_count"] == "3_to_4" + for contract in contracts.values(): + assert set(contract["section_semantics"]) <= set(JAPANESE_SECTIONS) + assert set(contract["recommended_subsections"]) <= set(JAPANESE_SECTIONS) + + +def test_japanese_grounding_accepts_japanese_section_plan() -> None: + plan = plan_payload() + plan["central_claims"][0]["supporting_evidence"] = [{"section_id": "sec:method"}] + plan["section_plan"] = [ + { + "section": section, + "focus": f"{section} に固有の証拠と分析上の役割を説明する。", + "evidence_sources": [{"section_id": "sec:method"}], + } + for section in ("研究課題", "データとタスク定義", "手法の骨子", "主要な結果", "深掘り分析", "限界") + ] + manifest = { + "coverage": {"total_pages": 10, "text_truncated": False}, + "sections": [{"section_id": "sec:method", "title": "Method", "page_start": 1, "page_end": 10}], + "pages": [], + } + assert validate_note_plan(plan, manifest, "ja") == [] + + +def test_japanese_figure_plan_uses_japanese_targets_and_reasons() -> None: + items = build_figure_items( + { + "figure_captions": [{"id": "Figure 1", "caption": "Overview of the system architecture."}], + "table_captions": [{"id": "Table 1", "caption": "Main results on three benchmarks."}], + }, + language="ja", + ) + by_id = {item["id"]: item for item in items} + assert by_id["Figure 1"]["section"] == "機構フロー" + assert by_id["Table 1"]["section"] == "主要な結果" + dumped = json.dumps(items, ensure_ascii=False) + assert "この" in dumped + for leftover in ("方法主线", "关键结果", "深度分析", "机制流程"): + assert leftover not in dumped + + +def test_japanese_note_passes_every_lint_gate_from_user_configuration( + tmp_path: Path, configured_user_home: Path +) -> None: + configured_user_home.write_text( + json.dumps({"output_language": "ja", "save_mode": "workspace"}), + encoding="utf-8", + ) + payload = run_lint(tmp_path, japanese_note()) + assert payload["output_language"] == "ja" + assert payload["warnings"] == [] + assert all(value is True for key, value in payload.items() if key.startswith("passes_")) + + +@pytest.mark.parametrize( + ("original", "invalid"), + [ + ("> 推奨位置：", "> 建议位置："), + ("> 現在の状態：", "> Current status:"), + ("### 機構フロー", "### 机制流程"), + ("- タイトル:", "- 标题:"), + ], +) +def test_japanese_lint_requires_exact_labels_and_mechanism_heading( + tmp_path: Path, original: str, invalid: str +) -> None: + payload = run_lint(tmp_path, japanese_note().replace(original, invalid), "--language", "ja") + all_gates_pass = all(value is True for key, value in payload.items() if key.startswith("passes_")) + assert payload["warnings"] or not all_gates_pass + + +@pytest.mark.parametrize( + "leftover_line", + [ + "### 机制流程", + "> 建议位置：手法の骨子", + "这张图解释方法内部机制。", + ], +) +def test_japanese_lint_rejects_simplified_chinese_leftovers(tmp_path: Path, leftover_line: str) -> None: + note = japanese_note().replace( + "重要な貢献はスコアの向上だけではない。", + f"重要な貢献はスコアの向上だけではない。\n\n{leftover_line}", + ) + payload = run_lint(tmp_path, note, "--language", "ja") + assert payload["passes_style_gate"] is False + assert "mixed_language_lines_present" in payload["warnings"] + assert any( + issue["reason"] == "simplified_chinese_text_present" for issue in payload["mixed_language_issues"] + ) + + +def test_japanese_lint_keeps_shared_kanji_and_source_metadata(tmp_path: Path) -> None: + # 日中で共有される漢字（数・与・会・後 など）と、参考文献の原語メタデータは許容する + note = japanese_note().replace( + "- Smith et al. (2024). Auditable Tool Use for Multi-hop Question Answering. DOI: 10.1234/example", + "- Smith et al. (2024). Auditable Tool Use for Multi-hop Question Answering. DOI: 10.1234/example\n" + "- 张伟 (2023). `可审计的工具使用`. DOI: 10.1234/example2", + ).replace( + "状態記録の設計は、", + "数値と証拠を与える設計は会議後にも再利用でき、状態記録の設計は、", + ) + payload = run_lint(tmp_path, note, "--language", "ja") + assert payload["passes_style_gate"] is True + + +def test_japanese_lint_requires_complete_top_level_heading_order(tmp_path: Path) -> None: + note = ( + japanese_note() + .replace("## 手法の骨子", "## TEMP") + .replace("## 主要な結果", "## 手法の骨子") + .replace("## TEMP", "## 主要な結果") + ) + payload = run_lint(tmp_path, note, "--language", "ja") + assert payload["passes_basic_structure"] is False + assert "top_level_section_profile_invalid" in payload["warnings"] + + +def test_japanese_lint_flags_english_prose_mixed_into_japanese(tmp_path: Path) -> None: + note = japanese_note().replace( + "重要な貢献はスコアの向上だけではない。", + "重要な貢献は the improvement of the score だけではない and it is also about the auditability。", + ) + payload = run_lint(tmp_path, note, "--language", "ja") + assert payload["passes_style_gate"] is False + assert "mixed_language_lines_present" in payload["warnings"] + + +def test_japanese_lint_rejects_generic_japanese_filler(tmp_path: Path) -> None: + note = japanese_note().replace( + "- 証拠選択とツール状態の追跡を一つの実行記録に統合し、失敗した証拠が黙って信頼できる入力に変わることを防ぐ。", + "- 本論文は新しい手法を提案する。", + ).replace( + "- 証拠の欠落と推論の誤りを区別する明示的なロールバック状態を追加し、最終回答の追跡を可能にする。", + "- 新規性が高い。", + ) + payload = run_lint(tmp_path, note, "--language", "ja") + assert not all(value is True for key, value in payload.items() if key.startswith("passes_")) + + +def test_run_pipeline_accepts_ja_language_override() -> None: + script = Path(__file__).resolve().parents[1] / "skills/deeppapernote/scripts/run_pipeline.py" + result = subprocess.run( + [sys.executable, str(script), "--help"], + capture_output=True, + text=True, + encoding="utf-8", + check=True, + ) + assert "ja" in result.stdout + + +JAPANESE_DOMAIN_RULES = """ +domains: + - label: 医療・健康 + aliases: + - healthcare + - medical + specialized_folders: + - メンタルヘルス + keywords: + - clinical + - patient + methods: [] + - label: メンタルヘルス + route_to: 医療・健康 + aliases: + - mental health + keywords: + - depression + methods: [] +fallback_domains: + - label: 機械学習 + aliases: + - machine learning + keywords: + - deep learning + - neural network + methods: [] + - label: 未分類 + aliases: + - unclassified + keywords: [] + methods: [] +""".strip() + "\n" + + +def test_domain_rules_can_be_overridden_by_environment_path(tmp_path: Path, monkeypatch) -> None: + import common + + rules_path = tmp_path / "ja_domain_rules.yaml" + rules_path.write_text(JAPANESE_DOMAIN_RULES, encoding="utf-8") + monkeypatch.setenv("DEEPPAPERNOTE_DOMAIN_RULES", str(rules_path)) + + assert common.infer_domain_label("Deep learning for depression screening", "patient cohort") == "医療・健康" + assert common.infer_domain_label("A neural network for image denoising") == "機械学習" + + +def test_domain_rules_next_to_user_configuration_are_preferred_over_skill_default( + tmp_path: Path, monkeypatch, configured_user_home: Path +) -> None: + import common + + monkeypatch.delenv("DEEPPAPERNOTE_DOMAIN_RULES", raising=False) + (configured_user_home.parent / "domain_rules.yaml").write_text(JAPANESE_DOMAIN_RULES, encoding="utf-8") + + assert common.infer_domain_label("A neural network for image denoising") == "機械学習" + + +def test_domain_fallback_labels_follow_loaded_rules(tmp_path: Path, monkeypatch) -> None: + """ルールにヒットしない場合の既定ラベルも、読み込んだ規則の alias から解決する。""" + import common + + rules_path = tmp_path / "ja_domain_rules.yaml" + rules_path.write_text(JAPANESE_DOMAIN_RULES, encoding="utf-8") + monkeypatch.setenv("DEEPPAPERNOTE_DOMAIN_RULES", str(rules_path)) + + # ルールのキーワードには一切ヒットしないが論文タイプが AI_method と推定されるタイトル + assert common.infer_domain_label( + "A new encoder-decoder model with attention for translation", + "We propose a model architecture and train it end to end; ablations show gains.", + ) == "機械学習" + assert common.infer_domain_label( + "Randomized controlled trial of cognitive therapy", + "We enrolled participants with anxiety and measured symptom scores at follow-up.", + ) == "医療・健康" + # 何にも該当しない場合（人文系の論文タイプ） + assert common.infer_domain_label( + "An ethnographic study of eighteenth-century poetry circles", + "An interpretive account of literary sociability.", + ) == "未分類" + + +# --- 同梱の日本語ドメイン規則（フォルダ名の日本語化） --- + +SHIPPED_JA_DOMAIN_RULES = ( + Path(__file__).resolve().parents[1] / "skills/deeppapernote/references/domain_rules.ja.yaml" +) + + +@pytest.fixture +def no_user_domain_rules(monkeypatch, configured_user_home: Path) -> Path: + """ENV も ~/.deeppapernote/domain_rules.yaml も存在しない、素の状態を作る。""" + monkeypatch.delenv("DEEPPAPERNOTE_DOMAIN_RULES", raising=False) + user_rules = configured_user_home.parent / "domain_rules.yaml" + assert not user_rules.exists() + return user_rules + + +def test_shipped_japanese_domain_rules_cover_every_default_label() -> None: + """同梱の ja 規則は zh-CN 規則と同じ alias 集合を持ち、ラベルは全て日本語である。""" + import common + + zh = common._parse_domain_rules_yaml(common.DOMAIN_RULES_PATH.read_text(encoding="utf-8-sig")) + ja = common._parse_domain_rules_yaml(SHIPPED_JA_DOMAIN_RULES.read_text(encoding="utf-8-sig")) + assert common._normalize_domain_rules(ja) is not None + + def alias_set(rules: dict) -> set[str]: + return { + alias.lower() + for section in common.DOMAIN_SECTIONS + for rule in rules.get(section, []) + for alias in rule.get("aliases", []) + } + + # ja は zh-CN の alias を全て持ち、さらに unclassified を明示する + assert alias_set(zh) <= alias_set(ja) + assert "unclassified" in alias_set(ja) + + zh_labels = {rule["label"] for section in common.DOMAIN_SECTIONS for rule in zh[section]} + ja_labels = {rule["label"] for section in common.DOMAIN_SECTIONS for rule in ja[section]} + # 日中で表記が同じラベル（法律・教育・金融・生物医学）以外は、中国語のまま残っていないこと + shared = {"法律", "教育", "金融", "生物医学"} + assert (zh_labels & ja_labels) <= shared + assert {"医療・健康", "機械学習", "大規模言語モデル", "未分類"} <= ja_labels + + +def test_infer_domain_label_uses_shipped_japanese_rules_for_ja(no_user_domain_rules: Path) -> None: + import common + + assert common.infer_domain_label( + "Deep learning for depression screening", "patient cohort", output_language="ja" + ) == "医療・健康" + assert common.infer_domain_label( + "A neural network for image denoising", output_language="ja" + ) == "機械学習" + assert common.infer_domain_label( + "Diffusion Policy for Robot Manipulation", "robotic control", output_language="ja" + ) == "ロボティクス" + # 言語未指定・zh-CN では従来どおり中国語ラベル + assert common.infer_domain_label("A neural network for image denoising") == "机器学习" + assert common.infer_domain_label( + "A neural network for image denoising", output_language="zh-CN" + ) == "机器学习" + + +def test_resolve_domain_subdir_follows_config_output_language( + tmp_path: Path, no_user_domain_rules: Path +) -> None: + """write_obsidian_note.py が渡す config の output_language でフォルダ名の言語が決まる。""" + import common + + vault = tmp_path / "vault" + (vault / "Research" / "Papers").mkdir(parents=True) + base = {"save_mode": "obsidian", "obsidian_vault": str(vault), "papers_dir": "Research/Papers"} + + assert common.resolve_domain_subdir( + {**base, "output_language": "ja"}, title="A neural network for image denoising" + ) == "機械学習" + assert common.resolve_domain_subdir( + {**base, "output_language": "zh-CN"}, title="A neural network for image denoising" + ) == "机器学习" + + +def test_resolve_domain_subdir_ja_still_reuses_existing_vault_folder( + tmp_path: Path, no_user_domain_rules: Path +) -> None: + """Vault に既存フォルダがあれば、言語に関係なくそれを再利用する（既存挙動の維持）。""" + import common + + vault = tmp_path / "vault" + (vault / "Research" / "Papers" / "Machine Learning").mkdir(parents=True) + config = { + "save_mode": "obsidian", + "obsidian_vault": str(vault), + "papers_dir": "Research/Papers", + "output_language": "ja", + } + assert common.resolve_domain_subdir( + config, title="A neural network for image denoising" + ) == "Machine Learning" + + +def test_user_overrides_still_beat_shipped_japanese_rules( + tmp_path: Path, monkeypatch, no_user_domain_rules: Path +) -> None: + import common + + custom = JAPANESE_DOMAIN_RULES.replace("label: 機械学習", "label: ML研究") + rules_path = tmp_path / "custom.yaml" + rules_path.write_text(custom, encoding="utf-8") + monkeypatch.setenv("DEEPPAPERNOTE_DOMAIN_RULES", str(rules_path)) + assert common.infer_domain_label( + "A neural network for image denoising", output_language="ja" + ) == "ML研究" + + monkeypatch.delenv("DEEPPAPERNOTE_DOMAIN_RULES") + no_user_domain_rules.write_text(custom, encoding="utf-8") + assert common.infer_domain_label( + "A neural network for image denoising", output_language="ja" + ) == "ML研究" + + +def test_ja_fallback_labels_stay_japanese_when_shipped_rules_are_unreadable( + tmp_path: Path, monkeypatch, no_user_domain_rules: Path +) -> None: + """同梱 YAML が壊れていても、ja の最終フォールバックラベルが中国語に戻らない。""" + import common + + broken = tmp_path / "domain_rules.ja.yaml" + broken.write_text("domains: [\n", encoding="utf-8") + monkeypatch.setattr(common, "DOMAIN_RULES_PATH", tmp_path / "domain_rules.yaml") + + label = common.infer_domain_label( + "An ethnographic study of eighteenth-century poetry circles", + "An interpretive account of literary sociability.", + output_language="ja", + ) + assert label == "未分類" + assert common.infer_domain_label( + "A new encoder-decoder model with attention for translation", + "We propose a model architecture and train it end to end; ablations show gains.", + output_language="ja", + ) == "機械学習"