CondonFM Skills

[ohadmo]

This repository is read-only and does not accept Pull Requests. Please open an Issue for reproducible bugs.

[github-actions]

Thanks for your interest. This repository is read-only and does not accept Pull Requests. Please open an Issue for reproducible bugs.

[github-actions]

Thanks for your interest. This repository is read-only and does not accept Pull Requests. Please open an Issue for reproducible bugs.

[ohadmo]

/nvskills-ci

[ohadmo]

@greptileai

[greptile-apps]

Confidence Score: 5/5

The changes since the previous review appear safe to merge, with no actionable new defect identified.

Summary

Adds generated publication artifacts for four CodonFM skills:

• Records Tier 1–3 benchmark results and publication recommendations.
• Adds skill cards describing ownership, use cases, dependencies, outputs, and evaluation results.
• Attaches Sigstore bundles covering each packaged skill’s resources.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[CodonFM skill source and evaluations] --> B[Tier 1-3 evaluation]
  B --> C[BENCHMARK.md]
  B --> D[skill-card.md]
  C --> E[Packaged skill resources]
  D --> E
  E --> F[skill.oms.sig]
  F --> G[NVSkills publication validation]

Loading

_{Reviews (8) · Last reviewed commit: "Attach NVSkills validation signatures"}

[ohadmo]

/nvskills-ci

[ohadmo]

/nvskills-ci

[ohadmo]

@greptileai

[ohadmo]

/nvskills-ci

[ohadmo]

/nvskills-ci

[ohadmo]

/nvskills-ci

[chrisknvidia]

@ohadmo : I did some debugging of the failures,
Suggestion for the resource-intensive Tier 3 evaluations:
Please add the following to skills/codonfm-score/evals/config.yml

schema_version: 1

harbor:
  timeout_multiplier: 4
  sandbox:
    templates:
      claude-code: harbor-eval-claude-code-8g
      codex: harbor-eval-codex-8g

• timeout_multiplier: 4 applies to both agents.
• Each template applies to both the with-skill and baseline arm for that agent.
• The Claude mapping is especially relevant because the latest remaining failure occurred in the Claude baseline.
• This provides equal resource headroom, but a rerun is still required.

[ohadmo]

/nvskills-ci

[ohadmo]

/nvskills-ci

[chrisknvidia]

@ohadmo : I tested the two previously failing codonfm-score cases end to end using Claude Opus 5 through NVCARPS → SkillEvaluator → Harbor → Astra.

• Resolved model: aws/anthropic/bedrock-claude-opus-5
• Coverage: valid_full
• Claude results: 4/4 scored, with zero failed attempts or policy refusals
• Attempts 2–3 were correctly skipped because every case passed on attempt 1
• Test MR !94
• Successful Tier 3 job

https://github.com/NVIDIA-BioNeMo/CodonFM/blob/3f95e4e01604dbe7fda83d5563ef7a55d3134772/skills/codonfm-score/evals/evals.json

Could you please add the following per-skill override to skills/codonfm-score/evals/config.yml and rerun the complete PR evaluation?

schema_version: 1

harbor:
  agents:
    claude-code:
      model: aws/anthropic/bedrock-claude-opus-5

This changes only the Claude model for codonfm-score; the Codex configuration remains unchanged. The controlled test confirms that Opus 5 avoids the refusals for the two previously failing cases. The complete PR rerun will confirm it across the full dataset.

[ohadmo]

/nvskills-ci

[ohadmo]

/nvskills-ci

[ohadmo]

/nvskills-ci