feat(grpo): add Binary-TV DPPO training support

[guyueh1]

Summary

• add Binary-TV DPPO as a ClippedPGLossFn formula variant selected by loss_fn.loss_mode=dppo_binary_tv
• add DPPO loss metrics and focused unit coverage for divergence masking and metric normalization
• add a GB200 nightly smoke recipe/test using the Qwen2.5-Math 1.5B GRPO recipe shape with DPPO defaults

Reference:

• https://arxiv.org/pdf/2602.04879 Qi et al, Rethinking the Trust Region in LLM Reinforcement Learning

Testing

• python3 -c "import ast, pathlib; files=[\"nemo_rl/algorithms/grpo.py\",\"nemo_rl/algorithms/loss/__init__.py\",\"nemo_rl/algorithms/loss/loss_functions.py\",\"tests/unit/algorithms/test_loss_functions.py\"]; [ast.parse(pathlib.Path(f).read_text(), filename=f) for f in files]; print(\"ast-ok\")"
• PYTHONPYCACHEPREFIX=/private/tmp/codex-pyc-nemo-rl python3 -m py_compile nemo_rl/algorithms/grpo.py nemo_rl/algorithms/loss/__init__.py nemo_rl/algorithms/loss/loss_functions.py tests/unit/algorithms/test_loss_functions.py
• git diff --check
• bash -n tests/test_suites/llm/grpo-dppo-binary-tv-qwen2.5-math-1.5b-instruct-1n4g-fsdp2tp1.sh
• TEST_DRYRUN=1 tests/test_suites/llm/grpo-dppo-binary-tv-qwen2.5-math-1.5b-instruct-1n4g-fsdp2tp1.sh
• suite accounting script: suite-accounting-ok entries=284 recipes=284

Note: full pytest was not run locally because this shell does not have pytest, uv, or torch on its default PATH/environment.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[guyueh1]

@NVIDIA-NeMo/rl_reviewers_algo please help review this PR, this algorithm can help make TMPE stable and avoid entropy collapse and is part of our low-precision RL experiment.