fix(openfold3): match AF3 pTM/ipTM reduction and confidence bin centers - #7
Closed
letientai299 wants to merge 2 commits into
Closed
letientai299 wants to merge 2 commits into
letientai299 wants to merge 2 commits into
Conversation
…ned tokens) _tm_score_from_pae_logits averaged the expected pairwise TM term over token pairs; AF3 (SI 5.9.1, eqs. 17-18) and upstream OpenFold3 compute_ptm take the mean over scored tokens per aligned token and then the max over aligned tokens with a valid frame. Use that reduction, evaluate the TM term at the aligned- error bin midpoints, add an optional has_frame (interim ~is_atomized at the call site; ligand-only inputs report NaN) and record the mask convention as ptm_frame_mask in get_scores(). Adds closed-form and upstream-parity tests. Signed-off-by: ashanehsazzadeh-dev <282092868+ashanehsazzadeh-dev@users.noreply.github.com>
_compute_pae, _compute_plddt and _select_best_sample weighted bin probabilities with torch.linspace end points (0 ... 32 A; 0 ... 1) instead of the bin mid- points used by AF3 / upstream OpenFold3 (get_bin_centers, probs_to_expected_ error, compute_plddt). Exact conversions for 0.1.0 outputs: PAE_new = 63/64 * PAE_old + 0.25 A; pLDDT_new = 49/50 * pLDDT_old + 1 (0-100). Best-sample selection (argmax of mean pLDDT) is invariant under this increasing affine map. Adds closed-form and upstream-parity tests. Signed-off-by: ashanehsazzadeh-dev <282092868+ashanehsazzadeh-dev@users.noreply.github.com>
Collaborator
Author
|
Same commits as #6 ( cc @ashanehsazzadeh-dev @ducta3141 @NVIDIA-BioNeMo/bioir |
11 tasks
letientai299
pushed a commit
that referenced
this pull request
Sep 17, 2026
pTM and ipTM maximize over aligned tokens, but only tokens with valid frames are eligible. The previous `~is_atomized` approximation excluded valid ligand frames and included polymers with missing backbone atoms, producing incorrect scores. Generate `valid_frame_mask` in the confidence head alongside PAE logits for each predicted sample. Also reduce memory use by searching for neighbors only from token start atoms, changing the working set from `N_atom²` to `N_token × N_atom` without changing results. Return only the mask because frame coordinates are used internally only for the angle check. Refer Github PR: #7 GitOrigin-RevId: 98263e8d21e8c312c8c2cb8551ef339eb8475de1
Contributor
|
Closed, see MR to address this problem: 86edd26 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reopened from #6 after an accidental squash merge to
mainwasforce-reset. Review comments remain on #6.
Description
Two deviations in
bionemo_ir/pipeline/models/openfold3/postprocessor.py(line numbers atbbf5f9ec) from the AlphaFold 3 definitions (Abramson et al., Nature 2024, SI §5.9.1, eqs. 17–18) and from upstream OpenFold3 (openfold3/core/metrics/confidence.py):_tm_score_from_pae_logits(lines 262–282) returns the mean of the expected TM term over all token pairs (pTM) or over all inter-chain pairs (ipTM). AF3 and upstreamcompute_ptmtake, for each aligned token i, the mean over scored tokens j and then the maximum over i (restricted to tokens with a valid frame). The pair mean is a lower bound of that quantity, so 0.1.0 reports systematically low pTM/ipTM — not comparable with the usual ipTM thresholds or with0.8·ipTM + 0.2·pTM— while the PAE matrix itself is correct. The Boltz-2, Protenix and OpenFold2 paths in this repository already use row-mean-then-max with bin midpoints; the OpenFold3 post-processor was the exception._compute_pae(298),_compute_plddt(209) and_select_best_sample(188) weight bin probabilities withtorch.linspace(lo, hi, n_bins)end points instead of bin midpoints (upstreamget_bin_centers: 0.25 … 31.75 Å for PAE, 0.01 … 0.99 for pLDDT). For PAE and pLDDT this is an exact increasing affine map, so the pipeline's own best-sample selection (mean-pLDDT argmax) is unchanged by this PR.Changes (one PR, two commits, bisectable)
fix(openfold3): match AF3 pTM/ipTM reduction (row mean, max over aligned tokens)— reduction fix, TM term at bin midpoints, optionalhas_frameargument. At the call site the mask is derived from the existingis_atomizedfeature (polymer tokens are frame-eligible; ligand atoms / ions are scored as j but not aligned on). This equals upstream's mask for polymer residues and single-atom ligands and is a lower bound for multi-atom ligands (upstream'sget_token_frame_atomsis not ported here). If no token is eligible the score isnan(→Noneinget_scores()), like the existing single-chain ipTM case.get_scores()gains one optional key,ptm_frame_mask, recording which convention produced the values — happy to drop it if you prefer the scores schema unchanged.fix(openfold3): use bin midpoints for PAE and pLDDT expectations—_compute_pae,_compute_plddt,_select_best_sample. Shifts every PAE entry by ≤ 0.25 Å and every pLDDT / B-factor / ModelCIF QA value by ≤ 1; kept separate from commit 1 because it touches many outputs but changes no ranking.Conversions for existing 0.1.0 outputs:
PAE_new = 63/64 · PAE_0.1.0 + 0.25 Å;pLDDT_new = 49/50 · pLDDT_0.1.0 + 1(0–100 scale);max_paeceiling 32.0 → 31.75 Å; pTM / ipTM have no closed form — recompute from the PAE logits.Example (public PDB entries)
Same PAE logits through the 0.1.0 and the patched post-processing, next to upstream OpenFold3 run separately on the same inputs (one seed, mean over 5 diffusion samples, so the upstream column agrees within sample-to-sample scatter rather than exactly):
On both entries pLDDT and PAE change only by the affine maps above, and the pipeline's selected sample is the same before and after.
Tests
tests/pipeline/models/openfold3/test_openfold3_ptm.py(new; synthetic inputs): max-over-aligned-tokens for pTM (0.856 vs 0.037 with the pair mean) and ipTM (inter-chain rows only); TM term at bin midpoints with the AF3d0(incl. the N < 19 clip);has_framesemantics incl. the all-masked / ligand-only case;(B, S, N, N, 64)slicing; PAE = 0.25 + 0.5k Å and pLDDT = 2k + 1 for one-hot bins; best-sample argmax invariance; parity with upstreamcompute_ptm,probs_to_expected_errorandcompute_plddt(|Δ| < 1e-5; skipped when the3rdparty/openfold-3submodule is not checked out). With both commits: 15 pass, 1 skipped (an env-gated report helper); on currentmain13 of these fail.ruff checkandruff format --checkare clean. The tests were run on CPU here — please let CI confirm on GPU.Type of change
Checklist
git commit -s