Skip to content

ci(monitoring): scheduled protocol drift report - #17

Draft
92Infinitus92 wants to merge 5 commits into
feat/scenarios/raw-layoutfrom
feat/scenarios/monitoring
Draft

92Infinitus92 wants to merge 5 commits into
feat/scenarios/raw-layoutfrom
feat/scenarios/monitoring

Conversation

@92Infinitus92

@92Infinitus92 92Infinitus92 commented Sep 19, 2026

Copy link
Copy Markdown
Collaborator

Milestone 3.3. Twice a day a workflow checks whether the protocols we integrate still look the way our templates assume, and commits a Markdown report to the scenarios/monitoring branch under scenarios/monitoring/reports/<timestamp>/. Published IDLs that differ from the committed ones land next to it under idls/, together with the diff.

Two files: a bash script that calls the tools a person would use, and the workflow that runs it.

What a run does

  1. Restores the previous observation from the Actions cache.
  2. Runs the integration suites with cargo nextest run --features integration-tests --no-fail-fast.
  3. monitor.sh checks three things against mainnet:
    • Program upgrades: solana program show for every program a template names (address in idl.json) plus the PMM programs listed at the top of the script. A changed deploy slot or upgrade authority since the previous run is drift.
    • IDLs: anchor idl fetch (legacy ones go through anchor idl convert), then jq compares the accounts, types and, where we commit them, instructions. Any entry we ship that the program no longer publishes identically is drift; entries only upstream has are listed as additions.
    • Markets: every address a template offers as a choice, checked with getSignaturesForAddress. No transaction ever is drift, none in three days is a warning.
  4. Writes the report and a one-word verdict. The report shows the nextest summary and failing test names only, never tool output, so an RPC key can not end up in the branch.
  5. A separate job with write access publishes the report and the run fails last, when the verdict is not clean or a suite failed.

What it shows today

14 of 17 committed IDLs differ from what the programs publish, listed by entry (for Kamino: types/Obligation, types/LendingMarket, types/ReserveConfig, ...; for Jupiter: accounts/TokenLedger and its instructions). 30 offered addresses have not seen a transaction in three days. All 21 programs are on their first observation until the cache holds a run.

Verified locally

Three consecutive runs against the public endpoint: first observation, then all 21 programs unchanged, then a corrupted slot in the snapshot reported as changed with old and new values. 111 markets checked without RPC errors. A real nextest log renders into the tests section. The report contains no RPC URL.

Not verified yet: the run on GitHub itself. One workflow_dispatch on this branch settles it. SURFPOOL_TEST_RPC_URL is optional; without it the public endpoint is used and Pump's surfnet-backed tests time out.

RetriggerConfidence Score: 5/5

The PR appears safe to merge; no outstanding previous findings or actionable new regressions remain.

Summary

Adds scheduled protocol monitoring and report publication, with the latest revision expanding IDL drift detection to compare complete committed account, type, and instruction entries.

  • Runs integration suites and mainnet program, IDL, and market checks twice daily.
  • Preserves observations through an Actions cache and publishes timestamped reports to a dedicated branch.
  • Normalizes legacy IDL account flags and reports changed entries by section and name.
  • Keeps third-party monitoring actions in a read-only job while isolating repository writes in the publication job.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart TD
  S[Scheduled or manual run] --> M[Read-only monitor job]
  M --> C[Restore previous snapshot]
  C --> T[Run integration suites]
  T --> D[Check programs, IDLs, and markets]
  D --> R[Render report and verdict]
  R --> A[Upload report artifact]
  D --> N[Save next snapshot]
  A --> P[Write-enabled publish job]
  P --> B[Commit report to scenarios/monitoring branch]
  B --> F{Clean verdict and suites passed?}
  F -->|Yes| OK[Workflow succeeds]
  F -->|No| FAIL[Workflow fails after publication]
Loading

Reviews (8) · Last reviewed commit: "ci(monitoring): compare IDL entries stru..."

Comment thread crates/core/src/tests/monitoring/mod.rs Outdated
Comment thread .github/workflows/protocol_monitoring.yml Outdated
Comment thread crates/core/src/tests/monitoring/idl_document.rs Outdated
Comment thread .github/workflows/protocol_monitoring.yml Outdated
Comment thread .github/scripts/render_monitoring_report.py Outdated
@92Infinitus92
92Infinitus92 force-pushed the feat/scenarios/monitoring branch from a5417bf to 96d21b2 Compare September 21, 2026 10:13
@92Infinitus92 92Infinitus92 changed the title feat(monitoring): add scheduled protocol drift monitoring ci(monitoring): scheduled protocol drift report from the workflow Sep 21, 2026
Comment thread .github/scripts/monitoring/monitor.mjs Outdated
Comment thread .github/scripts/monitoring/monitor.mjs Outdated
Comment thread .github/scripts/monitoring/monitor.mjs Outdated
@failfmi failfmi closed this Sep 21, 2026
@92Infinitus92 92Infinitus92 reopened this Sep 21, 2026
@92Infinitus92
92Infinitus92 marked this pull request as draft September 21, 2026 11:41
Comment thread .github/scripts/monitor.sh
@92Infinitus92 92Infinitus92 changed the title ci(monitoring): scheduled protocol drift report from the workflow ci(monitoring): scheduled protocol drift report Sep 21, 2026
@92Infinitus92

Copy link
Copy Markdown
Collaborator Author

@greptileai

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants