Skip to content

fix(ssm): batch housekeeper parameter deletions - #5469

Open
NickAnge wants to merge 1 commit into
github-aws-runners:mainfrom
NickAnge:fix/ssm-housekeeper-batch-delete
Open

NickAnge wants to merge 1 commit into
github-aws-runners:mainfrom
NickAnge:fix/ssm-housekeeper-batch-delete

Conversation

@NickAnge

@NickAnge NickAnge commented Sep 23, 2026

Copy link
Copy Markdown

The SSM housekeeper deletes stale runner tokens/JIT configurations individually, consuming one API request per parameter and contributing to throttling and cleanup backlogs. Delete up to 10 names per DeleteParameters request instead: 1,000 eligible parameters require 100 delete calls rather than 1,000, excluding retries.

Wait 350 ms before each batch to pace an invocation below the default three delete requests per second. Retain SDK retries and continue after exhausted batch failures, logging the affected names and any InvalidParameters returned by SSM. Account/Region quotas are shared, so simultaneous housekeepers can still throttle. Existing age filtering and dry-run behavior are preserved.

Both housekeeper IAM policies now grant ssm:DeleteParameters. Documentation covers cleanup behavior and the need to deploy the IAM update alongside the Lambda. This change is based on main and leaves listing behavior unchanged; page-by-page cleanup in #5464 and native expiration policies in #5265 are complementary work.

Related: #5243.

Tests added/updated (mocked SSM):

  • 1/10/11/25 candidates: batches never exceed 10; no skipped or duplicated names.
  • Age filtering across pages: fresh, cutoff-age, and incomplete entries are excluded.
  • Dry runs and empty selections: no delete requests.
  • Pacing: 350 ms before each batch.
  • Failed batches and invalid names: log failures and continue cleanup.
  • Listing failures and invalid settings: surface errors without deleting.

Validation:

  • All nine projects' test suites, lint, and formatting checks pass.
  • All five Lambda bundles build successfully; storage-provider production TypeScript checking passes.
  • Terraform formatting and git diff whitespace checks pass.
  • A broad tsc --noEmit check reports errors in untouched tests and Vitest configuration files.

No AWS deployment or live parameter deletion was performed.

@NickAnge
NickAnge marked this pull request as ready for review September 23, 2026 15:32
@NickAnge
NickAnge requested a review from a team as a code owner September 23, 2026 15:32
@NickAnge

Copy link
Copy Markdown
Author

I openend it for review but i think i this should be written on top of #5464. Once this is merged, i will rebase it

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant