Is your feature request related to a problem? Please describe.
Pluto currently appears to pull all API resource types from a cluster, followed by all objects matching those types, in an unpaginated list. In a large cluster with a significant number of resources, many of them unrelated to the rules/deprecations being checked, this leads to high memory usage.
Describe the solution you'd like
Short version: only pull the resources that appear in the ruleset we loaded, so we don't use as much memory.
Long version / suggestion:
- After loading the ruleset (built-in deprecations and custom/user-defined), build a list of resource types we need to check
- Only pull the resources that actually need to be checked (this can avoid loading thousands of events, for example)
- Paginate discovery of the remaining resource types, discarding ones we don't need before fetching the next set
Describe alternatives you've considered
- Smaller clusters with fewer resources: it's currently not feasible for me to reduce the number of resources in my cluster.
- Give pluto more resources: I've more than tripled the resources I gave to pluto to run against a cluster, I can technically keep going but improving the collection logic is more efficient
Additional context
N/A
Is your feature request related to a problem? Please describe.
Pluto currently appears to pull all API resource types from a cluster, followed by all objects matching those types, in an unpaginated list. In a large cluster with a significant number of resources, many of them unrelated to the rules/deprecations being checked, this leads to high memory usage.
Describe the solution you'd like
Short version: only pull the resources that appear in the ruleset we loaded, so we don't use as much memory.
Long version / suggestion:
Describe alternatives you've considered
Additional context
N/A