Route generation module installs through the private CFS feed (PS Gallery follow-up) [draft] - #3720
Draft
Gavin Barron (gavinbarron) wants to merge 9 commits into
Draft
Route generation module installs through the private CFS feed (PS Gallery follow-up) [draft]#3720Gavin Barron (gavinbarron) wants to merge 9 commits into
Gavin Barron (gavinbarron) wants to merge 9 commits into
Conversation
…hell Gallery follow-up) Phase 2 of the PowerShell CFSClean remediation: eliminate the CFSClean2-tier egress to www.powershellgallery.com / cdn.powershellgallery.com in the generation pipelines (187/221/663) by routing every generation-time Install-Module/Find-Module through the private Azure Artifacts feed PowerShell_V2_Build, which already has a PowerShell Gallery upstream (so it serves both the internally published Graph modules and public tooling modules). Mechanism: - Expose $(System.AccessToken) as a process-wide SYSTEM_ACCESSTOKEN variable in weekly-generation.yml, command-metadata-refresh.yml and ci-build.yml, so credentials are available to every step and to the ForEach-Object -Parallel runspaces used by GenerateModules.ps1. - install-tools.yml: new "Register private module feed" step registers PowerShell_V2_Build as a Trusted PSRepository (persisted for the job). - tools/Get-CfsFeedCredential.ps1: shared helper (Get-CfsFeedName / Get-CfsFeedCredential / Register-CfsFeed) that builds a PSCredential from SYSTEM_ACCESSTOKEN; returns $null locally so behaviour is unchanged off-CI. - Repoint sources from PSGallery to PowerShell_V2_Build and inject the credential via $PSDefaultParameterValues (runspace-local, so it works inside the parallel generation): ValidateUpdatedModuleVersion, GenerateRollUpModule, GenerateMetaModule, GenerateAuthenticationModule, BuildModule, Versions/BumpModuleVersion, plus the ad-hoc tooling installs (PlatyPS, Pester, powershell-yaml, PowerHTML) in GenerateHelp/TestModule/ImportExamples/ UpdateOpenApi. DRAFT: requires CI validation. Open questions to confirm in a pipeline run: - ValidateUpdatedModuleVersion's version gate now queries the private feed (with PSGallery upstream) instead of public PSGallery directly; confirm the published-version comparison still behaves as intended (the upstream proxies public versions, but the feed may also expose internally published versions). - Confirm www.powershellgallery.com / cdn.powershellgallery.com drop to 0 in the generation jobs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d3f8fec7-b00b-46be-ba39-7e1f3e7f7188
The 'Register private module feed (CFSClean)' step failed with 'The name specified has already been added to the list of available package sources' because a PackageManagement source named PowerShell_V2_Build already exists in the build job (not surfaced by Get-PSRepository). The repository still ends up registered, so wrap Register-PSRepository in try/catch and treat the already-exists collision as success when Get-PSRepository confirms the repo is present. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 6ea00b50-c719-469b-bf9c-72120184bfe0
…lery-private-feed
The 'already added' error is non-terminating and bypasses -ErrorAction Stop (so try/catch never fires) yet fails the PowerShell task under its default Stop preference. Pre-check Get-PackageSource (surfaces the collision that Get-PSRepository lazily misses in a fresh session), and suppress all streams on Register-PSRepository with a post-verify instead of relying on the call to not error. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 6ea00b50-c719-469b-bf9c-72120184bfe0
…d step The step still exited 1 despite a successful registration because PackageManagement emits benign non-terminating errors that trip the PowerShell task's default Stop preference. Set the task errorActionPreference to continue; Register-CfsFeed still throws (terminating) on genuine failure. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 6ea00b50-c719-469b-bf9c-72120184bfe0
Registration succeeds (repo is created + printed) but the step exited 1 because PowerShellGet's NuGet-provider bootstrap leaves a non-zero \0 that the task checks. Reset it at the end of the step; Register-CfsFeed throws on genuine failure before this line. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 6ea00b50-c719-469b-bf9c-72120184bfe0
BuildModule.ps1 read the existing module GUID from Find-Module's AdditionalMetadata.GUID, which the private Azure Artifacts feed does not populate (public PS Gallery does), producing a null GUID and failing Update-ModuleManifest. Add Get-CfsModuleGuid: Save-Package the published nupkg from the feed (NuGet provider, no dependency resolution) and read GUID from its .psd1; fall back to a fresh GUID only when the module is unpublished. Keeps the GUID lock working without any public PS Gallery call. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 6ea00b50-c719-469b-bf9c-72120184bfe0
…vate feed Generation failed installing PlatyPS: 'multiple modules matched platyPS. Please specify a single -Repository' because PSGallery and the private feed were both registered. Unregister PSGallery in the register step so the private feed (which has a PS Gallery upstream) is the sole source - eliminating install ambiguity and any residual public PS Gallery egress. Persists per-user for later generation steps/runspaces. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 6ea00b50-c719-469b-bf9c-72120184bfe0
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Generation runs
Install-Module/Find-Moduleagainst the public PowerShell Gallery (www.powershellgallery.com+cdn.powershellgallery.com), which trips CFSClean2. Telemetry (e.g. 221 build 230408, 663 build 232382) shows these hits in every workload generation job.Approach
The private Azure Artifacts feed
PowerShell_V2_Buildalready has a PowerShell Gallery upstream (confirmed via the feed's upstream sources), so it can serve both the internally published Graph modules and the public tooling modules (Pester, PlatyPS, powershell-yaml, PowerHTML). Route all generation module operations through it.Changes
SYSTEM_ACCESSTOKEN: $(System.AccessToken)tovariables:inweekly-generation.yml,command-metadata-refresh.yml,ci-build.yml— so credentials are available to every step and to theForEach-Object -Parallelrunspaces inGenerateModules.ps1.install-tools.yml: new Register private module feed step registersPowerShell_V2_Buildas a TrustedPSRepository(persisted for the job).tools/Get-CfsFeedCredential.ps1(new): shared helper —Get-CfsFeedName,Get-CfsFeedCredential(builds aPSCredentialfromSYSTEM_ACCESSTOKEN; returns$nulllocally so off-CI behaviour is unchanged),Register-CfsFeed.PSGallery→PowerShell_V2_Build, injecting the credential via$PSDefaultParameterValues(runspace-local, so it works inside parallel generation):ValidateUpdatedModuleVersion,GenerateRollUpModule,GenerateMetaModule,GenerateAuthenticationModule,BuildModule,Versions/BumpModuleVersion, and the ad-hoc tooling installs inGenerateHelp/TestModule/ImportExamples/UpdateOpenApi.Validation (in CI)
www.powershellgallery.com/cdn.powershellgallery.com= 0 in the generation jobs (187/221/663).ValidateUpdatedModuleVersionnow queries the private feed (PSGallery upstream) rather than public PSGallery directly — confirm the published-version comparison still gates correctly (the upstream proxies public versions, but the feed may also surface internally published versions).Related
s360-cfs/pipeline-cfs-compliance-status.md.