docx: Complete Refresh of the Examples and How To's

base.yml

@@ -201,22 +201,6 @@ nav:
               - docs/steps/modifier-and-hooks/ensure.md
       - docs/tasks.md
       - docs/builds.md
-      - How-To Guides:
-          - docs/how-to/index.md
-          - Pipelines:
-              - docs/how-to/pipeline-guides/managing-pipeline-configs.md
-              - docs/how-to/pipeline-guides/common-pipeline.md
-              - docs/how-to/pipeline-guides/task-inputs-outputs.md
-              - docs/how-to/pipeline-guides/gated-pipelines.md
-              - docs/how-to/pipeline-guides/time-triggered-pipelines.md
-              - docs/how-to/pipeline-guides/manual-approval.md
-          - Git:
-              - docs/how-to/git-guides/basic.md
-              - docs/how-to/git-guides/multi-branch.md
-              - docs/how-to/git-guides/monorepo.md
-          - Container Images:
-              - docs/how-to/container-image-guides/build-push.md
-              - docs/how-to/container-image-guides/build-use.md
       - Operation:
           - docs/operation/index.md
           - docs/operation/metrics.md
@@ -250,20 +234,53 @@ nav:
           - docs/internals/garbage-collector.md
   - Examples:
       - examples/index.md
-      - examples/hello-world.md
-      - examples/serial-job.md
-      - examples/pipeline-vars.md
-      - examples/set-pipeline.md
-      - examples/inputs-outputs.md
-      - examples/time-triggered.md
-      - examples/git-triggered.md
-      - examples/manually-triggered.md
-      - examples/job-and-task-hooks.md
-      - examples/golang-lib.md
-      - examples/rails-app.md
-      - examples/java-app.md
-      - examples/nodejs-app.md
-      - examples/php-app.md
+      - Fundamentals:
+          - examples/fundamentals/index.md
+          - examples/fundamentals/hello-world.md
+          - examples/fundamentals/inputs-outputs.md
+          - examples/fundamentals/set-pipeline.md
+      - Pipeline Patterns:
+          - examples/pipeline-patterns/index.md
+          - examples/pipeline-patterns/common-pipeline.md
+          - examples/pipeline-patterns/managing-pipeline-configs.md
+          - examples/pipeline-patterns/task-inputs-outputs.md
+          - examples/pipeline-patterns/gated-pipelines.md
+          - examples/pipeline-patterns/manual-approval.md
+          - examples/pipeline-patterns/time-triggered.md
+          - examples/pipeline-patterns/serial-job.md
+          - examples/pipeline-patterns/pipeline-vars.md
+          - examples/pipeline-patterns/manually-triggered.md
+      - Git Workflows:
+          - examples/git-workflows/index.md
+          - examples/git-workflows/basic.md
+          - examples/git-workflows/multi-branch.md
+          - examples/git-workflows/monorepo.md
+          - examples/git-workflows/git-triggered.md
+      - Container Images:
+          - examples/container-images/index.md
+          - examples/container-images/build-push.md
+          - examples/container-images/build-use.md
+      - Task Composition:
+          - examples/task-composition/index.md
+          - examples/task-composition/job-and-task-hooks.md
+      - Language Pipelines:
+          - examples/language-pipelines/index.md
+          - examples/language-pipelines/golang-lib.md
+          - examples/language-pipelines/rails-app.md
+          - examples/language-pipelines/java-app.md
+          - examples/language-pipelines/nodejs-app.md
+          - examples/language-pipelines/php-app.md
+      - Credentials:
+          - examples/credentials/index.md
+          - examples/credentials/idtoken-vault.md
+          - examples/credentials/idtoken-aws.md
+          - examples/credentials/idtoken-azure.md
+          - examples/credentials/idtoken-gcp.md
+      - Templating:
+          - examples/templating/index.md
+          - examples/templating/ytt-single-pipeline.md
+          - examples/templating/ytt-multi-file-pipeline.md
+
   - project/index.md
   - ecosystem/index.md
   - support/index.md

docs/blog/posts/2020/06/2020-06-19-how-to-build-and-publish-a-container-image.md

@@ -43,7 +43,7 @@ Now we can start building out our pipeline. Let's declare our [resources](../../
 first. We will need one resource to pull in the repo where our Dockerfile is located, and a second resource pointing to
 where we want to push the built container image to.
 
-_There are some [variables](../../../../examples/pipeline-vars.md#variables) in this file that we will
+_There are some [variables](../../../../examples/pipeline-patterns/pipeline-vars.md#variables) in this file that we will
 fill out later._
 
 ```yaml

docs/docs/getting-started/resources.md

@@ -562,10 +562,10 @@ at [this page](../../resource-types-list.md).
 This brings us to the end of the tutorial. You should have a basic understanding about how to read Concourse pipelines
 and start creating your own. Here are some other parts of the site to help you take off with Concourse:
 
-* [How-To Guides](../how-to/index.md) - Contains practical guides
-  for [working with pipelines](../how-to/pipeline-guides/common-pipeline.md) and examples of common pipeline workflows,
-  such as [git](../how-to/git-guides/basic.md)
-  and [container](../how-to/container-image-guides/build-push.md) workflows.
+* [How-To Guides](../../examples/index.md) - Contains practical guides
+  for [working with pipelines](../../examples/pipeline-patterns/common-pipeline.md) and examples of common pipeline workflows,
+  such as [git](../../examples/git-workflows/basic.md)
+  and [container](../../examples/container-images/build-push.md) workflows.
 * Check out all the reference documentation:
     * [Jobs](../jobs.md)
     * [Tasks](../tasks.md)

docs/docs/operation/creds/id-token.md

@@ -172,236 +172,11 @@ while keeping your Concourse instance private.
     out of sync with Concourse's JWKS, token verification will fail. The recommended approach is to use a reverse proxy 
     that forwards requests to your private Concourse in real-time, eliminating sync delays during key rotation.
 
-## Examples
+## Guides
 
-### Vault
+For end-to-end setup with a specific provider, see:
 
-You can use JWTs to authenticate
-with [HashiCorp Vault](https://developer.hashicorp.com/vault/docs/auth/jwt#jwt-authentication). This way your pipelines
-can directly communicate with Vault and use all of its features, beyond what Concourse's native Vault-integration
-offers.
-
-First enable the JWT auth method in your Vault Server:
-
-```shell
-vault auth enable jwt
-```
-
-Now configure the JWT auth method to accept JWTs issued by your Concourse (use your `--oidc-issuer-url` if configured,
-otherwise your external URL - see [Configuring a Separate OIDC Issuer](#configuring-a-separate-oidc-issuer)):
-
-```shell
-vault write auth/jwt/config \
-  oidc_discovery_url="https://<external_url_or_oidc_issuer_url>" \
-  default_role="demo"
-```
-
-Lastly, configure a role for JWT auth. Make sure to use the same value in your pipeline that you used for
-_bound_audiences_ (the best would be the URL of your Vault). _bound_subject_ must be the sub-claim value of your JWT, if
-you use the _subject_scope_ setting to change the contents of your sub-claim, adapt this accordingly!
-
-```shell
-vault write auth/jwt/role/demo \
-  role_type="jwt"\
-  user_claim="sub" \
-  bound_subject="main/your-pipeline" \
-  bound_audiences="my-vault-server.com" \
-  policies=webapps \
-  ttl=1h
-```
-
-This role will allow the holder of a JWT with aud: "`my-vault-server.com`" and sub: "`main/your-pipeline`" to get a
-Vault token with the Vault-policy `webapps`. If the policy you want to assign has a different name, simply change it in
-the above example. Make sure to adapt the value for `bound_subject` according to your team and pipeline name.
-
-Pipelines can now do the following:
-
-```yaml
-var_sources:
-  - name: vaulttoken
-    type: idtoken
-    config:
-      audience: [ "my-vault-server.com" ]
-
-jobs:
-  - name: vault-login
-    plan:
-      - task: login
-        config:
-          platform: linux
-          image_resource:
-            type: registry-image
-            source: { repository: hashicorp/vault }
-          run:
-            path: sh
-            args:
-              - -e
-              - -c
-              - |
-                export VAULT_ADDR=https://my-vault-server.com
-                vault write auth/jwt/login \
-                  role=demo \
-                  jwt=((vaulttoken:token)) \
-                  --format=json > vault-response.json
-                echo "Now do something with the token in vault-response.json"
-```
-
-You don't have to create a role and a policy for every single of your pipelines! You can use claims from the JWT with
-Vault's [policy templating](https://developer.hashicorp.com/vault/tutorials/policies/policy-templating) feature. This
-way you can define a policy that allows a pipeline read to all the secrets it would usually have access to using
-Concourse's native Vault-integration:
-
-```hcl
-path "concourse/metadata/{{ identity.entity.aliases.<JWT_ACCESSOR>.metadata.team }}" {
-  capabilities = ["list"]
-}
-
-path "concourse/data/{{ identity.entity.aliases.<JWT_ACCESSOR>.metadata.team }}/+" {
-  capabilities = ["read"]
-}
-
-path "concourse/metadata/{{ identity.entity.aliases.<JWT_ACCESSOR>.metadata.team }}/{{ identity.entity.aliases.<JWT_ACCESSOR>.metadata.pipeline }}" {
-  capabilities = ["list"]
-}
-
-path "concourse/metadata/{{ identity.entity.aliases.<JWT_ACCESSOR>.metadata.team }}/{{ identity.entity.aliases.<JWT_ACCESSOR>.metadata.pipeline }}/*" {
-  capabilities = ["read", "list"]
-}
-
-path "concourse/data/{{ identity.entity.aliases.<JWT_ACCESSOR>.metadata.team }}/{{ identity.entity.aliases.<JWT_ACCESSOR>.metadata.pipeline }}/*" {
-  capabilities = ["read", "list"]
-}
-```
-
-!!! note
-
-    Make sure to set `<JWT_ACCESSOR>` to the actual mount-accessor value of your JWT Auth method! You can use `vault 
-    auth list --format=json | jq -r '."jwt/".accessor'` to get the accessor for your jwt auth method.
-
-With a policy like this you don't need to configure `bound_subject` in your JWT auth role. Every single pipeline can
-simply use the same role and the policy will take care that they can only access secrets meant for them. However, you
-need to explicitly configure claim to metadata mapping:
-
-```shell
-vault write auth/jwt/role/demo \
-  role_type="jwt"\
-  user_claim="sub" \
-  bound_subject= \
-  bound_audiences="my-vault-server.com" \
-  policies=pipeline-new \
-  claim_mappings='team=team' \
-  claim_mappings='pipeline=pipeline' \
-  ttl=1h
-```
-
-### AWS
-
-AWS
-supports [federation with external identity providers](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers.html).
-Using this, you can allow identities managed by an external identity provider to perform actions in your AWS account.
-
-In this scenario the external identity provider is Concourse and the identities are teams/pipelines/jobs. This means you
-are able to grant a specific pipeline or job permission to perform actions in AWS (like deploying something), all
-without managing IAM users or dealing with long-lived credentials.
-
-First you need
-to [create an OpenID Connect identity provider](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_oidc.html)
-in your AWS Account. Set _Provider URL_ to the external URL of your Concourse server  (or the `--oidc-issuer-url` if
-you're using a separate OIDC issuer - see [Configuring a Separate OIDC Issuer](#configuring-a-separate-oidc-issuer)).
-For _Audience_, you can choose any string you like, but using a value like `sts.amazonaws.com` is recommended. You have
-to use the same string later in the configuration of your [`idtoken` var source](../../vars.md#id-token).
-
-Next you will need
-to [create an IAM-Role that can be assumed using your JWT](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-idp_oidc.html#idp_oidc_Create).
-Set _Identity Provider_ to the value you previously set _Audience_ to. Add a condition on the sub-claim with
-type `StringEquals` and value `yourteam/yourpipeline`. This will allow ONLY that specific pipeline (and any instanced
-versions of it) to assume that IAM Role using a JWT. If you use the `subject_scope` setting to change the contents of
-your sub-claim, adapt this condition accordingly! In the next step you will be able to choose which AWS permissions your
-role will get.
-
-Now you can use
-the [AWS AssumeRoleWithWebIdentity API operation](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRoleWithWebIdentity.html)
-to assume your role via a JWT issued by Concourse. The easiest way is to do this is via
-the [assume-role-with-web-identity AWS CLI command](https://docs.aws.amazon.com/cli/latest/reference/sts/assume-role-with-web-identity.html):
-
-```yaml
-var_sources:
-  - name: awstoken
-    type: idtoken
-    config:
-      audience: [ "sts.amazonaws.com" ]
-
-jobs:
-  - name: aws-login
-    plan:
-      - task: print
-        config:
-          platform: linux
-          image_resource:
-            type: registry-image
-            source: { repository: amazon/aws-cli }
-          run:
-            path: bash
-            args:
-              - -e
-              - -c
-              - |
-                aws sts assume-role-with-web-identity \
-                  --role-session-name Concourse \
-                  --role-arn arn:aws:iam::<your_account>:role/<your_role> \
-                  --web-identity-token ((awstoken:token)) > creds.json
-                echo "Now do something with the temporary credentials in creds.json"
-```
-
-### Azure
-
-Azure also supports a way to grant the holder of a JWT permissions in the Cloud. This is done via a feature
-called [Federated Credentials](https://learn.microsoft.com/en-us/graph/api/resources/federatedidentitycredentials-overview?view=graph-rest-1.0).
-
-First, [create an EntraID App Registration](https://learn.microsoft.com/en-us/entra/identity-platform/quickstart-register-app).
-This app registration will be the service principal used by your pipeline.
-
-Now [create a federated credential](https://learn.microsoft.com/en-us/entra/workload-id/workload-identity-federation-create-trust?pivots=identity-wif-apps-methods-azp#other-identity-providers)
-for the app registration you just created.
-
-For _Scenario_ select "Other". For Issuer set it to the external URL of your Concourse server (or the
-`--oidc-issuer-url` if you're using a separate OIDC issuer -
-see [Configuring a Separate OIDC Issuer](#configuring-a-separate-oidc-issuer)).
-For _Type_ select "Explicit subject identifier" and set _Value_ to `<teamname>/<pipelinename>` of the pipeline that
-should be able to use the identity. If you use the `subject_scope` setting to change the contents of your sub-claim,
-change this setting here accordingly.
-
-You can now assign IAM permissions to the identity of the app registration, which define what the identity is allowed to
-do in your Azure subscription.
-
-Your pipeline can now use the `az cli` to log in to Azure using a JWT generated by Concourse:
-
-```yaml
-var_sources:
-  - name: azuretoken
-    type: idtoken
-    config:
-      audience: [ "api://AzureADTokenExchange" ]
-
-jobs:
-  - name: azure-deploy
-    plan:
-      - task: login
-        config:
-          platform: linux
-          image_resource:
-            type: registry-image
-            source: { repository: mcr.microsoft.com/azure-cli }
-          run:
-            path: bash
-            args:
-              - -e
-              - -c
-              - |
-                echo ((azuretoken:token))
-                az login --service-principal \
-                  -u <client_id of your app registration> \
-                  --tenant <tenant_id of your app registration> \
-                  --federated-token ((azuretoken:token))
-                echo "You are now authenticated with Azure. Do something with it!"
-```
+- [Authenticating to Vault via IDToken](../../../examples/credentials/idtoken-vault.md)
+- [Authenticating to AWS via IDToken](../../../examples/credentials/idtoken-aws.md)
+- [Authenticating to Azure via IDToken](../../../examples/credentials/idtoken-azure.md)
+- [Authenticating to GCP via IDToken](../../../examples/credentials/idtoken-gcp.md)

docs/docs/steps/modifier-and-hooks/across.md

@@ -303,7 +303,7 @@ See [Across Step & Dynamic Vars](../../vars.md#across-step-dynamic-vars) for det
     When a new branch is added, a new pipeline will be created. When a branch is deleted, the pipeline will be 
     automatically archived as described in the [`set_pipeline` step](../set-pipeline.md).
 
-    For a more complete example, refer to [Multi-Branch Workflows](../../how-to/git-guides/multi-branch.md).
+    For a more complete example, refer to [Multi-Branch Workflows](../../../examples/git-workflows/multi-branch.md).
 
 ## Limitations
 

docs/docs/steps/task.md

@@ -143,7 +143,7 @@ the result of a task.
 
         This can be used to explicitly keep track of dependent images. You could also modify it to build and push the 
         image in one job and use it in later jobs. See [Building and Pushing an 
-        Image](../how-to/container-image-guides/build-push.md).
+        Image](../../examples/container-images/build-push.md).
 
         ```yaml
         resources:
@@ -169,7 +169,7 @@ the result of a task.
 
     ??? example "Building and using an image"
 
-        [Building an Image and Using it in a Task](../how-to/container-image-guides/build-use.md)
+        [Building an Image and Using it in a Task](../../examples/container-images/build-use.md)
 
 
 ??? info "`privileged`: [`boolean`](../config-basics.md#boolean-schema)"