Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 36 additions & 1 deletion .agents/skills/test-release-canary/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ The Release Canary (`.github/workflows/release-canary.yml`) smoke-tests the arti
|---|---|---|
| `macos` | `macos-latest-xlarge` | `install.sh` resolves the Homebrew formula, brew installs the cask, and `openshell status` reaches the brew-services–backed local gateway with the VM driver. |
| `ubuntu` | `ubuntu-latest` | `install.sh` installs the Debian package, the post-install systemd user service starts, and `openshell status` reaches the local gateway with the Docker driver. |
| `fedora` | `fedora:latest` container | `install.sh` installs the RPM packages, the local gateway starts under Podman, and `openshell status` succeeds. |
| `fedora` | `linux-amd64-cpu8` + Fedora Nix VM | `install.sh` installs the RPM packages, the root-owned local gateway starts with rootful Podman, and `openshell status` succeeds. |
| `kubernetes` | `ubuntu-latest` + kind | `helm install oci://ghcr.io/nvidia/openshell/helm-chart --version 0.0.0-dev` succeeds in a kind cluster, the gateway pod becomes Ready, port-forward exposes 8080, and the released CLI registers the in-cluster gateway and runs `openshell status` against it. |

All canary jobs disable anonymous OpenShell telemetry. Host package jobs inject
Expand Down Expand Up @@ -116,6 +116,41 @@ Swap `0.0.0-dev` for `0.0.0-dev.<sha>` to pin to a specific dev build. Tear down

Loopback registration auto-derives the gateway name to `openshell` if `--name` is omitted, which collides with the `install.sh`-installed local gateway — always pass `--name kind` (or another distinct name) when registering in addition to a local install.

## Local Fedora reproduction

The `fedora` job uses the repository's Nix test-guest harness instead of
running Fedora inside Docker. It can be reproduced on a Linux host with Nix,
KVM, and the repository checkout:

```shell
export INSTALL_SH_URL="https://raw.githubusercontent.com/NVIDIA/OpenShell/$(git rev-parse HEAD)/install.sh"
nix run .#test-guest -- \
--distro fedora \
--with podman-rootful \
-- \
sudo env \
SUDO_USER=root \
HOME=/root \
XDG_RUNTIME_DIR=/run/user/0 \
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/0/bus \
OPENSHELL_TELEMETRY_ENABLED=false \
INSTALL_SH_URL="$INSTALL_SH_URL" \
bash -s <<'EOF'
set -euo pipefail
mkdir -p "${XDG_RUNTIME_DIR}"
chmod 700 "${XDG_RUNTIME_DIR}"
systemctl start user-runtime-dir@0.service || true
systemctl start user@0.service
systemctl --user daemon-reload
mkdir -p "${HOME}/.config/openshell"
printf 'OPENSHELL_DRIVERS=podman\nOPENSHELL_PODMAN_SOCKET=/run/podman/podman.sock\nOPENSHELL_TELEMETRY_ENABLED=%s\n' \
"$OPENSHELL_TELEMETRY_ENABLED" > "${HOME}/.config/openshell/gateway.env"
podman --url unix:///run/podman/podman.sock info
curl -LsSf "${INSTALL_SH_URL}" | sh
openshell status
EOF
```

## Diagnosing failures

| Symptom | Likely cause | Where to look |
Expand Down
29 changes: 29 additions & 0 deletions .github/actions/setup-e2e-sandbox/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Setup E2E Sandbox
description: Download an architecture-matched prebuilt OpenShell sandbox binary for E2E tests

inputs:
artifact-prefix:
description: Artifact name prefix; linux-<arch> is appended automatically
required: true

runs:
using: composite
steps:
- name: Download prebuilt sandbox
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: ${{ format('{0}-linux-{1}', inputs.artifact-prefix, runner.arch == 'X64' && 'amd64' || 'arm64') }}
path: .e2e/prebuilt-sandbox

- name: Configure prebuilt sandbox
shell: bash
run: |
set -euo pipefail
sandbox="$GITHUB_WORKSPACE/.e2e/prebuilt-sandbox/openshell-sandbox"
if [[ ! -f "$sandbox" ]]; then
echo "downloaded artifact is missing $sandbox" >&2
exit 1
fi
chmod +x "$sandbox"
"$sandbox" --version
echo "OPENSHELL_SANDBOX_BIN=$sandbox" >> "$GITHUB_ENV"
132 changes: 131 additions & 1 deletion .github/workflows/branch-e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -263,6 +263,136 @@ jobs:
with:
image-tag: ${{ github.sha }}

podman-rootless-e2e:
name: E2E (rust-podman-rootless, Ubuntu 26.04 Nix VM)
needs: [pr_metadata, build-cli, build-gateway, build-sandbox]
if: needs.pr_metadata.outputs.should_run == 'true' && needs.pr_metadata.outputs.run_core_e2e == 'true'
permissions:
actions: read
contents: read
packages: read
runs-on: ubuntu-26.04
timeout-minutes: 60
env:
MISE_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false

- uses: ./.github/actions/setup-e2e-cli

- uses: ./.github/actions/setup-e2e-gateway

- uses: ./.github/actions/setup-e2e-sandbox
with:
artifact-prefix: rust-binary-supervisor

- uses: cachix/install-nix-action@13d8dd58da0234aa297dedd986986ccb8e7f3e24 # v31.11.1
with:
github_access_token: ${{ secrets.GITHUB_TOKEN }}

- uses: cachix/cachix-action@5f2d7c5294214f71b873db4b969586b980625e71 # v17
with:
name: openshell

- uses: ./.github/actions/setup-mise

- name: Install tools
run: mise install --locked

- name: Install system dependencies
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
build-essential \
clang \
cmake \
libclang-dev \
libssl-dev \
libz3-dev \
openssh-client \
pkg-config

- name: Run rootless Podman E2E
run: |
set -euo pipefail

mise x -- e2e/run.sh \
--vm ubuntu-26-04 \
--with podman-rootless \
--host-cli-bin "$OPENSHELL_BIN" \
--gateway-bin "$OPENSHELL_GATEWAY_BIN" \
--sandbox-bin "$OPENSHELL_SANDBOX_BIN" \
--gateway-config e2e/configs/gateway/podman.toml \
--features e2e-podman

podman-fedora-rootful-e2e:
name: E2E (rust-podman-rootful, Fedora Nix VM)
needs: [pr_metadata, build-cli, build-gateway, build-sandbox]
if: needs.pr_metadata.outputs.should_run == 'true' && needs.pr_metadata.outputs.run_core_e2e == 'true'
permissions:
actions: read
contents: read
packages: read
runs-on: ubuntu-26.04
timeout-minutes: 60
env:
MISE_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false

- uses: ./.github/actions/setup-e2e-cli

- uses: ./.github/actions/setup-e2e-gateway

- uses: ./.github/actions/setup-e2e-sandbox
with:
artifact-prefix: rust-binary-supervisor

- uses: cachix/install-nix-action@13d8dd58da0234aa297dedd986986ccb8e7f3e24 # v31.11.1
with:
github_access_token: ${{ secrets.GITHUB_TOKEN }}

- uses: cachix/cachix-action@5f2d7c5294214f71b873db4b969586b980625e71 # v17
with:
name: openshell

- uses: ./.github/actions/setup-mise

- name: Install tools
run: mise install --locked

- name: Install system dependencies
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
build-essential \
clang \
cmake \
libclang-dev \
libssl-dev \
libz3-dev \
openssh-client \
pkg-config

- name: Run Fedora rootful Podman E2E
run: |
set -euo pipefail

mise x -- e2e/run.sh \
--vm fedora \
--with podman-rootful \
--guest-gateway-user root \
--host-cli-bin "$OPENSHELL_BIN" \
--gateway-bin "$OPENSHELL_GATEWAY_BIN" \
--sandbox-bin "$OPENSHELL_SANDBOX_BIN" \
--gateway-config e2e/configs/gateway/podman-rootful.toml \
--features e2e-podman \
--suite sandbox_lifecycle

vm-e2e:
needs: [pr_metadata, build-cli, build-gateway, build-vm-driver]
if: needs.pr_metadata.outputs.should_run == 'true' && needs.pr_metadata.outputs.run_core_e2e == 'true'
Expand Down Expand Up @@ -428,7 +558,7 @@ jobs:

core-e2e-result:
name: Core E2E result
needs: [pr_metadata, docker-e2e, podman-e2e, vm-e2e, docker-external-driver-e2e, podman-external-driver-e2e, vm-external-driver-e2e, kubernetes-e2e, kubernetes-external-driver-e2e, kubernetes-workspace-managed-e2e, kubernetes-workspace-operator-e2e]
needs: [pr_metadata, docker-e2e, podman-e2e, podman-rootless-e2e, podman-fedora-rootful-e2e, vm-e2e, docker-external-driver-e2e, podman-external-driver-e2e, vm-external-driver-e2e, kubernetes-e2e, kubernetes-external-driver-e2e, kubernetes-workspace-managed-e2e, kubernetes-workspace-operator-e2e]
if: always() && needs.pr_metadata.outputs.should_run == 'true' && needs.pr_metadata.outputs.run_core_e2e == 'true'
runs-on: ubuntu-latest
steps:
Expand Down
86 changes: 29 additions & 57 deletions .github/workflows/release-canary.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,53 +61,43 @@ jobs:
name: Fedora RPM
if: ${{ github.event_name == 'workflow_dispatch' || github.event.workflow_run.conclusion == 'success' }}
runs-on: linux-amd64-cpu8
timeout-minutes: 20
timeout-minutes: 30
env:
FEDORA_CANARY_CONTAINER: openshell-fedora-canary-${{ github.run_id }}-${{ github.run_attempt }}
INSTALL_SH_URL: https://raw.githubusercontent.com/NVIDIA/OpenShell/${{ github.event.workflow_run.head_sha || github.sha }}/install.sh
steps:
- name: Start Fedora systemd container and root user manager
run: |
set -euo pipefail
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false

docker run --detach \
--name "${FEDORA_CANARY_CONTAINER}" \
--privileged \
--cgroupns=host \
--tmpfs /run \
--tmpfs /tmp \
--volume /sys/fs/cgroup:/sys/fs/cgroup:rw \
fedora:latest \
bash -lc 'dnf install -y curl dbus-daemon podman systemd && exec /usr/sbin/init'

for _ in $(seq 1 120); do
if docker exec "${FEDORA_CANARY_CONTAINER}" systemctl list-units --no-pager >/dev/null 2>&1; then
break
fi
if [ "$(docker inspect -f '{{.State.Running}}' "${FEDORA_CANARY_CONTAINER}")" != "true" ]; then
echo "::error::Fedora systemd container exited before systemd became reachable"
docker logs "${FEDORA_CANARY_CONTAINER}" >&2 || true
exit 1
fi
sleep 1
done
- uses: cachix/install-nix-action@13d8dd58da0234aa297dedd986986ccb8e7f3e24 # v31.11.1
with:
github_access_token: ${{ secrets.GITHUB_TOKEN }}

if ! docker exec "${FEDORA_CANARY_CONTAINER}" systemctl list-units --no-pager >/dev/null 2>&1; then
echo "::error::Fedora systemd container did not become reachable within 120s"
docker logs "${FEDORA_CANARY_CONTAINER}" >&2 || true
exit 1
fi
- uses: cachix/cachix-action@5f2d7c5294214f71b873db4b969586b980625e71 # v17
with:
name: openshell

- name: Install RPM in Fedora rootful Podman VM and check status
run: |
set -euo pipefail

docker exec --interactive "${FEDORA_CANARY_CONTAINER}" env \
nix run .#test-guest -- \
--distro fedora \
--with podman-rootful \
-- \
sudo env \
SUDO_USER=root \
HOME=/root \
XDG_RUNTIME_DIR=/run/user/0 \
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/0/bus \
OPENSHELL_TELEMETRY_ENABLED="$OPENSHELL_TELEMETRY_ENABLED" \
INSTALL_SH_URL="$INSTALL_SH_URL" \
bash -s <<'EOF'
set -euo pipefail
# install.sh manages the RPM gateway as a systemd user unit. This
# container is booted with systemd as PID 1, but it still has no
# login session. Start root's user manager explicitly so the
# installer can test service restart and gateway registration
# instead of its "restart later" fallback.

# install.sh manages the RPM gateway as a systemd user unit. Start
# root's user manager explicitly so the canary exercises the rootful
# service path on a real Fedora VM.
mkdir -p "${XDG_RUNTIME_DIR}"
chmod 700 "${XDG_RUNTIME_DIR}"
systemctl start user-runtime-dir@0.service || true
Expand All @@ -125,33 +115,15 @@ jobs:
systemctl --user status --no-pager >&2 || true
exit 1
fi
EOF

- name: Install and check status
run: |
set -euo pipefail

docker exec --interactive "${FEDORA_CANARY_CONTAINER}" env \
HOME=/root \
XDG_RUNTIME_DIR=/run/user/0 \
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/0/bus \
OPENSHELL_TELEMETRY_ENABLED="$OPENSHELL_TELEMETRY_ENABLED" \
INSTALL_SH_URL="https://raw.githubusercontent.com/NVIDIA/OpenShell/${{ github.event.workflow_run.head_sha || github.sha }}/install.sh" \
bash -s <<'EOF'
set -euo pipefail
mkdir -p "${HOME}/.config/openshell"
printf 'OPENSHELL_DRIVERS=podman\nOPENSHELL_TELEMETRY_ENABLED=%s\n' \
printf 'OPENSHELL_DRIVERS=podman\nOPENSHELL_PODMAN_SOCKET=/run/podman/podman.sock\nOPENSHELL_TELEMETRY_ENABLED=%s\n' \
"$OPENSHELL_TELEMETRY_ENABLED" > "${HOME}/.config/openshell/gateway.env"
podman info
podman --url unix:///run/podman/podman.sock info
curl -LsSf "${INSTALL_SH_URL}" | sh
openshell status
EOF

- name: Stop Fedora systemd container
if: always()
run: |
docker rm -f "${FEDORA_CANARY_CONTAINER}" >/dev/null 2>&1 || true

ubuntu-snap:
name: Ubuntu Snap
if: ${{ github.event.workflow_run.conclusion == 'success' }}
Expand Down
12 changes: 12 additions & 0 deletions TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,18 @@ Run the Podman-backed Rust CLI e2e suite:
mise run e2e:podman
```

Run the rootless Podman suite in an Ubuntu 26.04 Nix test guest:

```shell
mise run e2e:podman:rootless
```

Run the focused rootful Podman suite in a Fedora Nix test guest:

```shell
mise run e2e:podman:fedora-rootful
```

Run the VM-backed Rust CLI e2e suite:

```shell
Expand Down
29 changes: 29 additions & 0 deletions e2e/configs/gateway/podman-rootful.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

[openshell]
version = 1

[openshell.gateway]
bind_address = "127.0.0.1:8080"
log_level = "info"
compute_drivers = ["podman"]
disable_tls = true

[openshell.gateway.auth]
allow_unauthenticated_users = true

[openshell.gateway.gateway_jwt]
signing_key_path = ".cache/openshell-e2e/gateway-jwt/signing.pem"
public_key_path = ".cache/openshell-e2e/gateway-jwt/public.pem"
kid_path = ".cache/openshell-e2e/gateway-jwt/kid"
gateway_id = "openshell-e2e"
ttl_secs = 0

[openshell.drivers.podman]
socket_path = "/run/podman/podman.sock"
default_image = "ghcr.io/nvidia/openshell-community/sandboxes/base:latest"
image_pull_policy = "missing"
network_name = "openshell-e2e"
grpc_endpoint = "http://host.containers.internal:8080"
supervisor_image = "localhost/openshell/supervisor:e2e-vm"
Loading
Loading