Skip to content

Add dittosh server group — Ditto Server (portal) HTTP API client - #4

Open
Aaron LaBeau (biozal) wants to merge 2 commits into
mainfrom
portal-support
Open

Add dittosh server group — Ditto Server (portal) HTTP API client#4
Aaron LaBeau (biozal) wants to merge 2 commits into
mainfrom
portal-support

Conversation

@biozal

@biozal Aaron LaBeau (biozal) commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

What

Adds a dittosh server command group: a full HTTPS client for the Ditto Server (Big Peer) HTTP RPC API — the same API the portal's DQL editor uses. Endpoint inventory from the public docs (incl. the OpenAPI spec) and the portal's own client in cloud-services (portal/core/src/api/rpcClient.ts), which covers undocumented endpoints (RBAC, webhook secrets).

Commands

Command Endpoint
server execute (alias exec) POST /api/v5/store/execute (--api-version v4 = strict mode)
server remote-execute POST /api/v5/sync/remote_execute (peers, SYNC CONTEXT)
server attachment upload / get /api/v4/attachments (multipart up, raw bytes down)
server roles list/create/delete, server users list/set-roles/delete /api/v4/auth/* (portal RBAC)
server webhook-secrets list/create/rotate/delete /api/v4/auth/webhook/secret
server doctor validates config → connection → auth with a live probe

The legacy pre-DQL store API (find/findbyid/count/write) is deliberately omitted — execute covers it with full DQL (and its :param placeholders are rejected by the current deployment despite the OpenAPI text).

Configuration

--url/--api-key flags > shell env (DITTOSH_SERVER_URL/DITTOSH_SERVER_API_KEY, aliases DITTO_CLOUD_URL/DITTO_API_KEY) > .env in the current directory. The API key is never printed and is redacted from error messages. Exit codes follow the house contract (2 usage · 3 config/auth/connection · 1 query/API).

Every command's --help carries request body shapes, wire formats, and examples — most of these APIs aren't publicly documented, so the help text is the documentation.

Testing

  • 802 tests green. Unit: injectable FetchLike mock (no network), hermetic env scrubbing + tmpdir chdir (repo .env may hold real portal creds). e2e: real subprocess against a local node:http mock Ditto Server, execa extendEnv: false.
  • Coverage 90.8 / 86.2 / 96.2 / 92.1 vs the 85% hard gate.
  • Verified live against the retail app in the portal: SELECT/params/aggregates/EXPLAIN, full INSERT→SELECT→UPDATE→DELETE round-trip on a scratch collection, doctor with good + bad keys.
  • Adversarial review: 4 rounds, 2 independent reviewers, CONVERGED. ~25 issues fixed with regression tests (batch exit-code flattening, URL credential echo, --args - TTY hang, mid-body timeout mapping, roles-list pagination truncation, …). Ledger in plans/SDKS-4855-implementation-plan.md M9.
  • docs/testing-server.md: copy-pasteable manual checklist for the portal.

Notes

  • .env.sample documents the two new variables.
  • Not covered (deliberate): JWT http_login exchange, CDC/Kafka streams, server-side REPL.

@biozal
Add dittosh server group — Ditto Server (portal) HTTP API client
35fe631 
New command group covering the portal HTTP API surface (public docs +
the portal's own rpcClient wire shapes):
- server execute (alias exec): DQL over HTTP — positional/-e/-f/stdin
  batch, -p/--args, --txn-id, --api-version v4|v5, all output formats,
  -o export, pager, --max-rows, --time, --continue-on-error
- server remote-execute: DQL on connected peers (SYNC CONTEXT)
- server attachment upload/get (multipart; binary-safe download)
- server roles list/create/delete, users list/set-roles/delete (RBAC)
- server webhook-secrets list/create/rotate/delete
- server doctor: config -> connection -> auth probe with source labels

Config: --url/--api-key > shell env (DITTOSH_SERVER_URL/_API_KEY,
aliases DITTO_CLOUD_URL/DITTO_API_KEY) > cwd .env. Key never printed;
redacted from errors. Legacy pre-DQL store API (find/findbyid/count/
write) deliberately omitted — execute covers it.

802 tests green (unit: mock FetchLike, hermetic env scrubbing; e2e:
node:http mock server, execa extendEnv:false). Coverage 90.8/86.2/
96.2/92.1 vs 85 gate. docs/testing-server.md manual checklist. Verified
live against the portal (retail app). Adversarial review: 4 rounds,
2 reviewers, CONVERGED.
@biozal
Review round 5 (2 fresh adversarial reviewers, agree-before-fixing)
b24544c 
Agreed majors:
- Fail closed on non-contract 2xx bodies: execute/remote-execute require
  DQL hallmark keys (no more silent [] on proxy/SSO HTML 200s; no raw
  TypeError on items:'string'); roles/users require their envelope keys
  (portal parity); webhook list keeps portal's [] fallbacks + 404 -> [].
- Reject non-loopback http:// — the cloud's 308 sent the key cleartext
  on hop 1, then stripped it on redirect -> misleading 401.
- --timeout flag (default 120s) + PortalTimeoutError: honest 'may still
  be running' message, exit 1 (a timeout is not proof of failure; the
  portal leaves DQL unbounded — 30s stays only for admin endpoints).
- roles create --help example was a shell syntax error in zsh AND bash
  (unescaped single quotes) — now uses " " inside JSON.
- docs/testing-server.md webhook section rewritten: the provider must
  already exist (verified live: 400 Provider not found).

Agreed minors: .env UTF-8 BOM strip; per-key .env hints; batch checks
stdoutBroken() (| head); users-list 404 unsupported-endpoint hint;
--permissions JSON strings checked against the blanket whitelist;
remote-execute sets DITTOSH_JSON_OUT + accepts leading comments;
printWarnings never prints 'undefined'; -o help includes ADVISE;
usage-first --api-version ordering; stripEq comment corrected;
No-statements error is red; dead ?? removed.

Process: e2e coverage gaps closed per AGENTS.md (doctor, users,
webhook-secrets, roles create/delete, attachment upload, remote-execute
happy path). README gained the full dittosh server feature section;
exit-code table updated.

Gates by exit code: lint 0, tsc 0, 765+ tests 0, coverage 90.7/86.3/
96.3/92.1 (gate 85), build 0. Live-verified against the portal.
@biozal
Aaron LaBeau (biozal) marked this pull request as ready for review September 4, 2026 13:00
@biozal Aaron LaBeau (biozal) self-assigned this Sep 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ErikEverson Erik Everson (ErikEverson) Awaiting requested review from ErikEverson
@skylerjokiel Skyler Jokiel (skylerjokiel) Awaiting requested review from skylerjokiel
@draper-ditto Chris Draper (draper-ditto) Awaiting requested review from draper-ditto

At least 1 approving review is required to merge this pull request.

Assignees

@biozal Aaron LaBeau (biozal)

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@biozal