feat: an osw CLI and an MCP server for live OSL instances - #133
feat: an osw CLI and an MCP server for live OSL instances#133LukasGold wants to merge 30 commits into
Conversation
Add an in-repo `osw[mcp]` extra and an `osw-mcp` stdio console script that wraps OswExpress and serves it over the Model Context Protocol for clients such as Claude Code. Tools: semantic/SPARQL/full-text search, category schema introspection, entity read + JSON-LD export, create/update/delete, full page-slot access, and file up/download. - Delete is provenance-guarded: a local JSON ledger records pages the server created/modified; deleting anything untracked requires confirm_external_delete=true. - Credentials resolve from env/.env and are validated up front (fail fast, never prompts, so stdio is never corrupted by an input() call). - osw stdout is redirected to stderr so it never leaks onto the JSON-RPC channel. - OSW_MCP_READ_ONLY hides all mutating tools.
b46f046 to
23db777
Compare
Release previewMerging this PR would release v2.1.0 (current: Changelog preview (truncated)## v2.1.0 (2026-08-31)
### Bug Fixes
- Allow uploading a file from an in-memory stream
([`64d99d6`](https://github.com/OpenSemanticLab/osw-python/commit/64d99d66ea45bc56bdaa6d12f2502df0bbbaf88c))
- **service**: Validate read_only via pydantic instead of truthy set
([`56711ed`](https://github.com/OpenSemanticLab/osw-python/commit/56711ed878b260288ba6100c9fdf2623544d43b7))
### Documentation
- Drop remaining references to the separate MCP environment
([`be75e47`](https://github.com/OpenSemanticLab/osw-python/commit/be75e47ced3ad4692f5b14a30d509d09c0a39c2e))
### Refactoring
- De-isolate the mcp extra from the dev environment
([`8f81fec`](https://github.com/OpenSemanticLab/osw-python/commit/8f81fec145fda977544f9d95576b7edf225c26a1))
- **service**: Validate Settings with pydantic
([`2051c26`](https://github.com/OpenSemanticLab/osw-python/commit/2051c26174e149455bf7111c0e8671d13259eddb))
### Testing
- Do not assume the first ask-query hit carries jsondata
([`f4ef72c`](https://github.com/OpenSemanticLab/osw-python/commit/f4ef72cdd2e73e42f01d36717f25b8239d1ab8c1))
- Rename oold.py to oold_test.py so its tests are collected
([`20072a9`](https://github.com/OpenSemanticLab/osw-python/commit/20072a9249cd97126a222c62a70f84e0433343ef))
- Stop test_init_from_env_vars leaking OSW_CRED_FILEPATH
([`4b7dd7e`](https://github.com/OpenSemanticLab/osw-python/commit/4b7dd7e5670eee4a230751412ad756153a9f2387))
Preview via python-semantic-release and conventional commits. |
|
Credential loading bypasses the
|
Multi-instance credentials: selecting an OSL instance per sessionFollowing on from the credential-file comment above. Pinning one The multi-account file already exists. One correction on the interactive part. Prompting on the CLI cannot work for Elicit the choice, never the secret. Elicited values pass through the MCP Concretely:
Multiple registrations already cover part of this, with no code change: claude mcp add osw-dev --env OSW_MCP_ENV_FILE=... -- uvx ...
claude mcp add osw-prod --env OSW_MCP_ENV_FILE=... --env OSW_MCP_READ_ONLY=true -- uvx ...That has two advantages over in-session switching: the instance is visible in the Supplying a |
- FastMCP replaced by MCPServer, extra now requires mcp>=2 - mcp dropped from the all extra and the dev group: it needs anyio>=4.9, workflow pins anyio<4.7 (#139) - uv conflicts declare mcp exclusive with workflow and with dev - pytest stack moved to its own test group, so an environment with both pytest and mcp exists - src/osw/mcp excluded from ty, mcp tests guarded by importorskip
- OSW_MCP_CRED_FILEPATH configures it, OSL_CRED_FILEPATH is a fallback - an alternative to OSW_USERNAME/OSW_PASSWORD, so the password is not duplicated into a second plaintext file - existence and a matching domain entry are validated at startup - lookups use CredentialFallback.none, so osw never prompts and never blocks the stdio transport
- list_instances and select_instance tools, returning iris only and never any credential value - OSW_DOMAIN becomes optional when a credential file supplies the iris - auto-selects when OSW_DOMAIN is set or the file holds exactly one iri - switching rebuilds the connection and the per-domain provenance ledger - tools resolve the active domain and credentials at call time
|
All three addressed, one commit each: mcp>=1.2 resolves to 2.0.0Confirmed against the published wheel: The anyio clash you predicted is real, and it is a constraint the PR now has to Cost: the module is neither type-checked nor unit-tested in the default Credential loading bypasses the CredentialManager file patternAn iri-keyed credential file now configures the server, built into a Two things worth confirming:
Multi-instance credentialsAgreed on the constraint: elicited values pass through the client and into the Implemented as Note this piece was reversed later in the PR, see |
- move config/ledger/serialization from osw.mcp to osw.service - add errors.py (OpError), context.py (Context+Policy), registry.py - Operation validator rejects path-like params on the mcp surface - canonical OSW_* env names, OSW_MCP_*/OSL_* kept as aliases - config/ledger/serialization tests now run without the mcp extra
- add osw/service/ops/ with the search group as @operation functions - mcp/tools/search.py becomes a registry loop over bind() - add transitional legacy_context() so existing tests keep passing - bind() resolves annotations against the op module, not registry.py
- schema, status, entities and slots lifted as @operation functions - mcp/tools/*.py collapse to registry loops over bind() - error dicts become raised errors.*; ledger calls become records= hooks - normalize cli_name so every group reads as `osw <group> <verb>` - 172 passed in the dev env, 29 in the mcp extra env
- typer as a base dependency; osw = "osw.cli.main:app" console script - commands built from iter_operations(surface="cli"); Context built lazily - OpError.exit_code becomes the process exit status, no traceback - json_value parser lives in osw.service.params, so core never imports cli - set_slot coerces content per the sibling slot's content model
- add path-free get_file_info/read_file_text/write_file_text built on
WikiFileController.get()/.put(), never touching the local filesystem
- move file download/upload and ledger path to osw.cli.ops, the only
module allowed to name a path (surfaces={"cli"}); drop status's
ledger_path
- guard tests: no MCP-surfaced op names a path, and osw.mcp.server
never imports osw.cli
- read_file_text decodes incrementally so a byte cap splitting a
multi-byte character is not misreported as binary content
- delete list_instances/select_instance; each server process is pinned to one OSL instance and refuses to start if none resolves - map Operation's hints onto ToolAnnotations and _meta by explicit keyword, since the SDK silently absorbs a misspelled field name - set server instructions and version; make mcp.run(transport="stdio") explicit - add CLI --instance and osw instance list, returning iris only - retarget the "no instance selected" message at OSW_DOMAIN and --instance instead of the removed tool
Reversing in-session instance switching (
|
- delete osw/mcp/tools/ and connection.py; bodies live in osw.service.ops - fold registration.py into server.py, its sole consumer - replace test_mcp_tools.py with test_service_ops.py + test_mcp_server.py - rebuild the integration fixture on bind()/iter_operations - test_mcp_instances.py -> test_service_instances.py, no longer SDK-gated
- drop src/osw/mcp from [tool.ty.src] exclude - silence only the two unresolvable SDK imports inline (issue #139)
- add a Command line section: command tree, global options, exit behaviour - move credentials into a shared Configuration section with an alias table - state that no MCP tool takes a path, and where the path-based commands live - add stdio type, multi-instance and per-instance permission examples - give each CLI command group a one-line help string
- CLI now searches upward from the working directory, not from the installed package's directory (dotenv's default walks the call stack) - MCP server searches nowhere: its CWD is chosen by the client - both print the resolved .env and credential file to stderr at startup - a missing credential file whose path holds a control character now explains .env double-quote escape decoding
- new docs page "CLI and MCP tools", added to the zensical nav - README keeps a short pointer section, otherwise back to its old shape - get-started extras table gains the osw[mcp] row
- server entries now set OSW_CRED_FILEPATH plus OSW_DOMAIN, no .env needed - multi-instance example shares one credential file, one domain per server - register via claude mcp add-json, which takes the entry verbatim
- server no longer auto-selects a single-iri credential file; the CLI still does - docs present the env block and the .env file as two supported styles - multi-instance example shows one server of each style - drop the "OSW_DOMAIN may be omitted" note
- add a Setup section with the uv/pip installs up front - move Configuration below the MCP section, both adapters share it - state where the .env is looked for first, drop the always-loaded claim - note that --instance is optional and when it is required - collect the rationale in a Design notes section at the end
- lead with uv tool install, the mcp extra includes the base package - fold the pip, uv add and uvx variants into a details element - move the editable-install caveat to a Notes for developers section - document running the server from a local checkout via uvx --from
Ready for reviewThe description no longer matched the branch and has been rewritten around what Four things now differ from what was reviewed in the first round:
Two behaviours settled since then:
The CLI and MCP documentation moved out of the README into Default suite is 236 passed, 4 skipped, with |
- --instance is optional, so it is not what makes inference acceptable - the CLI resolves per invocation and reports the instance it resolved - state the --instance condition as OSW_DOMAIN unset, not .env-specific
Resolve the anyio conflict that forced the mcp isolation: - main widened the workflow pin to anyio>=4.9,<4.14, compatible with mcp>=2 - drop [tool.uv] conflicts and the separate test dependency group - add mcp to the all extra and to the dev group - relock
- drop the ty: ignore on the mcp SDK imports in server.py - run the MCP tests unconditionally instead of importorskip-ing them - update the config.py env-file hint (no more `test` group) - docs: mcp is part of osw[all]; replace the anyio design note
- test docstrings no longer contrast against a plain dev env - deptry comment no longer claims extras are absent from dev - README lists osw[mcp] among the extras
Closes #140. - express.py: replace the unreachable isinstance(source, IO) check with a duck-typed one; typing.IO is not runtime-checkable - InMemoryController: drop the __init__ that assigned stream before the model was initialised and overwrote a caller-supplied stream - default the stream to BytesIO, matching the byte-oriented get/put - declare IO in the upload_file / osw_upload_file signatures
- convert Settings from a frozen dataclass to a frozen pydantic model - add validators for domain, sparql_endpoint, state_dir, cred_filepath - constrain max_results/max_chars to positive integers via Field(gt=0) - drop _int_env in favour of one ValidationError -> RuntimeError site that still names the exact alias that was set Closes #143
- use monkeypatch.setenv so OSW_CRED_FILEPATH and OSW_DOMAIN are restored - the test unlinks its credential file, so the leaked path pointed every later test at a missing file - surfaced by the mcp de-isolation: tests/integration/test_mcp_server.py no longer skips for a missing SDK, so it hit the polluted environment
- SMW ask results have no defined order and Category:Item can hold pages without a jsondata slot - scan all returned titles, require at least one with the slot
- route OSW_READ_ONLY through Settings so an unparseable value raises - a typo like "ture" previously yielded False, silently enabling writes - "y"/"t" now parse as true; all documented spellings keep working - blank/whitespace-only still falls back to the default, as for the ints - drop the now-unused _TRUTHY set Follow-up to #143.
What this adds
Two ways to work against a live OpenSemanticLab instance without writing Python:
osw, a command line client, shipped with the base packageosw-mcp, a stdio MCP server (osw[mcp]extra) that exposes one instanceto agent clients such as Claude Code
Both are generated from a single declaration per operation, so a command and its
matching tool cannot drift apart in behaviour, argument names or help text.
Capabilities
search ask,search text,search instances,search sparqlsearch_entities,full_text_search,list_instances_of_category,sparql_queryschema getget_category_schemaentity get,put,export,deleteget_entity,create_or_update_entity,export_entity_jsonld,delete_entityslot list,get,setlist_page_slots,get_slot,set_slotfile info,cat,writeget_file_info,read_file_text,write_file_textfile download,file uploadstatus,instance list,ledger pathstatusBehaviour common to both:
--read-only,OSW_READ_ONLY) hides mutating tools ratherthan failing them when called, so an agent never sees a tool it cannot use.
normally; anything else requires
confirm_external_delete=true.OSW_MAX_RESULTS,OSW_MAX_CHARS) keep a broad query fromflooding an agent's context.
.envfile, or an iri-keyed oswcredential file. No tool ever returns one.
osw --jsonputs JSON on stdout and osw's ownprogress output on stderr. Failures exit non-zero with one line, no traceback.
Constraints that shaped it
Decisions that need agreement
1. No filesystem path reaches the MCP surface.
Rejected: documenting the server as local-use-only. That is a rule with no
enforcement behind it. Instead paths exist only in
osw.cli, and the operationmodel refuses to register a path-like parameter on an MCP-surfaced operation, so
the server fails at import rather than shipping such a tool.
2. One server process per instance, no switching at runtime.
This reverses
d393a66, which was agreed earlier in this PR. In-sessionswitching cannot keep the instance visible in the tool name, cannot make
read-only per instance, and is a prompt-injection target. Argument in full:
#133 (comment)
3. The server refuses to start unless an instance is named explicitly.
Rejected: failing at the first tool call, and inferring the sole iri of a
credential file. A server that cannot name its target would advertise tools that
all fail. The CLI still infers, because it resolves per invocation, reports what
it resolved, and
--instanceoverrides any single command.4.
typeras a base dependency, not an extra.Rejected: argparse plus a hand-written signature-to-parser translator (roughly 50
lines to own and keep honest), and click, which is decorator-per-option and would
mean writing every parameter twice. typer reads the same type hints and
docstrings the MCP SDK reads, which is what makes one declaration serve both. As
an extra it would let
pip install oswship a brokenoswconsole script.5.
mcpis an extra, not a base dependency.The SDK pulls in a server stack (starlette, uvicorn, sse-starlette) that nothing
in the Python API or the CLI needs, so only users who actually run the server pay
for it. It is included in
osw[all]and in thedevgroup, so the server and itstests share one environment with everything else. The earlier isolation, forced by
an anyio conflict with
osw[workflow]and declared through[tool.uv] conflicts,is gone: #139 is closed and
the pin is now
anyio>=4.9,<4.14.6. Canonical
OSW_*configuration names, old names kept as aliases.OSW_CRED_FILEPATHis already read bysrc/osw/express.py, so the previousOSW_MCP_CRED_FILEPATHdiverged from the library that owns the same setting.OSW_MCP_*andOSL_*remain accepted, so existing deployments keep working.Configuration
Required: an instance and credentials, either
OSW_DOMAINplusOSW_USERNAME/OSW_PASSWORD, orOSW_DOMAINplusOSW_CRED_FILEPATH..envhandling differs by adapter on purpose. The CLI searches upward from theworking directory. The server searches nowhere, because its working directory is
chosen by the client, and takes its settings from the
envblock of itsregistration. Both report the env file and credential file they resolved on
stderr before connecting.
Full reference, including registering one server per instance:
https://github.com/OpenSemanticLab/osw-python/blob/feat/mcp-server/docs/cli-and-mcp.md
Verification
One environment covers everything. That includes the 17 MCP tests in
tests/test_mcp_registration.py,tests/test_mcp_server.pyandtests/test_no_paths_on_mcp_surface.py, which CI now runs alongside the rest.They previously self-skipped in CI, because a plain
uv sync --frozenneverinstalled the
mcpextra, so the MCP code was effectively untested there.tests/integration/test_mcp_server.pyruns against a live instance.Follow-ups
Filed while planning this work.
Resolved:
that forced the separate MCP environment. Closed upstream of this branch.
fix: prevent silent page loss in store_entity parallel upload #132.
Settingsis now a frozen pydantic model with validators instead of adataclass with hand-rolled env parsing.
fix: allow uploading a file from an in-memory stream #148 and already merged into
this branch. Whichever of the two lands first, the other carries the change as
an ancestor, so fix: allow uploading a file from an in-memory stream #148 collapses to an empty diff if this PR merges first.
Open, deliberately not blocking this PR:
fetch_schemarewriting the installed
src/osw/model/entity.pyas a side effect. This ispre-existing
osw.corebehaviour, but this PR adds two new entry points to it(
create_or_update_entity, andexport_entity_jsonldviaautofetch_schema),so merging widens the exposure. The MCP
get_category_schematool is notaffected, it reads the
jsonschemaslot directly. Documented indocs/cli-and-mcp.md; design discussion running on the issue.ToolAnnotationssilently ignoring a misspelled hint. The guard is already in this branch, typed
Optional[bool]fields plus keyword construction plus a test; only the upstreamreport is outstanding.
The
store_entityparallel-upload fix that once rode along on this branch is nowtracked in #132.