fix: handle empty SMW ask results in semantic_search - #150
Open
LukasGold wants to merge 1 commit into
Open
Conversation
- SMW serialises an empty ask result set as a JSON array, not an object, so any zero-result query raised AttributeError instead of returning [] - normalise the payload once via _ask_results_as_dict() - warn when a query hits its limit, which silently truncated results - warn when entries are dropped by the exists != "1" filter - refs #145, #111
Contributor
Release previewMerging this PR would release v2.0.3 (current: Changelog preview (truncated)## v2.0.3 (2026-08-31)
### Bug Fixes
- Handle empty SMW ask results in semantic_search
([`e00ca8b`](https://github.com/OpenSemanticLab/osw-python/commit/e00ca8be80eb1575a520b609d5d988604ad59e24))
### Testing
- Rename oold.py to oold_test.py so its tests are collected
([`20072a9`](https://github.com/OpenSemanticLab/osw-python/commit/20072a9249cd97126a222c62a70f84e0433343ef))
Preview via python-semantic-release and conventional commits. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the crash behind #145 and explains the "random results" in #111.
Root causes
Three separate defects, all reproduced live against
healthbatt.projects01.open-semantic-lab.org.1. Zero-result queries raise
AttributeError(fixed here)SMW's
action=askserialises a non-empty result set as a JSON object keyed by page title, but an empty one as a JSON array.semantic_searchcalled.values()on it unconditionally (src/osw/wiki_tools.py:257), so every zero-result query raisedAttributeError: 'list' object has no attribute 'values'instead of returning[].Affects single queries, sequential batches and parallel batches.
return_json=Truehappened to survive because it returns before that line.OSW.query_instances()(src/osw/core.py:2007) is affected too, so querying a category with no instances raised instead of returning an empty list.2. Silent truncation at
limit(surfaced here)SearchParam.limitdefaults to 1000 and there is no pagination, so a larger match set is silently reduced to an arbitrary subset. Measured on healthbatt:[[Category:Item]]matches 2637 pages,semantic_searchreturned 1000 with no indication.3.
existsflickers server-side (surfaced here, not fixable in this library)The
exists == "1"filter silently discards results. The field only ever takes'1'or'', but it is not stable: across 8 identical calls, 8 pages flipped between the two, and no page was consistently''. The flips are perfectly correlated across pages (calls 0,2,3,4,7 agree; calls 1,5,6 agree), which points at inconsistent state between backends or an SMW query-result cache rather than at real page deletions. Consecutive identical calls returned 998 or 994 titles.This is the "random results" from #111. It needs to be addressed on the OSL instance side; this PR only makes the loss visible.
Changes
_ask_results_as_dict()normalises theaskpayload to a mapping, so an empty result set yields[]instead of raisingwarnings.warnwhen the result count meets the requested limit, emitted before thereturn_jsonearly return so both return modes get itwarnings.warnwhen entries are dropped by theexists != "1"filter, naming how manyFilter semantics, the default limit, and
prefix_searchare unchanged.Verification
pytest tests/ --ignore=tests/integration-> 61 passed, 1 skippedAttributeError/DID NOT WARNreturn_jsonwith a zero-result query all return results now instead of raising; both warnings fire with real countsOut of scope, noticed while probing
single_query += f"|limit={query.limit}"is appended unconditionally, so a caller-supplied limit in the query string is silently overridden:[[Category:Item]]|limit=2is sent as[[Category:Item]]|limit=2|limit=1000and SMW honours the last one. Verified live (returned 1000, not 2). Not changed here.