Skip to content

fix: preserve incomplete tool arguments in telemetry - #209

Open
SeanKudrna wants to merge 1 commit into
xai-org:mainfrom
SeanKudrna:fix/tool-argument-telemetry
Open

fix: preserve incomplete tool arguments in telemetry#209
SeanKudrna wants to merge 1 commit into
xai-org:mainfrom
SeanKudrna:fix/tool-argument-telemetry

Conversation

@SeanKudrna

@SeanKudrna SeanKudrna commented Sep 8, 2026

Copy link
Copy Markdown

Description

When a model reaches its token limit in the middle of a tool call, its arguments can contain incomplete JSON. Both prompt and completion telemetry unconditionally call json.loads, so chat.sample() raises JSONDecodeError instead of returning the response. Consuming chat.stream() also raises when it collects final span attributes, and replaying the response fails before the next request.

Parse valid arguments as before, but retain the original string in telemetry when decoding fails. This preserves the response for application-level handling without repairing or executing malformed arguments. The existing sensitive-attribute opt-out still applies.

Validation:

  • Reproduced the response failure before the fix in sync and async calls.
  • Added regression coverage for valid, incomplete, and empty arguments, response replay, stream completion, and the telemetry opt-out.
  • uv run ruff format --check and uv run ruff check: passed.
  • uv run pyright: 0 errors or warnings.
  • uv run pytest -n 4 -q: 843 passed on Python 3.13, with 20 existing batch-API deprecation warnings.

Related Issue

N/A. Submitted as a small, localized bug fix under the contribution guide's direct-PR guidance.

Type of Change

  • Bug fix

Checklist

  • Read the contribution guide and CLA.
  • Prior maintainer approval: not requested for this small fix.
  • Tested locally with formatting, lint, type checks, and the complete test suite.
  • Added regression tests and documented the fallback in code. No public API change requires a documentation update.

@SeanKudrna
SeanKudrna requested a review from a team as a code owner September 8, 2026 00:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant